diff options
| author | Erlang/OTP <[email protected]> | 2009-11-20 14:54:40 +0000 |
|---|---|---|
| committer | Erlang/OTP <[email protected]> | 2009-11-20 14:54:40 +0000 |
| commit | 84adefa331c4159d432d22840663c38f155cd4c1 (patch) | |
| tree | bff9a9c66adda4df2106dfd0e5c053ab182a12bd /lib/inets | |
| download | otp-84adefa331c4159d432d22840663c38f155cd4c1.tar.gz otp-84adefa331c4159d432d22840663c38f155cd4c1.tar.bz2 otp-84adefa331c4159d432d22840663c38f155cd4c1.zip | |
The R13B03 release.OTP_R13B03
Diffstat (limited to 'lib/inets')
274 files changed, 72872 insertions, 0 deletions
diff --git a/lib/inets/AUTHORS b/lib/inets/AUTHORS new file mode 100644 index 0000000000..a0dcc1fe92 --- /dev/null +++ b/lib/inets/AUTHORS @@ -0,0 +1,19 @@ +Original Authors: + +Ingela Anderton Andin - supported http client +Johan Blom - unsupported http client +H�kan Mattsson - tftp +Peter H�gfeldt - first version of ftp +Joakim Greben� - first version of httpd +Mattias Nilsson - first version of httpd + +Contributors: + +Ingela Anderton Andin +Martin Gustafsson +Johan Blom +Torbj�rn T�rnkvist +Joe Armstrong + + + diff --git a/lib/inets/Makefile b/lib/inets/Makefile new file mode 100644 index 0000000000..9a54bfb8e2 --- /dev/null +++ b/lib/inets/Makefile @@ -0,0 +1,38 @@ +# +# %CopyrightBegin% +# +# Copyright Ericsson AB 1996-2009. All Rights Reserved. +# +# The contents of this file are subject to the Erlang Public License, +# Version 1.1, (the "License"); you may not use this file except in +# compliance with the License. You should have received a copy of the +# Erlang Public License along with this software. If not, it can be +# retrieved online at http://www.erlang.org/. +# +# Software distributed under the License is distributed on an "AS IS" +# basis, WITHOUT WARRANTY OF ANY KIND, either express or implied. See +# the License for the specific language governing rights and limitations +# under the License. +# +# %CopyrightEnd% +# +# +include $(ERL_TOP)/make/target.mk +include $(ERL_TOP)/make/$(TARGET)/otp.mk + +# ---------------------------------------------------- +# Macros +# ---------------------------------------------------- + +SUB_DIRECTORIES = src examples priv doc/src + +include vsn.mk +VSN = $(INETS_VSN) + +SPECIAL_TARGETS = + +# ---------------------------------------------------- +# Default Subdir Targets +# ---------------------------------------------------- +include $(ERL_TOP)/make/otp_subdir.mk + diff --git a/lib/inets/doc/archive/includes.txt b/lib/inets/doc/archive/includes.txt new file mode 100644 index 0000000000..5222790be1 --- /dev/null +++ b/lib/inets/doc/archive/includes.txt @@ -0,0 +1,168 @@ + Server Side Includes (SSI) + +NCSA HTTPd allows users to create documents which provide simple information +to clients on the fly. Such information can include the current date, the +file's last modification date, and the size or last modification of other +files. In its more advanced usage, it can provide a powerful interface to +CGI and /bin/sh programs. + + * SSI Issues + * SSI Setup + * Converting INC SRV to SSI + * The SSI Format + * SSI Environment Variables + +------------------------------------------------------------------------ + +SSI Issues + +Having the server parse documents is a double edged sword. It can be costly +for heavily loaded servers to perform parsing of files while sending them. +Further, it can be considered a security risk to have average users +executing commands as the server's User. If you disable the exec option, +this danger is mitigated, but the performance issue remains. You should +consider these items carefully before activating server-side includes on +your server. + +------------------------------------------------------------------------ + +SSI Setup + +First, you should decide which directories you want to allow Includes in. +Most likely this will not include users' home directories or directories you +do not trust. You should then decide, of the directories you are allowing +includes in, which directories are safe enough to use exec in. + +For the directories in which you want to fully enable includes, you need to +use the Options directive to turn on the option Includes. Similarly for the +directories you want crippled (no exec) includes, you should use the option +IncludesNOEXEC. In any directory you want to disable includes, use the +Options directive without either option. + +Next, you need to tell the server what filename extension you are using for +the parsed files. These files, while very similar to HTML, are not HTML and +are thus not treated the same. Internally, the server uses the magic MIME +type text/x-server-parsed-html to identify parsed documents. It will then +perform a format conversion to change these files into HTML for the client. +To tell the server which extension you want to use for parsed files, use the +AddType directive. For instance: + +AddType text/x-server-parsed-html .shtml + +This makes any file ending with .shtml a parsed file. Alternatively, if you +don't care about the performance hit of having all .html files parsed, you +could use: + +AddType text/x-server-parsed-html .html + +This would make the server parse all .html files. + +------------------------------------------------------------------------ + +Converting your old INC SRV documents to the SSI Format + +You should use the program inc2shtml in the support subdirectory of the +HTTPd distribution to translate your documents from HTTPd 1.1 and earlier to +the new format. Usage is simple: inc2shtml file.html > file.shtml. + +------------------------------------------------------------------------ + +The SSI Format + +All directives to the server are formatted as SGML comments within the +document. This is in case the document should ever find itself in the +client's hands unparsed. Each directive has the following format: + +<!--#command tag1="value1" tag2="value2" --> + +Each command takes different arguments, most only accept one tag at a time. +Here is a breakdown of the commands and their associated tags: + + * config + + The config directive controls various aspects of the file parsing. + There are two valid tags: + + o errmsg controls what message is sent back to the client if an + error includes while parsing the document. When an error occurs, + it is logged in the server's error log. + + o timefmt gives the server a new format to use when providing dates. + This is a string compatible with the strftime library call under + most versions of UNIX. + + o sizefmt determines the formatting to be used when displaying the + size of a file. Valid choices are bytes, for a formatted byte + count (formatted as 1,234,567), or abbrev for an abbreviated + version displaying the number of kilobytes or megabytes the file + occupies. + + * include + + include will insert the text of a document into the parsed document. + Any included file is subject to the usual access control. This command + accepts two tags: + + o virtual gives a virtual path to a document on the server. You must + access a normal file this way, you cannot access a CGI script in + this fashion. You can, however, access another parsed document. + + o file gives a pathname relative to the current directory. ../ + cannot be used in this pathname, nor can absolute paths be used. + As above, you can send other parsed documents, but you cannot send + CGI scripts. + + * echo prints the value of one of the include variables (defined below). + Any dates are printed subject to the currently configured timefmt. The + only valid tag to this command is var, whose value is the name of the + variable you wish to echo. + + * fsize prints the size of the specified file. Valid tags are the same as + with the include command. The resulting format of this command is + subject to the sizefmt parameter to the config command. + + * flastmod prints the last modification date of the specified file, + subject to the formatting preference given by the timefmt parameter to + config. Valid tags are the same as with the include command. + + * exec executes a given shell command or CGI script. It must be activated + to be used. Valid tags are: + + o cmd will execute the given string using /bin/sh. All of the + variables defined below are defined, and can be used in the + command. + + o cgi will execute the given virtual path to a CGI script and + include its output. The server does not perform error checking to + make sure your script didn't output horrible things like a GIF, so + be careful. It will, however, interpret any URL Location: header + and translate it into an HTML anchor. + +------------------------------------------------------------------------ + +SSI Environment Variables + +A number of variables are made available to parsed documents. In addition to +the CGI variable set, the following variables are made available: + + * DOCUMENT_NAME: The current filename. + + * DOCUMENT_URI: The virtual path to this document (such as + /docs/tutorials/foo.shtml). + + * QUERY_STRING_UNESCAPED: The unescaped version of any search query the + client sent, with all shell-special characters escaped with \. + + * DATE_LOCAL: The current date, local time zone. Subject to the timefmt + parameter to the config command. + + * DATE_GMT: Same as DATE_LOCAL but in Greenwich mean time. + + * LAST_MODIFIED: The last modification date of the current document. + Subject to timefmt like the others. + +------------------------------------------------------------------------ + [Back] Return to tutorial index + +------------------------------------------------------------------------ +NCSA HTTPd Development Team / [email protected] / 9-28-95 diff --git a/lib/inets/doc/archive/pieces_of_code.txt b/lib/inets/doc/archive/pieces_of_code.txt new file mode 100644 index 0000000000..f18fd2a20c --- /dev/null +++ b/lib/inets/doc/archive/pieces_of_code.txt @@ -0,0 +1,83 @@ + +%script_name([],_) -> +% not_an_executable; +%script_name([""|Rest],SoFar) -> +% script_name(Rest,SoFar); +%script_name([First|Rest],SoFar) -> +% NewSoFar=SoFar++"/"++First, +% case file:read_file_info(NewSoFar) of +% %% Do the script exists and is it executable? To be done! +% {ok,FileInfo} when FileInfo#file_info.type == regular -> +% {yes,NewSoFar,[$/|Rest]}; +% {ok,FileInfo} -> +% script_name(Rest,NewSoFar); +% {error,Reason} -> +% script_name(Rest,NewSoFar) +% end. + + + + case httpd_util:key1search(Info#mod.data,real_name) of + undefined -> + DocumentRoot= + httpd_util:lookup(Info#mod.config_db,document_root,""), + DocumentRoot++"/"++httpd_util:decode_hex(Info#mod.request_uri); + RealName -> + RealName + end, + + +test() -> + disk_log:open({name,"/var/tmp/server_root/logs/access_disk_log"}, + {file,"/var/tmp/server_root/logs/access_disk_log"}, + {linkto,Pid}, + {format,external}, + {type,Type} + {size,{MaxBytes,MaxFiles}}, + {head,{M,F,A}}, + +%% order +parse_directory(Stream,ConfigDB,Directory,[$o,$r,$d,$e,$r|Order]) -> + case regexp:split(Order,",") of + {ok,[First,Second]} -> + CleanSecond=clean(Second), + case clean(First) of + "allow" when CleanSecond == "deny" -> + ets:insert(ConfigDB,{{directory,Directory},{"order",allow_deny}}), + parse_directory(Stream,ConfigDB,Directory); + "deny" when CleanSecond == "allow" -> + ets:insert(ConfigDB,{{directory,Directory},{"order",deny_allow}}), + parse_directory(Stream,ConfigDB,Directory); + _ -> + {error,atom_to_list(?MODULE)++": "++clean(Order)++ + " is an invalid order!"} + end; + {ok,_} -> + {error,atom_to_list(?MODULE)++": "++clean(Order)++ + " is an invalid order!"} + end; + +%% Redirect +parse(Stream,ConfigDB,[$R,$e,$d,$i,$r,$e,$c,$t|Redirect]) -> + case regexp:split(Redirect," ") of + {ok,[FakeName,URL]} -> + ets:insert(ConfigDB,{"Redirect",{FakeName,URL}}), + parse(Stream,ConfigDB); + {ok,_} -> + {error,atom_to_list(?MODULE)++": "++clean(Redirect)++ + " is an invalid Redirect!"} + end; + +%% deny +parse_directory(Stream,ConfigDB,Directory,[$d,$e,$n,$y|Deny]) -> + case regexp:split(Deny," ") of + {ok,["from","all"]} -> + ets:insert(ConfigDB,{{directory,Directory},{"deny",all}}), + parse_directory(Stream,ConfigDB,Directory); + {ok,["from"|Hosts]} -> + ets:insert(ConfigDB,{{directory,Directory},{"deny",Hosts}}), + parse_directory(Stream,ConfigDB,Directory); + {ok,_} -> + {error,atom_to_list(?MODULE)++": "++clean(Deny)++ + " is an invalid deny!"} + end; diff --git a/lib/inets/doc/archive/rfc1123.txt b/lib/inets/doc/archive/rfc1123.txt new file mode 100644 index 0000000000..4918137c88 --- /dev/null +++ b/lib/inets/doc/archive/rfc1123.txt @@ -0,0 +1,5782 @@ + + + + + + +Network Working Group Internet Engineering Task Force +Request for Comments: 1123 R. Braden, Editor + October 1989 + + + Requirements for Internet Hosts -- Application and Support + +Status of This Memo + + This RFC is an official specification for the Internet community. It + incorporates by reference, amends, corrects, and supplements the + primary protocol standards documents relating to hosts. Distribution + of this document is unlimited. + +Summary + + This RFC is one of a pair that defines and discusses the requirements + for Internet host software. This RFC covers the application and + support protocols; its companion RFC-1122 covers the communication + protocol layers: link layer, IP layer, and transport layer. + + + + Table of Contents + + + + + 1. INTRODUCTION ............................................... 5 + 1.1 The Internet Architecture .............................. 6 + 1.2 General Considerations ................................. 6 + 1.2.1 Continuing Internet Evolution ..................... 6 + 1.2.2 Robustness Principle .............................. 7 + 1.2.3 Error Logging ..................................... 8 + 1.2.4 Configuration ..................................... 8 + 1.3 Reading this Document .................................. 10 + 1.3.1 Organization ...................................... 10 + 1.3.2 Requirements ...................................... 10 + 1.3.3 Terminology ....................................... 11 + 1.4 Acknowledgments ........................................ 12 + + 2. GENERAL ISSUES ............................................. 13 + 2.1 Host Names and Numbers ................................. 13 + 2.2 Using Domain Name Service .............................. 13 + 2.3 Applications on Multihomed hosts ....................... 14 + 2.4 Type-of-Service ........................................ 14 + 2.5 GENERAL APPLICATION REQUIREMENTS SUMMARY ............... 15 + + + + +Internet Engineering Task Force [Page 1] + + + + +RFC1123 INTRODUCTION October 1989 + + + 3. REMOTE LOGIN -- TELNET PROTOCOL ............................ 16 + 3.1 INTRODUCTION ........................................... 16 + 3.2 PROTOCOL WALK-THROUGH .................................. 16 + 3.2.1 Option Negotiation ................................ 16 + 3.2.2 Telnet Go-Ahead Function .......................... 16 + 3.2.3 Control Functions ................................. 17 + 3.2.4 Telnet "Synch" Signal ............................. 18 + 3.2.5 NVT Printer and Keyboard .......................... 19 + 3.2.6 Telnet Command Structure .......................... 20 + 3.2.7 Telnet Binary Option .............................. 20 + 3.2.8 Telnet Terminal-Type Option ....................... 20 + 3.3 SPECIFIC ISSUES ........................................ 21 + 3.3.1 Telnet End-of-Line Convention ..................... 21 + 3.3.2 Data Entry Terminals .............................. 23 + 3.3.3 Option Requirements ............................... 24 + 3.3.4 Option Initiation ................................. 24 + 3.3.5 Telnet Linemode Option ............................ 25 + 3.4 TELNET/USER INTERFACE .................................. 25 + 3.4.1 Character Set Transparency ........................ 25 + 3.4.2 Telnet Commands ................................... 26 + 3.4.3 TCP Connection Errors ............................. 26 + 3.4.4 Non-Default Telnet Contact Port ................... 26 + 3.4.5 Flushing Output ................................... 26 + 3.5. TELNET REQUIREMENTS SUMMARY ........................... 27 + + 4. FILE TRANSFER .............................................. 29 + 4.1 FILE TRANSFER PROTOCOL -- FTP .......................... 29 + 4.1.1 INTRODUCTION ...................................... 29 + 4.1.2. PROTOCOL WALK-THROUGH ............................ 29 + 4.1.2.1 LOCAL Type ................................... 29 + 4.1.2.2 Telnet Format Control ........................ 30 + 4.1.2.3 Page Structure ............................... 30 + 4.1.2.4 Data Structure Transformations ............... 30 + 4.1.2.5 Data Connection Management ................... 31 + 4.1.2.6 PASV Command ................................. 31 + 4.1.2.7 LIST and NLST Commands ....................... 31 + 4.1.2.8 SITE Command ................................. 32 + 4.1.2.9 STOU Command ................................. 32 + 4.1.2.10 Telnet End-of-line Code ..................... 32 + 4.1.2.11 FTP Replies ................................. 33 + 4.1.2.12 Connections ................................. 34 + 4.1.2.13 Minimum Implementation; RFC-959 Section ..... 34 + 4.1.3 SPECIFIC ISSUES ................................... 35 + 4.1.3.1 Non-standard Command Verbs ................... 35 + 4.1.3.2 Idle Timeout ................................. 36 + 4.1.3.3 Concurrency of Data and Control .............. 36 + 4.1.3.4 FTP Restart Mechanism ........................ 36 + 4.1.4 FTP/USER INTERFACE ................................ 39 + + + +Internet Engineering Task Force [Page 2] + + + + +RFC1123 INTRODUCTION October 1989 + + + 4.1.4.1 Pathname Specification ....................... 39 + 4.1.4.2 "QUOTE" Command .............................. 40 + 4.1.4.3 Displaying Replies to User ................... 40 + 4.1.4.4 Maintaining Synchronization .................. 40 + 4.1.5 FTP REQUIREMENTS SUMMARY ......................... 41 + 4.2 TRIVIAL FILE TRANSFER PROTOCOL -- TFTP ................. 44 + 4.2.1 INTRODUCTION ...................................... 44 + 4.2.2 PROTOCOL WALK-THROUGH ............................. 44 + 4.2.2.1 Transfer Modes ............................... 44 + 4.2.2.2 UDP Header ................................... 44 + 4.2.3 SPECIFIC ISSUES ................................... 44 + 4.2.3.1 Sorcerer's Apprentice Syndrome ............... 44 + 4.2.3.2 Timeout Algorithms ........................... 46 + 4.2.3.3 Extensions ................................... 46 + 4.2.3.4 Access Control ............................... 46 + 4.2.3.5 Broadcast Request ............................ 46 + 4.2.4 TFTP REQUIREMENTS SUMMARY ......................... 47 + + 5. ELECTRONIC MAIL -- SMTP and RFC-822 ........................ 48 + 5.1 INTRODUCTION ........................................... 48 + 5.2 PROTOCOL WALK-THROUGH .................................. 48 + 5.2.1 The SMTP Model .................................... 48 + 5.2.2 Canonicalization .................................. 49 + 5.2.3 VRFY and EXPN Commands ............................ 50 + 5.2.4 SEND, SOML, and SAML Commands ..................... 50 + 5.2.5 HELO Command ...................................... 50 + 5.2.6 Mail Relay ........................................ 51 + 5.2.7 RCPT Command ...................................... 52 + 5.2.8 DATA Command ...................................... 53 + 5.2.9 Command Syntax .................................... 54 + 5.2.10 SMTP Replies ..................................... 54 + 5.2.11 Transparency ..................................... 55 + 5.2.12 WKS Use in MX Processing ......................... 55 + 5.2.13 RFC-822 Message Specification .................... 55 + 5.2.14 RFC-822 Date and Time Specification .............. 55 + 5.2.15 RFC-822 Syntax Change ............................ 56 + 5.2.16 RFC-822 Local-part .............................. 56 + 5.2.17 Domain Literals .................................. 57 + 5.2.18 Common Address Formatting Errors ................. 58 + 5.2.19 Explicit Source Routes ........................... 58 + 5.3 SPECIFIC ISSUES ........................................ 59 + 5.3.1 SMTP Queueing Strategies .......................... 59 + 5.3.1.1 Sending Strategy .............................. 59 + 5.3.1.2 Receiving strategy ........................... 61 + 5.3.2 Timeouts in SMTP .................................. 61 + 5.3.3 Reliable Mail Receipt ............................. 63 + 5.3.4 Reliable Mail Transmission ........................ 63 + 5.3.5 Domain Name Support ............................... 65 + + + +Internet Engineering Task Force [Page 3] + + + + +RFC1123 INTRODUCTION October 1989 + + + 5.3.6 Mailing Lists and Aliases ......................... 65 + 5.3.7 Mail Gatewaying ................................... 66 + 5.3.8 Maximum Message Size .............................. 68 + 5.4 SMTP REQUIREMENTS SUMMARY .............................. 69 + + 6. SUPPORT SERVICES ............................................ 72 + 6.1 DOMAIN NAME TRANSLATION ................................. 72 + 6.1.1 INTRODUCTION ....................................... 72 + 6.1.2 PROTOCOL WALK-THROUGH ............................. 72 + 6.1.2.1 Resource Records with Zero TTL ............... 73 + 6.1.2.2 QCLASS Values ................................ 73 + 6.1.2.3 Unused Fields ................................ 73 + 6.1.2.4 Compression .................................. 73 + 6.1.2.5 Misusing Configuration Info .................. 73 + 6.1.3 SPECIFIC ISSUES ................................... 74 + 6.1.3.1 Resolver Implementation ...................... 74 + 6.1.3.2 Transport Protocols .......................... 75 + 6.1.3.3 Efficient Resource Usage ..................... 77 + 6.1.3.4 Multihomed Hosts ............................. 78 + 6.1.3.5 Extensibility ................................ 79 + 6.1.3.6 Status of RR Types ........................... 79 + 6.1.3.7 Robustness ................................... 80 + 6.1.3.8 Local Host Table ............................. 80 + 6.1.4 DNS USER INTERFACE ................................ 81 + 6.1.4.1 DNS Administration ........................... 81 + 6.1.4.2 DNS User Interface ........................... 81 + 6.1.4.3 Interface Abbreviation Facilities ............. 82 + 6.1.5 DOMAIN NAME SYSTEM REQUIREMENTS SUMMARY ........... 84 + 6.2 HOST INITIALIZATION .................................... 87 + 6.2.1 INTRODUCTION ...................................... 87 + 6.2.2 REQUIREMENTS ...................................... 87 + 6.2.2.1 Dynamic Configuration ........................ 87 + 6.2.2.2 Loading Phase ................................ 89 + 6.3 REMOTE MANAGEMENT ...................................... 90 + 6.3.1 INTRODUCTION ...................................... 90 + 6.3.2 PROTOCOL WALK-THROUGH ............................. 90 + 6.3.3 MANAGEMENT REQUIREMENTS SUMMARY ................... 92 + + 7. REFERENCES ................................................. 93 + + + + + + + + + + + + +Internet Engineering Task Force [Page 4] + + + + +RFC1123 INTRODUCTION October 1989 + + +1. INTRODUCTION + + This document is one of a pair that defines and discusses the + requirements for host system implementations of the Internet protocol + suite. This RFC covers the applications layer and support protocols. + Its companion RFC, "Requirements for Internet Hosts -- Communications + Layers" [INTRO:1] covers the lower layer protocols: transport layer, + IP layer, and link layer. + + These documents are intended to provide guidance for vendors, + implementors, and users of Internet communication software. They + represent the consensus of a large body of technical experience and + wisdom, contributed by members of the Internet research and vendor + communities. + + This RFC enumerates standard protocols that a host connected to the + Internet must use, and it incorporates by reference the RFCs and + other documents describing the current specifications for these + protocols. It corrects errors in the referenced documents and adds + additional discussion and guidance for an implementor. + + For each protocol, this document also contains an explicit set of + requirements, recommendations, and options. The reader must + understand that the list of requirements in this document is + incomplete by itself; the complete set of requirements for an + Internet host is primarily defined in the standard protocol + specification documents, with the corrections, amendments, and + supplements contained in this RFC. + + A good-faith implementation of the protocols that was produced after + careful reading of the RFC's and with some interaction with the + Internet technical community, and that followed good communications + software engineering practices, should differ from the requirements + of this document in only minor ways. Thus, in many cases, the + "requirements" in this RFC are already stated or implied in the + standard protocol documents, so that their inclusion here is, in a + sense, redundant. However, they were included because some past + implementation has made the wrong choice, causing problems of + interoperability, performance, and/or robustness. + + This document includes discussion and explanation of many of the + requirements and recommendations. A simple list of requirements + would be dangerous, because: + + o Some required features are more important than others, and some + features are optional. + + o There may be valid reasons why particular vendor products that + + + +Internet Engineering Task Force [Page 5] + + + + +RFC1123 INTRODUCTION October 1989 + + + are designed for restricted contexts might choose to use + different specifications. + + However, the specifications of this document must be followed to meet + the general goal of arbitrary host interoperation across the + diversity and complexity of the Internet system. Although most + current implementations fail to meet these requirements in various + ways, some minor and some major, this specification is the ideal + towards which we need to move. + + These requirements are based on the current level of Internet + architecture. This document will be updated as required to provide + additional clarifications or to include additional information in + those areas in which specifications are still evolving. + + This introductory section begins with general advice to host software + vendors, and then gives some guidance on reading the rest of the + document. Section 2 contains general requirements that may be + applicable to all application and support protocols. Sections 3, 4, + and 5 contain the requirements on protocols for the three major + applications: Telnet, file transfer, and electronic mail, + respectively. Section 6 covers the support applications: the domain + name system, system initialization, and management. Finally, all + references will be found in Section 7. + + 1.1 The Internet Architecture + + For a brief introduction to the Internet architecture from a host + viewpoint, see Section 1.1 of [INTRO:1]. That section also + contains recommended references for general background on the + Internet architecture. + + 1.2 General Considerations + + There are two important lessons that vendors of Internet host + software have learned and which a new vendor should consider + seriously. + + 1.2.1 Continuing Internet Evolution + + The enormous growth of the Internet has revealed problems of + management and scaling in a large datagram-based packet + communication system. These problems are being addressed, and + as a result there will be continuing evolution of the + specifications described in this document. These changes will + be carefully planned and controlled, since there is extensive + participation in this planning by the vendors and by the + organizations responsible for operations of the networks. + + + +Internet Engineering Task Force [Page 6] + + + + +RFC1123 INTRODUCTION October 1989 + + + Development, evolution, and revision are characteristic of + computer network protocols today, and this situation will + persist for some years. A vendor who develops computer + communication software for the Internet protocol suite (or any + other protocol suite!) and then fails to maintain and update + that software for changing specifications is going to leave a + trail of unhappy customers. The Internet is a large + communication network, and the users are in constant contact + through it. Experience has shown that knowledge of + deficiencies in vendor software propagates quickly through the + Internet technical community. + + 1.2.2 Robustness Principle + + At every layer of the protocols, there is a general rule whose + application can lead to enormous benefits in robustness and + interoperability: + + "Be liberal in what you accept, and + conservative in what you send" + + Software should be written to deal with every conceivable + error, no matter how unlikely; sooner or later a packet will + come in with that particular combination of errors and + attributes, and unless the software is prepared, chaos can + ensue. In general, it is best to assume that the network is + filled with malevolent entities that will send in packets + designed to have the worst possible effect. This assumption + will lead to suitable protective design, although the most + serious problems in the Internet have been caused by + unenvisaged mechanisms triggered by low-probability events; + mere human malice would never have taken so devious a course! + + Adaptability to change must be designed into all levels of + Internet host software. As a simple example, consider a + protocol specification that contains an enumeration of values + for a particular header field -- e.g., a type field, a port + number, or an error code; this enumeration must be assumed to + be incomplete. Thus, if a protocol specification defines four + possible error codes, the software must not break when a fifth + code shows up. An undefined code might be logged (see below), + but it must not cause a failure. + + The second part of the principle is almost as important: + software on other hosts may contain deficiencies that make it + unwise to exploit legal but obscure protocol features. It is + unwise to stray far from the obvious and simple, lest untoward + effects result elsewhere. A corollary of this is "watch out + + + +Internet Engineering Task Force [Page 7] + + + + +RFC1123 INTRODUCTION October 1989 + + + for misbehaving hosts"; host software should be prepared, not + just to survive other misbehaving hosts, but also to cooperate + to limit the amount of disruption such hosts can cause to the + shared communication facility. + + 1.2.3 Error Logging + + The Internet includes a great variety of host and gateway + systems, each implementing many protocols and protocol layers, + and some of these contain bugs and mis-features in their + Internet protocol software. As a result of complexity, + diversity, and distribution of function, the diagnosis of user + problems is often very difficult. + + Problem diagnosis will be aided if host implementations include + a carefully designed facility for logging erroneous or + "strange" protocol events. It is important to include as much + diagnostic information as possible when an error is logged. In + particular, it is often useful to record the header(s) of a + packet that caused an error. However, care must be taken to + ensure that error logging does not consume prohibitive amounts + of resources or otherwise interfere with the operation of the + host. + + There is a tendency for abnormal but harmless protocol events + to overflow error logging files; this can be avoided by using a + "circular" log, or by enabling logging only while diagnosing a + known failure. It may be useful to filter and count duplicate + successive messages. One strategy that seems to work well is: + (1) always count abnormalities and make such counts accessible + through the management protocol (see Section 6.3); and (2) + allow the logging of a great variety of events to be + selectively enabled. For example, it might useful to be able + to "log everything" or to "log everything for host X". + + Note that different managements may have differing policies + about the amount of error logging that they want normally + enabled in a host. Some will say, "if it doesn't hurt me, I + don't want to know about it", while others will want to take a + more watchful and aggressive attitude about detecting and + removing protocol abnormalities. + + 1.2.4 Configuration + + It would be ideal if a host implementation of the Internet + protocol suite could be entirely self-configuring. This would + allow the whole suite to be implemented in ROM or cast into + silicon, it would simplify diskless workstations, and it would + + + +Internet Engineering Task Force [Page 8] + + + + +RFC1123 INTRODUCTION October 1989 + + + be an immense boon to harried LAN administrators as well as + system vendors. We have not reached this ideal; in fact, we + are not even close. + + At many points in this document, you will find a requirement + that a parameter be a configurable option. There are several + different reasons behind such requirements. In a few cases, + there is current uncertainty or disagreement about the best + value, and it may be necessary to update the recommended value + in the future. In other cases, the value really depends on + external factors -- e.g., the size of the host and the + distribution of its communication load, or the speeds and + topology of nearby networks -- and self-tuning algorithms are + unavailable and may be insufficient. In some cases, + configurability is needed because of administrative + requirements. + + Finally, some configuration options are required to communicate + with obsolete or incorrect implementations of the protocols, + distributed without sources, that unfortunately persist in many + parts of the Internet. To make correct systems coexist with + these faulty systems, administrators often have to "mis- + configure" the correct systems. This problem will correct + itself gradually as the faulty systems are retired, but it + cannot be ignored by vendors. + + When we say that a parameter must be configurable, we do not + intend to require that its value be explicitly read from a + configuration file at every boot time. We recommend that + implementors set up a default for each parameter, so a + configuration file is only necessary to override those defaults + that are inappropriate in a particular installation. Thus, the + configurability requirement is an assurance that it will be + POSSIBLE to override the default when necessary, even in a + binary-only or ROM-based product. + + This document requires a particular value for such defaults in + some cases. The choice of default is a sensitive issue when + the configuration item controls the accommodation to existing + faulty systems. If the Internet is to converge successfully to + complete interoperability, the default values built into + implementations must implement the official protocol, not + "mis-configurations" to accommodate faulty implementations. + Although marketing considerations have led some vendors to + choose mis-configuration defaults, we urge vendors to choose + defaults that will conform to the standard. + + Finally, we note that a vendor needs to provide adequate + + + +Internet Engineering Task Force [Page 9] + + + + +RFC1123 INTRODUCTION October 1989 + + + documentation on all configuration parameters, their limits and + effects. + + + 1.3 Reading this Document + + 1.3.1 Organization + + In general, each major section is organized into the following + subsections: + + (1) Introduction + + (2) Protocol Walk-Through -- considers the protocol + specification documents section-by-section, correcting + errors, stating requirements that may be ambiguous or + ill-defined, and providing further clarification or + explanation. + + (3) Specific Issues -- discusses protocol design and + implementation issues that were not included in the walk- + through. + + (4) Interfaces -- discusses the service interface to the next + higher layer. + + (5) Summary -- contains a summary of the requirements of the + section. + + Under many of the individual topics in this document, there is + parenthetical material labeled "DISCUSSION" or + "IMPLEMENTATION". This material is intended to give + clarification and explanation of the preceding requirements + text. It also includes some suggestions on possible future + directions or developments. The implementation material + contains suggested approaches that an implementor may want to + consider. + + The summary sections are intended to be guides and indexes to + the text, but are necessarily cryptic and incomplete. The + summaries should never be used or referenced separately from + the complete RFC. + + 1.3.2 Requirements + + In this document, the words that are used to define the + significance of each particular requirement are capitalized. + These words are: + + + +Internet Engineering Task Force [Page 10] + + + + +RFC1123 INTRODUCTION October 1989 + + + * "MUST" + + This word or the adjective "REQUIRED" means that the item + is an absolute requirement of the specification. + + * "SHOULD" + + This word or the adjective "RECOMMENDED" means that there + may exist valid reasons in particular circumstances to + ignore this item, but the full implications should be + understood and the case carefully weighed before choosing + a different course. + + * "MAY" + + This word or the adjective "OPTIONAL" means that this item + is truly optional. One vendor may choose to include the + item because a particular marketplace requires it or + because it enhances the product, for example; another + vendor may omit the same item. + + + An implementation is not compliant if it fails to satisfy one + or more of the MUST requirements for the protocols it + implements. An implementation that satisfies all the MUST and + all the SHOULD requirements for its protocols is said to be + "unconditionally compliant"; one that satisfies all the MUST + requirements but not all the SHOULD requirements for its + protocols is said to be "conditionally compliant". + + 1.3.3 Terminology + + This document uses the following technical terms: + + Segment + A segment is the unit of end-to-end transmission in the + TCP protocol. A segment consists of a TCP header followed + by application data. A segment is transmitted by + encapsulation in an IP datagram. + + Message + This term is used by some application layer protocols + (particularly SMTP) for an application data unit. + + Datagram + A [UDP] datagram is the unit of end-to-end transmission in + the UDP protocol. + + + + +Internet Engineering Task Force [Page 11] + + + + +RFC1123 INTRODUCTION October 1989 + + + Multihomed + A host is said to be multihomed if it has multiple IP + addresses to connected networks. + + + + 1.4 Acknowledgments + + This document incorporates contributions and comments from a large + group of Internet protocol experts, including representatives of + university and research labs, vendors, and government agencies. + It was assembled primarily by the Host Requirements Working Group + of the Internet Engineering Task Force (IETF). + + The Editor would especially like to acknowledge the tireless + dedication of the following people, who attended many long + meetings and generated 3 million bytes of electronic mail over the + past 18 months in pursuit of this document: Philip Almquist, Dave + Borman (Cray Research), Noel Chiappa, Dave Crocker (DEC), Steve + Deering (Stanford), Mike Karels (Berkeley), Phil Karn (Bellcore), + John Lekashman (NASA), Charles Lynn (BBN), Keith McCloghrie (TWG), + Paul Mockapetris (ISI), Thomas Narten (Purdue), Craig Partridge + (BBN), Drew Perkins (CMU), and James Van Bokkelen (FTP Software). + + In addition, the following people made major contributions to the + effort: Bill Barns (Mitre), Steve Bellovin (AT&T), Mike Brescia + (BBN), Ed Cain (DCA), Annette DeSchon (ISI), Martin Gross (DCA), + Phill Gross (NRI), Charles Hedrick (Rutgers), Van Jacobson (LBL), + John Klensin (MIT), Mark Lottor (SRI), Milo Medin (NASA), Bill + Melohn (Sun Microsystems), Greg Minshall (Kinetics), Jeff Mogul + (DEC), John Mullen (CMC), Jon Postel (ISI), John Romkey (Epilogue + Technology), and Mike StJohns (DCA). The following also made + significant contributions to particular areas: Eric Allman + (Berkeley), Rob Austein (MIT), Art Berggreen (ACC), Keith Bostic + (Berkeley), Vint Cerf (NRI), Wayne Hathaway (NASA), Matt Korn + (IBM), Erik Naggum (Naggum Software, Norway), Robert Ullmann + (Prime Computer), David Waitzman (BBN), Frank Wancho (USA), Arun + Welch (Ohio State), Bill Westfield (Cisco), and Rayan Zachariassen + (Toronto). + + We are grateful to all, including any contributors who may have + been inadvertently omitted from this list. + + + + + + + + + +Internet Engineering Task Force [Page 12] + + + + +RFC1123 APPLICATIONS LAYER -- GENERAL October 1989 + + +2. GENERAL ISSUES + + This section contains general requirements that may be applicable to + all application-layer protocols. + + 2.1 Host Names and Numbers + + The syntax of a legal Internet host name was specified in RFC-952 + [DNS:4]. One aspect of host name syntax is hereby changed: the + restriction on the first character is relaxed to allow either a + letter or a digit. Host software MUST support this more liberal + syntax. + + Host software MUST handle host names of up to 63 characters and + SHOULD handle host names of up to 255 characters. + + Whenever a user inputs the identity of an Internet host, it SHOULD + be possible to enter either (1) a host domain name or (2) an IP + address in dotted-decimal ("#.#.#.#") form. The host SHOULD check + the string syntactically for a dotted-decimal number before + looking it up in the Domain Name System. + + DISCUSSION: + This last requirement is not intended to specify the complete + syntactic form for entering a dotted-decimal host number; + that is considered to be a user-interface issue. For + example, a dotted-decimal number must be enclosed within + "[ ]" brackets for SMTP mail (see Section 5.2.17). This + notation could be made universal within a host system, + simplifying the syntactic checking for a dotted-decimal + number. + + If a dotted-decimal number can be entered without such + identifying delimiters, then a full syntactic check must be + made, because a segment of a host domain name is now allowed + to begin with a digit and could legally be entirely numeric + (see Section 6.1.2.4). However, a valid host name can never + have the dotted-decimal form #.#.#.#, since at least the + highest-level component label will be alphabetic. + + 2.2 Using Domain Name Service + + Host domain names MUST be translated to IP addresses as described + in Section 6.1. + + Applications using domain name services MUST be able to cope with + soft error conditions. Applications MUST wait a reasonable + interval between successive retries due to a soft error, and MUST + + + +Internet Engineering Task Force [Page 13] + + + + +RFC1123 APPLICATIONS LAYER -- GENERAL October 1989 + + + allow for the possibility that network problems may deny service + for hours or even days. + + An application SHOULD NOT rely on the ability to locate a WKS + record containing an accurate listing of all services at a + particular host address, since the WKS RR type is not often used + by Internet sites. To confirm that a service is present, simply + attempt to use it. + + 2.3 Applications on Multihomed hosts + + When the remote host is multihomed, the name-to-address + translation will return a list of alternative IP addresses. As + specified in Section 6.1.3.4, this list should be in order of + decreasing preference. Application protocol implementations + SHOULD be prepared to try multiple addresses from the list until + success is obtained. More specific requirements for SMTP are + given in Section 5.3.4. + + When the local host is multihomed, a UDP-based request/response + application SHOULD send the response with an IP source address + that is the same as the specific destination address of the UDP + request datagram. The "specific destination address" is defined + in the "IP Addressing" section of the companion RFC [INTRO:1]. + + Similarly, a server application that opens multiple TCP + connections to the same client SHOULD use the same local IP + address for all. + + 2.4 Type-of-Service + + Applications MUST select appropriate TOS values when they invoke + transport layer services, and these values MUST be configurable. + Note that a TOS value contains 5 bits, of which only the most- + significant 3 bits are currently defined; the other two bits MUST + be zero. + + DISCUSSION: + As gateway algorithms are developed to implement Type-of- + Service, the recommended values for various application + protocols may change. In addition, it is likely that + particular combinations of users and Internet paths will want + non-standard TOS values. For these reasons, the TOS values + must be configurable. + + See the latest version of the "Assigned Numbers" RFC + [INTRO:5] for the recommended TOS values for the major + application protocols. + + + +Internet Engineering Task Force [Page 14] + + + + +RFC1123 APPLICATIONS LAYER -- GENERAL October 1989 + + + 2.5 GENERAL APPLICATION REQUIREMENTS SUMMARY + + | | | | |S| | + | | | | |H| |F + | | | | |O|M|o + | | |S| |U|U|o + | | |H| |L|S|t + | |M|O| |D|T|n + | |U|U|M| | |o + | |S|L|A|N|N|t + | |T|D|Y|O|O|t +FEATURE |SECTION | | | |T|T|e +-----------------------------------------------|----------|-|-|-|-|-|-- + | | | | | | | +User interfaces: | | | | | | | + Allow host name to begin with digit |2.1 |x| | | | | + Host names of up to 635 characters |2.1 |x| | | | | + Host names of up to 255 characters |2.1 | |x| | | | + Support dotted-decimal host numbers |2.1 | |x| | | | + Check syntactically for dotted-dec first |2.1 | |x| | | | + | | | | | | | +Map domain names per Section 6.1 |2.2 |x| | | | | +Cope with soft DNS errors |2.2 |x| | | | | + Reasonable interval between retries |2.2 |x| | | | | + Allow for long outages |2.2 |x| | | | | +Expect WKS records to be available |2.2 | | | |x| | + | | | | | | | +Try multiple addr's for remote multihomed host |2.3 | |x| | | | +UDP reply src addr is specific dest of request |2.3 | |x| | | | +Use same IP addr for related TCP connections |2.3 | |x| | | | +Specify appropriate TOS values |2.4 |x| | | | | + TOS values configurable |2.4 |x| | | | | + Unused TOS bits zero |2.4 |x| | | | | + | | | | | | | + | | | | | | | + + + + + + + + + + + + + + + + +Internet Engineering Task Force [Page 15] + + + + +RFC1123 REMOTE LOGIN -- TELNET October 1989 + + +3. REMOTE LOGIN -- TELNET PROTOCOL + + 3.1 INTRODUCTION + + Telnet is the standard Internet application protocol for remote + login. It provides the encoding rules to link a user's + keyboard/display on a client ("user") system with a command + interpreter on a remote server system. A subset of the Telnet + protocol is also incorporated within other application protocols, + e.g., FTP and SMTP. + + Telnet uses a single TCP connection, and its normal data stream + ("Network Virtual Terminal" or "NVT" mode) is 7-bit ASCII with + escape sequences to embed control functions. Telnet also allows + the negotiation of many optional modes and functions. + + The primary Telnet specification is to be found in RFC-854 + [TELNET:1], while the options are defined in many other RFCs; see + Section 7 for references. + + 3.2 PROTOCOL WALK-THROUGH + + 3.2.1 Option Negotiation: RFC-854, pp. 2-3 + + Every Telnet implementation MUST include option negotiation and + subnegotiation machinery [TELNET:2]. + + A host MUST carefully follow the rules of RFC-854 to avoid + option-negotiation loops. A host MUST refuse (i.e, reply + WONT/DONT to a DO/WILL) an unsupported option. Option + negotiation SHOULD continue to function (even if all requests + are refused) throughout the lifetime of a Telnet connection. + + If all option negotiations fail, a Telnet implementation MUST + default to, and support, an NVT. + + DISCUSSION: + Even though more sophisticated "terminals" and supporting + option negotiations are becoming the norm, all + implementations must be prepared to support an NVT for any + user-server communication. + + 3.2.2 Telnet Go-Ahead Function: RFC-854, p. 5, and RFC-858 + + On a host that never sends the Telnet command Go Ahead (GA), + the Telnet Server MUST attempt to negotiate the Suppress Go + Ahead option (i.e., send "WILL Suppress Go Ahead"). A User or + Server Telnet MUST always accept negotiation of the Suppress Go + + + +Internet Engineering Task Force [Page 16] + + + + +RFC1123 REMOTE LOGIN -- TELNET October 1989 + + + Ahead option. + + When it is driving a full-duplex terminal for which GA has no + meaning, a User Telnet implementation MAY ignore GA commands. + + DISCUSSION: + Half-duplex ("locked-keyboard") line-at-a-time terminals + for which the Go-Ahead mechanism was designed have largely + disappeared from the scene. It turned out to be difficult + to implement sending the Go-Ahead signal in many operating + systems, even some systems that support native half-duplex + terminals. The difficulty is typically that the Telnet + server code does not have access to information about + whether the user process is blocked awaiting input from + the Telnet connection, i.e., it cannot reliably determine + when to send a GA command. Therefore, most Telnet Server + hosts do not send GA commands. + + The effect of the rules in this section is to allow either + end of a Telnet connection to veto the use of GA commands. + + There is a class of half-duplex terminals that is still + commercially important: "data entry terminals," which + interact in a full-screen manner. However, supporting + data entry terminals using the Telnet protocol does not + require the Go Ahead signal; see Section 3.3.2. + + 3.2.3 Control Functions: RFC-854, pp. 7-8 + + The list of Telnet commands has been extended to include EOR + (End-of-Record), with code 239 [TELNET:9]. + + Both User and Server Telnets MAY support the control functions + EOR, EC, EL, and Break, and MUST support AO, AYT, DM, IP, NOP, + SB, and SE. + + A host MUST be able to receive and ignore any Telnet control + functions that it does not support. + + DISCUSSION: + Note that a Server Telnet is required to support the + Telnet IP (Interrupt Process) function, even if the server + host has an equivalent in-stream function (e.g., Control-C + in many systems). The Telnet IP function may be stronger + than an in-stream interrupt command, because of the out- + of-band effect of TCP urgent data. + + The EOR control function may be used to delimit the + + + +Internet Engineering Task Force [Page 17] + + + + +RFC1123 REMOTE LOGIN -- TELNET October 1989 + + + stream. An important application is data entry terminal + support (see Section 3.3.2). There was concern that since + EOR had not been defined in RFC-854, a host that was not + prepared to correctly ignore unknown Telnet commands might + crash if it received an EOR. To protect such hosts, the + End-of-Record option [TELNET:9] was introduced; however, a + properly implemented Telnet program will not require this + protection. + + 3.2.4 Telnet "Synch" Signal: RFC-854, pp. 8-10 + + When it receives "urgent" TCP data, a User or Server Telnet + MUST discard all data except Telnet commands until the DM (and + end of urgent) is reached. + + When it sends Telnet IP (Interrupt Process), a User Telnet + SHOULD follow it by the Telnet "Synch" sequence, i.e., send as + TCP urgent data the sequence "IAC IP IAC DM". The TCP urgent + pointer points to the DM octet. + + When it receives a Telnet IP command, a Server Telnet MAY send + a Telnet "Synch" sequence back to the user, to flush the output + stream. The choice ought to be consistent with the way the + server operating system behaves when a local user interrupts a + process. + + When it receives a Telnet AO command, a Server Telnet MUST send + a Telnet "Synch" sequence back to the user, to flush the output + stream. + + A User Telnet SHOULD have the capability of flushing output + when it sends a Telnet IP; see also Section 3.4.5. + + DISCUSSION: + There are three possible ways for a User Telnet to flush + the stream of server output data: + + (1) Send AO after IP. + + This will cause the server host to send a "flush- + buffered-output" signal to its operating system. + However, the AO may not take effect locally, i.e., + stop terminal output at the User Telnet end, until + the Server Telnet has received and processed the AO + and has sent back a "Synch". + + (2) Send DO TIMING-MARK [TELNET:7] after IP, and discard + all output locally until a WILL/WONT TIMING-MARK is + + + +Internet Engineering Task Force [Page 18] + + + + +RFC1123 REMOTE LOGIN -- TELNET October 1989 + + + received from the Server Telnet. + + Since the DO TIMING-MARK will be processed after the + IP at the server, the reply to it should be in the + right place in the output data stream. However, the + TIMING-MARK will not send a "flush buffered output" + signal to the server operating system. Whether or + not this is needed is dependent upon the server + system. + + (3) Do both. + + The best method is not entirely clear, since it must + accommodate a number of existing server hosts that do not + follow the Telnet standards in various ways. The safest + approach is probably to provide a user-controllable option + to select (1), (2), or (3). + + 3.2.5 NVT Printer and Keyboard: RFC-854, p. 11 + + In NVT mode, a Telnet SHOULD NOT send characters with the + high-order bit 1, and MUST NOT send it as a parity bit. + Implementations that pass the high-order bit to applications + SHOULD negotiate binary mode (see Section 3.2.6). + + + DISCUSSION: + Implementors should be aware that a strict reading of + RFC-854 allows a client or server expecting NVT ASCII to + ignore characters with the high-order bit set. In + general, binary mode is expected to be used for + transmission of an extended (beyond 7-bit) character set + with Telnet. + + However, there exist applications that really need an 8- + bit NVT mode, which is currently not defined, and these + existing applications do set the high-order bit during + part or all of the life of a Telnet connection. Note that + binary mode is not the same as 8-bit NVT mode, since + binary mode turns off end-of-line processing. For this + reason, the requirements on the high-order bit are stated + as SHOULD, not MUST. + + RFC-854 defines a minimal set of properties of a "network + virtual terminal" or NVT; this is not meant to preclude + additional features in a real terminal. A Telnet + connection is fully transparent to all 7-bit ASCII + characters, including arbitrary ASCII control characters. + + + +Internet Engineering Task Force [Page 19] + + + + +RFC1123 REMOTE LOGIN -- TELNET October 1989 + + + For example, a terminal might support full-screen commands + coded as ASCII escape sequences; a Telnet implementation + would pass these sequences as uninterpreted data. Thus, + an NVT should not be conceived as a terminal type of a + highly-restricted device. + + 3.2.6 Telnet Command Structure: RFC-854, p. 13 + + Since options may appear at any point in the data stream, a + Telnet escape character (known as IAC, with the value 255) to + be sent as data MUST be doubled. + + 3.2.7 Telnet Binary Option: RFC-856 + + When the Binary option has been successfully negotiated, + arbitrary 8-bit characters are allowed. However, the data + stream MUST still be scanned for IAC characters, any embedded + Telnet commands MUST be obeyed, and data bytes equal to IAC + MUST be doubled. Other character processing (e.g., replacing + CR by CR NUL or by CR LF) MUST NOT be done. In particular, + there is no end-of-line convention (see Section 3.3.1) in + binary mode. + + DISCUSSION: + The Binary option is normally negotiated in both + directions, to change the Telnet connection from NVT mode + to "binary mode". + + The sequence IAC EOR can be used to delimit blocks of data + within a binary-mode Telnet stream. + + 3.2.8 Telnet Terminal-Type Option: RFC-1091 + + The Terminal-Type option MUST use the terminal type names + officially defined in the Assigned Numbers RFC [INTRO:5], when + they are available for the particular terminal. However, the + receiver of a Terminal-Type option MUST accept any name. + + DISCUSSION: + RFC-1091 [TELNET:10] updates an earlier version of the + Terminal-Type option defined in RFC-930. The earlier + version allowed a server host capable of supporting + multiple terminal types to learn the type of a particular + client's terminal, assuming that each physical terminal + had an intrinsic type. However, today a "terminal" is + often really a terminal emulator program running in a PC, + perhaps capable of emulating a range of terminal types. + Therefore, RFC-1091 extends the specification to allow a + + + +Internet Engineering Task Force [Page 20] + + + + +RFC1123 REMOTE LOGIN -- TELNET October 1989 + + + more general terminal-type negotiation between User and + Server Telnets. + + 3.3 SPECIFIC ISSUES + + 3.3.1 Telnet End-of-Line Convention + + The Telnet protocol defines the sequence CR LF to mean "end- + of-line". For terminal input, this corresponds to a command- + completion or "end-of-line" key being pressed on a user + terminal; on an ASCII terminal, this is the CR key, but it may + also be labelled "Return" or "Enter". + + When a Server Telnet receives the Telnet end-of-line sequence + CR LF as input from a remote terminal, the effect MUST be the + same as if the user had pressed the "end-of-line" key on a + local terminal. On server hosts that use ASCII, in particular, + receipt of the Telnet sequence CR LF must cause the same effect + as a local user pressing the CR key on a local terminal. Thus, + CR LF and CR NUL MUST have the same effect on an ASCII server + host when received as input over a Telnet connection. + + A User Telnet MUST be able to send any of the forms: CR LF, CR + NUL, and LF. A User Telnet on an ASCII host SHOULD have a + user-controllable mode to send either CR LF or CR NUL when the + user presses the "end-of-line" key, and CR LF SHOULD be the + default. + + The Telnet end-of-line sequence CR LF MUST be used to send + Telnet data that is not terminal-to-computer (e.g., for Server + Telnet sending output, or the Telnet protocol incorporated + another application protocol). + + DISCUSSION: + To allow interoperability between arbitrary Telnet clients + and servers, the Telnet protocol defined a standard + representation for a line terminator. Since the ASCII + character set includes no explicit end-of-line character, + systems have chosen various representations, e.g., CR, LF, + and the sequence CR LF. The Telnet protocol chose the CR + LF sequence as the standard for network transmission. + + Unfortunately, the Telnet protocol specification in RFC- + 854 [TELNET:1] has turned out to be somewhat ambiguous on + what character(s) should be sent from client to server for + the "end-of-line" key. The result has been a massive and + continuing interoperability headache, made worse by + various faulty implementations of both User and Server + + + +Internet Engineering Task Force [Page 21] + + + + +RFC1123 REMOTE LOGIN -- TELNET October 1989 + + + Telnets. + + Although the Telnet protocol is based on a perfectly + symmetric model, in a remote login session the role of the + user at a terminal differs from the role of the server + host. For example, RFC-854 defines the meaning of CR, LF, + and CR LF as output from the server, but does not specify + what the User Telnet should send when the user presses the + "end-of-line" key on the terminal; this turns out to be + the point at issue. + + When a user presses the "end-of-line" key, some User + Telnet implementations send CR LF, while others send CR + NUL (based on a different interpretation of the same + sentence in RFC-854). These will be equivalent for a + correctly-implemented ASCII server host, as discussed + above. For other servers, a mode in the User Telnet is + needed. + + The existence of User Telnets that send only CR NUL when + CR is pressed creates a dilemma for non-ASCII hosts: they + can either treat CR NUL as equivalent to CR LF in input, + thus precluding the possibility of entering a "bare" CR, + or else lose complete interworking. + + Suppose a user on host A uses Telnet to log into a server + host B, and then execute B's User Telnet program to log + into server host C. It is desirable for the Server/User + Telnet combination on B to be as transparent as possible, + i.e., to appear as if A were connected directly to C. In + particular, correct implementation will make B transparent + to Telnet end-of-line sequences, except that CR LF may be + translated to CR NUL or vice versa. + + IMPLEMENTATION: + To understand Telnet end-of-line issues, one must have at + least a general model of the relationship of Telnet to the + local operating system. The Server Telnet process is + typically coupled into the terminal driver software of the + operating system as a pseudo-terminal. A Telnet end-of- + line sequence received by the Server Telnet must have the + same effect as pressing the end-of-line key on a real + locally-connected terminal. + + Operating systems that support interactive character-at- + a-time applications (e.g., editors) typically have two + internal modes for their terminal I/O: a formatted mode, + in which local conventions for end-of-line and other + + + +Internet Engineering Task Force [Page 22] + + + + +RFC1123 REMOTE LOGIN -- TELNET October 1989 + + + formatting rules have been applied to the data stream, and + a "raw" mode, in which the application has direct access + to every character as it was entered. A Server Telnet + must be implemented in such a way that these modes have + the same effect for remote as for local terminals. For + example, suppose a CR LF or CR NUL is received by the + Server Telnet on an ASCII host. In raw mode, a CR + character is passed to the application; in formatted mode, + the local system's end-of-line convention is used. + + 3.3.2 Data Entry Terminals + + DISCUSSION: + In addition to the line-oriented and character-oriented + ASCII terminals for which Telnet was designed, there are + several families of video display terminals that are + sometimes known as "data entry terminals" or DETs. The + IBM 3270 family is a well-known example. + + Two Internet protocols have been designed to support + generic DETs: SUPDUP [TELNET:16, TELNET:17], and the DET + option [TELNET:18, TELNET:19]. The DET option drives a + data entry terminal over a Telnet connection using (sub-) + negotiation. SUPDUP is a completely separate terminal + protocol, which can be entered from Telnet by negotiation. + Although both SUPDUP and the DET option have been used + successfully in particular environments, neither has + gained general acceptance or wide implementation. + + A different approach to DET interaction has been developed + for supporting the IBM 3270 family through Telnet, + although the same approach would be applicable to any DET. + The idea is to enter a "native DET" mode, in which the + native DET input/output stream is sent as binary data. + The Telnet EOR command is used to delimit logical records + (e.g., "screens") within this binary stream. + + IMPLEMENTATION: + The rules for entering and leaving native DET mode are as + follows: + + o The Server uses the Terminal-Type option [TELNET:10] + to learn that the client is a DET. + + o It is conventional, but not required, that both ends + negotiate the EOR option [TELNET:9]. + + o Both ends negotiate the Binary option [TELNET:3] to + + + +Internet Engineering Task Force [Page 23] + + + + +RFC1123 REMOTE LOGIN -- TELNET October 1989 + + + enter native DET mode. + + o When either end negotiates out of binary mode, the + other end does too, and the mode then reverts to + normal NVT. + + + 3.3.3 Option Requirements + + Every Telnet implementation MUST support the Binary option + [TELNET:3] and the Suppress Go Ahead option [TELNET:5], and + SHOULD support the Echo [TELNET:4], Status [TELNET:6], End-of- + Record [TELNET:9], and Extended Options List [TELNET:8] + options. + + A User or Server Telnet SHOULD support the Window Size Option + [TELNET:12] if the local operating system provides the + corresponding capability. + + DISCUSSION: + Note that the End-of-Record option only signifies that a + Telnet can receive a Telnet EOR without crashing; + therefore, every Telnet ought to be willing to accept + negotiation of the End-of-Record option. See also the + discussion in Section 3.2.3. + + 3.3.4 Option Initiation + + When the Telnet protocol is used in a client/server situation, + the server SHOULD initiate negotiation of the terminal + interaction mode it expects. + + DISCUSSION: + The Telnet protocol was defined to be perfectly + symmetrical, but its application is generally asymmetric. + Remote login has been known to fail because NEITHER side + initiated negotiation of the required non-default terminal + modes. It is generally the server that determines the + preferred mode, so the server needs to initiate the + negotiation; since the negotiation is symmetric, the user + can also initiate it. + + A client (User Telnet) SHOULD provide a means for users to + enable and disable the initiation of option negotiation. + + DISCUSSION: + A user sometimes needs to connect to an application + service (e.g., FTP or SMTP) that uses Telnet for its + + + +Internet Engineering Task Force [Page 24] + + + + +RFC1123 REMOTE LOGIN -- TELNET October 1989 + + + control stream but does not support Telnet options. User + Telnet may be used for this purpose if initiation of + option negotiation is disabled. + + 3.3.5 Telnet Linemode Option + + DISCUSSION: + An important new Telnet option, LINEMODE [TELNET:12], has + been proposed. The LINEMODE option provides a standard + way for a User Telnet and a Server Telnet to agree that + the client rather than the server will perform terminal + character processing. When the client has prepared a + complete line of text, it will send it to the server in + (usually) one TCP packet. This option will greatly + decrease the packet cost of Telnet sessions and will also + give much better user response over congested or long- + delay networks. + + The LINEMODE option allows dynamic switching between local + and remote character processing. For example, the Telnet + connection will automatically negotiate into single- + character mode while a full screen editor is running, and + then return to linemode when the editor is finished. + + We expect that when this RFC is released, hosts should + implement the client side of this option, and may + implement the server side of this option. To properly + implement the server side, the server needs to be able to + tell the local system not to do any input character + processing, but to remember its current terminal state and + notify the Server Telnet process whenever the state + changes. This will allow password echoing and full screen + editors to be handled properly, for example. + + 3.4 TELNET/USER INTERFACE + + 3.4.1 Character Set Transparency + + User Telnet implementations SHOULD be able to send or receive + any 7-bit ASCII character. Where possible, any special + character interpretations by the user host's operating system + SHOULD be bypassed so that these characters can conveniently be + sent and received on the connection. + + Some character value MUST be reserved as "escape to command + mode"; conventionally, doubling this character allows it to be + entered as data. The specific character used SHOULD be user + selectable. + + + +Internet Engineering Task Force [Page 25] + + + + +RFC1123 REMOTE LOGIN -- TELNET October 1989 + + + On binary-mode connections, a User Telnet program MAY provide + an escape mechanism for entering arbitrary 8-bit values, if the + host operating system doesn't allow them to be entered directly + from the keyboard. + + IMPLEMENTATION: + The transparency issues are less pressing on servers, but + implementors should take care in dealing with issues like: + masking off parity bits (sent by an older, non-conforming + client) before they reach programs that expect only NVT + ASCII, and properly handling programs that request 8-bit + data streams. + + 3.4.2 Telnet Commands + + A User Telnet program MUST provide a user the capability of + entering any of the Telnet control functions IP, AO, or AYT, + and SHOULD provide the capability of entering EC, EL, and + Break. + + 3.4.3 TCP Connection Errors + + A User Telnet program SHOULD report to the user any TCP errors + that are reported by the transport layer (see "TCP/Application + Layer Interface" section in [INTRO:1]). + + 3.4.4 Non-Default Telnet Contact Port + + A User Telnet program SHOULD allow the user to optionally + specify a non-standard contact port number at the Server Telnet + host. + + 3.4.5 Flushing Output + + A User Telnet program SHOULD provide the user the ability to + specify whether or not output should be flushed when an IP is + sent; see Section 3.2.4. + + For any output flushing scheme that causes the User Telnet to + flush output locally until a Telnet signal is received from the + Server, there SHOULD be a way for the user to manually restore + normal output, in case the Server fails to send the expected + signal. + + + + + + + + +Internet Engineering Task Force [Page 26] + + + + +RFC1123 REMOTE LOGIN -- TELNET October 1989 + + + 3.5. TELNET REQUIREMENTS SUMMARY + + + | | | | |S| | + | | | | |H| |F + | | | | |O|M|o + | | |S| |U|U|o + | | |H| |L|S|t + | |M|O| |D|T|n + | |U|U|M| | |o + | |S|L|A|N|N|t + | |T|D|Y|O|O|t +FEATURE |SECTION | | | |T|T|e +-------------------------------------------------|--------|-|-|-|-|-|-- + | | | | | | | +Option Negotiation |3.2.1 |x| | | | | + Avoid negotiation loops |3.2.1 |x| | | | | + Refuse unsupported options |3.2.1 |x| | | | | + Negotiation OK anytime on connection |3.2.1 | |x| | | | + Default to NVT |3.2.1 |x| | | | | + Send official name in Term-Type option |3.2.8 |x| | | | | + Accept any name in Term-Type option |3.2.8 |x| | | | | + Implement Binary, Suppress-GA options |3.3.3 |x| | | | | + Echo, Status, EOL, Ext-Opt-List options |3.3.3 | |x| | | | + Implement Window-Size option if appropriate |3.3.3 | |x| | | | + Server initiate mode negotiations |3.3.4 | |x| | | | + User can enable/disable init negotiations |3.3.4 | |x| | | | + | | | | | | | +Go-Aheads | | | | | | | + Non-GA server negotiate SUPPRESS-GA option |3.2.2 |x| | | | | + User or Server accept SUPPRESS-GA option |3.2.2 |x| | | | | + User Telnet ignore GA's |3.2.2 | | |x| | | + | | | | | | | +Control Functions | | | | | | | + Support SE NOP DM IP AO AYT SB |3.2.3 |x| | | | | + Support EOR EC EL Break |3.2.3 | | |x| | | + Ignore unsupported control functions |3.2.3 |x| | | | | + User, Server discard urgent data up to DM |3.2.4 |x| | | | | + User Telnet send "Synch" after IP, AO, AYT |3.2.4 | |x| | | | + Server Telnet reply Synch to IP |3.2.4 | | |x| | | + Server Telnet reply Synch to AO |3.2.4 |x| | | | | + User Telnet can flush output when send IP |3.2.4 | |x| | | | + | | | | | | | +Encoding | | | | | | | + Send high-order bit in NVT mode |3.2.5 | | | |x| | + Send high-order bit as parity bit |3.2.5 | | | | |x| + Negot. BINARY if pass high-ord. bit to applic |3.2.5 | |x| | | | + Always double IAC data byte |3.2.6 |x| | | | | + + + +Internet Engineering Task Force [Page 27] + + + + +RFC1123 REMOTE LOGIN -- TELNET October 1989 + + + Double IAC data byte in binary mode |3.2.7 |x| | | | | + Obey Telnet cmds in binary mode |3.2.7 |x| | | | | + End-of-line, CR NUL in binary mode |3.2.7 | | | | |x| + | | | | | | | +End-of-Line | | | | | | | + EOL at Server same as local end-of-line |3.3.1 |x| | | | | + ASCII Server accept CR LF or CR NUL for EOL |3.3.1 |x| | | | | + User Telnet able to send CR LF, CR NUL, or LF |3.3.1 |x| | | | | + ASCII user able to select CR LF/CR NUL |3.3.1 | |x| | | | + User Telnet default mode is CR LF |3.3.1 | |x| | | | + Non-interactive uses CR LF for EOL |3.3.1 |x| | | | | + | | | | | | | +User Telnet interface | | | | | | | + Input & output all 7-bit characters |3.4.1 | |x| | | | + Bypass local op sys interpretation |3.4.1 | |x| | | | + Escape character |3.4.1 |x| | | | | + User-settable escape character |3.4.1 | |x| | | | + Escape to enter 8-bit values |3.4.1 | | |x| | | + Can input IP, AO, AYT |3.4.2 |x| | | | | + Can input EC, EL, Break |3.4.2 | |x| | | | + Report TCP connection errors to user |3.4.3 | |x| | | | + Optional non-default contact port |3.4.4 | |x| | | | + Can spec: output flushed when IP sent |3.4.5 | |x| | | | + Can manually restore output mode |3.4.5 | |x| | | | + | | | | | | | + + + + + + + + + + + + + + + + + + + + + + + + + + +Internet Engineering Task Force [Page 28] + + + + +RFC1123 FILE TRANSFER -- FTP October 1989 + + +4. FILE TRANSFER + + 4.1 FILE TRANSFER PROTOCOL -- FTP + + 4.1.1 INTRODUCTION + + The File Transfer Protocol FTP is the primary Internet standard + for file transfer. The current specification is contained in + RFC-959 [FTP:1]. + + FTP uses separate simultaneous TCP connections for control and + for data transfer. The FTP protocol includes many features, + some of which are not commonly implemented. However, for every + feature in FTP, there exists at least one implementation. The + minimum implementation defined in RFC-959 was too small, so a + somewhat larger minimum implementation is defined here. + + Internet users have been unnecessarily burdened for years by + deficient FTP implementations. Protocol implementors have + suffered from the erroneous opinion that implementing FTP ought + to be a small and trivial task. This is wrong, because FTP has + a user interface, because it has to deal (correctly) with the + whole variety of communication and operating system errors that + may occur, and because it has to handle the great diversity of + real file systems in the world. + + 4.1.2. PROTOCOL WALK-THROUGH + + 4.1.2.1 LOCAL Type: RFC-959 Section 3.1.1.4 + + An FTP program MUST support TYPE I ("IMAGE" or binary type) + as well as TYPE L 8 ("LOCAL" type with logical byte size 8). + A machine whose memory is organized into m-bit words, where + m is not a multiple of 8, MAY also support TYPE L m. + + DISCUSSION: + The command "TYPE L 8" is often required to transfer + binary data between a machine whose memory is organized + into (e.g.) 36-bit words and a machine with an 8-bit + byte organization. For an 8-bit byte machine, TYPE L 8 + is equivalent to IMAGE. + + "TYPE L m" is sometimes specified to the FTP programs + on two m-bit word machines to ensure the correct + transfer of a native-mode binary file from one machine + to the other. However, this command should have the + same effect on these machines as "TYPE I". + + + + +Internet Engineering Task Force [Page 29] + + + + +RFC1123 FILE TRANSFER -- FTP October 1989 + + + 4.1.2.2 Telnet Format Control: RFC-959 Section 3.1.1.5.2 + + A host that makes no distinction between TYPE N and TYPE T + SHOULD implement TYPE T to be identical to TYPE N. + + DISCUSSION: + This provision should ease interoperation with hosts + that do make this distinction. + + Many hosts represent text files internally as strings + of ASCII characters, using the embedded ASCII format + effector characters (LF, BS, FF, ...) to control the + format when a file is printed. For such hosts, there + is no distinction between "print" files and other + files. However, systems that use record structured + files typically need a special format for printable + files (e.g., ASA carriage control). For the latter + hosts, FTP allows a choice of TYPE N or TYPE T. + + 4.1.2.3 Page Structure: RFC-959 Section 3.1.2.3 and Appendix I + + Implementation of page structure is NOT RECOMMENDED in + general. However, if a host system does need to implement + FTP for "random access" or "holey" files, it MUST use the + defined page structure format rather than define a new + private FTP format. + + 4.1.2.4 Data Structure Transformations: RFC-959 Section 3.1.2 + + An FTP transformation between record-structure and file- + structure SHOULD be invertible, to the extent possible while + making the result useful on the target host. + + DISCUSSION: + RFC-959 required strict invertibility between record- + structure and file-structure, but in practice, + efficiency and convenience often preclude it. + Therefore, the requirement is being relaxed. There are + two different objectives for transferring a file: + processing it on the target host, or just storage. For + storage, strict invertibility is important. For + processing, the file created on the target host needs + to be in the format expected by application programs on + that host. + + As an example of the conflict, imagine a record- + oriented operating system that requires some data files + to have exactly 80 bytes in each record. While STORing + + + +Internet Engineering Task Force [Page 30] + + + + +RFC1123 FILE TRANSFER -- FTP October 1989 + + + a file on such a host, an FTP Server must be able to + pad each line or record to 80 bytes; a later retrieval + of such a file cannot be strictly invertible. + + 4.1.2.5 Data Connection Management: RFC-959 Section 3.3 + + A User-FTP that uses STREAM mode SHOULD send a PORT command + to assign a non-default data port before each transfer + command is issued. + + DISCUSSION: + This is required because of the long delay after a TCP + connection is closed until its socket pair can be + reused, to allow multiple transfers during a single FTP + session. Sending a port command can avoided if a + transfer mode other than stream is used, by leaving the + data transfer connection open between transfers. + + 4.1.2.6 PASV Command: RFC-959 Section 4.1.2 + + A server-FTP MUST implement the PASV command. + + If multiple third-party transfers are to be executed during + the same session, a new PASV command MUST be issued before + each transfer command, to obtain a unique port pair. + + IMPLEMENTATION: + The format of the 227 reply to a PASV command is not + well standardized. In particular, an FTP client cannot + assume that the parentheses shown on page 40 of RFC-959 + will be present (and in fact, Figure 3 on page 43 omits + them). Therefore, a User-FTP program that interprets + the PASV reply must scan the reply for the first digit + of the host and port numbers. + + Note that the host number h1,h2,h3,h4 is the IP address + of the server host that is sending the reply, and that + p1,p2 is a non-default data transfer port that PASV has + assigned. + + 4.1.2.7 LIST and NLST Commands: RFC-959 Section 4.1.3 + + The data returned by an NLST command MUST contain only a + simple list of legal pathnames, such that the server can use + them directly as the arguments of subsequent data transfer + commands for the individual files. + + The data returned by a LIST or NLST command SHOULD use an + + + +Internet Engineering Task Force [Page 31] + + + + +RFC1123 FILE TRANSFER -- FTP October 1989 + + + implied TYPE AN, unless the current type is EBCDIC, in which + case an implied TYPE EN SHOULD be used. + + DISCUSSION: + Many FTP clients support macro-commands that will get + or put files matching a wildcard specification, using + NLST to obtain a list of pathnames. The expansion of + "multiple-put" is local to the client, but "multiple- + get" requires cooperation by the server. + + The implied type for LIST and NLST is designed to + provide compatibility with existing User-FTPs, and in + particular with multiple-get commands. + + 4.1.2.8 SITE Command: RFC-959 Section 4.1.3 + + A Server-FTP SHOULD use the SITE command for non-standard + features, rather than invent new private commands or + unstandardized extensions to existing commands. + + 4.1.2.9 STOU Command: RFC-959 Section 4.1.3 + + The STOU command stores into a uniquely named file. When it + receives an STOU command, a Server-FTP MUST return the + actual file name in the "125 Transfer Starting" or the "150 + Opening Data Connection" message that precedes the transfer + (the 250 reply code mentioned in RFC-959 is incorrect). The + exact format of these messages is hereby defined to be as + follows: + + 125 FILE: pppp + 150 FILE: pppp + + where pppp represents the unique pathname of the file that + will be written. + + 4.1.2.10 Telnet End-of-line Code: RFC-959, Page 34 + + Implementors MUST NOT assume any correspondence between READ + boundaries on the control connection and the Telnet EOL + sequences (CR LF). + + DISCUSSION: + Thus, a server-FTP (or User-FTP) must continue reading + characters from the control connection until a complete + Telnet EOL sequence is encountered, before processing + the command (or response, respectively). Conversely, a + single READ from the control connection may include + + + +Internet Engineering Task Force [Page 32] + + + + +RFC1123 FILE TRANSFER -- FTP October 1989 + + + more than one FTP command. + + 4.1.2.11 FTP Replies: RFC-959 Section 4.2, Page 35 + + A Server-FTP MUST send only correctly formatted replies on + the control connection. Note that RFC-959 (unlike earlier + versions of the FTP spec) contains no provision for a + "spontaneous" reply message. + + A Server-FTP SHOULD use the reply codes defined in RFC-959 + whenever they apply. However, a server-FTP MAY use a + different reply code when needed, as long as the general + rules of Section 4.2 are followed. When the implementor has + a choice between a 4xx and 5xx reply code, a Server-FTP + SHOULD send a 4xx (temporary failure) code when there is any + reasonable possibility that a failed FTP will succeed a few + hours later. + + A User-FTP SHOULD generally use only the highest-order digit + of a 3-digit reply code for making a procedural decision, to + prevent difficulties when a Server-FTP uses non-standard + reply codes. + + A User-FTP MUST be able to handle multi-line replies. If + the implementation imposes a limit on the number of lines + and if this limit is exceeded, the User-FTP MUST recover, + e.g., by ignoring the excess lines until the end of the + multi-line reply is reached. + + A User-FTP SHOULD NOT interpret a 421 reply code ("Service + not available, closing control connection") specially, but + SHOULD detect closing of the control connection by the + server. + + DISCUSSION: + Server implementations that fail to strictly follow the + reply rules often cause FTP user programs to hang. + Note that RFC-959 resolved ambiguities in the reply + rules found in earlier FTP specifications and must be + followed. + + It is important to choose FTP reply codes that properly + distinguish between temporary and permanent failures, + to allow the successful use of file transfer client + daemons. These programs depend on the reply codes to + decide whether or not to retry a failed transfer; using + a permanent failure code (5xx) for a temporary error + will cause these programs to give up unnecessarily. + + + +Internet Engineering Task Force [Page 33] + + + + +RFC1123 FILE TRANSFER -- FTP October 1989 + + + When the meaning of a reply matches exactly the text + shown in RFC-959, uniformity will be enhanced by using + the RFC-959 text verbatim. However, a Server-FTP + implementor is encouraged to choose reply text that + conveys specific system-dependent information, when + appropriate. + + 4.1.2.12 Connections: RFC-959 Section 5.2 + + The words "and the port used" in the second paragraph of + this section of RFC-959 are erroneous (historical), and they + should be ignored. + + On a multihomed server host, the default data transfer port + (L-1) MUST be associated with the same local IP address as + the corresponding control connection to port L. + + A user-FTP MUST NOT send any Telnet controls other than + SYNCH and IP on an FTP control connection. In particular, it + MUST NOT attempt to negotiate Telnet options on the control + connection. However, a server-FTP MUST be capable of + accepting and refusing Telnet negotiations (i.e., sending + DONT/WONT). + + DISCUSSION: + Although the RFC says: "Server- and User- processes + should follow the conventions for the Telnet + protocol...[on the control connection]", it is not the + intent that Telnet option negotiation is to be + employed. + + 4.1.2.13 Minimum Implementation; RFC-959 Section 5.1 + + The following commands and options MUST be supported by + every server-FTP and user-FTP, except in cases where the + underlying file system or operating system does not allow or + support a particular command. + + Type: ASCII Non-print, IMAGE, LOCAL 8 + Mode: Stream + Structure: File, Record* + Commands: + USER, PASS, ACCT, + PORT, PASV, + TYPE, MODE, STRU, + RETR, STOR, APPE, + RNFR, RNTO, DELE, + CWD, CDUP, RMD, MKD, PWD, + + + +Internet Engineering Task Force [Page 34] + + + + +RFC1123 FILE TRANSFER -- FTP October 1989 + + + LIST, NLST, + SYST, STAT, + HELP, NOOP, QUIT. + + *Record structure is REQUIRED only for hosts whose file + systems support record structure. + + DISCUSSION: + Vendors are encouraged to implement a larger subset of + the protocol. For example, there are important + robustness features in the protocol (e.g., Restart, + ABOR, block mode) that would be an aid to some Internet + users but are not widely implemented. + + A host that does not have record structures in its file + system may still accept files with STRU R, recording + the byte stream literally. + + 4.1.3 SPECIFIC ISSUES + + 4.1.3.1 Non-standard Command Verbs + + FTP allows "experimental" commands, whose names begin with + "X". If these commands are subsequently adopted as + standards, there may still be existing implementations using + the "X" form. At present, this is true for the directory + commands: + + RFC-959 "Experimental" + + MKD XMKD + RMD XRMD + PWD XPWD + CDUP XCUP + CWD XCWD + + All FTP implementations SHOULD recognize both forms of these + commands, by simply equating them with extra entries in the + command lookup table. + + IMPLEMENTATION: + A User-FTP can access a server that supports only the + "X" forms by implementing a mode switch, or + automatically using the following procedure: if the + RFC-959 form of one of the above commands is rejected + with a 500 or 502 response code, then try the + experimental form; any other response would be passed + to the user. + + + +Internet Engineering Task Force [Page 35] + + + + +RFC1123 FILE TRANSFER -- FTP October 1989 + + + 4.1.3.2 Idle Timeout + + A Server-FTP process SHOULD have an idle timeout, which will + terminate the process and close the control connection if + the server is inactive (i.e., no command or data transfer in + progress) for a long period of time. The idle timeout time + SHOULD be configurable, and the default should be at least 5 + minutes. + + A client FTP process ("User-PI" in RFC-959) will need + timeouts on responses only if it is invoked from a program. + + DISCUSSION: + Without a timeout, a Server-FTP process may be left + pending indefinitely if the corresponding client + crashes without closing the control connection. + + 4.1.3.3 Concurrency of Data and Control + + DISCUSSION: + The intent of the designers of FTP was that a user + should be able to send a STAT command at any time while + data transfer was in progress and that the server-FTP + would reply immediately with status -- e.g., the number + of bytes transferred so far. Similarly, an ABOR + command should be possible at any time during a data + transfer. + + Unfortunately, some small-machine operating systems + make such concurrent programming difficult, and some + other implementers seek minimal solutions, so some FTP + implementations do not allow concurrent use of the data + and control connections. Even such a minimal server + must be prepared to accept and defer a STAT or ABOR + command that arrives during data transfer. + + 4.1.3.4 FTP Restart Mechanism + + The description of the 110 reply on pp. 40-41 of RFC-959 is + incorrect; the correct description is as follows. A restart + reply message, sent over the control connection from the + receiving FTP to the User-FTP, has the format: + + 110 MARK ssss = rrrr + + Here: + + * ssss is a text string that appeared in a Restart Marker + + + +Internet Engineering Task Force [Page 36] + + + + +RFC1123 FILE TRANSFER -- FTP October 1989 + + + in the data stream and encodes a position in the + sender's file system; + + * rrrr encodes the corresponding position in the + receiver's file system. + + The encoding, which is specific to a particular file system + and network implementation, is always generated and + interpreted by the same system, either sender or receiver. + + When an FTP that implements restart receives a Restart + Marker in the data stream, it SHOULD force the data to that + point to be written to stable storage before encoding the + corresponding position rrrr. An FTP sending Restart Markers + MUST NOT assume that 110 replies will be returned + synchronously with the data, i.e., it must not await a 110 + reply before sending more data. + + Two new reply codes are hereby defined for errors + encountered in restarting a transfer: + + 554 Requested action not taken: invalid REST parameter. + + A 554 reply may result from a FTP service command that + follows a REST command. The reply indicates that the + existing file at the Server-FTP cannot be repositioned + as specified in the REST. + + 555 Requested action not taken: type or stru mismatch. + + A 555 reply may result from an APPE command or from any + FTP service command following a REST command. The + reply indicates that there is some mismatch between the + current transfer parameters (type and stru) and the + attributes of the existing file. + + DISCUSSION: + Note that the FTP Restart mechanism requires that Block + or Compressed mode be used for data transfer, to allow + the Restart Markers to be included within the data + stream. The frequency of Restart Markers can be low. + + Restart Markers mark a place in the data stream, but + the receiver may be performing some transformation on + the data as it is stored into stable storage. In + general, the receiver's encoding must include any state + information necessary to restart this transformation at + any point of the FTP data stream. For example, in TYPE + + + +Internet Engineering Task Force [Page 37] + + + + +RFC1123 FILE TRANSFER -- FTP October 1989 + + + A transfers, some receiver hosts transform CR LF + sequences into a single LF character on disk. If a + Restart Marker happens to fall between CR and LF, the + receiver must encode in rrrr that the transfer must be + restarted in a "CR has been seen and discarded" state. + + Note that the Restart Marker is required to be encoded + as a string of printable ASCII characters, regardless + of the type of the data. + + RFC-959 says that restart information is to be returned + "to the user". This should not be taken literally. In + general, the User-FTP should save the restart + information (ssss,rrrr) in stable storage, e.g., append + it to a restart control file. An empty restart control + file should be created when the transfer first starts + and deleted automatically when the transfer completes + successfully. It is suggested that this file have a + name derived in an easily-identifiable manner from the + name of the file being transferred and the remote host + name; this is analogous to the means used by many text + editors for naming "backup" files. + + There are three cases for FTP restart. + + (1) User-to-Server Transfer + + The User-FTP puts Restart Markers <ssss> at + convenient places in the data stream. When the + Server-FTP receives a Marker, it writes all prior + data to disk, encodes its file system position and + transformation state as rrrr, and returns a "110 + MARK ssss = rrrr" reply over the control + connection. The User-FTP appends the pair + (ssss,rrrr) to its restart control file. + + To restart the transfer, the User-FTP fetches the + last (ssss,rrrr) pair from the restart control + file, repositions its local file system and + transformation state using ssss, and sends the + command "REST rrrr" to the Server-FTP. + + (2) Server-to-User Transfer + + The Server-FTP puts Restart Markers <ssss> at + convenient places in the data stream. When the + User-FTP receives a Marker, it writes all prior + data to disk, encodes its file system position and + + + +Internet Engineering Task Force [Page 38] + + + + +RFC1123 FILE TRANSFER -- FTP October 1989 + + + transformation state as rrrr, and appends the pair + (rrrr,ssss) to its restart control file. + + To restart the transfer, the User-FTP fetches the + last (rrrr,ssss) pair from the restart control + file, repositions its local file system and + transformation state using rrrr, and sends the + command "REST ssss" to the Server-FTP. + + (3) Server-to-Server ("Third-Party") Transfer + + The sending Server-FTP puts Restart Markers <ssss> + at convenient places in the data stream. When it + receives a Marker, the receiving Server-FTP writes + all prior data to disk, encodes its file system + position and transformation state as rrrr, and + sends a "110 MARK ssss = rrrr" reply over the + control connection to the User. The User-FTP + appends the pair (ssss,rrrr) to its restart + control file. + + To restart the transfer, the User-FTP fetches the + last (ssss,rrrr) pair from the restart control + file, sends "REST ssss" to the sending Server-FTP, + and sends "REST rrrr" to the receiving Server-FTP. + + + 4.1.4 FTP/USER INTERFACE + + This section discusses the user interface for a User-FTP + program. + + 4.1.4.1 Pathname Specification + + Since FTP is intended for use in a heterogeneous + environment, User-FTP implementations MUST support remote + pathnames as arbitrary character strings, so that their form + and content are not limited by the conventions of the local + operating system. + + DISCUSSION: + In particular, remote pathnames can be of arbitrary + length, and all the printing ASCII characters as well + as space (0x20) must be allowed. RFC-959 allows a + pathname to contain any 7-bit ASCII character except CR + or LF. + + + + + +Internet Engineering Task Force [Page 39] + + + + +RFC1123 FILE TRANSFER -- FTP October 1989 + + + 4.1.4.2 "QUOTE" Command + + A User-FTP program MUST implement a "QUOTE" command that + will pass an arbitrary character string to the server and + display all resulting response messages to the user. + + To make the "QUOTE" command useful, a User-FTP SHOULD send + transfer control commands to the server as the user enters + them, rather than saving all the commands and sending them + to the server only when a data transfer is started. + + DISCUSSION: + The "QUOTE" command is essential to allow the user to + access servers that require system-specific commands + (e.g., SITE or ALLO), or to invoke new or optional + features that are not implemented by the User-FTP. For + example, "QUOTE" may be used to specify "TYPE A T" to + send a print file to hosts that require the + distinction, even if the User-FTP does not recognize + that TYPE. + + 4.1.4.3 Displaying Replies to User + + A User-FTP SHOULD display to the user the full text of all + error reply messages it receives. It SHOULD have a + "verbose" mode in which all commands it sends and the full + text and reply codes it receives are displayed, for + diagnosis of problems. + + 4.1.4.4 Maintaining Synchronization + + The state machine in a User-FTP SHOULD be forgiving of + missing and unexpected reply messages, in order to maintain + command synchronization with the server. + + + + + + + + + + + + + + + + + +Internet Engineering Task Force [Page 40] + + + + +RFC1123 FILE TRANSFER -- FTP October 1989 + + + 4.1.5 FTP REQUIREMENTS SUMMARY + + | | | | |S| | + | | | | |H| |F + | | | | |O|M|o + | | |S| |U|U|o + | | |H| |L|S|t + | |M|O| |D|T|n + | |U|U|M| | |o + | |S|L|A|N|N|t + | |T|D|Y|O|O|t +FEATURE |SECTION | | | |T|T|e +-------------------------------------------|---------------|-|-|-|-|-|-- +Implement TYPE T if same as TYPE N |4.1.2.2 | |x| | | | +File/Record transform invertible if poss. |4.1.2.4 | |x| | | | +User-FTP send PORT cmd for stream mode |4.1.2.5 | |x| | | | +Server-FTP implement PASV |4.1.2.6 |x| | | | | + PASV is per-transfer |4.1.2.6 |x| | | | | +NLST reply usable in RETR cmds |4.1.2.7 |x| | | | | +Implied type for LIST and NLST |4.1.2.7 | |x| | | | +SITE cmd for non-standard features |4.1.2.8 | |x| | | | +STOU cmd return pathname as specified |4.1.2.9 |x| | | | | +Use TCP READ boundaries on control conn. |4.1.2.10 | | | | |x| + | | | | | | | +Server-FTP send only correct reply format |4.1.2.11 |x| | | | | +Server-FTP use defined reply code if poss. |4.1.2.11 | |x| | | | + New reply code following Section 4.2 |4.1.2.11 | | |x| | | +User-FTP use only high digit of reply |4.1.2.11 | |x| | | | +User-FTP handle multi-line reply lines |4.1.2.11 |x| | | | | +User-FTP handle 421 reply specially |4.1.2.11 | | | |x| | + | | | | | | | +Default data port same IP addr as ctl conn |4.1.2.12 |x| | | | | +User-FTP send Telnet cmds exc. SYNCH, IP |4.1.2.12 | | | | |x| +User-FTP negotiate Telnet options |4.1.2.12 | | | | |x| +Server-FTP handle Telnet options |4.1.2.12 |x| | | | | +Handle "Experimental" directory cmds |4.1.3.1 | |x| | | | +Idle timeout in server-FTP |4.1.3.2 | |x| | | | + Configurable idle timeout |4.1.3.2 | |x| | | | +Receiver checkpoint data at Restart Marker |4.1.3.4 | |x| | | | +Sender assume 110 replies are synchronous |4.1.3.4 | | | | |x| + | | | | | | | +Support TYPE: | | | | | | | + ASCII - Non-Print (AN) |4.1.2.13 |x| | | | | + ASCII - Telnet (AT) -- if same as AN |4.1.2.2 | |x| | | | + ASCII - Carriage Control (AC) |959 3.1.1.5.2 | | |x| | | + EBCDIC - (any form) |959 3.1.1.2 | | |x| | | + IMAGE |4.1.2.1 |x| | | | | + LOCAL 8 |4.1.2.1 |x| | | | | + + + +Internet Engineering Task Force [Page 41] + + + + +RFC1123 FILE TRANSFER -- FTP October 1989 + + + LOCAL m |4.1.2.1 | | |x| | |2 + | | | | | | | +Support MODE: | | | | | | | + Stream |4.1.2.13 |x| | | | | + Block |959 3.4.2 | | |x| | | + | | | | | | | +Support STRUCTURE: | | | | | | | + File |4.1.2.13 |x| | | | | + Record |4.1.2.13 |x| | | | |3 + Page |4.1.2.3 | | | |x| | + | | | | | | | +Support commands: | | | | | | | + USER |4.1.2.13 |x| | | | | + PASS |4.1.2.13 |x| | | | | + ACCT |4.1.2.13 |x| | | | | + CWD |4.1.2.13 |x| | | | | + CDUP |4.1.2.13 |x| | | | | + SMNT |959 5.3.1 | | |x| | | + REIN |959 5.3.1 | | |x| | | + QUIT |4.1.2.13 |x| | | | | + | | | | | | | + PORT |4.1.2.13 |x| | | | | + PASV |4.1.2.6 |x| | | | | + TYPE |4.1.2.13 |x| | | | |1 + STRU |4.1.2.13 |x| | | | |1 + MODE |4.1.2.13 |x| | | | |1 + | | | | | | | + RETR |4.1.2.13 |x| | | | | + STOR |4.1.2.13 |x| | | | | + STOU |959 5.3.1 | | |x| | | + APPE |4.1.2.13 |x| | | | | + ALLO |959 5.3.1 | | |x| | | + REST |959 5.3.1 | | |x| | | + RNFR |4.1.2.13 |x| | | | | + RNTO |4.1.2.13 |x| | | | | + ABOR |959 5.3.1 | | |x| | | + DELE |4.1.2.13 |x| | | | | + RMD |4.1.2.13 |x| | | | | + MKD |4.1.2.13 |x| | | | | + PWD |4.1.2.13 |x| | | | | + LIST |4.1.2.13 |x| | | | | + NLST |4.1.2.13 |x| | | | | + SITE |4.1.2.8 | | |x| | | + STAT |4.1.2.13 |x| | | | | + SYST |4.1.2.13 |x| | | | | + HELP |4.1.2.13 |x| | | | | + NOOP |4.1.2.13 |x| | | | | + | | | | | | | + + + +Internet Engineering Task Force [Page 42] + + + + +RFC1123 FILE TRANSFER -- FTP October 1989 + + +User Interface: | | | | | | | + Arbitrary pathnames |4.1.4.1 |x| | | | | + Implement "QUOTE" command |4.1.4.2 |x| | | | | + Transfer control commands immediately |4.1.4.2 | |x| | | | + Display error messages to user |4.1.4.3 | |x| | | | + Verbose mode |4.1.4.3 | |x| | | | + Maintain synchronization with server |4.1.4.4 | |x| | | | + +Footnotes: + +(1) For the values shown earlier. + +(2) Here m is number of bits in a memory word. + +(3) Required for host with record-structured file system, optional + otherwise. + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + +Internet Engineering Task Force [Page 43] + + + + +RFC1123 FILE TRANSFER -- TFTP October 1989 + + + 4.2 TRIVIAL FILE TRANSFER PROTOCOL -- TFTP + + 4.2.1 INTRODUCTION + + The Trivial File Transfer Protocol TFTP is defined in RFC-783 + [TFTP:1]. + + TFTP provides its own reliable delivery with UDP as its + transport protocol, using a simple stop-and-wait acknowledgment + system. Since TFTP has an effective window of only one 512 + octet segment, it can provide good performance only over paths + that have a small delay*bandwidth product. The TFTP file + interface is very simple, providing no access control or + security. + + TFTP's most important application is bootstrapping a host over + a local network, since it is simple and small enough to be + easily implemented in EPROM [BOOT:1, BOOT:2]. Vendors are + urged to support TFTP for booting. + + 4.2.2 PROTOCOL WALK-THROUGH + + The TFTP specification [TFTP:1] is written in an open style, + and does not fully specify many parts of the protocol. + + 4.2.2.1 Transfer Modes: RFC-783, Page 3 + + The transfer mode "mail" SHOULD NOT be supported. + + 4.2.2.2 UDP Header: RFC-783, Page 17 + + The Length field of a UDP header is incorrectly defined; it + includes the UDP header length (8). + + 4.2.3 SPECIFIC ISSUES + + 4.2.3.1 Sorcerer's Apprentice Syndrome + + There is a serious bug, known as the "Sorcerer's Apprentice + Syndrome," in the protocol specification. While it does not + cause incorrect operation of the transfer (the file will + always be transferred correctly if the transfer completes), + this bug may cause excessive retransmission, which may cause + the transfer to time out. + + Implementations MUST contain the fix for this problem: the + sender (i.e., the side originating the DATA packets) must + never resend the current DATA packet on receipt of a + + + +Internet Engineering Task Force [Page 44] + + + + +RFC1123 FILE TRANSFER -- TFTP October 1989 + + + duplicate ACK. + + DISCUSSION: + The bug is caused by the protocol rule that either + side, on receiving an old duplicate datagram, may + resend the current datagram. If a packet is delayed in + the network but later successfully delivered after + either side has timed out and retransmitted a packet, a + duplicate copy of the response may be generated. If + the other side responds to this duplicate with a + duplicate of its own, then every datagram will be sent + in duplicate for the remainder of the transfer (unless + a datagram is lost, breaking the repetition). Worse + yet, since the delay is often caused by congestion, + this duplicate transmission will usually causes more + congestion, leading to more delayed packets, etc. + + The following example may help to clarify this problem. + + TFTP A TFTP B + + (1) Receive ACK X-1 + Send DATA X + (2) Receive DATA X + Send ACK X + (ACK X is delayed in network, + and A times out): + (3) Retransmit DATA X + + (4) Receive DATA X again + Send ACK X again + (5) Receive (delayed) ACK X + Send DATA X+1 + (6) Receive DATA X+1 + Send ACK X+1 + (7) Receive ACK X again + Send DATA X+1 again + (8) Receive DATA X+1 again + Send ACK X+1 again + (9) Receive ACK X+1 + Send DATA X+2 + (10) Receive DATA X+2 + Send ACK X+3 + (11) Receive ACK X+1 again + Send DATA X+2 again + (12) Receive DATA X+2 again + Send ACK X+3 again + + + + +Internet Engineering Task Force [Page 45] + + + + +RFC1123 FILE TRANSFER -- TFTP October 1989 + + + Notice that once the delayed ACK arrives, the protocol + settles down to duplicate all further packets + (sequences 5-8 and 9-12). The problem is caused not by + either side timing out, but by both sides + retransmitting the current packet when they receive a + duplicate. + + The fix is to break the retransmission loop, as + indicated above. This is analogous to the behavior of + TCP. It is then possible to remove the retransmission + timer on the receiver, since the resent ACK will never + cause any action; this is a useful simplification where + TFTP is used in a bootstrap program. It is OK to allow + the timer to remain, and it may be helpful if the + retransmitted ACK replaces one that was genuinely lost + in the network. The sender still requires a retransmit + timer, of course. + + 4.2.3.2 Timeout Algorithms + + A TFTP implementation MUST use an adaptive timeout. + + IMPLEMENTATION: + TCP retransmission algorithms provide a useful base to + work from. At least an exponential backoff of + retransmission timeout is necessary. + + 4.2.3.3 Extensions + + A variety of non-standard extensions have been made to TFTP, + including additional transfer modes and a secure operation + mode (with passwords). None of these have been + standardized. + + 4.2.3.4 Access Control + + A server TFTP implementation SHOULD include some + configurable access control over what pathnames are allowed + in TFTP operations. + + 4.2.3.5 Broadcast Request + + A TFTP request directed to a broadcast address SHOULD be + silently ignored. + + DISCUSSION: + Due to the weak access control capability of TFTP, + directed broadcasts of TFTP requests to random networks + + + +Internet Engineering Task Force [Page 46] + + + + +RFC1123 FILE TRANSFER -- TFTP October 1989 + + + could create a significant security hole. + + 4.2.4 TFTP REQUIREMENTS SUMMARY + + | | | | |S| | + | | | | |H| |F + | | | | |O|M|o + | | |S| |U|U|o + | | |H| |L|S|t + | |M|O| |D|T|n + | |U|U|M| | |o + | |S|L|A|N|N|t + | |T|D|Y|O|O|t +FEATURE |SECTION | | | |T|T|e +-------------------------------------------------|--------|-|-|-|-|-|-- +Fix Sorcerer's Apprentice Syndrome |4.2.3.1 |x| | | | | +Transfer modes: | | | | | | | + netascii |RFC-783 |x| | | | | + octet |RFC-783 |x| | | | | + mail |4.2.2.1 | | | |x| | + extensions |4.2.3.3 | | |x| | | +Use adaptive timeout |4.2.3.2 |x| | | | | +Configurable access control |4.2.3.4 | |x| | | | +Silently ignore broadcast request |4.2.3.5 | |x| | | | +-------------------------------------------------|--------|-|-|-|-|-|-- +-------------------------------------------------|--------|-|-|-|-|-|-- + + + + + + + + + + + + + + + + + + + + + + + + + +Internet Engineering Task Force [Page 47] + + + + +RFC1123 MAIL -- SMTP & RFC-822 October 1989 + + +5. ELECTRONIC MAIL -- SMTP and RFC-822 + + 5.1 INTRODUCTION + + In the TCP/IP protocol suite, electronic mail in a format + specified in RFC-822 [SMTP:2] is transmitted using the Simple Mail + Transfer Protocol (SMTP) defined in RFC-821 [SMTP:1]. + + While SMTP has remained unchanged over the years, the Internet + community has made several changes in the way SMTP is used. In + particular, the conversion to the Domain Name System (DNS) has + caused changes in address formats and in mail routing. In this + section, we assume familiarity with the concepts and terminology + of the DNS, whose requirements are given in Section 6.1. + + RFC-822 specifies the Internet standard format for electronic mail + messages. RFC-822 supercedes an older standard, RFC-733, that may + still be in use in a few places, although it is obsolete. The two + formats are sometimes referred to simply by number ("822" and + "733"). + + RFC-822 is used in some non-Internet mail environments with + different mail transfer protocols than SMTP, and SMTP has also + been adapted for use in some non-Internet environments. Note that + this document presents the rules for the use of SMTP and RFC-822 + for the Internet environment only; other mail environments that + use these protocols may be expected to have their own rules. + + 5.2 PROTOCOL WALK-THROUGH + + This section covers both RFC-821 and RFC-822. + + The SMTP specification in RFC-821 is clear and contains numerous + examples, so implementors should not find it difficult to + understand. This section simply updates or annotates portions of + RFC-821 to conform with current usage. + + RFC-822 is a long and dense document, defining a rich syntax. + Unfortunately, incomplete or defective implementations of RFC-822 + are common. In fact, nearly all of the many formats of RFC-822 + are actually used, so an implementation generally needs to + recognize and correctly interpret all of the RFC-822 syntax. + + 5.2.1 The SMTP Model: RFC-821 Section 2 + + DISCUSSION: + Mail is sent by a series of request/response transactions + between a client, the "sender-SMTP," and a server, the + + + +Internet Engineering Task Force [Page 48] + + + + +RFC1123 MAIL -- SMTP & RFC-822 October 1989 + + + "receiver-SMTP". These transactions pass (1) the message + proper, which is composed of header and body, and (2) SMTP + source and destination addresses, referred to as the + "envelope". + + The SMTP programs are analogous to Message Transfer Agents + (MTAs) of X.400. There will be another level of protocol + software, closer to the end user, that is responsible for + composing and analyzing RFC-822 message headers; this + component is known as the "User Agent" in X.400, and we + use that term in this document. There is a clear logical + distinction between the User Agent and the SMTP + implementation, since they operate on different levels of + protocol. Note, however, that this distinction is may not + be exactly reflected the structure of typical + implementations of Internet mail. Often there is a + program known as the "mailer" that implements SMTP and + also some of the User Agent functions; the rest of the + User Agent functions are included in a user interface used + for entering and reading mail. + + The SMTP envelope is constructed at the originating site, + typically by the User Agent when the message is first + queued for the Sender-SMTP program. The envelope + addresses may be derived from information in the message + header, supplied by the user interface (e.g., to implement + a bcc: request), or derived from local configuration + information (e.g., expansion of a mailing list). The SMTP + envelope cannot in general be re-derived from the header + at a later stage in message delivery, so the envelope is + transmitted separately from the message itself using the + MAIL and RCPT commands of SMTP. + + The text of RFC-821 suggests that mail is to be delivered + to an individual user at a host. With the advent of the + domain system and of mail routing using mail-exchange (MX) + resource records, implementors should now think of + delivering mail to a user at a domain, which may or may + not be a particular host. This DOES NOT change the fact + that SMTP is a host-to-host mail exchange protocol. + + 5.2.2 Canonicalization: RFC-821 Section 3.1 + + The domain names that a Sender-SMTP sends in MAIL and RCPT + commands MUST have been "canonicalized," i.e., they must be + fully-qualified principal names or domain literals, not + nicknames or domain abbreviations. A canonicalized name either + identifies a host directly or is an MX name; it cannot be a + + + +Internet Engineering Task Force [Page 49] + + + + +RFC1123 MAIL -- SMTP & RFC-822 October 1989 + + + CNAME. + + 5.2.3 VRFY and EXPN Commands: RFC-821 Section 3.3 + + A receiver-SMTP MUST implement VRFY and SHOULD implement EXPN + (this requirement overrides RFC-821). However, there MAY be + configuration information to disable VRFY and EXPN in a + particular installation; this might even allow EXPN to be + disabled for selected lists. + + A new reply code is defined for the VRFY command: + + 252 Cannot VRFY user (e.g., info is not local), but will + take message for this user and attempt delivery. + + DISCUSSION: + SMTP users and administrators make regular use of these + commands for diagnosing mail delivery problems. With the + increasing use of multi-level mailing list expansion + (sometimes more than two levels), EXPN has been + increasingly important for diagnosing inadvertent mail + loops. On the other hand, some feel that EXPN represents + a significant privacy, and perhaps even a security, + exposure. + + 5.2.4 SEND, SOML, and SAML Commands: RFC-821 Section 3.4 + + An SMTP MAY implement the commands to send a message to a + user's terminal: SEND, SOML, and SAML. + + DISCUSSION: + It has been suggested that the use of mail relaying + through an MX record is inconsistent with the intent of + SEND to deliver a message immediately and directly to a + user's terminal. However, an SMTP receiver that is unable + to write directly to the user terminal can return a "251 + User Not Local" reply to the RCPT following a SEND, to + inform the originator of possibly deferred delivery. + + 5.2.5 HELO Command: RFC-821 Section 3.5 + + The sender-SMTP MUST ensure that the <domain> parameter in a + HELO command is a valid principal host domain name for the + client host. As a result, the receiver-SMTP will not have to + perform MX resolution on this name in order to validate the + HELO parameter. + + The HELO receiver MAY verify that the HELO parameter really + + + +Internet Engineering Task Force [Page 50] + + + + +RFC1123 MAIL -- SMTP & RFC-822 October 1989 + + + corresponds to the IP address of the sender. However, the + receiver MUST NOT refuse to accept a message, even if the + sender's HELO command fails verification. + + DISCUSSION: + Verifying the HELO parameter requires a domain name lookup + and may therefore take considerable time. An alternative + tool for tracking bogus mail sources is suggested below + (see "DATA Command"). + + Note also that the HELO argument is still required to have + valid <domain> syntax, since it will appear in a Received: + line; otherwise, a 501 error is to be sent. + + IMPLEMENTATION: + When HELO parameter validation fails, a suggested + procedure is to insert a note about the unknown + authenticity of the sender into the message header (e.g., + in the "Received:" line). + + 5.2.6 Mail Relay: RFC-821 Section 3.6 + + We distinguish three types of mail (store-and-) forwarding: + + (1) A simple forwarder or "mail exchanger" forwards a message + using private knowledge about the recipient; see section + 3.2 of RFC-821. + + (2) An SMTP mail "relay" forwards a message within an SMTP + mail environment as the result of an explicit source route + (as defined in section 3.6 of RFC-821). The SMTP relay + function uses the "@...:" form of source route from RFC- + 822 (see Section 5.2.19 below). + + (3) A mail "gateway" passes a message between different + environments. The rules for mail gateways are discussed + below in Section 5.3.7. + + An Internet host that is forwarding a message but is not a + gateway to a different mail environment (i.e., it falls under + (1) or (2)) SHOULD NOT alter any existing header fields, + although the host will add an appropriate Received: line as + required in Section 5.2.8. + + A Sender-SMTP SHOULD NOT send a RCPT TO: command containing an + explicit source route using the "@...:" address form. Thus, + the relay function defined in section 3.6 of RFC-821 should + not be used. + + + +Internet Engineering Task Force [Page 51] + + + + +RFC1123 MAIL -- SMTP & RFC-822 October 1989 + + + DISCUSSION: + The intent is to discourage all source routing and to + abolish explicit source routing for mail delivery within + the Internet environment. Source-routing is unnecessary; + the simple target address "user@domain" should always + suffice. This is the result of an explicit architectural + decision to use universal naming rather than source + routing for mail. Thus, SMTP provides end-to-end + connectivity, and the DNS provides globally-unique, + location-independent names. MX records handle the major + case where source routing might otherwise be needed. + + A receiver-SMTP MUST accept the explicit source route syntax in + the envelope, but it MAY implement the relay function as + defined in section 3.6 of RFC-821. If it does not implement + the relay function, it SHOULD attempt to deliver the message + directly to the host to the right of the right-most "@" sign. + + DISCUSSION: + For example, suppose a host that does not implement the + relay function receives a message with the SMTP command: + "RCPT TO:<@ALPHA,@BETA:joe@GAMMA>", where ALPHA, BETA, and + GAMMA represent domain names. Rather than immediately + refusing the message with a 550 error reply as suggested + on page 20 of RFC-821, the host should try to forward the + message to GAMMA directly, using: "RCPT TO:<joe@GAMMA>". + Since this host does not support relaying, it is not + required to update the reverse path. + + Some have suggested that source routing may be needed + occasionally for manually routing mail around failures; + however, the reality and importance of this need is + controversial. The use of explicit SMTP mail relaying for + this purpose is discouraged, and in fact it may not be + successful, as many host systems do not support it. Some + have used the "%-hack" (see Section 5.2.16) for this + purpose. + + 5.2.7 RCPT Command: RFC-821 Section 4.1.1 + + A host that supports a receiver-SMTP MUST support the reserved + mailbox "Postmaster". + + The receiver-SMTP MAY verify RCPT parameters as they arrive; + however, RCPT responses MUST NOT be delayed beyond a reasonable + time (see Section 5.3.2). + + Therefore, a "250 OK" response to a RCPT does not necessarily + + + +Internet Engineering Task Force [Page 52] + + + + +RFC1123 MAIL -- SMTP & RFC-822 October 1989 + + + imply that the delivery address(es) are valid. Errors found + after message acceptance will be reported by mailing a + notification message to an appropriate address (see Section + 5.3.3). + + DISCUSSION: + The set of conditions under which a RCPT parameter can be + validated immediately is an engineering design choice. + Reporting destination mailbox errors to the Sender-SMTP + before mail is transferred is generally desirable to save + time and network bandwidth, but this advantage is lost if + RCPT verification is lengthy. + + For example, the receiver can verify immediately any + simple local reference, such as a single locally- + registered mailbox. On the other hand, the "reasonable + time" limitation generally implies deferring verification + of a mailing list until after the message has been + transferred and accepted, since verifying a large mailing + list can take a very long time. An implementation might + or might not choose to defer validation of addresses that + are non-local and therefore require a DNS lookup. If a + DNS lookup is performed but a soft domain system error + (e.g., timeout) occurs, validity must be assumed. + + 5.2.8 DATA Command: RFC-821 Section 4.1.1 + + Every receiver-SMTP (not just one that "accepts a message for + relaying or for final delivery" [SMTP:1]) MUST insert a + "Received:" line at the beginning of a message. In this line, + called a "time stamp line" in RFC-821: + + * The FROM field SHOULD contain both (1) the name of the + source host as presented in the HELO command and (2) a + domain literal containing the IP address of the source, + determined from the TCP connection. + + * The ID field MAY contain an "@" as suggested in RFC-822, + but this is not required. + + * The FOR field MAY contain a list of <path> entries when + multiple RCPT commands have been given. + + + An Internet mail program MUST NOT change a Received: line that + was previously added to the message header. + + + + + +Internet Engineering Task Force [Page 53] + + + + +RFC1123 MAIL -- SMTP & RFC-822 October 1989 + + + DISCUSSION: + Including both the source host and the IP source address + in the Received: line may provide enough information for + tracking illicit mail sources and eliminate a need to + explicitly verify the HELO parameter. + + Received: lines are primarily intended for humans tracing + mail routes, primarily of diagnosis of faults. See also + the discussion under 5.3.7. + + When the receiver-SMTP makes "final delivery" of a message, + then it MUST pass the MAIL FROM: address from the SMTP envelope + with the message, for use if an error notification message must + be sent later (see Section 5.3.3). There is an analogous + requirement when gatewaying from the Internet into a different + mail environment; see Section 5.3.7. + + DISCUSSION: + Note that the final reply to the DATA command depends only + upon the successful transfer and storage of the message. + Any problem with the destination address(es) must either + (1) have been reported in an SMTP error reply to the RCPT + command(s), or (2) be reported in a later error message + mailed to the originator. + + IMPLEMENTATION: + The MAIL FROM: information may be passed as a parameter or + in a Return-Path: line inserted at the beginning of the + message. + + 5.2.9 Command Syntax: RFC-821 Section 4.1.2 + + The syntax shown in RFC-821 for the MAIL FROM: command omits + the case of an empty path: "MAIL FROM: <>" (see RFC-821 Page + 15). An empty reverse path MUST be supported. + + 5.2.10 SMTP Replies: RFC-821 Section 4.2 + + A receiver-SMTP SHOULD send only the reply codes listed in + section 4.2.2 of RFC-821 or in this document. A receiver-SMTP + SHOULD use the text shown in examples in RFC-821 whenever + appropriate. + + A sender-SMTP MUST determine its actions only by the reply + code, not by the text (except for 251 and 551 replies); any + text, including no text at all, must be acceptable. The space + (blank) following the reply code is considered part of the + text. Whenever possible, a sender-SMTP SHOULD test only the + + + +Internet Engineering Task Force [Page 54] + + + + +RFC1123 MAIL -- SMTP & RFC-822 October 1989 + + + first digit of the reply code, as specified in Appendix E of + RFC-821. + + DISCUSSION: + Interoperability problems have arisen with SMTP systems + using reply codes that are not listed explicitly in RFC- + 821 Section 4.3 but are legal according to the theory of + reply codes explained in Appendix E. + + 5.2.11 Transparency: RFC-821 Section 4.5.2 + + Implementors MUST be sure that their mail systems always add + and delete periods to ensure message transparency. + + 5.2.12 WKS Use in MX Processing: RFC-974, p. 5 + + RFC-974 [SMTP:3] recommended that the domain system be queried + for WKS ("Well-Known Service") records, to verify that each + proposed mail target does support SMTP. Later experience has + shown that WKS is not widely supported, so the WKS step in MX + processing SHOULD NOT be used. + + The following are notes on RFC-822, organized by section of that + document. + + 5.2.13 RFC-822 Message Specification: RFC-822 Section 4 + + The syntax shown for the Return-path line omits the possibility + of a null return path, which is used to prevent looping of + error notifications (see Section 5.3.3). The complete syntax + is: + + return = "Return-path" ":" route-addr + / "Return-path" ":" "<" ">" + + The set of optional header fields is hereby expanded to include + the Content-Type field defined in RFC-1049 [SMTP:7]. This + field "allows mail reading systems to automatically identify + the type of a structured message body and to process it for + display accordingly". [SMTP:7] A User Agent MAY support this + field. + + 5.2.14 RFC-822 Date and Time Specification: RFC-822 Section 5 + + The syntax for the date is hereby changed to: + + date = 1*2DIGIT month 2*4DIGIT + + + + +Internet Engineering Task Force [Page 55] + + + + +RFC1123 MAIL -- SMTP & RFC-822 October 1989 + + + All mail software SHOULD use 4-digit years in dates, to ease + the transition to the next century. + + There is a strong trend towards the use of numeric timezone + indicators, and implementations SHOULD use numeric timezones + instead of timezone names. However, all implementations MUST + accept either notation. If timezone names are used, they MUST + be exactly as defined in RFC-822. + + The military time zones are specified incorrectly in RFC-822: + they count the wrong way from UT (the signs are reversed). As + a result, military time zones in RFC-822 headers carry no + information. + + Finally, note that there is a typo in the definition of "zone" + in the syntax summary of appendix D; the correct definition + occurs in Section 3 of RFC-822. + + 5.2.15 RFC-822 Syntax Change: RFC-822 Section 6.1 + + The syntactic definition of "mailbox" in RFC-822 is hereby + changed to: + + mailbox = addr-spec ; simple address + / [phrase] route-addr ; name & addr-spec + + That is, the phrase preceding a route address is now OPTIONAL. + This change makes the following header field legal, for + example: + + From: <[email protected]> + + 5.2.16 RFC-822 Local-part: RFC-822 Section 6.2 + + The basic mailbox address specification has the form: "local- + part@domain". Here "local-part", sometimes called the "left- + hand side" of the address, is domain-dependent. + + A host that is forwarding the message but is not the + destination host implied by the right-hand side "domain" MUST + NOT interpret or modify the "local-part" of the address. + + When mail is to be gatewayed from the Internet mail environment + into a foreign mail environment (see Section 5.3.7), routing + information for that foreign environment MAY be embedded within + the "local-part" of the address. The gateway will then + interpret this local part appropriately for the foreign mail + environment. + + + +Internet Engineering Task Force [Page 56] + + + + +RFC1123 MAIL -- SMTP & RFC-822 October 1989 + + + DISCUSSION: + Although source routes are discouraged within the Internet + (see Section 5.2.6), there are non-Internet mail + environments whose delivery mechanisms do depend upon + source routes. Source routes for extra-Internet + environments can generally be buried in the "local-part" + of the address (see Section 5.2.16) while mail traverses + the Internet. When the mail reaches the appropriate + Internet mail gateway, the gateway will interpret the + local-part and build the necessary address or route for + the target mail environment. + + For example, an Internet host might send mail to: + "a!b!c!user@gateway-domain". The complex local part + "a!b!c!user" would be uninterpreted within the Internet + domain, but could be parsed and understood by the + specified mail gateway. + + An embedded source route is sometimes encoded in the + "local-part" using "%" as a right-binding routing + operator. For example, in: + + user%domain%relay3%relay2@relay1 + + the "%" convention implies that the mail is to be routed + from "relay1" through "relay2", "relay3", and finally to + "user" at "domain". This is commonly known as the "%- + hack". It is suggested that "%" have lower precedence + than any other routing operator (e.g., "!") hidden in the + local-part; for example, "a!b%c" would be interpreted as + "(a!b)%c". + + Only the target host (in this case, "relay1") is permitted + to analyze the local-part "user%domain%relay3%relay2". + + 5.2.17 Domain Literals: RFC-822 Section 6.2.3 + + A mailer MUST be able to accept and parse an Internet domain + literal whose content ("dtext"; see RFC-822) is a dotted- + decimal host address. This satisfies the requirement of + Section 2.1 for the case of mail. + + An SMTP MUST accept and recognize a domain literal for any of + its own IP addresses. + + + + + + + +Internet Engineering Task Force [Page 57] + + + + +RFC1123 MAIL -- SMTP & RFC-822 October 1989 + + + 5.2.18 Common Address Formatting Errors: RFC-822 Section 6.1 + + Errors in formatting or parsing 822 addresses are unfortunately + common. This section mentions only the most common errors. A + User Agent MUST accept all valid RFC-822 address formats, and + MUST NOT generate illegal address syntax. + + o A common error is to leave out the semicolon after a group + identifier. + + o Some systems fail to fully-qualify domain names in + messages they generate. The right-hand side of an "@" + sign in a header address field MUST be a fully-qualified + domain name. + + For example, some systems fail to fully-qualify the From: + address; this prevents a "reply" command in the user + interface from automatically constructing a return + address. + + DISCUSSION: + Although RFC-822 allows the local use of abbreviated + domain names within a domain, the application of + RFC-822 in Internet mail does not allow this. The + intent is that an Internet host must not send an SMTP + message header containing an abbreviated domain name + in an address field. This allows the address fields + of the header to be passed without alteration across + the Internet, as required in Section 5.2.6. + + o Some systems mis-parse multiple-hop explicit source routes + such as: + + @relay1,@relay2,@relay3:user@domain. + + + o Some systems over-qualify domain names by adding a + trailing dot to some or all domain names in addresses or + message-ids. This violates RFC-822 syntax. + + + 5.2.19 Explicit Source Routes: RFC-822 Section 6.2.7 + + Internet host software SHOULD NOT create an RFC-822 header + containing an address with an explicit source route, but MUST + accept such headers for compatibility with earlier systems. + + DISCUSSION: + + + +Internet Engineering Task Force [Page 58] + + + + +RFC1123 MAIL -- SMTP & RFC-822 October 1989 + + + In an understatement, RFC-822 says "The use of explicit + source routing is discouraged". Many hosts implemented + RFC-822 source routes incorrectly, so the syntax cannot be + used unambiguously in practice. Many users feel the + syntax is ugly. Explicit source routes are not needed in + the mail envelope for delivery; see Section 5.2.6. For + all these reasons, explicit source routes using the RFC- + 822 notations are not to be used in Internet mail headers. + + As stated in Section 5.2.16, it is necessary to allow an + explicit source route to be buried in the local-part of an + address, e.g., using the "%-hack", in order to allow mail + to be gatewayed into another environment in which explicit + source routing is necessary. The vigilant will observe + that there is no way for a User Agent to detect and + prevent the use of such implicit source routing when the + destination is within the Internet. We can only + discourage source routing of any kind within the Internet, + as unnecessary and undesirable. + + 5.3 SPECIFIC ISSUES + + 5.3.1 SMTP Queueing Strategies + + The common structure of a host SMTP implementation includes + user mailboxes, one or more areas for queueing messages in + transit, and one or more daemon processes for sending and + receiving mail. The exact structure will vary depending on the + needs of the users on the host and the number and size of + mailing lists supported by the host. We describe several + optimizations that have proved helpful, particularly for + mailers supporting high traffic levels. + + Any queueing strategy MUST include: + + o Timeouts on all activities. See Section 5.3.2. + + o Never sending error messages in response to error + messages. + + + 5.3.1.1 Sending Strategy + + The general model of a sender-SMTP is one or more processes + that periodically attempt to transmit outgoing mail. In a + typical system, the program that composes a message has some + method for requesting immediate attention for a new piece of + outgoing mail, while mail that cannot be transmitted + + + +Internet Engineering Task Force [Page 59] + + + + +RFC1123 MAIL -- SMTP & RFC-822 October 1989 + + + immediately MUST be queued and periodically retried by the + sender. A mail queue entry will include not only the + message itself but also the envelope information. + + The sender MUST delay retrying a particular destination + after one attempt has failed. In general, the retry + interval SHOULD be at least 30 minutes; however, more + sophisticated and variable strategies will be beneficial + when the sender-SMTP can determine the reason for non- + delivery. + + Retries continue until the message is transmitted or the + sender gives up; the give-up time generally needs to be at + least 4-5 days. The parameters to the retry algorithm MUST + be configurable. + + A sender SHOULD keep a list of hosts it cannot reach and + corresponding timeouts, rather than just retrying queued + mail items. + + DISCUSSION: + Experience suggests that failures are typically + transient (the target system has crashed), favoring a + policy of two connection attempts in the first hour the + message is in the queue, and then backing off to once + every two or three hours. + + The sender-SMTP can shorten the queueing delay by + cooperation with the receiver-SMTP. In particular, if + mail is received from a particular address, it is good + evidence that any mail queued for that host can now be + sent. + + The strategy may be further modified as a result of + multiple addresses per host (see Section 5.3.4), to + optimize delivery time vs. resource usage. + + A sender-SMTP may have a large queue of messages for + each unavailable destination host, and if it retried + all these messages in every retry cycle, there would be + excessive Internet overhead and the daemon would be + blocked for a long period. Note that an SMTP can + generally determine that a delivery attempt has failed + only after a timeout of a minute or more; a one minute + timeout per connection will result in a very large + delay if it is repeated for dozens or even hundreds of + queued messages. + + + + +Internet Engineering Task Force [Page 60] + + + + +RFC1123 MAIL -- SMTP & RFC-822 October 1989 + + + When the same message is to be delivered to several users on + the same host, only one copy of the message SHOULD be + transmitted. That is, the sender-SMTP should use the + command sequence: RCPT, RCPT,... RCPT, DATA instead of the + sequence: RCPT, DATA, RCPT, DATA,... RCPT, DATA. + Implementation of this efficiency feature is strongly urged. + + Similarly, the sender-SMTP MAY support multiple concurrent + outgoing mail transactions to achieve timely delivery. + However, some limit SHOULD be imposed to protect the host + from devoting all its resources to mail. + + The use of the different addresses of a multihomed host is + discussed below. + + 5.3.1.2 Receiving strategy + + The receiver-SMTP SHOULD attempt to keep a pending listen on + the SMTP port at all times. This will require the support + of multiple incoming TCP connections for SMTP. Some limit + MAY be imposed. + + IMPLEMENTATION: + When the receiver-SMTP receives mail from a particular + host address, it could notify the sender-SMTP to retry + any mail pending for that host address. + + 5.3.2 Timeouts in SMTP + + There are two approaches to timeouts in the sender-SMTP: (a) + limit the time for each SMTP command separately, or (b) limit + the time for the entire SMTP dialogue for a single mail + message. A sender-SMTP SHOULD use option (a), per-command + timeouts. Timeouts SHOULD be easily reconfigurable, preferably + without recompiling the SMTP code. + + DISCUSSION: + Timeouts are an essential feature of an SMTP + implementation. If the timeouts are too long (or worse, + there are no timeouts), Internet communication failures or + software bugs in receiver-SMTP programs can tie up SMTP + processes indefinitely. If the timeouts are too short, + resources will be wasted with attempts that time out part + way through message delivery. + + If option (b) is used, the timeout has to be very large, + e.g., an hour, to allow time to expand very large mailing + lists. The timeout may also need to increase linearly + + + +Internet Engineering Task Force [Page 61] + + + + +RFC1123 MAIL -- SMTP & RFC-822 October 1989 + + + with the size of the message, to account for the time to + transmit a very large message. A large fixed timeout + leads to two problems: a failure can still tie up the + sender for a very long time, and very large messages may + still spuriously time out (which is a wasteful failure!). + + Using the recommended option (a), a timer is set for each + SMTP command and for each buffer of the data transfer. + The latter means that the overall timeout is inherently + proportional to the size of the message. + + Based on extensive experience with busy mail-relay hosts, the + minimum per-command timeout values SHOULD be as follows: + + o Initial 220 Message: 5 minutes + + A Sender-SMTP process needs to distinguish between a + failed TCP connection and a delay in receiving the initial + 220 greeting message. Many receiver-SMTPs will accept a + TCP connection but delay delivery of the 220 message until + their system load will permit more mail to be processed. + + o MAIL Command: 5 minutes + + + o RCPT Command: 5 minutes + + A longer timeout would be required if processing of + mailing lists and aliases were not deferred until after + the message was accepted. + + o DATA Initiation: 2 minutes + + This is while awaiting the "354 Start Input" reply to a + DATA command. + + o Data Block: 3 minutes + + This is while awaiting the completion of each TCP SEND + call transmitting a chunk of data. + + o DATA Termination: 10 minutes. + + This is while awaiting the "250 OK" reply. When the + receiver gets the final period terminating the message + data, it typically performs processing to deliver the + message to a user mailbox. A spurious timeout at this + point would be very wasteful, since the message has been + + + +Internet Engineering Task Force [Page 62] + + + + +RFC1123 MAIL -- SMTP & RFC-822 October 1989 + + + successfully sent. + + A receiver-SMTP SHOULD have a timeout of at least 5 minutes + while it is awaiting the next command from the sender. + + 5.3.3 Reliable Mail Receipt + + When the receiver-SMTP accepts a piece of mail (by sending a + "250 OK" message in response to DATA), it is accepting + responsibility for delivering or relaying the message. It must + take this responsibility seriously, i.e., it MUST NOT lose the + message for frivolous reasons, e.g., because the host later + crashes or because of a predictable resource shortage. + + If there is a delivery failure after acceptance of a message, + the receiver-SMTP MUST formulate and mail a notification + message. This notification MUST be sent using a null ("<>") + reverse path in the envelope; see Section 3.6 of RFC-821. The + recipient of this notification SHOULD be the address from the + envelope return path (or the Return-Path: line). However, if + this address is null ("<>"), the receiver-SMTP MUST NOT send a + notification. If the address is an explicit source route, it + SHOULD be stripped down to its final hop. + + DISCUSSION: + For example, suppose that an error notification must be + sent for a message that arrived with: + "MAIL FROM:<@a,@b:user@d>". The notification message + should be sent to: "RCPT TO:<user@d>". + + Some delivery failures after the message is accepted by + SMTP will be unavoidable. For example, it may be + impossible for the receiver-SMTP to validate all the + delivery addresses in RCPT command(s) due to a "soft" + domain system error or because the target is a mailing + list (see earlier discussion of RCPT). + + To avoid receiving duplicate messages as the result of + timeouts, a receiver-SMTP MUST seek to minimize the time + required to respond to the final "." that ends a message + transfer. See RFC-1047 [SMTP:4] for a discussion of this + problem. + + 5.3.4 Reliable Mail Transmission + + To transmit a message, a sender-SMTP determines the IP address + of the target host from the destination address in the + envelope. Specifically, it maps the string to the right of the + + + +Internet Engineering Task Force [Page 63] + + + + +RFC1123 MAIL -- SMTP & RFC-822 October 1989 + + + "@" sign into an IP address. This mapping or the transfer + itself may fail with a soft error, in which case the sender- + SMTP will requeue the outgoing mail for a later retry, as + required in Section 5.3.1.1. + + When it succeeds, the mapping can result in a list of + alternative delivery addresses rather than a single address, + because of (a) multiple MX records, (b) multihoming, or both. + To provide reliable mail transmission, the sender-SMTP MUST be + able to try (and retry) each of the addresses in this list in + order, until a delivery attempt succeeds. However, there MAY + also be a configurable limit on the number of alternate + addresses that can be tried. In any case, a host SHOULD try at + least two addresses. + + The following information is to be used to rank the host + addresses: + + (1) Multiple MX Records -- these contain a preference + indication that should be used in sorting. If there are + multiple destinations with the same preference and there + is no clear reason to favor one (e.g., by address + preference), then the sender-SMTP SHOULD pick one at + random to spread the load across multiple mail exchanges + for a specific organization; note that this is a + refinement of the procedure in [DNS:3]. + + (2) Multihomed host -- The destination host (perhaps taken + from the preferred MX record) may be multihomed, in which + case the domain name resolver will return a list of + alternative IP addresses. It is the responsibility of the + domain name resolver interface (see Section 6.1.3.4 below) + to have ordered this list by decreasing preference, and + SMTP MUST try them in the order presented. + + DISCUSSION: + Although the capability to try multiple alternative + addresses is required, there may be circumstances where + specific installations want to limit or disable the use of + alternative addresses. The question of whether a sender + should attempt retries using the different addresses of a + multihomed host has been controversial. The main argument + for using the multiple addresses is that it maximizes the + probability of timely delivery, and indeed sometimes the + probability of any delivery; the counter argument is that + it may result in unnecessary resource use. + + Note that resource use is also strongly determined by the + + + +Internet Engineering Task Force [Page 64] + + + + +RFC1123 MAIL -- SMTP & RFC-822 October 1989 + + + sending strategy discussed in Section 5.3.1. + + 5.3.5 Domain Name Support + + SMTP implementations MUST use the mechanism defined in Section + 6.1 for mapping between domain names and IP addresses. This + means that every Internet SMTP MUST include support for the + Internet DNS. + + In particular, a sender-SMTP MUST support the MX record scheme + [SMTP:3]. See also Section 7.4 of [DNS:2] for information on + domain name support for SMTP. + + 5.3.6 Mailing Lists and Aliases + + An SMTP-capable host SHOULD support both the alias and the list + form of address expansion for multiple delivery. When a + message is delivered or forwarded to each address of an + expanded list form, the return address in the envelope + ("MAIL FROM:") MUST be changed to be the address of a person + who administers the list, but the message header MUST be left + unchanged; in particular, the "From" field of the message is + unaffected. + + DISCUSSION: + An important mail facility is a mechanism for multi- + destination delivery of a single message, by transforming + or "expanding" a pseudo-mailbox address into a list of + destination mailbox addresses. When a message is sent to + such a pseudo-mailbox (sometimes called an "exploder"), + copies are forwarded or redistributed to each mailbox in + the expanded list. We classify such a pseudo-mailbox as + an "alias" or a "list", depending upon the expansion + rules: + + (a) Alias + + To expand an alias, the recipient mailer simply + replaces the pseudo-mailbox address in the envelope + with each of the expanded addresses in turn; the rest + of the envelope and the message body are left + unchanged. The message is then delivered or + forwarded to each expanded address. + + (b) List + + A mailing list may be said to operate by + "redistribution" rather than by "forwarding". To + + + +Internet Engineering Task Force [Page 65] + + + + +RFC1123 MAIL -- SMTP & RFC-822 October 1989 + + + expand a list, the recipient mailer replaces the + pseudo-mailbox address in the envelope with each of + the expanded addresses in turn. The return address in + the envelope is changed so that all error messages + generated by the final deliveries will be returned to + a list administrator, not to the message originator, + who generally has no control over the contents of the + list and will typically find error messages annoying. + + + 5.3.7 Mail Gatewaying + + Gatewaying mail between different mail environments, i.e., + different mail formats and protocols, is complex and does not + easily yield to standardization. See for example [SMTP:5a], + [SMTP:5b]. However, some general requirements may be given for + a gateway between the Internet and another mail environment. + + (A) Header fields MAY be rewritten when necessary as messages + are gatewayed across mail environment boundaries. + + DISCUSSION: + This may involve interpreting the local-part of the + destination address, as suggested in Section 5.2.16. + + The other mail systems gatewayed to the Internet + generally use a subset of RFC-822 headers, but some + of them do not have an equivalent to the SMTP + envelope. Therefore, when a message leaves the + Internet environment, it may be necessary to fold the + SMTP envelope information into the message header. A + possible solution would be to create new header + fields to carry the envelope information (e.g., "X- + SMTP-MAIL:" and "X-SMTP-RCPT:"); however, this would + require changes in mail programs in the foreign + environment. + + (B) When forwarding a message into or out of the Internet + environment, a gateway MUST prepend a Received: line, but + it MUST NOT alter in any way a Received: line that is + already in the header. + + DISCUSSION: + This requirement is a subset of the general + "Received:" line requirement of Section 5.2.8; it is + restated here for emphasis. + + Received: fields of messages originating from other + + + +Internet Engineering Task Force [Page 66] + + + + +RFC1123 MAIL -- SMTP & RFC-822 October 1989 + + + environments may not conform exactly to RFC822. + However, the most important use of Received: lines is + for debugging mail faults, and this debugging can be + severely hampered by well-meaning gateways that try + to "fix" a Received: line. + + The gateway is strongly encouraged to indicate the + environment and protocol in the "via" clauses of + Received field(s) that it supplies. + + (C) From the Internet side, the gateway SHOULD accept all + valid address formats in SMTP commands and in RFC-822 + headers, and all valid RFC-822 messages. Although a + gateway must accept an RFC-822 explicit source route + ("@...:" format) in either the RFC-822 header or in the + envelope, it MAY or may not act on the source route; see + Sections 5.2.6 and 5.2.19. + + DISCUSSION: + It is often tempting to restrict the range of + addresses accepted at the mail gateway to simplify + the translation into addresses for the remote + environment. This practice is based on the + assumption that mail users have control over the + addresses their mailers send to the mail gateway. In + practice, however, users have little control over the + addresses that are finally sent; their mailers are + free to change addresses into any legal RFC-822 + format. + + (D) The gateway MUST ensure that all header fields of a + message that it forwards into the Internet meet the + requirements for Internet mail. In particular, all + addresses in "From:", "To:", "Cc:", etc., fields must be + transformed (if necessary) to satisfy RFC-822 syntax, and + they must be effective and useful for sending replies. + + + (E) The translation algorithm used to convert mail from the + Internet protocols to another environment's protocol + SHOULD try to ensure that error messages from the foreign + mail environment are delivered to the return path from the + SMTP envelope, not to the sender listed in the "From:" + field of the RFC-822 message. + + DISCUSSION: + Internet mail lists usually place the address of the + mail list maintainer in the envelope but leave the + + + +Internet Engineering Task Force [Page 67] + + + + +RFC1123 MAIL -- SMTP & RFC-822 October 1989 + + + original message header intact (with the "From:" + field containing the original sender). This yields + the behavior the average recipient expects: a reply + to the header gets sent to the original sender, not + to a mail list maintainer; however, errors get sent + to the maintainer (who can fix the problem) and not + the sender (who probably cannot). + + (F) Similarly, when forwarding a message from another + environment into the Internet, the gateway SHOULD set the + envelope return path in accordance with an error message + return address, if any, supplied by the foreign + environment. + + + 5.3.8 Maximum Message Size + + Mailer software MUST be able to send and receive messages of at + least 64K bytes in length (including header), and a much larger + maximum size is highly desirable. + + DISCUSSION: + Although SMTP does not define the maximum size of a + message, many systems impose implementation limits. + + The current de facto minimum limit in the Internet is 64K + bytes. However, electronic mail is used for a variety of + purposes that create much larger messages. For example, + mail is often used instead of FTP for transmitting ASCII + files, and in particular to transmit entire documents. As + a result, messages can be 1 megabyte or even larger. We + note that the present document together with its lower- + layer companion contains 0.5 megabytes. + + + + + + + + + + + + + + + + + + +Internet Engineering Task Force [Page 68] + + + + +RFC1123 MAIL -- SMTP & RFC-822 October 1989 + + + 5.4 SMTP REQUIREMENTS SUMMARY + + | | | | |S| | + | | | | |H| |F + | | | | |O|M|o + | | |S| |U|U|o + | | |H| |L|S|t + | |M|O| |D|T|n + | |U|U|M| | |o + | |S|L|A|N|N|t + | |T|D|Y|O|O|t +FEATURE |SECTION | | | |T|T|e +-----------------------------------------------|----------|-|-|-|-|-|-- + | | | | | | | +RECEIVER-SMTP: | | | | | | | + Implement VRFY |5.2.3 |x| | | | | + Implement EXPN |5.2.3 | |x| | | | + EXPN, VRFY configurable |5.2.3 | | |x| | | + Implement SEND, SOML, SAML |5.2.4 | | |x| | | + Verify HELO parameter |5.2.5 | | |x| | | + Refuse message with bad HELO |5.2.5 | | | | |x| + Accept explicit src-route syntax in env. |5.2.6 |x| | | | | + Support "postmaster" |5.2.7 |x| | | | | + Process RCPT when received (except lists) |5.2.7 | | |x| | | + Long delay of RCPT responses |5.2.7 | | | | |x| + | | | | | | | + Add Received: line |5.2.8 |x| | | | | + Received: line include domain literal |5.2.8 | |x| | | | + Change previous Received: line |5.2.8 | | | | |x| + Pass Return-Path info (final deliv/gwy) |5.2.8 |x| | | | | + Support empty reverse path |5.2.9 |x| | | | | + Send only official reply codes |5.2.10 | |x| | | | + Send text from RFC-821 when appropriate |5.2.10 | |x| | | | + Delete "." for transparency |5.2.11 |x| | | | | + Accept and recognize self domain literal(s) |5.2.17 |x| | | | | + | | | | | | | + Error message about error message |5.3.1 | | | | |x| + Keep pending listen on SMTP port |5.3.1.2 | |x| | | | + Provide limit on recv concurrency |5.3.1.2 | | |x| | | + Wait at least 5 mins for next sender cmd |5.3.2 | |x| | | | + Avoidable delivery failure after "250 OK" |5.3.3 | | | | |x| + Send error notification msg after accept |5.3.3 |x| | | | | + Send using null return path |5.3.3 |x| | | | | + Send to envelope return path |5.3.3 | |x| | | | + Send to null address |5.3.3 | | | | |x| + Strip off explicit src route |5.3.3 | |x| | | | + Minimize acceptance delay (RFC-1047) |5.3.3 |x| | | | | +-----------------------------------------------|----------|-|-|-|-|-|-- + + + +Internet Engineering Task Force [Page 69] + + + + +RFC1123 MAIL -- SMTP & RFC-822 October 1989 + + + | | | | | | | +SENDER-SMTP: | | | | | | | + Canonicalized domain names in MAIL, RCPT |5.2.2 |x| | | | | + Implement SEND, SOML, SAML |5.2.4 | | |x| | | + Send valid principal host name in HELO |5.2.5 |x| | | | | + Send explicit source route in RCPT TO: |5.2.6 | | | |x| | + Use only reply code to determine action |5.2.10 |x| | | | | + Use only high digit of reply code when poss. |5.2.10 | |x| | | | + Add "." for transparency |5.2.11 |x| | | | | + | | | | | | | + Retry messages after soft failure |5.3.1.1 |x| | | | | + Delay before retry |5.3.1.1 |x| | | | | + Configurable retry parameters |5.3.1.1 |x| | | | | + Retry once per each queued dest host |5.3.1.1 | |x| | | | + Multiple RCPT's for same DATA |5.3.1.1 | |x| | | | + Support multiple concurrent transactions |5.3.1.1 | | |x| | | + Provide limit on concurrency |5.3.1.1 | |x| | | | + | | | | | | | + Timeouts on all activities |5.3.1 |x| | | | | + Per-command timeouts |5.3.2 | |x| | | | + Timeouts easily reconfigurable |5.3.2 | |x| | | | + Recommended times |5.3.2 | |x| | | | + Try alternate addr's in order |5.3.4 |x| | | | | + Configurable limit on alternate tries |5.3.4 | | |x| | | + Try at least two alternates |5.3.4 | |x| | | | + Load-split across equal MX alternates |5.3.4 | |x| | | | + Use the Domain Name System |5.3.5 |x| | | | | + Support MX records |5.3.5 |x| | | | | + Use WKS records in MX processing |5.2.12 | | | |x| | +-----------------------------------------------|----------|-|-|-|-|-|-- + | | | | | | | +MAIL FORWARDING: | | | | | | | + Alter existing header field(s) |5.2.6 | | | |x| | + Implement relay function: 821/section 3.6 |5.2.6 | | |x| | | + If not, deliver to RHS domain |5.2.6 | |x| | | | + Interpret 'local-part' of addr |5.2.16 | | | | |x| + | | | | | | | +MAILING LISTS AND ALIASES | | | | | | | + Support both |5.3.6 | |x| | | | + Report mail list error to local admin. |5.3.6 |x| | | | | + | | | | | | | +MAIL GATEWAYS: | | | | | | | + Embed foreign mail route in local-part |5.2.16 | | |x| | | + Rewrite header fields when necessary |5.3.7 | | |x| | | + Prepend Received: line |5.3.7 |x| | | | | + Change existing Received: line |5.3.7 | | | | |x| + Accept full RFC-822 on Internet side |5.3.7 | |x| | | | + Act on RFC-822 explicit source route |5.3.7 | | |x| | | + + + +Internet Engineering Task Force [Page 70] + + + + +RFC1123 MAIL -- SMTP & RFC-822 October 1989 + + + Send only valid RFC-822 on Internet side |5.3.7 |x| | | | | + Deliver error msgs to envelope addr |5.3.7 | |x| | | | + Set env return path from err return addr |5.3.7 | |x| | | | + | | | | | | | +USER AGENT -- RFC-822 | | | | | | | + Allow user to enter <route> address |5.2.6 | | | |x| | + Support RFC-1049 Content Type field |5.2.13 | | |x| | | + Use 4-digit years |5.2.14 | |x| | | | + Generate numeric timezones |5.2.14 | |x| | | | + Accept all timezones |5.2.14 |x| | | | | + Use non-num timezones from RFC-822 |5.2.14 |x| | | | | + Omit phrase before route-addr |5.2.15 | | |x| | | + Accept and parse dot.dec. domain literals |5.2.17 |x| | | | | + Accept all RFC-822 address formats |5.2.18 |x| | | | | + Generate invalid RFC-822 address format |5.2.18 | | | | |x| + Fully-qualified domain names in header |5.2.18 |x| | | | | + Create explicit src route in header |5.2.19 | | | |x| | + Accept explicit src route in header |5.2.19 |x| | | | | + | | | | | | | +Send/recv at least 64KB messages |5.3.8 |x| | | | | + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + +Internet Engineering Task Force [Page 71] + + + + +RFC1123 SUPPORT SERVICES -- DOMAINS October 1989 + + +6. SUPPORT SERVICES + + 6.1 DOMAIN NAME TRANSLATION + + 6.1.1 INTRODUCTION + + Every host MUST implement a resolver for the Domain Name System + (DNS), and it MUST implement a mechanism using this DNS + resolver to convert host names to IP addresses and vice-versa + [DNS:1, DNS:2]. + + In addition to the DNS, a host MAY also implement a host name + translation mechanism that searches a local Internet host + table. See Section 6.1.3.8 for more information on this + option. + + DISCUSSION: + Internet host name translation was originally performed by + searching local copies of a table of all hosts. This + table became too large to update and distribute in a + timely manner and too large to fit into many hosts, so the + DNS was invented. + + The DNS creates a distributed database used primarily for + the translation between host names and host addresses. + Implementation of DNS software is required. The DNS + consists of two logically distinct parts: name servers and + resolvers (although implementations often combine these + two logical parts in the interest of efficiency) [DNS:2]. + + Domain name servers store authoritative data about certain + sections of the database and answer queries about the + data. Domain resolvers query domain name servers for data + on behalf of user processes. Every host therefore needs a + DNS resolver; some host machines will also need to run + domain name servers. Since no name server has complete + information, in general it is necessary to obtain + information from more than one name server to resolve a + query. + + 6.1.2 PROTOCOL WALK-THROUGH + + An implementor must study references [DNS:1] and [DNS:2] + carefully. They provide a thorough description of the theory, + protocol, and implementation of the domain name system, and + reflect several years of experience. + + + + + +Internet Engineering Task Force [Page 72] + + + + +RFC1123 SUPPORT SERVICES -- DOMAINS October 1989 + + + 6.1.2.1 Resource Records with Zero TTL: RFC-1035 Section 3.2.1 + + All DNS name servers and resolvers MUST properly handle RRs + with a zero TTL: return the RR to the client but do not + cache it. + + DISCUSSION: + Zero TTL values are interpreted to mean that the RR can + only be used for the transaction in progress, and + should not be cached; they are useful for extremely + volatile data. + + 6.1.2.2 QCLASS Values: RFC-1035 Section 3.2.5 + + A query with "QCLASS=*" SHOULD NOT be used unless the + requestor is seeking data from more than one class. In + particular, if the requestor is only interested in Internet + data types, QCLASS=IN MUST be used. + + 6.1.2.3 Unused Fields: RFC-1035 Section 4.1.1 + + Unused fields in a query or response message MUST be zero. + + 6.1.2.4 Compression: RFC-1035 Section 4.1.4 + + Name servers MUST use compression in responses. + + DISCUSSION: + Compression is essential to avoid overflowing UDP + datagrams; see Section 6.1.3.2. + + 6.1.2.5 Misusing Configuration Info: RFC-1035 Section 6.1.2 + + Recursive name servers and full-service resolvers generally + have some configuration information containing hints about + the location of root or local name servers. An + implementation MUST NOT include any of these hints in a + response. + + DISCUSSION: + Many implementors have found it convenient to store + these hints as if they were cached data, but some + neglected to ensure that this "cached data" was not + included in responses. This has caused serious + problems in the Internet when the hints were obsolete + or incorrect. + + + + + +Internet Engineering Task Force [Page 73] + + + + +RFC1123 SUPPORT SERVICES -- DOMAINS October 1989 + + + 6.1.3 SPECIFIC ISSUES + + 6.1.3.1 Resolver Implementation + + A name resolver SHOULD be able to multiplex concurrent + requests if the host supports concurrent processes. + + In implementing a DNS resolver, one of two different models + MAY optionally be chosen: a full-service resolver, or a stub + resolver. + + + (A) Full-Service Resolver + + A full-service resolver is a complete implementation of + the resolver service, and is capable of dealing with + communication failures, failure of individual name + servers, location of the proper name server for a given + name, etc. It must satisfy the following requirements: + + o The resolver MUST implement a local caching + function to avoid repeated remote access for + identical requests, and MUST time out information + in the cache. + + o The resolver SHOULD be configurable with start-up + information pointing to multiple root name servers + and multiple name servers for the local domain. + This insures that the resolver will be able to + access the whole name space in normal cases, and + will be able to access local domain information + should the local network become disconnected from + the rest of the Internet. + + + (B) Stub Resolver + + A "stub resolver" relies on the services of a recursive + name server on the connected network or a "nearby" + network. This scheme allows the host to pass on the + burden of the resolver function to a name server on + another host. This model is often essential for less + capable hosts, such as PCs, and is also recommended + when the host is one of several workstations on a local + network, because it allows all of the workstations to + share the cache of the recursive name server and hence + reduce the number of domain requests exported by the + local network. + + + +Internet Engineering Task Force [Page 74] + + + + +RFC1123 SUPPORT SERVICES -- DOMAINS October 1989 + + + At a minimum, the stub resolver MUST be capable of + directing its requests to redundant recursive name + servers. Note that recursive name servers are allowed + to restrict the sources of requests that they will + honor, so the host administrator must verify that the + service will be provided. Stub resolvers MAY implement + caching if they choose, but if so, MUST timeout cached + information. + + + 6.1.3.2 Transport Protocols + + DNS resolvers and recursive servers MUST support UDP, and + SHOULD support TCP, for sending (non-zone-transfer) queries. + Specifically, a DNS resolver or server that is sending a + non-zone-transfer query MUST send a UDP query first. If the + Answer section of the response is truncated and if the + requester supports TCP, it SHOULD try the query again using + TCP. + + DNS servers MUST be able to service UDP queries and SHOULD + be able to service TCP queries. A name server MAY limit the + resources it devotes to TCP queries, but it SHOULD NOT + refuse to service a TCP query just because it would have + succeeded with UDP. + + Truncated responses MUST NOT be saved (cached) and later + used in such a way that the fact that they are truncated is + lost. + + DISCUSSION: + UDP is preferred over TCP for queries because UDP + queries have much lower overhead, both in packet count + and in connection state. The use of UDP is essential + for heavily-loaded servers, especially the root + servers. UDP also offers additional robustness, since + a resolver can attempt several UDP queries to different + servers for the cost of a single TCP query. + + It is possible for a DNS response to be truncated, + although this is a very rare occurrence in the present + Internet DNS. Practically speaking, truncation cannot + be predicted, since it is data-dependent. The + dependencies include the number of RRs in the answer, + the size of each RR, and the savings in space realized + by the name compression algorithm. As a rule of thumb, + truncation in NS and MX lists should not occur for + answers containing 15 or fewer RRs. + + + +Internet Engineering Task Force [Page 75] + + + + +RFC1123 SUPPORT SERVICES -- DOMAINS October 1989 + + + Whether it is possible to use a truncated answer + depends on the application. A mailer must not use a + truncated MX response, since this could lead to mail + loops. + + Responsible practices can make UDP suffice in the vast + majority of cases. Name servers must use compression + in responses. Resolvers must differentiate truncation + of the Additional section of a response (which only + loses extra information) from truncation of the Answer + section (which for MX records renders the response + unusable by mailers). Database administrators should + list only a reasonable number of primary names in lists + of name servers, MX alternatives, etc. + + However, it is also clear that some new DNS record + types defined in the future will contain information + exceeding the 512 byte limit that applies to UDP, and + hence will require TCP. Thus, resolvers and name + servers should implement TCP services as a backup to + UDP today, with the knowledge that they will require + the TCP service in the future. + + By private agreement, name servers and resolvers MAY arrange + to use TCP for all traffic between themselves. TCP MUST be + used for zone transfers. + + A DNS server MUST have sufficient internal concurrency that + it can continue to process UDP queries while awaiting a + response or performing a zone transfer on an open TCP + connection [DNS:2]. + + A server MAY support a UDP query that is delivered using an + IP broadcast or multicast address. However, the Recursion + Desired bit MUST NOT be set in a query that is multicast, + and MUST be ignored by name servers receiving queries via a + broadcast or multicast address. A host that sends broadcast + or multicast DNS queries SHOULD send them only as occasional + probes, caching the IP address(es) it obtains from the + response(s) so it can normally send unicast queries. + + DISCUSSION: + Broadcast or (especially) IP multicast can provide a + way to locate nearby name servers without knowing their + IP addresses in advance. However, general broadcasting + of recursive queries can result in excessive and + unnecessary load on both network and servers. + + + + +Internet Engineering Task Force [Page 76] + + + + +RFC1123 SUPPORT SERVICES -- DOMAINS October 1989 + + + 6.1.3.3 Efficient Resource Usage + + The following requirements on servers and resolvers are very + important to the health of the Internet as a whole, + particularly when DNS services are invoked repeatedly by + higher level automatic servers, such as mailers. + + (1) The resolver MUST implement retransmission controls to + insure that it does not waste communication bandwidth, + and MUST impose finite bounds on the resources consumed + to respond to a single request. See [DNS:2] pages 43- + 44 for specific recommendations. + + (2) After a query has been retransmitted several times + without a response, an implementation MUST give up and + return a soft error to the application. + + (3) All DNS name servers and resolvers SHOULD cache + temporary failures, with a timeout period of the order + of minutes. + + DISCUSSION: + This will prevent applications that immediately + retry soft failures (in violation of Section 2.2 + of this document) from generating excessive DNS + traffic. + + (4) All DNS name servers and resolvers SHOULD cache + negative responses that indicate the specified name, or + data of the specified type, does not exist, as + described in [DNS:2]. + + (5) When a DNS server or resolver retries a UDP query, the + retry interval SHOULD be constrained by an exponential + backoff algorithm, and SHOULD also have upper and lower + bounds. + + IMPLEMENTATION: + A measured RTT and variance (if available) should + be used to calculate an initial retransmission + interval. If this information is not available, a + default of no less than 5 seconds should be used. + Implementations may limit the retransmission + interval, but this limit must exceed twice the + Internet maximum segment lifetime plus service + delay at the name server. + + (6) When a resolver or server receives a Source Quench for + + + +Internet Engineering Task Force [Page 77] + + + + +RFC1123 SUPPORT SERVICES -- DOMAINS October 1989 + + + a query it has issued, it SHOULD take steps to reduce + the rate of querying that server in the near future. A + server MAY ignore a Source Quench that it receives as + the result of sending a response datagram. + + IMPLEMENTATION: + One recommended action to reduce the rate is to + send the next query attempt to an alternate + server, if there is one available. Another is to + backoff the retry interval for the same server. + + + 6.1.3.4 Multihomed Hosts + + When the host name-to-address function encounters a host + with multiple addresses, it SHOULD rank or sort the + addresses using knowledge of the immediately connected + network number(s) and any other applicable performance or + history information. + + DISCUSSION: + The different addresses of a multihomed host generally + imply different Internet paths, and some paths may be + preferable to others in performance, reliability, or + administrative restrictions. There is no general way + for the domain system to determine the best path. A + recommended approach is to base this decision on local + configuration information set by the system + administrator. + + IMPLEMENTATION: + The following scheme has been used successfully: + + (a) Incorporate into the host configuration data a + Network-Preference List, that is simply a list of + networks in preferred order. This list may be + empty if there is no preference. + + (b) When a host name is mapped into a list of IP + addresses, these addresses should be sorted by + network number, into the same order as the + corresponding networks in the Network-Preference + List. IP addresses whose networks do not appear + in the Network-Preference List should be placed at + the end of the list. + + + + + + +Internet Engineering Task Force [Page 78] + + + + +RFC1123 SUPPORT SERVICES -- DOMAINS October 1989 + + + 6.1.3.5 Extensibility + + DNS software MUST support all well-known, class-independent + formats [DNS:2], and SHOULD be written to minimize the + trauma associated with the introduction of new well-known + types and local experimentation with non-standard types. + + DISCUSSION: + The data types and classes used by the DNS are + extensible, and thus new types will be added and old + types deleted or redefined. Introduction of new data + types ought to be dependent only upon the rules for + compression of domain names inside DNS messages, and + the translation between printable (i.e., master file) + and internal formats for Resource Records (RRs). + + Compression relies on knowledge of the format of data + inside a particular RR. Hence compression must only be + used for the contents of well-known, class-independent + RRs, and must never be used for class-specific RRs or + RR types that are not well-known. The owner name of an + RR is always eligible for compression. + + A name server may acquire, via zone transfer, RRs that + the server doesn't know how to convert to printable + format. A resolver can receive similar information as + the result of queries. For proper operation, this data + must be preserved, and hence the implication is that + DNS software cannot use textual formats for internal + storage. + + The DNS defines domain name syntax very generally -- a + string of labels each containing up to 63 8-bit octets, + separated by dots, and with a maximum total of 255 + octets. Particular applications of the DNS are + permitted to further constrain the syntax of the domain + names they use, although the DNS deployment has led to + some applications allowing more general names. In + particular, Section 2.1 of this document liberalizes + slightly the syntax of a legal Internet host name that + was defined in RFC-952 [DNS:4]. + + 6.1.3.6 Status of RR Types + + Name servers MUST be able to load all RR types except MD and + MF from configuration files. The MD and MF types are + obsolete and MUST NOT be implemented; in particular, name + servers MUST NOT load these types from configuration files. + + + +Internet Engineering Task Force [Page 79] + + + + +RFC1123 SUPPORT SERVICES -- DOMAINS October 1989 + + + DISCUSSION: + The RR types MB, MG, MR, NULL, MINFO and RP are + considered experimental, and applications that use the + DNS cannot expect these RR types to be supported by + most domains. Furthermore these types are subject to + redefinition. + + The TXT and WKS RR types have not been widely used by + Internet sites; as a result, an application cannot rely + on the the existence of a TXT or WKS RR in most + domains. + + 6.1.3.7 Robustness + + DNS software may need to operate in environments where the + root servers or other servers are unavailable due to network + connectivity or other problems. In this situation, DNS name + servers and resolvers MUST continue to provide service for + the reachable part of the name space, while giving temporary + failures for the rest. + + DISCUSSION: + Although the DNS is meant to be used primarily in the + connected Internet, it should be possible to use the + system in networks which are unconnected to the + Internet. Hence implementations must not depend on + access to root servers before providing service for + local names. + + 6.1.3.8 Local Host Table + + DISCUSSION: + A host may use a local host table as a backup or + supplement to the DNS. This raises the question of + which takes precedence, the DNS or the host table; the + most flexible approach would make this a configuration + option. + + Typically, the contents of such a supplementary host + table will be determined locally by the site. However, + a publically-available table of Internet hosts is + maintained by the DDN Network Information Center (DDN + NIC), with a format documented in [DNS:4]. This table + can be retrieved from the DDN NIC using a protocol + described in [DNS:5]. It must be noted that this table + contains only a small fraction of all Internet hosts. + Hosts using this protocol to retrieve the DDN NIC host + table should use the VERSION command to check if the + + + +Internet Engineering Task Force [Page 80] + + + + +RFC1123 SUPPORT SERVICES -- DOMAINS October 1989 + + + table has changed before requesting the entire table + with the ALL command. The VERSION identifier should be + treated as an arbitrary string and tested only for + equality; no numerical sequence may be assumed. + + The DDN NIC host table includes administrative + information that is not needed for host operation and + is therefore not currently included in the DNS + database; examples include network and gateway entries. + However, much of this additional information will be + added to the DNS in the future. Conversely, the DNS + provides essential services (in particular, MX records) + that are not available from the DDN NIC host table. + + 6.1.4 DNS USER INTERFACE + + 6.1.4.1 DNS Administration + + This document is concerned with design and implementation + issues in host software, not with administrative or + operational issues. However, administrative issues are of + particular importance in the DNS, since errors in particular + segments of this large distributed database can cause poor + or erroneous performance for many sites. These issues are + discussed in [DNS:6] and [DNS:7]. + + 6.1.4.2 DNS User Interface + + Hosts MUST provide an interface to the DNS for all + application programs running on the host. This interface + will typically direct requests to a system process to + perform the resolver function [DNS:1, 6.1:2]. + + At a minimum, the basic interface MUST support a request for + all information of a specific type and class associated with + a specific name, and it MUST return either all of the + requested information, a hard error code, or a soft error + indication. When there is no error, the basic interface + returns the complete response information without + modification, deletion, or ordering, so that the basic + interface will not need to be changed to accommodate new + data types. + + DISCUSSION: + The soft error indication is an essential part of the + interface, since it may not always be possible to + access particular information from the DNS; see Section + 6.1.3.3. + + + +Internet Engineering Task Force [Page 81] + + + + +RFC1123 SUPPORT SERVICES -- DOMAINS October 1989 + + + A host MAY provide other DNS interfaces tailored to + particular functions, transforming the raw domain data into + formats more suited to these functions. In particular, a + host MUST provide a DNS interface to facilitate translation + between host addresses and host names. + + 6.1.4.3 Interface Abbreviation Facilities + + User interfaces MAY provide a method for users to enter + abbreviations for commonly-used names. Although the + definition of such methods is outside of the scope of the + DNS specification, certain rules are necessary to insure + that these methods allow access to the entire DNS name space + and to prevent excessive use of Internet resources. + + If an abbreviation method is provided, then: + + (a) There MUST be some convention for denoting that a name + is already complete, so that the abbreviation method(s) + are suppressed. A trailing dot is the usual method. + + (b) Abbreviation expansion MUST be done exactly once, and + MUST be done in the context in which the name was + entered. + + + DISCUSSION: + For example, if an abbreviation is used in a mail + program for a destination, the abbreviation should be + expanded into a full domain name and stored in the + queued message with an indication that it is already + complete. Otherwise, the abbreviation might be + expanded with a mail system search list, not the + user's, or a name could grow due to repeated + canonicalizations attempts interacting with wildcards. + + The two most common abbreviation methods are: + + (1) Interface-level aliases + + Interface-level aliases are conceptually implemented as + a list of alias/domain name pairs. The list can be + per-user or per-host, and separate lists can be + associated with different functions, e.g. one list for + host name-to-address translation, and a different list + for mail domains. When the user enters a name, the + interface attempts to match the name to the alias + component of a list entry, and if a matching entry can + + + +Internet Engineering Task Force [Page 82] + + + + +RFC1123 SUPPORT SERVICES -- DOMAINS October 1989 + + + be found, the name is replaced by the domain name found + in the pair. + + Note that interface-level aliases and CNAMEs are + completely separate mechanisms; interface-level aliases + are a local matter while CNAMEs are an Internet-wide + aliasing mechanism which is a required part of any DNS + implementation. + + (2) Search Lists + + A search list is conceptually implemented as an ordered + list of domain names. When the user enters a name, the + domain names in the search list are used as suffixes to + the user-supplied name, one by one, until a domain name + with the desired associated data is found, or the + search list is exhausted. Search lists often contain + the name of the local host's parent domain or other + ancestor domains. Search lists are often per-user or + per-process. + + It SHOULD be possible for an administrator to disable a + DNS search-list facility. Administrative denial may be + warranted in some cases, to prevent abuse of the DNS. + + There is danger that a search-list mechanism will + generate excessive queries to the root servers while + testing whether user input is a complete domain name, + lacking a final period to mark it as complete. A + search-list mechanism MUST have one of, and SHOULD have + both of, the following two provisions to prevent this: + + (a) The local resolver/name server can implement + caching of negative responses (see Section + 6.1.3.3). + + (b) The search list expander can require two or more + interior dots in a generated domain name before it + tries using the name in a query to non-local + domain servers, such as the root. + + DISCUSSION: + The intent of this requirement is to avoid + excessive delay for the user as the search list is + tested, and more importantly to prevent excessive + traffic to the root and other high-level servers. + For example, if the user supplied a name "X" and + the search list contained the root as a component, + + + +Internet Engineering Task Force [Page 83] + + + + +RFC1123 SUPPORT SERVICES -- DOMAINS October 1989 + + + a query would have to consult a root server before + the next search list alternative could be tried. + The resulting load seen by the root servers and + gateways near the root would be multiplied by the + number of hosts in the Internet. + + The negative caching alternative limits the effect + to the first time a name is used. The interior + dot rule is simpler to implement but can prevent + easy use of some top-level names. + + + 6.1.5 DOMAIN NAME SYSTEM REQUIREMENTS SUMMARY + + | | | | |S| | + | | | | |H| |F + | | | | |O|M|o + | | |S| |U|U|o + | | |H| |L|S|t + | |M|O| |D|T|n + | |U|U|M| | |o + | |S|L|A|N|N|t + | |T|D|Y|O|O|t +FEATURE |SECTION | | | |T|T|e +-----------------------------------------------|-----------|-|-|-|-|-|-- +GENERAL ISSUES | | | | | | | + | | | | | | | +Implement DNS name-to-address conversion |6.1.1 |x| | | | | +Implement DNS address-to-name conversion |6.1.1 |x| | | | | +Support conversions using host table |6.1.1 | | |x| | | +Properly handle RR with zero TTL |6.1.2.1 |x| | | | | +Use QCLASS=* unnecessarily |6.1.2.2 | |x| | | | + Use QCLASS=IN for Internet class |6.1.2.2 |x| | | | | +Unused fields zero |6.1.2.3 |x| | | | | +Use compression in responses |6.1.2.4 |x| | | | | + | | | | | | | +Include config info in responses |6.1.2.5 | | | | |x| +Support all well-known, class-indep. types |6.1.3.5 |x| | | | | +Easily expand type list |6.1.3.5 | |x| | | | +Load all RR types (except MD and MF) |6.1.3.6 |x| | | | | +Load MD or MF type |6.1.3.6 | | | | |x| +Operate when root servers, etc. unavailable |6.1.3.7 |x| | | | | +-----------------------------------------------|-----------|-|-|-|-|-|-- +RESOLVER ISSUES: | | | | | | | + | | | | | | | +Resolver support multiple concurrent requests |6.1.3.1 | |x| | | | +Full-service resolver: |6.1.3.1 | | |x| | | + Local caching |6.1.3.1 |x| | | | | + + + +Internet Engineering Task Force [Page 84] + + + + +RFC1123 SUPPORT SERVICES -- DOMAINS October 1989 + + + Information in local cache times out |6.1.3.1 |x| | | | | + Configurable with starting info |6.1.3.1 | |x| | | | +Stub resolver: |6.1.3.1 | | |x| | | + Use redundant recursive name servers |6.1.3.1 |x| | | | | + Local caching |6.1.3.1 | | |x| | | + Information in local cache times out |6.1.3.1 |x| | | | | +Support for remote multi-homed hosts: | | | | | | | + Sort multiple addresses by preference list |6.1.3.4 | |x| | | | + | | | | | | | +-----------------------------------------------|-----------|-|-|-|-|-|-- +TRANSPORT PROTOCOLS: | | | | | | | + | | | | | | | +Support UDP queries |6.1.3.2 |x| | | | | +Support TCP queries |6.1.3.2 | |x| | | | + Send query using UDP first |6.1.3.2 |x| | | | |1 + Try TCP if UDP answers are truncated |6.1.3.2 | |x| | | | +Name server limit TCP query resources |6.1.3.2 | | |x| | | + Punish unnecessary TCP query |6.1.3.2 | | | |x| | +Use truncated data as if it were not |6.1.3.2 | | | | |x| +Private agreement to use only TCP |6.1.3.2 | | |x| | | +Use TCP for zone transfers |6.1.3.2 |x| | | | | +TCP usage not block UDP queries |6.1.3.2 |x| | | | | +Support broadcast or multicast queries |6.1.3.2 | | |x| | | + RD bit set in query |6.1.3.2 | | | | |x| + RD bit ignored by server is b'cast/m'cast |6.1.3.2 |x| | | | | + Send only as occasional probe for addr's |6.1.3.2 | |x| | | | +-----------------------------------------------|-----------|-|-|-|-|-|-- +RESOURCE USAGE: | | | | | | | + | | | | | | | +Transmission controls, per [DNS:2] |6.1.3.3 |x| | | | | + Finite bounds per request |6.1.3.3 |x| | | | | +Failure after retries => soft error |6.1.3.3 |x| | | | | +Cache temporary failures |6.1.3.3 | |x| | | | +Cache negative responses |6.1.3.3 | |x| | | | +Retries use exponential backoff |6.1.3.3 | |x| | | | + Upper, lower bounds |6.1.3.3 | |x| | | | +Client handle Source Quench |6.1.3.3 | |x| | | | +Server ignore Source Quench |6.1.3.3 | | |x| | | +-----------------------------------------------|-----------|-|-|-|-|-|-- +USER INTERFACE: | | | | | | | + | | | | | | | +All programs have access to DNS interface |6.1.4.2 |x| | | | | +Able to request all info for given name |6.1.4.2 |x| | | | | +Returns complete info or error |6.1.4.2 |x| | | | | +Special interfaces |6.1.4.2 | | |x| | | + Name<->Address translation |6.1.4.2 |x| | | | | + | | | | | | | +Abbreviation Facilities: |6.1.4.3 | | |x| | | + + + +Internet Engineering Task Force [Page 85] + + + + +RFC1123 SUPPORT SERVICES -- DOMAINS October 1989 + + + Convention for complete names |6.1.4.3 |x| | | | | + Conversion exactly once |6.1.4.3 |x| | | | | + Conversion in proper context |6.1.4.3 |x| | | | | + Search list: |6.1.4.3 | | |x| | | + Administrator can disable |6.1.4.3 | |x| | | | + Prevention of excessive root queries |6.1.4.3 |x| | | | | + Both methods |6.1.4.3 | |x| | | | +-----------------------------------------------|-----------|-|-|-|-|-|-- +-----------------------------------------------|-----------|-|-|-|-|-|-- + +1. Unless there is private agreement between particular resolver and + particular server. + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + +Internet Engineering Task Force [Page 86] + + + + +RFC1123 SUPPORT SERVICES -- INITIALIZATION October 1989 + + + 6.2 HOST INITIALIZATION + + 6.2.1 INTRODUCTION + + This section discusses the initialization of host software + across a connected network, or more generally across an + Internet path. This is necessary for a diskless host, and may + optionally be used for a host with disk drives. For a diskless + host, the initialization process is called "network booting" + and is controlled by a bootstrap program located in a boot ROM. + + To initialize a diskless host across the network, there are two + distinct phases: + + (1) Configure the IP layer. + + Diskless machines often have no permanent storage in which + to store network configuration information, so that + sufficient configuration information must be obtained + dynamically to support the loading phase that follows. + This information must include at least the IP addresses of + the host and of the boot server. To support booting + across a gateway, the address mask and a list of default + gateways are also required. + + (2) Load the host system code. + + During the loading phase, an appropriate file transfer + protocol is used to copy the system code across the + network from the boot server. + + A host with a disk may perform the first step, dynamic + configuration. This is important for microcomputers, whose + floppy disks allow network configuration information to be + mistakenly duplicated on more than one host. Also, + installation of new hosts is much simpler if they automatically + obtain their configuration information from a central server, + saving administrator time and decreasing the probability of + mistakes. + + 6.2.2 REQUIREMENTS + + 6.2.2.1 Dynamic Configuration + + A number of protocol provisions have been made for dynamic + configuration. + + o ICMP Information Request/Reply messages + + + +Internet Engineering Task Force [Page 87] + + + + +RFC1123 SUPPORT SERVICES -- INITIALIZATION October 1989 + + + This obsolete message pair was designed to allow a host + to find the number of the network it is on. + Unfortunately, it was useful only if the host already + knew the host number part of its IP address, + information that hosts requiring dynamic configuration + seldom had. + + o Reverse Address Resolution Protocol (RARP) [BOOT:4] + + RARP is a link-layer protocol for a broadcast medium + that allows a host to find its IP address given its + link layer address. Unfortunately, RARP does not work + across IP gateways and therefore requires a RARP server + on every network. In addition, RARP does not provide + any other configuration information. + + o ICMP Address Mask Request/Reply messages + + These ICMP messages allow a host to learn the address + mask for a particular network interface. + + o BOOTP Protocol [BOOT:2] + + This protocol allows a host to determine the IP + addresses of the local host and the boot server, the + name of an appropriate boot file, and optionally the + address mask and list of default gateways. To locate a + BOOTP server, the host broadcasts a BOOTP request using + UDP. Ad hoc gateway extensions have been used to + transmit the BOOTP broadcast through gateways, and in + the future the IP Multicasting facility will provide a + standard mechanism for this purpose. + + + The suggested approach to dynamic configuration is to use + the BOOTP protocol with the extensions defined in "BOOTP + Vendor Information Extensions" RFC-1084 [BOOT:3]. RFC-1084 + defines some important general (not vendor-specific) + extensions. In particular, these extensions allow the + address mask to be supplied in BOOTP; we RECOMMEND that the + address mask be supplied in this manner. + + DISCUSSION: + Historically, subnetting was defined long after IP, and + so a separate mechanism (ICMP Address Mask messages) + was designed to supply the address mask to a host. + However, the IP address mask and the corresponding IP + address conceptually form a pair, and for operational + + + +Internet Engineering Task Force [Page 88] + + + + +RFC1123 SUPPORT SERVICES -- INITIALIZATION October 1989 + + + simplicity they ought to be defined at the same time + and by the same mechanism, whether a configuration file + or a dynamic mechanism like BOOTP. + + Note that BOOTP is not sufficiently general to specify + the configurations of all interfaces of a multihomed + host. A multihomed host must either use BOOTP + separately for each interface, or configure one + interface using BOOTP to perform the loading, and + perform the complete initialization from a file later. + + Application layer configuration information is expected + to be obtained from files after loading of the system + code. + + 6.2.2.2 Loading Phase + + A suggested approach for the loading phase is to use TFTP + [BOOT:1] between the IP addresses established by BOOTP. + + TFTP to a broadcast address SHOULD NOT be used, for reasons + explained in Section 4.2.3.4. + + + + + + + + + + + + + + + + + + + + + + + + + + + + + +Internet Engineering Task Force [Page 89] + + + + +RFC1123 SUPPORT SERVICES -- MANAGEMENT October 1989 + + + 6.3 REMOTE MANAGEMENT + + 6.3.1 INTRODUCTION + + The Internet community has recently put considerable effort + into the development of network management protocols. The + result has been a two-pronged approach [MGT:1, MGT:6]: the + Simple Network Management Protocol (SNMP) [MGT:4] and the + Common Management Information Protocol over TCP (CMOT) [MGT:5]. + + In order to be managed using SNMP or CMOT, a host will need to + implement an appropriate management agent. An Internet host + SHOULD include an agent for either SNMP or CMOT. + + Both SNMP and CMOT operate on a Management Information Base + (MIB) that defines a collection of management values. By + reading and setting these values, a remote application may + query and change the state of the managed system. + + A standard MIB [MGT:3] has been defined for use by both + management protocols, using data types defined by the Structure + of Management Information (SMI) defined in [MGT:2]. Additional + MIB variables can be introduced under the "enterprises" and + "experimental" subtrees of the MIB naming space [MGT:2]. + + Every protocol module in the host SHOULD implement the relevant + MIB variables. A host SHOULD implement the MIB variables as + defined in the most recent standard MIB, and MAY implement + other MIB variables when appropriate and useful. + + 6.3.2 PROTOCOL WALK-THROUGH + + The MIB is intended to cover both hosts and gateways, although + there may be detailed differences in MIB application to the two + cases. This section contains the appropriate interpretation of + the MIB for hosts. It is likely that later versions of the MIB + will include more entries for host management. + + A managed host must implement the following groups of MIB + object definitions: System, Interfaces, Address Translation, + IP, ICMP, TCP, and UDP. + + The following specific interpretations apply to hosts: + + o ipInHdrErrors + + Note that the error "time-to-live exceeded" can occur in a + host only when it is forwarding a source-routed datagram. + + + +Internet Engineering Task Force [Page 90] + + + + +RFC1123 SUPPORT SERVICES -- MANAGEMENT October 1989 + + + o ipOutNoRoutes + + This object counts datagrams discarded because no route + can be found. This may happen in a host if all the + default gateways in the host's configuration are down. + + o ipFragOKs, ipFragFails, ipFragCreates + + A host that does not implement intentional fragmentation + (see "Fragmentation" section of [INTRO:1]) MUST return the + value zero for these three objects. + + o icmpOutRedirects + + For a host, this object MUST always be zero, since hosts + do not send Redirects. + + o icmpOutAddrMaskReps + + For a host, this object MUST always be zero, unless the + host is an authoritative source of address mask + information. + + o ipAddrTable + + For a host, the "IP Address Table" object is effectively a + table of logical interfaces. + + o ipRoutingTable + + For a host, the "IP Routing Table" object is effectively a + combination of the host's Routing Cache and the static + route table described in "Routing Outbound Datagrams" + section of [INTRO:1]. + + Within each ipRouteEntry, ipRouteMetric1...4 normally will + have no meaning for a host and SHOULD always be -1, while + ipRouteType will normally have the value "remote". + + If destinations on the connected network do not appear in + the Route Cache (see "Routing Outbound Datagrams section + of [INTRO:1]), there will be no entries with ipRouteType + of "direct". + + + DISCUSSION: + The current MIB does not include Type-of-Service in an + ipRouteEntry, but a future revision is expected to make + + + +Internet Engineering Task Force [Page 91] + + + + +RFC1123 SUPPORT SERVICES -- MANAGEMENT October 1989 + + + this addition. + + We also expect the MIB to be expanded to allow the remote + management of applications (e.g., the ability to partially + reconfigure mail systems). Network service applications + such as mail systems should therefore be written with the + "hooks" for remote management. + + 6.3.3 MANAGEMENT REQUIREMENTS SUMMARY + + | | | | |S| | + | | | | |H| |F + | | | | |O|M|o + | | |S| |U|U|o + | | |H| |L|S|t + | |M|O| |D|T|n + | |U|U|M| | |o + | |S|L|A|N|N|t + | |T|D|Y|O|O|t +FEATURE |SECTION | | | |T|T|e +-----------------------------------------------|-----------|-|-|-|-|-|-- +Support SNMP or CMOT agent |6.3.1 | |x| | | | +Implement specified objects in standard MIB |6.3.1 | |x| | | | + + + + + + + + + + + + + + + + + + + + + + + + + + + + +Internet Engineering Task Force [Page 92] + + + + +RFC1123 SUPPORT SERVICES -- MANAGEMENT October 1989 + + +7. REFERENCES + + This section lists the primary references with which every + implementer must be thoroughly familiar. It also lists some + secondary references that are suggested additional reading. + + INTRODUCTORY REFERENCES: + + + [INTRO:1] "Requirements for Internet Hosts -- Communication Layers," + IETF Host Requirements Working Group, R. Braden, Ed., RFC-1122, + October 1989. + + [INTRO:2] "DDN Protocol Handbook," NIC-50004, NIC-50005, NIC-50006, + (three volumes), SRI International, December 1985. + + [INTRO:3] "Official Internet Protocols," J. Reynolds and J. Postel, + RFC-1011, May 1987. + + This document is republished periodically with new RFC numbers; + the latest version must be used. + + [INTRO:4] "Protocol Document Order Information," O. Jacobsen and J. + Postel, RFC-980, March 1986. + + [INTRO:5] "Assigned Numbers," J. Reynolds and J. Postel, RFC-1010, + May 1987. + + This document is republished periodically with new RFC numbers; + the latest version must be used. + + + TELNET REFERENCES: + + + [TELNET:1] "Telnet Protocol Specification," J. Postel and J. + Reynolds, RFC-854, May 1983. + + [TELNET:2] "Telnet Option Specification," J. Postel and J. Reynolds, + RFC-855, May 1983. + + [TELNET:3] "Telnet Binary Transmission," J. Postel and J. Reynolds, + RFC-856, May 1983. + + [TELNET:4] "Telnet Echo Option," J. Postel and J. Reynolds, RFC-857, + May 1983. + + [TELNET:5] "Telnet Suppress Go Ahead Option," J. Postel and J. + + + +Internet Engineering Task Force [Page 93] + + + + +RFC1123 SUPPORT SERVICES -- MANAGEMENT October 1989 + + + Reynolds, RFC-858, May 1983. + + [TELNET:6] "Telnet Status Option," J. Postel and J. Reynolds, RFC- + 859, May 1983. + + [TELNET:7] "Telnet Timing Mark Option," J. Postel and J. Reynolds, + RFC-860, May 1983. + + [TELNET:8] "Telnet Extended Options List," J. Postel and J. + Reynolds, RFC-861, May 1983. + + [TELNET:9] "Telnet End-Of-Record Option," J. Postel, RFC-855, + December 1983. + + [TELNET:10] "Telnet Terminal-Type Option," J. VanBokkelen, RFC-1091, + February 1989. + + This document supercedes RFC-930. + + [TELNET:11] "Telnet Window Size Option," D. Waitzman, RFC-1073, + October 1988. + + [TELNET:12] "Telnet Linemode Option," D. Borman, RFC-1116, August + 1989. + + [TELNET:13] "Telnet Terminal Speed Option," C. Hedrick, RFC-1079, + December 1988. + + [TELNET:14] "Telnet Remote Flow Control Option," C. Hedrick, RFC- + 1080, November 1988. + + + SECONDARY TELNET REFERENCES: + + + [TELNET:15] "Telnet Protocol," MIL-STD-1782, U.S. Department of + Defense, May 1984. + + This document is intended to describe the same protocol as RFC- + 854. In case of conflict, RFC-854 takes precedence, and the + present document takes precedence over both. + + [TELNET:16] "SUPDUP Protocol," M. Crispin, RFC-734, October 1977. + + [TELNET:17] "Telnet SUPDUP Option," M. Crispin, RFC-736, October + 1977. + + [TELNET:18] "Data Entry Terminal Option," J. Day, RFC-732, June 1977. + + + +Internet Engineering Task Force [Page 94] + + + + +RFC1123 SUPPORT SERVICES -- MANAGEMENT October 1989 + + + [TELNET:19] "TELNET Data Entry Terminal option -- DODIIS + Implementation," A. Yasuda and T. Thompson, RFC-1043, February + 1988. + + + FTP REFERENCES: + + + [FTP:1] "File Transfer Protocol," J. Postel and J. Reynolds, RFC- + 959, October 1985. + + [FTP:2] "Document File Format Standards," J. Postel, RFC-678, + December 1974. + + [FTP:3] "File Transfer Protocol," MIL-STD-1780, U.S. Department of + Defense, May 1984. + + This document is based on an earlier version of the FTP + specification (RFC-765) and is obsolete. + + + TFTP REFERENCES: + + + [TFTP:1] "The TFTP Protocol Revision 2," K. Sollins, RFC-783, June + 1981. + + + MAIL REFERENCES: + + + [SMTP:1] "Simple Mail Transfer Protocol," J. Postel, RFC-821, August + 1982. + + [SMTP:2] "Standard For The Format of ARPA Internet Text Messages," + D. Crocker, RFC-822, August 1982. + + This document obsoleted an earlier specification, RFC-733. + + [SMTP:3] "Mail Routing and the Domain System," C. Partridge, RFC- + 974, January 1986. + + This RFC describes the use of MX records, a mandatory extension + to the mail delivery process. + + [SMTP:4] "Duplicate Messages and SMTP," C. Partridge, RFC-1047, + February 1988. + + + + +Internet Engineering Task Force [Page 95] + + + + +RFC1123 SUPPORT SERVICES -- MANAGEMENT October 1989 + + + [SMTP:5a] "Mapping between X.400 and RFC 822," S. Kille, RFC-987, + June 1986. + + [SMTP:5b] "Addendum to RFC-987," S. Kille, RFC-???, September 1987. + + The two preceding RFC's define a proposed standard for + gatewaying mail between the Internet and the X.400 environments. + + [SMTP:6] "Simple Mail Transfer Protocol," MIL-STD-1781, U.S. + Department of Defense, May 1984. + + This specification is intended to describe the same protocol as + does RFC-821. However, MIL-STD-1781 is incomplete; in + particular, it does not include MX records [SMTP:3]. + + [SMTP:7] "A Content-Type Field for Internet Messages," M. Sirbu, + RFC-1049, March 1988. + + + DOMAIN NAME SYSTEM REFERENCES: + + + [DNS:1] "Domain Names - Concepts and Facilities," P. Mockapetris, + RFC-1034, November 1987. + + This document and the following one obsolete RFC-882, RFC-883, + and RFC-973. + + [DNS:2] "Domain Names - Implementation and Specification," RFC-1035, + P. Mockapetris, November 1987. + + + [DNS:3] "Mail Routing and the Domain System," C. Partridge, RFC-974, + January 1986. + + + [DNS:4] "DoD Internet Host Table Specification," K. Harrenstein, + RFC-952, M. Stahl, E. Feinler, October 1985. + + SECONDARY DNS REFERENCES: + + + [DNS:5] "Hostname Server," K. Harrenstein, M. Stahl, E. Feinler, + RFC-953, October 1985. + + [DNS:6] "Domain Administrators Guide," M. Stahl, RFC-1032, November + 1987. + + + + +Internet Engineering Task Force [Page 96] + + + + +RFC1123 SUPPORT SERVICES -- MANAGEMENT October 1989 + + + [DNS:7] "Domain Administrators Operations Guide," M. Lottor, RFC- + 1033, November 1987. + + [DNS:8] "The Domain Name System Handbook," Vol. 4 of Internet + Protocol Handbook, NIC 50007, SRI Network Information Center, + August 1989. + + + SYSTEM INITIALIZATION REFERENCES: + + + [BOOT:1] "Bootstrap Loading Using TFTP," R. Finlayson, RFC-906, June + 1984. + + [BOOT:2] "Bootstrap Protocol (BOOTP)," W. Croft and J. Gilmore, RFC- + 951, September 1985. + + [BOOT:3] "BOOTP Vendor Information Extensions," J. Reynolds, RFC- + 1084, December 1988. + + Note: this RFC revised and obsoleted RFC-1048. + + [BOOT:4] "A Reverse Address Resolution Protocol," R. Finlayson, T. + Mann, J. Mogul, and M. Theimer, RFC-903, June 1984. + + + MANAGEMENT REFERENCES: + + + [MGT:1] "IAB Recommendations for the Development of Internet Network + Management Standards," V. Cerf, RFC-1052, April 1988. + + [MGT:2] "Structure and Identification of Management Information for + TCP/IP-based internets," M. Rose and K. McCloghrie, RFC-1065, + August 1988. + + [MGT:3] "Management Information Base for Network Management of + TCP/IP-based internets," M. Rose and K. McCloghrie, RFC-1066, + August 1988. + + [MGT:4] "A Simple Network Management Protocol," J. Case, M. Fedor, + M. Schoffstall, and C. Davin, RFC-1098, April 1989. + + [MGT:5] "The Common Management Information Services and Protocol + over TCP/IP," U. Warrier and L. Besaw, RFC-1095, April 1989. + + [MGT:6] "Report of the Second Ad Hoc Network Management Review + Group," V. Cerf, RFC-1109, August 1989. + + + +Internet Engineering Task Force [Page 97] + + + + +RFC1123 SUPPORT SERVICES -- MANAGEMENT October 1989 + + +Security Considerations + + There are many security issues in the application and support + programs of host software, but a full discussion is beyond the scope + of this RFC. Security-related issues are mentioned in sections + concerning TFTP (Sections 4.2.1, 4.2.3.4, 4.2.3.5), the SMTP VRFY and + EXPN commands (Section 5.2.3), the SMTP HELO command (5.2.5), and the + SMTP DATA command (Section 5.2.8). + +Author's Address + + Robert Braden + USC/Information Sciences Institute + 4676 Admiralty Way + Marina del Rey, CA 90292-6695 + + Phone: (213) 822 1511 + + EMail: [email protected] + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + +Internet Engineering Task Force [Page 98] + diff --git a/lib/inets/doc/archive/rfc1945.html b/lib/inets/doc/archive/rfc1945.html new file mode 100644 index 0000000000..cf88f53319 --- /dev/null +++ b/lib/inets/doc/archive/rfc1945.html @@ -0,0 +1,3035 @@ +<!-- + ``The contents of this file are subject to the Erlang Public License, + Version 1.1, (the "License"); you may not use this file except in + compliance with the License. You should have received a copy of the + Erlang Public License along with this software. If not, it can be + retrieved via the world wide web at http://www.erlang.org/. + + Software distributed under the License is distributed on an "AS IS" + basis, WITHOUT WARRANTY OF ANY KIND, either express or implied. See + the License for the specific language governing rights and limitations + under the License. + + The Initial Developer of the Original Code is Ericsson Utvecklings AB. + Portions created by Ericsson are Copyright 1999, Ericsson Utvecklings + AB. All Rights Reserved.'' + + $Id$ +--> +<HTML><HEAD> +<TITLE>Hypertext Transfer Protocol -- HTTP/1.0</TITLE> +</HEAD><BODY> +<PRE> +Network Working Group T. Berners-Lee, MIT/LCS +Request for Comments: 1945 R. Fielding, UC Irvine +Category: Informational H. Frystyk, MIT/LCS + May 1996 +</PRE> + +<H1 ALIGN=CENTER>Hypertext Transfer Protocol -- HTTP/1.0</H1> + +<H2><A NAME="Status">Status of this Memo</A></H2> + + This memo provides information for the Internet community. This memo + does not specify an Internet standard of any kind. Distribution of + this memo is unlimited. + +<H2>IESG Note:</H2> + + The IESG has concerns about this protocol, and expects this document + to be replaced relatively soon by a standards track document. + +<H2><A NAME="Abstract">Abstract</A></H2> + + The Hypertext Transfer Protocol (HTTP) is an application-level protocol with the lightness and + speed necessary for distributed, collaborative, hypermedia information systems. It is a generic, + stateless, object-oriented protocol which can be used for many tasks, such as name servers and + distributed object management systems, through extension of its request methods (commands). + A feature of HTTP is the typing of data representation, allowing systems to be built + independently of the data being transferred. +<P> + + HTTP has been in use by the World-Wide Web global information initiative since 1990. This + specification reflects common usage of the protocol referred to as "HTTP/1.0". +<P> + +<H2><A NAME="Contents">Table of Contents</A></H2> + +<PRE> +1. <A HREF="#Introduction">Introduction</A> + 1.1 <A HREF="#Purpose">Purpose</A> + 1.2 <A HREF="#Terminology">Terminology</A> + 1.3 <A HREF="#Operation">Overall Operation</A> + 1.4 <A HREF="#HTTP-and-MIME">HTTP and MIME</A> + +2. <A HREF="#Grammar">Notational Conventions and Generic Grammar</A> + 2.1 <A HREF="#Augmented-BNF">Augmented BNF</A> + 2.2 <A HREF="#Basic-Rules">Basic Rules</A> + +3. <A HREF="#Protocol-Parameters">Protocol Parameters</A> + 3.1 <A HREF="#HTTP-Version">HTTP Version</A> + 3.2 <A HREF="#URI">Uniform Resource Identifiers</A> + 3.2.1 <A HREF="#URI-syntax">General Syntax</A> + 3.2.2 <A HREF="#http-URL">http URL</A> + 3.3 <A HREF="#DateFormats">Date/Time Formats</A> + 3.4 <A HREF="#Charset">Character Sets</A> + 3.5 <A HREF="#Content-Codings">Content Codings</A> + 3.6 <A HREF="#Media-Types">Media Types</A> + 3.6.1 <A HREF="#TextCanonicalization">Canonicalization and Text Defaults</A> + 3.6.2 <A HREF="#Multipart">Multipart Types</A> + 3.7 <A HREF="#Product">Product Tokens</A> + +4. <A HREF="#Message">HTTP Message</A> + 4.1 <A HREF="#Message-Types">Message Types</A> + 4.2 <A HREF="#Message-Headers">Message Headers</A> + 4.3 <A HREF="#General-Header">General Header Fields</A> + +5. <A HREF="#Request">Request</A> + 5.1 <A HREF="#Request-Line">Request-Line</A> + 5.1.1 <A HREF="#Method">Method</A> + 5.1.2 <A HREF="#Request-URI">Request-URI</A> + 5.2 <A HREF="#Request-Header">Request Header Fields</A> + +6. <A HREF="#Response">Response</A> + 6.1 <A HREF="#Status-Line">Status-Line</A> + 6.1.1 <A HREF="#Status-Code">Status Code and Reason Phrase</A> + 6.2 <A HREF="#Response-Header">Response Header Fields</A> + +7. <A HREF="#Entity">Entity</A> + 7.1 <A HREF="#Entity-Header">Entity Header Fields</A> + 7.2 <A HREF="#Entity-Body">Entity Body</A> + 7.2.1 <A HREF="#BodyType">Type</A> + 7.2.2 <A HREF="#BodyLength">Length</A> + +8. <A HREF="#Methods">Method Definitions</A> + 8.1 <A HREF="#GET">GET</A> + 8.2 <A HREF="#HEAD">HEAD</A> + 8.3 <A HREF="#POST">POST</A> + +9. <A HREF="#Status-Codes">Status Code Definitions</A> + 9.1 <A HREF="#Code1xx">Informational 1xx</A> + 9.2 <A HREF="#Code2xx">Successful 2xx</A> + <A HREF="#Code200">200 OK</A> + <A HREF="#Code201">201 Created</A> + <A HREF="#Code202">202 Accepted</A> + <A HREF="#Code204">204 No Content</A> + 9.3 <A HREF="#Code3xx">Redirection 3xx</A> + <A HREF="#Code300">300 Multiple Choices</A> + <A HREF="#Code301">301 Moved Permanently</A> + <A HREF="#Code302">302 Moved Temporarily</A> + <A HREF="#Code304">304 Not Modified</A> + 9.4 <A HREF="#Code4xx">Client Error 4xx</A> + <A HREF="#Code400">400 Bad Request</A> + <A HREF="#Code401">401 Unauthorized</A> + <A HREF="#Code403">403 Forbidden</A> + <A HREF="#Code404">404 Not Found</A> + 9.5 <A HREF="#Code5xx">Server Error 5xx</A> + <A HREF="#Code500">500 Internal Server Error</A> + <A HREF="#Code501">501 Not Implemented</A> + <A HREF="#Code502">502 Bad Gateway</A> + <A HREF="#Code503">503 Service Unavailable</A> + +10. <A HREF="#HeaderFields">Header Field Definitions</A> + 10.1 <A HREF="#Allow">Allow</A> + 10.2 <A HREF="#Authorization">Authorization</A> + 10.3 <A HREF="#Content-Encoding">Content-Encoding</A> + 10.4 <A HREF="#Content-Length">Content-Length</A> + 10.5 <A HREF="#Content-Type">Content-Type</A> + 10.6 <A HREF="#Date">Date</A> + 10.7 <A HREF="#Expires">Expires</A> + 10.8 <A HREF="#From">From</A> + 10.9 <A HREF="#If-Modified-Since">If-Modified-Since</A> + 10.10 <A HREF="#Last-Modified">Last-Modified</A> + 10.11 <A HREF="#Location">Location</A> + 10.12 <A HREF="#Pragma">Pragma</A> + 10.13 <A HREF="#Referer">Referer</A> + 10.14 <A HREF="#Server">Server</A> + 10.15 <A HREF="#User-Agent">User-Agent</A> + 10.16 <A HREF="#WWW-Authenticate">WWW-Authenticate</A> + +11. <A HREF="#AA">Access Authentication</A> + 11.1 <A HREF="#BasicAA">Basic Authentication Scheme</A> + +12. <A HREF="#Security">Security Considerations</A> + 12.1 <A HREF="#AuthSecurity">Authentication of Clients</A> + 12.2 <A HREF="#SafeMethods">Safe Methods</A> + 12.3 <A HREF="#LogAbuse">Abuse of Server Log Information</A> + 12.4 <A HREF="#Sensitive">Transfer of Sensitive Information</A> + 12.5 <A HREF="#PathNameSecurity">Attacks Based On File and Path Names</A> + +13. <A HREF="#Acknowledgments">Acknowledgments</A> + +14. <A HREF="#References">References</A> + +15. <A HREF="#Authors">Authors' Addresses</A> + +Appendix A. <A HREF="#message_http">Internet Media Type message/http</A> + +Appendix B. <A HREF="#Tolerant">Tolerant Applications</A> + +Appendix C. <A HREF="#MIME">Relationship to MIME</A> + C.1 <A HREF="#MIME-Canonical">Conversion to Canonical Form</A> + C.2 <A HREF="#MIME-Date">Conversion of Date Formats</A> + C.3 <A HREF="#MIME-CE">Introduction of Content-Encoding</A> + C.4 <A HREF="#MIME-CTE">No Content-Transfer-Encoding</A> + C.5 <A HREF="#MIME-parts">HTTP Header Fields in Multipart Body-Parts</A> + +Appendix D. <A HREF="#Additional">Additional Features</A> + D.1 <A HREF="#Additional-Methods">Additional Request Methods</A> + D.1.1 <A HREF="#PUT">PUT</A> + D.1.2 <A HREF="#DELETE">DELETE</A> + D.1.3 <A HREF="#LINK">LINK</A> + D.1.4 <A HREF="#UNLINK">UNLINK</A> + D.2 <A HREF="#Additional-Headers">Additional Header Field Definitions</A> + D.2.1 <A HREF="#Accept">Accept</A> + D.2.2 <A HREF="#Accept-Charset">Accept-Charset</A> + D.2.3 <A HREF="#Accept-Encoding">Accept-Encoding</A> + D.2.4 <A HREF="#Accept-Language">Accept-Language</A> + D.2.5 <A HREF="#Content-Language">Content-Language</A> + D.2.6 <A HREF="#Link">Link</A> + D.2.7 <A HREF="#MIME-Version">MIME-Version</A> + D.2.8 <A HREF="#Retry-After">Retry-After</A> + D.2.9 <A HREF="#Title">Title</A> + D.2.10 <A HREF="#URI-header">URI</A> + + +</PRE> + +<H2>1. <A NAME="Introduction">Introduction</A></H2> + +<H3>1.1 <A NAME="Purpose">Purpose</A></H3> + + The Hypertext Transfer Protocol (HTTP) is an application-level protocol with the lightness and + speed necessary for distributed, collaborative, hypermedia information systems. HTTP has + been in use by the World-Wide Web global information initiative since 1990. This specification + reflects common usage of the protocol referred to as "HTTP/1.0". This specification describes + the features that seem to be consistently implemented in most HTTP/1.0 clients and servers. + The specification is split into two sections. Those features of HTTP for which implementations + are usually consistent are described in the main body of this document. Those features which + have few or inconsistent implementations are listed in <A HREF="#Additional">Appendix D</A>. +<P> + + Practical information systems require more functionality than simple retrieval, including + search, front-end update, and annotation. HTTP allows an open-ended set of methods to be + used to indicate the purpose of a request. It builds on the discipline of reference provided by + the Uniform Resource Identifier (URI)<A HREF="#RefURI"> [2]</A>, as a location (URL)<A HREF="#RefURL"> [4]</A> or name (URN)<A HREF="#RefURN"> [16]</A>, for + indicating the resource on which a method is to be applied. Messages are passed in a format + similar to that used by Internet Mail<A HREF="#RefSTD11"> [7]</A> and the Multipurpose Internet Mail Extensions + (MIME)<A HREF="#RefMIME1"> [5]</A>. +<P> + + HTTP is also used as a generic protocol for communication between user agents and + proxies/gateways to other Internet protocols, such as SMTP<A HREF="#RefSMTP"> [12]</A>, NNTP<A HREF="#RefNNTP"> [11]</A>, FTP<A HREF="#RefFTP"> [14]</A>, + Gopher<A HREF="#RefGopher"> [1]</A>, and WAIS<A HREF="#RefWAIS"> [8]</A>, allowing basic hypermedia access to resources available from + diverse applications and simplifying the implementation of user agents. +<P> + +<H3>1.2 <A NAME="Terminology">Terminology</A></H3> + + This specification uses a number of terms to refer to the roles played by participants in, and + objects of, the HTTP communication. +<P> + +<DL> +<DT>connection +<DD>A transport layer virtual circuit established between two application programs for the + purpose of communication. + +<P> +<DT>message +<DD>The basic unit of HTTP communication, consisting of a structured sequence of octets + matching the syntax defined in <A HREF="#Message">Section 4</A> and transmitted via the connection. + +<P> +<DT>request +<DD>An HTTP request message (as defined in <A HREF="#Request">Section 5</A>). + +<P> +<DT>response +<DD>An HTTP response message (as defined in <A HREF="#Response">Section 6</A>). + +<P> +<DT>resource +<DD>A network data object or service which can be identified by a URI (<A HREF="#URI">Section 3.2</A>). + +<P> +<DT>entity +<DD>A particular representation or rendition of a data resource, or reply from a service resource, + that may be enclosed within a request or response message. An entity consists of + metainformation in the form of entity headers and content in the form of an entity body. + +<P> +<DT>client +<DD>An application program that establishes connections for the purpose of sending requests. + +<P> +<DT>user agent +<DD>The client which initiates a request. These are often browsers, editors, spiders + (web-traversing robots), or other end user tools. + +<P> +<DT>server +<DD>An application program that accepts connections in order to service requests by sending + back responses. + +<P> +<DT>origin server +<DD>The server on which a given resource resides or is to be created. + +<P> +<DT>proxy +<DD>An intermediary program which acts as both a server and a client for the purpose of making + requests on behalf of other clients. Requests are serviced internally or by passing them, with + possible translation, on to other servers. A proxy must interpret and, if necessary, rewrite a + request message before forwarding it. Proxies are often used as client-side portals through + network firewalls and as helper applications for handling requests via protocols not + implemented by the user agent. + +<P> +<DT>gateway +<DD>A server which acts as an intermediary for some other server. Unlike a proxy, a gateway + receives requests as if it were the origin server for the requested resource; the requesting + client may not be aware that it is communicating with a gateway. Gateways are often used + as server-side portals through network firewalls and as protocol translators for access to + resources stored on non-HTTP systems. + +<P> +<DT>tunnel +<DD>A tunnel is an intermediary program which is acting as a blind relay between two + connections. Once active, a tunnel is not considered a party to the HTTP communication, + though the tunnel may have been initiated by an HTTP request. The tunnel ceases to exist + when both ends of the relayed connections are closed. Tunnels are used when a portal is + necessary and the intermediary cannot, or should not, interpret the relayed communication. + +<P> +<DT>cache +<DD>A program's local store of response messages and the subsystem that controls its message + storage, retrieval, and deletion. A cache stores cachable responses in order to reduce the + response time and network bandwidth consumption on future, equivalent requests. Any + client or server may include a cache, though a cache cannot be used by a server while it is + acting as a tunnel. +</DL> + + Any given program may be capable of being both a client and a server; our use of these terms + refers only to the role being performed by the program for a particular connection, rather than + to the program's capabilities in general. Likewise, any server may act as an origin server, proxy, + gateway, or tunnel, switching behavior based on the nature of each request. +<P> + +<H3>1.3 <A NAME="Operation">Overall Operation</A></H3> + + The HTTP protocol is based on a request/response paradigm. A client establishes a connection + with a server and sends a request to the server in the form of a request method, URI, and + protocol version, followed by a MIME-like message containing request modifiers, client + information, and possible body content. The server responds with a status line, including the + message's protocol version and a success or error code, followed by a MIME-like message + containing server information, entity metainformation, and possible body content. +<P> + + Most HTTP communication is initiated by a user agent and consists of a request to be applied + to a resource on some origin server. In the simplest case, this may be accomplished via a single + connection (v) between the user agent (UA) and the origin server (O). +<P> + +<PRE> + request chain ------------------------> + UA -------------------v------------------- O + <----------------------- response chain +</PRE> + + A more complicated situation occurs when one or more intermediaries are present in the + request/response chain. There are three common forms of intermediary: proxy, gateway, and + tunnel. A proxy is a forwarding agent, receiving requests for a URI in its absolute form, + rewriting all or parts of the message, and forwarding the reformatted request toward the server + identified by the URI. A gateway is a receiving agent, acting as a layer above some other + server(s) and, if necessary, translating the requests to the underlying server's protocol. A tunnel + acts as a relay point between two connections without changing the messages; tunnels are used + when the communication needs to pass through an intermediary (such as a firewall) even when + the intermediary cannot understand the contents of the messages. +<P> + +<PRE> + request chain --------------------------------------> + UA -----v----- A -----v----- B -----v----- C -----v----- O + <------------------------------------- response chain +</PRE> + + The figure above shows three intermediaries (A, B, and C) between the user agent and origin + server. A request or response message that travels the whole chain must pass through four + separate connections. This distinction is important because some HTTP communication + options may apply only to the connection with the nearest, non-tunnel neighbor, only to the + end-points of the chain, or to all connections along the chain. Although the diagram is linear, + each participant may be engaged in multiple, simultaneous communications. For example, B + may be receiving requests from many clients other than A, and/or forwarding requests to + servers other than C, at the same time that it is handling A's request. +<P> + + Any party to the communication which is not acting as a tunnel may employ an internal cache + for handling requests. The effect of a cache is that the request/response chain is shortened if + one of the participants along the chain has a cached response applicable to that request. The + following illustrates the resulting chain if B has a cached copy of an earlier response from O + (via C) for a request which has not been cached by UA or A. +<P> + +<PRE> + request chain ----------> + UA -----v----- A -----v----- B - - - - - - C - - - - - - O + <--------- response chain +</PRE> + + Not all responses are cachable, and some requests may contain modifiers which place special + requirements on cache behavior. Some HTTP/1.0 applications use heuristics to describe what + is or is not a "cachable" response, but these rules are not standardized. +<P> + + On the Internet, HTTP communication generally takes place over TCP/IP connections. The + default port is TCP 80<A HREF="#RefIANA"> [15]</A>, but other ports can be used. This does not preclude HTTP from + being implemented on top of any other protocol on the Internet, or on other networks. HTTP + only presumes a reliable transport; any protocol that provides such guarantees can be used, and + the mapping of the HTTP/1.0 request and response structures onto the transport data units of + the protocol in question is outside the scope of this specification. +<P> + + Except for experimental applications, current practice requires that the connection be + established by the client prior to each request and closed by the server after sending the + response. Both clients and servers should be aware that either party may close the connection + prematurely, due to user action, automated time-out, or program failure, and should handle + such closing in a predictable fashion. In any case, the closing of the connection by either or both + parties always terminates the current request, regardless of its status. +<P> + +<H3>1.4 <A NAME="HTTP-and-MIME">HTTP and MIME</A></H3> + + HTTP/1.0 uses many of the constructs defined for MIME, as defined in RFC 1521<A HREF="#RefMIME1"> [5]</A>. + <A HREF="#MIME">Appendix C</A> describes the ways in which the context of HTTP allows for different use of + Internet Media Types than is typically found in Internet mail, and gives the rationale for those + differences. +<P> + +<H2>2. <A NAME="Grammar">Notational Conventions and Generic Grammar</A></H2> + +<H3>2.1 <A NAME="Augmented-BNF">Augmented BNF</A></H3> + + All of the mechanisms specified in this document are described in both prose and an augmented + Backus-Naur Form (BNF) similar to that used by RFC 822<A HREF="#RefSTD11"> [7]</A>. Implementors will need to be + familiar with the notation in order to understand this specification. The augmented BNF + includes the following constructs: +<P> + +<DL> +<DT><CODE>name = definition</CODE> +<DD>The name of a rule is simply the name itself (without any enclosing <CODE>"<"</CODE> and <CODE>">"</CODE>) and is + separated from its definition by the equal character <CODE>"="</CODE>. Whitespace is only significant in + that indentation of continuation lines is used to indicate a rule definition that spans more + than one line. Certain basic rules are in uppercase, such as <CODE>SP</CODE>, <CODE>LWS</CODE>, <CODE>HT</CODE>, <CODE>CRLF</CODE>, <CODE>DIGIT</CODE>, + <CODE>ALPHA</CODE>, etc. Angle brackets are used within definitions whenever their presence will + facilitate discerning the use of rule names. + +<P> +<DT><CODE>"literal"</CODE> +<DD>Quotation marks surround literal text. Unless stated otherwise, the text is case-insensitive. + +<P> +<DT><CODE>rule1 | rule2</CODE> +<DD>Elements separated by a bar (<CODE>"I"</CODE>) are alternatives, e.g., <CODE>"yes | no"</CODE> will accept <CODE>yes</CODE> or <CODE>no</CODE>. + +<P> +<DT><CODE>(rule1 rule2)</CODE> +<DD>Elements enclosed in parentheses are treated as a single element. Thus, + <CODE>"(elem (foo | bar) elem)"</CODE> allows the token sequences <CODE>"elem foo elem"</CODE> and <CODE>"elem bar elem"</CODE>. + +<P> +<DT><CODE>*rule</CODE> +<DD>The character <CODE>"*"</CODE> preceding an element indicates repetition. The full form is + <CODE>"<n>*<m>element"</CODE> indicating at least <CODE><n></CODE> and at most <CODE><m></CODE> occurrences of <CODE>element</CODE>. Default + values are <CODE>0</CODE> and infinity so that <CODE>"*(element)"</CODE> allows any number, including zero; <CODE>"1*element"</CODE> + requires at least one; and <CODE>"1*2element"</CODE> allows one or two. + +<P> +<DT><CODE>[rule]</CODE> +<DD>Square brackets enclose optional elements; <CODE>"[foo bar]"</CODE> is equivalent to <CODE>"*1(foo bar)"</CODE>. + +<P> +<DT><EM>N</EM> <CODE>rule</CODE> +<DD>Specific repetition: <CODE>"<n>(element)"</CODE> is equivalent to <CODE>"<n>*<n>(element)"</CODE>; that is, exactly <CODE><n></CODE> + occurrences of <CODE>(element)</CODE>. Thus <CODE>2DIGIT</CODE> is a 2-digit number, and <CODE>3ALPHA</CODE> is a string of three + alphabetic characters. + +<P> +<DT><CODE>#rule</CODE> +<DD>A construct <CODE>"#"</CODE> is defined, similar to <CODE>"*"</CODE>, for defining lists of elements. The full form is + <CODE>"<n>#<m>element"</CODE> indicating at least <CODE><n></CODE> and at most <CODE><m></CODE> elements, each separated by one + or more commas (<CODE>","</CODE>) and optional linear whitespace (LWS). This makes the usual form of + lists very easy; a rule such as <CODE>"( *LWS element *( *LWS "," *LWS element ))</CODE>" can be shown as + <CODE>"1#element"</CODE>. Wherever this construct is used, null elements are allowed, but do not + contribute to the count of elements present. That is, <CODE>"(element), , (element)"</CODE> is permitted, but + counts as only two elements. Therefore, where at least one element is required, at least one + non-null element must be present. Default values are <CODE>0</CODE> and infinity so that <CODE>"#(element)"</CODE> + allows any number, including zero; <CODE>"1#element"</CODE> requires at least one; and <CODE>"1#2element"</CODE> + allows one or two. + +<P> +<DT><CODE>; comment</CODE> +<DD>A semi-colon, set off some distance to the right of rule text, starts a comment that continues + to the end of line. This is a simple way of including useful notes in parallel with the + specifications. + +<P> +<DT><CODE>implied *LWS</CODE> +<DD>The grammar described by this specification is word-based. Except where noted otherwise, + linear whitespace (<CODE>LWS</CODE>) can be included between any two adjacent words (<CODE>token</CODE> or + <CODE>quoted-string</CODE>), and between adjacent tokens and delimiters (<CODE>tspecials</CODE>), without changing the + interpretation of a field. At least one delimiter (<CODE>tspecials</CODE>) must exist between any two + tokens, since they would otherwise be interpreted as a single token. However, applications + should attempt to follow "common form" when generating HTTP constructs, since there + exist some implementations that fail to accept anything beyond the common forms. +</DL> + +<H3>2.2 <A NAME="Basic-Rules">Basic Rules</A></H3> + + The following rules are used throughout this specification to describe basic parsing constructs. + The US-ASCII coded character set is defined by<A HREF="#RefASCII"> [17]</A>. +<P> + +<PRE> + OCTET = <any 8-bit sequence of data> + CHAR = <any US-ASCII character (octets 0 - 127)> + UPALPHA = <any US-ASCII uppercase letter "A".."Z"> + LOALPHA = <any US-ASCII lowercase letter "a".."z"> + ALPHA = UPALPHA | LOALPHA + DIGIT = <any US-ASCII digit "0".."9"> + CTL = <any US-ASCII control character + (octets 0 - 31) and DEL (127)> + CR = <US-ASCII CR, carriage return (13)> + LF = <US-ASCII LF, linefeed (10)> + SP = <US-ASCII SP, space (32)> + HT = <US-ASCII HT, horizontal-tab (9)> + <"> = <US-ASCII double-quote mark (34)> +</PRE> + + HTTP/1.0 defines the octet sequence <CODE>CR LF</CODE> as the end-of-line marker for all protocol elements + except the <CODE>Entity-Body</CODE> (see <A HREF="#Tolerant">Appendix B</A> for tolerant applications). The end-of-line marker + within an <CODE>Entity-Body</CODE> is defined by its associated media type, as described in <A HREF="#Media-Types">Section 3.6</A>. +<P> + +<PRE> + CRLF = CR LF +</PRE> + + HTTP/1.0 headers may be folded onto multiple lines if each continuation line begins with a + space or horizontal tab. All linear whitespace, including folding, has the same semantics as <CODE>SP</CODE>. +<P> + +<PRE> + LWS = [CRLF] 1*( SP | HT ) +</PRE> + + However, folding of header lines is not expected by some applications, and should not be + generated by HTTP/1.0 applications. +<P> + + The <CODE>TEXT</CODE> rule is only used for descriptive field contents and values that are not intended to be + interpreted by the message parser. Words of <CODE>*TEXT</CODE> may contain octets from character sets other + than US-ASCII. +<P> + +<PRE> + TEXT = <any OCTET except CTLs, + but including LWS> +</PRE> + + Recipients of header field <CODE>TEXT</CODE> containing octets outside the US-ASCII character set may + assume that they represent ISO-8859-1 characters. +<P> + + Hexadecimal numeric characters are used in several protocol elements. +<P> + +<PRE> + HEX = "A" | "B" | "C" | "D" | "E" | "F" + | "a" | "b" | "c" | "d" | "e" | "f" | DIGIT +</PRE> + + Many HTTP/1.0 header field values consist of words separated by <CODE>LWS</CODE> or special characters. + These special characters must be in a quoted string to be used within a parameter value. +<P> + +<PRE> + word = token | quoted-string +</PRE> + +<PRE> + token = 1*<any CHAR except CTLs or tspecials> +</PRE> + +<PRE> + tspecials = "(" | ")" | "<" | ">" | "@" + | "," | ";" | ":" | "\" | <"> + | "/" | "[" | "]" | "?" | "=" + | "{" | "}" | SP | HT +</PRE> + + Comments may be included in some HTTP header fields by surrounding the comment text with + parentheses. Comments are only allowed in fields containing "<CODE>comment</CODE>" as part of their field + value definition. In all other fields, parentheses are considered part of the field value. +<P> + +<PRE> + comment = "(" *( ctext | comment ) ")" + ctext = <any TEXT excluding "(" and ")"> +</PRE> + + A string of text is parsed as a single word if it is quoted using double-quote marks. +<P> + +<PRE> + quoted-string = ( <"> *(qdtext) <"> ) +</PRE> + +<PRE> + qdtext = <any CHAR except <"> and CTLs, + but including LWS> +</PRE> + + Single-character quoting using the backslash ("\") character is not permitted in HTTP/1.0. +<P> + +<H2>3. <A NAME="Protocol-Parameters">Protocol Parameters</A></H2> + +<H3>3.1 <A NAME="HTTP-Version">HTTP Version</A></H3> + + HTTP uses a "<major>.<minor>" numbering scheme to indicate versions of the protocol. The + protocol versioning policy is intended to allow the sender to indicate the format of a message + and its capacity for understanding further HTTP communication, rather than the features + obtained via that communication. No change is made to the version number for the addition of + message components which do not affect communication behavior or which only add to + extensible field values. The <minor> number is incremented when the changes made to the + protocol add features which do not change the general message parsing algorithm, but which + may add to the message semantics and imply additional capabilities of the sender. The <major> + number is incremented when the format of a message within the protocol is changed. +<P> + + The version of an HTTP message is indicated by an <CODE>HTTP-Version</CODE> field in the first line of the + message. If the protocol version is not specified, the recipient must assume that the message is + in the simple HTTP/0.9 format. +<P> + +<PRE> + HTTP-Version = "HTTP" "/" 1*DIGIT "." 1*DIGIT +</PRE> + + Note that the major and minor numbers should be treated as separate integers and that each may + be incremented higher than a single digit. Thus, HTTP/2.4 is a lower version than HTTP/2.13, + which in turn is lower than HTTP/12.3. Leading zeros should be ignored by recipients and + never generated by senders. +<P> + + This document defines both the 0.9 and 1.0 versions of the HTTP protocol. Applications + sending <CODE>Full-Request</CODE> or <CODE>Full-Response</CODE> messages, as defined by this specification, must include + an <CODE>HTTP-Version</CODE> of "<CODE>HTTP/1.0</CODE>". +<P> + +HTTP/1.0 servers must: +<UL> +<LI>recognize the format of the <CODE>Request-Line</CODE> for HTTP/0.9 and HTTP/1.0 requests; +<LI>understand any valid request in the format of HTTP/0.9 or HTTP/1.0; +<LI>respond appropriately with a message in the same protocol version used by the client. +</UL> + +HTTP/1.0 clients must: +<UL> +<LI>recognize the format of the <CODE>Status-Line</CODE> for HTTP/1.0 responses; +<LI>understand any valid response in the format of HTTP/0.9 or HTTP/1.0. +</UL> + + Proxy and gateway applications must be careful in forwarding requests that are received in a + format different than that of the application's native HTTP version. Since the protocol version + indicates the protocol capability of the sender, a proxy/gateway must never send a message + with a version indicator which is greater than its native version; if a higher version request is + received, the proxy/gateway must either downgrade the request version or respond with an + error. Requests with a version lower than that of the application's native format may be + upgraded before being forwarded; the proxy/gateway's response to that request must follow the + server requirements listed above. +<P> + +<H3>3.2 <A NAME="URI">Uniform Resource Identifiers</A></H3> + + URIs have been known by many names: WWW addresses, Universal Document Identifiers, + Universal Resource Identifiers<A HREF="#RefURI"> [2]</A>, and finally the combination of Uniform Resource Locators + (URL)<A HREF="#RefURL"> [4]</A> and Names (URN)<A HREF="#RefURN"> [16]</A>. As far as HTTP is concerned, Uniform Resource Identifiers + are simply formatted strings which identify--via name, location, or any other characteristic--a + network resource. +<P> + +<H4>3.2.1 <A NAME="URI-syntax">General Syntax</A></H4> + + URIs in HTTP can be represented in absolute form or relative to some known base URI<A HREF="#RefRelURL"> [9]</A>, + depending upon the context of their use. The two forms are differentiated by the fact that + absolute URIs always begin with a scheme name followed by a colon. +<P> + +<PRE> + URI = ( absoluteURI | relativeURI ) [ "#" fragment ] +</PRE> + +<PRE> + absoluteURI = scheme ":" *( uchar | reserved ) +</PRE> + +<PRE> + relativeURI = net_path | abs_path | rel_path +</PRE> + +<PRE> + net_path = "//" net_loc [ abs_path ] + abs_path = "/" rel_path + rel_path = [ path ] [ ";" params ] [ "?" query ] +</PRE> + +<PRE> + path = fsegment *( "/" segment ) + fsegment = 1*pchar + segment = *pchar +</PRE> + +<PRE> + params = param *( ";" param ) + param = *( pchar | "/" ) +</PRE> + +<PRE> + scheme = 1*( ALPHA | DIGIT | "+" | "-" | "." ) + net_loc = *( pchar | ";" | "?" ) + query = *( uchar | reserved ) + fragment = *( uchar | reserved ) +</PRE> + +<PRE> + pchar = uchar | ":" | "@" | "&" | "=" | "+" + uchar = unreserved | escape + unreserved = ALPHA | DIGIT | safe | extra | national +</PRE> + +<PRE> + escape = "%" HEX HEX + reserved = ";" | "/" | "?" | ":" | "@" | "&" | "=" | "+" + extra = "!" | "*" | "'" | "(" | ")" | "," + safe = "$" | "-" | "_" | "." + unsafe = CTL | SP | <"> | "#" | "%" | "<" | ">" + national = <any OCTET excluding ALPHA, DIGIT, + reserved, extra, safe, and unsafe> +</PRE> + + For definitive information on URL syntax and semantics, see RFC 1738<A HREF="#RefURL"> [4]</A> and RFC 1808<A HREF="#RefRelURL"> [9]</A>. + The BNF above includes <CODE>national</CODE> characters not allowed in valid URLs as specified by + RFC 1738, since HTTP servers are not restricted in the set of <CODE>unreserved</CODE> characters allowed to + represent the <CODE>rel_path</CODE> part of addresses, and HTTP proxies may receive requests for URIs not + defined by RFC 1738. +<P> + +<H4>3.2.2 <A NAME="http-URL">http URL</A></H4> + + The "http" scheme is used to locate network resources via the HTTP protocol. This section + defines the scheme-specific syntax and semantics for http URLs. +<P> + +<PRE> + http_URL = "http:" "//" host [ ":" port ] [ abs_path ] +</PRE> + +<PRE> + host = <A legal Internet host domain name + or IP address (in dotted-decimal form), + as defined by Section 2.1 of RFC 1123> +</PRE> + +<PRE> + port = *DIGIT +</PRE> + + If the <CODE>port</CODE> is empty or not given, port 80 is assumed. The semantics are that the identified + resource is located at the server listening for TCP connections on that <CODE>port</CODE> of that <CODE>host</CODE>, and the + <CODE>Request-URI</CODE> for the resource is <CODE>abs_path</CODE>. If the <CODE>abs_path</CODE> is not present in the URL, it must be + given as "/" when used as a <CODE>Request-URI</CODE> (<A HREF="#Request-URI">Section 5.1.2</A>). +<P> + +<BLOCKQUOTE> + Note: Although the HTTP protocol is independent of the transport layer protocol, the + http URL only identifies resources by their TCP location, and thus non-TCP resources + must be identified by some other URI scheme. +</BLOCKQUOTE> + + The canonical form for "http" URLs is obtained by converting any <CODE>UPALPHA</CODE> characters in <CODE>host</CODE> + to their <CODE>LOALPHA</CODE> equivalent (hostnames are case-insensitive), eliding the <CODE>[ ":" port ]</CODE> if the port + is 80, and replacing an empty <CODE>abs_path</CODE> with "/". +<P> + +<H3>3.3 <A NAME="DateFormats">Date/Time Formats</A></H3> + + HTTP/1.0 applications have historically allowed three different formats for the representation + of date/time stamps: +<P> + +<PRE> + Sun, 06 Nov 1994 08:49:37 GMT ; RFC 822, updated by RFC 1123 + Sunday, 06-Nov-94 08:49:37 GMT ; RFC 850, obsoleted by RFC 1036 + Sun Nov 6 08:49:37 1994 ; ANSI C's asctime() format +</PRE> + + The first format is preferred as an Internet standard and represents a fixed-length subset of that + defined by RFC 1123<A HREF="#RefSTD3"> [6]</A> (an update to RFC 822<A HREF="#RefSTD11"> [7]</A>). The second format is in common use, but + is based on the obsolete RFC 850<A HREF="#RefUSENET"> [10]</A> date format and lacks a four-digit year. HTTP/1.0 clients + and servers that parse the date value should accept all three formats, though they must never + generate the third (asctime) format. +<P> + +<BLOCKQUOTE> + Note: Recipients of date values are encouraged to be robust in accepting date values + that may have been generated by non-HTTP applications, as is sometimes the case + when retrieving or posting messages via proxies/gateways to SMTP or NNTP. +</BLOCKQUOTE> + + All HTTP/1.0 date/time stamps must be represented in Universal Time (UT), also known as + Greenwich Mean Time (GMT), without exception. This is indicated in the first two formats by + the inclusion of "GMT" as the three-letter abbreviation for time zone, and should be assumed + when reading the asctime format. +<P> + +<PRE> + HTTP-date = rfc1123-date | rfc850-date | asctime-date +</PRE> + +<PRE> + rfc1123-date = wkday "," SP date1 SP time SP "GMT" + rfc850-date = weekday "," SP date2 SP time SP "GMT" + asctime-date = wkday SP date3 SP time SP 4DIGIT +</PRE> + +<PRE> + date1 = 2DIGIT SP month SP 4DIGIT + ; day month year (e.g., 02 Jun 1982) + date2 = 2DIGIT "-" month "-" 2DIGIT + ; day-month-year (e.g., 02-Jun-82) + date3 = month SP ( 2DIGIT | ( SP 1DIGIT )) + ; month day (e.g., Jun 2) +</PRE> + +<PRE> + time = 2DIGIT ":" 2DIGIT ":" 2DIGIT + ; 00:00:00 - 23:59:59 +</PRE> + +<PRE> + wkday = "Mon" | "Tue" | "Wed" + | "Thu" | "Fri" | "Sat" | "Sun" +</PRE> + +<PRE> + weekday = "Monday" | "Tuesday" | "Wednesday" + | "Thursday" | "Friday" | "Saturday" | "Sunday" +</PRE> + +<PRE> + month = "Jan" | "Feb" | "Mar" | "Apr" + | "May" | "Jun" | "Jul" | "Aug" + | "Sep" | "Oct" | "Nov" | "Dec" +</PRE> + +<BLOCKQUOTE> + Note: HTTP requirements for the date/time stamp format apply only to their usage + within the protocol stream. Clients and servers are not required to use these formats + for user presentation, request logging, etc. +</BLOCKQUOTE> + +<H3>3.4 <A NAME="Charset">Character Sets</A></H3> + + HTTP uses the same definition of the term "character set" as that described for MIME: +<P> + +<BLOCKQUOTE> + The term "character set" is used in this document to refer to a method used with + one or more tables to convert a sequence of octets into a sequence of characters. + Note that unconditional conversion in the other direction is not required, in that + not all characters may be available in a given character set and a character set may + provide more than one sequence of octets to represent a particular character. This + definition is intended to allow various kinds of character encodings, from simple + single-table mappings such as US-ASCII to complex table switching methods + such as those that use ISO 2022's techniques. However, the definition associated + with a MIME character set name must fully specify the mapping to be performed + from octets to characters. In particular, use of external profiling information to + determine the exact mapping is not permitted. +</BLOCKQUOTE> + +<BLOCKQUOTE> + Note: This use of the term "character set" is more commonly referred to as a + "character encoding." However, since HTTP and MIME share the same registry, it is + important that the terminology also be shared. +</BLOCKQUOTE> + + HTTP character sets are identified by case-insensitive tokens. The complete set of tokens are + defined by the IANA Character Set registry<A HREF="#RefIANA"> [15]</A>. However, because that registry does not + define a single, consistent token for each character set, we define here the preferred names for + those character sets most likely to be used with HTTP entities. These character sets include + those registered by RFC 1521<A HREF="#RefMIME1"> [5]</A> -- the US-ASCII<A HREF="#RefASCII"> [17]</A> + and ISO-8859<A HREF="#RefISO8859"> [18]</A> character sets -- + and other names specifically recommended for use within MIME charset parameters. +<P> + +<PRE> + charset = "US-ASCII" + | "ISO-8859-1" | "ISO-8859-2" | "ISO-8859-3" + | "ISO-8859-4" | "ISO-8859-5" | "ISO-8859-6" + | "ISO-8859-7" | "ISO-8859-8" | "ISO-8859-9" + | "ISO-2022-JP" | "ISO-2022-JP-2" | "ISO-2022-KR" + | "UNICODE-1-1" | "UNICODE-1-1-UTF-7" | "UNICODE-1-1-UTF-8" + | token +</PRE> + + Although HTTP allows an arbitrary token to be used as a charset value, any token that has a + predefined value within the IANA Character Set registry<A HREF="#RefIANA"> [15]</A> must represent the character set + defined by that registry. Applications should limit their use of character sets to those defined by + the IANA registry. +<P> + + The character set of an entity body should be labelled as the lowest common denominator of + the character codes used within that body, with the exception that no label is preferred over the + labels US-ASCII or ISO-8859-1. +<P> + +<H3>3.5 <A NAME="Content-Codings">Content Codings</A></H3> + + Content coding values are used to indicate an encoding transformation that has been applied to + a resource. Content codings are primarily used to allow a document to be compressed or + encrypted without losing the identity of its underlying media type. Typically, the resource is + stored in this encoding and only decoded before rendering or analogous usage. +<P> + +<PRE> + content-coding = "x-gzip" | "x-compress" | token +</PRE> + +<BLOCKQUOTE> + Note: For future compatibility, HTTP/1.0 applications should consider "gzip" and + "compress" to be equivalent to "x-gzip" and "x-compress", respectively. +</BLOCKQUOTE> + + All <CODE>content-coding</CODE> values are case-insensitive. HTTP/1.0 uses <CODE>content-coding</CODE> values in the + <CODE>Content-Encoding</CODE> (<A HREF="#Content-Encoding">Section 10.3</A>) header field. Although the value describes the content-coding, + what is more important is that it indicates what decoding mechanism will be required to remove + the encoding. Note that a single program may be capable of decoding multiple content-coding + formats. Two values are defined by this specification: +<P> + +<DL> +<DT><CODE>x-gzip</CODE> +<DD>An encoding format produced by the file compression program "gzip" (GNU zip) + developed by Jean-loup Gailly. This format is typically a Lempel-Ziv coding (LZ77) with + a 32 bit CRC. + +<P> +<DT><CODE>x-compress</CODE> +<DD>The encoding format produced by the file compression program "compress". This format + is an adaptive Lempel-Ziv-Welch coding (LZW). +</DL> + +<BLOCKQUOTE> + Note: Use of program names for the identification of encoding formats is not + desirable and should be discouraged for future encodings. Their use here is + representative of historical practice, not good design. +</BLOCKQUOTE> + +<H3>3.6 <A NAME="Media-Types">Media Types</A></H3> + + HTTP uses Internet Media Types<A HREF="#RefMediaType"> [13]</A> in the <CODE>Content-Type</CODE> header field (<A HREF="#Content-Type">Section 10.5</A>) in order + to provide open and extensible data typing. +<P> + +<PRE> + media-type = type "/" subtype *( ";" parameter ) + type = token + subtype = token +</PRE> + + Parameters may follow the type/subtype in the form of attribute/value pairs. +<P> + +<PRE> + parameter = attribute "=" value + attribute = token + value = token | quoted-string +</PRE> + + The type, subtype, and parameter attribute names are case-insensitive. Parameter values may + or may not be case-sensitive, depending on the semantics of the parameter name. <CODE>LWS</CODE> must not + be generated between the type and subtype, nor between an attribute and its value. Upon receipt + of a media type with an unrecognized parameter, a user agent should treat the media type as if + the unrecognized parameter and its value were not present. +<P> + + Some older HTTP applications do not recognize media type parameters. HTTP/1.0 + applications should only use media type parameters when they are necessary to define the + content of a message. +<P> + + Media-type values are registered with the Internet Assigned Number Authority (IANA<A HREF="#RefIANA"> [15]</A>). + The media type registration process is outlined in RFC 1590<A HREF="#RefMediaType"> [13]</A>. Use of non-registered media + types is discouraged. +<P> + +<H4>3.6.1 <A NAME="TextCanonicalization">Canonicalization and Text Defaults</A></H4> + + Internet media types are registered with a canonical form. In general, an <CODE>Entity-Body</CODE> transferred + via HTTP must be represented in the appropriate canonical form prior to its transmission. If the + body has been encoded with a <CODE>Content-Encoding</CODE>, the underlying data should be in canonical + form prior to being encoded. +<P> + + Media subtypes of the "text" type use <CODE>CRLF</CODE> as the text line break when in canonical form. + However, HTTP allows the transport of text media with plain <CODE>CR</CODE> or <CODE>LF</CODE> alone representing a line + break when used consistently within the <CODE>Entity-Body</CODE>. HTTP applications must accept <CODE>CRLF</CODE>, bare + <CODE>CR</CODE>, and bare <CODE>LF</CODE> as being representative of a line break in text media received via HTTP. +<P> + + In addition, if the text media is represented in a character set that does not use octets 13 and 10 + for <CODE>CR</CODE> and <CODE>LF</CODE> respectively, as is the case for some multi-byte character sets, HTTP allows the + use of whatever octet sequences are defined by that character set to represent the equivalent of + <CODE>CR</CODE> and <CODE>LF</CODE> for line breaks. This flexibility regarding line breaks applies only to text media in + the <CODE>Entity-Body</CODE>; a bare <CODE>CR</CODE> or <CODE>LF</CODE> should not be substituted for <CODE>CRLF</CODE> within any of the HTTP + control structures (such as header fields and multipart boundaries). +<P> + + The "charset" parameter is used with some media types to define the character set (<A HREF="#Charset">Section 3.4</A>) + of the data. When no explicit charset parameter is provided by the sender, media subtypes of + the "text" type are defined to have a default charset value of "ISO-8859-1" when received via + HTTP. Data in character sets other than "ISO-8859-1" or its subsets must be labelled with an + appropriate charset value in order to be consistently interpreted by the recipient. +<P> + +<BLOCKQUOTE> + Note: Many current HTTP servers provide data using charsets other than + "ISO-8859-1" without proper labelling. This situation reduces interoperability and is + not recommended. To compensate for this, some HTTP user agents provide a + configuration option to allow the user to change the default interpretation of the + media type character set when no charset parameter is given. +</BLOCKQUOTE> + +<H4>3.6.2 <A NAME="Multipart">Multipart Types</A></H4> + + MIME provides for a number of "multipart" types -- encapsulations of several entities within + a single message's <CODE>Entity-Body</CODE>. The multipart types registered by IANA<A HREF="#RefIANA"> [15]</A> do not have any + special meaning for HTTP/1.0, though user agents may need to understand each type in order + to correctly interpret the purpose of each body-part. An HTTP user agent should follow the + same or similar behavior as a MIME user agent does upon receipt of a multipart type. HTTP + servers should not assume that all HTTP clients are prepared to handle multipart types. +<P> + + All multipart types share a common syntax and must include a boundary parameter as part of + the media type value. The message body is itself a protocol element and must therefore use only + <CODE>CRLF</CODE> to represent line breaks between body-parts. Multipart body-parts may contain HTTP + header fields which are significant to the meaning of that part. +<P> + +<H3>3.7 <A NAME="Product">Product Tokens</A></H3> + + Product tokens are used to allow communicating applications to identify themselves via a + simple product token, with an optional slash and version designator. Most fields using product + tokens also allow subproducts which form a significant part of the application to be listed, + separated by whitespace. By convention, the products are listed in order of their significance + for identifying the application. +<P> + +<PRE> + product = token ["/" product-version] + product-version = token +</PRE> + + Examples: +<P> + +<PRE> + User-Agent: CERN-LineMode/2.15 libwww/2.17b3 + + Server: Apache/0.8.4 +</PRE> + + Product tokens should be short and to the point -- use of them for advertizing or other + non-essential information is explicitly forbidden. Although any token character may appear in + a <CODE>product-version</CODE>, this token should only be used for a version identifier (i.e., successive versions + of the same product should only differ in the <CODE>product-version</CODE> portion of the <CODE>product</CODE> value). +<P> + +<H2>4. <A NAME="Message">HTTP Message</A></H2> + +<H3>4.1 <A NAME="Message-Types">Message Types</A></H3> + + HTTP messages consist of requests from client to server and responses from server to client. +<P> + +<PRE> + HTTP-message = Simple-Request ; HTTP/0.9 messages + | Simple-Response + | Full-Request ; HTTP/1.0 messages + | Full-Response +</PRE> + + <CODE>Full-Request</CODE> and <CODE>Full-Response</CODE> use the generic message format of RFC 822<A HREF="#RefSTD11"> [7]</A> for transferring + entities. Both messages may include optional header fields (also known as "headers") and an + entity body. The entity body is separated from the headers by a null line (i.e., a line with nothing + preceding the <CODE>CRLF</CODE>). +<P> + +<PRE> + Full-Request = Request-Line ; <A HREF="#Request-Line">Section 5.1</A> + *( General-Header ; <A HREF="#General-Header">Section 4.3</A> + | Request-Header ; <A HREF="#Request-Header">Section 5.2</A> + | Entity-Header ) ; <A HREF="#Entity-Header">Section 7.1</A> + CRLF + [ Entity-Body ] ; <A HREF="#Entity-Body">Section 7.2</A> +</PRE> + +<PRE> + Full-Response = Status-Line ; <A HREF="#Status-Line">Section 6.1</A> + *( General-Header ; <A HREF="#General-Header">Section 4.3</A> + | Response-Header ; <A HREF="#Response-Header">Section 6.2</A> + | Entity-Header ) ; <A HREF="#Entity-Header">Section 7.1</A> + CRLF + [ Entity-Body ] ; <A HREF="#Entity-Body">Section 7.2</A> +</PRE> + + <CODE>Simple-Request</CODE> and <CODE>Simple-Response</CODE> do not allow the use of any header information and are + limited to a single request method (<CODE>GET</CODE>). +<P> + +<PRE> + Simple-Request = "GET" SP Request-URI CRLF +</PRE> + +<PRE> + Simple-Response = [ Entity-Body ] +</PRE> + + Use of the <CODE>Simple-Request</CODE> format is discouraged because it prevents the server from identifying + the media type of the returned entity. +<P> + +<H3>4.2 <A NAME="Message-Headers">Message Headers</A></H3> + + HTTP header fields, which include <CODE>General-Header</CODE> (<A HREF="#General-Header">Section 4.3</A>), <CODE>Request-Header</CODE> (<A HREF="#Request-Header">Section 5.2</A>), + <CODE>Response-Header</CODE> (<A HREF="#Response-Header">Section 6.2</A>), and <CODE>Entity-Header</CODE> (<A HREF="#Entity-Header">Section 7.1</A>) fields, follow the same generic + format as that given in Section 3.1 of RFC 822<A HREF="#RefSTD11"> [7]</A>. Each header field consists of a name + followed immediately by a colon (<CODE>":"</CODE>), a single space (<CODE>SP</CODE>) character, and the field value. Field + names are case-insensitive. Header fields can be extended over multiple lines by preceding + each extra line with at least one <CODE>SP</CODE> or <CODE>HT</CODE>, though this is not recommended. +<P> + +<PRE> + HTTP-header = field-name ":" [ field-value ] CRLF +</PRE> + +<PRE> + field-name = token + field-value = *( field-content | LWS ) +</PRE> + +<PRE> + field-content = <the OCTETs making up the field-value + and consisting of either *TEXT or combinations + of token, tspecials, and quoted-string> +</PRE> + + The order in which header fields are received is not significant. However, it is "good practice" + to send <CODE>General-Header</CODE> fields first, followed by <CODE>Request-Header</CODE> or <CODE>Response-Header</CODE> fields prior + to the <CODE>Entity-Header</CODE> fields. +<P> + + Multiple <CODE>HTTP-header</CODE> fields with the same <CODE>field-name</CODE> may be present in a message if and only + if the entire <CODE>field-value</CODE> for that header field is defined as a comma-separated list [i.e., <CODE>#(values)</CODE>]. + It must be possible to combine the multiple header fields into one "field-name: field-value" pair, + without changing the semantics of the message, by appending each subsequent field-value to + the first, each separated by a comma. +<P> + +<H3>4.3 <A NAME="General-Header">General Header Fields</A></H3> + + There are a few header fields which have general applicability for both request and response + messages, but which do not apply to the entity being transferred. These headers apply only to + the message being transmitted. +<P> + +<PRE> + General-Header = Date ; <A HREF="#Date">Section 10.6</A> + | Pragma ; <A HREF="#Pragma">Section 10.12</A> +</PRE> + + General header field names can be extended reliably only in combination with a change in the + protocol version. However, new or experimental header fields may be given the semantics of + general header fields if all parties in the communication recognize them to be general header + fields. Unrecognized header fields are treated as <CODE>Entity-Header</CODE> fields. +<P> + +<H2>5. <A NAME="Request">Request</A></H2> + + A request message from a client to a server includes, within the first line of that message, the + method to be applied to the resource, the identifier of the resource, and the protocol version in + use. For backwards compatibility with the more limited HTTP/0.9 protocol, there are two valid + formats for an HTTP request: +<P> + +<PRE> + Request = Simple-Request | Full-Request +</PRE> + +<PRE> + Simple-Request = "GET" SP Request-URI CRLF +</PRE> + +<PRE> + Full-Request = Request-Line ; <A HREF="#Request-Line">Section 5.1</A> + *( General-Header ; <A HREF="#General-Header">Section 4.3</A> + | Request-Header ; <A HREF="#Request-Header">Section 5.2</A> + | Entity-Header ) ; <A HREF="#Entity-Header">Section 7.1</A> + CRLF + [ Entity-Body ] ; <A HREF="#Entity-Body">Section 7.2</A> +</PRE> + + If an HTTP/1.0 server receives a <CODE>Simple-Request</CODE>, it must respond with an HTTP/0.9 + <CODE>Simple-Response</CODE>. An HTTP/1.0 client capable of receiving a <CODE>Full-Response</CODE> should never + generate a <CODE>Simple-Request</CODE>. +<P> + +<H3>5.1 <A NAME="Request-Line">Request-Line</A></H3> + + The <CODE>Request-Line</CODE> begins with a method token, followed by the <CODE>Request-URI</CODE> and the protocol + version, and ending with <CODE>CRLF</CODE>. The elements are separated by <CODE>SP</CODE> characters. + No <CODE>CR</CODE> or <CODE>LF</CODE> are allowed except in the final <CODE>CRLF</CODE> sequence. +<P> + +<PRE> + Request-Line = Method SP Request-URI SP HTTP-Version CRLF +</PRE> + + Note that the difference between a <CODE>Simple-Request</CODE> and the <CODE>Request-Line</CODE> of a <CODE>Full-Request</CODE> is the + presence of the <CODE>HTTP-Version</CODE> field and the availability of methods other than <CODE>GET</CODE>. +<P> + +<H4>5.1.1 <A NAME="Method">Method</A></H4> + + The <CODE>Method</CODE> token indicates the method to be performed on the resource identified by the + <CODE>Request-URI</CODE>. The method is case-sensitive. +<P> + +<PRE> + Method = "GET" ; <A HREF="#GET">Section 8.1</A> + | "HEAD" ; <A HREF="#HEAD">Section 8.2</A> + | "POST" ; <A HREF="#POST">Section 8.3</A> + | extension-method +</PRE> + +<PRE> + extension-method = token +</PRE> + + The list of methods acceptable by a specific resource can change dynamically; the client is + notified through the return code of the response if a method is not allowed on a resource. + Servers should return the status code 501 (not implemented) if the method is unrecognized or + not implemented. +<P> + + The methods commonly used by HTTP/1.0 applications are fully defined in <A HREF="#Methods">Section 8</A>. +<P> + +<H4>5.1.2 <A NAME="Request-URI">Request-URI</A></H4> + + The <CODE>Request-URI</CODE> is a Uniform Resource Identifier (<A HREF="#URI">Section 3.2</A>) and identifies the resource upon + which to apply the request. +<P> + +<PRE> + Request-URI = absoluteURI | abs_path +</PRE> + + The two options for <CODE>Request-URI</CODE> are dependent on the nature of the request. +<P> + + The <CODE>absoluteURI</CODE> form is only allowed when the request is being made to a proxy. The proxy is + requested to forward the request and return the response. If the request is <CODE>GET</CODE> or <CODE>HEAD</CODE> and a + prior response is cached, the proxy may use the cached message if it passes any restrictions in + the <CODE>Expires</CODE> header field. Note that the proxy may forward the request on to another proxy or + directly to the server specified by the <CODE>absoluteURI</CODE>. In order to avoid request loops, a proxy must + be able to recognize all of its server names, including any aliases, local variations, and the + numeric IP address. An example <CODE>Request-Line</CODE> would be: +<P> + +<PRE> + GET http://www.w3.org/pub/WWW/TheProject.html HTTP/1.0 +</PRE> + + The most common form of <CODE>Request-URI</CODE> is that used to identify a resource on an origin server + or gateway. In this case, only the absolute path of the URI is transmitted (see <A HREF="#URI-syntax">Section 3.2.1</A>, + <CODE>abs_path</CODE>). For example, a client wishing to retrieve the resource above directly from the origin + server would create a TCP connection to port 80 of the host "www.w3.org" and send the line: +<P> + +<PRE> + GET /pub/WWW/TheProject.html HTTP/1.0 +</PRE> + + followed by the remainder of the <CODE>Full-Request</CODE>. Note that the absolute path cannot be empty; if + none is present in the original URI, it must be given as "/" (the server root). +<P> + + The <CODE>Request-URI</CODE> is transmitted as an encoded string, where some characters may be escaped + using the "% HEX HEX" encoding defined by RFC 1738<A HREF="#RefURL"> [4]</A>. The origin server must decode + the <CODE>Request-URI</CODE> in order to properly interpret the request. +<P> + +<H3>5.2 <A NAME="Request-Header">Request Header Fields</A></H3> + + The request header fields allow the client to pass additional information about the request, and + about the client itself, to the server. These fields act as request modifiers, with semantics + equivalent to the parameters on a programming language method (procedure) invocation. +<P> + +<PRE> + Request-Header = Authorization ; <A HREF="#Authorization">Section 10.2</A> + | From ; <A HREF="#From">Section 10.8</A> + | If-Modified-Since ; <A HREF="#If-Modified-Since">Section 10.9</A> + | Referer ; <A HREF="#Referer">Section 10.13</A> + | User-Agent ; <A HREF="#User-Agent">Section 10.15</A> +</PRE> + + <CODE>Request-Header</CODE> field names can be extended reliably only in combination with a change in the + protocol version. However, new or experimental header fields may be given the semantics of + request header fields if all parties in the communication recognize them to be request header + fields. Unrecognized header fields are treated as <CODE>Entity-Header</CODE> fields. +<P> + +<H2>6. <A NAME="Response">Response</A></H2> + + After receiving and interpreting a request message, a server responds in the form of an HTTP + response message. +<P> + +<PRE> + Response = Simple-Response | Full-Response +</PRE> + +<PRE> + Simple-Response = [ Entity-Body ] +</PRE> + +<PRE> + Full-Response = Status-Line ; <A HREF="#Status-Line">Section 6.1</A> + *( General-Header ; <A HREF="#General-Header">Section 4.3</A> + | Response-Header ; <A HREF="#Response-Header">Section 6.2</A> + | Entity-Header ) ; <A HREF="#Entity-Header">Section 7.1</A> + CRLF + [ Entity-Body ] ; <A HREF="#Entity-Body">Section 7.2</A> +</PRE> + + A <CODE>Simple-Response</CODE> should only be sent in response to an HTTP/0.9 <CODE>Simple-Request</CODE> or if the + server only supports the more limited HTTP/0.9 protocol. If a client sends an HTTP/1.0 + <CODE>Full-Request</CODE> and receives a response that does not begin with a <CODE>Status-Line</CODE>, it should assume that + the response is a <CODE>Simple-Response</CODE> and parse it accordingly. Note that the <CODE>Simple-Response</CODE> + consists only of the entity body and is terminated by the server closing the connection. +<P> + +<H3>6.1 <A NAME="Status-Line">Status-Line</A></H3> + + The first line of a <CODE>Full-Response</CODE> message is the <CODE>Status-Line</CODE>, consisting of the protocol version + followed by a numeric status code and its associated textual phrase, with each element + separated by <CODE>SP</CODE> characters. No <CODE>CR</CODE> or <CODE>LF</CODE> is allowed except in the final <CODE>CRLF</CODE> sequence. +<P> + +<PRE> + Status-Line = HTTP-Version SP Status-Code SP Reason-Phrase CRLF +</PRE> + + Since a status line always begins with the protocol version and status code +<P> + +<PRE> + "HTTP/" 1*DIGIT "." 1*DIGIT SP 3DIGIT SP +</PRE> + + (e.g., <CODE>"HTTP/1.0 200 "</CODE>), the presence of that expression is sufficient to differentiate a + <CODE>Full-Response</CODE> from a <CODE>Simple-Response</CODE>. Although the <CODE>Simple-Response</CODE> format may allow such + an expression to occur at the beginning of an entity body, and thus cause a misinterpretation of + the message if it was given in response to a <CODE>Full-Request</CODE>, most HTTP/0.9 servers are limited to + responses of type "text/html" and therefore would never generate such a response. +<P> + +<H4>6.1.1 <A NAME="Status-Code">Status Code and Reason Phrase</A></H4> + + The <CODE>Status-Code</CODE> element is a 3-digit integer result code of the attempt to understand and satisfy + the request. The <CODE>Reason-Phrase</CODE> is intended to give a short textual description of the <CODE>Status-Code</CODE>. + The <CODE>Status-Code</CODE> is intended for use by automata and the <CODE>Reason-Phrase</CODE> is intended for the + human user. The client is not required to examine or display the <CODE>Reason-Phrase</CODE>. +<P> + + The first digit of the <CODE>Status-Code</CODE> defines the class of response. The last two digits do not have + any categorization role. There are 5 values for the first digit: +<P> + +<UL> +<LI>1xx: Informational - Not used, but reserved for future use +<LI>2xx: Success - The action was successfully received, understood, and accepted. +<LI>3xx: Redirection - Further action must be taken in order to complete the request +<LI>4xx: Client Error - The request contains bad syntax or cannot be fulfilled +<LI>5xx: Server Error - The server failed to fulfill an apparently valid request +</UL> + + The individual values of the numeric status codes defined for HTTP/1.0, and an example set of + corresponding <CODE>Reason-Phrase</CODE>'s, are presented below. The reason phrases listed here are only + recommended -- they may be replaced by local equivalents without affecting the protocol. + These codes are fully defined in <A HREF="#Status-Codes">Section 9</A>. +<P> + +<PRE> + Status-Code = "200" ; OK + | "201" ; Created + | "202" ; Accepted + | "204" ; No Content + | "301" ; Moved Permanently + | "302" ; Moved Temporarily + | "304" ; Not Modified + | "400" ; Bad Request + | "401" ; Unauthorized + | "403" ; Forbidden + | "404" ; Not Found + | "500" ; Internal Server Error + | "501" ; Not Implemented + | "502" ; Bad Gateway + | "503" ; Service Unavailable + | extension-code +</PRE> + +<PRE> + extension-code = 3DIGIT +</PRE> + +<PRE> + Reason-Phrase = *<TEXT, excluding CR, LF> +</PRE> + + HTTP status codes are extensible, but the above codes are the only ones generally recognized + in current practice. HTTP applications are not required to understand the meaning of all + registered status codes, though such understanding is obviously desirable. However, + applications must understand the class of any status code, as indicated by the first digit, and + treat any unrecognized response as being equivalent to the x00 status code of that class, with + the exception that an unrecognized response must not be cached. For example, if an + unrecognized status code of 431 is received by the client, it can safely assume that there was + something wrong with its request and treat the response as if it had received a 400 status code. + In such cases, user agents should present to the user the entity returned with the response, since + that entity is likely to include human-readable information which will explain the unusual + status. +<P> + +<H3>6.2 <A NAME="Response-Header">Response Header Fields</A></H3> + + The response header fields allow the server to pass additional information about the response + which cannot be placed in the <CODE>Status-Line</CODE>. These header fields give information about the server + and about further access to the resource identified by the <CODE>Request-URI</CODE>. +<P> + +<PRE> + Response-Header = Location ; <A HREF="#Location">Section 10.11</A> + | Server ; <A HREF="#Server">Section 10.14</A> + | WWW-Authenticate ; <A HREF="#WWW-Authenticate">Section 10.16</A> +</PRE> + + <CODE>Response-Header</CODE> field names can be extended reliably only in combination with a change in the + protocol version. However, new or experimental header fields may be given the semantics of + response header fields if all parties in the communication recognize them to be response header + fields. Unrecognized header fields are treated as <CODE>Entity-Header</CODE> fields. +<P> + +<H2>7. <A NAME="Entity">Entity</A></H2> + + <CODE>Full-Request</CODE> and <CODE>Full-Response</CODE> messages may transfer an entity within some requests and + responses. An entity consists of <CODE>Entity-Header</CODE> fields and (usually) an <CODE>Entity-Body</CODE>. In this section, + both <EM>sender</EM> and <EM>recipient</EM> refer to either the client or the server, depending on who sends and + who receives the entity. +<P> + +<H3>7.1 <A NAME="Entity-Header">Entity Header Fields</A></H3> + + <CODE>Entity-Header</CODE> fields define optional metainformation about the <CODE>Entity-Body</CODE> or, if no body is + present, about the resource identified by the request. +<P> + +<PRE> + Entity-Header = Allow ; <A HREF="#Allow">Section 10.1</A> + | Content-Encoding ; <A HREF="#Content-Encoding">Section 10.3</A> + | Content-Length ; <A HREF="#Content-Length">Section 10.4</A> + | Content-Type ; <A HREF="#Content-Type">Section 10.5</A> + | Expires ; <A HREF="#Expires">Section 10.7</A> + | Last-Modified ; <A HREF="#Last-Modified">Section 10.10</A> + | extension-header +</PRE> + +<PRE> + extension-header = HTTP-header +</PRE> + + The <CODE>extension-header</CODE> mechanism allows additional <CODE>Entity-Header</CODE> fields to be defined without + changing the protocol, but these fields cannot be assumed to be recognizable by the recipient. + Unrecognized header fields should be ignored by the recipient and forwarded by proxies. +<P> + +<H3>7.2 <A NAME="Entity-Body">Entity Body</A></H3> + + The entity body (if any) sent with an HTTP request or response is in a format and encoding + defined by the <CODE>Entity-Header</CODE> fields. +<P> + +<PRE> + Entity-Body = *OCTET +</PRE> + + An entity body is included with a request message only when the request method calls for one. + The presence of an entity body in a request is signaled by the inclusion of a <CODE>Content-Length</CODE> + header field in the request message headers. HTTP/1.0 requests containing an entity body must + include a valid <CODE>Content-Length</CODE> header field. +<P> + + For response messages, whether or not an entity body is included with a message is dependent + on both the request method and the response code. All responses to the <CODE>HEAD</CODE> request method + must not include a body, even though the presence of entity header fields may lead one to + believe they do. All 1xx (informational), 204 (no content), and 304 (not modified) responses + must not include a body. All other responses must include an entity body or a <CODE>Content-Length</CODE> + header field defined with a value of zero (0). +<P> + +<H4>7.2.1 <A NAME="BodyType">Type</A></H4> + + When an <CODE>Entity-Body</CODE> is included with a message, the data type of that body is determined via + the header fields <CODE>Content-Type</CODE> and <CODE>Content-Encoding</CODE>. These define a two-layer, ordered + encoding model: +<P> + +<PRE> + entity-body := Content-Encoding( Content-Type( data ) ) +</PRE> + + A <CODE>Content-Type</CODE> specifies the media type of the underlying data. A <CODE>Content-Encoding</CODE> may be used + to indicate any additional content coding applied to the type, usually for the purpose of data + compression, that is a property of the resource requested. The default for the content encoding + is none (i.e., the identity function). +<P> + + Any HTTP/1.0 message containing an entity body should include a <CODE>Content-Type</CODE> header field + defining the media type of that body. If and <STRONG>only if</STRONG> the media type is not given by a <CODE>Content-Type</CODE> + header, as is the case for <CODE>Simple-Response</CODE> messages, the recipient may attempt to guess the + media type via inspection of its content and/or the name extension(s) of the URL used to + identify the resource. If the media type remains unknown, the recipient should treat it as type + "<CODE>application/octet-stream</CODE>". +<P> + +<H4>7.2.2 <A NAME="BodyLength">Length</A></H4> + + When an <CODE>Entity-Body</CODE> is included with a message, the length of that body may be determined in + one of two ways. If a <CODE>Content-Length</CODE> header field is present, its value in bytes represents the + length of the <CODE>Entity-Body</CODE>. Otherwise, the body length is determined by the closing of the + connection by the server. +<P> + + Closing the connection cannot be used to indicate the end of a request body, since it leaves no + possibility for the server to send back a response. Therefore, HTTP/1.0 requests containing an + entity body must include a valid <CODE>Content-Length</CODE> header field. If a request contains an entity body + and <CODE>Content-Length</CODE> is not specified, and the server does not recognize or cannot calculate the + length from other fields, then the server should send a 400 (bad request) response. + +<BLOCKQUOTE> + Note: Some older servers supply an invalid Content-Length when sending a + document that contains server-side includes dynamically inserted into the data + stream. It must be emphasized that this will not be tolerated by future versions of + HTTP. Unless the client knows that it is receiving a response from a compliant server, + it should not depend on the Content-Length value being correct. +</BLOCKQUOTE> + +<H2>8. <A NAME="Methods">Method Definitions</A></H2> + + The set of common methods for HTTP/1.0 is defined below. Although this set can be expanded, + additional methods cannot be assumed to share the same semantics for separately extended + clients and servers. +<P> + +<H3>8.1 <A NAME="GET">GET</A></H3> + + The <CODE>GET</CODE> method means retrieve whatever information (in the form of an entity) is identified by + the <CODE>Request-URI</CODE>. If the <CODE>Request-URI</CODE> refers to a data-producing process, it is the produced data + which shall be returned as the entity in the response and not the source text of the process, + unless that text happens to be the output of the process. +<P> + + The semantics of the <CODE>GET</CODE> method changes to a "conditional <CODE>GET</CODE>" if the request message + includes an <CODE>If-Modified-Since</CODE> header field. A conditional <CODE>GET</CODE> method requests that the identified + resource be transferred only if it has been modified since the date given by the <CODE>If-Modified-Since</CODE> + header, as described in <A HREF="#If-Modified-Since">Section 10.9</A>. The conditional <CODE>GET</CODE> method is intended to reduce + network usage by allowing cached entities to be refreshed without requiring multiple requests + or transferring unnecessary data. +<P> + +<H3>8.2 <A NAME="HEAD">HEAD</A></H3> + + The <CODE>HEAD</CODE> method is identical to <CODE>GET</CODE> except that the server must not return any <CODE>Entity-Body</CODE> in + the response. The metainformation contained in the HTTP headers in response to a <CODE>HEAD</CODE> + request should be identical to the information sent in response to a <CODE>GET</CODE> request. This method + can be used for obtaining metainformation about the resource identified by the <CODE>Request-URI</CODE> + without transferring the <CODE>Entity-Body</CODE> itself. This method is often used for testing hypertext links + for validity, accessibility, and recent modification. +<P> + + There is no "conditional <CODE>HEAD</CODE>" request analogous to the conditional <CODE>GET</CODE>. If an <CODE>If-Modified-Since</CODE> + header field is included with a <CODE>HEAD</CODE> request, it should be ignored. +<P> + +<H3>8.3 <A NAME="POST">POST</A></H3> + + The <CODE>POST</CODE> method is used to request that the destination server accept the entity enclosed in the + request as a new subordinate of the resource identified by the <CODE>Request-URI</CODE> in the <CODE>Request-Line</CODE>. + <CODE>POST</CODE> is designed to allow a uniform method to cover the following functions: +<P> + +<UL> +<LI>Annotation of existing resources; +<LI>Posting a message to a bulletin board, newsgroup, mailing list, or similar group of + articles; +<LI>Providing a block of data, such as the result of submitting a form<A HREF="#RefHTML"> [3]</A>, to a data-handling + process; +<LI>Extending a database through an append operation. +</UL> + + The actual function performed by the <CODE>POST</CODE> method is determined by the server and is usually + dependent on the <CODE>Request-URI</CODE>. The posted entity is subordinate to that URI in the same way + that a file is subordinate to a directory containing it, a news article is subordinate to a + newsgroup to which it is posted, or a record is subordinate to a database. +<P> + + A successful <CODE>POST</CODE> does not require that the entity be created as a resource on the origin server + or made accessible for future reference. That is, the action performed by the <CODE>POST</CODE> method + might not result in a resource that can be identified by a URI. In this case, either 200 (ok) or + 204 (no content) is the appropriate response status, depending on whether or not the response + includes an entity that describes the result. +<P> + + If a resource has been created on the origin server, the response should be 201 (created) and + contain an entity (preferably of type "text/html") which describes the status of the request and + refers to the new resource. +<P> + + A valid <CODE>Content-Length</CODE> is required on all HTTP/1.0 <CODE>POST</CODE> requests. An HTTP/1.0 server should + respond with a 400 (bad request) message if it cannot determine the length of the request + message's content. +<P> + + Applications must not cache responses to a POST request because the application has no way + of knowing that the server would return an equivalent response on some future request. +<P> + +<H2>9. <A NAME="Status-Codes">Status Code Definitions</A></H2> + + Each <CODE>Status-Code</CODE> is described below, including a description of which <CODE>method</CODE>(s) it can follow + and any metainformation required in the response. +<P> + +<H3>9.1 <A NAME="Code1xx">Informational 1xx</A></H3> + + This class of status code indicates a provisional response, consisting only of the <CODE>Status-Line</CODE> and + optional headers, and is terminated by an empty line. HTTP/1.0 does not define any 1xx status + codes and they are not a valid response to a HTTP/1.0 request. However, they may be useful + for experimental applications which are outside the scope of this specification. +<P> + +<H3>9.2 <A NAME="Code2xx">Successful 2xx</A></H3> + + This class of status code indicates that the client's request was successfully received, + understood, and accepted. +<P> + +<H4> <A NAME="Code200">200 OK</A></H4> + + The request has succeeded. The information returned with the response is dependent on the + method used in the request, as follows: +<P> + +<DL COMPACT> +<DT><CODE>GET</CODE> +<DD>an entity corresponding to the requested resource is sent in the response; + +<DT><CODE>HEAD</CODE> +<DD>the response must only contain the header information and no <CODE>Entity-Body</CODE>; + +<DT><CODE>POST</CODE> +<DD>an entity describing or containing the result of the action. +</DL> + +<H4> <A NAME="Code201">201 Created</A></H4> + + The request has been fulfilled and resulted in a new resource being created. The newly created + resource can be referenced by the URI(s) returned in the entity of the response. The origin + server should create the resource before using this <CODE>Status-Code</CODE>. If the action cannot be carried + out immediately, the server must include in the response body a description of when the + resource will be available; otherwise, the server should respond with 202 (accepted). +<P> + + Of the methods defined by this specification, only <CODE>POST</CODE> can create a resource. +<P> + +<H4> <A NAME="Code202">202 Accepted</A></H4> + + The request has been accepted for processing, but the processing has not been completed. The + request may or may not eventually be acted upon, as it may be disallowed when processing + actually takes place. There is no facility for re-sending a status code from an asynchronous + operation such as this. +<P> + + The 202 response is intentionally non-committal. Its purpose is to allow a server to accept a + request for some other process (perhaps a batch-oriented process that is only run once per day) + without requiring that the user agent's connection to the server persist until the process is + completed. The entity returned with this response should include an indication of the request's + current status and either a pointer to a status monitor or some estimate of when the user can + expect the request to be fulfilled. +<P> + +<H4> <A NAME="Code204">204 No Content</A></H4> + + The server has fulfilled the request but there is no new information to send back. If the client is + a user agent, it should not change its document view from that which caused the request to be + generated. This response is primarily intended to allow input for scripts or other actions to take + place without causing a change to the user agent's active document view. The response may + include new metainformation in the form of entity headers, which should apply to the + document currently in the user agent's active view. +<P> + +<H3>9.3 <A NAME="Code3xx">Redirection 3xx</A></H3> + + This class of status code indicates that further action needs to be taken by the user agent in order + to fulfill the request. The action required may be carried out by the user agent without + interaction with the user if and only if the method used in the subsequent request is <CODE>GET</CODE> or + <CODE>HEAD</CODE>. A user agent should never automatically redirect a request more than 5 times, since such + redirections usually indicate an infinite loop. +<P> + +<H4> <A NAME="Code300">300 Multiple Choices</A></H4> + + This response code is not directly used by HTTP/1.0 applications, but serves as the default for + interpreting the 3xx class of responses. +<P> + + The requested resource is available at one or more locations. Unless it was a <CODE>HEAD</CODE> request, the + response should include an entity containing a list of resource characteristics and locations + from which the user or user agent can choose the one most appropriate. If the server has a + preferred choice, it should include the URL in a <CODE>Location</CODE> field; user agents may use this field + value for automatic redirection. +<P> + +<H4> <A NAME="Code301">301 Moved Permanently</A></H4> + + The requested resource has been assigned a new permanent URL and any future references to + this resource should be done using that URL. Clients with link editing capabilities should + automatically relink references to the <CODE>Request-URI</CODE> to the new reference returned by the server, + where possible. +<P> + + The new URL must be given by the <CODE>Location</CODE> field in the response. Unless it was a <CODE>HEAD</CODE> request, + the <CODE>Entity-Body</CODE> of the response should contain a short note with a hyperlink to the new URL. +<P> + + If the 301 status code is received in response to a request using the <CODE>POST</CODE> method, the user agent + must not automatically redirect the request unless it can be confirmed by the user, since this + might change the conditions under which the request was issued. +<P> + +<BLOCKQUOTE> + Note: When automatically redirecting a POST request after receiving a 301 status + code, some existing user agents will erroneously change it into a GET request. +</BLOCKQUOTE> + +<H4> <A NAME="Code302">302 Moved Temporarily</A></H4> + + The requested resource resides temporarily under a different URL. Since the redirection may + be altered on occasion, the client should continue to use the <CODE>Request-URI</CODE> for future requests. +<P> + + The URL must be given by the <CODE>Location</CODE> field in the response. Unless it was a <CODE>HEAD</CODE> request, the + <CODE>Entity-Body</CODE> of the response should contain a short note with a hyperlink to the new URI(s). +<P> + + If the 302 status code is received in response to a request using the <CODE>POST</CODE> method, the user agent + must not automatically redirect the request unless it can be confirmed by the user, since this + might change the conditions under which the request was issued. +<P> + +<BLOCKQUOTE> + Note: When automatically redirecting a POST request after receiving a 302 status + code, some existing user agents will erroneously change it into a GET request. +</BLOCKQUOTE> + +<H4> <A NAME="Code304">304 Not Modified</A></H4> + + If the client has performed a conditional <CODE>GET</CODE> request and access is allowed, but the document + has not been modified since the date and time specified in the <CODE>If-Modified-Since</CODE> field, the server + must respond with this status code and not send an <CODE>Entity-Body</CODE> to the client. Header fields + contained in the response should only include information which is relevant to cache managers + or which may have changed independently of the entity's <CODE>Last-Modified</CODE> date. Examples of + relevant header fields include: <CODE>Date</CODE>, <CODE>Server</CODE>, and <CODE>Expires</CODE>. A cache should update its cached entity + to reflect any new field values given in the 304 response. +<P> + +<H3>9.4 <A NAME="Code4xx">Client Error 4xx</A></H3> + + The 4xx class of status code is intended for cases in which the client seems to have erred. If the + client has not completed the request when a 4xx code is received, it should immediately cease + sending data to the server. Except when responding to a <CODE>HEAD</CODE> request, the server should + include an entity containing an explanation of the error situation, and whether it is a temporary + or permanent condition. These status codes are applicable to any request method. +<P> + +<BLOCKQUOTE> + Note: If the client is sending data, server implementations on TCP should be careful + to ensure that the client acknowledges receipt of the packet(s) containing the response + prior to closing the input connection. If the client continues sending data to the server + after the close, the server's controller will send a reset packet to the client, which may + erase the client's unacknowledged input buffers before they can be read and + interpreted by the HTTP application. +</BLOCKQUOTE> + +<H4> <A NAME="Code400">400 Bad Request</A></H4> + + The request could not be understood by the server due to malformed syntax. The client should + not repeat the request without modifications. +<P> + +<H4> <A NAME="Code401">401 Unauthorized</A></H4> + + The request requires user authentication. The response must include a <CODE>WWW-Authenticate</CODE> + header field (<A HREF="#WWW-Authenticate">Section 10.16</A>) containing a <CODE>challenge</CODE> applicable to the requested resource. The + client may repeat the request with a suitable <CODE>Authorization</CODE> header field (<A HREF="#Authorization">Section 10.2</A>). If the + request already included Authorization credentials, then the 401 response indicates that + authorization has been refused for those credentials. If the 401 response contains the same + challenge as the prior response, and the user agent has already attempted authentication at least + once, then the user should be presented the entity that was given in the response, since that + entity may include relevant diagnostic information. HTTP access authentication is explained + in <A HREF="#AA">Section 11</A>. +<P> + +<H4> <A NAME="Code403">403 Forbidden</A></H4> + + The server understood the request, but is refusing to fulfill it. Authorization will not help and + the request should not be repeated. If the request method was not <CODE>HEAD</CODE> and the server wishes + to make public why the request has not been fulfilled, it should describe the reason for the + refusal in the entity body. This status code is commonly used when the server does not wish to + reveal exactly why the request has been refused, or when no other response is applicable. +<P> + +<H4> <A NAME="Code404">404 Not Found</A></H4> + + The server has not found anything matching the <CODE>Request-URI</CODE>. No indication is given of whether + the condition is temporary or permanent. If the server does not wish to make this information + available to the client, the status code 403 (forbidden) can be used instead. +<P> + +<H3>9.5 <A NAME="Code5xx">Server Error 5xx</A></H3> + + Response status codes beginning with the digit "5" indicate cases in which the server is aware + that it has erred or is incapable of performing the request. If the client has not completed the + request when a 5xx code is received, it should immediately cease sending data to the server. + Except when responding to a <CODE>HEAD</CODE> request, the server should include an entity containing an + explanation of the error situation, and whether it is a temporary or permanent condition. These + response codes are applicable to any request method and there are no required header fields. +<P> + +<H4> <A NAME="Code500">500 Internal Server Error</A></H4> + + The server encountered an unexpected condition which prevented it from fulfilling the request. +<P> + +<H4> <A NAME="Code501">501 Not Implemented</A></H4> + + The server does not support the functionality required to fulfill the request. This is the + appropriate response when the server does not recognize the request method and is not capable + of supporting it for any resource. +<P> + +<H4> <A NAME="Code502">502 Bad Gateway</A></H4> + + The server, while acting as a gateway or proxy, received an invalid response from the upstream + server it accessed in attempting to fulfill the request. +<P> + +<H4> <A NAME="Code503">503 Service Unavailable</A></H4> + + The server is currently unable to handle the request due to a temporary overloading or + maintenance of the server. The implication is that this is a temporary condition which will be + alleviated after some delay. + +<BLOCKQUOTE> + Note: The existence of the 503 status code does not imply that a server must use it + when becoming overloaded. Some servers may wish to simply refuse the connection. +</BLOCKQUOTE> + +<H2>10. <A NAME="HeaderFields">Header Field Definitions</A></H2> + + This section defines the syntax and semantics of all commonly used HTTP/1.0 header fields. + For general and entity header fields, both <EM>sender </EM>and<EM> recipient</EM> refer to either the client or the + server, depending on who sends and who receives the message. +<P> + +<H3>10.1 <A NAME="Allow">Allow</A></H3> + + The <CODE>Allow</CODE> entity-header field lists the set of methods supported by the resource identified by the + <CODE>Request-URI</CODE>. The purpose of this field is strictly to inform the recipient of valid methods + associated with the resource. The <CODE>Allow</CODE> header field is not permitted in a request using the <CODE>POST</CODE> + method, and thus should be ignored if it is received as part of a <CODE>POST</CODE> entity. +<P> + +<PRE> + Allow = "Allow" ":" 1#method +</PRE> + + Example of use: +<P> + +<PRE> + Allow: GET, HEAD +</PRE> + + This field cannot prevent a client from trying other methods. However, the indications given by + the <CODE>Allow</CODE> header field value should be followed. The actual set of allowed methods is defined + by the origin server at the time of each request. +<P> + + A proxy must not modify the <CODE>Allow</CODE> header field even if it does not understand all the methods + specified, since the user agent may have other means of communicating with the origin server. +<P> + + The <CODE>Allow</CODE> header field does not indicate what methods are implemented by the server. +<P> + +<H3>10.2 <A NAME="Authorization">Authorization</A></H3> + + A user agent that wishes to authenticate itself with a server--usually, but not necessarily, after + receiving a 401 response--may do so by including an <CODE>Authorization</CODE> request-header field with + the request. The <CODE>Authorization</CODE> field value consists of <CODE>credentials</CODE> containing the authentication + information of the user agent for the realm of the resource being requested. +<P> + +<PRE> + Authorization = "Authorization" ":" credentials +</PRE> + + HTTP access authentication is described in <A HREF="#AA">Section 11</A>. If a request is authenticated and a <CODE>realm</CODE> + specified, the same <CODE>credentials</CODE> should be valid for all other requests within this <CODE>realm</CODE>. +<P> + + Responses to requests containing an <CODE>Authorization</CODE> field are not cachable. +<P> + +<H3>10.3 <A NAME="Content-Encoding">Content-Encoding</A></H3> + + The <CODE>Content-Encoding</CODE> entity-header field is used as a modifier to the <CODE>media-type</CODE>. When present, + its value indicates what additional content coding has been applied to the resource, and thus + what decoding mechanism must be applied in order to obtain the <CODE>media-type</CODE> referenced by the + <CODE>Content-Type</CODE> header field. The <CODE>Content-Encoding</CODE> is primarily used to allow a document to be + compressed without losing the identity of its underlying media type. +<P> + +<PRE> + Content-Encoding = "Content-Encoding" ":" content-coding +</PRE> + + Content codings are defined in <A HREF="#Content-Codings">Section 3.5</A>. An example of its use is +<P> + +<PRE> + Content-Encoding: x-gzip +</PRE> + + The <CODE>Content-Encoding</CODE> is a characteristic of the resource identified by the <CODE>Request-URI</CODE>. Typically, + the resource is stored with this encoding and is only decoded before rendering or analogous + usage. +<P> + +<H3>10.4 <A NAME="Content-Length">Content-Length</A></H3> + + The <CODE>Content-Length</CODE> entity-header field indicates the size of the <CODE>Entity-Body</CODE>, in decimal number + of octets, sent to the recipient or, in the case of the <CODE>HEAD</CODE> method, the size of the <CODE>Entity-Body</CODE> that + would have been sent had the request been a <CODE>GET</CODE>. +<P> + +<PRE> + Content-Length = "Content-Length" ":" 1*DIGIT +</PRE> + + An example is +<P> + +<PRE> + Content-Length: 3495 +</PRE> + + Applications should use this field to indicate the size of the <CODE>Entity-Body</CODE> to be transferred, + regardless of the media type of the entity. A valid <CODE>Content-Length</CODE> field value is required on all + HTTP/1.0 request messages containing an entity body. +<P> + + Any <CODE>Content-Length</CODE> greater than or equal to zero is a valid value. <A HREF="#BodyLength">Section 7.2.2</A> describes how + to determine the length of a response entity body if a <CODE>Content-Length</CODE> is not given. +<P> + +<BLOCKQUOTE> + Note: The meaning of this field is significantly different from the corresponding + definition in MIME, where it is an optional field used within the + "message/external-body" content-type. In HTTP, it should be used whenever the + entity's length can be determined prior to being transferred. +</BLOCKQUOTE> + +<H3>10.5 <A NAME="Content-Type">Content-Type</A></H3> + + The <CODE>Content-Type</CODE> entity-header field indicates the media type of the <CODE>Entity-Body</CODE> sent to the + recipient or, in the case of the <CODE>HEAD</CODE> method, the media type that would have been sent had the + request been a <CODE>GET</CODE>. +<P> + +<PRE> + Content-Type = "Content-Type" ":" media-type +</PRE> + + Media types are defined in <A HREF="#Media-Types">Section 3.6</A>. An example of the field is +<P> + +<PRE> + Content-Type: text/html +</PRE> + + Further discussion of methods for identifying the media type of an entity is provided in + <A HREF="#BodyType">Section 7.2.1</A>. +<P> + +<H3>10.6 <A NAME="Date">Date</A></H3> + + The <CODE>Date</CODE> general-header field represents the date and time at which the message was originated, + having the same semantics as <CODE>orig-date</CODE> in RFC 822. The field value is an <CODE>HTTP-date</CODE>, as + described in <A HREF="#DateFormats">Section 3.3</A>. +<P> + +<PRE> + Date = "Date" ":" HTTP-date +</PRE> + + An example is +<P> + +<PRE> + Date: Tue, 15 Nov 1994 08:12:31 GMT +</PRE> + + If a message is received via direct connection with the user agent (in the case of requests) or + the origin server (in the case of responses), then the date can be assumed to be the current date + at the receiving end. However, since the date--as it is believed by the origin--is important for + evaluating cached responses, origin servers should always include a <CODE>Date</CODE> header. Clients should + only send a <CODE>Date</CODE> header field in messages that include an entity body, as in the case of the <CODE>POST</CODE> + request, and even then it is optional. A received message which does not have a <CODE>Date</CODE> header + field should be assigned one by the recipient if the message will be cached by that recipient or + gatewayed via a protocol which requires a <CODE>Date</CODE>. +<P> + + In theory, the date should represent the moment just before the entity is generated. In practice, + the date can be generated at any time during the message origination without affecting its + semantic value. +<P> + +<BLOCKQUOTE> + Note: An earlier version of this document incorrectly specified that this field should + contain the creation date of the enclosed <CODE>Entity-Body</CODE>. This has been changed to reflect + actual (and proper) usage. +</BLOCKQUOTE> + +<H3>10.7 <A NAME="Expires">Expires</A></H3> + + The <CODE>Expires</CODE> entity-header field gives the date/time after which the entity should be considered + stale. This allows information providers to suggest the volatility of the resource, or a date after + which the information may no longer be valid. Applications must not cache this entity beyond + the date given. The presence of an Expires field does not imply that the original resource will + change or cease to exist at, before, or after that time. However, information providers that know + or even suspect that a resource will change by a certain date should include an Expires header + with that date. The format is an absolute date and time as defined by <CODE>HTTP-date</CODE> in <A HREF="#DateFormats">Section 3.3</A>. +<P> + +<PRE> + Expires = "Expires" ":" HTTP-date +</PRE> + + An example of its use is +<P> + +<PRE> + Expires: Thu, 01 Dec 1994 16:00:00 GMT +</PRE> + + If the date given is equal to or earlier than the value of the <CODE>Date</CODE> header, the recipient must not + cache the enclosed entity. If a resource is dynamic by nature, as is the case with many + data-producing processes, entities from that resource should be given an appropriate Expires + value which reflects that dynamism. +<P> + + The Expires field cannot be used to force a user agent to refresh its display or reload a resource; + its semantics apply only to caching mechanisms, and such mechanisms need only check a + resource's expiration status when a new request for that resource is initiated. +<P> + + User agents often have history mechanisms, such as "Back" buttons and history lists, which can + be used to redisplay an entity retrieved earlier in a session. By default, the <CODE>Expires</CODE> field does not + apply to history mechanisms. If the entity is still in storage, a history mechanism should display + it even if the entity has expired, unless the user has specifically configured the agent to refresh + expired history documents. +<P> + +<BLOCKQUOTE> + Note: Applications are encouraged to be tolerant of bad or misinformed + implementations of the Expires header. A value of zero (0) or an invalid date format + should be considered equivalent to an "expires immediately." Although these values + are not legitimate for HTTP/1.0, a robust implementation is always desirable. +</BLOCKQUOTE> + +<H3>10.8 <A NAME="From">From</A></H3> + + The <CODE>From</CODE> request-header field, if given, should contain an Internet e-mail address for the human + user who controls the requesting user agent. The address should be machine-usable, as defined + by <CODE>mailbox</CODE> in RFC 822<A HREF="#RefSTD11"> [7]</A> (as updated by RFC 1123<A HREF="#RefSTD3"> [6]</A>): +<P> + +<PRE> + From = "From" ":" mailbox +</PRE> + + An example is: +<P> + +<PRE> + From: [email protected] +</PRE> + + This header field may be used for logging purposes and as a means for identifying the source + of invalid or unwanted requests. It should not be used as an insecure form of access protection. + The interpretation of this field is that the request is being performed on behalf of the person + given, who accepts responsibility for the <CODE>method</CODE> performed. In particular, robot agents should + include this header so that the person responsible for running the robot can be contacted if + problems occur on the receiving end. +<P> + + The Internet e-mail address in this field may be separate from the Internet host which issued + the request. For example, when a request is passed through a proxy, the original issuer's address + should be used. +<P> + +<BLOCKQUOTE> + Note: The client should not send the <CODE>From</CODE> header field without the user's approval, as + it may conflict with the user's privacy interests or their site's security policy. It is + strongly recommended that the user be able to disable, enable, and modify the value + of this field at any time prior to a request. +</BLOCKQUOTE> + +<H3>10.9 <A NAME="If-Modified-Since">If-Modified-Since</A></H3> + + The <CODE>If-Modified-Since</CODE> request-header field is used with the <CODE>GET</CODE> method to make it conditional: + if the requested resource has not been modified since the time specified in this field, a copy of + the resource will not be returned from the server; instead, a 304 (not modified) response will + be returned without any <CODE>Entity-Body</CODE>. +<P> + +<PRE> + If-Modified-Since = "If-Modified-Since" ":" HTTP-date +</PRE> + + An example of the field is: + +<PRE> + If-Modified-Since: Sat, 29 Oct 1994 19:43:31 GMT +</PRE> + + A conditional <CODE>GET</CODE> method requests that the identified resource be transferred only if it has been + modified since the date given by the <CODE>If-Modified-Since</CODE> header. The algorithm for determining this + includes the following cases: +<P> + +<DL COMPACT> +<DT>a) +<DD>If the request would normally result in anything other than a 200 (ok) status, or if + the passed <CODE>If-Modified-Since</CODE> date is invalid, the response is exactly the same as for a + normal <CODE>GET</CODE>. A date which is later than the server's current time is invalid. + +<DT>b) +<DD>If the resource has been modified since the <CODE>If-Modified-Since</CODE> date, the response is + exactly the same as for a normal <CODE>GET</CODE>. + +<DT>c) +<DD>If the resource has not been modified since a valid <CODE>If-Modified-Since</CODE> date, the server + shall return a 304 (not modified) response. +</DL> + + The purpose of this feature is to allow efficient updates of cached information with a minimum + amount of transaction overhead. +<P> + +<H3>10.10 <A NAME="Last-Modified">Last-Modified</A></H3> + + The <CODE>Last-Modified</CODE> entity-header field indicates the date and time at which the sender believes + the resource was last modified. The exact semantics of this field are defined in terms of how the + recipient should interpret it: if the recipient has a copy of this resource which is older than the + date given by the <CODE>Last-Modified</CODE> field, that copy should be considered stale. +<P> + +<PRE> + Last-Modified = "Last-Modified" ":" HTTP-date +</PRE> + + An example of its use is +<P> + +<PRE> + Last-Modified: Tue, 15 Nov 1994 12:45:26 GMT +</PRE> + + The exact meaning of this header field depends on the implementation of the sender and the + nature of the original resource. For files, it may be just the file system last-modified time. For + entities with dynamically included parts, it may be the most recent of the set of last-modify + times for its component parts. For database gateways, it may be the last-update timestamp of + the record. For virtual objects, it may be the last time the internal state changed. +<P> + + An origin server must not send a Last-Modified date which is later than the server's time of + message origination. In such cases, where the resource's last modification would indicate some + time in the future, the server must replace that date with the message origination date. +<P> + +<H3>10.11 <A NAME="Location">Location</A></H3> + + The Location response-header field defines the exact location of the resource that was identified + by the <CODE>Request-URI</CODE>. For 3xx responses, the location must indicate the server's preferred URL + for automatic redirection to the resource. Only one absolute URL is allowed. +<P> + +<PRE> + Location = "Location" ":" absoluteURI +</PRE> + + An example is +<P> + +<PRE> + Location: http://www.w3.org/hypertext/WWW/NewLocation.html +</PRE> + +<H3>10.12 <A NAME="Pragma">Pragma</A></H3> + + The <CODE>Pragma</CODE> general-header field is used to include implementation-specific directives that may + apply to any recipient along the request/response chain. All pragma directives specify optional + behavior from the viewpoint of the protocol; however, some systems may require that behavior + be consistent with the directives. +<P> + +<PRE> + Pragma = "Pragma" ":" 1#pragma-directive +</PRE> + +<PRE> + pragma-directive = "no-cache" | extension-pragma + extension-pragma = token [ "=" word ] +</PRE> + + When the "<CODE>no-cache</CODE>" directive is present in a request message, an application should forward + the request toward the origin server even if it has a cached copy of what is being requested. This + allows a client to insist upon receiving an authoritative response to its request. It also allows a + client to refresh a cached copy which is known to be corrupted or stale. +<P> + + Pragma directives must be passed through by a proxy or gateway application, regardless of + their significance to that application, since the directives may be applicable to all recipients + along the request/response chain. It is not possible to specify a pragma for a specific recipient; + however, any pragma directive not relevant to a recipient should be ignored by that recipient. +<P> + +<H3>10.13 <A NAME="Referer">Referer</A></H3> + + The <CODE>Referer</CODE> request-header field allows the client to specify, for the server's benefit, the address + (URI) of the resource from which the <CODE>Request-URI</CODE> was obtained. This allows a server to + generate lists of back-links to resources for interest, logging, optimized caching, etc. It also + allows obsolete or mistyped links to be traced for maintenance. The <CODE>Referer</CODE> field must not be + sent if the <CODE>Request-URI</CODE> was obtained from a source that does not have its own URI, such as input + from the user keyboard. +<P> + +<PRE> + Referer = "Referer" ":" ( absoluteURI | relativeURI ) +</PRE> + + Example: +<P> + +<PRE> + Referer: http://www.w3.org/hypertext/DataSources/Overview.html +</PRE> + + If a partial URI is given, it should be interpreted relative to the <CODE>Request-URI</CODE>. The URI must not + include a fragment. +<P> + +<BLOCKQUOTE> + Note: Because the source of a link may be private information or may reveal an + otherwise private information source, it is strongly recommended that the user be + able to select whether or not the <CODE>Referer</CODE> field is sent. For example, a browser client + could have a toggle switch for browsing openly/anonymously, which would + respectively enable/disable the sending of <CODE>Referer</CODE> and <CODE>From</CODE> information. +</BLOCKQUOTE> + +<H3>10.14 <A NAME="Server">Server</A></H3> + + The <CODE>Server</CODE> response-header field contains information about the software used by the origin + server to handle the request. The field can contain multiple product tokens (<A HREF="#Product">Section 3.7</A>) and + comments identifying the server and any significant subproducts. By convention, the product + tokens are listed in order of their significance for identifying the application. +<P> + +<PRE> + Server = "Server" ":" 1*( product | comment ) +</PRE> + + Example: +<P> + +<PRE> + Server: CERN/3.0 libwww/2.17 +</PRE> + + If the response is being forwarded through a proxy, the proxy application must not add its data + to the product list. +<P> + +<BLOCKQUOTE> + Note: Revealing the specific software version of the server may allow the server + machine to become more vulnerable to attacks against software that is known to + contain security holes. Server implementors are encouraged to make this field a + configurable option. +</BLOCKQUOTE> + +<BLOCKQUOTE> + Note: Some existing servers fail to restrict themselves to the product token syntax + within the Server field. +</BLOCKQUOTE> + +<H3>10.15 <A NAME="User-Agent">User-Agent</A></H3> + + The <CODE>User-Agent</CODE> request-header field contains information about the user agent originating the + request. This is for statistical purposes, the tracing of protocol violations, and automated + recognition of user agents for the sake of tailoring responses to avoid particular user agent + limitations. Although it is not required, user agents should include this field with requests. The + field can contain multiple product tokens (<A HREF="#Product">Section 3.7</A>) and comments identifying the agent and + any subproducts which form a significant part of the user agent. By convention, the product + tokens are listed in order of their significance for identifying the application. +<P> + +<PRE> + User-Agent = "User-Agent" ":" 1*( product | comment ) +</PRE> + + Example: +<P> + +<PRE> + User-Agent: CERN-LineMode/2.15 libwww/2.17b3 +</PRE> + +<BLOCKQUOTE> + Note: Some current proxy applications append their product information to the list in + the User-Agent field. This is not recommended, since it makes machine interpretation + of these fields ambiguous. +</BLOCKQUOTE> + +<BLOCKQUOTE> + Note: Some existing clients fail to restrict themselves to the product token syntax + within the User-Agent field. +</BLOCKQUOTE> + +<H3>10.16 <A NAME="WWW-Authenticate">WWW-Authenticate</A></H3> + + The <CODE>WWW-Authenticate</CODE> response-header field must be included in 401 (unauthorized) response + messages. The field value consists of at least one <CODE>challenge</CODE> that indicates the authentication + scheme(s) and parameters applicable to the <CODE>Request-URI</CODE>. +<P> + +<PRE> + WWW-Authenticate = "WWW-Authenticate" ":" 1#challenge +</PRE> + + The HTTP access authentication process is described in <A HREF="#AA">Section 11</A>. User agents must take + special care in parsing the <CODE>WWW-Authenticate</CODE> field value if it contains more than one challenge, + or if more than one <CODE>WWW-Authenticate</CODE> header field is provided, since the contents of a challenge + may itself contain a comma-separated list of authentication parameters. +<P> + +<H2>11. <A NAME="AA">Access Authentication</A></H2> + + HTTP provides a simple challenge-response authentication mechanism which may be used by + a server to challenge a client request and by a client to provide authentication information. It + uses an extensible, case-insensitive token to identify the authentication scheme, followed by a + comma-separated list of attribute-value pairs which carry the parameters necessary for + achieving authentication via that scheme. +<P> + +<PRE> + auth-scheme = token +</PRE> + +<PRE> + auth-param = token "=" quoted-string +</PRE> + + The 401 (unauthorized) response message is used by an origin server to challenge the + authorization of a user agent. This response must include a <CODE>WWW-Authenticate</CODE> header field + containing at least one <CODE>challenge</CODE> applicable to the requested resource. +<P> + +<PRE> + challenge = auth-scheme 1*SP realm *( "," auth-param ) +</PRE> + +<PRE> + realm = "realm" "=" realm-value + realm-value = quoted-string +</PRE> + + The realm attribute (case-insensitive) is required for all authentication schemes which issue a + challenge. The realm value (case-sensitive), in combination with the canonical root URL of the + server being accessed, defines the protection space. These realms allow the protected resources + on a server to be partitioned into a set of protection spaces, each with its own authentication + scheme and/or authorization database. The realm value is a string, generally assigned by the + origin server, which may have additional semantics specific to the authentication scheme. +<P> + + A user agent that wishes to authenticate itself with a server--usually, but not necessarily, after + receiving a 401 response--may do so by including an <CODE>Authorization</CODE> header field with the + request. The <CODE>Authorization</CODE> field value consists of <CODE>credentials</CODE> containing the authentication + information of the user agent for the realm of the resource being requested. +<P> + +<PRE> + credentials = basic-credentials + | ( auth-scheme #auth-param ) +</PRE> + + The domain over which credentials can be automatically applied by a user agent is determined + by the protection space. If a prior request has been authorized, the same credentials may be + reused for all other requests within that protection space for a period of time determined by the + authentication scheme, parameters, and/or user preference. Unless otherwise defined by the + authentication scheme, a single protection space cannot extend outside the scope of its server. +<P> + + If the server does not wish to accept the credentials sent with a request, it should return a 403 + (forbidden) response. +<P> + + The HTTP protocol does not restrict applications to this simple challenge-response mechanism + for access authentication. Additional mechanisms may be used, such as encryption at the + transport level or via message encapsulation, and with additional header fields specifying + authentication information. However, these additional mechanisms are not defined by this + specification. +<P> + + Proxies must be completely transparent regarding user agent authentication. That is, they must + forward the <CODE>WWW-Authenticate</CODE> and <CODE>Authorization</CODE> headers untouched, and must not cache the + response to a request containing <CODE>Authorization</CODE>. HTTP/1.0 does not provide a means for a client + to be authenticated with a proxy. +<P> + +<H3>11.1 <A NAME="BasicAA">Basic Authentication Scheme</A></H3> + + The "basic" authentication scheme is based on the model that the user agent must authenticate + itself with a user-ID and a password for each realm. The realm value should be considered an + opaque string which can only be compared for equality with other realms on that server. The + server will authorize the request only if it can validate the user-ID and password for the + protection space of the <CODE>Request-URI</CODE>. There are no optional authentication parameters. +<P> + + Upon receipt of an unauthorized request for a URI within the protection space, the server + should respond with a challenge like the following: +<P> + +<PRE> + WWW-Authenticate: Basic realm="WallyWorld" +</PRE> + + where "WallyWorld" is the string assigned by the server to identify the protection space of the + <CODE>Request-URI</CODE>. +<P> + + To receive authorization, the client sends the user-ID and password, separated by a single colon + (":") character, within a base64<A HREF="#RefMIME1"> [5]</A> encoded string in the <CODE>credentials</CODE>. +<P> + +<PRE> + basic-credentials = "Basic" SP basic-cookie +</PRE> + +<PRE> + basic-cookie = <base64<A HREF="#RefMIME1"> [5]</A> encoding of userid-password, + except not limited to 76 char/line> +</PRE> + +<PRE> + userid-password = [ token ] ":" *TEXT +</PRE> + + If the user agent wishes to send the user-ID "Aladdin" and password "open sesame", it would + use the following header field: +<P> + +<PRE> + Authorization: Basic QWxhZGRpbjpvcGVuIHNlc2FtZQ== +</PRE> + + The basic authentication scheme is a non-secure method of filtering unauthorized access to + resources on an HTTP server. It is based on the assumption that the connection between the + client and the server can be regarded as a trusted carrier. As this is not generally true on an open + network, the basic authentication scheme should be used accordingly. In spite of this, clients + should implement the scheme in order to communicate with servers that use it. +<P> + +<H2>12. <A NAME="Security">Security Considerations</A></H2> + + This section is meant to inform application developers, information providers, and users of the + security limitations in HTTP/1.0 as described by this document. The discussion does not + include definitive solutions to the problems revealed, though it does make some suggestions for + reducing security risks. +<P> + +<H3>12.1 <A NAME="AuthSecurity">Authentication of Clients</A></H3> + + As mentioned in <A HREF="#BasicAA">Section 11.1</A>, the Basic authentication scheme is not a secure method of user + authentication, nor does it prevent the <CODE>Entity-Body</CODE> from being transmitted in clear text across the + physical network used as the carrier. HTTP/1.0 does not prevent additional authentication + schemes and encryption mechanisms from being employed to increase security. +<P> + +<H3>12.2 <A NAME="SafeMethods">Safe Methods</A></H3> + + The writers of client software should be aware that the software represents the user in their + interactions over the Internet, and should be careful to allow the user to be aware of any actions + they may take which may have an unexpected significance to themselves or others. +<P> + + In particular, the convention has been established that the <CODE>GET</CODE> and <CODE>HEAD</CODE> methods should never + have the significance of taking an action other than retrieval. These methods should be + considered "safe." This allows user agents to represent other methods, such as <CODE>POST</CODE>, in a + special way, so that the user is made aware of the fact that a possibly unsafe action is being + requested. +<P> + + Naturally, it is not possible to ensure that the server does not generate side-effects as a result of + performing a <CODE>GET</CODE> request; in fact, some dynamic resources consider that a feature. The + important distinction here is that the user did not request the side-effects, so therefore cannot + be held accountable for them. +<P> + +<H3>12.3 <A NAME="LogAbuse">Abuse of Server Log Information</A></H3> + + A server is in the position to save personal data about a user's requests which may identify their + reading patterns or subjects of interest. This information is clearly confidential in nature and its + handling may be constrained by law in certain countries. People using the HTTP protocol to + provide data are responsible for ensuring that such material is not distributed without the + permission of any individuals that are identifiable by the published results. +<P> + +<H3>12.4 <A NAME="Sensitive">Transfer of Sensitive Information</A></H3> + + Like any generic data transfer protocol, HTTP cannot regulate the content of the data that is + transferred, nor is there any a priori method of determining the sensitivity of any particular + piece of information within the context of any given request. Therefore, applications should + supply as much control over this information as possible to the provider of that information. + Three header fields are worth special mention in this context: <CODE>Server</CODE>, <CODE>Referer</CODE> and <CODE>From</CODE>. +<P> + + Revealing the specific software version of the server may allow the server machine to become + more vulnerable to attacks against software that is known to contain security holes. + Implementors should make the <CODE>Server</CODE> header field a configurable option. +<P> + + The <CODE>Referer</CODE> field allows reading patterns to be studied and reverse links drawn. Although it can + be very useful, its power can be abused if user details are not separated from the information + contained in the <CODE>Referer</CODE>. Even when the personal information has been removed, the <CODE>Referer</CODE> + field may indicate a private document's URI whose publication would be inappropriate. +<P> + + The information sent in the <CODE>From</CODE> field might conflict with the user's privacy interests or their + site's security policy, and hence it should not be transmitted without the user being able to + disable, enable, and modify the contents of the field. The user must be able to set the contents + of this field within a user preference or application defaults configuration. +<P> + + We suggest, though do not require, that a convenient toggle interface be provided for the user + to enable or disable the sending of <CODE>From</CODE> and <CODE>Referer</CODE> information. +<P> + +<H3>12.5 <A NAME="PathNameSecurity">Attacks Based On File and Path Names</A></H3> + + Implementations of HTTP origin servers should be careful to restrict the documents returned + by HTTP requests to be only those that were intended by the server administrators. If an HTTP + server translates HTTP URIs directly into file system calls, the server must take special care + not to serve files that were not intended to be delivered to HTTP clients. For example, Unix, + Microsoft Windows, and other operating systems use ".." as a path component to indicate a + directory level above the current one. On such a system, an HTTP server must disallow any + such construct in the <CODE>Request-URI</CODE> if it would otherwise allow access to a resource outside those + intended to be accessible via the HTTP server. Similarly, files intended for reference only + internally to the server (such as access control files, configuration files, and script code) must + be protected from inappropriate retrieval, since they might contain sensitive information. + Experience has shown that minor bugs in such HTTP server implementations have turned into + security risks. +<P> + +<H2>13. <A NAME="Acknowledgments">Acknowledgments</A></H2> + + This specification makes heavy use of the augmented BNF and generic constructs defined by + David H. Crocker for RFC 822<A HREF="#RefSTD11"> [7]</A>. Similarly, it reuses many of the definitions provided by + Nathaniel Borenstein and Ned Freed for MIME<A HREF="#RefMIME1"> [5]</A>. We hope that their inclusion in this + specification will help reduce past confusion over the relationship between HTTP/1.0 and + Internet mail message formats. +<P> + + The HTTP protocol has evolved considerably over the past four years. It has benefited from a + large and active developer community--the many people who have participated on the + <EM>www-talk</EM> mailing list--and it is that community which has been most responsible for the + success of HTTP and of the World-Wide Web in general. Marc Andreessen, Robert Cailliau, + Daniel W. Connolly, Bob Denny, Jean-Francois Groff, Phillip M. Hallam-Baker, Håkon W. Lie, + Ari Luotonen, Rob McCool, Lou Montulli, Dave Raggett, Tony Sanders, and + Marc VanHeyningen deserve special recognition for their efforts in defining aspects of the + protocol for early versions of this specification. +<P> + + Paul Hoffman contributed sections regarding the informational status of this document and + Appendices C and D. +<P> + + This document has benefited greatly from the comments of all those participating in the + HTTP-WG. In addition to those already mentioned, the following individuals have contributed + to this specification: +<P> + +<PRE> + Gary Adams Harald Tveit Alvestrand + Keith Ball Brian Behlendorf + Paul Burchard Maurizio Codogno + Mike Cowlishaw Roman Czyborra + Michael A. Dolan John Franks + Jim Gettys Marc Hedlund + Koen Holtman Alex Hopmann + Bob Jernigan Shel Kaphan + Martijn Koster Dave Kristol + Daniel LaLiberte Paul Leach + Albert Lunde John C. Mallery + Larry Masinter Mitra + Jeffrey Mogul Gavin Nicol + Bill Perry Jeffrey Perry + Owen Rees Luigi Rizzo + David Robinson Marc Salomon + Rich Salz Jim Seidman + Chuck Shotton Eric W. Sink + Simon E. Spero Robert S. Thau + François Yergeau Mary Ellen Zurko + Jean-Philippe Martin-Flatin +</PRE> + +<H2>14. <A NAME="References">References</A></H2> + +<DL COMPACT> +<DT>[1] +<DD><A NAME="RefGopher">Anklesaria, F., McCahill, M., Lindner, P., Johnson, D., Torrey, D., and B. Alberti</A>, + <A HREF="http://ds.internic.net/rfc/rfc1436.txt">"The Internet Gopher Protocol: + A distributed document search and retrieval protocol"</A>, + RFC 1436, University of Minnesota, March 1993. + +<DT>[2] +<DD><A NAME="RefURI">Berners-Lee, T.</A>, + <A HREF="http://ds.internic.net/rfc/rfc1630.txt">"Universal Resource Identifiers in WWW: + A Unifying Syntax for the Expression of Names and Addresses of Objects on the Network as used in the + World-Wide Web"</A>, RFC 1630, CERN, June 1994. + +<DT>[3] +<DD><A NAME="RefHTML">Berners-Lee, T., and D. Connolly</A>, + <A HREF="http://ds.internic.net/rfc/rfc1866.txt">"Hypertext Markup Language - 2.0"</A>, + RFC 1866, MIT/W3C, November 1995. + +<DT>[4] +<DD><A NAME="RefURL">Berners-Lee, T., Masinter, L., and M. McCahill</A>, + <A HREF="http://ds.internic.net/rfc/rfc1738.txt">"Uniform Resource Locators (URL)"</A>, + RFC 1738, CERN, Xerox PARC, University of Minnesota, December 1994. + +<DT>[5] +<DD><A NAME="RefMIME1">Borenstein, N., and N. Freed</A>, + <A HREF="http://ds.internic.net/rfc/rfc1521.ps">"MIME (Multipurpose Internet Mail Extensions) Part One: + Mechanisms for Specifying and Describing the Format of Internet Message Bodies"</A>, + RFC 1521, Bellcore, Innosoft, September 1993. + +<DT>[6] +<DD><A NAME="RefSTD3">Braden, R.</A>, + <A HREF="http://ds.internic.net/std/std3.txt">"Requirements for Internet hosts - application and support"</A>, + STD 3, RFC 1123, IETF, October 1989. + +<DT>[7] +<DD><A NAME="RefSTD11">Crocker, D.</A>, + <A HREF="http://ds.internic.net/std/std11.txt">"Standard for the Format of ARPA Internet Text Messages"</A>, + STD 11, RFC 822, UDEL, August 1982. + +<DT>[8] +<DD><A NAME="RefWAIS">F. Davis, B. Kahle, H. Morris, J. Salem, T. Shen, R. Wang, J. Sui, and M. Grinbaum</A>. + "WAIS Interface Protocol Prototype Functional Specification." (v1.5), Thinking + Machines Corporation, April 1990. + +<DT>[9] +<DD><A NAME="RefRelURL">Fielding, R.</A>, + <A HREF="http://ds.internic.net/rfc/rfc1808.txt">"Relative Uniform Resource Locators"</A>, + RFC 1808, UC Irvine, June 1995. + +<DT>[10] +<DD><A NAME="RefUSENET">Horton, M., and R. Adams</A>, + <A HREF="http://ds.internic.net/rfc/rfc1036.txt">"Standard for interchange of USENET messages"</A>, + RFC 1036 (Obsoletes RFC 850), AT&T Bell Laboratories, Center for Seismic Studies, December 1987. + +<DT>[11] +<DD><A NAME="RefNNTP">Kantor, B., and P. Lapsley</A>, + <A HREF="http://ds.internic.net/rfc/rfc977.txt">"Network News Transfer Protocol: + A Proposed Standard for the Stream-Based Transmission of News"</A>, + RFC 977, UC San Diego, UC Berkeley, February 1986. + +<DT>[12] +<DD><A NAME="RefSMTP">Postel, J.</A>, + <A HREF="http://ds.internic.net/std/std10.txt">"Simple Mail Transfer Protocol"</A>, + STD 10, RFC 821, USC/ISI, August 1982. + +<DT>[13] +<DD><A NAME="RefMediaType">Postel, J.</A>, + <A HREF="http://ds.internic.net/rfc/rfc1590.txt">"Media Type Registration Procedure"</A>, + RFC 1590, USC/ISI, March 1994. + +<DT>[14] +<DD><A NAME="RefFTP">Postel, J., and J. Reynolds</A>, + <A HREF="http://ds.internic.net/std/std9.txt">"File Transfer Protocol (FTP)"</A>, + STD 9, RFC 959, USC/ISI, October 1985. + +<DT>[15] +<DD><A NAME="RefIANA">Reynolds, J., and J. Postel</A>, + <A HREF="http://ds.internic.net/std/std2.txt">"Assigned Numbers"</A>, + STD 2, RFC 1700, USC/ISI, October 1994. + +<DT>[16] +<DD><A NAME="RefURN">Sollins, K., and L. Masinter</A>, + <A HREF="http://ds.internic.net/rfc/rfc1737.txt">"Functional Requirements for Uniform Resource Names."</A> + RFC 1737, MIT/LCS, Xerox Corporation, December 1994. + +<DT>[17] +<DD><A NAME="RefASCII">US-ASCII</A>. + Coded Character Set - 7-Bit American Standard Code for Information + Interchange. Standard ANSI X3.4-1986, ANSI, 1986. + +<DT>[18] +<DD><A NAME="RefISO8859">ISO-8859</A>. + International Standard -- Information Processing --<BR> + 8-bit Single-Byte Coded Graphic Character Sets --<BR> + Part 1: Latin alphabet No. 1, ISO 8859-1:1987.<BR> + Part 2: Latin alphabet No. 2, ISO 8859-2, 1987.<BR> + Part 3: Latin alphabet No. 3, ISO 8859-3, 1988.<BR> + Part 4: Latin alphabet No. 4, ISO 8859-4, 1988.<BR> + Part 5: Latin/Cyrillic alphabet, ISO 8859-5, 1988.<BR> + Part 6: Latin/Arabic alphabet, ISO 8859-6, 1987.<BR> + Part 7: Latin/Greek alphabet, ISO 8859-7, 1987.<BR> + Part 8: Latin/Hebrew alphabet, ISO 8859-8, 1988.<BR> + Part 9: Latin alphabet No. 5, ISO 8859-9, 1990. +</DL> + +<H2>15. <A NAME="Authors">Authors' Addresses</A></H2> + + <STRONG>Tim Berners-Lee</STRONG><BR> + Director, W3 Consortium<BR> + MIT Laboratory for Computer Science<BR> + 545 Technology Square<BR> + Cambridge, MA 02139, U.S.A.<BR> +<BR> + Fax: +1 (617) 258 8682<BR> + EMail: [email protected] +<P> + <STRONG>Roy T. Fielding</STRONG><BR> + Department of Information and Computer Science<BR> + University of California<BR> + Irvine, CA 92717-3425, U.S.A.<BR> +<BR> + Fax: +1 (714) 824-4056<BR> + EMail: [email protected] +<P> + <STRONG>Henrik Frystyk Nielsen</STRONG><BR> + W3 Consortium<BR> + MIT Laboratory for Computer Science<BR> + 545 Technology Square<BR> + Cambridge, MA 02139, U.S.A.<BR> +<BR> + Fax: +1 (617) 258 8682<BR> + EMail: [email protected] +<P> + +<H2><A NAME="Appendices">Appendices</A></H2> + + These appendices are provided for informational reasons only -- they do not form a part of the + HTTP/1.0 specification. +<P> + +<H2>A. <A NAME="message_http">Internet Media Type message/http</A></H2> + + In addition to defining the HTTP/1.0 protocol, this document serves as the specification for the + Internet media type "message/http". The following is to be registered with IANA<A HREF="#RefMediaType"> [13]</A>. +<P> + +<PRE> + Media Type name: message + + Media subtype name: http + + Required parameters: none + + Optional parameters: version, msgtype + + version: The HTTP-Version number of the enclosed message + (e.g., "1.0"). If not present, the version can be + determined from the first line of the body. + + msgtype: The message type -- "request" or "response". If not + present, the type can be determined from the first + line of the body. + + Encoding considerations: only "7bit", "8bit", or "binary" are + permitted + + Security considerations: none +</PRE> + +<H2>B. <A NAME="Tolerant">Tolerant Applications</A></H2> + + Although this document specifies the requirements for the generation of HTTP/1.0 messages, + not all applications will be correct in their implementation. We therefore recommend that + operational applications be tolerant of deviations whenever those deviations can be interpreted + unambiguously. +<P> + + Clients should be tolerant in parsing the <CODE>Status-Line</CODE> and servers tolerant when parsing the + <CODE>Request-Line</CODE>. In particular, they should accept any amount of <CODE>SP</CODE> or <CODE>HT</CODE> characters between + fields, even though only a single <CODE>SP</CODE> is required. +<P> + + The line terminator for <CODE>HTTP-header</CODE> fields is the sequence <CODE>CRLF</CODE>. However, we recommend that + applications, when parsing such headers, recognize a single <CODE>LF</CODE> as a line terminator and ignore + the leading <CODE>CR</CODE>. +<P> + +<H2>C. <A NAME="MIME">Relationship to MIME</A></H2> + + HTTP/1.0 uses many of the constructs defined for Internet Mail (RFC 822<A HREF="#RefSTD11"> [7]</A>) and the + Multipurpose Internet Mail Extensions (MIME<A HREF="#RefMIME1"> [5]</A>) to allow entities to be transmitted in an + open variety of representations and with extensible mechanisms. However, RFC 1521 + discusses mail, and HTTP has a few features that are different than those described in + RFC 1521. These differences were carefully chosen to optimize performance over binary + connections, to allow greater freedom in the use of new media types, to make date comparisons + easier, and to acknowledge the practice of some early HTTP servers and clients. +<P> + + At the time of this writing, it is expected that RFC 1521 will be revised. The revisions may + include some of the practices found in HTTP/1.0 but not in RFC 1521. +<P> + + This appendix describes specific areas where HTTP differs from RFC 1521. Proxies and + gateways to strict MIME environments should be aware of these differences and provide the + appropriate conversions where necessary. Proxies and gateways from MIME environments to + HTTP also need to be aware of the differences because some conversions may be required. +<P> + +<H3>C.1 <A NAME="MIME-Canonical">Conversion to Canonical Form</A></H3> + + RFC 1521 requires that an Internet mail entity be converted to canonical form prior to being + transferred, as described in Appendix G of RFC 1521<A HREF="#RefMIME1"> [5]</A>. + <A HREF="#TextCanonicalization">Section 3.6.1</A> of this document + describes the forms allowed for subtypes of the "text" media type when transmitted over HTTP. +<P> + + RFC 1521 requires that content with a Content-Type of "text" represent line breaks as CRLF + and forbids the use of CR or LF outside of line break sequences. HTTP allows CRLF, bare CR, + and bare LF to indicate a line break within text content when a message is transmitted over + HTTP. +<P> + + Where it is possible, a proxy or gateway from HTTP to a strict RFC 1521 environment should + translate all line breaks within the text media types described in <A HREF="#TextCanonicalization">Section 3.6.1</A> of this document + to the RFC 1521 canonical form of <CODE>CRLF</CODE>. Note, however, that this may be complicated by the + presence of a <CODE>Content-Encoding</CODE> and by the fact that HTTP allows the use of some character sets + which do not use octets 13 and 10 to represent <CODE>CR</CODE> and <CODE>LF</CODE>, as is the case for some multi-byte + character sets. +<P> + +<H3>C.2 <A NAME="MIME-Date">Conversion of Date Formats</A></H3> + + HTTP/1.0 uses a restricted set of date formats (<A HREF="#DateFormats">Section 3.3</A>) to simplify the process of date + comparison. Proxies and gateways from other protocols should ensure that any <CODE>Date</CODE> header + field present in a message conforms to one of the HTTP/1.0 formats and rewrite the date if + necessary. +<P> + +<H3>C.3 <A NAME="MIME-CE">Introduction of Content-Encoding</A></H3> + + RFC 1521 does not include any concept equivalent to HTTP/1.0's <CODE>Content-Encoding</CODE> header + field. Since this acts as a modifier on the media type, proxies and gateways from HTTP to + MIME-compliant protocols must either change the value of the <CODE>Content-Type</CODE> header field or + decode the <CODE>Entity-Body</CODE> before forwarding the message. (Some experimental applications of + <CODE>Content-Type</CODE> for Internet mail have used a media-type parameter of + <CODE>";conversions=<content-coding>"</CODE> to perform an equivalent function as + Content-Encoding. However, this parameter is not part of RFC 1521.) +<P> + +<H3>C.4 <A NAME="MIME-CTE">No Content-Transfer-Encoding</A></H3> + + HTTP does not use the Content-Transfer-Encoding (CTE) field of RFC 1521. Proxies and + gateways from MIME-compliant protocols to HTTP must remove any non-identity CTE + ("quoted-printable" or "base64") encoding prior to delivering the response message to an + HTTP client. +<P> + + Proxies and gateways from HTTP to MIME-compliant protocols are responsible for ensuring + that the message is in the correct format and encoding for safe transport on that protocol, where + "safe transport" is defined by the limitations of the protocol being used. Such a proxy or + gateway should label the data with an appropriate Content-Transfer-Encoding if doing so will + improve the likelihood of safe transport over the destination protocol. +<P> + +<H3>C.5 <A NAME="MIME-parts">HTTP Header Fields in Multipart Body-Parts</A></H3> + + In RFC 1521, most header fields in multipart body-parts are generally ignored unless the field + name begins with "Content-". In HTTP/1.0, multipart body-parts may contain any HTTP + header fields which are significant to the meaning of that part. +<P> + +<H2>D. <A NAME="Additional">Additional Features</A></H2> + + This appendix documents protocol elements used by some existing HTTP implementations, + but not consistently and correctly across most HTTP/1.0 applications. Implementors should be + aware of these features, but cannot rely upon their presence in, or interoperability with, other + HTTP/1.0 applications. +<P> + +<H3>D.1 <A NAME="Additional-Methods">Additional Request Methods</A></H3> + +<H4>D.1.1 <A NAME="PUT">PUT</A></H4> + + The PUT method requests that the enclosed entity be stored under the supplied <CODE>Request-URI</CODE>. If + the <CODE>Request-URI</CODE> refers to an already existing resource, the enclosed entity should be considered + as a modified version of the one residing on the origin server. If the <CODE>Request-URI</CODE> does not point + to an existing resource, and that URI is capable of being defined as a new resource by the + requesting user agent, the origin server can create the resource with that URI. +<P> + + The fundamental difference between the POST and PUT requests is reflected in the different + meaning of the <CODE>Request-URI</CODE>. The URI in a POST request identifies the resource that will handle + the enclosed entity as data to be processed. That resource may be a data-accepting process, a + gateway to some other protocol, or a separate entity that accepts annotations. In contrast, the + URI in a PUT request identifies the entity enclosed with the request -- the user agent knows + what URI is intended and the server should not apply the request to some other resource. +<P> + +<H4>D.1.2 <A NAME="DELETE">DELETE</A></H4> + + The DELETE method requests that the origin server delete the resource identified by the + <CODE>Request-URI</CODE>. +<P> + +<H4>D.1.3 <A NAME="LINK">LINK</A></H4> + + The LINK method establishes one or more Link relationships between the existing resource + identified by the <CODE>Request-URI</CODE> and other existing resources. +<P> + +<H4>D.1.4 <A NAME="UNLINK">UNLINK</A></H4> + + The UNLINK method removes one or more Link relationships from the existing resource + identified by the <CODE>Request-URI</CODE>. +<P> + +<H3>D.2 <A NAME="Additional-Headers">Additional Header Field Definitions</A></H3> + +<H4>D.2.1 <A NAME="Accept">Accept</A></H4> + + The Accept request-header field can be used to indicate a list of media ranges which are + acceptable as a response to the request. The asterisk "*" character is used to group media types + into ranges, with "*/*" indicating all media types and "type/*" indicating all subtypes of that + type. The set of ranges given by the client should represent what types are acceptable given the + context of the request. +<P> + +<H4>D.2.2 <A NAME="Accept-Charset">Accept-Charset</A></H4> + + The Accept-Charset request-header field can be used to indicate a list of preferred character sets + other than the default US-ASCII and ISO-8859-1. This field allows clients capable of + understanding more comprehensive or special-purpose character sets to signal that capability + to a server which is capable of representing documents in those character sets. +<P> + +<H4>D.2.3 <A NAME="Accept-Encoding">Accept-Encoding</A></H4> + + The Accept-Encoding request-header field is similar to Accept, but restricts the content-coding + values which are acceptable in the response. +<P> + +<H4>D.2.4 <A NAME="Accept-Language">Accept-Language</A></H4> + + The Accept-Language request-header field is similar to Accept, but restricts the set of natural + languages that are preferred as a response to the request. +<P> + +<H4>D.2.5 <A NAME="Content-Language">Content-Language</A></H4> + + The Content-Language entity-header field describes the natural language(s) of the intended + audience for the enclosed entity. Note that this may not be equivalent to all the languages used + within the entity. +<P> + +<H4>D.2.6 <A NAME="Link">Link</A></H4> + + The Link entity-header field provides a means for describing a relationship between the entity + and some other resource. An entity may include multiple Link values. Links at the + metainformation level typically indicate relationships like hierarchical structure and navigation + paths. +<P> + +<H4>D.2.7 <A NAME="MIME-Version">MIME-Version</A></H4> + + HTTP messages may include a single MIME-Version general-header field to indicate what + version of the MIME protocol was used to construct the message. Use of the MIME-Version + header field, as defined by RFC 1521<A HREF="#RefMIME1"> [5]</A>, should indicate that the message is + MIME-conformant. Unfortunately, some older HTTP/1.0 servers send it indiscriminately, and + thus this field should be ignored. +<P> + +<H4>D.2.8 <A NAME="Retry-After">Retry-After</A></H4> + + The Retry-After response-header field can be used with a 503 (service unavailable) response to + indicate how long the service is expected to be unavailable to the requesting client. The value + of this field can be either an HTTP-date or an integer number of seconds (in decimal) after the + time of the response. +<P> + +<H4>D.2.9 <A NAME="Title">Title</A></H4> + + The Title entity-header field indicates the title of the entity. +<P> + +<H4>D.2.10 <A NAME="URI-header">URI</A></H4> + + The URI entity-header field may contain some or all of the Uniform Resource Identifiers + (<A HREF="#URI">Section 3.2</A>) by which the <CODE>Request-URI</CODE> resource can be identified. There is no guarantee that + the resource can be accessed using the URI(s) specified. + +</BODY></HTML> diff --git a/lib/inets/doc/archive/rfc2068.txt b/lib/inets/doc/archive/rfc2068.txt new file mode 100644 index 0000000000..e16e4fdf7d --- /dev/null +++ b/lib/inets/doc/archive/rfc2068.txt @@ -0,0 +1,9075 @@ + + + + + + +Network Working Group R. Fielding +Request for Comments: 2068 UC Irvine +Category: Standards Track J. Gettys + J. Mogul + DEC + H. Frystyk + T. Berners-Lee + MIT/LCS + January 1997 + + + Hypertext Transfer Protocol -- HTTP/1.1 + +Status of this Memo + + This document specifies an Internet standards track protocol for the + Internet community, and requests discussion and suggestions for + improvements. Please refer to the current edition of the "Internet + Official Protocol Standards" (STD 1) for the standardization state + and status of this protocol. Distribution of this memo is unlimited. + +Abstract + + The Hypertext Transfer Protocol (HTTP) is an application-level + protocol for distributed, collaborative, hypermedia information + systems. It is a generic, stateless, object-oriented protocol which + can be used for many tasks, such as name servers and distributed + object management systems, through extension of its request methods. + A feature of HTTP is the typing and negotiation of data + representation, allowing systems to be built independently of the + data being transferred. + + HTTP has been in use by the World-Wide Web global information + initiative since 1990. This specification defines the protocol + referred to as "HTTP/1.1". + +Table of Contents + + 1 Introduction.............................................7 + 1.1 Purpose ..............................................7 + 1.2 Requirements .........................................7 + 1.3 Terminology ..........................................8 + 1.4 Overall Operation ...................................11 + 2 Notational Conventions and Generic Grammar..............13 + 2.1 Augmented BNF .......................................13 + 2.2 Basic Rules .........................................15 + 3 Protocol Parameters.....................................17 + 3.1 HTTP Version ........................................17 + + + +Fielding, et. al. Standards Track [Page 1] + +RFC 2068 HTTP/1.1 January 1997 + + + 3.2 Uniform Resource Identifiers ........................18 + 3.2.1 General Syntax ...................................18 + 3.2.2 http URL .........................................19 + 3.2.3 URI Comparison ...................................20 + 3.3 Date/Time Formats ...................................21 + 3.3.1 Full Date ........................................21 + 3.3.2 Delta Seconds ....................................22 + 3.4 Character Sets ......................................22 + 3.5 Content Codings .....................................23 + 3.6 Transfer Codings ....................................24 + 3.7 Media Types .........................................25 + 3.7.1 Canonicalization and Text Defaults ...............26 + 3.7.2 Multipart Types ..................................27 + 3.8 Product Tokens ......................................28 + 3.9 Quality Values ......................................28 + 3.10 Language Tags ......................................28 + 3.11 Entity Tags ........................................29 + 3.12 Range Units ........................................30 + 4 HTTP Message............................................30 + 4.1 Message Types .......................................30 + 4.2 Message Headers .....................................31 + 4.3 Message Body ........................................32 + 4.4 Message Length ......................................32 + 4.5 General Header Fields ...............................34 + 5 Request.................................................34 + 5.1 Request-Line ........................................34 + 5.1.1 Method ...........................................35 + 5.1.2 Request-URI ......................................35 + 5.2 The Resource Identified by a Request ................37 + 5.3 Request Header Fields ...............................37 + 6 Response................................................38 + 6.1 Status-Line .........................................38 + 6.1.1 Status Code and Reason Phrase ....................39 + 6.2 Response Header Fields ..............................41 + 7 Entity..................................................41 + 7.1 Entity Header Fields ................................41 + 7.2 Entity Body .........................................42 + 7.2.1 Type .............................................42 + 7.2.2 Length ...........................................43 + 8 Connections.............................................43 + 8.1 Persistent Connections ..............................43 + 8.1.1 Purpose ..........................................43 + 8.1.2 Overall Operation ................................44 + 8.1.3 Proxy Servers ....................................45 + 8.1.4 Practical Considerations .........................45 + 8.2 Message Transmission Requirements ...................46 + 9 Method Definitions......................................48 + 9.1 Safe and Idempotent Methods .........................48 + + + +Fielding, et. al. Standards Track [Page 2] + +RFC 2068 HTTP/1.1 January 1997 + + + 9.1.1 Safe Methods .....................................48 + 9.1.2 Idempotent Methods ...............................49 + 9.2 OPTIONS .............................................49 + 9.3 GET .................................................50 + 9.4 HEAD ................................................50 + 9.5 POST ................................................51 + 9.6 PUT .................................................52 + 9.7 DELETE ..............................................53 + 9.8 TRACE ...............................................53 + 10 Status Code Definitions................................53 + 10.1 Informational 1xx ..................................54 + 10.1.1 100 Continue ....................................54 + 10.1.2 101 Switching Protocols .........................54 + 10.2 Successful 2xx .....................................54 + 10.2.1 200 OK ..........................................54 + 10.2.2 201 Created .....................................55 + 10.2.3 202 Accepted ....................................55 + 10.2.4 203 Non-Authoritative Information ...............55 + 10.2.5 204 No Content ..................................55 + 10.2.6 205 Reset Content ...............................56 + 10.2.7 206 Partial Content .............................56 + 10.3 Redirection 3xx ....................................56 + 10.3.1 300 Multiple Choices ............................57 + 10.3.2 301 Moved Permanently ...........................57 + 10.3.3 302 Moved Temporarily ...........................58 + 10.3.4 303 See Other ...................................58 + 10.3.5 304 Not Modified ................................58 + 10.3.6 305 Use Proxy ...................................59 + 10.4 Client Error 4xx ...................................59 + 10.4.1 400 Bad Request .................................60 + 10.4.2 401 Unauthorized ................................60 + 10.4.3 402 Payment Required ............................60 + 10.4.4 403 Forbidden ...................................60 + 10.4.5 404 Not Found ...................................60 + 10.4.6 405 Method Not Allowed ..........................61 + 10.4.7 406 Not Acceptable ..............................61 + 10.4.8 407 Proxy Authentication Required ...............61 + 10.4.9 408 Request Timeout .............................62 + 10.4.10 409 Conflict ...................................62 + 10.4.11 410 Gone .......................................62 + 10.4.12 411 Length Required ............................63 + 10.4.13 412 Precondition Failed ........................63 + 10.4.14 413 Request Entity Too Large ...................63 + 10.4.15 414 Request-URI Too Long .......................63 + 10.4.16 415 Unsupported Media Type .....................63 + 10.5 Server Error 5xx ...................................64 + 10.5.1 500 Internal Server Error .......................64 + 10.5.2 501 Not Implemented .............................64 + + + +Fielding, et. al. Standards Track [Page 3] + +RFC 2068 HTTP/1.1 January 1997 + + + 10.5.3 502 Bad Gateway .................................64 + 10.5.4 503 Service Unavailable .........................64 + 10.5.5 504 Gateway Timeout .............................64 + 10.5.6 505 HTTP Version Not Supported ..................65 + 11 Access Authentication..................................65 + 11.1 Basic Authentication Scheme ........................66 + 11.2 Digest Authentication Scheme .......................67 + 12 Content Negotiation....................................67 + 12.1 Server-driven Negotiation ..........................68 + 12.2 Agent-driven Negotiation ...........................69 + 12.3 Transparent Negotiation ............................70 + 13 Caching in HTTP........................................70 + 13.1.1 Cache Correctness ...............................72 + 13.1.2 Warnings ........................................73 + 13.1.3 Cache-control Mechanisms ........................74 + 13.1.4 Explicit User Agent Warnings ....................74 + 13.1.5 Exceptions to the Rules and Warnings ............75 + 13.1.6 Client-controlled Behavior ......................75 + 13.2 Expiration Model ...................................75 + 13.2.1 Server-Specified Expiration .....................75 + 13.2.2 Heuristic Expiration ............................76 + 13.2.3 Age Calculations ................................77 + 13.2.4 Expiration Calculations .........................79 + 13.2.5 Disambiguating Expiration Values ................80 + 13.2.6 Disambiguating Multiple Responses ...............80 + 13.3 Validation Model ...................................81 + 13.3.1 Last-modified Dates .............................82 + 13.3.2 Entity Tag Cache Validators .....................82 + 13.3.3 Weak and Strong Validators ......................82 + 13.3.4 Rules for When to Use Entity Tags and Last- + modified Dates..........................................85 + 13.3.5 Non-validating Conditionals .....................86 + 13.4 Response Cachability ...............................86 + 13.5 Constructing Responses From Caches .................87 + 13.5.1 End-to-end and Hop-by-hop Headers ...............88 + 13.5.2 Non-modifiable Headers ..........................88 + 13.5.3 Combining Headers ...............................89 + 13.5.4 Combining Byte Ranges ...........................90 + 13.6 Caching Negotiated Responses .......................90 + 13.7 Shared and Non-Shared Caches .......................91 + 13.8 Errors or Incomplete Response Cache Behavior .......91 + 13.9 Side Effects of GET and HEAD .......................92 + 13.10 Invalidation After Updates or Deletions ...........92 + 13.11 Write-Through Mandatory ...........................93 + 13.12 Cache Replacement .................................93 + 13.13 History Lists .....................................93 + 14 Header Field Definitions...............................94 + 14.1 Accept .............................................95 + + + +Fielding, et. al. Standards Track [Page 4] + +RFC 2068 HTTP/1.1 January 1997 + + + 14.2 Accept-Charset .....................................97 + 14.3 Accept-Encoding ....................................97 + 14.4 Accept-Language ....................................98 + 14.5 Accept-Ranges ......................................99 + 14.6 Age ................................................99 + 14.7 Allow .............................................100 + 14.8 Authorization .....................................100 + 14.9 Cache-Control .....................................101 + 14.9.1 What is Cachable ...............................103 + 14.9.2 What May be Stored by Caches ...................103 + 14.9.3 Modifications of the Basic Expiration Mechanism 104 + 14.9.4 Cache Revalidation and Reload Controls .........105 + 14.9.5 No-Transform Directive .........................107 + 14.9.6 Cache Control Extensions .......................108 + 14.10 Connection .......................................109 + 14.11 Content-Base .....................................109 + 14.12 Content-Encoding .................................110 + 14.13 Content-Language .................................110 + 14.14 Content-Length ...................................111 + 14.15 Content-Location .................................112 + 14.16 Content-MD5 ......................................113 + 14.17 Content-Range ....................................114 + 14.18 Content-Type .....................................116 + 14.19 Date .............................................116 + 14.20 ETag .............................................117 + 14.21 Expires ..........................................117 + 14.22 From .............................................118 + 14.23 Host .............................................119 + 14.24 If-Modified-Since ................................119 + 14.25 If-Match .........................................121 + 14.26 If-None-Match ....................................122 + 14.27 If-Range .........................................123 + 14.28 If-Unmodified-Since ..............................124 + 14.29 Last-Modified ....................................124 + 14.30 Location .........................................125 + 14.31 Max-Forwards .....................................125 + 14.32 Pragma ...........................................126 + 14.33 Proxy-Authenticate ...............................127 + 14.34 Proxy-Authorization ..............................127 + 14.35 Public ...........................................127 + 14.36 Range ............................................128 + 14.36.1 Byte Ranges ...................................128 + 14.36.2 Range Retrieval Requests ......................130 + 14.37 Referer ..........................................131 + 14.38 Retry-After ......................................131 + 14.39 Server ...........................................132 + 14.40 Transfer-Encoding ................................132 + 14.41 Upgrade ..........................................132 + + + +Fielding, et. al. Standards Track [Page 5] + +RFC 2068 HTTP/1.1 January 1997 + + + 14.42 User-Agent .......................................134 + 14.43 Vary .............................................134 + 14.44 Via ..............................................135 + 14.45 Warning ..........................................137 + 14.46 WWW-Authenticate .................................139 + 15 Security Considerations...............................139 + 15.1 Authentication of Clients .........................139 + 15.2 Offering a Choice of Authentication Schemes .......140 + 15.3 Abuse of Server Log Information ...................141 + 15.4 Transfer of Sensitive Information .................141 + 15.5 Attacks Based On File and Path Names ..............142 + 15.6 Personal Information ..............................143 + 15.7 Privacy Issues Connected to Accept Headers ........143 + 15.8 DNS Spoofing ......................................144 + 15.9 Location Headers and Spoofing .....................144 + 16 Acknowledgments.......................................144 + 17 References............................................146 + 18 Authors' Addresses....................................149 + 19 Appendices............................................150 + 19.1 Internet Media Type message/http ..................150 + 19.2 Internet Media Type multipart/byteranges ..........150 + 19.3 Tolerant Applications .............................151 + 19.4 Differences Between HTTP Entities and + MIME Entities...........................................152 + 19.4.1 Conversion to Canonical Form ...................152 + 19.4.2 Conversion of Date Formats .....................153 + 19.4.3 Introduction of Content-Encoding ...............153 + 19.4.4 No Content-Transfer-Encoding ...................153 + 19.4.5 HTTP Header Fields in Multipart Body-Parts .....153 + 19.4.6 Introduction of Transfer-Encoding ..............154 + 19.4.7 MIME-Version ...................................154 + 19.5 Changes from HTTP/1.0 .............................154 + 19.5.1 Changes to Simplify Multi-homed Web Servers and + Conserve IP Addresses .................................155 + 19.6 Additional Features ...............................156 + 19.6.1 Additional Request Methods .....................156 + 19.6.2 Additional Header Field Definitions ............156 + 19.7 Compatibility with Previous Versions ..............160 + 19.7.1 Compatibility with HTTP/1.0 Persistent + Connections............................................161 + + + + + + + + + + + +Fielding, et. al. Standards Track [Page 6] + +RFC 2068 HTTP/1.1 January 1997 + + +1 Introduction + +1.1 Purpose + + The Hypertext Transfer Protocol (HTTP) is an application-level + protocol for distributed, collaborative, hypermedia information + systems. HTTP has been in use by the World-Wide Web global + information initiative since 1990. The first version of HTTP, + referred to as HTTP/0.9, was a simple protocol for raw data transfer + across the Internet. HTTP/1.0, as defined by RFC 1945 [6], improved + the protocol by allowing messages to be in the format of MIME-like + messages, containing metainformation about the data transferred and + modifiers on the request/response semantics. However, HTTP/1.0 does + not sufficiently take into consideration the effects of hierarchical + proxies, caching, the need for persistent connections, and virtual + hosts. In addition, the proliferation of incompletely-implemented + applications calling themselves "HTTP/1.0" has necessitated a + protocol version change in order for two communicating applications + to determine each other's true capabilities. + + This specification defines the protocol referred to as "HTTP/1.1". + This protocol includes more stringent requirements than HTTP/1.0 in + order to ensure reliable implementation of its features. + + Practical information systems require more functionality than simple + retrieval, including search, front-end update, and annotation. HTTP + allows an open-ended set of methods that indicate the purpose of a + request. It builds on the discipline of reference provided by the + Uniform Resource Identifier (URI) [3][20], as a location (URL) [4] or + name (URN) , for indicating the resource to which a method is to be + applied. Messages are passed in a format similar to that used by + Internet mail as defined by the Multipurpose Internet Mail Extensions + (MIME). + + HTTP is also used as a generic protocol for communication between + user agents and proxies/gateways to other Internet systems, including + those supported by the SMTP [16], NNTP [13], FTP [18], Gopher [2], + and WAIS [10] protocols. In this way, HTTP allows basic hypermedia + access to resources available from diverse applications. + +1.2 Requirements + + This specification uses the same words as RFC 1123 [8] for defining + the significance of each particular requirement. These words are: + + MUST + This word or the adjective "required" means that the item is an + absolute requirement of the specification. + + + +Fielding, et. al. Standards Track [Page 7] + +RFC 2068 HTTP/1.1 January 1997 + + + SHOULD + This word or the adjective "recommended" means that there may + exist valid reasons in particular circumstances to ignore this + item, but the full implications should be understood and the case + carefully weighed before choosing a different course. + + MAY + This word or the adjective "optional" means that this item is + truly optional. One vendor may choose to include the item because + a particular marketplace requires it or because it enhances the + product, for example; another vendor may omit the same item. + + An implementation is not compliant if it fails to satisfy one or more + of the MUST requirements for the protocols it implements. An + implementation that satisfies all the MUST and all the SHOULD + requirements for its protocols is said to be "unconditionally + compliant"; one that satisfies all the MUST requirements but not all + the SHOULD requirements for its protocols is said to be + "conditionally compliant." + +1.3 Terminology + + This specification uses a number of terms to refer to the roles + played by participants in, and objects of, the HTTP communication. + + connection + A transport layer virtual circuit established between two programs + for the purpose of communication. + + message + The basic unit of HTTP communication, consisting of a structured + sequence of octets matching the syntax defined in section 4 and + transmitted via the connection. + + request + An HTTP request message, as defined in section 5. + + response + An HTTP response message, as defined in section 6. + + resource + A network data object or service that can be identified by a URI, + as defined in section 3.2. Resources may be available in multiple + representations (e.g. multiple languages, data formats, size, + resolutions) or vary in other ways. + + + + + + +Fielding, et. al. Standards Track [Page 8] + +RFC 2068 HTTP/1.1 January 1997 + + + entity + The information transferred as the payload of a request or + response. An entity consists of metainformation in the form of + entity-header fields and content in the form of an entity-body, as + described in section 7. + + representation + An entity included with a response that is subject to content + negotiation, as described in section 12. There may exist multiple + representations associated with a particular response status. + + content negotiation + The mechanism for selecting the appropriate representation when + servicing a request, as described in section 12. The + representation of entities in any response can be negotiated + (including error responses). + + variant + A resource may have one, or more than one, representation(s) + associated with it at any given instant. Each of these + representations is termed a `variant.' Use of the term `variant' + does not necessarily imply that the resource is subject to content + negotiation. + + client + A program that establishes connections for the purpose of sending + requests. + + user agent + The client which initiates a request. These are often browsers, + editors, spiders (web-traversing robots), or other end user tools. + + server + An application program that accepts connections in order to + service requests by sending back responses. Any given program may + be capable of being both a client and a server; our use of these + terms refers only to the role being performed by the program for a + particular connection, rather than to the program's capabilities + in general. Likewise, any server may act as an origin server, + proxy, gateway, or tunnel, switching behavior based on the nature + of each request. + + origin server + The server on which a given resource resides or is to be created. + + + + + + + +Fielding, et. al. Standards Track [Page 9] + +RFC 2068 HTTP/1.1 January 1997 + + + proxy + An intermediary program which acts as both a server and a client + for the purpose of making requests on behalf of other clients. + Requests are serviced internally or by passing them on, with + possible translation, to other servers. A proxy must implement + both the client and server requirements of this specification. + + gateway + A server which acts as an intermediary for some other server. + Unlike a proxy, a gateway receives requests as if it were the + origin server for the requested resource; the requesting client + may not be aware that it is communicating with a gateway. + + tunnel + An intermediary program which is acting as a blind relay between + two connections. Once active, a tunnel is not considered a party + to the HTTP communication, though the tunnel may have been + initiated by an HTTP request. The tunnel ceases to exist when both + ends of the relayed connections are closed. + + cache + A program's local store of response messages and the subsystem + that controls its message storage, retrieval, and deletion. A + cache stores cachable responses in order to reduce the response + time and network bandwidth consumption on future, equivalent + requests. Any client or server may include a cache, though a cache + cannot be used by a server that is acting as a tunnel. + + cachable + A response is cachable if a cache is allowed to store a copy of + the response message for use in answering subsequent requests. The + rules for determining the cachability of HTTP responses are + defined in section 13. Even if a resource is cachable, there may + be additional constraints on whether a cache can use the cached + copy for a particular request. + + first-hand + A response is first-hand if it comes directly and without + unnecessary delay from the origin server, perhaps via one or more + proxies. A response is also first-hand if its validity has just + been checked directly with the origin server. + + explicit expiration time + The time at which the origin server intends that an entity should + no longer be returned by a cache without further validation. + + + + + + +Fielding, et. al. Standards Track [Page 10] + +RFC 2068 HTTP/1.1 January 1997 + + + heuristic expiration time + An expiration time assigned by a cache when no explicit expiration + time is available. + + age + The age of a response is the time since it was sent by, or + successfully validated with, the origin server. + + freshness lifetime + The length of time between the generation of a response and its + expiration time. + + fresh + A response is fresh if its age has not yet exceeded its freshness + lifetime. + + stale + A response is stale if its age has passed its freshness lifetime. + + semantically transparent + A cache behaves in a "semantically transparent" manner, with + respect to a particular response, when its use affects neither the + requesting client nor the origin server, except to improve + performance. When a cache is semantically transparent, the client + receives exactly the same response (except for hop-by-hop headers) + that it would have received had its request been handled directly + by the origin server. + + validator + A protocol element (e.g., an entity tag or a Last-Modified time) + that is used to find out whether a cache entry is an equivalent + copy of an entity. + +1.4 Overall Operation + + The HTTP protocol is a request/response protocol. A client sends a + request to the server in the form of a request method, URI, and + protocol version, followed by a MIME-like message containing request + modifiers, client information, and possible body content over a + connection with a server. The server responds with a status line, + including the message's protocol version and a success or error code, + followed by a MIME-like message containing server information, entity + metainformation, and possible entity-body content. The relationship + between HTTP and MIME is described in appendix 19.4. + + + + + + + +Fielding, et. al. Standards Track [Page 11] + +RFC 2068 HTTP/1.1 January 1997 + + + Most HTTP communication is initiated by a user agent and consists of + a request to be applied to a resource on some origin server. In the + simplest case, this may be accomplished via a single connection (v) + between the user agent (UA) and the origin server (O). + + request chain ------------------------> + UA -------------------v------------------- O + <----------------------- response chain + + A more complicated situation occurs when one or more intermediaries + are present in the request/response chain. There are three common + forms of intermediary: proxy, gateway, and tunnel. A proxy is a + forwarding agent, receiving requests for a URI in its absolute form, + rewriting all or part of the message, and forwarding the reformatted + request toward the server identified by the URI. A gateway is a + receiving agent, acting as a layer above some other server(s) and, if + necessary, translating the requests to the underlying server's + protocol. A tunnel acts as a relay point between two connections + without changing the messages; tunnels are used when the + communication needs to pass through an intermediary (such as a + firewall) even when the intermediary cannot understand the contents + of the messages. + + request chain --------------------------------------> + UA -----v----- A -----v----- B -----v----- C -----v----- O + <------------------------------------- response chain + + The figure above shows three intermediaries (A, B, and C) between the + user agent and origin server. A request or response message that + travels the whole chain will pass through four separate connections. + This distinction is important because some HTTP communication options + may apply only to the connection with the nearest, non-tunnel + neighbor, only to the end-points of the chain, or to all connections + along the chain. Although the diagram is linear, each participant + may be engaged in multiple, simultaneous communications. For example, + B may be receiving requests from many clients other than A, and/or + forwarding requests to servers other than C, at the same time that it + is handling A's request. + + Any party to the communication which is not acting as a tunnel may + employ an internal cache for handling requests. The effect of a cache + is that the request/response chain is shortened if one of the + participants along the chain has a cached response applicable to that + request. The following illustrates the resulting chain if B has a + cached copy of an earlier response from O (via C) for a request which + has not been cached by UA or A. + + + + + +Fielding, et. al. Standards Track [Page 12] + +RFC 2068 HTTP/1.1 January 1997 + + + request chain ----------> + UA -----v----- A -----v----- B - - - - - - C - - - - - - O + <--------- response chain + + Not all responses are usefully cachable, and some requests may + contain modifiers which place special requirements on cache behavior. + HTTP requirements for cache behavior and cachable responses are + defined in section 13. + + In fact, there are a wide variety of architectures and configurations + of caches and proxies currently being experimented with or deployed + across the World Wide Web; these systems include national hierarchies + of proxy caches to save transoceanic bandwidth, systems that + broadcast or multicast cache entries, organizations that distribute + subsets of cached data via CD-ROM, and so on. HTTP systems are used + in corporate intranets over high-bandwidth links, and for access via + PDAs with low-power radio links and intermittent connectivity. The + goal of HTTP/1.1 is to support the wide diversity of configurations + already deployed while introducing protocol constructs that meet the + needs of those who build web applications that require high + reliability and, failing that, at least reliable indications of + failure. + + HTTP communication usually takes place over TCP/IP connections. The + default port is TCP 80, but other ports can be used. This does not + preclude HTTP from being implemented on top of any other protocol on + the Internet, or on other networks. HTTP only presumes a reliable + transport; any protocol that provides such guarantees can be used; + the mapping of the HTTP/1.1 request and response structures onto the + transport data units of the protocol in question is outside the scope + of this specification. + + In HTTP/1.0, most implementations used a new connection for each + request/response exchange. In HTTP/1.1, a connection may be used for + one or more request/response exchanges, although connections may be + closed for a variety of reasons (see section 8.1). + +2 Notational Conventions and Generic Grammar + +2.1 Augmented BNF + + All of the mechanisms specified in this document are described in + both prose and an augmented Backus-Naur Form (BNF) similar to that + used by RFC 822 [9]. Implementers will need to be familiar with the + notation in order to understand this specification. The augmented BNF + includes the following constructs: + + + + + +Fielding, et. al. Standards Track [Page 13] + +RFC 2068 HTTP/1.1 January 1997 + + +name = definition + The name of a rule is simply the name itself (without any enclosing + "<" and ">") and is separated from its definition by the equal "=" + character. Whitespace is only significant in that indentation of + continuation lines is used to indicate a rule definition that spans + more than one line. Certain basic rules are in uppercase, such as + SP, LWS, HT, CRLF, DIGIT, ALPHA, etc. Angle brackets are used + within definitions whenever their presence will facilitate + discerning the use of rule names. + +"literal" + Quotation marks surround literal text. Unless stated otherwise, the + text is case-insensitive. + +rule1 | rule2 + Elements separated by a bar ("|") are alternatives, e.g., "yes | + no" will accept yes or no. + +(rule1 rule2) + Elements enclosed in parentheses are treated as a single element. + Thus, "(elem (foo | bar) elem)" allows the token sequences "elem + foo elem" and "elem bar elem". + +*rule + The character "*" preceding an element indicates repetition. The + full form is "<n>*<m>element" indicating at least <n> and at most + <m> occurrences of element. Default values are 0 and infinity so + that "*(element)" allows any number, including zero; "1*element" + requires at least one; and "1*2element" allows one or two. + +[rule] + Square brackets enclose optional elements; "[foo bar]" is + equivalent to "*1(foo bar)". + +N rule + Specific repetition: "<n>(element)" is equivalent to + "<n>*<n>(element)"; that is, exactly <n> occurrences of (element). + Thus 2DIGIT is a 2-digit number, and 3ALPHA is a string of three + alphabetic characters. + +#rule + A construct "#" is defined, similar to "*", for defining lists of + elements. The full form is "<n>#<m>element " indicating at least + <n> and at most <m> elements, each separated by one or more commas + (",") and optional linear whitespace (LWS). This makes the usual + form of lists very easy; a rule such as "( *LWS element *( *LWS "," + *LWS element )) " can be shown as "1#element". Wherever this + construct is used, null elements are allowed, but do not contribute + + + +Fielding, et. al. Standards Track [Page 14] + +RFC 2068 HTTP/1.1 January 1997 + + + to the count of elements present. That is, "(element), , (element) + " is permitted, but counts as only two elements. Therefore, where + at least one element is required, at least one non-null element + must be present. Default values are 0 and infinity so that + "#element" allows any number, including zero; "1#element" requires + at least one; and "1#2element" allows one or two. + +; comment + A semi-colon, set off some distance to the right of rule text, + starts a comment that continues to the end of line. This is a + simple way of including useful notes in parallel with the + specifications. + +implied *LWS + The grammar described by this specification is word-based. Except + where noted otherwise, linear whitespace (LWS) can be included + between any two adjacent words (token or quoted-string), and + between adjacent tokens and delimiters (tspecials), without + changing the interpretation of a field. At least one delimiter + (tspecials) must exist between any two tokens, since they would + otherwise be interpreted as a single token. + +2.2 Basic Rules + + The following rules are used throughout this specification to + describe basic parsing constructs. The US-ASCII coded character set + is defined by ANSI X3.4-1986 [21]. + + OCTET = <any 8-bit sequence of data> + CHAR = <any US-ASCII character (octets 0 - 127)> + UPALPHA = <any US-ASCII uppercase letter "A".."Z"> + LOALPHA = <any US-ASCII lowercase letter "a".."z"> + ALPHA = UPALPHA | LOALPHA + DIGIT = <any US-ASCII digit "0".."9"> + CTL = <any US-ASCII control character + (octets 0 - 31) and DEL (127)> + CR = <US-ASCII CR, carriage return (13)> + LF = <US-ASCII LF, linefeed (10)> + SP = <US-ASCII SP, space (32)> + HT = <US-ASCII HT, horizontal-tab (9)> + <"> = <US-ASCII double-quote mark (34)> + + + + + + + + + + +Fielding, et. al. Standards Track [Page 15] + +RFC 2068 HTTP/1.1 January 1997 + + + HTTP/1.1 defines the sequence CR LF as the end-of-line marker for all + protocol elements except the entity-body (see appendix 19.3 for + tolerant applications). The end-of-line marker within an entity-body + is defined by its associated media type, as described in section 3.7. + + CRLF = CR LF + + HTTP/1.1 headers can be folded onto multiple lines if the + continuation line begins with a space or horizontal tab. All linear + white space, including folding, has the same semantics as SP. + + LWS = [CRLF] 1*( SP | HT ) + + The TEXT rule is only used for descriptive field contents and values + that are not intended to be interpreted by the message parser. Words + of *TEXT may contain characters from character sets other than ISO + 8859-1 [22] only when encoded according to the rules of RFC 1522 + [14]. + + TEXT = <any OCTET except CTLs, + but including LWS> + + Hexadecimal numeric characters are used in several protocol elements. + + HEX = "A" | "B" | "C" | "D" | "E" | "F" + | "a" | "b" | "c" | "d" | "e" | "f" | DIGIT + + Many HTTP/1.1 header field values consist of words separated by LWS + or special characters. These special characters MUST be in a quoted + string to be used within a parameter value. + + token = 1*<any CHAR except CTLs or tspecials> + + tspecials = "(" | ")" | "<" | ">" | "@" + | "," | ";" | ":" | "\" | <"> + | "/" | "[" | "]" | "?" | "=" + | "{" | "}" | SP | HT + + Comments can be included in some HTTP header fields by surrounding + the comment text with parentheses. Comments are only allowed in + fields containing "comment" as part of their field value definition. + In all other fields, parentheses are considered part of the field + value. + + comment = "(" *( ctext | comment ) ")" + ctext = <any TEXT excluding "(" and ")"> + + + + + +Fielding, et. al. Standards Track [Page 16] + +RFC 2068 HTTP/1.1 January 1997 + + + A string of text is parsed as a single word if it is quoted using + double-quote marks. + + quoted-string = ( <"> *(qdtext) <"> ) + + qdtext = <any TEXT except <">> + + The backslash character ("\") may be used as a single-character quoting + mechanism only within quoted-string and comment constructs. + + quoted-pair = "\" CHAR + +3 Protocol Parameters + +3.1 HTTP Version + + HTTP uses a "<major>.<minor>" numbering scheme to indicate versions + of the protocol. The protocol versioning policy is intended to allow + the sender to indicate the format of a message and its capacity for + understanding further HTTP communication, rather than the features + obtained via that communication. No change is made to the version + number for the addition of message components which do not affect + communication behavior or which only add to extensible field values. + The <minor> number is incremented when the changes made to the + protocol add features which do not change the general message parsing + algorithm, but which may add to the message semantics and imply + additional capabilities of the sender. The <major> number is + incremented when the format of a message within the protocol is + changed. + + The version of an HTTP message is indicated by an HTTP-Version field + in the first line of the message. + + HTTP-Version = "HTTP" "/" 1*DIGIT "." 1*DIGIT + + Note that the major and minor numbers MUST be treated as separate + integers and that each may be incremented higher than a single digit. + Thus, HTTP/2.4 is a lower version than HTTP/2.13, which in turn is + lower than HTTP/12.3. Leading zeros MUST be ignored by recipients and + MUST NOT be sent. + + Applications sending Request or Response messages, as defined by this + specification, MUST include an HTTP-Version of "HTTP/1.1". Use of + this version number indicates that the sending application is at + least conditionally compliant with this specification. + + The HTTP version of an application is the highest HTTP version for + which the application is at least conditionally compliant. + + + +Fielding, et. al. Standards Track [Page 17] + +RFC 2068 HTTP/1.1 January 1997 + + + Proxy and gateway applications must be careful when forwarding + messages in protocol versions different from that of the application. + Since the protocol version indicates the protocol capability of the + sender, a proxy/gateway MUST never send a message with a version + indicator which is greater than its actual version; if a higher + version request is received, the proxy/gateway MUST either downgrade + the request version, respond with an error, or switch to tunnel + behavior. Requests with a version lower than that of the + proxy/gateway's version MAY be upgraded before being forwarded; the + proxy/gateway's response to that request MUST be in the same major + version as the request. + + Note: Converting between versions of HTTP may involve modification + of header fields required or forbidden by the versions involved. + +3.2 Uniform Resource Identifiers + + URIs have been known by many names: WWW addresses, Universal Document + Identifiers, Universal Resource Identifiers , and finally the + combination of Uniform Resource Locators (URL) and Names (URN). As + far as HTTP is concerned, Uniform Resource Identifiers are simply + formatted strings which identify--via name, location, or any other + characteristic--a resource. + +3.2.1 General Syntax + + URIs in HTTP can be represented in absolute form or relative to some + known base URI, depending upon the context of their use. The two + forms are differentiated by the fact that absolute URIs always begin + with a scheme name followed by a colon. + + URI = ( absoluteURI | relativeURI ) [ "#" fragment ] + + absoluteURI = scheme ":" *( uchar | reserved ) + + relativeURI = net_path | abs_path | rel_path + + net_path = "//" net_loc [ abs_path ] + abs_path = "/" rel_path + rel_path = [ path ] [ ";" params ] [ "?" query ] + + path = fsegment *( "/" segment ) + fsegment = 1*pchar + segment = *pchar + + params = param *( ";" param ) + param = *( pchar | "/" ) + + + + +Fielding, et. al. Standards Track [Page 18] + +RFC 2068 HTTP/1.1 January 1997 + + + scheme = 1*( ALPHA | DIGIT | "+" | "-" | "." ) + net_loc = *( pchar | ";" | "?" ) + + query = *( uchar | reserved ) + fragment = *( uchar | reserved ) + + pchar = uchar | ":" | "@" | "&" | "=" | "+" + uchar = unreserved | escape + unreserved = ALPHA | DIGIT | safe | extra | national + + escape = "%" HEX HEX + reserved = ";" | "/" | "?" | ":" | "@" | "&" | "=" | "+" + extra = "!" | "*" | "'" | "(" | ")" | "," + safe = "$" | "-" | "_" | "." + unsafe = CTL | SP | <"> | "#" | "%" | "<" | ">" + national = <any OCTET excluding ALPHA, DIGIT, + reserved, extra, safe, and unsafe> + + For definitive information on URL syntax and semantics, see RFC 1738 + [4] and RFC 1808 [11]. The BNF above includes national characters not + allowed in valid URLs as specified by RFC 1738, since HTTP servers + are not restricted in the set of unreserved characters allowed to + represent the rel_path part of addresses, and HTTP proxies may + receive requests for URIs not defined by RFC 1738. + + The HTTP protocol does not place any a priori limit on the length of + a URI. Servers MUST be able to handle the URI of any resource they + serve, and SHOULD be able to handle URIs of unbounded length if they + provide GET-based forms that could generate such URIs. A server + SHOULD return 414 (Request-URI Too Long) status if a URI is longer + than the server can handle (see section 10.4.15). + + Note: Servers should be cautious about depending on URI lengths + above 255 bytes, because some older client or proxy implementations + may not properly support these lengths. + +3.2.2 http URL + + The "http" scheme is used to locate network resources via the HTTP + protocol. This section defines the scheme-specific syntax and + semantics for http URLs. + + + + + + + + + + +Fielding, et. al. Standards Track [Page 19] + +RFC 2068 HTTP/1.1 January 1997 + + + http_URL = "http:" "//" host [ ":" port ] [ abs_path ] + + host = <A legal Internet host domain name + or IP address (in dotted-decimal form), + as defined by Section 2.1 of RFC 1123> + + port = *DIGIT + + If the port is empty or not given, port 80 is assumed. The semantics + are that the identified resource is located at the server listening + for TCP connections on that port of that host, and the Request-URI + for the resource is abs_path. The use of IP addresses in URL's SHOULD + be avoided whenever possible (see RFC 1900 [24]). If the abs_path is + not present in the URL, it MUST be given as "/" when used as a + Request-URI for a resource (section 5.1.2). + +3.2.3 URI Comparison + + When comparing two URIs to decide if they match or not, a client + SHOULD use a case-sensitive octet-by-octet comparison of the entire + URIs, with these exceptions: + + o A port that is empty or not given is equivalent to the default + port for that URI; + + o Comparisons of host names MUST be case-insensitive; + + o Comparisons of scheme names MUST be case-insensitive; + + o An empty abs_path is equivalent to an abs_path of "/". + + Characters other than those in the "reserved" and "unsafe" sets (see + section 3.2) are equivalent to their ""%" HEX HEX" encodings. + + For example, the following three URIs are equivalent: + + http://abc.com:80/~smith/home.html + http://ABC.com/%7Esmith/home.html + http://ABC.com:/%7esmith/home.html + + + + + + + + + + + + +Fielding, et. al. Standards Track [Page 20] + +RFC 2068 HTTP/1.1 January 1997 + + +3.3 Date/Time Formats + +3.3.1 Full Date + + HTTP applications have historically allowed three different formats + for the representation of date/time stamps: + + Sun, 06 Nov 1994 08:49:37 GMT ; RFC 822, updated by RFC 1123 + Sunday, 06-Nov-94 08:49:37 GMT ; RFC 850, obsoleted by RFC 1036 + Sun Nov 6 08:49:37 1994 ; ANSI C's asctime() format + + The first format is preferred as an Internet standard and represents + a fixed-length subset of that defined by RFC 1123 (an update to RFC + 822). The second format is in common use, but is based on the + obsolete RFC 850 [12] date format and lacks a four-digit year. + HTTP/1.1 clients and servers that parse the date value MUST accept + all three formats (for compatibility with HTTP/1.0), though they MUST + only generate the RFC 1123 format for representing HTTP-date values + in header fields. + + Note: Recipients of date values are encouraged to be robust in + accepting date values that may have been sent by non-HTTP + applications, as is sometimes the case when retrieving or posting + messages via proxies/gateways to SMTP or NNTP. + + All HTTP date/time stamps MUST be represented in Greenwich Mean Time + (GMT), without exception. This is indicated in the first two formats + by the inclusion of "GMT" as the three-letter abbreviation for time + zone, and MUST be assumed when reading the asctime format. + + HTTP-date = rfc1123-date | rfc850-date | asctime-date + + rfc1123-date = wkday "," SP date1 SP time SP "GMT" + rfc850-date = weekday "," SP date2 SP time SP "GMT" + asctime-date = wkday SP date3 SP time SP 4DIGIT + + date1 = 2DIGIT SP month SP 4DIGIT + ; day month year (e.g., 02 Jun 1982) + date2 = 2DIGIT "-" month "-" 2DIGIT + ; day-month-year (e.g., 02-Jun-82) + date3 = month SP ( 2DIGIT | ( SP 1DIGIT )) + ; month day (e.g., Jun 2) + + time = 2DIGIT ":" 2DIGIT ":" 2DIGIT + ; 00:00:00 - 23:59:59 + + wkday = "Mon" | "Tue" | "Wed" + | "Thu" | "Fri" | "Sat" | "Sun" + + + +Fielding, et. al. Standards Track [Page 21] + +RFC 2068 HTTP/1.1 January 1997 + + + weekday = "Monday" | "Tuesday" | "Wednesday" + | "Thursday" | "Friday" | "Saturday" | "Sunday" + + month = "Jan" | "Feb" | "Mar" | "Apr" + | "May" | "Jun" | "Jul" | "Aug" + | "Sep" | "Oct" | "Nov" | "Dec" + + Note: HTTP requirements for the date/time stamp format apply only + to their usage within the protocol stream. Clients and servers are + not required to use these formats for user presentation, request + logging, etc. + +3.3.2 Delta Seconds + + Some HTTP header fields allow a time value to be specified as an + integer number of seconds, represented in decimal, after the time + that the message was received. + + delta-seconds = 1*DIGIT + +3.4 Character Sets + + HTTP uses the same definition of the term "character set" as that + described for MIME: + + The term "character set" is used in this document to refer to a + method used with one or more tables to convert a sequence of octets + into a sequence of characters. Note that unconditional conversion + in the other direction is not required, in that not all characters + may be available in a given character set and a character set may + provide more than one sequence of octets to represent a particular + character. This definition is intended to allow various kinds of + character encodings, from simple single-table mappings such as US- + ASCII to complex table switching methods such as those that use ISO + 2022's techniques. However, the definition associated with a MIME + character set name MUST fully specify the mapping to be performed + from octets to characters. In particular, use of external profiling + information to determine the exact mapping is not permitted. + + Note: This use of the term "character set" is more commonly + referred to as a "character encoding." However, since HTTP and MIME + share the same registry, it is important that the terminology also + be shared. + + + + + + + + +Fielding, et. al. Standards Track [Page 22] + +RFC 2068 HTTP/1.1 January 1997 + + + HTTP character sets are identified by case-insensitive tokens. The + complete set of tokens is defined by the IANA Character Set registry + [19]. + + charset = token + + Although HTTP allows an arbitrary token to be used as a charset + value, any token that has a predefined value within the IANA + Character Set registry MUST represent the character set defined by + that registry. Applications SHOULD limit their use of character sets + to those defined by the IANA registry. + +3.5 Content Codings + + Content coding values indicate an encoding transformation that has + been or can be applied to an entity. Content codings are primarily + used to allow a document to be compressed or otherwise usefully + transformed without losing the identity of its underlying media type + and without loss of information. Frequently, the entity is stored in + coded form, transmitted directly, and only decoded by the recipient. + + content-coding = token + + All content-coding values are case-insensitive. HTTP/1.1 uses + content-coding values in the Accept-Encoding (section 14.3) and + Content-Encoding (section 14.12) header fields. Although the value + describes the content-coding, what is more important is that it + indicates what decoding mechanism will be required to remove the + encoding. + + The Internet Assigned Numbers Authority (IANA) acts as a registry for + content-coding value tokens. Initially, the registry contains the + following tokens: + + gzip An encoding format produced by the file compression program "gzip" + (GNU zip) as described in RFC 1952 [25]. This format is a Lempel- + Ziv coding (LZ77) with a 32 bit CRC. + + compress + The encoding format produced by the common UNIX file compression + program "compress". This format is an adaptive Lempel-Ziv-Welch + coding (LZW). + + + + + + + + + +Fielding, et. al. Standards Track [Page 23] + +RFC 2068 HTTP/1.1 January 1997 + + + Note: Use of program names for the identification of encoding + formats is not desirable and should be discouraged for future + encodings. Their use here is representative of historical practice, + not good design. For compatibility with previous implementations of + HTTP, applications should consider "x-gzip" and "x-compress" to be + equivalent to "gzip" and "compress" respectively. + + deflate The "zlib" format defined in RFC 1950[31] in combination with + the "deflate" compression mechanism described in RFC 1951[29]. + + New content-coding value tokens should be registered; to allow + interoperability between clients and servers, specifications of the + content coding algorithms needed to implement a new value should be + publicly available and adequate for independent implementation, and + conform to the purpose of content coding defined in this section. + +3.6 Transfer Codings + + Transfer coding values are used to indicate an encoding + transformation that has been, can be, or may need to be applied to an + entity-body in order to ensure "safe transport" through the network. + This differs from a content coding in that the transfer coding is a + property of the message, not of the original entity. + + transfer-coding = "chunked" | transfer-extension + + transfer-extension = token + + All transfer-coding values are case-insensitive. HTTP/1.1 uses + transfer coding values in the Transfer-Encoding header field (section + 14.40). + + Transfer codings are analogous to the Content-Transfer-Encoding + values of MIME , which were designed to enable safe transport of + binary data over a 7-bit transport service. However, safe transport + has a different focus for an 8bit-clean transfer protocol. In HTTP, + the only unsafe characteristic of message-bodies is the difficulty in + determining the exact body length (section 7.2.2), or the desire to + encrypt data over a shared transport. + + The chunked encoding modifies the body of a message in order to + transfer it as a series of chunks, each with its own size indicator, + followed by an optional footer containing entity-header fields. This + allows dynamically-produced content to be transferred along with the + information necessary for the recipient to verify that it has + received the full message. + + + + + +Fielding, et. al. Standards Track [Page 24] + +RFC 2068 HTTP/1.1 January 1997 + + + Chunked-Body = *chunk + "0" CRLF + footer + CRLF + + chunk = chunk-size [ chunk-ext ] CRLF + chunk-data CRLF + + hex-no-zero = <HEX excluding "0"> + + chunk-size = hex-no-zero *HEX + chunk-ext = *( ";" chunk-ext-name [ "=" chunk-ext-value ] ) + chunk-ext-name = token + chunk-ext-val = token | quoted-string + chunk-data = chunk-size(OCTET) + + footer = *entity-header + + The chunked encoding is ended by a zero-sized chunk followed by the + footer, which is terminated by an empty line. The purpose of the + footer is to provide an efficient way to supply information about an + entity that is generated dynamically; applications MUST NOT send + header fields in the footer which are not explicitly defined as being + appropriate for the footer, such as Content-MD5 or future extensions + to HTTP for digital signatures or other facilities. + + An example process for decoding a Chunked-Body is presented in + appendix 19.4.6. + + All HTTP/1.1 applications MUST be able to receive and decode the + "chunked" transfer coding, and MUST ignore transfer coding extensions + they do not understand. A server which receives an entity-body with a + transfer-coding it does not understand SHOULD return 501 + (Unimplemented), and close the connection. A server MUST NOT send + transfer-codings to an HTTP/1.0 client. + +3.7 Media Types + + HTTP uses Internet Media Types in the Content-Type (section 14.18) + and Accept (section 14.1) header fields in order to provide open and + extensible data typing and type negotiation. + + media-type = type "/" subtype *( ";" parameter ) + type = token + subtype = token + + Parameters may follow the type/subtype in the form of attribute/value + pairs. + + + +Fielding, et. al. Standards Track [Page 25] + +RFC 2068 HTTP/1.1 January 1997 + + + parameter = attribute "=" value + attribute = token + value = token | quoted-string + + The type, subtype, and parameter attribute names are case- + insensitive. Parameter values may or may not be case-sensitive, + depending on the semantics of the parameter name. Linear white space + (LWS) MUST NOT be used between the type and subtype, nor between an + attribute and its value. User agents that recognize the media-type + MUST process (or arrange to be processed by any external applications + used to process that type/subtype by the user agent) the parameters + for that MIME type as described by that type/subtype definition to + the and inform the user of any problems discovered. + + Note: some older HTTP applications do not recognize media type + parameters. When sending data to older HTTP applications, + implementations should only use media type parameters when they are + required by that type/subtype definition. + + Media-type values are registered with the Internet Assigned Number + Authority (IANA). The media type registration process is outlined in + RFC 2048 [17]. Use of non-registered media types is discouraged. + +3.7.1 Canonicalization and Text Defaults + + Internet media types are registered with a canonical form. In + general, an entity-body transferred via HTTP messages MUST be + represented in the appropriate canonical form prior to its + transmission; the exception is "text" types, as defined in the next + paragraph. + + When in canonical form, media subtypes of the "text" type use CRLF as + the text line break. HTTP relaxes this requirement and allows the + transport of text media with plain CR or LF alone representing a line + break when it is done consistently for an entire entity-body. HTTP + applications MUST accept CRLF, bare CR, and bare LF as being + representative of a line break in text media received via HTTP. In + addition, if the text is represented in a character set that does not + use octets 13 and 10 for CR and LF respectively, as is the case for + some multi-byte character sets, HTTP allows the use of whatever octet + sequences are defined by that character set to represent the + equivalent of CR and LF for line breaks. This flexibility regarding + line breaks applies only to text media in the entity-body; a bare CR + or LF MUST NOT be substituted for CRLF within any of the HTTP control + structures (such as header fields and multipart boundaries). + + If an entity-body is encoded with a Content-Encoding, the underlying + data MUST be in a form defined above prior to being encoded. + + + +Fielding, et. al. Standards Track [Page 26] + +RFC 2068 HTTP/1.1 January 1997 + + + The "charset" parameter is used with some media types to define the + character set (section 3.4) of the data. When no explicit charset + parameter is provided by the sender, media subtypes of the "text" + type are defined to have a default charset value of "ISO-8859-1" when + received via HTTP. Data in character sets other than "ISO-8859-1" or + its subsets MUST be labeled with an appropriate charset value. + + Some HTTP/1.0 software has interpreted a Content-Type header without + charset parameter incorrectly to mean "recipient should guess." + Senders wishing to defeat this behavior MAY include a charset + parameter even when the charset is ISO-8859-1 and SHOULD do so when + it is known that it will not confuse the recipient. + + Unfortunately, some older HTTP/1.0 clients did not deal properly with + an explicit charset parameter. HTTP/1.1 recipients MUST respect the + charset label provided by the sender; and those user agents that have + a provision to "guess" a charset MUST use the charset from the + content-type field if they support that charset, rather than the + recipient's preference, when initially displaying a document. + +3.7.2 Multipart Types + + MIME provides for a number of "multipart" types -- encapsulations of + one or more entities within a single message-body. All multipart + types share a common syntax, as defined in MIME [7], and MUST + include a boundary parameter as part of the media type value. The + message body is itself a protocol element and MUST therefore use only + CRLF to represent line breaks between body-parts. Unlike in MIME, the + epilogue of any multipart message MUST be empty; HTTP applications + MUST NOT transmit the epilogue (even if the original multipart + contains an epilogue). + + In HTTP, multipart body-parts MAY contain header fields which are + significant to the meaning of that part. A Content-Location header + field (section 14.15) SHOULD be included in the body-part of each + enclosed entity that can be identified by a URL. + + In general, an HTTP user agent SHOULD follow the same or similar + behavior as a MIME user agent would upon receipt of a multipart type. + If an application receives an unrecognized multipart subtype, the + application MUST treat it as being equivalent to "multipart/mixed". + + Note: The "multipart/form-data" type has been specifically defined + for carrying form data suitable for processing via the POST request + method, as described in RFC 1867 [15]. + + + + + + +Fielding, et. al. Standards Track [Page 27] + +RFC 2068 HTTP/1.1 January 1997 + + +3.8 Product Tokens + + Product tokens are used to allow communicating applications to + identify themselves by software name and version. Most fields using + product tokens also allow sub-products which form a significant part + of the application to be listed, separated by whitespace. By + convention, the products are listed in order of their significance + for identifying the application. + + product = token ["/" product-version] + product-version = token + + Examples: + + User-Agent: CERN-LineMode/2.15 libwww/2.17b3 + Server: Apache/0.8.4 + + Product tokens should be short and to the point -- use of them for + advertising or other non-essential information is explicitly + forbidden. Although any token character may appear in a product- + version, this token SHOULD only be used for a version identifier + (i.e., successive versions of the same product SHOULD only differ in + the product-version portion of the product value). + +3.9 Quality Values + + HTTP content negotiation (section 12) uses short "floating point" + numbers to indicate the relative importance ("weight") of various + negotiable parameters. A weight is normalized to a real number in the + range 0 through 1, where 0 is the minimum and 1 the maximum value. + HTTP/1.1 applications MUST NOT generate more than three digits after + the decimal point. User configuration of these values SHOULD also be + limited in this fashion. + + qvalue = ( "0" [ "." 0*3DIGIT ] ) + | ( "1" [ "." 0*3("0") ] ) + + "Quality values" is a misnomer, since these values merely represent + relative degradation in desired quality. + +3.10 Language Tags + + A language tag identifies a natural language spoken, written, or + otherwise conveyed by human beings for communication of information + to other human beings. Computer languages are explicitly excluded. + HTTP uses language tags within the Accept-Language and Content- + Language fields. + + + + +Fielding, et. al. Standards Track [Page 28] + +RFC 2068 HTTP/1.1 January 1997 + + + The syntax and registry of HTTP language tags is the same as that + defined by RFC 1766 [1]. In summary, a language tag is composed of 1 + or more parts: A primary language tag and a possibly empty series of + subtags: + + language-tag = primary-tag *( "-" subtag ) + + primary-tag = 1*8ALPHA + subtag = 1*8ALPHA + + Whitespace is not allowed within the tag and all tags are case- + insensitive. The name space of language tags is administered by the + IANA. Example tags include: + + en, en-US, en-cockney, i-cherokee, x-pig-latin + + where any two-letter primary-tag is an ISO 639 language abbreviation + and any two-letter initial subtag is an ISO 3166 country code. (The + last three tags above are not registered tags; all but the last are + examples of tags which could be registered in future.) + +3.11 Entity Tags + + Entity tags are used for comparing two or more entities from the same + requested resource. HTTP/1.1 uses entity tags in the ETag (section + 14.20), If-Match (section 14.25), If-None-Match (section 14.26), and + If-Range (section 14.27) header fields. The definition of how they + are used and compared as cache validators is in section 13.3.3. An + entity tag consists of an opaque quoted string, possibly prefixed by + a weakness indicator. + + entity-tag = [ weak ] opaque-tag + + weak = "W/" + opaque-tag = quoted-string + + A "strong entity tag" may be shared by two entities of a resource + only if they are equivalent by octet equality. + + A "weak entity tag," indicated by the "W/" prefix, may be shared by + two entities of a resource only if the entities are equivalent and + could be substituted for each other with no significant change in + semantics. A weak entity tag can only be used for weak comparison. + + An entity tag MUST be unique across all versions of all entities + associated with a particular resource. A given entity tag value may + be used for entities obtained by requests on different URIs without + implying anything about the equivalence of those entities. + + + +Fielding, et. al. Standards Track [Page 29] + +RFC 2068 HTTP/1.1 January 1997 + + +3.12 Range Units + + HTTP/1.1 allows a client to request that only part (a range of) the + response entity be included within the response. HTTP/1.1 uses range + units in the Range (section 14.36) and Content-Range (section 14.17) + header fields. An entity may be broken down into subranges according + to various structural units. + + range-unit = bytes-unit | other-range-unit + + bytes-unit = "bytes" + other-range-unit = token + +The only range unit defined by HTTP/1.1 is "bytes". HTTP/1.1 + implementations may ignore ranges specified using other units. + HTTP/1.1 has been designed to allow implementations of applications + that do not depend on knowledge of ranges. + +4 HTTP Message + +4.1 Message Types + + HTTP messages consist of requests from client to server and responses + from server to client. + + HTTP-message = Request | Response ; HTTP/1.1 messages + + Request (section 5) and Response (section 6) messages use the generic + message format of RFC 822 [9] for transferring entities (the payload + of the message). Both types of message consist of a start-line, one + or more header fields (also known as "headers"), an empty line (i.e., + a line with nothing preceding the CRLF) indicating the end of the + header fields, and an optional message-body. + + generic-message = start-line + *message-header + CRLF + [ message-body ] + + start-line = Request-Line | Status-Line + + In the interest of robustness, servers SHOULD ignore any empty + line(s) received where a Request-Line is expected. In other words, if + the server is reading the protocol stream at the beginning of a + message and receives a CRLF first, it should ignore the CRLF. + + + + + + +Fielding, et. al. Standards Track [Page 30] + +RFC 2068 HTTP/1.1 January 1997 + + + Note: certain buggy HTTP/1.0 client implementations generate an + extra CRLF's after a POST request. To restate what is explicitly + forbidden by the BNF, an HTTP/1.1 client must not preface or follow + a request with an extra CRLF. + +4.2 Message Headers + + HTTP header fields, which include general-header (section 4.5), + request-header (section 5.3), response-header (section 6.2), and + entity-header (section 7.1) fields, follow the same generic format as + that given in Section 3.1 of RFC 822 [9]. Each header field consists + of a name followed by a colon (":") and the field value. Field names + are case-insensitive. The field value may be preceded by any amount + of LWS, though a single SP is preferred. Header fields can be + extended over multiple lines by preceding each extra line with at + least one SP or HT. Applications SHOULD follow "common form" when + generating HTTP constructs, since there might exist some + implementations that fail to accept anything beyond the common forms. + + message-header = field-name ":" [ field-value ] CRLF + + field-name = token + field-value = *( field-content | LWS ) + + field-content = <the OCTETs making up the field-value + and consisting of either *TEXT or combinations + of token, tspecials, and quoted-string> + + The order in which header fields with differing field names are + received is not significant. However, it is "good practice" to send + general-header fields first, followed by request-header or response- + header fields, and ending with the entity-header fields. + + Multiple message-header fields with the same field-name may be + present in a message if and only if the entire field-value for that + header field is defined as a comma-separated list [i.e., #(values)]. + It MUST be possible to combine the multiple header fields into one + "field-name: field-value" pair, without changing the semantics of the + message, by appending each subsequent field-value to the first, each + separated by a comma. The order in which header fields with the same + field-name are received is therefore significant to the + interpretation of the combined field value, and thus a proxy MUST NOT + change the order of these field values when a message is forwarded. + + + + + + + + +Fielding, et. al. Standards Track [Page 31] + +RFC 2068 HTTP/1.1 January 1997 + + +4.3 Message Body + + The message-body (if any) of an HTTP message is used to carry the + entity-body associated with the request or response. The message-body + differs from the entity-body only when a transfer coding has been + applied, as indicated by the Transfer-Encoding header field (section + 14.40). + + message-body = entity-body + | <entity-body encoded as per Transfer-Encoding> + + Transfer-Encoding MUST be used to indicate any transfer codings + applied by an application to ensure safe and proper transfer of the + message. Transfer-Encoding is a property of the message, not of the + entity, and thus can be added or removed by any application along the + request/response chain. + + The rules for when a message-body is allowed in a message differ for + requests and responses. + + The presence of a message-body in a request is signaled by the + inclusion of a Content-Length or Transfer-Encoding header field in + the request's message-headers. A message-body MAY be included in a + request only when the request method (section 5.1.1) allows an + entity-body. + + For response messages, whether or not a message-body is included with + a message is dependent on both the request method and the response + status code (section 6.1.1). All responses to the HEAD request method + MUST NOT include a message-body, even though the presence of entity- + header fields might lead one to believe they do. All 1xx + (informational), 204 (no content), and 304 (not modified) responses + MUST NOT include a message-body. All other responses do include a + message-body, although it may be of zero length. + +4.4 Message Length + + When a message-body is included with a message, the length of that + body is determined by one of the following (in order of precedence): + + 1. Any response message which MUST NOT include a message-body + (such as the 1xx, 204, and 304 responses and any response to a HEAD + request) is always terminated by the first empty line after the + header fields, regardless of the entity-header fields present in the + message. + + 2. If a Transfer-Encoding header field (section 14.40) is present and + indicates that the "chunked" transfer coding has been applied, then + + + +Fielding, et. al. Standards Track [Page 32] + +RFC 2068 HTTP/1.1 January 1997 + + + the length is defined by the chunked encoding (section 3.6). + + 3. If a Content-Length header field (section 14.14) is present, its + value in bytes represents the length of the message-body. + + 4. If the message uses the media type "multipart/byteranges", which is + self-delimiting, then that defines the length. This media type MUST + NOT be used unless the sender knows that the recipient can parse it; + the presence in a request of a Range header with multiple byte-range + specifiers implies that the client can parse multipart/byteranges + responses. + + 5. By the server closing the connection. (Closing the connection + cannot be used to indicate the end of a request body, since that + would leave no possibility for the server to send back a response.) + + For compatibility with HTTP/1.0 applications, HTTP/1.1 requests + containing a message-body MUST include a valid Content-Length header + field unless the server is known to be HTTP/1.1 compliant. If a + request contains a message-body and a Content-Length is not given, + the server SHOULD respond with 400 (bad request) if it cannot + determine the length of the message, or with 411 (length required) if + it wishes to insist on receiving a valid Content-Length. + + All HTTP/1.1 applications that receive entities MUST accept the + "chunked" transfer coding (section 3.6), thus allowing this mechanism + to be used for messages when the message length cannot be determined + in advance. + + Messages MUST NOT include both a Content-Length header field and the + "chunked" transfer coding. If both are received, the Content-Length + MUST be ignored. + + When a Content-Length is given in a message where a message-body is + allowed, its field value MUST exactly match the number of OCTETs in + the message-body. HTTP/1.1 user agents MUST notify the user when an + invalid length is received and detected. + + + + + + + + + + + + + + +Fielding, et. al. Standards Track [Page 33] + +RFC 2068 HTTP/1.1 January 1997 + + +4.5 General Header Fields + + There are a few header fields which have general applicability for + both request and response messages, but which do not apply to the + entity being transferred. These header fields apply only to the + message being transmitted. + + general-header = Cache-Control ; Section 14.9 + | Connection ; Section 14.10 + | Date ; Section 14.19 + | Pragma ; Section 14.32 + | Transfer-Encoding ; Section 14.40 + | Upgrade ; Section 14.41 + | Via ; Section 14.44 + + General-header field names can be extended reliably only in + combination with a change in the protocol version. However, new or + experimental header fields may be given the semantics of general + header fields if all parties in the communication recognize them to + be general-header fields. Unrecognized header fields are treated as + entity-header fields. + +5 Request + + A request message from a client to a server includes, within the + first line of that message, the method to be applied to the resource, + the identifier of the resource, and the protocol version in use. + + Request = Request-Line ; Section 5.1 + *( general-header ; Section 4.5 + | request-header ; Section 5.3 + | entity-header ) ; Section 7.1 + CRLF + [ message-body ] ; Section 7.2 + +5.1 Request-Line + + The Request-Line begins with a method token, followed by the + Request-URI and the protocol version, and ending with CRLF. The + elements are separated by SP characters. No CR or LF are allowed + except in the final CRLF sequence. + + Request-Line = Method SP Request-URI SP HTTP-Version CRLF + + + + + + + + +Fielding, et. al. Standards Track [Page 34] + +RFC 2068 HTTP/1.1 January 1997 + + +5.1.1 Method + + The Method token indicates the method to be performed on the resource + identified by the Request-URI. The method is case-sensitive. + + Method = "OPTIONS" ; Section 9.2 + | "GET" ; Section 9.3 + | "HEAD" ; Section 9.4 + | "POST" ; Section 9.5 + | "PUT" ; Section 9.6 + | "DELETE" ; Section 9.7 + | "TRACE" ; Section 9.8 + | extension-method + + extension-method = token + + The list of methods allowed by a resource can be specified in an + Allow header field (section 14.7). The return code of the response + always notifies the client whether a method is currently allowed on a + resource, since the set of allowed methods can change dynamically. + Servers SHOULD return the status code 405 (Method Not Allowed) if the + method is known by the server but not allowed for the requested + resource, and 501 (Not Implemented) if the method is unrecognized or + not implemented by the server. The list of methods known by a server + can be listed in a Public response-header field (section 14.35). + + The methods GET and HEAD MUST be supported by all general-purpose + servers. All other methods are optional; however, if the above + methods are implemented, they MUST be implemented with the same + semantics as those specified in section 9. + +5.1.2 Request-URI + + The Request-URI is a Uniform Resource Identifier (section 3.2) and + identifies the resource upon which to apply the request. + + Request-URI = "*" | absoluteURI | abs_path + + The three options for Request-URI are dependent on the nature of the + request. The asterisk "*" means that the request does not apply to a + particular resource, but to the server itself, and is only allowed + when the method used does not necessarily apply to a resource. One + example would be + + OPTIONS * HTTP/1.1 + + The absoluteURI form is required when the request is being made to a + proxy. The proxy is requested to forward the request or service it + + + +Fielding, et. al. Standards Track [Page 35] + +RFC 2068 HTTP/1.1 January 1997 + + + from a valid cache, and return the response. Note that the proxy MAY + forward the request on to another proxy or directly to the server + specified by the absoluteURI. In order to avoid request loops, a + proxy MUST be able to recognize all of its server names, including + any aliases, local variations, and the numeric IP address. An example + Request-Line would be: + + GET http://www.w3.org/pub/WWW/TheProject.html HTTP/1.1 + + To allow for transition to absoluteURIs in all requests in future + versions of HTTP, all HTTP/1.1 servers MUST accept the absoluteURI + form in requests, even though HTTP/1.1 clients will only generate + them in requests to proxies. + + The most common form of Request-URI is that used to identify a + resource on an origin server or gateway. In this case the absolute + path of the URI MUST be transmitted (see section 3.2.1, abs_path) as + the Request-URI, and the network location of the URI (net_loc) MUST + be transmitted in a Host header field. For example, a client wishing + to retrieve the resource above directly from the origin server would + create a TCP connection to port 80 of the host "www.w3.org" and send + the lines: + + GET /pub/WWW/TheProject.html HTTP/1.1 + Host: www.w3.org + + followed by the remainder of the Request. Note that the absolute path + cannot be empty; if none is present in the original URI, it MUST be + given as "/" (the server root). + + If a proxy receives a request without any path in the Request-URI and + the method specified is capable of supporting the asterisk form of + request, then the last proxy on the request chain MUST forward the + request with "*" as the final Request-URI. For example, the request + + OPTIONS http://www.ics.uci.edu:8001 HTTP/1.1 + + would be forwarded by the proxy as + + OPTIONS * HTTP/1.1 + Host: www.ics.uci.edu:8001 + + after connecting to port 8001 of host "www.ics.uci.edu". + + The Request-URI is transmitted in the format specified in section + 3.2.1. The origin server MUST decode the Request-URI in order to + properly interpret the request. Servers SHOULD respond to invalid + Request-URIs with an appropriate status code. + + + +Fielding, et. al. Standards Track [Page 36] + +RFC 2068 HTTP/1.1 January 1997 + + + In requests that they forward, proxies MUST NOT rewrite the + "abs_path" part of a Request-URI in any way except as noted above to + replace a null abs_path with "*", no matter what the proxy does in + its internal implementation. + + Note: The "no rewrite" rule prevents the proxy from changing the + meaning of the request when the origin server is improperly using a + non-reserved URL character for a reserved purpose. Implementers + should be aware that some pre-HTTP/1.1 proxies have been known to + rewrite the Request-URI. + +5.2 The Resource Identified by a Request + + HTTP/1.1 origin servers SHOULD be aware that the exact resource + identified by an Internet request is determined by examining both the + Request-URI and the Host header field. + + An origin server that does not allow resources to differ by the + requested host MAY ignore the Host header field value. (But see + section 19.5.1 for other requirements on Host support in HTTP/1.1.) + + An origin server that does differentiate resources based on the host + requested (sometimes referred to as virtual hosts or vanity + hostnames) MUST use the following rules for determining the requested + resource on an HTTP/1.1 request: + + 1. If Request-URI is an absoluteURI, the host is part of the + Request-URI. Any Host header field value in the request MUST be + ignored. + + 2. If the Request-URI is not an absoluteURI, and the request + includes a Host header field, the host is determined by the Host + header field value. + + 3. If the host as determined by rule 1 or 2 is not a valid host on + the server, the response MUST be a 400 (Bad Request) error + message. + + Recipients of an HTTP/1.0 request that lacks a Host header field MAY + attempt to use heuristics (e.g., examination of the URI path for + something unique to a particular host) in order to determine what + exact resource is being requested. + +5.3 Request Header Fields + + The request-header fields allow the client to pass additional + information about the request, and about the client itself, to the + server. These fields act as request modifiers, with semantics + + + +Fielding, et. al. Standards Track [Page 37] + +RFC 2068 HTTP/1.1 January 1997 + + + equivalent to the parameters on a programming language method + invocation. + + request-header = Accept ; Section 14.1 + | Accept-Charset ; Section 14.2 + | Accept-Encoding ; Section 14.3 + | Accept-Language ; Section 14.4 + | Authorization ; Section 14.8 + | From ; Section 14.22 + | Host ; Section 14.23 + | If-Modified-Since ; Section 14.24 + | If-Match ; Section 14.25 + | If-None-Match ; Section 14.26 + | If-Range ; Section 14.27 + | If-Unmodified-Since ; Section 14.28 + | Max-Forwards ; Section 14.31 + | Proxy-Authorization ; Section 14.34 + | Range ; Section 14.36 + | Referer ; Section 14.37 + | User-Agent ; Section 14.42 + + Request-header field names can be extended reliably only in + combination with a change in the protocol version. However, new or + experimental header fields MAY be given the semantics of request- + header fields if all parties in the communication recognize them to + be request-header fields. Unrecognized header fields are treated as + entity-header fields. + +6 Response + + After receiving and interpreting a request message, a server responds + with an HTTP response message. + + Response = Status-Line ; Section 6.1 + *( general-header ; Section 4.5 + | response-header ; Section 6.2 + | entity-header ) ; Section 7.1 + CRLF + [ message-body ] ; Section 7.2 + +6.1 Status-Line + + The first line of a Response message is the Status-Line, consisting + of the protocol version followed by a numeric status code and its + associated textual phrase, with each element separated by SP + characters. No CR or LF is allowed except in the final CRLF + sequence. + + + + +Fielding, et. al. Standards Track [Page 38] + +RFC 2068 HTTP/1.1 January 1997 + + + Status-Line = HTTP-Version SP Status-Code SP Reason-Phrase CRLF + +6.1.1 Status Code and Reason Phrase + + The Status-Code element is a 3-digit integer result code of the + attempt to understand and satisfy the request. These codes are fully + defined in section 10. The Reason-Phrase is intended to give a short + textual description of the Status-Code. The Status-Code is intended + for use by automata and the Reason-Phrase is intended for the human + user. The client is not required to examine or display the Reason- + Phrase. + + The first digit of the Status-Code defines the class of response. The + last two digits do not have any categorization role. There are 5 + values for the first digit: + + o 1xx: Informational - Request received, continuing process + + o 2xx: Success - The action was successfully received, understood, + and accepted + + o 3xx: Redirection - Further action must be taken in order to + complete the request + + o 4xx: Client Error - The request contains bad syntax or cannot be + fulfilled + + o 5xx: Server Error - The server failed to fulfill an apparently + valid request + + The individual values of the numeric status codes defined for + HTTP/1.1, and an example set of corresponding Reason-Phrase's, are + presented below. The reason phrases listed here are only recommended + -- they may be replaced by local equivalents without affecting the + protocol. + + Status-Code = "100" ; Continue + | "101" ; Switching Protocols + | "200" ; OK + | "201" ; Created + | "202" ; Accepted + | "203" ; Non-Authoritative Information + | "204" ; No Content + | "205" ; Reset Content + | "206" ; Partial Content + | "300" ; Multiple Choices + | "301" ; Moved Permanently + | "302" ; Moved Temporarily + + + +Fielding, et. al. Standards Track [Page 39] + +RFC 2068 HTTP/1.1 January 1997 + + + | "303" ; See Other + | "304" ; Not Modified + | "305" ; Use Proxy + | "400" ; Bad Request + | "401" ; Unauthorized + | "402" ; Payment Required + | "403" ; Forbidden + | "404" ; Not Found + | "405" ; Method Not Allowed + | "406" ; Not Acceptable + | "407" ; Proxy Authentication Required + | "408" ; Request Time-out + | "409" ; Conflict + | "410" ; Gone + | "411" ; Length Required + | "412" ; Precondition Failed + | "413" ; Request Entity Too Large + | "414" ; Request-URI Too Large + | "415" ; Unsupported Media Type + | "500" ; Internal Server Error + | "501" ; Not Implemented + | "502" ; Bad Gateway + | "503" ; Service Unavailable + | "504" ; Gateway Time-out + | "505" ; HTTP Version not supported + | extension-code + + extension-code = 3DIGIT + + Reason-Phrase = *<TEXT, excluding CR, LF> + + HTTP status codes are extensible. HTTP applications are not required + to understand the meaning of all registered status codes, though such + understanding is obviously desirable. However, applications MUST + understand the class of any status code, as indicated by the first + digit, and treat any unrecognized response as being equivalent to the + x00 status code of that class, with the exception that an + unrecognized response MUST NOT be cached. For example, if an + unrecognized status code of 431 is received by the client, it can + safely assume that there was something wrong with its request and + treat the response as if it had received a 400 status code. In such + cases, user agents SHOULD present to the user the entity returned + with the response, since that entity is likely to include human- + readable information which will explain the unusual status. + + + + + + + +Fielding, et. al. Standards Track [Page 40] + +RFC 2068 HTTP/1.1 January 1997 + + +6.2 Response Header Fields + + The response-header fields allow the server to pass additional + information about the response which cannot be placed in the Status- + Line. These header fields give information about the server and about + further access to the resource identified by the Request-URI. + + response-header = Age ; Section 14.6 + | Location ; Section 14.30 + | Proxy-Authenticate ; Section 14.33 + | Public ; Section 14.35 + | Retry-After ; Section 14.38 + | Server ; Section 14.39 + | Vary ; Section 14.43 + | Warning ; Section 14.45 + | WWW-Authenticate ; Section 14.46 + + Response-header field names can be extended reliably only in + combination with a change in the protocol version. However, new or + experimental header fields MAY be given the semantics of response- + header fields if all parties in the communication recognize them to + be response-header fields. Unrecognized header fields are treated as + entity-header fields. + +7 Entity + + Request and Response messages MAY transfer an entity if not otherwise + restricted by the request method or response status code. An entity + consists of entity-header fields and an entity-body, although some + responses will only include the entity-headers. + + In this section, both sender and recipient refer to either the client + or the server, depending on who sends and who receives the entity. + +7.1 Entity Header Fields + + Entity-header fields define optional metainformation about the + entity-body or, if no body is present, about the resource identified + by the request. + + + + + + + + + + + + +Fielding, et. al. Standards Track [Page 41] + +RFC 2068 HTTP/1.1 January 1997 + + + entity-header = Allow ; Section 14.7 + | Content-Base ; Section 14.11 + | Content-Encoding ; Section 14.12 + | Content-Language ; Section 14.13 + | Content-Length ; Section 14.14 + | Content-Location ; Section 14.15 + | Content-MD5 ; Section 14.16 + | Content-Range ; Section 14.17 + | Content-Type ; Section 14.18 + | ETag ; Section 14.20 + | Expires ; Section 14.21 + | Last-Modified ; Section 14.29 + | extension-header + + extension-header = message-header + + The extension-header mechanism allows additional entity-header fields + to be defined without changing the protocol, but these fields cannot + be assumed to be recognizable by the recipient. Unrecognized header + fields SHOULD be ignored by the recipient and forwarded by proxies. + +7.2 Entity Body + + The entity-body (if any) sent with an HTTP request or response is in + a format and encoding defined by the entity-header fields. + + entity-body = *OCTET + + An entity-body is only present in a message when a message-body is + present, as described in section 4.3. The entity-body is obtained + from the message-body by decoding any Transfer-Encoding that may have + been applied to ensure safe and proper transfer of the message. + +7.2.1 Type + + When an entity-body is included with a message, the data type of that + body is determined via the header fields Content-Type and Content- + Encoding. These define a two-layer, ordered encoding model: + + entity-body := Content-Encoding( Content-Type( data ) ) + + Content-Type specifies the media type of the underlying data. + Content-Encoding may be used to indicate any additional content + codings applied to the data, usually for the purpose of data + compression, that are a property of the requested resource. There is + no default encoding. + + + + + +Fielding, et. al. Standards Track [Page 42] + +RFC 2068 HTTP/1.1 January 1997 + + + Any HTTP/1.1 message containing an entity-body SHOULD include a + Content-Type header field defining the media type of that body. If + and only if the media type is not given by a Content-Type field, the + recipient MAY attempt to guess the media type via inspection of its + content and/or the name extension(s) of the URL used to identify the + resource. If the media type remains unknown, the recipient SHOULD + treat it as type "application/octet-stream". + +7.2.2 Length + + The length of an entity-body is the length of the message-body after + any transfer codings have been removed. Section 4.4 defines how the + length of a message-body is determined. + +8 Connections + +8.1 Persistent Connections + +8.1.1 Purpose + + Prior to persistent connections, a separate TCP connection was + established to fetch each URL, increasing the load on HTTP servers + and causing congestion on the Internet. The use of inline images and + other associated data often requires a client to make multiple + requests of the same server in a short amount of time. Analyses of + these performance problems are available [30][27]; analysis and + results from a prototype implementation are in [26]. + + Persistent HTTP connections have a number of advantages: + + o By opening and closing fewer TCP connections, CPU time is saved, + and memory used for TCP protocol control blocks is also saved. + o HTTP requests and responses can be pipelined on a connection. + Pipelining allows a client to make multiple requests without + waiting for each response, allowing a single TCP connection to be + used much more efficiently, with much lower elapsed time. + o Network congestion is reduced by reducing the number of packets + caused by TCP opens, and by allowing TCP sufficient time to + determine the congestion state of the network. + o HTTP can evolve more gracefully; since errors can be reported + without the penalty of closing the TCP connection. Clients using + future versions of HTTP might optimistically try a new feature, but + if communicating with an older server, retry with old semantics + after an error is reported. + + HTTP implementations SHOULD implement persistent connections. + + + + + +Fielding, et. al. Standards Track [Page 43] + +RFC 2068 HTTP/1.1 January 1997 + + +8.1.2 Overall Operation + + A significant difference between HTTP/1.1 and earlier versions of + HTTP is that persistent connections are the default behavior of any + HTTP connection. That is, unless otherwise indicated, the client may + assume that the server will maintain a persistent connection. + + Persistent connections provide a mechanism by which a client and a + server can signal the close of a TCP connection. This signaling takes + place using the Connection header field. Once a close has been + signaled, the client MUST not send any more requests on that + connection. + +8.1.2.1 Negotiation + + An HTTP/1.1 server MAY assume that a HTTP/1.1 client intends to + maintain a persistent connection unless a Connection header including + the connection-token "close" was sent in the request. If the server + chooses to close the connection immediately after sending the + response, it SHOULD send a Connection header including the + connection-token close. + + An HTTP/1.1 client MAY expect a connection to remain open, but would + decide to keep it open based on whether the response from a server + contains a Connection header with the connection-token close. In case + the client does not want to maintain a connection for more than that + request, it SHOULD send a Connection header including the + connection-token close. + + If either the client or the server sends the close token in the + Connection header, that request becomes the last one for the + connection. + + Clients and servers SHOULD NOT assume that a persistent connection is + maintained for HTTP versions less than 1.1 unless it is explicitly + signaled. See section 19.7.1 for more information on backwards + compatibility with HTTP/1.0 clients. + + In order to remain persistent, all messages on the connection must + have a self-defined message length (i.e., one not defined by closure + of the connection), as described in section 4.4. + +8.1.2.2 Pipelining + + A client that supports persistent connections MAY "pipeline" its + requests (i.e., send multiple requests without waiting for each + response). A server MUST send its responses to those requests in the + same order that the requests were received. + + + +Fielding, et. al. Standards Track [Page 44] + +RFC 2068 HTTP/1.1 January 1997 + + + Clients which assume persistent connections and pipeline immediately + after connection establishment SHOULD be prepared to retry their + connection if the first pipelined attempt fails. If a client does + such a retry, it MUST NOT pipeline before it knows the connection is + persistent. Clients MUST also be prepared to resend their requests if + the server closes the connection before sending all of the + corresponding responses. + +8.1.3 Proxy Servers + + It is especially important that proxies correctly implement the + properties of the Connection header field as specified in 14.2.1. + + The proxy server MUST signal persistent connections separately with + its clients and the origin servers (or other proxy servers) that it + connects to. Each persistent connection applies to only one transport + link. + + A proxy server MUST NOT establish a persistent connection with an + HTTP/1.0 client. + +8.1.4 Practical Considerations + + Servers will usually have some time-out value beyond which they will + no longer maintain an inactive connection. Proxy servers might make + this a higher value since it is likely that the client will be making + more connections through the same server. The use of persistent + connections places no requirements on the length of this time-out for + either the client or the server. + + When a client or server wishes to time-out it SHOULD issue a graceful + close on the transport connection. Clients and servers SHOULD both + constantly watch for the other side of the transport close, and + respond to it as appropriate. If a client or server does not detect + the other side's close promptly it could cause unnecessary resource + drain on the network. + + A client, server, or proxy MAY close the transport connection at any + time. For example, a client MAY have started to send a new request at + the same time that the server has decided to close the "idle" + connection. From the server's point of view, the connection is being + closed while it was idle, but from the client's point of view, a + request is in progress. + + This means that clients, servers, and proxies MUST be able to recover + from asynchronous close events. Client software SHOULD reopen the + transport connection and retransmit the aborted request without user + interaction so long as the request method is idempotent (see section + + + +Fielding, et. al. Standards Track [Page 45] + +RFC 2068 HTTP/1.1 January 1997 + + + 9.1.2); other methods MUST NOT be automatically retried, although + user agents MAY offer a human operator the choice of retrying the + request. + + However, this automatic retry SHOULD NOT be repeated if the second + request fails. + + Servers SHOULD always respond to at least one request per connection, + if at all possible. Servers SHOULD NOT close a connection in the + middle of transmitting a response, unless a network or client failure + is suspected. + + Clients that use persistent connections SHOULD limit the number of + simultaneous connections that they maintain to a given server. A + single-user client SHOULD maintain AT MOST 2 connections with any + server or proxy. A proxy SHOULD use up to 2*N connections to another + server or proxy, where N is the number of simultaneously active + users. These guidelines are intended to improve HTTP response times + and avoid congestion of the Internet or other networks. + +8.2 Message Transmission Requirements + +General requirements: + +o HTTP/1.1 servers SHOULD maintain persistent connections and use + TCP's flow control mechanisms to resolve temporary overloads, + rather than terminating connections with the expectation that + clients will retry. The latter technique can exacerbate network + congestion. + +o An HTTP/1.1 (or later) client sending a message-body SHOULD monitor + the network connection for an error status while it is transmitting + the request. If the client sees an error status, it SHOULD + immediately cease transmitting the body. If the body is being sent + using a "chunked" encoding (section 3.6), a zero length chunk and + empty footer MAY be used to prematurely mark the end of the + message. If the body was preceded by a Content-Length header, the + client MUST close the connection. + +o An HTTP/1.1 (or later) client MUST be prepared to accept a 100 + (Continue) status followed by a regular response. + +o An HTTP/1.1 (or later) server that receives a request from a + HTTP/1.0 (or earlier) client MUST NOT transmit the 100 (continue) + response; it SHOULD either wait for the request to be completed + normally (thus avoiding an interrupted request) or close the + connection prematurely. + + + + +Fielding, et. al. Standards Track [Page 46] + +RFC 2068 HTTP/1.1 January 1997 + + + Upon receiving a method subject to these requirements from an + HTTP/1.1 (or later) client, an HTTP/1.1 (or later) server MUST either + respond with 100 (Continue) status and continue to read from the + input stream, or respond with an error status. If it responds with an + error status, it MAY close the transport (TCP) connection or it MAY + continue to read and discard the rest of the request. It MUST NOT + perform the requested method if it returns an error status. + + Clients SHOULD remember the version number of at least the most + recently used server; if an HTTP/1.1 client has seen an HTTP/1.1 or + later response from the server, and it sees the connection close + before receiving any status from the server, the client SHOULD retry + the request without user interaction so long as the request method is + idempotent (see section 9.1.2); other methods MUST NOT be + automatically retried, although user agents MAY offer a human + operator the choice of retrying the request.. If the client does + retry the request, the client + + o MUST first send the request header fields, and then + + o MUST wait for the server to respond with either a 100 (Continue) + response, in which case the client should continue, or with an + error status. + + If an HTTP/1.1 client has not seen an HTTP/1.1 or later response from + the server, it should assume that the server implements HTTP/1.0 or + older and will not use the 100 (Continue) response. If in this case + the client sees the connection close before receiving any status from + the server, the client SHOULD retry the request. If the client does + retry the request to this HTTP/1.0 server, it should use the + following "binary exponential backoff" algorithm to be assured of + obtaining a reliable response: + + 1. Initiate a new connection to the server + + 2. Transmit the request-headers + + 3. Initialize a variable R to the estimated round-trip time to the + server (e.g., based on the time it took to establish the + connection), or to a constant value of 5 seconds if the round-trip + time is not available. + + 4. Compute T = R * (2**N), where N is the number of previous retries + of this request. + + 5. Wait either for an error response from the server, or for T seconds + (whichever comes first) + + + + +Fielding, et. al. Standards Track [Page 47] + +RFC 2068 HTTP/1.1 January 1997 + + + 6. If no error response is received, after T seconds transmit the body + of the request. + + 7. If client sees that the connection is closed prematurely, repeat + from step 1 until the request is accepted, an error response is + received, or the user becomes impatient and terminates the retry + process. + + No matter what the server version, if an error status is received, + the client + + o MUST NOT continue and + + o MUST close the connection if it has not completed sending the + message. + + An HTTP/1.1 (or later) client that sees the connection close after + receiving a 100 (Continue) but before receiving any other status + SHOULD retry the request, and need not wait for 100 (Continue) + response (but MAY do so if this simplifies the implementation). + +9 Method Definitions + + The set of common methods for HTTP/1.1 is defined below. Although + this set can be expanded, additional methods cannot be assumed to + share the same semantics for separately extended clients and servers. + + The Host request-header field (section 14.23) MUST accompany all + HTTP/1.1 requests. + +9.1 Safe and Idempotent Methods + +9.1.1 Safe Methods + + Implementers should be aware that the software represents the user in + their interactions over the Internet, and should be careful to allow + the user to be aware of any actions they may take which may have an + unexpected significance to themselves or others. + + In particular, the convention has been established that the GET and + HEAD methods should never have the significance of taking an action + other than retrieval. These methods should be considered "safe." This + allows user agents to represent other methods, such as POST, PUT and + DELETE, in a special way, so that the user is made aware of the fact + that a possibly unsafe action is being requested. + + Naturally, it is not possible to ensure that the server does not + generate side-effects as a result of performing a GET request; in + + + +Fielding, et. al. Standards Track [Page 48] + +RFC 2068 HTTP/1.1 January 1997 + + + fact, some dynamic resources consider that a feature. The important + distinction here is that the user did not request the side-effects, + so therefore cannot be held accountable for them. + +9.1.2 Idempotent Methods + + Methods may also have the property of "idempotence" in that (aside + from error or expiration issues) the side-effects of N > 0 identical + requests is the same as for a single request. The methods GET, HEAD, + PUT and DELETE share this property. + +9.2 OPTIONS + + The OPTIONS method represents a request for information about the + communication options available on the request/response chain + identified by the Request-URI. This method allows the client to + determine the options and/or requirements associated with a resource, + or the capabilities of a server, without implying a resource action + or initiating a resource retrieval. + + Unless the server's response is an error, the response MUST NOT + include entity information other than what can be considered as + communication options (e.g., Allow is appropriate, but Content-Type + is not). Responses to this method are not cachable. + + If the Request-URI is an asterisk ("*"), the OPTIONS request is + intended to apply to the server as a whole. A 200 response SHOULD + include any header fields which indicate optional features + implemented by the server (e.g., Public), including any extensions + not defined by this specification, in addition to any applicable + general or response-header fields. As described in section 5.1.2, an + "OPTIONS *" request can be applied through a proxy by specifying the + destination server in the Request-URI without any path information. + + If the Request-URI is not an asterisk, the OPTIONS request applies + only to the options that are available when communicating with that + resource. A 200 response SHOULD include any header fields which + indicate optional features implemented by the server and applicable + to that resource (e.g., Allow), including any extensions not defined + by this specification, in addition to any applicable general or + response-header fields. If the OPTIONS request passes through a + proxy, the proxy MUST edit the response to exclude those options + which apply to a proxy's capabilities and which are known to be + unavailable through that proxy. + + + + + + + +Fielding, et. al. Standards Track [Page 49] + +RFC 2068 HTTP/1.1 January 1997 + + +9.3 GET + + The GET method means retrieve whatever information (in the form of an + entity) is identified by the Request-URI. If the Request-URI refers + to a data-producing process, it is the produced data which shall be + returned as the entity in the response and not the source text of the + process, unless that text happens to be the output of the process. + + The semantics of the GET method change to a "conditional GET" if the + request message includes an If-Modified-Since, If-Unmodified-Since, + If-Match, If-None-Match, or If-Range header field. A conditional GET + method requests that the entity be transferred only under the + circumstances described by the conditional header field(s). The + conditional GET method is intended to reduce unnecessary network + usage by allowing cached entities to be refreshed without requiring + multiple requests or transferring data already held by the client. + + The semantics of the GET method change to a "partial GET" if the + request message includes a Range header field. A partial GET requests + that only part of the entity be transferred, as described in section + 14.36. The partial GET method is intended to reduce unnecessary + network usage by allowing partially-retrieved entities to be + completed without transferring data already held by the client. + + The response to a GET request is cachable if and only if it meets the + requirements for HTTP caching described in section 13. + +9.4 HEAD + + The HEAD method is identical to GET except that the server MUST NOT + return a message-body in the response. The metainformation contained + in the HTTP headers in response to a HEAD request SHOULD be identical + to the information sent in response to a GET request. This method can + be used for obtaining metainformation about the entity implied by the + request without transferring the entity-body itself. This method is + often used for testing hypertext links for validity, accessibility, + and recent modification. + + The response to a HEAD request may be cachable in the sense that the + information contained in the response may be used to update a + previously cached entity from that resource. If the new field values + indicate that the cached entity differs from the current entity (as + would be indicated by a change in Content-Length, Content-MD5, ETag + or Last-Modified), then the cache MUST treat the cache entry as + stale. + + + + + + +Fielding, et. al. Standards Track [Page 50] + +RFC 2068 HTTP/1.1 January 1997 + + +9.5 POST + + The POST method is used to request that the destination server accept + the entity enclosed in the request as a new subordinate of the + resource identified by the Request-URI in the Request-Line. POST is + designed to allow a uniform method to cover the following functions: + + o Annotation of existing resources; + + o Posting a message to a bulletin board, newsgroup, mailing list, + or similar group of articles; + + o Providing a block of data, such as the result of submitting a + form, to a data-handling process; + + o Extending a database through an append operation. + + The actual function performed by the POST method is determined by the + server and is usually dependent on the Request-URI. The posted entity + is subordinate to that URI in the same way that a file is subordinate + to a directory containing it, a news article is subordinate to a + newsgroup to which it is posted, or a record is subordinate to a + database. + + The action performed by the POST method might not result in a + resource that can be identified by a URI. In this case, either 200 + (OK) or 204 (No Content) is the appropriate response status, + depending on whether or not the response includes an entity that + describes the result. + + If a resource has been created on the origin server, the response + SHOULD be 201 (Created) and contain an entity which describes the + status of the request and refers to the new resource, and a Location + header (see section 14.30). + + Responses to this method are not cachable, unless the response + includes appropriate Cache-Control or Expires header fields. However, + the 303 (See Other) response can be used to direct the user agent to + retrieve a cachable resource. + + POST requests must obey the message transmission requirements set out + in section 8.2. + + + + + + + + + +Fielding, et. al. Standards Track [Page 51] + +RFC 2068 HTTP/1.1 January 1997 + + +9.6 PUT + + The PUT method requests that the enclosed entity be stored under the + supplied Request-URI. If the Request-URI refers to an already + existing resource, the enclosed entity SHOULD be considered as a + modified version of the one residing on the origin server. If the + Request-URI does not point to an existing resource, and that URI is + capable of being defined as a new resource by the requesting user + agent, the origin server can create the resource with that URI. If a + new resource is created, the origin server MUST inform the user agent + via the 201 (Created) response. If an existing resource is modified, + either the 200 (OK) or 204 (No Content) response codes SHOULD be sent + to indicate successful completion of the request. If the resource + could not be created or modified with the Request-URI, an appropriate + error response SHOULD be given that reflects the nature of the + problem. The recipient of the entity MUST NOT ignore any Content-* + (e.g. Content-Range) headers that it does not understand or implement + and MUST return a 501 (Not Implemented) response in such cases. + + If the request passes through a cache and the Request-URI identifies + one or more currently cached entities, those entries should be + treated as stale. Responses to this method are not cachable. + + The fundamental difference between the POST and PUT requests is + reflected in the different meaning of the Request-URI. The URI in a + POST request identifies the resource that will handle the enclosed + entity. That resource may be a data-accepting process, a gateway to + some other protocol, or a separate entity that accepts annotations. + In contrast, the URI in a PUT request identifies the entity enclosed + with the request -- the user agent knows what URI is intended and the + server MUST NOT attempt to apply the request to some other resource. + If the server desires that the request be applied to a different URI, + it MUST send a 301 (Moved Permanently) response; the user agent MAY + then make its own decision regarding whether or not to redirect the + request. + + A single resource MAY be identified by many different URIs. For + example, an article may have a URI for identifying "the current + version" which is separate from the URI identifying each particular + version. In this case, a PUT request on a general URI may result in + several other URIs being defined by the origin server. + + HTTP/1.1 does not define how a PUT method affects the state of an + origin server. + + PUT requests must obey the message transmission requirements set out + in section 8.2. + + + + +Fielding, et. al. Standards Track [Page 52] + +RFC 2068 HTTP/1.1 January 1997 + + +9.7 DELETE + + The DELETE method requests that the origin server delete the resource + identified by the Request-URI. This method MAY be overridden by human + intervention (or other means) on the origin server. The client cannot + be guaranteed that the operation has been carried out, even if the + status code returned from the origin server indicates that the action + has been completed successfully. However, the server SHOULD not + indicate success unless, at the time the response is given, it + intends to delete the resource or move it to an inaccessible + location. + + A successful response SHOULD be 200 (OK) if the response includes an + entity describing the status, 202 (Accepted) if the action has not + yet been enacted, or 204 (No Content) if the response is OK but does + not include an entity. + + If the request passes through a cache and the Request-URI identifies + one or more currently cached entities, those entries should be + treated as stale. Responses to this method are not cachable. + +9.8 TRACE + + The TRACE method is used to invoke a remote, application-layer loop- + back of the request message. The final recipient of the request + SHOULD reflect the message received back to the client as the + entity-body of a 200 (OK) response. The final recipient is either the + origin server or the first proxy or gateway to receive a Max-Forwards + value of zero (0) in the request (see section 14.31). A TRACE request + MUST NOT include an entity. + + TRACE allows the client to see what is being received at the other + end of the request chain and use that data for testing or diagnostic + information. The value of the Via header field (section 14.44) is of + particular interest, since it acts as a trace of the request chain. + Use of the Max-Forwards header field allows the client to limit the + length of the request chain, which is useful for testing a chain of + proxies forwarding messages in an infinite loop. + + If successful, the response SHOULD contain the entire request message + in the entity-body, with a Content-Type of "message/http". Responses + to this method MUST NOT be cached. + +10 Status Code Definitions + + Each Status-Code is described below, including a description of which + method(s) it can follow and any metainformation required in the + response. + + + +Fielding, et. al. Standards Track [Page 53] + +RFC 2068 HTTP/1.1 January 1997 + + +10.1 Informational 1xx + + This class of status code indicates a provisional response, + consisting only of the Status-Line and optional headers, and is + terminated by an empty line. Since HTTP/1.0 did not define any 1xx + status codes, servers MUST NOT send a 1xx response to an HTTP/1.0 + client except under experimental conditions. + +10.1.1 100 Continue + + The client may continue with its request. This interim response is + used to inform the client that the initial part of the request has + been received and has not yet been rejected by the server. The client + SHOULD continue by sending the remainder of the request or, if the + request has already been completed, ignore this response. The server + MUST send a final response after the request has been completed. + +10.1.2 101 Switching Protocols + + The server understands and is willing to comply with the client's + request, via the Upgrade message header field (section 14.41), for a + change in the application protocol being used on this connection. The + server will switch protocols to those defined by the response's + Upgrade header field immediately after the empty line which + terminates the 101 response. + + The protocol should only be switched when it is advantageous to do + so. For example, switching to a newer version of HTTP is + advantageous over older versions, and switching to a real-time, + synchronous protocol may be advantageous when delivering resources + that use such features. + +10.2 Successful 2xx + + This class of status code indicates that the client's request was + successfully received, understood, and accepted. + +10.2.1 200 OK + + The request has succeeded. The information returned with the response + is dependent on the method used in the request, for example: + + GET an entity corresponding to the requested resource is sent in the + response; + + HEAD the entity-header fields corresponding to the requested resource + are sent in the response without any message-body; + + + + +Fielding, et. al. Standards Track [Page 54] + +RFC 2068 HTTP/1.1 January 1997 + + + POST an entity describing or containing the result of the action; + + TRACE an entity containing the request message as received by the end + server. + +10.2.2 201 Created + + The request has been fulfilled and resulted in a new resource being + created. The newly created resource can be referenced by the URI(s) + returned in the entity of the response, with the most specific URL + for the resource given by a Location header field. The origin server + MUST create the resource before returning the 201 status code. If the + action cannot be carried out immediately, the server should respond + with 202 (Accepted) response instead. + +10.2.3 202 Accepted + + The request has been accepted for processing, but the processing has + not been completed. The request MAY or MAY NOT eventually be acted + upon, as it MAY be disallowed when processing actually takes place. + There is no facility for re-sending a status code from an + asynchronous operation such as this. + + The 202 response is intentionally non-committal. Its purpose is to + allow a server to accept a request for some other process (perhaps a + batch-oriented process that is only run once per day) without + requiring that the user agent's connection to the server persist + until the process is completed. The entity returned with this + response SHOULD include an indication of the request's current status + and either a pointer to a status monitor or some estimate of when the + user can expect the request to be fulfilled. + +10.2.4 203 Non-Authoritative Information + + The returned metainformation in the entity-header is not the + definitive set as available from the origin server, but is gathered + from a local or a third-party copy. The set presented MAY be a subset + or superset of the original version. For example, including local + annotation information about the resource MAY result in a superset of + the metainformation known by the origin server. Use of this response + code is not required and is only appropriate when the response would + otherwise be 200 (OK). + +10.2.5 204 No Content + + The server has fulfilled the request but there is no new information + to send back. If the client is a user agent, it SHOULD NOT change its + document view from that which caused the request to be sent. This + + + +Fielding, et. al. Standards Track [Page 55] + +RFC 2068 HTTP/1.1 January 1997 + + + response is primarily intended to allow input for actions to take + place without causing a change to the user agent's active document + view. The response MAY include new metainformation in the form of + entity-headers, which SHOULD apply to the document currently in the + user agent's active view. + + The 204 response MUST NOT include a message-body, and thus is always + terminated by the first empty line after the header fields. + +10.2.6 205 Reset Content + + The server has fulfilled the request and the user agent SHOULD reset + the document view which caused the request to be sent. This response + is primarily intended to allow input for actions to take place via + user input, followed by a clearing of the form in which the input is + given so that the user can easily initiate another input action. The + response MUST NOT include an entity. + +10.2.7 206 Partial Content + + The server has fulfilled the partial GET request for the resource. + The request must have included a Range header field (section 14.36) + indicating the desired range. The response MUST include either a + Content-Range header field (section 14.17) indicating the range + included with this response, or a multipart/byteranges Content-Type + including Content-Range fields for each part. If multipart/byteranges + is not used, the Content-Length header field in the response MUST + match the actual number of OCTETs transmitted in the message-body. + + A cache that does not support the Range and Content-Range headers + MUST NOT cache 206 (Partial) responses. + +10.3 Redirection 3xx + + This class of status code indicates that further action needs to be + taken by the user agent in order to fulfill the request. The action + required MAY be carried out by the user agent without interaction + with the user if and only if the method used in the second request is + GET or HEAD. A user agent SHOULD NOT automatically redirect a request + more than 5 times, since such redirections usually indicate an + infinite loop. + + + + + + + + + + +Fielding, et. al. Standards Track [Page 56] + +RFC 2068 HTTP/1.1 January 1997 + + +10.3.1 300 Multiple Choices + + The requested resource corresponds to any one of a set of + representations, each with its own specific location, and agent- + driven negotiation information (section 12) is being provided so that + the user (or user agent) can select a preferred representation and + redirect its request to that location. + + Unless it was a HEAD request, the response SHOULD include an entity + containing a list of resource characteristics and location(s) from + which the user or user agent can choose the one most appropriate. The + entity format is specified by the media type given in the Content- + Type header field. Depending upon the format and the capabilities of + the user agent, selection of the most appropriate choice may be + performed automatically. However, this specification does not define + any standard for such automatic selection. + + If the server has a preferred choice of representation, it SHOULD + include the specific URL for that representation in the Location + field; user agents MAY use the Location field value for automatic + redirection. This response is cachable unless indicated otherwise. + +10.3.2 301 Moved Permanently + + The requested resource has been assigned a new permanent URI and any + future references to this resource SHOULD be done using one of the + returned URIs. Clients with link editing capabilities SHOULD + automatically re-link references to the Request-URI to one or more of + the new references returned by the server, where possible. This + response is cachable unless indicated otherwise. + + If the new URI is a location, its URL SHOULD be given by the Location + field in the response. Unless the request method was HEAD, the entity + of the response SHOULD contain a short hypertext note with a + hyperlink to the new URI(s). + + If the 301 status code is received in response to a request other + than GET or HEAD, the user agent MUST NOT automatically redirect the + request unless it can be confirmed by the user, since this might + change the conditions under which the request was issued. + + Note: When automatically redirecting a POST request after receiving + a 301 status code, some existing HTTP/1.0 user agents will + erroneously change it into a GET request. + + + + + + + +Fielding, et. al. Standards Track [Page 57] + +RFC 2068 HTTP/1.1 January 1997 + + +10.3.3 302 Moved Temporarily + + The requested resource resides temporarily under a different URI. + Since the redirection may be altered on occasion, the client SHOULD + continue to use the Request-URI for future requests. This response is + only cachable if indicated by a Cache-Control or Expires header + field. + + If the new URI is a location, its URL SHOULD be given by the Location + field in the response. Unless the request method was HEAD, the entity + of the response SHOULD contain a short hypertext note with a + hyperlink to the new URI(s). + + If the 302 status code is received in response to a request other + than GET or HEAD, the user agent MUST NOT automatically redirect the + request unless it can be confirmed by the user, since this might + change the conditions under which the request was issued. + + Note: When automatically redirecting a POST request after receiving + a 302 status code, some existing HTTP/1.0 user agents will + erroneously change it into a GET request. + +10.3.4 303 See Other + + The response to the request can be found under a different URI and + SHOULD be retrieved using a GET method on that resource. This method + exists primarily to allow the output of a POST-activated script to + redirect the user agent to a selected resource. The new URI is not a + substitute reference for the originally requested resource. The 303 + response is not cachable, but the response to the second (redirected) + request MAY be cachable. + + If the new URI is a location, its URL SHOULD be given by the Location + field in the response. Unless the request method was HEAD, the entity + of the response SHOULD contain a short hypertext note with a + hyperlink to the new URI(s). + +10.3.5 304 Not Modified + + If the client has performed a conditional GET request and access is + allowed, but the document has not been modified, the server SHOULD + respond with this status code. The response MUST NOT contain a + message-body. + + + + + + + + +Fielding, et. al. Standards Track [Page 58] + +RFC 2068 HTTP/1.1 January 1997 + + + The response MUST include the following header fields: + + o Date + + o ETag and/or Content-Location, if the header would have been sent in + a 200 response to the same request + + o Expires, Cache-Control, and/or Vary, if the field-value might + differ from that sent in any previous response for the same variant + + If the conditional GET used a strong cache validator (see section + 13.3.3), the response SHOULD NOT include other entity-headers. + Otherwise (i.e., the conditional GET used a weak validator), the + response MUST NOT include other entity-headers; this prevents + inconsistencies between cached entity-bodies and updated headers. + + If a 304 response indicates an entity not currently cached, then the + cache MUST disregard the response and repeat the request without the + conditional. + + If a cache uses a received 304 response to update a cache entry, the + cache MUST update the entry to reflect any new field values given in + the response. + + The 304 response MUST NOT include a message-body, and thus is always + terminated by the first empty line after the header fields. + +10.3.6 305 Use Proxy + + The requested resource MUST be accessed through the proxy given by + the Location field. The Location field gives the URL of the proxy. + The recipient is expected to repeat the request via the proxy. + +10.4 Client Error 4xx + + The 4xx class of status code is intended for cases in which the + client seems to have erred. Except when responding to a HEAD request, + the server SHOULD include an entity containing an explanation of the + error situation, and whether it is a temporary or permanent + condition. These status codes are applicable to any request method. + User agents SHOULD display any included entity to the user. + + Note: If the client is sending data, a server implementation using + TCP should be careful to ensure that the client acknowledges + receipt of the packet(s) containing the response, before the server + closes the input connection. If the client continues sending data + to the server after the close, the server's TCP stack will send a + reset packet to the client, which may erase the client's + + + +Fielding, et. al. Standards Track [Page 59] + +RFC 2068 HTTP/1.1 January 1997 + + + unacknowledged input buffers before they can be read and + interpreted by the HTTP application. + +10.4.1 400 Bad Request + + The request could not be understood by the server due to malformed + syntax. The client SHOULD NOT repeat the request without + modifications. + +10.4.2 401 Unauthorized + + The request requires user authentication. The response MUST include a + WWW-Authenticate header field (section 14.46) containing a challenge + applicable to the requested resource. The client MAY repeat the + request with a suitable Authorization header field (section 14.8). If + the request already included Authorization credentials, then the 401 + response indicates that authorization has been refused for those + credentials. If the 401 response contains the same challenge as the + prior response, and the user agent has already attempted + authentication at least once, then the user SHOULD be presented the + entity that was given in the response, since that entity MAY include + relevant diagnostic information. HTTP access authentication is + explained in section 11. + +10.4.3 402 Payment Required + + This code is reserved for future use. + +10.4.4 403 Forbidden + + The server understood the request, but is refusing to fulfill it. + Authorization will not help and the request SHOULD NOT be repeated. + If the request method was not HEAD and the server wishes to make + public why the request has not been fulfilled, it SHOULD describe the + reason for the refusal in the entity. This status code is commonly + used when the server does not wish to reveal exactly why the request + has been refused, or when no other response is applicable. + +10.4.5 404 Not Found + + The server has not found anything matching the Request-URI. No + indication is given of whether the condition is temporary or + permanent. + + + + + + + + +Fielding, et. al. Standards Track [Page 60] + +RFC 2068 HTTP/1.1 January 1997 + + + If the server does not wish to make this information available to the + client, the status code 403 (Forbidden) can be used instead. The 410 + (Gone) status code SHOULD be used if the server knows, through some + internally configurable mechanism, that an old resource is + permanently unavailable and has no forwarding address. + +10.4.6 405 Method Not Allowed + + The method specified in the Request-Line is not allowed for the + resource identified by the Request-URI. The response MUST include an + Allow header containing a list of valid methods for the requested + resource. + +10.4.7 406 Not Acceptable + + The resource identified by the request is only capable of generating + response entities which have content characteristics not acceptable + according to the accept headers sent in the request. + + Unless it was a HEAD request, the response SHOULD include an entity + containing a list of available entity characteristics and location(s) + from which the user or user agent can choose the one most + appropriate. The entity format is specified by the media type given + in the Content-Type header field. Depending upon the format and the + capabilities of the user agent, selection of the most appropriate + choice may be performed automatically. However, this specification + does not define any standard for such automatic selection. + + Note: HTTP/1.1 servers are allowed to return responses which are + not acceptable according to the accept headers sent in the request. + In some cases, this may even be preferable to sending a 406 + response. User agents are encouraged to inspect the headers of an + incoming response to determine if it is acceptable. If the response + could be unacceptable, a user agent SHOULD temporarily stop receipt + of more data and query the user for a decision on further actions. + +10.4.8 407 Proxy Authentication Required + + This code is similar to 401 (Unauthorized), but indicates that the + client MUST first authenticate itself with the proxy. The proxy MUST + return a Proxy-Authenticate header field (section 14.33) containing a + challenge applicable to the proxy for the requested resource. The + client MAY repeat the request with a suitable Proxy-Authorization + header field (section 14.34). HTTP access authentication is explained + in section 11. + + + + + + +Fielding, et. al. Standards Track [Page 61] + +RFC 2068 HTTP/1.1 January 1997 + + +10.4.9 408 Request Timeout + + The client did not produce a request within the time that the server + was prepared to wait. The client MAY repeat the request without + modifications at any later time. + +10.4.10 409 Conflict + + The request could not be completed due to a conflict with the current + state of the resource. This code is only allowed in situations where + it is expected that the user might be able to resolve the conflict + and resubmit the request. The response body SHOULD include enough + information for the user to recognize the source of the conflict. + Ideally, the response entity would include enough information for the + user or user agent to fix the problem; however, that may not be + possible and is not required. + + Conflicts are most likely to occur in response to a PUT request. If + versioning is being used and the entity being PUT includes changes to + a resource which conflict with those made by an earlier (third-party) + request, the server MAY use the 409 response to indicate that it + can't complete the request. In this case, the response entity SHOULD + contain a list of the differences between the two versions in a + format defined by the response Content-Type. + +10.4.11 410 Gone + + The requested resource is no longer available at the server and no + forwarding address is known. This condition SHOULD be considered + permanent. Clients with link editing capabilities SHOULD delete + references to the Request-URI after user approval. If the server does + not know, or has no facility to determine, whether or not the + condition is permanent, the status code 404 (Not Found) SHOULD be + used instead. This response is cachable unless indicated otherwise. + + The 410 response is primarily intended to assist the task of web + maintenance by notifying the recipient that the resource is + intentionally unavailable and that the server owners desire that + remote links to that resource be removed. Such an event is common for + limited-time, promotional services and for resources belonging to + individuals no longer working at the server's site. It is not + necessary to mark all permanently unavailable resources as "gone" or + to keep the mark for any length of time -- that is left to the + discretion of the server owner. + + + + + + + +Fielding, et. al. Standards Track [Page 62] + +RFC 2068 HTTP/1.1 January 1997 + + +10.4.12 411 Length Required + + The server refuses to accept the request without a defined Content- + Length. The client MAY repeat the request if it adds a valid + Content-Length header field containing the length of the message-body + in the request message. + +10.4.13 412 Precondition Failed + + The precondition given in one or more of the request-header fields + evaluated to false when it was tested on the server. This response + code allows the client to place preconditions on the current resource + metainformation (header field data) and thus prevent the requested + method from being applied to a resource other than the one intended. + +10.4.14 413 Request Entity Too Large + + The server is refusing to process a request because the request + entity is larger than the server is willing or able to process. The + server may close the connection to prevent the client from continuing + the request. + + If the condition is temporary, the server SHOULD include a Retry- + After header field to indicate that it is temporary and after what + time the client may try again. + +10.4.15 414 Request-URI Too Long + + The server is refusing to service the request because the Request-URI + is longer than the server is willing to interpret. This rare + condition is only likely to occur when a client has improperly + converted a POST request to a GET request with long query + information, when the client has descended into a URL "black hole" of + redirection (e.g., a redirected URL prefix that points to a suffix of + itself), or when the server is under attack by a client attempting to + exploit security holes present in some servers using fixed-length + buffers for reading or manipulating the Request-URI. + +10.4.16 415 Unsupported Media Type + + The server is refusing to service the request because the entity of + the request is in a format not supported by the requested resource + for the requested method. + + + + + + + + +Fielding, et. al. Standards Track [Page 63] + +RFC 2068 HTTP/1.1 January 1997 + + +10.5 Server Error 5xx + + Response status codes beginning with the digit "5" indicate cases in + which the server is aware that it has erred or is incapable of + performing the request. Except when responding to a HEAD request, the + server SHOULD include an entity containing an explanation of the + error situation, and whether it is a temporary or permanent + condition. User agents SHOULD display any included entity to the + user. These response codes are applicable to any request method. + +10.5.1 500 Internal Server Error + + The server encountered an unexpected condition which prevented it + from fulfilling the request. + +10.5.2 501 Not Implemented + + The server does not support the functionality required to fulfill the + request. This is the appropriate response when the server does not + recognize the request method and is not capable of supporting it for + any resource. + +10.5.3 502 Bad Gateway + + The server, while acting as a gateway or proxy, received an invalid + response from the upstream server it accessed in attempting to + fulfill the request. + +10.5.4 503 Service Unavailable + + The server is currently unable to handle the request due to a + temporary overloading or maintenance of the server. The implication + is that this is a temporary condition which will be alleviated after + some delay. If known, the length of the delay may be indicated in a + Retry-After header. If no Retry-After is given, the client SHOULD + handle the response as it would for a 500 response. + + Note: The existence of the 503 status code does not imply that a + server must use it when becoming overloaded. Some servers may wish + to simply refuse the connection. + +10.5.5 504 Gateway Timeout + + The server, while acting as a gateway or proxy, did not receive a + timely response from the upstream server it accessed in attempting to + complete the request. + + + + + +Fielding, et. al. Standards Track [Page 64] + +RFC 2068 HTTP/1.1 January 1997 + + +10.5.6 505 HTTP Version Not Supported + + The server does not support, or refuses to support, the HTTP protocol + version that was used in the request message. The server is + indicating that it is unable or unwilling to complete the request + using the same major version as the client, as described in section + 3.1, other than with this error message. The response SHOULD contain + an entity describing why that version is not supported and what other + protocols are supported by that server. + +11 Access Authentication + + HTTP provides a simple challenge-response authentication mechanism + which MAY be used by a server to challenge a client request and by a + client to provide authentication information. It uses an extensible, + case-insensitive token to identify the authentication scheme, + followed by a comma-separated list of attribute-value pairs which + carry the parameters necessary for achieving authentication via that + scheme. + + auth-scheme = token + + auth-param = token "=" quoted-string + + The 401 (Unauthorized) response message is used by an origin server + to challenge the authorization of a user agent. This response MUST + include a WWW-Authenticate header field containing at least one + challenge applicable to the requested resource. + + challenge = auth-scheme 1*SP realm *( "," auth-param ) + + realm = "realm" "=" realm-value + realm-value = quoted-string + + The realm attribute (case-insensitive) is required for all + authentication schemes which issue a challenge. The realm value + (case-sensitive), in combination with the canonical root URL (see + section 5.1.2) of the server being accessed, defines the protection + space. These realms allow the protected resources on a server to be + partitioned into a set of protection spaces, each with its own + authentication scheme and/or authorization database. The realm value + is a string, generally assigned by the origin server, which may have + additional semantics specific to the authentication scheme. + + A user agent that wishes to authenticate itself with a server-- + usually, but not necessarily, after receiving a 401 or 411 response- + -MAY do so by including an Authorization header field with the + request. The Authorization field value consists of credentials + + + +Fielding, et. al. Standards Track [Page 65] + +RFC 2068 HTTP/1.1 January 1997 + + + containing the authentication information of the user agent for the + realm of the resource being requested. + + credentials = basic-credentials + | auth-scheme #auth-param + + The domain over which credentials can be automatically applied by a + user agent is determined by the protection space. If a prior request + has been authorized, the same credentials MAY be reused for all other + requests within that protection space for a period of time determined + by the authentication scheme, parameters, and/or user preference. + Unless otherwise defined by the authentication scheme, a single + protection space cannot extend outside the scope of its server. + + If the server does not wish to accept the credentials sent with a + request, it SHOULD return a 401 (Unauthorized) response. The response + MUST include a WWW-Authenticate header field containing the (possibly + new) challenge applicable to the requested resource and an entity + explaining the refusal. + + The HTTP protocol does not restrict applications to this simple + challenge-response mechanism for access authentication. Additional + mechanisms MAY be used, such as encryption at the transport level or + via message encapsulation, and with additional header fields + specifying authentication information. However, these additional + mechanisms are not defined by this specification. + + Proxies MUST be completely transparent regarding user agent + authentication. That is, they MUST forward the WWW-Authenticate and + Authorization headers untouched, and follow the rules found in + section 14.8. + + HTTP/1.1 allows a client to pass authentication information to and + from a proxy via the Proxy-Authenticate and Proxy-Authorization + headers. + +11.1 Basic Authentication Scheme + + The "basic" authentication scheme is based on the model that the user + agent must authenticate itself with a user-ID and a password for each + realm. The realm value should be considered an opaque string which + can only be compared for equality with other realms on that server. + The server will service the request only if it can validate the + user-ID and password for the protection space of the Request-URI. + There are no optional authentication parameters. + + + + + + +Fielding, et. al. Standards Track [Page 66] + +RFC 2068 HTTP/1.1 January 1997 + + + Upon receipt of an unauthorized request for a URI within the + protection space, the server MAY respond with a challenge like the + following: + + WWW-Authenticate: Basic realm="WallyWorld" + + where "WallyWorld" is the string assigned by the server to identify + the protection space of the Request-URI. + + To receive authorization, the client sends the userid and password, + separated by a single colon (":") character, within a base64 encoded + string in the credentials. + + basic-credentials = "Basic" SP basic-cookie + + basic-cookie = <base64 [7] encoding of user-pass, + except not limited to 76 char/line> + + user-pass = userid ":" password + + userid = *<TEXT excluding ":"> + + password = *TEXT + + Userids might be case sensitive. + + If the user agent wishes to send the userid "Aladdin" and password + "open sesame", it would use the following header field: + + Authorization: Basic QWxhZGRpbjpvcGVuIHNlc2FtZQ== + + See section 15 for security considerations associated with Basic + authentication. + +11.2 Digest Authentication Scheme + + A digest authentication for HTTP is specified in RFC 2069 [32]. + +12 Content Negotiation + + Most HTTP responses include an entity which contains information for + interpretation by a human user. Naturally, it is desirable to supply + the user with the "best available" entity corresponding to the + request. Unfortunately for servers and caches, not all users have + the same preferences for what is "best," and not all user agents are + equally capable of rendering all entity types. For that reason, HTTP + has provisions for several mechanisms for "content negotiation" -- + the process of selecting the best representation for a given response + + + +Fielding, et. al. Standards Track [Page 67] + +RFC 2068 HTTP/1.1 January 1997 + + + when there are multiple representations available. + + Note: This is not called "format negotiation" because the alternate + representations may be of the same media type, but use different + capabilities of that type, be in different languages, etc. + + Any response containing an entity-body MAY be subject to negotiation, + including error responses. + + There are two kinds of content negotiation which are possible in + HTTP: server-driven and agent-driven negotiation. These two kinds of + negotiation are orthogonal and thus may be used separately or in + combination. One method of combination, referred to as transparent + negotiation, occurs when a cache uses the agent-driven negotiation + information provided by the origin server in order to provide + server-driven negotiation for subsequent requests. + +12.1 Server-driven Negotiation + + If the selection of the best representation for a response is made by + an algorithm located at the server, it is called server-driven + negotiation. Selection is based on the available representations of + the response (the dimensions over which it can vary; e.g. language, + content-coding, etc.) and the contents of particular header fields in + the request message or on other information pertaining to the request + (such as the network address of the client). + + Server-driven negotiation is advantageous when the algorithm for + selecting from among the available representations is difficult to + describe to the user agent, or when the server desires to send its + "best guess" to the client along with the first response (hoping to + avoid the round-trip delay of a subsequent request if the "best + guess" is good enough for the user). In order to improve the server's + guess, the user agent MAY include request header fields (Accept, + Accept-Language, Accept-Encoding, etc.) which describe its + preferences for such a response. + + Server-driven negotiation has disadvantages: + +1. It is impossible for the server to accurately determine what might be + "best" for any given user, since that would require complete + knowledge of both the capabilities of the user agent and the intended + use for the response (e.g., does the user want to view it on screen + or print it on paper?). + +2. Having the user agent describe its capabilities in every request can + be both very inefficient (given that only a small percentage of + responses have multiple representations) and a potential violation of + + + +Fielding, et. al. Standards Track [Page 68] + +RFC 2068 HTTP/1.1 January 1997 + + + the user's privacy. + +3. It complicates the implementation of an origin server and the + algorithms for generating responses to a request. + +4. It may limit a public cache's ability to use the same response for + multiple user's requests. + + HTTP/1.1 includes the following request-header fields for enabling + server-driven negotiation through description of user agent + capabilities and user preferences: Accept (section 14.1), Accept- + Charset (section 14.2), Accept-Encoding (section 14.3), Accept- + Language (section 14.4), and User-Agent (section 14.42). However, an + origin server is not limited to these dimensions and MAY vary the + response based on any aspect of the request, including information + outside the request-header fields or within extension header fields + not defined by this specification. + + HTTP/1.1 origin servers MUST include an appropriate Vary header field + (section 14.43) in any cachable response based on server-driven + negotiation. The Vary header field describes the dimensions over + which the response might vary (i.e. the dimensions over which the + origin server picks its "best guess" response from multiple + representations). + + HTTP/1.1 public caches MUST recognize the Vary header field when it + is included in a response and obey the requirements described in + section 13.6 that describes the interactions between caching and + content negotiation. + +12.2 Agent-driven Negotiation + + With agent-driven negotiation, selection of the best representation + for a response is performed by the user agent after receiving an + initial response from the origin server. Selection is based on a list + of the available representations of the response included within the + header fields (this specification reserves the field-name Alternates, + as described in appendix 19.6.2.1) or entity-body of the initial + response, with each representation identified by its own URI. + Selection from among the representations may be performed + automatically (if the user agent is capable of doing so) or manually + by the user selecting from a generated (possibly hypertext) menu. + + Agent-driven negotiation is advantageous when the response would vary + over commonly-used dimensions (such as type, language, or encoding), + when the origin server is unable to determine a user agent's + capabilities from examining the request, and generally when public + caches are used to distribute server load and reduce network usage. + + + +Fielding, et. al. Standards Track [Page 69] + +RFC 2068 HTTP/1.1 January 1997 + + + Agent-driven negotiation suffers from the disadvantage of needing a + second request to obtain the best alternate representation. This + second request is only efficient when caching is used. In addition, + this specification does not define any mechanism for supporting + automatic selection, though it also does not prevent any such + mechanism from being developed as an extension and used within + HTTP/1.1. + + HTTP/1.1 defines the 300 (Multiple Choices) and 406 (Not Acceptable) + status codes for enabling agent-driven negotiation when the server is + unwilling or unable to provide a varying response using server-driven + negotiation. + +12.3 Transparent Negotiation + + Transparent negotiation is a combination of both server-driven and + agent-driven negotiation. When a cache is supplied with a form of the + list of available representations of the response (as in agent-driven + negotiation) and the dimensions of variance are completely understood + by the cache, then the cache becomes capable of performing server- + driven negotiation on behalf of the origin server for subsequent + requests on that resource. + + Transparent negotiation has the advantage of distributing the + negotiation work that would otherwise be required of the origin + server and also removing the second request delay of agent-driven + negotiation when the cache is able to correctly guess the right + response. + + This specification does not define any mechanism for transparent + negotiation, though it also does not prevent any such mechanism from + being developed as an extension and used within HTTP/1.1. An HTTP/1.1 + cache performing transparent negotiation MUST include a Vary header + field in the response (defining the dimensions of its variance) if it + is cachable to ensure correct interoperation with all HTTP/1.1 + clients. The agent-driven negotiation information supplied by the + origin server SHOULD be included with the transparently negotiated + response. + +13 Caching in HTTP + + HTTP is typically used for distributed information systems, where + performance can be improved by the use of response caches. The + HTTP/1.1 protocol includes a number of elements intended to make + caching work as well as possible. Because these elements are + inextricable from other aspects of the protocol, and because they + interact with each other, it is useful to describe the basic caching + design of HTTP separately from the detailed descriptions of methods, + + + +Fielding, et. al. Standards Track [Page 70] + +RFC 2068 HTTP/1.1 January 1997 + + + headers, response codes, etc. + + Caching would be useless if it did not significantly improve + performance. The goal of caching in HTTP/1.1 is to eliminate the need + to send requests in many cases, and to eliminate the need to send + full responses in many other cases. The former reduces the number of + network round-trips required for many operations; we use an + "expiration" mechanism for this purpose (see section 13.2). The + latter reduces network bandwidth requirements; we use a "validation" + mechanism for this purpose (see section 13.3). + + Requirements for performance, availability, and disconnected + operation require us to be able to relax the goal of semantic + transparency. The HTTP/1.1 protocol allows origin servers, caches, + and clients to explicitly reduce transparency when necessary. + However, because non-transparent operation may confuse non-expert + users, and may be incompatible with certain server applications (such + as those for ordering merchandise), the protocol requires that + transparency be relaxed + + o only by an explicit protocol-level request when relaxed by client + or origin server + + o only with an explicit warning to the end user when relaxed by cache + or client + + + + + + + + + + + + + + + + + + + + + + + + + + +Fielding, et. al. Standards Track [Page 71] + +RFC 2068 HTTP/1.1 January 1997 + + + Therefore, the HTTP/1.1 protocol provides these important elements: + + 1. Protocol features that provide full semantic transparency when this + is required by all parties. + + 2. Protocol features that allow an origin server or user agent to + explicitly request and control non-transparent operation. + + 3. Protocol features that allow a cache to attach warnings to + responses that do not preserve the requested approximation of + semantic transparency. + + A basic principle is that it must be possible for the clients to + detect any potential relaxation of semantic transparency. + + Note: The server, cache, or client implementer may be faced with + design decisions not explicitly discussed in this specification. If + a decision may affect semantic transparency, the implementer ought + to err on the side of maintaining transparency unless a careful and + complete analysis shows significant benefits in breaking + transparency. + +13.1.1 Cache Correctness + + A correct cache MUST respond to a request with the most up-to-date + response held by the cache that is appropriate to the request (see + sections 13.2.5, 13.2.6, and 13.12) which meets one of the following + conditions: + + 1. It has been checked for equivalence with what the origin server + would have returned by revalidating the response with the origin + server (section 13.3); + + 2. It is "fresh enough" (see section 13.2). In the default case, this + means it meets the least restrictive freshness requirement of the + client, server, and cache (see section 14.9); if the origin server + so specifies, it is the freshness requirement of the origin server + alone. + + 3. It includes a warning if the freshness demand of the client or the + origin server is violated (see section 13.1.5 and 14.45). + + 4. It is an appropriate 304 (Not Modified), 305 (Proxy Redirect), or + error (4xx or 5xx) response message. + + If the cache can not communicate with the origin server, then a + correct cache SHOULD respond as above if the response can be + correctly served from the cache; if not it MUST return an error or + + + +Fielding, et. al. Standards Track [Page 72] + +RFC 2068 HTTP/1.1 January 1997 + + + warning indicating that there was a communication failure. + + If a cache receives a response (either an entire response, or a 304 + (Not Modified) response) that it would normally forward to the + requesting client, and the received response is no longer fresh, the + cache SHOULD forward it to the requesting client without adding a new + Warning (but without removing any existing Warning headers). A cache + SHOULD NOT attempt to revalidate a response simply because that + response became stale in transit; this might lead to an infinite + loop. An user agent that receives a stale response without a Warning + MAY display a warning indication to the user. + +13.1.2 Warnings + + Whenever a cache returns a response that is neither first-hand nor + "fresh enough" (in the sense of condition 2 in section 13.1.1), it + must attach a warning to that effect, using a Warning response- + header. This warning allows clients to take appropriate action. + + Warnings may be used for other purposes, both cache-related and + otherwise. The use of a warning, rather than an error status code, + distinguish these responses from true failures. + + Warnings are always cachable, because they never weaken the + transparency of a response. This means that warnings can be passed to + HTTP/1.0 caches without danger; such caches will simply pass the + warning along as an entity-header in the response. + + Warnings are assigned numbers between 0 and 99. This specification + defines the code numbers and meanings of each currently assigned + warnings, allowing a client or cache to take automated action in some + (but not all) cases. + + Warnings also carry a warning text. The text may be in any + appropriate natural language (perhaps based on the client's Accept + headers), and include an optional indication of what character set is + used. + + Multiple warnings may be attached to a response (either by the origin + server or by a cache), including multiple warnings with the same code + number. For example, a server may provide the same warning with texts + in both English and Basque. + + When multiple warnings are attached to a response, it may not be + practical or reasonable to display all of them to the user. This + version of HTTP does not specify strict priority rules for deciding + which warnings to display and in what order, but does suggest some + heuristics. + + + +Fielding, et. al. Standards Track [Page 73] + +RFC 2068 HTTP/1.1 January 1997 + + + The Warning header and the currently defined warnings are described + in section 14.45. + +13.1.3 Cache-control Mechanisms + + The basic cache mechanisms in HTTP/1.1 (server-specified expiration + times and validators) are implicit directives to caches. In some + cases, a server or client may need to provide explicit directives to + the HTTP caches. We use the Cache-Control header for this purpose. + + The Cache-Control header allows a client or server to transmit a + variety of directives in either requests or responses. These + directives typically override the default caching algorithms. As a + general rule, if there is any apparent conflict between header + values, the most restrictive interpretation should be applied (that + is, the one that is most likely to preserve semantic transparency). + However, in some cases, Cache-Control directives are explicitly + specified as weakening the approximation of semantic transparency + (for example, "max-stale" or "public"). + + The Cache-Control directives are described in detail in section 14.9. + +13.1.4 Explicit User Agent Warnings + + Many user agents make it possible for users to override the basic + caching mechanisms. For example, the user agent may allow the user to + specify that cached entities (even explicitly stale ones) are never + validated. Or the user agent might habitually add "Cache-Control: + max-stale=3600" to every request. The user should have to explicitly + request either non-transparent behavior, or behavior that results in + abnormally ineffective caching. + + If the user has overridden the basic caching mechanisms, the user + agent should explicitly indicate to the user whenever this results in + the display of information that might not meet the server's + transparency requirements (in particular, if the displayed entity is + known to be stale). Since the protocol normally allows the user agent + to determine if responses are stale or not, this indication need only + be displayed when this actually happens. The indication need not be a + dialog box; it could be an icon (for example, a picture of a rotting + fish) or some other visual indicator. + + If the user has overridden the caching mechanisms in a way that would + abnormally reduce the effectiveness of caches, the user agent should + continually display an indication (for example, a picture of currency + in flames) so that the user does not inadvertently consume excess + resources or suffer from excessive latency. + + + + +Fielding, et. al. Standards Track [Page 74] + +RFC 2068 HTTP/1.1 January 1997 + + +13.1.5 Exceptions to the Rules and Warnings + + In some cases, the operator of a cache may choose to configure it to + return stale responses even when not requested by clients. This + decision should not be made lightly, but may be necessary for reasons + of availability or performance, especially when the cache is poorly + connected to the origin server. Whenever a cache returns a stale + response, it MUST mark it as such (using a Warning header). This + allows the client software to alert the user that there may be a + potential problem. + + It also allows the user agent to take steps to obtain a first-hand or + fresh response. For this reason, a cache SHOULD NOT return a stale + response if the client explicitly requests a first-hand or fresh one, + unless it is impossible to comply for technical or policy reasons. + +13.1.6 Client-controlled Behavior + + While the origin server (and to a lesser extent, intermediate caches, + by their contribution to the age of a response) are the primary + source of expiration information, in some cases the client may need + to control a cache's decision about whether to return a cached + response without validating it. Clients do this using several + directives of the Cache-Control header. + + A client's request may specify the maximum age it is willing to + accept of an unvalidated response; specifying a value of zero forces + the cache(s) to revalidate all responses. A client may also specify + the minimum time remaining before a response expires. Both of these + options increase constraints on the behavior of caches, and so cannot + further relax the cache's approximation of semantic transparency. + + A client may also specify that it will accept stale responses, up to + some maximum amount of staleness. This loosens the constraints on the + caches, and so may violate the origin server's specified constraints + on semantic transparency, but may be necessary to support + disconnected operation, or high availability in the face of poor + connectivity. + +13.2 Expiration Model + +13.2.1 Server-Specified Expiration + + HTTP caching works best when caches can entirely avoid making + requests to the origin server. The primary mechanism for avoiding + requests is for an origin server to provide an explicit expiration + time in the future, indicating that a response may be used to satisfy + subsequent requests. In other words, a cache can return a fresh + + + +Fielding, et. al. Standards Track [Page 75] + +RFC 2068 HTTP/1.1 January 1997 + + + response without first contacting the server. + + Our expectation is that servers will assign future explicit + expiration times to responses in the belief that the entity is not + likely to change, in a semantically significant way, before the + expiration time is reached. This normally preserves semantic + transparency, as long as the server's expiration times are carefully + chosen. + + The expiration mechanism applies only to responses taken from a cache + and not to first-hand responses forwarded immediately to the + requesting client. + + If an origin server wishes to force a semantically transparent cache + to validate every request, it may assign an explicit expiration time + in the past. This means that the response is always stale, and so the + cache SHOULD validate it before using it for subsequent requests. See + section 14.9.4 for a more restrictive way to force revalidation. + + If an origin server wishes to force any HTTP/1.1 cache, no matter how + it is configured, to validate every request, it should use the + "must-revalidate" Cache-Control directive (see section 14.9). + + Servers specify explicit expiration times using either the Expires + header, or the max-age directive of the Cache-Control header. + + An expiration time cannot be used to force a user agent to refresh + its display or reload a resource; its semantics apply only to caching + mechanisms, and such mechanisms need only check a resource's + expiration status when a new request for that resource is initiated. + See section 13.13 for explanation of the difference between caches + and history mechanisms. + +13.2.2 Heuristic Expiration + + Since origin servers do not always provide explicit expiration times, + HTTP caches typically assign heuristic expiration times, employing + algorithms that use other header values (such as the Last-Modified + time) to estimate a plausible expiration time. The HTTP/1.1 + specification does not provide specific algorithms, but does impose + worst-case constraints on their results. Since heuristic expiration + times may compromise semantic transparency, they should be used + cautiously, and we encourage origin servers to provide explicit + expiration times as much as possible. + + + + + + + +Fielding, et. al. Standards Track [Page 76] + +RFC 2068 HTTP/1.1 January 1997 + + +13.2.3 Age Calculations + + In order to know if a cached entry is fresh, a cache needs to know if + its age exceeds its freshness lifetime. We discuss how to calculate + the latter in section 13.2.4; this section describes how to calculate + the age of a response or cache entry. + + In this discussion, we use the term "now" to mean "the current value + of the clock at the host performing the calculation." Hosts that use + HTTP, but especially hosts running origin servers and caches, should + use NTP [28] or some similar protocol to synchronize their clocks to + a globally accurate time standard. + + Also note that HTTP/1.1 requires origin servers to send a Date header + with every response, giving the time at which the response was + generated. We use the term "date_value" to denote the value of the + Date header, in a form appropriate for arithmetic operations. + + HTTP/1.1 uses the Age response-header to help convey age information + between caches. The Age header value is the sender's estimate of the + amount of time since the response was generated at the origin server. + In the case of a cached response that has been revalidated with the + origin server, the Age value is based on the time of revalidation, + not of the original response. + + In essence, the Age value is the sum of the time that the response + has been resident in each of the caches along the path from the + origin server, plus the amount of time it has been in transit along + network paths. + + We use the term "age_value" to denote the value of the Age header, in + a form appropriate for arithmetic operations. + + A response's age can be calculated in two entirely independent ways: + + 1. now minus date_value, if the local clock is reasonably well + synchronized to the origin server's clock. If the result is + negative, the result is replaced by zero. + + 2. age_value, if all of the caches along the response path + implement HTTP/1.1. + + Given that we have two independent ways to compute the age of a + response when it is received, we can combine these as + + corrected_received_age = max(now - date_value, age_value) + + and as long as we have either nearly synchronized clocks or all- + + + +Fielding, et. al. Standards Track [Page 77] + +RFC 2068 HTTP/1.1 January 1997 + + + HTTP/1.1 paths, one gets a reliable (conservative) result. + + Note that this correction is applied at each HTTP/1.1 cache along the + path, so that if there is an HTTP/1.0 cache in the path, the correct + received age is computed as long as the receiving cache's clock is + nearly in sync. We don't need end-to-end clock synchronization + (although it is good to have), and there is no explicit clock + synchronization step. + + Because of network-imposed delays, some significant interval may pass + from the time that a server generates a response and the time it is + received at the next outbound cache or client. If uncorrected, this + delay could result in improperly low ages. + + Because the request that resulted in the returned Age value must have + been initiated prior to that Age value's generation, we can correct + for delays imposed by the network by recording the time at which the + request was initiated. Then, when an Age value is received, it MUST + be interpreted relative to the time the request was initiated, not + the time that the response was received. This algorithm results in + conservative behavior no matter how much delay is experienced. So, we + compute: + + corrected_initial_age = corrected_received_age + + (now - request_time) + + where "request_time" is the time (according to the local clock) when + the request that elicited this response was sent. + + Summary of age calculation algorithm, when a cache receives a + response: + + /* + * age_value + * is the value of Age: header received by the cache with + * this response. + * date_value + * is the value of the origin server's Date: header + * request_time + * is the (local) time when the cache made the request + * that resulted in this cached response + * response_time + * is the (local) time when the cache received the + * response + * now + * is the current (local) time + */ + apparent_age = max(0, response_time - date_value); + + + +Fielding, et. al. Standards Track [Page 78] + +RFC 2068 HTTP/1.1 January 1997 + + + corrected_received_age = max(apparent_age, age_value); + response_delay = response_time - request_time; + corrected_initial_age = corrected_received_age + response_delay; + resident_time = now - response_time; + current_age = corrected_initial_age + resident_time; + + When a cache sends a response, it must add to the + corrected_initial_age the amount of time that the response was + resident locally. It must then transmit this total age, using the Age + header, to the next recipient cache. + + Note that a client cannot reliably tell that a response is first- + hand, but the presence of an Age header indicates that a response + is definitely not first-hand. Also, if the Date in a response is + earlier than the client's local request time, the response is + probably not first-hand (in the absence of serious clock skew). + +13.2.4 Expiration Calculations + + In order to decide whether a response is fresh or stale, we need to + compare its freshness lifetime to its age. The age is calculated as + described in section 13.2.3; this section describes how to calculate + the freshness lifetime, and to determine if a response has expired. + In the discussion below, the values can be represented in any form + appropriate for arithmetic operations. + + We use the term "expires_value" to denote the value of the Expires + header. We use the term "max_age_value" to denote an appropriate + value of the number of seconds carried by the max-age directive of + the Cache-Control header in a response (see section 14.10. + + The max-age directive takes priority over Expires, so if max-age is + present in a response, the calculation is simply: + + freshness_lifetime = max_age_value + + Otherwise, if Expires is present in the response, the calculation is: + + freshness_lifetime = expires_value - date_value + + Note that neither of these calculations is vulnerable to clock skew, + since all of the information comes from the origin server. + + If neither Expires nor Cache-Control: max-age appears in the + response, and the response does not include other restrictions on + caching, the cache MAY compute a freshness lifetime using a + heuristic. If the value is greater than 24 hours, the cache must + attach Warning 13 to any response whose age is more than 24 hours if + + + +Fielding, et. al. Standards Track [Page 79] + +RFC 2068 HTTP/1.1 January 1997 + + + such warning has not already been added. + + Also, if the response does have a Last-Modified time, the heuristic + expiration value SHOULD be no more than some fraction of the interval + since that time. A typical setting of this fraction might be 10%. + + The calculation to determine if a response has expired is quite + simple: + + response_is_fresh = (freshness_lifetime > current_age) + +13.2.5 Disambiguating Expiration Values + + Because expiration values are assigned optimistically, it is possible + for two caches to contain fresh values for the same resource that are + different. + + If a client performing a retrieval receives a non-first-hand response + for a request that was already fresh in its own cache, and the Date + header in its existing cache entry is newer than the Date on the new + response, then the client MAY ignore the response. If so, it MAY + retry the request with a "Cache-Control: max-age=0" directive (see + section 14.9), to force a check with the origin server. + + If a cache has two fresh responses for the same representation with + different validators, it MUST use the one with the more recent Date + header. This situation may arise because the cache is pooling + responses from other caches, or because a client has asked for a + reload or a revalidation of an apparently fresh cache entry. + +13.2.6 Disambiguating Multiple Responses + + Because a client may be receiving responses via multiple paths, so + that some responses flow through one set of caches and other + responses flow through a different set of caches, a client may + receive responses in an order different from that in which the origin + server sent them. We would like the client to use the most recently + generated response, even if older responses are still apparently + fresh. + + Neither the entity tag nor the expiration value can impose an + ordering on responses, since it is possible that a later response + intentionally carries an earlier expiration time. However, the + HTTP/1.1 specification requires the transmission of Date headers on + every response, and the Date values are ordered to a granularity of + one second. + + + + + +Fielding, et. al. Standards Track [Page 80] + +RFC 2068 HTTP/1.1 January 1997 + + + When a client tries to revalidate a cache entry, and the response it + receives contains a Date header that appears to be older than the one + for the existing entry, then the client SHOULD repeat the request + unconditionally, and include + + Cache-Control: max-age=0 + + to force any intermediate caches to validate their copies directly + with the origin server, or + + Cache-Control: no-cache + + to force any intermediate caches to obtain a new copy from the origin + server. + + If the Date values are equal, then the client may use either response + (or may, if it is being extremely prudent, request a new response). + Servers MUST NOT depend on clients being able to choose + deterministically between responses generated during the same second, + if their expiration times overlap. + +13.3 Validation Model + + When a cache has a stale entry that it would like to use as a + response to a client's request, it first has to check with the origin + server (or possibly an intermediate cache with a fresh response) to + see if its cached entry is still usable. We call this "validating" + the cache entry. Since we do not want to have to pay the overhead of + retransmitting the full response if the cached entry is good, and we + do not want to pay the overhead of an extra round trip if the cached + entry is invalid, the HTTP/1.1 protocol supports the use of + conditional methods. + + The key protocol features for supporting conditional methods are + those concerned with "cache validators." When an origin server + generates a full response, it attaches some sort of validator to it, + which is kept with the cache entry. When a client (user agent or + proxy cache) makes a conditional request for a resource for which it + has a cache entry, it includes the associated validator in the + request. + + The server then checks that validator against the current validator + for the entity, and, if they match, it responds with a special status + code (usually, 304 (Not Modified)) and no entity-body. Otherwise, it + returns a full response (including entity-body). Thus, we avoid + transmitting the full response if the validator matches, and we avoid + an extra round trip if it does not match. + + + + +Fielding, et. al. Standards Track [Page 81] + +RFC 2068 HTTP/1.1 January 1997 + + + Note: the comparison functions used to decide if validators match + are defined in section 13.3.3. + + In HTTP/1.1, a conditional request looks exactly the same as a normal + request for the same resource, except that it carries a special + header (which includes the validator) that implicitly turns the + method (usually, GET) into a conditional. + + The protocol includes both positive and negative senses of cache- + validating conditions. That is, it is possible to request either that + a method be performed if and only if a validator matches or if and + only if no validators match. + + Note: a response that lacks a validator may still be cached, and + served from cache until it expires, unless this is explicitly + prohibited by a Cache-Control directive. However, a cache cannot do + a conditional retrieval if it does not have a validator for the + entity, which means it will not be refreshable after it expires. + +13.3.1 Last-modified Dates + + The Last-Modified entity-header field value is often used as a cache + validator. In simple terms, a cache entry is considered to be valid + if the entity has not been modified since the Last-Modified value. + +13.3.2 Entity Tag Cache Validators + + The ETag entity-header field value, an entity tag, provides for an + "opaque" cache validator. This may allow more reliable validation in + situations where it is inconvenient to store modification dates, + where the one-second resolution of HTTP date values is not + sufficient, or where the origin server wishes to avoid certain + paradoxes that may arise from the use of modification dates. + + Entity Tags are described in section 3.11. The headers used with + entity tags are described in sections 14.20, 14.25, 14.26 and 14.43. + +13.3.3 Weak and Strong Validators + + Since both origin servers and caches will compare two validators to + decide if they represent the same or different entities, one normally + would expect that if the entity (the entity-body or any entity- + headers) changes in any way, then the associated validator would + change as well. If this is true, then we call this validator a + "strong validator." + + However, there may be cases when a server prefers to change the + validator only on semantically significant changes, and not when + + + +Fielding, et. al. Standards Track [Page 82] + +RFC 2068 HTTP/1.1 January 1997 + + + insignificant aspects of the entity change. A validator that does not + always change when the resource changes is a "weak validator." + + Entity tags are normally "strong validators," but the protocol + provides a mechanism to tag an entity tag as "weak." One can think of + a strong validator as one that changes whenever the bits of an entity + changes, while a weak value changes whenever the meaning of an entity + changes. Alternatively, one can think of a strong validator as part + of an identifier for a specific entity, while a weak validator is + part of an identifier for a set of semantically equivalent entities. + + Note: One example of a strong validator is an integer that is + incremented in stable storage every time an entity is changed. + + An entity's modification time, if represented with one-second + resolution, could be a weak validator, since it is possible that + the resource may be modified twice during a single second. + + Support for weak validators is optional; however, weak validators + allow for more efficient caching of equivalent objects; for + example, a hit counter on a site is probably good enough if it is + updated every few days or weeks, and any value during that period + is likely "good enough" to be equivalent. + + A "use" of a validator is either when a client generates a request + and includes the validator in a validating header field, or when a + server compares two validators. + + Strong validators are usable in any context. Weak validators are only + usable in contexts that do not depend on exact equality of an entity. + For example, either kind is usable for a conditional GET of a full + entity. However, only a strong validator is usable for a sub-range + retrieval, since otherwise the client may end up with an internally + inconsistent entity. + + The only function that the HTTP/1.1 protocol defines on validators is + comparison. There are two validator comparison functions, depending + on whether the comparison context allows the use of weak validators + or not: + + o The strong comparison function: in order to be considered equal, + both validators must be identical in every way, and neither may be + weak. + o The weak comparison function: in order to be considered equal, both + validators must be identical in every way, but either or both of + them may be tagged as "weak" without affecting the result. + + The weak comparison function MAY be used for simple (non-subrange) + + + +Fielding, et. al. Standards Track [Page 83] + +RFC 2068 HTTP/1.1 January 1997 + + + GET requests. The strong comparison function MUST be used in all + other cases. + + An entity tag is strong unless it is explicitly tagged as weak. + Section 3.11 gives the syntax for entity tags. + + A Last-Modified time, when used as a validator in a request, is + implicitly weak unless it is possible to deduce that it is strong, + using the following rules: + + o The validator is being compared by an origin server to the actual + current validator for the entity and, + o That origin server reliably knows that the associated entity did + not change twice during the second covered by the presented + validator. +or + + o The validator is about to be used by a client in an If-Modified- + Since or If-Unmodified-Since header, because the client has a cache + entry for the associated entity, and + o That cache entry includes a Date value, which gives the time when + the origin server sent the original response, and + o The presented Last-Modified time is at least 60 seconds before the + Date value. +or + + o The validator is being compared by an intermediate cache to the + validator stored in its cache entry for the entity, and + o That cache entry includes a Date value, which gives the time when + the origin server sent the original response, and + o The presented Last-Modified time is at least 60 seconds before the + Date value. + + This method relies on the fact that if two different responses were + sent by the origin server during the same second, but both had the + same Last-Modified time, then at least one of those responses would + have a Date value equal to its Last-Modified time. The arbitrary 60- + second limit guards against the possibility that the Date and Last- + Modified values are generated from different clocks, or at somewhat + different times during the preparation of the response. An + implementation may use a value larger than 60 seconds, if it is + believed that 60 seconds is too short. + + If a client wishes to perform a sub-range retrieval on a value for + which it has only a Last-Modified time and no opaque validator, it + may do this only if the Last-Modified time is strong in the sense + described here. + + + + +Fielding, et. al. Standards Track [Page 84] + +RFC 2068 HTTP/1.1 January 1997 + + + A cache or origin server receiving a cache-conditional request, other + than a full-body GET request, MUST use the strong comparison function + to evaluate the condition. + + These rules allow HTTP/1.1 caches and clients to safely perform sub- + range retrievals on values that have been obtained from HTTP/1.0 + servers. + +13.3.4 Rules for When to Use Entity Tags and Last-modified Dates + + We adopt a set of rules and recommendations for origin servers, + clients, and caches regarding when various validator types should be + used, and for what purposes. + + HTTP/1.1 origin servers: + + o SHOULD send an entity tag validator unless it is not feasible to + generate one. + o MAY send a weak entity tag instead of a strong entity tag, if + performance considerations support the use of weak entity tags, or + if it is unfeasible to send a strong entity tag. + o SHOULD send a Last-Modified value if it is feasible to send one, + unless the risk of a breakdown in semantic transparency that could + result from using this date in an If-Modified-Since header would + lead to serious problems. + + In other words, the preferred behavior for an HTTP/1.1 origin server + is to send both a strong entity tag and a Last-Modified value. + + In order to be legal, a strong entity tag MUST change whenever the + associated entity value changes in any way. A weak entity tag SHOULD + change whenever the associated entity changes in a semantically + significant way. + + Note: in order to provide semantically transparent caching, an + origin server must avoid reusing a specific strong entity tag value + for two different entities, or reusing a specific weak entity tag + value for two semantically different entities. Cache entries may + persist for arbitrarily long periods, regardless of expiration + times, so it may be inappropriate to expect that a cache will never + again attempt to validate an entry using a validator that it + obtained at some point in the past. + + HTTP/1.1 clients: + + o If an entity tag has been provided by the origin server, MUST + use that entity tag in any cache-conditional request (using + If-Match or If-None-Match). + + + +Fielding, et. al. Standards Track [Page 85] + +RFC 2068 HTTP/1.1 January 1997 + + + o If only a Last-Modified value has been provided by the origin + server, SHOULD use that value in non-subrange cache-conditional + requests (using If-Modified-Since). + o If only a Last-Modified value has been provided by an HTTP/1.0 + origin server, MAY use that value in subrange cache-conditional + requests (using If-Unmodified-Since:). The user agent should + provide a way to disable this, in case of difficulty. + o If both an entity tag and a Last-Modified value have been + provided by the origin server, SHOULD use both validators in + cache-conditional requests. This allows both HTTP/1.0 and + HTTP/1.1 caches to respond appropriately. + + An HTTP/1.1 cache, upon receiving a request, MUST use the most + restrictive validator when deciding whether the client's cache entry + matches the cache's own cache entry. This is only an issue when the + request contains both an entity tag and a last-modified-date + validator (If-Modified-Since or If-Unmodified-Since). + + A note on rationale: The general principle behind these rules is + that HTTP/1.1 servers and clients should transmit as much non- + redundant information as is available in their responses and + requests. HTTP/1.1 systems receiving this information will make the + most conservative assumptions about the validators they receive. + + HTTP/1.0 clients and caches will ignore entity tags. Generally, + last-modified values received or used by these systems will support + transparent and efficient caching, and so HTTP/1.1 origin servers + should provide Last-Modified values. In those rare cases where the + use of a Last-Modified value as a validator by an HTTP/1.0 system + could result in a serious problem, then HTTP/1.1 origin servers + should not provide one. + +13.3.5 Non-validating Conditionals + + The principle behind entity tags is that only the service author + knows the semantics of a resource well enough to select an + appropriate cache validation mechanism, and the specification of any + validator comparison function more complex than byte-equality would + open up a can of worms. Thus, comparisons of any other headers + (except Last-Modified, for compatibility with HTTP/1.0) are never + used for purposes of validating a cache entry. + +13.4 Response Cachability + + Unless specifically constrained by a Cache-Control (section 14.9) + directive, a caching system may always store a successful response + (see section 13.8) as a cache entry, may return it without validation + if it is fresh, and may return it after successful validation. If + + + +Fielding, et. al. Standards Track [Page 86] + +RFC 2068 HTTP/1.1 January 1997 + + + there is neither a cache validator nor an explicit expiration time + associated with a response, we do not expect it to be cached, but + certain caches may violate this expectation (for example, when little + or no network connectivity is available). A client can usually detect + that such a response was taken from a cache by comparing the Date + header to the current time. + + Note that some HTTP/1.0 caches are known to violate this + expectation without providing any Warning. + + However, in some cases it may be inappropriate for a cache to retain + an entity, or to return it in response to a subsequent request. This + may be because absolute semantic transparency is deemed necessary by + the service author, or because of security or privacy considerations. + Certain Cache-Control directives are therefore provided so that the + server can indicate that certain resource entities, or portions + thereof, may not be cached regardless of other considerations. + + Note that section 14.8 normally prevents a shared cache from saving + and returning a response to a previous request if that request + included an Authorization header. + + A response received with a status code of 200, 203, 206, 300, 301 or + 410 may be stored by a cache and used in reply to a subsequent + request, subject to the expiration mechanism, unless a Cache-Control + directive prohibits caching. However, a cache that does not support + the Range and Content-Range headers MUST NOT cache 206 (Partial + Content) responses. + + A response received with any other status code MUST NOT be returned + in a reply to a subsequent request unless there are Cache-Control + directives or another header(s) that explicitly allow it. For + example, these include the following: an Expires header (section + 14.21); a "max-age", "must-revalidate", "proxy-revalidate", "public" + or "private" Cache-Control directive (section 14.9). + +13.5 Constructing Responses From Caches + + The purpose of an HTTP cache is to store information received in + response to requests, for use in responding to future requests. In + many cases, a cache simply returns the appropriate parts of a + response to the requester. However, if the cache holds a cache entry + based on a previous response, it may have to combine parts of a new + response with what is held in the cache entry. + + + + + + + +Fielding, et. al. Standards Track [Page 87] + +RFC 2068 HTTP/1.1 January 1997 + + +13.5.1 End-to-end and Hop-by-hop Headers + + For the purpose of defining the behavior of caches and non-caching + proxies, we divide HTTP headers into two categories: + + o End-to-end headers, which must be transmitted to the + ultimate recipient of a request or response. End-to-end + headers in responses must be stored as part of a cache entry + and transmitted in any response formed from a cache entry. + o Hop-by-hop headers, which are meaningful only for a single + transport-level connection, and are not stored by caches or + forwarded by proxies. + + The following HTTP/1.1 headers are hop-by-hop headers: + + o Connection + o Keep-Alive + o Public + o Proxy-Authenticate + o Transfer-Encoding + o Upgrade + + All other headers defined by HTTP/1.1 are end-to-end headers. + + Hop-by-hop headers introduced in future versions of HTTP MUST be + listed in a Connection header, as described in section 14.10. + +13.5.2 Non-modifiable Headers + + Some features of the HTTP/1.1 protocol, such as Digest + Authentication, depend on the value of certain end-to-end headers. A + cache or non-caching proxy SHOULD NOT modify an end-to-end header + unless the definition of that header requires or specifically allows + that. + + A cache or non-caching proxy MUST NOT modify any of the following + fields in a request or response, nor may it add any of these fields + if not already present: + + o Content-Location + o ETag + o Expires + o Last-Modified + + + + + + + + +Fielding, et. al. Standards Track [Page 88] + +RFC 2068 HTTP/1.1 January 1997 + + + A cache or non-caching proxy MUST NOT modify or add any of the + following fields in a response that contains the no-transform Cache- + Control directive, or in any request: + + o Content-Encoding + o Content-Length + o Content-Range + o Content-Type + + A cache or non-caching proxy MAY modify or add these fields in a + response that does not include no-transform, but if it does so, it + MUST add a Warning 14 (Transformation applied) if one does not + already appear in the response. + + Warning: unnecessary modification of end-to-end headers may cause + authentication failures if stronger authentication mechanisms are + introduced in later versions of HTTP. Such authentication + mechanisms may rely on the values of header fields not listed here. + +13.5.3 Combining Headers + + When a cache makes a validating request to a server, and the server + provides a 304 (Not Modified) response, the cache must construct a + response to send to the requesting client. The cache uses the + entity-body stored in the cache entry as the entity-body of this + outgoing response. The end-to-end headers stored in the cache entry + are used for the constructed response, except that any end-to-end + headers provided in the 304 response MUST replace the corresponding + headers from the cache entry. Unless the cache decides to remove the + cache entry, it MUST also replace the end-to-end headers stored with + the cache entry with corresponding headers received in the incoming + response. + + In other words, the set of end-to-end headers received in the + incoming response overrides all corresponding end-to-end headers + stored with the cache entry. The cache may add Warning headers (see + section 14.45) to this set. + + If a header field-name in the incoming response matches more than one + header in the cache entry, all such old headers are replaced. + + Note: this rule allows an origin server to use a 304 (Not Modified) + response to update any header associated with a previous response + for the same entity, although it might not always be meaningful or + correct to do so. This rule does not allow an origin server to use + a 304 (not Modified) response to entirely delete a header that it + had provided with a previous response. + + + + +Fielding, et. al. Standards Track [Page 89] + +RFC 2068 HTTP/1.1 January 1997 + + +13.5.4 Combining Byte Ranges + + A response may transfer only a subrange of the bytes of an entity- + body, either because the request included one or more Range + specifications, or because a connection was broken prematurely. After + several such transfers, a cache may have received several ranges of + the same entity-body. + + If a cache has a stored non-empty set of subranges for an entity, and + an incoming response transfers another subrange, the cache MAY + combine the new subrange with the existing set if both the following + conditions are met: + + o Both the incoming response and the cache entry must have a cache + validator. + o The two cache validators must match using the strong comparison + function (see section 13.3.3). + + If either requirement is not meant, the cache must use only the most + recent partial response (based on the Date values transmitted with + every response, and using the incoming response if these values are + equal or missing), and must discard the other partial information. + +13.6 Caching Negotiated Responses + + Use of server-driven content negotiation (section 12), as indicated + by the presence of a Vary header field in a response, alters the + conditions and procedure by which a cache can use the response for + subsequent requests. + + A server MUST use the Vary header field (section 14.43) to inform a + cache of what header field dimensions are used to select among + multiple representations of a cachable response. A cache may use the + selected representation (the entity included with that particular + response) for replying to subsequent requests on that resource only + when the subsequent requests have the same or equivalent values for + all header fields specified in the Vary response-header. Requests + with a different value for one or more of those header fields would + be forwarded toward the origin server. + + If an entity tag was assigned to the representation, the forwarded + request SHOULD be conditional and include the entity tags in an If- + None-Match header field from all its cache entries for the Request- + URI. This conveys to the server the set of entities currently held by + the cache, so that if any one of these entities matches the requested + entity, the server can use the ETag header in its 304 (Not Modified) + response to tell the cache which entry is appropriate. If the + entity-tag of the new response matches that of an existing entry, the + + + +Fielding, et. al. Standards Track [Page 90] + +RFC 2068 HTTP/1.1 January 1997 + + + new response SHOULD be used to update the header fields of the + existing entry, and the result MUST be returned to the client. + + The Vary header field may also inform the cache that the + representation was selected using criteria not limited to the + request-headers; in this case, a cache MUST NOT use the response in a + reply to a subsequent request unless the cache relays the new request + to the origin server in a conditional request and the server responds + with 304 (Not Modified), including an entity tag or Content-Location + that indicates which entity should be used. + + If any of the existing cache entries contains only partial content + for the associated entity, its entity-tag SHOULD NOT be included in + the If-None-Match header unless the request is for a range that would + be fully satisfied by that entry. + + If a cache receives a successful response whose Content-Location + field matches that of an existing cache entry for the same Request- + URI, whose entity-tag differs from that of the existing entry, and + whose Date is more recent than that of the existing entry, the + existing entry SHOULD NOT be returned in response to future requests, + and should be deleted from the cache. + +13.7 Shared and Non-Shared Caches + + For reasons of security and privacy, it is necessary to make a + distinction between "shared" and "non-shared" caches. A non-shared + cache is one that is accessible only to a single user. Accessibility + in this case SHOULD be enforced by appropriate security mechanisms. + All other caches are considered to be "shared." Other sections of + this specification place certain constraints on the operation of + shared caches in order to prevent loss of privacy or failure of + access controls. + +13.8 Errors or Incomplete Response Cache Behavior + + A cache that receives an incomplete response (for example, with fewer + bytes of data than specified in a Content-Length header) may store + the response. However, the cache MUST treat this as a partial + response. Partial responses may be combined as described in section + 13.5.4; the result might be a full response or might still be + partial. A cache MUST NOT return a partial response to a client + without explicitly marking it as such, using the 206 (Partial + Content) status code. A cache MUST NOT return a partial response + using a status code of 200 (OK). + + If a cache receives a 5xx response while attempting to revalidate an + entry, it may either forward this response to the requesting client, + + + +Fielding, et. al. Standards Track [Page 91] + +RFC 2068 HTTP/1.1 January 1997 + + + or act as if the server failed to respond. In the latter case, it MAY + return a previously received response unless the cached entry + includes the "must-revalidate" Cache-Control directive (see section + 14.9). + +13.9 Side Effects of GET and HEAD + + Unless the origin server explicitly prohibits the caching of their + responses, the application of GET and HEAD methods to any resources + SHOULD NOT have side effects that would lead to erroneous behavior if + these responses are taken from a cache. They may still have side + effects, but a cache is not required to consider such side effects in + its caching decisions. Caches are always expected to observe an + origin server's explicit restrictions on caching. + + We note one exception to this rule: since some applications have + traditionally used GETs and HEADs with query URLs (those containing a + "?" in the rel_path part) to perform operations with significant side + effects, caches MUST NOT treat responses to such URLs as fresh unless + the server provides an explicit expiration time. This specifically + means that responses from HTTP/1.0 servers for such URIs should not + be taken from a cache. See section 9.1.1 for related information. + +13.10 Invalidation After Updates or Deletions + + The effect of certain methods at the origin server may cause one or + more existing cache entries to become non-transparently invalid. That + is, although they may continue to be "fresh," they do not accurately + reflect what the origin server would return for a new request. + + There is no way for the HTTP protocol to guarantee that all such + cache entries are marked invalid. For example, the request that + caused the change at the origin server may not have gone through the + proxy where a cache entry is stored. However, several rules help + reduce the likelihood of erroneous behavior. + + In this section, the phrase "invalidate an entity" means that the + cache should either remove all instances of that entity from its + storage, or should mark these as "invalid" and in need of a mandatory + revalidation before they can be returned in response to a subsequent + request. + + + + + + + + + + +Fielding, et. al. Standards Track [Page 92] + +RFC 2068 HTTP/1.1 January 1997 + + + Some HTTP methods may invalidate an entity. This is either the entity + referred to by the Request-URI, or by the Location or Content- + Location response-headers (if present). These methods are: + + o PUT + o DELETE + o POST + + In order to prevent denial of service attacks, an invalidation based + on the URI in a Location or Content-Location header MUST only be + performed if the host part is the same as in the Request-URI. + +13.11 Write-Through Mandatory + + All methods that may be expected to cause modifications to the origin + server's resources MUST be written through to the origin server. This + currently includes all methods except for GET and HEAD. A cache MUST + NOT reply to such a request from a client before having transmitted + the request to the inbound server, and having received a + corresponding response from the inbound server. This does not prevent + a cache from sending a 100 (Continue) response before the inbound + server has replied. + + The alternative (known as "write-back" or "copy-back" caching) is not + allowed in HTTP/1.1, due to the difficulty of providing consistent + updates and the problems arising from server, cache, or network + failure prior to write-back. + +13.12 Cache Replacement + + If a new cachable (see sections 14.9.2, 13.2.5, 13.2.6 and 13.8) + response is received from a resource while any existing responses for + the same resource are cached, the cache SHOULD use the new response + to reply to the current request. It may insert it into cache storage + and may, if it meets all other requirements, use it to respond to any + future requests that would previously have caused the old response to + be returned. If it inserts the new response into cache storage it + should follow the rules in section 13.5.3. + + Note: a new response that has an older Date header value than + existing cached responses is not cachable. + +13.13 History Lists + + User agents often have history mechanisms, such as "Back" buttons and + history lists, which can be used to redisplay an entity retrieved + earlier in a session. + + + + +Fielding, et. al. Standards Track [Page 93] + +RFC 2068 HTTP/1.1 January 1997 + + + History mechanisms and caches are different. In particular history + mechanisms SHOULD NOT try to show a semantically transparent view of + the current state of a resource. Rather, a history mechanism is meant + to show exactly what the user saw at the time when the resource was + retrieved. + + By default, an expiration time does not apply to history mechanisms. + If the entity is still in storage, a history mechanism should display + it even if the entity has expired, unless the user has specifically + configured the agent to refresh expired history documents. + + This should not be construed to prohibit the history mechanism from + telling the user that a view may be stale. + + Note: if history list mechanisms unnecessarily prevent users from + viewing stale resources, this will tend to force service authors to + avoid using HTTP expiration controls and cache controls when they + would otherwise like to. Service authors may consider it important + that users not be presented with error messages or warning messages + when they use navigation controls (such as BACK) to view previously + fetched resources. Even though sometimes such resources ought not + to cached, or ought to expire quickly, user interface + considerations may force service authors to resort to other means + of preventing caching (e.g. "once-only" URLs) in order not to + suffer the effects of improperly functioning history mechanisms. + +14 Header Field Definitions + + This section defines the syntax and semantics of all standard + HTTP/1.1 header fields. For entity-header fields, both sender and + recipient refer to either the client or the server, depending on who + sends and who receives the entity. + + + + + + + + + + + + + + + + + + + +Fielding, et. al. Standards Track [Page 94] + +RFC 2068 HTTP/1.1 January 1997 + + +14.1 Accept + + The Accept request-header field can be used to specify certain media + types which are acceptable for the response. Accept headers can be + used to indicate that the request is specifically limited to a small + set of desired types, as in the case of a request for an in-line + image. + + Accept = "Accept" ":" + #( media-range [ accept-params ] ) + + media-range = ( "*/*" + | ( type "/" "*" ) + | ( type "/" subtype ) + ) *( ";" parameter ) + + accept-params = ";" "q" "=" qvalue *( accept-extension ) + + accept-extension = ";" token [ "=" ( token | quoted-string ) ] + + The asterisk "*" character is used to group media types into ranges, + with "*/*" indicating all media types and "type/*" indicating all + subtypes of that type. The media-range MAY include media type + parameters that are applicable to that range. + + Each media-range MAY be followed by one or more accept-params, + beginning with the "q" parameter for indicating a relative quality + factor. The first "q" parameter (if any) separates the media-range + parameter(s) from the accept-params. Quality factors allow the user + or user agent to indicate the relative degree of preference for that + media-range, using the qvalue scale from 0 to 1 (section 3.9). The + default value is q=1. + + Note: Use of the "q" parameter name to separate media type + parameters from Accept extension parameters is due to historical + practice. Although this prevents any media type parameter named + "q" from being used with a media range, such an event is believed + to be unlikely given the lack of any "q" parameters in the IANA + media type registry and the rare usage of any media type parameters + in Accept. Future media types should be discouraged from + registering any parameter named "q". + + The example + + Accept: audio/*; q=0.2, audio/basic + + SHOULD be interpreted as "I prefer audio/basic, but send me any audio + type if it is the best available after an 80% mark-down in quality." + + + +Fielding, et. al. Standards Track [Page 95] + +RFC 2068 HTTP/1.1 January 1997 + + + If no Accept header field is present, then it is assumed that the + client accepts all media types. If an Accept header field is present, + and if the server cannot send a response which is acceptable + according to the combined Accept field value, then the server SHOULD + send a 406 (not acceptable) response. + + A more elaborate example is + + Accept: text/plain; q=0.5, text/html, + text/x-dvi; q=0.8, text/x-c + + Verbally, this would be interpreted as "text/html and text/x-c are + the preferred media types, but if they do not exist, then send the + text/x-dvi entity, and if that does not exist, send the text/plain + entity." + + Media ranges can be overridden by more specific media ranges or + specific media types. If more than one media range applies to a given + type, the most specific reference has precedence. For example, + + Accept: text/*, text/html, text/html;level=1, */* + + have the following precedence: + + 1) text/html;level=1 + 2) text/html + 3) text/* + 4) */* + + The media type quality factor associated with a given type is + determined by finding the media range with the highest precedence + which matches that type. For example, + + Accept: text/*;q=0.3, text/html;q=0.7, text/html;level=1, + text/html;level=2;q=0.4, */*;q=0.5 + + would cause the following values to be associated: + + text/html;level=1 = 1 + text/html = 0.7 + text/plain = 0.3 + image/jpeg = 0.5 + text/html;level=2 = 0.4 + text/html;level=3 = 0.7 + + Note: A user agent may be provided with a default set of quality + values for certain media ranges. However, unless the user agent is + a closed system which cannot interact with other rendering agents, + + + +Fielding, et. al. Standards Track [Page 96] + +RFC 2068 HTTP/1.1 January 1997 + + + this default set should be configurable by the user. + +14.2 Accept-Charset + + The Accept-Charset request-header field can be used to indicate what + character sets are acceptable for the response. This field allows + clients capable of understanding more comprehensive or special- + purpose character sets to signal that capability to a server which is + capable of representing documents in those character sets. The ISO- + 8859-1 character set can be assumed to be acceptable to all user + agents. + + Accept-Charset = "Accept-Charset" ":" + 1#( charset [ ";" "q" "=" qvalue ] ) + + Character set values are described in section 3.4. Each charset may + be given an associated quality value which represents the user's + preference for that charset. The default value is q=1. An example is + + Accept-Charset: iso-8859-5, unicode-1-1;q=0.8 + + If no Accept-Charset header is present, the default is that any + character set is acceptable. If an Accept-Charset header is present, + and if the server cannot send a response which is acceptable + according to the Accept-Charset header, then the server SHOULD send + an error response with the 406 (not acceptable) status code, though + the sending of an unacceptable response is also allowed. + +14.3 Accept-Encoding + + The Accept-Encoding request-header field is similar to Accept, but + restricts the content-coding values (section 14.12) which are + acceptable in the response. + + Accept-Encoding = "Accept-Encoding" ":" + #( content-coding ) + + An example of its use is + + Accept-Encoding: compress, gzip + + If no Accept-Encoding header is present in a request, the server MAY + assume that the client will accept any content coding. If an Accept- + Encoding header is present, and if the server cannot send a response + which is acceptable according to the Accept-Encoding header, then the + server SHOULD send an error response with the 406 (Not Acceptable) + status code. + + + + +Fielding, et. al. Standards Track [Page 97] + +RFC 2068 HTTP/1.1 January 1997 + + + An empty Accept-Encoding value indicates none are acceptable. + +14.4 Accept-Language + + The Accept-Language request-header field is similar to Accept, but + restricts the set of natural languages that are preferred as a + response to the request. + + Accept-Language = "Accept-Language" ":" + 1#( language-range [ ";" "q" "=" qvalue ] ) + + language-range = ( ( 1*8ALPHA *( "-" 1*8ALPHA ) ) | "*" ) + + Each language-range MAY be given an associated quality value which + represents an estimate of the user's preference for the languages + specified by that range. The quality value defaults to "q=1". For + example, + + Accept-Language: da, en-gb;q=0.8, en;q=0.7 + + would mean: "I prefer Danish, but will accept British English and + other types of English." A language-range matches a language-tag if + it exactly equals the tag, or if it exactly equals a prefix of the + tag such that the first tag character following the prefix is "-". + The special range "*", if present in the Accept-Language field, + matches every tag not matched by any other range present in the + Accept-Language field. + + Note: This use of a prefix matching rule does not imply that + language tags are assigned to languages in such a way that it is + always true that if a user understands a language with a certain + tag, then this user will also understand all languages with tags + for which this tag is a prefix. The prefix rule simply allows the + use of prefix tags if this is the case. + + The language quality factor assigned to a language-tag by the + Accept-Language field is the quality value of the longest language- + range in the field that matches the language-tag. If no language- + range in the field matches the tag, the language quality factor + assigned is 0. If no Accept-Language header is present in the + request, the server SHOULD assume that all languages are equally + acceptable. If an Accept-Language header is present, then all + languages which are assigned a quality factor greater than 0 are + acceptable. + + It may be contrary to the privacy expectations of the user to send an + Accept-Language header with the complete linguistic preferences of + the user in every request. For a discussion of this issue, see + + + +Fielding, et. al. Standards Track [Page 98] + +RFC 2068 HTTP/1.1 January 1997 + + + section 15.7. + + Note: As intelligibility is highly dependent on the individual + user, it is recommended that client applications make the choice of + linguistic preference available to the user. If the choice is not + made available, then the Accept-Language header field must not be + given in the request. + +14.5 Accept-Ranges + + The Accept-Ranges response-header field allows the server to indicate + its acceptance of range requests for a resource: + + Accept-Ranges = "Accept-Ranges" ":" acceptable-ranges + + acceptable-ranges = 1#range-unit | "none" + + Origin servers that accept byte-range requests MAY send + + Accept-Ranges: bytes + + but are not required to do so. Clients MAY generate byte-range + requests without having received this header for the resource + involved. + + Servers that do not accept any kind of range request for a resource + MAY send + + Accept-Ranges: none + + to advise the client not to attempt a range request. + +14.6 Age + + The Age response-header field conveys the sender's estimate of the + amount of time since the response (or its revalidation) was generated + at the origin server. A cached response is "fresh" if its age does + not exceed its freshness lifetime. Age values are calculated as + specified in section 13.2.3. + + Age = "Age" ":" age-value + + age-value = delta-seconds + + Age values are non-negative decimal integers, representing time in + seconds. + + + + + +Fielding, et. al. Standards Track [Page 99] + +RFC 2068 HTTP/1.1 January 1997 + + + If a cache receives a value larger than the largest positive integer + it can represent, or if any of its age calculations overflows, it + MUST transmit an Age header with a value of 2147483648 (2^31). + HTTP/1.1 caches MUST send an Age header in every response. Caches + SHOULD use an arithmetic type of at least 31 bits of range. + +14.7 Allow + + The Allow entity-header field lists the set of methods supported by + the resource identified by the Request-URI. The purpose of this field + is strictly to inform the recipient of valid methods associated with + the resource. An Allow header field MUST be present in a 405 (Method + Not Allowed) response. + + Allow = "Allow" ":" 1#method + + Example of use: + + Allow: GET, HEAD, PUT + + This field cannot prevent a client from trying other methods. + However, the indications given by the Allow header field value SHOULD + be followed. The actual set of allowed methods is defined by the + origin server at the time of each request. + + The Allow header field MAY be provided with a PUT request to + recommend the methods to be supported by the new or modified + resource. The server is not required to support these methods and + SHOULD include an Allow header in the response giving the actual + supported methods. + + A proxy MUST NOT modify the Allow header field even if it does not + understand all the methods specified, since the user agent MAY have + other means of communicating with the origin server. + + The Allow header field does not indicate what methods are implemented + at the server level. Servers MAY use the Public response-header field + (section 14.35) to describe what methods are implemented on the + server as a whole. + +14.8 Authorization + + A user agent that wishes to authenticate itself with a server-- + usually, but not necessarily, after receiving a 401 response--MAY do + so by including an Authorization request-header field with the + request. The Authorization field value consists of credentials + containing the authentication information of the user agent for the + realm of the resource being requested. + + + +Fielding, et. al. Standards Track [Page 100] + +RFC 2068 HTTP/1.1 January 1997 + + + Authorization = "Authorization" ":" credentials + + HTTP access authentication is described in section 11. If a request + is authenticated and a realm specified, the same credentials SHOULD + be valid for all other requests within this realm. + + When a shared cache (see section 13.7) receives a request containing + an Authorization field, it MUST NOT return the corresponding response + as a reply to any other request, unless one of the following specific + exceptions holds: + + 1. If the response includes the "proxy-revalidate" Cache-Control + directive, the cache MAY use that response in replying to a + subsequent request, but a proxy cache MUST first revalidate it with + the origin server, using the request-headers from the new request + to allow the origin server to authenticate the new request. + 2. If the response includes the "must-revalidate" Cache-Control + directive, the cache MAY use that response in replying to a + subsequent request, but all caches MUST first revalidate it with + the origin server, using the request-headers from the new request + to allow the origin server to authenticate the new request. + 3. If the response includes the "public" Cache-Control directive, it + may be returned in reply to any subsequent request. + +14.9 Cache-Control + + The Cache-Control general-header field is used to specify directives + that MUST be obeyed by all caching mechanisms along the + request/response chain. The directives specify behavior intended to + prevent caches from adversely interfering with the request or + response. These directives typically override the default caching + algorithms. Cache directives are unidirectional in that the presence + of a directive in a request does not imply that the same directive + should be given in the response. + + Note that HTTP/1.0 caches may not implement Cache-Control and may + only implement Pragma: no-cache (see section 14.32). + + Cache directives must be passed through by a proxy or gateway + application, regardless of their significance to that application, + since the directives may be applicable to all recipients along the + request/response chain. It is not possible to specify a cache- + directive for a specific cache. + + Cache-Control = "Cache-Control" ":" 1#cache-directive + + cache-directive = cache-request-directive + | cache-response-directive + + + +Fielding, et. al. Standards Track [Page 101] + +RFC 2068 HTTP/1.1 January 1997 + + + cache-request-directive = + "no-cache" [ "=" <"> 1#field-name <"> ] + | "no-store" + | "max-age" "=" delta-seconds + | "max-stale" [ "=" delta-seconds ] + | "min-fresh" "=" delta-seconds + | "only-if-cached" + | cache-extension + + cache-response-directive = + "public" + | "private" [ "=" <"> 1#field-name <"> ] + | "no-cache" [ "=" <"> 1#field-name <"> ] + | "no-store" + | "no-transform" + | "must-revalidate" + | "proxy-revalidate" + | "max-age" "=" delta-seconds + | cache-extension + + cache-extension = token [ "=" ( token | quoted-string ) ] + + When a directive appears without any 1#field-name parameter, the + directive applies to the entire request or response. When such a + directive appears with a 1#field-name parameter, it applies only to + the named field or fields, and not to the rest of the request or + response. This mechanism supports extensibility; implementations of + future versions of the HTTP protocol may apply these directives to + header fields not defined in HTTP/1.1. + + The cache-control directives can be broken down into these general + categories: + + o Restrictions on what is cachable; these may only be imposed by the + origin server. + o Restrictions on what may be stored by a cache; these may be imposed + by either the origin server or the user agent. + o Modifications of the basic expiration mechanism; these may be + imposed by either the origin server or the user agent. + o Controls over cache revalidation and reload; these may only be + imposed by a user agent. + o Control over transformation of entities. + o Extensions to the caching system. + + + + + + + + +Fielding, et. al. Standards Track [Page 102] + +RFC 2068 HTTP/1.1 January 1997 + + +14.9.1 What is Cachable + + By default, a response is cachable if the requirements of the request + method, request header fields, and the response status indicate that + it is cachable. Section 13.4 summarizes these defaults for + cachability. The following Cache-Control response directives allow an + origin server to override the default cachability of a response: + +public + Indicates that the response is cachable by any cache, even if it + would normally be non-cachable or cachable only within a non-shared + cache. (See also Authorization, section 14.8, for additional + details.) + +private + Indicates that all or part of the response message is intended for a + single user and MUST NOT be cached by a shared cache. This allows an + origin server to state that the specified parts of the response are + intended for only one user and are not a valid response for requests + by other users. A private (non-shared) cache may cache the response. + + Note: This usage of the word private only controls where the + response may be cached, and cannot ensure the privacy of the + message content. + +no-cache + Indicates that all or part of the response message MUST NOT be cached + anywhere. This allows an origin server to prevent caching even by + caches that have been configured to return stale responses to client + requests. + + Note: Most HTTP/1.0 caches will not recognize or obey this + directive. + +14.9.2 What May be Stored by Caches + + The purpose of the no-store directive is to prevent the inadvertent + release or retention of sensitive information (for example, on backup + tapes). The no-store directive applies to the entire message, and may + be sent either in a response or in a request. If sent in a request, a + cache MUST NOT store any part of either this request or any response + to it. If sent in a response, a cache MUST NOT store any part of + either this response or the request that elicited it. This directive + applies to both non-shared and shared caches. "MUST NOT store" in + this context means that the cache MUST NOT intentionally store the + information in non-volatile storage, and MUST make a best-effort + attempt to remove the information from volatile storage as promptly + as possible after forwarding it. + + + +Fielding, et. al. Standards Track [Page 103] + +RFC 2068 HTTP/1.1 January 1997 + + + Even when this directive is associated with a response, users may + explicitly store such a response outside of the caching system (e.g., + with a "Save As" dialog). History buffers may store such responses as + part of their normal operation. + + The purpose of this directive is to meet the stated requirements of + certain users and service authors who are concerned about accidental + releases of information via unanticipated accesses to cache data + structures. While the use of this directive may improve privacy in + some cases, we caution that it is NOT in any way a reliable or + sufficient mechanism for ensuring privacy. In particular, malicious + or compromised caches may not recognize or obey this directive; and + communications networks may be vulnerable to eavesdropping. + +14.9.3 Modifications of the Basic Expiration Mechanism + + The expiration time of an entity may be specified by the origin + server using the Expires header (see section 14.21). Alternatively, + it may be specified using the max-age directive in a response. + + If a response includes both an Expires header and a max-age + directive, the max-age directive overrides the Expires header, even + if the Expires header is more restrictive. This rule allows an origin + server to provide, for a given response, a longer expiration time to + an HTTP/1.1 (or later) cache than to an HTTP/1.0 cache. This may be + useful if certain HTTP/1.0 caches improperly calculate ages or + expiration times, perhaps due to desynchronized clocks. + + Note: most older caches, not compliant with this specification, do + not implement any Cache-Control directives. An origin server + wishing to use a Cache-Control directive that restricts, but does + not prevent, caching by an HTTP/1.1-compliant cache may exploit the + requirement that the max-age directive overrides the Expires + header, and the fact that non-HTTP/1.1-compliant caches do not + observe the max-age directive. + + Other directives allow an user agent to modify the basic expiration + mechanism. These directives may be specified on a request: + + max-age + Indicates that the client is willing to accept a response whose age + is no greater than the specified time in seconds. Unless max-stale + directive is also included, the client is not willing to accept a + stale response. + + min-fresh + Indicates that the client is willing to accept a response whose + freshness lifetime is no less than its current age plus the + + + +Fielding, et. al. Standards Track [Page 104] + +RFC 2068 HTTP/1.1 January 1997 + + + specified time in seconds. That is, the client wants a response + that will still be fresh for at least the specified number of + seconds. + + max-stale + Indicates that the client is willing to accept a response that has + exceeded its expiration time. If max-stale is assigned a value, + then the client is willing to accept a response that has exceeded + its expiration time by no more than the specified number of + seconds. If no value is assigned to max-stale, then the client is + willing to accept a stale response of any age. + + If a cache returns a stale response, either because of a max-stale + directive on a request, or because the cache is configured to + override the expiration time of a response, the cache MUST attach a + Warning header to the stale response, using Warning 10 (Response is + stale). + +14.9.4 Cache Revalidation and Reload Controls + + Sometimes an user agent may want or need to insist that a cache + revalidate its cache entry with the origin server (and not just with + the next cache along the path to the origin server), or to reload its + cache entry from the origin server. End-to-end revalidation may be + necessary if either the cache or the origin server has overestimated + the expiration time of the cached response. End-to-end reload may be + necessary if the cache entry has become corrupted for some reason. + + End-to-end revalidation may be requested either when the client does + not have its own local cached copy, in which case we call it + "unspecified end-to-end revalidation", or when the client does have a + local cached copy, in which case we call it "specific end-to-end + revalidation." + + The client can specify these three kinds of action using Cache- + Control request directives: + + End-to-end reload + The request includes a "no-cache" Cache-Control directive or, for + compatibility with HTTP/1.0 clients, "Pragma: no-cache". No field + names may be included with the no-cache directive in a request. The + server MUST NOT use a cached copy when responding to such a + request. + + Specific end-to-end revalidation + The request includes a "max-age=0" Cache-Control directive, which + forces each cache along the path to the origin server to revalidate + its own entry, if any, with the next cache or server. The initial + + + +Fielding, et. al. Standards Track [Page 105] + +RFC 2068 HTTP/1.1 January 1997 + + + request includes a cache-validating conditional with the client's + current validator. + + Unspecified end-to-end revalidation + The request includes "max-age=0" Cache-Control directive, which + forces each cache along the path to the origin server to revalidate + its own entry, if any, with the next cache or server. The initial + request does not include a cache-validating conditional; the first + cache along the path (if any) that holds a cache entry for this + resource includes a cache-validating conditional with its current + validator. + + When an intermediate cache is forced, by means of a max-age=0 + directive, to revalidate its own cache entry, and the client has + supplied its own validator in the request, the supplied validator may + differ from the validator currently stored with the cache entry. In + this case, the cache may use either validator in making its own + request without affecting semantic transparency. + + However, the choice of validator may affect performance. The best + approach is for the intermediate cache to use its own validator when + making its request. If the server replies with 304 (Not Modified), + then the cache should return its now validated copy to the client + with a 200 (OK) response. If the server replies with a new entity and + cache validator, however, the intermediate cache should compare the + returned validator with the one provided in the client's request, + using the strong comparison function. If the client's validator is + equal to the origin server's, then the intermediate cache simply + returns 304 (Not Modified). Otherwise, it returns the new entity with + a 200 (OK) response. + + If a request includes the no-cache directive, it should not include + min-fresh, max-stale, or max-age. + + In some cases, such as times of extremely poor network connectivity, + a client may want a cache to return only those responses that it + currently has stored, and not to reload or revalidate with the origin + server. To do this, the client may include the only-if-cached + directive in a request. If it receives this directive, a cache SHOULD + either respond using a cached entry that is consistent with the other + constraints of the request, or respond with a 504 (Gateway Timeout) + status. However, if a group of caches is being operated as a unified + system with good internal connectivity, such a request MAY be + forwarded within that group of caches. + + Because a cache may be configured to ignore a server's specified + expiration time, and because a client request may include a max-stale + directive (which has a similar effect), the protocol also includes a + + + +Fielding, et. al. Standards Track [Page 106] + +RFC 2068 HTTP/1.1 January 1997 + + + mechanism for the origin server to require revalidation of a cache + entry on any subsequent use. When the must-revalidate directive is + present in a response received by a cache, that cache MUST NOT use + the entry after it becomes stale to respond to a subsequent request + without first revalidating it with the origin server. (I.e., the + cache must do an end-to-end revalidation every time, if, based solely + on the origin server's Expires or max-age value, the cached response + is stale.) + + The must-revalidate directive is necessary to support reliable + operation for certain protocol features. In all circumstances an + HTTP/1.1 cache MUST obey the must-revalidate directive; in + particular, if the cache cannot reach the origin server for any + reason, it MUST generate a 504 (Gateway Timeout) response. + + Servers should send the must-revalidate directive if and only if + failure to revalidate a request on the entity could result in + incorrect operation, such as a silently unexecuted financial + transaction. Recipients MUST NOT take any automated action that + violates this directive, and MUST NOT automatically provide an + unvalidated copy of the entity if revalidation fails. + + Although this is not recommended, user agents operating under severe + connectivity constraints may violate this directive but, if so, MUST + explicitly warn the user that an unvalidated response has been + provided. The warning MUST be provided on each unvalidated access, + and SHOULD require explicit user confirmation. + + The proxy-revalidate directive has the same meaning as the must- + revalidate directive, except that it does not apply to non-shared + user agent caches. It can be used on a response to an authenticated + request to permit the user's cache to store and later return the + response without needing to revalidate it (since it has already been + authenticated once by that user), while still requiring proxies that + service many users to revalidate each time (in order to make sure + that each user has been authenticated). Note that such authenticated + responses also need the public cache control directive in order to + allow them to be cached at all. + +14.9.5 No-Transform Directive + + Implementers of intermediate caches (proxies) have found it useful to + convert the media type of certain entity bodies. A proxy might, for + example, convert between image formats in order to save cache space + or to reduce the amount of traffic on a slow link. HTTP has to date + been silent on these transformations. + + + + + +Fielding, et. al. Standards Track [Page 107] + +RFC 2068 HTTP/1.1 January 1997 + + + Serious operational problems have already occurred, however, when + these transformations have been applied to entity bodies intended for + certain kinds of applications. For example, applications for medical + imaging, scientific data analysis and those using end-to-end + authentication, all depend on receiving an entity body that is bit + for bit identical to the original entity-body. + + Therefore, if a response includes the no-transform directive, an + intermediate cache or proxy MUST NOT change those headers that are + listed in section 13.5.2 as being subject to the no-transform + directive. This implies that the cache or proxy must not change any + aspect of the entity-body that is specified by these headers. + +14.9.6 Cache Control Extensions + + The Cache-Control header field can be extended through the use of one + or more cache-extension tokens, each with an optional assigned value. + Informational extensions (those which do not require a change in + cache behavior) may be added without changing the semantics of other + directives. Behavioral extensions are designed to work by acting as + modifiers to the existing base of cache directives. Both the new + directive and the standard directive are supplied, such that + applications which do not understand the new directive will default + to the behavior specified by the standard directive, and those that + understand the new directive will recognize it as modifying the + requirements associated with the standard directive. In this way, + extensions to the Cache-Control directives can be made without + requiring changes to the base protocol. + + This extension mechanism depends on a HTTP cache obeying all of the + cache-control directives defined for its native HTTP-version, obeying + certain extensions, and ignoring all directives that it does not + understand. + + For example, consider a hypothetical new response directive called + "community" which acts as a modifier to the "private" directive. We + define this new directive to mean that, in addition to any non-shared + cache, any cache which is shared only by members of the community + named within its value may cache the response. An origin server + wishing to allow the "UCI" community to use an otherwise private + response in their shared cache(s) may do so by including + + Cache-Control: private, community="UCI" + + A cache seeing this header field will act correctly even if the cache + does not understand the "community" cache-extension, since it will + also see and understand the "private" directive and thus default to + the safe behavior. + + + +Fielding, et. al. Standards Track [Page 108] + +RFC 2068 HTTP/1.1 January 1997 + + + Unrecognized cache-directives MUST be ignored; it is assumed that any + cache-directive likely to be unrecognized by an HTTP/1.1 cache will + be combined with standard directives (or the response's default + cachability) such that the cache behavior will remain minimally + correct even if the cache does not understand the extension(s). + +14.10 Connection + + The Connection general-header field allows the sender to specify + options that are desired for that particular connection and MUST NOT + be communicated by proxies over further connections. + + The Connection header has the following grammar: + + Connection-header = "Connection" ":" 1#(connection-token) + connection-token = token + + HTTP/1.1 proxies MUST parse the Connection header field before a + message is forwarded and, for each connection-token in this field, + remove any header field(s) from the message with the same name as the + connection-token. Connection options are signaled by the presence of + a connection-token in the Connection header field, not by any + corresponding additional header field(s), since the additional header + field may not be sent if there are no parameters associated with that + connection option. HTTP/1.1 defines the "close" connection option + for the sender to signal that the connection will be closed after + completion of the response. For example, + + Connection: close + + in either the request or the response header fields indicates that + the connection should not be considered `persistent' (section 8.1) + after the current request/response is complete. + + HTTP/1.1 applications that do not support persistent connections MUST + include the "close" connection option in every message. + +14.11 Content-Base + + The Content-Base entity-header field may be used to specify the base + URI for resolving relative URLs within the entity. This header field + is described as Base in RFC 1808, which is expected to be revised. + + Content-Base = "Content-Base" ":" absoluteURI + + If no Content-Base field is present, the base URI of an entity is + defined either by its Content-Location (if that Content-Location URI + is an absolute URI) or the URI used to initiate the request, in that + + + +Fielding, et. al. Standards Track [Page 109] + +RFC 2068 HTTP/1.1 January 1997 + + + order of precedence. Note, however, that the base URI of the contents + within the entity-body may be redefined within that entity-body. + +14.12 Content-Encoding + + The Content-Encoding entity-header field is used as a modifier to the + media-type. When present, its value indicates what additional content + codings have been applied to the entity-body, and thus what decoding + mechanisms MUST be applied in order to obtain the media-type + referenced by the Content-Type header field. Content-Encoding is + primarily used to allow a document to be compressed without losing + the identity of its underlying media type. + + Content-Encoding = "Content-Encoding" ":" 1#content-coding + + Content codings are defined in section 3.5. An example of its use is + + Content-Encoding: gzip + + The Content-Encoding is a characteristic of the entity identified by + the Request-URI. Typically, the entity-body is stored with this + encoding and is only decoded before rendering or analogous usage. + + If multiple encodings have been applied to an entity, the content + codings MUST be listed in the order in which they were applied. + + Additional information about the encoding parameters MAY be provided + by other entity-header fields not defined by this specification. + +14.13 Content-Language + + The Content-Language entity-header field describes the natural + language(s) of the intended audience for the enclosed entity. Note + that this may not be equivalent to all the languages used within the + entity-body. + + Content-Language = "Content-Language" ":" 1#language-tag + + Language tags are defined in section 3.10. The primary purpose of + Content-Language is to allow a user to identify and differentiate + entities according to the user's own preferred language. Thus, if the + body content is intended only for a Danish-literate audience, the + appropriate field is + + Content-Language: da + + If no Content-Language is specified, the default is that the content + is intended for all language audiences. This may mean that the sender + + + +Fielding, et. al. Standards Track [Page 110] + +RFC 2068 HTTP/1.1 January 1997 + + + does not consider it to be specific to any natural language, or that + the sender does not know for which language it is intended. + + Multiple languages MAY be listed for content that is intended for + multiple audiences. For example, a rendition of the "Treaty of + Waitangi," presented simultaneously in the original Maori and English + versions, would call for + + Content-Language: mi, en + + However, just because multiple languages are present within an entity + does not mean that it is intended for multiple linguistic audiences. + An example would be a beginner's language primer, such as "A First + Lesson in Latin," which is clearly intended to be used by an + English-literate audience. In this case, the Content-Language should + only include "en". + + Content-Language may be applied to any media type -- it is not + limited to textual documents. + +14.14 Content-Length + + The Content-Length entity-header field indicates the size of the + message-body, in decimal number of octets, sent to the recipient or, + in the case of the HEAD method, the size of the entity-body that + would have been sent had the request been a GET. + + Content-Length = "Content-Length" ":" 1*DIGIT + + An example is + + Content-Length: 3495 + + Applications SHOULD use this field to indicate the size of the + message-body to be transferred, regardless of the media type of the + entity. It must be possible for the recipient to reliably determine + the end of HTTP/1.1 requests containing an entity-body, e.g., because + the request has a valid Content-Length field, uses Transfer-Encoding: + chunked or a multipart body. + + Any Content-Length greater than or equal to zero is a valid value. + Section 4.4 describes how to determine the length of a message-body + if a Content-Length is not given. + + + + + + + + +Fielding, et. al. Standards Track [Page 111] + +RFC 2068 HTTP/1.1 January 1997 + + + Note: The meaning of this field is significantly different from the + corresponding definition in MIME, where it is an optional field + used within the "message/external-body" content-type. In HTTP, it + SHOULD be sent whenever the message's length can be determined + prior to being transferred. + +14.15 Content-Location + + The Content-Location entity-header field may be used to supply the + resource location for the entity enclosed in the message. In the case + where a resource has multiple entities associated with it, and those + entities actually have separate locations by which they might be + individually accessed, the server should provide a Content-Location + for the particular variant which is returned. In addition, a server + SHOULD provide a Content-Location for the resource corresponding to + the response entity. + + Content-Location = "Content-Location" ":" + ( absoluteURI | relativeURI ) + + If no Content-Base header field is present, the value of Content- + Location also defines the base URL for the entity (see section + 14.11). + + The Content-Location value is not a replacement for the original + requested URI; it is only a statement of the location of the resource + corresponding to this particular entity at the time of the request. + Future requests MAY use the Content-Location URI if the desire is to + identify the source of that particular entity. + + A cache cannot assume that an entity with a Content-Location + different from the URI used to retrieve it can be used to respond to + later requests on that Content-Location URI. However, the Content- + Location can be used to differentiate between multiple entities + retrieved from a single requested resource, as described in section + 13.6. + + If the Content-Location is a relative URI, the URI is interpreted + relative to any Content-Base URI provided in the response. If no + Content-Base is provided, the relative URI is interpreted relative to + the Request-URI. + + + + + + + + + + +Fielding, et. al. Standards Track [Page 112] + +RFC 2068 HTTP/1.1 January 1997 + + +14.16 Content-MD5 + + The Content-MD5 entity-header field, as defined in RFC 1864 [23], is + an MD5 digest of the entity-body for the purpose of providing an + end-to-end message integrity check (MIC) of the entity-body. (Note: a + MIC is good for detecting accidental modification of the entity-body + in transit, but is not proof against malicious attacks.) + + Content-MD5 = "Content-MD5" ":" md5-digest + + md5-digest = <base64 of 128 bit MD5 digest as per RFC 1864> + + The Content-MD5 header field may be generated by an origin server to + function as an integrity check of the entity-body. Only origin + servers may generate the Content-MD5 header field; proxies and + gateways MUST NOT generate it, as this would defeat its value as an + end-to-end integrity check. Any recipient of the entity-body, + including gateways and proxies, MAY check that the digest value in + this header field matches that of the entity-body as received. + + The MD5 digest is computed based on the content of the entity-body, + including any Content-Encoding that has been applied, but not + including any Transfer-Encoding that may have been applied to the + message-body. If the message is received with a Transfer-Encoding, + that encoding must be removed prior to checking the Content-MD5 value + against the received entity. + + This has the result that the digest is computed on the octets of the + entity-body exactly as, and in the order that, they would be sent if + no Transfer-Encoding were being applied. + + HTTP extends RFC 1864 to permit the digest to be computed for MIME + composite media-types (e.g., multipart/* and message/rfc822), but + this does not change how the digest is computed as defined in the + preceding paragraph. + + Note: There are several consequences of this. The entity-body for + composite types may contain many body-parts, each with its own MIME + and HTTP headers (including Content-MD5, Content-Transfer-Encoding, + and Content-Encoding headers). If a body-part has a Content- + Transfer-Encoding or Content-Encoding header, it is assumed that + the content of the body-part has had the encoding applied, and the + body-part is included in the Content-MD5 digest as is -- i.e., + after the application. The Transfer-Encoding header field is not + allowed within body-parts. + + Note: while the definition of Content-MD5 is exactly the same for + HTTP as in RFC 1864 for MIME entity-bodies, there are several ways + + + +Fielding, et. al. Standards Track [Page 113] + +RFC 2068 HTTP/1.1 January 1997 + + + in which the application of Content-MD5 to HTTP entity-bodies + differs from its application to MIME entity-bodies. One is that + HTTP, unlike MIME, does not use Content-Transfer-Encoding, and does + use Transfer-Encoding and Content-Encoding. Another is that HTTP + more frequently uses binary content types than MIME, so it is worth + noting that, in such cases, the byte order used to compute the + digest is the transmission byte order defined for the type. Lastly, + HTTP allows transmission of text types with any of several line + break conventions and not just the canonical form using CRLF. + Conversion of all line breaks to CRLF should not be done before + computing or checking the digest: the line break convention used in + the text actually transmitted should be left unaltered when + computing the digest. + +14.17 Content-Range + + The Content-Range entity-header is sent with a partial entity-body to + specify where in the full entity-body the partial body should be + inserted. It also indicates the total size of the full entity-body. + When a server returns a partial response to a client, it must + describe both the extent of the range covered by the response, and + the length of the entire entity-body. + + Content-Range = "Content-Range" ":" content-range-spec + + content-range-spec = byte-content-range-spec + + byte-content-range-spec = bytes-unit SP first-byte-pos "-" + last-byte-pos "/" entity-length + + entity-length = 1*DIGIT + + Unlike byte-ranges-specifier values, a byte-content-range-spec may + only specify one range, and must contain absolute byte positions for + both the first and last byte of the range. + + A byte-content-range-spec whose last-byte-pos value is less than its + first-byte-pos value, or whose entity-length value is less than or + equal to its last-byte-pos value, is invalid. The recipient of an + invalid byte-content-range-spec MUST ignore it and any content + transferred along with it. + + + + + + + + + + +Fielding, et. al. Standards Track [Page 114] + +RFC 2068 HTTP/1.1 January 1997 + + + Examples of byte-content-range-spec values, assuming that the entity + contains a total of 1234 bytes: + + o The first 500 bytes: + + bytes 0-499/1234 + + o The second 500 bytes: + + bytes 500-999/1234 + + o All except for the first 500 bytes: + + bytes 500-1233/1234 + + o The last 500 bytes: + + bytes 734-1233/1234 + + When an HTTP message includes the content of a single range (for + example, a response to a request for a single range, or to a request + for a set of ranges that overlap without any holes), this content is + transmitted with a Content-Range header, and a Content-Length header + showing the number of bytes actually transferred. For example, + + HTTP/1.1 206 Partial content + Date: Wed, 15 Nov 1995 06:25:24 GMT + Last-modified: Wed, 15 Nov 1995 04:58:08 GMT + Content-Range: bytes 21010-47021/47022 + Content-Length: 26012 + Content-Type: image/gif + + When an HTTP message includes the content of multiple ranges (for + example, a response to a request for multiple non-overlapping + ranges), these are transmitted as a multipart MIME message. The + multipart MIME content-type used for this purpose is defined in this + specification to be "multipart/byteranges". See appendix 19.2 for its + definition. + + A client that cannot decode a MIME multipart/byteranges message + should not ask for multiple byte-ranges in a single request. + + When a client requests multiple byte-ranges in one request, the + server SHOULD return them in the order that they appeared in the + request. + + If the server ignores a byte-range-spec because it is invalid, the + server should treat the request as if the invalid Range header field + + + +Fielding, et. al. Standards Track [Page 115] + +RFC 2068 HTTP/1.1 January 1997 + + + did not exist. (Normally, this means return a 200 response containing + the full entity). The reason is that the only time a client will make + such an invalid request is when the entity is smaller than the entity + retrieved by a prior request. + +14.18 Content-Type + + The Content-Type entity-header field indicates the media type of the + entity-body sent to the recipient or, in the case of the HEAD method, + the media type that would have been sent had the request been a GET. + + Content-Type = "Content-Type" ":" media-type + Media types are defined in section 3.7. An example of the field is + + Content-Type: text/html; charset=ISO-8859-4 + + Further discussion of methods for identifying the media type of an + entity is provided in section 7.2.1. + +14.19 Date + + The Date general-header field represents the date and time at which + the message was originated, having the same semantics as orig-date in + RFC 822. The field value is an HTTP-date, as described in section + 3.3.1. + + Date = "Date" ":" HTTP-date + + An example is + + Date: Tue, 15 Nov 1994 08:12:31 GMT + + If a message is received via direct connection with the user agent + (in the case of requests) or the origin server (in the case of + responses), then the date can be assumed to be the current date at + the receiving end. However, since the date--as it is believed by the + origin--is important for evaluating cached responses, origin servers + MUST include a Date header field in all responses. Clients SHOULD + only send a Date header field in messages that include an entity- + body, as in the case of the PUT and POST requests, and even then it + is optional. A received message which does not have a Date header + field SHOULD be assigned one by the recipient if the message will be + cached by that recipient or gatewayed via a protocol which requires a + Date. + + + + + + + +Fielding, et. al. Standards Track [Page 116] + +RFC 2068 HTTP/1.1 January 1997 + + + In theory, the date SHOULD represent the moment just before the + entity is generated. In practice, the date can be generated at any + time during the message origination without affecting its semantic + value. + + The format of the Date is an absolute date and time as defined by + HTTP-date in section 3.3; it MUST be sent in RFC1123 [8]-date format. + +14.20 ETag + + The ETag entity-header field defines the entity tag for the + associated entity. The headers used with entity tags are described in + sections 14.20, 14.25, 14.26 and 14.43. The entity tag may be used + for comparison with other entities from the same resource (see + section 13.3.2). + + ETag = "ETag" ":" entity-tag + + Examples: + + ETag: "xyzzy" + ETag: W/"xyzzy" + ETag: "" + +14.21 Expires + + The Expires entity-header field gives the date/time after which the + response should be considered stale. A stale cache entry may not + normally be returned by a cache (either a proxy cache or an user + agent cache) unless it is first validated with the origin server (or + with an intermediate cache that has a fresh copy of the entity). See + section 13.2 for further discussion of the expiration model. + + The presence of an Expires field does not imply that the original + resource will change or cease to exist at, before, or after that + time. + + The format is an absolute date and time as defined by HTTP-date in + section 3.3; it MUST be in RFC1123-date format: + + Expires = "Expires" ":" HTTP-date + + + + + + + + + + +Fielding, et. al. Standards Track [Page 117] + +RFC 2068 HTTP/1.1 January 1997 + + + An example of its use is + + Expires: Thu, 01 Dec 1994 16:00:00 GMT + + Note: if a response includes a Cache-Control field with the max-age + directive, that directive overrides the Expires field. + + HTTP/1.1 clients and caches MUST treat other invalid date formats, + especially including the value "0", as in the past (i.e., "already + expired"). + + To mark a response as "already expired," an origin server should use + an Expires date that is equal to the Date header value. (See the + rules for expiration calculations in section 13.2.4.) + + To mark a response as "never expires," an origin server should use an + Expires date approximately one year from the time the response is + sent. HTTP/1.1 servers should not send Expires dates more than one + year in the future. + + The presence of an Expires header field with a date value of some + time in the future on an response that otherwise would by default be + non-cacheable indicates that the response is cachable, unless + indicated otherwise by a Cache-Control header field (section 14.9). + +14.22 From + + The From request-header field, if given, SHOULD contain an Internet + e-mail address for the human user who controls the requesting user + agent. The address SHOULD be machine-usable, as defined by mailbox + in RFC 822 (as updated by RFC 1123 ): + + From = "From" ":" mailbox + + An example is: + + From: [email protected] + + This header field MAY be used for logging purposes and as a means for + identifying the source of invalid or unwanted requests. It SHOULD NOT + be used as an insecure form of access protection. The interpretation + of this field is that the request is being performed on behalf of the + person given, who accepts responsibility for the method performed. In + particular, robot agents SHOULD include this header so that the + person responsible for running the robot can be contacted if problems + occur on the receiving end. + + + + + +Fielding, et. al. Standards Track [Page 118] + +RFC 2068 HTTP/1.1 January 1997 + + + The Internet e-mail address in this field MAY be separate from the + Internet host which issued the request. For example, when a request + is passed through a proxy the original issuer's address SHOULD be + used. + + Note: The client SHOULD not send the From header field without the + user's approval, as it may conflict with the user's privacy + interests or their site's security policy. It is strongly + recommended that the user be able to disable, enable, and modify + the value of this field at any time prior to a request. + +14.23 Host + + The Host request-header field specifies the Internet host and port + number of the resource being requested, as obtained from the original + URL given by the user or referring resource (generally an HTTP URL, + as described in section 3.2.2). The Host field value MUST represent + the network location of the origin server or gateway given by the + original URL. This allows the origin server or gateway to + differentiate between internally-ambiguous URLs, such as the root "/" + URL of a server for multiple host names on a single IP address. + + Host = "Host" ":" host [ ":" port ] ; Section 3.2.2 + + A "host" without any trailing port information implies the default + port for the service requested (e.g., "80" for an HTTP URL). For + example, a request on the origin server for + <http://www.w3.org/pub/WWW/> MUST include: + + GET /pub/WWW/ HTTP/1.1 + Host: www.w3.org + + A client MUST include a Host header field in all HTTP/1.1 request + messages on the Internet (i.e., on any message corresponding to a + request for a URL which includes an Internet host address for the + service being requested). If the Host field is not already present, + an HTTP/1.1 proxy MUST add a Host field to the request message prior + to forwarding it on the Internet. All Internet-based HTTP/1.1 servers + MUST respond with a 400 status code to any HTTP/1.1 request message + which lacks a Host header field. + + See sections 5.2 and 19.5.1 for other requirements relating to Host. + +14.24 If-Modified-Since + + The If-Modified-Since request-header field is used with the GET + method to make it conditional: if the requested variant has not been + modified since the time specified in this field, an entity will not + + + +Fielding, et. al. Standards Track [Page 119] + +RFC 2068 HTTP/1.1 January 1997 + + + be returned from the server; instead, a 304 (not modified) response + will be returned without any message-body. + + If-Modified-Since = "If-Modified-Since" ":" HTTP-date + + An example of the field is: + + If-Modified-Since: Sat, 29 Oct 1994 19:43:31 GMT + + A GET method with an If-Modified-Since header and no Range header + requests that the identified entity be transferred only if it has + been modified since the date given by the If-Modified-Since header. + The algorithm for determining this includes the following cases: + + a)If the request would normally result in anything other than a 200 + (OK) status, or if the passed If-Modified-Since date is invalid, the + response is exactly the same as for a normal GET. A date which is + later than the server's current time is invalid. + + b)If the variant has been modified since the If-Modified-Since date, + the response is exactly the same as for a normal GET. + + c)If the variant has not been modified since a valid If-Modified-Since + date, the server MUST return a 304 (Not Modified) response. + + The purpose of this feature is to allow efficient updates of cached + information with a minimum amount of transaction overhead. + + Note that the Range request-header field modifies the meaning of + If-Modified-Since; see section 14.36 for full details. + + Note that If-Modified-Since times are interpreted by the server, + whose clock may not be synchronized with the client. + + Note that if a client uses an arbitrary date in the If-Modified-Since + header instead of a date taken from the Last-Modified header for the + same request, the client should be aware of the fact that this date + is interpreted in the server's understanding of time. The client + should consider unsynchronized clocks and rounding problems due to + the different encodings of time between the client and server. This + includes the possibility of race conditions if the document has + changed between the time it was first requested and the If-Modified- + Since date of a subsequent request, and the possibility of clock- + skew-related problems if the If-Modified-Since date is derived from + the client's clock without correction to the server's clock. + Corrections for different time bases between client and server are at + best approximate due to network latency. + + + + +Fielding, et. al. Standards Track [Page 120] + +RFC 2068 HTTP/1.1 January 1997 + + +14.25 If-Match + + The If-Match request-header field is used with a method to make it + conditional. A client that has one or more entities previously + obtained from the resource can verify that one of those entities is + current by including a list of their associated entity tags in the + If-Match header field. The purpose of this feature is to allow + efficient updates of cached information with a minimum amount of + transaction overhead. It is also used, on updating requests, to + prevent inadvertent modification of the wrong version of a resource. + As a special case, the value "*" matches any current entity of the + resource. + + If-Match = "If-Match" ":" ( "*" | 1#entity-tag ) + + If any of the entity tags match the entity tag of the entity that + would have been returned in the response to a similar GET request + (without the If-Match header) on that resource, or if "*" is given + and any current entity exists for that resource, then the server MAY + perform the requested method as if the If-Match header field did not + exist. + + A server MUST use the strong comparison function (see section 3.11) + to compare the entity tags in If-Match. + + If none of the entity tags match, or if "*" is given and no current + entity exists, the server MUST NOT perform the requested method, and + MUST return a 412 (Precondition Failed) response. This behavior is + most useful when the client wants to prevent an updating method, such + as PUT, from modifying a resource that has changed since the client + last retrieved it. + + If the request would, without the If-Match header field, result in + anything other than a 2xx status, then the If-Match header MUST be + ignored. + + The meaning of "If-Match: *" is that the method SHOULD be performed + if the representation selected by the origin server (or by a cache, + possibly using the Vary mechanism, see section 14.43) exists, and + MUST NOT be performed if the representation does not exist. + + + + + + + + + + + +Fielding, et. al. Standards Track [Page 121] + +RFC 2068 HTTP/1.1 January 1997 + + + A request intended to update a resource (e.g., a PUT) MAY include an + If-Match header field to signal that the request method MUST NOT be + applied if the entity corresponding to the If-Match value (a single + entity tag) is no longer a representation of that resource. This + allows the user to indicate that they do not wish the request to be + successful if the resource has been changed without their knowledge. + Examples: + + If-Match: "xyzzy" + If-Match: "xyzzy", "r2d2xxxx", "c3piozzzz" + If-Match: * + +14.26 If-None-Match + + The If-None-Match request-header field is used with a method to make + it conditional. A client that has one or more entities previously + obtained from the resource can verify that none of those entities is + current by including a list of their associated entity tags in the + If-None-Match header field. The purpose of this feature is to allow + efficient updates of cached information with a minimum amount of + transaction overhead. It is also used, on updating requests, to + prevent inadvertent modification of a resource which was not known to + exist. + + As a special case, the value "*" matches any current entity of the + resource. + + If-None-Match = "If-None-Match" ":" ( "*" | 1#entity-tag ) + + If any of the entity tags match the entity tag of the entity that + would have been returned in the response to a similar GET request + (without the If-None-Match header) on that resource, or if "*" is + given and any current entity exists for that resource, then the + server MUST NOT perform the requested method. Instead, if the request + method was GET or HEAD, the server SHOULD respond with a 304 (Not + Modified) response, including the cache-related entity-header fields + (particularly ETag) of one of the entities that matched. For all + other request methods, the server MUST respond with a status of 412 + (Precondition Failed). + + See section 13.3.3 for rules on how to determine if two entity tags + match. The weak comparison function can only be used with GET or HEAD + requests. + + If none of the entity tags match, or if "*" is given and no current + entity exists, then the server MAY perform the requested method as if + the If-None-Match header field did not exist. + + + + +Fielding, et. al. Standards Track [Page 122] + +RFC 2068 HTTP/1.1 January 1997 + + + If the request would, without the If-None-Match header field, result + in anything other than a 2xx status, then the If-None-Match header + MUST be ignored. + + The meaning of "If-None-Match: *" is that the method MUST NOT be + performed if the representation selected by the origin server (or by + a cache, possibly using the Vary mechanism, see section 14.43) + exists, and SHOULD be performed if the representation does not exist. + This feature may be useful in preventing races between PUT + operations. + + Examples: + + If-None-Match: "xyzzy" + If-None-Match: W/"xyzzy" + If-None-Match: "xyzzy", "r2d2xxxx", "c3piozzzz" + If-None-Match: W/"xyzzy", W/"r2d2xxxx", W/"c3piozzzz" + If-None-Match: * + +14.27 If-Range + + If a client has a partial copy of an entity in its cache, and wishes + to have an up-to-date copy of the entire entity in its cache, it + could use the Range request-header with a conditional GET (using + either or both of If-Unmodified-Since and If-Match.) However, if the + condition fails because the entity has been modified, the client + would then have to make a second request to obtain the entire current + entity-body. + + The If-Range header allows a client to "short-circuit" the second + request. Informally, its meaning is `if the entity is unchanged, send + me the part(s) that I am missing; otherwise, send me the entire new + entity.' + + If-Range = "If-Range" ":" ( entity-tag | HTTP-date ) + + If the client has no entity tag for an entity, but does have a Last- + Modified date, it may use that date in a If-Range header. (The server + can distinguish between a valid HTTP-date and any form of entity-tag + by examining no more than two characters.) The If-Range header should + only be used together with a Range header, and must be ignored if the + request does not include a Range header, or if the server does not + support the sub-range operation. + + + + + + + + +Fielding, et. al. Standards Track [Page 123] + +RFC 2068 HTTP/1.1 January 1997 + + + If the entity tag given in the If-Range header matches the current + entity tag for the entity, then the server should provide the + specified sub-range of the entity using a 206 (Partial content) + response. If the entity tag does not match, then the server should + return the entire entity using a 200 (OK) response. + +14.28 If-Unmodified-Since + + The If-Unmodified-Since request-header field is used with a method to + make it conditional. If the requested resource has not been modified + since the time specified in this field, the server should perform the + requested operation as if the If-Unmodified-Since header were not + present. + + If the requested variant has been modified since the specified time, + the server MUST NOT perform the requested operation, and MUST return + a 412 (Precondition Failed). + + If-Unmodified-Since = "If-Unmodified-Since" ":" HTTP-date + + An example of the field is: + + If-Unmodified-Since: Sat, 29 Oct 1994 19:43:31 GMT + + If the request normally (i.e., without the If-Unmodified-Since + header) would result in anything other than a 2xx status, the If- + Unmodified-Since header should be ignored. + + If the specified date is invalid, the header is ignored. + +14.29 Last-Modified + + The Last-Modified entity-header field indicates the date and time at + which the origin server believes the variant was last modified. + + Last-Modified = "Last-Modified" ":" HTTP-date + + An example of its use is + + Last-Modified: Tue, 15 Nov 1994 12:45:26 GMT + + The exact meaning of this header field depends on the implementation + of the origin server and the nature of the original resource. For + files, it may be just the file system last-modified time. For + entities with dynamically included parts, it may be the most recent + of the set of last-modify times for its component parts. For database + gateways, it may be the last-update time stamp of the record. For + virtual objects, it may be the last time the internal state changed. + + + +Fielding, et. al. Standards Track [Page 124] + +RFC 2068 HTTP/1.1 January 1997 + + + An origin server MUST NOT send a Last-Modified date which is later + than the server's time of message origination. In such cases, where + the resource's last modification would indicate some time in the + future, the server MUST replace that date with the message + origination date. + + An origin server should obtain the Last-Modified value of the entity + as close as possible to the time that it generates the Date value of + its response. This allows a recipient to make an accurate assessment + of the entity's modification time, especially if the entity changes + near the time that the response is generated. + + HTTP/1.1 servers SHOULD send Last-Modified whenever feasible. + +14.30 Location + + The Location response-header field is used to redirect the recipient + to a location other than the Request-URI for completion of the + request or identification of a new resource. For 201 (Created) + responses, the Location is that of the new resource which was created + by the request. For 3xx responses, the location SHOULD indicate the + server's preferred URL for automatic redirection to the resource. The + field value consists of a single absolute URL. + + Location = "Location" ":" absoluteURI + + An example is + + Location: http://www.w3.org/pub/WWW/People.html + + Note: The Content-Location header field (section 14.15) differs + from Location in that the Content-Location identifies the original + location of the entity enclosed in the request. It is therefore + possible for a response to contain header fields for both Location + and Content-Location. Also see section 13.10 for cache requirements + of some methods. + +14.31 Max-Forwards + + The Max-Forwards request-header field may be used with the TRACE + method (section 14.31) to limit the number of proxies or gateways + that can forward the request to the next inbound server. This can be + useful when the client is attempting to trace a request chain which + appears to be failing or looping in mid-chain. + + Max-Forwards = "Max-Forwards" ":" 1*DIGIT + + + + + +Fielding, et. al. Standards Track [Page 125] + +RFC 2068 HTTP/1.1 January 1997 + + + The Max-Forwards value is a decimal integer indicating the remaining + number of times this request message may be forwarded. + + Each proxy or gateway recipient of a TRACE request containing a Max- + Forwards header field SHOULD check and update its value prior to + forwarding the request. If the received value is zero (0), the + recipient SHOULD NOT forward the request; instead, it SHOULD respond + as the final recipient with a 200 (OK) response containing the + received request message as the response entity-body (as described in + section 9.8). If the received Max-Forwards value is greater than + zero, then the forwarded message SHOULD contain an updated Max- + Forwards field with a value decremented by one (1). + + The Max-Forwards header field SHOULD be ignored for all other methods + defined by this specification and for any extension methods for which + it is not explicitly referred to as part of that method definition. + +14.32 Pragma + + The Pragma general-header field is used to include implementation- + specific directives that may apply to any recipient along the + request/response chain. All pragma directives specify optional + behavior from the viewpoint of the protocol; however, some systems + MAY require that behavior be consistent with the directives. + + Pragma = "Pragma" ":" 1#pragma-directive + + pragma-directive = "no-cache" | extension-pragma + extension-pragma = token [ "=" ( token | quoted-string ) ] + + When the no-cache directive is present in a request message, an + application SHOULD forward the request toward the origin server even + if it has a cached copy of what is being requested. This pragma + directive has the same semantics as the no-cache cache-directive (see + section 14.9) and is defined here for backwards compatibility with + HTTP/1.0. Clients SHOULD include both header fields when a no-cache + request is sent to a server not known to be HTTP/1.1 compliant. + + Pragma directives MUST be passed through by a proxy or gateway + application, regardless of their significance to that application, + since the directives may be applicable to all recipients along the + request/response chain. It is not possible to specify a pragma for a + specific recipient; however, any pragma directive not relevant to a + recipient SHOULD be ignored by that recipient. + + + + + + + +Fielding, et. al. Standards Track [Page 126] + +RFC 2068 HTTP/1.1 January 1997 + + + HTTP/1.1 clients SHOULD NOT send the Pragma request-header. HTTP/1.1 + caches SHOULD treat "Pragma: no-cache" as if the client had sent + "Cache-Control: no-cache". No new Pragma directives will be defined + in HTTP. + +14.33 Proxy-Authenticate + + The Proxy-Authenticate response-header field MUST be included as part + of a 407 (Proxy Authentication Required) response. The field value + consists of a challenge that indicates the authentication scheme and + parameters applicable to the proxy for this Request-URI. + + Proxy-Authenticate = "Proxy-Authenticate" ":" challenge + + The HTTP access authentication process is described in section 11. + Unlike WWW-Authenticate, the Proxy-Authenticate header field applies + only to the current connection and SHOULD NOT be passed on to + downstream clients. However, an intermediate proxy may need to obtain + its own credentials by requesting them from the downstream client, + which in some circumstances will appear as if the proxy is forwarding + the Proxy-Authenticate header field. + +14.34 Proxy-Authorization + + The Proxy-Authorization request-header field allows the client to + identify itself (or its user) to a proxy which requires + authentication. The Proxy-Authorization field value consists of + credentials containing the authentication information of the user + agent for the proxy and/or realm of the resource being requested. + + Proxy-Authorization = "Proxy-Authorization" ":" credentials + + The HTTP access authentication process is described in section 11. + Unlike Authorization, the Proxy-Authorization header field applies + only to the next outbound proxy that demanded authentication using + the Proxy-Authenticate field. When multiple proxies are used in a + chain, the Proxy-Authorization header field is consumed by the first + outbound proxy that was expecting to receive credentials. A proxy MAY + relay the credentials from the client request to the next proxy if + that is the mechanism by which the proxies cooperatively authenticate + a given request. + +14.35 Public + + The Public response-header field lists the set of methods supported + by the server. The purpose of this field is strictly to inform the + recipient of the capabilities of the server regarding unusual + methods. The methods listed may or may not be applicable to the + + + +Fielding, et. al. Standards Track [Page 127] + +RFC 2068 HTTP/1.1 January 1997 + + + Request-URI; the Allow header field (section 14.7) MAY be used to + indicate methods allowed for a particular URI. + + Public = "Public" ":" 1#method + + Example of use: + + Public: OPTIONS, MGET, MHEAD, GET, HEAD + + This header field applies only to the server directly connected to + the client (i.e., the nearest neighbor in a chain of connections). If + the response passes through a proxy, the proxy MUST either remove the + Public header field or replace it with one applicable to its own + capabilities. + +14.36 Range + +14.36.1 Byte Ranges + + Since all HTTP entities are represented in HTTP messages as sequences + of bytes, the concept of a byte range is meaningful for any HTTP + entity. (However, not all clients and servers need to support byte- + range operations.) + + Byte range specifications in HTTP apply to the sequence of bytes in + the entity-body (not necessarily the same as the message-body). + + A byte range operation may specify a single range of bytes, or a set + of ranges within a single entity. + + ranges-specifier = byte-ranges-specifier + + byte-ranges-specifier = bytes-unit "=" byte-range-set + + byte-range-set = 1#( byte-range-spec | suffix-byte-range-spec ) + + byte-range-spec = first-byte-pos "-" [last-byte-pos] + + first-byte-pos = 1*DIGIT + + last-byte-pos = 1*DIGIT + + The first-byte-pos value in a byte-range-spec gives the byte-offset + of the first byte in a range. The last-byte-pos value gives the + byte-offset of the last byte in the range; that is, the byte + positions specified are inclusive. Byte offsets start at zero. + + + + + +Fielding, et. al. Standards Track [Page 128] + +RFC 2068 HTTP/1.1 January 1997 + + + If the last-byte-pos value is present, it must be greater than or + equal to the first-byte-pos in that byte-range-spec, or the byte- + range-spec is invalid. The recipient of an invalid byte-range-spec + must ignore it. + + If the last-byte-pos value is absent, or if the value is greater than + or equal to the current length of the entity-body, last-byte-pos is + taken to be equal to one less than the current length of the entity- + body in bytes. + + By its choice of last-byte-pos, a client can limit the number of + bytes retrieved without knowing the size of the entity. + + suffix-byte-range-spec = "-" suffix-length + + suffix-length = 1*DIGIT + + A suffix-byte-range-spec is used to specify the suffix of the + entity-body, of a length given by the suffix-length value. (That is, + this form specifies the last N bytes of an entity-body.) If the + entity is shorter than the specified suffix-length, the entire + entity-body is used. + + Examples of byte-ranges-specifier values (assuming an entity-body of + length 10000): + + o The first 500 bytes (byte offsets 0-499, inclusive): + + bytes=0-499 + + o The second 500 bytes (byte offsets 500-999, inclusive): + + bytes=500-999 + + o The final 500 bytes (byte offsets 9500-9999, inclusive): + + bytes=-500 + + o Or + + bytes=9500- + + o The first and last bytes only (bytes 0 and 9999): + + bytes=0-0,-1 + + + + + + +Fielding, et. al. Standards Track [Page 129] + +RFC 2068 HTTP/1.1 January 1997 + + + o Several legal but not canonical specifications of the second + 500 bytes (byte offsets 500-999, inclusive): + + bytes=500-600,601-999 + + bytes=500-700,601-999 + +14.36.2 Range Retrieval Requests + + HTTP retrieval requests using conditional or unconditional GET + methods may request one or more sub-ranges of the entity, instead of + the entire entity, using the Range request header, which applies to + the entity returned as the result of the request: + + Range = "Range" ":" ranges-specifier + + A server MAY ignore the Range header. However, HTTP/1.1 origin + servers and intermediate caches SHOULD support byte ranges when + possible, since Range supports efficient recovery from partially + failed transfers, and supports efficient partial retrieval of large + entities. + + If the server supports the Range header and the specified range or + ranges are appropriate for the entity: + + o The presence of a Range header in an unconditional GET modifies + what is returned if the GET is otherwise successful. In other + words, the response carries a status code of 206 (Partial + Content) instead of 200 (OK). + + o The presence of a Range header in a conditional GET (a request + using one or both of If-Modified-Since and If-None-Match, or + one or both of If-Unmodified-Since and If-Match) modifies what + is returned if the GET is otherwise successful and the condition + is true. It does not affect the 304 (Not Modified) response + returned if the conditional is false. + + In some cases, it may be more appropriate to use the If-Range header + (see section 14.27) in addition to the Range header. + + If a proxy that supports ranges receives a Range request, forwards + the request to an inbound server, and receives an entire entity in + reply, it SHOULD only return the requested range to its client. It + SHOULD store the entire received response in its cache, if that is + consistent with its cache allocation policies. + + + + + + +Fielding, et. al. Standards Track [Page 130] + +RFC 2068 HTTP/1.1 January 1997 + + +14.37 Referer + + The Referer[sic] request-header field allows the client to specify, + for the server's benefit, the address (URI) of the resource from + which the Request-URI was obtained (the "referrer", although the + header field is misspelled.) The Referer request-header allows a + server to generate lists of back-links to resources for interest, + logging, optimized caching, etc. It also allows obsolete or mistyped + links to be traced for maintenance. The Referer field MUST NOT be + sent if the Request-URI was obtained from a source that does not have + its own URI, such as input from the user keyboard. + + Referer = "Referer" ":" ( absoluteURI | relativeURI ) + + Example: + + Referer: http://www.w3.org/hypertext/DataSources/Overview.html + + If the field value is a partial URI, it SHOULD be interpreted + relative to the Request-URI. The URI MUST NOT include a fragment. + + Note: Because the source of a link may be private information or + may reveal an otherwise private information source, it is strongly + recommended that the user be able to select whether or not the + Referer field is sent. For example, a browser client could have a + toggle switch for browsing openly/anonymously, which would + respectively enable/disable the sending of Referer and From + information. + +14.38 Retry-After + + The Retry-After response-header field can be used with a 503 (Service + Unavailable) response to indicate how long the service is expected to + be unavailable to the requesting client. The value of this field can + be either an HTTP-date or an integer number of seconds (in decimal) + after the time of the response. + + Retry-After = "Retry-After" ":" ( HTTP-date | delta-seconds ) + + Two examples of its use are + + Retry-After: Fri, 31 Dec 1999 23:59:59 GMT + Retry-After: 120 + + In the latter example, the delay is 2 minutes. + + + + + + +Fielding, et. al. Standards Track [Page 131] + +RFC 2068 HTTP/1.1 January 1997 + + +14.39 Server + + The Server response-header field contains information about the + software used by the origin server to handle the request. The field + can contain multiple product tokens (section 3.8) and comments + identifying the server and any significant subproducts. The product + tokens are listed in order of their significance for identifying the + application. + + Server = "Server" ":" 1*( product | comment ) + + Example: + + Server: CERN/3.0 libwww/2.17 + + If the response is being forwarded through a proxy, the proxy + application MUST NOT modify the Server response-header. Instead, it + SHOULD include a Via field (as described in section 14.44). + + Note: Revealing the specific software version of the server may + allow the server machine to become more vulnerable to attacks + against software that is known to contain security holes. Server + implementers are encouraged to make this field a configurable + option. + +14.40 Transfer-Encoding + + The Transfer-Encoding general-header field indicates what (if any) + type of transformation has been applied to the message body in order + to safely transfer it between the sender and the recipient. This + differs from the Content-Encoding in that the transfer coding is a + property of the message, not of the entity. + + Transfer-Encoding = "Transfer-Encoding" ":" 1#transfer- + coding + + Transfer codings are defined in section 3.6. An example is: + + Transfer-Encoding: chunked + + Many older HTTP/1.0 applications do not understand the Transfer- + Encoding header. + +14.41 Upgrade + + The Upgrade general-header allows the client to specify what + additional communication protocols it supports and would like to use + if the server finds it appropriate to switch protocols. The server + + + +Fielding, et. al. Standards Track [Page 132] + +RFC 2068 HTTP/1.1 January 1997 + + + MUST use the Upgrade header field within a 101 (Switching Protocols) + response to indicate which protocol(s) are being switched. + + Upgrade = "Upgrade" ":" 1#product + + For example, + + Upgrade: HTTP/2.0, SHTTP/1.3, IRC/6.9, RTA/x11 + + The Upgrade header field is intended to provide a simple mechanism + for transition from HTTP/1.1 to some other, incompatible protocol. It + does so by allowing the client to advertise its desire to use another + protocol, such as a later version of HTTP with a higher major version + number, even though the current request has been made using HTTP/1.1. + This eases the difficult transition between incompatible protocols by + allowing the client to initiate a request in the more commonly + supported protocol while indicating to the server that it would like + to use a "better" protocol if available (where "better" is determined + by the server, possibly according to the nature of the method and/or + resource being requested). + + The Upgrade header field only applies to switching application-layer + protocols upon the existing transport-layer connection. Upgrade + cannot be used to insist on a protocol change; its acceptance and use + by the server is optional. The capabilities and nature of the + application-layer communication after the protocol change is entirely + dependent upon the new protocol chosen, although the first action + after changing the protocol MUST be a response to the initial HTTP + request containing the Upgrade header field. + + The Upgrade header field only applies to the immediate connection. + Therefore, the upgrade keyword MUST be supplied within a Connection + header field (section 14.10) whenever Upgrade is present in an + HTTP/1.1 message. + + The Upgrade header field cannot be used to indicate a switch to a + protocol on a different connection. For that purpose, it is more + appropriate to use a 301, 302, 303, or 305 redirection response. + + This specification only defines the protocol name "HTTP" for use by + the family of Hypertext Transfer Protocols, as defined by the HTTP + version rules of section 3.1 and future updates to this + specification. Any token can be used as a protocol name; however, it + will only be useful if both the client and server associate the name + with the same protocol. + + + + + + +Fielding, et. al. Standards Track [Page 133] + +RFC 2068 HTTP/1.1 January 1997 + + +14.42 User-Agent + + The User-Agent request-header field contains information about the + user agent originating the request. This is for statistical purposes, + the tracing of protocol violations, and automated recognition of user + agents for the sake of tailoring responses to avoid particular user + agent limitations. User agents SHOULD include this field with + requests. The field can contain multiple product tokens (section 3.8) + and comments identifying the agent and any subproducts which form a + significant part of the user agent. By convention, the product tokens + are listed in order of their significance for identifying the + application. + + User-Agent = "User-Agent" ":" 1*( product | comment ) + + Example: + + User-Agent: CERN-LineMode/2.15 libwww/2.17b3 + +14.43 Vary + + The Vary response-header field is used by a server to signal that the + response entity was selected from the available representations of + the response using server-driven negotiation (section 12). Field- + names listed in Vary headers are those of request-headers. The Vary + field value indicates either that the given set of header fields + encompass the dimensions over which the representation might vary, or + that the dimensions of variance are unspecified ("*") and thus may + vary over any aspect of future requests. + + Vary = "Vary" ":" ( "*" | 1#field-name ) + + An HTTP/1.1 server MUST include an appropriate Vary header field with + any cachable response that is subject to server-driven negotiation. + Doing so allows a cache to properly interpret future requests on that + resource and informs the user agent about the presence of negotiation + on that resource. A server SHOULD include an appropriate Vary header + field with a non-cachable response that is subject to server-driven + negotiation, since this might provide the user agent with useful + information about the dimensions over which the response might vary. + + The set of header fields named by the Vary field value is known as + the "selecting" request-headers. + + When the cache receives a subsequent request whose Request-URI + specifies one or more cache entries including a Vary header, the + cache MUST NOT use such a cache entry to construct a response to the + new request unless all of the headers named in the cached Vary header + + + +Fielding, et. al. Standards Track [Page 134] + +RFC 2068 HTTP/1.1 January 1997 + + + are present in the new request, and all of the stored selecting + request-headers from the previous request match the corresponding + headers in the new request. + + The selecting request-headers from two requests are defined to match + if and only if the selecting request-headers in the first request can + be transformed to the selecting request-headers in the second request + by adding or removing linear whitespace (LWS) at places where this is + allowed by the corresponding BNF, and/or combining multiple message- + header fields with the same field name following the rules about + message headers in section 4.2. + + A Vary field value of "*" signals that unspecified parameters, + possibly other than the contents of request-header fields (e.g., the + network address of the client), play a role in the selection of the + response representation. Subsequent requests on that resource can + only be properly interpreted by the origin server, and thus a cache + MUST forward a (possibly conditional) request even when it has a + fresh response cached for the resource. See section 13.6 for use of + the Vary header by caches. + + A Vary field value consisting of a list of field-names signals that + the representation selected for the response is based on a selection + algorithm which considers ONLY the listed request-header field values + in selecting the most appropriate representation. A cache MAY assume + that the same selection will be made for future requests with the + same values for the listed field names, for the duration of time in + which the response is fresh. + + The field-names given are not limited to the set of standard + request-header fields defined by this specification. Field names are + case-insensitive. + +14.44 Via + + The Via general-header field MUST be used by gateways and proxies to + indicate the intermediate protocols and recipients between the user + agent and the server on requests, and between the origin server and + the client on responses. It is analogous to the "Received" field of + RFC 822 and is intended to be used for tracking message forwards, + avoiding request loops, and identifying the protocol capabilities of + all senders along the request/response chain. + + + + + + + + + +Fielding, et. al. Standards Track [Page 135] + +RFC 2068 HTTP/1.1 January 1997 + + + Via = "Via" ":" 1#( received-protocol received-by [ comment ] ) + + received-protocol = [ protocol-name "/" ] protocol-version + protocol-name = token + protocol-version = token + received-by = ( host [ ":" port ] ) | pseudonym + pseudonym = token + + The received-protocol indicates the protocol version of the message + received by the server or client along each segment of the + request/response chain. The received-protocol version is appended to + the Via field value when the message is forwarded so that information + about the protocol capabilities of upstream applications remains + visible to all recipients. + + The protocol-name is optional if and only if it would be "HTTP". The + received-by field is normally the host and optional port number of a + recipient server or client that subsequently forwarded the message. + However, if the real host is considered to be sensitive information, + it MAY be replaced by a pseudonym. If the port is not given, it MAY + be assumed to be the default port of the received-protocol. + + Multiple Via field values represent each proxy or gateway that has + forwarded the message. Each recipient MUST append its information + such that the end result is ordered according to the sequence of + forwarding applications. + + Comments MAY be used in the Via header field to identify the software + of the recipient proxy or gateway, analogous to the User-Agent and + Server header fields. However, all comments in the Via field are + optional and MAY be removed by any recipient prior to forwarding the + message. + + For example, a request message could be sent from an HTTP/1.0 user + agent to an internal proxy code-named "fred", which uses HTTP/1.1 to + forward the request to a public proxy at nowhere.com, which completes + the request by forwarding it to the origin server at www.ics.uci.edu. + The request received by www.ics.uci.edu would then have the following + Via header field: + + Via: 1.0 fred, 1.1 nowhere.com (Apache/1.1) + + Proxies and gateways used as a portal through a network firewall + SHOULD NOT, by default, forward the names and ports of hosts within + the firewall region. This information SHOULD only be propagated if + explicitly enabled. If not enabled, the received-by host of any host + behind the firewall SHOULD be replaced by an appropriate pseudonym + for that host. + + + +Fielding, et. al. Standards Track [Page 136] + +RFC 2068 HTTP/1.1 January 1997 + + + For organizations that have strong privacy requirements for hiding + internal structures, a proxy MAY combine an ordered subsequence of + Via header field entries with identical received-protocol values into + a single such entry. For example, + + Via: 1.0 ricky, 1.1 ethel, 1.1 fred, 1.0 lucy + + could be collapsed to + + Via: 1.0 ricky, 1.1 mertz, 1.0 lucy + + Applications SHOULD NOT combine multiple entries unless they are all + under the same organizational control and the hosts have already been + replaced by pseudonyms. Applications MUST NOT combine entries which + have different received-protocol values. + +14.45 Warning + + The Warning response-header field is used to carry additional + information about the status of a response which may not be reflected + by the response status code. This information is typically, though + not exclusively, used to warn about a possible lack of semantic + transparency from caching operations. + + Warning headers are sent with responses using: + + Warning = "Warning" ":" 1#warning-value + + warning-value = warn-code SP warn-agent SP warn-text + warn-code = 2DIGIT + warn-agent = ( host [ ":" port ] ) | pseudonym + ; the name or pseudonym of the server adding + ; the Warning header, for use in debugging + warn-text = quoted-string + + A response may carry more than one Warning header. + + The warn-text should be in a natural language and character set that + is most likely to be intelligible to the human user receiving the + response. This decision may be based on any available knowledge, + such as the location of the cache or user, the Accept-Language field + in a request, the Content-Language field in a response, etc. The + default language is English and the default character set is ISO- + 8859-1. + + If a character set other than ISO-8859-1 is used, it MUST be encoded + in the warn-text using the method described in RFC 1522 [14]. + + + + +Fielding, et. al. Standards Track [Page 137] + +RFC 2068 HTTP/1.1 January 1997 + + + Any server or cache may add Warning headers to a response. New + Warning headers should be added after any existing Warning headers. A + cache MUST NOT delete any Warning header that it received with a + response. However, if a cache successfully validates a cache entry, + it SHOULD remove any Warning headers previously attached to that + entry except as specified for specific Warning codes. It MUST then + add any Warning headers received in the validating response. In other + words, Warning headers are those that would be attached to the most + recent relevant response. + + When multiple Warning headers are attached to a response, the user + agent SHOULD display as many of them as possible, in the order that + they appear in the response. If it is not possible to display all of + the warnings, the user agent should follow these heuristics: + + o Warnings that appear early in the response take priority over those + appearing later in the response. + o Warnings in the user's preferred character set take priority over + warnings in other character sets but with identical warn-codes and + warn-agents. + + Systems that generate multiple Warning headers should order them with + this user agent behavior in mind. + + This is a list of the currently-defined warn-codes, each with a + recommended warn-text in English, and a description of its meaning. + +10 Response is stale + MUST be included whenever the returned response is stale. A cache may + add this warning to any response, but may never remove it until the + response is known to be fresh. + +11 Revalidation failed + MUST be included if a cache returns a stale response because an + attempt to revalidate the response failed, due to an inability to + reach the server. A cache may add this warning to any response, but + may never remove it until the response is successfully revalidated. + +12 Disconnected operation + SHOULD be included if the cache is intentionally disconnected from + the rest of the network for a period of time. + +13 Heuristic expiration + MUST be included if the cache heuristically chose a freshness + lifetime greater than 24 hours and the response's age is greater than + 24 hours. + + + + + +Fielding, et. al. Standards Track [Page 138] + +RFC 2068 HTTP/1.1 January 1997 + + +14 Transformation applied + MUST be added by an intermediate cache or proxy if it applies any + transformation changing the content-coding (as specified in the + Content-Encoding header) or media-type (as specified in the + Content-Type header) of the response, unless this Warning code + already appears in the response. MUST NOT be deleted from a response + even after revalidation. + +99 Miscellaneous warning + The warning text may include arbitrary information to be presented to + a human user, or logged. A system receiving this warning MUST NOT + take any automated action. + +14.46 WWW-Authenticate + + The WWW-Authenticate response-header field MUST be included in 401 + (Unauthorized) response messages. The field value consists of at + least one challenge that indicates the authentication scheme(s) and + parameters applicable to the Request-URI. + + WWW-Authenticate = "WWW-Authenticate" ":" 1#challenge + + The HTTP access authentication process is described in section 11. + User agents MUST take special care in parsing the WWW-Authenticate + field value if it contains more than one challenge, or if more than + one WWW-Authenticate header field is provided, since the contents of + a challenge may itself contain a comma-separated list of + authentication parameters. + +15 Security Considerations + + This section is meant to inform application developers, information + providers, and users of the security limitations in HTTP/1.1 as + described by this document. The discussion does not include + definitive solutions to the problems revealed, though it does make + some suggestions for reducing security risks. + +15.1 Authentication of Clients + + The Basic authentication scheme is not a secure method of user + authentication, nor does it in any way protect the entity, which is + transmitted in clear text across the physical network used as the + carrier. HTTP does not prevent additional authentication schemes and + encryption mechanisms from being employed to increase security or the + addition of enhancements (such as schemes to use one-time passwords) + to Basic authentication. + + + + + +Fielding, et. al. Standards Track [Page 139] + +RFC 2068 HTTP/1.1 January 1997 + + + The most serious flaw in Basic authentication is that it results in + the essentially clear text transmission of the user's password over + the physical network. It is this problem which Digest Authentication + attempts to address. + + Because Basic authentication involves the clear text transmission of + passwords it SHOULD never be used (without enhancements) to protect + sensitive or valuable information. + + A common use of Basic authentication is for identification purposes + -- requiring the user to provide a user name and password as a means + of identification, for example, for purposes of gathering accurate + usage statistics on a server. When used in this way it is tempting to + think that there is no danger in its use if illicit access to the + protected documents is not a major concern. This is only correct if + the server issues both user name and password to the users and in + particular does not allow the user to choose his or her own password. + The danger arises because naive users frequently reuse a single + password to avoid the task of maintaining multiple passwords. + + If a server permits users to select their own passwords, then the + threat is not only illicit access to documents on the server but also + illicit access to the accounts of all users who have chosen to use + their account password. If users are allowed to choose their own + password that also means the server must maintain files containing + the (presumably encrypted) passwords. Many of these may be the + account passwords of users perhaps at distant sites. The owner or + administrator of such a system could conceivably incur liability if + this information is not maintained in a secure fashion. + + Basic Authentication is also vulnerable to spoofing by counterfeit + servers. If a user can be led to believe that he is connecting to a + host containing information protected by basic authentication when in + fact he is connecting to a hostile server or gateway then the + attacker can request a password, store it for later use, and feign an + error. This type of attack is not possible with Digest Authentication + [32]. Server implementers SHOULD guard against the possibility of + this sort of counterfeiting by gateways or CGI scripts. In particular + it is very dangerous for a server to simply turn over a connection to + a gateway since that gateway can then use the persistent connection + mechanism to engage in multiple transactions with the client while + impersonating the original server in a way that is not detectable by + the client. + +15.2 Offering a Choice of Authentication Schemes + + An HTTP/1.1 server may return multiple challenges with a 401 + (Authenticate) response, and each challenge may use a different + + + +Fielding, et. al. Standards Track [Page 140] + +RFC 2068 HTTP/1.1 January 1997 + + + scheme. The order of the challenges returned to the user agent is in + the order that the server would prefer they be chosen. The server + should order its challenges with the "most secure" authentication + scheme first. A user agent should choose as the challenge to be made + to the user the first one that the user agent understands. + + When the server offers choices of authentication schemes using the + WWW-Authenticate header, the "security" of the authentication is only + as malicious user could capture the set of challenges and try to + authenticate him/herself using the weakest of the authentication + schemes. Thus, the ordering serves more to protect the user's + credentials than the server's information. + + A possible man-in-the-middle (MITM) attack would be to add a weak + authentication scheme to the set of choices, hoping that the client + will use one that exposes the user's credentials (e.g. password). For + this reason, the client should always use the strongest scheme that + it understands from the choices accepted. + + An even better MITM attack would be to remove all offered choices, + and to insert a challenge that requests Basic authentication. For + this reason, user agents that are concerned about this kind of attack + could remember the strongest authentication scheme ever requested by + a server and produce a warning message that requires user + confirmation before using a weaker one. A particularly insidious way + to mount such a MITM attack would be to offer a "free" proxy caching + service to gullible users. + +15.3 Abuse of Server Log Information + + A server is in the position to save personal data about a user's + requests which may identify their reading patterns or subjects of + interest. This information is clearly confidential in nature and its + handling may be constrained by law in certain countries. People using + the HTTP protocol to provide data are responsible for ensuring that + such material is not distributed without the permission of any + individuals that are identifiable by the published results. + +15.4 Transfer of Sensitive Information + + Like any generic data transfer protocol, HTTP cannot regulate the + content of the data that is transferred, nor is there any a priori + method of determining the sensitivity of any particular piece of + information within the context of any given request. Therefore, + applications SHOULD supply as much control over this information as + possible to the provider of that information. Four header fields are + worth special mention in this context: Server, Via, Referer and From. + + + + +Fielding, et. al. Standards Track [Page 141] + +RFC 2068 HTTP/1.1 January 1997 + + + Revealing the specific software version of the server may allow the + server machine to become more vulnerable to attacks against software + that is known to contain security holes. Implementers SHOULD make the + Server header field a configurable option. + + Proxies which serve as a portal through a network firewall SHOULD + take special precautions regarding the transfer of header information + that identifies the hosts behind the firewall. In particular, they + SHOULD remove, or replace with sanitized versions, any Via fields + generated behind the firewall. + + The Referer field allows reading patterns to be studied and reverse + links drawn. Although it can be very useful, its power can be abused + if user details are not separated from the information contained in + the Referer. Even when the personal information has been removed, the + Referer field may indicate a private document's URI whose publication + would be inappropriate. + + The information sent in the From field might conflict with the user's + privacy interests or their site's security policy, and hence it + SHOULD NOT be transmitted without the user being able to disable, + enable, and modify the contents of the field. The user MUST be able + to set the contents of this field within a user preference or + application defaults configuration. + + We suggest, though do not require, that a convenient toggle interface + be provided for the user to enable or disable the sending of From and + Referer information. + +15.5 Attacks Based On File and Path Names + + Implementations of HTTP origin servers SHOULD be careful to restrict + the documents returned by HTTP requests to be only those that were + intended by the server administrators. If an HTTP server translates + HTTP URIs directly into file system calls, the server MUST take + special care not to serve files that were not intended to be + delivered to HTTP clients. For example, UNIX, Microsoft Windows, and + other operating systems use ".." as a path component to indicate a + directory level above the current one. On such a system, an HTTP + server MUST disallow any such construct in the Request-URI if it + would otherwise allow access to a resource outside those intended to + be accessible via the HTTP server. Similarly, files intended for + reference only internally to the server (such as access control + files, configuration files, and script code) MUST be protected from + inappropriate retrieval, since they might contain sensitive + information. Experience has shown that minor bugs in such HTTP server + implementations have turned into security risks. + + + + +Fielding, et. al. Standards Track [Page 142] + +RFC 2068 HTTP/1.1 January 1997 + + +15.6 Personal Information + + HTTP clients are often privy to large amounts of personal information + (e.g. the user's name, location, mail address, passwords, encryption + keys, etc.), and SHOULD be very careful to prevent unintentional + leakage of this information via the HTTP protocol to other sources. + We very strongly recommend that a convenient interface be provided + for the user to control dissemination of such information, and that + designers and implementers be particularly careful in this area. + History shows that errors in this area are often both serious + security and/or privacy problems, and often generate highly adverse + publicity for the implementer's company. + +15.7 Privacy Issues Connected to Accept Headers + + Accept request-headers can reveal information about the user to all + servers which are accessed. The Accept-Language header in particular + can reveal information the user would consider to be of a private + nature, because the understanding of particular languages is often + strongly correlated to the membership of a particular ethnic group. + User agents which offer the option to configure the contents of an + Accept-Language header to be sent in every request are strongly + encouraged to let the configuration process include a message which + makes the user aware of the loss of privacy involved. + + An approach that limits the loss of privacy would be for a user agent + to omit the sending of Accept-Language headers by default, and to ask + the user whether it should start sending Accept-Language headers to a + server if it detects, by looking for any Vary response-header fields + generated by the server, that such sending could improve the quality + of service. + + Elaborate user-customized accept header fields sent in every request, + in particular if these include quality values, can be used by servers + as relatively reliable and long-lived user identifiers. Such user + identifiers would allow content providers to do click-trail tracking, + and would allow collaborating content providers to match cross-server + click-trails or form submissions of individual users. Note that for + many users not behind a proxy, the network address of the host + running the user agent will also serve as a long-lived user + identifier. In environments where proxies are used to enhance + privacy, user agents should be conservative in offering accept header + configuration options to end users. As an extreme privacy measure, + proxies could filter the accept headers in relayed requests. General + purpose user agents which provide a high degree of header + configurability should warn users about the loss of privacy which can + be involved. + + + + +Fielding, et. al. Standards Track [Page 143] + +RFC 2068 HTTP/1.1 January 1997 + + +15.8 DNS Spoofing + + Clients using HTTP rely heavily on the Domain Name Service, and are + thus generally prone to security attacks based on the deliberate + mis-association of IP addresses and DNS names. Clients need to be + cautious in assuming the continuing validity of an IP number/DNS name + association. + + In particular, HTTP clients SHOULD rely on their name resolver for + confirmation of an IP number/DNS name association, rather than + caching the result of previous host name lookups. Many platforms + already can cache host name lookups locally when appropriate, and + they SHOULD be configured to do so. These lookups should be cached, + however, only when the TTL (Time To Live) information reported by the + name server makes it likely that the cached information will remain + useful. + + If HTTP clients cache the results of host name lookups in order to + achieve a performance improvement, they MUST observe the TTL + information reported by DNS. + + If HTTP clients do not observe this rule, they could be spoofed when + a previously-accessed server's IP address changes. As network + renumbering is expected to become increasingly common, the + possibility of this form of attack will grow. Observing this + requirement thus reduces this potential security vulnerability. + + This requirement also improves the load-balancing behavior of clients + for replicated servers using the same DNS name and reduces the + likelihood of a user's experiencing failure in accessing sites which + use that strategy. + +15.9 Location Headers and Spoofing + + If a single server supports multiple organizations that do not trust + one another, then it must check the values of Location and Content- + Location headers in responses that are generated under control of + said organizations to make sure that they do not attempt to + invalidate resources over which they have no authority. + +16 Acknowledgments + + This specification makes heavy use of the augmented BNF and generic + constructs defined by David H. Crocker for RFC 822. Similarly, it + reuses many of the definitions provided by Nathaniel Borenstein and + Ned Freed for MIME. We hope that their inclusion in this + specification will help reduce past confusion over the relationship + between HTTP and Internet mail message formats. + + + +Fielding, et. al. Standards Track [Page 144] + +RFC 2068 HTTP/1.1 January 1997 + + + The HTTP protocol has evolved considerably over the past four years. + It has benefited from a large and active developer community--the + many people who have participated on the www-talk mailing list--and + it is that community which has been most responsible for the success + of HTTP and of the World-Wide Web in general. Marc Andreessen, Robert + Cailliau, Daniel W. Connolly, Bob Denny, John Franks, Jean-Francois + Groff, Phillip M. Hallam-Baker, Hakon W. Lie, Ari Luotonen, Rob + McCool, Lou Montulli, Dave Raggett, Tony Sanders, and Marc + VanHeyningen deserve special recognition for their efforts in + defining early aspects of the protocol. + + This document has benefited greatly from the comments of all those + participating in the HTTP-WG. In addition to those already mentioned, + the following individuals have contributed to this specification: + + Gary Adams Albert Lunde + Harald Tveit Alvestrand John C. Mallery + Keith Ball Jean-Philippe Martin-Flatin + Brian Behlendorf Larry Masinter + Paul Burchard Mitra + Maurizio Codogno David Morris + Mike Cowlishaw Gavin Nicol + Roman Czyborra Bill Perry + Michael A. Dolan Jeffrey Perry + David J. Fiander Scott Powers + Alan Freier Owen Rees + Marc Hedlund Luigi Rizzo + Greg Herlihy David Robinson + Koen Holtman Marc Salomon + Alex Hopmann Rich Salz + Bob Jernigan Allan M. Schiffman + Shel Kaphan Jim Seidman + Rohit Khare Chuck Shotton + John Klensin Eric W. Sink + Martijn Koster Simon E. Spero + Alexei Kosut Richard N. Taylor + David M. Kristol Robert S. Thau + Daniel LaLiberte Bill (BearHeart) Weinman + Ben Laurie Francois Yergeau + Paul J. Leach Mary Ellen Zurko + Daniel DuBois + + Much of the content and presentation of the caching design is due to + suggestions and comments from individuals including: Shel Kaphan, + Paul Leach, Koen Holtman, David Morris, and Larry Masinter. + + + + + + +Fielding, et. al. Standards Track [Page 145] + +RFC 2068 HTTP/1.1 January 1997 + + + Most of the specification of ranges is based on work originally done + by Ari Luotonen and John Franks, with additional input from Steve + Zilles. + + Thanks to the "cave men" of Palo Alto. You know who you are. + + Jim Gettys (the current editor of this document) wishes particularly + to thank Roy Fielding, the previous editor of this document, along + with John Klensin, Jeff Mogul, Paul Leach, Dave Kristol, Koen + Holtman, John Franks, Alex Hopmann, and Larry Masinter for their + help. + +17 References + + [1] Alvestrand, H., "Tags for the identification of languages", RFC + 1766, UNINETT, March 1995. + + [2] Anklesaria, F., McCahill, M., Lindner, P., Johnson, D., Torrey, + D., and B. Alberti. "The Internet Gopher Protocol: (a distributed + document search and retrieval protocol)", RFC 1436, University of + Minnesota, March 1993. + + [3] Berners-Lee, T., "Universal Resource Identifiers in WWW", A + Unifying Syntax for the Expression of Names and Addresses of Objects + on the Network as used in the World-Wide Web", RFC 1630, CERN, June + 1994. + + [4] Berners-Lee, T., Masinter, L., and M. McCahill, "Uniform Resource + Locators (URL)", RFC 1738, CERN, Xerox PARC, University of Minnesota, + December 1994. + + [5] Berners-Lee, T., and D. Connolly, "HyperText Markup Language + Specification - 2.0", RFC 1866, MIT/LCS, November 1995. + + [6] Berners-Lee, T., Fielding, R., and H. Frystyk, "Hypertext + Transfer Protocol -- HTTP/1.0.", RFC 1945 MIT/LCS, UC Irvine, May + 1996. + + [7] Freed, N., and N. Borenstein, "Multipurpose Internet Mail + Extensions (MIME) Part One: Format of Internet Message Bodies", RFC + 2045, Innosoft, First Virtual, November 1996. + + [8] Braden, R., "Requirements for Internet hosts - application and + support", STD 3, RFC 1123, IETF, October 1989. + + [9] Crocker, D., "Standard for the Format of ARPA Internet Text + Messages", STD 11, RFC 822, UDEL, August 1982. + + + + +Fielding, et. al. Standards Track [Page 146] + +RFC 2068 HTTP/1.1 January 1997 + + + [10] Davis, F., Kahle, B., Morris, H., Salem, J., Shen, T., Wang, R., + Sui, J., and M. Grinbaum. "WAIS Interface Protocol Prototype + Functional Specification", (v1.5), Thinking Machines Corporation, + April 1990. + + [11] Fielding, R., "Relative Uniform Resource Locators", RFC 1808, UC + Irvine, June 1995. + + [12] Horton, M., and R. Adams. "Standard for interchange of USENET + messages", RFC 1036, AT&T Bell Laboratories, Center for Seismic + Studies, December 1987. + + [13] Kantor, B., and P. Lapsley. "Network News Transfer Protocol." A + Proposed Standard for the Stream-Based Transmission of News", RFC + 977, UC San Diego, UC Berkeley, February 1986. + + [14] Moore, K., "MIME (Multipurpose Internet Mail Extensions) Part + Three: Message Header Extensions for Non-ASCII Text", RFC 2047, + University of Tennessee, November 1996. + + [15] Nebel, E., and L. Masinter. "Form-based File Upload in HTML", + RFC 1867, Xerox Corporation, November 1995. + + [16] Postel, J., "Simple Mail Transfer Protocol", STD 10, RFC 821, + USC/ISI, August 1982. + + [17] Postel, J., "Media Type Registration Procedure", RFC 2048, + USC/ISI, November 1996. + + [18] Postel, J., and J. Reynolds, "File Transfer Protocol (FTP)", STD + 9, RFC 959, USC/ISI, October 1985. + + [19] Reynolds, J., and J. Postel, "Assigned Numbers", STD 2, RFC + 1700, USC/ISI, October 1994. + + [20] Sollins, K., and L. Masinter, "Functional Requirements for + Uniform Resource Names", RFC 1737, MIT/LCS, Xerox Corporation, + December 1994. + + [21] US-ASCII. Coded Character Set - 7-Bit American Standard Code for + Information Interchange. Standard ANSI X3.4-1986, ANSI, 1986. + + [22] ISO-8859. International Standard -- Information Processing -- + 8-bit Single-Byte Coded Graphic Character Sets -- + Part 1: Latin alphabet No. 1, ISO 8859-1:1987. + Part 2: Latin alphabet No. 2, ISO 8859-2, 1987. + Part 3: Latin alphabet No. 3, ISO 8859-3, 1988. + Part 4: Latin alphabet No. 4, ISO 8859-4, 1988. + + + +Fielding, et. al. Standards Track [Page 147] + +RFC 2068 HTTP/1.1 January 1997 + + + Part 5: Latin/Cyrillic alphabet, ISO 8859-5, 1988. + Part 6: Latin/Arabic alphabet, ISO 8859-6, 1987. + Part 7: Latin/Greek alphabet, ISO 8859-7, 1987. + Part 8: Latin/Hebrew alphabet, ISO 8859-8, 1988. + Part 9: Latin alphabet No. 5, ISO 8859-9, 1990. + + [23] Meyers, J., and M. Rose "The Content-MD5 Header Field", RFC + 1864, Carnegie Mellon, Dover Beach Consulting, October, 1995. + + [24] Carpenter, B., and Y. Rekhter, "Renumbering Needs Work", RFC + 1900, IAB, February 1996. + + [25] Deutsch, P., "GZIP file format specification version 4.3." RFC + 1952, Aladdin Enterprises, May 1996. + + [26] Venkata N. Padmanabhan and Jeffrey C. Mogul. Improving HTTP + Latency. Computer Networks and ISDN Systems, v. 28, pp. 25-35, Dec. + 1995. Slightly revised version of paper in Proc. 2nd International + WWW Conf. '94: Mosaic and the Web, Oct. 1994, which is available at + http://www.ncsa.uiuc.edu/SDG/IT94/Proceedings/DDay/mogul/ + HTTPLatency.html. + + [27] Joe Touch, John Heidemann, and Katia Obraczka, "Analysis of HTTP + Performance", <URL: http://www.isi.edu/lsam/ib/http-perf/>, + USC/Information Sciences Institute, June 1996 + + [28] Mills, D., "Network Time Protocol, Version 3, Specification, + Implementation and Analysis", RFC 1305, University of Delaware, March + 1992. + + [29] Deutsch, P., "DEFLATE Compressed Data Format Specification + version 1.3." RFC 1951, Aladdin Enterprises, May 1996. + + [30] Spero, S., "Analysis of HTTP Performance Problems" + <URL:http://sunsite.unc.edu/mdma-release/http-prob.html>. + + [31] Deutsch, P., and J-L. Gailly, "ZLIB Compressed Data Format + Specification version 3.3", RFC 1950, Aladdin Enterprises, Info-ZIP, + May 1996. + + [32] Franks, J., Hallam-Baker, P., Hostetler, J., Leach, P., + Luotonen, A., Sink, E., and L. Stewart, "An Extension to HTTP : + Digest Access Authentication", RFC 2069, January 1997. + + + + + + + + +Fielding, et. al. Standards Track [Page 148] + +RFC 2068 HTTP/1.1 January 1997 + + +18 Authors' Addresses + + Roy T. Fielding + Department of Information and Computer Science + University of California + Irvine, CA 92717-3425, USA + + Fax: +1 (714) 824-4056 + EMail: [email protected] + + + Jim Gettys + MIT Laboratory for Computer Science + 545 Technology Square + Cambridge, MA 02139, USA + + Fax: +1 (617) 258 8682 + EMail: [email protected] + + + Jeffrey C. Mogul + Western Research Laboratory + Digital Equipment Corporation + 250 University Avenue + Palo Alto, California, 94305, USA + + EMail: [email protected] + + + Henrik Frystyk Nielsen + W3 Consortium + MIT Laboratory for Computer Science + 545 Technology Square + Cambridge, MA 02139, USA + + Fax: +1 (617) 258 8682 + EMail: [email protected] + + + Tim Berners-Lee + Director, W3 Consortium + MIT Laboratory for Computer Science + 545 Technology Square + Cambridge, MA 02139, USA + + Fax: +1 (617) 258 8682 + EMail: [email protected] + + + + +Fielding, et. al. Standards Track [Page 149] + +RFC 2068 HTTP/1.1 January 1997 + + +19 Appendices + +19.1 Internet Media Type message/http + + In addition to defining the HTTP/1.1 protocol, this document serves + as the specification for the Internet media type "message/http". The + following is to be registered with IANA. + + Media Type name: message + Media subtype name: http + Required parameters: none + Optional parameters: version, msgtype + + version: The HTTP-Version number of the enclosed message + (e.g., "1.1"). If not present, the version can be + determined from the first line of the body. + + msgtype: The message type -- "request" or "response". If not + present, the type can be determined from the first + line of the body. + + Encoding considerations: only "7bit", "8bit", or "binary" are + permitted + + Security considerations: none + +19.2 Internet Media Type multipart/byteranges + + When an HTTP message includes the content of multiple ranges (for + example, a response to a request for multiple non-overlapping + ranges), these are transmitted as a multipart MIME message. The + multipart media type for this purpose is called + "multipart/byteranges". + + The multipart/byteranges media type includes two or more parts, each + with its own Content-Type and Content-Range fields. The parts are + separated using a MIME boundary parameter. + + Media Type name: multipart + Media subtype name: byteranges + Required parameters: boundary + Optional parameters: none + + Encoding considerations: only "7bit", "8bit", or "binary" are + permitted + + Security considerations: none + + + + +Fielding, et. al. Standards Track [Page 150] + +RFC 2068 HTTP/1.1 January 1997 + + +For example: + + HTTP/1.1 206 Partial content + Date: Wed, 15 Nov 1995 06:25:24 GMT + Last-modified: Wed, 15 Nov 1995 04:58:08 GMT + Content-type: multipart/byteranges; boundary=THIS_STRING_SEPARATES + + --THIS_STRING_SEPARATES + Content-type: application/pdf + Content-range: bytes 500-999/8000 + + ...the first range... + --THIS_STRING_SEPARATES + Content-type: application/pdf + Content-range: bytes 7000-7999/8000 + + ...the second range + --THIS_STRING_SEPARATES-- + +19.3 Tolerant Applications + + Although this document specifies the requirements for the generation + of HTTP/1.1 messages, not all applications will be correct in their + implementation. We therefore recommend that operational applications + be tolerant of deviations whenever those deviations can be + interpreted unambiguously. + + Clients SHOULD be tolerant in parsing the Status-Line and servers + tolerant when parsing the Request-Line. In particular, they SHOULD + accept any amount of SP or HT characters between fields, even though + only a single SP is required. + + The line terminator for message-header fields is the sequence CRLF. + However, we recommend that applications, when parsing such headers, + recognize a single LF as a line terminator and ignore the leading CR. + + The character set of an entity-body should be labeled as the lowest + common denominator of the character codes used within that body, with + the exception that no label is preferred over the labels US-ASCII or + ISO-8859-1. + + Additional rules for requirements on parsing and encoding of dates + and other potential problems with date encodings include: + + o HTTP/1.1 clients and caches should assume that an RFC-850 date + which appears to be more than 50 years in the future is in fact + in the past (this helps solve the "year 2000" problem). + + + + +Fielding, et. al. Standards Track [Page 151] + +RFC 2068 HTTP/1.1 January 1997 + + + o An HTTP/1.1 implementation may internally represent a parsed + Expires date as earlier than the proper value, but MUST NOT + internally represent a parsed Expires date as later than the + proper value. + + o All expiration-related calculations must be done in GMT. The + local time zone MUST NOT influence the calculation or comparison + of an age or expiration time. + + o If an HTTP header incorrectly carries a date value with a time + zone other than GMT, it must be converted into GMT using the most + conservative possible conversion. + +19.4 Differences Between HTTP Entities and MIME Entities + + HTTP/1.1 uses many of the constructs defined for Internet Mail (RFC + 822) and the Multipurpose Internet Mail Extensions (MIME ) to allow + entities to be transmitted in an open variety of representations and + with extensible mechanisms. However, MIME [7] discusses mail, and + HTTP has a few features that are different from those described in + MIME. These differences were carefully chosen to optimize + performance over binary connections, to allow greater freedom in the + use of new media types, to make date comparisons easier, and to + acknowledge the practice of some early HTTP servers and clients. + + This appendix describes specific areas where HTTP differs from MIME. + Proxies and gateways to strict MIME environments SHOULD be aware of + these differences and provide the appropriate conversions where + necessary. Proxies and gateways from MIME environments to HTTP also + need to be aware of the differences because some conversions may be + required. + +19.4.1 Conversion to Canonical Form + + MIME requires that an Internet mail entity be converted to canonical + form prior to being transferred. Section 3.7.1 of this document + describes the forms allowed for subtypes of the "text" media type + when transmitted over HTTP. MIME requires that content with a type of + "text" represent line breaks as CRLF and forbids the use of CR or LF + outside of line break sequences. HTTP allows CRLF, bare CR, and bare + LF to indicate a line break within text content when a message is + transmitted over HTTP. + + Where it is possible, a proxy or gateway from HTTP to a strict MIME + environment SHOULD translate all line breaks within the text media + types described in section 3.7.1 of this document to the MIME + canonical form of CRLF. Note, however, that this may be complicated + by the presence of a Content-Encoding and by the fact that HTTP + + + +Fielding, et. al. Standards Track [Page 152] + +RFC 2068 HTTP/1.1 January 1997 + + + allows the use of some character sets which do not use octets 13 and + 10 to represent CR and LF, as is the case for some multi-byte + character sets. + +19.4.2 Conversion of Date Formats + + HTTP/1.1 uses a restricted set of date formats (section 3.3.1) to + simplify the process of date comparison. Proxies and gateways from + other protocols SHOULD ensure that any Date header field present in a + message conforms to one of the HTTP/1.1 formats and rewrite the date + if necessary. + +19.4.3 Introduction of Content-Encoding + + MIME does not include any concept equivalent to HTTP/1.1's Content- + Encoding header field. Since this acts as a modifier on the media + type, proxies and gateways from HTTP to MIME-compliant protocols MUST + either change the value of the Content-Type header field or decode + the entity-body before forwarding the message. (Some experimental + applications of Content-Type for Internet mail have used a media-type + parameter of ";conversions=<content-coding>" to perform an equivalent + function as Content-Encoding. However, this parameter is not part of + MIME.) + +19.4.4 No Content-Transfer-Encoding + + HTTP does not use the Content-Transfer-Encoding (CTE) field of MIME. + Proxies and gateways from MIME-compliant protocols to HTTP MUST + remove any non-identity CTE ("quoted-printable" or "base64") encoding + prior to delivering the response message to an HTTP client. + + Proxies and gateways from HTTP to MIME-compliant protocols are + responsible for ensuring that the message is in the correct format + and encoding for safe transport on that protocol, where "safe + transport" is defined by the limitations of the protocol being used. + Such a proxy or gateway SHOULD label the data with an appropriate + Content-Transfer-Encoding if doing so will improve the likelihood of + safe transport over the destination protocol. + +19.4.5 HTTP Header Fields in Multipart Body-Parts + + In MIME, most header fields in multipart body-parts are generally + ignored unless the field name begins with "Content-". In HTTP/1.1, + multipart body-parts may contain any HTTP header fields which are + significant to the meaning of that part. + + + + + + +Fielding, et. al. Standards Track [Page 153] + +RFC 2068 HTTP/1.1 January 1997 + + +19.4.6 Introduction of Transfer-Encoding + + HTTP/1.1 introduces the Transfer-Encoding header field (section + 14.40). Proxies/gateways MUST remove any transfer coding prior to + forwarding a message via a MIME-compliant protocol. + + A process for decoding the "chunked" transfer coding (section 3.6) + can be represented in pseudo-code as: + + length := 0 + read chunk-size, chunk-ext (if any) and CRLF + while (chunk-size > 0) { + read chunk-data and CRLF + append chunk-data to entity-body + length := length + chunk-size + read chunk-size and CRLF + } + read entity-header + while (entity-header not empty) { + append entity-header to existing header fields + read entity-header + } + Content-Length := length + Remove "chunked" from Transfer-Encoding + +19.4.7 MIME-Version + + HTTP is not a MIME-compliant protocol (see appendix 19.4). However, + HTTP/1.1 messages may include a single MIME-Version general-header + field to indicate what version of the MIME protocol was used to + construct the message. Use of the MIME-Version header field indicates + that the message is in full compliance with the MIME protocol. + Proxies/gateways are responsible for ensuring full compliance (where + possible) when exporting HTTP messages to strict MIME environments. + + MIME-Version = "MIME-Version" ":" 1*DIGIT "." 1*DIGIT + + MIME version "1.0" is the default for use in HTTP/1.1. However, + HTTP/1.1 message parsing and semantics are defined by this document + and not the MIME specification. + +19.5 Changes from HTTP/1.0 + + This section summarizes major differences between versions HTTP/1.0 + and HTTP/1.1. + + + + + + +Fielding, et. al. Standards Track [Page 154] + +RFC 2068 HTTP/1.1 January 1997 + + +19.5.1 Changes to Simplify Multi-homed Web Servers and Conserve IP + Addresses + + The requirements that clients and servers support the Host request- + header, report an error if the Host request-header (section 14.23) is + missing from an HTTP/1.1 request, and accept absolute URIs (section + 5.1.2) are among the most important changes defined by this + specification. + + Older HTTP/1.0 clients assumed a one-to-one relationship of IP + addresses and servers; there was no other established mechanism for + distinguishing the intended server of a request than the IP address + to which that request was directed. The changes outlined above will + allow the Internet, once older HTTP clients are no longer common, to + support multiple Web sites from a single IP address, greatly + simplifying large operational Web servers, where allocation of many + IP addresses to a single host has created serious problems. The + Internet will also be able to recover the IP addresses that have been + allocated for the sole purpose of allowing special-purpose domain + names to be used in root-level HTTP URLs. Given the rate of growth of + the Web, and the number of servers already deployed, it is extremely + important that all implementations of HTTP (including updates to + existing HTTP/1.0 applications) correctly implement these + requirements: + + o Both clients and servers MUST support the Host request-header. + + o Host request-headers are required in HTTP/1.1 requests. + + o Servers MUST report a 400 (Bad Request) error if an HTTP/1.1 + request does not include a Host request-header. + + o Servers MUST accept absolute URIs. + + + + + + + + + + + + + + + + + + +Fielding, et. al. Standards Track [Page 155] + +RFC 2068 HTTP/1.1 January 1997 + + +19.6 Additional Features + + This appendix documents protocol elements used by some existing HTTP + implementations, but not consistently and correctly across most + HTTP/1.1 applications. Implementers should be aware of these + features, but cannot rely upon their presence in, or interoperability + with, other HTTP/1.1 applications. Some of these describe proposed + experimental features, and some describe features that experimental + deployment found lacking that are now addressed in the base HTTP/1.1 + specification. + +19.6.1 Additional Request Methods + +19.6.1.1 PATCH + + The PATCH method is similar to PUT except that the entity contains a + list of differences between the original version of the resource + identified by the Request-URI and the desired content of the resource + after the PATCH action has been applied. The list of differences is + in a format defined by the media type of the entity (e.g., + "application/diff") and MUST include sufficient information to allow + the server to recreate the changes necessary to convert the original + version of the resource to the desired version. + + If the request passes through a cache and the Request-URI identifies + a currently cached entity, that entity MUST be removed from the + cache. Responses to this method are not cachable. + + The actual method for determining how the patched resource is placed, + and what happens to its predecessor, is defined entirely by the + origin server. If the original version of the resource being patched + included a Content-Version header field, the request entity MUST + include a Derived-From header field corresponding to the value of the + original Content-Version header field. Applications are encouraged to + use these fields for constructing versioning relationships and + resolving version conflicts. + + PATCH requests must obey the message transmission requirements set + out in section 8.2. + + Caches that implement PATCH should invalidate cached responses as + defined in section 13.10 for PUT. + +19.6.1.2 LINK + + The LINK method establishes one or more Link relationships between + the existing resource identified by the Request-URI and other + existing resources. The difference between LINK and other methods + + + +Fielding, et. al. Standards Track [Page 156] + +RFC 2068 HTTP/1.1 January 1997 + + + allowing links to be established between resources is that the LINK + method does not allow any message-body to be sent in the request and + does not directly result in the creation of new resources. + + If the request passes through a cache and the Request-URI identifies + a currently cached entity, that entity MUST be removed from the + cache. Responses to this method are not cachable. + + Caches that implement LINK should invalidate cached responses as + defined in section 13.10 for PUT. + +19.6.1.3 UNLINK + + The UNLINK method removes one or more Link relationships from the + existing resource identified by the Request-URI. These relationships + may have been established using the LINK method or by any other + method supporting the Link header. The removal of a link to a + resource does not imply that the resource ceases to exist or becomes + inaccessible for future references. + + If the request passes through a cache and the Request-URI identifies + a currently cached entity, that entity MUST be removed from the + cache. Responses to this method are not cachable. + + Caches that implement UNLINK should invalidate cached responses as + defined in section 13.10 for PUT. + +19.6.2 Additional Header Field Definitions + +19.6.2.1 Alternates + + The Alternates response-header field has been proposed as a means for + the origin server to inform the client about other available + representations of the requested resource, along with their + distinguishing attributes, and thus providing a more reliable means + for a user agent to perform subsequent selection of another + representation which better fits the desires of its user (described + as agent-driven negotiation in section 12). + + + + + + + + + + + + + +Fielding, et. al. Standards Track [Page 157] + +RFC 2068 HTTP/1.1 January 1997 + + + The Alternates header field is orthogonal to the Vary header field in + that both may coexist in a message without affecting the + interpretation of the response or the available representations. It + is expected that Alternates will provide a significant improvement + over the server-driven negotiation provided by the Vary field for + those resources that vary over common dimensions like type and + language. + + The Alternates header field will be defined in a future + specification. + +19.6.2.2 Content-Version + + The Content-Version entity-header field defines the version tag + associated with a rendition of an evolving entity. Together with the + Derived-From field described in section 19.6.2.3, it allows a group + of people to work simultaneously on the creation of a work as an + iterative process. The field should be used to allow evolution of a + particular work along a single path rather than derived works or + renditions in different representations. + + Content-Version = "Content-Version" ":" quoted-string + + Examples of the Content-Version field include: + + Content-Version: "2.1.2" + Content-Version: "Fred 19950116-12:26:48" + Content-Version: "2.5a4-omega7" + +19.6.2.3 Derived-From + + The Derived-From entity-header field can be used to indicate the + version tag of the resource from which the enclosed entity was + derived before modifications were made by the sender. This field is + used to help manage the process of merging successive changes to a + resource, particularly when such changes are being made in parallel + and from multiple sources. + + Derived-From = "Derived-From" ":" quoted-string + + An example use of the field is: + + Derived-From: "2.1.1" + + The Derived-From field is required for PUT and PATCH requests if the + entity being sent was previously retrieved from the same URI and a + Content-Version header was included with the entity when it was last + retrieved. + + + +Fielding, et. al. Standards Track [Page 158] + +RFC 2068 HTTP/1.1 January 1997 + + +19.6.2.4 Link + + The Link entity-header field provides a means for describing a + relationship between two resources, generally between the requested + resource and some other resource. An entity MAY include multiple Link + values. Links at the metainformation level typically indicate + relationships like hierarchical structure and navigation paths. The + Link field is semantically equivalent to the <LINK> element in + HTML.[5] + + Link = "Link" ":" #("<" URI ">" *( ";" link-param ) + + link-param = ( ( "rel" "=" relationship ) + | ( "rev" "=" relationship ) + | ( "title" "=" quoted-string ) + | ( "anchor" "=" <"> URI <"> ) + | ( link-extension ) ) + + link-extension = token [ "=" ( token | quoted-string ) ] + + relationship = sgml-name + | ( <"> sgml-name *( SP sgml-name) <"> ) + + sgml-name = ALPHA *( ALPHA | DIGIT | "." | "-" ) + + Relationship values are case-insensitive and MAY be extended within + the constraints of the sgml-name syntax. The title parameter MAY be + used to label the destination of a link such that it can be used as + identification within a human-readable menu. The anchor parameter MAY + be used to indicate a source anchor other than the entire current + resource, such as a fragment of this resource or a third resource. + + Examples of usage include: + + Link: <http://www.cern.ch/TheBook/chapter2>; rel="Previous" + + Link: <mailto:[email protected]>; rev="Made"; title="Tim Berners-Lee" + + The first example indicates that chapter2 is previous to this + resource in a logical navigation path. The second indicates that the + person responsible for making the resource available is identified by + the given e-mail address. + +19.6.2.5 URI + + The URI header field has, in past versions of this specification, + been used as a combination of the existing Location, Content- + Location, and Vary header fields as well as the future Alternates + + + +Fielding, et. al. Standards Track [Page 159] + +RFC 2068 HTTP/1.1 January 1997 + + + field (above). Its primary purpose has been to include a list of + additional URIs for the resource, including names and mirror + locations. However, it has become clear that the combination of many + different functions within this single field has been a barrier to + consistently and correctly implementing any of those functions. + Furthermore, we believe that the identification of names and mirror + locations would be better performed via the Link header field. The + URI header field is therefore deprecated in favor of those other + fields. + + URI-header = "URI" ":" 1#( "<" URI ">" ) + +19.7 Compatibility with Previous Versions + + It is beyond the scope of a protocol specification to mandate + compliance with previous versions. HTTP/1.1 was deliberately + designed, however, to make supporting previous versions easy. It is + worth noting that at the time of composing this specification, we + would expect commercial HTTP/1.1 servers to: + + o recognize the format of the Request-Line for HTTP/0.9, 1.0, and 1.1 + requests; + + o understand any valid request in the format of HTTP/0.9, 1.0, or + 1.1; + + o respond appropriately with a message in the same major version used + by the client. + + And we would expect HTTP/1.1 clients to: + + o recognize the format of the Status-Line for HTTP/1.0 and 1.1 + responses; + + o understand any valid response in the format of HTTP/0.9, 1.0, or + 1.1. + + For most implementations of HTTP/1.0, each connection is established + by the client prior to the request and closed by the server after + sending the response. A few implementations implement the Keep-Alive + version of persistent connections described in section 19.7.1.1. + + + + + + + + + + +Fielding, et. al. Standards Track [Page 160] + +RFC 2068 HTTP/1.1 January 1997 + + +19.7.1 Compatibility with HTTP/1.0 Persistent Connections + + Some clients and servers may wish to be compatible with some previous + implementations of persistent connections in HTTP/1.0 clients and + servers. Persistent connections in HTTP/1.0 must be explicitly + negotiated as they are not the default behavior. HTTP/1.0 + experimental implementations of persistent connections are faulty, + and the new facilities in HTTP/1.1 are designed to rectify these + problems. The problem was that some existing 1.0 clients may be + sending Keep-Alive to a proxy server that doesn't understand + Connection, which would then erroneously forward it to the next + inbound server, which would establish the Keep-Alive connection and + result in a hung HTTP/1.0 proxy waiting for the close on the + response. The result is that HTTP/1.0 clients must be prevented from + using Keep-Alive when talking to proxies. + + However, talking to proxies is the most important use of persistent + connections, so that prohibition is clearly unacceptable. Therefore, + we need some other mechanism for indicating a persistent connection + is desired, which is safe to use even when talking to an old proxy + that ignores Connection. Persistent connections are the default for + HTTP/1.1 messages; we introduce a new keyword (Connection: close) for + declaring non-persistence. + + The following describes the original HTTP/1.0 form of persistent + connections. + + When it connects to an origin server, an HTTP client MAY send the + Keep-Alive connection-token in addition to the Persist connection- + token: + + Connection: Keep-Alive + + An HTTP/1.0 server would then respond with the Keep-Alive connection + token and the client may proceed with an HTTP/1.0 (or Keep-Alive) + persistent connection. + + An HTTP/1.1 server may also establish persistent connections with + HTTP/1.0 clients upon receipt of a Keep-Alive connection token. + However, a persistent connection with an HTTP/1.0 client cannot make + use of the chunked transfer-coding, and therefore MUST use a + Content-Length for marking the ending boundary of each message. + + A client MUST NOT send the Keep-Alive connection token to a proxy + server as HTTP/1.0 proxy servers do not obey the rules of HTTP/1.1 + for parsing the Connection header field. + + + + + +Fielding, et. al. Standards Track [Page 161] + +RFC 2068 HTTP/1.1 January 1997 + + +19.7.1.1 The Keep-Alive Header + + When the Keep-Alive connection-token has been transmitted with a + request or a response, a Keep-Alive header field MAY also be + included. The Keep-Alive header field takes the following form: + + Keep-Alive-header = "Keep-Alive" ":" 0# keepalive-param + + keepalive-param = param-name "=" value + + The Keep-Alive header itself is optional, and is used only if a + parameter is being sent. HTTP/1.1 does not define any parameters. + + If the Keep-Alive header is sent, the corresponding connection token + MUST be transmitted. The Keep-Alive header MUST be ignored if + received without the connection token. + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + +Fielding, et. al. Standards Track [Page 162] + diff --git a/lib/inets/doc/archive/rfc2145.txt b/lib/inets/doc/archive/rfc2145.txt new file mode 100644 index 0000000000..b6db4d588d --- /dev/null +++ b/lib/inets/doc/archive/rfc2145.txt @@ -0,0 +1,395 @@ + + + + + + +Network Working Group J. C. Mogul +Request for Comments: 2145 DEC +Category: Informational R. Fielding + UC Irvine + J. Gettys + DEC + H. Frystyk + MIT/LCS + May 1997 + + Use and Interpretation of + HTTP Version Numbers + +Status of this Memo + + This memo provides information for the Internet community. This memo + does not specify an Internet standard of any kind. Distribution of + this memo is unlimited. + + Distribution of this document is unlimited. Please send comments to + the HTTP working group at <[email protected]>. Discussions + of the working group are archived at + <URL:http://www.ics.uci.edu/pub/ietf/http/>. General discussions + about HTTP and the applications which use HTTP should take place on + the <[email protected]> mailing list. + +Abstract + + HTTP request and response messages include an HTTP protocol version + number. Some confusion exists concerning the proper use and + interpretation of HTTP version numbers, and concerning + interoperability of HTTP implementations of different protocol + versions. This document is an attempt to clarify the situation. It + is not a modification of the intended meaning of the existing + HTTP/1.0 and HTTP/1.1 documents, but it does describe the intention + of the authors of those documents, and can be considered definitive + when there is any ambiguity in those documents concerning HTTP + version numbers, for all versions of HTTP. + + + + + + + + + + + + + +Mogul, et. al. Informational [Page 1] + +RFC 2145 HTTP Version Numbers May 1997 + + +TABLE OF CONTENTS + + 1 Introduction. . . . . . . . . . . . . . . . . . . . . . . . . . 2 + 1.1 Robustness Principle . . . . . . . . . . . . . . . . . . 3 + 2 HTTP version numbers. . . . . . . . . . . . . . . . . . . . . . 3 + 2.1 Proxy behavior. . . . . . . . . . . . . . . . . . . . . . . . 4 + 2.2 Compatibility between minor versions of the same major + version. . . . . . . . . . . . . . . . . . . . . . . . 4 + 2.3 Which version number to send in a message. . . . . . . . 5 + 3 Security Considerations . . . . . . . . . . . . . . . . . . . . 6 + 4 References. . . . . . . . . . . . . . . . . . . . . . . . . . . 6 + 5 Authors' addresses. . . . . . . . . . . . . . . . . . . . . . . 6 + +1 Introduction + + HTTP request and response messages include an HTTP protocol version + number. According to section 3.1 of the HTTP/1.1 specification [2], + + HTTP uses a "<major>.<minor>" numbering scheme to indicate + versions of the protocol. The protocol versioning policy is + intended to allow the sender to indicate the format of a message + and its capacity for understanding further HTTP communication, + rather than the features obtained via that communication. No + change is made to the version number for the addition of message + components which do not affect communication behavior or which + only add to extensible field values. The <minor> number is + incremented when the changes made to the protocol add features + which do not change the general message parsing algorithm, but + which may add to the message semantics and imply additional + capabilities of the sender. The <major> number is incremented when + the format of a message within the protocol is changed. + + The same language appears in the description of HTTP/1.0 [1]. + + Many readers of these documents have expressed some confusion about + the intended meaning of this policy. Also, some people who wrote + HTTP implementations before RFC1945 [1] was issued were not aware of + the intentions behind the introduction of version numbers in + HTTP/1.0. This has led to debate and inconsistency regarding the use + and interpretation of HTTP version numbers, and has led to + interoperability problems in certain cases. + + + + + + + + + + +Mogul, et. al. Informational [Page 2] + +RFC 2145 HTTP Version Numbers May 1997 + + + This document is an attempt to clarify the situation. It is not a + modification of the intended meaning of the existing HTTP/1.0 and + HTTP/1.1 documents, but it does describe the intention of the authors + of those documents. In any case where either of those two documents + is ambiguous regarding the use and interpretation of HTTP version + numbers, this document should be considered the definitive as to the + intentions of the designers of HTTP. + + The specification described in this document is not part of the + specification of any individual version of HTTP, such as HTTP/1.0 or + HTTP/1.1. Rather, this document describes the use of HTTP version + numbers in any version of HTTP (except for HTTP/0.9, which did not + include version numbers). + + No vendor or other provider of an HTTP implementation should claim + any compliance with any IETF HTTP specification unless the + implementation conditionally complies with the rules in this + document. + +1.1 Robustness Principle + + RFC791 [4] defines the "robustness principle" in section 3.2: + + an implementation must be conservative in its sending + behavior, and liberal in its receiving behavior. + + This principle applies to HTTP, as well. It is the fundamental basis + for interpreting any part of the HTTP specification that might still + be ambiguous. In particular, implementations of HTTP SHOULD NOT + reject messages or generate errors unnecessarily. + +2 HTTP version numbers + + We start by restating the language quoted above from section 3.1 of + the HTTP/1.1 specification [2]: + + It is, and has always been, the explicit intent of the + HTTP specification that the interpretation of an HTTP message + header does not change between minor versions of the same major + version. + + It is, and has always been, the explicit intent of the + HTTP specification that an implementation receiving a message + header that it does not understand MUST ignore that header. (The + word "ignore" has a special meaning for proxies; see section 2.1 + below.) + + + + + +Mogul, et. al. Informational [Page 3] + +RFC 2145 HTTP Version Numbers May 1997 + + + To make this as clear as possible: The major version sent in a + message MAY indicate the interpretation of other header fields. The + minor version sent in a message MUST NOT indicate the interpretation + of other header fields. This reflects the principle that the minor + version labels the capability of the sender, not the interpretation + of the message. + + Note: In a future version of HTTP, we may introduce a mechanism + that explicitly requires a receiving implementation to reject a + message if it does not understand certain headers. For example, + this might be implemented by means of a header that lists a set of + other message headers that must be understood by the recipient. + Any implementation claiming at least conditional compliance with + this future version of HTTP would have to implement this + mechanism. However, no implementation claiming compliance with a + lower HTTP version (in particular, HTTP/1.1) will have to + implement this mechanism. + + This future change may be required to support the Protocol + Extension Protocol (PEP) [3]. + + One consequence of these rules is that an HTTP/1.1 message sent to an + HTTP/1.0 recipient (or a recipient whose version is unknown) MUST be + constructed so that it remains a valid HTTP/1.0 message when all + headers not defined in the HTTP/1.0 specification [1] are removed. + +2.1 Proxy behavior + + A proxy MUST forward an unknown header, unless it is protected by a + Connection header. A proxy implementing an HTTP version >= 1.1 MUST + NOT forward unknown headers that are protected by a Connection + header, as described in section 14.10 of the HTTP/1.1 specification + [2]. + + We remind the reader that that HTTP version numbers are hop-by-hop + components of HTTP messages, and are not end-to-end. That is, an + HTTP proxy never "forwards" an HTTP version number in either a + request or response. + +2.2 Compatibility between minor versions of the same major version + + An implementation of HTTP/x.b sending a message to a recipient whose + version is known to be HTTP/x.a, a < b, MAY send a header that is not + defined in the specification for HTTP/x.a. For example, an HTTP/1.1 + server may send a "Cache-control" header to an HTTP/1.0 client; this + may be useful if the immediate recipient is an HTTP/1.0 proxy, but + the ultimate recipient is an HTTP/1.1 client. + + + + +Mogul, et. al. Informational [Page 4] + +RFC 2145 HTTP Version Numbers May 1997 + + + An implementation of HTTP/x.b sending a message to a recipient whose + version is known to be HTTP/x.a, a < b, MUST NOT depend on the + recipient understanding a header not defined in the specification for + HTTP/x.a. For example, HTTP/1.0 clients cannot be expected to + understand chunked encodings, and so an HTTP/1.1 server must never + send "Transfer-Encoding: chunked" in response to an HTTP/1.0 request. + +2.3 Which version number to send in a message + + The most strenuous debate over the use of HTTP version numbers has + centered on the problem of implementations that do not follow the + robustness principle, and which fail to produce useful results when + they receive a message with an HTTP minor version higher than the + minor version they implement. We consider these implementations + buggy, but we recognize that the robustness principle also implies + that message senders should make concessions to buggy implementations + when this is truly necessary for interoperation. + + An HTTP client SHOULD send a request version equal to the highest + version for which the client is at least conditionally compliant, and + whose major version is no higher than the highest version supported + by the server, if this is known. An HTTP client MUST NOT send a + version for which it is not at least conditionally compliant. + + An HTTP client MAY send a lower request version, if it is known that + the server incorrectly implements the HTTP specification, but only + after the client has determined that the server is actually buggy. + + An HTTP server SHOULD send a response version equal to the highest + version for which the server is at least conditionally compliant, and + whose major version is less than or equal to the one received in the + request. An HTTP server MUST NOT send a version for which it is not + at least conditionally compliant. A server MAY send a 505 (HTTP + Version Not Supported) response if cannot send a response using the + major version used in the client's request. + + An HTTP server MAY send a lower response version, if it is known or + suspected that the client incorrectly implements the HTTP + specification, but this should not be the default, and this SHOULD + NOT be done if the request version is HTTP/1.1 or greater. + + + + + + + + + + + +Mogul, et. al. Informational [Page 5] + +RFC 2145 HTTP Version Numbers May 1997 + + +3 Security Considerations + + None, except to the extent that security mechanisms introduced in one + version of HTTP might depend on the proper interpretation of HTTP + version numbers in older implementations. + +4 References + + 1. Berners-Lee, T., R. Fielding, and H. Frystyk. Hypertext + Transfer Protocol -- HTTP/1.0. RFC 1945, HTTP Working Group, May, + 1996. + + 2. Fielding, Roy T., Jim Gettys, Jeffrey C. Mogul, Henrik Frystyk + Nielsen, and Tim Berners-Lee. Hypertext Transfer Protocol -- + HTTP/1.1. RFC 2068, HTTP Working Group, January, 1997. + + 3. Khare, Rohit. HTTP/1.2 Extension Protocol (PEP). HTTP Working + Group, Work in Progress. + + 4. Postel, Jon. Internet Protocol. RFC 791, NIC, September, 1981. + +5 Authors' addresses + + Jeffrey C. Mogul + Western Research Laboratory + Digital Equipment Corporation + 250 University Avenue + Palo Alto, California, 94305, USA + Email: [email protected] + + Roy T. Fielding + Department of Information and Computer Science + University of California + Irvine, CA 92717-3425, USA + Fax: +1 (714) 824-4056 + Email: [email protected] + + Jim Gettys + MIT Laboratory for Computer Science + 545 Technology Square + Cambridge, MA 02139, USA + Fax: +1 (617) 258 8682 + Email: [email protected] + + + + + + + + +Mogul, et. al. Informational [Page 6] + +RFC 2145 HTTP Version Numbers May 1997 + + + Henrik Frystyk Nielsen + W3 Consortium + MIT Laboratory for Computer Science + 545 Technology Square + Cambridge, MA 02139, USA + Fax: +1 (617) 258 8682 + Email: [email protected] + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + +Mogul, et. al. Informational [Page 7] + diff --git a/lib/inets/doc/archive/rfc2428.txt b/lib/inets/doc/archive/rfc2428.txt new file mode 100644 index 0000000000..a6ec3535ed --- /dev/null +++ b/lib/inets/doc/archive/rfc2428.txt @@ -0,0 +1,451 @@ + + + + + + +Network Working Group M. Allman +Request for Comments: 2428 NASA Lewis/Sterling Software +Category: Standards Track S. Ostermann + Ohio University + C. Metz + The Inner Net + September 1998 + + + FTP Extensions for IPv6 and NATs + +Status of this Memo + + This document specifies an Internet standards track protocol for the + Internet community, and requests discussion and suggestions for + improvements. Please refer to the current edition of the "Internet + Official Protocol Standards" (STD 1) for the standardization state + and status of this protocol. Distribution of this memo is unlimited. + +Copyright Notice + + Copyright (C) The Internet Society (1998). All Rights Reserved. + +Abstract + + The specification for the File Transfer Protocol assumes that the + underlying network protocol uses a 32-bit network address + (specifically IP version 4). With the deployment of version 6 of the + Internet Protocol, network addresses will no longer be 32-bits. This + paper specifies extensions to FTP that will allow the protocol to + work over IPv4 and IPv6. In addition, the framework defined can + support additional network protocols in the future. + +1. Introduction + + The keywords, such as MUST and SHOULD, found in this document are + used as defined in RFC 2119 [Bra97]. + + The File Transfer Protocol [PR85] only provides the ability to + communicate information about IPv4 data connections. FTP assumes + network addresses will be 32 bits in length. However, with the + deployment of version 6 of the Internet Protocol [DH96] addresses + will no longer be 32 bits long. RFC 1639 [Pis94] specifies + extensions to FTP to enable its use over various network protocols. + Unfortunately, the mechanism can fail in a multi-protocol + environment. During the transition between IPv4 and IPv6, FTP needs + the ability to negotiate the network protocol that will be used for + data transfer. + + + +Allman, et. al. Standards Track [Page 1] + +RFC 2428 FTP Extensions for IPv6 and NATs September 1998 + + + This document provides a specification for a way that FTP can + communicate data connection endpoint information for network + protocols other than IPv4. In this specification, the FTP commands + PORT and PASV are replaced with EPRT and EPSV, respectively. This + document is organized as follows. Section 2 outlines the EPRT + command and Section 3 outlines the EPSV command. Section 4 defines + the utilization of these two new FTP commands. Section 5 briefly + presents security considerations. Finally, Section 6 provides + conclusions. + +2. The EPRT Command + + The EPRT command allows for the specification of an extended address + for the data connection. The extended address MUST consist of the + network protocol as well as the network and transport addresses. The + format of EPRT is: + + EPRT<space><d><net-prt><d><net-addr><d><tcp-port><d> + + The EPRT command keyword MUST be followed by a single space (ASCII + 32). Following the space, a delimiter character (<d>) MUST be + specified. The delimiter character MUST be one of the ASCII + characters in range 33-126 inclusive. The character "|" (ASCII 124) + is recommended unless it coincides with a character needed to encode + the network address. + + The <net-prt> argument MUST be an address family number defined by + IANA in the latest Assigned Numbers RFC (RFC 1700 [RP94] as of the + writing of this document). This number indicates the protocol to be + used (and, implicitly, the address length). This document will use + two of address family numbers from [RP94] as examples, according to + the following table: + + AF Number Protocol + --------- -------- + 1 Internet Protocol, Version 4 [Pos81a] + 2 Internet Protocol, Version 6 [DH96] + + The <net-addr> is a protocol specific string representation of the + network address. For the two address families specified above (AF + Number 1 and 2), addresses MUST be in the following format: + + AF Number Address Format Example + --------- -------------- ------- + 1 dotted decimal 132.235.1.2 + 2 IPv6 string 1080::8:800:200C:417A + representations + defined in [HD96] + + + +Allman, et. al. Standards Track [Page 2] + +RFC 2428 FTP Extensions for IPv6 and NATs September 1998 + + + The <tcp-port> argument must be the string representation of the + number of the TCP port on which the host is listening for the data + connection. + + The following are sample EPRT commands: + + EPRT |1|132.235.1.2|6275| + + EPRT |2|1080::8:800:200C:417A|5282| + + The first command specifies that the server should use IPv4 to open a + data connection to the host "132.235.1.2" on TCP port 6275. The + second command specifies that the server should use the IPv6 network + protocol and the network address "1080::8:800:200C:417A" to open a + TCP data connection on port 5282. + + Upon receipt of a valid EPRT command, the server MUST return a code + of 200 (Command OK). The standard negative error code 500 and 501 + [PR85] are sufficient to handle most errors (e.g., syntax errors) + involving the EPRT command. However, an additional error code is + needed. The response code 522 indicates that the server does not + support the requested network protocol. The interpretation of this + new error code is: + + 5yz Negative Completion + x2z Connections + xy2 Extended Port Failure - unknown network protocol + + The text portion of the response MUST indicate which network + protocols the server does support. If the network protocol is + unsupported, the format of the response string MUST be: + + <text stating that the network protocol is unsupported> \ + (prot1,prot2,...,protn) + + Both the numeric code specified above and the protocol information + between the characters '(' and ')' are intended for the software + automata receiving the response; the textual message between the + numeric code and the '(' is intended for the human user and can be + any arbitrary text, but MUST NOT include the characters '(' and ')'. + In the above case, the text SHOULD indicate that the network protocol + in the EPRT command is not supported by the server. The list of + protocols inside the parenthesis MUST be a comma separated list of + address family numbers. Two example response strings follow: + + Network protocol not supported, use (1) + + Network protocol not supported, use (1,2) + + + +Allman, et. al. Standards Track [Page 3] + +RFC 2428 FTP Extensions for IPv6 and NATs September 1998 + + +3. The EPSV Command + + The EPSV command requests that a server listen on a data port and + wait for a connection. The EPSV command takes an optional argument. + The response to this command includes only the TCP port number of the + listening connection. The format of the response, however, is + similar to the argument of the EPRT command. This allows the same + parsing routines to be used for both commands. In addition, the + format leaves a place holder for the network protocol and/or network + address, which may be needed in the EPSV response in the future. The + response code for entering passive mode using an extended address + MUST be 229. The interpretation of this code, according to [PR85] + is: + + 2yz Positive Completion + x2z Connections + xy9 Extended Passive Mode Entered + + The text returned in response to the EPSV command MUST be: + + <text indicating server is entering extended passive mode> \ + (<d><d><d><tcp-port><d>) + + The portion of the string enclosed in parentheses MUST be the exact + string needed by the EPRT command to open the data connection, as + specified above. + + The first two fields contained in the parenthesis MUST be blank. The + third field MUST be the string representation of the TCP port number + on which the server is listening for a data connection. The network + protocol used by the data connection will be the same network + protocol used by the control connection. In addition, the network + address used to establish the data connection will be the same + network address used for the control connection. An example response + string follows: + + Entering Extended Passive Mode (|||6446|) + + The standard negative error codes 500 and 501 are sufficient to + handle all errors involving the EPSV command (e.g., syntax errors). + + When the EPSV command is issued with no argument, the server will + choose the network protocol for the data connection based on the + protocol used for the control connection. However, in the case of + proxy FTP, this protocol might not be appropriate for communication + between the two servers. Therefore, the client needs to be able to + request a specific protocol. If the server returns a protocol that + is not supported by the host that will be connecting to the port, the + + + +Allman, et. al. Standards Track [Page 4] + +RFC 2428 FTP Extensions for IPv6 and NATs September 1998 + + + client MUST issue an ABOR (abort) command to allow the server to + close down the listening connection. The client can then send an + EPSV command requesting the use of a specific network protocol, as + follows: + + EPSV<space><net-prt> + + If the requested protocol is supported by the server, it SHOULD use + the protocol. If not, the server MUST return the 522 error messages + as outlined in section 2. + + Finally, the EPSV command can be used with the argument "ALL" to + inform Network Address Translators that the EPRT command (as well as + other data commands) will no longer be used. An example of this + command follows: + + EPSV<space>ALL + + Upon receipt of an EPSV ALL command, the server MUST reject all data + connection setup commands other than EPSV (i.e., EPRT, PORT, PASV, et + al.). This use of the EPSV command is further explained in section + 4. + +4. Command Usage + + For all FTP transfers where the control and data connection(s) are + being established between the same two machines, the EPSV command + MUST be used. Using the EPSV command benefits performance of + transfers that traverse firewalls or Network Address Translators + (NATs). RFC 1579 [Bel94] recommends using the passive command when + behind firewalls since firewalls do not generally allow incoming + connections (which are required when using the PORT (EPRT) command). + In addition, using EPSV as defined in this document does not require + NATs to change the network address in the traffic as it is forwarded. + The NAT would have to change the address if the EPRT command was + used. Finally, if the client issues an "EPSV ALL" command, NATs may + be able to put the connection on a "fast path" through the + translator, as the EPRT command will never be used and therefore, + translation of the data portion of the segments will never be needed. + When a client only expects to do two-way FTP transfers, it SHOULD + issue this command as soon as possible. If a client later finds that + it must do a three-way FTP transfer after issuing an EPSV ALL + command, a new FTP session MUST be started. + + + + + + + + +Allman, et. al. Standards Track [Page 5] + +RFC 2428 FTP Extensions for IPv6 and NATs September 1998 + + +5. Security Issues + + The authors do not believe that these changes to FTP introduce new + security problems. A companion Work in Progress [AO98] is a more + general discussion of FTP security issues and techniques to reduce + these security problems. + +6. Conclusions + + The extensions specified in this paper will enable FTP to operate + over a variety of network protocols. + +References + + [AO98] Allman, M., and S. Ostermann, "FTP Security + Considerations", Work in Progress. + + [Bel94] Bellovin, S., "Firewall-Friendly FTP", RFC 1579, February + 1994. + + [Bra97] Bradner, S., "Key words for use in RFCs to Indicate + Requirement Levels", BCP 14, RFC 2119, March 1997. + + [DH96] Deering, S., and R. Hinden, "Internet Protocol, Version 6 + (IPv6) Specification", RFC 1883, December 1995. + + [HD96] Hinden, R., and S. Deering, "IP Version 6 Addressing + Architecture", RFC 2373, July 1998. + + [Pis94] Piscitello, D., "FTP Operation Over Big Address Records + (FOOBAR)", RFC 1639, June 1994. + + [Pos81a] Postel, J., "Internet Protocol", STD 5, RFC 791, September + 1981. + + [Pos81b] Postel, J., "Transmission Control Protocol", STD 7, RFC 793, + September 1981. + + [PR85] Postel, J., and J. Reynolds, "File Transfer Protocol (FTP)", + STD 9, RFC 959, October 1985. + + [RP94] Reynolds, J., and J. Postel, "Assigned Numbers", STD 2, RFC + 1700, October 1994. See also: + http://www.iana.org/numbers.html + + + + + + + +Allman, et. al. Standards Track [Page 6] + +RFC 2428 FTP Extensions for IPv6 and NATs September 1998 + + +Authors' Addresses + + Mark Allman + NASA Lewis Research Center/Sterling Software + 21000 Brookpark Rd. MS 54-2 + Cleveland, OH 44135 + + Phone: (216) 433-6586 + EMail: [email protected] + http://gigahertz.lerc.nasa.gov/~mallman/ + + + Shawn Ostermann + School of Electrical Engineering and Computer Science + Ohio University + 416 Morton Hall + Athens, OH 45701 + + Phone: (740) 593-1234 + EMail: [email protected] + + + Craig Metz + The Inner Net + Box 10314-1954 + Blacksburg, VA 24062-0314 + + Phone: (DSN) 754-8590 + EMail: [email protected] + + + + + + + + + + + + + + + + + + + + + + +Allman, et. al. Standards Track [Page 7] + +RFC 2428 FTP Extensions for IPv6 and NATs September 1998 + + +Full Copyright Statement + + Copyright (C) The Internet Society (1998). All Rights Reserved. + + This document and translations of it may be copied and furnished to + others, and derivative works that comment on or otherwise explain it + or assist in its implementation may be prepared, copied, published + and distributed, in whole or in part, without restriction of any + kind, provided that the above copyright notice and this paragraph are + included on all such copies and derivative works. However, this + document itself may not be modified in any way, such as by removing + the copyright notice or references to the Internet Society or other + Internet organizations, except as needed for the purpose of + developing Internet standards in which case the procedures for + copyrights defined in the Internet Standards process must be + followed, or as required to translate it into languages other than + English. + + The limited permissions granted above are perpetual and will not be + revoked by the Internet Society or its successors or assigns. + + This document and the information contained herein is provided on an + "AS IS" basis and THE INTERNET SOCIETY AND THE INTERNET ENGINEERING + TASK FORCE DISCLAIMS ALL WARRANTIES, EXPRESS OR IMPLIED, INCLUDING + BUT NOT LIMITED TO ANY WARRANTY THAT THE USE OF THE INFORMATION + HEREIN WILL NOT INFRINGE ANY RIGHTS OR ANY IMPLIED WARRANTIES OF + MERCHANTABILITY OR FITNESS FOR A PARTICULAR PURPOSE. + + + + + + + + + + + + + + + + + + + + + + + + +Allman, et. al. Standards Track [Page 8] + diff --git a/lib/inets/doc/archive/rfc2577.txt b/lib/inets/doc/archive/rfc2577.txt new file mode 100644 index 0000000000..83ba203130 --- /dev/null +++ b/lib/inets/doc/archive/rfc2577.txt @@ -0,0 +1,451 @@ + + + + + + +Network Working Group M. Allman +Request for Comments: 2577 NASA Glenn/Sterling Software +Category: Informational S. Ostermann + Ohio University + May 1999 + + + FTP Security Considerations + +Status of this Memo + + This memo provides information for the Internet community. It does + not specify an Internet standard of any kind. Distribution of this + memo is unlimited. + +Copyright Notice + + Copyright (C) The Internet Society (1999). All Rights Reserved. + +Abstract + + The specification for the File Transfer Protocol (FTP) contains a + number of mechanisms that can be used to compromise network security. + The FTP specification allows a client to instruct a server to + transfer files to a third machine. This third-party mechanism, known + as proxy FTP, causes a well known security problem. The FTP + specification also allows an unlimited number of attempts at entering + a user's password. This allows brute force "password guessing" + attacks. This document provides suggestions for system + administrators and those implementing FTP servers that will decrease + the security problems associated with FTP. + +1 Introduction + + The File Transfer Protocol specification (FTP) [PR85] provides a + mechanism that allows a client to establish an FTP control connection + and transfer a file between two FTP servers. This "proxy FTP" + mechanism can be used to decrease the amount of traffic on the + network; the client instructs one server to transfer a file to + another server, rather than transferring the file from the first + server to the client and then from the client to the second server. + This is particularly useful when the client connects to the network + using a slow link (e.g., a modem). While useful, proxy FTP provides + a security problem known as a "bounce attack" [CERT97:27]. In + addition to the bounce attack, FTP servers can be used by attackers + to guess passwords using brute force. + + + + + +Allman & Ostermann Informational [Page 1] + +RFC 2577 FTP Security Considerations May 1999 + + + This document does not contain a discussion of FTP when used in + conjunction with strong security protocols, such as IP Security. + These security concerns should be documented, however they are out of + the scope of this document. + + This paper provides information for FTP server implementers and + system administrators, as follows. Section 2 describes the FTP + "bounce attack". Section 3 provides suggestions for minimizing the + bounce attack. Section 4 provides suggestions for servers which + limit access based on network address. Section 5 provides + recommendations for limiting brute force "password guessing" by + clients. Next, section 6 provides a brief discussion of mechanisms + to improve privacy. Section 7 provides a mechanism to prevent user + identity guessing. Section 8 discusses the practice of port + stealing. Finally, section 9 provides an overview of other FTP + security issues related to software bugs rather than protocol issues. + +2 The Bounce Attack + + The version of FTP specified in the standard [PR85] provides a method + for attacking well known network servers, while making the + perpetrators difficult to track down. The attack involves sending an + FTP "PORT" command to an FTP server containing the network address + and the port number of the machine and service being attacked. At + this point, the original client can instruct the FTP server to send a + file to the service being attacked. Such a file would contain + commands relevant to the service being attacked (SMTP, NNTP, etc.). + Instructing a third party to connect to the service, rather than + connecting directly, makes tracking down the perpetrator difficult + and can circumvent network-address-based access restrictions. + + As an example, a client uploads a file containing SMTP commands to an + FTP server. Then, using an appropriate PORT command, the client + instructs the server to open a connection to a third machine's SMTP + port. Finally, the client instructs the server to transfer the + uploaded file containing SMTP commands to the third machine. This + may allow the client to forge mail on the third machine without + making a direct connection. This makes it difficult to track + attackers. + +3 Protecting Against the Bounce Attack + + The original FTP specification [PR85] assumes that data connections + will be made using the Transmission Control Protocol (TCP) [Pos81]. + TCP port numbers in the range 0 - 1023 are reserved for well known + services such as mail, network news and FTP control connections + [RP94]. The FTP specification makes no restrictions on the TCP port + number used for the data connection. Therefore, using proxy FTP, + + + +Allman & Ostermann Informational [Page 2] + +RFC 2577 FTP Security Considerations May 1999 + + + clients have the ability to tell the server to attack a well known + service on any machine. + + To avoid such bounce attacks, it is suggested that servers not open + data connections to TCP ports less than 1024. If a server receives a + PORT command containing a TCP port number less than 1024, the + suggested response is 504 (defined as "Command not implemented for + that parameter" by [PR85]). Note that this still leaves non-well + known servers (those running on ports greater than 1023) vulnerable + to bounce attacks. + + Several proposals (e.g., [AOM98] and [Pis94]) provide a mechanism + that would allow data connections to be made using a transport + protocol other than TCP. Similar precautions should be taken to + protect well known services when using these protocols. + + Also note that the bounce attack generally requires that a + perpetrator be able to upload a file to an FTP server and later + download it to the service being attacked. Using proper file + protections will prevent this behavior. However, attackers can also + attack services by sending random data from a remote FTP server which + may cause problems for some services. + + Disabling the PORT command is also an option for protecting against + the bounce attack. Most file transfers can be made using only the + PASV command [Bel94]. The disadvantage of disabling the PORT command + is that one loses the ability to use proxy FTP, but proxy FTP may not + be necessary in a particular environment. + +4 Restricted Access + + For some FTP servers, it is desirable to restrict access based on + network address. For example, a server might want to restrict access + to certain files from certain places (e.g., a certain file should not + be transferred out of an organization). In such a situation, the + server should confirm that the network address of the remote hosts on + both the control connection and the data connection are within the + organization before sending a restricted file. By checking both + connections, a server is protected against the case when the control + connection is established with a trusted host and the data connection + is not. Likewise, the client should verify the IP address of the + remote host after accepting a connection on a port opened in listen + mode to verify that the connection was made by the expected server. + + Note that restricting access based on network address leaves the FTP + server vulnerable to "spoof" attacks. In a spoof attack, for + example, an attacking machine could assume the host address of + another machine inside an organization and download files that are + + + +Allman & Ostermann Informational [Page 3] + +RFC 2577 FTP Security Considerations May 1999 + + + not accessible from outside the organization. Whenever possible, + secure authentication mechanisms should be used, such as those + outlined in [HL97]. + +5 Protecting Passwords + + To minimize the risk of brute force password guessing through the FTP + server, it is suggested that servers limit the number of attempts + that can be made at sending a correct password. After a small number + of attempts (3-5), the server should close the control connection + with the client. Before closing the control connection the server + must send a return code of 421 ("Service not available, closing + control connection." [PR85]) to the client. In addition, it is + suggested that the server impose a 5 second delay before replying to + an invalid "PASS" command to diminish the efficiency of a brute force + attack. If available, mechanisms already provided by the target + operating system should be used to implement the above suggestions. + + An intruder can subvert the above mechanisms by establishing + multiple, parallel control connections to a server. To combat the + use of multiple concurrent connections, the server could either limit + the total number of control connections possible or attempt to detect + suspicious activity across sessions and refuse further connections + from the site. However, both of these mechanisms open the door to + "denial of service" attacks, in which an attacker purposely initiates + the attack to disable access by a valid user. + + Standard FTP [PR85] sends passwords in clear text using the "PASS" + command. It is suggested that FTP clients and servers use alternate + authentication mechanisms that are not subject to eavesdropping (such + as the mechanisms being developed by the IETF Common Authentication + Technology Working Group [HL97]). + +6 Privacy + + All data and control information (including passwords) is sent across + the network in unencrypted form by standard FTP [PR85]. To guarantee + the privacy of the information FTP transmits, a strong encryption + scheme should be used whenever possible. One such mechanism is + defined in [HL97]. + +7 Protecting Usernames + + Standard FTP [PR85] specifies a 530 response to the USER command when + the username is rejected. If the username is valid and a password is + required FTP returns a 331 response instead. In order to prevent a + malicious client from determining valid usernames on a server, it is + suggested that a server always return 331 to the USER command and + + + +Allman & Ostermann Informational [Page 4] + +RFC 2577 FTP Security Considerations May 1999 + + + then reject the combination of username and password for an invalid + username. + +8 Port Stealing + + Many operating systems assign dynamic port numbers in increasing + order. By making a legitimate transfer, an attacker can observe the + current port number allocated by the server and "guess" the next one + that will be used. The attacker can make a connection to this port, + thus denying another legitimate client the ability to make a + transfer. Alternatively, the attacker can steal a file meant for a + legitimate user. In addition, an attacker can insert a forged file + into a data stream thought to come from an authenticated client. + This problem can be mitigated by making FTP clients and servers use + random local port numbers for data connections, either by requesting + random ports from the operating system or using system dependent + mechanisms. + +9 Software-Base Security Problems + + The emphasis in this document is on protocol-related security issues. + There are a number of documented FTP security-related problems that + are due to poor implementation as well. Although the details of + these types of problems are beyond the scope of this document, it + should be pointed out that the following FTP features has been abused + in the past and should be treated with great care by future + implementers: + + Anonymous FTP + + Anonymous FTP refers to the ability of a client to connect to an + FTP server with minimal authentication and gain access to public + files. Security problems arise when such a user can read all + files on the system or can create files. [CERT92:09] [CERT93:06] + + Remote Command Execution + + An optional FTP extension, "SITE EXEC", allows clients to execute + arbitrary commands on the server. This feature should obviously + be implemented with great care. There are several documented + cases of the FTP "SITE EXEC" command being used to subvert server + security [CERT94:08] [CERT95:16] + + Debug Code + + Several previous security compromises related to FTP can be + attributed to software that was installed with debugging features + enabled [CERT88:01]. + + + +Allman & Ostermann Informational [Page 5] + +RFC 2577 FTP Security Considerations May 1999 + + + This document recommends that implementors of FTP servers with these + capabilities review all of the CERT advisories for attacks on these + or similar mechanisms before releasing their software. + +10 Conclusion + + Using the above suggestions can decrease the security problems + associated with FTP servers without eliminating functionality. + +11 Security Considerations + + Security issues are discussed throughout this memo. + +Acknowledgments + + We would like to thank Alex Belits, Jim Bound, William Curtin, Robert + Elz, Paul Hethmon, Alun Jones and Stephen Tihor for their helpful + comments on this paper. Also, we thank the FTPEXT WG members who + gave many useful suggestions at the Memphis IETF meeting. + +References + + [AOM98] Allman, M., Ostermann, S. and C. Metz, "FTP Extensions + for IPv6 and NATs", RFC 2428, September 1998. + + [Bel94] Bellovin. S., "Firewall-Friendly FTP", RFC 1579, February + 1994. + + [CERT88:01] CERT Advisory CA-88:01. ftpd Vulnerability. December, + 1988 ftp://info.cert.org/pub/cert_advisories/ + + [CERT92:09] CERT Advisory CA-92:09. AIX Anonymous FTP Vulnerability. + April 27, 1992. ftp://info.cert.org/pub/cert_advisories/ + + [CERT93:06] CERT Advisory CA-93:06. Wuarchive ftpd Vulnerability. + September 19,1997 + ftp://info.cert.org/pub/cert_advisories/ + + [CERT94:08] CERT Advisory CA-94:08. ftpd Vulnerabilities. September + 23, 1997. ftp://info.cert.org/pub/cert_advisories/ + + [CERT95:16] CERT Advisory CA-95:16. wu-ftpd Misconfiguration + Vulnerability. September 23, 1997 + ftp://info.cert.org/pub/cert_advisories/ + + [CERT97:27] CERT Advisory CA-97.27. FTP Bounce. January 8, 1998. + ftp://info.cert.org/pub/cert_advisories/ + + + + +Allman & Ostermann Informational [Page 6] + +RFC 2577 FTP Security Considerations May 1999 + + + [HL97] Horowitz, M. and S. Lunt, "FTP Security Extensions", RFC + 2228, October 1997. + + [Pis94] Piscitello, D., "FTP Operation Over Big Address Records + (FOOBAR), RFC 1639, June 1994. + + [Pos81] Postel, J., "Transmission Control Protocol", STD 7, RFC + 793, September 1981. + + [PR85] Postel, J. and J. Reynolds, "File Transfer Protocol + (FTP)", STD 9, RFC 959, October 1985. + + [RP94] Reynolds, J. and J. Postel, "Assigned Numbers", STD 2, + RFC 1700, October 1994. See also: + http://www.iana.org/numbers.html + +Authors' Addresses + + Mark Allman + NASA Glenn Research Center/Sterling Software + 21000 Brookpark Rd. MS 54-2 + Cleveland, OH 44135 + + EMail: [email protected] + + + Shawn Ostermann + School of Electrical Engineering and Computer Science + Ohio University + 416 Morton Hall + Athens, OH 45701 + + EMail: [email protected] + + + + + + + + + + + + + + + + + + +Allman & Ostermann Informational [Page 7] + +RFC 2577 FTP Security Considerations May 1999 + + +Full Copyright Statement + + Copyright (C) The Internet Society (1999). All Rights Reserved. + + This document and translations of it may be copied and furnished to + others, and derivative works that comment on or otherwise explain it + or assist in its implementation may be prepared, copied, published + and distributed, in whole or in part, without restriction of any + kind, provided that the above copyright notice and this paragraph are + included on all such copies and derivative works. However, this + document itself may not be modified in any way, such as by removing + the copyright notice or references to the Internet Society or other + Internet organizations, except as needed for the purpose of + developing Internet standards in which case the procedures for + copyrights defined in the Internet Standards process must be + followed, or as required to translate it into languages other than + English. + + The limited permissions granted above are perpetual and will not be + revoked by the Internet Society or its successors or assigns. + + This document and the information contained herein is provided on an + "AS IS" basis and THE INTERNET SOCIETY AND THE INTERNET ENGINEERING + TASK FORCE DISCLAIMS ALL WARRANTIES, EXPRESS OR IMPLIED, INCLUDING + BUT NOT LIMITED TO ANY WARRANTY THAT THE USE OF THE INFORMATION + HEREIN WILL NOT INFRINGE ANY RIGHTS OR ANY IMPLIED WARRANTIES OF + MERCHANTABILITY OR FITNESS FOR A PARTICULAR PURPOSE. + +Acknowledgement + + Funding for the RFC Editor function is currently provided by the + Internet Society. + + + + + + + + + + + + + + + + + + + +Allman & Ostermann Informational [Page 8] + diff --git a/lib/inets/doc/archive/rfc2616.pdf b/lib/inets/doc/archive/rfc2616.pdf Binary files differnew file mode 100644 index 0000000000..d3d4bae2c8 --- /dev/null +++ b/lib/inets/doc/archive/rfc2616.pdf diff --git a/lib/inets/doc/archive/rfc2616.txt b/lib/inets/doc/archive/rfc2616.txt new file mode 100644 index 0000000000..45d7d08b8f --- /dev/null +++ b/lib/inets/doc/archive/rfc2616.txt @@ -0,0 +1,9859 @@ + + + + + + +Network Working Group R. Fielding +Request for Comments: 2616 UC Irvine +Obsoletes: 2068 J. Gettys +Category: Standards Track Compaq/W3C + J. Mogul + Compaq + H. Frystyk + W3C/MIT + L. Masinter + Xerox + P. Leach + Microsoft + T. Berners-Lee + W3C/MIT + June 1999 + + + Hypertext Transfer Protocol -- HTTP/1.1 + +Status of this Memo + + This document specifies an Internet standards track protocol for the + Internet community, and requests discussion and suggestions for + improvements. Please refer to the current edition of the "Internet + Official Protocol Standards" (STD 1) for the standardization state + and status of this protocol. Distribution of this memo is unlimited. + +Copyright Notice + + Copyright (C) The Internet Society (1999). All Rights Reserved. + +Abstract + + The Hypertext Transfer Protocol (HTTP) is an application-level + protocol for distributed, collaborative, hypermedia information + systems. It is a generic, stateless, protocol which can be used for + many tasks beyond its use for hypertext, such as name servers and + distributed object management systems, through extension of its + request methods, error codes and headers [47]. A feature of HTTP is + the typing and negotiation of data representation, allowing systems + to be built independently of the data being transferred. + + HTTP has been in use by the World-Wide Web global information + initiative since 1990. This specification defines the protocol + referred to as "HTTP/1.1", and is an update to RFC 2068 [33]. + + + + + + +Fielding, et al. Standards Track [Page 1] + +RFC 2616 HTTP/1.1 June 1999 + + +Table of Contents + + 1 Introduction ...................................................7 + 1.1 Purpose......................................................7 + 1.2 Requirements .................................................8 + 1.3 Terminology ..................................................8 + 1.4 Overall Operation ...........................................12 + 2 Notational Conventions and Generic Grammar ....................14 + 2.1 Augmented BNF ...............................................14 + 2.2 Basic Rules .................................................15 + 3 Protocol Parameters ...........................................17 + 3.1 HTTP Version ................................................17 + 3.2 Uniform Resource Identifiers ................................18 + 3.2.1 General Syntax ...........................................19 + 3.2.2 http URL .................................................19 + 3.2.3 URI Comparison ...........................................20 + 3.3 Date/Time Formats ...........................................20 + 3.3.1 Full Date ................................................20 + 3.3.2 Delta Seconds ............................................21 + 3.4 Character Sets ..............................................21 + 3.4.1 Missing Charset ..........................................22 + 3.5 Content Codings .............................................23 + 3.6 Transfer Codings ............................................24 + 3.6.1 Chunked Transfer Coding ..................................25 + 3.7 Media Types .................................................26 + 3.7.1 Canonicalization and Text Defaults .......................27 + 3.7.2 Multipart Types ..........................................27 + 3.8 Product Tokens ..............................................28 + 3.9 Quality Values ..............................................29 + 3.10 Language Tags ...............................................29 + 3.11 Entity Tags .................................................30 + 3.12 Range Units .................................................30 + 4 HTTP Message ..................................................31 + 4.1 Message Types ...............................................31 + 4.2 Message Headers .............................................31 + 4.3 Message Body ................................................32 + 4.4 Message Length ..............................................33 + 4.5 General Header Fields .......................................34 + 5 Request .......................................................35 + 5.1 Request-Line ................................................35 + 5.1.1 Method ...................................................36 + 5.1.2 Request-URI ..............................................36 + 5.2 The Resource Identified by a Request ........................38 + 5.3 Request Header Fields .......................................38 + 6 Response ......................................................39 + 6.1 Status-Line .................................................39 + 6.1.1 Status Code and Reason Phrase ............................39 + 6.2 Response Header Fields ......................................41 + + + +Fielding, et al. Standards Track [Page 2] + +RFC 2616 HTTP/1.1 June 1999 + + + 7 Entity ........................................................42 + 7.1 Entity Header Fields ........................................42 + 7.2 Entity Body .................................................43 + 7.2.1 Type .....................................................43 + 7.2.2 Entity Length ............................................43 + 8 Connections ...................................................44 + 8.1 Persistent Connections ......................................44 + 8.1.1 Purpose ..................................................44 + 8.1.2 Overall Operation ........................................45 + 8.1.3 Proxy Servers ............................................46 + 8.1.4 Practical Considerations .................................46 + 8.2 Message Transmission Requirements ...........................47 + 8.2.1 Persistent Connections and Flow Control ..................47 + 8.2.2 Monitoring Connections for Error Status Messages .........48 + 8.2.3 Use of the 100 (Continue) Status .........................48 + 8.2.4 Client Behavior if Server Prematurely Closes Connection ..50 + 9 Method Definitions ............................................51 + 9.1 Safe and Idempotent Methods .................................51 + 9.1.1 Safe Methods .............................................51 + 9.1.2 Idempotent Methods .......................................51 + 9.2 OPTIONS .....................................................52 + 9.3 GET .........................................................53 + 9.4 HEAD ........................................................54 + 9.5 POST ........................................................54 + 9.6 PUT .........................................................55 + 9.7 DELETE ......................................................56 + 9.8 TRACE .......................................................56 + 9.9 CONNECT .....................................................57 + 10 Status Code Definitions ......................................57 + 10.1 Informational 1xx ...........................................57 + 10.1.1 100 Continue .............................................58 + 10.1.2 101 Switching Protocols ..................................58 + 10.2 Successful 2xx ..............................................58 + 10.2.1 200 OK ...................................................58 + 10.2.2 201 Created ..............................................59 + 10.2.3 202 Accepted .............................................59 + 10.2.4 203 Non-Authoritative Information ........................59 + 10.2.5 204 No Content ...........................................60 + 10.2.6 205 Reset Content ........................................60 + 10.2.7 206 Partial Content ......................................60 + 10.3 Redirection 3xx .............................................61 + 10.3.1 300 Multiple Choices .....................................61 + 10.3.2 301 Moved Permanently ....................................62 + 10.3.3 302 Found ................................................62 + 10.3.4 303 See Other ............................................63 + 10.3.5 304 Not Modified .........................................63 + 10.3.6 305 Use Proxy ............................................64 + 10.3.7 306 (Unused) .............................................64 + + + +Fielding, et al. Standards Track [Page 3] + +RFC 2616 HTTP/1.1 June 1999 + + + 10.3.8 307 Temporary Redirect ...................................65 + 10.4 Client Error 4xx ............................................65 + 10.4.1 400 Bad Request .........................................65 + 10.4.2 401 Unauthorized ........................................66 + 10.4.3 402 Payment Required ....................................66 + 10.4.4 403 Forbidden ...........................................66 + 10.4.5 404 Not Found ...........................................66 + 10.4.6 405 Method Not Allowed ..................................66 + 10.4.7 406 Not Acceptable ......................................67 + 10.4.8 407 Proxy Authentication Required .......................67 + 10.4.9 408 Request Timeout .....................................67 + 10.4.10 409 Conflict ............................................67 + 10.4.11 410 Gone ................................................68 + 10.4.12 411 Length Required .....................................68 + 10.4.13 412 Precondition Failed .................................68 + 10.4.14 413 Request Entity Too Large ............................69 + 10.4.15 414 Request-URI Too Long ................................69 + 10.4.16 415 Unsupported Media Type ..............................69 + 10.4.17 416 Requested Range Not Satisfiable .....................69 + 10.4.18 417 Expectation Failed ..................................70 + 10.5 Server Error 5xx ............................................70 + 10.5.1 500 Internal Server Error ................................70 + 10.5.2 501 Not Implemented ......................................70 + 10.5.3 502 Bad Gateway ..........................................70 + 10.5.4 503 Service Unavailable ..................................70 + 10.5.5 504 Gateway Timeout ......................................71 + 10.5.6 505 HTTP Version Not Supported ...........................71 + 11 Access Authentication ........................................71 + 12 Content Negotiation ..........................................71 + 12.1 Server-driven Negotiation ...................................72 + 12.2 Agent-driven Negotiation ....................................73 + 12.3 Transparent Negotiation .....................................74 + 13 Caching in HTTP ..............................................74 + 13.1.1 Cache Correctness ........................................75 + 13.1.2 Warnings .................................................76 + 13.1.3 Cache-control Mechanisms .................................77 + 13.1.4 Explicit User Agent Warnings .............................78 + 13.1.5 Exceptions to the Rules and Warnings .....................78 + 13.1.6 Client-controlled Behavior ...............................79 + 13.2 Expiration Model ............................................79 + 13.2.1 Server-Specified Expiration ..............................79 + 13.2.2 Heuristic Expiration .....................................80 + 13.2.3 Age Calculations .........................................80 + 13.2.4 Expiration Calculations ..................................83 + 13.2.5 Disambiguating Expiration Values .........................84 + 13.2.6 Disambiguating Multiple Responses ........................84 + 13.3 Validation Model ............................................85 + 13.3.1 Last-Modified Dates ......................................86 + + + +Fielding, et al. Standards Track [Page 4] + +RFC 2616 HTTP/1.1 June 1999 + + + 13.3.2 Entity Tag Cache Validators ..............................86 + 13.3.3 Weak and Strong Validators ...............................86 + 13.3.4 Rules for When to Use Entity Tags and Last-Modified Dates.89 + 13.3.5 Non-validating Conditionals ..............................90 + 13.4 Response Cacheability .......................................91 + 13.5 Constructing Responses From Caches ..........................92 + 13.5.1 End-to-end and Hop-by-hop Headers ........................92 + 13.5.2 Non-modifiable Headers ...................................92 + 13.5.3 Combining Headers ........................................94 + 13.5.4 Combining Byte Ranges ....................................95 + 13.6 Caching Negotiated Responses ................................95 + 13.7 Shared and Non-Shared Caches ................................96 + 13.8 Errors or Incomplete Response Cache Behavior ................97 + 13.9 Side Effects of GET and HEAD ................................97 + 13.10 Invalidation After Updates or Deletions ...................97 + 13.11 Write-Through Mandatory ...................................98 + 13.12 Cache Replacement .........................................99 + 13.13 History Lists .............................................99 + 14 Header Field Definitions ....................................100 + 14.1 Accept .....................................................100 + 14.2 Accept-Charset .............................................102 + 14.3 Accept-Encoding ............................................102 + 14.4 Accept-Language ............................................104 + 14.5 Accept-Ranges ..............................................105 + 14.6 Age ........................................................106 + 14.7 Allow ......................................................106 + 14.8 Authorization ..............................................107 + 14.9 Cache-Control ..............................................108 + 14.9.1 What is Cacheable .......................................109 + 14.9.2 What May be Stored by Caches ............................110 + 14.9.3 Modifications of the Basic Expiration Mechanism .........111 + 14.9.4 Cache Revalidation and Reload Controls ..................113 + 14.9.5 No-Transform Directive ..................................115 + 14.9.6 Cache Control Extensions ................................116 + 14.10 Connection ...............................................117 + 14.11 Content-Encoding .........................................118 + 14.12 Content-Language .........................................118 + 14.13 Content-Length ...........................................119 + 14.14 Content-Location .........................................120 + 14.15 Content-MD5 ..............................................121 + 14.16 Content-Range ............................................122 + 14.17 Content-Type .............................................124 + 14.18 Date .....................................................124 + 14.18.1 Clockless Origin Server Operation ......................125 + 14.19 ETag .....................................................126 + 14.20 Expect ...................................................126 + 14.21 Expires ..................................................127 + 14.22 From .....................................................128 + + + +Fielding, et al. Standards Track [Page 5] + +RFC 2616 HTTP/1.1 June 1999 + + + 14.23 Host .....................................................128 + 14.24 If-Match .................................................129 + 14.25 If-Modified-Since ........................................130 + 14.26 If-None-Match ............................................132 + 14.27 If-Range .................................................133 + 14.28 If-Unmodified-Since ......................................134 + 14.29 Last-Modified ............................................134 + 14.30 Location .................................................135 + 14.31 Max-Forwards .............................................136 + 14.32 Pragma ...................................................136 + 14.33 Proxy-Authenticate .......................................137 + 14.34 Proxy-Authorization ......................................137 + 14.35 Range ....................................................138 + 14.35.1 Byte Ranges ...........................................138 + 14.35.2 Range Retrieval Requests ..............................139 + 14.36 Referer ..................................................140 + 14.37 Retry-After ..............................................141 + 14.38 Server ...................................................141 + 14.39 TE .......................................................142 + 14.40 Trailer ..................................................143 + 14.41 Transfer-Encoding..........................................143 + 14.42 Upgrade ..................................................144 + 14.43 User-Agent ...............................................145 + 14.44 Vary .....................................................145 + 14.45 Via ......................................................146 + 14.46 Warning ..................................................148 + 14.47 WWW-Authenticate .........................................150 + 15 Security Considerations .......................................150 + 15.1 Personal Information....................................151 + 15.1.1 Abuse of Server Log Information .........................151 + 15.1.2 Transfer of Sensitive Information .......................151 + 15.1.3 Encoding Sensitive Information in URI's .................152 + 15.1.4 Privacy Issues Connected to Accept Headers ..............152 + 15.2 Attacks Based On File and Path Names .......................153 + 15.3 DNS Spoofing ...............................................154 + 15.4 Location Headers and Spoofing ..............................154 + 15.5 Content-Disposition Issues .................................154 + 15.6 Authentication Credentials and Idle Clients ................155 + 15.7 Proxies and Caching ........................................155 + 15.7.1 Denial of Service Attacks on Proxies....................156 + 16 Acknowledgments .............................................156 + 17 References ..................................................158 + 18 Authors' Addresses ..........................................162 + 19 Appendices ..................................................164 + 19.1 Internet Media Type message/http and application/http ......164 + 19.2 Internet Media Type multipart/byteranges ...................165 + 19.3 Tolerant Applications ......................................166 + 19.4 Differences Between HTTP Entities and RFC 2045 Entities ....167 + + + +Fielding, et al. Standards Track [Page 6] + +RFC 2616 HTTP/1.1 June 1999 + + + 19.4.1 MIME-Version ............................................167 + 19.4.2 Conversion to Canonical Form ............................167 + 19.4.3 Conversion of Date Formats ..............................168 + 19.4.4 Introduction of Content-Encoding ........................168 + 19.4.5 No Content-Transfer-Encoding ............................168 + 19.4.6 Introduction of Transfer-Encoding .......................169 + 19.4.7 MHTML and Line Length Limitations .......................169 + 19.5 Additional Features ........................................169 + 19.5.1 Content-Disposition .....................................170 + 19.6 Compatibility with Previous Versions .......................170 + 19.6.1 Changes from HTTP/1.0 ...................................171 + 19.6.2 Compatibility with HTTP/1.0 Persistent Connections ......172 + 19.6.3 Changes from RFC 2068 ...................................172 + 20 Index .......................................................175 + 21 Full Copyright Statement ....................................176 + +1 Introduction + +1.1 Purpose + + The Hypertext Transfer Protocol (HTTP) is an application-level + protocol for distributed, collaborative, hypermedia information + systems. HTTP has been in use by the World-Wide Web global + information initiative since 1990. The first version of HTTP, + referred to as HTTP/0.9, was a simple protocol for raw data transfer + across the Internet. HTTP/1.0, as defined by RFC 1945 [6], improved + the protocol by allowing messages to be in the format of MIME-like + messages, containing metainformation about the data transferred and + modifiers on the request/response semantics. However, HTTP/1.0 does + not sufficiently take into consideration the effects of hierarchical + proxies, caching, the need for persistent connections, or virtual + hosts. In addition, the proliferation of incompletely-implemented + applications calling themselves "HTTP/1.0" has necessitated a + protocol version change in order for two communicating applications + to determine each other's true capabilities. + + This specification defines the protocol referred to as "HTTP/1.1". + This protocol includes more stringent requirements than HTTP/1.0 in + order to ensure reliable implementation of its features. + + Practical information systems require more functionality than simple + retrieval, including search, front-end update, and annotation. HTTP + allows an open-ended set of methods and headers that indicate the + purpose of a request [47]. It builds on the discipline of reference + provided by the Uniform Resource Identifier (URI) [3], as a location + (URL) [4] or name (URN) [20], for indicating the resource to which a + + + + + +Fielding, et al. Standards Track [Page 7] + +RFC 2616 HTTP/1.1 June 1999 + + + method is to be applied. Messages are passed in a format similar to + that used by Internet mail [9] as defined by the Multipurpose + Internet Mail Extensions (MIME) [7]. + + HTTP is also used as a generic protocol for communication between + user agents and proxies/gateways to other Internet systems, including + those supported by the SMTP [16], NNTP [13], FTP [18], Gopher [2], + and WAIS [10] protocols. In this way, HTTP allows basic hypermedia + access to resources available from diverse applications. + +1.2 Requirements + + The key words "MUST", "MUST NOT", "REQUIRED", "SHALL", "SHALL NOT", + "SHOULD", "SHOULD NOT", "RECOMMENDED", "MAY", and "OPTIONAL" in this + document are to be interpreted as described in RFC 2119 [34]. + + An implementation is not compliant if it fails to satisfy one or more + of the MUST or REQUIRED level requirements for the protocols it + implements. An implementation that satisfies all the MUST or REQUIRED + level and all the SHOULD level requirements for its protocols is said + to be "unconditionally compliant"; one that satisfies all the MUST + level requirements but not all the SHOULD level requirements for its + protocols is said to be "conditionally compliant." + +1.3 Terminology + + This specification uses a number of terms to refer to the roles + played by participants in, and objects of, the HTTP communication. + + connection + A transport layer virtual circuit established between two programs + for the purpose of communication. + + message + The basic unit of HTTP communication, consisting of a structured + sequence of octets matching the syntax defined in section 4 and + transmitted via the connection. + + request + An HTTP request message, as defined in section 5. + + response + An HTTP response message, as defined in section 6. + + + + + + + + +Fielding, et al. Standards Track [Page 8] + +RFC 2616 HTTP/1.1 June 1999 + + + resource + A network data object or service that can be identified by a URI, + as defined in section 3.2. Resources may be available in multiple + representations (e.g. multiple languages, data formats, size, and + resolutions) or vary in other ways. + + entity + The information transferred as the payload of a request or + response. An entity consists of metainformation in the form of + entity-header fields and content in the form of an entity-body, as + described in section 7. + + representation + An entity included with a response that is subject to content + negotiation, as described in section 12. There may exist multiple + representations associated with a particular response status. + + content negotiation + The mechanism for selecting the appropriate representation when + servicing a request, as described in section 12. The + representation of entities in any response can be negotiated + (including error responses). + + variant + A resource may have one, or more than one, representation(s) + associated with it at any given instant. Each of these + representations is termed a `varriant'. Use of the term `variant' + does not necessarily imply that the resource is subject to content + negotiation. + + client + A program that establishes connections for the purpose of sending + requests. + + user agent + The client which initiates a request. These are often browsers, + editors, spiders (web-traversing robots), or other end user tools. + + server + An application program that accepts connections in order to + service requests by sending back responses. Any given program may + be capable of being both a client and a server; our use of these + terms refers only to the role being performed by the program for a + particular connection, rather than to the program's capabilities + in general. Likewise, any server may act as an origin server, + proxy, gateway, or tunnel, switching behavior based on the nature + of each request. + + + + +Fielding, et al. Standards Track [Page 9] + +RFC 2616 HTTP/1.1 June 1999 + + + origin server + The server on which a given resource resides or is to be created. + + proxy + An intermediary program which acts as both a server and a client + for the purpose of making requests on behalf of other clients. + Requests are serviced internally or by passing them on, with + possible translation, to other servers. A proxy MUST implement + both the client and server requirements of this specification. A + "transparent proxy" is a proxy that does not modify the request or + response beyond what is required for proxy authentication and + identification. A "non-transparent proxy" is a proxy that modifies + the request or response in order to provide some added service to + the user agent, such as group annotation services, media type + transformation, protocol reduction, or anonymity filtering. Except + where either transparent or non-transparent behavior is explicitly + stated, the HTTP proxy requirements apply to both types of + proxies. + + gateway + A server which acts as an intermediary for some other server. + Unlike a proxy, a gateway receives requests as if it were the + origin server for the requested resource; the requesting client + may not be aware that it is communicating with a gateway. + + tunnel + An intermediary program which is acting as a blind relay between + two connections. Once active, a tunnel is not considered a party + to the HTTP communication, though the tunnel may have been + initiated by an HTTP request. The tunnel ceases to exist when both + ends of the relayed connections are closed. + + cache + A program's local store of response messages and the subsystem + that controls its message storage, retrieval, and deletion. A + cache stores cacheable responses in order to reduce the response + time and network bandwidth consumption on future, equivalent + requests. Any client or server may include a cache, though a cache + cannot be used by a server that is acting as a tunnel. + + cacheable + A response is cacheable if a cache is allowed to store a copy of + the response message for use in answering subsequent requests. The + rules for determining the cacheability of HTTP responses are + defined in section 13. Even if a resource is cacheable, there may + be additional constraints on whether a cache can use the cached + copy for a particular request. + + + + +Fielding, et al. Standards Track [Page 10] + +RFC 2616 HTTP/1.1 June 1999 + + + first-hand + A response is first-hand if it comes directly and without + unnecessary delay from the origin server, perhaps via one or more + proxies. A response is also first-hand if its validity has just + been checked directly with the origin server. + + explicit expiration time + The time at which the origin server intends that an entity should + no longer be returned by a cache without further validation. + + heuristic expiration time + An expiration time assigned by a cache when no explicit expiration + time is available. + + age + The age of a response is the time since it was sent by, or + successfully validated with, the origin server. + + freshness lifetime + The length of time between the generation of a response and its + expiration time. + + fresh + A response is fresh if its age has not yet exceeded its freshness + lifetime. + + stale + A response is stale if its age has passed its freshness lifetime. + + semantically transparent + A cache behaves in a "semantically transparent" manner, with + respect to a particular response, when its use affects neither the + requesting client nor the origin server, except to improve + performance. When a cache is semantically transparent, the client + receives exactly the same response (except for hop-by-hop headers) + that it would have received had its request been handled directly + by the origin server. + + validator + A protocol element (e.g., an entity tag or a Last-Modified time) + that is used to find out whether a cache entry is an equivalent + copy of an entity. + + upstream/downstream + Upstream and downstream describe the flow of a message: all + messages flow from upstream to downstream. + + + + + +Fielding, et al. Standards Track [Page 11] + +RFC 2616 HTTP/1.1 June 1999 + + + inbound/outbound + Inbound and outbound refer to the request and response paths for + messages: "inbound" means "traveling toward the origin server", + and "outbound" means "traveling toward the user agent" + +1.4 Overall Operation + + The HTTP protocol is a request/response protocol. A client sends a + request to the server in the form of a request method, URI, and + protocol version, followed by a MIME-like message containing request + modifiers, client information, and possible body content over a + connection with a server. The server responds with a status line, + including the message's protocol version and a success or error code, + followed by a MIME-like message containing server information, entity + metainformation, and possible entity-body content. The relationship + between HTTP and MIME is described in appendix 19.4. + + Most HTTP communication is initiated by a user agent and consists of + a request to be applied to a resource on some origin server. In the + simplest case, this may be accomplished via a single connection (v) + between the user agent (UA) and the origin server (O). + + request chain ------------------------> + UA -------------------v------------------- O + <----------------------- response chain + + A more complicated situation occurs when one or more intermediaries + are present in the request/response chain. There are three common + forms of intermediary: proxy, gateway, and tunnel. A proxy is a + forwarding agent, receiving requests for a URI in its absolute form, + rewriting all or part of the message, and forwarding the reformatted + request toward the server identified by the URI. A gateway is a + receiving agent, acting as a layer above some other server(s) and, if + necessary, translating the requests to the underlying server's + protocol. A tunnel acts as a relay point between two connections + without changing the messages; tunnels are used when the + communication needs to pass through an intermediary (such as a + firewall) even when the intermediary cannot understand the contents + of the messages. + + request chain --------------------------------------> + UA -----v----- A -----v----- B -----v----- C -----v----- O + <------------------------------------- response chain + + The figure above shows three intermediaries (A, B, and C) between the + user agent and origin server. A request or response message that + travels the whole chain will pass through four separate connections. + This distinction is important because some HTTP communication options + + + +Fielding, et al. Standards Track [Page 12] + +RFC 2616 HTTP/1.1 June 1999 + + + may apply only to the connection with the nearest, non-tunnel + neighbor, only to the end-points of the chain, or to all connections + along the chain. Although the diagram is linear, each participant may + be engaged in multiple, simultaneous communications. For example, B + may be receiving requests from many clients other than A, and/or + forwarding requests to servers other than C, at the same time that it + is handling A's request. + + Any party to the communication which is not acting as a tunnel may + employ an internal cache for handling requests. The effect of a cache + is that the request/response chain is shortened if one of the + participants along the chain has a cached response applicable to that + request. The following illustrates the resulting chain if B has a + cached copy of an earlier response from O (via C) for a request which + has not been cached by UA or A. + + request chain ----------> + UA -----v----- A -----v----- B - - - - - - C - - - - - - O + <--------- response chain + + Not all responses are usefully cacheable, and some requests may + contain modifiers which place special requirements on cache behavior. + HTTP requirements for cache behavior and cacheable responses are + defined in section 13. + + In fact, there are a wide variety of architectures and configurations + of caches and proxies currently being experimented with or deployed + across the World Wide Web. These systems include national hierarchies + of proxy caches to save transoceanic bandwidth, systems that + broadcast or multicast cache entries, organizations that distribute + subsets of cached data via CD-ROM, and so on. HTTP systems are used + in corporate intranets over high-bandwidth links, and for access via + PDAs with low-power radio links and intermittent connectivity. The + goal of HTTP/1.1 is to support the wide diversity of configurations + already deployed while introducing protocol constructs that meet the + needs of those who build web applications that require high + reliability and, failing that, at least reliable indications of + failure. + + HTTP communication usually takes place over TCP/IP connections. The + default port is TCP 80 [19], but other ports can be used. This does + not preclude HTTP from being implemented on top of any other protocol + on the Internet, or on other networks. HTTP only presumes a reliable + transport; any protocol that provides such guarantees can be used; + the mapping of the HTTP/1.1 request and response structures onto the + transport data units of the protocol in question is outside the scope + of this specification. + + + + +Fielding, et al. Standards Track [Page 13] + +RFC 2616 HTTP/1.1 June 1999 + + + In HTTP/1.0, most implementations used a new connection for each + request/response exchange. In HTTP/1.1, a connection may be used for + one or more request/response exchanges, although connections may be + closed for a variety of reasons (see section 8.1). + +2 Notational Conventions and Generic Grammar + +2.1 Augmented BNF + + All of the mechanisms specified in this document are described in + both prose and an augmented Backus-Naur Form (BNF) similar to that + used by RFC 822 [9]. Implementors will need to be familiar with the + notation in order to understand this specification. The augmented BNF + includes the following constructs: + + name = definition + The name of a rule is simply the name itself (without any + enclosing "<" and ">") and is separated from its definition by the + equal "=" character. White space is only significant in that + indentation of continuation lines is used to indicate a rule + definition that spans more than one line. Certain basic rules are + in uppercase, such as SP, LWS, HT, CRLF, DIGIT, ALPHA, etc. Angle + brackets are used within definitions whenever their presence will + facilitate discerning the use of rule names. + + "literal" + Quotation marks surround literal text. Unless stated otherwise, + the text is case-insensitive. + + rule1 | rule2 + Elements separated by a bar ("|") are alternatives, e.g., "yes | + no" will accept yes or no. + + (rule1 rule2) + Elements enclosed in parentheses are treated as a single element. + Thus, "(elem (foo | bar) elem)" allows the token sequences "elem + foo elem" and "elem bar elem". + + *rule + The character "*" preceding an element indicates repetition. The + full form is "<n>*<m>element" indicating at least <n> and at most + <m> occurrences of element. Default values are 0 and infinity so + that "*(element)" allows any number, including zero; "1*element" + requires at least one; and "1*2element" allows one or two. + + [rule] + Square brackets enclose optional elements; "[foo bar]" is + equivalent to "*1(foo bar)". + + + +Fielding, et al. Standards Track [Page 14] + +RFC 2616 HTTP/1.1 June 1999 + + + N rule + Specific repetition: "<n>(element)" is equivalent to + "<n>*<n>(element)"; that is, exactly <n> occurrences of (element). + Thus 2DIGIT is a 2-digit number, and 3ALPHA is a string of three + alphabetic characters. + + #rule + A construct "#" is defined, similar to "*", for defining lists of + elements. The full form is "<n>#<m>element" indicating at least + <n> and at most <m> elements, each separated by one or more commas + (",") and OPTIONAL linear white space (LWS). This makes the usual + form of lists very easy; a rule such as + ( *LWS element *( *LWS "," *LWS element )) + can be shown as + 1#element + Wherever this construct is used, null elements are allowed, but do + not contribute to the count of elements present. That is, + "(element), , (element) " is permitted, but counts as only two + elements. Therefore, where at least one element is required, at + least one non-null element MUST be present. Default values are 0 + and infinity so that "#element" allows any number, including zero; + "1#element" requires at least one; and "1#2element" allows one or + two. + + ; comment + A semi-colon, set off some distance to the right of rule text, + starts a comment that continues to the end of line. This is a + simple way of including useful notes in parallel with the + specifications. + + implied *LWS + The grammar described by this specification is word-based. Except + where noted otherwise, linear white space (LWS) can be included + between any two adjacent words (token or quoted-string), and + between adjacent words and separators, without changing the + interpretation of a field. At least one delimiter (LWS and/or + + separators) MUST exist between any two tokens (for the definition + of "token" below), since they would otherwise be interpreted as a + single token. + +2.2 Basic Rules + + The following rules are used throughout this specification to + describe basic parsing constructs. The US-ASCII coded character set + is defined by ANSI X3.4-1986 [21]. + + + + + +Fielding, et al. Standards Track [Page 15] + +RFC 2616 HTTP/1.1 June 1999 + + + OCTET = <any 8-bit sequence of data> + CHAR = <any US-ASCII character (octets 0 - 127)> + UPALPHA = <any US-ASCII uppercase letter "A".."Z"> + LOALPHA = <any US-ASCII lowercase letter "a".."z"> + ALPHA = UPALPHA | LOALPHA + DIGIT = <any US-ASCII digit "0".."9"> + CTL = <any US-ASCII control character + (octets 0 - 31) and DEL (127)> + CR = <US-ASCII CR, carriage return (13)> + LF = <US-ASCII LF, linefeed (10)> + SP = <US-ASCII SP, space (32)> + HT = <US-ASCII HT, horizontal-tab (9)> + <"> = <US-ASCII double-quote mark (34)> + + HTTP/1.1 defines the sequence CR LF as the end-of-line marker for all + protocol elements except the entity-body (see appendix 19.3 for + tolerant applications). The end-of-line marker within an entity-body + is defined by its associated media type, as described in section 3.7. + + CRLF = CR LF + + HTTP/1.1 header field values can be folded onto multiple lines if the + continuation line begins with a space or horizontal tab. All linear + white space, including folding, has the same semantics as SP. A + recipient MAY replace any linear white space with a single SP before + interpreting the field value or forwarding the message downstream. + + LWS = [CRLF] 1*( SP | HT ) + + The TEXT rule is only used for descriptive field contents and values + that are not intended to be interpreted by the message parser. Words + of *TEXT MAY contain characters from character sets other than ISO- + 8859-1 [22] only when encoded according to the rules of RFC 2047 + [14]. + + TEXT = <any OCTET except CTLs, + but including LWS> + + A CRLF is allowed in the definition of TEXT only as part of a header + field continuation. It is expected that the folding LWS will be + replaced with a single SP before interpretation of the TEXT value. + + Hexadecimal numeric characters are used in several protocol elements. + + HEX = "A" | "B" | "C" | "D" | "E" | "F" + | "a" | "b" | "c" | "d" | "e" | "f" | DIGIT + + + + + +Fielding, et al. Standards Track [Page 16] + +RFC 2616 HTTP/1.1 June 1999 + + + Many HTTP/1.1 header field values consist of words separated by LWS + or special characters. These special characters MUST be in a quoted + string to be used within a parameter value (as defined in section + 3.6). + + token = 1*<any CHAR except CTLs or separators> + separators = "(" | ")" | "<" | ">" | "@" + | "," | ";" | ":" | "\" | <"> + | "/" | "[" | "]" | "?" | "=" + | "{" | "}" | SP | HT + + Comments can be included in some HTTP header fields by surrounding + the comment text with parentheses. Comments are only allowed in + fields containing "comment" as part of their field value definition. + In all other fields, parentheses are considered part of the field + value. + + comment = "(" *( ctext | quoted-pair | comment ) ")" + ctext = <any TEXT excluding "(" and ")"> + + A string of text is parsed as a single word if it is quoted using + double-quote marks. + + quoted-string = ( <"> *(qdtext | quoted-pair ) <"> ) + qdtext = <any TEXT except <">> + + The backslash character ("\") MAY be used as a single-character + quoting mechanism only within quoted-string and comment constructs. + + quoted-pair = "\" CHAR + +3 Protocol Parameters + +3.1 HTTP Version + + HTTP uses a "<major>.<minor>" numbering scheme to indicate versions + of the protocol. The protocol versioning policy is intended to allow + the sender to indicate the format of a message and its capacity for + understanding further HTTP communication, rather than the features + obtained via that communication. No change is made to the version + number for the addition of message components which do not affect + communication behavior or which only add to extensible field values. + The <minor> number is incremented when the changes made to the + protocol add features which do not change the general message parsing + algorithm, but which may add to the message semantics and imply + additional capabilities of the sender. The <major> number is + incremented when the format of a message within the protocol is + changed. See RFC 2145 [36] for a fuller explanation. + + + +Fielding, et al. Standards Track [Page 17] + +RFC 2616 HTTP/1.1 June 1999 + + + The version of an HTTP message is indicated by an HTTP-Version field + in the first line of the message. + + HTTP-Version = "HTTP" "/" 1*DIGIT "." 1*DIGIT + + Note that the major and minor numbers MUST be treated as separate + integers and that each MAY be incremented higher than a single digit. + Thus, HTTP/2.4 is a lower version than HTTP/2.13, which in turn is + lower than HTTP/12.3. Leading zeros MUST be ignored by recipients and + MUST NOT be sent. + + An application that sends a request or response message that includes + HTTP-Version of "HTTP/1.1" MUST be at least conditionally compliant + with this specification. Applications that are at least conditionally + compliant with this specification SHOULD use an HTTP-Version of + "HTTP/1.1" in their messages, and MUST do so for any message that is + not compatible with HTTP/1.0. For more details on when to send + specific HTTP-Version values, see RFC 2145 [36]. + + The HTTP version of an application is the highest HTTP version for + which the application is at least conditionally compliant. + + Proxy and gateway applications need to be careful when forwarding + messages in protocol versions different from that of the application. + Since the protocol version indicates the protocol capability of the + sender, a proxy/gateway MUST NOT send a message with a version + indicator which is greater than its actual version. If a higher + version request is received, the proxy/gateway MUST either downgrade + the request version, or respond with an error, or switch to tunnel + behavior. + + Due to interoperability problems with HTTP/1.0 proxies discovered + since the publication of RFC 2068[33], caching proxies MUST, gateways + MAY, and tunnels MUST NOT upgrade the request to the highest version + they support. The proxy/gateway's response to that request MUST be in + the same major version as the request. + + Note: Converting between versions of HTTP may involve modification + of header fields required or forbidden by the versions involved. + +3.2 Uniform Resource Identifiers + + URIs have been known by many names: WWW addresses, Universal Document + Identifiers, Universal Resource Identifiers [3], and finally the + combination of Uniform Resource Locators (URL) [4] and Names (URN) + [20]. As far as HTTP is concerned, Uniform Resource Identifiers are + simply formatted strings which identify--via name, location, or any + other characteristic--a resource. + + + +Fielding, et al. Standards Track [Page 18] + +RFC 2616 HTTP/1.1 June 1999 + + +3.2.1 General Syntax + + URIs in HTTP can be represented in absolute form or relative to some + known base URI [11], depending upon the context of their use. The two + forms are differentiated by the fact that absolute URIs always begin + with a scheme name followed by a colon. For definitive information on + URL syntax and semantics, see "Uniform Resource Identifiers (URI): + Generic Syntax and Semantics," RFC 2396 [42] (which replaces RFCs + 1738 [4] and RFC 1808 [11]). This specification adopts the + definitions of "URI-reference", "absoluteURI", "relativeURI", "port", + "host","abs_path", "rel_path", and "authority" from that + specification. + + The HTTP protocol does not place any a priori limit on the length of + a URI. Servers MUST be able to handle the URI of any resource they + serve, and SHOULD be able to handle URIs of unbounded length if they + provide GET-based forms that could generate such URIs. A server + SHOULD return 414 (Request-URI Too Long) status if a URI is longer + than the server can handle (see section 10.4.15). + + Note: Servers ought to be cautious about depending on URI lengths + above 255 bytes, because some older client or proxy + implementations might not properly support these lengths. + +3.2.2 http URL + + The "http" scheme is used to locate network resources via the HTTP + protocol. This section defines the scheme-specific syntax and + semantics for http URLs. + + http_URL = "http:" "//" host [ ":" port ] [ abs_path [ "?" query ]] + + If the port is empty or not given, port 80 is assumed. The semantics + are that the identified resource is located at the server listening + for TCP connections on that port of that host, and the Request-URI + for the resource is abs_path (section 5.1.2). The use of IP addresses + in URLs SHOULD be avoided whenever possible (see RFC 1900 [24]). If + the abs_path is not present in the URL, it MUST be given as "/" when + used as a Request-URI for a resource (section 5.1.2). If a proxy + receives a host name which is not a fully qualified domain name, it + MAY add its domain to the host name it received. If a proxy receives + a fully qualified domain name, the proxy MUST NOT change the host + name. + + + + + + + + +Fielding, et al. Standards Track [Page 19] + +RFC 2616 HTTP/1.1 June 1999 + + +3.2.3 URI Comparison + + When comparing two URIs to decide if they match or not, a client + SHOULD use a case-sensitive octet-by-octet comparison of the entire + URIs, with these exceptions: + + - A port that is empty or not given is equivalent to the default + port for that URI-reference; + + - Comparisons of host names MUST be case-insensitive; + + - Comparisons of scheme names MUST be case-insensitive; + + - An empty abs_path is equivalent to an abs_path of "/". + + Characters other than those in the "reserved" and "unsafe" sets (see + RFC 2396 [42]) are equivalent to their ""%" HEX HEX" encoding. + + For example, the following three URIs are equivalent: + + http://abc.com:80/~smith/home.html + http://ABC.com/%7Esmith/home.html + http://ABC.com:/%7esmith/home.html + +3.3 Date/Time Formats + +3.3.1 Full Date + + HTTP applications have historically allowed three different formats + for the representation of date/time stamps: + + Sun, 06 Nov 1994 08:49:37 GMT ; RFC 822, updated by RFC 1123 + Sunday, 06-Nov-94 08:49:37 GMT ; RFC 850, obsoleted by RFC 1036 + Sun Nov 6 08:49:37 1994 ; ANSI C's asctime() format + + The first format is preferred as an Internet standard and represents + a fixed-length subset of that defined by RFC 1123 [8] (an update to + RFC 822 [9]). The second format is in common use, but is based on the + obsolete RFC 850 [12] date format and lacks a four-digit year. + HTTP/1.1 clients and servers that parse the date value MUST accept + all three formats (for compatibility with HTTP/1.0), though they MUST + only generate the RFC 1123 format for representing HTTP-date values + in header fields. See section 19.3 for further information. + + Note: Recipients of date values are encouraged to be robust in + accepting date values that may have been sent by non-HTTP + applications, as is sometimes the case when retrieving or posting + messages via proxies/gateways to SMTP or NNTP. + + + +Fielding, et al. Standards Track [Page 20] + +RFC 2616 HTTP/1.1 June 1999 + + + All HTTP date/time stamps MUST be represented in Greenwich Mean Time + (GMT), without exception. For the purposes of HTTP, GMT is exactly + equal to UTC (Coordinated Universal Time). This is indicated in the + first two formats by the inclusion of "GMT" as the three-letter + abbreviation for time zone, and MUST be assumed when reading the + asctime format. HTTP-date is case sensitive and MUST NOT include + additional LWS beyond that specifically included as SP in the + grammar. + + HTTP-date = rfc1123-date | rfc850-date | asctime-date + rfc1123-date = wkday "," SP date1 SP time SP "GMT" + rfc850-date = weekday "," SP date2 SP time SP "GMT" + asctime-date = wkday SP date3 SP time SP 4DIGIT + date1 = 2DIGIT SP month SP 4DIGIT + ; day month year (e.g., 02 Jun 1982) + date2 = 2DIGIT "-" month "-" 2DIGIT + ; day-month-year (e.g., 02-Jun-82) + date3 = month SP ( 2DIGIT | ( SP 1DIGIT )) + ; month day (e.g., Jun 2) + time = 2DIGIT ":" 2DIGIT ":" 2DIGIT + ; 00:00:00 - 23:59:59 + wkday = "Mon" | "Tue" | "Wed" + | "Thu" | "Fri" | "Sat" | "Sun" + weekday = "Monday" | "Tuesday" | "Wednesday" + | "Thursday" | "Friday" | "Saturday" | "Sunday" + month = "Jan" | "Feb" | "Mar" | "Apr" + | "May" | "Jun" | "Jul" | "Aug" + | "Sep" | "Oct" | "Nov" | "Dec" + + Note: HTTP requirements for the date/time stamp format apply only + to their usage within the protocol stream. Clients and servers are + not required to use these formats for user presentation, request + logging, etc. + +3.3.2 Delta Seconds + + Some HTTP header fields allow a time value to be specified as an + integer number of seconds, represented in decimal, after the time + that the message was received. + + delta-seconds = 1*DIGIT + +3.4 Character Sets + + HTTP uses the same definition of the term "character set" as that + described for MIME: + + + + + +Fielding, et al. Standards Track [Page 21] + +RFC 2616 HTTP/1.1 June 1999 + + + The term "character set" is used in this document to refer to a + method used with one or more tables to convert a sequence of octets + into a sequence of characters. Note that unconditional conversion in + the other direction is not required, in that not all characters may + be available in a given character set and a character set may provide + more than one sequence of octets to represent a particular character. + This definition is intended to allow various kinds of character + encoding, from simple single-table mappings such as US-ASCII to + complex table switching methods such as those that use ISO-2022's + techniques. However, the definition associated with a MIME character + set name MUST fully specify the mapping to be performed from octets + to characters. In particular, use of external profiling information + to determine the exact mapping is not permitted. + + Note: This use of the term "character set" is more commonly + referred to as a "character encoding." However, since HTTP and + MIME share the same registry, it is important that the terminology + also be shared. + + HTTP character sets are identified by case-insensitive tokens. The + complete set of tokens is defined by the IANA Character Set registry + [19]. + + charset = token + + Although HTTP allows an arbitrary token to be used as a charset + value, any token that has a predefined value within the IANA + Character Set registry [19] MUST represent the character set defined + by that registry. Applications SHOULD limit their use of character + sets to those defined by the IANA registry. + + Implementors should be aware of IETF character set requirements [38] + [41]. + +3.4.1 Missing Charset + + Some HTTP/1.0 software has interpreted a Content-Type header without + charset parameter incorrectly to mean "recipient should guess." + Senders wishing to defeat this behavior MAY include a charset + parameter even when the charset is ISO-8859-1 and SHOULD do so when + it is known that it will not confuse the recipient. + + Unfortunately, some older HTTP/1.0 clients did not deal properly with + an explicit charset parameter. HTTP/1.1 recipients MUST respect the + charset label provided by the sender; and those user agents that have + a provision to "guess" a charset MUST use the charset from the + + + + + +Fielding, et al. Standards Track [Page 22] + +RFC 2616 HTTP/1.1 June 1999 + + + content-type field if they support that charset, rather than the + recipient's preference, when initially displaying a document. See + section 3.7.1. + +3.5 Content Codings + + Content coding values indicate an encoding transformation that has + been or can be applied to an entity. Content codings are primarily + used to allow a document to be compressed or otherwise usefully + transformed without losing the identity of its underlying media type + and without loss of information. Frequently, the entity is stored in + coded form, transmitted directly, and only decoded by the recipient. + + content-coding = token + + All content-coding values are case-insensitive. HTTP/1.1 uses + content-coding values in the Accept-Encoding (section 14.3) and + Content-Encoding (section 14.11) header fields. Although the value + describes the content-coding, what is more important is that it + indicates what decoding mechanism will be required to remove the + encoding. + + The Internet Assigned Numbers Authority (IANA) acts as a registry for + content-coding value tokens. Initially, the registry contains the + following tokens: + + gzip An encoding format produced by the file compression program + "gzip" (GNU zip) as described in RFC 1952 [25]. This format is a + Lempel-Ziv coding (LZ77) with a 32 bit CRC. + + compress + The encoding format produced by the common UNIX file compression + program "compress". This format is an adaptive Lempel-Ziv-Welch + coding (LZW). + + Use of program names for the identification of encoding formats + is not desirable and is discouraged for future encodings. Their + use here is representative of historical practice, not good + design. For compatibility with previous implementations of HTTP, + applications SHOULD consider "x-gzip" and "x-compress" to be + equivalent to "gzip" and "compress" respectively. + + deflate + The "zlib" format defined in RFC 1950 [31] in combination with + the "deflate" compression mechanism described in RFC 1951 [29]. + + + + + + +Fielding, et al. Standards Track [Page 23] + +RFC 2616 HTTP/1.1 June 1999 + + + identity + The default (identity) encoding; the use of no transformation + whatsoever. This content-coding is used only in the Accept- + Encoding header, and SHOULD NOT be used in the Content-Encoding + header. + + New content-coding value tokens SHOULD be registered; to allow + interoperability between clients and servers, specifications of the + content coding algorithms needed to implement a new value SHOULD be + publicly available and adequate for independent implementation, and + conform to the purpose of content coding defined in this section. + +3.6 Transfer Codings + + Transfer-coding values are used to indicate an encoding + transformation that has been, can be, or may need to be applied to an + entity-body in order to ensure "safe transport" through the network. + This differs from a content coding in that the transfer-coding is a + property of the message, not of the original entity. + + transfer-coding = "chunked" | transfer-extension + transfer-extension = token *( ";" parameter ) + + Parameters are in the form of attribute/value pairs. + + parameter = attribute "=" value + attribute = token + value = token | quoted-string + + All transfer-coding values are case-insensitive. HTTP/1.1 uses + transfer-coding values in the TE header field (section 14.39) and in + the Transfer-Encoding header field (section 14.41). + + Whenever a transfer-coding is applied to a message-body, the set of + transfer-codings MUST include "chunked", unless the message is + terminated by closing the connection. When the "chunked" transfer- + coding is used, it MUST be the last transfer-coding applied to the + message-body. The "chunked" transfer-coding MUST NOT be applied more + than once to a message-body. These rules allow the recipient to + determine the transfer-length of the message (section 4.4). + + Transfer-codings are analogous to the Content-Transfer-Encoding + values of MIME [7], which were designed to enable safe transport of + binary data over a 7-bit transport service. However, safe transport + has a different focus for an 8bit-clean transfer protocol. In HTTP, + the only unsafe characteristic of message-bodies is the difficulty in + determining the exact body length (section 7.2.2), or the desire to + encrypt data over a shared transport. + + + +Fielding, et al. Standards Track [Page 24] + +RFC 2616 HTTP/1.1 June 1999 + + + The Internet Assigned Numbers Authority (IANA) acts as a registry for + transfer-coding value tokens. Initially, the registry contains the + following tokens: "chunked" (section 3.6.1), "identity" (section + 3.6.2), "gzip" (section 3.5), "compress" (section 3.5), and "deflate" + (section 3.5). + + New transfer-coding value tokens SHOULD be registered in the same way + as new content-coding value tokens (section 3.5). + + A server which receives an entity-body with a transfer-coding it does + not understand SHOULD return 501 (Unimplemented), and close the + connection. A server MUST NOT send transfer-codings to an HTTP/1.0 + client. + +3.6.1 Chunked Transfer Coding + + The chunked encoding modifies the body of a message in order to + transfer it as a series of chunks, each with its own size indicator, + followed by an OPTIONAL trailer containing entity-header fields. This + allows dynamically produced content to be transferred along with the + information necessary for the recipient to verify that it has + received the full message. + + Chunked-Body = *chunk + last-chunk + trailer + CRLF + + chunk = chunk-size [ chunk-extension ] CRLF + chunk-data CRLF + chunk-size = 1*HEX + last-chunk = 1*("0") [ chunk-extension ] CRLF + + chunk-extension= *( ";" chunk-ext-name [ "=" chunk-ext-val ] ) + chunk-ext-name = token + chunk-ext-val = token | quoted-string + chunk-data = chunk-size(OCTET) + trailer = *(entity-header CRLF) + + The chunk-size field is a string of hex digits indicating the size of + the chunk. The chunked encoding is ended by any chunk whose size is + zero, followed by the trailer, which is terminated by an empty line. + + The trailer allows the sender to include additional HTTP header + fields at the end of the message. The Trailer header field can be + used to indicate which header fields are included in a trailer (see + section 14.40). + + + + +Fielding, et al. Standards Track [Page 25] + +RFC 2616 HTTP/1.1 June 1999 + + + A server using chunked transfer-coding in a response MUST NOT use the + trailer for any header fields unless at least one of the following is + true: + + a)the request included a TE header field that indicates "trailers" is + acceptable in the transfer-coding of the response, as described in + section 14.39; or, + + b)the server is the origin server for the response, the trailer + fields consist entirely of optional metadata, and the recipient + could use the message (in a manner acceptable to the origin server) + without receiving this metadata. In other words, the origin server + is willing to accept the possibility that the trailer fields might + be silently discarded along the path to the client. + + This requirement prevents an interoperability failure when the + message is being received by an HTTP/1.1 (or later) proxy and + forwarded to an HTTP/1.0 recipient. It avoids a situation where + compliance with the protocol would have necessitated a possibly + infinite buffer on the proxy. + + An example process for decoding a Chunked-Body is presented in + appendix 19.4.6. + + All HTTP/1.1 applications MUST be able to receive and decode the + "chunked" transfer-coding, and MUST ignore chunk-extension extensions + they do not understand. + +3.7 Media Types + + HTTP uses Internet Media Types [17] in the Content-Type (section + 14.17) and Accept (section 14.1) header fields in order to provide + open and extensible data typing and type negotiation. + + media-type = type "/" subtype *( ";" parameter ) + type = token + subtype = token + + Parameters MAY follow the type/subtype in the form of attribute/value + pairs (as defined in section 3.6). + + The type, subtype, and parameter attribute names are case- + insensitive. Parameter values might or might not be case-sensitive, + depending on the semantics of the parameter name. Linear white space + (LWS) MUST NOT be used between the type and subtype, nor between an + attribute and its value. The presence or absence of a parameter might + be significant to the processing of a media-type, depending on its + definition within the media type registry. + + + +Fielding, et al. Standards Track [Page 26] + +RFC 2616 HTTP/1.1 June 1999 + + + Note that some older HTTP applications do not recognize media type + parameters. When sending data to older HTTP applications, + implementations SHOULD only use media type parameters when they are + required by that type/subtype definition. + + Media-type values are registered with the Internet Assigned Number + Authority (IANA [19]). The media type registration process is + outlined in RFC 1590 [17]. Use of non-registered media types is + discouraged. + +3.7.1 Canonicalization and Text Defaults + + Internet media types are registered with a canonical form. An + entity-body transferred via HTTP messages MUST be represented in the + appropriate canonical form prior to its transmission except for + "text" types, as defined in the next paragraph. + + When in canonical form, media subtypes of the "text" type use CRLF as + the text line break. HTTP relaxes this requirement and allows the + transport of text media with plain CR or LF alone representing a line + break when it is done consistently for an entire entity-body. HTTP + applications MUST accept CRLF, bare CR, and bare LF as being + representative of a line break in text media received via HTTP. In + addition, if the text is represented in a character set that does not + use octets 13 and 10 for CR and LF respectively, as is the case for + some multi-byte character sets, HTTP allows the use of whatever octet + sequences are defined by that character set to represent the + equivalent of CR and LF for line breaks. This flexibility regarding + line breaks applies only to text media in the entity-body; a bare CR + or LF MUST NOT be substituted for CRLF within any of the HTTP control + structures (such as header fields and multipart boundaries). + + If an entity-body is encoded with a content-coding, the underlying + data MUST be in a form defined above prior to being encoded. + + The "charset" parameter is used with some media types to define the + character set (section 3.4) of the data. When no explicit charset + parameter is provided by the sender, media subtypes of the "text" + type are defined to have a default charset value of "ISO-8859-1" when + received via HTTP. Data in character sets other than "ISO-8859-1" or + its subsets MUST be labeled with an appropriate charset value. See + section 3.4.1 for compatibility problems. + +3.7.2 Multipart Types + + MIME provides for a number of "multipart" types -- encapsulations of + one or more entities within a single message-body. All multipart + types share a common syntax, as defined in section 5.1.1 of RFC 2046 + + + +Fielding, et al. Standards Track [Page 27] + +RFC 2616 HTTP/1.1 June 1999 + + + [40], and MUST include a boundary parameter as part of the media type + value. The message body is itself a protocol element and MUST + therefore use only CRLF to represent line breaks between body-parts. + Unlike in RFC 2046, the epilogue of any multipart message MUST be + empty; HTTP applications MUST NOT transmit the epilogue (even if the + original multipart contains an epilogue). These restrictions exist in + order to preserve the self-delimiting nature of a multipart message- + body, wherein the "end" of the message-body is indicated by the + ending multipart boundary. + + In general, HTTP treats a multipart message-body no differently than + any other media type: strictly as payload. The one exception is the + "multipart/byteranges" type (appendix 19.2) when it appears in a 206 + (Partial Content) response, which will be interpreted by some HTTP + caching mechanisms as described in sections 13.5.4 and 14.16. In all + other cases, an HTTP user agent SHOULD follow the same or similar + behavior as a MIME user agent would upon receipt of a multipart type. + The MIME header fields within each body-part of a multipart message- + body do not have any significance to HTTP beyond that defined by + their MIME semantics. + + In general, an HTTP user agent SHOULD follow the same or similar + behavior as a MIME user agent would upon receipt of a multipart type. + If an application receives an unrecognized multipart subtype, the + application MUST treat it as being equivalent to "multipart/mixed". + + Note: The "multipart/form-data" type has been specifically defined + for carrying form data suitable for processing via the POST + request method, as described in RFC 1867 [15]. + +3.8 Product Tokens + + Product tokens are used to allow communicating applications to + identify themselves by software name and version. Most fields using + product tokens also allow sub-products which form a significant part + of the application to be listed, separated by white space. By + convention, the products are listed in order of their significance + for identifying the application. + + product = token ["/" product-version] + product-version = token + + Examples: + + User-Agent: CERN-LineMode/2.15 libwww/2.17b3 + Server: Apache/0.8.4 + + + + + +Fielding, et al. Standards Track [Page 28] + +RFC 2616 HTTP/1.1 June 1999 + + + Product tokens SHOULD be short and to the point. They MUST NOT be + used for advertising or other non-essential information. Although any + token character MAY appear in a product-version, this token SHOULD + only be used for a version identifier (i.e., successive versions of + the same product SHOULD only differ in the product-version portion of + the product value). + +3.9 Quality Values + + HTTP content negotiation (section 12) uses short "floating point" + numbers to indicate the relative importance ("weight") of various + negotiable parameters. A weight is normalized to a real number in + the range 0 through 1, where 0 is the minimum and 1 the maximum + value. If a parameter has a quality value of 0, then content with + this parameter is `not acceptable' for the client. HTTP/1.1 + applications MUST NOT generate more than three digits after the + decimal point. User configuration of these values SHOULD also be + limited in this fashion. + + qvalue = ( "0" [ "." 0*3DIGIT ] ) + | ( "1" [ "." 0*3("0") ] ) + + "Quality values" is a misnomer, since these values merely represent + relative degradation in desired quality. + +3.10 Language Tags + + A language tag identifies a natural language spoken, written, or + otherwise conveyed by human beings for communication of information + to other human beings. Computer languages are explicitly excluded. + HTTP uses language tags within the Accept-Language and Content- + Language fields. + + The syntax and registry of HTTP language tags is the same as that + defined by RFC 1766 [1]. In summary, a language tag is composed of 1 + or more parts: A primary language tag and a possibly empty series of + subtags: + + language-tag = primary-tag *( "-" subtag ) + primary-tag = 1*8ALPHA + subtag = 1*8ALPHA + + White space is not allowed within the tag and all tags are case- + insensitive. The name space of language tags is administered by the + IANA. Example tags include: + + en, en-US, en-cockney, i-cherokee, x-pig-latin + + + + +Fielding, et al. Standards Track [Page 29] + +RFC 2616 HTTP/1.1 June 1999 + + + where any two-letter primary-tag is an ISO-639 language abbreviation + and any two-letter initial subtag is an ISO-3166 country code. (The + last three tags above are not registered tags; all but the last are + examples of tags which could be registered in future.) + +3.11 Entity Tags + + Entity tags are used for comparing two or more entities from the same + requested resource. HTTP/1.1 uses entity tags in the ETag (section + 14.19), If-Match (section 14.24), If-None-Match (section 14.26), and + If-Range (section 14.27) header fields. The definition of how they + are used and compared as cache validators is in section 13.3.3. An + entity tag consists of an opaque quoted string, possibly prefixed by + a weakness indicator. + + entity-tag = [ weak ] opaque-tag + weak = "W/" + opaque-tag = quoted-string + + A "strong entity tag" MAY be shared by two entities of a resource + only if they are equivalent by octet equality. + + A "weak entity tag," indicated by the "W/" prefix, MAY be shared by + two entities of a resource only if the entities are equivalent and + could be substituted for each other with no significant change in + semantics. A weak entity tag can only be used for weak comparison. + + An entity tag MUST be unique across all versions of all entities + associated with a particular resource. A given entity tag value MAY + be used for entities obtained by requests on different URIs. The use + of the same entity tag value in conjunction with entities obtained by + requests on different URIs does not imply the equivalence of those + entities. + +3.12 Range Units + + HTTP/1.1 allows a client to request that only part (a range of) the + response entity be included within the response. HTTP/1.1 uses range + units in the Range (section 14.35) and Content-Range (section 14.16) + header fields. An entity can be broken down into subranges according + to various structural units. + + range-unit = bytes-unit | other-range-unit + bytes-unit = "bytes" + other-range-unit = token + + The only range unit defined by HTTP/1.1 is "bytes". HTTP/1.1 + implementations MAY ignore ranges specified using other units. + + + +Fielding, et al. Standards Track [Page 30] + +RFC 2616 HTTP/1.1 June 1999 + + + HTTP/1.1 has been designed to allow implementations of applications + that do not depend on knowledge of ranges. + +4 HTTP Message + +4.1 Message Types + + HTTP messages consist of requests from client to server and responses + from server to client. + + HTTP-message = Request | Response ; HTTP/1.1 messages + + Request (section 5) and Response (section 6) messages use the generic + message format of RFC 822 [9] for transferring entities (the payload + of the message). Both types of message consist of a start-line, zero + or more header fields (also known as "headers"), an empty line (i.e., + a line with nothing preceding the CRLF) indicating the end of the + header fields, and possibly a message-body. + + generic-message = start-line + *(message-header CRLF) + CRLF + [ message-body ] + start-line = Request-Line | Status-Line + + In the interest of robustness, servers SHOULD ignore any empty + line(s) received where a Request-Line is expected. In other words, if + the server is reading the protocol stream at the beginning of a + message and receives a CRLF first, it should ignore the CRLF. + + Certain buggy HTTP/1.0 client implementations generate extra CRLF's + after a POST request. To restate what is explicitly forbidden by the + BNF, an HTTP/1.1 client MUST NOT preface or follow a request with an + extra CRLF. + +4.2 Message Headers + + HTTP header fields, which include general-header (section 4.5), + request-header (section 5.3), response-header (section 6.2), and + entity-header (section 7.1) fields, follow the same generic format as + that given in Section 3.1 of RFC 822 [9]. Each header field consists + of a name followed by a colon (":") and the field value. Field names + are case-insensitive. The field value MAY be preceded by any amount + of LWS, though a single SP is preferred. Header fields can be + extended over multiple lines by preceding each extra line with at + least one SP or HT. Applications ought to follow "common form", where + one is known or indicated, when generating HTTP constructs, since + there might exist some implementations that fail to accept anything + + + +Fielding, et al. Standards Track [Page 31] + +RFC 2616 HTTP/1.1 June 1999 + + + beyond the common forms. + + message-header = field-name ":" [ field-value ] + field-name = token + field-value = *( field-content | LWS ) + field-content = <the OCTETs making up the field-value + and consisting of either *TEXT or combinations + of token, separators, and quoted-string> + + The field-content does not include any leading or trailing LWS: + linear white space occurring before the first non-whitespace + character of the field-value or after the last non-whitespace + character of the field-value. Such leading or trailing LWS MAY be + removed without changing the semantics of the field value. Any LWS + that occurs between field-content MAY be replaced with a single SP + before interpreting the field value or forwarding the message + downstream. + + The order in which header fields with differing field names are + received is not significant. However, it is "good practice" to send + general-header fields first, followed by request-header or response- + header fields, and ending with the entity-header fields. + + Multiple message-header fields with the same field-name MAY be + present in a message if and only if the entire field-value for that + header field is defined as a comma-separated list [i.e., #(values)]. + It MUST be possible to combine the multiple header fields into one + "field-name: field-value" pair, without changing the semantics of the + message, by appending each subsequent field-value to the first, each + separated by a comma. The order in which header fields with the same + field-name are received is therefore significant to the + interpretation of the combined field value, and thus a proxy MUST NOT + change the order of these field values when a message is forwarded. + +4.3 Message Body + + The message-body (if any) of an HTTP message is used to carry the + entity-body associated with the request or response. The message-body + differs from the entity-body only when a transfer-coding has been + applied, as indicated by the Transfer-Encoding header field (section + 14.41). + + message-body = entity-body + | <entity-body encoded as per Transfer-Encoding> + + Transfer-Encoding MUST be used to indicate any transfer-codings + applied by an application to ensure safe and proper transfer of the + message. Transfer-Encoding is a property of the message, not of the + + + +Fielding, et al. Standards Track [Page 32] + +RFC 2616 HTTP/1.1 June 1999 + + + entity, and thus MAY be added or removed by any application along the + request/response chain. (However, section 3.6 places restrictions on + when certain transfer-codings may be used.) + + The rules for when a message-body is allowed in a message differ for + requests and responses. + + The presence of a message-body in a request is signaled by the + inclusion of a Content-Length or Transfer-Encoding header field in + the request's message-headers. A message-body MUST NOT be included in + a request if the specification of the request method (section 5.1.1) + does not allow sending an entity-body in requests. A server SHOULD + read and forward a message-body on any request; if the request method + does not include defined semantics for an entity-body, then the + message-body SHOULD be ignored when handling the request. + + For response messages, whether or not a message-body is included with + a message is dependent on both the request method and the response + status code (section 6.1.1). All responses to the HEAD request method + MUST NOT include a message-body, even though the presence of entity- + header fields might lead one to believe they do. All 1xx + (informational), 204 (no content), and 304 (not modified) responses + MUST NOT include a message-body. All other responses do include a + message-body, although it MAY be of zero length. + +4.4 Message Length + + The transfer-length of a message is the length of the message-body as + it appears in the message; that is, after any transfer-codings have + been applied. When a message-body is included with a message, the + transfer-length of that body is determined by one of the following + (in order of precedence): + + 1.Any response message which "MUST NOT" include a message-body (such + as the 1xx, 204, and 304 responses and any response to a HEAD + request) is always terminated by the first empty line after the + header fields, regardless of the entity-header fields present in + the message. + + 2.If a Transfer-Encoding header field (section 14.41) is present and + has any value other than "identity", then the transfer-length is + defined by use of the "chunked" transfer-coding (section 3.6), + unless the message is terminated by closing the connection. + + 3.If a Content-Length header field (section 14.13) is present, its + decimal value in OCTETs represents both the entity-length and the + transfer-length. The Content-Length header field MUST NOT be sent + if these two lengths are different (i.e., if a Transfer-Encoding + + + +Fielding, et al. Standards Track [Page 33] + +RFC 2616 HTTP/1.1 June 1999 + + + header field is present). If a message is received with both a + Transfer-Encoding header field and a Content-Length header field, + the latter MUST be ignored. + + 4.If the message uses the media type "multipart/byteranges", and the + ransfer-length is not otherwise specified, then this self- + elimiting media type defines the transfer-length. This media type + UST NOT be used unless the sender knows that the recipient can arse + it; the presence in a request of a Range header with ultiple byte- + range specifiers from a 1.1 client implies that the lient can parse + multipart/byteranges responses. + + A range header might be forwarded by a 1.0 proxy that does not + understand multipart/byteranges; in this case the server MUST + delimit the message using methods defined in items 1,3 or 5 of + this section. + + 5.By the server closing the connection. (Closing the connection + cannot be used to indicate the end of a request body, since that + would leave no possibility for the server to send back a response.) + + For compatibility with HTTP/1.0 applications, HTTP/1.1 requests + containing a message-body MUST include a valid Content-Length header + field unless the server is known to be HTTP/1.1 compliant. If a + request contains a message-body and a Content-Length is not given, + the server SHOULD respond with 400 (bad request) if it cannot + determine the length of the message, or with 411 (length required) if + it wishes to insist on receiving a valid Content-Length. + + All HTTP/1.1 applications that receive entities MUST accept the + "chunked" transfer-coding (section 3.6), thus allowing this mechanism + to be used for messages when the message length cannot be determined + in advance. + + Messages MUST NOT include both a Content-Length header field and a + non-identity transfer-coding. If the message does include a non- + identity transfer-coding, the Content-Length MUST be ignored. + + When a Content-Length is given in a message where a message-body is + allowed, its field value MUST exactly match the number of OCTETs in + the message-body. HTTP/1.1 user agents MUST notify the user when an + invalid length is received and detected. + +4.5 General Header Fields + + There are a few header fields which have general applicability for + both request and response messages, but which do not apply to the + entity being transferred. These header fields apply only to the + + + +Fielding, et al. Standards Track [Page 34] + +RFC 2616 HTTP/1.1 June 1999 + + + message being transmitted. + + general-header = Cache-Control ; Section 14.9 + | Connection ; Section 14.10 + | Date ; Section 14.18 + | Pragma ; Section 14.32 + | Trailer ; Section 14.40 + | Transfer-Encoding ; Section 14.41 + | Upgrade ; Section 14.42 + | Via ; Section 14.45 + | Warning ; Section 14.46 + + General-header field names can be extended reliably only in + combination with a change in the protocol version. However, new or + experimental header fields may be given the semantics of general + header fields if all parties in the communication recognize them to + be general-header fields. Unrecognized header fields are treated as + entity-header fields. + +5 Request + + A request message from a client to a server includes, within the + first line of that message, the method to be applied to the resource, + the identifier of the resource, and the protocol version in use. + + Request = Request-Line ; Section 5.1 + *(( general-header ; Section 4.5 + | request-header ; Section 5.3 + | entity-header ) CRLF) ; Section 7.1 + CRLF + [ message-body ] ; Section 4.3 + +5.1 Request-Line + + The Request-Line begins with a method token, followed by the + Request-URI and the protocol version, and ending with CRLF. The + elements are separated by SP characters. No CR or LF is allowed + except in the final CRLF sequence. + + Request-Line = Method SP Request-URI SP HTTP-Version CRLF + + + + + + + + + + + +Fielding, et al. Standards Track [Page 35] + +RFC 2616 HTTP/1.1 June 1999 + + +5.1.1 Method + + The Method token indicates the method to be performed on the + resource identified by the Request-URI. The method is case-sensitive. + + Method = "OPTIONS" ; Section 9.2 + | "GET" ; Section 9.3 + | "HEAD" ; Section 9.4 + | "POST" ; Section 9.5 + | "PUT" ; Section 9.6 + | "DELETE" ; Section 9.7 + | "TRACE" ; Section 9.8 + | "CONNECT" ; Section 9.9 + | extension-method + extension-method = token + + The list of methods allowed by a resource can be specified in an + Allow header field (section 14.7). The return code of the response + always notifies the client whether a method is currently allowed on a + resource, since the set of allowed methods can change dynamically. An + origin server SHOULD return the status code 405 (Method Not Allowed) + if the method is known by the origin server but not allowed for the + requested resource, and 501 (Not Implemented) if the method is + unrecognized or not implemented by the origin server. The methods GET + and HEAD MUST be supported by all general-purpose servers. All other + methods are OPTIONAL; however, if the above methods are implemented, + they MUST be implemented with the same semantics as those specified + in section 9. + +5.1.2 Request-URI + + The Request-URI is a Uniform Resource Identifier (section 3.2) and + identifies the resource upon which to apply the request. + + Request-URI = "*" | absoluteURI | abs_path | authority + + The four options for Request-URI are dependent on the nature of the + request. The asterisk "*" means that the request does not apply to a + particular resource, but to the server itself, and is only allowed + when the method used does not necessarily apply to a resource. One + example would be + + OPTIONS * HTTP/1.1 + + The absoluteURI form is REQUIRED when the request is being made to a + proxy. The proxy is requested to forward the request or service it + from a valid cache, and return the response. Note that the proxy MAY + forward the request on to another proxy or directly to the server + + + +Fielding, et al. Standards Track [Page 36] + +RFC 2616 HTTP/1.1 June 1999 + + + specified by the absoluteURI. In order to avoid request loops, a + proxy MUST be able to recognize all of its server names, including + any aliases, local variations, and the numeric IP address. An example + Request-Line would be: + + GET http://www.w3.org/pub/WWW/TheProject.html HTTP/1.1 + + To allow for transition to absoluteURIs in all requests in future + versions of HTTP, all HTTP/1.1 servers MUST accept the absoluteURI + form in requests, even though HTTP/1.1 clients will only generate + them in requests to proxies. + + The authority form is only used by the CONNECT method (section 9.9). + + The most common form of Request-URI is that used to identify a + resource on an origin server or gateway. In this case the absolute + path of the URI MUST be transmitted (see section 3.2.1, abs_path) as + the Request-URI, and the network location of the URI (authority) MUST + be transmitted in a Host header field. For example, a client wishing + to retrieve the resource above directly from the origin server would + create a TCP connection to port 80 of the host "www.w3.org" and send + the lines: + + GET /pub/WWW/TheProject.html HTTP/1.1 + Host: www.w3.org + + followed by the remainder of the Request. Note that the absolute path + cannot be empty; if none is present in the original URI, it MUST be + given as "/" (the server root). + + The Request-URI is transmitted in the format specified in section + 3.2.1. If the Request-URI is encoded using the "% HEX HEX" encoding + [42], the origin server MUST decode the Request-URI in order to + properly interpret the request. Servers SHOULD respond to invalid + Request-URIs with an appropriate status code. + + A transparent proxy MUST NOT rewrite the "abs_path" part of the + received Request-URI when forwarding it to the next inbound server, + except as noted above to replace a null abs_path with "/". + + Note: The "no rewrite" rule prevents the proxy from changing the + meaning of the request when the origin server is improperly using + a non-reserved URI character for a reserved purpose. Implementors + should be aware that some pre-HTTP/1.1 proxies have been known to + rewrite the Request-URI. + + + + + + +Fielding, et al. Standards Track [Page 37] + +RFC 2616 HTTP/1.1 June 1999 + + +5.2 The Resource Identified by a Request + + The exact resource identified by an Internet request is determined by + examining both the Request-URI and the Host header field. + + An origin server that does not allow resources to differ by the + requested host MAY ignore the Host header field value when + determining the resource identified by an HTTP/1.1 request. (But see + section 19.6.1.1 for other requirements on Host support in HTTP/1.1.) + + An origin server that does differentiate resources based on the host + requested (sometimes referred to as virtual hosts or vanity host + names) MUST use the following rules for determining the requested + resource on an HTTP/1.1 request: + + 1. If Request-URI is an absoluteURI, the host is part of the + Request-URI. Any Host header field value in the request MUST be + ignored. + + 2. If the Request-URI is not an absoluteURI, and the request includes + a Host header field, the host is determined by the Host header + field value. + + 3. If the host as determined by rule 1 or 2 is not a valid host on + the server, the response MUST be a 400 (Bad Request) error message. + + Recipients of an HTTP/1.0 request that lacks a Host header field MAY + attempt to use heuristics (e.g., examination of the URI path for + something unique to a particular host) in order to determine what + exact resource is being requested. + +5.3 Request Header Fields + + The request-header fields allow the client to pass additional + information about the request, and about the client itself, to the + server. These fields act as request modifiers, with semantics + equivalent to the parameters on a programming language method + invocation. + + request-header = Accept ; Section 14.1 + | Accept-Charset ; Section 14.2 + | Accept-Encoding ; Section 14.3 + | Accept-Language ; Section 14.4 + | Authorization ; Section 14.8 + | Expect ; Section 14.20 + | From ; Section 14.22 + | Host ; Section 14.23 + | If-Match ; Section 14.24 + + + +Fielding, et al. Standards Track [Page 38] + +RFC 2616 HTTP/1.1 June 1999 + + + | If-Modified-Since ; Section 14.25 + | If-None-Match ; Section 14.26 + | If-Range ; Section 14.27 + | If-Unmodified-Since ; Section 14.28 + | Max-Forwards ; Section 14.31 + | Proxy-Authorization ; Section 14.34 + | Range ; Section 14.35 + | Referer ; Section 14.36 + | TE ; Section 14.39 + | User-Agent ; Section 14.43 + + Request-header field names can be extended reliably only in + combination with a change in the protocol version. However, new or + experimental header fields MAY be given the semantics of request- + header fields if all parties in the communication recognize them to + be request-header fields. Unrecognized header fields are treated as + entity-header fields. + +6 Response + + After receiving and interpreting a request message, a server responds + with an HTTP response message. + + Response = Status-Line ; Section 6.1 + *(( general-header ; Section 4.5 + | response-header ; Section 6.2 + | entity-header ) CRLF) ; Section 7.1 + CRLF + [ message-body ] ; Section 7.2 + +6.1 Status-Line + + The first line of a Response message is the Status-Line, consisting + of the protocol version followed by a numeric status code and its + associated textual phrase, with each element separated by SP + characters. No CR or LF is allowed except in the final CRLF sequence. + + Status-Line = HTTP-Version SP Status-Code SP Reason-Phrase CRLF + +6.1.1 Status Code and Reason Phrase + + The Status-Code element is a 3-digit integer result code of the + attempt to understand and satisfy the request. These codes are fully + defined in section 10. The Reason-Phrase is intended to give a short + textual description of the Status-Code. The Status-Code is intended + for use by automata and the Reason-Phrase is intended for the human + user. The client is not required to examine or display the Reason- + Phrase. + + + +Fielding, et al. Standards Track [Page 39] + +RFC 2616 HTTP/1.1 June 1999 + + + The first digit of the Status-Code defines the class of response. The + last two digits do not have any categorization role. There are 5 + values for the first digit: + + - 1xx: Informational - Request received, continuing process + + - 2xx: Success - The action was successfully received, + understood, and accepted + + - 3xx: Redirection - Further action must be taken in order to + complete the request + + - 4xx: Client Error - The request contains bad syntax or cannot + be fulfilled + + - 5xx: Server Error - The server failed to fulfill an apparently + valid request + + The individual values of the numeric status codes defined for + HTTP/1.1, and an example set of corresponding Reason-Phrase's, are + presented below. The reason phrases listed here are only + recommendations -- they MAY be replaced by local equivalents without + affecting the protocol. + + Status-Code = + "100" ; Section 10.1.1: Continue + | "101" ; Section 10.1.2: Switching Protocols + | "200" ; Section 10.2.1: OK + | "201" ; Section 10.2.2: Created + | "202" ; Section 10.2.3: Accepted + | "203" ; Section 10.2.4: Non-Authoritative Information + | "204" ; Section 10.2.5: No Content + | "205" ; Section 10.2.6: Reset Content + | "206" ; Section 10.2.7: Partial Content + | "300" ; Section 10.3.1: Multiple Choices + | "301" ; Section 10.3.2: Moved Permanently + | "302" ; Section 10.3.3: Found + | "303" ; Section 10.3.4: See Other + | "304" ; Section 10.3.5: Not Modified + | "305" ; Section 10.3.6: Use Proxy + | "307" ; Section 10.3.8: Temporary Redirect + | "400" ; Section 10.4.1: Bad Request + | "401" ; Section 10.4.2: Unauthorized + | "402" ; Section 10.4.3: Payment Required + | "403" ; Section 10.4.4: Forbidden + | "404" ; Section 10.4.5: Not Found + | "405" ; Section 10.4.6: Method Not Allowed + | "406" ; Section 10.4.7: Not Acceptable + + + +Fielding, et al. Standards Track [Page 40] + +RFC 2616 HTTP/1.1 June 1999 + + + | "407" ; Section 10.4.8: Proxy Authentication Required + | "408" ; Section 10.4.9: Request Time-out + | "409" ; Section 10.4.10: Conflict + | "410" ; Section 10.4.11: Gone + | "411" ; Section 10.4.12: Length Required + | "412" ; Section 10.4.13: Precondition Failed + | "413" ; Section 10.4.14: Request Entity Too Large + | "414" ; Section 10.4.15: Request-URI Too Large + | "415" ; Section 10.4.16: Unsupported Media Type + | "416" ; Section 10.4.17: Requested range not satisfiable + | "417" ; Section 10.4.18: Expectation Failed + | "500" ; Section 10.5.1: Internal Server Error + | "501" ; Section 10.5.2: Not Implemented + | "502" ; Section 10.5.3: Bad Gateway + | "503" ; Section 10.5.4: Service Unavailable + | "504" ; Section 10.5.5: Gateway Time-out + | "505" ; Section 10.5.6: HTTP Version not supported + | extension-code + + extension-code = 3DIGIT + Reason-Phrase = *<TEXT, excluding CR, LF> + + HTTP status codes are extensible. HTTP applications are not required + to understand the meaning of all registered status codes, though such + understanding is obviously desirable. However, applications MUST + understand the class of any status code, as indicated by the first + digit, and treat any unrecognized response as being equivalent to the + x00 status code of that class, with the exception that an + unrecognized response MUST NOT be cached. For example, if an + unrecognized status code of 431 is received by the client, it can + safely assume that there was something wrong with its request and + treat the response as if it had received a 400 status code. In such + cases, user agents SHOULD present to the user the entity returned + with the response, since that entity is likely to include human- + readable information which will explain the unusual status. + +6.2 Response Header Fields + + The response-header fields allow the server to pass additional + information about the response which cannot be placed in the Status- + Line. These header fields give information about the server and about + further access to the resource identified by the Request-URI. + + response-header = Accept-Ranges ; Section 14.5 + | Age ; Section 14.6 + | ETag ; Section 14.19 + | Location ; Section 14.30 + | Proxy-Authenticate ; Section 14.33 + + + +Fielding, et al. Standards Track [Page 41] + +RFC 2616 HTTP/1.1 June 1999 + + + | Retry-After ; Section 14.37 + | Server ; Section 14.38 + | Vary ; Section 14.44 + | WWW-Authenticate ; Section 14.47 + + Response-header field names can be extended reliably only in + combination with a change in the protocol version. However, new or + experimental header fields MAY be given the semantics of response- + header fields if all parties in the communication recognize them to + be response-header fields. Unrecognized header fields are treated as + entity-header fields. + +7 Entity + + Request and Response messages MAY transfer an entity if not otherwise + restricted by the request method or response status code. An entity + consists of entity-header fields and an entity-body, although some + responses will only include the entity-headers. + + In this section, both sender and recipient refer to either the client + or the server, depending on who sends and who receives the entity. + +7.1 Entity Header Fields + + Entity-header fields define metainformation about the entity-body or, + if no body is present, about the resource identified by the request. + Some of this metainformation is OPTIONAL; some might be REQUIRED by + portions of this specification. + + entity-header = Allow ; Section 14.7 + | Content-Encoding ; Section 14.11 + | Content-Language ; Section 14.12 + | Content-Length ; Section 14.13 + | Content-Location ; Section 14.14 + | Content-MD5 ; Section 14.15 + | Content-Range ; Section 14.16 + | Content-Type ; Section 14.17 + | Expires ; Section 14.21 + | Last-Modified ; Section 14.29 + | extension-header + + extension-header = message-header + + The extension-header mechanism allows additional entity-header fields + to be defined without changing the protocol, but these fields cannot + be assumed to be recognizable by the recipient. Unrecognized header + fields SHOULD be ignored by the recipient and MUST be forwarded by + transparent proxies. + + + +Fielding, et al. Standards Track [Page 42] + +RFC 2616 HTTP/1.1 June 1999 + + +7.2 Entity Body + + The entity-body (if any) sent with an HTTP request or response is in + a format and encoding defined by the entity-header fields. + + entity-body = *OCTET + + An entity-body is only present in a message when a message-body is + present, as described in section 4.3. The entity-body is obtained + from the message-body by decoding any Transfer-Encoding that might + have been applied to ensure safe and proper transfer of the message. + +7.2.1 Type + + When an entity-body is included with a message, the data type of that + body is determined via the header fields Content-Type and Content- + Encoding. These define a two-layer, ordered encoding model: + + entity-body := Content-Encoding( Content-Type( data ) ) + + Content-Type specifies the media type of the underlying data. + Content-Encoding may be used to indicate any additional content + codings applied to the data, usually for the purpose of data + compression, that are a property of the requested resource. There is + no default encoding. + + Any HTTP/1.1 message containing an entity-body SHOULD include a + Content-Type header field defining the media type of that body. If + and only if the media type is not given by a Content-Type field, the + recipient MAY attempt to guess the media type via inspection of its + content and/or the name extension(s) of the URI used to identify the + resource. If the media type remains unknown, the recipient SHOULD + treat it as type "application/octet-stream". + +7.2.2 Entity Length + + The entity-length of a message is the length of the message-body + before any transfer-codings have been applied. Section 4.4 defines + how the transfer-length of a message-body is determined. + + + + + + + + + + + + +Fielding, et al. Standards Track [Page 43] + +RFC 2616 HTTP/1.1 June 1999 + + +8 Connections + +8.1 Persistent Connections + +8.1.1 Purpose + + Prior to persistent connections, a separate TCP connection was + established to fetch each URL, increasing the load on HTTP servers + and causing congestion on the Internet. The use of inline images and + other associated data often require a client to make multiple + requests of the same server in a short amount of time. Analysis of + these performance problems and results from a prototype + implementation are available [26] [30]. Implementation experience and + measurements of actual HTTP/1.1 (RFC 2068) implementations show good + results [39]. Alternatives have also been explored, for example, + T/TCP [27]. + + Persistent HTTP connections have a number of advantages: + + - By opening and closing fewer TCP connections, CPU time is saved + in routers and hosts (clients, servers, proxies, gateways, + tunnels, or caches), and memory used for TCP protocol control + blocks can be saved in hosts. + + - HTTP requests and responses can be pipelined on a connection. + Pipelining allows a client to make multiple requests without + waiting for each response, allowing a single TCP connection to + be used much more efficiently, with much lower elapsed time. + + - Network congestion is reduced by reducing the number of packets + caused by TCP opens, and by allowing TCP sufficient time to + determine the congestion state of the network. + + - Latency on subsequent requests is reduced since there is no time + spent in TCP's connection opening handshake. + + - HTTP can evolve more gracefully, since errors can be reported + without the penalty of closing the TCP connection. Clients using + future versions of HTTP might optimistically try a new feature, + but if communicating with an older server, retry with old + semantics after an error is reported. + + HTTP implementations SHOULD implement persistent connections. + + + + + + + + +Fielding, et al. Standards Track [Page 44] + +RFC 2616 HTTP/1.1 June 1999 + + +8.1.2 Overall Operation + + A significant difference between HTTP/1.1 and earlier versions of + HTTP is that persistent connections are the default behavior of any + HTTP connection. That is, unless otherwise indicated, the client + SHOULD assume that the server will maintain a persistent connection, + even after error responses from the server. + + Persistent connections provide a mechanism by which a client and a + server can signal the close of a TCP connection. This signaling takes + place using the Connection header field (section 14.10). Once a close + has been signaled, the client MUST NOT send any more requests on that + connection. + +8.1.2.1 Negotiation + + An HTTP/1.1 server MAY assume that a HTTP/1.1 client intends to + maintain a persistent connection unless a Connection header including + the connection-token "close" was sent in the request. If the server + chooses to close the connection immediately after sending the + response, it SHOULD send a Connection header including the + connection-token close. + + An HTTP/1.1 client MAY expect a connection to remain open, but would + decide to keep it open based on whether the response from a server + contains a Connection header with the connection-token close. In case + the client does not want to maintain a connection for more than that + request, it SHOULD send a Connection header including the + connection-token close. + + If either the client or the server sends the close token in the + Connection header, that request becomes the last one for the + connection. + + Clients and servers SHOULD NOT assume that a persistent connection is + maintained for HTTP versions less than 1.1 unless it is explicitly + signaled. See section 19.6.2 for more information on backward + compatibility with HTTP/1.0 clients. + + In order to remain persistent, all messages on the connection MUST + have a self-defined message length (i.e., one not defined by closure + of the connection), as described in section 4.4. + + + + + + + + + +Fielding, et al. Standards Track [Page 45] + +RFC 2616 HTTP/1.1 June 1999 + + +8.1.2.2 Pipelining + + A client that supports persistent connections MAY "pipeline" its + requests (i.e., send multiple requests without waiting for each + response). A server MUST send its responses to those requests in the + same order that the requests were received. + + Clients which assume persistent connections and pipeline immediately + after connection establishment SHOULD be prepared to retry their + connection if the first pipelined attempt fails. If a client does + such a retry, it MUST NOT pipeline before it knows the connection is + persistent. Clients MUST also be prepared to resend their requests if + the server closes the connection before sending all of the + corresponding responses. + + Clients SHOULD NOT pipeline requests using non-idempotent methods or + non-idempotent sequences of methods (see section 9.1.2). Otherwise, a + premature termination of the transport connection could lead to + indeterminate results. A client wishing to send a non-idempotent + request SHOULD wait to send that request until it has received the + response status for the previous request. + +8.1.3 Proxy Servers + + It is especially important that proxies correctly implement the + properties of the Connection header field as specified in section + 14.10. + + The proxy server MUST signal persistent connections separately with + its clients and the origin servers (or other proxy servers) that it + connects to. Each persistent connection applies to only one transport + link. + + A proxy server MUST NOT establish a HTTP/1.1 persistent connection + with an HTTP/1.0 client (but see RFC 2068 [33] for information and + discussion of the problems with the Keep-Alive header implemented by + many HTTP/1.0 clients). + +8.1.4 Practical Considerations + + Servers will usually have some time-out value beyond which they will + no longer maintain an inactive connection. Proxy servers might make + this a higher value since it is likely that the client will be making + more connections through the same server. The use of persistent + connections places no requirements on the length (or existence) of + this time-out for either the client or the server. + + + + + +Fielding, et al. Standards Track [Page 46] + +RFC 2616 HTTP/1.1 June 1999 + + + When a client or server wishes to time-out it SHOULD issue a graceful + close on the transport connection. Clients and servers SHOULD both + constantly watch for the other side of the transport close, and + respond to it as appropriate. If a client or server does not detect + the other side's close promptly it could cause unnecessary resource + drain on the network. + + A client, server, or proxy MAY close the transport connection at any + time. For example, a client might have started to send a new request + at the same time that the server has decided to close the "idle" + connection. From the server's point of view, the connection is being + closed while it was idle, but from the client's point of view, a + request is in progress. + + This means that clients, servers, and proxies MUST be able to recover + from asynchronous close events. Client software SHOULD reopen the + transport connection and retransmit the aborted sequence of requests + without user interaction so long as the request sequence is + idempotent (see section 9.1.2). Non-idempotent methods or sequences + MUST NOT be automatically retried, although user agents MAY offer a + human operator the choice of retrying the request(s). Confirmation by + user-agent software with semantic understanding of the application + MAY substitute for user confirmation. The automatic retry SHOULD NOT + be repeated if the second sequence of requests fails. + + Servers SHOULD always respond to at least one request per connection, + if at all possible. Servers SHOULD NOT close a connection in the + middle of transmitting a response, unless a network or client failure + is suspected. + + Clients that use persistent connections SHOULD limit the number of + simultaneous connections that they maintain to a given server. A + single-user client SHOULD NOT maintain more than 2 connections with + any server or proxy. A proxy SHOULD use up to 2*N connections to + another server or proxy, where N is the number of simultaneously + active users. These guidelines are intended to improve HTTP response + times and avoid congestion. + +8.2 Message Transmission Requirements + +8.2.1 Persistent Connections and Flow Control + + HTTP/1.1 servers SHOULD maintain persistent connections and use TCP's + flow control mechanisms to resolve temporary overloads, rather than + terminating connections with the expectation that clients will retry. + The latter technique can exacerbate network congestion. + + + + + +Fielding, et al. Standards Track [Page 47] + +RFC 2616 HTTP/1.1 June 1999 + + +8.2.2 Monitoring Connections for Error Status Messages + + An HTTP/1.1 (or later) client sending a message-body SHOULD monitor + the network connection for an error status while it is transmitting + the request. If the client sees an error status, it SHOULD + immediately cease transmitting the body. If the body is being sent + using a "chunked" encoding (section 3.6), a zero length chunk and + empty trailer MAY be used to prematurely mark the end of the message. + If the body was preceded by a Content-Length header, the client MUST + close the connection. + +8.2.3 Use of the 100 (Continue) Status + + The purpose of the 100 (Continue) status (see section 10.1.1) is to + allow a client that is sending a request message with a request body + to determine if the origin server is willing to accept the request + (based on the request headers) before the client sends the request + body. In some cases, it might either be inappropriate or highly + inefficient for the client to send the body if the server will reject + the message without looking at the body. + + Requirements for HTTP/1.1 clients: + + - If a client will wait for a 100 (Continue) response before + sending the request body, it MUST send an Expect request-header + field (section 14.20) with the "100-continue" expectation. + + - A client MUST NOT send an Expect request-header field (section + 14.20) with the "100-continue" expectation if it does not intend + to send a request body. + + Because of the presence of older implementations, the protocol allows + ambiguous situations in which a client may send "Expect: 100- + continue" without receiving either a 417 (Expectation Failed) status + or a 100 (Continue) status. Therefore, when a client sends this + header field to an origin server (possibly via a proxy) from which it + has never seen a 100 (Continue) status, the client SHOULD NOT wait + for an indefinite period before sending the request body. + + Requirements for HTTP/1.1 origin servers: + + - Upon receiving a request which includes an Expect request-header + field with the "100-continue" expectation, an origin server MUST + either respond with 100 (Continue) status and continue to read + from the input stream, or respond with a final status code. The + origin server MUST NOT wait for the request body before sending + the 100 (Continue) response. If it responds with a final status + code, it MAY close the transport connection or it MAY continue + + + +Fielding, et al. Standards Track [Page 48] + +RFC 2616 HTTP/1.1 June 1999 + + + to read and discard the rest of the request. It MUST NOT + perform the requested method if it returns a final status code. + + - An origin server SHOULD NOT send a 100 (Continue) response if + the request message does not include an Expect request-header + field with the "100-continue" expectation, and MUST NOT send a + 100 (Continue) response if such a request comes from an HTTP/1.0 + (or earlier) client. There is an exception to this rule: for + compatibility with RFC 2068, a server MAY send a 100 (Continue) + status in response to an HTTP/1.1 PUT or POST request that does + not include an Expect request-header field with the "100- + continue" expectation. This exception, the purpose of which is + to minimize any client processing delays associated with an + undeclared wait for 100 (Continue) status, applies only to + HTTP/1.1 requests, and not to requests with any other HTTP- + version value. + + - An origin server MAY omit a 100 (Continue) response if it has + already received some or all of the request body for the + corresponding request. + + - An origin server that sends a 100 (Continue) response MUST + ultimately send a final status code, once the request body is + received and processed, unless it terminates the transport + connection prematurely. + + - If an origin server receives a request that does not include an + Expect request-header field with the "100-continue" expectation, + the request includes a request body, and the server responds + with a final status code before reading the entire request body + from the transport connection, then the server SHOULD NOT close + the transport connection until it has read the entire request, + or until the client closes the connection. Otherwise, the client + might not reliably receive the response message. However, this + requirement is not be construed as preventing a server from + defending itself against denial-of-service attacks, or from + badly broken client implementations. + + Requirements for HTTP/1.1 proxies: + + - If a proxy receives a request that includes an Expect request- + header field with the "100-continue" expectation, and the proxy + either knows that the next-hop server complies with HTTP/1.1 or + higher, or does not know the HTTP version of the next-hop + server, it MUST forward the request, including the Expect header + field. + + + + + +Fielding, et al. Standards Track [Page 49] + +RFC 2616 HTTP/1.1 June 1999 + + + - If the proxy knows that the version of the next-hop server is + HTTP/1.0 or lower, it MUST NOT forward the request, and it MUST + respond with a 417 (Expectation Failed) status. + + - Proxies SHOULD maintain a cache recording the HTTP version + numbers received from recently-referenced next-hop servers. + + - A proxy MUST NOT forward a 100 (Continue) response if the + request message was received from an HTTP/1.0 (or earlier) + client and did not include an Expect request-header field with + the "100-continue" expectation. This requirement overrides the + general rule for forwarding of 1xx responses (see section 10.1). + +8.2.4 Client Behavior if Server Prematurely Closes Connection + + If an HTTP/1.1 client sends a request which includes a request body, + but which does not include an Expect request-header field with the + "100-continue" expectation, and if the client is not directly + connected to an HTTP/1.1 origin server, and if the client sees the + connection close before receiving any status from the server, the + client SHOULD retry the request. If the client does retry this + request, it MAY use the following "binary exponential backoff" + algorithm to be assured of obtaining a reliable response: + + 1. Initiate a new connection to the server + + 2. Transmit the request-headers + + 3. Initialize a variable R to the estimated round-trip time to the + server (e.g., based on the time it took to establish the + connection), or to a constant value of 5 seconds if the round- + trip time is not available. + + 4. Compute T = R * (2**N), where N is the number of previous + retries of this request. + + 5. Wait either for an error response from the server, or for T + seconds (whichever comes first) + + 6. If no error response is received, after T seconds transmit the + body of the request. + + 7. If client sees that the connection is closed prematurely, + repeat from step 1 until the request is accepted, an error + response is received, or the user becomes impatient and + terminates the retry process. + + + + + +Fielding, et al. Standards Track [Page 50] + +RFC 2616 HTTP/1.1 June 1999 + + + If at any point an error status is received, the client + + - SHOULD NOT continue and + + - SHOULD close the connection if it has not completed sending the + request message. + +9 Method Definitions + + The set of common methods for HTTP/1.1 is defined below. Although + this set can be expanded, additional methods cannot be assumed to + share the same semantics for separately extended clients and servers. + + The Host request-header field (section 14.23) MUST accompany all + HTTP/1.1 requests. + +9.1 Safe and Idempotent Methods + +9.1.1 Safe Methods + + Implementors should be aware that the software represents the user in + their interactions over the Internet, and should be careful to allow + the user to be aware of any actions they might take which may have an + unexpected significance to themselves or others. + + In particular, the convention has been established that the GET and + HEAD methods SHOULD NOT have the significance of taking an action + other than retrieval. These methods ought to be considered "safe". + This allows user agents to represent other methods, such as POST, PUT + and DELETE, in a special way, so that the user is made aware of the + fact that a possibly unsafe action is being requested. + + Naturally, it is not possible to ensure that the server does not + generate side-effects as a result of performing a GET request; in + fact, some dynamic resources consider that a feature. The important + distinction here is that the user did not request the side-effects, + so therefore cannot be held accountable for them. + +9.1.2 Idempotent Methods + + Methods can also have the property of "idempotence" in that (aside + from error or expiration issues) the side-effects of N > 0 identical + requests is the same as for a single request. The methods GET, HEAD, + PUT and DELETE share this property. Also, the methods OPTIONS and + TRACE SHOULD NOT have side effects, and so are inherently idempotent. + + + + + + +Fielding, et al. Standards Track [Page 51] + +RFC 2616 HTTP/1.1 June 1999 + + + However, it is possible that a sequence of several requests is non- + idempotent, even if all of the methods executed in that sequence are + idempotent. (A sequence is idempotent if a single execution of the + entire sequence always yields a result that is not changed by a + reexecution of all, or part, of that sequence.) For example, a + sequence is non-idempotent if its result depends on a value that is + later modified in the same sequence. + + A sequence that never has side effects is idempotent, by definition + (provided that no concurrent operations are being executed on the + same set of resources). + +9.2 OPTIONS + + The OPTIONS method represents a request for information about the + communication options available on the request/response chain + identified by the Request-URI. This method allows the client to + determine the options and/or requirements associated with a resource, + or the capabilities of a server, without implying a resource action + or initiating a resource retrieval. + + Responses to this method are not cacheable. + + If the OPTIONS request includes an entity-body (as indicated by the + presence of Content-Length or Transfer-Encoding), then the media type + MUST be indicated by a Content-Type field. Although this + specification does not define any use for such a body, future + extensions to HTTP might use the OPTIONS body to make more detailed + queries on the server. A server that does not support such an + extension MAY discard the request body. + + If the Request-URI is an asterisk ("*"), the OPTIONS request is + intended to apply to the server in general rather than to a specific + resource. Since a server's communication options typically depend on + the resource, the "*" request is only useful as a "ping" or "no-op" + type of method; it does nothing beyond allowing the client to test + the capabilities of the server. For example, this can be used to test + a proxy for HTTP/1.1 compliance (or lack thereof). + + If the Request-URI is not an asterisk, the OPTIONS request applies + only to the options that are available when communicating with that + resource. + + A 200 response SHOULD include any header fields that indicate + optional features implemented by the server and applicable to that + resource (e.g., Allow), possibly including extensions not defined by + this specification. The response body, if any, SHOULD also include + information about the communication options. The format for such a + + + +Fielding, et al. Standards Track [Page 52] + +RFC 2616 HTTP/1.1 June 1999 + + + body is not defined by this specification, but might be defined by + future extensions to HTTP. Content negotiation MAY be used to select + the appropriate response format. If no response body is included, the + response MUST include a Content-Length field with a field-value of + "0". + + The Max-Forwards request-header field MAY be used to target a + specific proxy in the request chain. When a proxy receives an OPTIONS + request on an absoluteURI for which request forwarding is permitted, + the proxy MUST check for a Max-Forwards field. If the Max-Forwards + field-value is zero ("0"), the proxy MUST NOT forward the message; + instead, the proxy SHOULD respond with its own communication options. + If the Max-Forwards field-value is an integer greater than zero, the + proxy MUST decrement the field-value when it forwards the request. If + no Max-Forwards field is present in the request, then the forwarded + request MUST NOT include a Max-Forwards field. + +9.3 GET + + The GET method means retrieve whatever information (in the form of an + entity) is identified by the Request-URI. If the Request-URI refers + to a data-producing process, it is the produced data which shall be + returned as the entity in the response and not the source text of the + process, unless that text happens to be the output of the process. + + The semantics of the GET method change to a "conditional GET" if the + request message includes an If-Modified-Since, If-Unmodified-Since, + If-Match, If-None-Match, or If-Range header field. A conditional GET + method requests that the entity be transferred only under the + circumstances described by the conditional header field(s). The + conditional GET method is intended to reduce unnecessary network + usage by allowing cached entities to be refreshed without requiring + multiple requests or transferring data already held by the client. + + The semantics of the GET method change to a "partial GET" if the + request message includes a Range header field. A partial GET requests + that only part of the entity be transferred, as described in section + 14.35. The partial GET method is intended to reduce unnecessary + network usage by allowing partially-retrieved entities to be + completed without transferring data already held by the client. + + The response to a GET request is cacheable if and only if it meets + the requirements for HTTP caching described in section 13. + + See section 15.1.3 for security considerations when used for forms. + + + + + + +Fielding, et al. Standards Track [Page 53] + +RFC 2616 HTTP/1.1 June 1999 + + +9.4 HEAD + + The HEAD method is identical to GET except that the server MUST NOT + return a message-body in the response. The metainformation contained + in the HTTP headers in response to a HEAD request SHOULD be identical + to the information sent in response to a GET request. This method can + be used for obtaining metainformation about the entity implied by the + request without transferring the entity-body itself. This method is + often used for testing hypertext links for validity, accessibility, + and recent modification. + + The response to a HEAD request MAY be cacheable in the sense that the + information contained in the response MAY be used to update a + previously cached entity from that resource. If the new field values + indicate that the cached entity differs from the current entity (as + would be indicated by a change in Content-Length, Content-MD5, ETag + or Last-Modified), then the cache MUST treat the cache entry as + stale. + +9.5 POST + + The POST method is used to request that the origin server accept the + entity enclosed in the request as a new subordinate of the resource + identified by the Request-URI in the Request-Line. POST is designed + to allow a uniform method to cover the following functions: + + - Annotation of existing resources; + + - Posting a message to a bulletin board, newsgroup, mailing list, + or similar group of articles; + + - Providing a block of data, such as the result of submitting a + form, to a data-handling process; + + - Extending a database through an append operation. + + The actual function performed by the POST method is determined by the + server and is usually dependent on the Request-URI. The posted entity + is subordinate to that URI in the same way that a file is subordinate + to a directory containing it, a news article is subordinate to a + newsgroup to which it is posted, or a record is subordinate to a + database. + + The action performed by the POST method might not result in a + resource that can be identified by a URI. In this case, either 200 + (OK) or 204 (No Content) is the appropriate response status, + depending on whether or not the response includes an entity that + describes the result. + + + +Fielding, et al. Standards Track [Page 54] + +RFC 2616 HTTP/1.1 June 1999 + + + If a resource has been created on the origin server, the response + SHOULD be 201 (Created) and contain an entity which describes the + status of the request and refers to the new resource, and a Location + header (see section 14.30). + + Responses to this method are not cacheable, unless the response + includes appropriate Cache-Control or Expires header fields. However, + the 303 (See Other) response can be used to direct the user agent to + retrieve a cacheable resource. + + POST requests MUST obey the message transmission requirements set out + in section 8.2. + + See section 15.1.3 for security considerations. + +9.6 PUT + + The PUT method requests that the enclosed entity be stored under the + supplied Request-URI. If the Request-URI refers to an already + existing resource, the enclosed entity SHOULD be considered as a + modified version of the one residing on the origin server. If the + Request-URI does not point to an existing resource, and that URI is + capable of being defined as a new resource by the requesting user + agent, the origin server can create the resource with that URI. If a + new resource is created, the origin server MUST inform the user agent + via the 201 (Created) response. If an existing resource is modified, + either the 200 (OK) or 204 (No Content) response codes SHOULD be sent + to indicate successful completion of the request. If the resource + could not be created or modified with the Request-URI, an appropriate + error response SHOULD be given that reflects the nature of the + problem. The recipient of the entity MUST NOT ignore any Content-* + (e.g. Content-Range) headers that it does not understand or implement + and MUST return a 501 (Not Implemented) response in such cases. + + If the request passes through a cache and the Request-URI identifies + one or more currently cached entities, those entries SHOULD be + treated as stale. Responses to this method are not cacheable. + + The fundamental difference between the POST and PUT requests is + reflected in the different meaning of the Request-URI. The URI in a + POST request identifies the resource that will handle the enclosed + entity. That resource might be a data-accepting process, a gateway to + some other protocol, or a separate entity that accepts annotations. + In contrast, the URI in a PUT request identifies the entity enclosed + with the request -- the user agent knows what URI is intended and the + server MUST NOT attempt to apply the request to some other resource. + If the server desires that the request be applied to a different URI, + + + + +Fielding, et al. Standards Track [Page 55] + +RFC 2616 HTTP/1.1 June 1999 + + + it MUST send a 301 (Moved Permanently) response; the user agent MAY + then make its own decision regarding whether or not to redirect the + request. + + A single resource MAY be identified by many different URIs. For + example, an article might have a URI for identifying "the current + version" which is separate from the URI identifying each particular + version. In this case, a PUT request on a general URI might result in + several other URIs being defined by the origin server. + + HTTP/1.1 does not define how a PUT method affects the state of an + origin server. + + PUT requests MUST obey the message transmission requirements set out + in section 8.2. + + Unless otherwise specified for a particular entity-header, the + entity-headers in the PUT request SHOULD be applied to the resource + created or modified by the PUT. + +9.7 DELETE + + The DELETE method requests that the origin server delete the resource + identified by the Request-URI. This method MAY be overridden by human + intervention (or other means) on the origin server. The client cannot + be guaranteed that the operation has been carried out, even if the + status code returned from the origin server indicates that the action + has been completed successfully. However, the server SHOULD NOT + indicate success unless, at the time the response is given, it + intends to delete the resource or move it to an inaccessible + location. + + A successful response SHOULD be 200 (OK) if the response includes an + entity describing the status, 202 (Accepted) if the action has not + yet been enacted, or 204 (No Content) if the action has been enacted + but the response does not include an entity. + + If the request passes through a cache and the Request-URI identifies + one or more currently cached entities, those entries SHOULD be + treated as stale. Responses to this method are not cacheable. + +9.8 TRACE + + The TRACE method is used to invoke a remote, application-layer loop- + back of the request message. The final recipient of the request + SHOULD reflect the message received back to the client as the + entity-body of a 200 (OK) response. The final recipient is either the + + + + +Fielding, et al. Standards Track [Page 56] + +RFC 2616 HTTP/1.1 June 1999 + + + origin server or the first proxy or gateway to receive a Max-Forwards + value of zero (0) in the request (see section 14.31). A TRACE request + MUST NOT include an entity. + + TRACE allows the client to see what is being received at the other + end of the request chain and use that data for testing or diagnostic + information. The value of the Via header field (section 14.45) is of + particular interest, since it acts as a trace of the request chain. + Use of the Max-Forwards header field allows the client to limit the + length of the request chain, which is useful for testing a chain of + proxies forwarding messages in an infinite loop. + + If the request is valid, the response SHOULD contain the entire + request message in the entity-body, with a Content-Type of + "message/http". Responses to this method MUST NOT be cached. + +9.9 CONNECT + + This specification reserves the method name CONNECT for use with a + proxy that can dynamically switch to being a tunnel (e.g. SSL + tunneling [44]). + +10 Status Code Definitions + + Each Status-Code is described below, including a description of which + method(s) it can follow and any metainformation required in the + response. + +10.1 Informational 1xx + + This class of status code indicates a provisional response, + consisting only of the Status-Line and optional headers, and is + terminated by an empty line. There are no required headers for this + class of status code. Since HTTP/1.0 did not define any 1xx status + codes, servers MUST NOT send a 1xx response to an HTTP/1.0 client + except under experimental conditions. + + A client MUST be prepared to accept one or more 1xx status responses + prior to a regular response, even if the client does not expect a 100 + (Continue) status message. Unexpected 1xx status responses MAY be + ignored by a user agent. + + Proxies MUST forward 1xx responses, unless the connection between the + proxy and its client has been closed, or unless the proxy itself + requested the generation of the 1xx response. (For example, if a + + + + + + +Fielding, et al. Standards Track [Page 57] + +RFC 2616 HTTP/1.1 June 1999 + + + proxy adds a "Expect: 100-continue" field when it forwards a request, + then it need not forward the corresponding 100 (Continue) + response(s).) + +10.1.1 100 Continue + + The client SHOULD continue with its request. This interim response is + used to inform the client that the initial part of the request has + been received and has not yet been rejected by the server. The client + SHOULD continue by sending the remainder of the request or, if the + request has already been completed, ignore this response. The server + MUST send a final response after the request has been completed. See + section 8.2.3 for detailed discussion of the use and handling of this + status code. + +10.1.2 101 Switching Protocols + + The server understands and is willing to comply with the client's + request, via the Upgrade message header field (section 14.42), for a + change in the application protocol being used on this connection. The + server will switch protocols to those defined by the response's + Upgrade header field immediately after the empty line which + terminates the 101 response. + + The protocol SHOULD be switched only when it is advantageous to do + so. For example, switching to a newer version of HTTP is advantageous + over older versions, and switching to a real-time, synchronous + protocol might be advantageous when delivering resources that use + such features. + +10.2 Successful 2xx + + This class of status code indicates that the client's request was + successfully received, understood, and accepted. + +10.2.1 200 OK + + The request has succeeded. The information returned with the response + is dependent on the method used in the request, for example: + + GET an entity corresponding to the requested resource is sent in + the response; + + HEAD the entity-header fields corresponding to the requested + resource are sent in the response without any message-body; + + POST an entity describing or containing the result of the action; + + + + +Fielding, et al. Standards Track [Page 58] + +RFC 2616 HTTP/1.1 June 1999 + + + TRACE an entity containing the request message as received by the + end server. + +10.2.2 201 Created + + The request has been fulfilled and resulted in a new resource being + created. The newly created resource can be referenced by the URI(s) + returned in the entity of the response, with the most specific URI + for the resource given by a Location header field. The response + SHOULD include an entity containing a list of resource + characteristics and location(s) from which the user or user agent can + choose the one most appropriate. The entity format is specified by + the media type given in the Content-Type header field. The origin + server MUST create the resource before returning the 201 status code. + If the action cannot be carried out immediately, the server SHOULD + respond with 202 (Accepted) response instead. + + A 201 response MAY contain an ETag response header field indicating + the current value of the entity tag for the requested variant just + created, see section 14.19. + +10.2.3 202 Accepted + + The request has been accepted for processing, but the processing has + not been completed. The request might or might not eventually be + acted upon, as it might be disallowed when processing actually takes + place. There is no facility for re-sending a status code from an + asynchronous operation such as this. + + The 202 response is intentionally non-committal. Its purpose is to + allow a server to accept a request for some other process (perhaps a + batch-oriented process that is only run once per day) without + requiring that the user agent's connection to the server persist + until the process is completed. The entity returned with this + response SHOULD include an indication of the request's current status + and either a pointer to a status monitor or some estimate of when the + user can expect the request to be fulfilled. + +10.2.4 203 Non-Authoritative Information + + The returned metainformation in the entity-header is not the + definitive set as available from the origin server, but is gathered + from a local or a third-party copy. The set presented MAY be a subset + or superset of the original version. For example, including local + annotation information about the resource might result in a superset + of the metainformation known by the origin server. Use of this + response code is not required and is only appropriate when the + response would otherwise be 200 (OK). + + + +Fielding, et al. Standards Track [Page 59] + +RFC 2616 HTTP/1.1 June 1999 + + +10.2.5 204 No Content + + The server has fulfilled the request but does not need to return an + entity-body, and might want to return updated metainformation. The + response MAY include new or updated metainformation in the form of + entity-headers, which if present SHOULD be associated with the + requested variant. + + If the client is a user agent, it SHOULD NOT change its document view + from that which caused the request to be sent. This response is + primarily intended to allow input for actions to take place without + causing a change to the user agent's active document view, although + any new or updated metainformation SHOULD be applied to the document + currently in the user agent's active view. + + The 204 response MUST NOT include a message-body, and thus is always + terminated by the first empty line after the header fields. + +10.2.6 205 Reset Content + + The server has fulfilled the request and the user agent SHOULD reset + the document view which caused the request to be sent. This response + is primarily intended to allow input for actions to take place via + user input, followed by a clearing of the form in which the input is + given so that the user can easily initiate another input action. The + response MUST NOT include an entity. + +10.2.7 206 Partial Content + + The server has fulfilled the partial GET request for the resource. + The request MUST have included a Range header field (section 14.35) + indicating the desired range, and MAY have included an If-Range + header field (section 14.27) to make the request conditional. + + The response MUST include the following header fields: + + - Either a Content-Range header field (section 14.16) indicating + the range included with this response, or a multipart/byteranges + Content-Type including Content-Range fields for each part. If a + Content-Length header field is present in the response, its + value MUST match the actual number of OCTETs transmitted in the + message-body. + + - Date + + - ETag and/or Content-Location, if the header would have been sent + in a 200 response to the same request + + + + +Fielding, et al. Standards Track [Page 60] + +RFC 2616 HTTP/1.1 June 1999 + + + - Expires, Cache-Control, and/or Vary, if the field-value might + differ from that sent in any previous response for the same + variant + + If the 206 response is the result of an If-Range request that used a + strong cache validator (see section 13.3.3), the response SHOULD NOT + include other entity-headers. If the response is the result of an + If-Range request that used a weak validator, the response MUST NOT + include other entity-headers; this prevents inconsistencies between + cached entity-bodies and updated headers. Otherwise, the response + MUST include all of the entity-headers that would have been returned + with a 200 (OK) response to the same request. + + A cache MUST NOT combine a 206 response with other previously cached + content if the ETag or Last-Modified headers do not match exactly, + see 13.5.4. + + A cache that does not support the Range and Content-Range headers + MUST NOT cache 206 (Partial) responses. + +10.3 Redirection 3xx + + This class of status code indicates that further action needs to be + taken by the user agent in order to fulfill the request. The action + required MAY be carried out by the user agent without interaction + with the user if and only if the method used in the second request is + GET or HEAD. A client SHOULD detect infinite redirection loops, since + such loops generate network traffic for each redirection. + + Note: previous versions of this specification recommended a + maximum of five redirections. Content developers should be aware + that there might be clients that implement such a fixed + limitation. + +10.3.1 300 Multiple Choices + + The requested resource corresponds to any one of a set of + representations, each with its own specific location, and agent- + driven negotiation information (section 12) is being provided so that + the user (or user agent) can select a preferred representation and + redirect its request to that location. + + Unless it was a HEAD request, the response SHOULD include an entity + containing a list of resource characteristics and location(s) from + which the user or user agent can choose the one most appropriate. The + entity format is specified by the media type given in the Content- + Type header field. Depending upon the format and the capabilities of + + + + +Fielding, et al. Standards Track [Page 61] + +RFC 2616 HTTP/1.1 June 1999 + + + the user agent, selection of the most appropriate choice MAY be + performed automatically. However, this specification does not define + any standard for such automatic selection. + + If the server has a preferred choice of representation, it SHOULD + include the specific URI for that representation in the Location + field; user agents MAY use the Location field value for automatic + redirection. This response is cacheable unless indicated otherwise. + +10.3.2 301 Moved Permanently + + The requested resource has been assigned a new permanent URI and any + future references to this resource SHOULD use one of the returned + URIs. Clients with link editing capabilities ought to automatically + re-link references to the Request-URI to one or more of the new + references returned by the server, where possible. This response is + cacheable unless indicated otherwise. + + The new permanent URI SHOULD be given by the Location field in the + response. Unless the request method was HEAD, the entity of the + response SHOULD contain a short hypertext note with a hyperlink to + the new URI(s). + + If the 301 status code is received in response to a request other + than GET or HEAD, the user agent MUST NOT automatically redirect the + request unless it can be confirmed by the user, since this might + change the conditions under which the request was issued. + + Note: When automatically redirecting a POST request after + receiving a 301 status code, some existing HTTP/1.0 user agents + will erroneously change it into a GET request. + +10.3.3 302 Found + + The requested resource resides temporarily under a different URI. + Since the redirection might be altered on occasion, the client SHOULD + continue to use the Request-URI for future requests. This response + is only cacheable if indicated by a Cache-Control or Expires header + field. + + The temporary URI SHOULD be given by the Location field in the + response. Unless the request method was HEAD, the entity of the + response SHOULD contain a short hypertext note with a hyperlink to + the new URI(s). + + + + + + + +Fielding, et al. Standards Track [Page 62] + +RFC 2616 HTTP/1.1 June 1999 + + + If the 302 status code is received in response to a request other + than GET or HEAD, the user agent MUST NOT automatically redirect the + request unless it can be confirmed by the user, since this might + change the conditions under which the request was issued. + + Note: RFC 1945 and RFC 2068 specify that the client is not allowed + to change the method on the redirected request. However, most + existing user agent implementations treat 302 as if it were a 303 + response, performing a GET on the Location field-value regardless + of the original request method. The status codes 303 and 307 have + been added for servers that wish to make unambiguously clear which + kind of reaction is expected of the client. + +10.3.4 303 See Other + + The response to the request can be found under a different URI and + SHOULD be retrieved using a GET method on that resource. This method + exists primarily to allow the output of a POST-activated script to + redirect the user agent to a selected resource. The new URI is not a + substitute reference for the originally requested resource. The 303 + response MUST NOT be cached, but the response to the second + (redirected) request might be cacheable. + + The different URI SHOULD be given by the Location field in the + response. Unless the request method was HEAD, the entity of the + response SHOULD contain a short hypertext note with a hyperlink to + the new URI(s). + + Note: Many pre-HTTP/1.1 user agents do not understand the 303 + status. When interoperability with such clients is a concern, the + 302 status code may be used instead, since most user agents react + to a 302 response as described here for 303. + +10.3.5 304 Not Modified + + If the client has performed a conditional GET request and access is + allowed, but the document has not been modified, the server SHOULD + respond with this status code. The 304 response MUST NOT contain a + message-body, and thus is always terminated by the first empty line + after the header fields. + + The response MUST include the following header fields: + + - Date, unless its omission is required by section 14.18.1 + + + + + + + +Fielding, et al. Standards Track [Page 63] + +RFC 2616 HTTP/1.1 June 1999 + + + If a clockless origin server obeys these rules, and proxies and + clients add their own Date to any response received without one (as + already specified by [RFC 2068], section 14.19), caches will operate + correctly. + + - ETag and/or Content-Location, if the header would have been sent + in a 200 response to the same request + + - Expires, Cache-Control, and/or Vary, if the field-value might + differ from that sent in any previous response for the same + variant + + If the conditional GET used a strong cache validator (see section + 13.3.3), the response SHOULD NOT include other entity-headers. + Otherwise (i.e., the conditional GET used a weak validator), the + response MUST NOT include other entity-headers; this prevents + inconsistencies between cached entity-bodies and updated headers. + + If a 304 response indicates an entity not currently cached, then the + cache MUST disregard the response and repeat the request without the + conditional. + + If a cache uses a received 304 response to update a cache entry, the + cache MUST update the entry to reflect any new field values given in + the response. + +10.3.6 305 Use Proxy + + The requested resource MUST be accessed through the proxy given by + the Location field. The Location field gives the URI of the proxy. + The recipient is expected to repeat this single request via the + proxy. 305 responses MUST only be generated by origin servers. + + Note: RFC 2068 was not clear that 305 was intended to redirect a + single request, and to be generated by origin servers only. Not + observing these limitations has significant security consequences. + +10.3.7 306 (Unused) + + The 306 status code was used in a previous version of the + specification, is no longer used, and the code is reserved. + + + + + + + + + + +Fielding, et al. Standards Track [Page 64] + +RFC 2616 HTTP/1.1 June 1999 + + +10.3.8 307 Temporary Redirect + + The requested resource resides temporarily under a different URI. + Since the redirection MAY be altered on occasion, the client SHOULD + continue to use the Request-URI for future requests. This response + is only cacheable if indicated by a Cache-Control or Expires header + field. + + The temporary URI SHOULD be given by the Location field in the + response. Unless the request method was HEAD, the entity of the + response SHOULD contain a short hypertext note with a hyperlink to + the new URI(s) , since many pre-HTTP/1.1 user agents do not + understand the 307 status. Therefore, the note SHOULD contain the + information necessary for a user to repeat the original request on + the new URI. + + If the 307 status code is received in response to a request other + than GET or HEAD, the user agent MUST NOT automatically redirect the + request unless it can be confirmed by the user, since this might + change the conditions under which the request was issued. + +10.4 Client Error 4xx + + The 4xx class of status code is intended for cases in which the + client seems to have erred. Except when responding to a HEAD request, + the server SHOULD include an entity containing an explanation of the + error situation, and whether it is a temporary or permanent + condition. These status codes are applicable to any request method. + User agents SHOULD display any included entity to the user. + + If the client is sending data, a server implementation using TCP + SHOULD be careful to ensure that the client acknowledges receipt of + the packet(s) containing the response, before the server closes the + input connection. If the client continues sending data to the server + after the close, the server's TCP stack will send a reset packet to + the client, which may erase the client's unacknowledged input buffers + before they can be read and interpreted by the HTTP application. + +10.4.1 400 Bad Request + + The request could not be understood by the server due to malformed + syntax. The client SHOULD NOT repeat the request without + modifications. + + + + + + + + +Fielding, et al. Standards Track [Page 65] + +RFC 2616 HTTP/1.1 June 1999 + + +10.4.2 401 Unauthorized + + The request requires user authentication. The response MUST include a + WWW-Authenticate header field (section 14.47) containing a challenge + applicable to the requested resource. The client MAY repeat the + request with a suitable Authorization header field (section 14.8). If + the request already included Authorization credentials, then the 401 + response indicates that authorization has been refused for those + credentials. If the 401 response contains the same challenge as the + prior response, and the user agent has already attempted + authentication at least once, then the user SHOULD be presented the + entity that was given in the response, since that entity might + include relevant diagnostic information. HTTP access authentication + is explained in "HTTP Authentication: Basic and Digest Access + Authentication" [43]. + +10.4.3 402 Payment Required + + This code is reserved for future use. + +10.4.4 403 Forbidden + + The server understood the request, but is refusing to fulfill it. + Authorization will not help and the request SHOULD NOT be repeated. + If the request method was not HEAD and the server wishes to make + public why the request has not been fulfilled, it SHOULD describe the + reason for the refusal in the entity. If the server does not wish to + make this information available to the client, the status code 404 + (Not Found) can be used instead. + +10.4.5 404 Not Found + + The server has not found anything matching the Request-URI. No + indication is given of whether the condition is temporary or + permanent. The 410 (Gone) status code SHOULD be used if the server + knows, through some internally configurable mechanism, that an old + resource is permanently unavailable and has no forwarding address. + This status code is commonly used when the server does not wish to + reveal exactly why the request has been refused, or when no other + response is applicable. + +10.4.6 405 Method Not Allowed + + The method specified in the Request-Line is not allowed for the + resource identified by the Request-URI. The response MUST include an + Allow header containing a list of valid methods for the requested + resource. + + + + +Fielding, et al. Standards Track [Page 66] + +RFC 2616 HTTP/1.1 June 1999 + + +10.4.7 406 Not Acceptable + + The resource identified by the request is only capable of generating + response entities which have content characteristics not acceptable + according to the accept headers sent in the request. + + Unless it was a HEAD request, the response SHOULD include an entity + containing a list of available entity characteristics and location(s) + from which the user or user agent can choose the one most + appropriate. The entity format is specified by the media type given + in the Content-Type header field. Depending upon the format and the + capabilities of the user agent, selection of the most appropriate + choice MAY be performed automatically. However, this specification + does not define any standard for such automatic selection. + + Note: HTTP/1.1 servers are allowed to return responses which are + not acceptable according to the accept headers sent in the + request. In some cases, this may even be preferable to sending a + 406 response. User agents are encouraged to inspect the headers of + an incoming response to determine if it is acceptable. + + If the response could be unacceptable, a user agent SHOULD + temporarily stop receipt of more data and query the user for a + decision on further actions. + +10.4.8 407 Proxy Authentication Required + + This code is similar to 401 (Unauthorized), but indicates that the + client must first authenticate itself with the proxy. The proxy MUST + return a Proxy-Authenticate header field (section 14.33) containing a + challenge applicable to the proxy for the requested resource. The + client MAY repeat the request with a suitable Proxy-Authorization + header field (section 14.34). HTTP access authentication is explained + in "HTTP Authentication: Basic and Digest Access Authentication" + [43]. + +10.4.9 408 Request Timeout + + The client did not produce a request within the time that the server + was prepared to wait. The client MAY repeat the request without + modifications at any later time. + +10.4.10 409 Conflict + + The request could not be completed due to a conflict with the current + state of the resource. This code is only allowed in situations where + it is expected that the user might be able to resolve the conflict + and resubmit the request. The response body SHOULD include enough + + + +Fielding, et al. Standards Track [Page 67] + +RFC 2616 HTTP/1.1 June 1999 + + + information for the user to recognize the source of the conflict. + Ideally, the response entity would include enough information for the + user or user agent to fix the problem; however, that might not be + possible and is not required. + + Conflicts are most likely to occur in response to a PUT request. For + example, if versioning were being used and the entity being PUT + included changes to a resource which conflict with those made by an + earlier (third-party) request, the server might use the 409 response + to indicate that it can't complete the request. In this case, the + response entity would likely contain a list of the differences + between the two versions in a format defined by the response + Content-Type. + +10.4.11 410 Gone + + The requested resource is no longer available at the server and no + forwarding address is known. This condition is expected to be + considered permanent. Clients with link editing capabilities SHOULD + delete references to the Request-URI after user approval. If the + server does not know, or has no facility to determine, whether or not + the condition is permanent, the status code 404 (Not Found) SHOULD be + used instead. This response is cacheable unless indicated otherwise. + + The 410 response is primarily intended to assist the task of web + maintenance by notifying the recipient that the resource is + intentionally unavailable and that the server owners desire that + remote links to that resource be removed. Such an event is common for + limited-time, promotional services and for resources belonging to + individuals no longer working at the server's site. It is not + necessary to mark all permanently unavailable resources as "gone" or + to keep the mark for any length of time -- that is left to the + discretion of the server owner. + +10.4.12 411 Length Required + + The server refuses to accept the request without a defined Content- + Length. The client MAY repeat the request if it adds a valid + Content-Length header field containing the length of the message-body + in the request message. + +10.4.13 412 Precondition Failed + + The precondition given in one or more of the request-header fields + evaluated to false when it was tested on the server. This response + code allows the client to place preconditions on the current resource + metainformation (header field data) and thus prevent the requested + method from being applied to a resource other than the one intended. + + + +Fielding, et al. Standards Track [Page 68] + +RFC 2616 HTTP/1.1 June 1999 + + +10.4.14 413 Request Entity Too Large + + The server is refusing to process a request because the request + entity is larger than the server is willing or able to process. The + server MAY close the connection to prevent the client from continuing + the request. + + If the condition is temporary, the server SHOULD include a Retry- + After header field to indicate that it is temporary and after what + time the client MAY try again. + +10.4.15 414 Request-URI Too Long + + The server is refusing to service the request because the Request-URI + is longer than the server is willing to interpret. This rare + condition is only likely to occur when a client has improperly + converted a POST request to a GET request with long query + information, when the client has descended into a URI "black hole" of + redirection (e.g., a redirected URI prefix that points to a suffix of + itself), or when the server is under attack by a client attempting to + exploit security holes present in some servers using fixed-length + buffers for reading or manipulating the Request-URI. + +10.4.16 415 Unsupported Media Type + + The server is refusing to service the request because the entity of + the request is in a format not supported by the requested resource + for the requested method. + +10.4.17 416 Requested Range Not Satisfiable + + A server SHOULD return a response with this status code if a request + included a Range request-header field (section 14.35), and none of + the range-specifier values in this field overlap the current extent + of the selected resource, and the request did not include an If-Range + request-header field. (For byte-ranges, this means that the first- + byte-pos of all of the byte-range-spec values were greater than the + current length of the selected resource.) + + When this status code is returned for a byte-range request, the + response SHOULD include a Content-Range entity-header field + specifying the current length of the selected resource (see section + 14.16). This response MUST NOT use the multipart/byteranges content- + type. + + + + + + + +Fielding, et al. Standards Track [Page 69] + +RFC 2616 HTTP/1.1 June 1999 + + +10.4.18 417 Expectation Failed + + The expectation given in an Expect request-header field (see section + 14.20) could not be met by this server, or, if the server is a proxy, + the server has unambiguous evidence that the request could not be met + by the next-hop server. + +10.5 Server Error 5xx + + Response status codes beginning with the digit "5" indicate cases in + which the server is aware that it has erred or is incapable of + performing the request. Except when responding to a HEAD request, the + server SHOULD include an entity containing an explanation of the + error situation, and whether it is a temporary or permanent + condition. User agents SHOULD display any included entity to the + user. These response codes are applicable to any request method. + +10.5.1 500 Internal Server Error + + The server encountered an unexpected condition which prevented it + from fulfilling the request. + +10.5.2 501 Not Implemented + + The server does not support the functionality required to fulfill the + request. This is the appropriate response when the server does not + recognize the request method and is not capable of supporting it for + any resource. + +10.5.3 502 Bad Gateway + + The server, while acting as a gateway or proxy, received an invalid + response from the upstream server it accessed in attempting to + fulfill the request. + +10.5.4 503 Service Unavailable + + The server is currently unable to handle the request due to a + temporary overloading or maintenance of the server. The implication + is that this is a temporary condition which will be alleviated after + some delay. If known, the length of the delay MAY be indicated in a + Retry-After header. If no Retry-After is given, the client SHOULD + handle the response as it would for a 500 response. + + Note: The existence of the 503 status code does not imply that a + server must use it when becoming overloaded. Some servers may wish + to simply refuse the connection. + + + + +Fielding, et al. Standards Track [Page 70] + +RFC 2616 HTTP/1.1 June 1999 + + +10.5.5 504 Gateway Timeout + + The server, while acting as a gateway or proxy, did not receive a + timely response from the upstream server specified by the URI (e.g. + HTTP, FTP, LDAP) or some other auxiliary server (e.g. DNS) it needed + to access in attempting to complete the request. + + Note: Note to implementors: some deployed proxies are known to + return 400 or 500 when DNS lookups time out. + +10.5.6 505 HTTP Version Not Supported + + The server does not support, or refuses to support, the HTTP protocol + version that was used in the request message. The server is + indicating that it is unable or unwilling to complete the request + using the same major version as the client, as described in section + 3.1, other than with this error message. The response SHOULD contain + an entity describing why that version is not supported and what other + protocols are supported by that server. + +11 Access Authentication + + HTTP provides several OPTIONAL challenge-response authentication + mechanisms which can be used by a server to challenge a client + request and by a client to provide authentication information. The + general framework for access authentication, and the specification of + "basic" and "digest" authentication, are specified in "HTTP + Authentication: Basic and Digest Access Authentication" [43]. This + specification adopts the definitions of "challenge" and "credentials" + from that specification. + +12 Content Negotiation + + Most HTTP responses include an entity which contains information for + interpretation by a human user. Naturally, it is desirable to supply + the user with the "best available" entity corresponding to the + request. Unfortunately for servers and caches, not all users have the + same preferences for what is "best," and not all user agents are + equally capable of rendering all entity types. For that reason, HTTP + has provisions for several mechanisms for "content negotiation" -- + the process of selecting the best representation for a given response + when there are multiple representations available. + + Note: This is not called "format negotiation" because the + alternate representations may be of the same media type, but use + different capabilities of that type, be in different languages, + etc. + + + + +Fielding, et al. Standards Track [Page 71] + +RFC 2616 HTTP/1.1 June 1999 + + + Any response containing an entity-body MAY be subject to negotiation, + including error responses. + + There are two kinds of content negotiation which are possible in + HTTP: server-driven and agent-driven negotiation. These two kinds of + negotiation are orthogonal and thus may be used separately or in + combination. One method of combination, referred to as transparent + negotiation, occurs when a cache uses the agent-driven negotiation + information provided by the origin server in order to provide + server-driven negotiation for subsequent requests. + +12.1 Server-driven Negotiation + + If the selection of the best representation for a response is made by + an algorithm located at the server, it is called server-driven + negotiation. Selection is based on the available representations of + the response (the dimensions over which it can vary; e.g. language, + content-coding, etc.) and the contents of particular header fields in + the request message or on other information pertaining to the request + (such as the network address of the client). + + Server-driven negotiation is advantageous when the algorithm for + selecting from among the available representations is difficult to + describe to the user agent, or when the server desires to send its + "best guess" to the client along with the first response (hoping to + avoid the round-trip delay of a subsequent request if the "best + guess" is good enough for the user). In order to improve the server's + guess, the user agent MAY include request header fields (Accept, + Accept-Language, Accept-Encoding, etc.) which describe its + preferences for such a response. + + Server-driven negotiation has disadvantages: + + 1. It is impossible for the server to accurately determine what + might be "best" for any given user, since that would require + complete knowledge of both the capabilities of the user agent + and the intended use for the response (e.g., does the user want + to view it on screen or print it on paper?). + + 2. Having the user agent describe its capabilities in every + request can be both very inefficient (given that only a small + percentage of responses have multiple representations) and a + potential violation of the user's privacy. + + 3. It complicates the implementation of an origin server and the + algorithms for generating responses to a request. + + + + + +Fielding, et al. Standards Track [Page 72] + +RFC 2616 HTTP/1.1 June 1999 + + + 4. It may limit a public cache's ability to use the same response + for multiple user's requests. + + HTTP/1.1 includes the following request-header fields for enabling + server-driven negotiation through description of user agent + capabilities and user preferences: Accept (section 14.1), Accept- + Charset (section 14.2), Accept-Encoding (section 14.3), Accept- + Language (section 14.4), and User-Agent (section 14.43). However, an + origin server is not limited to these dimensions and MAY vary the + response based on any aspect of the request, including information + outside the request-header fields or within extension header fields + not defined by this specification. + + The Vary header field can be used to express the parameters the + server uses to select a representation that is subject to server- + driven negotiation. See section 13.6 for use of the Vary header field + by caches and section 14.44 for use of the Vary header field by + servers. + +12.2 Agent-driven Negotiation + + With agent-driven negotiation, selection of the best representation + for a response is performed by the user agent after receiving an + initial response from the origin server. Selection is based on a list + of the available representations of the response included within the + header fields or entity-body of the initial response, with each + representation identified by its own URI. Selection from among the + representations may be performed automatically (if the user agent is + capable of doing so) or manually by the user selecting from a + generated (possibly hypertext) menu. + + Agent-driven negotiation is advantageous when the response would vary + over commonly-used dimensions (such as type, language, or encoding), + when the origin server is unable to determine a user agent's + capabilities from examining the request, and generally when public + caches are used to distribute server load and reduce network usage. + + Agent-driven negotiation suffers from the disadvantage of needing a + second request to obtain the best alternate representation. This + second request is only efficient when caching is used. In addition, + this specification does not define any mechanism for supporting + automatic selection, though it also does not prevent any such + mechanism from being developed as an extension and used within + HTTP/1.1. + + + + + + + +Fielding, et al. Standards Track [Page 73] + +RFC 2616 HTTP/1.1 June 1999 + + + HTTP/1.1 defines the 300 (Multiple Choices) and 406 (Not Acceptable) + status codes for enabling agent-driven negotiation when the server is + unwilling or unable to provide a varying response using server-driven + negotiation. + +12.3 Transparent Negotiation + + Transparent negotiation is a combination of both server-driven and + agent-driven negotiation. When a cache is supplied with a form of the + list of available representations of the response (as in agent-driven + negotiation) and the dimensions of variance are completely understood + by the cache, then the cache becomes capable of performing server- + driven negotiation on behalf of the origin server for subsequent + requests on that resource. + + Transparent negotiation has the advantage of distributing the + negotiation work that would otherwise be required of the origin + server and also removing the second request delay of agent-driven + negotiation when the cache is able to correctly guess the right + response. + + This specification does not define any mechanism for transparent + negotiation, though it also does not prevent any such mechanism from + being developed as an extension that could be used within HTTP/1.1. + +13 Caching in HTTP + + HTTP is typically used for distributed information systems, where + performance can be improved by the use of response caches. The + HTTP/1.1 protocol includes a number of elements intended to make + caching work as well as possible. Because these elements are + inextricable from other aspects of the protocol, and because they + interact with each other, it is useful to describe the basic caching + design of HTTP separately from the detailed descriptions of methods, + headers, response codes, etc. + + Caching would be useless if it did not significantly improve + performance. The goal of caching in HTTP/1.1 is to eliminate the need + to send requests in many cases, and to eliminate the need to send + full responses in many other cases. The former reduces the number of + network round-trips required for many operations; we use an + "expiration" mechanism for this purpose (see section 13.2). The + latter reduces network bandwidth requirements; we use a "validation" + mechanism for this purpose (see section 13.3). + + Requirements for performance, availability, and disconnected + operation require us to be able to relax the goal of semantic + transparency. The HTTP/1.1 protocol allows origin servers, caches, + + + +Fielding, et al. Standards Track [Page 74] + +RFC 2616 HTTP/1.1 June 1999 + + + and clients to explicitly reduce transparency when necessary. + However, because non-transparent operation may confuse non-expert + users, and might be incompatible with certain server applications + (such as those for ordering merchandise), the protocol requires that + transparency be relaxed + + - only by an explicit protocol-level request when relaxed by + client or origin server + + - only with an explicit warning to the end user when relaxed by + cache or client + + Therefore, the HTTP/1.1 protocol provides these important elements: + + 1. Protocol features that provide full semantic transparency when + this is required by all parties. + + 2. Protocol features that allow an origin server or user agent to + explicitly request and control non-transparent operation. + + 3. Protocol features that allow a cache to attach warnings to + responses that do not preserve the requested approximation of + semantic transparency. + + A basic principle is that it must be possible for the clients to + detect any potential relaxation of semantic transparency. + + Note: The server, cache, or client implementor might be faced with + design decisions not explicitly discussed in this specification. + If a decision might affect semantic transparency, the implementor + ought to err on the side of maintaining transparency unless a + careful and complete analysis shows significant benefits in + breaking transparency. + +13.1.1 Cache Correctness + + A correct cache MUST respond to a request with the most up-to-date + response held by the cache that is appropriate to the request (see + sections 13.2.5, 13.2.6, and 13.12) which meets one of the following + conditions: + + 1. It has been checked for equivalence with what the origin server + would have returned by revalidating the response with the + origin server (section 13.3); + + + + + + + +Fielding, et al. Standards Track [Page 75] + +RFC 2616 HTTP/1.1 June 1999 + + + 2. It is "fresh enough" (see section 13.2). In the default case, + this means it meets the least restrictive freshness requirement + of the client, origin server, and cache (see section 14.9); if + the origin server so specifies, it is the freshness requirement + of the origin server alone. + + If a stored response is not "fresh enough" by the most + restrictive freshness requirement of both the client and the + origin server, in carefully considered circumstances the cache + MAY still return the response with the appropriate Warning + header (see section 13.1.5 and 14.46), unless such a response + is prohibited (e.g., by a "no-store" cache-directive, or by a + "no-cache" cache-request-directive; see section 14.9). + + 3. It is an appropriate 304 (Not Modified), 305 (Proxy Redirect), + or error (4xx or 5xx) response message. + + If the cache can not communicate with the origin server, then a + correct cache SHOULD respond as above if the response can be + correctly served from the cache; if not it MUST return an error or + warning indicating that there was a communication failure. + + If a cache receives a response (either an entire response, or a 304 + (Not Modified) response) that it would normally forward to the + requesting client, and the received response is no longer fresh, the + cache SHOULD forward it to the requesting client without adding a new + Warning (but without removing any existing Warning headers). A cache + SHOULD NOT attempt to revalidate a response simply because that + response became stale in transit; this might lead to an infinite + loop. A user agent that receives a stale response without a Warning + MAY display a warning indication to the user. + +13.1.2 Warnings + + Whenever a cache returns a response that is neither first-hand nor + "fresh enough" (in the sense of condition 2 in section 13.1.1), it + MUST attach a warning to that effect, using a Warning general-header. + The Warning header and the currently defined warnings are described + in section 14.46. The warning allows clients to take appropriate + action. + + Warnings MAY be used for other purposes, both cache-related and + otherwise. The use of a warning, rather than an error status code, + distinguish these responses from true failures. + + Warnings are assigned three digit warn-codes. The first digit + indicates whether the Warning MUST or MUST NOT be deleted from a + stored cache entry after a successful revalidation: + + + +Fielding, et al. Standards Track [Page 76] + +RFC 2616 HTTP/1.1 June 1999 + + + 1xx Warnings that describe the freshness or revalidation status of + the response, and so MUST be deleted after a successful + revalidation. 1XX warn-codes MAY be generated by a cache only when + validating a cached entry. It MUST NOT be generated by clients. + + 2xx Warnings that describe some aspect of the entity body or entity + headers that is not rectified by a revalidation (for example, a + lossy compression of the entity bodies) and which MUST NOT be + deleted after a successful revalidation. + + See section 14.46 for the definitions of the codes themselves. + + HTTP/1.0 caches will cache all Warnings in responses, without + deleting the ones in the first category. Warnings in responses that + are passed to HTTP/1.0 caches carry an extra warning-date field, + which prevents a future HTTP/1.1 recipient from believing an + erroneously cached Warning. + + Warnings also carry a warning text. The text MAY be in any + appropriate natural language (perhaps based on the client's Accept + headers), and include an OPTIONAL indication of what character set is + used. + + Multiple warnings MAY be attached to a response (either by the origin + server or by a cache), including multiple warnings with the same code + number. For example, a server might provide the same warning with + texts in both English and Basque. + + When multiple warnings are attached to a response, it might not be + practical or reasonable to display all of them to the user. This + version of HTTP does not specify strict priority rules for deciding + which warnings to display and in what order, but does suggest some + heuristics. + +13.1.3 Cache-control Mechanisms + + The basic cache mechanisms in HTTP/1.1 (server-specified expiration + times and validators) are implicit directives to caches. In some + cases, a server or client might need to provide explicit directives + to the HTTP caches. We use the Cache-Control header for this purpose. + + The Cache-Control header allows a client or server to transmit a + variety of directives in either requests or responses. These + directives typically override the default caching algorithms. As a + general rule, if there is any apparent conflict between header + values, the most restrictive interpretation is applied (that is, the + one that is most likely to preserve semantic transparency). However, + + + + +Fielding, et al. Standards Track [Page 77] + +RFC 2616 HTTP/1.1 June 1999 + + + in some cases, cache-control directives are explicitly specified as + weakening the approximation of semantic transparency (for example, + "max-stale" or "public"). + + The cache-control directives are described in detail in section 14.9. + +13.1.4 Explicit User Agent Warnings + + Many user agents make it possible for users to override the basic + caching mechanisms. For example, the user agent might allow the user + to specify that cached entities (even explicitly stale ones) are + never validated. Or the user agent might habitually add "Cache- + Control: max-stale=3600" to every request. The user agent SHOULD NOT + default to either non-transparent behavior, or behavior that results + in abnormally ineffective caching, but MAY be explicitly configured + to do so by an explicit action of the user. + + If the user has overridden the basic caching mechanisms, the user + agent SHOULD explicitly indicate to the user whenever this results in + the display of information that might not meet the server's + transparency requirements (in particular, if the displayed entity is + known to be stale). Since the protocol normally allows the user agent + to determine if responses are stale or not, this indication need only + be displayed when this actually happens. The indication need not be a + dialog box; it could be an icon (for example, a picture of a rotting + fish) or some other indicator. + + If the user has overridden the caching mechanisms in a way that would + abnormally reduce the effectiveness of caches, the user agent SHOULD + continually indicate this state to the user (for example, by a + display of a picture of currency in flames) so that the user does not + inadvertently consume excess resources or suffer from excessive + latency. + +13.1.5 Exceptions to the Rules and Warnings + + In some cases, the operator of a cache MAY choose to configure it to + return stale responses even when not requested by clients. This + decision ought not be made lightly, but may be necessary for reasons + of availability or performance, especially when the cache is poorly + connected to the origin server. Whenever a cache returns a stale + response, it MUST mark it as such (using a Warning header) enabling + the client software to alert the user that there might be a potential + problem. + + + + + + + +Fielding, et al. Standards Track [Page 78] + +RFC 2616 HTTP/1.1 June 1999 + + + It also allows the user agent to take steps to obtain a first-hand or + fresh response. For this reason, a cache SHOULD NOT return a stale + response if the client explicitly requests a first-hand or fresh one, + unless it is impossible to comply for technical or policy reasons. + +13.1.6 Client-controlled Behavior + + While the origin server (and to a lesser extent, intermediate caches, + by their contribution to the age of a response) are the primary + source of expiration information, in some cases the client might need + to control a cache's decision about whether to return a cached + response without validating it. Clients do this using several + directives of the Cache-Control header. + + A client's request MAY specify the maximum age it is willing to + accept of an unvalidated response; specifying a value of zero forces + the cache(s) to revalidate all responses. A client MAY also specify + the minimum time remaining before a response expires. Both of these + options increase constraints on the behavior of caches, and so cannot + further relax the cache's approximation of semantic transparency. + + A client MAY also specify that it will accept stale responses, up to + some maximum amount of staleness. This loosens the constraints on the + caches, and so might violate the origin server's specified + constraints on semantic transparency, but might be necessary to + support disconnected operation, or high availability in the face of + poor connectivity. + +13.2 Expiration Model + +13.2.1 Server-Specified Expiration + + HTTP caching works best when caches can entirely avoid making + requests to the origin server. The primary mechanism for avoiding + requests is for an origin server to provide an explicit expiration + time in the future, indicating that a response MAY be used to satisfy + subsequent requests. In other words, a cache can return a fresh + response without first contacting the server. + + Our expectation is that servers will assign future explicit + expiration times to responses in the belief that the entity is not + likely to change, in a semantically significant way, before the + expiration time is reached. This normally preserves semantic + transparency, as long as the server's expiration times are carefully + chosen. + + + + + + +Fielding, et al. Standards Track [Page 79] + +RFC 2616 HTTP/1.1 June 1999 + + + The expiration mechanism applies only to responses taken from a cache + and not to first-hand responses forwarded immediately to the + requesting client. + + If an origin server wishes to force a semantically transparent cache + to validate every request, it MAY assign an explicit expiration time + in the past. This means that the response is always stale, and so the + cache SHOULD validate it before using it for subsequent requests. See + section 14.9.4 for a more restrictive way to force revalidation. + + If an origin server wishes to force any HTTP/1.1 cache, no matter how + it is configured, to validate every request, it SHOULD use the "must- + revalidate" cache-control directive (see section 14.9). + + Servers specify explicit expiration times using either the Expires + header, or the max-age directive of the Cache-Control header. + + An expiration time cannot be used to force a user agent to refresh + its display or reload a resource; its semantics apply only to caching + mechanisms, and such mechanisms need only check a resource's + expiration status when a new request for that resource is initiated. + See section 13.13 for an explanation of the difference between caches + and history mechanisms. + +13.2.2 Heuristic Expiration + + Since origin servers do not always provide explicit expiration times, + HTTP caches typically assign heuristic expiration times, employing + algorithms that use other header values (such as the Last-Modified + time) to estimate a plausible expiration time. The HTTP/1.1 + specification does not provide specific algorithms, but does impose + worst-case constraints on their results. Since heuristic expiration + times might compromise semantic transparency, they ought to used + cautiously, and we encourage origin servers to provide explicit + expiration times as much as possible. + +13.2.3 Age Calculations + + In order to know if a cached entry is fresh, a cache needs to know if + its age exceeds its freshness lifetime. We discuss how to calculate + the latter in section 13.2.4; this section describes how to calculate + the age of a response or cache entry. + + In this discussion, we use the term "now" to mean "the current value + of the clock at the host performing the calculation." Hosts that use + HTTP, but especially hosts running origin servers and caches, SHOULD + use NTP [28] or some similar protocol to synchronize their clocks to + a globally accurate time standard. + + + +Fielding, et al. Standards Track [Page 80] + +RFC 2616 HTTP/1.1 June 1999 + + + HTTP/1.1 requires origin servers to send a Date header, if possible, + with every response, giving the time at which the response was + generated (see section 14.18). We use the term "date_value" to denote + the value of the Date header, in a form appropriate for arithmetic + operations. + + HTTP/1.1 uses the Age response-header to convey the estimated age of + the response message when obtained from a cache. The Age field value + is the cache's estimate of the amount of time since the response was + generated or revalidated by the origin server. + + In essence, the Age value is the sum of the time that the response + has been resident in each of the caches along the path from the + origin server, plus the amount of time it has been in transit along + network paths. + + We use the term "age_value" to denote the value of the Age header, in + a form appropriate for arithmetic operations. + + A response's age can be calculated in two entirely independent ways: + + 1. now minus date_value, if the local clock is reasonably well + synchronized to the origin server's clock. If the result is + negative, the result is replaced by zero. + + 2. age_value, if all of the caches along the response path + implement HTTP/1.1. + + Given that we have two independent ways to compute the age of a + response when it is received, we can combine these as + + corrected_received_age = max(now - date_value, age_value) + + and as long as we have either nearly synchronized clocks or all- + HTTP/1.1 paths, one gets a reliable (conservative) result. + + Because of network-imposed delays, some significant interval might + pass between the time that a server generates a response and the time + it is received at the next outbound cache or client. If uncorrected, + this delay could result in improperly low ages. + + Because the request that resulted in the returned Age value must have + been initiated prior to that Age value's generation, we can correct + for delays imposed by the network by recording the time at which the + request was initiated. Then, when an Age value is received, it MUST + be interpreted relative to the time the request was initiated, not + + + + + +Fielding, et al. Standards Track [Page 81] + +RFC 2616 HTTP/1.1 June 1999 + + + the time that the response was received. This algorithm results in + conservative behavior no matter how much delay is experienced. So, we + compute: + + corrected_initial_age = corrected_received_age + + (now - request_time) + + where "request_time" is the time (according to the local clock) when + the request that elicited this response was sent. + + Summary of age calculation algorithm, when a cache receives a + response: + + /* + * age_value + * is the value of Age: header received by the cache with + * this response. + * date_value + * is the value of the origin server's Date: header + * request_time + * is the (local) time when the cache made the request + * that resulted in this cached response + * response_time + * is the (local) time when the cache received the + * response + * now + * is the current (local) time + */ + + apparent_age = max(0, response_time - date_value); + corrected_received_age = max(apparent_age, age_value); + response_delay = response_time - request_time; + corrected_initial_age = corrected_received_age + response_delay; + resident_time = now - response_time; + current_age = corrected_initial_age + resident_time; + + The current_age of a cache entry is calculated by adding the amount + of time (in seconds) since the cache entry was last validated by the + origin server to the corrected_initial_age. When a response is + generated from a cache entry, the cache MUST include a single Age + header field in the response with a value equal to the cache entry's + current_age. + + The presence of an Age header field in a response implies that a + response is not first-hand. However, the converse is not true, since + the lack of an Age header field in a response does not imply that the + + + + + +Fielding, et al. Standards Track [Page 82] + +RFC 2616 HTTP/1.1 June 1999 + + + response is first-hand unless all caches along the request path are + compliant with HTTP/1.1 (i.e., older HTTP caches did not implement + the Age header field). + +13.2.4 Expiration Calculations + + In order to decide whether a response is fresh or stale, we need to + compare its freshness lifetime to its age. The age is calculated as + described in section 13.2.3; this section describes how to calculate + the freshness lifetime, and to determine if a response has expired. + In the discussion below, the values can be represented in any form + appropriate for arithmetic operations. + + We use the term "expires_value" to denote the value of the Expires + header. We use the term "max_age_value" to denote an appropriate + value of the number of seconds carried by the "max-age" directive of + the Cache-Control header in a response (see section 14.9.3). + + The max-age directive takes priority over Expires, so if max-age is + present in a response, the calculation is simply: + + freshness_lifetime = max_age_value + + Otherwise, if Expires is present in the response, the calculation is: + + freshness_lifetime = expires_value - date_value + + Note that neither of these calculations is vulnerable to clock skew, + since all of the information comes from the origin server. + + If none of Expires, Cache-Control: max-age, or Cache-Control: s- + maxage (see section 14.9.3) appears in the response, and the response + does not include other restrictions on caching, the cache MAY compute + a freshness lifetime using a heuristic. The cache MUST attach Warning + 113 to any response whose age is more than 24 hours if such warning + has not already been added. + + Also, if the response does have a Last-Modified time, the heuristic + expiration value SHOULD be no more than some fraction of the interval + since that time. A typical setting of this fraction might be 10%. + + The calculation to determine if a response has expired is quite + simple: + + response_is_fresh = (freshness_lifetime > current_age) + + + + + + +Fielding, et al. Standards Track [Page 83] + +RFC 2616 HTTP/1.1 June 1999 + + +13.2.5 Disambiguating Expiration Values + + Because expiration values are assigned optimistically, it is possible + for two caches to contain fresh values for the same resource that are + different. + + If a client performing a retrieval receives a non-first-hand response + for a request that was already fresh in its own cache, and the Date + header in its existing cache entry is newer than the Date on the new + response, then the client MAY ignore the response. If so, it MAY + retry the request with a "Cache-Control: max-age=0" directive (see + section 14.9), to force a check with the origin server. + + If a cache has two fresh responses for the same representation with + different validators, it MUST use the one with the more recent Date + header. This situation might arise because the cache is pooling + responses from other caches, or because a client has asked for a + reload or a revalidation of an apparently fresh cache entry. + +13.2.6 Disambiguating Multiple Responses + + Because a client might be receiving responses via multiple paths, so + that some responses flow through one set of caches and other + responses flow through a different set of caches, a client might + receive responses in an order different from that in which the origin + server sent them. We would like the client to use the most recently + generated response, even if older responses are still apparently + fresh. + + Neither the entity tag nor the expiration value can impose an + ordering on responses, since it is possible that a later response + intentionally carries an earlier expiration time. The Date values are + ordered to a granularity of one second. + + When a client tries to revalidate a cache entry, and the response it + receives contains a Date header that appears to be older than the one + for the existing entry, then the client SHOULD repeat the request + unconditionally, and include + + Cache-Control: max-age=0 + + to force any intermediate caches to validate their copies directly + with the origin server, or + + Cache-Control: no-cache + + to force any intermediate caches to obtain a new copy from the origin + server. + + + +Fielding, et al. Standards Track [Page 84] + +RFC 2616 HTTP/1.1 June 1999 + + + If the Date values are equal, then the client MAY use either response + (or MAY, if it is being extremely prudent, request a new response). + Servers MUST NOT depend on clients being able to choose + deterministically between responses generated during the same second, + if their expiration times overlap. + +13.3 Validation Model + + When a cache has a stale entry that it would like to use as a + response to a client's request, it first has to check with the origin + server (or possibly an intermediate cache with a fresh response) to + see if its cached entry is still usable. We call this "validating" + the cache entry. Since we do not want to have to pay the overhead of + retransmitting the full response if the cached entry is good, and we + do not want to pay the overhead of an extra round trip if the cached + entry is invalid, the HTTP/1.1 protocol supports the use of + conditional methods. + + The key protocol features for supporting conditional methods are + those concerned with "cache validators." When an origin server + generates a full response, it attaches some sort of validator to it, + which is kept with the cache entry. When a client (user agent or + proxy cache) makes a conditional request for a resource for which it + has a cache entry, it includes the associated validator in the + request. + + The server then checks that validator against the current validator + for the entity, and, if they match (see section 13.3.3), it responds + with a special status code (usually, 304 (Not Modified)) and no + entity-body. Otherwise, it returns a full response (including + entity-body). Thus, we avoid transmitting the full response if the + validator matches, and we avoid an extra round trip if it does not + match. + + In HTTP/1.1, a conditional request looks exactly the same as a normal + request for the same resource, except that it carries a special + header (which includes the validator) that implicitly turns the + method (usually, GET) into a conditional. + + The protocol includes both positive and negative senses of cache- + validating conditions. That is, it is possible to request either that + a method be performed if and only if a validator matches or if and + only if no validators match. + + + + + + + + +Fielding, et al. Standards Track [Page 85] + +RFC 2616 HTTP/1.1 June 1999 + + + Note: a response that lacks a validator may still be cached, and + served from cache until it expires, unless this is explicitly + prohibited by a cache-control directive. However, a cache cannot + do a conditional retrieval if it does not have a validator for the + entity, which means it will not be refreshable after it expires. + +13.3.1 Last-Modified Dates + + The Last-Modified entity-header field value is often used as a cache + validator. In simple terms, a cache entry is considered to be valid + if the entity has not been modified since the Last-Modified value. + +13.3.2 Entity Tag Cache Validators + + The ETag response-header field value, an entity tag, provides for an + "opaque" cache validator. This might allow more reliable validation + in situations where it is inconvenient to store modification dates, + where the one-second resolution of HTTP date values is not + sufficient, or where the origin server wishes to avoid certain + paradoxes that might arise from the use of modification dates. + + Entity Tags are described in section 3.11. The headers used with + entity tags are described in sections 14.19, 14.24, 14.26 and 14.44. + +13.3.3 Weak and Strong Validators + + Since both origin servers and caches will compare two validators to + decide if they represent the same or different entities, one normally + would expect that if the entity (the entity-body or any entity- + headers) changes in any way, then the associated validator would + change as well. If this is true, then we call this validator a + "strong validator." + + However, there might be cases when a server prefers to change the + validator only on semantically significant changes, and not when + insignificant aspects of the entity change. A validator that does not + always change when the resource changes is a "weak validator." + + Entity tags are normally "strong validators," but the protocol + provides a mechanism to tag an entity tag as "weak." One can think of + a strong validator as one that changes whenever the bits of an entity + changes, while a weak value changes whenever the meaning of an entity + changes. Alternatively, one can think of a strong validator as part + of an identifier for a specific entity, while a weak validator is + part of an identifier for a set of semantically equivalent entities. + + Note: One example of a strong validator is an integer that is + incremented in stable storage every time an entity is changed. + + + +Fielding, et al. Standards Track [Page 86] + +RFC 2616 HTTP/1.1 June 1999 + + + An entity's modification time, if represented with one-second + resolution, could be a weak validator, since it is possible that + the resource might be modified twice during a single second. + + Support for weak validators is optional. However, weak validators + allow for more efficient caching of equivalent objects; for + example, a hit counter on a site is probably good enough if it is + updated every few days or weeks, and any value during that period + is likely "good enough" to be equivalent. + + A "use" of a validator is either when a client generates a request + and includes the validator in a validating header field, or when a + server compares two validators. + + Strong validators are usable in any context. Weak validators are only + usable in contexts that do not depend on exact equality of an entity. + For example, either kind is usable for a conditional GET of a full + entity. However, only a strong validator is usable for a sub-range + retrieval, since otherwise the client might end up with an internally + inconsistent entity. + + Clients MAY issue simple (non-subrange) GET requests with either weak + validators or strong validators. Clients MUST NOT use weak validators + in other forms of request. + + The only function that the HTTP/1.1 protocol defines on validators is + comparison. There are two validator comparison functions, depending + on whether the comparison context allows the use of weak validators + or not: + + - The strong comparison function: in order to be considered equal, + both validators MUST be identical in every way, and both MUST + NOT be weak. + + - The weak comparison function: in order to be considered equal, + both validators MUST be identical in every way, but either or + both of them MAY be tagged as "weak" without affecting the + result. + + An entity tag is strong unless it is explicitly tagged as weak. + Section 3.11 gives the syntax for entity tags. + + A Last-Modified time, when used as a validator in a request, is + implicitly weak unless it is possible to deduce that it is strong, + using the following rules: + + - The validator is being compared by an origin server to the + actual current validator for the entity and, + + + +Fielding, et al. Standards Track [Page 87] + +RFC 2616 HTTP/1.1 June 1999 + + + - That origin server reliably knows that the associated entity did + not change twice during the second covered by the presented + validator. + + or + + - The validator is about to be used by a client in an If- + Modified-Since or If-Unmodified-Since header, because the client + has a cache entry for the associated entity, and + + - That cache entry includes a Date value, which gives the time + when the origin server sent the original response, and + + - The presented Last-Modified time is at least 60 seconds before + the Date value. + + or + + - The validator is being compared by an intermediate cache to the + validator stored in its cache entry for the entity, and + + - That cache entry includes a Date value, which gives the time + when the origin server sent the original response, and + + - The presented Last-Modified time is at least 60 seconds before + the Date value. + + This method relies on the fact that if two different responses were + sent by the origin server during the same second, but both had the + same Last-Modified time, then at least one of those responses would + have a Date value equal to its Last-Modified time. The arbitrary 60- + second limit guards against the possibility that the Date and Last- + Modified values are generated from different clocks, or at somewhat + different times during the preparation of the response. An + implementation MAY use a value larger than 60 seconds, if it is + believed that 60 seconds is too short. + + If a client wishes to perform a sub-range retrieval on a value for + which it has only a Last-Modified time and no opaque validator, it + MAY do this only if the Last-Modified time is strong in the sense + described here. + + A cache or origin server receiving a conditional request, other than + a full-body GET request, MUST use the strong comparison function to + evaluate the condition. + + These rules allow HTTP/1.1 caches and clients to safely perform sub- + range retrievals on values that have been obtained from HTTP/1.0 + + + +Fielding, et al. Standards Track [Page 88] + +RFC 2616 HTTP/1.1 June 1999 + + + servers. + +13.3.4 Rules for When to Use Entity Tags and Last-Modified Dates + + We adopt a set of rules and recommendations for origin servers, + clients, and caches regarding when various validator types ought to + be used, and for what purposes. + + HTTP/1.1 origin servers: + + - SHOULD send an entity tag validator unless it is not feasible to + generate one. + + - MAY send a weak entity tag instead of a strong entity tag, if + performance considerations support the use of weak entity tags, + or if it is unfeasible to send a strong entity tag. + + - SHOULD send a Last-Modified value if it is feasible to send one, + unless the risk of a breakdown in semantic transparency that + could result from using this date in an If-Modified-Since header + would lead to serious problems. + + In other words, the preferred behavior for an HTTP/1.1 origin server + is to send both a strong entity tag and a Last-Modified value. + + In order to be legal, a strong entity tag MUST change whenever the + associated entity value changes in any way. A weak entity tag SHOULD + change whenever the associated entity changes in a semantically + significant way. + + Note: in order to provide semantically transparent caching, an + origin server must avoid reusing a specific strong entity tag + value for two different entities, or reusing a specific weak + entity tag value for two semantically different entities. Cache + entries might persist for arbitrarily long periods, regardless of + expiration times, so it might be inappropriate to expect that a + cache will never again attempt to validate an entry using a + validator that it obtained at some point in the past. + + HTTP/1.1 clients: + + - If an entity tag has been provided by the origin server, MUST + use that entity tag in any cache-conditional request (using If- + Match or If-None-Match). + + - If only a Last-Modified value has been provided by the origin + server, SHOULD use that value in non-subrange cache-conditional + requests (using If-Modified-Since). + + + +Fielding, et al. Standards Track [Page 89] + +RFC 2616 HTTP/1.1 June 1999 + + + - If only a Last-Modified value has been provided by an HTTP/1.0 + origin server, MAY use that value in subrange cache-conditional + requests (using If-Unmodified-Since:). The user agent SHOULD + provide a way to disable this, in case of difficulty. + + - If both an entity tag and a Last-Modified value have been + provided by the origin server, SHOULD use both validators in + cache-conditional requests. This allows both HTTP/1.0 and + HTTP/1.1 caches to respond appropriately. + + An HTTP/1.1 origin server, upon receiving a conditional request that + includes both a Last-Modified date (e.g., in an If-Modified-Since or + If-Unmodified-Since header field) and one or more entity tags (e.g., + in an If-Match, If-None-Match, or If-Range header field) as cache + validators, MUST NOT return a response status of 304 (Not Modified) + unless doing so is consistent with all of the conditional header + fields in the request. + + An HTTP/1.1 caching proxy, upon receiving a conditional request that + includes both a Last-Modified date and one or more entity tags as + cache validators, MUST NOT return a locally cached response to the + client unless that cached response is consistent with all of the + conditional header fields in the request. + + Note: The general principle behind these rules is that HTTP/1.1 + servers and clients should transmit as much non-redundant + information as is available in their responses and requests. + HTTP/1.1 systems receiving this information will make the most + conservative assumptions about the validators they receive. + + HTTP/1.0 clients and caches will ignore entity tags. Generally, + last-modified values received or used by these systems will + support transparent and efficient caching, and so HTTP/1.1 origin + servers should provide Last-Modified values. In those rare cases + where the use of a Last-Modified value as a validator by an + HTTP/1.0 system could result in a serious problem, then HTTP/1.1 + origin servers should not provide one. + +13.3.5 Non-validating Conditionals + + The principle behind entity tags is that only the service author + knows the semantics of a resource well enough to select an + appropriate cache validation mechanism, and the specification of any + validator comparison function more complex than byte-equality would + open up a can of worms. Thus, comparisons of any other headers + (except Last-Modified, for compatibility with HTTP/1.0) are never + used for purposes of validating a cache entry. + + + + +Fielding, et al. Standards Track [Page 90] + +RFC 2616 HTTP/1.1 June 1999 + + +13.4 Response Cacheability + + Unless specifically constrained by a cache-control (section 14.9) + directive, a caching system MAY always store a successful response + (see section 13.8) as a cache entry, MAY return it without validation + if it is fresh, and MAY return it after successful validation. If + there is neither a cache validator nor an explicit expiration time + associated with a response, we do not expect it to be cached, but + certain caches MAY violate this expectation (for example, when little + or no network connectivity is available). A client can usually detect + that such a response was taken from a cache by comparing the Date + header to the current time. + + Note: some HTTP/1.0 caches are known to violate this expectation + without providing any Warning. + + However, in some cases it might be inappropriate for a cache to + retain an entity, or to return it in response to a subsequent + request. This might be because absolute semantic transparency is + deemed necessary by the service author, or because of security or + privacy considerations. Certain cache-control directives are + therefore provided so that the server can indicate that certain + resource entities, or portions thereof, are not to be cached + regardless of other considerations. + + Note that section 14.8 normally prevents a shared cache from saving + and returning a response to a previous request if that request + included an Authorization header. + + A response received with a status code of 200, 203, 206, 300, 301 or + 410 MAY be stored by a cache and used in reply to a subsequent + request, subject to the expiration mechanism, unless a cache-control + directive prohibits caching. However, a cache that does not support + the Range and Content-Range headers MUST NOT cache 206 (Partial + Content) responses. + + A response received with any other status code (e.g. status codes 302 + and 307) MUST NOT be returned in a reply to a subsequent request + unless there are cache-control directives or another header(s) that + explicitly allow it. For example, these include the following: an + Expires header (section 14.21); a "max-age", "s-maxage", "must- + revalidate", "proxy-revalidate", "public" or "private" cache-control + directive (section 14.9). + + + + + + + + +Fielding, et al. Standards Track [Page 91] + +RFC 2616 HTTP/1.1 June 1999 + + +13.5 Constructing Responses From Caches + + The purpose of an HTTP cache is to store information received in + response to requests for use in responding to future requests. In + many cases, a cache simply returns the appropriate parts of a + response to the requester. However, if the cache holds a cache entry + based on a previous response, it might have to combine parts of a new + response with what is held in the cache entry. + +13.5.1 End-to-end and Hop-by-hop Headers + + For the purpose of defining the behavior of caches and non-caching + proxies, we divide HTTP headers into two categories: + + - End-to-end headers, which are transmitted to the ultimate + recipient of a request or response. End-to-end headers in + responses MUST be stored as part of a cache entry and MUST be + transmitted in any response formed from a cache entry. + + - Hop-by-hop headers, which are meaningful only for a single + transport-level connection, and are not stored by caches or + forwarded by proxies. + + The following HTTP/1.1 headers are hop-by-hop headers: + + - Connection + - Keep-Alive + - Proxy-Authenticate + - Proxy-Authorization + - TE + - Trailers + - Transfer-Encoding + - Upgrade + + All other headers defined by HTTP/1.1 are end-to-end headers. + + Other hop-by-hop headers MUST be listed in a Connection header, + (section 14.10) to be introduced into HTTP/1.1 (or later). + +13.5.2 Non-modifiable Headers + + Some features of the HTTP/1.1 protocol, such as Digest + Authentication, depend on the value of certain end-to-end headers. A + transparent proxy SHOULD NOT modify an end-to-end header unless the + definition of that header requires or specifically allows that. + + + + + + +Fielding, et al. Standards Track [Page 92] + +RFC 2616 HTTP/1.1 June 1999 + + + A transparent proxy MUST NOT modify any of the following fields in a + request or response, and it MUST NOT add any of these fields if not + already present: + + - Content-Location + + - Content-MD5 + + - ETag + + - Last-Modified + + A transparent proxy MUST NOT modify any of the following fields in a + response: + + - Expires + + but it MAY add any of these fields if not already present. If an + Expires header is added, it MUST be given a field-value identical to + that of the Date header in that response. + + A proxy MUST NOT modify or add any of the following fields in a + message that contains the no-transform cache-control directive, or in + any request: + + - Content-Encoding + + - Content-Range + + - Content-Type + + A non-transparent proxy MAY modify or add these fields to a message + that does not include no-transform, but if it does so, it MUST add a + Warning 214 (Transformation applied) if one does not already appear + in the message (see section 14.46). + + Warning: unnecessary modification of end-to-end headers might + cause authentication failures if stronger authentication + mechanisms are introduced in later versions of HTTP. Such + authentication mechanisms MAY rely on the values of header fields + not listed here. + + The Content-Length field of a request or response is added or deleted + according to the rules in section 4.4. A transparent proxy MUST + preserve the entity-length (section 7.2.2) of the entity-body, + although it MAY change the transfer-length (section 4.4). + + + + + +Fielding, et al. Standards Track [Page 93] + +RFC 2616 HTTP/1.1 June 1999 + + +13.5.3 Combining Headers + + When a cache makes a validating request to a server, and the server + provides a 304 (Not Modified) response or a 206 (Partial Content) + response, the cache then constructs a response to send to the + requesting client. + + If the status code is 304 (Not Modified), the cache uses the entity- + body stored in the cache entry as the entity-body of this outgoing + response. If the status code is 206 (Partial Content) and the ETag or + Last-Modified headers match exactly, the cache MAY combine the + contents stored in the cache entry with the new contents received in + the response and use the result as the entity-body of this outgoing + response, (see 13.5.4). + + The end-to-end headers stored in the cache entry are used for the + constructed response, except that + + - any stored Warning headers with warn-code 1xx (see section + 14.46) MUST be deleted from the cache entry and the forwarded + response. + + - any stored Warning headers with warn-code 2xx MUST be retained + in the cache entry and the forwarded response. + + - any end-to-end headers provided in the 304 or 206 response MUST + replace the corresponding headers from the cache entry. + + Unless the cache decides to remove the cache entry, it MUST also + replace the end-to-end headers stored with the cache entry with + corresponding headers received in the incoming response, except for + Warning headers as described immediately above. If a header field- + name in the incoming response matches more than one header in the + cache entry, all such old headers MUST be replaced. + + In other words, the set of end-to-end headers received in the + incoming response overrides all corresponding end-to-end headers + stored with the cache entry (except for stored Warning headers with + warn-code 1xx, which are deleted even if not overridden). + + Note: this rule allows an origin server to use a 304 (Not + Modified) or a 206 (Partial Content) response to update any header + associated with a previous response for the same entity or sub- + ranges thereof, although it might not always be meaningful or + correct to do so. This rule does not allow an origin server to use + a 304 (Not Modified) or a 206 (Partial Content) response to + entirely delete a header that it had provided with a previous + response. + + + +Fielding, et al. Standards Track [Page 94] + +RFC 2616 HTTP/1.1 June 1999 + + +13.5.4 Combining Byte Ranges + + A response might transfer only a subrange of the bytes of an entity- + body, either because the request included one or more Range + specifications, or because a connection was broken prematurely. After + several such transfers, a cache might have received several ranges of + the same entity-body. + + If a cache has a stored non-empty set of subranges for an entity, and + an incoming response transfers another subrange, the cache MAY + combine the new subrange with the existing set if both the following + conditions are met: + + - Both the incoming response and the cache entry have a cache + validator. + + - The two cache validators match using the strong comparison + function (see section 13.3.3). + + If either requirement is not met, the cache MUST use only the most + recent partial response (based on the Date values transmitted with + every response, and using the incoming response if these values are + equal or missing), and MUST discard the other partial information. + +13.6 Caching Negotiated Responses + + Use of server-driven content negotiation (section 12.1), as indicated + by the presence of a Vary header field in a response, alters the + conditions and procedure by which a cache can use the response for + subsequent requests. See section 14.44 for use of the Vary header + field by servers. + + A server SHOULD use the Vary header field to inform a cache of what + request-header fields were used to select among multiple + representations of a cacheable response subject to server-driven + negotiation. The set of header fields named by the Vary field value + is known as the "selecting" request-headers. + + When the cache receives a subsequent request whose Request-URI + specifies one or more cache entries including a Vary header field, + the cache MUST NOT use such a cache entry to construct a response to + the new request unless all of the selecting request-headers present + in the new request match the corresponding stored request-headers in + the original request. + + The selecting request-headers from two requests are defined to match + if and only if the selecting request-headers in the first request can + be transformed to the selecting request-headers in the second request + + + +Fielding, et al. Standards Track [Page 95] + +RFC 2616 HTTP/1.1 June 1999 + + + by adding or removing linear white space (LWS) at places where this + is allowed by the corresponding BNF, and/or combining multiple + message-header fields with the same field name following the rules + about message headers in section 4.2. + + A Vary header field-value of "*" always fails to match and subsequent + requests on that resource can only be properly interpreted by the + origin server. + + If the selecting request header fields for the cached entry do not + match the selecting request header fields of the new request, then + the cache MUST NOT use a cached entry to satisfy the request unless + it first relays the new request to the origin server in a conditional + request and the server responds with 304 (Not Modified), including an + entity tag or Content-Location that indicates the entity to be used. + + If an entity tag was assigned to a cached representation, the + forwarded request SHOULD be conditional and include the entity tags + in an If-None-Match header field from all its cache entries for the + resource. This conveys to the server the set of entities currently + held by the cache, so that if any one of these entities matches the + requested entity, the server can use the ETag header field in its 304 + (Not Modified) response to tell the cache which entry is appropriate. + If the entity-tag of the new response matches that of an existing + entry, the new response SHOULD be used to update the header fields of + the existing entry, and the result MUST be returned to the client. + + If any of the existing cache entries contains only partial content + for the associated entity, its entity-tag SHOULD NOT be included in + the If-None-Match header field unless the request is for a range that + would be fully satisfied by that entry. + + If a cache receives a successful response whose Content-Location + field matches that of an existing cache entry for the same Request- + ]URI, whose entity-tag differs from that of the existing entry, and + whose Date is more recent than that of the existing entry, the + existing entry SHOULD NOT be returned in response to future requests + and SHOULD be deleted from the cache. + +13.7 Shared and Non-Shared Caches + + For reasons of security and privacy, it is necessary to make a + distinction between "shared" and "non-shared" caches. A non-shared + cache is one that is accessible only to a single user. Accessibility + in this case SHOULD be enforced by appropriate security mechanisms. + All other caches are considered to be "shared." Other sections of + + + + + +Fielding, et al. Standards Track [Page 96] + +RFC 2616 HTTP/1.1 June 1999 + + + this specification place certain constraints on the operation of + shared caches in order to prevent loss of privacy or failure of + access controls. + +13.8 Errors or Incomplete Response Cache Behavior + + A cache that receives an incomplete response (for example, with fewer + bytes of data than specified in a Content-Length header) MAY store + the response. However, the cache MUST treat this as a partial + response. Partial responses MAY be combined as described in section + 13.5.4; the result might be a full response or might still be + partial. A cache MUST NOT return a partial response to a client + without explicitly marking it as such, using the 206 (Partial + Content) status code. A cache MUST NOT return a partial response + using a status code of 200 (OK). + + If a cache receives a 5xx response while attempting to revalidate an + entry, it MAY either forward this response to the requesting client, + or act as if the server failed to respond. In the latter case, it MAY + return a previously received response unless the cached entry + includes the "must-revalidate" cache-control directive (see section + 14.9). + +13.9 Side Effects of GET and HEAD + + Unless the origin server explicitly prohibits the caching of their + responses, the application of GET and HEAD methods to any resources + SHOULD NOT have side effects that would lead to erroneous behavior if + these responses are taken from a cache. They MAY still have side + effects, but a cache is not required to consider such side effects in + its caching decisions. Caches are always expected to observe an + origin server's explicit restrictions on caching. + + We note one exception to this rule: since some applications have + traditionally used GETs and HEADs with query URLs (those containing a + "?" in the rel_path part) to perform operations with significant side + effects, caches MUST NOT treat responses to such URIs as fresh unless + the server provides an explicit expiration time. This specifically + means that responses from HTTP/1.0 servers for such URIs SHOULD NOT + be taken from a cache. See section 9.1.1 for related information. + +13.10 Invalidation After Updates or Deletions + + The effect of certain methods performed on a resource at the origin + server might cause one or more existing cache entries to become non- + transparently invalid. That is, although they might continue to be + "fresh," they do not accurately reflect what the origin server would + return for a new request on that resource. + + + +Fielding, et al. Standards Track [Page 97] + +RFC 2616 HTTP/1.1 June 1999 + + + There is no way for the HTTP protocol to guarantee that all such + cache entries are marked invalid. For example, the request that + caused the change at the origin server might not have gone through + the proxy where a cache entry is stored. However, several rules help + reduce the likelihood of erroneous behavior. + + In this section, the phrase "invalidate an entity" means that the + cache will either remove all instances of that entity from its + storage, or will mark these as "invalid" and in need of a mandatory + revalidation before they can be returned in response to a subsequent + request. + + Some HTTP methods MUST cause a cache to invalidate an entity. This is + either the entity referred to by the Request-URI, or by the Location + or Content-Location headers (if present). These methods are: + + - PUT + + - DELETE + + - POST + + In order to prevent denial of service attacks, an invalidation based + on the URI in a Location or Content-Location header MUST only be + performed if the host part is the same as in the Request-URI. + + A cache that passes through requests for methods it does not + understand SHOULD invalidate any entities referred to by the + Request-URI. + +13.11 Write-Through Mandatory + + All methods that might be expected to cause modifications to the + origin server's resources MUST be written through to the origin + server. This currently includes all methods except for GET and HEAD. + A cache MUST NOT reply to such a request from a client before having + transmitted the request to the inbound server, and having received a + corresponding response from the inbound server. This does not prevent + a proxy cache from sending a 100 (Continue) response before the + inbound server has sent its final reply. + + The alternative (known as "write-back" or "copy-back" caching) is not + allowed in HTTP/1.1, due to the difficulty of providing consistent + updates and the problems arising from server, cache, or network + failure prior to write-back. + + + + + + +Fielding, et al. Standards Track [Page 98] + +RFC 2616 HTTP/1.1 June 1999 + + +13.12 Cache Replacement + + If a new cacheable (see sections 14.9.2, 13.2.5, 13.2.6 and 13.8) + response is received from a resource while any existing responses for + the same resource are cached, the cache SHOULD use the new response + to reply to the current request. It MAY insert it into cache storage + and MAY, if it meets all other requirements, use it to respond to any + future requests that would previously have caused the old response to + be returned. If it inserts the new response into cache storage the + rules in section 13.5.3 apply. + + Note: a new response that has an older Date header value than + existing cached responses is not cacheable. + +13.13 History Lists + + User agents often have history mechanisms, such as "Back" buttons and + history lists, which can be used to redisplay an entity retrieved + earlier in a session. + + History mechanisms and caches are different. In particular history + mechanisms SHOULD NOT try to show a semantically transparent view of + the current state of a resource. Rather, a history mechanism is meant + to show exactly what the user saw at the time when the resource was + retrieved. + + By default, an expiration time does not apply to history mechanisms. + If the entity is still in storage, a history mechanism SHOULD display + it even if the entity has expired, unless the user has specifically + configured the agent to refresh expired history documents. + + This is not to be construed to prohibit the history mechanism from + telling the user that a view might be stale. + + Note: if history list mechanisms unnecessarily prevent users from + viewing stale resources, this will tend to force service authors + to avoid using HTTP expiration controls and cache controls when + they would otherwise like to. Service authors may consider it + important that users not be presented with error messages or + warning messages when they use navigation controls (such as BACK) + to view previously fetched resources. Even though sometimes such + resources ought not to cached, or ought to expire quickly, user + interface considerations may force service authors to resort to + other means of preventing caching (e.g. "once-only" URLs) in order + not to suffer the effects of improperly functioning history + mechanisms. + + + + + +Fielding, et al. Standards Track [Page 99] + +RFC 2616 HTTP/1.1 June 1999 + + +14 Header Field Definitions + + This section defines the syntax and semantics of all standard + HTTP/1.1 header fields. For entity-header fields, both sender and + recipient refer to either the client or the server, depending on who + sends and who receives the entity. + +14.1 Accept + + The Accept request-header field can be used to specify certain media + types which are acceptable for the response. Accept headers can be + used to indicate that the request is specifically limited to a small + set of desired types, as in the case of a request for an in-line + image. + + Accept = "Accept" ":" + #( media-range [ accept-params ] ) + + media-range = ( "*/*" + | ( type "/" "*" ) + | ( type "/" subtype ) + ) *( ";" parameter ) + accept-params = ";" "q" "=" qvalue *( accept-extension ) + accept-extension = ";" token [ "=" ( token | quoted-string ) ] + + The asterisk "*" character is used to group media types into ranges, + with "*/*" indicating all media types and "type/*" indicating all + subtypes of that type. The media-range MAY include media type + parameters that are applicable to that range. + + Each media-range MAY be followed by one or more accept-params, + beginning with the "q" parameter for indicating a relative quality + factor. The first "q" parameter (if any) separates the media-range + parameter(s) from the accept-params. Quality factors allow the user + or user agent to indicate the relative degree of preference for that + media-range, using the qvalue scale from 0 to 1 (section 3.9). The + default value is q=1. + + Note: Use of the "q" parameter name to separate media type + parameters from Accept extension parameters is due to historical + practice. Although this prevents any media type parameter named + "q" from being used with a media range, such an event is believed + to be unlikely given the lack of any "q" parameters in the IANA + media type registry and the rare usage of any media type + parameters in Accept. Future media types are discouraged from + registering any parameter named "q". + + + + + +Fielding, et al. Standards Track [Page 100] + +RFC 2616 HTTP/1.1 June 1999 + + + The example + + Accept: audio/*; q=0.2, audio/basic + + SHOULD be interpreted as "I prefer audio/basic, but send me any audio + type if it is the best available after an 80% mark-down in quality." + + If no Accept header field is present, then it is assumed that the + client accepts all media types. If an Accept header field is present, + and if the server cannot send a response which is acceptable + according to the combined Accept field value, then the server SHOULD + send a 406 (not acceptable) response. + + A more elaborate example is + + Accept: text/plain; q=0.5, text/html, + text/x-dvi; q=0.8, text/x-c + + Verbally, this would be interpreted as "text/html and text/x-c are + the preferred media types, but if they do not exist, then send the + text/x-dvi entity, and if that does not exist, send the text/plain + entity." + + Media ranges can be overridden by more specific media ranges or + specific media types. If more than one media range applies to a given + type, the most specific reference has precedence. For example, + + Accept: text/*, text/html, text/html;level=1, */* + + have the following precedence: + + 1) text/html;level=1 + 2) text/html + 3) text/* + 4) */* + + The media type quality factor associated with a given type is + determined by finding the media range with the highest precedence + which matches that type. For example, + + Accept: text/*;q=0.3, text/html;q=0.7, text/html;level=1, + text/html;level=2;q=0.4, */*;q=0.5 + + would cause the following values to be associated: + + text/html;level=1 = 1 + text/html = 0.7 + text/plain = 0.3 + + + +Fielding, et al. Standards Track [Page 101] + +RFC 2616 HTTP/1.1 June 1999 + + + image/jpeg = 0.5 + text/html;level=2 = 0.4 + text/html;level=3 = 0.7 + + Note: A user agent might be provided with a default set of quality + values for certain media ranges. However, unless the user agent is + a closed system which cannot interact with other rendering agents, + this default set ought to be configurable by the user. + +14.2 Accept-Charset + + The Accept-Charset request-header field can be used to indicate what + character sets are acceptable for the response. This field allows + clients capable of understanding more comprehensive or special- + purpose character sets to signal that capability to a server which is + capable of representing documents in those character sets. + + Accept-Charset = "Accept-Charset" ":" + 1#( ( charset | "*" )[ ";" "q" "=" qvalue ] ) + + + Character set values are described in section 3.4. Each charset MAY + be given an associated quality value which represents the user's + preference for that charset. The default value is q=1. An example is + + Accept-Charset: iso-8859-5, unicode-1-1;q=0.8 + + The special value "*", if present in the Accept-Charset field, + matches every character set (including ISO-8859-1) which is not + mentioned elsewhere in the Accept-Charset field. If no "*" is present + in an Accept-Charset field, then all character sets not explicitly + mentioned get a quality value of 0, except for ISO-8859-1, which gets + a quality value of 1 if not explicitly mentioned. + + If no Accept-Charset header is present, the default is that any + character set is acceptable. If an Accept-Charset header is present, + and if the server cannot send a response which is acceptable + according to the Accept-Charset header, then the server SHOULD send + an error response with the 406 (not acceptable) status code, though + the sending of an unacceptable response is also allowed. + +14.3 Accept-Encoding + + The Accept-Encoding request-header field is similar to Accept, but + restricts the content-codings (section 3.5) that are acceptable in + the response. + + Accept-Encoding = "Accept-Encoding" ":" + + + +Fielding, et al. Standards Track [Page 102] + +RFC 2616 HTTP/1.1 June 1999 + + + 1#( codings [ ";" "q" "=" qvalue ] ) + codings = ( content-coding | "*" ) + + Examples of its use are: + + Accept-Encoding: compress, gzip + Accept-Encoding: + Accept-Encoding: * + Accept-Encoding: compress;q=0.5, gzip;q=1.0 + Accept-Encoding: gzip;q=1.0, identity; q=0.5, *;q=0 + + A server tests whether a content-coding is acceptable, according to + an Accept-Encoding field, using these rules: + + 1. If the content-coding is one of the content-codings listed in + the Accept-Encoding field, then it is acceptable, unless it is + accompanied by a qvalue of 0. (As defined in section 3.9, a + qvalue of 0 means "not acceptable.") + + 2. The special "*" symbol in an Accept-Encoding field matches any + available content-coding not explicitly listed in the header + field. + + 3. If multiple content-codings are acceptable, then the acceptable + content-coding with the highest non-zero qvalue is preferred. + + 4. The "identity" content-coding is always acceptable, unless + specifically refused because the Accept-Encoding field includes + "identity;q=0", or because the field includes "*;q=0" and does + not explicitly include the "identity" content-coding. If the + Accept-Encoding field-value is empty, then only the "identity" + encoding is acceptable. + + If an Accept-Encoding field is present in a request, and if the + server cannot send a response which is acceptable according to the + Accept-Encoding header, then the server SHOULD send an error response + with the 406 (Not Acceptable) status code. + + If no Accept-Encoding field is present in a request, the server MAY + assume that the client will accept any content coding. In this case, + if "identity" is one of the available content-codings, then the + server SHOULD use the "identity" content-coding, unless it has + additional information that a different content-coding is meaningful + to the client. + + Note: If the request does not include an Accept-Encoding field, + and if the "identity" content-coding is unavailable, then + content-codings commonly understood by HTTP/1.0 clients (i.e., + + + +Fielding, et al. Standards Track [Page 103] + +RFC 2616 HTTP/1.1 June 1999 + + + "gzip" and "compress") are preferred; some older clients + improperly display messages sent with other content-codings. The + server might also make this decision based on information about + the particular user-agent or client. + + Note: Most HTTP/1.0 applications do not recognize or obey qvalues + associated with content-codings. This means that qvalues will not + work and are not permitted with x-gzip or x-compress. + +14.4 Accept-Language + + The Accept-Language request-header field is similar to Accept, but + restricts the set of natural languages that are preferred as a + response to the request. Language tags are defined in section 3.10. + + Accept-Language = "Accept-Language" ":" + 1#( language-range [ ";" "q" "=" qvalue ] ) + language-range = ( ( 1*8ALPHA *( "-" 1*8ALPHA ) ) | "*" ) + + Each language-range MAY be given an associated quality value which + represents an estimate of the user's preference for the languages + specified by that range. The quality value defaults to "q=1". For + example, + + Accept-Language: da, en-gb;q=0.8, en;q=0.7 + + would mean: "I prefer Danish, but will accept British English and + other types of English." A language-range matches a language-tag if + it exactly equals the tag, or if it exactly equals a prefix of the + tag such that the first tag character following the prefix is "-". + The special range "*", if present in the Accept-Language field, + matches every tag not matched by any other range present in the + Accept-Language field. + + Note: This use of a prefix matching rule does not imply that + language tags are assigned to languages in such a way that it is + always true that if a user understands a language with a certain + tag, then this user will also understand all languages with tags + for which this tag is a prefix. The prefix rule simply allows the + use of prefix tags if this is the case. + + The language quality factor assigned to a language-tag by the + Accept-Language field is the quality value of the longest language- + range in the field that matches the language-tag. If no language- + range in the field matches the tag, the language quality factor + assigned is 0. If no Accept-Language header is present in the + request, the server + + + + +Fielding, et al. Standards Track [Page 104] + +RFC 2616 HTTP/1.1 June 1999 + + + SHOULD assume that all languages are equally acceptable. If an + Accept-Language header is present, then all languages which are + assigned a quality factor greater than 0 are acceptable. + + It might be contrary to the privacy expectations of the user to send + an Accept-Language header with the complete linguistic preferences of + the user in every request. For a discussion of this issue, see + section 15.1.4. + + As intelligibility is highly dependent on the individual user, it is + recommended that client applications make the choice of linguistic + preference available to the user. If the choice is not made + available, then the Accept-Language header field MUST NOT be given in + the request. + + Note: When making the choice of linguistic preference available to + the user, we remind implementors of the fact that users are not + familiar with the details of language matching as described above, + and should provide appropriate guidance. As an example, users + might assume that on selecting "en-gb", they will be served any + kind of English document if British English is not available. A + user agent might suggest in such a case to add "en" to get the + best matching behavior. + +14.5 Accept-Ranges + + The Accept-Ranges response-header field allows the server to + indicate its acceptance of range requests for a resource: + + Accept-Ranges = "Accept-Ranges" ":" acceptable-ranges + acceptable-ranges = 1#range-unit | "none" + + Origin servers that accept byte-range requests MAY send + + Accept-Ranges: bytes + + but are not required to do so. Clients MAY generate byte-range + requests without having received this header for the resource + involved. Range units are defined in section 3.12. + + Servers that do not accept any kind of range request for a + resource MAY send + + Accept-Ranges: none + + to advise the client not to attempt a range request. + + + + + +Fielding, et al. Standards Track [Page 105] + +RFC 2616 HTTP/1.1 June 1999 + + +14.6 Age + + The Age response-header field conveys the sender's estimate of the + amount of time since the response (or its revalidation) was + generated at the origin server. A cached response is "fresh" if + its age does not exceed its freshness lifetime. Age values are + calculated as specified in section 13.2.3. + + Age = "Age" ":" age-value + age-value = delta-seconds + + Age values are non-negative decimal integers, representing time in + seconds. + + If a cache receives a value larger than the largest positive + integer it can represent, or if any of its age calculations + overflows, it MUST transmit an Age header with a value of + 2147483648 (2^31). An HTTP/1.1 server that includes a cache MUST + include an Age header field in every response generated from its + own cache. Caches SHOULD use an arithmetic type of at least 31 + bits of range. + +14.7 Allow + + The Allow entity-header field lists the set of methods supported + by the resource identified by the Request-URI. The purpose of this + field is strictly to inform the recipient of valid methods + associated with the resource. An Allow header field MUST be + present in a 405 (Method Not Allowed) response. + + Allow = "Allow" ":" #Method + + Example of use: + + Allow: GET, HEAD, PUT + + This field cannot prevent a client from trying other methods. + However, the indications given by the Allow header field value + SHOULD be followed. The actual set of allowed methods is defined + by the origin server at the time of each request. + + The Allow header field MAY be provided with a PUT request to + recommend the methods to be supported by the new or modified + resource. The server is not required to support these methods and + SHOULD include an Allow header in the response giving the actual + supported methods. + + + + + +Fielding, et al. Standards Track [Page 106] + +RFC 2616 HTTP/1.1 June 1999 + + + A proxy MUST NOT modify the Allow header field even if it does not + understand all the methods specified, since the user agent might + have other means of communicating with the origin server. + +14.8 Authorization + + A user agent that wishes to authenticate itself with a server-- + usually, but not necessarily, after receiving a 401 response--does + so by including an Authorization request-header field with the + request. The Authorization field value consists of credentials + containing the authentication information of the user agent for + the realm of the resource being requested. + + Authorization = "Authorization" ":" credentials + + HTTP access authentication is described in "HTTP Authentication: + Basic and Digest Access Authentication" [43]. If a request is + authenticated and a realm specified, the same credentials SHOULD + be valid for all other requests within this realm (assuming that + the authentication scheme itself does not require otherwise, such + as credentials that vary according to a challenge value or using + synchronized clocks). + + When a shared cache (see section 13.7) receives a request + containing an Authorization field, it MUST NOT return the + corresponding response as a reply to any other request, unless one + of the following specific exceptions holds: + + 1. If the response includes the "s-maxage" cache-control + directive, the cache MAY use that response in replying to a + subsequent request. But (if the specified maximum age has + passed) a proxy cache MUST first revalidate it with the origin + server, using the request-headers from the new request to allow + the origin server to authenticate the new request. (This is the + defined behavior for s-maxage.) If the response includes "s- + maxage=0", the proxy MUST always revalidate it before re-using + it. + + 2. If the response includes the "must-revalidate" cache-control + directive, the cache MAY use that response in replying to a + subsequent request. But if the response is stale, all caches + MUST first revalidate it with the origin server, using the + request-headers from the new request to allow the origin server + to authenticate the new request. + + 3. If the response includes the "public" cache-control directive, + it MAY be returned in reply to any subsequent request. + + + + +Fielding, et al. Standards Track [Page 107] + +RFC 2616 HTTP/1.1 June 1999 + + +14.9 Cache-Control + + The Cache-Control general-header field is used to specify directives + that MUST be obeyed by all caching mechanisms along the + request/response chain. The directives specify behavior intended to + prevent caches from adversely interfering with the request or + response. These directives typically override the default caching + algorithms. Cache directives are unidirectional in that the presence + of a directive in a request does not imply that the same directive is + to be given in the response. + + Note that HTTP/1.0 caches might not implement Cache-Control and + might only implement Pragma: no-cache (see section 14.32). + + Cache directives MUST be passed through by a proxy or gateway + application, regardless of their significance to that application, + since the directives might be applicable to all recipients along the + request/response chain. It is not possible to specify a cache- + directive for a specific cache. + + Cache-Control = "Cache-Control" ":" 1#cache-directive + + cache-directive = cache-request-directive + | cache-response-directive + + cache-request-directive = + "no-cache" ; Section 14.9.1 + | "no-store" ; Section 14.9.2 + | "max-age" "=" delta-seconds ; Section 14.9.3, 14.9.4 + | "max-stale" [ "=" delta-seconds ] ; Section 14.9.3 + | "min-fresh" "=" delta-seconds ; Section 14.9.3 + | "no-transform" ; Section 14.9.5 + | "only-if-cached" ; Section 14.9.4 + | cache-extension ; Section 14.9.6 + + cache-response-directive = + "public" ; Section 14.9.1 + | "private" [ "=" <"> 1#field-name <"> ] ; Section 14.9.1 + | "no-cache" [ "=" <"> 1#field-name <"> ]; Section 14.9.1 + | "no-store" ; Section 14.9.2 + | "no-transform" ; Section 14.9.5 + | "must-revalidate" ; Section 14.9.4 + | "proxy-revalidate" ; Section 14.9.4 + | "max-age" "=" delta-seconds ; Section 14.9.3 + | "s-maxage" "=" delta-seconds ; Section 14.9.3 + | cache-extension ; Section 14.9.6 + + cache-extension = token [ "=" ( token | quoted-string ) ] + + + +Fielding, et al. Standards Track [Page 108] + +RFC 2616 HTTP/1.1 June 1999 + + + When a directive appears without any 1#field-name parameter, the + directive applies to the entire request or response. When such a + directive appears with a 1#field-name parameter, it applies only to + the named field or fields, and not to the rest of the request or + response. This mechanism supports extensibility; implementations of + future versions of the HTTP protocol might apply these directives to + header fields not defined in HTTP/1.1. + + The cache-control directives can be broken down into these general + categories: + + - Restrictions on what are cacheable; these may only be imposed by + the origin server. + + - Restrictions on what may be stored by a cache; these may be + imposed by either the origin server or the user agent. + + - Modifications of the basic expiration mechanism; these may be + imposed by either the origin server or the user agent. + + - Controls over cache revalidation and reload; these may only be + imposed by a user agent. + + - Control over transformation of entities. + + - Extensions to the caching system. + +14.9.1 What is Cacheable + + By default, a response is cacheable if the requirements of the + request method, request header fields, and the response status + indicate that it is cacheable. Section 13.4 summarizes these defaults + for cacheability. The following Cache-Control response directives + allow an origin server to override the default cacheability of a + response: + + public + Indicates that the response MAY be cached by any cache, even if it + would normally be non-cacheable or cacheable only within a non- + shared cache. (See also Authorization, section 14.8, for + additional details.) + + private + Indicates that all or part of the response message is intended for + a single user and MUST NOT be cached by a shared cache. This + allows an origin server to state that the specified parts of the + + + + + +Fielding, et al. Standards Track [Page 109] + +RFC 2616 HTTP/1.1 June 1999 + + + response are intended for only one user and are not a valid + response for requests by other users. A private (non-shared) cache + MAY cache the response. + + Note: This usage of the word private only controls where the + response may be cached, and cannot ensure the privacy of the + message content. + + no-cache + If the no-cache directive does not specify a field-name, then a + cache MUST NOT use the response to satisfy a subsequent request + without successful revalidation with the origin server. This + allows an origin server to prevent caching even by caches that + have been configured to return stale responses to client requests. + + If the no-cache directive does specify one or more field-names, + then a cache MAY use the response to satisfy a subsequent request, + subject to any other restrictions on caching. However, the + specified field-name(s) MUST NOT be sent in the response to a + subsequent request without successful revalidation with the origin + server. This allows an origin server to prevent the re-use of + certain header fields in a response, while still allowing caching + of the rest of the response. + + Note: Most HTTP/1.0 caches will not recognize or obey this + directive. + +14.9.2 What May be Stored by Caches + + no-store + The purpose of the no-store directive is to prevent the + inadvertent release or retention of sensitive information (for + example, on backup tapes). The no-store directive applies to the + entire message, and MAY be sent either in a response or in a + request. If sent in a request, a cache MUST NOT store any part of + either this request or any response to it. If sent in a response, + a cache MUST NOT store any part of either this response or the + request that elicited it. This directive applies to both non- + shared and shared caches. "MUST NOT store" in this context means + that the cache MUST NOT intentionally store the information in + non-volatile storage, and MUST make a best-effort attempt to + remove the information from volatile storage as promptly as + possible after forwarding it. + + Even when this directive is associated with a response, users + might explicitly store such a response outside of the caching + system (e.g., with a "Save As" dialog). History buffers MAY store + such responses as part of their normal operation. + + + +Fielding, et al. Standards Track [Page 110] + +RFC 2616 HTTP/1.1 June 1999 + + + The purpose of this directive is to meet the stated requirements + of certain users and service authors who are concerned about + accidental releases of information via unanticipated accesses to + cache data structures. While the use of this directive might + improve privacy in some cases, we caution that it is NOT in any + way a reliable or sufficient mechanism for ensuring privacy. In + particular, malicious or compromised caches might not recognize or + obey this directive, and communications networks might be + vulnerable to eavesdropping. + +14.9.3 Modifications of the Basic Expiration Mechanism + + The expiration time of an entity MAY be specified by the origin + server using the Expires header (see section 14.21). Alternatively, + it MAY be specified using the max-age directive in a response. When + the max-age cache-control directive is present in a cached response, + the response is stale if its current age is greater than the age + value given (in seconds) at the time of a new request for that + resource. The max-age directive on a response implies that the + response is cacheable (i.e., "public") unless some other, more + restrictive cache directive is also present. + + If a response includes both an Expires header and a max-age + directive, the max-age directive overrides the Expires header, even + if the Expires header is more restrictive. This rule allows an origin + server to provide, for a given response, a longer expiration time to + an HTTP/1.1 (or later) cache than to an HTTP/1.0 cache. This might be + useful if certain HTTP/1.0 caches improperly calculate ages or + expiration times, perhaps due to desynchronized clocks. + + Many HTTP/1.0 cache implementations will treat an Expires value that + is less than or equal to the response Date value as being equivalent + to the Cache-Control response directive "no-cache". If an HTTP/1.1 + cache receives such a response, and the response does not include a + Cache-Control header field, it SHOULD consider the response to be + non-cacheable in order to retain compatibility with HTTP/1.0 servers. + + Note: An origin server might wish to use a relatively new HTTP + cache control feature, such as the "private" directive, on a + network including older caches that do not understand that + feature. The origin server will need to combine the new feature + with an Expires field whose value is less than or equal to the + Date value. This will prevent older caches from improperly + caching the response. + + + + + + + +Fielding, et al. Standards Track [Page 111] + +RFC 2616 HTTP/1.1 June 1999 + + + s-maxage + If a response includes an s-maxage directive, then for a shared + cache (but not for a private cache), the maximum age specified by + this directive overrides the maximum age specified by either the + max-age directive or the Expires header. The s-maxage directive + also implies the semantics of the proxy-revalidate directive (see + section 14.9.4), i.e., that the shared cache must not use the + entry after it becomes stale to respond to a subsequent request + without first revalidating it with the origin server. The s- + maxage directive is always ignored by a private cache. + + Note that most older caches, not compliant with this specification, + do not implement any cache-control directives. An origin server + wishing to use a cache-control directive that restricts, but does not + prevent, caching by an HTTP/1.1-compliant cache MAY exploit the + requirement that the max-age directive overrides the Expires header, + and the fact that pre-HTTP/1.1-compliant caches do not observe the + max-age directive. + + Other directives allow a user agent to modify the basic expiration + mechanism. These directives MAY be specified on a request: + + max-age + Indicates that the client is willing to accept a response whose + age is no greater than the specified time in seconds. Unless max- + stale directive is also included, the client is not willing to + accept a stale response. + + min-fresh + Indicates that the client is willing to accept a response whose + freshness lifetime is no less than its current age plus the + specified time in seconds. That is, the client wants a response + that will still be fresh for at least the specified number of + seconds. + + max-stale + Indicates that the client is willing to accept a response that has + exceeded its expiration time. If max-stale is assigned a value, + then the client is willing to accept a response that has exceeded + its expiration time by no more than the specified number of + seconds. If no value is assigned to max-stale, then the client is + willing to accept a stale response of any age. + + If a cache returns a stale response, either because of a max-stale + directive on a request, or because the cache is configured to + override the expiration time of a response, the cache MUST attach a + Warning header to the stale response, using Warning 110 (Response is + stale). + + + +Fielding, et al. Standards Track [Page 112] + +RFC 2616 HTTP/1.1 June 1999 + + + A cache MAY be configured to return stale responses without + validation, but only if this does not conflict with any "MUST"-level + requirements concerning cache validation (e.g., a "must-revalidate" + cache-control directive). + + If both the new request and the cached entry include "max-age" + directives, then the lesser of the two values is used for determining + the freshness of the cached entry for that request. + +14.9.4 Cache Revalidation and Reload Controls + + Sometimes a user agent might want or need to insist that a cache + revalidate its cache entry with the origin server (and not just with + the next cache along the path to the origin server), or to reload its + cache entry from the origin server. End-to-end revalidation might be + necessary if either the cache or the origin server has overestimated + the expiration time of the cached response. End-to-end reload may be + necessary if the cache entry has become corrupted for some reason. + + End-to-end revalidation may be requested either when the client does + not have its own local cached copy, in which case we call it + "unspecified end-to-end revalidation", or when the client does have a + local cached copy, in which case we call it "specific end-to-end + revalidation." + + The client can specify these three kinds of action using Cache- + Control request directives: + + End-to-end reload + The request includes a "no-cache" cache-control directive or, for + compatibility with HTTP/1.0 clients, "Pragma: no-cache". Field + names MUST NOT be included with the no-cache directive in a + request. The server MUST NOT use a cached copy when responding to + such a request. + + Specific end-to-end revalidation + The request includes a "max-age=0" cache-control directive, which + forces each cache along the path to the origin server to + revalidate its own entry, if any, with the next cache or server. + The initial request includes a cache-validating conditional with + the client's current validator. + + Unspecified end-to-end revalidation + The request includes "max-age=0" cache-control directive, which + forces each cache along the path to the origin server to + revalidate its own entry, if any, with the next cache or server. + The initial request does not include a cache-validating + + + + +Fielding, et al. Standards Track [Page 113] + +RFC 2616 HTTP/1.1 June 1999 + + + conditional; the first cache along the path (if any) that holds a + cache entry for this resource includes a cache-validating + conditional with its current validator. + + max-age + When an intermediate cache is forced, by means of a max-age=0 + directive, to revalidate its own cache entry, and the client has + supplied its own validator in the request, the supplied validator + might differ from the validator currently stored with the cache + entry. In this case, the cache MAY use either validator in making + its own request without affecting semantic transparency. + + However, the choice of validator might affect performance. The + best approach is for the intermediate cache to use its own + validator when making its request. If the server replies with 304 + (Not Modified), then the cache can return its now validated copy + to the client with a 200 (OK) response. If the server replies with + a new entity and cache validator, however, the intermediate cache + can compare the returned validator with the one provided in the + client's request, using the strong comparison function. If the + client's validator is equal to the origin server's, then the + intermediate cache simply returns 304 (Not Modified). Otherwise, + it returns the new entity with a 200 (OK) response. + + If a request includes the no-cache directive, it SHOULD NOT + include min-fresh, max-stale, or max-age. + + only-if-cached + In some cases, such as times of extremely poor network + connectivity, a client may want a cache to return only those + responses that it currently has stored, and not to reload or + revalidate with the origin server. To do this, the client may + include the only-if-cached directive in a request. If it receives + this directive, a cache SHOULD either respond using a cached entry + that is consistent with the other constraints of the request, or + respond with a 504 (Gateway Timeout) status. However, if a group + of caches is being operated as a unified system with good internal + connectivity, such a request MAY be forwarded within that group of + caches. + + must-revalidate + Because a cache MAY be configured to ignore a server's specified + expiration time, and because a client request MAY include a max- + stale directive (which has a similar effect), the protocol also + includes a mechanism for the origin server to require revalidation + of a cache entry on any subsequent use. When the must-revalidate + directive is present in a response received by a cache, that cache + MUST NOT use the entry after it becomes stale to respond to a + + + +Fielding, et al. Standards Track [Page 114] + +RFC 2616 HTTP/1.1 June 1999 + + + subsequent request without first revalidating it with the origin + server. (I.e., the cache MUST do an end-to-end revalidation every + time, if, based solely on the origin server's Expires or max-age + value, the cached response is stale.) + + The must-revalidate directive is necessary to support reliable + operation for certain protocol features. In all circumstances an + HTTP/1.1 cache MUST obey the must-revalidate directive; in + particular, if the cache cannot reach the origin server for any + reason, it MUST generate a 504 (Gateway Timeout) response. + + Servers SHOULD send the must-revalidate directive if and only if + failure to revalidate a request on the entity could result in + incorrect operation, such as a silently unexecuted financial + transaction. Recipients MUST NOT take any automated action that + violates this directive, and MUST NOT automatically provide an + unvalidated copy of the entity if revalidation fails. + + Although this is not recommended, user agents operating under + severe connectivity constraints MAY violate this directive but, if + so, MUST explicitly warn the user that an unvalidated response has + been provided. The warning MUST be provided on each unvalidated + access, and SHOULD require explicit user confirmation. + + proxy-revalidate + The proxy-revalidate directive has the same meaning as the must- + revalidate directive, except that it does not apply to non-shared + user agent caches. It can be used on a response to an + authenticated request to permit the user's cache to store and + later return the response without needing to revalidate it (since + it has already been authenticated once by that user), while still + requiring proxies that service many users to revalidate each time + (in order to make sure that each user has been authenticated). + Note that such authenticated responses also need the public cache + control directive in order to allow them to be cached at all. + +14.9.5 No-Transform Directive + + no-transform + Implementors of intermediate caches (proxies) have found it useful + to convert the media type of certain entity bodies. A non- + transparent proxy might, for example, convert between image + formats in order to save cache space or to reduce the amount of + traffic on a slow link. + + Serious operational problems occur, however, when these + transformations are applied to entity bodies intended for certain + kinds of applications. For example, applications for medical + + + +Fielding, et al. Standards Track [Page 115] + +RFC 2616 HTTP/1.1 June 1999 + + + imaging, scientific data analysis and those using end-to-end + authentication, all depend on receiving an entity body that is bit + for bit identical to the original entity-body. + + Therefore, if a message includes the no-transform directive, an + intermediate cache or proxy MUST NOT change those headers that are + listed in section 13.5.2 as being subject to the no-transform + directive. This implies that the cache or proxy MUST NOT change + any aspect of the entity-body that is specified by these headers, + including the value of the entity-body itself. + +14.9.6 Cache Control Extensions + + The Cache-Control header field can be extended through the use of one + or more cache-extension tokens, each with an optional assigned value. + Informational extensions (those which do not require a change in + cache behavior) MAY be added without changing the semantics of other + directives. Behavioral extensions are designed to work by acting as + modifiers to the existing base of cache directives. Both the new + directive and the standard directive are supplied, such that + applications which do not understand the new directive will default + to the behavior specified by the standard directive, and those that + understand the new directive will recognize it as modifying the + requirements associated with the standard directive. In this way, + extensions to the cache-control directives can be made without + requiring changes to the base protocol. + + This extension mechanism depends on an HTTP cache obeying all of the + cache-control directives defined for its native HTTP-version, obeying + certain extensions, and ignoring all directives that it does not + understand. + + For example, consider a hypothetical new response directive called + community which acts as a modifier to the private directive. We + define this new directive to mean that, in addition to any non-shared + cache, any cache which is shared only by members of the community + named within its value may cache the response. An origin server + wishing to allow the UCI community to use an otherwise private + response in their shared cache(s) could do so by including + + Cache-Control: private, community="UCI" + + A cache seeing this header field will act correctly even if the cache + does not understand the community cache-extension, since it will also + see and understand the private directive and thus default to the safe + behavior. + + + + + +Fielding, et al. Standards Track [Page 116] + +RFC 2616 HTTP/1.1 June 1999 + + + Unrecognized cache-directives MUST be ignored; it is assumed that any + cache-directive likely to be unrecognized by an HTTP/1.1 cache will + be combined with standard directives (or the response's default + cacheability) such that the cache behavior will remain minimally + correct even if the cache does not understand the extension(s). + +14.10 Connection + + The Connection general-header field allows the sender to specify + options that are desired for that particular connection and MUST NOT + be communicated by proxies over further connections. + + The Connection header has the following grammar: + + Connection = "Connection" ":" 1#(connection-token) + connection-token = token + + HTTP/1.1 proxies MUST parse the Connection header field before a + message is forwarded and, for each connection-token in this field, + remove any header field(s) from the message with the same name as the + connection-token. Connection options are signaled by the presence of + a connection-token in the Connection header field, not by any + corresponding additional header field(s), since the additional header + field may not be sent if there are no parameters associated with that + connection option. + + Message headers listed in the Connection header MUST NOT include + end-to-end headers, such as Cache-Control. + + HTTP/1.1 defines the "close" connection option for the sender to + signal that the connection will be closed after completion of the + response. For example, + + Connection: close + + in either the request or the response header fields indicates that + the connection SHOULD NOT be considered `persistent' (section 8.1) + after the current request/response is complete. + + HTTP/1.1 applications that do not support persistent connections MUST + include the "close" connection option in every message. + + A system receiving an HTTP/1.0 (or lower-version) message that + includes a Connection header MUST, for each connection-token in this + field, remove and ignore any header field(s) from the message with + the same name as the connection-token. This protects against mistaken + forwarding of such header fields by pre-HTTP/1.1 proxies. See section + 19.6.2. + + + +Fielding, et al. Standards Track [Page 117] + +RFC 2616 HTTP/1.1 June 1999 + + +14.11 Content-Encoding + + The Content-Encoding entity-header field is used as a modifier to the + media-type. When present, its value indicates what additional content + codings have been applied to the entity-body, and thus what decoding + mechanisms must be applied in order to obtain the media-type + referenced by the Content-Type header field. Content-Encoding is + primarily used to allow a document to be compressed without losing + the identity of its underlying media type. + + Content-Encoding = "Content-Encoding" ":" 1#content-coding + + Content codings are defined in section 3.5. An example of its use is + + Content-Encoding: gzip + + The content-coding is a characteristic of the entity identified by + the Request-URI. Typically, the entity-body is stored with this + encoding and is only decoded before rendering or analogous usage. + However, a non-transparent proxy MAY modify the content-coding if the + new coding is known to be acceptable to the recipient, unless the + "no-transform" cache-control directive is present in the message. + + If the content-coding of an entity is not "identity", then the + response MUST include a Content-Encoding entity-header (section + 14.11) that lists the non-identity content-coding(s) used. + + If the content-coding of an entity in a request message is not + acceptable to the origin server, the server SHOULD respond with a + status code of 415 (Unsupported Media Type). + + If multiple encodings have been applied to an entity, the content + codings MUST be listed in the order in which they were applied. + Additional information about the encoding parameters MAY be provided + by other entity-header fields not defined by this specification. + +14.12 Content-Language + + The Content-Language entity-header field describes the natural + language(s) of the intended audience for the enclosed entity. Note + that this might not be equivalent to all the languages used within + the entity-body. + + Content-Language = "Content-Language" ":" 1#language-tag + + + + + + + +Fielding, et al. Standards Track [Page 118] + +RFC 2616 HTTP/1.1 June 1999 + + + Language tags are defined in section 3.10. The primary purpose of + Content-Language is to allow a user to identify and differentiate + entities according to the user's own preferred language. Thus, if the + body content is intended only for a Danish-literate audience, the + appropriate field is + + Content-Language: da + + If no Content-Language is specified, the default is that the content + is intended for all language audiences. This might mean that the + sender does not consider it to be specific to any natural language, + or that the sender does not know for which language it is intended. + + Multiple languages MAY be listed for content that is intended for + multiple audiences. For example, a rendition of the "Treaty of + Waitangi," presented simultaneously in the original Maori and English + versions, would call for + + Content-Language: mi, en + + However, just because multiple languages are present within an entity + does not mean that it is intended for multiple linguistic audiences. + An example would be a beginner's language primer, such as "A First + Lesson in Latin," which is clearly intended to be used by an + English-literate audience. In this case, the Content-Language would + properly only include "en". + + Content-Language MAY be applied to any media type -- it is not + limited to textual documents. + +14.13 Content-Length + + The Content-Length entity-header field indicates the size of the + entity-body, in decimal number of OCTETs, sent to the recipient or, + in the case of the HEAD method, the size of the entity-body that + would have been sent had the request been a GET. + + Content-Length = "Content-Length" ":" 1*DIGIT + + An example is + + Content-Length: 3495 + + Applications SHOULD use this field to indicate the transfer-length of + the message-body, unless this is prohibited by the rules in section + 4.4. + + + + + +Fielding, et al. Standards Track [Page 119] + +RFC 2616 HTTP/1.1 June 1999 + + + Any Content-Length greater than or equal to zero is a valid value. + Section 4.4 describes how to determine the length of a message-body + if a Content-Length is not given. + + Note that the meaning of this field is significantly different from + the corresponding definition in MIME, where it is an optional field + used within the "message/external-body" content-type. In HTTP, it + SHOULD be sent whenever the message's length can be determined prior + to being transferred, unless this is prohibited by the rules in + section 4.4. + +14.14 Content-Location + + The Content-Location entity-header field MAY be used to supply the + resource location for the entity enclosed in the message when that + entity is accessible from a location separate from the requested + resource's URI. A server SHOULD provide a Content-Location for the + variant corresponding to the response entity; especially in the case + where a resource has multiple entities associated with it, and those + entities actually have separate locations by which they might be + individually accessed, the server SHOULD provide a Content-Location + for the particular variant which is returned. + + Content-Location = "Content-Location" ":" + ( absoluteURI | relativeURI ) + + The value of Content-Location also defines the base URI for the + entity. + + The Content-Location value is not a replacement for the original + requested URI; it is only a statement of the location of the resource + corresponding to this particular entity at the time of the request. + Future requests MAY specify the Content-Location URI as the request- + URI if the desire is to identify the source of that particular + entity. + + A cache cannot assume that an entity with a Content-Location + different from the URI used to retrieve it can be used to respond to + later requests on that Content-Location URI. However, the Content- + Location can be used to differentiate between multiple entities + retrieved from a single requested resource, as described in section + 13.6. + + If the Content-Location is a relative URI, the relative URI is + interpreted relative to the Request-URI. + + The meaning of the Content-Location header in PUT or POST requests is + undefined; servers are free to ignore it in those cases. + + + +Fielding, et al. Standards Track [Page 120] + +RFC 2616 HTTP/1.1 June 1999 + + +14.15 Content-MD5 + + The Content-MD5 entity-header field, as defined in RFC 1864 [23], is + an MD5 digest of the entity-body for the purpose of providing an + end-to-end message integrity check (MIC) of the entity-body. (Note: a + MIC is good for detecting accidental modification of the entity-body + in transit, but is not proof against malicious attacks.) + + Content-MD5 = "Content-MD5" ":" md5-digest + md5-digest = <base64 of 128 bit MD5 digest as per RFC 1864> + + The Content-MD5 header field MAY be generated by an origin server or + client to function as an integrity check of the entity-body. Only + origin servers or clients MAY generate the Content-MD5 header field; + proxies and gateways MUST NOT generate it, as this would defeat its + value as an end-to-end integrity check. Any recipient of the entity- + body, including gateways and proxies, MAY check that the digest value + in this header field matches that of the entity-body as received. + + The MD5 digest is computed based on the content of the entity-body, + including any content-coding that has been applied, but not including + any transfer-encoding applied to the message-body. If the message is + received with a transfer-encoding, that encoding MUST be removed + prior to checking the Content-MD5 value against the received entity. + + This has the result that the digest is computed on the octets of the + entity-body exactly as, and in the order that, they would be sent if + no transfer-encoding were being applied. + + HTTP extends RFC 1864 to permit the digest to be computed for MIME + composite media-types (e.g., multipart/* and message/rfc822), but + this does not change how the digest is computed as defined in the + preceding paragraph. + + There are several consequences of this. The entity-body for composite + types MAY contain many body-parts, each with its own MIME and HTTP + headers (including Content-MD5, Content-Transfer-Encoding, and + Content-Encoding headers). If a body-part has a Content-Transfer- + Encoding or Content-Encoding header, it is assumed that the content + of the body-part has had the encoding applied, and the body-part is + included in the Content-MD5 digest as is -- i.e., after the + application. The Transfer-Encoding header field is not allowed within + body-parts. + + Conversion of all line breaks to CRLF MUST NOT be done before + computing or checking the digest: the line break convention used in + the text actually transmitted MUST be left unaltered when computing + the digest. + + + +Fielding, et al. Standards Track [Page 121] + +RFC 2616 HTTP/1.1 June 1999 + + + Note: while the definition of Content-MD5 is exactly the same for + HTTP as in RFC 1864 for MIME entity-bodies, there are several ways + in which the application of Content-MD5 to HTTP entity-bodies + differs from its application to MIME entity-bodies. One is that + HTTP, unlike MIME, does not use Content-Transfer-Encoding, and + does use Transfer-Encoding and Content-Encoding. Another is that + HTTP more frequently uses binary content types than MIME, so it is + worth noting that, in such cases, the byte order used to compute + the digest is the transmission byte order defined for the type. + Lastly, HTTP allows transmission of text types with any of several + line break conventions and not just the canonical form using CRLF. + +14.16 Content-Range + + The Content-Range entity-header is sent with a partial entity-body to + specify where in the full entity-body the partial body should be + applied. Range units are defined in section 3.12. + + Content-Range = "Content-Range" ":" content-range-spec + + content-range-spec = byte-content-range-spec + byte-content-range-spec = bytes-unit SP + byte-range-resp-spec "/" + ( instance-length | "*" ) + + byte-range-resp-spec = (first-byte-pos "-" last-byte-pos) + | "*" + instance-length = 1*DIGIT + + The header SHOULD indicate the total length of the full entity-body, + unless this length is unknown or difficult to determine. The asterisk + "*" character means that the instance-length is unknown at the time + when the response was generated. + + Unlike byte-ranges-specifier values (see section 14.35.1), a byte- + range-resp-spec MUST only specify one range, and MUST contain + absolute byte positions for both the first and last byte of the + range. + + A byte-content-range-spec with a byte-range-resp-spec whose last- + byte-pos value is less than its first-byte-pos value, or whose + instance-length value is less than or equal to its last-byte-pos + value, is invalid. The recipient of an invalid byte-content-range- + spec MUST ignore it and any content transferred along with it. + + A server sending a response with status code 416 (Requested range not + satisfiable) SHOULD include a Content-Range field with a byte-range- + resp-spec of "*". The instance-length specifies the current length of + + + +Fielding, et al. Standards Track [Page 122] + +RFC 2616 HTTP/1.1 June 1999 + + + the selected resource. A response with status code 206 (Partial + Content) MUST NOT include a Content-Range field with a byte-range- + resp-spec of "*". + + Examples of byte-content-range-spec values, assuming that the entity + contains a total of 1234 bytes: + + . The first 500 bytes: + bytes 0-499/1234 + + . The second 500 bytes: + bytes 500-999/1234 + + . All except for the first 500 bytes: + bytes 500-1233/1234 + + . The last 500 bytes: + bytes 734-1233/1234 + + When an HTTP message includes the content of a single range (for + example, a response to a request for a single range, or to a request + for a set of ranges that overlap without any holes), this content is + transmitted with a Content-Range header, and a Content-Length header + showing the number of bytes actually transferred. For example, + + HTTP/1.1 206 Partial content + Date: Wed, 15 Nov 1995 06:25:24 GMT + Last-Modified: Wed, 15 Nov 1995 04:58:08 GMT + Content-Range: bytes 21010-47021/47022 + Content-Length: 26012 + Content-Type: image/gif + + When an HTTP message includes the content of multiple ranges (for + example, a response to a request for multiple non-overlapping + ranges), these are transmitted as a multipart message. The multipart + media type used for this purpose is "multipart/byteranges" as defined + in appendix 19.2. See appendix 19.6.3 for a compatibility issue. + + A response to a request for a single range MUST NOT be sent using the + multipart/byteranges media type. A response to a request for + multiple ranges, whose result is a single range, MAY be sent as a + multipart/byteranges media type with one part. A client that cannot + decode a multipart/byteranges message MUST NOT ask for multiple + byte-ranges in a single request. + + When a client requests multiple byte-ranges in one request, the + server SHOULD return them in the order that they appeared in the + request. + + + +Fielding, et al. Standards Track [Page 123] + +RFC 2616 HTTP/1.1 June 1999 + + + If the server ignores a byte-range-spec because it is syntactically + invalid, the server SHOULD treat the request as if the invalid Range + header field did not exist. (Normally, this means return a 200 + response containing the full entity). + + If the server receives a request (other than one including an If- + Range request-header field) with an unsatisfiable Range request- + header field (that is, all of whose byte-range-spec values have a + first-byte-pos value greater than the current length of the selected + resource), it SHOULD return a response code of 416 (Requested range + not satisfiable) (section 10.4.17). + + Note: clients cannot depend on servers to send a 416 (Requested + range not satisfiable) response instead of a 200 (OK) response for + an unsatisfiable Range request-header, since not all servers + implement this request-header. + +14.17 Content-Type + + The Content-Type entity-header field indicates the media type of the + entity-body sent to the recipient or, in the case of the HEAD method, + the media type that would have been sent had the request been a GET. + + Content-Type = "Content-Type" ":" media-type + + Media types are defined in section 3.7. An example of the field is + + Content-Type: text/html; charset=ISO-8859-4 + + Further discussion of methods for identifying the media type of an + entity is provided in section 7.2.1. + +14.18 Date + + The Date general-header field represents the date and time at which + the message was originated, having the same semantics as orig-date in + RFC 822. The field value is an HTTP-date, as described in section + 3.3.1; it MUST be sent in RFC 1123 [8]-date format. + + Date = "Date" ":" HTTP-date + + An example is + + Date: Tue, 15 Nov 1994 08:12:31 GMT + + Origin servers MUST include a Date header field in all responses, + except in these cases: + + + + +Fielding, et al. Standards Track [Page 124] + +RFC 2616 HTTP/1.1 June 1999 + + + 1. If the response status code is 100 (Continue) or 101 (Switching + Protocols), the response MAY include a Date header field, at + the server's option. + + 2. If the response status code conveys a server error, e.g. 500 + (Internal Server Error) or 503 (Service Unavailable), and it is + inconvenient or impossible to generate a valid Date. + + 3. If the server does not have a clock that can provide a + reasonable approximation of the current time, its responses + MUST NOT include a Date header field. In this case, the rules + in section 14.18.1 MUST be followed. + + A received message that does not have a Date header field MUST be + assigned one by the recipient if the message will be cached by that + recipient or gatewayed via a protocol which requires a Date. An HTTP + implementation without a clock MUST NOT cache responses without + revalidating them on every use. An HTTP cache, especially a shared + cache, SHOULD use a mechanism, such as NTP [28], to synchronize its + clock with a reliable external standard. + + Clients SHOULD only send a Date header field in messages that include + an entity-body, as in the case of the PUT and POST requests, and even + then it is optional. A client without a clock MUST NOT send a Date + header field in a request. + + The HTTP-date sent in a Date header SHOULD NOT represent a date and + time subsequent to the generation of the message. It SHOULD represent + the best available approximation of the date and time of message + generation, unless the implementation has no means of generating a + reasonably accurate date and time. In theory, the date ought to + represent the moment just before the entity is generated. In + practice, the date can be generated at any time during the message + origination without affecting its semantic value. + +14.18.1 Clockless Origin Server Operation + + Some origin server implementations might not have a clock available. + An origin server without a clock MUST NOT assign Expires or Last- + Modified values to a response, unless these values were associated + with the resource by a system or user with a reliable clock. It MAY + assign an Expires value that is known, at or before server + configuration time, to be in the past (this allows "pre-expiration" + of responses without storing separate Expires values for each + resource). + + + + + + +Fielding, et al. Standards Track [Page 125] + +RFC 2616 HTTP/1.1 June 1999 + + +14.19 ETag + + The ETag response-header field provides the current value of the + entity tag for the requested variant. The headers used with entity + tags are described in sections 14.24, 14.26 and 14.44. The entity tag + MAY be used for comparison with other entities from the same resource + (see section 13.3.3). + + ETag = "ETag" ":" entity-tag + + Examples: + + ETag: "xyzzy" + ETag: W/"xyzzy" + ETag: "" + +14.20 Expect + + The Expect request-header field is used to indicate that particular + server behaviors are required by the client. + + Expect = "Expect" ":" 1#expectation + + expectation = "100-continue" | expectation-extension + expectation-extension = token [ "=" ( token | quoted-string ) + *expect-params ] + expect-params = ";" token [ "=" ( token | quoted-string ) ] + + + A server that does not understand or is unable to comply with any of + the expectation values in the Expect field of a request MUST respond + with appropriate error status. The server MUST respond with a 417 + (Expectation Failed) status if any of the expectations cannot be met + or, if there are other problems with the request, some other 4xx + status. + + This header field is defined with extensible syntax to allow for + future extensions. If a server receives a request containing an + Expect field that includes an expectation-extension that it does not + support, it MUST respond with a 417 (Expectation Failed) status. + + Comparison of expectation values is case-insensitive for unquoted + tokens (including the 100-continue token), and is case-sensitive for + quoted-string expectation-extensions. + + + + + + + +Fielding, et al. Standards Track [Page 126] + +RFC 2616 HTTP/1.1 June 1999 + + + The Expect mechanism is hop-by-hop: that is, an HTTP/1.1 proxy MUST + return a 417 (Expectation Failed) status if it receives a request + with an expectation that it cannot meet. However, the Expect + request-header itself is end-to-end; it MUST be forwarded if the + request is forwarded. + + Many older HTTP/1.0 and HTTP/1.1 applications do not understand the + Expect header. + + See section 8.2.3 for the use of the 100 (continue) status. + +14.21 Expires + + The Expires entity-header field gives the date/time after which the + response is considered stale. A stale cache entry may not normally be + returned by a cache (either a proxy cache or a user agent cache) + unless it is first validated with the origin server (or with an + intermediate cache that has a fresh copy of the entity). See section + 13.2 for further discussion of the expiration model. + + The presence of an Expires field does not imply that the original + resource will change or cease to exist at, before, or after that + time. + + The format is an absolute date and time as defined by HTTP-date in + section 3.3.1; it MUST be in RFC 1123 date format: + + Expires = "Expires" ":" HTTP-date + + An example of its use is + + Expires: Thu, 01 Dec 1994 16:00:00 GMT + + Note: if a response includes a Cache-Control field with the max- + age directive (see section 14.9.3), that directive overrides the + Expires field. + + HTTP/1.1 clients and caches MUST treat other invalid date formats, + especially including the value "0", as in the past (i.e., "already + expired"). + + To mark a response as "already expired," an origin server sends an + Expires date that is equal to the Date header value. (See the rules + for expiration calculations in section 13.2.4.) + + + + + + + +Fielding, et al. Standards Track [Page 127] + +RFC 2616 HTTP/1.1 June 1999 + + + To mark a response as "never expires," an origin server sends an + Expires date approximately one year from the time the response is + sent. HTTP/1.1 servers SHOULD NOT send Expires dates more than one + year in the future. + + The presence of an Expires header field with a date value of some + time in the future on a response that otherwise would by default be + non-cacheable indicates that the response is cacheable, unless + indicated otherwise by a Cache-Control header field (section 14.9). + +14.22 From + + The From request-header field, if given, SHOULD contain an Internet + e-mail address for the human user who controls the requesting user + agent. The address SHOULD be machine-usable, as defined by "mailbox" + in RFC 822 [9] as updated by RFC 1123 [8]: + + From = "From" ":" mailbox + + An example is: + + From: [email protected] + + This header field MAY be used for logging purposes and as a means for + identifying the source of invalid or unwanted requests. It SHOULD NOT + be used as an insecure form of access protection. The interpretation + of this field is that the request is being performed on behalf of the + person given, who accepts responsibility for the method performed. In + particular, robot agents SHOULD include this header so that the + person responsible for running the robot can be contacted if problems + occur on the receiving end. + + The Internet e-mail address in this field MAY be separate from the + Internet host which issued the request. For example, when a request + is passed through a proxy the original issuer's address SHOULD be + used. + + The client SHOULD NOT send the From header field without the user's + approval, as it might conflict with the user's privacy interests or + their site's security policy. It is strongly recommended that the + user be able to disable, enable, and modify the value of this field + at any time prior to a request. + +14.23 Host + + The Host request-header field specifies the Internet host and port + number of the resource being requested, as obtained from the original + URI given by the user or referring resource (generally an HTTP URL, + + + +Fielding, et al. Standards Track [Page 128] + +RFC 2616 HTTP/1.1 June 1999 + + + as described in section 3.2.2). The Host field value MUST represent + the naming authority of the origin server or gateway given by the + original URL. This allows the origin server or gateway to + differentiate between internally-ambiguous URLs, such as the root "/" + URL of a server for multiple host names on a single IP address. + + Host = "Host" ":" host [ ":" port ] ; Section 3.2.2 + + A "host" without any trailing port information implies the default + port for the service requested (e.g., "80" for an HTTP URL). For + example, a request on the origin server for + <http://www.w3.org/pub/WWW/> would properly include: + + GET /pub/WWW/ HTTP/1.1 + Host: www.w3.org + + A client MUST include a Host header field in all HTTP/1.1 request + messages . If the requested URI does not include an Internet host + name for the service being requested, then the Host header field MUST + be given with an empty value. An HTTP/1.1 proxy MUST ensure that any + request message it forwards does contain an appropriate Host header + field that identifies the service being requested by the proxy. All + Internet-based HTTP/1.1 servers MUST respond with a 400 (Bad Request) + status code to any HTTP/1.1 request message which lacks a Host header + field. + + See sections 5.2 and 19.6.1.1 for other requirements relating to + Host. + +14.24 If-Match + + The If-Match request-header field is used with a method to make it + conditional. A client that has one or more entities previously + obtained from the resource can verify that one of those entities is + current by including a list of their associated entity tags in the + If-Match header field. Entity tags are defined in section 3.11. The + purpose of this feature is to allow efficient updates of cached + information with a minimum amount of transaction overhead. It is also + used, on updating requests, to prevent inadvertent modification of + the wrong version of a resource. As a special case, the value "*" + matches any current entity of the resource. + + If-Match = "If-Match" ":" ( "*" | 1#entity-tag ) + + If any of the entity tags match the entity tag of the entity that + would have been returned in the response to a similar GET request + (without the If-Match header) on that resource, or if "*" is given + + + + +Fielding, et al. Standards Track [Page 129] + +RFC 2616 HTTP/1.1 June 1999 + + + and any current entity exists for that resource, then the server MAY + perform the requested method as if the If-Match header field did not + exist. + + A server MUST use the strong comparison function (see section 13.3.3) + to compare the entity tags in If-Match. + + If none of the entity tags match, or if "*" is given and no current + entity exists, the server MUST NOT perform the requested method, and + MUST return a 412 (Precondition Failed) response. This behavior is + most useful when the client wants to prevent an updating method, such + as PUT, from modifying a resource that has changed since the client + last retrieved it. + + If the request would, without the If-Match header field, result in + anything other than a 2xx or 412 status, then the If-Match header + MUST be ignored. + + The meaning of "If-Match: *" is that the method SHOULD be performed + if the representation selected by the origin server (or by a cache, + possibly using the Vary mechanism, see section 14.44) exists, and + MUST NOT be performed if the representation does not exist. + + A request intended to update a resource (e.g., a PUT) MAY include an + If-Match header field to signal that the request method MUST NOT be + applied if the entity corresponding to the If-Match value (a single + entity tag) is no longer a representation of that resource. This + allows the user to indicate that they do not wish the request to be + successful if the resource has been changed without their knowledge. + Examples: + + If-Match: "xyzzy" + If-Match: "xyzzy", "r2d2xxxx", "c3piozzzz" + If-Match: * + + The result of a request having both an If-Match header field and + either an If-None-Match or an If-Modified-Since header fields is + undefined by this specification. + +14.25 If-Modified-Since + + The If-Modified-Since request-header field is used with a method to + make it conditional: if the requested variant has not been modified + since the time specified in this field, an entity will not be + returned from the server; instead, a 304 (not modified) response will + be returned without any message-body. + + If-Modified-Since = "If-Modified-Since" ":" HTTP-date + + + +Fielding, et al. Standards Track [Page 130] + +RFC 2616 HTTP/1.1 June 1999 + + + An example of the field is: + + If-Modified-Since: Sat, 29 Oct 1994 19:43:31 GMT + + A GET method with an If-Modified-Since header and no Range header + requests that the identified entity be transferred only if it has + been modified since the date given by the If-Modified-Since header. + The algorithm for determining this includes the following cases: + + a) If the request would normally result in anything other than a + 200 (OK) status, or if the passed If-Modified-Since date is + invalid, the response is exactly the same as for a normal GET. + A date which is later than the server's current time is + invalid. + + b) If the variant has been modified since the If-Modified-Since + date, the response is exactly the same as for a normal GET. + + c) If the variant has not been modified since a valid If- + Modified-Since date, the server SHOULD return a 304 (Not + Modified) response. + + The purpose of this feature is to allow efficient updates of cached + information with a minimum amount of transaction overhead. + + Note: The Range request-header field modifies the meaning of If- + Modified-Since; see section 14.35 for full details. + + Note: If-Modified-Since times are interpreted by the server, whose + clock might not be synchronized with the client. + + Note: When handling an If-Modified-Since header field, some + servers will use an exact date comparison function, rather than a + less-than function, for deciding whether to send a 304 (Not + Modified) response. To get best results when sending an If- + Modified-Since header field for cache validation, clients are + advised to use the exact date string received in a previous Last- + Modified header field whenever possible. + + Note: If a client uses an arbitrary date in the If-Modified-Since + header instead of a date taken from the Last-Modified header for + the same request, the client should be aware of the fact that this + date is interpreted in the server's understanding of time. The + client should consider unsynchronized clocks and rounding problems + due to the different encodings of time between the client and + server. This includes the possibility of race conditions if the + document has changed between the time it was first requested and + the If-Modified-Since date of a subsequent request, and the + + + +Fielding, et al. Standards Track [Page 131] + +RFC 2616 HTTP/1.1 June 1999 + + + possibility of clock-skew-related problems if the If-Modified- + Since date is derived from the client's clock without correction + to the server's clock. Corrections for different time bases + between client and server are at best approximate due to network + latency. + + The result of a request having both an If-Modified-Since header field + and either an If-Match or an If-Unmodified-Since header fields is + undefined by this specification. + +14.26 If-None-Match + + The If-None-Match request-header field is used with a method to make + it conditional. A client that has one or more entities previously + obtained from the resource can verify that none of those entities is + current by including a list of their associated entity tags in the + If-None-Match header field. The purpose of this feature is to allow + efficient updates of cached information with a minimum amount of + transaction overhead. It is also used to prevent a method (e.g. PUT) + from inadvertently modifying an existing resource when the client + believes that the resource does not exist. + + As a special case, the value "*" matches any current entity of the + resource. + + If-None-Match = "If-None-Match" ":" ( "*" | 1#entity-tag ) + + If any of the entity tags match the entity tag of the entity that + would have been returned in the response to a similar GET request + (without the If-None-Match header) on that resource, or if "*" is + given and any current entity exists for that resource, then the + server MUST NOT perform the requested method, unless required to do + so because the resource's modification date fails to match that + supplied in an If-Modified-Since header field in the request. + Instead, if the request method was GET or HEAD, the server SHOULD + respond with a 304 (Not Modified) response, including the cache- + related header fields (particularly ETag) of one of the entities that + matched. For all other request methods, the server MUST respond with + a status of 412 (Precondition Failed). + + See section 13.3.3 for rules on how to determine if two entities tags + match. The weak comparison function can only be used with GET or HEAD + requests. + + + + + + + + +Fielding, et al. Standards Track [Page 132] + +RFC 2616 HTTP/1.1 June 1999 + + + If none of the entity tags match, then the server MAY perform the + requested method as if the If-None-Match header field did not exist, + but MUST also ignore any If-Modified-Since header field(s) in the + request. That is, if no entity tags match, then the server MUST NOT + return a 304 (Not Modified) response. + + If the request would, without the If-None-Match header field, result + in anything other than a 2xx or 304 status, then the If-None-Match + header MUST be ignored. (See section 13.3.4 for a discussion of + server behavior when both If-Modified-Since and If-None-Match appear + in the same request.) + + The meaning of "If-None-Match: *" is that the method MUST NOT be + performed if the representation selected by the origin server (or by + a cache, possibly using the Vary mechanism, see section 14.44) + exists, and SHOULD be performed if the representation does not exist. + This feature is intended to be useful in preventing races between PUT + operations. + + Examples: + + If-None-Match: "xyzzy" + If-None-Match: W/"xyzzy" + If-None-Match: "xyzzy", "r2d2xxxx", "c3piozzzz" + If-None-Match: W/"xyzzy", W/"r2d2xxxx", W/"c3piozzzz" + If-None-Match: * + + The result of a request having both an If-None-Match header field and + either an If-Match or an If-Unmodified-Since header fields is + undefined by this specification. + +14.27 If-Range + + If a client has a partial copy of an entity in its cache, and wishes + to have an up-to-date copy of the entire entity in its cache, it + could use the Range request-header with a conditional GET (using + either or both of If-Unmodified-Since and If-Match.) However, if the + condition fails because the entity has been modified, the client + would then have to make a second request to obtain the entire current + entity-body. + + The If-Range header allows a client to "short-circuit" the second + request. Informally, its meaning is `if the entity is unchanged, send + me the part(s) that I am missing; otherwise, send me the entire new + entity'. + + If-Range = "If-Range" ":" ( entity-tag | HTTP-date ) + + + + +Fielding, et al. Standards Track [Page 133] + +RFC 2616 HTTP/1.1 June 1999 + + + If the client has no entity tag for an entity, but does have a Last- + Modified date, it MAY use that date in an If-Range header. (The + server can distinguish between a valid HTTP-date and any form of + entity-tag by examining no more than two characters.) The If-Range + header SHOULD only be used together with a Range header, and MUST be + ignored if the request does not include a Range header, or if the + server does not support the sub-range operation. + + If the entity tag given in the If-Range header matches the current + entity tag for the entity, then the server SHOULD provide the + specified sub-range of the entity using a 206 (Partial content) + response. If the entity tag does not match, then the server SHOULD + return the entire entity using a 200 (OK) response. + +14.28 If-Unmodified-Since + + The If-Unmodified-Since request-header field is used with a method to + make it conditional. If the requested resource has not been modified + since the time specified in this field, the server SHOULD perform the + requested operation as if the If-Unmodified-Since header were not + present. + + If the requested variant has been modified since the specified time, + the server MUST NOT perform the requested operation, and MUST return + a 412 (Precondition Failed). + + If-Unmodified-Since = "If-Unmodified-Since" ":" HTTP-date + + An example of the field is: + + If-Unmodified-Since: Sat, 29 Oct 1994 19:43:31 GMT + + If the request normally (i.e., without the If-Unmodified-Since + header) would result in anything other than a 2xx or 412 status, the + If-Unmodified-Since header SHOULD be ignored. + + If the specified date is invalid, the header is ignored. + + The result of a request having both an If-Unmodified-Since header + field and either an If-None-Match or an If-Modified-Since header + fields is undefined by this specification. + +14.29 Last-Modified + + The Last-Modified entity-header field indicates the date and time at + which the origin server believes the variant was last modified. + + Last-Modified = "Last-Modified" ":" HTTP-date + + + +Fielding, et al. Standards Track [Page 134] + +RFC 2616 HTTP/1.1 June 1999 + + + An example of its use is + + Last-Modified: Tue, 15 Nov 1994 12:45:26 GMT + + The exact meaning of this header field depends on the implementation + of the origin server and the nature of the original resource. For + files, it may be just the file system last-modified time. For + entities with dynamically included parts, it may be the most recent + of the set of last-modify times for its component parts. For database + gateways, it may be the last-update time stamp of the record. For + virtual objects, it may be the last time the internal state changed. + + An origin server MUST NOT send a Last-Modified date which is later + than the server's time of message origination. In such cases, where + the resource's last modification would indicate some time in the + future, the server MUST replace that date with the message + origination date. + + An origin server SHOULD obtain the Last-Modified value of the entity + as close as possible to the time that it generates the Date value of + its response. This allows a recipient to make an accurate assessment + of the entity's modification time, especially if the entity changes + near the time that the response is generated. + + HTTP/1.1 servers SHOULD send Last-Modified whenever feasible. + +14.30 Location + + The Location response-header field is used to redirect the recipient + to a location other than the Request-URI for completion of the + request or identification of a new resource. For 201 (Created) + responses, the Location is that of the new resource which was created + by the request. For 3xx responses, the location SHOULD indicate the + server's preferred URI for automatic redirection to the resource. The + field value consists of a single absolute URI. + + Location = "Location" ":" absoluteURI + + An example is: + + Location: http://www.w3.org/pub/WWW/People.html + + Note: The Content-Location header field (section 14.14) differs + from Location in that the Content-Location identifies the original + location of the entity enclosed in the request. It is therefore + possible for a response to contain header fields for both Location + and Content-Location. Also see section 13.10 for cache + requirements of some methods. + + + +Fielding, et al. Standards Track [Page 135] + +RFC 2616 HTTP/1.1 June 1999 + + +14.31 Max-Forwards + + The Max-Forwards request-header field provides a mechanism with the + TRACE (section 9.8) and OPTIONS (section 9.2) methods to limit the + number of proxies or gateways that can forward the request to the + next inbound server. This can be useful when the client is attempting + to trace a request chain which appears to be failing or looping in + mid-chain. + + Max-Forwards = "Max-Forwards" ":" 1*DIGIT + + The Max-Forwards value is a decimal integer indicating the remaining + number of times this request message may be forwarded. + + Each proxy or gateway recipient of a TRACE or OPTIONS request + containing a Max-Forwards header field MUST check and update its + value prior to forwarding the request. If the received value is zero + (0), the recipient MUST NOT forward the request; instead, it MUST + respond as the final recipient. If the received Max-Forwards value is + greater than zero, then the forwarded message MUST contain an updated + Max-Forwards field with a value decremented by one (1). + + The Max-Forwards header field MAY be ignored for all other methods + defined by this specification and for any extension methods for which + it is not explicitly referred to as part of that method definition. + +14.32 Pragma + + The Pragma general-header field is used to include implementation- + specific directives that might apply to any recipient along the + request/response chain. All pragma directives specify optional + behavior from the viewpoint of the protocol; however, some systems + MAY require that behavior be consistent with the directives. + + Pragma = "Pragma" ":" 1#pragma-directive + pragma-directive = "no-cache" | extension-pragma + extension-pragma = token [ "=" ( token | quoted-string ) ] + + When the no-cache directive is present in a request message, an + application SHOULD forward the request toward the origin server even + if it has a cached copy of what is being requested. This pragma + directive has the same semantics as the no-cache cache-directive (see + section 14.9) and is defined here for backward compatibility with + HTTP/1.0. Clients SHOULD include both header fields when a no-cache + request is sent to a server not known to be HTTP/1.1 compliant. + + + + + + +Fielding, et al. Standards Track [Page 136] + +RFC 2616 HTTP/1.1 June 1999 + + + Pragma directives MUST be passed through by a proxy or gateway + application, regardless of their significance to that application, + since the directives might be applicable to all recipients along the + request/response chain. It is not possible to specify a pragma for a + specific recipient; however, any pragma directive not relevant to a + recipient SHOULD be ignored by that recipient. + + HTTP/1.1 caches SHOULD treat "Pragma: no-cache" as if the client had + sent "Cache-Control: no-cache". No new Pragma directives will be + defined in HTTP. + + Note: because the meaning of "Pragma: no-cache as a response + header field is not actually specified, it does not provide a + reliable replacement for "Cache-Control: no-cache" in a response + +14.33 Proxy-Authenticate + + The Proxy-Authenticate response-header field MUST be included as part + of a 407 (Proxy Authentication Required) response. The field value + consists of a challenge that indicates the authentication scheme and + parameters applicable to the proxy for this Request-URI. + + Proxy-Authenticate = "Proxy-Authenticate" ":" 1#challenge + + The HTTP access authentication process is described in "HTTP + Authentication: Basic and Digest Access Authentication" [43]. Unlike + WWW-Authenticate, the Proxy-Authenticate header field applies only to + the current connection and SHOULD NOT be passed on to downstream + clients. However, an intermediate proxy might need to obtain its own + credentials by requesting them from the downstream client, which in + some circumstances will appear as if the proxy is forwarding the + Proxy-Authenticate header field. + +14.34 Proxy-Authorization + + The Proxy-Authorization request-header field allows the client to + identify itself (or its user) to a proxy which requires + authentication. The Proxy-Authorization field value consists of + credentials containing the authentication information of the user + agent for the proxy and/or realm of the resource being requested. + + Proxy-Authorization = "Proxy-Authorization" ":" credentials + + The HTTP access authentication process is described in "HTTP + Authentication: Basic and Digest Access Authentication" [43] . Unlike + Authorization, the Proxy-Authorization header field applies only to + the next outbound proxy that demanded authentication using the Proxy- + Authenticate field. When multiple proxies are used in a chain, the + + + +Fielding, et al. Standards Track [Page 137] + +RFC 2616 HTTP/1.1 June 1999 + + + Proxy-Authorization header field is consumed by the first outbound + proxy that was expecting to receive credentials. A proxy MAY relay + the credentials from the client request to the next proxy if that is + the mechanism by which the proxies cooperatively authenticate a given + request. + +14.35 Range + +14.35.1 Byte Ranges + + Since all HTTP entities are represented in HTTP messages as sequences + of bytes, the concept of a byte range is meaningful for any HTTP + entity. (However, not all clients and servers need to support byte- + range operations.) + + Byte range specifications in HTTP apply to the sequence of bytes in + the entity-body (not necessarily the same as the message-body). + + A byte range operation MAY specify a single range of bytes, or a set + of ranges within a single entity. + + ranges-specifier = byte-ranges-specifier + byte-ranges-specifier = bytes-unit "=" byte-range-set + byte-range-set = 1#( byte-range-spec | suffix-byte-range-spec ) + byte-range-spec = first-byte-pos "-" [last-byte-pos] + first-byte-pos = 1*DIGIT + last-byte-pos = 1*DIGIT + + The first-byte-pos value in a byte-range-spec gives the byte-offset + of the first byte in a range. The last-byte-pos value gives the + byte-offset of the last byte in the range; that is, the byte + positions specified are inclusive. Byte offsets start at zero. + + If the last-byte-pos value is present, it MUST be greater than or + equal to the first-byte-pos in that byte-range-spec, or the byte- + range-spec is syntactically invalid. The recipient of a byte-range- + set that includes one or more syntactically invalid byte-range-spec + values MUST ignore the header field that includes that byte-range- + set. + + If the last-byte-pos value is absent, or if the value is greater than + or equal to the current length of the entity-body, last-byte-pos is + taken to be equal to one less than the current length of the entity- + body in bytes. + + By its choice of last-byte-pos, a client can limit the number of + bytes retrieved without knowing the size of the entity. + + + + +Fielding, et al. Standards Track [Page 138] + +RFC 2616 HTTP/1.1 June 1999 + + + suffix-byte-range-spec = "-" suffix-length + suffix-length = 1*DIGIT + + A suffix-byte-range-spec is used to specify the suffix of the + entity-body, of a length given by the suffix-length value. (That is, + this form specifies the last N bytes of an entity-body.) If the + entity is shorter than the specified suffix-length, the entire + entity-body is used. + + If a syntactically valid byte-range-set includes at least one byte- + range-spec whose first-byte-pos is less than the current length of + the entity-body, or at least one suffix-byte-range-spec with a non- + zero suffix-length, then the byte-range-set is satisfiable. + Otherwise, the byte-range-set is unsatisfiable. If the byte-range-set + is unsatisfiable, the server SHOULD return a response with a status + of 416 (Requested range not satisfiable). Otherwise, the server + SHOULD return a response with a status of 206 (Partial Content) + containing the satisfiable ranges of the entity-body. + + Examples of byte-ranges-specifier values (assuming an entity-body of + length 10000): + + - The first 500 bytes (byte offsets 0-499, inclusive): bytes=0- + 499 + + - The second 500 bytes (byte offsets 500-999, inclusive): + bytes=500-999 + + - The final 500 bytes (byte offsets 9500-9999, inclusive): + bytes=-500 + + - Or bytes=9500- + + - The first and last bytes only (bytes 0 and 9999): bytes=0-0,-1 + + - Several legal but not canonical specifications of the second 500 + bytes (byte offsets 500-999, inclusive): + bytes=500-600,601-999 + bytes=500-700,601-999 + +14.35.2 Range Retrieval Requests + + HTTP retrieval requests using conditional or unconditional GET + methods MAY request one or more sub-ranges of the entity, instead of + the entire entity, using the Range request header, which applies to + the entity returned as the result of the request: + + Range = "Range" ":" ranges-specifier + + + +Fielding, et al. Standards Track [Page 139] + +RFC 2616 HTTP/1.1 June 1999 + + + A server MAY ignore the Range header. However, HTTP/1.1 origin + servers and intermediate caches ought to support byte ranges when + possible, since Range supports efficient recovery from partially + failed transfers, and supports efficient partial retrieval of large + entities. + + If the server supports the Range header and the specified range or + ranges are appropriate for the entity: + + - The presence of a Range header in an unconditional GET modifies + what is returned if the GET is otherwise successful. In other + words, the response carries a status code of 206 (Partial + Content) instead of 200 (OK). + + - The presence of a Range header in a conditional GET (a request + using one or both of If-Modified-Since and If-None-Match, or + one or both of If-Unmodified-Since and If-Match) modifies what + is returned if the GET is otherwise successful and the + condition is true. It does not affect the 304 (Not Modified) + response returned if the conditional is false. + + In some cases, it might be more appropriate to use the If-Range + header (see section 14.27) in addition to the Range header. + + If a proxy that supports ranges receives a Range request, forwards + the request to an inbound server, and receives an entire entity in + reply, it SHOULD only return the requested range to its client. It + SHOULD store the entire received response in its cache if that is + consistent with its cache allocation policies. + +14.36 Referer + + The Referer[sic] request-header field allows the client to specify, + for the server's benefit, the address (URI) of the resource from + which the Request-URI was obtained (the "referrer", although the + header field is misspelled.) The Referer request-header allows a + server to generate lists of back-links to resources for interest, + logging, optimized caching, etc. It also allows obsolete or mistyped + links to be traced for maintenance. The Referer field MUST NOT be + sent if the Request-URI was obtained from a source that does not have + its own URI, such as input from the user keyboard. + + Referer = "Referer" ":" ( absoluteURI | relativeURI ) + + Example: + + Referer: http://www.w3.org/hypertext/DataSources/Overview.html + + + + +Fielding, et al. Standards Track [Page 140] + +RFC 2616 HTTP/1.1 June 1999 + + + If the field value is a relative URI, it SHOULD be interpreted + relative to the Request-URI. The URI MUST NOT include a fragment. See + section 15.1.3 for security considerations. + +14.37 Retry-After + + The Retry-After response-header field can be used with a 503 (Service + Unavailable) response to indicate how long the service is expected to + be unavailable to the requesting client. This field MAY also be used + with any 3xx (Redirection) response to indicate the minimum time the + user-agent is asked wait before issuing the redirected request. The + value of this field can be either an HTTP-date or an integer number + of seconds (in decimal) after the time of the response. + + Retry-After = "Retry-After" ":" ( HTTP-date | delta-seconds ) + + Two examples of its use are + + Retry-After: Fri, 31 Dec 1999 23:59:59 GMT + Retry-After: 120 + + In the latter example, the delay is 2 minutes. + +14.38 Server + + The Server response-header field contains information about the + software used by the origin server to handle the request. The field + can contain multiple product tokens (section 3.8) and comments + identifying the server and any significant subproducts. The product + tokens are listed in order of their significance for identifying the + application. + + Server = "Server" ":" 1*( product | comment ) + + Example: + + Server: CERN/3.0 libwww/2.17 + + If the response is being forwarded through a proxy, the proxy + application MUST NOT modify the Server response-header. Instead, it + SHOULD include a Via field (as described in section 14.45). + + Note: Revealing the specific software version of the server might + allow the server machine to become more vulnerable to attacks + against software that is known to contain security holes. Server + implementors are encouraged to make this field a configurable + option. + + + + +Fielding, et al. Standards Track [Page 141] + +RFC 2616 HTTP/1.1 June 1999 + + +14.39 TE + + The TE request-header field indicates what extension transfer-codings + it is willing to accept in the response and whether or not it is + willing to accept trailer fields in a chunked transfer-coding. Its + value may consist of the keyword "trailers" and/or a comma-separated + list of extension transfer-coding names with optional accept + parameters (as described in section 3.6). + + TE = "TE" ":" #( t-codings ) + t-codings = "trailers" | ( transfer-extension [ accept-params ] ) + + The presence of the keyword "trailers" indicates that the client is + willing to accept trailer fields in a chunked transfer-coding, as + defined in section 3.6.1. This keyword is reserved for use with + transfer-coding values even though it does not itself represent a + transfer-coding. + + Examples of its use are: + + TE: deflate + TE: + TE: trailers, deflate;q=0.5 + + The TE header field only applies to the immediate connection. + Therefore, the keyword MUST be supplied within a Connection header + field (section 14.10) whenever TE is present in an HTTP/1.1 message. + + A server tests whether a transfer-coding is acceptable, according to + a TE field, using these rules: + + 1. The "chunked" transfer-coding is always acceptable. If the + keyword "trailers" is listed, the client indicates that it is + willing to accept trailer fields in the chunked response on + behalf of itself and any downstream clients. The implication is + that, if given, the client is stating that either all + downstream clients are willing to accept trailer fields in the + forwarded response, or that it will attempt to buffer the + response on behalf of downstream recipients. + + Note: HTTP/1.1 does not define any means to limit the size of a + chunked response such that a client can be assured of buffering + the entire response. + + 2. If the transfer-coding being tested is one of the transfer- + codings listed in the TE field, then it is acceptable unless it + is accompanied by a qvalue of 0. (As defined in section 3.9, a + qvalue of 0 means "not acceptable.") + + + +Fielding, et al. Standards Track [Page 142] + +RFC 2616 HTTP/1.1 June 1999 + + + 3. If multiple transfer-codings are acceptable, then the + acceptable transfer-coding with the highest non-zero qvalue is + preferred. The "chunked" transfer-coding always has a qvalue + of 1. + + If the TE field-value is empty or if no TE field is present, the only + transfer-coding is "chunked". A message with no transfer-coding is + always acceptable. + +14.40 Trailer + + The Trailer general field value indicates that the given set of + header fields is present in the trailer of a message encoded with + chunked transfer-coding. + + Trailer = "Trailer" ":" 1#field-name + + An HTTP/1.1 message SHOULD include a Trailer header field in a + message using chunked transfer-coding with a non-empty trailer. Doing + so allows the recipient to know which header fields to expect in the + trailer. + + If no Trailer header field is present, the trailer SHOULD NOT include + any header fields. See section 3.6.1 for restrictions on the use of + trailer fields in a "chunked" transfer-coding. + + Message header fields listed in the Trailer header field MUST NOT + include the following header fields: + + . Transfer-Encoding + + . Content-Length + + . Trailer + +14.41 Transfer-Encoding + + The Transfer-Encoding general-header field indicates what (if any) + type of transformation has been applied to the message body in order + to safely transfer it between the sender and the recipient. This + differs from the content-coding in that the transfer-coding is a + property of the message, not of the entity. + + Transfer-Encoding = "Transfer-Encoding" ":" 1#transfer-coding + + Transfer-codings are defined in section 3.6. An example is: + + Transfer-Encoding: chunked + + + +Fielding, et al. Standards Track [Page 143] + +RFC 2616 HTTP/1.1 June 1999 + + + If multiple encodings have been applied to an entity, the transfer- + codings MUST be listed in the order in which they were applied. + Additional information about the encoding parameters MAY be provided + by other entity-header fields not defined by this specification. + + Many older HTTP/1.0 applications do not understand the Transfer- + Encoding header. + +14.42 Upgrade + + The Upgrade general-header allows the client to specify what + additional communication protocols it supports and would like to use + if the server finds it appropriate to switch protocols. The server + MUST use the Upgrade header field within a 101 (Switching Protocols) + response to indicate which protocol(s) are being switched. + + Upgrade = "Upgrade" ":" 1#product + + For example, + + Upgrade: HTTP/2.0, SHTTP/1.3, IRC/6.9, RTA/x11 + + The Upgrade header field is intended to provide a simple mechanism + for transition from HTTP/1.1 to some other, incompatible protocol. It + does so by allowing the client to advertise its desire to use another + protocol, such as a later version of HTTP with a higher major version + number, even though the current request has been made using HTTP/1.1. + This eases the difficult transition between incompatible protocols by + allowing the client to initiate a request in the more commonly + supported protocol while indicating to the server that it would like + to use a "better" protocol if available (where "better" is determined + by the server, possibly according to the nature of the method and/or + resource being requested). + + The Upgrade header field only applies to switching application-layer + protocols upon the existing transport-layer connection. Upgrade + cannot be used to insist on a protocol change; its acceptance and use + by the server is optional. The capabilities and nature of the + application-layer communication after the protocol change is entirely + dependent upon the new protocol chosen, although the first action + after changing the protocol MUST be a response to the initial HTTP + request containing the Upgrade header field. + + The Upgrade header field only applies to the immediate connection. + Therefore, the upgrade keyword MUST be supplied within a Connection + header field (section 14.10) whenever Upgrade is present in an + HTTP/1.1 message. + + + + +Fielding, et al. Standards Track [Page 144] + +RFC 2616 HTTP/1.1 June 1999 + + + The Upgrade header field cannot be used to indicate a switch to a + protocol on a different connection. For that purpose, it is more + appropriate to use a 301, 302, 303, or 305 redirection response. + + This specification only defines the protocol name "HTTP" for use by + the family of Hypertext Transfer Protocols, as defined by the HTTP + version rules of section 3.1 and future updates to this + specification. Any token can be used as a protocol name; however, it + will only be useful if both the client and server associate the name + with the same protocol. + +14.43 User-Agent + + The User-Agent request-header field contains information about the + user agent originating the request. This is for statistical purposes, + the tracing of protocol violations, and automated recognition of user + agents for the sake of tailoring responses to avoid particular user + agent limitations. User agents SHOULD include this field with + requests. The field can contain multiple product tokens (section 3.8) + and comments identifying the agent and any subproducts which form a + significant part of the user agent. By convention, the product tokens + are listed in order of their significance for identifying the + application. + + User-Agent = "User-Agent" ":" 1*( product | comment ) + + Example: + + User-Agent: CERN-LineMode/2.15 libwww/2.17b3 + +14.44 Vary + + The Vary field value indicates the set of request-header fields that + fully determines, while the response is fresh, whether a cache is + permitted to use the response to reply to a subsequent request + without revalidation. For uncacheable or stale responses, the Vary + field value advises the user agent about the criteria that were used + to select the representation. A Vary field value of "*" implies that + a cache cannot determine from the request headers of a subsequent + request whether this response is the appropriate representation. See + section 13.6 for use of the Vary header field by caches. + + Vary = "Vary" ":" ( "*" | 1#field-name ) + + An HTTP/1.1 server SHOULD include a Vary header field with any + cacheable response that is subject to server-driven negotiation. + Doing so allows a cache to properly interpret future requests on that + resource and informs the user agent about the presence of negotiation + + + +Fielding, et al. Standards Track [Page 145] + +RFC 2616 HTTP/1.1 June 1999 + + + on that resource. A server MAY include a Vary header field with a + non-cacheable response that is subject to server-driven negotiation, + since this might provide the user agent with useful information about + the dimensions over which the response varies at the time of the + response. + + A Vary field value consisting of a list of field-names signals that + the representation selected for the response is based on a selection + algorithm which considers ONLY the listed request-header field values + in selecting the most appropriate representation. A cache MAY assume + that the same selection will be made for future requests with the + same values for the listed field names, for the duration of time for + which the response is fresh. + + The field-names given are not limited to the set of standard + request-header fields defined by this specification. Field names are + case-insensitive. + + A Vary field value of "*" signals that unspecified parameters not + limited to the request-headers (e.g., the network address of the + client), play a role in the selection of the response representation. + The "*" value MUST NOT be generated by a proxy server; it may only be + generated by an origin server. + +14.45 Via + + The Via general-header field MUST be used by gateways and proxies to + indicate the intermediate protocols and recipients between the user + agent and the server on requests, and between the origin server and + the client on responses. It is analogous to the "Received" field of + RFC 822 [9] and is intended to be used for tracking message forwards, + avoiding request loops, and identifying the protocol capabilities of + all senders along the request/response chain. + + Via = "Via" ":" 1#( received-protocol received-by [ comment ] ) + received-protocol = [ protocol-name "/" ] protocol-version + protocol-name = token + protocol-version = token + received-by = ( host [ ":" port ] ) | pseudonym + pseudonym = token + + The received-protocol indicates the protocol version of the message + received by the server or client along each segment of the + request/response chain. The received-protocol version is appended to + the Via field value when the message is forwarded so that information + about the protocol capabilities of upstream applications remains + visible to all recipients. + + + + +Fielding, et al. Standards Track [Page 146] + +RFC 2616 HTTP/1.1 June 1999 + + + The protocol-name is optional if and only if it would be "HTTP". The + received-by field is normally the host and optional port number of a + recipient server or client that subsequently forwarded the message. + However, if the real host is considered to be sensitive information, + it MAY be replaced by a pseudonym. If the port is not given, it MAY + be assumed to be the default port of the received-protocol. + + Multiple Via field values represents each proxy or gateway that has + forwarded the message. Each recipient MUST append its information + such that the end result is ordered according to the sequence of + forwarding applications. + + Comments MAY be used in the Via header field to identify the software + of the recipient proxy or gateway, analogous to the User-Agent and + Server header fields. However, all comments in the Via field are + optional and MAY be removed by any recipient prior to forwarding the + message. + + For example, a request message could be sent from an HTTP/1.0 user + agent to an internal proxy code-named "fred", which uses HTTP/1.1 to + forward the request to a public proxy at nowhere.com, which completes + the request by forwarding it to the origin server at www.ics.uci.edu. + The request received by www.ics.uci.edu would then have the following + Via header field: + + Via: 1.0 fred, 1.1 nowhere.com (Apache/1.1) + + Proxies and gateways used as a portal through a network firewall + SHOULD NOT, by default, forward the names and ports of hosts within + the firewall region. This information SHOULD only be propagated if + explicitly enabled. If not enabled, the received-by host of any host + behind the firewall SHOULD be replaced by an appropriate pseudonym + for that host. + + For organizations that have strong privacy requirements for hiding + internal structures, a proxy MAY combine an ordered subsequence of + Via header field entries with identical received-protocol values into + a single such entry. For example, + + Via: 1.0 ricky, 1.1 ethel, 1.1 fred, 1.0 lucy + + could be collapsed to + + Via: 1.0 ricky, 1.1 mertz, 1.0 lucy + + + + + + + +Fielding, et al. Standards Track [Page 147] + +RFC 2616 HTTP/1.1 June 1999 + + + Applications SHOULD NOT combine multiple entries unless they are all + under the same organizational control and the hosts have already been + replaced by pseudonyms. Applications MUST NOT combine entries which + have different received-protocol values. + +14.46 Warning + + The Warning general-header field is used to carry additional + information about the status or transformation of a message which + might not be reflected in the message. This information is typically + used to warn about a possible lack of semantic transparency from + caching operations or transformations applied to the entity body of + the message. + + Warning headers are sent with responses using: + + Warning = "Warning" ":" 1#warning-value + + warning-value = warn-code SP warn-agent SP warn-text + [SP warn-date] + + warn-code = 3DIGIT + warn-agent = ( host [ ":" port ] ) | pseudonym + ; the name or pseudonym of the server adding + ; the Warning header, for use in debugging + warn-text = quoted-string + warn-date = <"> HTTP-date <"> + + A response MAY carry more than one Warning header. + + The warn-text SHOULD be in a natural language and character set that + is most likely to be intelligible to the human user receiving the + response. This decision MAY be based on any available knowledge, such + as the location of the cache or user, the Accept-Language field in a + request, the Content-Language field in a response, etc. The default + language is English and the default character set is ISO-8859-1. + + If a character set other than ISO-8859-1 is used, it MUST be encoded + in the warn-text using the method described in RFC 2047 [14]. + + Warning headers can in general be applied to any message, however + some specific warn-codes are specific to caches and can only be + applied to response messages. New Warning headers SHOULD be added + after any existing Warning headers. A cache MUST NOT delete any + Warning header that it received with a message. However, if a cache + successfully validates a cache entry, it SHOULD remove any Warning + headers previously attached to that entry except as specified for + + + + +Fielding, et al. Standards Track [Page 148] + +RFC 2616 HTTP/1.1 June 1999 + + + specific Warning codes. It MUST then add any Warning headers received + in the validating response. In other words, Warning headers are those + that would be attached to the most recent relevant response. + + When multiple Warning headers are attached to a response, the user + agent ought to inform the user of as many of them as possible, in the + order that they appear in the response. If it is not possible to + inform the user of all of the warnings, the user agent SHOULD follow + these heuristics: + + - Warnings that appear early in the response take priority over + those appearing later in the response. + + - Warnings in the user's preferred character set take priority + over warnings in other character sets but with identical warn- + codes and warn-agents. + + Systems that generate multiple Warning headers SHOULD order them with + this user agent behavior in mind. + + Requirements for the behavior of caches with respect to Warnings are + stated in section 13.1.2. + + This is a list of the currently-defined warn-codes, each with a + recommended warn-text in English, and a description of its meaning. + + 110 Response is stale + MUST be included whenever the returned response is stale. + + 111 Revalidation failed + MUST be included if a cache returns a stale response because an + attempt to revalidate the response failed, due to an inability to + reach the server. + + 112 Disconnected operation + SHOULD be included if the cache is intentionally disconnected from + the rest of the network for a period of time. + + 113 Heuristic expiration + MUST be included if the cache heuristically chose a freshness + lifetime greater than 24 hours and the response's age is greater + than 24 hours. + + 199 Miscellaneous warning + The warning text MAY include arbitrary information to be presented + to a human user, or logged. A system receiving this warning MUST + NOT take any automated action, besides presenting the warning to + the user. + + + +Fielding, et al. Standards Track [Page 149] + +RFC 2616 HTTP/1.1 June 1999 + + + 214 Transformation applied + MUST be added by an intermediate cache or proxy if it applies any + transformation changing the content-coding (as specified in the + Content-Encoding header) or media-type (as specified in the + Content-Type header) of the response, or the entity-body of the + response, unless this Warning code already appears in the response. + + 299 Miscellaneous persistent warning + The warning text MAY include arbitrary information to be presented + to a human user, or logged. A system receiving this warning MUST + NOT take any automated action. + + If an implementation sends a message with one or more Warning headers + whose version is HTTP/1.0 or lower, then the sender MUST include in + each warning-value a warn-date that matches the date in the response. + + If an implementation receives a message with a warning-value that + includes a warn-date, and that warn-date is different from the Date + value in the response, then that warning-value MUST be deleted from + the message before storing, forwarding, or using it. (This prevents + bad consequences of naive caching of Warning header fields.) If all + of the warning-values are deleted for this reason, the Warning header + MUST be deleted as well. + +14.47 WWW-Authenticate + + The WWW-Authenticate response-header field MUST be included in 401 + (Unauthorized) response messages. The field value consists of at + least one challenge that indicates the authentication scheme(s) and + parameters applicable to the Request-URI. + + WWW-Authenticate = "WWW-Authenticate" ":" 1#challenge + + The HTTP access authentication process is described in "HTTP + Authentication: Basic and Digest Access Authentication" [43]. User + agents are advised to take special care in parsing the WWW- + Authenticate field value as it might contain more than one challenge, + or if more than one WWW-Authenticate header field is provided, the + contents of a challenge itself can contain a comma-separated list of + authentication parameters. + +15 Security Considerations + + This section is meant to inform application developers, information + providers, and users of the security limitations in HTTP/1.1 as + described by this document. The discussion does not include + definitive solutions to the problems revealed, though it does make + some suggestions for reducing security risks. + + + +Fielding, et al. Standards Track [Page 150] + +RFC 2616 HTTP/1.1 June 1999 + + +15.1 Personal Information + + HTTP clients are often privy to large amounts of personal information + (e.g. the user's name, location, mail address, passwords, encryption + keys, etc.), and SHOULD be very careful to prevent unintentional + leakage of this information via the HTTP protocol to other sources. + We very strongly recommend that a convenient interface be provided + for the user to control dissemination of such information, and that + designers and implementors be particularly careful in this area. + History shows that errors in this area often create serious security + and/or privacy problems and generate highly adverse publicity for the + implementor's company. + +15.1.1 Abuse of Server Log Information + + A server is in the position to save personal data about a user's + requests which might identify their reading patterns or subjects of + interest. This information is clearly confidential in nature and its + handling can be constrained by law in certain countries. People using + the HTTP protocol to provide data are responsible for ensuring that + such material is not distributed without the permission of any + individuals that are identifiable by the published results. + +15.1.2 Transfer of Sensitive Information + + Like any generic data transfer protocol, HTTP cannot regulate the + content of the data that is transferred, nor is there any a priori + method of determining the sensitivity of any particular piece of + information within the context of any given request. Therefore, + applications SHOULD supply as much control over this information as + possible to the provider of that information. Four header fields are + worth special mention in this context: Server, Via, Referer and From. + + Revealing the specific software version of the server might allow the + server machine to become more vulnerable to attacks against software + that is known to contain security holes. Implementors SHOULD make the + Server header field a configurable option. + + Proxies which serve as a portal through a network firewall SHOULD + take special precautions regarding the transfer of header information + that identifies the hosts behind the firewall. In particular, they + SHOULD remove, or replace with sanitized versions, any Via fields + generated behind the firewall. + + The Referer header allows reading patterns to be studied and reverse + links drawn. Although it can be very useful, its power can be abused + if user details are not separated from the information contained in + + + + +Fielding, et al. Standards Track [Page 151] + +RFC 2616 HTTP/1.1 June 1999 + + + the Referer. Even when the personal information has been removed, the + Referer header might indicate a private document's URI whose + publication would be inappropriate. + + The information sent in the From field might conflict with the user's + privacy interests or their site's security policy, and hence it + SHOULD NOT be transmitted without the user being able to disable, + enable, and modify the contents of the field. The user MUST be able + to set the contents of this field within a user preference or + application defaults configuration. + + We suggest, though do not require, that a convenient toggle interface + be provided for the user to enable or disable the sending of From and + Referer information. + + The User-Agent (section 14.43) or Server (section 14.38) header + fields can sometimes be used to determine that a specific client or + server have a particular security hole which might be exploited. + Unfortunately, this same information is often used for other valuable + purposes for which HTTP currently has no better mechanism. + +15.1.3 Encoding Sensitive Information in URI's + + Because the source of a link might be private information or might + reveal an otherwise private information source, it is strongly + recommended that the user be able to select whether or not the + Referer field is sent. For example, a browser client could have a + toggle switch for browsing openly/anonymously, which would + respectively enable/disable the sending of Referer and From + information. + + Clients SHOULD NOT include a Referer header field in a (non-secure) + HTTP request if the referring page was transferred with a secure + protocol. + + Authors of services which use the HTTP protocol SHOULD NOT use GET + based forms for the submission of sensitive data, because this will + cause this data to be encoded in the Request-URI. Many existing + servers, proxies, and user agents will log the request URI in some + place where it might be visible to third parties. Servers can use + POST-based form submission instead + +15.1.4 Privacy Issues Connected to Accept Headers + + Accept request-headers can reveal information about the user to all + servers which are accessed. The Accept-Language header in particular + can reveal information the user would consider to be of a private + nature, because the understanding of particular languages is often + + + +Fielding, et al. Standards Track [Page 152] + +RFC 2616 HTTP/1.1 June 1999 + + + strongly correlated to the membership of a particular ethnic group. + User agents which offer the option to configure the contents of an + Accept-Language header to be sent in every request are strongly + encouraged to let the configuration process include a message which + makes the user aware of the loss of privacy involved. + + An approach that limits the loss of privacy would be for a user agent + to omit the sending of Accept-Language headers by default, and to ask + the user whether or not to start sending Accept-Language headers to a + server if it detects, by looking for any Vary response-header fields + generated by the server, that such sending could improve the quality + of service. + + Elaborate user-customized accept header fields sent in every request, + in particular if these include quality values, can be used by servers + as relatively reliable and long-lived user identifiers. Such user + identifiers would allow content providers to do click-trail tracking, + and would allow collaborating content providers to match cross-server + click-trails or form submissions of individual users. Note that for + many users not behind a proxy, the network address of the host + running the user agent will also serve as a long-lived user + identifier. In environments where proxies are used to enhance + privacy, user agents ought to be conservative in offering accept + header configuration options to end users. As an extreme privacy + measure, proxies could filter the accept headers in relayed requests. + General purpose user agents which provide a high degree of header + configurability SHOULD warn users about the loss of privacy which can + be involved. + +15.2 Attacks Based On File and Path Names + + Implementations of HTTP origin servers SHOULD be careful to restrict + the documents returned by HTTP requests to be only those that were + intended by the server administrators. If an HTTP server translates + HTTP URIs directly into file system calls, the server MUST take + special care not to serve files that were not intended to be + delivered to HTTP clients. For example, UNIX, Microsoft Windows, and + other operating systems use ".." as a path component to indicate a + directory level above the current one. On such a system, an HTTP + server MUST disallow any such construct in the Request-URI if it + would otherwise allow access to a resource outside those intended to + be accessible via the HTTP server. Similarly, files intended for + reference only internally to the server (such as access control + files, configuration files, and script code) MUST be protected from + inappropriate retrieval, since they might contain sensitive + information. Experience has shown that minor bugs in such HTTP server + implementations have turned into security risks. + + + + +Fielding, et al. Standards Track [Page 153] + +RFC 2616 HTTP/1.1 June 1999 + + +15.3 DNS Spoofing + + Clients using HTTP rely heavily on the Domain Name Service, and are + thus generally prone to security attacks based on the deliberate + mis-association of IP addresses and DNS names. Clients need to be + cautious in assuming the continuing validity of an IP number/DNS name + association. + + In particular, HTTP clients SHOULD rely on their name resolver for + confirmation of an IP number/DNS name association, rather than + caching the result of previous host name lookups. Many platforms + already can cache host name lookups locally when appropriate, and + they SHOULD be configured to do so. It is proper for these lookups to + be cached, however, only when the TTL (Time To Live) information + reported by the name server makes it likely that the cached + information will remain useful. + + If HTTP clients cache the results of host name lookups in order to + achieve a performance improvement, they MUST observe the TTL + information reported by DNS. + + If HTTP clients do not observe this rule, they could be spoofed when + a previously-accessed server's IP address changes. As network + renumbering is expected to become increasingly common [24], the + possibility of this form of attack will grow. Observing this + requirement thus reduces this potential security vulnerability. + + This requirement also improves the load-balancing behavior of clients + for replicated servers using the same DNS name and reduces the + likelihood of a user's experiencing failure in accessing sites which + use that strategy. + +15.4 Location Headers and Spoofing + + If a single server supports multiple organizations that do not trust + one another, then it MUST check the values of Location and Content- + Location headers in responses that are generated under control of + said organizations to make sure that they do not attempt to + invalidate resources over which they have no authority. + +15.5 Content-Disposition Issues + + RFC 1806 [35], from which the often implemented Content-Disposition + (see section 19.5.1) header in HTTP is derived, has a number of very + serious security considerations. Content-Disposition is not part of + the HTTP standard, but since it is widely implemented, we are + documenting its use and risks for implementors. See RFC 2183 [49] + (which updates RFC 1806) for details. + + + +Fielding, et al. Standards Track [Page 154] + +RFC 2616 HTTP/1.1 June 1999 + + +15.6 Authentication Credentials and Idle Clients + + Existing HTTP clients and user agents typically retain authentication + information indefinitely. HTTP/1.1. does not provide a method for a + server to direct clients to discard these cached credentials. This is + a significant defect that requires further extensions to HTTP. + Circumstances under which credential caching can interfere with the + application's security model include but are not limited to: + + - Clients which have been idle for an extended period following + which the server might wish to cause the client to reprompt the + user for credentials. + + - Applications which include a session termination indication + (such as a `logout' or `commit' button on a page) after which + the server side of the application `knows' that there is no + further reason for the client to retain the credentials. + + This is currently under separate study. There are a number of work- + arounds to parts of this problem, and we encourage the use of + password protection in screen savers, idle time-outs, and other + methods which mitigate the security problems inherent in this + problem. In particular, user agents which cache credentials are + encouraged to provide a readily accessible mechanism for discarding + cached credentials under user control. + +15.7 Proxies and Caching + + By their very nature, HTTP proxies are men-in-the-middle, and + represent an opportunity for man-in-the-middle attacks. Compromise of + the systems on which the proxies run can result in serious security + and privacy problems. Proxies have access to security-related + information, personal information about individual users and + organizations, and proprietary information belonging to users and + content providers. A compromised proxy, or a proxy implemented or + configured without regard to security and privacy considerations, + might be used in the commission of a wide range of potential attacks. + + Proxy operators should protect the systems on which proxies run as + they would protect any system that contains or transports sensitive + information. In particular, log information gathered at proxies often + contains highly sensitive personal information, and/or information + about organizations. Log information should be carefully guarded, and + appropriate guidelines for use developed and followed. (Section + 15.1.1). + + + + + + +Fielding, et al. Standards Track [Page 155] + +RFC 2616 HTTP/1.1 June 1999 + + + Caching proxies provide additional potential vulnerabilities, since + the contents of the cache represent an attractive target for + malicious exploitation. Because cache contents persist after an HTTP + request is complete, an attack on the cache can reveal information + long after a user believes that the information has been removed from + the network. Therefore, cache contents should be protected as + sensitive information. + + Proxy implementors should consider the privacy and security + implications of their design and coding decisions, and of the + configuration options they provide to proxy operators (especially the + default configuration). + + Users of a proxy need to be aware that they are no trustworthier than + the people who run the proxy; HTTP itself cannot solve this problem. + + The judicious use of cryptography, when appropriate, may suffice to + protect against a broad range of security and privacy attacks. Such + cryptography is beyond the scope of the HTTP/1.1 specification. + +15.7.1 Denial of Service Attacks on Proxies + + They exist. They are hard to defend against. Research continues. + Beware. + +16 Acknowledgments + + This specification makes heavy use of the augmented BNF and generic + constructs defined by David H. Crocker for RFC 822 [9]. Similarly, it + reuses many of the definitions provided by Nathaniel Borenstein and + Ned Freed for MIME [7]. We hope that their inclusion in this + specification will help reduce past confusion over the relationship + between HTTP and Internet mail message formats. + + The HTTP protocol has evolved considerably over the years. It has + benefited from a large and active developer community--the many + people who have participated on the www-talk mailing list--and it is + that community which has been most responsible for the success of + HTTP and of the World-Wide Web in general. Marc Andreessen, Robert + Cailliau, Daniel W. Connolly, Bob Denny, John Franks, Jean-Francois + Groff, Phillip M. Hallam-Baker, Hakon W. Lie, Ari Luotonen, Rob + McCool, Lou Montulli, Dave Raggett, Tony Sanders, and Marc + VanHeyningen deserve special recognition for their efforts in + defining early aspects of the protocol. + + This document has benefited greatly from the comments of all those + participating in the HTTP-WG. In addition to those already mentioned, + the following individuals have contributed to this specification: + + + +Fielding, et al. Standards Track [Page 156] + +RFC 2616 HTTP/1.1 June 1999 + + + Gary Adams Ross Patterson + Harald Tveit Alvestrand Albert Lunde + Keith Ball John C. Mallery + Brian Behlendorf Jean-Philippe Martin-Flatin + Paul Burchard Mitra + Maurizio Codogno David Morris + Mike Cowlishaw Gavin Nicol + Roman Czyborra Bill Perry + Michael A. Dolan Jeffrey Perry + David J. Fiander Scott Powers + Alan Freier Owen Rees + Marc Hedlund Luigi Rizzo + Greg Herlihy David Robinson + Koen Holtman Marc Salomon + Alex Hopmann Rich Salz + Bob Jernigan Allan M. Schiffman + Shel Kaphan Jim Seidman + Rohit Khare Chuck Shotton + John Klensin Eric W. Sink + Martijn Koster Simon E. Spero + Alexei Kosut Richard N. Taylor + David M. Kristol Robert S. Thau + Daniel LaLiberte Bill (BearHeart) Weinman + Ben Laurie Francois Yergeau + Paul J. Leach Mary Ellen Zurko + Daniel DuBois Josh Cohen + + + Much of the content and presentation of the caching design is due to + suggestions and comments from individuals including: Shel Kaphan, + Paul Leach, Koen Holtman, David Morris, and Larry Masinter. + + Most of the specification of ranges is based on work originally done + by Ari Luotonen and John Franks, with additional input from Steve + Zilles. + + Thanks to the "cave men" of Palo Alto. You know who you are. + + Jim Gettys (the current editor of this document) wishes particularly + to thank Roy Fielding, the previous editor of this document, along + with John Klensin, Jeff Mogul, Paul Leach, Dave Kristol, Koen + Holtman, John Franks, Josh Cohen, Alex Hopmann, Scott Lawrence, and + Larry Masinter for their help. And thanks go particularly to Jeff + Mogul and Scott Lawrence for performing the "MUST/MAY/SHOULD" audit. + + + + + + + +Fielding, et al. Standards Track [Page 157] + +RFC 2616 HTTP/1.1 June 1999 + + + The Apache Group, Anselm Baird-Smith, author of Jigsaw, and Henrik + Frystyk implemented RFC 2068 early, and we wish to thank them for the + discovery of many of the problems that this document attempts to + rectify. + +17 References + + [1] Alvestrand, H., "Tags for the Identification of Languages", RFC + 1766, March 1995. + + [2] Anklesaria, F., McCahill, M., Lindner, P., Johnson, D., Torrey, + D. and B. Alberti, "The Internet Gopher Protocol (a distributed + document search and retrieval protocol)", RFC 1436, March 1993. + + [3] Berners-Lee, T., "Universal Resource Identifiers in WWW", RFC + 1630, June 1994. + + [4] Berners-Lee, T., Masinter, L. and M. McCahill, "Uniform Resource + Locators (URL)", RFC 1738, December 1994. + + [5] Berners-Lee, T. and D. Connolly, "Hypertext Markup Language - + 2.0", RFC 1866, November 1995. + + [6] Berners-Lee, T., Fielding, R. and H. Frystyk, "Hypertext Transfer + Protocol -- HTTP/1.0", RFC 1945, May 1996. + + [7] Freed, N. and N. Borenstein, "Multipurpose Internet Mail + Extensions (MIME) Part One: Format of Internet Message Bodies", + RFC 2045, November 1996. + + [8] Braden, R., "Requirements for Internet Hosts -- Communication + Layers", STD 3, RFC 1123, October 1989. + + [9] Crocker, D., "Standard for The Format of ARPA Internet Text + Messages", STD 11, RFC 822, August 1982. + + [10] Davis, F., Kahle, B., Morris, H., Salem, J., Shen, T., Wang, R., + Sui, J., and M. Grinbaum, "WAIS Interface Protocol Prototype + Functional Specification," (v1.5), Thinking Machines + Corporation, April 1990. + + [11] Fielding, R., "Relative Uniform Resource Locators", RFC 1808, + June 1995. + + [12] Horton, M. and R. Adams, "Standard for Interchange of USENET + Messages", RFC 1036, December 1987. + + + + + +Fielding, et al. Standards Track [Page 158] + +RFC 2616 HTTP/1.1 June 1999 + + + [13] Kantor, B. and P. Lapsley, "Network News Transfer Protocol", RFC + 977, February 1986. + + [14] Moore, K., "MIME (Multipurpose Internet Mail Extensions) Part + Three: Message Header Extensions for Non-ASCII Text", RFC 2047, + November 1996. + + [15] Nebel, E. and L. Masinter, "Form-based File Upload in HTML", RFC + 1867, November 1995. + + [16] Postel, J., "Simple Mail Transfer Protocol", STD 10, RFC 821, + August 1982. + + [17] Postel, J., "Media Type Registration Procedure", RFC 1590, + November 1996. + + [18] Postel, J. and J. Reynolds, "File Transfer Protocol", STD 9, RFC + 959, October 1985. + + [19] Reynolds, J. and J. Postel, "Assigned Numbers", STD 2, RFC 1700, + October 1994. + + [20] Sollins, K. and L. Masinter, "Functional Requirements for + Uniform Resource Names", RFC 1737, December 1994. + + [21] US-ASCII. Coded Character Set - 7-Bit American Standard Code for + Information Interchange. Standard ANSI X3.4-1986, ANSI, 1986. + + [22] ISO-8859. International Standard -- Information Processing -- + 8-bit Single-Byte Coded Graphic Character Sets -- + Part 1: Latin alphabet No. 1, ISO-8859-1:1987. + Part 2: Latin alphabet No. 2, ISO-8859-2, 1987. + Part 3: Latin alphabet No. 3, ISO-8859-3, 1988. + Part 4: Latin alphabet No. 4, ISO-8859-4, 1988. + Part 5: Latin/Cyrillic alphabet, ISO-8859-5, 1988. + Part 6: Latin/Arabic alphabet, ISO-8859-6, 1987. + Part 7: Latin/Greek alphabet, ISO-8859-7, 1987. + Part 8: Latin/Hebrew alphabet, ISO-8859-8, 1988. + Part 9: Latin alphabet No. 5, ISO-8859-9, 1990. + + [23] Meyers, J. and M. Rose, "The Content-MD5 Header Field", RFC + 1864, October 1995. + + [24] Carpenter, B. and Y. Rekhter, "Renumbering Needs Work", RFC + 1900, February 1996. + + [25] Deutsch, P., "GZIP file format specification version 4.3", RFC + 1952, May 1996. + + + +Fielding, et al. Standards Track [Page 159] + +RFC 2616 HTTP/1.1 June 1999 + + + [26] Venkata N. Padmanabhan, and Jeffrey C. Mogul. "Improving HTTP + Latency", Computer Networks and ISDN Systems, v. 28, pp. 25-35, + Dec. 1995. Slightly revised version of paper in Proc. 2nd + International WWW Conference '94: Mosaic and the Web, Oct. 1994, + which is available at + http://www.ncsa.uiuc.edu/SDG/IT94/Proceedings/DDay/mogul/HTTPLat + ency.html. + + [27] Joe Touch, John Heidemann, and Katia Obraczka. "Analysis of HTTP + Performance", <URL: http://www.isi.edu/touch/pubs/http-perf96/>, + ISI Research Report ISI/RR-98-463, (original report dated Aug. + 1996), USC/Information Sciences Institute, August 1998. + + [28] Mills, D., "Network Time Protocol (Version 3) Specification, + Implementation and Analysis", RFC 1305, March 1992. + + [29] Deutsch, P., "DEFLATE Compressed Data Format Specification + version 1.3", RFC 1951, May 1996. + + [30] S. Spero, "Analysis of HTTP Performance Problems," + http://sunsite.unc.edu/mdma-release/http-prob.html. + + [31] Deutsch, P. and J. Gailly, "ZLIB Compressed Data Format + Specification version 3.3", RFC 1950, May 1996. + + [32] Franks, J., Hallam-Baker, P., Hostetler, J., Leach, P., + Luotonen, A., Sink, E. and L. Stewart, "An Extension to HTTP: + Digest Access Authentication", RFC 2069, January 1997. + + [33] Fielding, R., Gettys, J., Mogul, J., Frystyk, H. and T. + Berners-Lee, "Hypertext Transfer Protocol -- HTTP/1.1", RFC + 2068, January 1997. + + [34] Bradner, S., "Key words for use in RFCs to Indicate Requirement + Levels", BCP 14, RFC 2119, March 1997. + + [35] Troost, R. and Dorner, S., "Communicating Presentation + Information in Internet Messages: The Content-Disposition + Header", RFC 1806, June 1995. + + [36] Mogul, J., Fielding, R., Gettys, J. and H. Frystyk, "Use and + Interpretation of HTTP Version Numbers", RFC 2145, May 1997. + [jg639] + + [37] Palme, J., "Common Internet Message Headers", RFC 2076, February + 1997. [jg640] + + + + + +Fielding, et al. Standards Track [Page 160] + +RFC 2616 HTTP/1.1 June 1999 + + + [38] Yergeau, F., "UTF-8, a transformation format of Unicode and + ISO-10646", RFC 2279, January 1998. [jg641] + + [39] Nielsen, H.F., Gettys, J., Baird-Smith, A., Prud'hommeaux, E., + Lie, H., and C. Lilley. "Network Performance Effects of + HTTP/1.1, CSS1, and PNG," Proceedings of ACM SIGCOMM '97, Cannes + France, September 1997.[jg642] + + [40] Freed, N. and N. Borenstein, "Multipurpose Internet Mail + Extensions (MIME) Part Two: Media Types", RFC 2046, November + 1996. [jg643] + + [41] Alvestrand, H., "IETF Policy on Character Sets and Languages", + BCP 18, RFC 2277, January 1998. [jg644] + + [42] Berners-Lee, T., Fielding, R. and L. Masinter, "Uniform Resource + Identifiers (URI): Generic Syntax and Semantics", RFC 2396, + August 1998. [jg645] + + [43] Franks, J., Hallam-Baker, P., Hostetler, J., Lawrence, S., + Leach, P., Luotonen, A., Sink, E. and L. Stewart, "HTTP + Authentication: Basic and Digest Access Authentication", RFC + 2617, June 1999. [jg646] + + [44] Luotonen, A., "Tunneling TCP based protocols through Web proxy + servers," Work in Progress. [jg647] + + [45] Palme, J. and A. Hopmann, "MIME E-mail Encapsulation of + Aggregate Documents, such as HTML (MHTML)", RFC 2110, March + 1997. + + [46] Bradner, S., "The Internet Standards Process -- Revision 3", BCP + 9, RFC 2026, October 1996. + + [47] Masinter, L., "Hyper Text Coffee Pot Control Protocol + (HTCPCP/1.0)", RFC 2324, 1 April 1998. + + [48] Freed, N. and N. Borenstein, "Multipurpose Internet Mail + Extensions (MIME) Part Five: Conformance Criteria and Examples", + RFC 2049, November 1996. + + [49] Troost, R., Dorner, S. and K. Moore, "Communicating Presentation + Information in Internet Messages: The Content-Disposition Header + Field", RFC 2183, August 1997. + + + + + + + +Fielding, et al. Standards Track [Page 161] + +RFC 2616 HTTP/1.1 June 1999 + + +18 Authors' Addresses + + Roy T. Fielding + Information and Computer Science + University of California, Irvine + Irvine, CA 92697-3425, USA + + Fax: +1 (949) 824-1715 + EMail: [email protected] + + + James Gettys + World Wide Web Consortium + MIT Laboratory for Computer Science + 545 Technology Square + Cambridge, MA 02139, USA + + Fax: +1 (617) 258 8682 + EMail: [email protected] + + + Jeffrey C. Mogul + Western Research Laboratory + Compaq Computer Corporation + 250 University Avenue + Palo Alto, California, 94305, USA + + EMail: [email protected] + + + Henrik Frystyk Nielsen + World Wide Web Consortium + MIT Laboratory for Computer Science + 545 Technology Square + Cambridge, MA 02139, USA + + Fax: +1 (617) 258 8682 + EMail: [email protected] + + + Larry Masinter + Xerox Corporation + 3333 Coyote Hill Road + Palo Alto, CA 94034, USA + + EMail: [email protected] + + + + + +Fielding, et al. Standards Track [Page 162] + +RFC 2616 HTTP/1.1 June 1999 + + + Paul J. Leach + Microsoft Corporation + 1 Microsoft Way + Redmond, WA 98052, USA + + EMail: [email protected] + + + Tim Berners-Lee + Director, World Wide Web Consortium + MIT Laboratory for Computer Science + 545 Technology Square + Cambridge, MA 02139, USA + + Fax: +1 (617) 258 8682 + EMail: [email protected] + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + +Fielding, et al. Standards Track [Page 163] + +RFC 2616 HTTP/1.1 June 1999 + + +19 Appendices + +19.1 Internet Media Type message/http and application/http + + In addition to defining the HTTP/1.1 protocol, this document serves + as the specification for the Internet media type "message/http" and + "application/http". The message/http type can be used to enclose a + single HTTP request or response message, provided that it obeys the + MIME restrictions for all "message" types regarding line length and + encodings. The application/http type can be used to enclose a + pipeline of one or more HTTP request or response messages (not + intermixed). The following is to be registered with IANA [17]. + + Media Type name: message + Media subtype name: http + Required parameters: none + Optional parameters: version, msgtype + version: The HTTP-Version number of the enclosed message + (e.g., "1.1"). If not present, the version can be + determined from the first line of the body. + msgtype: The message type -- "request" or "response". If not + present, the type can be determined from the first + line of the body. + Encoding considerations: only "7bit", "8bit", or "binary" are + permitted + Security considerations: none + + Media Type name: application + Media subtype name: http + Required parameters: none + Optional parameters: version, msgtype + version: The HTTP-Version number of the enclosed messages + (e.g., "1.1"). If not present, the version can be + determined from the first line of the body. + msgtype: The message type -- "request" or "response". If not + present, the type can be determined from the first + line of the body. + Encoding considerations: HTTP messages enclosed by this type + are in "binary" format; use of an appropriate + Content-Transfer-Encoding is required when + transmitted via E-mail. + Security considerations: none + + + + + + + + + +Fielding, et al. Standards Track [Page 164] + +RFC 2616 HTTP/1.1 June 1999 + + +19.2 Internet Media Type multipart/byteranges + + When an HTTP 206 (Partial Content) response message includes the + content of multiple ranges (a response to a request for multiple + non-overlapping ranges), these are transmitted as a multipart + message-body. The media type for this purpose is called + "multipart/byteranges". + + The multipart/byteranges media type includes two or more parts, each + with its own Content-Type and Content-Range fields. The required + boundary parameter specifies the boundary string used to separate + each body-part. + + Media Type name: multipart + Media subtype name: byteranges + Required parameters: boundary + Optional parameters: none + Encoding considerations: only "7bit", "8bit", or "binary" are + permitted + Security considerations: none + + + For example: + + HTTP/1.1 206 Partial Content + Date: Wed, 15 Nov 1995 06:25:24 GMT + Last-Modified: Wed, 15 Nov 1995 04:58:08 GMT + Content-type: multipart/byteranges; boundary=THIS_STRING_SEPARATES + + --THIS_STRING_SEPARATES + Content-type: application/pdf + Content-range: bytes 500-999/8000 + + ...the first range... + --THIS_STRING_SEPARATES + Content-type: application/pdf + Content-range: bytes 7000-7999/8000 + + ...the second range + --THIS_STRING_SEPARATES-- + + Notes: + + 1) Additional CRLFs may precede the first boundary string in the + entity. + + + + + + +Fielding, et al. Standards Track [Page 165] + +RFC 2616 HTTP/1.1 June 1999 + + + 2) Although RFC 2046 [40] permits the boundary string to be + quoted, some existing implementations handle a quoted boundary + string incorrectly. + + 3) A number of browsers and servers were coded to an early draft + of the byteranges specification to use a media type of + multipart/x-byteranges, which is almost, but not quite + compatible with the version documented in HTTP/1.1. + +19.3 Tolerant Applications + + Although this document specifies the requirements for the generation + of HTTP/1.1 messages, not all applications will be correct in their + implementation. We therefore recommend that operational applications + be tolerant of deviations whenever those deviations can be + interpreted unambiguously. + + Clients SHOULD be tolerant in parsing the Status-Line and servers + tolerant when parsing the Request-Line. In particular, they SHOULD + accept any amount of SP or HT characters between fields, even though + only a single SP is required. + + The line terminator for message-header fields is the sequence CRLF. + However, we recommend that applications, when parsing such headers, + recognize a single LF as a line terminator and ignore the leading CR. + + The character set of an entity-body SHOULD be labeled as the lowest + common denominator of the character codes used within that body, with + the exception that not labeling the entity is preferred over labeling + the entity with the labels US-ASCII or ISO-8859-1. See section 3.7.1 + and 3.4.1. + + Additional rules for requirements on parsing and encoding of dates + and other potential problems with date encodings include: + + - HTTP/1.1 clients and caches SHOULD assume that an RFC-850 date + which appears to be more than 50 years in the future is in fact + in the past (this helps solve the "year 2000" problem). + + - An HTTP/1.1 implementation MAY internally represent a parsed + Expires date as earlier than the proper value, but MUST NOT + internally represent a parsed Expires date as later than the + proper value. + + - All expiration-related calculations MUST be done in GMT. The + local time zone MUST NOT influence the calculation or comparison + of an age or expiration time. + + + + +Fielding, et al. Standards Track [Page 166] + +RFC 2616 HTTP/1.1 June 1999 + + + - If an HTTP header incorrectly carries a date value with a time + zone other than GMT, it MUST be converted into GMT using the + most conservative possible conversion. + +19.4 Differences Between HTTP Entities and RFC 2045 Entities + + HTTP/1.1 uses many of the constructs defined for Internet Mail (RFC + 822 [9]) and the Multipurpose Internet Mail Extensions (MIME [7]) to + allow entities to be transmitted in an open variety of + representations and with extensible mechanisms. However, RFC 2045 + discusses mail, and HTTP has a few features that are different from + those described in RFC 2045. These differences were carefully chosen + to optimize performance over binary connections, to allow greater + freedom in the use of new media types, to make date comparisons + easier, and to acknowledge the practice of some early HTTP servers + and clients. + + This appendix describes specific areas where HTTP differs from RFC + 2045. Proxies and gateways to strict MIME environments SHOULD be + aware of these differences and provide the appropriate conversions + where necessary. Proxies and gateways from MIME environments to HTTP + also need to be aware of the differences because some conversions + might be required. + +19.4.1 MIME-Version + + HTTP is not a MIME-compliant protocol. However, HTTP/1.1 messages MAY + include a single MIME-Version general-header field to indicate what + version of the MIME protocol was used to construct the message. Use + of the MIME-Version header field indicates that the message is in + full compliance with the MIME protocol (as defined in RFC 2045[7]). + Proxies/gateways are responsible for ensuring full compliance (where + possible) when exporting HTTP messages to strict MIME environments. + + MIME-Version = "MIME-Version" ":" 1*DIGIT "." 1*DIGIT + + MIME version "1.0" is the default for use in HTTP/1.1. However, + HTTP/1.1 message parsing and semantics are defined by this document + and not the MIME specification. + +19.4.2 Conversion to Canonical Form + + RFC 2045 [7] requires that an Internet mail entity be converted to + canonical form prior to being transferred, as described in section 4 + of RFC 2049 [48]. Section 3.7.1 of this document describes the forms + allowed for subtypes of the "text" media type when transmitted over + HTTP. RFC 2046 requires that content with a type of "text" represent + line breaks as CRLF and forbids the use of CR or LF outside of line + + + +Fielding, et al. Standards Track [Page 167] + +RFC 2616 HTTP/1.1 June 1999 + + + break sequences. HTTP allows CRLF, bare CR, and bare LF to indicate a + line break within text content when a message is transmitted over + HTTP. + + Where it is possible, a proxy or gateway from HTTP to a strict MIME + environment SHOULD translate all line breaks within the text media + types described in section 3.7.1 of this document to the RFC 2049 + canonical form of CRLF. Note, however, that this might be complicated + by the presence of a Content-Encoding and by the fact that HTTP + allows the use of some character sets which do not use octets 13 and + 10 to represent CR and LF, as is the case for some multi-byte + character sets. + + Implementors should note that conversion will break any cryptographic + checksums applied to the original content unless the original content + is already in canonical form. Therefore, the canonical form is + recommended for any content that uses such checksums in HTTP. + +19.4.3 Conversion of Date Formats + + HTTP/1.1 uses a restricted set of date formats (section 3.3.1) to + simplify the process of date comparison. Proxies and gateways from + other protocols SHOULD ensure that any Date header field present in a + message conforms to one of the HTTP/1.1 formats and rewrite the date + if necessary. + +19.4.4 Introduction of Content-Encoding + + RFC 2045 does not include any concept equivalent to HTTP/1.1's + Content-Encoding header field. Since this acts as a modifier on the + media type, proxies and gateways from HTTP to MIME-compliant + protocols MUST either change the value of the Content-Type header + field or decode the entity-body before forwarding the message. (Some + experimental applications of Content-Type for Internet mail have used + a media-type parameter of ";conversions=<content-coding>" to perform + a function equivalent to Content-Encoding. However, this parameter is + not part of RFC 2045.) + +19.4.5 No Content-Transfer-Encoding + + HTTP does not use the Content-Transfer-Encoding (CTE) field of RFC + 2045. Proxies and gateways from MIME-compliant protocols to HTTP MUST + remove any non-identity CTE ("quoted-printable" or "base64") encoding + prior to delivering the response message to an HTTP client. + + Proxies and gateways from HTTP to MIME-compliant protocols are + responsible for ensuring that the message is in the correct format + and encoding for safe transport on that protocol, where "safe + + + +Fielding, et al. Standards Track [Page 168] + +RFC 2616 HTTP/1.1 June 1999 + + + transport" is defined by the limitations of the protocol being used. + Such a proxy or gateway SHOULD label the data with an appropriate + Content-Transfer-Encoding if doing so will improve the likelihood of + safe transport over the destination protocol. + +19.4.6 Introduction of Transfer-Encoding + + HTTP/1.1 introduces the Transfer-Encoding header field (section + 14.41). Proxies/gateways MUST remove any transfer-coding prior to + forwarding a message via a MIME-compliant protocol. + + A process for decoding the "chunked" transfer-coding (section 3.6) + can be represented in pseudo-code as: + + length := 0 + read chunk-size, chunk-extension (if any) and CRLF + while (chunk-size > 0) { + read chunk-data and CRLF + append chunk-data to entity-body + length := length + chunk-size + read chunk-size and CRLF + } + read entity-header + while (entity-header not empty) { + append entity-header to existing header fields + read entity-header + } + Content-Length := length + Remove "chunked" from Transfer-Encoding + +19.4.7 MHTML and Line Length Limitations + + HTTP implementations which share code with MHTML [45] implementations + need to be aware of MIME line length limitations. Since HTTP does not + have this limitation, HTTP does not fold long lines. MHTML messages + being transported by HTTP follow all conventions of MHTML, including + line length limitations and folding, canonicalization, etc., since + HTTP transports all message-bodies as payload (see section 3.7.2) and + does not interpret the content or any MIME header lines that might be + contained therein. + +19.5 Additional Features + + RFC 1945 and RFC 2068 document protocol elements used by some + existing HTTP implementations, but not consistently and correctly + across most HTTP/1.1 applications. Implementors are advised to be + aware of these features, but cannot rely upon their presence in, or + interoperability with, other HTTP/1.1 applications. Some of these + + + +Fielding, et al. Standards Track [Page 169] + +RFC 2616 HTTP/1.1 June 1999 + + + describe proposed experimental features, and some describe features + that experimental deployment found lacking that are now addressed in + the base HTTP/1.1 specification. + + A number of other headers, such as Content-Disposition and Title, + from SMTP and MIME are also often implemented (see RFC 2076 [37]). + +19.5.1 Content-Disposition + + The Content-Disposition response-header field has been proposed as a + means for the origin server to suggest a default filename if the user + requests that the content is saved to a file. This usage is derived + from the definition of Content-Disposition in RFC 1806 [35]. + + content-disposition = "Content-Disposition" ":" + disposition-type *( ";" disposition-parm ) + disposition-type = "attachment" | disp-extension-token + disposition-parm = filename-parm | disp-extension-parm + filename-parm = "filename" "=" quoted-string + disp-extension-token = token + disp-extension-parm = token "=" ( token | quoted-string ) + + An example is + + Content-Disposition: attachment; filename="fname.ext" + + The receiving user agent SHOULD NOT respect any directory path + information present in the filename-parm parameter, which is the only + parameter believed to apply to HTTP implementations at this time. The + filename SHOULD be treated as a terminal component only. + + If this header is used in a response with the application/octet- + stream content-type, the implied suggestion is that the user agent + should not display the response, but directly enter a `save response + as...' dialog. + + See section 15.5 for Content-Disposition security issues. + +19.6 Compatibility with Previous Versions + + It is beyond the scope of a protocol specification to mandate + compliance with previous versions. HTTP/1.1 was deliberately + designed, however, to make supporting previous versions easy. It is + worth noting that, at the time of composing this specification + (1996), we would expect commercial HTTP/1.1 servers to: + + - recognize the format of the Request-Line for HTTP/0.9, 1.0, and + 1.1 requests; + + + +Fielding, et al. Standards Track [Page 170] + +RFC 2616 HTTP/1.1 June 1999 + + + - understand any valid request in the format of HTTP/0.9, 1.0, or + 1.1; + + - respond appropriately with a message in the same major version + used by the client. + + And we would expect HTTP/1.1 clients to: + + - recognize the format of the Status-Line for HTTP/1.0 and 1.1 + responses; + + - understand any valid response in the format of HTTP/0.9, 1.0, or + 1.1. + + For most implementations of HTTP/1.0, each connection is established + by the client prior to the request and closed by the server after + sending the response. Some implementations implement the Keep-Alive + version of persistent connections described in section 19.7.1 of RFC + 2068 [33]. + +19.6.1 Changes from HTTP/1.0 + + This section summarizes major differences between versions HTTP/1.0 + and HTTP/1.1. + +19.6.1.1 Changes to Simplify Multi-homed Web Servers and Conserve IP + Addresses + + The requirements that clients and servers support the Host request- + header, report an error if the Host request-header (section 14.23) is + missing from an HTTP/1.1 request, and accept absolute URIs (section + 5.1.2) are among the most important changes defined by this + specification. + + Older HTTP/1.0 clients assumed a one-to-one relationship of IP + addresses and servers; there was no other established mechanism for + distinguishing the intended server of a request than the IP address + to which that request was directed. The changes outlined above will + allow the Internet, once older HTTP clients are no longer common, to + support multiple Web sites from a single IP address, greatly + simplifying large operational Web servers, where allocation of many + IP addresses to a single host has created serious problems. The + Internet will also be able to recover the IP addresses that have been + allocated for the sole purpose of allowing special-purpose domain + names to be used in root-level HTTP URLs. Given the rate of growth of + the Web, and the number of servers already deployed, it is extremely + + + + + +Fielding, et al. Standards Track [Page 171] + +RFC 2616 HTTP/1.1 June 1999 + + + important that all implementations of HTTP (including updates to + existing HTTP/1.0 applications) correctly implement these + requirements: + + - Both clients and servers MUST support the Host request-header. + + - A client that sends an HTTP/1.1 request MUST send a Host header. + + - Servers MUST report a 400 (Bad Request) error if an HTTP/1.1 + request does not include a Host request-header. + + - Servers MUST accept absolute URIs. + +19.6.2 Compatibility with HTTP/1.0 Persistent Connections + + Some clients and servers might wish to be compatible with some + previous implementations of persistent connections in HTTP/1.0 + clients and servers. Persistent connections in HTTP/1.0 are + explicitly negotiated as they are not the default behavior. HTTP/1.0 + experimental implementations of persistent connections are faulty, + and the new facilities in HTTP/1.1 are designed to rectify these + problems. The problem was that some existing 1.0 clients may be + sending Keep-Alive to a proxy server that doesn't understand + Connection, which would then erroneously forward it to the next + inbound server, which would establish the Keep-Alive connection and + result in a hung HTTP/1.0 proxy waiting for the close on the + response. The result is that HTTP/1.0 clients must be prevented from + using Keep-Alive when talking to proxies. + + However, talking to proxies is the most important use of persistent + connections, so that prohibition is clearly unacceptable. Therefore, + we need some other mechanism for indicating a persistent connection + is desired, which is safe to use even when talking to an old proxy + that ignores Connection. Persistent connections are the default for + HTTP/1.1 messages; we introduce a new keyword (Connection: close) for + declaring non-persistence. See section 14.10. + + The original HTTP/1.0 form of persistent connections (the Connection: + Keep-Alive and Keep-Alive header) is documented in RFC 2068. [33] + +19.6.3 Changes from RFC 2068 + + This specification has been carefully audited to correct and + disambiguate key word usage; RFC 2068 had many problems in respect to + the conventions laid out in RFC 2119 [34]. + + Clarified which error code should be used for inbound server failures + (e.g. DNS failures). (Section 10.5.5). + + + +Fielding, et al. Standards Track [Page 172] + +RFC 2616 HTTP/1.1 June 1999 + + + CREATE had a race that required an Etag be sent when a resource is + first created. (Section 10.2.2). + + Content-Base was deleted from the specification: it was not + implemented widely, and there is no simple, safe way to introduce it + without a robust extension mechanism. In addition, it is used in a + similar, but not identical fashion in MHTML [45]. + + Transfer-coding and message lengths all interact in ways that + required fixing exactly when chunked encoding is used (to allow for + transfer encoding that may not be self delimiting); it was important + to straighten out exactly how message lengths are computed. (Sections + 3.6, 4.4, 7.2.2, 13.5.2, 14.13, 14.16) + + A content-coding of "identity" was introduced, to solve problems + discovered in caching. (section 3.5) + + Quality Values of zero should indicate that "I don't want something" + to allow clients to refuse a representation. (Section 3.9) + + The use and interpretation of HTTP version numbers has been clarified + by RFC 2145. Require proxies to upgrade requests to highest protocol + version they support to deal with problems discovered in HTTP/1.0 + implementations (Section 3.1) + + Charset wildcarding is introduced to avoid explosion of character set + names in accept headers. (Section 14.2) + + A case was missed in the Cache-Control model of HTTP/1.1; s-maxage + was introduced to add this missing case. (Sections 13.4, 14.8, 14.9, + 14.9.3) + + The Cache-Control: max-age directive was not properly defined for + responses. (Section 14.9.3) + + There are situations where a server (especially a proxy) does not + know the full length of a response but is capable of serving a + byterange request. We therefore need a mechanism to allow byteranges + with a content-range not indicating the full length of the message. + (Section 14.16) + + Range request responses would become very verbose if all meta-data + were always returned; by allowing the server to only send needed + headers in a 206 response, this problem can be avoided. (Section + 10.2.7, 13.5.3, and 14.27) + + + + + + +Fielding, et al. Standards Track [Page 173] + +RFC 2616 HTTP/1.1 June 1999 + + + Fix problem with unsatisfiable range requests; there are two cases: + syntactic problems, and range doesn't exist in the document. The 416 + status code was needed to resolve this ambiguity needed to indicate + an error for a byte range request that falls outside of the actual + contents of a document. (Section 10.4.17, 14.16) + + Rewrite of message transmission requirements to make it much harder + for implementors to get it wrong, as the consequences of errors here + can have significant impact on the Internet, and to deal with the + following problems: + + 1. Changing "HTTP/1.1 or later" to "HTTP/1.1", in contexts where + this was incorrectly placing a requirement on the behavior of + an implementation of a future version of HTTP/1.x + + 2. Made it clear that user-agents should retry requests, not + "clients" in general. + + 3. Converted requirements for clients to ignore unexpected 100 + (Continue) responses, and for proxies to forward 100 responses, + into a general requirement for 1xx responses. + + 4. Modified some TCP-specific language, to make it clearer that + non-TCP transports are possible for HTTP. + + 5. Require that the origin server MUST NOT wait for the request + body before it sends a required 100 (Continue) response. + + 6. Allow, rather than require, a server to omit 100 (Continue) if + it has already seen some of the request body. + + 7. Allow servers to defend against denial-of-service attacks and + broken clients. + + This change adds the Expect header and 417 status code. The message + transmission requirements fixes are in sections 8.2, 10.4.18, + 8.1.2.2, 13.11, and 14.20. + + Proxies should be able to add Content-Length when appropriate. + (Section 13.5.2) + + Clean up confusion between 403 and 404 responses. (Section 10.4.4, + 10.4.5, and 10.4.11) + + Warnings could be cached incorrectly, or not updated appropriately. + (Section 13.1.2, 13.2.4, 13.5.2, 13.5.3, 14.9.3, and 14.46) Warning + also needed to be a general header, as PUT or other methods may have + need for it in requests. + + + +Fielding, et al. Standards Track [Page 174] + +RFC 2616 HTTP/1.1 June 1999 + + + Transfer-coding had significant problems, particularly with + interactions with chunked encoding. The solution is that transfer- + codings become as full fledged as content-codings. This involves + adding an IANA registry for transfer-codings (separate from content + codings), a new header field (TE) and enabling trailer headers in the + future. Transfer encoding is a major performance benefit, so it was + worth fixing [39]. TE also solves another, obscure, downward + interoperability problem that could have occurred due to interactions + between authentication trailers, chunked encoding and HTTP/1.0 + clients.(Section 3.6, 3.6.1, and 14.39) + + The PATCH, LINK, UNLINK methods were defined but not commonly + implemented in previous versions of this specification. See RFC 2068 + [33]. + + The Alternates, Content-Version, Derived-From, Link, URI, Public and + Content-Base header fields were defined in previous versions of this + specification, but not commonly implemented. See RFC 2068 [33]. + +20 Index + + Please see the PostScript version of this RFC for the INDEX. + + + + + + + + + + + + + + + + + + + + + + + + + + + + + +Fielding, et al. Standards Track [Page 175] + +RFC 2616 HTTP/1.1 June 1999 + + +21. Full Copyright Statement + + Copyright (C) The Internet Society (1999). All Rights Reserved. + + This document and translations of it may be copied and furnished to + others, and derivative works that comment on or otherwise explain it + or assist in its implementation may be prepared, copied, published + and distributed, in whole or in part, without restriction of any + kind, provided that the above copyright notice and this paragraph are + included on all such copies and derivative works. However, this + document itself may not be modified in any way, such as by removing + the copyright notice or references to the Internet Society or other + Internet organizations, except as needed for the purpose of + developing Internet standards in which case the procedures for + copyrights defined in the Internet Standards process must be + followed, or as required to translate it into languages other than + English. + + The limited permissions granted above are perpetual and will not be + revoked by the Internet Society or its successors or assigns. + + This document and the information contained herein is provided on an + "AS IS" basis and THE INTERNET SOCIETY AND THE INTERNET ENGINEERING + TASK FORCE DISCLAIMS ALL WARRANTIES, EXPRESS OR IMPLIED, INCLUDING + BUT NOT LIMITED TO ANY WARRANTY THAT THE USE OF THE INFORMATION + HEREIN WILL NOT INFRINGE ANY RIGHTS OR ANY IMPLIED WARRANTIES OF + MERCHANTABILITY OR FITNESS FOR A PARTICULAR PURPOSE. + +Acknowledgement + + Funding for the RFC Editor function is currently provided by the + Internet Society. + + + + + + + + + + + + + + + + + + + +Fielding, et al. Standards Track [Page 176] + diff --git a/lib/inets/doc/archive/rfc959.txt b/lib/inets/doc/archive/rfc959.txt new file mode 100644 index 0000000000..5c9f11af5d --- /dev/null +++ b/lib/inets/doc/archive/rfc959.txt @@ -0,0 +1,3933 @@ + + +Network Working Group J. Postel +Request for Comments: 959 J. Reynolds + ISI +Obsoletes RFC: 765 (IEN 149) October 1985 + + FILE TRANSFER PROTOCOL (FTP) + + +Status of this Memo + + This memo is the official specification of the File Transfer + Protocol (FTP). Distribution of this memo is unlimited. + + The following new optional commands are included in this edition of + the specification: + + CDUP (Change to Parent Directory), SMNT (Structure Mount), STOU + (Store Unique), RMD (Remove Directory), MKD (Make Directory), PWD + (Print Directory), and SYST (System). + + Note that this specification is compatible with the previous edition. + +1. INTRODUCTION + + The objectives of FTP are 1) to promote sharing of files (computer + programs and/or data), 2) to encourage indirect or implicit (via + programs) use of remote computers, 3) to shield a user from + variations in file storage systems among hosts, and 4) to transfer + data reliably and efficiently. FTP, though usable directly by a user + at a terminal, is designed mainly for use by programs. + + The attempt in this specification is to satisfy the diverse needs of + users of maxi-hosts, mini-hosts, personal workstations, and TACs, + with a simple, and easily implemented protocol design. + + This paper assumes knowledge of the Transmission Control Protocol + (TCP) [2] and the Telnet Protocol [3]. These documents are contained + in the ARPA-Internet protocol handbook [1]. + +2. OVERVIEW + + In this section, the history, the terminology, and the FTP model are + discussed. The terms defined in this section are only those that + have special significance in FTP. Some of the terminology is very + specific to the FTP model; some readers may wish to turn to the + section on the FTP model while reviewing the terminology. + + + + + + + +Postel & Reynolds [Page 1] + + + +RFC 959 October 1985 +File Transfer Protocol + + + 2.1. HISTORY + + FTP has had a long evolution over the years. Appendix III is a + chronological compilation of Request for Comments documents + relating to FTP. These include the first proposed file transfer + mechanisms in 1971 that were developed for implementation on hosts + at M.I.T. (RFC 114), plus comments and discussion in RFC 141. + + RFC 172 provided a user-level oriented protocol for file transfer + between host computers (including terminal IMPs). A revision of + this as RFC 265, restated FTP for additional review, while RFC 281 + suggested further changes. The use of a "Set Data Type" + transaction was proposed in RFC 294 in January 1982. + + RFC 354 obsoleted RFCs 264 and 265. The File Transfer Protocol + was now defined as a protocol for file transfer between HOSTs on + the ARPANET, with the primary function of FTP defined as + transfering files efficiently and reliably among hosts and + allowing the convenient use of remote file storage capabilities. + RFC 385 further commented on errors, emphasis points, and + additions to the protocol, while RFC 414 provided a status report + on the working server and user FTPs. RFC 430, issued in 1973, + (among other RFCs too numerous to mention) presented further + comments on FTP. Finally, an "official" FTP document was + published as RFC 454. + + By July 1973, considerable changes from the last versions of FTP + were made, but the general structure remained the same. RFC 542 + was published as a new "official" specification to reflect these + changes. However, many implementations based on the older + specification were not updated. + + In 1974, RFCs 607 and 614 continued comments on FTP. RFC 624 + proposed further design changes and minor modifications. In 1975, + RFC 686 entitled, "Leaving Well Enough Alone", discussed the + differences between all of the early and later versions of FTP. + RFC 691 presented a minor revision of RFC 686, regarding the + subject of print files. + + Motivated by the transition from the NCP to the TCP as the + underlying protocol, a phoenix was born out of all of the above + efforts in RFC 765 as the specification of FTP for use on TCP. + + This current edition of the FTP specification is intended to + correct some minor documentation errors, to improve the + explanation of some protocol features, and to add some new + optional commands. + + +Postel & Reynolds [Page 2] + + + +RFC 959 October 1985 +File Transfer Protocol + + + In particular, the following new optional commands are included in + this edition of the specification: + + CDUP - Change to Parent Directory + + SMNT - Structure Mount + + STOU - Store Unique + + RMD - Remove Directory + + MKD - Make Directory + + PWD - Print Directory + + SYST - System + + This specification is compatible with the previous edition. A + program implemented in conformance to the previous specification + should automatically be in conformance to this specification. + + 2.2. TERMINOLOGY + + ASCII + + The ASCII character set is as defined in the ARPA-Internet + Protocol Handbook. In FTP, ASCII characters are defined to be + the lower half of an eight-bit code set (i.e., the most + significant bit is zero). + + access controls + + Access controls define users' access privileges to the use of a + system, and to the files in that system. Access controls are + necessary to prevent unauthorized or accidental use of files. + It is the prerogative of a server-FTP process to invoke access + controls. + + byte size + + There are two byte sizes of interest in FTP: the logical byte + size of the file, and the transfer byte size used for the + transmission of the data. The transfer byte size is always 8 + bits. The transfer byte size is not necessarily the byte size + in which data is to be stored in a system, nor the logical byte + size for interpretation of the structure of the data. + + + +Postel & Reynolds [Page 3] + + + +RFC 959 October 1985 +File Transfer Protocol + + + control connection + + The communication path between the USER-PI and SERVER-PI for + the exchange of commands and replies. This connection follows + the Telnet Protocol. + + data connection + + A full duplex connection over which data is transferred, in a + specified mode and type. The data transferred may be a part of + a file, an entire file or a number of files. The path may be + between a server-DTP and a user-DTP, or between two + server-DTPs. + + data port + + The passive data transfer process "listens" on the data port + for a connection from the active transfer process in order to + open the data connection. + + DTP + + The data transfer process establishes and manages the data + connection. The DTP can be passive or active. + + End-of-Line + + The end-of-line sequence defines the separation of printing + lines. The sequence is Carriage Return, followed by Line Feed. + + EOF + + The end-of-file condition that defines the end of a file being + transferred. + + EOR + + The end-of-record condition that defines the end of a record + being transferred. + + error recovery + + A procedure that allows a user to recover from certain errors + such as failure of either host system or transfer process. In + FTP, error recovery may involve restarting a file transfer at a + given checkpoint. + + + +Postel & Reynolds [Page 4] + + + +RFC 959 October 1985 +File Transfer Protocol + + + FTP commands + + A set of commands that comprise the control information flowing + from the user-FTP to the server-FTP process. + + file + + An ordered set of computer data (including programs), of + arbitrary length, uniquely identified by a pathname. + + mode + + The mode in which data is to be transferred via the data + connection. The mode defines the data format during transfer + including EOR and EOF. The transfer modes defined in FTP are + described in the Section on Transmission Modes. + + NVT + + The Network Virtual Terminal as defined in the Telnet Protocol. + + NVFS + + The Network Virtual File System. A concept which defines a + standard network file system with standard commands and + pathname conventions. + + page + + A file may be structured as a set of independent parts called + pages. FTP supports the transmission of discontinuous files as + independent indexed pages. + + pathname + + Pathname is defined to be the character string which must be + input to a file system by a user in order to identify a file. + Pathname normally contains device and/or directory names, and + file name specification. FTP does not yet specify a standard + pathname convention. Each user must follow the file naming + conventions of the file systems involved in the transfer. + + PI + + The protocol interpreter. The user and server sides of the + protocol have distinct roles implemented in a user-PI and a + server-PI. + + +Postel & Reynolds [Page 5] + + + +RFC 959 October 1985 +File Transfer Protocol + + + record + + A sequential file may be structured as a number of contiguous + parts called records. Record structures are supported by FTP + but a file need not have record structure. + + reply + + A reply is an acknowledgment (positive or negative) sent from + server to user via the control connection in response to FTP + commands. The general form of a reply is a completion code + (including error codes) followed by a text string. The codes + are for use by programs and the text is usually intended for + human users. + + server-DTP + + The data transfer process, in its normal "active" state, + establishes the data connection with the "listening" data port. + It sets up parameters for transfer and storage, and transfers + data on command from its PI. The DTP can be placed in a + "passive" state to listen for, rather than initiate a + connection on the data port. + + server-FTP process + + A process or set of processes which perform the function of + file transfer in cooperation with a user-FTP process and, + possibly, another server. The functions consist of a protocol + interpreter (PI) and a data transfer process (DTP). + + server-PI + + The server protocol interpreter "listens" on Port L for a + connection from a user-PI and establishes a control + communication connection. It receives standard FTP commands + from the user-PI, sends replies, and governs the server-DTP. + + type + + The data representation type used for data transfer and + storage. Type implies certain transformations between the time + of data storage and data transfer. The representation types + defined in FTP are described in the Section on Establishing + Data Connections. + + + + +Postel & Reynolds [Page 6] + + + +RFC 959 October 1985 +File Transfer Protocol + + + user + + A person or a process on behalf of a person wishing to obtain + file transfer service. The human user may interact directly + with a server-FTP process, but use of a user-FTP process is + preferred since the protocol design is weighted towards + automata. + + user-DTP + + The data transfer process "listens" on the data port for a + connection from a server-FTP process. If two servers are + transferring data between them, the user-DTP is inactive. + + user-FTP process + + A set of functions including a protocol interpreter, a data + transfer process and a user interface which together perform + the function of file transfer in cooperation with one or more + server-FTP processes. The user interface allows a local + language to be used in the command-reply dialogue with the + user. + + user-PI + + The user protocol interpreter initiates the control connection + from its port U to the server-FTP process, initiates FTP + commands, and governs the user-DTP if that process is part of + the file transfer. + + + + + + + + + + + + + + + + + + + + +Postel & Reynolds [Page 7] + + + +RFC 959 October 1985 +File Transfer Protocol + + + 2.3. THE FTP MODEL + + With the above definitions in mind, the following model (shown in + Figure 1) may be diagrammed for an FTP service. + + ------------- + |/---------\| + || User || -------- + ||Interface|<--->| User | + |\----^----/| -------- + ---------- | | | + |/------\| FTP Commands |/----V----\| + ||Server|<---------------->| User || + || PI || FTP Replies || PI || + |\--^---/| |\----^----/| + | | | | | | + -------- |/--V---\| Data |/----V----\| -------- + | File |<--->|Server|<---------------->| User |<--->| File | + |System| || DTP || Connection || DTP || |System| + -------- |\------/| |\---------/| -------- + ---------- ------------- + + Server-FTP USER-FTP + + NOTES: 1. The data connection may be used in either direction. + 2. The data connection need not exist all of the time. + + Figure 1 Model for FTP Use + + In the model described in Figure 1, the user-protocol interpreter + initiates the control connection. The control connection follows + the Telnet protocol. At the initiation of the user, standard FTP + commands are generated by the user-PI and transmitted to the + server process via the control connection. (The user may + establish a direct control connection to the server-FTP, from a + TAC terminal for example, and generate standard FTP commands + independently, bypassing the user-FTP process.) Standard replies + are sent from the server-PI to the user-PI over the control + connection in response to the commands. + + The FTP commands specify the parameters for the data connection + (data port, transfer mode, representation type, and structure) and + the nature of file system operation (store, retrieve, append, + delete, etc.). The user-DTP or its designate should "listen" on + the specified data port, and the server initiate the data + connection and data transfer in accordance with the specified + parameters. It should be noted that the data port need not be in + + +Postel & Reynolds [Page 8] + + + +RFC 959 October 1985 +File Transfer Protocol + + + the same host that initiates the FTP commands via the control + connection, but the user or the user-FTP process must ensure a + "listen" on the specified data port. It ought to also be noted + that the data connection may be used for simultaneous sending and + receiving. + + In another situation a user might wish to transfer files between + two hosts, neither of which is a local host. The user sets up + control connections to the two servers and then arranges for a + data connection between them. In this manner, control information + is passed to the user-PI but data is transferred between the + server data transfer processes. Following is a model of this + server-server interaction. + + + Control ------------ Control + ---------->| User-FTP |<----------- + | | User-PI | | + | | "C" | | + V ------------ V + -------------- -------------- + | Server-FTP | Data Connection | Server-FTP | + | "A" |<---------------------->| "B" | + -------------- Port (A) Port (B) -------------- + + + Figure 2 + + The protocol requires that the control connections be open while + data transfer is in progress. It is the responsibility of the + user to request the closing of the control connections when + finished using the FTP service, while it is the server who takes + the action. The server may abort data transfer if the control + connections are closed without command. + + The Relationship between FTP and Telnet: + + The FTP uses the Telnet protocol on the control connection. + This can be achieved in two ways: first, the user-PI or the + server-PI may implement the rules of the Telnet Protocol + directly in their own procedures; or, second, the user-PI or + the server-PI may make use of the existing Telnet module in the + system. + + Ease of implementaion, sharing code, and modular programming + argue for the second approach. Efficiency and independence + + + +Postel & Reynolds [Page 9] + + + +RFC 959 October 1985 +File Transfer Protocol + + + argue for the first approach. In practice, FTP relies on very + little of the Telnet Protocol, so the first approach does not + necessarily involve a large amount of code. + +3. DATA TRANSFER FUNCTIONS + + Files are transferred only via the data connection. The control + connection is used for the transfer of commands, which describe the + functions to be performed, and the replies to these commands (see the + Section on FTP Replies). Several commands are concerned with the + transfer of data between hosts. These data transfer commands include + the MODE command which specify how the bits of the data are to be + transmitted, and the STRUcture and TYPE commands, which are used to + define the way in which the data are to be represented. The + transmission and representation are basically independent but the + "Stream" transmission mode is dependent on the file structure + attribute and if "Compressed" transmission mode is used, the nature + of the filler byte depends on the representation type. + + 3.1. DATA REPRESENTATION AND STORAGE + + Data is transferred from a storage device in the sending host to a + storage device in the receiving host. Often it is necessary to + perform certain transformations on the data because data storage + representations in the two systems are different. For example, + NVT-ASCII has different data storage representations in different + systems. DEC TOPS-20s's generally store NVT-ASCII as five 7-bit + ASCII characters, left-justified in a 36-bit word. IBM Mainframe's + store NVT-ASCII as 8-bit EBCDIC codes. Multics stores NVT-ASCII + as four 9-bit characters in a 36-bit word. It is desirable to + convert characters into the standard NVT-ASCII representation when + transmitting text between dissimilar systems. The sending and + receiving sites would have to perform the necessary + transformations between the standard representation and their + internal representations. + + A different problem in representation arises when transmitting + binary data (not character codes) between host systems with + different word lengths. It is not always clear how the sender + should send data, and the receiver store it. For example, when + transmitting 32-bit bytes from a 32-bit word-length system to a + 36-bit word-length system, it may be desirable (for reasons of + efficiency and usefulness) to store the 32-bit bytes + right-justified in a 36-bit word in the latter system. In any + case, the user should have the option of specifying data + representation and transformation functions. It should be noted + + + +Postel & Reynolds [Page 10] + + + +RFC 959 October 1985 +File Transfer Protocol + + + that FTP provides for very limited data type representations. + Transformations desired beyond this limited capability should be + performed by the user directly. + + 3.1.1. DATA TYPES + + Data representations are handled in FTP by a user specifying a + representation type. This type may implicitly (as in ASCII or + EBCDIC) or explicitly (as in Local byte) define a byte size for + interpretation which is referred to as the "logical byte size." + Note that this has nothing to do with the byte size used for + transmission over the data connection, called the "transfer + byte size", and the two should not be confused. For example, + NVT-ASCII has a logical byte size of 8 bits. If the type is + Local byte, then the TYPE command has an obligatory second + parameter specifying the logical byte size. The transfer byte + size is always 8 bits. + + 3.1.1.1. ASCII TYPE + + This is the default type and must be accepted by all FTP + implementations. It is intended primarily for the transfer + of text files, except when both hosts would find the EBCDIC + type more convenient. + + The sender converts the data from an internal character + representation to the standard 8-bit NVT-ASCII + representation (see the Telnet specification). The receiver + will convert the data from the standard form to his own + internal form. + + In accordance with the NVT standard, the <CRLF> sequence + should be used where necessary to denote the end of a line + of text. (See the discussion of file structure at the end + of the Section on Data Representation and Storage.) + + Using the standard NVT-ASCII representation means that data + must be interpreted as 8-bit bytes. + + The Format parameter for ASCII and EBCDIC types is discussed + below. + + + + + + + + +Postel & Reynolds [Page 11] + + + +RFC 959 October 1985 +File Transfer Protocol + + + 3.1.1.2. EBCDIC TYPE + + This type is intended for efficient transfer between hosts + which use EBCDIC for their internal character + representation. + + For transmission, the data are represented as 8-bit EBCDIC + characters. The character code is the only difference + between the functional specifications of EBCDIC and ASCII + types. + + End-of-line (as opposed to end-of-record--see the discussion + of structure) will probably be rarely used with EBCDIC type + for purposes of denoting structure, but where it is + necessary the <NL> character should be used. + + 3.1.1.3. IMAGE TYPE + + The data are sent as contiguous bits which, for transfer, + are packed into the 8-bit transfer bytes. The receiving + site must store the data as contiguous bits. The structure + of the storage system might necessitate the padding of the + file (or of each record, for a record-structured file) to + some convenient boundary (byte, word or block). This + padding, which must be all zeros, may occur only at the end + of the file (or at the end of each record) and there must be + a way of identifying the padding bits so that they may be + stripped off if the file is retrieved. The padding + transformation should be well publicized to enable a user to + process a file at the storage site. + + Image type is intended for the efficient storage and + retrieval of files and for the transfer of binary data. It + is recommended that this type be accepted by all FTP + implementations. + + 3.1.1.4. LOCAL TYPE + + The data is transferred in logical bytes of the size + specified by the obligatory second parameter, Byte size. + The value of Byte size must be a decimal integer; there is + no default value. The logical byte size is not necessarily + the same as the transfer byte size. If there is a + difference in byte sizes, then the logical bytes should be + packed contiguously, disregarding transfer byte boundaries + and with any necessary padding at the end. + + + +Postel & Reynolds [Page 12] + + + +RFC 959 October 1985 +File Transfer Protocol + + + When the data reaches the receiving host, it will be + transformed in a manner dependent on the logical byte size + and the particular host. This transformation must be + invertible (i.e., an identical file can be retrieved if the + same parameters are used) and should be well publicized by + the FTP implementors. + + For example, a user sending 36-bit floating-point numbers to + a host with a 32-bit word could send that data as Local byte + with a logical byte size of 36. The receiving host would + then be expected to store the logical bytes so that they + could be easily manipulated; in this example putting the + 36-bit logical bytes into 64-bit double words should + suffice. + + In another example, a pair of hosts with a 36-bit word size + may send data to one another in words by using TYPE L 36. + The data would be sent in the 8-bit transmission bytes + packed so that 9 transmission bytes carried two host words. + + 3.1.1.5. FORMAT CONTROL + + The types ASCII and EBCDIC also take a second (optional) + parameter; this is to indicate what kind of vertical format + control, if any, is associated with a file. The following + data representation types are defined in FTP: + + A character file may be transferred to a host for one of + three purposes: for printing, for storage and later + retrieval, or for processing. If a file is sent for + printing, the receiving host must know how the vertical + format control is represented. In the second case, it must + be possible to store a file at a host and then retrieve it + later in exactly the same form. Finally, it should be + possible to move a file from one host to another and process + the file at the second host without undue trouble. A single + ASCII or EBCDIC format does not satisfy all these + conditions. Therefore, these types have a second parameter + specifying one of the following three formats: + + 3.1.1.5.1. NON PRINT + + This is the default format to be used if the second + (format) parameter is omitted. Non-print format must be + accepted by all FTP implementations. + + + + +Postel & Reynolds [Page 13] + + + +RFC 959 October 1985 +File Transfer Protocol + + + The file need contain no vertical format information. If + it is passed to a printer process, this process may + assume standard values for spacing and margins. + + Normally, this format will be used with files destined + for processing or just storage. + + 3.1.1.5.2. TELNET FORMAT CONTROLS + + The file contains ASCII/EBCDIC vertical format controls + (i.e., <CR>, <LF>, <NL>, <VT>, <FF>) which the printer + process will interpret appropriately. <CRLF>, in exactly + this sequence, also denotes end-of-line. + + 3.1.1.5.2. CARRIAGE CONTROL (ASA) + + The file contains ASA (FORTRAN) vertical format control + characters. (See RFC 740 Appendix C; and Communications + of the ACM, Vol. 7, No. 10, p. 606, October 1964.) In a + line or a record formatted according to the ASA Standard, + the first character is not to be printed. Instead, it + should be used to determine the vertical movement of the + paper which should take place before the rest of the + record is printed. + + The ASA Standard specifies the following control + characters: + + Character Vertical Spacing + + blank Move paper up one line + 0 Move paper up two lines + 1 Move paper to top of next page + + No movement, i.e., overprint + + Clearly there must be some way for a printer process to + distinguish the end of the structural entity. If a file + has record structure (see below) this is no problem; + records will be explicitly marked during transfer and + storage. If the file has no record structure, the <CRLF> + end-of-line sequence is used to separate printing lines, + but these format effectors are overridden by the ASA + controls. + + + + + + +Postel & Reynolds [Page 14] + + + +RFC 959 October 1985 +File Transfer Protocol + + + 3.1.2. DATA STRUCTURES + + In addition to different representation types, FTP allows the + structure of a file to be specified. Three file structures are + defined in FTP: + + file-structure, where there is no internal structure and + the file is considered to be a + continuous sequence of data bytes, + + record-structure, where the file is made up of sequential + records, + + and page-structure, where the file is made up of independent + indexed pages. + + File-structure is the default to be assumed if the STRUcture + command has not been used but both file and record structures + must be accepted for "text" files (i.e., files with TYPE ASCII + or EBCDIC) by all FTP implementations. The structure of a file + will affect both the transfer mode of a file (see the Section + on Transmission Modes) and the interpretation and storage of + the file. + + The "natural" structure of a file will depend on which host + stores the file. A source-code file will usually be stored on + an IBM Mainframe in fixed length records but on a DEC TOPS-20 + as a stream of characters partitioned into lines, for example + by <CRLF>. If the transfer of files between such disparate + sites is to be useful, there must be some way for one site to + recognize the other's assumptions about the file. + + With some sites being naturally file-oriented and others + naturally record-oriented there may be problems if a file with + one structure is sent to a host oriented to the other. If a + text file is sent with record-structure to a host which is file + oriented, then that host should apply an internal + transformation to the file based on the record structure. + Obviously, this transformation should be useful, but it must + also be invertible so that an identical file may be retrieved + using record structure. + + In the case of a file being sent with file-structure to a + record-oriented host, there exists the question of what + criteria the host should use to divide the file into records + which can be processed locally. If this division is necessary, + the FTP implementation should use the end-of-line sequence, + + +Postel & Reynolds [Page 15] + + + +RFC 959 October 1985 +File Transfer Protocol + + + <CRLF> for ASCII, or <NL> for EBCDIC text files, as the + delimiter. If an FTP implementation adopts this technique, it + must be prepared to reverse the transformation if the file is + retrieved with file-structure. + + 3.1.2.1. FILE STRUCTURE + + File structure is the default to be assumed if the STRUcture + command has not been used. + + In file-structure there is no internal structure and the + file is considered to be a continuous sequence of data + bytes. + + 3.1.2.2. RECORD STRUCTURE + + Record structures must be accepted for "text" files (i.e., + files with TYPE ASCII or EBCDIC) by all FTP implementations. + + In record-structure the file is made up of sequential + records. + + 3.1.2.3. PAGE STRUCTURE + + To transmit files that are discontinuous, FTP defines a page + structure. Files of this type are sometimes known as + "random access files" or even as "holey files". In these + files there is sometimes other information associated with + the file as a whole (e.g., a file descriptor), or with a + section of the file (e.g., page access controls), or both. + In FTP, the sections of the file are called pages. + + To provide for various page sizes and associated + information, each page is sent with a page header. The page + header has the following defined fields: + + Header Length + + The number of logical bytes in the page header + including this byte. The minimum header length is 4. + + Page Index + + The logical page number of this section of the file. + This is not the transmission sequence number of this + page, but the index used to identify this page of the + file. + + +Postel & Reynolds [Page 16] + + + +RFC 959 October 1985 +File Transfer Protocol + + + Data Length + + The number of logical bytes in the page data. The + minimum data length is 0. + + Page Type + + The type of page this is. The following page types + are defined: + + 0 = Last Page + + This is used to indicate the end of a paged + structured transmission. The header length must + be 4, and the data length must be 0. + + 1 = Simple Page + + This is the normal type for simple paged files + with no page level associated control + information. The header length must be 4. + + 2 = Descriptor Page + + This type is used to transmit the descriptive + information for the file as a whole. + + 3 = Access Controlled Page + + This type includes an additional header field + for paged files with page level access control + information. The header length must be 5. + + Optional Fields + + Further header fields may be used to supply per page + control information, for example, per page access + control. + + All fields are one logical byte in length. The logical byte + size is specified by the TYPE command. See Appendix I for + further details and a specific case at the page structure. + + A note of caution about parameters: a file must be stored and + retrieved with the same parameters if the retrieved version is to + + + + +Postel & Reynolds [Page 17] + + + +RFC 959 October 1985 +File Transfer Protocol + + + be identical to the version originally transmitted. Conversely, + FTP implementations must return a file identical to the original + if the parameters used to store and retrieve a file are the same. + + 3.2. ESTABLISHING DATA CONNECTIONS + + The mechanics of transferring data consists of setting up the data + connection to the appropriate ports and choosing the parameters + for transfer. Both the user and the server-DTPs have a default + data port. The user-process default data port is the same as the + control connection port (i.e., U). The server-process default + data port is the port adjacent to the control connection port + (i.e., L-1). + + The transfer byte size is 8-bit bytes. This byte size is relevant + only for the actual transfer of the data; it has no bearing on + representation of the data within a host's file system. + + The passive data transfer process (this may be a user-DTP or a + second server-DTP) shall "listen" on the data port prior to + sending a transfer request command. The FTP request command + determines the direction of the data transfer. The server, upon + receiving the transfer request, will initiate the data connection + to the port. When the connection is established, the data + transfer begins between DTP's, and the server-PI sends a + confirming reply to the user-PI. + + Every FTP implementation must support the use of the default data + ports, and only the USER-PI can initiate a change to non-default + ports. + + It is possible for the user to specify an alternate data port by + use of the PORT command. The user may want a file dumped on a TAC + line printer or retrieved from a third party host. In the latter + case, the user-PI sets up control connections with both + server-PI's. One server is then told (by an FTP command) to + "listen" for a connection which the other will initiate. The + user-PI sends one server-PI a PORT command indicating the data + port of the other. Finally, both are sent the appropriate + transfer commands. The exact sequence of commands and replies + sent between the user-controller and the servers is defined in the + Section on FTP Replies. + + In general, it is the server's responsibility to maintain the data + connection--to initiate it and to close it. The exception to this + + + + +Postel & Reynolds [Page 18] + + + +RFC 959 October 1985 +File Transfer Protocol + + + is when the user-DTP is sending the data in a transfer mode that + requires the connection to be closed to indicate EOF. The server + MUST close the data connection under the following conditions: + + 1. The server has completed sending data in a transfer mode + that requires a close to indicate EOF. + + 2. The server receives an ABORT command from the user. + + 3. The port specification is changed by a command from the + user. + + 4. The control connection is closed legally or otherwise. + + 5. An irrecoverable error condition occurs. + + Otherwise the close is a server option, the exercise of which the + server must indicate to the user-process by either a 250 or 226 + reply only. + + 3.3. DATA CONNECTION MANAGEMENT + + Default Data Connection Ports: All FTP implementations must + support use of the default data connection ports, and only the + User-PI may initiate the use of non-default ports. + + Negotiating Non-Default Data Ports: The User-PI may specify a + non-default user side data port with the PORT command. The + User-PI may request the server side to identify a non-default + server side data port with the PASV command. Since a connection + is defined by the pair of addresses, either of these actions is + enough to get a different data connection, still it is permitted + to do both commands to use new ports on both ends of the data + connection. + + Reuse of the Data Connection: When using the stream mode of data + transfer the end of the file must be indicated by closing the + connection. This causes a problem if multiple files are to be + transfered in the session, due to need for TCP to hold the + connection record for a time out period to guarantee the reliable + communication. Thus the connection can not be reopened at once. + + There are two solutions to this problem. The first is to + negotiate a non-default port. The second is to use another + transfer mode. + + A comment on transfer modes. The stream transfer mode is + + +Postel & Reynolds [Page 19] + + + +RFC 959 October 1985 +File Transfer Protocol + + + inherently unreliable, since one can not determine if the + connection closed prematurely or not. The other transfer modes + (Block, Compressed) do not close the connection to indicate the + end of file. They have enough FTP encoding that the data + connection can be parsed to determine the end of the file. + Thus using these modes one can leave the data connection open + for multiple file transfers. + + 3.4. TRANSMISSION MODES + + The next consideration in transferring data is choosing the + appropriate transmission mode. There are three modes: one which + formats the data and allows for restart procedures; one which also + compresses the data for efficient transfer; and one which passes + the data with little or no processing. In this last case the mode + interacts with the structure attribute to determine the type of + processing. In the compressed mode, the representation type + determines the filler byte. + + All data transfers must be completed with an end-of-file (EOF) + which may be explicitly stated or implied by the closing of the + data connection. For files with record structure, all the + end-of-record markers (EOR) are explicit, including the final one. + For files transmitted in page structure a "last-page" page type is + used. + + NOTE: In the rest of this section, byte means "transfer byte" + except where explicitly stated otherwise. + + For the purpose of standardized transfer, the sending host will + translate its internal end of line or end of record denotation + into the representation prescribed by the transfer mode and file + structure, and the receiving host will perform the inverse + translation to its internal denotation. An IBM Mainframe record + count field may not be recognized at another host, so the + end-of-record information may be transferred as a two byte control + code in Stream mode or as a flagged bit in a Block or Compressed + mode descriptor. End-of-line in an ASCII or EBCDIC file with no + record structure should be indicated by <CRLF> or <NL>, + respectively. Since these transformations imply extra work for + some systems, identical systems transferring non-record structured + text files might wish to use a binary representation and stream + mode for the transfer. + + + + + + +Postel & Reynolds [Page 20] + + + +RFC 959 October 1985 +File Transfer Protocol + + + The following transmission modes are defined in FTP: + + 3.4.1. STREAM MODE + + The data is transmitted as a stream of bytes. There is no + restriction on the representation type used; record structures + are allowed. + + In a record structured file EOR and EOF will each be indicated + by a two-byte control code. The first byte of the control code + will be all ones, the escape character. The second byte will + have the low order bit on and zeros elsewhere for EOR and the + second low order bit on for EOF; that is, the byte will have + value 1 for EOR and value 2 for EOF. EOR and EOF may be + indicated together on the last byte transmitted by turning both + low order bits on (i.e., the value 3). If a byte of all ones + was intended to be sent as data, it should be repeated in the + second byte of the control code. + + If the structure is a file structure, the EOF is indicated by + the sending host closing the data connection and all bytes are + data bytes. + + 3.4.2. BLOCK MODE + + The file is transmitted as a series of data blocks preceded by + one or more header bytes. The header bytes contain a count + field, and descriptor code. The count field indicates the + total length of the data block in bytes, thus marking the + beginning of the next data block (there are no filler bits). + The descriptor code defines: last block in the file (EOF) last + block in the record (EOR), restart marker (see the Section on + Error Recovery and Restart) or suspect data (i.e., the data + being transferred is suspected of errors and is not reliable). + This last code is NOT intended for error control within FTP. + It is motivated by the desire of sites exchanging certain types + of data (e.g., seismic or weather data) to send and receive all + the data despite local errors (such as "magnetic tape read + errors"), but to indicate in the transmission that certain + portions are suspect). Record structures are allowed in this + mode, and any representation type may be used. + + The header consists of the three bytes. Of the 24 bits of + header information, the 16 low order bits shall represent byte + count, and the 8 high order bits shall represent descriptor + codes as shown below. + + + +Postel & Reynolds [Page 21] + + + +RFC 959 October 1985 +File Transfer Protocol + + + Block Header + + +----------------+----------------+----------------+ + | Descriptor | Byte Count | + | 8 bits | 16 bits | + +----------------+----------------+----------------+ + + + The descriptor codes are indicated by bit flags in the + descriptor byte. Four codes have been assigned, where each + code number is the decimal value of the corresponding bit in + the byte. + + Code Meaning + + 128 End of data block is EOR + 64 End of data block is EOF + 32 Suspected errors in data block + 16 Data block is a restart marker + + With this encoding, more than one descriptor coded condition + may exist for a particular block. As many bits as necessary + may be flagged. + + The restart marker is embedded in the data stream as an + integral number of 8-bit bytes representing printable + characters in the language being used over the control + connection (e.g., default--NVT-ASCII). <SP> (Space, in the + appropriate language) must not be used WITHIN a restart marker. + + For example, to transmit a six-character marker, the following + would be sent: + + +--------+--------+--------+ + |Descrptr| Byte count | + |code= 16| = 6 | + +--------+--------+--------+ + + +--------+--------+--------+ + | Marker | Marker | Marker | + | 8 bits | 8 bits | 8 bits | + +--------+--------+--------+ + + +--------+--------+--------+ + | Marker | Marker | Marker | + | 8 bits | 8 bits | 8 bits | + +--------+--------+--------+ + + +Postel & Reynolds [Page 22] + + + +RFC 959 October 1985 +File Transfer Protocol + + + 3.4.3. COMPRESSED MODE + + There are three kinds of information to be sent: regular data, + sent in a byte string; compressed data, consisting of + replications or filler; and control information, sent in a + two-byte escape sequence. If n>0 bytes (up to 127) of regular + data are sent, these n bytes are preceded by a byte with the + left-most bit set to 0 and the right-most 7 bits containing the + number n. + + Byte string: + + 1 7 8 8 + +-+-+-+-+-+-+-+-+ +-+-+-+-+-+-+-+-+ +-+-+-+-+-+-+-+-+ + |0| n | | d(1) | ... | d(n) | + +-+-+-+-+-+-+-+-+ +-+-+-+-+-+-+-+-+ +-+-+-+-+-+-+-+-+ + ^ ^ + |---n bytes---| + of data + + String of n data bytes d(1),..., d(n) + Count n must be positive. + + To compress a string of n replications of the data byte d, the + following 2 bytes are sent: + + Replicated Byte: + + 2 6 8 + +-+-+-+-+-+-+-+-+ +-+-+-+-+-+-+-+-+ + |1 0| n | | d | + +-+-+-+-+-+-+-+-+ +-+-+-+-+-+-+-+-+ + + A string of n filler bytes can be compressed into a single + byte, where the filler byte varies with the representation + type. If the type is ASCII or EBCDIC the filler byte is <SP> + (Space, ASCII code 32, EBCDIC code 64). If the type is Image + or Local byte the filler is a zero byte. + + Filler String: + + 2 6 + +-+-+-+-+-+-+-+-+ + |1 1| n | + +-+-+-+-+-+-+-+-+ + + The escape sequence is a double byte, the first of which is the + + +Postel & Reynolds [Page 23] + + + +RFC 959 October 1985 +File Transfer Protocol + + + escape byte (all zeros) and the second of which contains + descriptor codes as defined in Block mode. The descriptor + codes have the same meaning as in Block mode and apply to the + succeeding string of bytes. + + Compressed mode is useful for obtaining increased bandwidth on + very large network transmissions at a little extra CPU cost. + It can be most effectively used to reduce the size of printer + files such as those generated by RJE hosts. + + 3.5. ERROR RECOVERY AND RESTART + + There is no provision for detecting bits lost or scrambled in data + transfer; this level of error control is handled by the TCP. + However, a restart procedure is provided to protect users from + gross system failures (including failures of a host, an + FTP-process, or the underlying network). + + The restart procedure is defined only for the block and compressed + modes of data transfer. It requires the sender of data to insert + a special marker code in the data stream with some marker + information. The marker information has meaning only to the + sender, but must consist of printable characters in the default or + negotiated language of the control connection (ASCII or EBCDIC). + The marker could represent a bit-count, a record-count, or any + other information by which a system may identify a data + checkpoint. The receiver of data, if it implements the restart + procedure, would then mark the corresponding position of this + marker in the receiving system, and return this information to the + user. + + In the event of a system failure, the user can restart the data + transfer by identifying the marker point with the FTP restart + procedure. The following example illustrates the use of the + restart procedure. + + The sender of the data inserts an appropriate marker block in the + data stream at a convenient point. The receiving host marks the + corresponding data point in its file system and conveys the last + known sender and receiver marker information to the user, either + directly or over the control connection in a 110 reply (depending + on who is the sender). In the event of a system failure, the user + or controller process restarts the server at the last server + marker by sending a restart command with server's marker code as + its argument. The restart command is transmitted over the control + + + + +Postel & Reynolds [Page 24] + + + +RFC 959 October 1985 +File Transfer Protocol + + + connection and is immediately followed by the command (such as + RETR, STOR or LIST) which was being executed when the system + failure occurred. + +4. FILE TRANSFER FUNCTIONS + + The communication channel from the user-PI to the server-PI is + established as a TCP connection from the user to the standard server + port. The user protocol interpreter is responsible for sending FTP + commands and interpreting the replies received; the server-PI + interprets commands, sends replies and directs its DTP to set up the + data connection and transfer the data. If the second party to the + data transfer (the passive transfer process) is the user-DTP, then it + is governed through the internal protocol of the user-FTP host; if it + is a second server-DTP, then it is governed by its PI on command from + the user-PI. The FTP replies are discussed in the next section. In + the description of a few of the commands in this section, it is + helpful to be explicit about the possible replies. + + 4.1. FTP COMMANDS + + 4.1.1. ACCESS CONTROL COMMANDS + + The following commands specify access control identifiers + (command codes are shown in parentheses). + + USER NAME (USER) + + The argument field is a Telnet string identifying the user. + The user identification is that which is required by the + server for access to its file system. This command will + normally be the first command transmitted by the user after + the control connections are made (some servers may require + this). Additional identification information in the form of + a password and/or an account command may also be required by + some servers. Servers may allow a new USER command to be + entered at any point in order to change the access control + and/or accounting information. This has the effect of + flushing any user, password, and account information already + supplied and beginning the login sequence again. All + transfer parameters are unchanged and any file transfer in + progress is completed under the old access control + parameters. + + + + + + +Postel & Reynolds [Page 25] + + + +RFC 959 October 1985 +File Transfer Protocol + + + PASSWORD (PASS) + + The argument field is a Telnet string specifying the user's + password. This command must be immediately preceded by the + user name command, and, for some sites, completes the user's + identification for access control. Since password + information is quite sensitive, it is desirable in general + to "mask" it or suppress typeout. It appears that the + server has no foolproof way to achieve this. It is + therefore the responsibility of the user-FTP process to hide + the sensitive password information. + + ACCOUNT (ACCT) + + The argument field is a Telnet string identifying the user's + account. The command is not necessarily related to the USER + command, as some sites may require an account for login and + others only for specific access, such as storing files. In + the latter case the command may arrive at any time. + + There are reply codes to differentiate these cases for the + automation: when account information is required for login, + the response to a successful PASSword command is reply code + 332. On the other hand, if account information is NOT + required for login, the reply to a successful PASSword + command is 230; and if the account information is needed for + a command issued later in the dialogue, the server should + return a 332 or 532 reply depending on whether it stores + (pending receipt of the ACCounT command) or discards the + command, respectively. + + CHANGE WORKING DIRECTORY (CWD) + + This command allows the user to work with a different + directory or dataset for file storage or retrieval without + altering his login or accounting information. Transfer + parameters are similarly unchanged. The argument is a + pathname specifying a directory or other system dependent + file group designator. + + CHANGE TO PARENT DIRECTORY (CDUP) + + This command is a special case of CWD, and is included to + simplify the implementation of programs for transferring + directory trees between operating systems having different + + + + +Postel & Reynolds [Page 26] + + + +RFC 959 October 1985 +File Transfer Protocol + + + syntaxes for naming the parent directory. The reply codes + shall be identical to the reply codes of CWD. See + Appendix II for further details. + + STRUCTURE MOUNT (SMNT) + + This command allows the user to mount a different file + system data structure without altering his login or + accounting information. Transfer parameters are similarly + unchanged. The argument is a pathname specifying a + directory or other system dependent file group designator. + + REINITIALIZE (REIN) + + This command terminates a USER, flushing all I/O and account + information, except to allow any transfer in progress to be + completed. All parameters are reset to the default settings + and the control connection is left open. This is identical + to the state in which a user finds himself immediately after + the control connection is opened. A USER command may be + expected to follow. + + LOGOUT (QUIT) + + This command terminates a USER and if file transfer is not + in progress, the server closes the control connection. If + file transfer is in progress, the connection will remain + open for result response and the server will then close it. + If the user-process is transferring files for several USERs + but does not wish to close and then reopen connections for + each, then the REIN command should be used instead of QUIT. + + An unexpected close on the control connection will cause the + server to take the effective action of an abort (ABOR) and a + logout (QUIT). + + 4.1.2. TRANSFER PARAMETER COMMANDS + + All data transfer parameters have default values, and the + commands specifying data transfer parameters are required only + if the default parameter values are to be changed. The default + value is the last specified value, or if no value has been + specified, the standard default value is as stated here. This + implies that the server must "remember" the applicable default + values. The commands may be in any order except that they must + precede the FTP service request. The following commands + specify data transfer parameters: + + +Postel & Reynolds [Page 27] + + + +RFC 959 October 1985 +File Transfer Protocol + + + DATA PORT (PORT) + + The argument is a HOST-PORT specification for the data port + to be used in data connection. There are defaults for both + the user and server data ports, and under normal + circumstances this command and its reply are not needed. If + this command is used, the argument is the concatenation of a + 32-bit internet host address and a 16-bit TCP port address. + This address information is broken into 8-bit fields and the + value of each field is transmitted as a decimal number (in + character string representation). The fields are separated + by commas. A port command would be: + + PORT h1,h2,h3,h4,p1,p2 + + where h1 is the high order 8 bits of the internet host + address. + + PASSIVE (PASV) + + This command requests the server-DTP to "listen" on a data + port (which is not its default data port) and to wait for a + connection rather than initiate one upon receipt of a + transfer command. The response to this command includes the + host and port address this server is listening on. + + REPRESENTATION TYPE (TYPE) + + The argument specifies the representation type as described + in the Section on Data Representation and Storage. Several + types take a second parameter. The first parameter is + denoted by a single Telnet character, as is the second + Format parameter for ASCII and EBCDIC; the second parameter + for local byte is a decimal integer to indicate Bytesize. + The parameters are separated by a <SP> (Space, ASCII code + 32). + + The following codes are assigned for type: + + \ / + A - ASCII | | N - Non-print + |-><-| T - Telnet format effectors + E - EBCDIC| | C - Carriage Control (ASA) + / \ + I - Image + + L <byte size> - Local byte Byte size + + +Postel & Reynolds [Page 28] + + + +RFC 959 October 1985 +File Transfer Protocol + + + The default representation type is ASCII Non-print. If the + Format parameter is changed, and later just the first + argument is changed, Format then returns to the Non-print + default. + + FILE STRUCTURE (STRU) + + The argument is a single Telnet character code specifying + file structure described in the Section on Data + Representation and Storage. + + The following codes are assigned for structure: + + F - File (no record structure) + R - Record structure + P - Page structure + + The default structure is File. + + TRANSFER MODE (MODE) + + The argument is a single Telnet character code specifying + the data transfer modes described in the Section on + Transmission Modes. + + The following codes are assigned for transfer modes: + + S - Stream + B - Block + C - Compressed + + The default transfer mode is Stream. + + 4.1.3. FTP SERVICE COMMANDS + + The FTP service commands define the file transfer or the file + system function requested by the user. The argument of an FTP + service command will normally be a pathname. The syntax of + pathnames must conform to server site conventions (with + standard defaults applicable), and the language conventions of + the control connection. The suggested default handling is to + use the last specified device, directory or file name, or the + standard default defined for local users. The commands may be + in any order except that a "rename from" command must be + followed by a "rename to" command and the restart command must + be followed by the interrupted service command (e.g., STOR or + RETR). The data, when transferred in response to FTP service + + +Postel & Reynolds [Page 29] + + + +RFC 959 October 1985 +File Transfer Protocol + + + commands, shall always be sent over the data connection, except + for certain informative replies. The following commands + specify FTP service requests: + + RETRIEVE (RETR) + + This command causes the server-DTP to transfer a copy of the + file, specified in the pathname, to the server- or user-DTP + at the other end of the data connection. The status and + contents of the file at the server site shall be unaffected. + + STORE (STOR) + + This command causes the server-DTP to accept the data + transferred via the data connection and to store the data as + a file at the server site. If the file specified in the + pathname exists at the server site, then its contents shall + be replaced by the data being transferred. A new file is + created at the server site if the file specified in the + pathname does not already exist. + + STORE UNIQUE (STOU) + + This command behaves like STOR except that the resultant + file is to be created in the current directory under a name + unique to that directory. The 250 Transfer Started response + must include the name generated. + + APPEND (with create) (APPE) + + This command causes the server-DTP to accept the data + transferred via the data connection and to store the data in + a file at the server site. If the file specified in the + pathname exists at the server site, then the data shall be + appended to that file; otherwise the file specified in the + pathname shall be created at the server site. + + ALLOCATE (ALLO) + + This command may be required by some servers to reserve + sufficient storage to accommodate the new file to be + transferred. The argument shall be a decimal integer + representing the number of bytes (using the logical byte + size) of storage to be reserved for the file. For files + sent with record or page structure a maximum record or page + size (in logical bytes) might also be necessary; this is + indicated by a decimal integer in a second argument field of + + +Postel & Reynolds [Page 30] + + + +RFC 959 October 1985 +File Transfer Protocol + + + the command. This second argument is optional, but when + present should be separated from the first by the three + Telnet characters <SP> R <SP>. This command shall be + followed by a STORe or APPEnd command. The ALLO command + should be treated as a NOOP (no operation) by those servers + which do not require that the maximum size of the file be + declared beforehand, and those servers interested in only + the maximum record or page size should accept a dummy value + in the first argument and ignore it. + + RESTART (REST) + + The argument field represents the server marker at which + file transfer is to be restarted. This command does not + cause file transfer but skips over the file to the specified + data checkpoint. This command shall be immediately followed + by the appropriate FTP service command which shall cause + file transfer to resume. + + RENAME FROM (RNFR) + + This command specifies the old pathname of the file which is + to be renamed. This command must be immediately followed by + a "rename to" command specifying the new file pathname. + + RENAME TO (RNTO) + + This command specifies the new pathname of the file + specified in the immediately preceding "rename from" + command. Together the two commands cause a file to be + renamed. + + ABORT (ABOR) + + This command tells the server to abort the previous FTP + service command and any associated transfer of data. The + abort command may require "special action", as discussed in + the Section on FTP Commands, to force recognition by the + server. No action is to be taken if the previous command + has been completed (including data transfer). The control + connection is not to be closed by the server, but the data + connection must be closed. + + There are two cases for the server upon receipt of this + command: (1) the FTP service command was already completed, + or (2) the FTP service command is still in progress. + + + +Postel & Reynolds [Page 31] + + + +RFC 959 October 1985 +File Transfer Protocol + + + In the first case, the server closes the data connection + (if it is open) and responds with a 226 reply, indicating + that the abort command was successfully processed. + + In the second case, the server aborts the FTP service in + progress and closes the data connection, returning a 426 + reply to indicate that the service request terminated + abnormally. The server then sends a 226 reply, + indicating that the abort command was successfully + processed. + + DELETE (DELE) + + This command causes the file specified in the pathname to be + deleted at the server site. If an extra level of protection + is desired (such as the query, "Do you really wish to + delete?"), it should be provided by the user-FTP process. + + REMOVE DIRECTORY (RMD) + + This command causes the directory specified in the pathname + to be removed as a directory (if the pathname is absolute) + or as a subdirectory of the current working directory (if + the pathname is relative). See Appendix II. + + MAKE DIRECTORY (MKD) + + This command causes the directory specified in the pathname + to be created as a directory (if the pathname is absolute) + or as a subdirectory of the current working directory (if + the pathname is relative). See Appendix II. + + PRINT WORKING DIRECTORY (PWD) + + This command causes the name of the current working + directory to be returned in the reply. See Appendix II. + + LIST (LIST) + + This command causes a list to be sent from the server to the + passive DTP. If the pathname specifies a directory or other + group of files, the server should transfer a list of files + in the specified directory. If the pathname specifies a + file then the server should send current information on the + file. A null argument implies the user's current working or + default directory. The data transfer is over the data + connection in type ASCII or type EBCDIC. (The user must + + +Postel & Reynolds [Page 32] + + + +RFC 959 October 1985 +File Transfer Protocol + + + ensure that the TYPE is appropriately ASCII or EBCDIC). + Since the information on a file may vary widely from system + to system, this information may be hard to use automatically + in a program, but may be quite useful to a human user. + + NAME LIST (NLST) + + This command causes a directory listing to be sent from + server to user site. The pathname should specify a + directory or other system-specific file group descriptor; a + null argument implies the current directory. The server + will return a stream of names of files and no other + information. The data will be transferred in ASCII or + EBCDIC type over the data connection as valid pathname + strings separated by <CRLF> or <NL>. (Again the user must + ensure that the TYPE is correct.) This command is intended + to return information that can be used by a program to + further process the files automatically. For example, in + the implementation of a "multiple get" function. + + SITE PARAMETERS (SITE) + + This command is used by the server to provide services + specific to his system that are essential to file transfer + but not sufficiently universal to be included as commands in + the protocol. The nature of these services and the + specification of their syntax can be stated in a reply to + the HELP SITE command. + + SYSTEM (SYST) + + This command is used to find out the type of operating + system at the server. The reply shall have as its first + word one of the system names listed in the current version + of the Assigned Numbers document [4]. + + STATUS (STAT) + + This command shall cause a status response to be sent over + the control connection in the form of a reply. The command + may be sent during a file transfer (along with the Telnet IP + and Synch signals--see the Section on FTP Commands) in which + case the server will respond with the status of the + operation in progress, or it may be sent between file + transfers. In the latter case, the command may have an + argument field. If the argument is a pathname, the command + is analogous to the "list" command except that data shall be + + +Postel & Reynolds [Page 33] + + + +RFC 959 October 1985 +File Transfer Protocol + + + transferred over the control connection. If a partial + pathname is given, the server may respond with a list of + file names or attributes associated with that specification. + If no argument is given, the server should return general + status information about the server FTP process. This + should include current values of all transfer parameters and + the status of connections. + + HELP (HELP) + + This command shall cause the server to send helpful + information regarding its implementation status over the + control connection to the user. The command may take an + argument (e.g., any command name) and return more specific + information as a response. The reply is type 211 or 214. + It is suggested that HELP be allowed before entering a USER + command. The server may use this reply to specify + site-dependent parameters, e.g., in response to HELP SITE. + + NOOP (NOOP) + + This command does not affect any parameters or previously + entered commands. It specifies no action other than that the + server send an OK reply. + + The File Transfer Protocol follows the specifications of the Telnet + protocol for all communications over the control connection. Since + the language used for Telnet communication may be a negotiated + option, all references in the next two sections will be to the + "Telnet language" and the corresponding "Telnet end-of-line code". + Currently, one may take these to mean NVT-ASCII and <CRLF>. No other + specifications of the Telnet protocol will be cited. + + FTP commands are "Telnet strings" terminated by the "Telnet end of + line code". The command codes themselves are alphabetic characters + terminated by the character <SP> (Space) if parameters follow and + Telnet-EOL otherwise. The command codes and the semantics of + commands are described in this section; the detailed syntax of + commands is specified in the Section on Commands, the reply sequences + are discussed in the Section on Sequencing of Commands and Replies, + and scenarios illustrating the use of commands are provided in the + Section on Typical FTP Scenarios. + + FTP commands may be partitioned as those specifying access-control + identifiers, data transfer parameters, or FTP service requests. + Certain commands (such as ABOR, STAT, QUIT) may be sent over the + control connection while a data transfer is in progress. Some + + +Postel & Reynolds [Page 34] + + + +RFC 959 October 1985 +File Transfer Protocol + + + servers may not be able to monitor the control and data connections + simultaneously, in which case some special action will be necessary + to get the server's attention. The following ordered format is + tentatively recommended: + + 1. User system inserts the Telnet "Interrupt Process" (IP) signal + in the Telnet stream. + + 2. User system sends the Telnet "Synch" signal. + + 3. User system inserts the command (e.g., ABOR) in the Telnet + stream. + + 4. Server PI, after receiving "IP", scans the Telnet stream for + EXACTLY ONE FTP command. + + (For other servers this may not be necessary but the actions listed + above should have no unusual effect.) + + 4.2. FTP REPLIES + + Replies to File Transfer Protocol commands are devised to ensure + the synchronization of requests and actions in the process of file + transfer, and to guarantee that the user process always knows the + state of the Server. Every command must generate at least one + reply, although there may be more than one; in the latter case, + the multiple replies must be easily distinguished. In addition, + some commands occur in sequential groups, such as USER, PASS and + ACCT, or RNFR and RNTO. The replies show the existence of an + intermediate state if all preceding commands have been successful. + A failure at any point in the sequence necessitates the repetition + of the entire sequence from the beginning. + + The details of the command-reply sequence are made explicit in + a set of state diagrams below. + + An FTP reply consists of a three digit number (transmitted as + three alphanumeric characters) followed by some text. The number + is intended for use by automata to determine what state to enter + next; the text is intended for the human user. It is intended + that the three digits contain enough encoded information that the + user-process (the User-PI) will not need to examine the text and + may either discard it or pass it on to the user, as appropriate. + In particular, the text may be server-dependent, so there are + likely to be varying texts for each reply code. + + A reply is defined to contain the 3-digit code, followed by Space + + +Postel & Reynolds [Page 35] + + + +RFC 959 October 1985 +File Transfer Protocol + + + <SP>, followed by one line of text (where some maximum line length + has been specified), and terminated by the Telnet end-of-line + code. There will be cases however, where the text is longer than + a single line. In these cases the complete text must be bracketed + so the User-process knows when it may stop reading the reply (i.e. + stop processing input on the control connection) and go do other + things. This requires a special format on the first line to + indicate that more than one line is coming, and another on the + last line to designate it as the last. At least one of these must + contain the appropriate reply code to indicate the state of the + transaction. To satisfy all factions, it was decided that both + the first and last line codes should be the same. + + Thus the format for multi-line replies is that the first line + will begin with the exact required reply code, followed + immediately by a Hyphen, "-" (also known as Minus), followed by + text. The last line will begin with the same code, followed + immediately by Space <SP>, optionally some text, and the Telnet + end-of-line code. + + For example: + 123-First line + Second line + 234 A line beginning with numbers + 123 The last line + + The user-process then simply needs to search for the second + occurrence of the same reply code, followed by <SP> (Space), at + the beginning of a line, and ignore all intermediary lines. If + an intermediary line begins with a 3-digit number, the Server + must pad the front to avoid confusion. + + This scheme allows standard system routines to be used for + reply information (such as for the STAT reply), with + "artificial" first and last lines tacked on. In rare cases + where these routines are able to generate three digits and a + Space at the beginning of any line, the beginning of each + text line should be offset by some neutral text, like Space. + + This scheme assumes that multi-line replies may not be nested. + + The three digits of the reply each have a special significance. + This is intended to allow a range of very simple to very + sophisticated responses by the user-process. The first digit + denotes whether the response is good, bad or incomplete. + (Referring to the state diagram), an unsophisticated user-process + will be able to determine its next action (proceed as planned, + + +Postel & Reynolds [Page 36] + + + +RFC 959 October 1985 +File Transfer Protocol + + + redo, retrench, etc.) by simply examining this first digit. A + user-process that wants to know approximately what kind of error + occurred (e.g. file system error, command syntax error) may + examine the second digit, reserving the third digit for the finest + gradation of information (e.g., RNTO command without a preceding + RNFR). + + There are five values for the first digit of the reply code: + + 1yz Positive Preliminary reply + + The requested action is being initiated; expect another + reply before proceeding with a new command. (The + user-process sending another command before the + completion reply would be in violation of protocol; but + server-FTP processes should queue any commands that + arrive while a preceding command is in progress.) This + type of reply can be used to indicate that the command + was accepted and the user-process may now pay attention + to the data connections, for implementations where + simultaneous monitoring is difficult. The server-FTP + process may send at most, one 1yz reply per command. + + 2yz Positive Completion reply + + The requested action has been successfully completed. A + new request may be initiated. + + 3yz Positive Intermediate reply + + The command has been accepted, but the requested action + is being held in abeyance, pending receipt of further + information. The user should send another command + specifying this information. This reply is used in + command sequence groups. + + 4yz Transient Negative Completion reply + + The command was not accepted and the requested action did + not take place, but the error condition is temporary and + the action may be requested again. The user should + return to the beginning of the command sequence, if any. + It is difficult to assign a meaning to "transient", + particularly when two distinct sites (Server- and + User-processes) have to agree on the interpretation. + Each reply in the 4yz category might have a slightly + different time value, but the intent is that the + + +Postel & Reynolds [Page 37] + + + +RFC 959 October 1985 +File Transfer Protocol + + + user-process is encouraged to try again. A rule of thumb + in determining if a reply fits into the 4yz or the 5yz + (Permanent Negative) category is that replies are 4yz if + the commands can be repeated without any change in + command form or in properties of the User or Server + (e.g., the command is spelled the same with the same + arguments used; the user does not change his file access + or user name; the server does not put up a new + implementation.) + + 5yz Permanent Negative Completion reply + + The command was not accepted and the requested action did + not take place. The User-process is discouraged from + repeating the exact request (in the same sequence). Even + some "permanent" error conditions can be corrected, so + the human user may want to direct his User-process to + reinitiate the command sequence by direct action at some + point in the future (e.g., after the spelling has been + changed, or the user has altered his directory status.) + + The following function groupings are encoded in the second + digit: + + x0z Syntax - These replies refer to syntax errors, + syntactically correct commands that don't fit any + functional category, unimplemented or superfluous + commands. + + x1z Information - These are replies to requests for + information, such as status or help. + + x2z Connections - Replies referring to the control and + data connections. + + x3z Authentication and accounting - Replies for the login + process and accounting procedures. + + x4z Unspecified as yet. + + x5z File system - These replies indicate the status of the + Server file system vis-a-vis the requested transfer or + other file system action. + + The third digit gives a finer gradation of meaning in each of + the function categories, specified by the second digit. The + list of replies below will illustrate this. Note that the text + + +Postel & Reynolds [Page 38] + + + +RFC 959 October 1985 +File Transfer Protocol + + + associated with each reply is recommended, rather than + mandatory, and may even change according to the command with + which it is associated. The reply codes, on the other hand, + must strictly follow the specifications in the last section; + that is, Server implementations should not invent new codes for + situations that are only slightly different from the ones + described here, but rather should adapt codes already defined. + + A command such as TYPE or ALLO whose successful execution + does not offer the user-process any new information will + cause a 200 reply to be returned. If the command is not + implemented by a particular Server-FTP process because it + has no relevance to that computer system, for example ALLO + at a TOPS20 site, a Positive Completion reply is still + desired so that the simple User-process knows it can proceed + with its course of action. A 202 reply is used in this case + with, for example, the reply text: "No storage allocation + necessary." If, on the other hand, the command requests a + non-site-specific action and is unimplemented, the response + is 502. A refinement of that is the 504 reply for a command + that is implemented, but that requests an unimplemented + parameter. + + 4.2.1 Reply Codes by Function Groups + + 200 Command okay. + 500 Syntax error, command unrecognized. + This may include errors such as command line too long. + 501 Syntax error in parameters or arguments. + 202 Command not implemented, superfluous at this site. + 502 Command not implemented. + 503 Bad sequence of commands. + 504 Command not implemented for that parameter. + + + + + + + + + + + + + + + + +Postel & Reynolds [Page 39] + + + +RFC 959 October 1985 +File Transfer Protocol + + + 110 Restart marker reply. + In this case, the text is exact and not left to the + particular implementation; it must read: + MARK yyyy = mmmm + Where yyyy is User-process data stream marker, and mmmm + server's equivalent marker (note the spaces between markers + and "="). + 211 System status, or system help reply. + 212 Directory status. + 213 File status. + 214 Help message. + On how to use the server or the meaning of a particular + non-standard command. This reply is useful only to the + human user. + 215 NAME system type. + Where NAME is an official system name from the list in the + Assigned Numbers document. + + 120 Service ready in nnn minutes. + 220 Service ready for new user. + 221 Service closing control connection. + Logged out if appropriate. + 421 Service not available, closing control connection. + This may be a reply to any command if the service knows it + must shut down. + 125 Data connection already open; transfer starting. + 225 Data connection open; no transfer in progress. + 425 Can't open data connection. + 226 Closing data connection. + Requested file action successful (for example, file + transfer or file abort). + 426 Connection closed; transfer aborted. + 227 Entering Passive Mode (h1,h2,h3,h4,p1,p2). + + 230 User logged in, proceed. + 530 Not logged in. + 331 User name okay, need password. + 332 Need account for login. + 532 Need account for storing files. + + + + + + + + + + +Postel & Reynolds [Page 40] + + + +RFC 959 October 1985 +File Transfer Protocol + + + 150 File status okay; about to open data connection. + 250 Requested file action okay, completed. + 257 "PATHNAME" created. + 350 Requested file action pending further information. + 450 Requested file action not taken. + File unavailable (e.g., file busy). + 550 Requested action not taken. + File unavailable (e.g., file not found, no access). + 451 Requested action aborted. Local error in processing. + 551 Requested action aborted. Page type unknown. + 452 Requested action not taken. + Insufficient storage space in system. + 552 Requested file action aborted. + Exceeded storage allocation (for current directory or + dataset). + 553 Requested action not taken. + File name not allowed. + + + 4.2.2 Numeric Order List of Reply Codes + + 110 Restart marker reply. + In this case, the text is exact and not left to the + particular implementation; it must read: + MARK yyyy = mmmm + Where yyyy is User-process data stream marker, and mmmm + server's equivalent marker (note the spaces between markers + and "="). + 120 Service ready in nnn minutes. + 125 Data connection already open; transfer starting. + 150 File status okay; about to open data connection. + + + + + + + + + + + + + + + + + + +Postel & Reynolds [Page 41] + + + +RFC 959 October 1985 +File Transfer Protocol + + + 200 Command okay. + 202 Command not implemented, superfluous at this site. + 211 System status, or system help reply. + 212 Directory status. + 213 File status. + 214 Help message. + On how to use the server or the meaning of a particular + non-standard command. This reply is useful only to the + human user. + 215 NAME system type. + Where NAME is an official system name from the list in the + Assigned Numbers document. + 220 Service ready for new user. + 221 Service closing control connection. + Logged out if appropriate. + 225 Data connection open; no transfer in progress. + 226 Closing data connection. + Requested file action successful (for example, file + transfer or file abort). + 227 Entering Passive Mode (h1,h2,h3,h4,p1,p2). + 230 User logged in, proceed. + 250 Requested file action okay, completed. + 257 "PATHNAME" created. + + 331 User name okay, need password. + 332 Need account for login. + 350 Requested file action pending further information. + + 421 Service not available, closing control connection. + This may be a reply to any command if the service knows it + must shut down. + 425 Can't open data connection. + 426 Connection closed; transfer aborted. + 450 Requested file action not taken. + File unavailable (e.g., file busy). + 451 Requested action aborted: local error in processing. + 452 Requested action not taken. + Insufficient storage space in system. + + + + + + + + + + + +Postel & Reynolds [Page 42] + + + +RFC 959 October 1985 +File Transfer Protocol + + + 500 Syntax error, command unrecognized. + This may include errors such as command line too long. + 501 Syntax error in parameters or arguments. + 502 Command not implemented. + 503 Bad sequence of commands. + 504 Command not implemented for that parameter. + 530 Not logged in. + 532 Need account for storing files. + 550 Requested action not taken. + File unavailable (e.g., file not found, no access). + 551 Requested action aborted: page type unknown. + 552 Requested file action aborted. + Exceeded storage allocation (for current directory or + dataset). + 553 Requested action not taken. + File name not allowed. + + +5. DECLARATIVE SPECIFICATIONS + + 5.1. MINIMUM IMPLEMENTATION + + In order to make FTP workable without needless error messages, the + following minimum implementation is required for all servers: + + TYPE - ASCII Non-print + MODE - Stream + STRUCTURE - File, Record + COMMANDS - USER, QUIT, PORT, + TYPE, MODE, STRU, + for the default values + RETR, STOR, + NOOP. + + The default values for transfer parameters are: + + TYPE - ASCII Non-print + MODE - Stream + STRU - File + + All hosts must accept the above as the standard defaults. + + + + + + + + +Postel & Reynolds [Page 43] + + + +RFC 959 October 1985 +File Transfer Protocol + + + 5.2. CONNECTIONS + + The server protocol interpreter shall "listen" on Port L. The + user or user protocol interpreter shall initiate the full-duplex + control connection. Server- and user- processes should follow the + conventions of the Telnet protocol as specified in the + ARPA-Internet Protocol Handbook [1]. Servers are under no + obligation to provide for editing of command lines and may require + that it be done in the user host. The control connection shall be + closed by the server at the user's request after all transfers and + replies are completed. + + The user-DTP must "listen" on the specified data port; this may be + the default user port (U) or a port specified in the PORT command. + The server shall initiate the data connection from his own default + data port (L-1) using the specified user data port. The direction + of the transfer and the port used will be determined by the FTP + service command. + + Note that all FTP implementation must support data transfer using + the default port, and that only the USER-PI may initiate the use + of non-default ports. + + When data is to be transferred between two servers, A and B (refer + to Figure 2), the user-PI, C, sets up control connections with + both server-PI's. One of the servers, say A, is then sent a PASV + command telling him to "listen" on his data port rather than + initiate a connection when he receives a transfer service command. + When the user-PI receives an acknowledgment to the PASV command, + which includes the identity of the host and port being listened + on, the user-PI then sends A's port, a, to B in a PORT command; a + reply is returned. The user-PI may then send the corresponding + service commands to A and B. Server B initiates the connection + and the transfer proceeds. The command-reply sequence is listed + below where the messages are vertically synchronous but + horizontally asynchronous: + + + + + + + + + + + + + +Postel & Reynolds [Page 44] + + + +RFC 959 October 1985 +File Transfer Protocol + + + User-PI - Server A User-PI - Server B + ------------------ ------------------ + + C->A : Connect C->B : Connect + C->A : PASV + A->C : 227 Entering Passive Mode. A1,A2,A3,A4,a1,a2 + C->B : PORT A1,A2,A3,A4,a1,a2 + B->C : 200 Okay + C->A : STOR C->B : RETR + B->A : Connect to HOST-A, PORT-a + + Figure 3 + + The data connection shall be closed by the server under the + conditions described in the Section on Establishing Data + Connections. If the data connection is to be closed following a + data transfer where closing the connection is not required to + indicate the end-of-file, the server must do so immediately. + Waiting until after a new transfer command is not permitted + because the user-process will have already tested the data + connection to see if it needs to do a "listen"; (remember that the + user must "listen" on a closed data port BEFORE sending the + transfer request). To prevent a race condition here, the server + sends a reply (226) after closing the data connection (or if the + connection is left open, a "file transfer completed" reply (250) + and the user-PI should wait for one of these replies before + issuing a new transfer command). + + Any time either the user or server see that the connection is + being closed by the other side, it should promptly read any + remaining data queued on the connection and issue the close on its + own side. + + 5.3. COMMANDS + + The commands are Telnet character strings transmitted over the + control connections as described in the Section on FTP Commands. + The command functions and semantics are described in the Section + on Access Control Commands, Transfer Parameter Commands, FTP + Service Commands, and Miscellaneous Commands. The command syntax + is specified here. + + The commands begin with a command code followed by an argument + field. The command codes are four or fewer alphabetic characters. + Upper and lower case alphabetic characters are to be treated + identically. Thus, any of the following may represent the + retrieve command: + + +Postel & Reynolds [Page 45] + + + +RFC 959 October 1985 +File Transfer Protocol + + + RETR Retr retr ReTr rETr + + This also applies to any symbols representing parameter values, + such as A or a for ASCII TYPE. The command codes and the argument + fields are separated by one or more spaces. + + The argument field consists of a variable length character string + ending with the character sequence <CRLF> (Carriage Return, Line + Feed) for NVT-ASCII representation; for other negotiated languages + a different end of line character might be used. It should be + noted that the server is to take no action until the end of line + code is received. + + The syntax is specified below in NVT-ASCII. All characters in the + argument field are ASCII characters including any ASCII + represented decimal integers. Square brackets denote an optional + argument field. If the option is not taken, the appropriate + default is implied. + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + +Postel & Reynolds [Page 46] + + + +RFC 959 October 1985 +File Transfer Protocol + + + 5.3.1. FTP COMMANDS + + The following are the FTP commands: + + USER <SP> <username> <CRLF> + PASS <SP> <password> <CRLF> + ACCT <SP> <account-information> <CRLF> + CWD <SP> <pathname> <CRLF> + CDUP <CRLF> + SMNT <SP> <pathname> <CRLF> + QUIT <CRLF> + REIN <CRLF> + PORT <SP> <host-port> <CRLF> + PASV <CRLF> + TYPE <SP> <type-code> <CRLF> + STRU <SP> <structure-code> <CRLF> + MODE <SP> <mode-code> <CRLF> + RETR <SP> <pathname> <CRLF> + STOR <SP> <pathname> <CRLF> + STOU <CRLF> + APPE <SP> <pathname> <CRLF> + ALLO <SP> <decimal-integer> + [<SP> R <SP> <decimal-integer>] <CRLF> + REST <SP> <marker> <CRLF> + RNFR <SP> <pathname> <CRLF> + RNTO <SP> <pathname> <CRLF> + ABOR <CRLF> + DELE <SP> <pathname> <CRLF> + RMD <SP> <pathname> <CRLF> + MKD <SP> <pathname> <CRLF> + PWD <CRLF> + LIST [<SP> <pathname>] <CRLF> + NLST [<SP> <pathname>] <CRLF> + SITE <SP> <string> <CRLF> + SYST <CRLF> + STAT [<SP> <pathname>] <CRLF> + HELP [<SP> <string>] <CRLF> + NOOP <CRLF> + + + + + + + + + + + +Postel & Reynolds [Page 47] + + + +RFC 959 October 1985 +File Transfer Protocol + + + 5.3.2. FTP COMMAND ARGUMENTS + + The syntax of the above argument fields (using BNF notation + where applicable) is: + + <username> ::= <string> + <password> ::= <string> + <account-information> ::= <string> + <string> ::= <char> | <char><string> + <char> ::= any of the 128 ASCII characters except <CR> and + <LF> + <marker> ::= <pr-string> + <pr-string> ::= <pr-char> | <pr-char><pr-string> + <pr-char> ::= printable characters, any + ASCII code 33 through 126 + <byte-size> ::= <number> + <host-port> ::= <host-number>,<port-number> + <host-number> ::= <number>,<number>,<number>,<number> + <port-number> ::= <number>,<number> + <number> ::= any decimal integer 1 through 255 + <form-code> ::= N | T | C + <type-code> ::= A [<sp> <form-code>] + | E [<sp> <form-code>] + | I + | L <sp> <byte-size> + <structure-code> ::= F | R | P + <mode-code> ::= S | B | C + <pathname> ::= <string> + <decimal-integer> ::= any decimal integer + + + + + + + + + + + + + + + + + + + + +Postel & Reynolds [Page 48] + + + +RFC 959 October 1985 +File Transfer Protocol + + + 5.4. SEQUENCING OF COMMANDS AND REPLIES + + The communication between the user and server is intended to be an + alternating dialogue. As such, the user issues an FTP command and + the server responds with a prompt primary reply. The user should + wait for this initial primary success or failure response before + sending further commands. + + Certain commands require a second reply for which the user should + also wait. These replies may, for example, report on the progress + or completion of file transfer or the closing of the data + connection. They are secondary replies to file transfer commands. + + One important group of informational replies is the connection + greetings. Under normal circumstances, a server will send a 220 + reply, "awaiting input", when the connection is completed. The + user should wait for this greeting message before sending any + commands. If the server is unable to accept input right away, a + 120 "expected delay" reply should be sent immediately and a 220 + reply when ready. The user will then know not to hang up if there + is a delay. + + Spontaneous Replies + + Sometimes "the system" spontaneously has a message to be sent + to a user (usually all users). For example, "System going down + in 15 minutes". There is no provision in FTP for such + spontaneous information to be sent from the server to the user. + It is recommended that such information be queued in the + server-PI and delivered to the user-PI in the next reply + (possibly making it a multi-line reply). + + The table below lists alternative success and failure replies for + each command. These must be strictly adhered to; a server may + substitute text in the replies, but the meaning and action implied + by the code numbers and by the specific command reply sequence + cannot be altered. + + Command-Reply Sequences + + In this section, the command-reply sequence is presented. Each + command is listed with its possible replies; command groups are + listed together. Preliminary replies are listed first (with + their succeeding replies indented and under them), then + positive and negative completion, and finally intermediary + + + + +Postel & Reynolds [Page 49] + + + +RFC 959 October 1985 +File Transfer Protocol + + + replies with the remaining commands from the sequence + following. This listing forms the basis for the state + diagrams, which will be presented separately. + + Connection Establishment + 120 + 220 + 220 + 421 + Login + USER + 230 + 530 + 500, 501, 421 + 331, 332 + PASS + 230 + 202 + 530 + 500, 501, 503, 421 + 332 + ACCT + 230 + 202 + 530 + 500, 501, 503, 421 + CWD + 250 + 500, 501, 502, 421, 530, 550 + CDUP + 200 + 500, 501, 502, 421, 530, 550 + SMNT + 202, 250 + 500, 501, 502, 421, 530, 550 + Logout + REIN + 120 + 220 + 220 + 421 + 500, 502 + QUIT + 221 + 500 + + + + +Postel & Reynolds [Page 50] + + + +RFC 959 October 1985 +File Transfer Protocol + + + Transfer parameters + PORT + 200 + 500, 501, 421, 530 + PASV + 227 + 500, 501, 502, 421, 530 + MODE + 200 + 500, 501, 504, 421, 530 + TYPE + 200 + 500, 501, 504, 421, 530 + STRU + 200 + 500, 501, 504, 421, 530 + File action commands + ALLO + 200 + 202 + 500, 501, 504, 421, 530 + REST + 500, 501, 502, 421, 530 + 350 + STOR + 125, 150 + (110) + 226, 250 + 425, 426, 451, 551, 552 + 532, 450, 452, 553 + 500, 501, 421, 530 + STOU + 125, 150 + (110) + 226, 250 + 425, 426, 451, 551, 552 + 532, 450, 452, 553 + 500, 501, 421, 530 + RETR + 125, 150 + (110) + 226, 250 + 425, 426, 451 + 450, 550 + 500, 501, 421, 530 + + + + +Postel & Reynolds [Page 51] + + + +RFC 959 October 1985 +File Transfer Protocol + + + LIST + 125, 150 + 226, 250 + 425, 426, 451 + 450 + 500, 501, 502, 421, 530 + NLST + 125, 150 + 226, 250 + 425, 426, 451 + 450 + 500, 501, 502, 421, 530 + APPE + 125, 150 + (110) + 226, 250 + 425, 426, 451, 551, 552 + 532, 450, 550, 452, 553 + 500, 501, 502, 421, 530 + RNFR + 450, 550 + 500, 501, 502, 421, 530 + 350 + RNTO + 250 + 532, 553 + 500, 501, 502, 503, 421, 530 + DELE + 250 + 450, 550 + 500, 501, 502, 421, 530 + RMD + 250 + 500, 501, 502, 421, 530, 550 + MKD + 257 + 500, 501, 502, 421, 530, 550 + PWD + 257 + 500, 501, 502, 421, 550 + ABOR + 225, 226 + 500, 501, 502, 421 + + + + + + +Postel & Reynolds [Page 52] + + + +RFC 959 October 1985 +File Transfer Protocol + + + Informational commands + SYST + 215 + 500, 501, 502, 421 + STAT + 211, 212, 213 + 450 + 500, 501, 502, 421, 530 + HELP + 211, 214 + 500, 501, 502, 421 + Miscellaneous commands + SITE + 200 + 202 + 500, 501, 530 + NOOP + 200 + 500 421 + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + +Postel & Reynolds [Page 53] + + + +RFC 959 October 1985 +File Transfer Protocol + + +6. STATE DIAGRAMS + + Here we present state diagrams for a very simple minded FTP + implementation. Only the first digit of the reply codes is used. + There is one state diagram for each group of FTP commands or command + sequences. + + The command groupings were determined by constructing a model for + each command then collecting together the commands with structurally + identical models. + + For each command or command sequence there are three possible + outcomes: success (S), failure (F), and error (E). In the state + diagrams below we use the symbol B for "begin", and the symbol W for + "wait for reply". + + We first present the diagram that represents the largest group of FTP + commands: + + + 1,3 +---+ + ----------->| E | + | +---+ + | + +---+ cmd +---+ 2 +---+ + | B |---------->| W |---------->| S | + +---+ +---+ +---+ + | + | 4,5 +---+ + ----------->| F | + +---+ + + + This diagram models the commands: + + ABOR, ALLO, DELE, CWD, CDUP, SMNT, HELP, MODE, NOOP, PASV, + QUIT, SITE, PORT, SYST, STAT, RMD, MKD, PWD, STRU, and TYPE. + + + + + + + + + + + + +Postel & Reynolds [Page 54] + + + +RFC 959 October 1985 +File Transfer Protocol + + + The other large group of commands is represented by a very similar + diagram: + + + 3 +---+ + ----------->| E | + | +---+ + | + +---+ cmd +---+ 2 +---+ + | B |---------->| W |---------->| S | + +---+ --->+---+ +---+ + | | | + | | | 4,5 +---+ + | 1 | ----------->| F | + ----- +---+ + + + This diagram models the commands: + + APPE, LIST, NLST, REIN, RETR, STOR, and STOU. + + Note that this second model could also be used to represent the first + group of commands, the only difference being that in the first group + the 100 series replies are unexpected and therefore treated as error, + while the second group expects (some may require) 100 series replies. + Remember that at most, one 100 series reply is allowed per command. + + The remaining diagrams model command sequences, perhaps the simplest + of these is the rename sequence: + + + +---+ RNFR +---+ 1,2 +---+ + | B |---------->| W |---------->| E | + +---+ +---+ -->+---+ + | | | + 3 | | 4,5 | + -------------- ------ | + | | | +---+ + | ------------->| S | + | | 1,3 | | +---+ + | 2| -------- + | | | | + V | | | + +---+ RNTO +---+ 4,5 ----->+---+ + | |---------->| W |---------->| F | + +---+ +---+ +---+ + + + +Postel & Reynolds [Page 55] + + + +RFC 959 October 1985 +File Transfer Protocol + + + The next diagram is a simple model of the Restart command: + + + +---+ REST +---+ 1,2 +---+ + | B |---------->| W |---------->| E | + +---+ +---+ -->+---+ + | | | + 3 | | 4,5 | + -------------- ------ | + | | | +---+ + | ------------->| S | + | | 3 | | +---+ + | 2| -------- + | | | | + V | | | + +---+ cmd +---+ 4,5 ----->+---+ + | |---------->| W |---------->| F | + +---+ -->+---+ +---+ + | | + | 1 | + ------ + + + Where "cmd" is APPE, STOR, or RETR. + + We note that the above three models are similar. The Restart differs + from the Rename two only in the treatment of 100 series replies at + the second stage, while the second group expects (some may require) + 100 series replies. Remember that at most, one 100 series reply is + allowed per command. + + + + + + + + + + + + + + + + + + + +Postel & Reynolds [Page 56] + + + +RFC 959 October 1985 +File Transfer Protocol + + + The most complicated diagram is for the Login sequence: + + + 1 + +---+ USER +---+------------->+---+ + | B |---------->| W | 2 ---->| E | + +---+ +---+------ | -->+---+ + | | | | | + 3 | | 4,5 | | | + -------------- ----- | | | + | | | | | + | | | | | + | --------- | + | 1| | | | + V | | | | + +---+ PASS +---+ 2 | ------>+---+ + | |---------->| W |------------->| S | + +---+ +---+ ---------->+---+ + | | | | | + 3 | |4,5| | | + -------------- -------- | + | | | | | + | | | | | + | ----------- + | 1,3| | | | + V | 2| | | + +---+ ACCT +---+-- | ----->+---+ + | |---------->| W | 4,5 -------->| F | + +---+ +---+------------->+---+ + + + + + + + + + + + + + + + + + + + + +Postel & Reynolds [Page 57] + + + +RFC 959 October 1985 +File Transfer Protocol + + + Finally, we present a generalized diagram that could be used to model + the command and reply interchange: + + + ------------------------------------ + | | + Begin | | + | V | + | +---+ cmd +---+ 2 +---+ | + -->| |------->| |---------->| | | + | | | W | | S |-----| + -->| | -->| |----- | | | + | +---+ | +---+ 4,5 | +---+ | + | | | | | | | + | | | 1| |3 | +---+ | + | | | | | | | | | + | | ---- | ---->| F |----- + | | | | | + | | | +---+ + ------------------- + | + | + V + End + + + + + + + + + + + + + + + + + + + + + + + + + +Postel & Reynolds [Page 58] + + + +RFC 959 October 1985 +File Transfer Protocol + + +7. TYPICAL FTP SCENARIO + + User at host U wanting to transfer files to/from host S: + + In general, the user will communicate to the server via a mediating + user-FTP process. The following may be a typical scenario. The + user-FTP prompts are shown in parentheses, '---->' represents + commands from host U to host S, and '<----' represents replies from + host S to host U. + + LOCAL COMMANDS BY USER ACTION INVOLVED + + ftp (host) multics<CR> Connect to host S, port L, + establishing control connections. + <---- 220 Service ready <CRLF>. + username Doe <CR> USER Doe<CRLF>----> + <---- 331 User name ok, + need password<CRLF>. + password mumble <CR> PASS mumble<CRLF>----> + <---- 230 User logged in<CRLF>. + retrieve (local type) ASCII<CR> + (local pathname) test 1 <CR> User-FTP opens local file in ASCII. + (for. pathname) test.pl1<CR> RETR test.pl1<CRLF> ----> + <---- 150 File status okay; + about to open data + connection<CRLF>. + Server makes data connection + to port U. + + <---- 226 Closing data connection, + file transfer successful<CRLF>. + type Image<CR> TYPE I<CRLF> ----> + <---- 200 Command OK<CRLF> + store (local type) image<CR> + (local pathname) file dump<CR> User-FTP opens local file in Image. + (for.pathname) >udd>cn>fd<CR> STOR >udd>cn>fd<CRLF> ----> + <---- 550 Access denied<CRLF> + terminate QUIT <CRLF> ----> + Server closes all + connections. + +8. CONNECTION ESTABLISHMENT + + The FTP control connection is established via TCP between the user + process port U and the server process port L. This protocol is + assigned the service port 21 (25 octal), that is L=21. + + + +Postel & Reynolds [Page 59] + + + +RFC 959 October 1985 +File Transfer Protocol + + +APPENDIX I - PAGE STRUCTURE + + The need for FTP to support page structure derives principally from + the need to support efficient transmission of files between TOPS-20 + systems, particularly the files used by NLS. + + The file system of TOPS-20 is based on the concept of pages. The + operating system is most efficient at manipulating files as pages. + The operating system provides an interface to the file system so that + many applications view files as sequential streams of characters. + However, a few applications use the underlying page structures + directly, and some of these create holey files. + + A TOPS-20 disk file consists of four things: a pathname, a page + table, a (possibly empty) set of pages, and a set of attributes. + + The pathname is specified in the RETR or STOR command. It includes + the directory name, file name, file name extension, and generation + number. + + The page table contains up to 2**18 entries. Each entry may be + EMPTY, or may point to a page. If it is not empty, there are also + some page-specific access bits; not all pages of a file need have the + same access protection. + + A page is a contiguous set of 512 words of 36 bits each. + + The attributes of the file, in the File Descriptor Block (FDB), + contain such things as creation time, write time, read time, writer's + byte-size, end-of-file pointer, count of reads and writes, backup + system tape numbers, etc. + + Note that there is NO requirement that entries in the page table be + contiguous. There may be empty page table slots between occupied + ones. Also, the end of file pointer is simply a number. There is no + requirement that it in fact point at the "last" datum in the file. + Ordinary sequential I/O calls in TOPS-20 will cause the end of file + pointer to be left after the last datum written, but other operations + may cause it not to be so, if a particular programming system so + requires. + + In fact, in both of these special cases, "holey" files and + end-of-file pointers NOT at the end of the file, occur with NLS data + files. + + + + + +Postel & Reynolds [Page 60] + + + +RFC 959 October 1985 +File Transfer Protocol + + + The TOPS-20 paged files can be sent with the FTP transfer parameters: + TYPE L 36, STRU P, and MODE S (in fact, any mode could be used). + + Each page of information has a header. Each header field, which is a + logical byte, is a TOPS-20 word, since the TYPE is L 36. + + The header fields are: + + Word 0: Header Length. + + The header length is 5. + + Word 1: Page Index. + + If the data is a disk file page, this is the number of that + page in the file's page map. Empty pages (holes) in the file + are simply not sent. Note that a hole is NOT the same as a + page of zeros. + + Word 2: Data Length. + + The number of data words in this page, following the header. + Thus, the total length of the transmission unit is the Header + Length plus the Data Length. + + Word 3: Page Type. + + A code for what type of chunk this is. A data page is type 3, + the FDB page is type 2. + + Word 4: Page Access Control. + + The access bits associated with the page in the file's page + map. (This full word quantity is put into AC2 of an SPACS by + the program reading from net to disk.) + + After the header are Data Length data words. Data Length is + currently either 512 for a data page or 31 for an FDB. Trailing + zeros in a disk file page may be discarded, making Data Length less + than 512 in that case. + + + + + + + + + +Postel & Reynolds [Page 61] + + + +RFC 959 October 1985 +File Transfer Protocol + + +APPENDIX II - DIRECTORY COMMANDS + + Since UNIX has a tree-like directory structure in which directories + are as easy to manipulate as ordinary files, it is useful to expand + the FTP servers on these machines to include commands which deal with + the creation of directories. Since there are other hosts on the + ARPA-Internet which have tree-like directories (including TOPS-20 and + Multics), these commands are as general as possible. + + Four directory commands have been added to FTP: + + MKD pathname + + Make a directory with the name "pathname". + + RMD pathname + + Remove the directory with the name "pathname". + + PWD + + Print the current working directory name. + + CDUP + + Change to the parent of the current working directory. + + The "pathname" argument should be created (removed) as a + subdirectory of the current working directory, unless the "pathname" + string contains sufficient information to specify otherwise to the + server, e.g., "pathname" is an absolute pathname (in UNIX and + Multics), or pathname is something like "<abso.lute.path>" to + TOPS-20. + + REPLY CODES + + The CDUP command is a special case of CWD, and is included to + simplify the implementation of programs for transferring directory + trees between operating systems having different syntaxes for + naming the parent directory. The reply codes for CDUP be + identical to the reply codes of CWD. + + The reply codes for RMD be identical to the reply codes for its + file analogue, DELE. + + The reply codes for MKD, however, are a bit more complicated. A + freshly created directory will probably be the object of a future + + +Postel & Reynolds [Page 62] + + + +RFC 959 October 1985 +File Transfer Protocol + + + CWD command. Unfortunately, the argument to MKD may not always be + a suitable argument for CWD. This is the case, for example, when + a TOPS-20 subdirectory is created by giving just the subdirectory + name. That is, with a TOPS-20 server FTP, the command sequence + + MKD MYDIR + CWD MYDIR + + will fail. The new directory may only be referred to by its + "absolute" name; e.g., if the MKD command above were issued while + connected to the directory <DFRANKLIN>, the new subdirectory + could only be referred to by the name <DFRANKLIN.MYDIR>. + + Even on UNIX and Multics, however, the argument given to MKD may + not be suitable. If it is a "relative" pathname (i.e., a pathname + which is interpreted relative to the current directory), the user + would need to be in the same current directory in order to reach + the subdirectory. Depending on the application, this may be + inconvenient. It is not very robust in any case. + + To solve these problems, upon successful completion of an MKD + command, the server should return a line of the form: + + 257<space>"<directory-name>"<space><commentary> + + That is, the server will tell the user what string to use when + referring to the created directory. The directory name can + contain any character; embedded double-quotes should be escaped by + double-quotes (the "quote-doubling" convention). + + For example, a user connects to the directory /usr/dm, and creates + a subdirectory, named pathname: + + CWD /usr/dm + 200 directory changed to /usr/dm + MKD pathname + 257 "/usr/dm/pathname" directory created + + An example with an embedded double quote: + + MKD foo"bar + 257 "/usr/dm/foo""bar" directory created + CWD /usr/dm/foo"bar + 200 directory changed to /usr/dm/foo"bar + + + + + +Postel & Reynolds [Page 63] + + + +RFC 959 October 1985 +File Transfer Protocol + + + The prior existence of a subdirectory with the same name is an + error, and the server must return an "access denied" error reply + in that case. + + CWD /usr/dm + 200 directory changed to /usr/dm + MKD pathname + 521-"/usr/dm/pathname" directory already exists; + 521 taking no action. + + The failure replies for MKD are analogous to its file creating + cousin, STOR. Also, an "access denied" return is given if a file + name with the same name as the subdirectory will conflict with the + creation of the subdirectory (this is a problem on UNIX, but + shouldn't be one on TOPS-20). + + Essentially because the PWD command returns the same type of + information as the successful MKD command, the successful PWD + command uses the 257 reply code as well. + + SUBTLETIES + + Because these commands will be most useful in transferring + subtrees from one machine to another, carefully observe that the + argument to MKD is to be interpreted as a sub-directory of the + current working directory, unless it contains enough information + for the destination host to tell otherwise. A hypothetical + example of its use in the TOPS-20 world: + + CWD <some.where> + 200 Working directory changed + MKD overrainbow + 257 "<some.where.overrainbow>" directory created + CWD overrainbow + 431 No such directory + CWD <some.where.overrainbow> + 200 Working directory changed + + CWD <some.where> + 200 Working directory changed to <some.where> + MKD <unambiguous> + 257 "<unambiguous>" directory created + CWD <unambiguous> + + Note that the first example results in a subdirectory of the + connected directory. In contrast, the argument in the second + example contains enough information for TOPS-20 to tell that the + + +Postel & Reynolds [Page 64] + + + +RFC 959 October 1985 +File Transfer Protocol + + + <unambiguous> directory is a top-level directory. Note also that + in the first example the user "violated" the protocol by + attempting to access the freshly created directory with a name + other than the one returned by TOPS-20. Problems could have + resulted in this case had there been an <overrainbow> directory; + this is an ambiguity inherent in some TOPS-20 implementations. + Similar considerations apply to the RMD command. The point is + this: except where to do so would violate a host's conventions for + denoting relative versus absolute pathnames, the host should treat + the operands of the MKD and RMD commands as subdirectories. The + 257 reply to the MKD command must always contain the absolute + pathname of the created directory. + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + +Postel & Reynolds [Page 65] + + + +RFC 959 October 1985 +File Transfer Protocol + + +APPENDIX III - RFCs on FTP + + Bhushan, Abhay, "A File Transfer Protocol", RFC 114 (NIC 5823), + MIT-Project MAC, 16 April 1971. + + Harslem, Eric, and John Heafner, "Comments on RFC 114 (A File + Transfer Protocol)", RFC 141 (NIC 6726), RAND, 29 April 1971. + + Bhushan, Abhay, et al, "The File Transfer Protocol", RFC 172 + (NIC 6794), MIT-Project MAC, 23 June 1971. + + Braden, Bob, "Comments on DTP and FTP Proposals", RFC 238 (NIC 7663), + UCLA/CCN, 29 September 1971. + + Bhushan, Abhay, et al, "The File Transfer Protocol", RFC 265 + (NIC 7813), MIT-Project MAC, 17 November 1971. + + McKenzie, Alex, "A Suggested Addition to File Transfer Protocol", + RFC 281 (NIC 8163), BBN, 8 December 1971. + + Bhushan, Abhay, "The Use of "Set Data Type" Transaction in File + Transfer Protocol", RFC 294 (NIC 8304), MIT-Project MAC, + 25 January 1972. + + Bhushan, Abhay, "The File Transfer Protocol", RFC 354 (NIC 10596), + MIT-Project MAC, 8 July 1972. + + Bhushan, Abhay, "Comments on the File Transfer Protocol (RFC 354)", + RFC 385 (NIC 11357), MIT-Project MAC, 18 August 1972. + + Hicks, Greg, "User FTP Documentation", RFC 412 (NIC 12404), Utah, + 27 November 1972. + + Bhushan, Abhay, "File Transfer Protocol (FTP) Status and Further + Comments", RFC 414 (NIC 12406), MIT-Project MAC, 20 November 1972. + + Braden, Bob, "Comments on File Transfer Protocol", RFC 430 + (NIC 13299), UCLA/CCN, 7 February 1973. + + Thomas, Bob, and Bob Clements, "FTP Server-Server Interaction", + RFC 438 (NIC 13770), BBN, 15 January 1973. + + Braden, Bob, "Print Files in FTP", RFC 448 (NIC 13299), UCLA/CCN, + 27 February 1973. + + McKenzie, Alex, "File Transfer Protocol", RFC 454 (NIC 14333), BBN, + 16 February 1973. + + +Postel & Reynolds [Page 66] + + + +RFC 959 October 1985 +File Transfer Protocol + + + Bressler, Bob, and Bob Thomas, "Mail Retrieval via FTP", RFC 458 + (NIC 14378), BBN-NET and BBN-TENEX, 20 February 1973. + + Neigus, Nancy, "File Transfer Protocol", RFC 542 (NIC 17759), BBN, + 12 July 1973. + + Krilanovich, Mark, and George Gregg, "Comments on the File Transfer + Protocol", RFC 607 (NIC 21255), UCSB, 7 January 1974. + + Pogran, Ken, and Nancy Neigus, "Response to RFC 607 - Comments on the + File Transfer Protocol", RFC 614 (NIC 21530), BBN, 28 January 1974. + + Krilanovich, Mark, George Gregg, Wayne Hathaway, and Jim White, + "Comments on the File Transfer Protocol", RFC 624 (NIC 22054), UCSB, + Ames Research Center, SRI-ARC, 28 February 1974. + + Bhushan, Abhay, "FTP Comments and Response to RFC 430", RFC 463 + (NIC 14573), MIT-DMCG, 21 February 1973. + + Braden, Bob, "FTP Data Compression", RFC 468 (NIC 14742), UCLA/CCN, + 8 March 1973. + + Bhushan, Abhay, "FTP and Network Mail System", RFC 475 (NIC 14919), + MIT-DMCG, 6 March 1973. + + Bressler, Bob, and Bob Thomas "FTP Server-Server Interaction - II", + RFC 478 (NIC 14947), BBN-NET and BBN-TENEX, 26 March 1973. + + White, Jim, "Use of FTP by the NIC Journal", RFC 479 (NIC 14948), + SRI-ARC, 8 March 1973. + + White, Jim, "Host-Dependent FTP Parameters", RFC 480 (NIC 14949), + SRI-ARC, 8 March 1973. + + Padlipsky, Mike, "An FTP Command-Naming Problem", RFC 506 + (NIC 16157), MIT-Multics, 26 June 1973. + + Day, John, "Memo to FTP Group (Proposal for File Access Protocol)", + RFC 520 (NIC 16819), Illinois, 25 June 1973. + + Merryman, Robert, "The UCSD-CC Server-FTP Facility", RFC 532 + (NIC 17451), UCSD-CC, 22 June 1973. + + Braden, Bob, "TENEX FTP Problem", RFC 571 (NIC 18974), UCLA/CCN, + 15 November 1973. + + + + +Postel & Reynolds [Page 67] + + + +RFC 959 October 1985 +File Transfer Protocol + + + McKenzie, Alex, and Jon Postel, "Telnet and FTP Implementation - + Schedule Change", RFC 593 (NIC 20615), BBN and MITRE, + 29 November 1973. + + Sussman, Julie, "FTP Error Code Usage for More Reliable Mail + Service", RFC 630 (NIC 30237), BBN, 10 April 1974. + + Postel, Jon, "Revised FTP Reply Codes", RFC 640 (NIC 30843), + UCLA/NMC, 5 June 1974. + + Harvey, Brian, "Leaving Well Enough Alone", RFC 686 (NIC 32481), + SU-AI, 10 May 1975. + + Harvey, Brian, "One More Try on the FTP", RFC 691 (NIC 32700), SU-AI, + 28 May 1975. + + Lieb, J., "CWD Command of FTP", RFC 697 (NIC 32963), 14 July 1975. + + Harrenstien, Ken, "FTP Extension: XSEN", RFC 737 (NIC 42217), SRI-KL, + 31 October 1977. + + Harrenstien, Ken, "FTP Extension: XRSQ/XRCP", RFC 743 (NIC 42758), + SRI-KL, 30 December 1977. + + Lebling, P. David, "Survey of FTP Mail and MLFL", RFC 751, MIT, + 10 December 1978. + + Postel, Jon, "File Transfer Protocol Specification", RFC 765, ISI, + June 1980. + + Mankins, David, Dan Franklin, and Buzz Owen, "Directory Oriented FTP + Commands", RFC 776, BBN, December 1980. + + Padlipsky, Michael, "FTP Unique-Named Store Command", RFC 949, MITRE, + July 1985. + + + + + + + + + + + + + + +Postel & Reynolds [Page 68] + + + +RFC 959 October 1985 +File Transfer Protocol + + +REFERENCES + + [1] Feinler, Elizabeth, "Internet Protocol Transition Workbook", + Network Information Center, SRI International, March 1982. + + [2] Postel, Jon, "Transmission Control Protocol - DARPA Internet + Program Protocol Specification", RFC 793, DARPA, September 1981. + + [3] Postel, Jon, and Joyce Reynolds, "Telnet Protocol + Specification", RFC 854, ISI, May 1983. + + [4] Reynolds, Joyce, and Jon Postel, "Assigned Numbers", RFC 943, + ISI, April 1985. + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + +Postel & Reynolds [Page 69] + diff --git a/lib/inets/doc/html/.gitignore b/lib/inets/doc/html/.gitignore new file mode 100644 index 0000000000..e69de29bb2 --- /dev/null +++ b/lib/inets/doc/html/.gitignore diff --git a/lib/inets/doc/man3/.gitignore b/lib/inets/doc/man3/.gitignore new file mode 100644 index 0000000000..e69de29bb2 --- /dev/null +++ b/lib/inets/doc/man3/.gitignore diff --git a/lib/inets/doc/man6/.gitignore b/lib/inets/doc/man6/.gitignore new file mode 100644 index 0000000000..e69de29bb2 --- /dev/null +++ b/lib/inets/doc/man6/.gitignore diff --git a/lib/inets/doc/pdf/.gitignore b/lib/inets/doc/pdf/.gitignore new file mode 100644 index 0000000000..e69de29bb2 --- /dev/null +++ b/lib/inets/doc/pdf/.gitignore diff --git a/lib/inets/doc/src/Makefile b/lib/inets/doc/src/Makefile new file mode 100644 index 0000000000..5b5a818db8 --- /dev/null +++ b/lib/inets/doc/src/Makefile @@ -0,0 +1,272 @@ +# +# %CopyrightBegin% +# +# Copyright Ericsson AB 1997-2009. All Rights Reserved. +# +# The contents of this file are subject to the Erlang Public License, +# Version 1.1, (the "License"); you may not use this file except in +# compliance with the License. You should have received a copy of the +# Erlang Public License along with this software. If not, it can be +# retrieved online at http://www.erlang.org/. +# +# Software distributed under the License is distributed on an "AS IS" +# basis, WITHOUT WARRANTY OF ANY KIND, either express or implied. See +# the License for the specific language governing rights and limitations +# under the License. +# +# %CopyrightEnd% +# +# +include $(ERL_TOP)/make/target.mk +include $(ERL_TOP)/make/$(TARGET)/otp.mk + +# ---------------------------------------------------- +# Application version +# ---------------------------------------------------- +include ../../vsn.mk +VSN=$(INETS_VSN) +APPLICATION=inets + +# ---------------------------------------------------- +# Include dependency +# ---------------------------------------------------- + +ifndef DOCSUPPORT +include make.dep +endif + +# ---------------------------------------------------- +# Release directory specification +# ---------------------------------------------------- +RELSYSDIR = $(RELEASE_PATH)/lib/$(APPLICATION)-$(VSN) + +# ---------------------------------------------------- +# Target Specs +# ---------------------------------------------------- +XML_APPLICATION_FILES = ref_man.xml + +XML_CHAPTER_FILES = \ + inets_services.xml \ + http_client.xml \ + http_server.xml \ + ftp_client.xml \ + notes.xml + +XML_REF3_FILES = \ + inets.xml \ + ftp.xml \ + tftp.xml \ + http.xml\ + httpd.xml \ + httpd_conf.xml \ + httpd_socket.xml \ + httpd_util.xml \ + mod_alias.xml \ + mod_auth.xml \ + mod_esi.xml \ + mod_security.xml + +XML_PART_FILES = \ + part.xml \ + part_notes.xml \ + part_notes_history.xml +XML_HTML_FILES = \ + notes_history.xml + +BOOK_FILES = book.xml + +XML_FILES = $(BOOK_FILES) \ + $(XML_CHAPTER_FILES) \ + $(XML_PART_FILES) \ + $(XML_REF6_FILES) \ + $(XML_REF3_FILES) \ + $(XML_APPLICATION_FILES) + +GIF_FILES = \ + inets.gif \ + notes.gif \ + ref_man.gif \ + book.gif \ + warning.gif \ + note.gif + + +# ---------------------------------------------------- + +HTML_FILES = \ + $(XML_APPLICATION_FILES:%.xml=$(HTMLDIR)/%.html) \ + $(XML_HTML_FILES:%.xml=$(HTMLDIR)/%.html) \ + $(XML_PART_FILES:%.xml=$(HTMLDIR)/%.html) + +INFO_FILE = ../../info +EXTRA_FILES = summary.html.src \ + $(DEFAULT_GIF_FILES) \ + $(DEFAULT_HTML_FILES) \ + $(XML_REF3_FILES:%.xml=$(HTMLDIR)/%.html) \ + $(XML_REF6_FILES:%.xml=$(HTMLDIR)/%.html) \ + $(XML_CHAPTER_FILES:%.xml=$(HTMLDIR)/%.html) + +MAN3_FILES = $(XML_REF3_FILES:%.xml=$(MAN3DIR)/%.3) + +ifdef DOCSUPPORT + +HTML_REF_MAN_FILE = $(HTMLDIR)/index.html + +TOP_PDF_FILE = $(PDFDIR)/$(APPLICATION)-$(VSN).pdf + +else + +TEX_FILES_BOOK = \ + $(BOOK_FILES:%.xml=%.tex) +TEX_FILES_REF_MAN = \ + $(XML_PART_FILES:%.xml=%.tex) \ + $(XML_REF3_FILES:%.xml=%.tex) \ + $(XML_REF6_FILES:%.xml=%.tex) \ + $(XML_APPLICATION_FILES:%.xml=%.tex) +TEX_FILES_USERS_GUIDE = \ + $(XML_CHAPTER_FILES:%.xml=%.tex) + +TOP_PDF_FILE = $(APPLICATION)-$(VSN).pdf +TOP_PS_FILE = $(APPLICATION)-$(VSN).ps + +$(TOP_PDF_FILE): book.dvi ../../vsn.mk + $(DVI2PS) $(DVIPS_FLAGS) -f $< | $(DISTILL) $(DISTILL_FLAGS) > $@ + +$(TOP_PS_FILE): book.dvi ../../vsn.mk + $(DVI2PS) $(DVIPS_FLAGS) -f $< > $@ + +TOP_HTML_FILES = + +endif + +# ---------------------------------------------------- +# FLAGS +# ---------------------------------------------------- +XML_FLAGS += +DVIPS_FLAGS += + +# ---------------------------------------------------- +# Targets +# ---------------------------------------------------- +$(HTMLDIR)/%.gif: %.gif + $(INSTALL_DATA) $< $@ + +ifdef DOCSUPPORT + +docs: pdf html man + +ldocs: local_docs + +$(TOP_PDF_FILE): $(XML_FILES) + +pdf: $(TOP_PDF_FILE) + +html: gifs $(HTML_REF_MAN_FILE) + +clean clean_docs: clean_html clean_man clean_pdf + rm -f errs core *~ + +else + +ifeq ($(DOCTYPE),pdf) +docs: pdf +else +ifeq ($(DOCTYPE),ps) +docs: ps +else +docs: html man +endif +endif + +pdf: $(TOP_PDF_FILE) + +ps: $(TOP_PS_FILE) + +html: $(HTML_FILES) $(TOP_HTML_FILES) gifs + +clean_tex: + rm -f $(TEX_FILES_USERS_GUIDE) $(TEX_FILES_REF_MAN) $(TEX_FILES_BOOK) + +clean: clean_tex clean_html clean_man + rm -f *.xmls_output *.xmls_errs + rm -f $(TOP_PDF_FILE) + rm -f errs core *~ +endif + +man: $(MAN3_FILES) + +gifs: $(GIF_FILES:%=$(HTMLDIR)/%) + +debug opt: + +clean_pdf: + rm -f $(TOP_PDF_FILE) $(TOP_PDF_FILE:%.pdf=%.fo) + +clean_html: + rm -rf $(TOP_HTML_FILES) $(HTMLDIR)/* + +clean_man: + rm -f $(MAN3_FILES) + +# ---------------------------------------------------- +# Release Target +# ---------------------------------------------------- +include $(ERL_TOP)/make/otp_release_targets.mk + +ifdef DOCSUPPORT + +release_docs_spec: docs + $(INSTALL_DIR) $(RELSYSDIR)/doc/pdf + $(INSTALL_DATA) $(TOP_PDF_FILE) $(RELSYSDIR)/doc/pdf + $(INSTALL_DIR) $(RELSYSDIR)/doc/html + $(INSTALL_DATA) $(HTMLDIR)/* \ + $(RELSYSDIR)/doc/html + $(INSTALL_DATA) $(INFO_FILE) $(RELSYSDIR) + $(INSTALL_DIR) $(RELEASE_PATH)/man/man3 + $(INSTALL_DATA) $(MAN3DIR)/* $(RELEASE_PATH)/man/man3 +else + +ifeq ($(DOCTYPE),pdf) +release_docs_spec: pdf + $(INSTALL_DIR) $(RELEASE_PATH)/pdf + $(INSTALL_DATA) $(TOP_PDF_FILE) $(RELEASE_PATH)/pdf +else +ifeq ($(DOCTYPE),ps) +release_docs_spec: ps + $(INSTALL_DIR) $(RELEASE_PATH)/ps + $(INSTALL_DATA) $(TOP_PS_FILE) $(RELEASE_PATH)/ps +else +release_docs_spec: docs + $(INSTALL_DIR) $(RELSYSDIR)/doc/html + $(INSTALL_DATA) $(GIF_FILES) $(EXTRA_FILES) $(HTML_FILES) \ + $(RELSYSDIR)/doc/html + $(INSTALL_DATA) $(INFO_FILE) $(RELSYSDIR) + $(INSTALL_DIR) $(RELEASE_PATH)/man/man3 + $(INSTALL_DATA) $(MAN3_FILES) $(RELEASE_PATH)/man/man3 + +endif +endif + +endif + +release_spec: + +info: + @echo "GIF_FILES:\n$(GIF_FILES)" + @echo "" + @echo "EXTRA_FILES:\n$(EXTRA_FILES)" + @echo "" + @echo "HTML_FILES:\n$(HTML_FILES)" + @echo "" + @echo "TOP_HTML_FILES:\n$(TOP_HTML_FILES)" + @echo "" + @echo "DEFAULT_GIF_FILES:\n$(DEFAULT_GIF_FILES)" + @echo "" + @echo "DEFAULT_HTML_FILES:\n$(DEFAULT_HTML_FILES)" + @echo "" + @echo "XML_REF3_FILES:\n$(XML_REF3_FILES)" + @echo "" + @echo "XML_REF6_FILES:\n$(XML_REF6_FILES)" + @echo "" + @echo "XML_CHAPTER_FILES:\n$(XML_CHAPTER_FILES)" + @echo "" diff --git a/lib/inets/doc/src/book.gif b/lib/inets/doc/src/book.gif Binary files differnew file mode 100644 index 0000000000..94b3868792 --- /dev/null +++ b/lib/inets/doc/src/book.gif diff --git a/lib/inets/doc/src/book.xml b/lib/inets/doc/src/book.xml new file mode 100644 index 0000000000..7da0abd98f --- /dev/null +++ b/lib/inets/doc/src/book.xml @@ -0,0 +1,50 @@ +<?xml version="1.0" encoding="latin1" ?> +<!DOCTYPE book SYSTEM "book.dtd"> + +<book xmlns:xi="http://www.w3.org/2001/XInclude"> + <header titlestyle="normal"> + <copyright> + <year>1997</year><year>2009</year> + <holder>Ericsson AB. All Rights Reserved.</holder> + </copyright> + <legalnotice> + The contents of this file are subject to the Erlang Public License, + Version 1.1, (the "License"); you may not use this file except in + compliance with the License. You should have received a copy of the + Erlang Public License along with this software. If not, it can be + retrieved online at http://www.erlang.org/. + + Software distributed under the License is distributed on an "AS IS" + basis, WITHOUT WARRANTY OF ANY KIND, either express or implied. See + the License for the specific language governing rights and limitations + under the License. + + </legalnotice> + + <title>inets</title> + <prepared>Mattias Nilsson</prepared> + <docno></docno> + <date>1997-07-16</date> + <rev>2.3</rev> + <file>book.sgml</file> + </header> + <insidecover> + </insidecover> + <pagetext>Inets</pagetext> + <preamble> + <contents level="2"></contents> + </preamble> + <parts lift="no"> + <xi:include href="part.xml"/> + </parts> + <applications> + <xi:include href="ref_man.xml"/> + </applications> + <releasenotes> + <xi:include href="notes.xml"/> + </releasenotes> + <listofterms></listofterms> + <index></index> +</book> + + diff --git a/lib/inets/doc/src/fascicules.xml b/lib/inets/doc/src/fascicules.xml new file mode 100644 index 0000000000..101e745722 --- /dev/null +++ b/lib/inets/doc/src/fascicules.xml @@ -0,0 +1,19 @@ +<?xml version="1.0" encoding="latin1" ?> +<!DOCTYPE fascicules SYSTEM "fascicules.dtd"> + +<fascicules> + <fascicule file="part" href="part_frame.html" entry="no"> + User's Guide + </fascicule> + <fascicule file="ref_man" href="ref_man_frame.html" entry="yes"> + Reference Manual + </fascicule> + <fascicule file="part_notes" href="part_notes_frame.html" entry="no"> + Release Notes + </fascicule> + <fascicule file="" href="../../../../doc/print.html" entry="no"> + Off-Print + </fascicule> +</fascicules> + + diff --git a/lib/inets/doc/src/ftp.xml b/lib/inets/doc/src/ftp.xml new file mode 100644 index 0000000000..9ecca3dde1 --- /dev/null +++ b/lib/inets/doc/src/ftp.xml @@ -0,0 +1,939 @@ +<?xml version="1.0" encoding="latin1" ?> +<!DOCTYPE erlref SYSTEM "erlref.dtd"> + +<erlref> + <header> + <copyright> + <year>1997</year><year>2009</year> + <holder>Ericsson AB. All Rights Reserved.</holder> + </copyright> + <legalnotice> + The contents of this file are subject to the Erlang Public License, + Version 1.1, (the "License"); you may not use this file except in + compliance with the License. You should have received a copy of the + Erlang Public License along with this software. If not, it can be + retrieved online at http://www.erlang.org/. + + Software distributed under the License is distributed on an "AS IS" + basis, WITHOUT WARRANTY OF ANY KIND, either express or implied. See + the License for the specific language governing rights and limitations + under the License. + + </legalnotice> + + <title>ftp</title> + <prepared>Peter Högfeldt</prepared> + <docno></docno> + <date>1997-11-05</date> + <rev>B</rev> + <file>ftp.xml</file> + </header> + <module>ftp</module> + <modulesummary>A File Transfer Protocol client</modulesummary> + + <description> + + <p>The <c>ftp</c> module implements a client for file transfer + according to a subset of the File Transfer Protocol (see <term + id="RFC"></term>959). </p> + + <p>Starting from inets version 4.4.1 the ftp + client will always try to use passive ftp mode and only resort + to active ftp mode if this fails. There is a start option + <seealso marker="#mode">mode</seealso> where this default behavior + may be changed. </p> + + <marker id="two_start"></marker> + + <p>There are two ways to start an ftp client. One is using the + <seealso marker="#service_start">Inets service framework</seealso> + and the other is to start it directy as a standalone process + using the <seealso marker="#open">open</seealso> function. </p> + + <p>For a simple example of an ftp session see + <seealso marker="ftp_client">Inets User's Guide.</seealso></p> + + <p>In addition to the ordinary functions for receiving and sending + files (see <c>recv/2</c>, <c>recv/3</c>, <c>send/2</c> and + <c>send/3</c>) there are functions for receiving remote files as + binaries (see <c>recv_bin/2</c>) and for sending binaries to to be + stored as remote files (see <c>send_bin/3</c>).</p> + + <p>There is also a set of functions for sending and receiving + contiguous parts of a file to be stored in a remote file (for send + see <c>send_chunk_start/2</c>, <c>send_chunk/2</c> and + <c>send_chunk_end/1</c> and for receive see + <c>recv_chunk_start/2</c> and <c>recv_chunk/</c>).</p> + + <p>The particular return values of the functions below depend very + much on the implementation of the FTP server at the remote + host. In particular the results from <c>ls</c> and <c>nlist</c> + varies. Often real errors are not reported as errors by <c>ls</c>, + even if for instance a file or directory does not + exist. <c>nlist</c> is usually more strict, but some + implementations have the peculiar behaviour of responding with an + error, if the request is a listing of the contents of directory + which exists but is empty.</p> + + <marker id="service_start"></marker> + </description> + + <section> + <title>FTP CLIENT SERVICE START/STOP </title> + + <p>The FTP client can be started and stopped dynamically in runtime by + calling the Inets application API + <c>inets:start(ftpc, ServiceConfig)</c>, + or <c>inets:start(ftpc, ServiceConfig, How)</c>, and + <c>inets:stop(ftpc, Pid)</c>. + See <seealso marker="inets">inets(3)</seealso> for more info. </p> + <p>Below follows a description of + the available configuration options.</p> + + <taglist> + <tag>{host, Host}</tag> + <item> + <marker id="host"></marker> + <p>Host = <c>string() | ip_address()</c> </p> + </item> + + <tag>{port, Port}</tag> + <item> + <marker id="port"></marker> + <p>Port = <c>integer() > 0</c> </p> + <p>Default is 21.</p> + </item> + + <tag>{mode, Mode}</tag> + <item> + <marker id="mode"></marker> + <p>Mode = <c>active | passive</c> </p>> + <p>Default is <c>passive</c>. </p> + </item> + + <tag>{verbose, Verbose}</tag> + <item> + <marker id="verbose"></marker> + <p>Verbose = <c>boolean()</c> </p> + <p>This determines if the FTP communication should be + verbose or not. </p> + <p>Default is <c>false</c>. </p> + </item> + + <tag>{debug, Debug}</tag> + <item> + <marker id="debug"></marker> + <p>Debug = <c>trace | debug | disable</c> </p> + <p>Debugging using the dbg toolkit. </p> + <p>Default is <c>disable</c>. </p> + </item> + + <tag>{ipfamily, IpFamily}</tag> + <item> + <marker id="ipfamily"></marker> + <p>IpFamily = <c>inet | inet6 | inet6fb4</c> </p> + <p>With <c>inet6fb4</c> the client behaves as before + (it tries to use IPv6 and only if that does not work, it + uses IPv4). </p> + <p>Default is <c>inet</c> (IPv4). </p> + </item> + + <tag>{timeout, Timeout}</tag> + <item> + <marker id="timeout"></marker> + <p>Timeout = <c>integer() >= 0</c> </p> + <p>Connection timeout. </p> + <p>Default is 60000 (milliseconds). </p> + </item> + + <tag>{progress, Progress}</tag> + <item> + <marker id="progress"></marker> + <p>Progress = <c>ignore | {CBModule, CBFunction, InitProgress}</c></p> + <p>CBModule = <c>atom()</c>, CBFunction = <c>atom()</c> </p> + <p>InitProgress = <c>term()</c> </p> + <p>Default is <c>ignore</c>. </p> + </item> + + </taglist> + + <p>The progress option is intended to be used by applications that + want to create some type of progress report such as a progress bar in + a GUI. The default value for the progress option is ignore + e.i. the option is not used. When the progress option is + specified the following will happen when ftp:send/[3,4] or + ftp:recv/[3,4] are called.</p> + + <list type="bulleted"> + <item> + <p>Before a file is transfered the following call will + be made to indicate the start of the file transfer and how big + the file is. The return value of the callback function + should be a new value for the UserProgressTerm that will + bu used as input next time the callback function is + called.</p> + <br></br> + <p><c> + CBModule:CBFunction(InitProgress, File, {file_size, FileSize}) + </c></p> + <br></br> + </item> + + <item> + <p>Every time a chunk of bytes is transfered the + following call will be made:</p> + <br></br> + <p><c> + CBModule:CBFunction(UserProgressTerm, File, {transfer_size, TransferSize}) </c></p> + <br></br> + </item> + + <item> + <p>At the end of the file the following call will be + made to indicate the end of the transfer.</p> + <br></br> + <p><c> + CBModule:CBFunction(UserProgressTerm, File, {transfer_size, 0}) </c></p> + <br></br> + </item> + </list> + + <p>The callback function should be defined as </p> + + <p><c> + CBModule:CBFunction(UserProgressTerm, File, Size) -> UserProgressTerm </c></p> + + <p><c> + CBModule = CBFunction = atom() + </c></p> + + <p><c> + UserProgressTerm = term() + </c></p> + + <p><c> + File = string() + </c></p> + + <p><c> + Size = {transfer_size, integer()} | {file_size, integer()} | {file_size, unknown} </c></p> + + <p>Alas for remote files it is not possible for ftp to determine the + file size in a platform independent way. In this case the size + will be <c>unknown</c> and it is left to the application to find + out the size. </p> + + <note> + <p>The callback is made by a middleman process, hence the + file transfer will not be affected by the code in the progress + callback function. If the callback should crash this will be + detected by the ftp connection process that will print an + info-report and then go one as if the progress option was set + to ignore. </p> + </note> + + <p>The file transfer type is set to the default of the FTP server + when the session is opened. This is usually ASCCI-mode. + </p> + + <p>The current local working directory (cf. <c>lpwd/1</c>) is set to + the value reported by <c>file:get_cwd/1</c>. the wanted + local directory. + </p> + + <p>The return value <c>Pid</c> is used as a reference to the + newly created ftp client in all other functions, and they should + be called by the process that created the connection. The ftp + client process monitors the process that created it and + will terminate if that process terminates.</p> + </section> + + <section> + <title>COMMON DATA TYPES </title> + <p>Here follows type definitions that are used by more than one + function in the FTP client API. </p> + <p><c> pid() - identifier of an ftp connection.</c></p> + <p><c> string() = list of ASCII characters.</c></p> + <p><c> shortage_reason() = etnospc | epnospc</c></p> + <p><c> restriction_reason() = epath | efnamena | elogin | enotbinary + - note not all restrictions may always relevant to all functions + </c></p> + <p><c>common_reason() = econn | eclosed | term() - some kind of + explanation of what went wrong.</c></p> + + <marker id="account"></marker> + </section> + + <funcs> + <func> + <name>account(Pid, Account) -> ok | {error, Reason}</name> + <fsummary>Specify which account to use.</fsummary> + <type> + <v>Pid = pid()</v> + <v>Account = string()</v> + <v>Reason = eacct | common_reason()</v> + </type> + <desc> + <p>If an account is needed for an operation set the account + with this operation.</p> + + <marker id="append"></marker> + <marker id="append2"></marker> + <marker id="append3"></marker> + </desc> + </func> + + <func> + <name>append(Pid, LocalFile) -> </name> + <name>append(Pid, LocalFile, RemoteFile) -> ok | {error, Reason}</name> + <fsummary>Transfer file to remote server, and append it to Remotefile.</fsummary> + <type> + <v>Pid = pid()</v> + <v>LocalFile = RemoteFile = string()</v> + <v>Reason = epath | elogin | etnospc | epnospc | efnamena | common_reason</v> + </type> + <desc> + <p>Transfers the file <c>LocalFile</c> to the remote server. If + <c>RemoteFile</c> is specified, the name of the remote file that the + file will be appended to is set to <c>RemoteFile</c>; otherwise + the name is set to <c>LocalFile</c> If the file does not exists the + file will be created.</p> + + <marker id="append_bin"></marker> + </desc> + </func> + + <func> + <name>append_bin(Pid, Bin, RemoteFile) -> ok | {error, Reason}</name> + <fsummary>Transfer a binary into a remote file.</fsummary> + <type> + <v>Pid = pid()</v> + <v>Bin = binary()()</v> + <v>RemoteFile = string()</v> + <v>Reason = restriction_reason()| shortage_reason() | common_reason()</v> + </type> + <desc> + <p>Transfers the binary <c>Bin</c> to the remote server and append + it to the file <c>RemoteFile</c>. If the file does not exists it + will be created.</p> + + <marker id="append_chunk"></marker> + </desc> + </func> + + <func> + <name>append_chunk(Pid, Bin) -> ok | {error, Reason}</name> + <fsummary>append a chunk to the remote file.</fsummary> + <type> + <v>Pid = pid()</v> + <v>Bin = binary()</v> + <v>Reason = echunk | restriction_reason() | common_reason()</v> + </type> + <desc> + <p>Transfer the chunk <c>Bin</c> to the remote server, which + append it into the file specified in the call to + <c>append_chunk_start/2</c>. </p> + <p>Note that for some errors, e.g. file system full, it is + necessary to to call <c>append_chunk_end</c> to get the + proper reason.</p> + + <marker id="append_chunk_start"></marker> + </desc> + </func> + + <func> + <name>append_chunk_start(Pid, File) -> ok | {error, Reason}</name> + <fsummary>Start transfer of file chunks for appending to File.</fsummary> + <type> + <v>Pid = pid()</v> + <v>File = string()</v> + <v>Reason = restriction_reason() | common_reason()</v> + </type> + <desc> + <p>Start the transfer of chunks for appending to the file + <c>File</c> at the remote server. If the file does not exists + it will be created.</p> + + <marker id="append_chunk_end"></marker> + </desc> + </func> + + <func> + <name>append_chunk_end(Pid) -> ok | {error, Reason}</name> + <fsummary>Stop transfer of chunks for appending.</fsummary> + <type> + <v>Pid = pid()</v> + <v>Reason = echunk | restriction_reason() | shortage_reason() </v> + </type> + <desc> + <p>Stops transfer of chunks for appending to the remote server. + The file at the remote server, specified in the call to + <c>append_chunk_start/2</c> is closed by the server.</p> + + <marker id="cd"></marker> + </desc> + </func> + + <func> + <name>cd(Pid, Dir) -> ok | {error, Reason}</name> + <fsummary>Change remote working directory.</fsummary> + <type> + <v>Pid = pid()</v> + <v>Dir = string()</v> + <v>Reason = restriction_reason() | common_reason() </v> + </type> + <desc> + <p>Changes the working directory at the remote server to + <c>Dir</c>.</p> + + <marker id="close"></marker> + </desc> + </func> + + <func> + <name>close(Pid) -> ok</name> + <fsummary>End the ftp session.</fsummary> + <type> + <v>Pid = pid()</v> + </type> + <desc> + <p>Ends an ftp session, created using the + <seealso marker="#open">open</seealso> function. </p> + + <marker id="delete"></marker> + </desc> + </func> + + <func> + <name>delete(Pid, File) -> ok | {error, Reason}</name> + <fsummary>Delete a file at the remote server..</fsummary> + <type> + <v>Pid = pid()</v> + <v>File = string()</v> + <v>Reason = restriction_reason() | common_reason()</v> + </type> + <desc> + <p>Deletes the file <c>File</c> at the remote server.</p> + + <marker id="append"></marker> + </desc> + </func> + + <func> + <name>formaterror(Tag) -> string()</name> + <fsummary>Return error diagnostics.</fsummary> + <type> + <v>Tag = {error, atom()} | atom()</v> + </type> + <desc> + <p>Given an error return value <c>{error, AtomReason}</c>, + this function returns a readable string describing the error.</p> + + <marker id="lcd"></marker> + </desc> + </func> + + <func> + <name>lcd(Pid, Dir) -> ok | {error, Reason}</name> + <fsummary>Change local working directory.</fsummary> + <type> + <v>Pid = pid()</v> + <v>Dir = string()</v> + <v>Reason = restriction_reason()</v> + </type> + <desc> + <p>Changes the working directory to <c>Dir</c> for the local client. </p> + + <marker id="lpwd"></marker> + </desc> + </func> + + <func> + <name>lpwd(Pid) -> {ok, Dir}</name> + <fsummary>Get local current working directory.</fsummary> + <type> + <v>Pid = pid()</v> + </type> + <desc> + <p>Returns the current working directory at the local client.</p> + + <marker id="ls"></marker> + <marker id="ls1"></marker> + <marker id="ls2"></marker> + </desc> + </func> + + <func> + <name>ls(Pid) -> </name> + <name>ls(Pid, Pathname) -> {ok, Listing} | {error, Reason}</name> + <fsummary>List of files.</fsummary> + <type> + <v>Pid = pid()</v> + <v>Pathname = string()</v> + <v>Listing = string()</v> + <v>Reason = restriction_reason() | common_reason()</v> + </type> + <desc> + <p>Returns a list of files in long format. </p> + <p><c>Pathname</c> can be a directory, a group of files or + even a file. The <c>Pathname</c> string can contain wildcard(s). </p> + <p><c>ls/1</c> implies the user's current remote directory. </p> + <p>The format of <c>Listing</c> is operating system dependent + (on UNIX it is typically produced from the output of the + <c>ls -l</c> shell command).</p> + + <marker id="mkdir"></marker> + </desc> + </func> + + <func> + <name>mkdir(Pid, Dir) -> ok | {error, Reason}</name> + <fsummary>Create remote directory.</fsummary> + <type> + <v>Pid = pid()</v> + <v>Dir = string()</v> + <v>Reason = restriction_reason() | common_reason()</v> + </type> + <desc> + <p>Creates the directory <c>Dir</c> at the remote server.</p> + + <marker id="nlist"></marker> + <marker id="nlist1"></marker> + <marker id="nlist2"></marker> + </desc> + </func> + + <func> + <name>nlist(Pid) -> </name> + <name>nlist(Pid, Pathname) -> {ok, Listing} | {error, Reason}</name> + <fsummary>List of files.</fsummary> + <type> + <v>Pid = pid()</v> + <v>Pathname = string()</v> + <v>Listing = string()</v> + <v>Reason = restriction_reason() | common_reason()</v> + </type> + <desc> + <p>Returns a list of files in short format. </p> + <p><c>Pathname</c> can be a directory, a group of files or + even a file. The <c>Pathname</c> string can contain wildcard(s). </p> + <p><c>nlist/1</c> implies the user's current remote directory. </p> + <p>The format of <c>Listing</c> is a stream of + file names, where each name is separated by <CRLF> or + <NL>. Contrary to the <c>ls</c> function, the purpose of + <c>nlist</c> is to make it possible for a program to + automatically process file name information.</p> + + <marker id="open"></marker> + </desc> + </func> + + <func> + <name>open(Host) -> {ok, Pid} | {error, Reason}</name> + <name>open(Host, Opts) -> {ok, Pid} | {error, Reason}</name> + <fsummary>Start an standalone ftp client.</fsummary> + <type> + <v>Host = string() | ip_address()</v> + <v>Opts = start_options() | open_options()</v> + <v>start_options() = [start_option()]</v> + <v>start_option() = {verbose, verbose()} | {debug, debug()}</v> + <v>verbose() = boolean() (defaults to false)</v> + <v>debug() = disable | debug | trace (defaults to disable)</v> + <v>open_options() = [open_option()]</v> + <v>open_option() = {ipfamily, ipfamily()} | {port, port()} | {mode, mode()} | {timeout, timeout()} | {progress, progress()}</v> + <v>ipfamily() = inet | inet6 | inet6fb4 (defaults to inet)</v> + <v>port() = integer() > 0 (defaults to 21)</v> + <v>mode() = active | passive (defaults to passive)</v> + <v>timeout() = integer() >= 0 (defaults to 60000 milliseconds)</v> + <v>pogress() = ignore | {module(), function(), initial_data()} (defaults to ignore)</v> + <v>module() = atom()</v> + <v>function() = atom()</v> + <v>initial_data() = term()</v> + <v>Reason = ehost | term()</v> + </type> + + <desc> + <p>This function is used to start a standalone ftp client process + (without the inets service framework) and + open a session with the FTP server at <c>Host</c>. </p> + + <p>A session opened in this way, is closed using the + <seealso marker="#close">close</seealso> function. </p> + + <marker id="pwd"></marker> + </desc> + </func> + + <func> + <name>pwd(Pid) -> {ok, Dir} | {error, Reason}</name> + <fsummary>Get remote current working directory.</fsummary> + <type> + <v>Pid = pid()</v> + <v>Reason = restriction_reason() | common_reason() </v> + </type> + <desc> + <p>Returns the current working directory at the remote server. </p> + </desc> + </func> + + <func> + <name>pwd(Pid) -> {ok, Dir} | {error, Reason}</name> + <fsummary>Get remote current working directory.</fsummary> + <type> + <v>Pid = pid()</v> + <v>Reason = restriction_reason() | common_reason() </v> + </type> + <desc> + <p>Returns the current working directory at the remote server.</p> + + <marker id="recv"></marker> + <marker id="recv2"></marker> + <marker id="recv3"></marker> + </desc> + </func> + + <func> + <name>recv(Pid, RemoteFile) -> </name> + <name>recv(Pid, RemoteFile, LocalFile) -> ok | {error, Reason}</name> + <fsummary>Transfer file from remote server.</fsummary> + <type> + <v>Pid = pid()</v> + <v>RemoteFile = LocalFile = string()</v> + <v>Reason = restriction_reason() | common_reason() | file_write_error_reason() </v> + <v>file_write_error_reason() = see file:write/2</v> + </type> + <desc> + <p>Transfer the file <c>RemoteFile</c> from the remote server + to the the file system of the local client. If + <c>LocalFile</c> is specified, the local file will be + <c>LocalFile</c>; otherwise it will be + <c>RemoteFile</c>.</p> + <p>If the file write fails + (e.g. enospc), then the command is aborted and <c>{error, file_write_error_reason()}</c> is returned. The file is + however <em>not</em> removed.</p> + + <marker id="recv_bin"></marker> + </desc> + </func> + + <func> + <name>recv_bin(Pid, RemoteFile) -> {ok, Bin} | {error, Reason}</name> + <fsummary>Transfer file from remote server as a binary.</fsummary> + <type> + <v>Pid = pid()</v> + <v>Bin = binary()</v> + <v>RemoteFile = string()</v> + <v>Reason = restriction_reason() | common_reason()</v> + </type> + <desc> + <p>Transfers the file <c>RemoteFile</c> from the remote server and + receives it as a binary.</p> + + <marker id="recv_chunk_start"></marker> + </desc> + </func> + + <func> + <name>recv_chunk_start(Pid, RemoteFile) -> ok | {error, Reason}</name> + <fsummary>Start chunk-reading of the remote file.</fsummary> + <type> + <v>Pid = pid()</v> + <v>RemoteFile = string()</v> + <v>Reason = restriction_reason() | common_reason()</v> + </type> + <desc> + <p>Start transfer of the file <c>RemoteFile</c> from the + remote server.</p> + + <marker id="recv_chunk"></marker> + </desc> + </func> + + <func> + <name>recv_chunk(Pid) -> ok | {ok, Bin} | {error, Reason}</name> + <fsummary>Receive a chunk of the remote file.</fsummary> + <type> + <v>Pid = pid()</v> + <v>Bin = binary()</v> + <v>Reason = restriction_reason() | common_reason()</v> + </type> + <desc> + <p>Receive a chunk of the remote file (<c>RemoteFile</c> of + <c>recv_chunk_start</c>). The return values has the following + meaning:</p> + <list type="bulleted"> + <item><c>ok</c> the transfer is complete.</item> + <item><c>{ok, Bin}</c> just another chunk of the file.</item> + <item><c>{error, Reason}</c> transfer failed.</item> + </list> + + <marker id="rename"></marker> + </desc> + </func> + + <func> + <name>rename(Pid, Old, New) -> ok | {error, Reason}</name> + <fsummary>Rename a file at the remote server..</fsummary> + <type> + <v>Pid = pid()</v> + <v>CurrFile = NewFile = string()</v> + <v>Reason = restriction_reason() | common_reason()</v> + </type> + <desc> + <p>Renames <c>Old</c> to <c>New</c> at the remote server.</p> + + <marker id="rmdir"></marker> + </desc> + </func> + + <func> + <name>rmdir(Pid, Dir) -> ok | {error, Reason}</name> + <fsummary>Remove a remote directory.</fsummary> + <type> + <v>Pid = pid()</v> + <v>Dir = string()</v> + <v>Reason = restriction_reason() | common_reason()</v> + </type> + <desc> + <p>Removes directory <c>Dir</c> at the remote server.</p> + + <marker id="send"></marker> + <marker id="send2"></marker> + <marker id="send3"></marker> + </desc> + </func> + + <func> + <name>send(Pid, LocalFile) -></name> + <name>send(Pid, LocalFile, RemoteFile) -> ok | {error, Reason}</name> + <fsummary>Transfer file to remote server.</fsummary> + <type> + <v>Pid = pid()</v> + <v>LocalFile = RemoteFile = string()</v> + <v>Reason = restriction_reason() | common_reason() | shortage_reason()</v> + </type> + <desc> + <p>Transfers the file <c>LocalFile</c> to the remote server. If + <c>RemoteFile</c> is specified, the name of the remote file is set + to <c>RemoteFile</c>; otherwise the name is set to <c>LocalFile</c>.</p> + + <marker id="send_bin"></marker> + </desc> + </func> + + <func> + <name>send_bin(Pid, Bin, RemoteFile) -> ok | {error, Reason}</name> + <fsummary>Transfer a binary into a remote file.</fsummary> + <type> + <v>Pid = pid()</v> + <v>Bin = binary()()</v> + <v>RemoteFile = string()</v> + <v>Reason = restriction_reason() | common_reason() | shortage_reason()</v> + </type> + <desc> + <p>Transfers the binary <c>Bin</c> into the file <c>RemoteFile</c> + at the remote server.</p> + + <marker id="send_chunk"></marker> + </desc> + </func> + + <func> + <name>send_chunk(Pid, Bin) -> ok | {error, Reason}</name> + <fsummary>Write a chunk to the remote file.</fsummary> + <type> + <v>Pid = pid()</v> + <v>Bin = binary()</v> + <v>Reason = echunk | restriction_reason() | common_reason()</v> + </type> + <desc> + <p>Transfer the chunk <c>Bin</c> to the remote server, which + writes it into the file specified in the call to + <c>send_chunk_start/2</c>. </p> + <p>Note that for some errors, e.g. file system full, it is + necessary to to call <c>send_chunk_end</c> to get the + proper reason.</p> + + <marker id="send_chunk_start"></marker> + </desc> + </func> + + <func> + <name>send_chunk_start(Pid, File) -> ok | {error, Reason}</name> + <fsummary>Start transfer of file chunks.</fsummary> + <type> + <v>Pid = pid()</v> + <v>File = string()</v> + <v>Reason = restriction_reason() | common_reason()</v> + </type> + <desc> + <p>Start transfer of chunks into the file <c>File</c> at the + remote server.</p> + + <marker id="send_chunk_end"></marker> + </desc> + </func> + + <func> + <name>send_chunk_end(Pid) -> ok | {error, Reason}</name> + <fsummary>Stop transfer of chunks.</fsummary> + <type> + <v>Pid = pid()</v> + <v>Reason = restriction_reason() | common_reason() | shortage_reason()</v> + </type> + <desc> + <p>Stops transfer of chunks to the remote server. The file at the + remote server, specified in the call to <c>send_chunk_start/2</c> + is closed by the server.</p> + + <marker id="type"></marker> + </desc> + </func> + + <func> + <name>type(Pid, Type) -> ok | {error, Reason}</name> + <fsummary>Set transfer type to <c>ascii</c>or <c>binary</c>.</fsummary> + <type> + <v>Pid = pid()</v> + <v>Type = ascii | binary</v> + <v>Reason = etype | restriction_reason() | common_reason()</v> + </type> + <desc> + <p>Sets the file transfer type to <c>ascii</c> or <c>binary</c>. When + an ftp session is opened, the default transfer type of the + server is used, most often <c>ascii</c>, which is the default + according to RFC 959.</p> + + <marker id="user3"></marker> + </desc> + </func> + + <func> + <name>user(Pid, User, Password) -> ok | {error, Reason}</name> + <fsummary>User login.</fsummary> + <type> + <v>Pid = pid()</v> + <v>User = Password = string()</v> + <v>Reason = euser | common_reason()</v> + </type> + <desc> + <p>Performs login of <c>User</c> with <c>Password</c>.</p> + + <marker id="user4"></marker> + </desc> + </func> + + <func> + <name>user(Pid, User, Password, Account) -> ok | {error, Reason}</name> + <fsummary>User login.</fsummary> + <type> + <v>Pid = pid()</v> + <v>User = Password = string()</v> + <v>Reason = euser | common_reason() </v> + </type> + <desc> + <p>Performs login of <c>User</c> with <c>Password</c> to the account + specified by <c>Account</c>.</p> + + <marker id="quote"></marker> + </desc> + </func> + + <func> + <name>quote(Pid, Command) -> [FTPLine]</name> + <fsummary>Sends an arbitrary FTP command. </fsummary> + <type> + <v>Pid = pid()</v> + <v>Command = string()</v> + <v>FTPLine = string() - Note the telnet end of line characters, from the ftp protocol definition, CRLF e.g. "\\r\ " has been removed.</v> + </type> + <desc> + <p>Sends an arbitrary FTP command and returns verbatimly a list + of the lines sent back by the FTP server. This functions is + intended to give an application accesses to FTP commands + that are server specific or that may not be provided by + this FTP client. </p> + <note> + <p>FTP commands that require a data connection can not be + successfully issued with this function. </p> + </note> + </desc> + </func> + </funcs> + + <section> + <title>ERRORS</title> + <p>The possible error reasons and the corresponding diagnostic strings + returned by <c>formaterror/1</c> are as follows: + </p> + <taglist> + <tag><c>echunk</c></tag> + <item> + <p>Synchronisation error during chunk sending. + </p> + <p>A call has been made to <c>send_chunk/2</c> or + <c>send_chunk_end/1</c>, before a call to + <c>send_chunk_start/2</c>; or a call has been made to another + transfer function during chunk sending, i.e. before a call + to <c>send_chunk_end/1</c>.</p> + </item> + <tag><c>eclosed</c></tag> + <item> + <p>The session has been closed.</p> + </item> + <tag><c>econn</c></tag> + <item> + <p>Connection to remote server prematurely closed.</p> + </item> + <tag><c>ehost</c></tag> + <item> + <p>Host not found, FTP server not found, or connection rejected + by FTP server.</p> + </item> + <tag><c>elogin</c></tag> + <item> + <p>User not logged in.</p> + </item> + <tag><c>enotbinary</c></tag> + <item> + <p>Term is not a binary.</p> + </item> + <tag><c>epath</c></tag> + <item> + <p>No such file or directory, or directory already exists, or + permission denied.</p> + </item> + <tag><c>etype</c></tag> + <item> + <p>No such type.</p> + </item> + <tag><c>euser</c></tag> + <item> + <p>User name or password not valid.</p> + </item> + <tag><c>etnospc</c></tag> + <item> + <p>Insufficient storage space in system [452].</p> + </item> + <tag><c>epnospc</c></tag> + <item> + <p>Exceeded storage allocation (for current directory or + dataset) [552].</p> + </item> + <tag><c>efnamena</c></tag> + <item> + <p>File name not allowed [553].</p> + </item> + </taglist> + </section> + + <section> + <title>SEE ALSO</title> + <p>file, filename, J. Postel and J. Reynolds: File Transfer Protocol + (RFC 959). + </p> + </section> + +</erlref> + + diff --git a/lib/inets/doc/src/ftp_client.xml b/lib/inets/doc/src/ftp_client.xml new file mode 100644 index 0000000000..7f62a453a6 --- /dev/null +++ b/lib/inets/doc/src/ftp_client.xml @@ -0,0 +1,91 @@ +<?xml version="1.0" encoding="latin1" ?> +<!DOCTYPE chapter SYSTEM "chapter.dtd"> + +<chapter> + <header> + <copyright> + <year>2004</year><year>2009</year> + <holder>Ericsson AB. All Rights Reserved.</holder> + </copyright> + <legalnotice> + The contents of this file are subject to the Erlang Public License, + Version 1.1, (the "License"); you may not use this file except in + compliance with the License. You should have received a copy of the + Erlang Public License along with this software. If not, it can be + retrieved online at http://www.erlang.org/. + + Software distributed under the License is distributed on an "AS IS" + basis, WITHOUT WARRANTY OF ANY KIND, either express or implied. See + the License for the specific language governing rights and limitations + under the License. + + </legalnotice> + + <title>FTP Client</title> + <prepared>Ingela Anderton Andin</prepared> + <responsible></responsible> + <docno></docno> + <approved></approved> + <checked></checked> + <date></date> + <rev></rev> + <file>ftp_client.xml</file> + </header> + + <section> + <title>Introduction</title> + + <p>Ftp clients are consider to be rather temporary and are + for that reason only started and stopped during + runtime and can not be started at application startup. + Due to the design of FTP client API, letting some + functions return intermediate results, only the process + that started the ftp client will be able to access it in + order to preserve sane semantics. (This could be solved + by changing the API and using the concept of a controlling + process more in line with other OTP applications, but + that is perhaps something for the future.) + If the process that started the ftp session + dies the ftp client process will terminate.</p> + + <p>The client supports ipv6 as long as the underlying mechanisms + also do so. </p> + + </section> + + <section> + <title>Using the FTP Client API</title> + <p>The following is a simple example of an ftp session, where + the user <c>guest</c> with password <c>password</c> logs on to + the remote host <c>erlang.org</c>, and where the file + <c>appl.erl</c> is transferred from the remote to the local + host. When the session is opened, the current directory at + the remote host is <c>/home/guest</c>, and <c>/home/fred</c> + at the local host. Before transferring the file, the current + local directory is changed to <c>/home/eproj/examples</c>, and + the remote directory is set to + <c>/home/guest/appl/examples</c>.</p> + <code type="erl"><![CDATA[ + 1> inets:start(). + ok + 2> {ok, Pid} = inets:start(ftpc, [{host, "erlang.org"}]). + {ok,<0.22.0>} + 3> ftp:user(Pid, "guest", "password"). + ok + 4> ftp:pwd(Pid). + {ok, "/home/guest"} + 5> ftp:cd(Pid, "appl/examples"). + ok + 6> ftp:lpwd(Pid). + {ok, "/home/fred"}. + 7> ftp:lcd(Pid, "/home/eproj/examples"). + ok + 8> ftp:recv(Pid, "appl.erl"). + ok + 9> inets:stop(ftpc, Pid). + ok + ]]></code> + </section> +</chapter> + + diff --git a/lib/inets/doc/src/http.xml b/lib/inets/doc/src/http.xml new file mode 100644 index 0000000000..f6f8338113 --- /dev/null +++ b/lib/inets/doc/src/http.xml @@ -0,0 +1,491 @@ +<?xml version="1.0" encoding="latin1" ?> +<!DOCTYPE erlref SYSTEM "erlref.dtd"> + +<erlref> + <header> + <copyright> + <year>2004</year><year>2009</year> + <holder>Ericsson AB. All Rights Reserved.</holder> + </copyright> + <legalnotice> + The contents of this file are subject to the Erlang Public License, + Version 1.1, (the "License"); you may not use this file except in + compliance with the License. You should have received a copy of the + Erlang Public License along with this software. If not, it can be + retrieved online at http://www.erlang.org/. + + Software distributed under the License is distributed on an "AS IS" + basis, WITHOUT WARRANTY OF ANY KIND, either express or implied. See + the License for the specific language governing rights and limitations + under the License. + + </legalnotice> + + <title>http</title> + <prepared>Ingela Anderton Andin</prepared> + <responsible></responsible> + <docno></docno> + <date></date> + <rev></rev> + </header> + <module>http</module> + <modulesummary>An HTTP/1.1 client </modulesummary> + <description> + <p>This module provides the API to a HTTP/1.1 client according to + RFC 2616, caching is currently not supported.</p> + <note> + <p>When starting the Inets application a manager process for the + default profile will be started. The functions in this API + that does not explicitly use a profile will accesses the + default profile. A profile keeps track of proxy options, + cookies and other options that can be applied to more than one + request. </p> + + <p>If the scheme + https is used the ssl application needs to be started.</p> + + <p>Also note that pipelining will only be used if the pipeline + timeout is set, otherwise persistent connections without + pipelining will be used e.i. the client always waits for + the previous response before sending the next request.</p> + </note> + <p>There are some usage examples in the <seealso + marker="http_client">Inets User's Guide.</seealso></p> + </description> + + <section> + <title>COMMON DATA TYPES </title> + <p>Type definitions that are used more than once in + this module:</p> + <code type="none"><![CDATA[ +boolean() = true | false +string() = list of ASCII characters +request_id() = ref() +profile() = atom() +path() = string() representing a file path or directory path +ip_address() = See inet(3) + ]]></code> + + </section> + + <section> + <title>HTTP DATA TYPES </title> + <p>Type definitions that are related to HTTP:</p> + <p>For more information about HTTP see rfc 2616</p> + + <code type="none"><![CDATA[ +method() = head | get | put | post | trace | options | delete +request() = {url(), headers()} | + {url(), headers(), content_type(), body()} +url() = string() - Syntax according to the URI definition in rfc 2396, ex: "http://www.erlang.org" +status_line() = {http_version(), status_code(), reason_phrase()} +http_version() = string() ex: "HTTP/1.1" +status_code() = integer() +reason_phrase() = string() +content_type() = string() +headers() = [header()] +header() = {field(), value()} +field() = string() +value() = string() +body() = string() | binary() +filename() = string() + ]]></code> + + </section> + + <section> + <title>SSL DATA TYPES </title> + <p>Some type definitions relevant when using https, + for details <seealso marker="ssl:ssl">ssl(3)</seealso>: </p> + <code type="none"><![CDATA[ +ssl_options() = {verify, code()} | + {depth, depth()} | + {certfile, path()} | + {keyfile, path()} | + {password, string()} | + {cacertfile, path()} | + {ciphers, string()} + ]]></code> + </section> + + <section> + <title>HTTP CLIENT SERVICE START/STOP </title> + + <p>A HTTP client can be configured to start when starting the inets + application or started dynamically in runtime by calling the + inets application API <c>inets:start(httpc, ServiceConfig)</c>, or + <c>inets:start(httpc, ServiceConfig, How)</c> + see <seealso marker="inets">inets(3)</seealso> Below follows a + description of the available configuration options.</p> + <taglist> + <tag>{profile, profile()}</tag> + <item>Name of the profile, see + common data types below, this option is mandatory.</item> + <tag>{data_dir, path()}</tag> + <item>Directory where the profile + may save persistent data, if omitted all cookies will be treated + as session cookies.</item> + </taglist> + + <p>The client can be stopped using inets:stop(httpc, Pid) or + inets:stop(httpc, Profile).</p> + + <marker id="cancel_request"></marker> + </section> + + <funcs> + <func> + <name>cancel_request(RequestId) -> </name> + <name>cancel_request(RequestId, Profile) -> ok</name> + <fsummary>Cancels an asynchronous HTTP-request.</fsummary> + <type> + <v>RequestId = request_id() - A unique identifier as returned + by request/4</v> + <v>Profile = profile()</v> + </type> + <desc> + <p>Cancels an asynchronous HTTP-request. </p> + + <marker id="request1"></marker> + </desc> + </func> + + <func> + <name>request(Url) -> </name> + <name>request(Url, Profile) -> {ok, Result} | {error, Reason}</name> + <fsummary>Sends a get HTTP-request</fsummary> + <type> + <v>Url = url() </v> <v>Result = {status_line(), headers(), + body()} | {status_code(), body()} | request_id() </v> + <v>Profile = profile()</v> + <v>Reason = term() </v> + </type> + <desc> + <p>Equivalent to http:request(get, {Url, []}, [], []).</p> + + <marker id="request2"></marker> + </desc> + </func> + + <func> + <name>request(Method, Request, HTTPOptions, Options) -> </name> + <name>request(Method, Request, HTTPOptions, Options, Profile) -> {ok, Result} | {ok, saved_to_file} | {error, Reason}</name> + + <fsummary>Sends a HTTP-request</fsummary> + <type> + <v>Method = method() </v> + <v>Request = request()</v> + <v>HTTPOptions = http_options()</v> + <v>http_options() = [http_option()]</v> + <v>http_option() = {timeout, timeout()} | + {connect_timeout, timeout()} | + {ssl, ssl_options()} | + {autoredirect, boolean()} | + {proxy_auth, {userstring(), passwordstring()}} | + {version, http_version()} | + {relaxed, boolean()}</v> + <v>timeout() = integer() >= 0 | infinity</v> + <v>Options = options()</v> + <v>options() = [option()]</v> + <v>option() = {sync, boolean()} | + {stream, stream_to()} | + {body_format, body_format()} | + {full_result, boolean()} | + {headers_as_is, boolean()}</v> + <v>stream_to() = self | {self, once} | filename() </v> + <v>body_format() = string() | binary() </v> + <v>Result = {status_line(), headers(), body()} | + {status_code(), body()} | request_id() </v> + <v>Profile = profile() </v> + <v>Reason = term() </v> + </type> + + <desc> + <p>Sends a HTTP-request. The function can be both synchronous + and asynchronous. In the later case the function will return + {ok, RequestId} and later on message(s) will be sent to the + calling process on the format: </p> +<pre> + {http, {RequestId, Result}} + {http, {RequestId, {error, Reason}}} + {http, {RequestId, stream_start, Headers} + {http, {RequestId, stream, BinBodyPart} + {http, {RequestId, stream_end, Headers} + {http, {RequestId, saved_to_file}}. +</pre> + + <p>Http option (<c>http_option()</c>) details: </p> + <taglist> + <tag><c><![CDATA[timeout]]></c></tag> + <item> + <p>Timeout time for the request. </p> + <p>Defaults to <c>infinity</c>. </p> + </item> + + <tag><c><![CDATA[connect_timeout]]></c></tag> + <item> + <p>Connection timeout time, used during the initial request, + when the client is connecting to the server. </p> + <p>Defaults to the value of the <c>timeout</c> option. </p> + </item> + + <tag><c><![CDATA[ssl]]></c></tag> + <item> + <p>If using SSL, these SSL-specific options are used. </p> + <p>Defaults to <c>[]</c>. </p> + </item> + + <tag><c><![CDATA[autoredirect]]></c></tag> + <item> + <p>Should the client automatically retreive the information + from the new URI and return that as the result instead + of a 30X-result code. </p> + <p>Note that for some 30X-result codes automatic redirect + is not allowed in these cases the 30X-result will always + be returned. </p> + <p>Defaults to <c>true</c>. </p> + </item> + + <tag><c><![CDATA[proxy_auth]]></c></tag> + <item> + <p>A proxy-authorization header using the provided user name and + password will be added to the request. </p> + </item> + + <tag><c><![CDATA[version]]></c></tag> + <item> + <p>Can be used to make the client act as an <c>HTTP/1.0</c> or + <c>HTTP/0.9</c> client. By default this is an <c>HTTP/1.1</c> + client. When using <c>HTTP/1.0</c> persistent connections will + not be used. </p> + <p>Defaults to the trsing <c>"HTTP/1.1"</c>. </p> + </item> + + <tag><c><![CDATA[relaxed]]></c></tag> + <item> + <p>If set to true workarounds for known server deviations from + the HTTP-standard are enabled. </p> + <p>Defaults to <c>false</c>. </p> + </item> + + </taglist> + + <p>Option (<c>option()</c>) details: </p> + <taglist> + <tag><c><![CDATA[sync]]></c></tag> + <item> + <p>Shall the request be synchronous or asynchronous. </p> + <p>Defaults to <c>true</c>. </p> + </item> + + <tag><c><![CDATA[stream]]></c></tag> + <item> + <p>Streams the body of a 200 or 206 response to the calling + process or to a file. When streaming to the calling process + using the option <c>self</c> the the following stream messages + will be sent to that process: {http, {RequestId, + stream_start, Headers}, {http, {RequestId, stream, + BinBodyPart}, {http, {RequestId, stream_end, Headers}. When + streaming to to the calling processes using the option + <c>{self once}</c> the first message will have an additional + element e.i. {http, {RequestId, stream_start, Headers, Pid}, + this is the process id that should be used as an argument to + http:stream_next/1 to trigger the next message to be sent to + the calling process. </p> + <p>Note that it is possible that chunked encoding will add + headers so that there are more headers in the stream_end + message than in the stream_start. + When streaming to a file and the request is asynchronous the + message {http, {RequestId, saved_to_file}} will be sent. </p> + <p>Defaults to <c>none</c>. </p> + </item> + + <tag><c><![CDATA[body_format]]></c></tag> + <item> + <p>Defines if the body shall be delivered as a string or as a + binary. This option is only valid for the synchronous + request. </p> + <p>Defaults to <c>string</c>. </p> + </item> + + <tag><c><![CDATA[full_result]]></c></tag> + <item> + <p>Should a "full result" be returned to the caller (that is, + the body, the headers and the entire status-line) or not + (the body and the status code). </p> + <p>Defaults to <c>true</c>. </p> + </item> + + <tag><c><![CDATA[header_as_is]]></c></tag> + <item> + <p>Shall the headers provided by the user be made + lower case or be regarded as case sensitive. </p> + <p>Note that the http standard requires them to be + case insenstive. This feature should only be used if there is + no other way to communicate with the server or for testing + purpose. Also note that when this option is used no headers + will be automatically added, all necessary headers has to be + provided by the user. </p> + <p>Defaults to <c>false</c>. </p> + </item> + + </taglist> + + <marker id="set_options"></marker> + </desc> + </func> + + <func> + <name>set_options(Options) -> </name> + <name>set_options(Options, Profile) -> ok | {error, Reason}</name> + <fsummary>Sets options to be used for subsequent requests.</fsummary> + <type> + <v>Options = [Option]</v> + <v>Option = {proxy, {Proxy, NoProxy}} | {max_sessions, MaxSessions} | + {max_keep_alive_length, MaxKeepAlive} | {keep_alive_timeout, KeepAliveTimeout} | + {max_pipeline_length, MaxPipeline} | {pipeline_timeout, PipelineTimeout} | + {cookies | CookieMode} | + {ipfamily, IpFamily} | {ip, IpAddress} | {port, Port} | + {verbose, VerboseMode} </v> + <v>Proxy = {Hostname, Port}</v> + <v>Hostname = string() </v> + <d>ex: "localhost" or "foo.bar.se"</d> + <v>Port = integer()</v> + <d>ex: 8080 </d> + <v>NoProxy = [NoProxyDesc]</v> + <v>NoProxyDesc = DomainDesc | HostName | IPDesc</v> + <v>DomainDesc = "*.Domain"</v> + <d>ex: "*.ericsson.se"</d> + <v>IpDesc = string()</v> + <d>ex: "134.138" or "[FEDC:BA98" (all IP-addresses starting with 134.138 or FEDC:BA98), "66.35.250.150" or "[2010:836B:4179::836B:4179]" (a complete IP-address).</d> + <v>MaxSessions = integer() </v> + <d>Default is <em>2</em>. + Maximum number of persistent connections to a host.</d> + <v>MaxKeepAlive = integer() </v> + <d>Default is <em>5</em>. + Maximum number of outstanding requests on the same connection to + a host.</d> + <v>KeepAliveTimeout = integer() </v> + <d>Default is <em>120000</em> (= 2 min). + If a persistent connection is idle longer than the + keep_alive_timeout the client will close the connection. + The server may also have a such a time out but you should + not count on it!</d> + <v>MaxPipeline = integer() </v> + <d>Default is <em>2</em>. + Maximum number of outstanding requests on a pipelined connection to a host.</d> + <v>PipelineTimeout = integer() </v> + <d>Default is <em>0</em>, + which will result in pipelining not being used. + If a persistent connection is idle longer than the + pipeline_timeout the client will close the connection. </d> + <v>CookieMode = enabled | disabled | verify </v> + <d>Default is <em>disabled</em>. + If Cookies are enabled all valid cookies will automatically be + saved in the client manager's cookie database. + If the option verify is used the function http:verify_cookie/2 + has to be called for the cookie to be saved.</d> + <v>IpFamily = inet | inet6 | inet6fb4 </v> + <d>By default <em>inet</em>. + When it is set to <c>inet6fb4</c> you can use both ipv4 and ipv6. + It first tries <c>inet6</c> and if that does not works falls back to <c>inet</c>. + The option is here to provide a workaround for buggy ipv6 stacks to ensure that + ipv4 will always work.</d> + <v>IpAddress = ip_address() </v> + <d>If the host has several network interfaces, this option specifies which one to use. + See gen_tcp:connect/3,4 for more info. </d> + <v>Port = integer() </v> + <d>Specify which local port number to use. + See gen_tcp:connect/3,4 for more info. </d> + <v>VerboseMode = false | verbose | debug | trace </v> + <d>Default is <em>false</em>. + This option is used to switch on (or off) + different levels of erlang trace on the client. + It is a debug feature.</d> + <v>Profile = profile()</v> + </type> + <desc> + <p>Sets options to be used for subsequent + requests.</p> + <note> + <p>If possible the client will keep its connections + alive and use persistent connections + with or without pipeline depending on configuration + and current circumstances. The HTTP/1.1 specification does not + provide a guideline for how many requests that would be + ideal to be sent on a persistent connection, + this very much depends on the + application. Note that a very long queue of requests may cause a + user perceived delays as earlier request may take a long time + to complete. The HTTP/1.1 specification does suggest a + limit of 2 persistent connections per server, which is the + default value of the max_sessions option. </p> + </note> + + <marker id="stream_next"></marker> + </desc> + </func> + + <func> + <name>stream_next(Pid) -> ok</name> + <fsummary> Triggers the next message to be streamed, e.i. + same behavior as active once for sockets. + </fsummary> + <type> + <v>Pid = pid() - as received in the stream_start message</v> + </type> + <desc> + <p>Triggers the next message to be streamed, e.i. + same behavior as active once for sockets.</p> + + <marker id="verify_cookie"></marker> + </desc> + </func> + + <func> + <name>verify_cookie(SetCookieHeaders, Url) -> </name> + <name>verify_cookie(SetCookieHeaders, Url, Profile) -> ok | {error, Reason}</name> + <fsummary>Saves the cookies defined in SetCookieHeaders in the client profile's cookie database.</fsummary> + <type> + <v>SetCookieHeaders = headers() - where field = "set-cookie"</v> + <v>Url = url()</v> + <v>Profile = profile()</v> + </type> + <desc> + <p>Saves the cookies defined in SetCookieHeaders + in the client profile's cookie database. You need to + call this function if you set the option cookies to verify. + If no profile is specified the default profile will be used. + </p> + + <marker id="cookie_header"></marker> + </desc> + </func> + + <func> + <name>cookie_header(Url) -> </name> + <name>cookie_header(Url, Profile) -> header() | {error, Rason}</name> + <fsummary>Returns the cookie header that would be sent when + making a request to Url using the profile Profile.</fsummary> + <type> + <v>Url = url()</v> + <v>Profile = profile()</v> + </type> + <desc> + <p>Returns the cookie header that would be sent + when making a request to Url using the profile Profile. + If no profile is specified the default profile will be used. + </p> + </desc> + </func> + </funcs> + + <section> + <title>SEE ALSO</title> + <p>RFC 2616, <seealso marker="inets">inets(3)</seealso>, + <seealso marker="ssl:ssl">ssl(3)</seealso> + </p> + </section> + +</erlref> + diff --git a/lib/inets/doc/src/http_client.xml b/lib/inets/doc/src/http_client.xml new file mode 100644 index 0000000000..510c30eb35 --- /dev/null +++ b/lib/inets/doc/src/http_client.xml @@ -0,0 +1,140 @@ +<?xml version="1.0" encoding="latin1" ?> +<!DOCTYPE chapter SYSTEM "chapter.dtd"> + +<chapter> + <header> + <copyright> + <year>2004</year><year>2009</year> + <holder>Ericsson AB. All Rights Reserved.</holder> + </copyright> + <legalnotice> + The contents of this file are subject to the Erlang Public License, + Version 1.1, (the "License"); you may not use this file except in + compliance with the License. You should have received a copy of the + Erlang Public License along with this software. If not, it can be + retrieved online at http://www.erlang.org/. + + Software distributed under the License is distributed on an "AS IS" + basis, WITHOUT WARRANTY OF ANY KIND, either express or implied. See + the License for the specific language governing rights and limitations + under the License. + + </legalnotice> + + <title>HTTP Client</title> + <prepared>Ingela Anderton Andin</prepared> + <responsible></responsible> + <docno></docno> + <approved></approved> + <checked></checked> + <date></date> + <rev></rev> + <file>http_client.xml</file> + </header> + + <section> + <title>Introduction</title> + + <p>The HTTP client default profile will be started when the inets + application is started and is then available to all processes on + that erlang node. Other profiles may also be started at + application startup, or profiles can be started and stopped + dynamically in runtime. Each client profile will spawn a new + process to handle each request unless there is a possibility to use + a persistent connection with or without pipelining. + The client will add a host header and an empty + te header if there are no such headers present in the request.</p> + + <p>The clients supports ipv6 as long as the underlying mechanisms also do + so.</p> + </section> + + <section> + <title>Configuration</title> + <p> What to put in the erlang node application configuration file + in order to start a profile at application startup.</p> + <pre> + [{inets, [{services, [{httpc, PropertyList}]}]}] + </pre> + <p>For valid properties see <seealso + marker="http">http(3)</seealso></p> + </section> + + <section> + <title>Using the HTTP Client API</title> + <code type="erl"> + 1 > inets:start(). + ok + </code> + <p> The following calls uses the default client profile. + Use the proxy "www-proxy.mycompany.com:8000", + but not for requests to localhost. This will apply to all subsequent + requests</p> + <code type="erl"> + 2 > http:set_options([{proxy, {{"www-proxy.mycompany.com", 8000}, + ["localhost"]}}]). + ok + </code> + <p>An ordinary synchronous request. </p> + <code type="erl"> + 3 > {ok, {{Version, 200, ReasonPhrase}, Headers, Body}} = + http:request(get, {"http://www.erlang.org", []}, [], []). + </code> + <p>With all default values, as above, a get request can also be written + like this.</p> + <code type="erl"> + 4 > {ok, {{Version, 200, ReasonPhrase}, Headers, Body}} = + http:request("http://www.erlang.org"). + </code> + <p>An ordinary asynchronous request. The result will be sent + to the calling process on the form {http, {ReqestId, Result}}</p> + <code type="erl"> + 5 > {ok, RequestId} = + http:request(get, {"http://www.erlang.org", []}, [], [{sync, false}]). + </code> + <p>In this case the calling process is the shell, so we receive the + result.</p> + <code type="erl"> + 6 > receive {http, {RequestId, Result}} -> ok after 500 -> error end. + ok + </code> + <p>Send a request with a specified connection header. </p> + <code type="erl"> + 7 > {ok, {{NewVersion, 200, NewReasonPhrase}, NewHeaders, NewBody}} = + http:request(get, {"http://www.erlang.org", [{"connection", "close"}]}, + [], []). + </code> + + <p>Start a HTTP client profile. </p> + + <code><![CDATA[ + 8 > {ok, Pid} = inets:start(httpc, [{profile, foo}]). + {ok, <0.45.0>} + ]]></code> + + <p>The new profile has no proxy settings so the connection will + be refused</p> + <code type="erl"> + 9 > http:request("http://www.erlang.org", foo). + {error,econnrefused} + </code> + + <p>Stop a HTTP client profile. </p> + <code type="erl"> + 10 > inets:stop(httpc, foo). + ok + </code> + + <p>Alternatively:</p> + <code type="erl"> + 10 > inets:stop(httpc, Pid). + ok + </code> + + + </section> +</chapter> + + + + diff --git a/lib/inets/doc/src/http_server.xml b/lib/inets/doc/src/http_server.xml new file mode 100644 index 0000000000..56317d647c --- /dev/null +++ b/lib/inets/doc/src/http_server.xml @@ -0,0 +1,1004 @@ +<?xml version="1.0" encoding="latin1" ?> +<!DOCTYPE chapter SYSTEM "chapter.dtd"> + +<chapter> + <header> + <copyright> + <year>2004</year><year>2009</year> + <holder>Ericsson AB. All Rights Reserved.</holder> + </copyright> + <legalnotice> + The contents of this file are subject to the Erlang Public License, + Version 1.1, (the "License"); you may not use this file except in + compliance with the License. You should have received a copy of the + Erlang Public License along with this software. If not, it can be + retrieved online at http://www.erlang.org/. + + Software distributed under the License is distributed on an "AS IS" + basis, WITHOUT WARRANTY OF ANY KIND, either express or implied. See + the License for the specific language governing rights and limitations + under the License. + + </legalnotice> + + <title>HTTP server </title> + <prepared>Ingela Anderton Andin</prepared> + <responsible></responsible> + <docno></docno> + <approved></approved> + <checked></checked> + <date></date> + <rev></rev> + <file>http_server.xml</file> + </header> + + <section> + <title>Introduction</title> + + <p>The HTTP server, also referred to as httpd, handles HTTP requests + as described in RFC 2616 with a few exceptions such as gateway + and proxy functionality. The server supports ipv6 as long as the + underlying mechanisms also do so. </p> + + <p>The server implements numerous features such as SSL (Secure Sockets + Layer), ESI (Erlang Scripting Interface), CGI (Common Gateway + Interface), User Authentication(using Mnesia, dets or plain text + database), Common Logfile Format (with or without disk_log(3) + support), URL Aliasing, Action Mappings, Directory Listings and SSI + (Server-Side Includes).</p> + + <p>The configuration of the server is provided as an erlang + property list, and for backwards compatibility also a configuration + file using apache-style configuration directives is + supported.</p> + + <p>As of inets version 5.0 the HTTP server is an easy to + start/stop and customize web server that provides the most basic + web server functionality. Depending on your needs there + are also other erlang based web servers that may be of interest + such as Yaws, http://yaws.hyber.org, that for instance has its own + markup support to generate html, and supports certain buzzword + technologies such as SOAP.</p> + + <p>Allmost all server functionality has been implemented using an + especially crafted server API, it is described in the Erlang Web + Server API. This API can be used to advantage by all who wants + to enhance the server core functionality, for example custom + logging and authentication.</p> + </section> + + <section> + <title>Configuration</title> + + <p> What to put in the erlang node application configuration file + in order to start a http server at application startup.</p> + + <code type="erl"> + [{inets, [{services, [{httpd, [{proplist_file, + "/var/tmp/server_root/conf/8888_props.conf"}]}, + {httpd, [{proplist_file, + "/var/tmp/server_root/conf/8080_props.conf"}]}]}]}]. + </code> + + <p>The server is configured using an erlang property list. + For the available properties see + <seealso marker="inets:inets">httpd(3)</seealso> + For backwards compatibility also apache-like config files + are supported. + </p> + + <p>All possible config properties are as follows </p> + <code type="none"> + httpd_service() -> {httpd, httpd()} + httpd() -> [httpd_config()] + httpd_config() -> {file, file()} | + {proplist_file, file()} + {debug, debug()} | + {accept_timeout, integer()} + debug() -> disable | [debug_options()] + debug_options() -> {all_functions, modules()} | + {exported_functions, modules()} | + {disable, modules()} + modules() -> [atom()] + </code> + <p>{proplist_file, file()} File containing an erlang property + list, followed by a full stop, describing the HTTP server + configuration.</p> + <p>{file, file()} If you use an old apace-like configuration file.</p> + <p>{debug, debug()} - Can enable trace on all + functions or only exported functions on chosen modules.</p> + <p>{accept_timeout, integer()} sets the wanted timeout value for + the server to set up a request connection.</p> + </section> + + <section> + <title>Using the HTTP Server API</title> + <code type="none"> + 1 > inets:start(). + ok + </code> + <p> Start a HTTP server with minimal + required configuration. Note that if you + specify port 0 an arbitrary available port will be + used and you can use the info function to find out + which port number that was picked. + </p> + + <code type="none"> + 2 > {ok, Pid} = inets:start(httpd, [{port, 0}, + {server_name,"httpd_test"}, {server_root,"/tmp"}, + {document_root,"/tmp/htdocs"}, {bind_address, "localhost"}]). + {ok, 0.79.0} + </code> + + <code type="none"> + 3 > httpd:info(Pid). + [{mime_types,[{"html","text/html"},{"htm","text/html"}]}, + {server_name,"httpd_test"}, + {bind_address, {127,0,0,1}}, + {server_root,"/tmp"}, + {port,59408}, + {document_root,"/tmp/htdocs"}] + </code> + + <p> Reload the configuration without restarting the server. + Note port and bind_address can not be changed. Clients + trying to access the server during the reload will + get a service temporary unavailable answer. + </p> + <code type="none"> + 4 > httpd:reload_config([{port, 59408}, + {server_name,"httpd_test"}, {server_root,"/tmp/www_test"}, + {document_root,"/tmp/www_test/htdocs"}, + {bind_address, "localhost"}], non_disturbing). + ok. + </code> + + <code type="none"> + 5 > httpd:info(Pid, [server_root, document_root]). + [{server_root,"/tmp/www_test"},{document_root,"/tmp/www_test/htdocs"}] + </code> + + <code type="none"> + 6 > ok = inets:stop(httpd, Pid). + </code> + + <p> Alternative:</p> + + <code type="none"> + 6 > ok = inets:stop(httpd, {{127,0,0,1}, 59408}). + </code> + + <p>Note that bind_address has to be + the ip address reported by the info function and can + not be the hostname that is allowed when inputting bind_address.</p> + + </section> + + <section> + <title>Htaccess - User Configurable Authentication.</title> + <p>If users of the web server needs to manage authentication of + web pages that are local to their user and do not have + server administrative privileges. They can use the + per-directory runtime configurable user-authentication scheme + that Inets calls htaccess. It works the following way: </p> + <list type="bulleted"> + <item>Each directory in the path to the requested asset is + searched for an access-file (default .htaccess), that restricts + the web servers rights to respond to a request. If an access-file + is found the rules in that file is applied to the + request. </item> + <item>The rules in an access-file applies both to files in the same + directories and in subdirectories. If there exists more than one + access-file in the path to an asset, the rules in the + access-file nearest the requested asset will be applied.</item> + <item>To change the rules that restricts the use of + an asset. The user only needs to have write access + to the directory where the asset exists.</item> + <item>All the access-files in the path to a requested asset is read + once per request, this means that the load on the server will + increase when this scheme is used.</item> + <item>If a directory is + limited both by auth directives in the HTTP server configuration + file and by the htaccess files. The user must be allowed to get + access the file by both methods for the request to succeed.</item> + </list> + + <section> + <title>Access Files Directives</title> + <p>In every directory under the <c>DocumentRoot</c> or under an + <c>Alias</c> a user can place an access-file. An access-file + is a plain text file that specify the restrictions that + shall be considered before the web server answer to a + request. If there are more than one access-file in the path + to the requested asset, the directives in the access-file in + the directory nearest the asset will be used.</p> + <list type="bulleted"> + <item> + <p><em>DIRECTIVE: "allow"</em></p> + <p><em>Syntax:</em><c>Allow</c> from subnet subnet|from all <br></br> +<em>Default:</em><c>from all </c> <br></br> +</p> + <p>Same as the directive allow for the server config file. </p> + </item> + <item> + <p><em>DIRECTIVE: "AllowOverRide"</em></p> + <p><em>Syntax:</em><c>AllowOverRide</c> all | none | + Directives <br></br> +<em>Default:</em><c>- None -</c> <br></br> +<c>AllowOverRide</c> Specify which parameters that not + access-files in subdirectories are allowed to alter the value + for. If the parameter is set to none no more + access-files will be parsed. + </p> + <p>If only one access-file exists setting this parameter to + none can lessen the burden on the server since the server + will stop looking for access-files.</p> + </item> + <item> + <p><em>DIRECTIVE: "AuthGroupfile"</em></p> + <p><em>Syntax:</em><c>AuthGroupFile</c> Filename <br></br> +<em>Default:</em><c>- None -</c> <br></br> +</p> + <p>AuthGroupFile indicates which file that contains the list + of groups. Filename must contain the absolute path to the + file. The format of the file is one group per row and + every row contains the name of the group and the members + of the group separated by a space, for example:</p> + <pre> +\011 GroupName: Member1 Member2 .... MemberN + </pre> + </item> + <item> + <p><em>DIRECTIVE: "AuthName"</em></p> + <p><em>Syntax:</em><c>AuthName</c> auth-domain <br></br> +<em>Default:</em><c>- None -</c> <br></br> +</p> + <p>Same as the directive AuthName for the server config file. </p> + </item> + <item> + <p><em>DIRECTIVE: "AuthType"</em></p> + <p><em>Syntax:</em><c>AuthType</c> Basic <br></br> +<em>Default:</em><c>Basic</c> <br></br> +</p> + <p><c>AuthType</c> Specify which authentication scheme that shall + be used. Today only Basic Authenticating using UUEncoding of + the password and user ID is implemented. </p> + </item> + <item> + <p><em>DIRECTIVE: "AuthUserFile"</em></p> + <p><em>Syntax:</em><c>AuthUserFile</c> Filename <br></br> +<em>Default:</em><c>- None -</c> <br></br> +</p> + <p><c>AuthUserFile</c> indicate which file that contains the list + of users. Filename must contain the absolute path to the + file. The users name and password are not encrypted so do not + place the file with users in a directory that is accessible + via the web server. The format of the file is one user per row + and every row contains User Name and Password separated by a + colon, for example:</p> + <pre> +\011 UserName:Password +\011 UserName:Password + </pre> + </item> + <item> + <p><em>DIRECTIVE: "deny"</em></p> + <p><em>Syntax:</em><c>deny</c> from subnet subnet|from all <br></br> +<em>Context:</em> Limit</p> + <p>Same as the directive deny for the server config file. </p> + </item> + <item> + <p><em>DIRECTIVE: "Limit"</em> <br></br> +</p> + <p><em>Syntax:</em><c><![CDATA[<Limit]]></c> RequestMethods<c>></c> <br></br> +<em>Default:</em> - None - <br></br> +</p> + <p><c><![CDATA[<Limit>]]></c> and </Limit> are used to enclose + a group of directives which applies only to requests using + the specified methods. If no request method is specified + all request methods are verified against the restrictions.</p> + <pre> +\011 <Limit POST GET HEAD> +\011 order allow deny +\011 require group group1 +\011 allow from 123.145.244.5 +\011 </Limit> + </pre> + </item> + <item> + <p><em>DIRECTIVE: "order"</em> <br></br> +<em>Syntax:</em><c>order</c> allow deny | deny allow <br></br> +<em>Default:</em> allow deny <br></br> +</p> + <p><c>order</c>, defines if the deny or allow control shall + be preformed first.</p> + <p>If the order is set to allow deny, then first the users + network address is controlled to be in the allow subset. If + the users network address is not in the allowed subset he will + be denied to get the asset. If the network-address is in the + allowed subset then a second control will be preformed, that + the users network address is not in the subset of network + addresses that shall be denied as specified by the deny + parameter.</p> + <p>If the order is set to deny allow then only users from networks + specified to be in the allowed subset will succeed to request + assets in the limited area.</p> + </item> + <item> + <p><em>DIRECTIVE: "require"</em></p> + <p><em>Syntax:</em><c>require</c> + group group1 group2...|user user1 user2... <br></br> +<em>Default:</em><c>- None -</c> <br></br> +<em>Context:</em> Limit <br></br> +</p> + <p>See the require directive in the documentation of mod_auth(3) + for more information.</p> + </item> + </list> + </section> + </section> + + <section> + <title>Dynamic Web Pages</title> + <p>The Inets HTTP server provides two ways of creating dynamic web + pages, each with its own advantages and disadvantages. </p> + <p>First there are CGI-scripts that can be written in any programming + language. CGI-scripts are standardized and supported by most + web servers. The drawback with CGI-scripts is that they are resource + intensive because of their design. CGI requires the server to fork a + new OS process for each executable it needs to start. </p> + <p>Second there are ESI-functions that provide a tight and efficient + interface to the execution of Erlang functions, this interface + on the other hand is Inets specific. </p> + + <section> + <title>The Common Gateway Interface (CGI) Version 1.1, RFC 3875.</title> + <p>The mod_cgi module makes it possible to execute CGI scripts + in the server. A file that matches the definition of a + ScriptAlias config directive is treated as a CGI script. A CGI + script is executed by the server and it's output is returned to + the client. </p> + <p>The CGI Script response comprises a message-header and a + message-body, separated by a blank line. The message-header + contains one or more header fields. The body may be + empty. Example: </p> + <code type="none"> +"Content-Type:text/plain\ +Accept-Ranges:none\ +\ +some very +\011plain text" </code> + <p>The server will interpret the cgi-headers and most of them + will be transformed into HTTP headers and sent back to the + client together with the body.</p> + <p>Support for CGI-1.1 is implemented in accordance with the RFC + 3875. </p> + </section> + + <section> + <title>Erlang Server Interface (ESI)</title> + <p>The erlang server interface is implemented by the + module mod_esi.</p> + + <section> + <title>ERL Scheme </title> + <p>The erl scheme is designed to mimic plain CGI, but without + the extra overhead. An URL which calls an Erlang erl function + has the following syntax (regular expression): </p> + <code type="none"> +\011 http://your.server.org/***/Module[:/]Function(?QueryString|/PathInfo) + </code> + <p>*** above depends on how the ErlScriptAlias config + directive has been used</p> + <p>The module (Module) referred to must be found in the code + path, and it must define a function (Function) with an arity + of two or three. It is preferable to implement a funtion + with arity three as it permits you to send chunks of the + webpage beeing generated to the client during the generation + phase instead of first generating the whole web page and + then sending it to the client. The option to implement a + function with arity two is only kept for + backwardcompatibilty reasons. + See <seealso marker="mod_esi">mod_esi(3)</seealso> for + implementation details of the esi callback function.</p> + </section> + + <section> + <title>EVAL Scheme </title> + <p>The eval scheme is straight-forward and does not mimic the + behavior of plain CGI. An URL which calls an Erlang eval + function has the following syntax:</p> + <code type="none"> +http://your.server.org/***/Mod:Func(Arg1,...,ArgN) + </code> + <p>*** above depends on how the ErlScriptAlias config + directive has been used</p> + <p>The module (Mod) referred to must be found in the code + path, and data returned by the function (Func) is passed + back to the client. Data returned from the + function must furthermore take the form as specified in + the CGI specification. See <seealso marker="mod_esi">mod_esi(3)</seealso> for implementation details of the esi + callback function.</p> + <note> + <p>The eval scheme can seriously threaten the + integrity of the Erlang node housing a Web server, for + example: </p> + <code type="none"> +http://your.server.org/eval?httpd_example:print(atom_to_list(apply(erlang,halt,[]))) + </code> + <p>which effectively will close down the Erlang node, + that is use the erl scheme instead, until this + security breach has been fixed.</p> + <p>Today there are no good way of solving this problem + and therefore Eval Scheme may be removed in future + release of Inets. </p> + </note> + </section> + </section> + </section> + + <section> + <title>Logging </title> + <p>There are three types of logs supported. Transfer logs, + security logs and error logs. The de-facto standard Common + Logfile Format is used for the transfer and security logging. + There are numerous statistics programs available to analyze Common + Logfile Format. The Common Logfile Format looks as follows: + </p> + <p><em>remotehost rfc931 authuser [date] "request" status bytes</em></p> + <taglist> + <tag><em>remotehost</em></tag> + <item>Remote hostname</item> + <tag><em>rfc931</em></tag> + <item>The client's remote username (RFC 931).</item> + <tag><em>authuser</em></tag> + <item>The username with which the user authenticated himself.</item> + <tag><em>[date]</em></tag> + <item>Date and time of the request (RFC 1123).</item> + <tag><em>"request"</em></tag> + <item>The request line exactly as it came from the client (RFC 1945).</item> + <tag><em>status</em></tag> + <item>The HTTP status code returned to the client (RFC 1945).</item> + <tag><em>bytes</em></tag> + <item>The content-length of the document transferred. </item> + </taglist> + <p>Internal server errors are recorde in the error log file. The + format of this file is a more ad hoc format than the logs using + Common Logfile Format, but conforms to the following syntax: + </p> + <p><em>[date]</em> access to <em>path</em> failed for + <em>remotehost</em>, reason: <em>reason</em></p> + </section> + + <section> + <title>Server Side Includes</title> + <p>Server Side Includes enables the server to run code embedded + in HTML pages to generate the response to the client.</p> + <note> + <p>Having the server parse HTML pages is a double edged sword! + It can be costly for a heavily loaded server to perform + parsing of HTML pages while sending them. Furthermore, it can + be considered a security risk to have average users executing + commands in the name of the Erlang node user. Carefully + consider these items before activating server-side includes.</p> + </note> + + <section> + <marker id="ssi_setup"></marker> + <title>SERVER-SIDE INCLUDES (SSI) SETUP</title> + <p>The server must be told which filename extensions to be used + for the parsed files. These files, while very similar to HTML, + are not HTML and are thus not treated the same. Internally, the + server uses the magic MIME type <c>text/x-server-parsed-html</c> + to identify parsed documents. It will then perform a format + conversion to change these files into HTML for the + client. Update the <c>mime.types</c> file, as described in the + Mime Type Settings, to tell the server which extension to use + for parsed files, for example: + </p> + <pre> +\011text/x-server-parsed-html shtml shtm + </pre> + <p>This makes files ending with <c>.shtml</c> and <c>.shtm</c> + into parsed files. Alternatively, if the performance hit is not a + problem, <em>all</em> HTML pages can be marked as parsed: + </p> + <pre> +\011text/x-server-parsed-html html htm + </pre> + </section> + + <section> + <marker id="ssi_format"></marker> + <title>Server-Side Includes (SSI) Format</title> + <p>All server-side include directives to the server are formatted + as SGML comments within the HTML page. This is in case the + document should ever find itself in the client's hands + unparsed. Each directive has the following format: + </p> + <pre> +\011<!--#command tag1="value1" tag2="value2" --> + </pre> + <p>Each command takes different arguments, most only accept one + tag at a time. Here is a breakdown of the commands and their + associated tags: + </p> + <p>The config directive controls various aspects of the + file parsing. There are two valid tags: + </p> + <taglist> + <tag><c>errmsg</c></tag> + <item> + <p>controls the message sent back to the client if an + error occurred while parsing the document. All errors are + logged in the server's error log.</p> + </item> + <tag><c>sizefmt</c></tag> + <item> + <p>determines the format used to display the size of + a file. Valid choices are <c>bytes</c> or + <c>abbrev</c>. <c>bytes</c> for a formatted byte count + or <c>abbrev</c> for an abbreviated version displaying + the number of kilobytes.</p> + </item> + </taglist> + <p>The include directory + will insert the text of a document into the parsed + document. This command accepts two tags:</p> + <taglist> + <tag><c>virtual</c></tag> + <item> + <p>gives a virtual path to a document on the + server. Only normal files and other parsed documents can + be accessed in this way.</p> + </item> + <tag><c>file</c></tag> + <item> + <p>gives a pathname relative to the current + directory. <c>../</c> cannot be used in this pathname, nor + can absolute paths. As above, you can send other parsed + documents, but you cannot send CGI scripts.</p> + </item> + </taglist> + <p>The echo directive prints the value of one of the include + variables (defined below). The only valid tag to this + command is <c>var</c>, whose value is the name of the + variable you wish to echo.</p> + <p>The fsize directive prints the size of the specified + file. Valid tags are the same as with the <c>include</c> + command. The resulting format of this command is subject + to the <c>sizefmt</c> parameter to the <c>config</c> + command.</p> + <p>The lastmod directive prints the last modification date of + the specified file. Valid tags are the same as with the + <c>include</c> command.</p> + <p>The exec directive executes a given shell command or CGI + script. Valid tags are:</p> + <taglist> + <tag><c>cmd</c></tag> + <item> + <p>executes the given string using <c>/bin/sh</c>. All + of the variables defined below are defined, and can be + used in the command.</p> + </item> + <tag><c>cgi</c></tag> + <item> + <p>executes the given virtual path to a CGI script and + includes its output. The server does not perform error + checking on the script output.</p> + </item> + </taglist> + </section> + + <section> + <marker id="ssi_environment_variables"></marker> + <title>Server-Side Includes (SSI) Environment Variables</title> + <p>A number of variables are made available to parsed + documents. In addition to the CGI variable set, the following + variables are made available: + </p> + <taglist> + <tag><c>DOCUMENT_NAME</c></tag> + <item> + <p>The current filename.</p> + </item> + <tag><c>DOCUMENT_URI</c></tag> + <item> + <p>The virtual path to this document (such as + <c>/docs/tutorials/foo.shtml</c>).</p> + </item> + <tag><c>QUERY_STRING_UNESCAPED</c></tag> + <item> + <p>The unescaped version of any search query the client + sent, with all shell-special characters escaped with + <c>\\</c>.</p> + </item> + <tag><c>DATE_LOCAL</c></tag> + <item> + <p>The current date, local time zone.</p> + </item> + <tag><c>DATE_GMT</c></tag> + <item> + <p>Same as DATE_LOCAL but in Greenwich mean time.</p> + </item> + <tag><c>LAST_MODIFIED</c></tag> + <item> + <p>The last modification date of the current document.</p> + </item> + </taglist> + </section> + </section> + + <section> + <title>The Erlang Web Server API</title> + <p>The process of handling a HTTP request involves several steps + such as:</p> + <list type="bulleted"> + <item>Seting up connections, sending and receiving data.</item> + <item>URI to filename translation</item> + <item>Authenication/access checks.</item> + <item>Retriving/generating the response.</item> + <item>Logging</item> + </list> + <p>To provide customization and extensibility of the HTTP servers + request handling most of these steps are handled by one or more + modules that may be replaced or removed at runtime, and of course + new ones can be added. For each request all modules will be + traversed in the order specified by the modules directive in the + server configuration file. Some parts mainly the communication + related steps are considered server core functionality and are + not implemented using the Erlang Web Server API. A description of + functionality implemented by the Erlang Webserver API is described + in the section Inets Webserver Modules.</p> + <p>A module can use data generated by previous modules in the + Erlang Webserver API module sequence or generate data to be used + by consecutive Erlang Web Server API modules. This is made + possible due to an internal list of key-value tuples, also referred to + as interaction data. </p> + <note> + <p>Interaction data enforces module dependencies and + should be avoided if possible. This means the order + of modules in the Modules property is significant.</p> + </note> + + <section> + <title>API Description</title> + <p>Each module implements server functionality + using the Erlang Web Server API should implement the following + call back functions:</p> + <list type="bulleted"> + <item>do/1 (mandatory) - the function called when + a request should be handled.</item> + <item>load/2</item> + <item>store/2</item> + <item>remove/1</item> + </list> + <p>The latter functions are needed only when new config + directives are to be introduced. For details see + <seealso marker="httpd">httpd(3)</seealso></p> + </section> + </section> + + <section> + <title>Inets Web Server Modules</title> <p>The convention is that + all modules implementing some webserver functionality has the + name mod_*. When configuring the web server an appropriate + selection of these modules should be present in the Module + directive. Please note that there are some interaction dependencies + to take into account so the order of the modules can not be + totally random.</p> + + <section> + <title>mod_action - Filetype/Method-Based Script Execution.</title> + <p>Runs CGI scripts whenever a file of a + certain type or HTTP method (See RFC 1945) is requested. + </p> + <p>Uses the following Erlang Web Server API interaction data: + </p> + <list type="bulleted"> + <item>real_name - from mod_alias</item> + </list> + <p>Exports the following Erlang Web Server API interaction data, if possible: + </p> + <taglist> + <tag><c>{new_request_uri, RequestURI}</c></tag> + <item>An alternative <c>RequestURI</c> has been generated.</item> + </taglist> + </section> + + <section> + <title>mod_alias - URL Aliasing</title> + <p>This module makes it possible to map different parts of the + host file system into the document tree e.i. creates aliases and + redirections.</p> + <p>Exports the following Erlang Web Server API interaction data, if possible: + </p> + <taglist> + <tag><c>{real_name, PathData}</c></tag> + <item>PathData is the argument used for API function mod_alias:path/3.</item> + </taglist> + </section> + + <section> + <title>mod_auth - User Authentication </title> + <p>This module provides for basic user authentication using + textual files, dets databases as well as mnesia databases.</p> + <p>Uses the following Erlang Web Server API interaction data: + </p> + <list type="bulleted"> + <item>real_name - from mod_alias</item> + </list> + <p>Exports the following Erlang Web Server API interaction data: + </p> + <taglist> + <tag><c>{remote_user, User}</c></tag> + <item>The user name with which the user has authenticated himself.</item> + </taglist> + + + <section> + <title>Mnesia as Authentication Database</title> + + <p> If Mnesia is used as storage method, Mnesia must be + started prio to the HTTP server. The first time Mnesia is + started the schema and the tables must be created before + Mnesia is started. A naive example of a module with two + functions that creates and start mnesia is provided + here. The function shall be used the first + time. first_start/0 creates the schema and the tables. The + second function start/0 shall be used in consecutive + startups. start/0 Starts Mnesia and wait for the tables to + be initiated. This function must only be used when the + schema and the tables already is created. </p> + + <code> + -module(mnesia_test). + -export([start/0,load_data/0]). + -include("mod_auth.hrl"). + + first_start()-> + mnesia:create_schema([node()]), + mnesia:start(), + mnesia:create_table(httpd_user, + [{type,bag},{disc_copies,[node()]}, + {attributes,record_info(fields,httpd_user)}]), + mnesia:create_table(httpd_group, + [{type,bag},{disc_copies,[node()]}, + {attributes,record_info(fields,httpd_group)}]), + mnesia:wait_for_tables([httpd_user,httpd_group],60000). + + start()-> + mnesia:start(), + mnesia:wait_for_tables([httpd_user,httpd_group],60000). + </code> + + <p>To create the Mnesia tables we use two records defined in + mod_auth.hrl so the file must be included. The first + function first_start/0 creates a schema that specify on + which nodes the database shall reside. Then it starts Mnesia + and creates the tables. The first argument is the name of + the tables, the second argument is a list of options how the + table will be created, see Mnesia documentation for more + information. Since the current implementation of the + mod_auth_mnesia saves one row for each user the type must be + bag. When the schema and the tables is created the second + function start/0 shall be used to start Mensia. It starts + Mnesia and wait for the tables to be loaded. Mnesia use the + directory specified as mnesia_dir at startup if specified, + otherwise Mnesia use the current directory. For security + reasons, make sure that the Mnesia tables are stored outside + the document tree of the HTTP server. If it is placed in the + directory which it protects, clients will be able to + download the tables. Only the dets and mnesia storage + methods allow writing of dynamic user data to disk. plain is + a read only method.</p> + </section> + + </section> + + <section> + <title>mod_cgi - CGI Scripts</title> + <p>This module handles invoking of CGI scripts</p> + </section> + + <section> + <title>mod_dir - Directories</title> + <p>This module generates an HTML directory listing + (Apache-style) if a client sends a request for a directory + instead of a file. This module needs to be removed from the + Modules config directive if directory listings is unwanted.</p> + <p>Uses the following Erlang Web Server API interaction data: + </p> + <list type="bulleted"> + <item>real_name - from mod_alias</item> + </list> + <p>Exports the following Erlang Web Server API interaction data: + </p> + <taglist> + <tag><c>{mime_type, MimeType}</c></tag> + <item>The file suffix of the incoming URL mapped into a + <c>MimeType</c>.</item> + </taglist> + </section> + + <section> + <title>mod_disk_log - Logging Using disk_log.</title> + <p>Standard logging using the "Common Logfile Format" and + disk_log(3).</p> + <p>Uses the following Erlang Web Server API interaction data: + </p> + <list type="bulleted"> + <item>remote_user - from mod_auth</item> + </list> + </section> + + <section> + <title>mod_esi - Erlang Server Interface</title> + <p>This module implements + the Erlang Server Interface (ESI) that provides a tight and + efficient interface to the execution of Erlang functions. </p> + <p>Uses the following Erlang Web Server API interaction data: + </p> + <list type="bulleted"> + <item>remote_user - from mod_auth</item> + </list> + <p>Exports the following Erlang Web Server API interaction data: + </p> + <taglist> + <tag><c>{mime_type, MimeType}</c></tag> + <item>The file suffix of the incoming URL mapped into a + <c>MimeType</c></item> + </taglist> + </section> + + <section> + <title>mod_get - Regular GET Requests</title> + <p>This module is responsible for handling GET requests to regular + files. GET requests for parts of files is handled by mod_range.</p> + <p>Uses the following Erlang Web Server API interaction data: + </p> + <list type="bulleted"> + <item>real_name - from mod_alias</item> + </list> + </section> + + <section> + <title>mod_head - Regular HEAD Requests</title> + <p>This module is responsible for handling HEAD requests to regular + files. HEAD requests for dynamic content is handled by each module + responsible for dynamic content.</p> + <p>Uses the following Erlang Web Server API interaction data: + </p> + <list type="bulleted"> + <item>real_name - from mod_alias</item> + </list> + </section> + + <section> + <title>mod_htaccess - User Configurable Access</title> + <p>This module provides per-directory user configurable access + control.</p> + <p>Uses the following Erlang Web Server API interaction data: + </p> + <list type="bulleted"> + <item>real_name - from mod_alias</item> + </list> + <p>Exports the following Erlang Web Server API interaction data: + </p> + <taglist> + <tag><c>{remote_user_name, User}</c></tag> + <item>The user name with which the user has authenticated himself.</item> + </taglist> + </section> + + <section> + <title>mod_include - SSI</title> + <p>This module makes it possible to expand "macros" embedded in + HTML pages before they are delivered to the client, that is + Server-Side Includes (SSI). + </p> + <p>Uses the following Erlang Webserver API interaction data: + </p> + <list type="bulleted"> + <item>real_name - from mod_alias</item> + <item>remote_user - from mod_auth</item> + </list> + <p>Exports the following Erlang Webserver API interaction data: + </p> + <taglist> + <tag><c>{mime_type, MimeType}</c></tag> + <item>The file suffix of the incoming URL mapped into a + <c>MimeType</c> as defined in the Mime Type Settings + section.</item> + </taglist> + </section> + + <section> + <title>mod_log - Logging Using Text Files.</title> + <p>Standard logging using the "Common Logfile Format" and text + files.</p> + <p>Uses the following Erlang Webserver API interaction data: + </p> + <list type="bulleted"> + <item>remote_user - from mod_auth</item> + </list> + </section> + + <section> + <title>mod_range - Requests with Range Headers</title> + <p>This module response to requests for one or many ranges of a + file. This is especially useful when downloading large files, + since a broken download may be resumed.</p> + <p>Note that request for multiple parts of a document will report a + size of zero to the log file.</p> + <p>Uses the following Erlang Webserver API interaction data: + </p> + <list type="bulleted"> + <item>real_name - from mod_alias</item> + </list> + </section> + + <section> + <title>mod_response_control - Requests with If* Headers</title> + <p>This module controls that the conditions in the requests is + fulfilled. For example a request may specify that the answer + only is of interest if the content is unchanged since last + retrieval. Or if the content is changed the range-request shall + be converted to a request for the whole file instead.</p> <p>If + a client sends more then one of the header fields that restricts + the servers right to respond, the standard does not specify how + this shall be handled. httpd will control each field in the + following order and if one of the fields not match the current + state the request will be rejected with a proper response. + <br></br> + + 1.If-modified <br></br> + + 2.If-Unmodified <br></br> + + 3.If-Match <br></br> + + 4.If-Nomatch <br></br> +</p> + <p>Uses the following Erlang Webserver API interaction data: + </p> + <list type="bulleted"> + <item>real_name - from mod_alias</item> + </list> + <p>Exports the following Erlang Webserver API interaction data: + </p> + <taglist> + <tag><c>{if_range, send_file}</c></tag> + <item>The conditions for the range request was not fulfilled. + The response must not be treated as a range request, instead it + must be treated as a ordinary get request. </item> + </taglist> + </section> + + <section> + <title>mod_security - Security Filter</title> + <p>This module serves as a filter for authenticated requests + handled in mod_auth. It provides possibility to restrict users + from access for a specified amount of time if they fail to + authenticate several times. It logs failed authentication as + well as blocking of users, and it also calls a configurable + call-back module when the events occur. </p> + <p>There is also an + API to manually block, unblock and list blocked users or users, + who have been authenticated within a configurable amount of + time.</p> + </section> + + <section> + <title>mod_trace - TRACE Request</title> + <p>mod_trace is responsible for handling of TRACE requests. + Trace is a new request method in HTTP/1.1. The intended use of + trace requests is for testing. The body of the trace response is + the request message that the responding Web server or proxy + received.</p> + </section> + </section> +</chapter> + + diff --git a/lib/inets/doc/src/httpd.xml b/lib/inets/doc/src/httpd.xml new file mode 100644 index 0000000000..60afcc6cfe --- /dev/null +++ b/lib/inets/doc/src/httpd.xml @@ -0,0 +1,1089 @@ +<?xml version="1.0" encoding="latin1" ?> +<!DOCTYPE erlref SYSTEM "erlref.dtd"> + +<erlref> + <header> + <copyright> + <year>1997</year><year>2009</year> + <holder>Ericsson AB. All Rights Reserved.</holder> + </copyright> + <legalnotice> + The contents of this file are subject to the Erlang Public License, + Version 1.1, (the "License"); you may not use this file except in + compliance with the License. You should have received a copy of the + Erlang Public License along with this software. If not, it can be + retrieved online at http://www.erlang.org/. + + Software distributed under the License is distributed on an "AS IS" + basis, WITHOUT WARRANTY OF ANY KIND, either express or implied. See + the License for the specific language governing rights and limitations + under the License. + + </legalnotice> + + <title>httpd</title> + <prepared>Ingela Anderton Andin</prepared> + <docno></docno> + <date>1997-10-14</date> + <rev>2.2</rev> + <file>httpd.sgml</file> + </header> + <module>httpd</module> + <modulesummary>An implementation of an HTTP + 1.1 compliant Web server, as defined in RFC 2616. + </modulesummary> + <description> + <p>Documents the HTTP server start options, some administrative + functions and also specifies the Erlang Web server callback + API</p> + </description> + + <section> + <title>COMMON DATA TYPES </title> + <p>Type definitions that are used more than once in + this module:</p> + <p><c>boolean() = true | false </c></p> + <p><c>string() = list of ASCII characters</c></p> + <p><c>path() = string() - representing a file or directory path.</c></p> + <p><c> ip_address() = {N1,N2,N3,N4} % IPv4 + | {K1,K2,K3,K4,K5,K6,K7,K8} % IPv6</c></p> + <p><c>hostname() = string() - representing a host ex "foo.bar.com"</c></p> + <p><c>property() = atom()</c></p> + + </section> + + <section> + <title>ERLANG HTTP SERVER SERVICE START/STOP </title> + <p>A web server can be configured to start when starting the inets + application or started dynamically in runtime by calling the + Inets application API <c>inets:start(httpd, ServiceConfig)</c>, or + <c>inets:start(httpd, ServiceConfig, How)</c>, + see <seealso marker="inets">inets(3)</seealso> Below follows a + description of the available configuration options, also called + properties.</p> + + <marker id="file_prop"></marker> + <p><em>File properties</em></p> + + <p>When the web server is started + at application start time the properties should be fetched from a + configuration file that could consist of a regular erlang property + list, e.i. <c>[{Option, Value}] </c> where <c> Option = property() + </c> and <c>Value = term()</c>, followed by a full stop, or for + backwards compatibility an Apache like configuration file. If the + web server is started dynamically at runtime you may still specify + a file but you could also just specify the complete property + list.</p> + + <taglist> + <tag>{proplist_file, path()}</tag> + <item> + If this property is defined inets will expect to find + all other properties defined in this file. Note that the + file must include all properties listed under mandatory + properties. </item> + <tag>{file, path()}</tag> + + <item> If this property is defined + inets will expect to find all other properties defined in this + file, that uses Apache like syntax. Note that the file must + include all properties listed under mandatory properties. The + Apache like syntax is the property, written as one word where + each new word begins with a capital, followed by a white-space + followed by the value followed by a new line. Ex: + + <code> + {server_root, "/urs/local/www"} -> ServerRoot /usr/local/www + </code> + + <p>With a few exceptions, that are documented + for each property that behaves differently, + and the special case {directory, {path(), PropertyList}} and + {security_directory, {Dir, PropertyList}} that are represented + as:</p> + <pre> + <![CDATA[ + <Directory Dir> + <Properties handled as described above> + </Directory> + ]]> + </pre> + </item> + </taglist> + <note> + <p>The properties proplist_file and file are mutually exclusive.</p> + </note> + + <marker id="mand_prop"></marker> + <p><em>Mandatory properties</em></p> + <taglist> + <tag>{port, integer()} </tag> + <item> + The port that the HTTP server shall listen on. + If zero is specified as port, an arbitrary available port + will be picked and you can use the httpd:info/2 function to find + out which port was picked. </item> + <tag>{server_name, string()} </tag> + <item> + The name of your server, normally a fully qualified domain + name. + </item> + <tag>{server_root, path()} </tag> + <item> + Defines the servers home directory where log files etc can + be stored. Relative paths specified in other properties refer + to this directory.</item> + <tag>{document_root, path()}</tag> + <item> + Defines the top directory for the documents that + are available on the HTTP server.</item> + </taglist> + + <marker id="comm_prop"></marker> + <p><em>Communication properties</em> </p> + <taglist> + <tag>{bind_address, ip_address() | hostname() | any} </tag> + <item> + Defaults to <c>any</c>. Note that <c>any</c> is denoted <em>*</em> + in the apache like configuration file. + </item> + + <tag>{socket_type, ip_comm | ssl}</tag> + <item> + <p>Defaults to <c>ip_comm</c>. </p> + </item> + + <tag>{ipfamily, inet | inet6 | inet6fb4}</tag> + <item> + <p>Defaults to <c>inet6fb4. </c> </p> + <p>Note that this option is only used when the option + <c>socket_type</c> has the value <c>ip_comm</c>. </p> + </item> + + </taglist> + + <p><em>Erlang Web server API modules</em> </p> + <taglist> + <tag>{modules, [atom()]} </tag> + <item> + Defines which modules the HTTP server will use to handle + requests. Defaults to: <c>[mod_alias, mod_auth, mod_esi, + mod_actions, mod_cgi, mod_dir, mod_get, mod_head, mod_log, + mod_disk_log] </c> + Note that some mod-modules are dependent on + others, so the order can not be entirely arbitrary. See the + <seealso marker="http_server"> Inets Web server Modules in the + Users guide</seealso> for more information. + </item> + </taglist> + + <marker id="limit_prop"></marker> + <p><em>Limit properties</em> </p> + <taglist> + <tag>{disable_chunked_transfer_encoding_send, boolean()}</tag> + <item> + This property allows you to disable chunked + transfer-encoding when sending a response to a HTTP/1.1 + client, by default this is false.</item> + + <tag>{keep_alive, boolean()}</tag> + <item> + Instructs the server whether or not to use persistent + connections when the client claims to be HTTP/1.1 + compliant, default is true.</item> + + <tag>{keep_alive_timeout, integer()}</tag> + <item> + The number of seconds the server will wait for a + subsequent request from the client before closing the + connection. Default is 150.</item> + + <tag>{max_body_size, integer()}</tag> + <item> + Limits the size of the message body of HTTP request. + By the default there is no limit.</item> + + <tag>{max_clients, integer()}</tag> + <item> + Limits the number of simultaneous requests that can be + supported. Defaults to 150. </item> + + <tag>{max_header_size, integer()}</tag> + <item> + Limits the size of the message header of HTTP request. + Defaults to 10240. + </item> + + <tag>{max_uri, integer()}</tag> + <item> + Limits the size of the HTTP request URI. By + default there is no limit.</item> + + <tag>{max_keep_alive_requests, integer()}</tag> + <item> The number of request that a client can do on one + connection. When the server has responded to the number of + requests defined by max_keep_alive_requests the server close the + connection. The server will close it even if there are queued + request. Defaults to no limit.</item> + </taglist> + + <marker id="admin_prop"></marker> + <p><em>Administrative properties</em></p> + <taglist> + <tag>{mime_types, [{MimeType, Extension}] | path()}</tag> + <item> + <p>Where MimeType = string() and Extension = string(). + Files delivered to the client are MIME typed according to RFC + 1590. File suffixes are mapped to MIME types before file delivery. + The mapping between file suffixes and MIME types can be specified + as an Apache like file as well as directly in the property list. Such + a file may look like:</p> + <pre> + \011 # MIME type\011\011\011Extension + \011 text/html\011\011\011html htm + \011 text/plain\011\011\011asc txt + </pre> + + <p>Defaults to [{"html","text/html"},{"htm","text/html"}]</p> + </item> + + <tag>{mime_type, string()}</tag> + + <item> + When the server is asked to provide a document type which + cannot be determined by the MIME Type Settings, the server will + use this default type. </item> + + <tag>{server_admin, string()}</tag> + <item> + ServerAdmin defines the email-address of the server + administrator, to be included in any error messages returned by + the server.</item> + + <tag>{log_format, common | combined}</tag> + <item> + <p>Defines if access logs should be written according to the common + log format or to the extended common log format. + The <c>common</c> format is one line that looks like this: + <c>remotehost rfc931 authuser [date] "request" status bytes</c></p> + + <pre>remotehost + Remote +rfc931 + The client's remote username (RFC 931). +authuser + The username with which the user authenticated himself. +[date] + Date and time of the request (RFC 1123). +"request" + The request line exactly as it came from the client(RFC 1945). +status + The HTTP status code returned to the client (RFC 1945). +bytes + The content-length of the document transferred. + </pre> + + <p>The <c>combined</c> format is on line that look like this: + <c>remotehost rfc931 authuser [date] "request" status bytes "referer" "user_agent" </c></p> + + <pre>"referer" + The url the client was on before + requesting your url. (If it could not be determined a minus + sign will be placed in this field) +"user_agent" + The software the client claims to be using. (If it + could not be determined a minus sign will be placed in + this field) + </pre> + + <p>This affects the access logs written by mod_log and mod_disk_log. + </p> + + </item> + + <tag>{error_log_format, pretty | compact}</tag> + <item> + <p>Defaults to pretty. If the error log is meant to be read + directly by a human <c>pretty</c> will be the best + option. <c>pretty</c> has the format corresponding to: + </p> + + <code>io:format("[~s] ~s, reason: ~n ~p ~n~n", [Date, Msg, Reason]). + </code> + + <p><c>compact</c> has the format corresponding to:</p> + + <code>io:format("[~s] ~s, reason: ~w ~n", [Date, Msg, Reason]). + </code> + + <p>This affects the error logs written by mod_log and mod_disk_log. + </p> + </item> + + </taglist> + + <marker id="ssl_prop"></marker> + <p><em>ssl properties</em></p> + <taglist> + <tag>{ssl_ca_certificate_file, path()}</tag> + <item> + Used as cacertfile option in ssl:listen/2 see + <seealso marker="ssl:ssl">ssl(3)</seealso> </item> + + <tag>{ssl_certificate_file, path()}</tag> + <item> + Used as certfile option in ssl:listen/2 see + <seealso marker="ssl:ssl">ssl(3)</seealso> + </item> + + <tag>{ssl_ciphers, list()}</tag> + <item> + Used as ciphers option in ssl:listen/2 see + <seealso marker="ssl:ssl">ssl(3)</seealso> + </item> + + <tag>{ssl_verify_client, integer()}</tag> + <item> + Used as verify option in ssl:listen/2 see + <seealso marker="ssl:ssl">ssl(3)</seealso> </item> + + <tag>{ssl_verify_depth, integer()}</tag> + <item> + Used as depth option in ssl:listen/2 see + <seealso marker="ssl:ssl">ssl(3)</seealso> </item> + + <tag>{ssl_password_callback_function, atom()}</tag> + <item> + Used together with ssl_password_callback_module + to retrieve a value to use as password option to ssl:listen/2 + see <seealso marker="ssl:ssl">ssl(3)</seealso> + </item> + + <tag>{ssl_password_callback_arguments, list()}</tag> + <item> + Used together with ssl_password_callback_function to supply a + list of arguments to the callback function. If not specified + the callback function will be assumed to have arity 0. </item> + + <tag>{ssl_password_callback_module, atom()}</tag> + <item> + Used together with ssl_password_callback_function + to retrieve a value to use as password option to ssl:listen/2 + see <seealso marker="ssl:ssl">ssl(3)</seealso></item> + + </taglist> + + <marker id="alias_prop"></marker> + <p><em>URL aliasing properties - requires mod_alias</em></p> + <taglist> + <tag>{alias, {Alias, RealName}}</tag> + + <item> Where Alias = string() and RealName = string(). + The Alias property allows documents to be stored in the local file + system instead of the document_root location. URLs with a path that + begins with url-path is mapped to local files that begins with + directory-filename, for example: + + <code>{alias, {"/image", "/ftp/pub/image"}</code> + + and an access to http://your.server.org/image/foo.gif would refer to + the file /ftp/pub/image/foo.gif.</item> + + <tag>{directory_index, [string()]}</tag> + + <item> + DirectoryIndex specifies a list of resources to look for + if a client requests a directory using a / at the end of the + directory name. file depicts the name of a file in the + directory. Several files may be given, in which case the server + will return the first it finds, for example: + + <code>{directory_index, ["index.hml", "welcome.html"]}</code> + + and access to http://your.server.org/docs/ would return + http://your.server.org/docs/index.html or + http://your.server.org/docs/welcome.html if index.html do not + exist. + </item> + </taglist> + + <marker id="cgi_prop"></marker> + <p><em>CGI properties - requires mod_cgi</em></p> + <taglist> + <tag>{script_alias, {Alias, RealName}}</tag> + <item> Where Alias = string() and RealName = string(). + Has the same behavior as the Alias property, except that + it also marks the target directory as containing CGI + scripts. URLs with a path beginning with url-path are mapped to + scripts beginning with directory-filename, for example: + + <code> {script_alias, {"/cgi-bin/", "/web/cgi-bin/"}</code> + + and an access to http://your.server.org/cgi-bin/foo would cause + the server to run the script /web/cgi-bin/foo. + </item> + + <tag>{script_nocache, boolean()}</tag> + + <item> + If ScriptNoCache is set to true the HTTP server will by + default add the header fields necessary to prevent proxies from + caching the page. Generally this is something you want. Defaults + to false.</item> + + <tag>{script_timeout, integer()}</tag> + + <item> + The time in seconds the web server will wait between each + chunk of data from the script. If the CGI-script not delivers + any data before the timeout the connection to the client will be + closed. Defaults to 15. </item> + + <tag>{action, {MimeType, CgiScript}} - requires mod_action</tag> + + <item>Where MimeType = string() and CgiScript = string(). + Action adds an action, which will activate a cgi-script + whenever a file of a certain mime-type is requested. It + propagates the URL and file path of the requested document using + the standard CGI PATH_INFO and PATH_TRANSLATED environment + variables. + <code> {action, {"text/plain", "/cgi-bin/log_and_deliver_text"} + </code> + </item> + + <tag>{script, {Method, CgiScript}} - requires mod_action</tag> + + <item>Where Method = string() and CgiScript = string(). + Script adds an action, which will activate a cgi-script + whenever a file is requested using a certain HTTP method. The + method is either GET or POST as defined in RFC 1945. It + propagates the URL and file path of the requested document using + the standard CGI PATH_INFO and PATH_TRANSLATED environment + variables. + + <code> {script, {"PUT", "/cgi-bin/put"} + </code> + + </item> + </taglist> + + <marker id="esi_prop"></marker> + <p><em>ESI properties - requires mod_esi</em></p> + <taglist> + <tag>{erl_script_alias, {URLPath, [AllowedModule]}}</tag> + + <item>Where URLPath = string() and AllowedModule = atom(). + erl_script_alias marks all URLs matching url-path as erl + scheme scripts. A matching URL is mapped into a specific module + and function. For example: + + <code>{erl_script_alias, {"/cgi-bin/example" [httpd_example]} + </code> + + and a request to + http://your.server.org/cgi-bin/example/httpd_example:yahoo + would refer to httpd_example:yahoo/2 and + http://your.server.org/cgi-bin/example/other:yahoo would + not be allowed to execute. + </item> + + <tag>{erl_script_nocache, boolean()}</tag> + + <item> + If erl_script_nocache is set to true the server will add + http header fields that prevents proxies from caching the + page. This is generally a good idea for dynamic content, since + the content often vary between each request. Defaults to false. + </item> + + <tag>{erl_script_timeout, integer()}</tag> + + <item> + If erl_script_timeout sets the time in seconds the server will + wait between each chunk of data to be delivered through + mod_esi:deliver/2. Defaults to 15. This is only relevant + for scripts that uses the erl scheme. + </item> + + <tag>{eval_script_alias, {URLPath, [AllowedModule]}}</tag> + + <item>Where URLPath = string() and AllowedModule = atom(). + Same as erl_script_alias but for scripts + using the eval scheme. Note that this is only supported + for backwards compatibility. The eval scheme is deprecated.</item> + </taglist> + + <marker id="log_prop"></marker> + <p><em>Log properties - requires mod_log</em></p> + <taglist> + <tag>{error_log, path()}</tag> + + <item> + Defines the filename of the error log file to be used to log + server errors. If the filename does not begin with a slash (/) + it is assumed to be relative to the server_root</item> + + <tag>{security_log, path()}</tag> + + <item> + Defines the filename of the access log file to be used to + log security events. If the filename does not begin with a slash + (/) it is assumed to be relative to the server_root. + </item> + + <tag>{transfer_log, path()}</tag> + + <item> + Defines the filename of the access log file to be used to + log incoming requests. If the filename does not begin with a + slash (/) it is assumed to be relative to the server_root. + </item> + </taglist> + + <marker id="dlog_prop"></marker> + <p><em>Disk Log properties - requires mod_disk_log</em></p> + <taglist> + <tag>{disk_log_format, internal | external}</tag> + + <item> + Defines the file-format of the log files see disk_log for + more information. If the internal file-format is used, the + logfile will be repaired after a crash. When a log file is + repaired data might get lost. When the external file-format is + used httpd will not start if the log file is broken. Defaults to + external. + </item> + + <tag>{error_disk_log, internal | external}</tag> + + <item> + Defines the filename of the (disk_log(3)) error log file + to be used to log server errors. If the filename does not begin + with a slash (/) it is assumed to be relative to the server_root. + </item> + + <tag>{error_disk_log_size, {MaxBytes, MaxFiles}}</tag> + + <item>Where MaxBytes = integer() and MaxFiles = integer(). + Defines the properties of the (disk_log(3)) error log + file. The disk_log(3) error log file is of type wrap log and + max-bytes will be written to each file and max-files will be + used before the first file is truncated and reused. </item> + + <tag>{security_disk_log, path()}</tag> + + <item> + Defines the filename of the (disk_log(3)) access log file + which logs incoming security events i.e authenticated + requests. If the filename does not begin with a slash (/) it + is assumed to be relative to the server_root. + </item> + + <tag>{security_disk_log_size, {MaxBytes, MaxFiles}}</tag> + + <item>Where MaxBytes = integer() and MaxFiles = integer(). + Defines the properties of the disk_log(3) access log + file. The disk_log(3) access log file is of type wrap log and + max-bytes will be written to each file and max-files will be + used before the first file is truncated and reused.</item> + + <tag>{transfer_disk_log, path()}</tag> + + <item> + Defines the filename of the (disk_log(3)) access log file + which logs incoming requests. If the filename does not begin + with a slash (/) it is assumed to be relative to the + server_root. + </item> + + <tag>{transfer_disk_log_size, {MaxBytes, MaxFiles}}</tag> + + <item>Where MaxBytes = integer() and MaxFiles = integer(). + Defines the properties of the disk_log(3) access log + file. The disk_log(3) access log file is of type wrap log and + max-bytes will be written to each file and max-files will be + used before the first file is truncated and reused.</item> + </taglist> + + <marker id="auth_prop"></marker> + <p><em>Authentication properties - requires mod_auth</em></p> + + <p><em>{directory, {path(), [{property(), term()}]}}</em></p> + + <marker id="dir_prop"></marker> + <p>Here follows the valid properties for directories </p> + + <taglist> + <tag>{allow_from, all | [RegxpHostString]}</tag> + + <item> + Defines a set of hosts which should be granted access to a + given directory. + + For example: + + <code>{allow_from, ["123.34.56.11", "150.100.23"] </code> + + The host 123.34.56.11 and all machines on the 150.100.23 + subnet are allowed access.</item> + + <tag>{deny_from, all | [RegxpHostString]}</tag> + + <item> + Defines a set of hosts + which should be denied access to a given directory. + For example: + + <code>{deny_from, ["123.34.56.11", "150.100.23"] </code> + + The host 123.34.56.11 and all machines on the 150.100.23 + subnet are not allowed access.</item> + + <tag>{auth_type, plain | dets | mnesia}</tag> + + <item> + Sets the type of authentication database that is used for the + directory.The key difference between the different methods is + that dynamic data can be saved when Mnesia and Dets is used. + This property is called AuthDbType in the Apache like + configuration files. + </item> + + <tag>{auth_user_file, path()}</tag> + + <item> + Sets the name of a file which contains the list of users and + passwords for user authentication. filename can be either + absolute or relative to the <c>server_root</c>. If using the + plain storage method, this file is a plain text file, where + each line contains a user name followed by a colon, followed + by the non-encrypted password. If user names are duplicated, + the behavior is undefined. For example: + + <code> ragnar:s7Xxv7 + edward:wwjau8 </code> + + If using the dets storage method, the user database is + maintained by dets and should not be edited by hand. Use the + API functions in mod_auth module to create / edit the user + database. This directive is ignored if using the mnesia + storage method. For security reasons, make sure that the + <c>auth_user_file</c> is stored outside the document tree of the Web + server. If it is placed in the directory which it protects, + clients will be able to download it. + </item> + + <tag>{auth_group_file, path()}</tag> + + <item> Sets the name of a file which contains the list of user + groups for user authentication. Filename can be either + absolute or relative to the <c>server_root</c>. If you use the plain + storage method, the group file is a plain text file, where + each line contains a group name followed by a colon, followed + by the member user names separated by spaces. For example: + + <code>group1: bob joe ante</code> + + If using the dets storage method, the group database is + maintained by dets and should not be edited by hand. Use the + API for mod_auth module to create / edit the group database. + This directive is ignored if using the mnesia storage method. + For security reasons, make sure that the <c>auth_group_file</c> is + stored outside the document tree of the Web server. If it is + placed in the directory which it protects, clients will be + able to download it.</item> + + <tag>{auth_name, string()}</tag> + + <item> + Sets the name of the authorization realm (auth-domain) for + a directory. This string informs the client about which user + name and password to use. </item> + + <tag>{auth_access_password, string()}</tag> + + <item> If set to other than "NoPassword" the password is required + for all API calls. If the password is set to "DummyPassword" the + password must be changed before any other API calls. To secure + the authenticating data the password must be changed after the + web server is started since it otherwise is written in clear + text in the configuration file.</item> + + <tag>{require_user, [string()]}</tag> + <item> + Defines users which should be granted access to a given + directory using a secret password. + </item> + + <tag>{require_group, [string()]}</tag> + <item> + Defines users which should be granted access to a given + directory using a secret password. + </item> + + </taglist> + + <marker id="htaccess_prop"></marker> + <p><em>Htaccess authentication properties - requires mod_htaccess</em></p> + <taglist> + <tag>{access_files, [path()]}</tag> + + <item> Specify which filenames that are used for + access-files. When a request comes every directory in the path + to the requested asset will be searched after files with the + names specified by this parameter. If such a file is found the + file will be parsed and the restrictions specified in it will + be applied to the request. + </item> + </taglist> + + <marker id="sec_prop"></marker> + <p><em>Security properties - requires mod_security </em></p> + + <p><em>{security_directory, {path(), [{property(), term()}]}</em></p> + + <marker id="sdir_prop"></marker> + <p> Here follows the valid properties for security directories</p> + <taglist> + <tag>{security_data_file, path()}</tag> + + <item> + Name of the security data file. The filename can either + absolute or relative to the server_root. This file is used to + store persistent data for the mod_security module. </item> + + <tag>{security_max_retries, integer()}</tag> + + <item> Specifies the maximum number of tries to authenticate a + user has before the user is blocked out. If a user + successfully authenticates when the user has been blocked, the + user will receive a 403 (Forbidden) response from the + server. If the user makes a failed attempt while blocked the + server will return 401 (Unauthorized), for security + reasons. Defaults to 3 may also be set to infinity.</item> + + <tag>{security_block_time, integer()}</tag> + + <item> Specifies the number of minutes a user is blocked. After + this amount of time, he automatically regains access. + Defaults to 60</item> + + <tag>{security_fail_expire_time, integer()}</tag> + + <item> + Specifies the number of minutes a failed user authentication + is remembered. If a user authenticates after this amount of + time, his previous failed authentications are + forgotten. Defaults to 30</item> + + <tag>{security_auth_timeout, integer()}</tag> + + <item> + Specifies the number of seconds a successful user + authentication is remembered. After this time has passed, the + authentication will no longer be reported. Defaults to 30. + </item> + </taglist> + </section> + + <funcs> + <func> + <name>info(Pid) -></name> + <name>info(Pid, Properties) -> [{Option, Value}]</name> + <fsummary>Fetches information about the HTTP server</fsummary> + <type> + <v>Properties = [property()]</v> + <v>Option = property()</v> + <v>Value = term()</v> + </type> + <desc> + <p>Fetches information about the HTTP server. When called + with only the pid all properties are fetched, when called + with a list of specific properties they are fetched. + Available properties are the same as the servers start options. + </p> + + <note><p>Pid is the pid returned from inets:start/[2,3]. + Can also be retrieved form inets:services/0, inets:services_info/0 + see <seealso marker="inets">inets(3)</seealso> + </p></note> + </desc> + </func> + + <func> + <name>info(Address, Port) -> </name> + <name>info(Address, Port, Properties) -> [{Option, Value}] </name> + <fsummary>Fetches information about the HTTP server</fsummary> + <type> + <v>Address = ip_address()</v> + <v>Port = integer()</v> + <v>Properties = [property()]</v> + <v>Option = property()</v> + <v>Value = term()</v> + </type> + <desc> + <p>Fetches information about the HTTP server. When called with + only the Address and Port all properties are fetched, when + called with a list of specific properties they are fetched. + Available properties are the same as the servers start + options. + </p> + + <note><p> Address has to be the ip-address and can not be + the hostname. + </p></note> + </desc> + </func> + + <func> + <name>reload_config(Config, Mode) -> ok | {error, Reason}</name> + <fsummary>Reloads the HTTP server configuration without + restarting the server.</fsummary> + <type> + <v>Config = path() | [{Option, Value}]</v> + <v>Option = property()</v> + <v>Value = term()</v> + <v>Mode = non_disturbing | disturbing</v> + </type> + <desc> + <p>Reloads the HTTP server configuration without restarting the + server. Incoming requests will be answered with a temporary + down message during the time the it takes to reload.</p> + + <note><p>Available properties are the same as the servers + start options, although the properties bind_address and + port can not be changed.</p></note> + + <p>If mode is disturbing, the server is blocked forcefully and + all ongoing requests are terminated and the reload will + start immediately. If mode is non-disturbing, no new + connections are accepted, but the ongoing requests are + allowed to complete before the reload is done.</p> + </desc> + </func> + </funcs> + + <section> + <title>ERLANG WEB SERVER API DATA TYPES </title> + <code type="none"> + ModData = #mod{} + + -record(mod, { +\011\011data = [], +\011\011socket_type = ip_comm, +\011\011socket, +\011\011config_db, +\011\011method, +\011\011absolute_uri, +\011\011request_uri, +\011\011http_version, +\011\011request_line, +\011\011parsed_header = [], +\011\011entity_body, +\011\011connection +\011 }). + </code> + <p>The fields of the <c>mod</c> record has the following meaning: + </p> + <taglist> + <tag><c>data</c></tag> + <item>Type <c>[{InteractionKey,InteractionValue}]</c> is used to + propagate data between modules. Depicted + <c>interaction_data()</c> in function type declarations. + </item> + <tag><c>socket_type</c></tag> + <item><c>socket_type()</c>, + Indicates whether it is an ip socket or a ssl socket. + </item> + <tag><c>socket</c></tag> + <item>The actual socket in <c>ip_comm</c> or <c>ssl</c> format + depending on the <c>socket_type</c>. + </item> + <tag><c>config_db</c></tag> + <item>The config file directives stored as key-value tuples in + an ETS-table. Depicted <c>config_db()</c> in function type + declarations. + </item> + <tag><c>method</c></tag> + <item>Type <c>"GET" | "POST" | "HEAD" | "TRACE"</c>, that is the + HTTP method. + </item> + <tag><c>absolute_uri</c></tag> + <item>If the request is a HTTP/1.1 + request the URI might be in the absolute URI format. In that + case httpd will save the absolute URI in this field. An Example + of an absolute URI could + be<c>"http://ServerName:Part/cgi-bin/find.pl?person=jocke"</c></item> + <tag><c>request_uri</c></tag> + <item>The <c>Request-URI</c> as defined + in RFC 1945, for example <c>"/cgi-bin/find.pl?person=jocke"</c></item> + <tag><c>http_version</c></tag> + <item>The <c>HTTP</c> version of the + request, that is "HTTP/0.9", "HTTP/1.0", or "HTTP/1.1". + </item> + <tag><c>request_line</c></tag> + <item>The <c>Request-Line</c> as + defined in RFC 1945, for example <c>"GET /cgi-bin/find.pl?person=jocke HTTP/1.0"</c>. + </item> + <tag><c>parsed_header</c></tag> + <item>Type <c>[{HeaderKey,HeaderValue}]</c>, + <c>parsed_header</c> contains all HTTP header fields from the + HTTP-request stored in a list as key-value tuples. See RFC 2616 + for a listing of all header fields. For example the date field + would be stored as: <c>{"date","Wed, 15 Oct 1997 14:35:17 GMT"}. + RFC 2616 defines that HTTP is a case insensitive protocol and + the header fields may be in lower case or upper case. Httpd will + ensure that all header field names are in lower case. </c>. + </item> + <tag><c>entity_body</c></tag> + <item>The <c>Entity-Body</c> as defined + in RFC 2616, for example data sent from a CGI-script using the + POST method. + </item> + <tag><c>connection</c></tag> + <item><c>true | false</c> If set to true the connection to the + client is a persistent connection and will not be closed when + the request is served.</item> + </taglist> + </section> + + <section> + <title>ERLANG WEB SERVER API CALLBACK FUNCTIONS</title> + </section> + <funcs> + <func> + <name>Module:do(ModData)-> {proceed, OldData} | {proceed, NewData} | {break, NewData} | done</name> + <fsummary>Called for each request to the Web server.</fsummary> + <type> + <v>OldData = list()</v> + <v>NewData = [{response,{StatusCode,Body}}] | [{response,{response,Head,Body}}] | [{response,{already_sent,Statuscode,Size}] </v> + <v>StausCode = integer()</v> + <v>Body = io_list() | nobody | {Fun, Arg}</v> + <v>Head = [HeaderOption]</v> + <v>HeaderOption = {Option, Value} | {code, StatusCode}</v> + <v>Option = accept_ranges | allow | cache_control | content_MD5 | content_encoding | content_language | content_length | content_location | content_range | content_type | date | etag | expires | last_modified | location | pragma | retry_after | server | trailer | transfer_encoding</v> + <v>Value = string()</v> + <v>Fun = fun( Arg ) -> sent| close | Body </v> + <v>Arg = [term()]</v> + </type> + <desc> + <p>When a valid request reaches httpd it calls <c>do/1</c> in + each module defined by the Modules configuration + option. The function may generate data for other modules + or a response that can be sent back to the client.</p> + <p>The field <c>data</c> in ModData is a list. This list will be + the list returned from the last call to + <c>do/1</c>.</p> + <p><c>Body</c> is the body of the http-response that will be + sent back to the client an appropriate header will be + appended to the message. <c>StatusCode</c> will be the + status code of the response see RFC2616 for the appropriate + values.</p> + <p><c>Head</c> is a key value list of HTTP header fields. The + server will construct a HTTP header from this data. See RFC + 2616 for the appropriate value for each header field. If the + client is a HTTP/1.0 client then the server will filter the + list so that only HTTP/1.0 header fields will be sent back + to the client.</p> + <p>If <c>Body</c> is returned and equal to <c>{Fun,Arg}</c>, + the Web server will try <c>apply/2</c> on <c>Fun</c> with + <c>Arg</c> as argument and expect that the fun either + returns a list <c>(Body)</c> that is a HTTP-repsonse or the + atom sent if the HTTP-response is sent back to the + client. If close is returned from the fun something has gone + wrong and the server will signal this to the client by + closing the connection.</p> + </desc> + </func> + <func> + <name>Module:load(Line, AccIn)-> eof | ok | {ok, AccOut} | {ok, AccOut, {Option, Value}} | {ok, AccOut, [{Option, Value}]} | {error, Reason} </name> + <fsummary>Load is used to convert a line in a Apache like config + file to a <c>{Option, Value}</c> tuple.</fsummary> + <type> + <v>Line = string()</v> + <v>AccIn = [{Option, Value}]</v> + <v>AccOut = [{Option, Value}]</v> + <v>Option = property()</v> + <v>Value = term() </v> + <v>Reason = term()</v> + </type> + <desc> + <p>Load is used to convert a line in a Apache like + configuration file to a <c>{Option, Value}</c> tuple. Some + more complex configuration options such as <c>directory</c> + and <c>security_directory</c> will create an + accumulator.This function does only need clauses for the + options implemented by this particular callback module. + </p> + </desc> + </func> + <func> + <name>Module:store({Option, Value}, Config)-> {ok, {Option, NewValue}} | {error, Reason} </name> + <fsummary></fsummary> + <type> + <v>Line = string()</v> + <v>Option = property()</v> + <v>Config = [{Option, Value}]</v> + <v>Value = term() </v> + <v>Reason = term()</v> + </type> + <desc> + <p>This function is used to check the validity of the + configuration options before saving them in the internal + database. This function may also have a side effect + e.i. setup necessary extra resources implied by the + configuration option. It can also + resolve possible dependencies among + configuration options by changing the value of the option. + This function does only need clauses for the options + implemented by this particular callback module.</p> + </desc> + </func> + + <func> + <name>Module:remove(ConfigDB) -> ok | {error, Reason} </name> + <fsummary>Callback function that is called when the Web server is closed.</fsummary> + <type> + <v>ConfigDB = ets_table()</v> + <v>Reason = term()</v> + </type> + <desc> + <p>When httpd is shutdown it will try to execute + <c>remove/1</c> in each Erlang web server callback module. The + programmer may use this function to clean up resources + that may have been created in the store function.</p> + </desc> + </func> + </funcs> + + <section> + <title>ERLANG WEB SERVER API HELP FUNCTIONS</title> + </section> + <funcs> + <func> + <name>parse_query(QueryString) -> [{Key,Value}]</name> + <fsummary>Parse incoming data to <c>erl </c>and <c>eval </c>scripts.</fsummary> + <type> + <v>QueryString = string()</v> + <v>Key = string()</v> + <v>Value = string()</v> + </type> + <desc> + <marker id="parse_query"></marker> + <p><c>parse_query/1</c> parses incoming data to <c>erl</c> and + <c>eval</c> scripts (See <seealso marker="mod_esi">mod_esi(3)</seealso>) as defined in the standard + URL format, that is '+' becomes 'space' and decoding of + hexadecimal characters (<c>%xx</c>).</p> + </desc> + </func> + </funcs> + + <section> + <title>SEE ALSO</title> + <p>RFC 2616, <seealso marker="inets">inets(3)</seealso>, + <seealso marker="ssl:ssl">ssl(3)</seealso> + </p> + </section> + +</erlref> + diff --git a/lib/inets/doc/src/httpd_conf.xml b/lib/inets/doc/src/httpd_conf.xml new file mode 100644 index 0000000000..a1ad76a8ae --- /dev/null +++ b/lib/inets/doc/src/httpd_conf.xml @@ -0,0 +1,144 @@ +<?xml version="1.0" encoding="latin1" ?> +<!DOCTYPE erlref SYSTEM "erlref.dtd"> + +<erlref> + <header> + <copyright> + <year>1997</year><year>2009</year> + <holder>Ericsson AB. All Rights Reserved.</holder> + </copyright> + <legalnotice> + The contents of this file are subject to the Erlang Public License, + Version 1.1, (the "License"); you may not use this file except in + compliance with the License. You should have received a copy of the + Erlang Public License along with this software. If not, it can be + retrieved online at http://www.erlang.org/. + + Software distributed under the License is distributed on an "AS IS" + basis, WITHOUT WARRANTY OF ANY KIND, either express or implied. See + the License for the specific language governing rights and limitations + under the License. + + </legalnotice> + + <title>httpd_conf</title> + <prepared>Joakim Grebenö</prepared> + <docno></docno> + <date>1997-10-14</date> + <rev>2.2</rev> + <file>httpd_conf.sgml</file> + </header> + <module>httpd_conf</module> + <modulesummary>Configuration utility functions to be used by the Erlang + Web server API programmer.</modulesummary> + <description> + <p>This module provides the Erlang Webserver API programmer with + utility functions for adding run-time configuration directives.</p> + </description> + <funcs> + <func> + <name>check_enum(EnumString,ValidEnumStrings) -> Result</name> + <fsummary>Check if string is a valid enumeration.</fsummary> + <type> + <v>EnumString = string()</v> + <v>ValidEnumStrings = [string()]</v> + <v>Result = {ok,atom()} | {error,not_valid}</v> + </type> + <desc> + <marker id="check_enum"></marker> + <p><c>check_enum/2</c> checks if <c>EnumString</c> is a valid + enumeration of <c>ValidEnumStrings</c> in which case it is + returned as an atom.</p> + </desc> + </func> + <func> + <name>clean(String) -> Stripped</name> + <fsummary>Remove leading and/or trailing white spaces.</fsummary> + <type> + <v>String = Stripped = string()</v> + </type> + <desc> + <marker id="clean"></marker> + <p><c>clean/1</c> removes leading and/or trailing white spaces + from <c>String</c>.</p> + </desc> + </func> + <func> + <name>custom_clean(String,Before,After) -> Stripped</name> + <fsummary>Remove leading and/or trailing white spaces and custom characters.</fsummary> + <type> + <v>Before = After = regexp()</v> + <v>String = Stripped = string()</v> + </type> + <desc> + <marker id="custom_clean"></marker> + <p><c>custom_clean/3</c> removes leading and/or trailing white + spaces and custom characters from <c>String</c>. <c>Before</c> + and <c>After</c> are regular expressions, as defined in + <c>regexp(3)</c>, describing the custom characters.</p> + </desc> + </func> + <func> + <name>is_directory(FilePath) -> Result</name> + <fsummary>Check if a file path is a directory.</fsummary> + <type> + <v>FilePath = string()</v> + <v>Result = {ok,Directory} | {error,Reason}</v> + <v>Directory = string()</v> + <v>Reason = string() | enoent | eaccess | enotdir | FileInfo</v> + <v>FileInfo = File info record</v> + </type> + <desc> + <marker id="is_directory"></marker> + <p><c>is_directory/1</c> checks if <c>FilePath</c> is a + directory in which case it is returned. Please read + <c>file(3)</c> for a description of <c>enoent</c>, + <c>eaccess</c> and <c>enotdir</c>. The definition of + the file info record can be found by including <c>file.hrl</c> + from the kernel application, see file(3).</p> + </desc> + </func> + <func> + <name>is_file(FilePath) -> Result</name> + <fsummary>Check if a file path is a regular file.</fsummary> + <type> + <v>FilePath = string()</v> + <v>Result = {ok,File} | {error,Reason}</v> + <v>File = string()</v> + <v>Reason = string() | enoent | eaccess | enotdir | FileInfo</v> + <v>FileInfo = File info record</v> + </type> + <desc> + <marker id="is_file"></marker> + <p><c>is_file/1</c> checks if <c>FilePath</c> is a regular + file in which case it is returned. Read <c>file(3)</c> for a + description of <c>enoent</c>, <c>eaccess</c> and + <c>enotdir</c>. The definition of the file info record can be + found by including <c>file.hrl</c> from the kernel application, + see file(3).</p> + </desc> + </func> + <func> + <name>make_integer(String) -> Result</name> + <fsummary>Return an integer representation of a string.</fsummary> + <type> + <v>String = string()</v> + <v>Result = {ok,integer()} | {error,nomatch}</v> + </type> + <desc> + <marker id="make_integer"></marker> + <p><c>make_integer/1</c> returns an integer representation of + <c>String</c>.</p> + </desc> + </func> + </funcs> + + <section> + <marker id="see_also"></marker> + <title>SEE ALSO</title> + <p><seealso marker="httpd">httpd(3)</seealso></p> + </section> + +</erlref> + + diff --git a/lib/inets/doc/src/httpd_socket.xml b/lib/inets/doc/src/httpd_socket.xml new file mode 100644 index 0000000000..fba1a58d3a --- /dev/null +++ b/lib/inets/doc/src/httpd_socket.xml @@ -0,0 +1,95 @@ +<?xml version="1.0" encoding="latin1" ?> +<!DOCTYPE erlref SYSTEM "erlref.dtd"> + +<erlref> + <header> + <copyright> + <year>1997</year><year>2009</year> + <holder>Ericsson AB. All Rights Reserved.</holder> + </copyright> + <legalnotice> + The contents of this file are subject to the Erlang Public License, + Version 1.1, (the "License"); you may not use this file except in + compliance with the License. You should have received a copy of the + Erlang Public License along with this software. If not, it can be + retrieved online at http://www.erlang.org/. + + Software distributed under the License is distributed on an "AS IS" + basis, WITHOUT WARRANTY OF ANY KIND, either express or implied. See + the License for the specific language governing rights and limitations + under the License. + + </legalnotice> + + <title>httpd_socket</title> + <prepared>Joakim Grebenö</prepared> + <docno></docno> + <date>1997-10-14</date> + <rev>2.2</rev> + <file>httpd_socket.sgml</file> + </header> + <module>httpd_socket</module> + <modulesummary>Communication utility functions to be used by the Erlang + Web server API programmer.</modulesummary> + <description> + <p>This module provides the Erlang Web server API module programmer + with utility functions for generic sockets communication. The + appropriate communication mechanism is transparently used, that + is <c>ip_comm</c> or <c>ssl</c>.</p> + </description> + <funcs> + <func> + <name>deliver(SocketType, Socket, Data) -> Result</name> + <fsummary>Send binary data over socket.</fsummary> + <type> + <v>SocketType = socket_type()</v> + <v>Socket = socket()</v> + <v>Data = io_list() | binary()</v> + <v>Result = socket_closed | void()</v> + </type> + <desc> + <marker id="deliver"></marker> + <p><c>deliver/3</c> sends the <c>Binary</c> over the + <c>Socket</c> using the specified <c>SocketType</c>. Socket + and SocketType should be the socket and the socket_type form + the mod record as defined in httpd.hrl</p> + </desc> + </func> + <func> + <name>peername(SocketType,Socket) -> {Port,IPAddress}</name> + <fsummary>Return the port and IP-address of the remote socket.</fsummary> + <type> + <v>SocketType = socket_type()</v> + <v>Socket = socket()</v> + <v>Port = integer()</v> + <v>IPAddress = string()</v> + </type> + <desc> + <marker id="peername"></marker> + <p><c>peername/3</c> returns the <c>Port</c> and + <c>IPAddress</c> of the remote <c>Socket</c>. </p> + </desc> + </func> + <func> + <name>resolve() -> HostName</name> + <fsummary>Return the official name of the current host.</fsummary> + <type> + <v>HostName = string()</v> + </type> + <desc> + <marker id="resolve"></marker> + <p><c>resolve/0</c> returns the official <c>HostName</c> of + the current host. </p> + </desc> + </func> + </funcs> + + <section> + <marker id="see_also"></marker> + <title>SEE ALSO</title> + <p><seealso marker="httpd">httpd(3)</seealso></p> + </section> + +</erlref> + + diff --git a/lib/inets/doc/src/httpd_util.xml b/lib/inets/doc/src/httpd_util.xml new file mode 100644 index 0000000000..1566ee29d1 --- /dev/null +++ b/lib/inets/doc/src/httpd_util.xml @@ -0,0 +1,459 @@ +<?xml version="1.0" encoding="latin1" ?> +<!DOCTYPE erlref SYSTEM "erlref.dtd"> + +<erlref> + <header> + <copyright> + <year>1997</year><year>2009</year> + <holder>Ericsson AB. All Rights Reserved.</holder> + </copyright> + <legalnotice> + The contents of this file are subject to the Erlang Public License, + Version 1.1, (the "License"); you may not use this file except in + compliance with the License. You should have received a copy of the + Erlang Public License along with this software. If not, it can be + retrieved online at http://www.erlang.org/. + + Software distributed under the License is distributed on an "AS IS" + basis, WITHOUT WARRANTY OF ANY KIND, either express or implied. See + the License for the specific language governing rights and limitations + under the License. + + </legalnotice> + + <title>httpd_util</title> + <prepared>Joakim Grebenö</prepared> + <docno></docno> + <date>1997-10-14</date> + <rev>2.2</rev> + <file>httpd_util.sgml</file> + </header> + <module>httpd_util</module> + <modulesummary>Miscellaneous utility functions to be used when implementing Erlang Web server API modules.</modulesummary> + <description> + <p>This module provides the Erlang Web Server API module + programmer with miscellaneous utility functions.</p> + + <marker id="convert_request_date"></marker> + </description> + + <funcs> + <func> + <name>convert_request_date(DateString) -> ErlDate|bad_date</name> + <fsummary>Convert The the date to the Erlang date format.</fsummary> + <type> + <v>DateString = string()</v> + <v>ErlDate = {{Year,Month,Date},{Hour,Min,Sec}}</v> + <v>Year = Month = Date = Hour = Min = Sec = integer()</v> + </type> + <desc> + <p><c>convert_request_date/1</c> converts <c>DateString</c> to + the Erlang date format. DateString must be in one of the three + date formats that is defined in the RFC 2616.</p> + + <marker id="create_etag"></marker> + </desc> + </func> + + <func> + <name>create_etag(FileInfo) -> Etag</name> + <fsummary>Calculates the Etag for a file.</fsummary> + <type> + <v>FileInfo = file_info()</v> + <v>Etag = string()</v> + </type> + <desc> + <p><c>create_etag/1</c> calculates the Etag for a file, from it's + size and time for last modification. fileinfo is a record defined + in <c>kernel/include/file.hrl</c></p> + + <marker id="decode_hex"></marker> + </desc> + </func> + + <func> + <name>decode_hex(HexValue) -> DecValue</name> + <fsummary>Convert a hex value into its decimal equivalent.</fsummary> + <type> + <v>HexValue = DecValue = string()</v> + </type> + <desc> + <p>Converts the hexadecimal value <c>HexValue</c> into it's + decimal equivalent (<c>DecValue</c>).</p> + + <marker id="day"></marker> + </desc> + </func> + + <func> + <name>day(NthDayOfWeek) -> DayOfWeek</name> + <fsummary>Convert the day of the week (integer [1-7]) to an abbreviated string.</fsummary> + <type> + <v>NthDayOfWeek = 1-7</v> + <v>DayOfWeek = string()</v> + </type> + <desc> + <marker id="day"></marker> + <p><c>day/1</c> converts the day of the week + (<c>NthDayOfWeek</c>) as an integer (1-7) to an abbreviated + string, that is: </p> + <p>1 = "Mon", 2 = "Tue", ..., 7 = "Sat".</p> + + <marker id="flatlength"></marker> + </desc> + </func> + + <func> + <name>flatlength(NestedList) -> Size</name> + <fsummary>Compute the size of a possibly nested list.</fsummary> + <type> + <v>NestedList = list()</v> + <v>Size = integer()</v> + </type> + <desc> + <marker id="flatlength"></marker> + <p><c>flatlength/1</c> computes the size of the possibly nested + list <c>NestedList</c>. Which may contain binaries.</p> + + <marker id="hexlist_to_integer"></marker> + </desc> + </func> + +<!-- + <func> + <name>header(StatusCode,PersistentConn)</name> + <name>header(StatusCode,Date)</name> + <name>header(StatusCode,MimeType,Date)</name> + <name>header(StatusCode,MimeType,PersistentConn,Date) -> HTTPHeader</name> + <fsummary>Generate a HTTP 1.1 header.</fsummary> + <type> + <v>StatusCode = integer()</v> + <v>Date = rfc1123_date()</v> + <v>MimeType = string()</v> + <v>PersistentConn = true | false</v> + </type> + <desc> + <marker id="header"></marker> + <p><c>header</c> returns a HTTP 1.1 header string. The + <c>StatusCode</c> is one of the status codes defined in RFC + 2616 and the <c>Date</c> string is RFC 1123 + compliant. (See <seealso marker="#rfc1123_date">rfc1123_date/0</seealso>). + </p> + <p>Note that the two version of <c>header/n</c> that does not + has a <c>PersistentConn</c> argument is there only for + backward compatibility, and must not be used in new Erlang + Webserver API modules. that will support persistent + connections.</p> + + <marker id="hexlist_to_integer"></marker> + </desc> + </func> +--> + + <func> + <name>hexlist_to_integer(HexString) -> Number</name> + <fsummary>Convert a hexadecimal string to an integer.</fsummary> + <type> + <v>Number = integer()</v> + <v>HexString = string()</v> + </type> + <desc> + <p><c>hexlist_to_integer</c> Convert the Hexadecimal value of + HexString to an integer.</p> + + <marker id="integer_to_hexlist"></marker> + </desc> + </func> + + <func> + <name>integer_to_hexlist(Number) -> HexString</name> + <fsummary>Convert an integer to a hexadecimal string.</fsummary> + <type> + <v>Number = integer()</v> + <v>HexString = string()</v> + </type> + <desc> + <marker id="integer_to_hexlist"></marker> + <p><c>integer_to_hexlist/1</c> Returns a string that represents + the Number in a Hexadecimal form.</p> + + <marker id="lookup"></marker> + </desc> + </func> + + <func> + <name>lookup(ETSTable,Key) -> Result</name> + <name>lookup(ETSTable,Key,Undefined) -> Result</name> + <fsummary>Extract the first value associated with a key in an ETS table.</fsummary> + <type> + <v>ETSTable = ets_table()</v> + <v>Key = term()</v> + <v>Result = term() | undefined | Undefined</v> + <v>Undefined = term()</v> + </type> + <desc> + <p><c>lookup</c> extracts <c>{Key,Value}</c> tuples from + <c>ETSTable</c> and returns the <c>Value</c> associated + with <c>Key</c>. If <c>ETSTable</c> is of type <c>bag</c> + only the first <c>Value</c> associated with <c>Key</c> is + returned. <c>lookup/2</c> returns <c>undefined</c> and + <c>lookup/3</c> returns <c>Undefined</c> if no <c>Value</c> + is found.</p> + + <marker id="lookup_mime"></marker> + </desc> + </func> + + <func> + <name>lookup_mime(ConfigDB,Suffix)</name> + <name>lookup_mime(ConfigDB,Suffix,Undefined) -> MimeType</name> + <fsummary>Return the mime type associated with a specific file suffix. </fsummary> + <type> + <v>ConfigDB = ets_table()</v> + <v>Suffix = string()</v> + <v>MimeType = string() | undefined | Undefined</v> + <v>Undefined = term()</v> + </type> + <desc> + <marker id="lookup_mime"></marker> + <p><c>lookup_mime</c> returns the mime type associated with a + specific file suffix as specified in the <c>mime.types</c> + file (located in the <path unix="$SERVER_ROOT/conf/mime.types" windows="%SERVER_ROOT%\\\\conf\\\\mime.types">config directory</path>).</p> + + <marker id="lookup_mime_default"></marker> + </desc> + </func> + + <func> + <name>lookup_mime_default(ConfigDB,Suffix)</name> + <name>lookup_mime_default(ConfigDB,Suffix,Undefined) -> MimeType</name> + <fsummary>Return the mime type associated with a specific file suffix or the value of the DefaultType.</fsummary> + <type> + <v>ConfigDB = ets_table()</v> + <v>Suffix = string()</v> + <v>MimeType = string() | undefined | Undefined</v> + <v>Undefined = term()</v> + </type> + <desc> + <marker id="lookup_mime_default"></marker> + <p><c>lookup_mime_default</c> returns the mime type associated + with a specific file suffix as specified in the + <c>mime.types</c> file (located in the + <path unix="$SERVER_ROOT/conf/mime.types" windows="%SERVER_ROOT%\\\\conf\\\\mime.types">config directory</path>). + If no appropriate association can be found + the value of DefaultType is + returned.</p> + + <marker id="message"></marker> + </desc> + </func> + + <func> + <name>message(StatusCode,PhraseArgs,ConfigDB) -> Message</name> + <fsummary>Return an informative HTTP 1.1 status string in HTML.</fsummary> + <type> + <v>StatusCode = 301 | 400 | 403 | 404 | 500 | 501 | 504</v> + <v>PhraseArgs = term()</v> + <v>ConfigDB = ets_table</v> + <v>Message = string()</v> + </type> + <desc> + <marker id="message"></marker> + <p><c>message/3</c> returns an informative HTTP 1.1 status + string in HTML. Each <c>StatusCode</c> requires a specific + <c>PhraseArgs</c>: + </p> + <taglist> + <tag><c>301</c></tag> + <item><c>string()</c>: A URL pointing at the new document + position.</item> + <tag><c>400 | 401 | 500</c></tag> + <item><c>none</c> (No <c>PhraseArgs</c>)</item> + <tag><c>403 | 404</c></tag> + <item><c>string()</c>: A <c>Request-URI</c> as described in + RFC 2616.</item> + <tag><c>501</c></tag> + <item><c>{Method,RequestURI,HTTPVersion}</c>: The HTTP + <c>Method</c>, <c>Request-URI</c> and <c>HTTP-Version</c> + as defined in RFC 2616.</item> + <tag><c>504</c></tag> + <item><c>string()</c>: A string describing why the service + was unavailable.</item> + </taglist> + + <marker id="month"></marker> + </desc> + </func> + + <func> + <name>month(NthMonth) -> Month</name> + <fsummary>Convert the month as an integer (1-12) to an abbreviated string.</fsummary> + <type> + <v>NthMonth = 1-12</v> + <v>Month = string()</v> + </type> + <desc> + <marker id="month"></marker> + <p><c>month/1</c> converts the month <c>NthMonth</c> as an + integer (1-12) to an abbreviated string, that is: </p> + <p>1 = "Jan", 2 = "Feb", ..., 12 = "Dec".</p> + + <marker id="multi_lookup"></marker> + </desc> + </func> + + <func> + <name>multi_lookup(ETSTable,Key) -> Result</name> + <fsummary>Extract the values associated with a key in a ETS table.</fsummary> + <type> + <v>ETSTable = ets_table()</v> + <v>Key = term()</v> + <v>Result = [term()]</v> + </type> + <desc> + <p><c>multi_lookup</c> extracts all <c>{Key,Value}</c> tuples + from an <c>ETSTable</c> and returns <em>all</em><c>Values</c> associated with the <c>Key</c> in a list.</p> + + <marker id="reason phrase"></marker> + </desc> + </func> + + <func> + <name>reason_phrase(StatusCode) -> Description</name> + <fsummary>Return the description of an HTTP 1.1 status code.</fsummary> + <type> + <v>StatusCode = 100| 200 | 201 | 202 | 204 | 205 | 206 | 300 | 301 | 302 | 303 | 304 | 400 | 401 | 402 | 403 | 404 | 405 | 406 | 410 411 | 412 | 413 | 414 415 | 416 | 417 | 500 | 501 | 502 | 503 | 504 | 505</v> + <v>Description = string()</v> + </type> + <desc> + <p><c>reason_phrase</c> returns the <c>Description</c> of an + HTTP 1.1 <c>StatusCode</c>, for example 200 is "OK" and 201 + is "Created". Read RFC 2616 for further information.</p> + + <marker id="rfc1123_date"></marker> + </desc> + </func> + + <func> + <name>rfc1123_date() -> RFC1123Date</name> + <name>rfc1123_date({{YYYY,MM,DD},{Hour,Min,Sec}}}) -> RFC1123Date</name> + <fsummary>Return the current date in RFC 1123 format.</fsummary> + <type> + <v>YYYY = MM = DD = Hour = Min =Sec = integer()</v> + <v>RFC1123Date = string()</v> + </type> + <desc> + <marker id="rfc1123_date"></marker> + <p><c>rfc1123_date/0</c> returns the current date in RFC 1123 + format. <c>rfc_date/1</c> converts the date in the Erlang format + to the RFC 1123 date format.</p> + + <marker id="split"></marker> + </desc> + </func> + + <func> + <name>split(String,RegExp,N) -> SplitRes</name> + <fsummary>Split a string in N chunks using a regular expression.</fsummary> + <type> + <v>String = RegExp = string()</v> + <v>SplitRes = {ok, FieldList} | {error, errordesc()}</v> + <v>Fieldlist = [string()]</v> + <v>N = integer</v> + </type> + <desc> + <marker id="split"></marker> + <p><c>split/3</c> splits the <c>String</c> in <c>N</c> chunks + using the <c>RegExp</c>. <c>split/3</c> is is equivalent to + <c>regexp:split/2</c> with one exception, that is <c>N</c> + defines the number of maximum number of fields in the + <c>FieldList</c>.</p> + + <marker id="split_script_path"></marker> + </desc> + </func> + + <func> + <name>split_script_path(RequestLine) -> Splitted</name> + <fsummary>Split a <c>RequestLine</c>in a file reference to an executable and a<c>QueryString</c>or a <c>PathInfo</c>string.</fsummary> + <type> + <v>RequestLine = string()</v> + <v>Splitted = not_a_script | {Path, PathInfo, QueryString}</v> + <v>Path = QueryString = PathInfo = string()</v> + </type> + <desc> + <marker id="split_script_path"></marker> + <p><c>split_script_path/1</c> is equivalent to + <c>split_path/1</c> with one exception. If the longest + possible path is not a regular, accessible and executable + file <c>not_a_script</c> is returned.</p> + + <marker id="split_path"></marker> + </desc> + </func> + + <func> + <name>split_path(RequestLine) -> {Path,QueryStringOrPathInfo}</name> + <fsummary>Split a <c>RequestLine</c>in a file reference and a <c>QueryString</c>or a<c>PathInfo</c>string.</fsummary> + <type> + <v>RequestLine = Path = QueryStringOrPathInfo = string()</v> + </type> + <desc> + <marker id="split_path"></marker> + <p><c>split_path/1</c> splits the <c>RequestLine</c> in a file + reference (<c>Path</c>) and a <c>QueryString</c> or a + <c>PathInfo</c> string as specified in RFC 2616. A + <c>QueryString</c> is isolated from the <c>Path</c> with a + question mark (<c>?</c>) and <c>PathInfo</c> with a slash + (/). In the case of a <c>QueryString</c>, everything before + the <c>?</c> is a <c>Path</c> and everything after a + <c>QueryString</c>. In the case of a <c>PathInfo</c> the + <c>RequestLine</c> is scanned from left-to-right on the hunt + for longest possible <c>Path</c> being a file or a + directory. Everything after the longest possible + <c>Path</c>, isolated with a <c>/</c>, is regarded as + <c>PathInfo</c>. The resulting <c>Path</c> is decoded using + <c>decode_hex/1</c> before delivery.</p> + + <marker id="strip"></marker> + </desc> + </func> + + <func> + <name>strip(String) -> Stripped</name> + <fsummary>Returns String where the leading and trailing space and tabs has been removed.</fsummary> + <type> + <v>String = Stripped = string()</v> + </type> + <desc> + <marker id="strip"></marker> + <p><c>strip/1</c> removes any leading or trailing linear white + space from the string. Linear white space should be read as + horizontal tab or space.</p> + + <marker id="suffix"></marker> + </desc> + </func> + + <func> + <name>suffix(FileName) -> Suffix</name> + <fsummary>Extract the file suffix from a given filename.</fsummary> + <type> + <v>FileName = Suffix = string()</v> + </type> + <desc> + <marker id="suffix"></marker> + <p><c>suffix/1</c> is equivalent to + <c>filename:extension/1</c> with one exception, that is + <c>Suffix</c> is returned without a leading dot (<c>.</c>). </p> + </desc> + </func> + </funcs> + + <section> + <marker id="see_also"></marker> + <title>SEE ALSO</title> + <p><seealso marker="httpd">httpd(3)</seealso></p> + </section> + +</erlref> diff --git a/lib/inets/doc/src/inets.gif b/lib/inets/doc/src/inets.gif Binary files differnew file mode 100644 index 0000000000..64968ae68a --- /dev/null +++ b/lib/inets/doc/src/inets.gif diff --git a/lib/inets/doc/src/inets.xml b/lib/inets/doc/src/inets.xml new file mode 100644 index 0000000000..e5fe32f32f --- /dev/null +++ b/lib/inets/doc/src/inets.xml @@ -0,0 +1,179 @@ +<?xml version="1.0" encoding="latin1" ?> +<!DOCTYPE erlref SYSTEM "erlref.dtd"> + +<erlref> + <header> + <copyright> + <year>2007</year><year>2009</year> + <holder>Ericsson AB. All Rights Reserved.</holder> + </copyright> + <legalnotice> + The contents of this file are subject to the Erlang Public License, + Version 1.1, (the "License"); you may not use this file except in + compliance with the License. You should have received a copy of the + Erlang Public License along with this software. If not, it can be + retrieved online at http://www.erlang.org/. + + Software distributed under the License is distributed on an "AS IS" + basis, WITHOUT WARRANTY OF ANY KIND, either express or implied. See + the License for the specific language governing rights and limitations + under the License. + + </legalnotice> + + <title>inets</title> + <prepared>Ingela Anderton Andin</prepared> + <responsible></responsible> + <docno></docno> + <date></date> + <rev></rev> + </header> + <module>inets</module> + <modulesummary>The inets services API</modulesummary> + <description> + <p>This module provides the most basic API to the + clients and servers, that are part of the Inets application, + such as start and stop. </p> + </description> + + <section> + <title>COMMON DATA TYPES </title> + <p>Type definitions that are used more than once in + this module: </p> + <p><c> service() = ftpc | tfptd | httpc | httpd</c></p> + <p><c> property() = atom() </c></p> + </section> + <funcs> + <func> + <name>services() -> [{Service, Pid}]</name> + <fsummary>Returns a list of currently running services. </fsummary> + <type> + <v>Service = service()</v> + <v>Pid = pid()</v> + </type> + <desc> + <p>Returns a list of currently running services.</p> + <note> + <p>Services started as <c>stand_alone</c> will not + be listed.</p> + </note> + </desc> + </func> + <func> + <name>services_info() -> [{Service, Pid, Info}]</name> + <fsummary>Returns a list of currently running services where + each service is described by a [{Option, Value}] + list. </fsummary> + <type> + <v>Service = service()</v> + <v>Pid = pid()</v> + <v>Info = [{Option, Value}]</v> + <v>Option = property()</v> + <v>Value = term()</v> + </type> + <desc> + <p>Returns a list of currently running services where each + service is described by a [{Option, Value}] list. The + information given in the list is specific for each service + and it is probable that each service will have its own info + function that gives you even more details about the + service.</p> + </desc> + </func> + + <func> + <name>service_names() -> [Service] </name> + <fsummary>Returns a list of available service names.</fsummary> + <type> + <v>Service = service()</v> + </type> + <desc> + <p>Returns a list of available service names.</p> + </desc> + </func> + + <func> + <name>start() -> </name> + <name>start(Type) -> ok | {error, Reason}</name> + <fsummary>Starts the Inets application. </fsummary> + <type> + <v>Type = permanent | transient | temporary</v> + </type> + <desc> + <p>Starts the Inets application. Default type + is temporary. See also + <seealso marker="kernel:application">application(3)</seealso></p> + </desc> + </func> + <func> + <name>stop() -> ok </name> + <fsummary>Stops the inets application.</fsummary> + <desc> + <p>Stops the inets application. See also + <seealso marker="kernel:application">application(3)</seealso></p> + </desc> + </func> + <func> + <name>start(Service, ServiceConfig) -> {ok, Pid} | {error, Reason}</name> + <name>start(Service, ServiceConfig, How) -> {ok, Pid} | + {error, Reason}</name> + <fsummary>Dynamically starts an inets + service after the inets application has been + started. </fsummary> + <type> + <v>Service = service()</v> + <v>ServiceConfig = [{Option, Value}]</v> + <v>Option = property()</v> + <v>Value = term()</v> + <v>How = inets | stand_alone - default is inets</v> + </type> + <desc> + <p>Dynamically starts an inets service after the inets + application has been started.\011</p> + <note> + <p>Dynamically started services will not be handled by + application takeover and failover behavior when inets is + run as a distributed application. Nor will they be + automatically restarted when the inets application is + restarted, but as long as the inets application is up and + running they will be supervised and may be soft code + upgraded. Services started as <c>stand_alone</c>, + e.i. the service is not started as part of the inets + application, will lose all OTP application benefits such + as soft upgrade. The "stand_alone-service" will be linked to + the process that started it. In most cases some of the + supervision functionality will still be in place and in + some sense the calling process has now become the top + supervisor.</p> + </note> + </desc> + </func> + <func> + <name>stop(Service, Reference) -> ok | {error, Reason} </name> + <fsummary>Stops a started service of the inets application or takes + down a "stand_alone-service" gracefully.</fsummary> + <type> + <v>Service = service() | stand_alone</v> + <v>Reference = pid() | term() - service specified reference</v> + <v>Reason = term()</v> + </type> + <desc> + <p>Stops a started service of the inets application or takes + down a "stand_alone-service" gracefully. When the + <c>stand_alone</c> option is used in start, + only the pid is a valid argument to stop.</p> + </desc> + </func> + </funcs> + + <section> + <title>SEE ALSO</title> + <p><seealso marker="ftp">ftp(3)</seealso>, + <seealso marker="http">http(3)</seealso>, + <seealso marker="httpd">httpd(3)</seealso>, + <seealso marker="tftp">tftp(3)</seealso></p> + </section> + +</erlref> + + diff --git a/lib/inets/doc/src/inets_services.xml b/lib/inets/doc/src/inets_services.xml new file mode 100644 index 0000000000..c274d67f19 --- /dev/null +++ b/lib/inets/doc/src/inets_services.xml @@ -0,0 +1,78 @@ +<?xml version="1.0" encoding="latin1" ?> +<!DOCTYPE chapter SYSTEM "chapter.dtd"> + +<chapter> + <header> + <copyright> + <year>1997</year><year>2009</year> + <holder>Ericsson AB. All Rights Reserved.</holder> + </copyright> + <legalnotice> + The contents of this file are subject to the Erlang Public License, + Version 1.1, (the "License"); you may not use this file except in + compliance with the License. You should have received a copy of the + Erlang Public License along with this software. If not, it can be + retrieved online at http://www.erlang.org/. + + Software distributed under the License is distributed on an "AS IS" + basis, WITHOUT WARRANTY OF ANY KIND, either express or implied. See + the License for the specific language governing rights and limitations + under the License. + + </legalnotice> + + <title>Introduction</title> + <prepared>Ingela Anderton Andin</prepared> + <responsible></responsible> + <docno></docno> + <approved></approved> + <checked></checked> + <date>2004-09-28</date> + <rev>A</rev> + <file>inets_services.xml</file> + </header> + + <section> + <title>Purpose</title> + <p>Inets is a container for Internet clients and + servers. Currently, an <term id="HTTP"></term>client and server, a + TFPT client and server, and a FTP client has been incorporated + into Inets. The HTTP server and client is HTTP 1.1 compliant as + defined in <term id="RFC"></term>2616.</p> + </section> + + <section> + <title>Prerequisites</title> + <p>It is assumed that the reader is familiar with the Erlang + programming language, concepts of OTP and has a basic + understanding of the HTTP, TFTP and FTP protocols.</p> + </section> + + <section> + <title>The Service Concept</title> + <p>Each client and server in inets is viewed as service. Services + may be configured to be started at application startup or + started dynamically in runtime. If you want to run inets as an + distributed application that should handle application failover + and takeover, services should be configured to be started at + application startup. When starting the inets application + the inets top supervisor will start a number of subsupervisors + and worker processes for handling the different services + provided. When starting services dynamically new children will + be added to the supervision tree, unless the service is started + with the stand alone option, in which case the service is linked + to the calling process and all OTP application features such as + soft upgrade are lost.</p> + <p>Services that should be configured for startup at application + startup time should be put into the erlang node configuration file + on the form: </p> + <pre> + [{inets, [{services, ListofConfiguredServices}]}]. + </pre> + <p>For details of exactly what to put in the list of configured + services see the documentation for the services that should be + configured.</p> + </section> +</chapter> + + diff --git a/lib/inets/doc/src/make.dep b/lib/inets/doc/src/make.dep new file mode 100644 index 0000000000..d96c6dc5b8 --- /dev/null +++ b/lib/inets/doc/src/make.dep @@ -0,0 +1,27 @@ +# ---------------------------------------------------- +# >>>> Do not edit this file <<<< +# This file was automaticly generated by +# /home/otp/bin/docdepend +# ---------------------------------------------------- + + +# ---------------------------------------------------- +# TeX files that the DVI file depend on +# ---------------------------------------------------- + +book.dvi: book.tex ftp.tex ftp_client.tex http.tex http_client.tex \ + http_server.tex httpd.tex httpd_conf.tex httpd_socket.tex \ + httpd_util.tex inets.tex inets_services.tex \ + mod_alias.tex mod_auth.tex mod_esi.tex mod_security.tex \ + part.tex ref_man.tex tftp.tex + +# ---------------------------------------------------- +# Source inlined when transforming from source to LaTeX +# ---------------------------------------------------- + +book.tex: ref_man.xml + +ftp.tex: ../../../../system/doc/definitions/term.defs + +inets_services.tex: ../../../../system/doc/definitions/term.defs + diff --git a/lib/inets/doc/src/marting_tankar.sdw b/lib/inets/doc/src/marting_tankar.sdw Binary files differnew file mode 100644 index 0000000000..90e1d2130d --- /dev/null +++ b/lib/inets/doc/src/marting_tankar.sdw diff --git a/lib/inets/doc/src/min_head.gif b/lib/inets/doc/src/min_head.gif Binary files differnew file mode 100644 index 0000000000..67948a6378 --- /dev/null +++ b/lib/inets/doc/src/min_head.gif diff --git a/lib/inets/doc/src/mod_alias.xml b/lib/inets/doc/src/mod_alias.xml new file mode 100644 index 0000000000..c783b99b23 --- /dev/null +++ b/lib/inets/doc/src/mod_alias.xml @@ -0,0 +1,132 @@ +<?xml version="1.0" encoding="latin1" ?> +<!DOCTYPE erlref SYSTEM "erlref.dtd"> + +<erlref> + <header> + <copyright> + <year>1997</year><year>2009</year> + <holder>Ericsson AB. All Rights Reserved.</holder> + </copyright> + <legalnotice> + The contents of this file are subject to the Erlang Public License, + Version 1.1, (the "License"); you may not use this file except in + compliance with the License. You should have received a copy of the + Erlang Public License along with this software. If not, it can be + retrieved online at http://www.erlang.org/. + + Software distributed under the License is distributed on an "AS IS" + basis, WITHOUT WARRANTY OF ANY KIND, either express or implied. See + the License for the specific language governing rights and limitations + under the License. + + </legalnotice> + + <title>mod_alias</title> + <prepared>Joakim Grebenö</prepared> + <docno></docno> + <date>1997-10-14</date> + <rev>2.2</rev> + <file>mod_alias.sgml</file> + </header> + <module>mod_alias</module> + <modulesummary>URL aliasing.</modulesummary> + <description> + <p>Erlang Webserver Server internal API for handling of things + such as interaction data exported by the mod_alias module.</p> + </description> + <funcs> + <func> + <name>default_index(ConfigDB, Path) -> NewPath</name> + <fsummary>Return a new path with the default resource or file appended.</fsummary> + <type> + <v>ConfigDB = config_db()</v> + <v>Path = NewPath = string()</v> + </type> + <desc> + <marker id="default_index"></marker> + <p>If <c>Path</c> is a directory, <c>default_index/2</c>, it starts + searching for resources or files that are specified in the config + directive DirectoryIndex. + If an appropriate resource or file is found, it is appended to + the end of <c>Path</c> and then returned. <c>Path</c> is + returned unaltered, if no appropriate + file is found, or if <c>Path</c> is not a directory. + <c>config_db()</c> is the server config file in ETS table format + as described in + <seealso marker="http_server">Inets Users Guide.</seealso>.</p> + </desc> + </func> + <func> + <name>path(PathData, ConfigDB, RequestURI) -> Path</name> + <fsummary>Return the actual file path to a URL.</fsummary> + <type> + <v>PathData = interaction_data()</v> + <v>ConfigDB = config_db()</v> + <v>RequestURI = Path = string()</v> + </type> + <desc> + <marker id="path"></marker> + <p><c>path/3</c> returns the actual file <c>Path</c> in the + <c>RequestURI</c> (See RFC 1945). If the interaction data + <c>{real_name,{Path,AfterPath}}</c> has been exported by + mod_alias; + <c>Path</c> is returned. If no interaction data has been + exported, ServerRoot is used to + generate a file <c>Path</c>. <c>config_db()</c> and + <c>interaction_data()</c> are as defined in <seealso marker="http_server">Inets Users Guide</seealso>.</p> + </desc> + </func> + <func> + <name>real_name(ConfigDB, RequestURI, Aliases) -> Ret</name> + <fsummary>Expand a request uri using Alias config directives.</fsummary> + <type> + <v>ConfigDB = config_db()</v> + <v>RequestURI = string()</v> + <v>Aliases = [{FakeName,RealName}]</v> + <v>Ret = {ShortPath,Path,AfterPath}</v> + <v>ShortPath = Path = AfterPath = string()</v> + </type> + <desc> + <marker id="real_name"></marker> + <p><c>real_name/3</c> traverses <c>Aliases</c>, typically + extracted from <c>ConfigDB</c>, and matches each + <c>FakeName</c> with <c>RequestURI</c>. If a match is found + <c>FakeName</c> is replaced with <c>RealName</c> in the + match. The resulting path is split into two parts, that + is <c>ShortPath</c> and <c>AfterPath</c> as defined in <seealso marker="httpd_util#split_path">httpd_util:split_path/1</seealso>. + <c>Path</c> is generated from <c>ShortPath</c>, that is + the result from <seealso marker="#default_index">default_index/2</seealso> with + <c>ShortPath</c> as an argument. + <c>config_db()</c> is the server config file in ETS table + format as described in <seealso marker="http_server">Inets User Guide.</seealso>. </p> + </desc> + </func> + <func> + <name>real_script_name(ConfigDB,RequestURI,ScriptAliases) -> Ret</name> + <fsummary>Expand a request uri using ScriptAlias config directives.</fsummary> + <type> + <v>ConfigDB = config_db()</v> + <v>RequestURI = string()</v> + <v>ScriptAliases = [{FakeName,RealName}]</v> + <v>Ret = {ShortPath,AfterPath} | not_a_script</v> + <v>ShortPath = AfterPath = string()</v> + </type> + <desc> + <marker id="real_script_name"></marker> + <p><c>real_name/3</c> traverses <c>ScriptAliases</c>, + typically extracted from <c>ConfigDB</c>, and matches each + <c>FakeName</c> with <c>RequestURI</c>. If a match is found + <c>FakeName</c> is replaced with <c>RealName</c> in the + match. If the resulting match is not an executable script + <c>not_a_script</c> is returned. If it is a script the + resulting script path is in two parts, that is + <c>ShortPath</c> and <c>AfterPath</c> as defined in <seealso marker="httpd_util#split_script_path">httpd_util:split_script_path/1</seealso>. + <c>config_db()</c> is the server config file in ETS table + format as described in <seealso marker="http_server">Inets Users Guide.</seealso>.</p> + </desc> + </func> + </funcs> + +</erlref> + + diff --git a/lib/inets/doc/src/mod_auth.xml b/lib/inets/doc/src/mod_auth.xml new file mode 100644 index 0000000000..f3628c8297 --- /dev/null +++ b/lib/inets/doc/src/mod_auth.xml @@ -0,0 +1,287 @@ +<?xml version="1.0" encoding="latin1" ?> +<!DOCTYPE erlref SYSTEM "erlref.dtd"> + +<erlref> + <header> + <copyright> + <year>1997</year><year>2009</year> + <holder>Ericsson AB. All Rights Reserved.</holder> + </copyright> + <legalnotice> + The contents of this file are subject to the Erlang Public License, + Version 1.1, (the "License"); you may not use this file except in + compliance with the License. You should have received a copy of the + Erlang Public License along with this software. If not, it can be + retrieved online at http://www.erlang.org/. + + Software distributed under the License is distributed on an "AS IS" + basis, WITHOUT WARRANTY OF ANY KIND, either express or implied. See + the License for the specific language governing rights and limitations + under the License. + + </legalnotice> + + <title>mod_auth</title> + <prepared>Joakim Grebenö</prepared> + <docno></docno> + <date>1997-10-14</date> + <rev>2.3</rev> + <file>mod_auth.sgml</file> + </header> + <module>mod_auth</module> + <modulesummary>User authentication using text files, dets or mnesia database.</modulesummary> + <description> + <p>This module provides for basic user authentication using + textual files, dets databases as well as mnesia databases. </p> + </description> + <funcs> + <func> + <name>add_user(UserName, Options) -> true| {error, Reason}</name> + <name>add_user(UserName, Password, UserData, Port, Dir) -> true | {error, Reason}</name> + <name>add_user(UserName, Password, UserData, Address, Port, Dir) -> true | {error, Reason}</name> + <fsummary>Add a user to the user database.</fsummary> + <type> + <v>UserName = string()</v> + <v>Options = [Option]</v> + <v>Option = {password,Password} | {userData,UserData} | {port,Port} | {addr,Address} | {dir,Directory} | {authPassword,AuthPassword}</v> + <v>Password = string()</v> + <v>UserData = term()</v> + <v>Port = integer()</v> + <v>Address = {A,B,C,D} | string() | undefined</v> + <v>Dir = string()</v> + <v>AuthPassword =string()</v> + <v>Reason = term()</v> + </type> + <desc> + <marker id="user_api"></marker> + <marker id="add_user"></marker> + <p><c>add_user/2, add_user/5</c> and <c>add_user/6</c> adds a user to the user + database. If the operation is successful, this function returns + <c>true</c>. If an error occurs, <c>{error,Reason}</c> is returned. When <c>add_user/2</c> + is called the Password, UserData Port and Dir options is mandatory.</p> + </desc> + </func> + <func> + <name>delete_user(UserName,Options) -> true | {error, Reason}</name> + <name>delete_user(UserName, Port, Dir) -> true | {error, Reason}</name> + <name>delete_user(UserName, Address, Port, Dir) -> true | {error, Reason}</name> + <fsummary>Delete a user from the user database.</fsummary> + <type> + <v>UserName = string()</v> + <v>Options = [Option]</v> + <v>Option = {port,Port} | {addr,Address} | {dir,Directory} | {authPassword,AuthPassword}</v> + <v>Port = integer()</v> + <v>Address = {A,B,C,D} | string() | undefined</v> + <v>Dir = string()</v> + <v>AuthPassword = string()</v> + <v>Reason = term()</v> + </type> + <desc> + <marker id="delete_user"></marker> + <p><c>delete_user/2, delete_user/3</c> and <c>delete_user/4</c> + deletes a user + from the user database. If the operation is succesfull, this + function returns <c>true</c>. If an error occurs, + <c>{error,Reason}</c> is returned. When <c>delete_user/2</c> is + called the Port and Dir options are mandatory.</p> + </desc> + </func> + <func> + <name>get_user(UserName,Options) -> {ok, #httpd_user} |{error, Reason}</name> + <name>get_user(UserName, Port, Dir) -> {ok, #httpd_user} | {error, Reason}</name> + <name>get_user(UserName, Address, Port, Dir) -> {ok, #httpd_user} | {error, Reason}</name> + <fsummary>Returns a user from the user database.</fsummary> + <type> + <v>UserName = string()</v> + <v>Options = [Option]</v> + <v>Option = {port,Port} | {addr,Address} | {dir,Directory} | {authPassword,AuthPassword}</v> + <v>Port = integer()</v> + <v>Address = {A,B,C,D} | string() | undefined</v> + <v>Dir = string()</v> + <v>AuthPassword = string()</v> + <v>Reason = term()</v> + </type> + <desc> + <marker id="get_user"></marker> + <p><c>get_user/2, get_user/3</c> and <c>get_user/4</c> returns a + <c>httpd_user</c> record containing the userdata for a + specific user. If the user cannot be found, <c>{error, Reason}</c> + is returned. When <c>get_user/2</c> is called the Port and Dir + options are mandatory.</p> + </desc> + </func> + <func> + <name>list_users(Options) -> {ok, Users} | {error, Reason} <name>list_users(Port, Dir) -> {ok, Users} | {error, Reason}</name> + <name>list_users(Address, Port, Dir) -> {ok, Users} | {error, Reason}</name> + <fsummary>List users in the user database.</fsummary> + <type> + <v>Options = [Option]</v> + <v>Option = {port,Port} | {addr,Address} | {dir,Directory} | {authPassword,AuthPassword}</v> + <v>Port = integer()</v> + <v>Address = {A,B,C,D} | string() | undefined</v> + <v>Dir = string()</v> + <v>Users = list()</v> + <v>AuthPassword = string()</v> + <v>Reason = atom()</v> + </type> + <desc> + <marker id="list_users"></marker> + <p><c>list_users/1, list_users/2</c> and <c>list_users/3</c> returns a list + of users in the user database for a specific <c>Port/Dir</c>. + When <c>list_users/1</c> is called the Port and Dir + options are mandatory.</p> + </desc> + </func> + <func> + <name>add_group_member(GroupName, UserName, Options) -> true | {error, Reason}</name> + <name>add_group_member(GroupName, UserName, Port, Dir) -> true | {error, Reason}</name> + <name>add_group_member(GroupName, UserName, Address, Port, Dir) -> true | {error, Reason}</name> + <fsummary>Add a user to a group.</fsummary> + <type> + <v>GroupName = string()</v> + <v>UserName = string()</v> + <v>Options = [Option]</v> + <v>Option = {port,Port} | {addr,Address} | {dir,Directory} | {authPassword,AuthPassword}</v> + <v>Port = integer()</v> + <v>Address = {A,B,C,D} | string() | undefined</v> + <v>Dir = string()</v> + <v>AuthPassword = string()</v> + <v>Reason = term()</v> + </type> + <desc> + <marker id="add_group_member"></marker> + <p><c>add_group_member/3, add_group_member/4</c> and <c>add_group_member/5</c> + adds a user to a group. If the group does not exist, it + is created and the user is added to the group. Upon successful + operation, this function returns <c>true</c>. When <c>add_group_members/3</c> + is called the Port and Dir options are mandatory.</p> + </desc> + </func> + <func> + <name>delete_group_member(GroupName, UserName, Options) -> true | {error, Reason}</name> + <name>delete_group_member(GroupName, UserName, Port, Dir) -> true | {error, Reason}</name> + <name>delete_group_member(GroupName, UserName, Address, Port, Dir) -> true | {error, Reason}</name> + <fsummary>Remove a user from a group.</fsummary> + <type> + <v>GroupName = string()</v> + <v>UserName = string()</v> + <v>Options = [Option]</v> + <v>Option = {port,Port} | {addr,Address} | {dir,Directory} | {authPassword,AuthPassword}</v> + <v>Port = integer()</v> + <v>Address = {A,B,C,D} | string() | undefined</v> + <v>Dir = string()</v> + <v>AuthPassword = string()</v> + <v>Reason = term()</v> + </type> + <desc> + <marker id="delete_group_member"></marker> + <p><c>delete_group_member/3, delete_group_member/4</c> and <c>delete_group_member/5</c> deletes a user from a group. + If the group or the user does not exist, + this function returns an error, otherwise it returns <c>true</c>. + When <c>delete_group_member/3</c> is called the Port and Dir options + are mandatory.</p> + </desc> + </func> + <func> + <name>list_group_members(GroupName, Options) -> {ok, Users} | {error, Reason}</name> + <name>list_group_members(GroupName, Port, Dir) -> {ok, Users} | {error, Reason}</name> + <name>list_group_members(GroupName, Address, Port, Dir) -> {ok, Users} | {error, Reason}</name> + <fsummary>List the members of a group.</fsummary> + <type> + <v>GroupName = string()</v> + <v>Options = [Option]</v> + <v>Option = {port,Port} | {addr,Address} | {dir,Directory} | {authPassword,AuthPassword}</v> + <v>Port = integer()</v> + <v>Address = {A,B,C,D} | string() | undefined</v> + <v>Dir = string()</v> + <v>Users = list()</v> + <v>AuthPassword = string()</v> + <v>Reason = term()</v> + </type> + <desc> + <marker id="list_group_members"></marker> + <p><c>list_group_members/2, list_group_members/3</c> and <c>list_group_members/4</c> + lists the members of a specified group. If the group does not + exist or there is an error, <c>{error, Reason}</c> is returned. + When <c>list_group_members/2</c> is called the Port and Dir options + are mandatory.</p> + </desc> + </func> + <func> + <name>list_groups(Options) -> {ok, Groups} | {error, Reason}</name> + <name>list_groups(Port, Dir) -> {ok, Groups} | {error, Reason}</name> + <name>list_groups(Address, Port, Dir) -> {ok, Groups} | {error, Reason}</name> + <fsummary>List all the groups.</fsummary> + <type> + <v>Options = [Option]</v> + <v>Option = {port,Port} | {addr,Address} | {dir,Directory} | {authPassword,AuthPassword}</v> + <v>Port = integer()</v> + <v>Address = {A,B,C,D} | string() | undefined</v> + <v>Dir = string()</v> + <v>Groups = list()</v> + <v>AuthPassword = string()</v> + <v>Reason = term()</v> + </type> + <desc> + <marker id="list_groups"></marker> + <p><c>list_groups/1, list_groups/2</c> and <c>list_groups/3</c> lists all + the groups available. If there is an error, <c>{error, Reason}</c> + is returned. When <c>list_groups/1</c> is called the Port and Dir options + are mandatory.</p> + </desc> + </func> + <func> + <name>delete_group(GroupName, Options) -> true | {error,Reason} <name>delete_group(GroupName, Port, Dir) -> true | {error, Reason}</name> + <name>delete_group(GroupName, Address, Port, Dir) -> true | {error, Reason}</name> + <fsummary>Deletes a group</fsummary> + <type> + <v>Options = [Option]</v> + <v>Option = {port,Port} | {addr,Address} | {dir,Directory} | {authPassword,AuthPassword}</v> + <v>Port = integer()</v> + <v>Address = {A,B,C,D} | string() | undefined</v> + <v>Dir = string()</v> + <v>GroupName = string()</v> + <v>AuthPassword = string()</v> + <v>Reason = term()</v> + </type> + <desc> + <marker id="delete_group"></marker> + <p><c>delete_group/2, delete_group/3</c> and <c>delete_group/4</c> deletes the + group specified and returns <c>true</c>. If there is an error, + <c>{error, Reason}</c> is returned. When <c>delete_group/2</c> is called the + Port and Dir options are mandatory.</p> + </desc> + </func> + <func> + <name>update_password(Port, Dir, OldPassword, NewPassword, NewPassword) -> ok | {error, Reason}</name> + <name>update_password(Address,Port, Dir, OldPassword, NewPassword, NewPassword) -> ok | {error, Reason}</name> + <fsummary>Change the AuthAcessPassword</fsummary> + <type> + <v>Port = integer()</v> + <v>Address = {A,B,C,D} | string() | undefined</v> + <v>Dir = string()</v> + <v>GroupName = string()</v> + <v>OldPassword = string()</v> + <v>NewPassword = string()</v> + <v>Reason = term()</v> + </type> + <desc> + <marker id="update_password"></marker> + <p><c>update_password/5</c> and <c>update_password/6</c> Updates the AuthAccessPassword + for the specified directory. If NewPassword is equal to "NoPassword" no password is requires to + change authorisation data. If NewPassword is equal to "DummyPassword" no changes can be done + without changing the password first.</p> + </desc> + </func> + </funcs> + + <section> + <marker id="see_also"></marker> + <title>SEE ALSO</title> + <p><seealso marker="httpd">httpd(3)</seealso>, + <seealso marker="mod_alias">mod_alias(3)</seealso>,</p> + </section> + +</erlref> + + diff --git a/lib/inets/doc/src/mod_esi.xml b/lib/inets/doc/src/mod_esi.xml new file mode 100644 index 0000000000..d4541a1d15 --- /dev/null +++ b/lib/inets/doc/src/mod_esi.xml @@ -0,0 +1,123 @@ +<?xml version="1.0" encoding="latin1" ?> +<!DOCTYPE erlref SYSTEM "erlref.dtd"> + +<erlref> + <header> + <copyright> + <year>1997</year><year>2009</year> + <holder>Ericsson AB. All Rights Reserved.</holder> + </copyright> + <legalnotice> + The contents of this file are subject to the Erlang Public License, + Version 1.1, (the "License"); you may not use this file except in + compliance with the License. You should have received a copy of the + Erlang Public License along with this software. If not, it can be + retrieved online at http://www.erlang.org/. + + Software distributed under the License is distributed on an "AS IS" + basis, WITHOUT WARRANTY OF ANY KIND, either express or implied. See + the License for the specific language governing rights and limitations + under the License. + + </legalnotice> + + <title>mod_esi</title> + <prepared>Joakim Grebenö</prepared> + <docno></docno> + <date>1997-10-14</date> + <rev>2.2</rev> + <file>mod_esi.sgml</file> + </header> + <module>mod_esi</module> + <modulesummary>Erlang Server Interface </modulesummary> + <description> + <p>This module defines the API - Erlang Server Interface (ESI). + Which is a more efficient way of writing erlang scripts + for your Inets web server than writing them as common CGI scripts.</p> + </description> + <funcs> + <func> + <name>deliver(SessionID, Data) -> ok | {error, Reason}</name> + <fsummary>Sends Data back to client.</fsummary> + <type> + <v>SessionID = term()</v> + <v>Data = string() | io_list()</v> + <v>Reason = term()</v> + </type> + <desc> + <marker id="deliver"></marker> + <p>This function is <em>only</em> intended to be used from + functions called by the Erl Scheme interface to deliver + parts of the content to the user.</p> + <p>Sends data from a Erl Scheme script back to the client.</p> + + <note><p>Note + that if any HTTP-header fields should be added by the + script they must be in the first call to deliver/2 and the + data in the call must be a string. Do not + assume anything about the data type of SessionID, the + SessionID must be the value given as input to the esi + call back function that you implemented.</p></note> + </desc> + </func> + </funcs> + + <section> + <title>ESI Callback Functions</title> + </section> + <funcs> + <func> + <name>Module:Function(SessionID, Env, Input)-> _ </name> + <fsummary>Creates a dynamic web page and returns it chunk by chunk to the server process by calling mod_esi:deliver/2.</fsummary> + <type> + <v>SessionID = term()</v> + <v>Env = [EnvironmentDirectives] ++ ParsedHeader</v> + <v>EnvironmentDirectives = {Key,Value}</v> + <v>Key = query_string | content_length | server_software | gateway_interface | server_protocol | server_port | request_method | remote_addr | script_name. <v>Input = string()</v> + </type> + <desc> + <p>The <c>Module</c> must be found in the code path and export + <c>Function</c> with an arity of two. An erlScriptAlias must + also be set up in the configuration file for the Web server.</p> + <p>If the HTTP request is a post request and a body is sent + then content_length will be the length of the posted + data. If get is used query_string will be the data after + <em>?</em> in the url.</p> + <p>ParsedHeader is the HTTP request as a key value tuple + list. The keys in parsed header will be the in lower case.</p> + <p>SessionID is a identifier + the server use when <c>deliver/2</c> is called, do not + assume any-thing about the datatype.</p> + <p>Use this callback function to dynamically generate dynamic web + content. when a part of the page is generated send the + data back to the client through <c>deliver/2</c>. Note + that the first chunk of data sent to the client must at + least contain all HTTP header fields that the response + will generate. If the first chunk not contains + <em>End of HTTP header</em> that is <c>"\\r\ \\r\ "</c> + the server will + assume that no HTTP header fields will be generated.</p> + </desc> + </func> + <func> + <name>Module:Function(Env, Input)-> Response </name> + <fsummary>Creates a dynamic web page and return it as a list. This functions is deprecated and only kept for backwards compatibility.</fsummary> + <type> + <v>Env = [EnvironmentDirectives] ++ ParsedHeader</v> + <v>EnvironmentDirectives = {Key,Value}</v> + <v>Key = query_string | content_length | server_software | gateway_interface | server_protocol | server_port | request_method | remote_addr | script_name. <v>Input = string()</v> + <v>Response = string()</v> + </type> + <desc> + <p>This callback format consumes quite much memory since the + whole response must be generated before it is sent to the + user. This functions is deprecated and only keept for backwards + compatibility. + For new development Module:Function/3 should be used.</p> + </desc> + </func> + </funcs> + +</erlref> + + diff --git a/lib/inets/doc/src/mod_security.xml b/lib/inets/doc/src/mod_security.xml new file mode 100644 index 0000000000..5f9f88071e --- /dev/null +++ b/lib/inets/doc/src/mod_security.xml @@ -0,0 +1,158 @@ +<?xml version="1.0" encoding="latin1" ?> +<!DOCTYPE erlref SYSTEM "erlref.dtd"> + +<erlref> + <header> + <copyright> + <year>1998</year><year>2009</year> + <holder>Ericsson AB. All Rights Reserved.</holder> + </copyright> + <legalnotice> + The contents of this file are subject to the Erlang Public License, + Version 1.1, (the "License"); you may not use this file except in + compliance with the License. You should have received a copy of the + Erlang Public License along with this software. If not, it can be + retrieved online at http://www.erlang.org/. + + Software distributed under the License is distributed on an "AS IS" + basis, WITHOUT WARRANTY OF ANY KIND, either express or implied. See + the License for the specific language governing rights and limitations + under the License. + + </legalnotice> + + <title>mod_security</title> + <prepared>Mattias Nilsson</prepared> + <docno></docno> + <date>1997-11-18</date> + <rev>1.0</rev> + <file>mod_security.sgml</file> + </header> + <module>mod_security</module> + <modulesummary>Security Audit and Trailing Functionality</modulesummary> + <description> + <p>Security Audit and Trailing Functionality</p> + </description> + <funcs> + <func> + <name>list_auth_users(Port) -> Users | []</name> + <name>list_auth_users(Address, Port) -> Users | []</name> + <name>list_auth_users(Port, Dir) -> Users | []</name> + <name>list_auth_users(Address, Port, Dir) -> Users | []</name> + <fsummary>List users that have authenticated within the SecurityAuthTimeout time for a given address (if specified), port number and directory (if specified).</fsummary> + <type> + <v>Port = integer()</v> + <v>Address = {A,B,C,D} | string() | undefined</v> + <v>Dir = string()</v> + <v>Users = list() = [string()]</v> + </type> + <desc> + <marker id="list_auth_users"></marker> + <p><c>list_auth_users/1</c>, <c>list_auth_users/2</c> and + <c>list_auth_users/3</c> returns a list of users that are + currently authenticated. Authentications are stored for + SecurityAuthTimeout seconds, and are then discarded.</p> + </desc> + </func> + <func> + <name>list_blocked_users(Port) -> Users | []</name> + <name>list_blocked_users(Address, Port) -> Users | []</name> + <name>list_blocked_users(Port, Dir) -> Users | []</name> + <name>list_blocked_users(Address, Port, Dir) -> Users | []</name> + <fsummary>List users that are currently blocked from access to a specified port number, for a given address (if specified).</fsummary> + <type> + <v>Port = integer()</v> + <v>Address = {A,B,C,D} | string() | undefined</v> + <v>Dir = string()</v> + <v>Users = list() = [string()]</v> + </type> + <desc> + <marker id="list_blocked_users"></marker> + <p><c>list_blocked_users/1</c>, <c>list_blocked_users/2</c> and + <c>list_blocked_users/3</c> returns a list of users that are + currently blocked from access.</p> + </desc> + </func> + <func> + <name>block_user(User, Port, Dir, Seconds) -> true | {error, Reason}</name> + <name>block_user(User, Address, Port, Dir, Seconds) -> true | {error, Reason}</name> + <fsummary>Block user from access to a directory for a certain amount of time.</fsummary> + <type> + <v>User = string()</v> + <v>Port = integer()</v> + <v>Address = {A,B,C,D} | string() | undefined</v> + <v>Dir = string()</v> + <v>Seconds = integer() | infinity</v> + <v>Reason = no_such_directory</v> + </type> + <desc> + <marker id="block_user"></marker> + <p><c>block_user/4</c> and <c>block_user/5</c> blocks the user + <c>User</c> from the directory <c>Dir</c> for a specified + amount of time.</p> + </desc> + </func> + <func> + <name>unblock_user(User, Port) -> true | {error, Reason}</name> + <name>unblock_user(User, Address, Port) -> true | {error, Reason}</name> + <name>unblock_user(User, Port, Dir) -> true | {error, Reason}</name> + <name>unblock_user(User, Address, Port, Dir) -> true | {error, Reason}</name> + <fsummary>Remove a blocked user from the block list</fsummary> + <type> + <v>User = string()</v> + <v>Port = integer()</v> + <v>Address = {A,B,C,D} | string() | undefined</v> + <v>Dir = string()</v> + <v>Reason = term()</v> + </type> + <desc> + <marker id="unblock_user"></marker> + <p><c>unblock_user/2</c>, <c>unblock_user/3</c> and + <c>unblock_user/4</c> removes the user <c>User</c> from + the list of blocked users for the Port (and Dir) specified.</p> + </desc> + </func> + </funcs> + + <section> + <marker id="callback_module"></marker> + <title>The SecurityCallbackModule</title> + <p>The SecurityCallbackModule is a user written module that can receive events from + the mod_security Erlang Webserver API module. This module only exports one function, + <seealso marker="#callback_module_event">event/4</seealso>, which is described below. + </p> + </section> + <funcs> + <func> + <name>event(What, Port, Dir, Data) -> ignored</name> + <name>event(What, Address, Port, Dir, Data) -> ignored</name> + <fsummary>This function is called whenever an event occurs in mod_security</fsummary> + <type> + <v>What = atom()</v> + <v>Port = integer()</v> + <v>Address = {A,B,C,D} | string() <v>Dir = string()</v> + <v>What = [Info]</v> + <v>Info = {Name, Value}</v> + </type> + <desc> + <marker id="callback_module_event"></marker> + <p><c>event/4</c> or <c>event/4</c> is called whenever an event + occurs in the mod_security Erlang Webserver API module (<c>event/4</c> is + called if Address is undefined and <c>event/5</c> otherwise). + The <c>What</c> argument specifies the type of event that has + occurred, and should be one of the following reasons; + <c>auth_fail</c> (a failed user authentication), + <c>user_block</c> (a user is being blocked from access) or + <c>user_unblock</c> (a user is being removed from the block list).</p> + <note> + <p>Note that the <c>user_unblock</c> event is not triggered when + a user is removed from the block list explicitly using the + <c>unblock_user</c> function.</p> + </note> + </desc> + </func> + </funcs> + +</erlref> + + diff --git a/lib/inets/doc/src/note.gif b/lib/inets/doc/src/note.gif Binary files differnew file mode 100644 index 0000000000..6fffe30419 --- /dev/null +++ b/lib/inets/doc/src/note.gif diff --git a/lib/inets/doc/src/notes.gif b/lib/inets/doc/src/notes.gif Binary files differnew file mode 100644 index 0000000000..e000cca26a --- /dev/null +++ b/lib/inets/doc/src/notes.gif diff --git a/lib/inets/doc/src/notes.xml b/lib/inets/doc/src/notes.xml new file mode 100644 index 0000000000..489e88cbe5 --- /dev/null +++ b/lib/inets/doc/src/notes.xml @@ -0,0 +1,929 @@ +<?xml version="1.0" encoding="latin1" ?> +<!DOCTYPE chapter SYSTEM "chapter.dtd"> + +<chapter> + <header> + <copyright> + <year>2002</year><year>2009</year> + <holder>Ericsson AB. All Rights Reserved.</holder> + </copyright> + <legalnotice> + The contents of this file are subject to the Erlang Public License, + Version 1.1, (the "License"); you may not use this file except in + compliance with the License. You should have received a copy of the + Erlang Public License along with this software. If not, it can be + retrieved online at http://www.erlang.org/. + + Software distributed under the License is distributed on an "AS IS" + basis, WITHOUT WARRANTY OF ANY KIND, either express or implied. See + the License for the specific language governing rights and limitations + under the License. + + </legalnotice> + + <title>Inets Release Notes</title> + <prepared></prepared> + <responsible></responsible> + <docno></docno> + <approved></approved> + <checked></checked> + <date>2002-02-28</date> + <rev>A</rev> + <file>notes.xml</file> + </header> + + <section><title>Inets 5.2</title> + + <section><title>Improvements and New Features</title> +<!-- + <p>-</p> +--> + <list> + <item> + <p>[ftpc] - Start of the FTP client has been changed in + the following way: </p> + <list type="bulleted"> + <item> + <p>It is now also possible to start a standalone FTP client + process using the re-introduced + <seealso marker="ftp#open">ftp:open</seealso> + function. </p> + <p>This is an alternative to starting the client using the + <seealso marker="ftp#service_start">inets service framework</seealso>. </p> + <p>The old <c>ftp:open/1</c>, undocumented, function, + caused the client to be hooken into the inets service + supervision framework. This is <em>no</em> longer the + case. </p> + <p>*** POTENTIAL INCOMPATIBILITY ***</p> + </item> + <item> + <p>Previously, the FTP client attempted to use IPv6, + unless otherwise instructed (the <c>ip_v6_disabled</c> + flag), and only used IPv4 if this did not work. + This has now been <em>changed</em>. </p> + <p>A new option, + <seealso marker="ftp#ipfamily">ipfamily</seealso>, + has been introduced, with the default value + <c>inet</c> (IPv4). </p> + <p>See <seealso marker="ftp#open">ftp:open</seealso> + for more info.</p> + <p>*** POTENTIAL INCOMPATIBILITY ***</p> + </item> + </list> + <p>Own Id: OTP-8258</p> + <!-- <p>Aux Id: seq11407</p> --> + </item> + + <item> + <p>The documentation is now built with open source tools + (<em>xsltproc</em> and <em>fop</em>) that exists on most + platforms. One visible change is that the frames are removed.</p> + <p>Own Id: OTP-8249</p> + </item> + + </list> + </section> + + <section><title>Fixed Bugs and Malfunctions</title> + +<!-- + <p>-</p> +--> + + <list> + <item> + <p>[httpc] - Streaming to file did not work.</p> + <p>[email protected]</p> + <p>Own Id: OTP-8204</p> + </item> + + <item> + <p>[ftpc] - The + <seealso marker="ftp#ls2">ls/2</seealso> function (LIST command) + and the + <seealso marker="ftp#nlist2">nlist/2</seealso> function + (NLST command) + with wildcards did + not work properly. </p> + <p>These functions is documented as working on directories, but + this is actually not according the standard. The LIST and NLST + commands are specified to operate on a directory or other + group of files, or a file.</p> + <p>Previously, an attempt was made to check if the listing + returned by the server was actually an error message. This + was done by changing remote directory (cd) into the + (assumed) "directory". This may work if Pathname was actually + a directory, but as this is not always the case, this test + does not work. Instead, we now return the actual server + result and leave the interpretation to the caller. </p> + <p>*** POTENTIAL INCOMPATIBILITY ***</p> + <p>Own Id: OTP-8247</p> + <p>Aux Id: seq11407</p> + </item> + + <item> + <p>[httpc] - Fixes various bugs in timeout and keep-alive queue + handling. </p> + <list type="bulleted"> + <item> + <p>When a queued request times, out the error mssage is sent + the owner of the active request. </p> + </item> + + <item> + <p>Requests in the keep-alive queue is forgotten when + handler terminates. </p> + </item> + + <item> + <p>Timeout out requests are retried. </p> + </item> + </list> + <p>Jean-S�bastien P�dron</p> + <p>Own Id: OTP-8248</p> + </item> + + <item> + <p>[httpd] - Unnecessarily strict matching when handling closing sockets. </p> + <p>Own Id: OTP-8280</p> + </item> + + </list> + </section> + + </section> <!-- 5.2 --> + + + <section><title>Inets 5.1.3</title> + + <section><title>Improvements and New Features</title> + <p>-</p> +<!-- + <list> + <item> + <p>[httpc] Added support for web services using only basic auth, + with a token as the user part and no password part.</p> + <p>[email protected]</p> + <p>Own Id: OTP-7998</p> + </item> + + </list> +--> + </section> + + <section><title>Fixed Bugs and Malfunctions</title> + +<!-- + <p>-</p> +--> + + <list> + <item> + <p>[httpc] - Raise condition. + When http:request is called and httpc_manager selects a session + where there's already a pending request, then the connection + handler for that session effectively resets its parser, readying + it for the response to the second request. But if there are still + some inbound packets for the response to the first request, things + get confused.</p> + <p>[email protected]</p> + <p>Own Id: OTP-8154</p> + </item> + + </list> + </section> + + </section> <!-- 5.1.3 --> + + + <section><title>Inets 5.1.2</title> +<!-- + <p>-</p> +--> + + <section><title>Improvements and New Features</title> +<!-- + <p>-</p> +--> + + <list> + <item> + <p>[httpc] - Added http option <c>connect_timeout</c> for http + client request. + The <c>connect_timeout</c> option is used for the initial + request, when the client connects to the server. Default + value is that of the <c>timeout</c> option. </p> + <p>See the + <seealso marker="http#request2">request/4,5</seealso> + function for more info. </p> + <p>Own Id: OTP-7298</p> + <!-- <p>Aux Id: seq11086</p> --> + </item> + + </list> + </section> + + + <section><title>Fixed Bugs and Malfunctions</title> +<!-- + <p>-</p> +--> + + <list> + <item> + <p>[httpd] - Failed to create listen socket with invalid + option combo. The http-server failed to create its listen + socket when the bind-address was an IPv4-address (a tuple of + size 4) and the ipfamily option was inet6fb4. </p> + <p>Own Id: OTP-8118</p> + <p>Aux Id: seq11321</p> + </item> + + <item> + <p>[httpd] - Removed documentation for non-existing function + (httpd_util:header/2,3,4). </p> + <p>Own Id: OTP-8101</p> + <!-- <p>Aux Id: seq11321</p> --> + </item> + + </list> + </section> + + </section> <!-- 5.1.2 --> + + + <section><title>Inets 5.1.1</title> +<!-- + <p>-</p> +--> + + <section><title>Improvements and New Features</title> + <list> + <item> + <p>[httpd] - When starting inets (the web-server) and supplying + a descriptor on the command line + (example: erl -httpd_8888 <descriptor>) + it is now possible to specify which ip-family to use: + <c>inet | inet6 | inet6fb4</c>. </p> + <p>Example: erl -httpd_8888 10|inet6</p> + <p>When starting the web-server either using a file with + property list (the proplist_file) or a an property list, + using the ipfamily option: + <c>{ipfamily, inet | inet6 | inet6fb4}</c>. </p> + <p>Finally, when starting the web-server using the classical + apache-style config file, the <c>BindAddress</c> directive + has been augmented to allow the specification of the + IpFamily: <c>BindAddress blirk.ericsson.se|inet</c></p> + <p>Default is <c>inet6fb4</c> which emulates the + behaviour of the previous version. </p> + <p>See the + <seealso marker="httpd#comm_prop">Communication properties</seealso> + section for more info. </p> + <p>Own Id: OTP-8069</p> + <p>Aux Id: seq11086</p> + </item> + + </list> + </section> + + + <section><title>Fixed Bugs and Malfunctions</title> +<!-- + <p>-</p> +--> + + <list> + <item> + <p>[httpc] - Reception of unexpected data causes handler crash. </p> + <p>Own Id: OTP-8052</p> + </item> + + </list> + </section> + + </section> <!-- 5.1.1 --> + + + <section><title>Inets 5.1</title> + + <section><title>Improvements and New Features</title> + <list> + <item> + <p>[httpc] Added support for web services using only basic auth, + with a token as the user part and no password part.</p> + <p>[email protected]</p> + <p>Own Id: OTP-7998</p> + </item> + + <item> + <p>[httpc] - Bind HTTP client to IP-addr. It is now possible + to specify an alternate ip-address and port to be used when + the client connects to the server. </p> + <p>As a side-effect of this, the option <c>ipv6</c> has been + removed and replaced by the <c>ipfamily</c> option. </p> + <p>See <seealso marker="http#set_options">http:set_options/1,2</seealso> + for more info. </p> + <p>*** POTENTIAL INCOMPATIBILITY ***</p> + <p>Own Id: OTP-8004</p> + </item> + + </list> + </section> + + + <section><title>Fixed Bugs and Malfunctions</title> +<!-- + <p>-</p> +--> + + <list> + <item> + <p>Updated guard tests (i.e. is_list(L) instead of + list(L) and possibly andalso/orelse instead of ","/";"). </p> + <p>Own Id: OTP-7994</p> + </item> + + <item> + <p>[httpc] - Remove use of the deprecated regexp module. </p> + <p>Own Id: OTP-8001</p> + </item> + + <item> + <p>[httpc] - The option <c>max_keep_alive_length</c> was + not handled properly. </p> + <p>Own Id: OTP-8005</p> + </item> + + + </list> + </section> + + </section> <!-- 5.1 --> + + + <section><title>Inets 5.0.14</title> + + <section><title>Improvements and New Features</title> + <list> + <item> + <p> + [tftp] The callback watchdog has been removed, as it + turned out to be counter productive when the disk was + overloaded. Earlier a connection was aborted when a + callback (which performs the file access in the TFTP + server) took too long time.</p> + <p> + [tftp] The error message "Too many connections" has been + reclassified to be a warning.</p> + <p> + Own Id: OTP-7888</p> + </item> + </list> + </section> + + + <section><title>Fixed Bugs and Malfunctions</title> + <list> + <item> + <p>[httpc] - Incorrect http version option check. </p> + <p>Mats Cronqvist</p> + <p>Own Id: OTP-7882</p> + </item> + + <item> + <p>[httpc] - Unnecessary error report when client + terminating as a result of the server closed the + socket unexpectedly. </p> + <p>Own Id: OTP-7883</p> + </item> + + <item> + <p>[httpc] - Failed transforming a relative URI to + an absolute URI. </p> + <p>[email protected]</p> + <p>Own Id: OTP-7950</p> + </item> + + <item> + <p>[httpd] - The HTTP server did not handle the config + option ssl_ca_certificate_file. </p> + <p>[email protected]</p> + <p>Own Id: OTP-7976</p> + </item> + + </list> + </section> + + </section> <!-- 5.0.14 --> + + + <section><title>Inets 5.0.13</title> + + <section><title>Fixed Bugs and Malfunctions</title> + <list> + <item> + <p> + Ssl did not work correctly with the use of new style + configuration due to sn old internal format that was not + changed correctly in all places.</p> + <p> + Own Id: OTP-7723 Aux Id: seq11143 </p> + </item> + <item> + <p> + [httpc] - Now streams 200 and 206 results and not only + 200 results.</p> + <p> + Own Id: OTP-7857</p> + </item> + </list> + </section> + + + <section><title>Improvements and New Features</title> + <list> + <item> + <p> + [httpc] - The inets http client will now use persistent + connections without pipelining as default and if a + pipeline timeout is set it will pipeline the requests on + the persistent connections.</p> + <p> + *** POTENTIAL INCOMPATIBILITY ***</p> + <p> + Own Id: OTP-7463</p> + </item> + <item> + <p> + [httpd] - added option ssl_password_callback_arguments.</p> + <p> + Own Id: OTP-7724 Aux Id: seq11151 </p> + </item> + <item> + <p> + Changed the socket use so that it will become more robust + to non-functional ipv6 and fallback on ipv4. This changes + may for very special os-configurations cause a problem + when used with erts-versions pre R13.</p> + <p> + Own Id: OTP-7726</p> + </item> + <item> + <p> + Removed deprecated function httpd_util:key1search/[2,3]</p> + <p> + Own Id: OTP-7815</p> + </item> + </list> + </section> + + </section> + + <section><title>Inets 5.0.12</title> + + <section><title>Improvements and New Features</title> + <list> + <item> + <p> + [httpd] - Updated inets so that it not uses the deprecated + function ssl:accept/[2,3].</p> + <p> + Own Id: OTP-7636 Aux Id: seq11086 </p> + </item> + </list> + </section> + + </section> + + + <section><title>Inets 5.0.11</title> + + <section><title>Fixed Bugs and Malfunctions</title> + <list> + <item> + <p> + Transient bug related to hot code swap of the TFTP server is + now fixed. It could happen that the first TFTP server that was + started after a code upgrade to Inets-5.0.6 crashed with a + function clause error in tftp_engine:service_init/2.</p> + <p> Own Id: OTP-7574 Aux Id: seq11069 </p> + </item> + <item> + <p> + [httpd] - Validation of ssl_password_callback_module was + incorrect.</p> + <p> + Own Id: OTP-7597 Aux Id: seq11074 </p> + </item> + <item> + <p> + [httpd] - Misspelling in old apachelike configuration + directive TransferDiskLogSize has been corrected.</p> + <p> Own Id: OTP-7598 Aux Id: seq11059 </p> + </item> + <item> + <p> + Minor problems found by dialyzer has been fixed.</p> + <p> + Own Id: OTP-7605</p> + </item> + </list> + </section> + + </section> + +<section><title>Inets 5.0.10</title> + + <section><title>Fixed Bugs and Malfunctions</title> + <list> + <item> + <p> + Enhanched an info report.</p> + <p> + Own Id: OTP-7450</p> + </item> + </list> + </section> + + + <section><title>Improvements and New Features</title> + <list> + <item> + <p> + Changed errro message from + {wrong_type,{document_root,"/tmp/htdocs"}} to + {invalid_option,{non_existing, + document_root,"/tmp/htdocs"}}.</p> + <p> + Own Id: OTP-7454</p> + </item> + <item> + <p> + Relative paths in directory authentication did not work + as intended, this has now been fixed.</p> + <p> + Own Id: OTP-7490</p> + </item> + <item> + <p> + The query-string passed to the callback function was not + compliant with the documentation, it is now.</p> + <p> + Own Id: OTP-7512</p> + </item> + </list> + </section> + +</section> + + <section><title>Inets 5.0.9</title> + + <section><title>Fixed Bugs and Malfunctions</title> + <list> + <item> + <p> + Parameters to error_logger:error_report/1 has been + corrected.</p> + <p> + Own Id: OTP-7257 Aux Id: OTP-7294, OTP-7258 </p> + </item> + <item> + <p> + [httpd] - If a Module/Function request matching an + erl_script_alias registration does not exist as a function in + the module registered a 404 error will now be issued instead of a + 500 error.</p> + <p> + Own Id: OTP-7323</p> + </item> + <item> + <p> + [httpd] -The option auth_type for mod_auth is no longer + mandatory, for backward-compatibility reasons.</p> + <p> + Own Id: OTP-7341</p> + </item> + </list> + </section> + + </section> + + <section><title>Inets 5.0.8</title> + + <section><title>Fixed Bugs and Malfunctions</title> + <list> + <item> + <p> + [httpd] - Spelling error caused client connection header + to be ignored.</p> + <p> + Own Id: OTP-7315 Aux Id: seq10951 </p> + </item> + <item> + <p> + [httpd] - Call to the function + mod_get:get_modification_date/1 was made too early + resulting in that httpd did not send the 404 file missing + response.</p> + <p> + Own Id: OTP-7321</p> + </item> + </list> + </section> + + </section> + + <section><title>Inets 5.0.7</title> + + <section><title>Improvements and New Features</title> + <list> + <item> + <p> + [httpc, httpd] - Now follows the recommendation regarding + line terminators in section 19.3 in RFC 2616 e.i: "The + line terminator for message-header fields is the sequence + CRLF. However, we recommend that applications, when + parsing such headers, recognize a single LF as a line + terminator and ignore the leading CR".</p> + <p> + Own Id: OTP-7304 Aux Id: seq10944 </p> + </item> + </list> + </section> + + </section> + + <section><title>Inets 5.0.6</title> + + <section><title>Improvements and New Features</title> + <list> + <item> + <p> + [tftp] If a callback (which performs the file access in + the TFTP server) takes too long time (more than the + double TFTP timeout), the server will abort the + connection and send an error reply to the client. This + implies that the server will release resources attached + to the connection faster than before. The server simply + assumes that the client has given up.</p> + <p> + [tftp] If the TFTP server receives yet another request + from the same client (same host and port) while it + already has an active connection to the client, it will + simply ignore the new request if the request is equal + with the first one (same filename and options). This + implies that the (new) client will be served by the + already ongoing connection on the server side. By not + setting up yet another connection, in parallel with the + ongoing one, the server will consumer lesser resources.</p> + <p> + [tftp] netascii mode is now supported when the + client/server has native ascii support (Windows). The new + optional parameter native_ascii in the tftp_binary and + tftp_file callback modules can be used to override the + default behavior.</p> + <p> + [tftp] Yet another callback module has been added in + order to allow customized handling of error, warning and + info messages. See the new configuration parameter, + logger.</p> + <p> + [tftp] Yet another configuration parameter, max_retries, + has been added in order to control the number of times a + packet can be resent. The default is 5.</p> + <p> + [tftp] tftp:info/1 and tftp:change_config/2 can now be + applied to all daemons or all servers in one command + without bothering about their process identifiers.</p> + <p> + External TR HI89527.</p> + <p> + Own Id: OTP-7266</p> + </item> + </list> + </section> + +</section> + +<section><title>Inets 5.0.5</title> + + <section><title>Improvements and New Features</title> + <list> + <item> + <p> + [tftp] Blocks with too low block numbers are silently + discarded. For example if a server receives block #5 when + it expects block #7 it will discard the block without + interrupting the file transfer. Too high block numbers + does still imply an error. External TR HI96072.</p> + <p> + Own Id: OTP-7220</p> + </item> + <item> + <p> + [tftp] The problem with occasional case_clause errors in + tftp_engine:common_read/7 has been fixed. External TR + HI97362.</p> + <p> + Own Id: OTP-7221</p> + </item> + </list> + </section> + +</section> + + <section><title>Inets 5.0.4</title> + + <section><title>Improvements and New Features</title> + <list> + <item> + <p> + Changed calls to file open to concur with the API and not + use deprecated syntax.</p> + <p> + Own Id: OTP-7172</p> + </item> + <item> + <p> + [tftp] Server lost the first packet when the client timed + out</p> + <p> + Own Id: OTP-7173</p> + </item> + </list> + </section> + + </section> + + <section><title>Inets 5.0.3</title> + + <section><title>Improvements and New Features</title> + <list> + <item> + <p> + Updated copyright headers and fixed backwards + compatibility for an undocumented feature, for now. This + feature will later be removed and a new and documented + option will take its place.</p> + <p> + Own Id: OTP-7144</p> + </item> + </list> + </section> + + </section> + + <section><title>Inets 5.0.2</title> + + <section><title>Improvements and New Features</title> + <list> + <item> + <p> + [httpd] - Error logs now has a pretty and a compact + format and access logs can be written on the common log + format or the extended common log format.</p> + <p> + Own Id: OTP-6661 Aux Id: Seq 7764 </p> + </item> + <item> + <p> + [httpc] - Added acceptance of missing reason phrase to + the relaxed mode.</p> + <p> + Own Id: OTP-7024</p> + </item> + <item> + <p> + [httpc] - A new option has been added to enable the + client to act as lower version clients, by default the + client is an HTTP/1.1 client.</p> + <p> + Own Id: OTP-7043</p> + </item> + </list> + </section> + + </section> + + <section><title>Inets 5.0.1</title> + + <section><title>Fixed Bugs and Malfunctions</title> + <list> + <item> + <p> + [httpd] - Deprecated function httpd:start/1 did not + accept all inputs that it had done previously. This + should now work again.</p> + <p> + Own Id: OTP-7040</p> + </item> + </list> + </section> + + <section><title>Improvements and New Features</title> + <list> + <item> + <p> + [httpd] - Changed validity check on bind_address so that + it uses inet:getaddr instead of inet:gethostbyaddr as the + former puts a too hard restriction on the bind_address.</p> + <p> + Own Id: OTP-7041 Aux Id: seq10829 </p> + </item> + <item> + <p> + [httpc] - Internal process now does try-catch and + terminates normally in case of HTTP parse errors. + Semantical the client works just as before returning an + error message to the client, even if the error massage + has been enhanced, but there is no supervisor report in + the shell of a internal process crashing. (Which was the + expected behavior and not a fault.)</p> + <p> + Own Id: OTP-7042</p> + </item> + </list> + </section> + + </section> + + <section><title>Inets 5.0</title> + + <section><title>Improvements and New Features</title> + <list> + <item> + <p> + [httpd, httpc] - Deprecated base64 decode/encode + functions have been removed. Inets uses base64 in STDLIB + instead.</p> + <p> + *** POTENTIAL INCOMPATIBILITY ***</p> + <p> + Own Id: OTP-6485</p> + </item> + <item> + <p> + [httpd] - It is now possible to restrict the length of + acceptable URI:s in the HTTP server.</p> + <p> + Own Id: OTP-6572</p> + </item> + <item> + <p> + [httpc] - Profiles are now supported i.e. the options + available in set_options/1 can be set locally for a + certain profile and do not have to affect all + HTTP-requests issued in the Erlang node. Calls to the + HTTP client API functions not using the profile argument + will use the default profile.</p> + <p> + Own Id: OTP-6690</p> + </item> + <item> + <p> + A new uniform Inets interface provides a flexible way to + start/stop Inets services and get information about + running services. See inets(3). This also means that + inflexibilities in the HTTP server has been removed and + more default values has been added.</p> + <p> + Own Id: OTP-6705</p> + </item> + <item> + <p> + [tftp] Logged errors have been changed to be logged + warnings.</p> + <p> + Own Id: OTP-6916 Aux Id: seq10737 </p> + </item> + <item> + <p> + [httpc] - The client will now return the proper value + when receiving a HTTP 204 code instead of hanging.</p> + <p> + Own Id: OTP-6982</p> + </item> + <item> + <p> + The Inets application now has to be explicitly started + and stopped i.e. it will not automatically be started as + a temporary application as it did before. Although a + practical feature when testing things in the shell it is + not desirable that people take advantage of this and not + start the Inets application in a correct way in their + products. Added functions to the Inets API that call + application:start/stop.</p> + <p> + *** POTENTIAL INCOMPATIBILITY ***</p> + <p> + Own Id: OTP-6993</p> + </item> + </list> + </section> + + <!-- p>For information about older versions see + <url href="part_notes_history_frame.html">release notes history</url>.</p --> + </section> +</chapter> + + diff --git a/lib/inets/doc/src/notes_history.xml b/lib/inets/doc/src/notes_history.xml new file mode 100644 index 0000000000..53375c9aa7 --- /dev/null +++ b/lib/inets/doc/src/notes_history.xml @@ -0,0 +1,1826 @@ +<?xml version="1.0" encoding="latin1" ?> +<!DOCTYPE chapter SYSTEM "chapter.dtd"> + +<chapter> + <header> + <copyright> + <year>2004</year><year>2009</year> + <holder>Ericsson AB. All Rights Reserved.</holder> + </copyright> + <legalnotice> + The contents of this file are subject to the Erlang Public License, + Version 1.1, (the "License"); you may not use this file except in + compliance with the License. You should have received a copy of the + Erlang Public License along with this software. If not, it can be + retrieved online at http://www.erlang.org/. + + Software distributed under the License is distributed on an "AS IS" + basis, WITHOUT WARRANTY OF ANY KIND, either express or implied. See + the License for the specific language governing rights and limitations + under the License. + + </legalnotice> + + <title>Inets Release Notes History</title> + <prepared></prepared> + <responsible></responsible> + <docno></docno> + <approved></approved> + <checked></checked> + <date>04-09-30</date> + <rev>A</rev> + <file>notes_history.sgml</file> + </header> + + <section> + <title>Inets 4.7.17</title> + + <section> + <title>Fixed Bugs and Malfunctions</title> + <list type="bulleted"> + <item> + <p>[HTTP client] - Could fail in retry_pipeline/2 due to an + incorrect assumption of an internal returnvalue.</p> + <p>Own Id: OTP-6791</p> + </item> + <item> + <p>[HTTP client] - The check of the value of the transfer + encoding has been updated so that it is case insensitive.</p> + <p>Own Id: OTP-6807</p> + </item> + <item> + <p>[HTTP client] - When receiving a 304 "Not Modified" + reply, the httpc_handler will no longer expect to receive + a HTTP body.</p> + <p>Own Id: OTP-6821</p> + </item> + <item> + <p>[HTTP client] - Parsing of the HTTP response failed when + there was no headers fields in the response.</p> + <p>Own Id: OTP-6830</p> + </item> + </list> + </section> + + <section> + <title>Improvements and New Features</title> + <list type="bulleted"> + <item> + <p>[HTTP server] - Started clean up of code so that it uses + stdlib functions instead of reinventing them.</p> + <p>Own Id: OTP-6808</p> + </item> + </list> + </section> + + </section> + + <section> + <title>Inets 4.7.16</title> + + <section> + <title>Improvements and New Features</title> + <list type="bulleted"> + <item> + <p>Minor Makefile changes.</p> + <p>Own Id: OTP-6689</p> + </item> + <item> + <p>http_base_64:encode/decode and + httpd_util:to_upper/to_lower are now deprecated, as these + functions has been moved to stdlib, using them will now + cause an compiler warning and the documentation of them + has been removed.</p> + <p>Own Id: OTP-6716</p> + </item> + <item> + <p>When making an asynchronous HTTP request and the + underlying gen_tcp:connect failed with timeout (not a + very common case) the return of the asynchronous HTTP + request was delayed for "timeout" seconds. This happened + due to the fact that when spawning a gen_server process + the spawn will wait for the init function to complete. + This is now avoided using proc_lib and + gen_server:enter_loop/3, hence the asynchronous HTTP + request return will not be delayed. Also the request + timeout value is now propagated to gen_tcp rather that + relying on the system default.</p> + <p>Own Id: OTP-6735</p> + </item> + </list> + </section> + </section> + + <section> + <title>Inets 4.7.15</title> + + <section> + <title>Fixed Bugs and Malfunctions</title> + <list type="bulleted"> + <item> + <p>[tftp] The TFTP client/server implements now a more + relaxed interpretation of the RFC 1350 regarding + re-receive of acknowledgments. If multiple copies of the + same acknowledgments is received the spurious ones are + silently ignored. This fix was intended for inets-4.7.14 + but accidentaly it was not included in that release.</p> + <p>Own Id: OTP-6706 Aux Id: OTP-6691 </p> + </item> + </list> + </section> + + <section> + <title>Improvements and New Features</title> + <list type="bulleted"> + <item> + <p>Minor Makefile changes.</p> + <p>Own Id: OTP-6689</p> + </item> + </list> + </section> + </section> + + <section> + <title>Inets 4.7.14</title> + + <section> + <title>Fixed Bugs and Malfunctions</title> + <list type="bulleted"> + <item> + <p>[tftp] The TFTP client/server implements now a more + relaxed interpretation of the RFC 1350 regarding + re-receive of acknowledgments. If multiple copies of the + same acknowledgments is received the spurious ones are + silently ignored.</p> + <p>Own Id: OTP-6691</p> + </item> + </list> + </section> + </section> + + <section> + <title>Inets 4.7.13</title> + + <section> + <title>Improvements and New Features</title> + <list type="bulleted"> + <item> + <p>[tftp] The TFTP client/server implements now a more + relaxed interpretation of the RFC 1350 regarding + re-receive of data packets. If multiple copies of the + same data packet is received the spurious ones are + silently ignored.</p> + <p>Own Id: OTP-6642</p> + </item> + </list> + </section> + </section> + + <section> + <title>Inets 4.7.12</title> + + <section> + <title>Improvements and New Features</title> + <list type="bulleted"> + <item> + <p>[httpd] - When calling httpd:restart there was a file + descriptor and ets table leek due to the fact that the + semantics of the function restart is not restart but + reload.</p> + <p>Own Id: OTP-6573 Aux Id: seq10607 </p> + </item> + <item> + <p>[tftp] Crash in tftp_engine:terminate/3 caused big crash + report. The file was however transferred as normal, but + the ugly printout is now gone.</p> + <p>Own Id: OTP-6596 Aux Id: seq10618 </p> + </item> + </list> + </section> + </section> + + <section> + <title>Inets 4.7.11</title> + + <section> + <title>Fixed Bugs and Malfunctions</title> + <list type="bulleted"> + <item> + <p>[http] - Chunked encoding updated to handle the empty + chunk.</p> + <p>Own Id: OTP-6511</p> + </item> + <item> + <p>[http] - Removed minor bugs and dead code found by + dialyzer.</p> + <p>Own Id: OTP-6522</p> + </item> + </list> + </section> + </section> + + <section> + <title>Inets 4.7.10</title> + + <section> + <title>Improvements and New Features</title> + <list type="bulleted"> + <item> + <p>[httpd] - The server no longer produces error messages + when the client resets the the connection. This is not an + error as far as the server is concerned.</p> + <p>Own Id: OTP-6484 Aux Id: seq10568 </p> + </item> + <item> + <p>[tftp] The server is now silent by default. Error + messages can however be displayed by setting the debug + level to 'error' (new) or higher.</p> + <p>Own Id: OTP-6498</p> + </item> + </list> + </section> + </section> + + <section> + <title>Inets 4.7.9</title> + + <section> + <title>Fixed Bugs and Malfunctions</title> + <list type="bulleted"> + <item> + <p>Bug in in handling of request timers has been corrected.</p> + <p>Own Id: OTP-6476</p> + </item> + </list> + </section> + + <section> + <title>Improvements and New Features</title> + <list type="bulleted"> + <item> + <p>[tftp] - Daemon was only able to process one request when + fd option was given.</p> + <p>Own Id: OTP-6480</p> + </item> + </list> + </section> + </section> + + <section> + <title>Inets 4.7.8</title> + + <section> + <title>Fixed Bugs and Malfunctions</title> + <list type="bulleted"> + <item> + <p>[ftp, httpc] - Updated the internal ensure_started + functions to handle that inets was started as an included + application.</p> + <p>Own Id: OTP-6409 Aux Id: seq10546 </p> + </item> + </list> + </section> + + <section> + <title>Improvements and New Features</title> + <list type="bulleted"> + <item> + <p>[httpd] - Guard added to API function + httpd_util:integer_to_hexlist/1.</p> + <p>Own Id: OTP-6397</p> + </item> + <item> + <p>[tftp] - Introduced ability to use prebound ports (see + the option {port, Port} for more info). Added peer info + (host and port) as new optional argument to prepare and + open functions in tftp callback modules.</p> + <p>Own Id: OTP-6413</p> + </item> + </list> + </section> + </section> + + <section> + <title>Inets 4.7.7</title> + + <section> + <title>Fixed Bugs and Malfunctions</title> + <list type="bulleted"> + <item> + <p>[httpd] - The http server could throw away pipelined + requests leaving the client hanging.</p> + <p>Own Id: OTP-6310</p> + </item> + <item> + <p>[ftp] - Code for handling fallback to ipv4 was not + exhaustive.</p> + <p>Own Id: OTP-6312</p> + </item> + <item> + <p>[httpd] - Incorrect handling of ipv6 address would crash + the loading of the BindAddress parameter, which had the + consequence that you could not start more than one HTTP + server on an erlang node.</p> + <p>Own Id: OTP-6323</p> + </item> + <item> + <p>[httpc] - Some 30X codes, as for instance 302, should not + always be automatically redirected but in inets-4.7.6 some + restrictions where made too hard, never allowing an + automatic redirection. (Automatic redirect should be + allowed for get and head.)</p> + <p>Own Id: OTP-6332</p> + </item> + <item> + <p>[ftp] - The mode flag that could be used to force ftp + active mode was ignored. Only the deprecated function + force_active/1 works for prior releases to this one.</p> + <p>Own Id: OTP-6342</p> + </item> + </list> + </section> + + <section> + <title>Improvements and New Features</title> + <list type="bulleted"> + <item> + <p>[httpc] - Added debug feature to turn on/off some basic + erlang tracing on client processes.</p> + <p>Own Id: OTP-6326</p> + </item> + </list> + </section> + </section> + + <section> + <title>Inets 4.7.6</title> + + <section> + <title>Fixed Bugs and Malfunctions</title> + <list type="bulleted"> + <item> + <p>[httpc] - The parsing of uri's was rewritten so that it + should handle all types of uri's including ipv6 uri's.</p> + <p>Own Id: OTP-5677</p> + </item> + <item> + <p>[httpc, httpd] - Extensions and trailers where not + properly handled by the chunk decoding implementation.</p> + <p>Own Id: OTP-6005 Aux Id: OTP-6264 </p> + </item> + <item> + <p>[httpc] - A request resulting in an empty body is now + returned without any delays.</p> + <p>Own Id: OTP-6243</p> + </item> + <item> + <p>[httpc] - When http:request/4 was used with a configured + proxy, and the webserver returns 3XX code, http:request/4 + entered an endless loop. Two problems was solved in this + area, the absolute uri is now updated when a redirect is + issued, so that the problem in this case will not arise, + and the redirection endless loop detection was fixed so + that will actually detect potential endless loops.</p> + <p>Own Id: OTP-6244</p> + </item> + <item> + <p>[httpc, httpd] - In some cases if a body contained the + sequence "\\r\ + 0" and was chunked encoded this sequence + was incorrectly interpreted as the last chunk.</p> + <p>Own Id: OTP-6264 Aux Id: OTP-6005 </p> + </item> + <item> + <p>[httpc, httpd] - http_request.erl didn't handle https + URIs, which meant that redirects from ESI did not work.</p> + <p>Own Id: OTP-6274</p> + </item> + <item> + <p>[httpc, httpd] - The base 64 decoder was missing a guard + so that invalid input lead to an emulator crash instead + of a function clause as expected. Also the http server + has been improved to handle the function clause error + returning a bad credentials reply to the client.</p> + <p>Own Id: OTP-6279</p> + </item> + <item> + <p>[httpc] - Changed internal default value as it sometimes + would be interpreted incorrectly causing the client to + return an incomplete body.</p> + <p>Own Id: OTP-6283</p> + </item> + <item> + <p>[httpc] - Handling of 30X codes was changed so that it + works according to the documentation. For instance 301 + and 302 codes will not be automatically redirected.</p> + <p>Own Id: OTP-6297</p> + </item> + <item> + <p>[httpc] - A bug in the pipeline-handling code could cause + a response to be sent to the client with an incorrect + request id.</p> + <p>Own Id: OTP-6303</p> + </item> + </list> + </section> + + <section> + <title>Improvements and New Features</title> + <list type="bulleted"> + <item> + <p>[httpc] - Added feature to send header values as they + where typed by the user of the client. Note that the http + standard requires them to be case insensitive. This + feature should only be used if there is no other way to + communicate with the server or for testing purpose.</p> + <p>Own Id: OTP-5527</p> + </item> + <item> + <p>[httpc] - When using asynchronous HTTP-request it is now + possible to receive "200-responses" as streams instead of + having to wait until the whole response has been + delivered. It also possible to stream "200-response + bodys" to a file both for synchronous and asynchronous + requests.</p> + <p>Own Id: OTP-6263</p> + </item> + <item> + <p>[httpc] - Added option to generate Proxy-Authorization + header from provided proxy username and password.</p> + <p>Own Id: OTP-6280</p> + </item> + </list> + </section> + </section> + + <section> + <title>Inets 4.7.5</title> + + <section> + <title>Fixed Bugs and Malfunctions</title> + <list type="bulleted"> + <item> + <p>[FTP] Change documentation so that it agrees with the + default behaviour of the code regarding the use of + transfer type, which is right according to rfc959.</p> + <p>Own Id: OTP-6018</p> + </item> + <item> + <p>[ftp] The application handles the case if the owning + process terminates with the reason 'shutdown'.</p> + <p>Own Id: OTP-6035</p> + </item> + <item> + <p>[ftp] Handle file errors from the FTP server.</p> + <p>Own Id: OTP-6036</p> + </item> + <item> + <p>[httpd] Header parsing of reply from cgi script incorrect.</p> + <p>Own Id: OTP-6145</p> + </item> + <item> + <p>[ftp] The timeout given in the ftp:open call was not + properly used, which could leave the caller hanging + forever.</p> + <p>Own Id: OTP-6184</p> + <p>Aux Id: seq10388</p> + </item> + <item> + <p>[httpd] HTTPD request handler does not handle unexpected info + properly, which causes an unnecessarily obscure error + message.</p> + <p>Own Id: OTP-6189</p> + <p>Aux Id: seq10395</p> + </item> + <item> + <p>[http] Misc fixes in the URI parsing module.</p> + <p>Igor Goryachev</p> + <p>Own Id: OTP-6191</p> + </item> + </list> + </section> + + <section> + <title>Improvements and New Features</title> + <list type="bulleted"> + <item> + <p>Added application interface module: <c>inets</c>.</p> + <p>Own Id: OTP-6135</p> + </item> + </list> + </section> + </section> + + <section> + <title>Inets 4.7.4</title> + + <section> + <title>Fixed Bugs and Malfunctions</title> + <list type="bulleted"> + <item> + <p>Removed code generating compiler warnings</p> + <p>Own Id: OTP-6069</p> + </item> + </list> + </section> + + <section> + <title>Improvements and New Features</title> + <list type="bulleted"> + <item> + <p>[tftp] Added documentation (manual page) for + <seealso marker="tftp">TFTP</seealso>.</p> + <p>Own Id: OTP-6082</p> + </item> + </list> + </section> + </section> + + <section> + <title>Inets 4.7.3</title> + + <section> + <title>Fixed Bugs and Malfunctions</title> + <list type="bulleted"> + <item> + <p>[http,server] started dbg even though no debugging was + desired.</p> + <p>Own Id: OTP-5984 Aux Id: seq10290 </p> + </item> + <item> + <p>[http,server] request handler process died ugly due to a + parse error when validating a bad request.</p> + <p>Own Id: OTP-6003 Aux Id: seq10260 </p> + </item> + </list> + </section> + </section> + + <section> + <title>Inets 4.7.2</title> + + <section> + <title>Fixed Bugs and Malfunctions</title> + <list type="bulleted"> + <item> + <p>[http,server] Server falsely sets ipv6 enabled when it + gets an ipv4-mapped ipv6 address.</p> + <p>Own Id: OTP-5941 Aux Id: seq10221 </p> + </item> + <item> + <p>[http,server] In case http version is unknown of client + use HTTP1.0 to send status.</p> + <p>Own Id: OTP-5943 Aux Id: seq10198 </p> + </item> + <item> + <p>[http,server] The process handling a request now ignores + garbage messages.</p> + <p>Own Id: OTP-5961 Aux Id: seq10198 </p> + </item> + <item> + <p>[http,server] Changed some actions taken if config data + in httpd services was faulty.</p> + <p>Own Id: OTP-5962 Aux Id: seq10198 </p> + </item> + </list> + </section> + </section> + + <section> + <title>inets 4.7.1</title> + + <section> + <title>Fixed Bugs and Malfunctions</title> + <list type="bulleted"> + <item> + <p>[http,server] It was possible to read arbitrary files on + server by prepending ././ and ../../ in front of the file + name.</p> + <p>Own Id: OTP-5938</p> + </item> + </list> + </section> + </section> + + <section> + <title>Inets 4.7</title> + + <section> + <title>Fixed Bugs and Malfunctions</title> + <list type="bulleted"> + <item> + <p>[http,server] Handling of undefined file times (e.g. + modification time: the mtime field on the file_info + record).</p> + <p>Own Id: OTP-5865 Aux Id: OTP-5844 </p> + </item> + </list> + </section> + + <section> + <title>Improvements and New Features</title> + <list type="bulleted"> + <item> + <p>[http,server]It is now possible to set the wanted timeout + for the server to setup a request connection. A new + syntax for inets.config is provided in the users guide + documentation. This syntax also allows to set tracing of + the server for debug purposes.</p> + <p>Own Id: OTP-5913 Aux Id: seq10198 </p> + </item> + </list> + </section> + </section> + + <section> + <title>Inets 4.6.2</title> + + <section> + <title>Fixed Bugs and Malfunctions</title> + <list type="bulleted"> + <item> + <p>[http,server] Had earlier forgot to convert a value in + entity_body to a binary.</p> + <p>Own Id: OTP-5796</p> + </item> + <item> + <p>[http,server] Now application checks whether the + necessary directives under directive Directory exist.</p> + <p>Own Id: OTP-5821</p> + </item> + </list> + </section> + </section> + + <section> + <title>Inets 4.6.1</title> + + <section> + <title>Fixed Bugs and Malfunctions</title> + <list type="bulleted"> + <item> + <p>[http, client] If an Ipv4-mapped ipv6 address is used the + client now falls back on ipv4.</p> + <p>Own Id: OTP-5773 Aux Id: OTP-5765, OTP-5764 </p> + </item> + <item> + <p>[http,server] Content-length may got a too low value, + causing loss of data in clients.</p> + <p>Own Id: OTP-5775 Aux Id: seq10110 </p> + </item> + </list> + </section> + + <section> + <title>Improvements and New Features</title> + <list type="bulleted"> + <item> + <p>[http, client] The verbose mode prints what was sent and + received during a request. Added a <c>verbose</c> option + that can be used by <c>http:set_options/1</c></p> + <p>Own Id: OTP-5766</p> + </item> + </list> + </section> + </section> + + <section> + <title>Inets 4.6</title> + + <section> + <title>Fixed Bugs and Malfunctions</title> + <list type="bulleted"> + <item> + <p>[ftp, client] - Improvement of error handling if + something goes wrong while handling the option list in + ftp:open/[1,2,3].</p> + <p>Own Id: OTP-5711</p> + </item> + <item> + <p>[ftp, client] - Error when parsing a multiple FTP + response line. The last line in a multiple response must + be the response code followed by a space. A server may + have intermediate lines that start with the response code + even if this is not recommended. The parsing missed to + make sure that that space was present in what it + considered to be the last line.</p> + <p>Own Id: OTP-5712</p> + </item> + <item> + <p>[http, client] - The HTTP client will now retry a + pipelined request that was unsuccessful due to the fact + that the server unexpectedly closed the pipeline + connection.</p> + <p>Own Id: OTP-5728</p> + </item> + <item> + <p>[http, server, esi] - Under some circumstances mod_esi + would send a corrupted content-length header.</p> + <p>Own Id: OTP-5735</p> + </item> + <item> + <p>[http, server, get] - Removed debug printout which + caused a confusing "Socket closed"-printout at high + load.</p> + <p>Own Id: OTP-5762 Aux Id: seq10101 </p> + </item> + <item> + <p>FTP: a data connection setup to the ftp server that + failed caused a crash of the client. Now it is handled + smoothly.</p> + <p>Own Id: OTP-5738</p> + </item> + <item> + <p>[ftp] If host name is a ipv4 tuple ftp erroneous tries to + connect as a ipv6 address with the ipv4 address.</p> + <p>Own Id: OTP-5764</p> + </item> + <item> + <p>[ftp] Handles connect to ipv6/ipv4 address differently + according to a change in <c>inet:getaddr</c>.</p> + <p>Own Id: OTP-5765</p> + </item> + </list> + </section> + + <section> + <title>Improvements and New Features</title> + <list type="bulleted"> + <item> + <p>[http, server] - The HTTP request handling was remoduled + to have a more straight forward error-handling. And the + internal debug strategy was changed to use tracing + instead of debug macros, which means we do not have to + write special debug code.</p> + <p>Own Id: OTP-5729</p> + </item> + <item> + <p>ftp:ls towards different ftp servers resulted in + different return results. E.g. the solaris 9 default + server caused <c>{ok,[]}</c> while older servers caused + <c>{error,epath}</c> as the result. For backwards + compatibility the behaviour has been changed to the old + result.</p> + <p>Own Id: OTP-5731</p> + </item> + <item> + <p>[http, server] - The documentation for the HTTP server + has been partly rewritten and very restructured too + provide a better overall picture. Lots of information + provided by "semi manual pages" has been moved to the + Users Guide.</p> + <p>Own Id: OTP-5752</p> + </item> + <item> + <p>FTP: verbose mode now also prints what the client sends + on the control channel.</p> + <p>Own Id: OTP-5753</p> + </item> + </list> + </section> + </section> + + <section> + <title>Inets 4.5.4</title> + + <section> + <title>Fixed Bugs and Malfunctions</title> + <list type="bulleted"> + <item> + <p>[ftp, client] - Timing related issues could sometimes + cause the ftp response to be delayed.</p> + <p>Own Id: OTP-5705 Aux Id: seq10055 </p> + </item> + <item> + <p>[http, server, esi] - The dispatching of the post body to + the esi callback function was broken.</p> + <p>Own Id: OTP-5706</p> + </item> + </list> + </section> + </section> + + <section> + <title>Inets 4.5.3</title> + + <section> + <title>Fixed Bugs and Malfunctions</title> + <list type="bulleted"> + <item> + <p>[ftp, client] - The FTP error code 550 was handled in an + unexpected way. Some earlier versions of inets had a + workaround for this in ftp:recv_bin/2 that was eliminated + during restructuring of the ftp module while implementing + ipv6 capabilities. The problem is now fixed.</p> + <p>Own Id: OTP-5682 Aux Id: seq10048 </p> + </item> + <item> + <p>[http, client] Post request with a body in binary format + failed as length was used instead of size.</p> + <p>Own Id: OTP-5686</p> + </item> + <item> + <p>[ftp, client] - For some FTP commands the FTP server will + send more than one reply on the FTP control channel. In + the case of a fast FTP server the client would sometimes + wrongly disregard the second answer as trailing garbage + attached to the first reply.</p> + <p>Own Id: OTP-5690 Aux Id: seq10055 </p> + </item> + </list> + </section> + + <section> + <title>Improvements and New Features</title> + <list type="bulleted"> + <item> + <p>[ftp, client] - A new option {progress, {CBmodule, + CBFunction, InitProgressTerm} has been added to allow + users to create things such as progress bars in there + GUI's. The option affects ftp:send/[3,4] and + ftp:recv/[3,4].</p> + <p>Own Id: OTP-5680</p> + </item> + <item> + <p>[http, client] - Added new API function http:request/1</p> + <p>Own Id: OTP-5691</p> + </item> + <item> + <p>[httpd, server] - mod_cgi is implemented according to + CGI-1.1 RFC 3875, an early implementation was based on + some draft that is not totally compliant to the RFC. + Documentation was updated. Also some code was + restructured to facilitate testing and maintenance of the + server.</p> + <p>Own Id: OTP-5694</p> + </item> + </list> + </section> + </section> + + <section> + <title>Inets 4.5.2</title> + + <section> + <title>Fixed Bugs and Malfunctions</title> + <list type="bulleted"> + <item> + <p>[ftp, client] Calling ftp:recv/2 twice on the same + connection failed due to that the last message on the + ctrl channel was not appropriately taken care of. This + could potentially cause a problem for any operation + performed on the same connection where there had + previously been an ftp:recv/2 call. Also, in some cases, + when the process tries to close the data connection, it + does not take into account that the data connection may + actually not have been established.</p> + <p>Own Id: OTP-5662 Aux Id: seq10004, seq9988 </p> + </item> + <item> + <p>[ftp, client] Enhanced error handling, mainly so the ftp + client behaves gracefully when the user does strange + things such as violate the user API.</p> + <p>Own Id: OTP-5665</p> + </item> + </list> + </section> + + <section> + <title>Improvements and New Features</title> + <list type="bulleted"> + <item> + <p>[ftp, client] Added open option: mode.Deprecates + function force_active/1.</p> + <p>Own Id: OTP-5663</p> + </item> + </list> + </section> + </section> + + <section> + <title>Inets 4.5.1</title> + + <section> + <title>Fixed Bugs and Malfunctions</title> + <list type="bulleted"> + <item> + <p>[http, server] The server did not handle the config + directive BindAddress value "*" properly. + When creating the option list for the listen call, + everything beside the atom undefined (if BindAddress + was never given) and an 4-tuple (e.g. BindAddress + value is 192.168.0.30) was incorrectly assumed to be + an ipv6 address. + For undefined (no BindAddress), Inets attempts to + figure out if it is running on a ipv6-machine, and if + so, add the inet6 option when calling listen. The + same approach should be used when BindAddress is + assigned the value "*".</p> + <p>Own Id: OTP-5642</p> + </item> + <item> + <p>Some data doesn't pass through http_base_64:decode/1 correctly. + The decoding routine fails whenever a 4-character group of the + encoding ends with "9" or "99". If it ends with 99, two bytes + will be lost in the decode routine. If it ends with a single 9, + one byte will be lost in the decode.</p> + <p>Own Id: OTP-5635 Aux Id: seq9971</p> + </item> + <item> + <p>[http, server,esi] Web server does not handle status-code + returned by an esi function. I.e. the esi-function + can no longer control the status code.</p> + <p>Own Id: OTP-5648 Aux Id: seq9982</p> + </item> + <item> + <p>[http, server,esi] Corrected header format. First character + was lower case, and there where no space after the ":" + character, example: content-length:0. Now, first character + was upper case, and a space after the ":" character, + example: Content-Length: 0 (To preserve + backward compatibility with the de-facto standard as the + new way does not break the HTTP standard!)</p> + <p>Own Id: OTP-5649 Aux Id: seq9982</p> + </item> + <item> + <p>[http, server, cgi] Parsing of the status header field could + cause a crash.</p> + <p>Own Id: OTP-5650 Aux Id: seq9982</p> + </item> + </list> + </section> + </section> + + <section> + <title>Inets 4.5</title> + + <section> + <title>Fixed Bugs and Malfunctions</title> + <list type="bulleted"> + <item> + <p>The internal design of using blocking gen_tcp:recv with a + timeout and retries resulted in code that was hard to get + a good overview of, and ultimate led to situations where + the client got the wrong answer or no answer at all. The + errors where many times very timing dependent and mainly + effected the chunk-related functions, so if you where + lucky you probably would not have noticed. The internal + design was changed to use gen_tcp active once semantics. + The API is not effected except for the function + ftp:quote/2 which now returns a list of strings (ftp + result lines) where the line endings "\\r\ + " has been + removed. This was the original intention for the return + value of ftp:quote/2 but it was non trivial to make a + good such solution with the old design and a compromise + was made.</p> + <p>*** POTENTIAL INCOMPATIBILITY ***</p> + <p>Own Id: OTP-5623</p> + </item> + <item> + <p>The new implementation of pipelining in inets-4.1 alas + was slightly broken and unfortunately not caught by the + test suite that apparently needs some additions. The + result was that requests that ought to have been + pipelined where not, this has now been fixed.</p> + <p>Own Id: OTP-5624</p> + </item> + <item> + <p>When using the latest esi interface with the callback + interface of arity 3, HTTP Content-Type headers where + ignored, this due to a subtle difference between this + interface and the old one in how they viewed HTTP + delimiters.</p> + <p>Own Id: OTP-5627</p> + </item> + </list> + </section> + + <section> + <title>Improvements and New Features</title> + <list type="bulleted"> + <item> + <p>The HTTP server now supports ipv6 in the case that the + underlying mechanisms also do so. (ssl does not yet + support ipv6.)</p> + <p>Own Id: OTP-5141</p> + </item> + <item> + <p>The FTP client now supports ipv6 in the case that the + underlying mechanisms also do so.</p> + <p>Own Id: OTP-5142</p> + </item> + <item> + <p>An option was added to disable the ipv6 support in the + HTTP client. This to provide a workaround possibility for + buggy ipv6-stacks.</p> + <p>Own Id: OTP-5625 Aux Id: seq9872</p> + </item> + <item> + <p>When generating dynamic HTTP response bodies the the + default content-type is now set to "text/html" instead of + "text/plain" which is more intuitive.</p> + <p>*** POTENTIAL INCOMPATIBILITY ***</p> + <p>Own Id: OTP-5626</p> + </item> + </list> + </section> + </section> + + <section> + <title>Inets 4.4.1</title> + + <section> + <title>Fixed Bugs and Malfunctions</title> + <list type="bulleted"> + <item> + <p>Wrapper function http_transport:accept/3 did + error-handling that it should not do because only the + caller of the wrapper can determine what action to take. + So timeouts where handled twice, once in http_transport + and once in httpd_acceptor. Clean up of the wrapper + module http_transport changed the action of the wrapper + module and made the unwanted behavior noticeable in in + OTP error logs. And now the unwanted error handling has + been removed. The cleanup helped us find bad code but + alas it also generates a lot of log printouts that are + quite disturbing to the user of the HTTP-server.</p> + <p>Own Id: OTP-5549 Aux Id: seq9851 </p> + </item> + <item> + <p>In the rewrite for 4.4 some mod_esi-environment values + where mistaken for ordinary header values and where + incorrectly transformed to strings. They are now atoms + again.</p> + <p>Own Id: OTP-5551 Aux Id: seq9854 </p> + </item> + <item> + <p>The HTTP server now handles "GET /\\r\ + \\r\ + " as well as + "GET / \\r\ + \\r\ + ". According to the RFC the whitespace is + not needed.</p> + <p>Own Id: OTP-5552 Aux Id: seq8426 </p> + </item> + </list> + </section> + + <section> + <title>Improvements and New Features</title> + <list type="bulleted"> + <item> + <p>The ftp client now supports passive mode. Actually the + ftp client will always try to use passive mode and if it + fails it will use active mode instead. It is also + possible to force the ftp-client to use active mode, if + that is desired, by calling ftp:force_active/1 this way + you can get the old behavior.</p> + <p>*** POTENTIAL INCOMPATIBILITY ***</p> + <p>Own Id: OTP-5148</p> + </item> + </list> + </section> + </section> + + <section> + <title>Inets 4.4</title> + + <section> + <title>Fixed Bugs and Malfunctions</title> + <list type="bulleted"> + <item> + <p>The server did not handle HTTP-0.9 messages with an + implicit version.</p> + <p>Own Id: OTP-5513</p> + </item> + <item> + <p>An internal server timeout killed the request handling + process without sending a message back to the client. As + this timeout only affects a single request it has been + set to infinity (if the main server process dies the + request handling process will also die and the client + will receive an error). This might make a client that + does not use a timeout hang for a longer period of time, + but that is an expected behavior!</p> + <p>Own Id: OTP-5514 Aux Id: seq9806 </p> + </item> + <item> + <p>That a third party closes the http servers accept socket + is recoverable for inets, hence inets will only produce + an info report as there was no error in Inets but + measures where taken to avoid failure due to errors + elsewhere.</p> + <p>Own Id: OTP-5516 Aux Id: seq9806 </p> + </item> + <item> + <p>The HTTP client proxy settings where ignored. Bug + introduced in inets-4.3.</p> + <p>Own Id: OTP-5517</p> + </item> + <item> + <p>Inets only sent the "WWW-Authenticate" header at the + first attempt to get a page, if the user supplied the + wrong user/password combination the header was not sent + again. This forces the user to kill the browser entirely + after a failed login attempt, before the user may try to + login again. Inets now always send the authentication + header.</p> + <p>Own Id: OTP-5521</p> + </item> + <item> + <p>A major rewrite of big parts of the HTTP server code was + performed. There where many things that did not work + satisfactory. Cgi script handling can never have worked + properly and the cases when it did sort of work, a big + unnecessary delay was enforced. Headers where not always + treated as expected and HTTP version handling did not + work, all responses where sent as version HTTP/1.1 no + matter what.</p> + <p>Own Id: OTP-5537</p> + </item> + </list> + </section> + </section> + + <section> + <title>Inets 4.3.1</title> + + <section> + <title>Fixed Bugs and Malfunctions</title> + <list type="bulleted"> + <item> + <p>When further testing the functionality of https requests + that goes through a proxy. We realised that alas this can + not currently be supported as it requires features from + the ssl implementation that is not currently available. + So for now an error message will be returned when trying + to use this functionality.</p> + <p>Own Id: OTP-5453</p> + </item> + <item> + <p>When trying to get a url from a server that does not + exist the client hanged instead of returning an error + message. Bug introduced in inets-4.3.</p> + <p>Own Id: OTP-5454</p> + </item> + </list> + </section> + </section> + + <section> + <title>Inets 4.3</title> + + <section> + <title>Fixed Bugs and Malfunctions</title> + <list type="bulleted"> + <item> + <p>Tunneling of SSL through a proxy has now been + implemented. However due to lack of test sites this has only + partially been verified, it is likely that there will + have to be future improvements in this area.</p> + <p>Own Id: OTP-5443</p> + </item> + </list> + </section> + + <section> + <title>Improvements and New Features</title> + <list type="bulleted"> + <item> + <p>The pipeline timeout was changed to be zero by default to + avoid that people by accident would create connection + processes that never dies and eats up the socket + resources.</p> + <p>*** POTENTIAL INCOMPATIBILITY ***</p> + <p>Own Id: OTP-5442</p> + </item> + <item> + <p>Altered the way spawn_link is used in mod_esi to avoid + getting, in this scenario unwanted error reports, from + spawn_link. (The behavior of spawn_link was altered in a + not backwards compatible way.)</p> + <p>Own Id: OTP-5444</p> + </item> + </list> + </section> + </section> + + <section> + <title>Inets 4.2.1</title> + + <section> + <title>Fixed Bugs and Malfunctions</title> + <list type="bulleted"> + <item> + <p>Sometimes EWS modules where called with an Info record + where the peername field was {-1, "unknown"}. This could + happen when a client was making a lot of requests which + it discards before they where answered by the server. The + server now ignores such requests and does not call the + EWS modules in this case.</p> + <p>Own Id: OTP-5380 Aux Id: seq9739 </p> + </item> + <item> + <p>The HTTP-server now returns the 408 status code upon a + request timeout as expected instead of the previous + faulty behavior of sending a 500 status code.</p> + <p>Own Id: OTP-5409</p> + </item> + <item> + <p>The content length was put in to the HTTP-headers as an + integer instead of as a string.</p> + <p>Own Id: OTP-5410</p> + </item> + <item> + <p>It was wrongly presumed that code:priv_dir would always + be writable due to how the test-server works. The + directory is now a configuration parameter in the + inets-application configuration file. Failing to + configure it will result in that all cookies are treated + as session cookies.</p> + <p>Own Id: OTP-5411</p> + </item> + </list> + </section> + + <section> + <title>Improvements and New Features</title> + <list type="bulleted"> + <item> + <p>An undocumented beta version of tftp is included.</p> + <p>Own Id: OTP-5419</p> + </item> + </list> + </section> + </section> + + <section> + <title>Inets 4.2</title> + + <section> + <title>Fixed Bugs and Malfunctions</title> + <list type="bulleted"> + <item> + <p>When sending a request through a proxy the absolute URI + must be used.</p> + <p>Own Id: OTP-5368</p> + </item> + </list> + </section> + + <section> + <title>Improvements and New Features</title> + <list type="bulleted"> + <item> + <p>Basic support for cookies was implemented. Later some + more functions to inspect cookies may be added.</p> + <p>Own Id: OTP-5331</p> + </item> + <item> + <p>A top tftp supervisor was added in preparation for adding + a tftp service in a future Inets release.</p> + <p>Own Id: OTP-5379</p> + </item> + </list> + </section> + </section> + + <section> + <title>Inets 4.1</title> + + <section> + <title>Fixed Bugs and Malfunctions</title> + <list type="bulleted"> + <item> + <p>The URI check that disables relative links that goes + outside the server-root still missed a few cases, in + spite of the improvement in OTP-5140.</p> + <p>Own Id: OTP-5249</p> + </item> + <item> + <p>The http client pipelining implementation has been + rewritten as the old implementation was too optimistic + about when to pipeline. In the process of doing this also + the error handling was improved, better clean up is + performed when the request handling process terminates + and better handling of the case that the httpc_manager + process dies and is restarted.</p> + <p>Own Id: OTP-5303</p> + </item> + <item> + <p>Improved handling of status codes 30X and 50X.</p> + <p>Own Id: OTP-5309</p> + </item> + </list> + </section> + + <section> + <title>Improvements and New Features</title> + <list type="bulleted"> + <item> + <p>The Inets supervision tree has been reorganized to create + a better balance between the Inets services. Preferably + they should not effect each other. The ftp service has + also been included in the Inets supervision tree, it was + for reasons unknown, not included before.</p> + <p>Own Id: OTP-5188</p> + </item> + <item> + <p>The service concept in Inets is now better documented.</p> + <p>Own Id: OTP-5189</p> + </item> + <item> + <p>The Inets shutdown times have proven to be too short + under some circumstances, as a heavy load, therefore they + have been prolonged.</p> + <p>Own Id: OTP-5261 Aux Id: seq9624</p> + </item> + <item> + <p>Options for automatic redirection and pipelining is now + available in the http client API.</p> + <p>Own Id: OTP-5304</p> + </item> + </list> + </section> + </section> + + <section> + <title>Inets 4.0.1</title> + + <section> + <title>Fixed Bugs and Malfunctions</title> + <list type="bulleted"> + <item> + <p>A programming error could cause a badmatch in the + http-client when the http response was chunk decoded.</p> + <p>Own Id: OTP-5101</p> + </item> + <item> + <p>The parsing of HTTP messages was missing a base case. + This caused unexpected behavior when the separator CR and + LF where received in different tcp packets.</p> + <p>Own Id: OTP-5239</p> + </item> + </list> + </section> + </section> + + <section> + <title>Inets 4.0</title> + + <section> + <title>Fixed Bugs and Malfunctions</title> + <list type="bulleted"> + <item> + <p>When receiving a status 100 code, the client should only + respond by sending the message body, if the client sent + an expect header in the first place. Failing to do so may + result in that the server receives the body twice.</p> + <p>Own Id: OTP-4848</p> + </item> + <item> + <p>mod_get now also handles http version HTTP/0.9</p> + <p>Own Id: OTP-4935 Aux Id: seq8426 </p> + </item> + <item> + <p>"Last-modified" field was incorrectly set to local time + with the tag GMT, it is now corrected so that the time + reflected is in fact GMT.</p> + <p>Own Id: OTP-4936</p> + </item> + <item> + <p>The client will only add a host-field to the request if + there is not one already present.</p> + <p>Own Id: OTP-4984</p> + </item> + <item> + <p>The Inets application tries to be compatible with + Apache. To be more compatible the option + 'MaxKeepAliveRequest' is renamed 'MaxKeepAliveRequests'. + The old name is kept for backward compatibility.</p> + <p>Own Id: OTP-5024</p> + </item> + <item> + <p>Changing the base 64 decoding to not accept invalid + input, uncovered a logical error in mod_security.erl An + already decoded string was sent as input to decode. In + this case, as it so happened, the two errors worked + together creating the illusion that everything was right. + This has now been corrected.</p> + <p>Own Id: OTP-5083</p> + </item> + <item> + <p>URLs where not properly scrutinised for relative paths. A + malicious user could exploit this to read files outside + the document root. This is no longer the case.</p> + <p>Own Id: OTP-5140</p> + </item> + </list> + </section> + + <section> + <title>Improvements and New Features</title> + <list type="bulleted"> + <item> + <p>A HTTP 1.1 client is officially included in Inets. It is + loosely based on the previously unsupported code + contributed by Johan Blom. In this first version only the + most basic HTTP functionality is supported. The user API + has been changed.</p> + <p>*** POTENTIAL INCOMPATIBILITY ***</p> + <p>Own Id: OTP-5047</p> + </item> + <item> + <p>Fixed erroneous link in documentation.</p> + <p>Own Id: OTP-5089 Aux Id: seq8887 </p> + </item> + <item> + <p>Added the function quote/2 that lets you send an + arbitrary FTP command to the FTP client.</p> + <p>Own Id: OTP-5099 Aux Id: seq8961 </p> + </item> + <item> + <p>Started integration of the HTTP client and server code + too facilitate maintenance and further development.</p> + <p>Own Id: OTP-5110</p> + </item> + <item> + <p>Due to several possibilities to interpret the ftp + standard some newer ftp-servers have interpreted the + standard in such a way that the documented return value + of ftp:nlist/2 does not always match the actual return + value. Some extra checks have now been added to ensure + the documented return value. This will also result in + that ftp:nlist is not bug compatible in the case that + nlist is given a filename instead of a directory it will + now return an error instead of {ok, FileName}.</p> + <p>*** POTENTIAL INCOMPATIBILITY ***</p> + <p>Own Id: OTP-5165</p> + </item> + <item> + <p>Created a Users Guide for Inets. Earlier there where some + fake manual pages and information was scattered + everywhere and hard to find.</p> + <p>Own Id: OTP-5180</p> + </item> + </list> + </section> + </section> + + <section> + <title>Inets 3.0.10</title> + + <section> + <title>Fixed Bugs and Malfunctions</title> + <list type="bulleted"> + <item> + <p>[httpd] - When calling httpd:restart there was a file + descriptor and ets table leek due to the fact that the + semantics of the function restart is not restart but + reload. This is solved in inets-4.7.12 and this special + inets-3.0.10 release is intended for old systems only.</p> + <p>Own Id: OTP-6579 Aux Id: OTP-6573, seq10607 </p> + </item> + </list> + </section> + </section> + + <section> + <title>Inets 3.0.9</title> + + <section> + <title>Fixed Bugs and Malfunctions</title> + <list type="bulleted"> + <item> + <p>[http,server] Illegal hexadecimal code in URL was not + handled. The validation of URI:s are therefore updated.</p> + <p>Own Id: OTP-6078 Aux Id: seq10306 </p> + </item> + </list> + </section> + </section> + + <section> + <title>Inets 3.0.8 </title> + <p>Special version featuring some small 4.1 improvements + without enforcing the big changes of the 4.X releases. </p> + + <section> + <title>Fixed Bugs and Malfunctions</title> + <list type="bulleted"> + <item> + <p>The URI check that disables relative links that goes + outside the server-root missed a few cases.</p> + <p>Own Id: OTP-5249</p> + </item> + </list> + </section> + + <section> + <title>Improvements and New Features</title> + <list type="bulleted"> + <item> + <p>The inets shutdown times have proven to be too short + under some circumstances, as a heavy load, therefore they + have been prolonged.</p> + <p>Own Id: OTP-5261 Aux Id: seq9624 </p> + </item> + </list> + </section> + </section> + + <section> + <title>Inets 3.0.7</title> + + <section> + <title>Reported Fixed Bugs and Malfunctions</title> + <list type="bulleted"> + <item> + <p>[httpd] Fixed a number of transfer-encoding problems.</p> + <p>First part of the data received from a CGI-script was sent + as chunked even if the client was HTTP/1.0.</p> + <p>Introduced new directive + (<c>DisableChunkedTransferEncodingSend</c>) to turn off usage + of chunked transfer-encoding (when sending) since it appear's + some browser's have problems handling this. This applies + if the client is HTTP/1.1.</p> + <p>(Own Id: OTP-4806) + <br></br> + Aux Id: Seq 8150</p> + </item> + <item> + <p>[httpc] HTTP client reformats some URLs (e.g. containing %20, + space).</p> + <p>Updated client from sowap.sf.net as of 2003-09-08.</p> + <p>Johan Blom of Mobile Arts AB</p> + <p>(Own Id: OTP-4807)</p> + </item> + <item> + <p>[httpd] In module mod_browser, malformed search for parsed + header, user-agent.</p> + <p>Also added new os and browser</p> + <p>(Own Id: OTP-4808)</p> + </item> + <item> + <p>[ftp] FTP client doesn't notice when disk is full.</p> + <p>(Own Id: OTP-4822) + <br></br> + Aux Id: Seq 8175</p> + </item> + </list> + </section> + </section> + + <section> + <title>Inets 3.0.6</title> + + <section> + <title>Reported Fixed Bugs and Malfunctions</title> + <list type="bulleted"> + <item> + <p>On Windows the <c>ftp:ls</c> function sometimes exits.</p> + <p>Workaround for a problem that seems to happen only on Windows + when calling the ls function. Closing an already closed socket + will result in enotsock returned which will result in an exit.</p> + <p>(Own Id: OTP-4770)</p> + </item> + </list> + </section> + </section> + + <section> + <title>Inets 3.0.5</title> + + <section> + <title>Reported Fixed Bugs and Malfunctions</title> + <list type="bulleted"> + <item> + <p>Web server does not handle econnaborted accept result.</p> + <p>This results in an unnecessary acceptor process restart.</p> + <p>(Own Id: OTP-4732)</p> + </item> + </list> + </section> + </section> + + <section> + <title>Inets 3.0.4</title> + + <section> + <title>Reported Fixed Bugs and Malfunctions</title> + <list type="bulleted"> + <item> + <p>ESI callback generates broken HTTP.</p> + <p>This was a problem for (at least) Netscape 4.75. It worked for + Mozilla 1.4a (on Solaris 8) and rumor has it that it also worked + for IE.</p> + <p> <br></br> + Sean Hinde</p> + <p>(Own Id: OTP-4696)</p> + </item> + <item> + <p>Log-size values ignored for security- and error disk-logs + (SecurityDiskLogSize and ErrorDiskLogSize in mod_disk_log). + Instead the TransferDiskLogSize was used. + <br></br> + Thomas Lange</p> + <p>(Own Id: OTP-4659)</p> + </item> + </list> + </section> + </section> + + <section> + <title>Inets 3.0.3</title> + + <section> + <title>Reported Fixed Bugs and Malfunctions</title> + <list type="bulleted"> + <item> + <p>HTTP client called undefined function in HTTP server + (httpd_response:send_status/3).</p> + <p>(Own Id: OTP-4628)</p> + </item> + </list> + </section> + </section> + + <section> + <title>Inets 3.0.2</title> + + <section> + <title>Reported Fixed Bugs and Malfunctions</title> + <list type="bulleted"> + <item> + <p>HTTP client <em>now</em> updated to jnets-0.1. + <br></br> + Auther: Johan Blom of Mobile Arts AB.</p> + <p>(Own Id: OTP-4588)</p> + </item> + </list> + </section> + </section> + + <section> + <title>Inets 3.0.1</title> + + <section> + <title>Improvements and new features</title> + <list type="bulleted"> + <item> + <p>FTP client now supports chunked receive. Proposal of Luke Gorrie + provided inspiration but not algorithm.</p> + <p>(Own Id: OTP-4549)</p> + </item> + <item> + <p>HTTP client updated to jnets-0.1, now supporting proxy. + <br></br> + Auther: Johan Blom of Mobile Arts AB.</p> + <p>(Own Id: OTP-4552)</p> + </item> + </list> + </section> + + <section> + <title>Reported Fixed Bugs and Malfunctions</title> + <list type="bulleted"> + <item> + <p>Body length calculation incorrect.</p> + <p>(Own Id: OTP-4548)</p> + <p>(Aux Id: OTP-4207, Seq 7209)</p> + </item> + <item> + <p>HTTP-request with a BODY length longer than 0 does not work.</p> + <p>(Own Id: OTP-4550)</p> + <p>(Aux Id: Seq 7653)</p> + </item> + <item> + <p>Calculation of remaining chunk size incorrect.</p> + <p>(Own Id: OTP-4551)</p> + </item> + <item> + <p>Wrong module name used when attempting to stop the security + server (mod_sec_server instead of mod_security_server).</p> + <p>(Own Id: OTP-4556)</p> + </item> + </list> + </section> + </section> + + <section> + <title>Inets 3.0</title> + + <section> + <title>Improvements and new features</title> + <list type="bulleted"> + <item>Added HTTP client to the application. + <br></br> + Auther: Johan Blom of Mobile Arts AB.</item> + <item>FTP: More info in exit reason when socket operation fails. + <br></br> + (Own Id: OTP-4429)</item> + <item>Make install targets corrected (INSTALL_SCRIPT is used instead + of INSTALL_PROGRAMS for scripts). + <br></br> + (Own Id: OTP-4428)</item> + <item>In inets, mod_cgi crashes when a directory is protected for + a group or for a user and we try to execute a CGI script + inside this protected directory. + <br></br> + Guillaume Bongenaar. + <br></br> + (Own Id: OTP-4416)</item> + <item>Removed crypto application dependency. + <br></br> + Matthias Lang + <br></br> + (Own Id: OTP-4417)</item> + <item>Use the same read algorithm for socket type ssl as is used + for ip_comm. As of version 2.3.5 of the ssl application it + is possible to use socket option {active, once}, so the same + algorithm can be used for both ip_comm and ssl. + <br></br> + (Own Id: OTP-4374) + <br></br> + (Aux Id: Seq 7417)</item> + <item>Added inets test suite to the release. Including the + lightweight inets test server.</item> + <item>Incorrectly formated disk log entries. + <em>term_to_binary</em> was (incorrectly) used for the + external format. + <br></br> + Own Id: OTP-4228 + <br></br> + Aux Id: Seq 7239</item> + <item>Adding verbosity printouts to 'catch' cgi problems on some + platforms.</item> + <item> + <p>Updated to handle HTTP/1.1.</p> + <list type="bulleted"> + <item>Persistent connections are now default for http/1.1 clients</item> + <item>Module <c>mod_esi</c> can send data to the client in chunks.</item> + <item>Updated configuration directives <em>KeepAlive</em></item> + <item> + <p>New configuration directives:</p> + <list type="bulleted"> + <item><em>MaxKeepAliveRequest</em></item> + <item><em>ErlScriptTimeout</em></item> + <item><em>ErlScriptNoCache</em></item> + <item><em>ScriptTimeout</em></item> + <item><em>ScriptNoCache</em></item> + </list> + </item> + <item>New functions in httpd_utility to ease the development of + http/1.1 complaint modules.</item> + <item>Record mod has a new field absolute_uri.</item> + <item>All header field names in parsed_header is in lowercase.</item> + <item>httpd handles chunked requests.</item> + <item>New module <em>mod_range</em> that handles range-requests.</item> + <item>New module <em>mod_responsecontrol</em> that controls how + the request will be handled the due to the If-Modified, + If-Match and If-Range http header fields.</item> + </list> + </item> + </list> + </section> + + <section> + <title>Reported Fixed Bugs and Malfunctions</title> + <list type="bulleted"> + <item>POST requests not properly handled. + <br></br> + (Own Id: OTP-4409) + <br></br> + (Aux Id: Seq 7485)</item> + <item> + <p>Change in the inets API. + <br></br> + (Own Id: OTP-4408) + <br></br> + (Aux Id: Seq 7485)</p> + <p>*** POTENTIAL INCOMPATIBILITY ***</p> + <p></p> + </item> + <item>When opening the disk log (mod_disk_log), an open attempt + is made without a size option. If the file exist, then it is + opened. If the file does not exist, then another attempt is + made, this time with the size option. + <br></br> + (Own Id: OTP-4281) + <br></br> + (Aux Id: Seq 7312)</item> + <item>Changing of disk log format fails. Restart of webserver after + change of disk log format (DiskLogFormat) fails with + <em>arg_mismatch</em>. + <br></br> + (Own Id: OTP-4231) + <br></br> + (Aux Id: Seq 7244)</item> + </list> + </section> + </section> +</chapter> + + diff --git a/lib/inets/doc/src/part.xml b/lib/inets/doc/src/part.xml new file mode 100644 index 0000000000..36955df6b3 --- /dev/null +++ b/lib/inets/doc/src/part.xml @@ -0,0 +1,42 @@ +<?xml version="1.0" encoding="latin1" ?> +<!DOCTYPE part SYSTEM "part.dtd"> + +<part xmlns:xi="http://www.w3.org/2001/XInclude"> + <header> + <copyright> + <year>2004</year><year>2009</year> + <holder>Ericsson AB. All Rights Reserved.</holder> + </copyright> + <legalnotice> + The contents of this file are subject to the Erlang Public License, + Version 1.1, (the "License"); you may not use this file except in + compliance with the License. You should have received a copy of the + Erlang Public License along with this software. If not, it can be + retrieved online at http://www.erlang.org/. + + Software distributed under the License is distributed on an "AS IS" + basis, WITHOUT WARRANTY OF ANY KIND, either express or implied. See + the License for the specific language governing rights and limitations + under the License. + + </legalnotice> + + <title>Inets User's Guide</title> + <prepared>Ingela Anderton Andin</prepared> + <docno></docno> + <date>2002-09-17</date> + <rev>A</rev> + <file>part.sgml</file> + </header> + <description> + <p>The <em>Inets Application </em> provides a set of Internet + related services. Currently supported are a HTTP client, a HTTP + server a FTP client and a TFTP client and server.</p> + </description> + <xi:include href="inets_services.xml"/> + <xi:include href="ftp_client.xml"/> + <xi:include href="http_client.xml"/> + <xi:include href="http_server.xml"/> +</part> + + diff --git a/lib/inets/doc/src/part_notes.xml b/lib/inets/doc/src/part_notes.xml new file mode 100644 index 0000000000..21f464318b --- /dev/null +++ b/lib/inets/doc/src/part_notes.xml @@ -0,0 +1,39 @@ +<?xml version="1.0" encoding="latin1" ?> +<!DOCTYPE part SYSTEM "part.dtd"> + +<part xmlns:xi="http://www.w3.org/2001/XInclude"> + <header> + <copyright> + <year>2002</year><year>2009</year> + <holder>Ericsson AB. All Rights Reserved.</holder> + </copyright> + <legalnotice> + The contents of this file are subject to the Erlang Public License, + Version 1.1, (the "License"); you may not use this file except in + compliance with the License. You should have received a copy of the + Erlang Public License along with this software. If not, it can be + retrieved online at http://www.erlang.org/. + + Software distributed under the License is distributed on an "AS IS" + basis, WITHOUT WARRANTY OF ANY KIND, either express or implied. See + the License for the specific language governing rights and limitations + under the License. + + </legalnotice> + + <title>Inets</title> + <prepared>Micael Karlberg</prepared> + <docno></docno> + <date>2002-02-28</date> + <rev>3.0</rev> + <file>part_notes.sgml</file> + </header> + <description> + <p>A set of services such as a Web server and a ftp client etc. </p> + <p>For information about older versions see + <url href="part_notes_history_frame.html">release notes history</url>.</p> + </description> + <xi:include file="notes.xml"/> +</part> + + diff --git a/lib/inets/doc/src/part_notes_history.xml b/lib/inets/doc/src/part_notes_history.xml new file mode 100644 index 0000000000..3c1e6f5232 --- /dev/null +++ b/lib/inets/doc/src/part_notes_history.xml @@ -0,0 +1,34 @@ +<?xml version="1.0" encoding="latin1" ?> +<!DOCTYPE part SYSTEM "part.dtd"> + +<part> + <header> + <copyright> + <year>2004</year><year>2009</year> + <holder>Ericsson AB. All Rights Reserved.</holder> + </copyright> + <legalnotice> + The contents of this file are subject to the Erlang Public License, + Version 1.1, (the "License"); you may not use this file except in + compliance with the License. You should have received a copy of the + Erlang Public License along with this software. If not, it can be + retrieved online at http://www.erlang.org/. + + Software distributed under the License is distributed on an "AS IS" + basis, WITHOUT WARRANTY OF ANY KIND, either express or implied. See + the License for the specific language governing rights and limitations + under the License. + + </legalnotice> + + <title>Inets</title> + <prepared>Micael Karlberg</prepared> + <docno></docno> + <date>2002-02-28</date> + <rev>3.0</rev> + <file>part_notes.sgml</file> + </header> + <include file="notes_history"></include> +</part> + + diff --git a/lib/inets/doc/src/ref_man.gif b/lib/inets/doc/src/ref_man.gif Binary files differnew file mode 100644 index 0000000000..b13c4efd53 --- /dev/null +++ b/lib/inets/doc/src/ref_man.gif diff --git a/lib/inets/doc/src/ref_man.xml b/lib/inets/doc/src/ref_man.xml new file mode 100644 index 0000000000..7ec2c041c8 --- /dev/null +++ b/lib/inets/doc/src/ref_man.xml @@ -0,0 +1,53 @@ +<?xml version="1.0" encoding="latin1" ?> +<!DOCTYPE application SYSTEM "application.dtd"> + +<application xmlns:xi="http://www.w3.org/2001/XInclude"> + <header> + <copyright> + <year>1997</year><year>2009</year> + <holder>Ericsson AB. All Rights Reserved.</holder> + </copyright> + <legalnotice> + The contents of this file are subject to the Erlang Public License, + Version 1.1, (the "License"); you may not use this file except in + compliance with the License. You should have received a copy of the + Erlang Public License along with this software. If not, it can be + retrieved online at http://www.erlang.org/. + + Software distributed under the License is distributed on an "AS IS" + basis, WITHOUT WARRANTY OF ANY KIND, either express or implied. See + the License for the specific language governing rights and limitations + under the License. + + </legalnotice> + + <title>Inets Reference Manual</title> + <prepared>Joakim Grebenö</prepared> + <docno></docno> + <date>1997-07-16</date> + <rev>2.1</rev> + <file>ref_man.xml</file> + </header> + <description> + <p>Inets is a container for Internet clients and + servers. Currently a FTP client, a HTTP client and server, and + a tftp client and server has been incorporated in Inets.</p> + </description> + <xi:include href="inets.xml"/> + <xi:include href="ftp.xml"/> + <xi:include href="tftp.xml"/> + <xi:include href="http.xml"/> + <xi:include href="httpd.xml"/> + <xi:include href="httpd_conf.xml"/> + <xi:include href="httpd_socket.xml"/> + <xi:include href="httpd_util.xml"/> + <xi:include href="mod_alias.xml"/> + <xi:include href="mod_auth.xml"/> + <xi:include href="mod_esi.xml"/> + <xi:include href="mod_security.xml"/> +</application> + + + + + diff --git a/lib/inets/doc/src/summary.html.src b/lib/inets/doc/src/summary.html.src new file mode 100644 index 0000000000..17637a0787 --- /dev/null +++ b/lib/inets/doc/src/summary.html.src @@ -0,0 +1 @@ +A set of services such as a web server and a ftp client etc
\ No newline at end of file diff --git a/lib/inets/doc/src/tftp.xml b/lib/inets/doc/src/tftp.xml new file mode 100644 index 0000000000..96d6ae0dd5 --- /dev/null +++ b/lib/inets/doc/src/tftp.xml @@ -0,0 +1,637 @@ +<?xml version="1.0" encoding="latin1" ?> +<!DOCTYPE erlref SYSTEM "erlref.dtd"> + +<erlref> + <header> + <copyright> + <year>2006</year><year>2009</year> + <holder>Ericsson AB. All Rights Reserved.</holder> + </copyright> + <legalnotice> + The contents of this file are subject to the Erlang Public License, + Version 1.1, (the "License"); you may not use this file except in + compliance with the License. You should have received a copy of the + Erlang Public License along with this software. If not, it can be + retrieved online at http://www.erlang.org/. + + Software distributed under the License is distributed on an "AS IS" + basis, WITHOUT WARRANTY OF ANY KIND, either express or implied. See + the License for the specific language governing rights and limitations + under the License. + + </legalnotice> + + <title>tftp</title> + <prepared></prepared> + <docno></docno> + <date></date> + <rev></rev> + </header> + <module>tftp</module> + <modulesummary>Trivial FTP</modulesummary> + <description> + <p>This is a complete implementation of the following IETF standards:</p> + <list type="bulleted"> + <item>RFC 1350, The TFTP Protocol (revision 2).</item> + <item>RFC 2347, TFTP Option Extension.</item> + <item>RFC 2348, TFTP Blocksize Option.</item> + <item>RFC 2349, TFTP Timeout Interval and Transfer Size Options.</item> + </list> + <p>The only feature that not is implemented in this release is + the "netascii" transfer mode.</p> + <p>The <seealso marker="#start/1">start/1</seealso> function starts + a daemon process which listens for UDP packets on a port. When it + receives a request for read or write it spawns a temporary server + process which handles the actual transfer of the file.</p> + <p>On the client side + the <seealso marker="#read_file/3">read_file/3</seealso> + and <seealso marker="#write_file/3">write_file/3</seealso> + functions spawns a temporary client process which establishes + contact with a TFTP daemon and performs the actual transfer of + the file.</p> + <p><c>tftp</c> uses a callback module to handle the actual file + transfer. Two such callback modules are provided, + <c>tftp_binary</c> and <c>tftp_file</c>. See + <seealso marker="#read_file/3">read_file/3</seealso> and + <seealso marker="#write_file/3">write_file/3</seealso> for + more information about these. The user can also implement own + callback modules, see <seealso marker="#tftp_callback">CALLBACK FUNCTIONS</seealso> below. A callback module provided by + the user is registered using the <c>callback</c> option, see + <seealso marker="#options">DATA TYPES</seealso> below.</p> + </description> + + <section> + <title>TFTP SERVER SERVICE START/STOP </title> + + <p>A TFTP server can be configured to start statically when starting + the Inets application. Alternatively it can be started dynamically + (when Inets already is started) by calling the Inets application API + <c>inets:start(tftpd, ServiceConfig)</c>, or + <c>inets:start(tftpd, ServiceConfig, How)</c>, + see <seealso marker="inets">inets(3)</seealso> for details. + The <c>ServiceConfig</c> for TFTP is described below in + the <seealso marker="#options">COMMON DATA TYPES</seealso> + section.</p> + + <p>The TFTP server can be stopped using <c>inets:stop(tftpd, Pid)</c>, + see <seealso marker="inets">inets(3)</seealso> for details.</p> + + <p>The TPFT client is of such a temporary nature that it is not + handled as a service in the Inets service framework.</p> + + </section> + + <section> + <marker id="options"></marker> + <title>COMMON DATA TYPES</title> + <pre> + ServiceConfig = Options + Options = [option()] + option() -- see below + </pre> + <p>Most of the options are common for both the client and the server + side, but some of them differs a little. Here are the available + options:</p> + <taglist> + <tag><c>{debug, Level}</c></tag> + <item> + <p><c>Level = none | error | warning | brief | normal | verbose | all</c></p> + <p>Controls the level of debug printouts. The default is + <c>none</c>.</p> + </item> + <tag><c>{host, Host}</c></tag> + <item> + <p><c>Host = hostname()</c> see + <seealso marker="kernel:inet">inet(3)</seealso></p> + <p>The name or IP address of the host where the TFTP daemon + resides. This option is only used by the client.</p> + </item> + <tag><c>{port, Port}</c></tag> + <item> + <p><c>Port = int()</c></p> + <p>The TFTP port where the daemon listens. It defaults to + the standardized number 69. On the server side it may + sometimes make sense to set it to 0, which means that + the daemon just will pick a free port (which one is + returned by the <c>info/1</c> function).</p> + <p>If a socket has somehow already has been connected, the + {udp, [{fd, integer()}]} option can be used to pass the + open file descriptor to gen_udp. This can be automated + a bit by using a command line argument stating the + prebound file descriptor number. For example, if the + Port is 69 and the file descriptor 22 has been opened by + setuid_socket_wrap. Then the command line argument + "-tftpd_69 22" will trigger the prebound file + descriptor 22 to be used instead of opening port 69. + The UDP option {udp, [{fd, 22}]} automatically be added. + See init:get_argument/ about command line arguments and + gen_udp:open/2 about UDP options.</p> + </item> + <tag><c>{port_policy, Policy}</c></tag> + <item> + <p><c>Policy = random | Port | {range, MinPort, MaxPort}</c> <br></br> +<c>Port = MinPort = MaxPort = int()</c></p> + <p>Policy for the selection of the temporary port which is used + by the server/client during the file transfer. It defaults to + <c>random</c> which is the standardized policy. With this + policy a randomized free port used. A single port or a range + of ports can be useful if the protocol should pass through a + firewall.</p> + </item> + <tag><c>{udp, Options}</c></tag> + <item> + <p><c>Options = [Opt]</c> see + <seealso marker="kernel:gen_udp#open/1">gen_udp:open/2</seealso></p> + </item> + <tag><c>{use_tsize, Bool}</c></tag> + <item> + <p><c>Bool = bool()</c></p> + <p>Flag for automated usage of the <c>tsize</c> option. With + this set to true, the <c>write_file/3</c> client will + determine the filesize and send it to the server as + the standardized <c>tsize</c> option. A <c>read_file/3</c> + client will just acquire filesize from the server by sending + a zero <c>tsize</c>.</p> + </item> + <tag><c>{max_tsize, MaxTsize}</c></tag> + <item> + <p><c>MaxTsize = int() | infinity</c></p> + <p>Threshold for the maximal filesize in bytes. The transfer + will be aborted if the limit is exceeded. It defaults to + <c>infinity</c>.</p> + </item> + <tag><c>{max_conn, MaxConn}</c></tag> + <item> + <p><c>MaxConn = int() | infinity</c></p> + <p>Threshold for the maximal number of active connections. + The daemon will reject the setup of new connections if + the limit is exceeded. It defaults to <c>infinity</c>.</p> + </item> + <tag><c>{TftpKey, TftpVal}</c></tag> + <item> + <p><c>TftpKey = string()</c> <br></br> +<c>TftpVal = string()</c></p> + <p>The name and value of a TFTP option.</p> + </item> + <tag><c>{reject, Feature}</c></tag> + <item> + <p><c>Feature = Mode | TftpKey</c> <br></br> +<c> Mode = read | write</c> <br></br> +<c> TftpKey = string()</c></p> + <p>Control which features that should be rejected. This is + mostly useful for the server as it may restrict usage of + certain TFTP options or read/write access.</p> + </item> + <tag><c>{callback, {RegExp, Module, State}}</c></tag> + <item> + <p><c>RegExp = string()</c> <br></br> +<c>Module = atom()</c> <br></br> +<c>State = term()</c></p> + <p>Registration of a callback module. When a file is to be + transferred, its local filename will be matched to the regular + expressions of the registered callbacks. The first matching + callback will be used the during the transfer. See + <seealso marker="#read_file/3">read_file/3</seealso> and + <seealso marker="#write_file/3">write_file/3</seealso>. + </p> + <p>The callback module must implement the <c>tftp</c> behavior, + <seealso marker="#tftp_callback">CALLBACK FUNCTIONS</seealso>.</p> + </item> + + <tag><c>{logger, Module}</c></tag> + <item> + <p><c>Module = module()()</c></p> + + <p>Callback module for customized logging of error, warning and + info messages. >The callback module must implement the + <c>tftp_logger</c> behavior, + <seealso marker="#tftp_logger">LOGGER FUNCTIONS</seealso>. + The default module is <c>tftp_logger</c>.</p> + </item> + + <tag><c>{max_retries, MaxRetries}</c></tag> + <item> + <p><c>MaxRetries = int()</c></p> + + <p>Threshold for the maximal number of retries. By default + the server/client will try to resend a message up to + <c>5</c> times when the timeout expires.</p> + </item> + </taglist> + </section> + + <funcs> + <func> + <name>start(Options) -> {ok, Pid} | {error, Reason}</name> + <fsummary>Start a daemon process</fsummary> + <type> + <v>Options = [option()]</v> + <v>Pid = pid()</v> + <v>Reason = term()</v> + </type> + <desc> + <p>Starts a daemon process which listens for udp packets on a + port. When it receives a request for read or write it spawns + a temporary server process which handles the actual transfer + of the (virtual) file.</p> + </desc> + </func> + <func> + <name>read_file(RemoteFilename, LocalFilename, Options) -> {ok, LastCallbackState} | {error, Reason}</name> + <fsummary>Read a (virtual) file from a TFTP server</fsummary> + <type> + <v>RemoteFilename = string()</v> + <v>LocalFilename = binary | string()</v> + <v>Options = [option()]</v> + <v>LastCallbackState = term()</v> + <v>Reason = term()</v> + </type> + <desc> + <p>Reads a (virtual) file <c>RemoteFilename</c> from a TFTP + server.</p> + <p>If <c>LocalFilename</c> is the atom <c>binary</c>, + <c>tftp_binary</c> is used as callback module. It concatenates + all transferred blocks and returns them as one single binary + in <c>LastCallbackState</c>.</p> + <p>If <c>LocalFilename</c> is a string and there are no + registered callback modules, <c>tftp_file</c> is used as + callback module. It writes each transferred block to the file + named <c>LocalFilename</c> and returns the number of + transferred bytes in <c>LastCallbackState</c>.</p> + <p>If <c>LocalFilename</c> is a string and there are registered + callback modules, <c>LocalFilename</c> is tested against + the regexps of these and the callback module corresponding to + the first match is used, or an error tuple is returned if no + matching regexp is found.</p> + </desc> + </func> + <func> + <name>write_file(RemoteFilename, LocalFilename, Options) -> {ok, LastCallbackState} | {error, Reason}</name> + <fsummary>Write a (virtual) file to a TFTP server</fsummary> + <type> + <v>RemoteFilename = string()</v> + <v>LocalFilename = binary() | string()</v> + <v>Options = [option()]</v> + <v>LastCallbackState = term()</v> + <v>Reason = term()</v> + </type> + <desc> + <p>Writes a (virtual) file <c>RemoteFilename</c> to a TFTP + server.</p> + <p>If <c>LocalFilename</c> is a binary, <c>tftp_binary</c> is + used as callback module. The binary is transferred block by + block and the number of transferred bytes is returned in + <c>LastCallbackState</c>.</p> + <p>If <c>LocalFilename</c> is a string and there are no + registered callback modules, <c>tftp_file</c> is used as + callback module. It reads the file named <c>LocalFilename</c> + block by block and returns the number of transferred bytes + in <c>LastCallbackState</c>.</p> + <p>If <c>LocalFilename</c> is a string and there are registered + callback modules, <c>LocalFilename</c> is tested against + the regexps of these and the callback module corresponding to + the first match is used, or an error tuple is returned if no + matching regexp is found.</p> + </desc> + </func> + + <func> + <name>info(daemons) -> [{Pid, Options}]</name> + <fsummary>Return information about all daemons</fsummary> + <type> + <v>Pid = [pid()()]</v> + <v>Options = [option()]</v> + <v>Reason = term()</v> + </type> + <desc> + <p>Returns info about all TFTP daemon processes. + </p> + </desc> + </func> + + <func> + <name>info(servers) -> [{Pid, Options}]</name> + <fsummary>Return information about all servers</fsummary> + <type> + <v>Pid = [pid()()]</v> + <v>Options = [option()]</v> + <v>Reason = term()</v> + </type> + <desc> + <p>Returns info about all TFTP server processes. + </p> + </desc> + </func> + + <func> + <name>info(Pid) -> {ok, Options} | {error, Reason}</name> + <fsummary>Return information about a daemon, server or client process</fsummary> + <type> + <v>Options = [option()]</v> + <v>Reason = term()</v> + </type> + <desc> + <p>Returns info about a TFTP daemon, server or client process.</p> + </desc> + </func> + + <func> + <name>change_config(daemons, Options) -> [{Pid, Result}]</name> + <fsummary>Changes config for all daemons + </fsummary> + <type> + <v>Options = [option()]</v> + <v>Pid = pid()</v> + <v>Result = ok | {error, Reason}</v> + <v>Reason = term()</v> + </type> + <desc> + <p>Changes config for all TFTP daemon processes + </p> + </desc> + </func> + + <func> + <name>change_config(servers, Options) -> [{Pid, Result}]</name> + <fsummary>Changes config for all servers + </fsummary> + <type> + <v>Options = [option()]</v> + <v>Pid = pid()</v> + <v>Result = ok | {error, Reason}</v> + <v>Reason = term()</v> + </type> + <desc> + <p>Changes config for all TFTP server processes + </p> + </desc> + </func> + + <func> + <name>change_config(Pid, Options) -> Result</name> + <fsummary>Changes config for a TFTP daemon, server or client process</fsummary> + <type> + <v>Pid = pid()</v> + <v>Options = [option()]</v> + <v>Result = ok | {error, Reason}</v> + <v>Reason = term()</v> + </type> + <desc> + <p>Changes config for a TFTP daemon, server or client process</p> + </desc> + </func> + <func> + <name>start() -> ok | {error, Reason}</name> + <fsummary>Start the Inets application</fsummary> + <type> + <v>Reason = term()</v> + </type> + <desc> + <p>Starts the Inets application.</p> + </desc> + </func> + </funcs> + + <section> + <marker id="tftp_callback"></marker> + <title>CALLBACK FUNCTIONS</title> + <p>A <c>tftp</c> callback module should be implemented as a + <c>tftp</c> behavior and export the functions listed below.</p> + <p>On the server side the callback interaction starts with a call to + <c>open/5</c> with the registered initial callback state. + <c>open/5</c> is expected to open the (virtual) file. Then either + the <c>read/1</c> or <c>write/2</c> functions are invoked + repeatedly, once per transferred block. At each function call + the state returned from the previous call is obtained. When + the last block has been encountered the <c>read/1</c> or + <c>write/2</c> functions is expected to close the (virtual) file + and return its last state. The <c>abort/3</c> function is only + used in error situations. <c>prepare/5</c> is not used on + the server side.</p> + <p>On the client side the callback interaction is the same, but it + starts and ends a bit differently. It starts with a call to + <c>prepare/5</c> with the same arguments as <c>open/5</c> takes. + <c>prepare/5</c> is expected to validate the TFTP options, + suggested by the user and return the subset of them that it + accepts. Then the options is sent to the server which will perform + the same TFTP option negotiation procedure. The options that are + accepted by the server are forwarded to the <c>open/5</c> function + on the client side. On the client side the <c>open/5</c> function + must accept all option as is or reject the transfer. Then + the callback interaction follows the same pattern as described + above for the server side. When the last block is encountered in + <c>read/1</c> or <c>write/2</c> the returned state is forwarded to + the user and returned from <c>read_file</c>/3 or + <c>write_file/3</c>.</p> + + <p> If a callback (which performs the file access + in the TFTP server) takes too long time (more than + the double TFTP timeout), the server will abort the + connection and send an error reply to the client. + This implies that the server will release resources + attached to the connection faster than before. The + server simply assumes that the client has given + up.</p> + + <p>If the TFTP server receives yet another request from + the same client (same host and port) while it + already has an active connection to the client, it + will simply ignore the new request if the request is + equal with the first one (same filename and options). + This implies that the (new) client will be served + by the already ongoing connection on the server + side. By not setting up yet another connection, in + parallel with the ongoing one, the server will + consumer lesser resources. + </p> + </section> + + <funcs> + <func> + <name>prepare(Peer, Access, Filename, Mode, SuggestedOptions, InitialState) -> {ok, AcceptedOptions, NewState} | {error, {Code, Text}}</name> + <fsummary>Prepare to open a file on the client side</fsummary> + <type> + <v>Peer = {PeerType, PeerHost, PeerPort}</v> + <v>PeerType = inet | inet6</v> + <v>PeerHost = ip_address()</v> + <v>PeerPort = integer()</v> + <v>Access = read | write</v> + <v>Filename = string()</v> + <v>Mode = string()</v> + <v>SuggestedOptions = AcceptedOptions = [{Key, Value}]</v> + <v> Key = Value = string()</v> + <v>InitialState = [] | [{root_dir, string()}]</v> + <v>NewState = term()</v> + <v>Code = undef | enoent | eacces | enospc</v> + <v> | badop | eexist | baduser | badopt</v> + <v> | int()</v> + <v>Text = string()</v> + </type> + <desc> + <p>Prepares to open a file on the client side.</p> + <p>No new options may be added, but the ones that are present in + <c>SuggestedOptions</c> may be omitted or replaced with new + values in <c>AcceptedOptions</c>.</p> + <p>Will be followed by a call to <c>open/4</c> before any + read/write access is performed. <c>AcceptedOptions</c> is + sent to the server which replies with those options that it + accepts. These will be forwarded to <c>open/4</c> as + <c>SuggestedOptions</c>.</p> + </desc> + </func> + <func> + <name>open(Peer, Access, Filename, Mode, SuggestedOptions, State) -> {ok, AcceptedOptions, NewState} | {error, {Code, Text}}</name> + <fsummary>Open a file for read or write access</fsummary> + <type> + <v>Peer = {PeerType, PeerHost, PeerPort}</v> + <v>PeerType = inet | inet6</v> + <v>PeerHost = ip_address()</v> + <v>PeerPort = integer()</v> + <v>Access = read | write</v> + <v>Filename = string()</v> + <v>Mode = string()</v> + <v>SuggestedOptions = AcceptedOptions = [{Key, Value}]</v> + <v> Key = Value = string()</v> + <v>State = InitialState | term()</v> + <v> InitialState = [] | [{root_dir, string()}]</v> + <v>NewState = term()</v> + <v>Code = undef | enoent | eacces | enospc</v> + <v> | badop | eexist | baduser | badopt</v> + <v> | int()</v> + <v>Text = string()</v> + </type> + <desc> + <p>Opens a file for read or write access.</p> + <p>On the client side where the <c>open/5</c> call has been + preceded by a call to <c>prepare/5</c>, all options must be + accepted or rejected.</p> + <p>On the server side, where there is no preceding + <c>prepare/5</c> call, no new options may be added, but + the ones that are present in <c>SuggestedOptions</c> may be + omitted or replaced with new values in <c>AcceptedOptions</c>.</p> + </desc> + </func> + <func> + <name>read(State) -> {more, Bin, NewState} | {last, Bin, FileSize} | {error, {Code, Text}}</name> + <fsummary>Read a chunk from the file</fsummary> + <type> + <v>State = NewState = term()</v> + <v>Bin = binary()</v> + <v>FileSize = int()</v> + <v>Code = undef | enoent | eacces | enospc</v> + <v> | badop | eexist | baduser | badopt</v> + <v> | int()</v> + <v>Text = string()</v> + </type> + <desc> + <p>Read a chunk from the file.</p> + <p>The callback function is expected to close + the file when the last file chunk is + encountered. When an error is encountered + the callback function is expected to clean + up after the aborted file transfer, such as + closing open file descriptors etc. In both + cases there will be no more calls to any of + the callback functions.</p> + </desc> + </func> + <func> + <name>write(Bin, State) -> {more, NewState} | {last, FileSize} | {error, {Code, Text}}</name> + <fsummary>Write a chunk to the file</fsummary> + <type> + <v>Bin = binary()</v> + <v>State = NewState = term()</v> + <v>FileSize = int()</v> + <v>Code = undef | enoent | eacces | enospc</v> + <v> | badop | eexist | baduser | badopt</v> + <v> | int()</v> + <v>Text = string()</v> + </type> + <desc> + <p>Write a chunk to the file.</p> + <p>The callback function is expected to close + the file when the last file chunk is + encountered. When an error is encountered + the callback function is expected to clean + up after the aborted file transfer, such as + closing open file descriptors etc. In both + cases there will be no more calls to any of + the callback functions.</p> + </desc> + </func> + <func> + <name>abort(Code, Text, State) -> ok</name> + <fsummary>Abort the file transfer</fsummary> + <type> + <v>Code = undef | enoent | eacces | enospc</v> + <v> | badop | eexist | baduser | badopt</v> + <v> | int()</v> + <v>Text = string()</v> + <v>State = term()</v> + </type> + <desc> + <p>Invoked when the file transfer is aborted.</p> + <p>The callback function is expected to clean + up its used resources after the aborted file + transfer, such as closing open file + descriptors etc. The function will not be + invoked if any of the other callback + functions returns an error, as it is + expected that they already have cleaned up + the necessary resources. It will however be + invoked if the functions fails (crashes).</p> + </desc> + </func> + </funcs> + + <section> + <marker id="tftp_logger"></marker> + <title>LOGGER FUNCTIONS</title> + + <p>A <c>tftp_logger</c> callback module should be implemented as a + <c>tftp_logger</c> behavior and export the functions listed below.</p> + </section> + + <funcs> + <func> + <name>error_msg(Format, Data) -> ok | exit(Reason)</name> + <fsummary>Log an error message</fsummary> + <type> + <v>Format = string()</v> + <v>Data = [term()]</v> + <v>Reason = term()</v> + </type> + <desc> + <p>Log an error message. See <c>error_logger:error_msg/2 for details.</c> </p> + </desc> + </func> + + <func> + <name>warning_msg(Format, Data) -> ok | exit(Reason)</name> + <fsummary>Log an error message</fsummary> + <type> + <v>Format = string()</v> + <v>Data = [term()]</v> + <v>Reason = term()</v> + </type> + <desc> + <p>Log a warning message. See <c>error_logger:warning_msg/2 for details.</c> </p> + </desc> + </func> + + <func> + <name>info_msg(Format, Data) -> ok | exit(Reason)</name> + <fsummary>Log an error message</fsummary> + <type> + <v>Format = string()</v> + <v>Data = [term()]</v> + <v>Reason = term()</v> + </type> + <desc> + <p>Log an info message. See <c>error_logger:info_msg/2 for details.</c> </p> + </desc> + </func> + </funcs> +</erlref> + + diff --git a/lib/inets/doc/src/user_guide.gif b/lib/inets/doc/src/user_guide.gif Binary files differnew file mode 100644 index 0000000000..e6275a803d --- /dev/null +++ b/lib/inets/doc/src/user_guide.gif diff --git a/lib/inets/doc/src/warning.gif b/lib/inets/doc/src/warning.gif Binary files differnew file mode 100644 index 0000000000..96af52360e --- /dev/null +++ b/lib/inets/doc/src/warning.gif diff --git a/lib/inets/ebin/.gitignore b/lib/inets/ebin/.gitignore new file mode 100644 index 0000000000..e69de29bb2 --- /dev/null +++ b/lib/inets/ebin/.gitignore diff --git a/lib/inets/examples/Makefile b/lib/inets/examples/Makefile new file mode 100644 index 0000000000..a42b0e38b6 --- /dev/null +++ b/lib/inets/examples/Makefile @@ -0,0 +1,209 @@ +# +# %CopyrightBegin% +# +# Copyright Ericsson AB 1997-2009. All Rights Reserved. +# +# The contents of this file are subject to the Erlang Public License, +# Version 1.1, (the "License"); you may not use this file except in +# compliance with the License. You should have received a copy of the +# Erlang Public License along with this software. If not, it can be +# retrieved online at http://www.erlang.org/. +# +# Software distributed under the License is distributed on an "AS IS" +# basis, WITHOUT WARRANTY OF ANY KIND, either express or implied. See +# the License for the specific language governing rights and limitations +# under the License. +# +# %CopyrightEnd% +# +# +include $(ERL_TOP)/make/target.mk +include $(ERL_TOP)/make/$(TARGET)/otp.mk + +# ---------------------------------------------------- +# Application version +# ---------------------------------------------------- +include ../vsn.mk +VSN=$(INETS_VSN) + +# ---------------------------------------------------- +# Release directory specification +# ---------------------------------------------------- +RELSYSDIR = $(RELEASE_PATH)/lib/inets-$(VSN) + +# ---------------------------------------------------- +# Target Specs +# ---------------------------------------------------- +MODULE= + +AUTH_FILES = server_root/auth/group \ + server_root/auth/passwd +CGI_FILES = server_root/cgi-bin/printenv.sh +CONF_FILES = server_root/conf/8080.conf \ + server_root/conf/8888.conf \ + server_root/conf/httpd.conf \ + server_root/conf/ssl.conf \ + server_root/conf/mime.types +OPEN_FILES = server_root/htdocs/open/dummy.html +MNESIA_OPEN_FILES = server_root/htdocs/mnesia_open/dummy.html +MISC_FILES = server_root/htdocs/misc/friedrich.html \ + server_root/htdocs/misc/oech.html +SECRET_FILES = server_root/htdocs/secret/dummy.html +MNESIA_SECRET_FILES = server_root/htdocs/mnesia_secret/dummy.html +HTDOCS_FILES = server_root/htdocs/index.html \ + server_root/htdocs/config.shtml \ + server_root/htdocs/echo.shtml \ + server_root/htdocs/exec.shtml \ + server_root/htdocs/flastmod.shtml \ + server_root/htdocs/fsize.shtml \ + server_root/htdocs/include.shtml +ICON_FILES = server_root/icons/README \ + server_root/icons/a.gif \ + server_root/icons/alert.black.gif \ + server_root/icons/alert.red.gif \ + server_root/icons/apache_pb.gif \ + server_root/icons/back.gif \ + server_root/icons/ball.gray.gif \ + server_root/icons/ball.red.gif \ + server_root/icons/binary.gif \ + server_root/icons/binhex.gif \ + server_root/icons/blank.gif \ + server_root/icons/bomb.gif \ + server_root/icons/box1.gif \ + server_root/icons/box2.gif \ + server_root/icons/broken.gif \ + server_root/icons/burst.gif \ + server_root/icons/button1.gif \ + server_root/icons/button10.gif \ + server_root/icons/button2.gif \ + server_root/icons/button3.gif \ + server_root/icons/button4.gif \ + server_root/icons/button5.gif \ + server_root/icons/button6.gif \ + server_root/icons/button7.gif \ + server_root/icons/button8.gif \ + server_root/icons/button9.gif \ + server_root/icons/buttonl.gif \ + server_root/icons/buttonr.gif \ + server_root/icons/c.gif \ + server_root/icons/comp.blue.gif \ + server_root/icons/comp.gray.gif \ + server_root/icons/compressed.gif \ + server_root/icons/continued.gif \ + server_root/icons/dir.gif \ + server_root/icons/down.gif \ + server_root/icons/dvi.gif \ + server_root/icons/f.gif \ + server_root/icons/folder.gif \ + server_root/icons/folder.open.gif \ + server_root/icons/folder.sec.gif \ + server_root/icons/forward.gif \ + server_root/icons/generic.gif \ + server_root/icons/generic.red.gif \ + server_root/icons/generic.sec.gif \ + server_root/icons/hand.right.gif \ + server_root/icons/hand.up.gif \ + server_root/icons/htdig.gif \ + server_root/icons/icon.sheet.gif \ + server_root/icons/image1.gif \ + server_root/icons/image2.gif \ + server_root/icons/image3.gif \ + server_root/icons/index.gif \ + server_root/icons/layout.gif \ + server_root/icons/left.gif \ + server_root/icons/link.gif \ + server_root/icons/movie.gif \ + server_root/icons/p.gif \ + server_root/icons/patch.gif \ + server_root/icons/pdf.gif \ + server_root/icons/pie0.gif \ + server_root/icons/pie1.gif \ + server_root/icons/pie2.gif \ + server_root/icons/pie3.gif \ + server_root/icons/pie4.gif \ + server_root/icons/pie5.gif \ + server_root/icons/pie6.gif \ + server_root/icons/pie7.gif \ + server_root/icons/pie8.gif \ + server_root/icons/portal.gif \ + server_root/icons/poweredby.gif \ + server_root/icons/ps.gif \ + server_root/icons/quill.gif \ + server_root/icons/right.gif \ + server_root/icons/screw1.gif \ + server_root/icons/screw2.gif \ + server_root/icons/script.gif \ + server_root/icons/sound1.gif \ + server_root/icons/sound2.gif \ + server_root/icons/sphere1.gif \ + server_root/icons/sphere2.gif \ + server_root/icons/star.gif \ + server_root/icons/star_blank.gif \ + server_root/icons/tar.gif \ + server_root/icons/tex.gif \ + server_root/icons/text.gif \ + server_root/icons/transfer.gif \ + server_root/icons/unknown.gif \ + server_root/icons/up.gif \ + server_root/icons/uu.gif \ + server_root/icons/uuencoded.gif \ + server_root/icons/world1.gif \ + server_root/icons/world2.gif + +SSL_FILES = server_root/ssl/ssl_client.pem \ + server_root/ssl/ssl_server.pem + +# ---------------------------------------------------- +# FLAGS +# ---------------------------------------------------- +ERL_COMPILE_FLAGS += + +# ---------------------------------------------------- +# Targets +# ---------------------------------------------------- + +debug opt: + +clean: + +docs: + +# ---------------------------------------------------- +# Release Target +# ---------------------------------------------------- +include $(ERL_TOP)/make/otp_release_targets.mk + +release_spec: opt + $(INSTALL_DIR) $(RELSYSDIR)/examples/server_root/auth + $(INSTALL_DATA) $(AUTH_FILES) $(RELSYSDIR)/examples/server_root/auth + $(INSTALL_DIR) $(RELSYSDIR)/examples/server_root/cgi-bin + $(INSTALL_SCRIPT) $(CGI_FILES) $(RELSYSDIR)/examples/server_root/cgi-bin + $(INSTALL_DIR) $(RELSYSDIR)/examples/server_root/conf + $(INSTALL_DATA) $(CONF_FILES) $(RELSYSDIR)/examples/server_root/conf + $(INSTALL_DIR) $(RELSYSDIR)/examples/server_root/htdocs/open + $(INSTALL_DATA) $(OPEN_FILES) \ + $(RELSYSDIR)/examples/server_root/htdocs/open + $(INSTALL_DIR) $(RELSYSDIR)/examples/server_root/htdocs/mnesia_open + $(INSTALL_DATA) $(MNESIA_OPEN_FILES) \ + $(RELSYSDIR)/examples/server_root/htdocs/mnesia_open + $(INSTALL_DIR) $(RELSYSDIR)/examples/server_root/htdocs/misc + $(INSTALL_DATA) $(MISC_FILES) \ + $(RELSYSDIR)/examples/server_root/htdocs/misc + $(INSTALL_DIR) \ + $(RELSYSDIR)/examples/server_root/htdocs/secret/top_secret + $(INSTALL_DIR) \ + $(RELSYSDIR)/examples/server_root/htdocs/mnesia_secret/top_secret + $(INSTALL_DATA) $(SECRET_FILES) \ + $(RELSYSDIR)/examples/server_root/htdocs/secret + $(INSTALL_DATA) $(MNESIA_SECRET_FILES) \ + $(RELSYSDIR)/examples/server_root/htdocs/mnesia_secret + $(INSTALL_DIR) $(RELSYSDIR)/examples/server_root/htdocs + $(INSTALL_DATA) $(HTDOCS_FILES) $(RELSYSDIR)/examples/server_root/htdocs + $(INSTALL_DIR) $(RELSYSDIR)/examples/server_root/icons + $(INSTALL_DATA) $(ICON_FILES) $(RELSYSDIR)/examples/server_root/icons + $(INSTALL_DIR) $(RELSYSDIR)/examples/server_root/ssl + $(INSTALL_DATA) $(SSL_FILES) $(RELSYSDIR)/examples/server_root/ssl + $(INSTALL_DIR) $(RELSYSDIR)/examples/server_root/logs + +release_docs_spec: + diff --git a/lib/inets/examples/server_root/auth/group b/lib/inets/examples/server_root/auth/group new file mode 100644 index 0000000000..b3da0ccbd3 --- /dev/null +++ b/lib/inets/examples/server_root/auth/group @@ -0,0 +1,3 @@ +group1: one two +group2: two three +group3: three Aladdin diff --git a/lib/inets/examples/server_root/auth/passwd b/lib/inets/examples/server_root/auth/passwd new file mode 100644 index 0000000000..8c980ff547 --- /dev/null +++ b/lib/inets/examples/server_root/auth/passwd @@ -0,0 +1,4 @@ +one:onePassword +two:twoPassword +three:threePassword +Aladdin:AladdinPassword diff --git a/lib/inets/examples/server_root/cgi-bin/printenv.bat b/lib/inets/examples/server_root/cgi-bin/printenv.bat new file mode 100644 index 0000000000..25a49a1536 --- /dev/null +++ b/lib/inets/examples/server_root/cgi-bin/printenv.bat @@ -0,0 +1,9 @@ +@echo off +echo tomrad > c:\cygwin\tmp\hej +echo Content-type: text/html +echo. +echo ^<HTML^> ^<HEAD^> ^<TITLE^>OS Environment^</TITLE^> ^</HEAD^> ^<BODY^>^<PRE^> +set +echo ^</PRE^>^</BODY^>^</HTML^> + + diff --git a/lib/inets/examples/server_root/cgi-bin/printenv.sh b/lib/inets/examples/server_root/cgi-bin/printenv.sh new file mode 100755 index 0000000000..de81de9bde --- /dev/null +++ b/lib/inets/examples/server_root/cgi-bin/printenv.sh @@ -0,0 +1,6 @@ +#!/bin/sh +echo "Content-type: text/html" +echo "" +echo "<HTML> <HEAD> <TITLE>OS Environment</TITLE> </HEAD> <BODY><PRE>" +env +echo "</PRE></BODY></HTML>"
\ No newline at end of file diff --git a/lib/inets/examples/server_root/conf/8080.conf b/lib/inets/examples/server_root/conf/8080.conf new file mode 100644 index 0000000000..48e66f0114 --- /dev/null +++ b/lib/inets/examples/server_root/conf/8080.conf @@ -0,0 +1,79 @@ +Port 8080 +#ServerName your.server.net +SocketType ip_comm +Modules mod_alias mod_auth mod_esi mod_actions mod_cgi mod_include mod_dir mod_get mod_head mod_log mod_disk_log +ServerAdmin [email protected] +ServerRoot /var/tmp/server_root +ErrorLog logs/error_log_8080 +TransferLog logs/access_log_8080 +SecurityLog logs/security_log_8080 +ErrorDiskLog logs/error_disk_log_8080 +ErrorDiskLogSize 200000 10 +TransferDiskLog logs/access_disk_log_8080 +TransferDiskLogSize 200000 10 +SecurityDiskLog logs/security_disk_log +SecurityDiskLogSize 200000 10 +MaxClients 50 +#KeepAlive 5 +#KeepAliveTimeout 10 +DocumentRoot /var/tmp/server_root/htdocs +DirectoryIndex index.html welcome.html +DefaultType text/plain +Alias /icons/ /var/tmp/server_root/icons/ +Alias /pics/ /var/tmp/server_root/icons/ +ScriptAlias /cgi-bin/ /var/tmp/server_root/cgi-bin/ +ScriptAlias /htbin/ /var/tmp/server_root/cgi-bin/ +ErlScriptAlias /cgi-bin/erl httpd_example io +EvalScriptAlias /eval httpd_example io +#Script HEAD /cgi-bin/printenv.sh +#Action image/gif /cgi-bin/printenv.sh + +<Directory /var/tmp/server_root/htdocs/open> +AuthDBType plain +AuthName Open Area +AuthUserFile /var/tmp/server_root/auth/passwd +AuthGroupFile /var/tmp/server_root/auth/group +require user one Aladdin +</Directory> + +<Directory /var/tmp/server_root/htdocs/secret> +AuthDBType plain +AuthName Secret Area +AuthUserFile /var/tmp/server_root/auth/passwd +AuthGroupFile /var/tmp/server_root/auth/group +require group group1 group2 +</Directory> + +<Directory /var/tmp/server_root/htdocs/secret/top_secret> +AuthDBType plain +AuthName Top Secret Area +AuthUserFile /var/tmp/server_root/auth/passwd +AuthGroupFile /var/tmp/server_root/auth/group +require group group3 +</Directory> + +<Directory /var/tmp/server_root/htdocs/mnesia_open> +AuthDBType mnesia +AuthName Open Area +require user one Aladdin +</Directory> + +<Directory /var/tmp/server_root/htdocs/mnesia_secret> +AuthDBType mnesia +AuthName Secret Area +require group group1 group2 +</Directory> + +<Directory /var/tmp/server_root/htdocs/mnesia_secret/top_secret> +AuthDBType mnesia +AuthName Top Secret Area +require group group3 +allow from 130.100.34 130.100.35 +deny from 100.234.22.12 194.100.34.1 130.100.34.25 +SecurityDataFile logs/security_data +SecurityMaxRetries 3 +SecurityBlockTime 10 +SecurityFailExpireTime 1 +SecurityAuthTimeout 1 +SecurityCallbackModule security_callback +</Directory> diff --git a/lib/inets/examples/server_root/conf/8888.conf b/lib/inets/examples/server_root/conf/8888.conf new file mode 100644 index 0000000000..79bb7fcca4 --- /dev/null +++ b/lib/inets/examples/server_root/conf/8888.conf @@ -0,0 +1,63 @@ +Port 8888 +#ServerName your.server.net +SocketType ip_comm +Modules mod_alias mod_auth mod_esi mod_actions mod_cgi mod_include mod_dir mod_get mod_head mod_log mod_disk_log +ServerAdmin [email protected] +ServerRoot /var/tmp/server_root +ErrorLog logs/error_log_8888 +TransferLog logs/access_log_8888 +ErrorDiskLog logs/error_disk_log_8888 +ErrorDiskLogSize 200000 10 +TransferDiskLog logs/access_disk_log_8888 +TransferDiskLogSize 200000 10 +MaxClients 150 +DocumentRoot /var/tmp/server_root/htdocs +DirectoryIndex index.html welcome.html +DefaultType text/plain +Alias /icons/ /var/tmp/server_root/icons/ +Alias /pics/ /var/tmp/server_root/icons/ +ScriptAlias /cgi-bin/ /var/tmp/server_root/cgi-bin/ +ScriptAlias /htbin/ /var/tmp/server_root/cgi-bin/ +ErlScriptAlias /cgi-bin/erl httpd_example io +EvalScriptAlias /eval httpd_example io +#Script HEAD /cgi-bin/printenv.sh +#Action image/gif /cgi-bin/printenv.sh + +<Directory /var/tmp/server_root/htdocs/open> +AuthName Open Area +AuthUserFile /var/tmp/server_root/auth/passwd +AuthGroupFile /var/tmp/server_root/auth/group +require user one Aladdin +</Directory> + +<Directory /var/tmp/server_root/htdocs/secret> +AuthName Secret Area +AuthUserFile /var/tmp/server_root/auth/passwd +AuthGroupFile /var/tmp/server_root/auth/group +require group group1 group2 +</Directory> + +<Directory /var/tmp/server_root/htdocs/secret/top_secret> +AuthName Top Secret Area +AuthUserFile /var/tmp/server_root/auth/passwd +AuthGroupFile /var/tmp/server_root/auth/group +require group group3 +</Directory> + +<Directory /var/tmp/server_root/htdocs/mnesia_open> +AuthName Open Area +AuthMnesiaDB On +require user one Aladdin +</Directory> + +<Directory /var/tmp/server_root/htdocs/mnesia_secret> +AuthName Secret Area +AuthMnesiaDB On +require group group1 group2 +</Directory> + +<Directory /var/tmp/server_root/htdocs/mnesia_secret/top_secret> +AuthName Top Secret Area +AuthMnesiaDB On +require group group3 +</Directory> diff --git a/lib/inets/examples/server_root/conf/httpd.conf b/lib/inets/examples/server_root/conf/httpd.conf new file mode 100644 index 0000000000..8a74ed1afd --- /dev/null +++ b/lib/inets/examples/server_root/conf/httpd.conf @@ -0,0 +1,268 @@ +# +# %CopyrightBegin% +# +# Copyright Ericsson AB 1997-2009. All Rights Reserved. +# +# The contents of this file are subject to the Erlang Public License, +# Version 1.1, (the "License"); you may not use this file except in +# compliance with the License. You should have received a copy of the +# Erlang Public License along with this software. If not, it can be +# retrieved online at http://www.erlang.org/. +# +# Software distributed under the License is distributed on an "AS IS" +# basis, WITHOUT WARRANTY OF ANY KIND, either express or implied. See +# the License for the specific language governing rights and limitations +# under the License. +# +# %CopyrightEnd% +# +# + +# Port: The port the standalone listens to. For ports < 1023, you will +# need httpd to be run as root initially. + +Port 8888 + +# BindAddress: This directive is used to tell the server which IP address +# to listen to. It can either contain "*", an IP address, or a fully +# qualified Internet domain name. +# +# It is also possible to specify the ip-family with the directive. +# There ar three possible value: inet, inet6 and inet6fb4 +# inet: Use IpFamily inet when retreiving the address and +# fail if that does not work. +# inet6: Use IpFamily inet6 when retreiving the address and +# fail if that does not work. +# inet6fb4: First IpFamily inet6 is tried and if that does not work, +# inet is used as fallback. +# Default value for ip-family is inet6fb4 +# +# The syntax is: <address>[|<ip-family>] +# +#BindAddress * +#BindAddress *|inet + + +# ServerName allows you to set a host name which is sent back to clients for +# your server if it's different than the one the program would get (i.e. use +# "www" instead of the host's real name). +# +# Note: You cannot just invent host names and hope they work. The name you +# define here must be a valid DNS name for your host. If you don't understand +# this, ask your network administrator. + +#ServerName your.server.net + +# SocketType is either ip_comm, sockets or ssl. + +SocketType ip_comm + +# Modules: Server run-time plug-in modules written using the Erlang +# Web Server API (EWSAPI). The server API make it easy to add functionality +# to the server. Read more about EWSAPI in the Reference Manual. +# WARNING! Do not tamper with this directive unless you are familiar with +# EWSAPI. + +Modules mod_alias mod_auth mod_esi mod_actions mod_cgi mod_responsecontrol mod_trace mod_range mod_head mod_include mod_dir mod_get mod_log mod_disk_log + +# ServerAdmin: Your address, where problems with the server should be +# e-mailed. + +ServerAdmin [email protected] + +# ServerRoot: The directory the server's config, error, and log files +# are kept in + +ServerRoot /var/tmp/server_root + +# ErrorLog: The location of the error log file. If this does not start +# with /, ServerRoot is prepended to it. + +ErrorLog logs/error_log + +# TransferLog: The location of the transfer log file. If this does not +# start with /, ServerRoot is prepended to it. + +TransferLog logs/access_log + +# SecurityLog: The location of the security log file (mod_security required) +# +SecurityLog logs/security_log + +# ErrorDiskLog: The location of the error log file. If this does not +# start with /, ServerRoot is prepended to it. This log file is managed +# with the disk_log module [See disk_log(3)]. The ErrorDiskLogSize directive +# takes two argument, i.e. MaxBytes and MaxFiles. The wrap log writes at most +# MaxBytes bytes on each file, and it uses MaxFiles files before it wraps, and +# truncates the first file. + +ErrorDiskLog logs/error_disk_log +ErrorDiskLogSize 200000 10 + +# TransferDiskLog: The location of the transfer log file. If this does not +# start with /, ServerRoot is prepended to it. This log file is managed +# with the disk_log module [See disk_log(3)]. The TransferDiskLogSize directive +# takes two argument, i.e. MaxBytes and MaxFiles. The wrap log writes at most +# MaxBytes bytes on each file, and it uses MaxFiles files before it wraps, and +# truncates the first file. + +TransferDiskLog logs/access_disk_log +TransferDiskLogSize 200000 10 + +# SecurityDiskLog: The location of the security log file. If this does not +# start with /, ServerRoot is prepended to it. This log file is managed +# with the disk_log module [See disk_log(3)]. The SecurityDiskLogSize directive +# takes two argument, i.e. MaxBytes and MaxFiles. The wrap log writes at most +# MaxBytes bytes on each file, and it uses MaxFiles files before it wraps, and +# truncates the first file. + +SecurityDiskLog logs/security_disk_log +SecurityDiskLogSize 200000 10 + +# Limit on total number of servers running, i.e., limit on the number +# of clients who can simultaneously connect --- if this limit is ever +# reached, clients will be LOCKED OUT, so it should NOT BE SET TOO LOW. +# It is intended mainly as a brake to keep a runaway server from taking +# the server with it as it spirals down... + +MaxClients 50 + +# KeepAlive set the flag for persistent connections. For peristent connections +# set KeepAlive to on. To use One request per connection set the flag to off +# Note: The value has changed since previous version of INETS. +KeepAlive on + +# KeepAliveTimeout sets the number of seconds before a persistent connection +# times out and closes. +KeepAliveTimeout 10 + +# MaxKeepAliveRequests sets the number of seconds before a persistent connection +# times out and closes. +MaxKeepAliveRequests 10 + + + +# DocumentRoot: The directory out of which you will serve your +# documents. By default, all requests are taken from this directory, but +# symbolic links and aliases may be used to point to other locations. + +DocumentRoot /var/tmp/server_root/htdocs + +# DirectoryIndex: Name of the file or files to use as a pre-written HTML +# directory index. Separate multiple entries with spaces. + +DirectoryIndex index.html welcome.html + +# DefaultType is the default MIME type for documents which the server +# cannot find the type of from filename extensions. + +DefaultType text/plain + +# Aliases: Add here as many aliases as you need (with no limit). The format is +# Alias fakename realname + +Alias /icons/ /var/tmp/server_root/icons/ +Alias /pics/ /var/tmp/server_root/icons/ + +# ScriptAlias: This controls which directories contain server scripts. +# Format: ScriptAlias fakename realname + +ScriptAlias /cgi-bin/ /var/tmp/server_root/cgi-bin/ +ScriptAlias /htbin/ /var/tmp/server_root/cgi-bin/ + +# This directive adds an action, which will activate cgi-script when a +# file is requested using the method of method, which can be one of +# GET, POST and HEAD. It sends the URL and file path of the requested +# document using the standard CGI PATH_INFO and PATH_TRANSLATED +# environment variables. + +#Script HEAD /cgi-bin/printenv.sh + +# This directive adds an action, which will activate cgi-script when a +# file of content type mime-type is requested. It sends the URL and +# file path of the requested document using the standard CGI PATH_INFO +# and PATH_TRANSLATED environment variables. + +#Action image/gif /cgi-bin/printenv.sh + +# ErlScriptAlias: This specifies how "Erl" server scripts are called. +# Format: ErlScriptAlias fakename realname allowed_modules + +ErlScriptAlias /down/erl httpd_example io + +# EvalScriptAlias: This specifies how "Eval" server scripts are called. +# Format: EvalScriptAlias fakename realname allowed_modules + +EvalScriptAlias /eval httpd_example io + +# Point SSLCertificateFile at a PEM encoded certificate. + +SSLCertificateFile /var/tmp/server_root/ssl/ssl_server.pem + +# If the key is not combined with the certificate, use this directive to +# point at the key file. + +SSLCertificateKeyFile /var/tmp/server_root/ssl/ssl_server.pem + +# Set SSLVerifyClient to: +# 0 if no certicate is required +# 1 if the client may present a valid certificate +# 2 if the client must present a valid certificate +# 3 if the client may present a valid certificate but it is not required to +# have a valid CA + +SSLVerifyClient 0 + +# Each directory to which INETS has access, can be configured with respect +# to which services and features are allowed and/or disabled in that +# directory (and its subdirectories). + +<Directory /var/tmp/server_root/htdocs/open> +AuthDBType plain +AuthName Open Area +AuthUserFile /var/tmp/server_root/auth/passwd +AuthGroupFile /var/tmp/server_root/auth/group +require user one Aladdin +</Directory> + +<Directory /var/tmp/server_root/htdocs/secret> +AuthDBType plain +AuthName Secret Area +AuthUserFile /var/tmp/server_root/auth/passwd +AuthGroupFile /var/tmp/server_root/auth/group +require group group1 group2 +</Directory> + +<Directory /var/tmp/server_root/htdocs/secret/top_secret> +AuthDBType plain +AuthName Top Secret Area +AuthUserFile /var/tmp/server_root/auth/passwd +AuthGroupFile /var/tmp/server_root/auth/group +require group group3 +</Directory> + +<Directory /var/tmp/server_root/htdocs/mnesia_open> +AuthDBType mnesia +AuthName Open Area +require user one Aladdin +</Directory> + +<Directory /var/tmp/server_root/htdocs/mnesia_secret> +AuthDBType mnesia +AuthName Secret Area +require group group1 group2 +</Directory> + +<Directory /var/tmp/server_root/htdocs/mnesia_secret/top_secret> +AuthDBType mnesia +AuthName Top Secret Area +require group group3 +allow from 130.100.34 130.100.35 +deny from 100.234.22.12 194.100.34.1 130.100.34.25 +SecurityDataFile logs/security_data +SecurityMaxRetries 3 +SecurityBlockTime 10 +SecurityFailExpireTime 1 +SecurityAuthTimeout 1 +SecurityCallbackModule security_callback +</Directory> diff --git a/lib/inets/examples/server_root/conf/mime.types b/lib/inets/examples/server_root/conf/mime.types new file mode 100644 index 0000000000..d2f81e4e5e --- /dev/null +++ b/lib/inets/examples/server_root/conf/mime.types @@ -0,0 +1,465 @@ +# This is a comment. I love comments. + +# MIME type Extension +application/EDI-Consent +application/EDI-X12 +application/EDIFACT +application/activemessage +application/andrew-inset ez +application/applefile +application/atomicmail +application/batch-SMTP +application/beep+xml +application/cals-1840 +application/commonground +application/cybercash +application/dca-rft +application/dec-dx +application/dvcs +application/eshop +application/http +application/hyperstudio +application/iges +application/index +application/index.cmd +application/index.obj +application/index.response +application/index.vnd +application/iotp +application/ipp +application/isup +application/font-tdpfr +application/mac-binhex40 hqx +application/mac-compactpro cpt +application/macwriteii +application/marc +application/mathematica +application/mathematica-old +application/msword doc +application/news-message-id +application/news-transmission +application/ocsp-request +application/ocsp-response +application/octet-stream bin dms lha lzh exe class so dll +application/oda oda +application/parityfec +application/pdf pdf +application/pgp-encrypted +application/pgp-keys +application/pgp-signature +application/pkcs10 +application/pkcs7-mime +application/pkcs7-signature +application/pkix-cert +application/pkix-crl +application/pkixcmp +application/postscript ai eps ps +application/prs.alvestrand.titrax-sheet +application/prs.cww +application/prs.nprend +application/qsig +application/remote-printing +application/riscos +application/rtf +application/sdp +application/set-payment +application/set-payment-initiation +application/set-registration +application/set-registration-initiation +application/sgml +application/sgml-open-catalog +application/sieve +application/slate +application/smil smi smil +application/timestamp-query +application/timestamp-reply +application/vemmi +application/vnd.3M.Post-it-Notes +application/vnd.FloGraphIt +application/vnd.accpac.simply.aso +application/vnd.accpac.simply.imp +application/vnd.acucobol +application/vnd.aether.imp +application/vnd.anser-web-certificate-issue-initiation +application/vnd.anser-web-funds-transfer-initiation +application/vnd.audiograph +application/vnd.businessobjects +application/vnd.bmi +application/vnd.canon-cpdl +application/vnd.canon-lips +application/vnd.claymore +application/vnd.commerce-battelle +application/vnd.commonspace +application/vnd.comsocaller +application/vnd.contact.cmsg +application/vnd.cosmocaller +application/vnd.cups-postscript +application/vnd.cups-raster +application/vnd.cups-raw +application/vnd.ctc-posml +application/vnd.cybank +application/vnd.dna +application/vnd.dpgraph +application/vnd.dxr +application/vnd.ecdis-update +application/vnd.ecowin.chart +application/vnd.ecowin.filerequest +application/vnd.ecowin.fileupdate +application/vnd.ecowin.series +application/vnd.ecowin.seriesrequest +application/vnd.ecowin.seriesupdate +application/vnd.enliven +application/vnd.epson.esf +application/vnd.epson.msf +application/vnd.epson.quickanime +application/vnd.epson.salt +application/vnd.epson.ssf +application/vnd.ericsson.quickcall +application/vnd.eudora.data +application/vnd.fdf +application/vnd.ffsns +application/vnd.framemaker +application/vnd.fsc.weblaunch +application/vnd.fujitsu.oasys +application/vnd.fujitsu.oasys2 +application/vnd.fujitsu.oasys3 +application/vnd.fujitsu.oasysgp +application/vnd.fujitsu.oasysprs +application/vnd.fujixerox.ddd +application/vnd.fujixerox.docuworks +application/vnd.fujixerox.docuworks.binder +application/vnd.fut-misnet +application/vnd.grafeq +application/vnd.groove-account +application/vnd.groove-identity-message +application/vnd.groove-injector +application/vnd.groove-tool-message +application/vnd.groove-tool-template +application/vnd.groove-vcard +application/vnd.hhe.lesson-player +application/vnd.hp-HPGL +application/vnd.hp-PCL +application/vnd.hp-PCLXL +application/vnd.hp-hpid +application/vnd.hp-hps +application/vnd.httphone +application/vnd.hzn-3d-crossword +application/vnd.ibm.afplinedata +application/vnd.ibm.MiniPay +application/vnd.ibm.modcap +application/vnd.informix-visionary +application/vnd.intercon.formnet +application/vnd.intertrust.digibox +application/vnd.intertrust.nncp +application/vnd.intu.qbo +application/vnd.intu.qfx +application/vnd.irepository.package+xml +application/vnd.is-xpr +application/vnd.japannet-directory-service +application/vnd.japannet-jpnstore-wakeup +application/vnd.japannet-payment-wakeup +application/vnd.japannet-registration +application/vnd.japannet-registration-wakeup +application/vnd.japannet-setstore-wakeup +application/vnd.japannet-verification +application/vnd.japannet-verification-wakeup +application/vnd.koan +application/vnd.lotus-1-2-3 +application/vnd.lotus-approach +application/vnd.lotus-freelance +application/vnd.lotus-notes +application/vnd.lotus-organizer +application/vnd.lotus-screencam +application/vnd.lotus-wordpro +application/vnd.mcd +application/vnd.mediastation.cdkey +application/vnd.meridian-slingshot +application/vnd.mif mif +application/vnd.minisoft-hp3000-save +application/vnd.mitsubishi.misty-guard.trustweb +application/vnd.mobius.daf +application/vnd.mobius.dis +application/vnd.mobius.msl +application/vnd.mobius.plc +application/vnd.mobius.txf +application/vnd.motorola.flexsuite +application/vnd.motorola.flexsuite.adsi +application/vnd.motorola.flexsuite.fis +application/vnd.motorola.flexsuite.gotap +application/vnd.motorola.flexsuite.kmr +application/vnd.motorola.flexsuite.ttc +application/vnd.motorola.flexsuite.wem +application/vnd.mozilla.xul+xml +application/vnd.ms-artgalry +application/vnd.ms-asf +application/vnd.ms-excel xls +application/vnd.ms-lrm +application/vnd.ms-powerpoint ppt +application/vnd.ms-project +application/vnd.ms-tnef +application/vnd.ms-works +application/vnd.mseq +application/vnd.msign +application/vnd.music-niff +application/vnd.musician +application/vnd.netfpx +application/vnd.noblenet-directory +application/vnd.noblenet-sealer +application/vnd.noblenet-web +application/vnd.novadigm.EDM +application/vnd.novadigm.EDX +application/vnd.novadigm.EXT +application/vnd.osa.netdeploy +application/vnd.palm +application/vnd.pg.format +application/vnd.pg.osasli +application/vnd.powerbuilder6 +application/vnd.powerbuilder6-s +application/vnd.powerbuilder7 +application/vnd.powerbuilder7-s +application/vnd.powerbuilder75 +application/vnd.powerbuilder75-s +application/vnd.previewsystems.box +application/vnd.publishare-delta-tree +application/vnd.pvi.ptid1 +application/vnd.pwg-xhtml-print+xml +application/vnd.rapid +application/vnd.s3sms +application/vnd.seemail +application/vnd.shana.informed.formdata +application/vnd.shana.informed.formtemplate +application/vnd.shana.informed.interchange +application/vnd.shana.informed.package +application/vnd.sss-cod +application/vnd.sss-dtf +application/vnd.sss-ntf +application/vnd.street-stream +application/vnd.svd +application/vnd.swiftview-ics +application/vnd.triscape.mxs +application/vnd.trueapp +application/vnd.truedoc +application/vnd.tve-trigger +application/vnd.ufdl +application/vnd.uplanet.alert +application/vnd.uplanet.alert-wbxml +application/vnd.uplanet.bearer-choice-wbxml +application/vnd.uplanet.bearer-choice +application/vnd.uplanet.cacheop +application/vnd.uplanet.cacheop-wbxml +application/vnd.uplanet.channel +application/vnd.uplanet.channel-wbxml +application/vnd.uplanet.list +application/vnd.uplanet.list-wbxml +application/vnd.uplanet.listcmd +application/vnd.uplanet.listcmd-wbxml +application/vnd.uplanet.signal +application/vnd.vcx +application/vnd.vectorworks +application/vnd.vidsoft.vidconference +application/vnd.visio +application/vnd.vividence.scriptfile +application/vnd.wap.sic +application/vnd.wap.slc +application/vnd.wap.wbxml wbxml +application/vnd.wap.wmlc wmlc +application/vnd.wap.wmlscriptc wmlsc +application/vnd.webturbo +application/vnd.wrq-hp3000-labelled +application/vnd.wt.stf +application/vnd.xara +application/vnd.xfdl +application/vnd.yellowriver-custom-menu +application/whoispp-query +application/whoispp-response +application/wita +application/wordperfect5.1 +application/x-bcpio bcpio +application/x-cdlink vcd +application/x-chess-pgn pgn +application/x-compress +application/x-cpio cpio +application/x-csh csh +application/x-director dcr dir dxr +application/x-dvi dvi +application/x-futuresplash spl +application/x-gtar gtar +application/x-gzip +application/x-hdf hdf +application/x-javascript js +application/x-koan skp skd skt skm +application/x-latex latex +application/x-netcdf nc cdf +application/x-sh sh +application/x-shar shar +application/x-shockwave-flash swf +application/x-stuffit sit +application/x-sv4cpio sv4cpio +application/x-sv4crc sv4crc +application/x-tar tar +application/x-tcl tcl +application/x-tex tex +application/x-texinfo texinfo texi +application/x-troff t tr roff +application/x-troff-man man +application/x-troff-me me +application/x-troff-ms ms +application/x-ustar ustar +application/x-wais-source src +application/x400-bp +application/xml +application/xml-dtd +application/xml-external-parsed-entity +application/zip zip +audio/32kadpcm +audio/basic au snd +audio/g.722.1 +audio/l16 +audio/midi mid midi kar +audio/mp4a-latm +audio/mpa-robust +audio/mpeg mpga mp2 mp3 +audio/parityfec +audio/prs.sid +audio/telephone-event +audio/tone +audio/vnd.cisco.nse +audio/vnd.cns.anp1 +audio/vnd.cns.inf1 +audio/vnd.digital-winds +audio/vnd.everad.plj +audio/vnd.lucent.voice +audio/vnd.nortel.vbk +audio/vnd.nuera.ecelp4800 +audio/vnd.nuera.ecelp7470 +audio/vnd.nuera.ecelp9600 +audio/vnd.octel.sbc +audio/vnd.qcelp +audio/vnd.rhetorex.32kadpcm +audio/vnd.vmx.cvsd +audio/x-aiff aif aiff aifc +audio/x-mpegurl m3u +audio/x-pn-realaudio ram rm +audio/x-pn-realaudio-plugin rpm +audio/x-realaudio ra +audio/x-wav wav +chemical/x-pdb pdb +chemical/x-xyz xyz +image/bmp bmp +image/cgm +image/g3fax +image/gif gif +image/ief ief +image/jpeg jpeg jpg jpe +image/naplps +image/png png +image/prs.btif +image/prs.pti +image/tiff tiff tif +image/vnd.cns.inf2 +image/vnd.dwg +image/vnd.dxf +image/vnd.fastbidsheet +image/vnd.fpx +image/vnd.fst +image/vnd.fujixerox.edmics-mmr +image/vnd.fujixerox.edmics-rlc +image/vnd.mix +image/vnd.net-fpx +image/vnd.svf +image/vnd.wap.wbmp wbmp +image/vnd.xiff +image/x-cmu-raster ras +image/x-portable-anymap pnm +image/x-portable-bitmap pbm +image/x-portable-graymap pgm +image/x-portable-pixmap ppm +image/x-rgb rgb +image/x-xbitmap xbm +image/x-xpixmap xpm +image/x-xwindowdump xwd +message/delivery-status +message/disposition-notification +message/external-body +message/http +message/news +message/partial +message/rfc822 +message/s-http +model/iges igs iges +model/mesh msh mesh silo +model/vnd.dwf +model/vnd.flatland.3dml +model/vnd.gdl +model/vnd.gs-gdl +model/vnd.gtw +model/vnd.mts +model/vnd.vtu +model/vrml wrl vrml +multipart/alternative +multipart/appledouble +multipart/byteranges +multipart/digest +multipart/encrypted +multipart/form-data +multipart/header-set +multipart/mixed +multipart/parallel +multipart/related +multipart/report +multipart/signed +multipart/voice-message +text/calendar +text/css css +text/directory +text/enriched +text/html html htm +text/parityfec +text/plain asc txt +text/prs.lines.tag +text/rfc822-headers +text/richtext rtx +text/rtf rtf +text/sgml sgml sgm +text/tab-separated-values tsv +text/t140 +text/uri-list +text/vnd.DMClientScript +text/vnd.IPTC.NITF +text/vnd.IPTC.NewsML +text/vnd.abc +text/vnd.curl +text/vnd.flatland.3dml +text/vnd.fly +text/vnd.fmi.flexstor +text/vnd.in3d.3dml +text/vnd.in3d.spot +text/vnd.latex-z +text/vnd.motorola.reflex +text/vnd.ms-mediapackage +text/vnd.wap.si +text/vnd.wap.sl +text/vnd.wap.wml wml +text/vnd.wap.wmlscript wmls +text/x-setext etx +text/x-server-parsed-html shtml +text/xml xml xsl +text/xml-external-parsed-entity +video/mp4v-es +video/mpeg mpeg mpg mpe +video/parityfec +video/pointer +video/quicktime qt mov +video/vnd.fvt +video/vnd.motorola.video +video/vnd.motorola.videop +video/vnd.mpegurl mxu +video/vnd.mts +video/vnd.nokia.interleaved-multimedia +video/vnd.vivo +video/x-msvideo avi +video/x-sgi-movie movie +x-conference/x-cooltalk ice + + + diff --git a/lib/inets/examples/server_root/conf/ssl.conf b/lib/inets/examples/server_root/conf/ssl.conf new file mode 100644 index 0000000000..8b8c57a98b --- /dev/null +++ b/lib/inets/examples/server_root/conf/ssl.conf @@ -0,0 +1,66 @@ +Port 8088 +#ServerName your.server.net +SocketType ssl +Modules mod_alias mod_auth mod_esi mod_actions mod_cgi mod_include mod_dir mod_get mod_head mod_log mod_disk_log +ServerAdmin [email protected] +ServerRoot /var/tmp/server_root +ErrorLog logs/error_log_8088 +TransferLog logs/access_log_8088 +ErrorDiskLog logs/error_disk_log_8088 +ErrorDiskLogSize 200000 10 +TransferDiskLog logs/access_disk_log_8088 +TransferDiskLogSize 200000 10 +MaxClients 150 +DocumentRoot /var/tmp/server_root/htdocs +DirectoryIndex index.html welcome.html +DefaultType text/plain +Alias /icons/ /var/tmp/server_root/icons/ +Alias /pics/ /var/tmp/server_root/icons/ +ScriptAlias /cgi-bin/ /var/tmp/server_root/cgi-bin/ +ScriptAlias /htbin/ /var/tmp/server_root/cgi-bin/ +ErlScriptAlias /cgi-bin/erl httpd_example io +EvalScriptAlias /eval httpd_example io +SSLCertificateFile /var/tmp/server_root/ssl/ssl_server.pem +SSLCertificateKeyFile /var/tmp/server_root/ssl/ssl_server.pem +SSLVerifyClient 0 +#Script HEAD /cgi-bin/printenv.sh +#Action image/gif /cgi-bin/printenv.sh + +<Directory /var/tmp/server_root/htdocs/open> +AuthName Open Area +AuthUserFile /var/tmp/server_root/auth/passwd +AuthGroupFile /var/tmp/server_root/auth/group +require user one Aladdin +</Directory> + +<Directory /var/tmp/server_root/htdocs/secret> +AuthName Secret Area +AuthUserFile /var/tmp/server_root/auth/passwd +AuthGroupFile /var/tmp/server_root/auth/group +require group group1 group2 +</Directory> + +<Directory /var/tmp/server_root/htdocs/secret/top_secret> +AuthName Top Secret Area +AuthUserFile /var/tmp/server_root/auth/passwd +AuthGroupFile /var/tmp/server_root/auth/group +require group group3 +</Directory> + +<Directory /var/tmp/server_root/htdocs/mnesia_open> +AuthName Open Area +AuthMnesiaDB On +require user one Aladdin +</Directory> + +<Directory /var/tmp/server_root/htdocs/mnesia_secret> +AuthName Secret Area +AuthMnesiaDB On +require group group1 group2 +</Directory> + +<Directory /var/tmp/server_root/htdocs/mnesia_secret/top_secret> +AuthName Top Secret Area +AuthMnesiaDB On +require group group3 +</Directory> diff --git a/lib/inets/examples/server_root/htdocs/config.shtml b/lib/inets/examples/server_root/htdocs/config.shtml new file mode 100644 index 0000000000..107e3ff610 --- /dev/null +++ b/lib/inets/examples/server_root/htdocs/config.shtml @@ -0,0 +1,70 @@ +<HTML> +<HEAD> +<TITLE>/ssi.html (17-Apr-1997)</TITLE> +</HEAD> +<BODY> +<H1>/ssi.html</H1> + +<!-- ************* CONFIG ************* --> + +<!--#config timefmt="%a %b %e %T %Z %Y" sizefmt="abbrev"--> +<!--#config errmsg="[an especially ugly error occurred while processing this directive]"--> + +<!-- ************* INCLUDE ************* --> + +<P>Include /misc/friedrich.html: +<!--#include virtual="/misc/friedrich.html"--> +<P>Include /misc/not_defined.html: <!--#include virtual="/misc/not_defined.html"--> +<P>Include misc/friedrich.html: +<!--#include file="misc/friedrich.html"--> +<P>Include not_defined.html: <!--#include file="not_defined.html"--> + +<P><HR> + +<!-- ************* ECHO ************* --> + +<P>DOCUMENT_NAME: <!--#echo var="DOCUMENT_NAME"--> +<P>DOCUMENT_URI: <!--#echo var="DOCUMENT_URI"--> +<P>QUERY_STRING_UNESCAPED: <!--#echo var="QUERY_STRING_UNESCAPED"--> +<P>DATE_LOCAL: <!--#echo var="DATE_LOCAL"--> +<P>DATE_GMT: <!--#echo var="DATE_GMT"--> +<P>LAST_MODIFIED: <!--#echo var="LAST_MODIFIED"--> +<P>NOT_DEFINED: <!--#echo var="NOT_DEFINED"--> + +<P><HR> + +<!-- ************* FSIZE ************* --> + +<P>Size of index.html: <!--#fsize file="index.html"--> +<P>Size of not_defined.html: <!--#fsize file="not_defined.html"--> +<!--#config sizefmt="bytes"--> +<P>Size of /misc/friedrich.html: <!--#fsize virtual="/misc/friedrich.html"--> +<P>Size of /misc/not_defined.html: <!--#fsize virtual="/misc/not_defined.html"--> + +<P><HR> + +<!-- ************* FLASTMOD ************* --> + +<P>Last modification of index.html: <!--#flastmod file="index.html"--> +<P>Last modification of not_defined.html: <!--#flastmod file="not_defined.html"--> +<P>Last modification of /misc/friedrich.html: <!--#flastmod virtual="/misc/friedrich.html"--> +<P>Last modification of /misc/not_defined.html: <!--#flastmod virtual="/misc/not_defined.html"--> + +<!--#exec cmd="ls"--> +<!--#exec cmd="printenv"--> +<!--#exec cmd="sunemaja"--> + +<!--#exec cgi="/cgi-bin/printenv.sh"--> + +</BODY> +</HTML> + + + + + + + + + + diff --git a/lib/inets/examples/server_root/htdocs/dets_open/dummy.html b/lib/inets/examples/server_root/htdocs/dets_open/dummy.html new file mode 100644 index 0000000000..a6e8a35a04 --- /dev/null +++ b/lib/inets/examples/server_root/htdocs/dets_open/dummy.html @@ -0,0 +1,10 @@ +<HTML> +<HEAD> +<TITLE>/open/dummy.html (17-Apr-1997)</TITLE> +<!-- Created by: Joakim Greben�, 17-Apr-1997 --> +<!-- Changed by: Joakim Greben�, 17-Apr-1997 --> +</HEAD> +<BODY> +<H1>/open/dummy.html</H1> +</BODY> +</HTML> diff --git a/lib/inets/examples/server_root/htdocs/dets_secret/dummy.html b/lib/inets/examples/server_root/htdocs/dets_secret/dummy.html new file mode 100644 index 0000000000..016b04e540 --- /dev/null +++ b/lib/inets/examples/server_root/htdocs/dets_secret/dummy.html @@ -0,0 +1,10 @@ +<HTML> +<HEAD> +<TITLE>/secret/dummy.html (17-Apr-1997)</TITLE> +<!-- Created by: Joakim Greben�, 17-Apr-1997 --> +<!-- Changed by: Joakim Greben�, 17-Apr-1997 --> +</HEAD> +<BODY> +<H1>/secret/dummy.html</H1> +</BODY> +</HTML> diff --git a/lib/inets/examples/server_root/htdocs/dets_secret/top_secret/index.html b/lib/inets/examples/server_root/htdocs/dets_secret/top_secret/index.html new file mode 100644 index 0000000000..34db3d5d1a --- /dev/null +++ b/lib/inets/examples/server_root/htdocs/dets_secret/top_secret/index.html @@ -0,0 +1,9 @@ +<HTML> +<HEAD> +<TITLE>/secret/top_secret/index.html (04-Feb-1998)</TITLE> +<!-- Created by: Mattias Nilsson, 04-Feb-1998 --> +</HEAD> +<BODY> +<H1>/secret/top_secret/index.html</H1> +</BODY> +</HTML> diff --git a/lib/inets/examples/server_root/htdocs/echo.shtml b/lib/inets/examples/server_root/htdocs/echo.shtml new file mode 100644 index 0000000000..141db5be59 --- /dev/null +++ b/lib/inets/examples/server_root/htdocs/echo.shtml @@ -0,0 +1,35 @@ +<HTML> +<HEAD> +<TITLE>/echo.shtml</TITLE> +</HEAD> +<BODY> +<H1>/echo.shtml</H1> + +<P>DOCUMENT_NAME: <!--#echo var="DOCUMENT_NAME"--> + +<P>DOCUMENT_URI: <!--#echo var="DOCUMENT_URI"--> + +<P>QUERY_STRING_UNESCAPED: <!--#echo var="QUERY_STRING_UNESCAPED"--> + +<P>DATE_LOCAL: <!--#echo var="DATE_LOCAL"--> + +<P>DATE_GMT: <!--#echo var="DATE_GMT"--> + +<P>LAST_MODIFIED: <!--#echo var="LAST_MODIFIED"--> + +<P>NOT_DEFINED: <!--#echo var="NOT_DEFINED"--> + +<P>[<A HREF="ssi.html">Back</A>] + +</BODY> +</HTML> + + + + + + + + + + diff --git a/lib/inets/examples/server_root/htdocs/exec.shtml b/lib/inets/examples/server_root/htdocs/exec.shtml new file mode 100644 index 0000000000..97333da898 --- /dev/null +++ b/lib/inets/examples/server_root/htdocs/exec.shtml @@ -0,0 +1,30 @@ +<HTML> +<HEAD> +<TITLE>/exec.shtml</TITLE> +</HEAD> +<BODY> +<H1>/exec.shtml</H1> +<PRE> +<!--#exec cmd="ls"--> +<HR> +<!--#exec cmd="printenv"--> +<HR> +<!--#exec cmd="sunemaja"--> +<HR> +<!--#exec cgi="/cgi-bin/printenv.sh"--> +</PRE> + +<P>[<A HREF="ssi.html">Back</A>] + +</BODY> +</HTML> + + + + + + + + + + diff --git a/lib/inets/examples/server_root/htdocs/flastmod.shtml b/lib/inets/examples/server_root/htdocs/flastmod.shtml new file mode 100644 index 0000000000..d54c36fe50 --- /dev/null +++ b/lib/inets/examples/server_root/htdocs/flastmod.shtml @@ -0,0 +1,29 @@ +<HTML> +<HEAD> +<TITLE>/flastmod.shtml</TITLE> +</HEAD> +<BODY> +<H1>/flastmod.shtml</H1> + +<P>Last modification of index.html: <!--#flastmod file="index.html"--> + +<P>Last modification of not_defined.html: <!--#flastmod file="not_defined.html"--> + +<P>Last modification of /misc/friedrich.html: <!--#flastmod virtual="/misc/friedrich.html"--> + +<P>Last modification of /misc/not_defined.html: <!--#flastmod virtual="/misc/not_defined.html"--> + +<P>[<A HREF="ssi.html">Back</A>] + +</BODY> +</HTML> + + + + + + + + + + diff --git a/lib/inets/examples/server_root/htdocs/fsize.shtml b/lib/inets/examples/server_root/htdocs/fsize.shtml new file mode 100644 index 0000000000..570ee9cf6d --- /dev/null +++ b/lib/inets/examples/server_root/htdocs/fsize.shtml @@ -0,0 +1,29 @@ +<HTML> +<HEAD> +<TITLE>/fsize.shtml</TITLE> +</HEAD> +<BODY> +<H1>/fsize.shtml</H1> + +<P>Size of index.html: <!--#fsize file="index.html"--> + +<P>Size of not_defined.html: <!--#fsize file="not_defined.html"--> + +<P>Size of /misc/friedrich.html: <!--#fsize virtual="/misc/friedrich.html"--> + +<P>Size of /misc/not_defined.html: <!--#fsize virtual="/misc/not_defined.html"--> + +<P>[<A HREF="ssi.html">Back</A>] + +</BODY> +</HTML> + + + + + + + + + + diff --git a/lib/inets/examples/server_root/htdocs/include.shtml b/lib/inets/examples/server_root/htdocs/include.shtml new file mode 100644 index 0000000000..529aad0437 --- /dev/null +++ b/lib/inets/examples/server_root/htdocs/include.shtml @@ -0,0 +1,33 @@ +<HTML> +<HEAD> +<TITLE>/include.shtml</TITLE> +</HEAD> +<BODY> +<H1>/include.shtml</H1> + +<P>Include /misc/friedrich.html: +<!--#include virtual="/misc/friedrich.html"--> + +<P>Include /misc/not_defined.html: +<!--#include virtual="/misc/not_defined.html"--> + +<P>Include misc/friedrich.html: +<!--#include file="misc/friedrich.html"--> + +<P>Include not_defined.html: +<!--#include file="not_defined.html"--> + +<P>[<A HREF="ssi.html">Back</A>] + +</BODY> +</HTML> + + + + + + + + + + diff --git a/lib/inets/examples/server_root/htdocs/index.html b/lib/inets/examples/server_root/htdocs/index.html new file mode 100644 index 0000000000..cfdc9f9ab7 --- /dev/null +++ b/lib/inets/examples/server_root/htdocs/index.html @@ -0,0 +1,25 @@ +<HTML> +<HEAD> +<TITLE>/index.html</TITLE> +</HEAD> +<BODY> +<H1>/index.html</H1> + +<STRONG>Server-Side Include (SSI) commands:</STRONG><BR> +<A HREF="config.shtml">config</A><BR> +<A HREF="echo.shtml">echo</A><BR> +<A HREF="exec.shtml">exec</A><BR> +<A HREF="flastmod.shtml">flastmod</A><BR> +<A HREF="fsize.shtml">fsize</A><BR> +<A HREF="include.shtml">include</A><BR> + +<BR> +<BR> + +<STRONG>ESI callback:</STRING><BR> +<A HREF="cgi-bin/erl/httpd_example/get">cgi-bin/erl/httpd_example/get</A><BR> +<A HREF="cgi-bin/erl/httpd_example/yahoo">cgi-bin/erl/httpd_example/yahoo</A><BR> +<A HREF="cgi-bin/erl/httpd_example/test1">cgi-bin/erl/httpd_example/test1</A><BR> + +</BODY> +</HTML> diff --git a/lib/inets/examples/server_root/htdocs/last_modified.html b/lib/inets/examples/server_root/htdocs/last_modified.html new file mode 100644 index 0000000000..65c1790813 --- /dev/null +++ b/lib/inets/examples/server_root/htdocs/last_modified.html @@ -0,0 +1,22 @@ +<HTML> +<HEAD> +<TITLE>/last_modified.html</TITLE> +</HEAD> +<BODY> +<H1>/last_modified.html</H1> + +<P>This document is only used for test of illegal last-modified date.</P> + + +</BODY> +</HTML> + + + + + + + + + + diff --git a/lib/inets/examples/server_root/htdocs/misc/friedrich.html b/lib/inets/examples/server_root/htdocs/misc/friedrich.html new file mode 100644 index 0000000000..d7953d5df4 --- /dev/null +++ b/lib/inets/examples/server_root/htdocs/misc/friedrich.html @@ -0,0 +1,7 @@ +<P><CITE> +Talking much about oneself can also be a means to conceal oneself.<BR> +-- Friedrich Nietzsche +</CITE> + +<P>Nested Include: +<!--#include file="misc/oech.html"--> diff --git a/lib/inets/examples/server_root/htdocs/misc/oech.html b/lib/inets/examples/server_root/htdocs/misc/oech.html new file mode 100644 index 0000000000..506064bf04 --- /dev/null +++ b/lib/inets/examples/server_root/htdocs/misc/oech.html @@ -0,0 +1,4 @@ +<P><CITE> +What excuses stand in your way? How can you eliminate them?<BR> +-- Roger von Oech +</CITE> diff --git a/lib/inets/examples/server_root/htdocs/misc/welcome.html b/lib/inets/examples/server_root/htdocs/misc/welcome.html new file mode 100644 index 0000000000..8c17451f91 --- /dev/null +++ b/lib/inets/examples/server_root/htdocs/misc/welcome.html @@ -0,0 +1 @@ +<HTML></HTML> diff --git a/lib/inets/examples/server_root/htdocs/mnesia_open/dummy.html b/lib/inets/examples/server_root/htdocs/mnesia_open/dummy.html new file mode 100644 index 0000000000..a6e8a35a04 --- /dev/null +++ b/lib/inets/examples/server_root/htdocs/mnesia_open/dummy.html @@ -0,0 +1,10 @@ +<HTML> +<HEAD> +<TITLE>/open/dummy.html (17-Apr-1997)</TITLE> +<!-- Created by: Joakim Greben�, 17-Apr-1997 --> +<!-- Changed by: Joakim Greben�, 17-Apr-1997 --> +</HEAD> +<BODY> +<H1>/open/dummy.html</H1> +</BODY> +</HTML> diff --git a/lib/inets/examples/server_root/htdocs/mnesia_secret/dummy.html b/lib/inets/examples/server_root/htdocs/mnesia_secret/dummy.html new file mode 100644 index 0000000000..016b04e540 --- /dev/null +++ b/lib/inets/examples/server_root/htdocs/mnesia_secret/dummy.html @@ -0,0 +1,10 @@ +<HTML> +<HEAD> +<TITLE>/secret/dummy.html (17-Apr-1997)</TITLE> +<!-- Created by: Joakim Greben�, 17-Apr-1997 --> +<!-- Changed by: Joakim Greben�, 17-Apr-1997 --> +</HEAD> +<BODY> +<H1>/secret/dummy.html</H1> +</BODY> +</HTML> diff --git a/lib/inets/examples/server_root/htdocs/mnesia_secret/top_secret/index.html b/lib/inets/examples/server_root/htdocs/mnesia_secret/top_secret/index.html new file mode 100644 index 0000000000..2d17e8b596 --- /dev/null +++ b/lib/inets/examples/server_root/htdocs/mnesia_secret/top_secret/index.html @@ -0,0 +1,9 @@ +<HTML> +<HEAD> +<TITLE>/mnesia_secret/top_secret/index.html (04-Feb-1998)</TITLE> +<!-- Created by: Mattias Nilsson, 04-Feb-1998 --> +</HEAD> +<BODY> +<H1>/mnesia_secret/top_secret/index.html</H1> +</BODY> +</HTML> diff --git a/lib/inets/examples/server_root/htdocs/open/dummy.html b/lib/inets/examples/server_root/htdocs/open/dummy.html new file mode 100644 index 0000000000..a6e8a35a04 --- /dev/null +++ b/lib/inets/examples/server_root/htdocs/open/dummy.html @@ -0,0 +1,10 @@ +<HTML> +<HEAD> +<TITLE>/open/dummy.html (17-Apr-1997)</TITLE> +<!-- Created by: Joakim Greben�, 17-Apr-1997 --> +<!-- Changed by: Joakim Greben�, 17-Apr-1997 --> +</HEAD> +<BODY> +<H1>/open/dummy.html</H1> +</BODY> +</HTML> diff --git a/lib/inets/examples/server_root/htdocs/secret/dummy.html b/lib/inets/examples/server_root/htdocs/secret/dummy.html new file mode 100644 index 0000000000..016b04e540 --- /dev/null +++ b/lib/inets/examples/server_root/htdocs/secret/dummy.html @@ -0,0 +1,10 @@ +<HTML> +<HEAD> +<TITLE>/secret/dummy.html (17-Apr-1997)</TITLE> +<!-- Created by: Joakim Greben�, 17-Apr-1997 --> +<!-- Changed by: Joakim Greben�, 17-Apr-1997 --> +</HEAD> +<BODY> +<H1>/secret/dummy.html</H1> +</BODY> +</HTML> diff --git a/lib/inets/examples/server_root/htdocs/secret/top_secret/index.html b/lib/inets/examples/server_root/htdocs/secret/top_secret/index.html new file mode 100644 index 0000000000..34db3d5d1a --- /dev/null +++ b/lib/inets/examples/server_root/htdocs/secret/top_secret/index.html @@ -0,0 +1,9 @@ +<HTML> +<HEAD> +<TITLE>/secret/top_secret/index.html (04-Feb-1998)</TITLE> +<!-- Created by: Mattias Nilsson, 04-Feb-1998 --> +</HEAD> +<BODY> +<H1>/secret/top_secret/index.html</H1> +</BODY> +</HTML> diff --git a/lib/inets/examples/server_root/icons/README b/lib/inets/examples/server_root/icons/README new file mode 100644 index 0000000000..a1fc5a5a9c --- /dev/null +++ b/lib/inets/examples/server_root/icons/README @@ -0,0 +1,161 @@ +Public Domain Icons + + These icons were originally made for Mosaic for X and have been + included in the NCSA httpd and Apache server distributions in the + past. They are in the public domain and may be freely included in any + application. The originals were done by Kevin Hughes ([email protected]). + + Many thanks to Andy Polyakov for tuning the icon colors and adding a + few new images. If you'd like to contribute additions or ideas to + this set, please let me know. + + The distribution site for these icons is at: + + http://www.eit.com/goodies/www.icons/ + + Kevin Hughes + September 11, 1995 + + +Suggested Uses + +The following are a few suggestions, to serve as a starting point for ideas. +Please feel free to tweak and rename the icons as you like. + + a.gif + This might be used to represent PostScript or text layout + languages. + + alert.black.gif, alert.red.gif + These can be used to highlight any important items, such as a + README file in a directory. + + back.gif, forward.gif + These can be used as links to go to previous and next areas. + + ball.gray.gif, ball.red.gif + These might be used as bullets. + + binary.gif + This can be used to represent binary files. + + binhex.gif + This can represent BinHex-encoded data. + + blank.gif + This can be used as a placeholder or a spacing element. + + bomb.gif + This can be used to repreesnt core files. + + box1.gif, box2.gif + These icons can be used to represent generic 3D applications and + related files. + + broken.gif + This can represent corrupted data. + + burst.gif + This can call attention to new and important items. + + c.gif + This might represent C source code. + + comp.blue.gif, comp.red.gif + These little computer icons can stand for telnet or FTP + sessions. + + compressed.gif + This may represent compressed data. + + continued.gif + This can be a link to a continued listing of a directory. + + down.gif, up.gif, left.gif, right.gif + These can be used to scroll up, down, left and right in a + listing or may be used to denote items in an outline. + + dvi.gif + This can represent DVI files. + + f.gif + This might represent FORTRAN or Forth source code. + + folder.gif, folder.open.gif, folder.sec.gif + The folder can represent directories. There is also a version + that can represent secure directories or directories that cannot + be viewed. + + generic.gif, generic.sec.gif, generic.red.gif + These can represent generic files, secure files, and important + files, respectively. + + hand.right.gif, hand.up.gif + These can point out important items (pun intended). + + image1.gif, image2.gif, image3.gif + These can represent image formats of various types. + + index.gif + This might represent a WAIS index or search facility. + + layout.gif + This might represent files and formats that contain graphics as + well as text layout, such as HTML and PDF files. + + link.gif + This might represent files that are symbolic links. + + movie.gif + This can represent various movie formats. + + p.gif + This may stand for Perl or Python source code. + + pie0.gif ... pie8.gif + These icons can be used in applications where a list of + documents is returned from a search. The little pie chart images + can denote how relevant the documents may be to your search + query. + + patch.gif + This may stand for patches and diff files. + + portal.gif + This might be a link to an online service or a 3D world. + + ps.gif, quill.gif + These may represent PostScript files. + + screw1.gif, screw2.gif + These may represent CAD or engineering data and formats. + + script.gif + This can represent any of various interpreted languages, such as + Perl, python, TCL, and shell scripts, as well as server + configuration files. + + sound1.gif, sound2.gif + These can represent sound files. + + sphere1.gif, sphere2.gif + These can represent 3D worlds or rendering applications and + formats. + + tex.gif + This can represent TeX files. + + text.gif + This can represent generic (plain) text files. + + transfer.gif + This can represent FTP transfers or uploads/downloads. + + unknown.gif + This may represent a file of an unknown type. + + uuencoded.gif + This can stand for uuencoded data. + + world1.gif, world2.gif + These can represent 3D worlds or other 3D formats. diff --git a/lib/inets/examples/server_root/icons/a.gif b/lib/inets/examples/server_root/icons/a.gif Binary files differnew file mode 100644 index 0000000000..bb23d971f4 --- /dev/null +++ b/lib/inets/examples/server_root/icons/a.gif diff --git a/lib/inets/examples/server_root/icons/alert.black.gif b/lib/inets/examples/server_root/icons/alert.black.gif Binary files differnew file mode 100644 index 0000000000..eaecd2172a --- /dev/null +++ b/lib/inets/examples/server_root/icons/alert.black.gif diff --git a/lib/inets/examples/server_root/icons/alert.red.gif b/lib/inets/examples/server_root/icons/alert.red.gif Binary files differnew file mode 100644 index 0000000000..a423894043 --- /dev/null +++ b/lib/inets/examples/server_root/icons/alert.red.gif diff --git a/lib/inets/examples/server_root/icons/apache_pb.gif b/lib/inets/examples/server_root/icons/apache_pb.gif Binary files differnew file mode 100644 index 0000000000..3a1c139fc4 --- /dev/null +++ b/lib/inets/examples/server_root/icons/apache_pb.gif diff --git a/lib/inets/examples/server_root/icons/back.gif b/lib/inets/examples/server_root/icons/back.gif Binary files differnew file mode 100644 index 0000000000..a694ae1ec3 --- /dev/null +++ b/lib/inets/examples/server_root/icons/back.gif diff --git a/lib/inets/examples/server_root/icons/ball.gray.gif b/lib/inets/examples/server_root/icons/ball.gray.gif Binary files differnew file mode 100644 index 0000000000..eb84268c4c --- /dev/null +++ b/lib/inets/examples/server_root/icons/ball.gray.gif diff --git a/lib/inets/examples/server_root/icons/ball.red.gif b/lib/inets/examples/server_root/icons/ball.red.gif Binary files differnew file mode 100644 index 0000000000..a8425cb574 --- /dev/null +++ b/lib/inets/examples/server_root/icons/ball.red.gif diff --git a/lib/inets/examples/server_root/icons/binary.gif b/lib/inets/examples/server_root/icons/binary.gif Binary files differnew file mode 100644 index 0000000000..9a15cbae04 --- /dev/null +++ b/lib/inets/examples/server_root/icons/binary.gif diff --git a/lib/inets/examples/server_root/icons/binhex.gif b/lib/inets/examples/server_root/icons/binhex.gif Binary files differnew file mode 100644 index 0000000000..62d0363108 --- /dev/null +++ b/lib/inets/examples/server_root/icons/binhex.gif diff --git a/lib/inets/examples/server_root/icons/blank.gif b/lib/inets/examples/server_root/icons/blank.gif Binary files differnew file mode 100644 index 0000000000..0ccf01e198 --- /dev/null +++ b/lib/inets/examples/server_root/icons/blank.gif diff --git a/lib/inets/examples/server_root/icons/bomb.gif b/lib/inets/examples/server_root/icons/bomb.gif Binary files differnew file mode 100644 index 0000000000..270fdb1c06 --- /dev/null +++ b/lib/inets/examples/server_root/icons/bomb.gif diff --git a/lib/inets/examples/server_root/icons/box1.gif b/lib/inets/examples/server_root/icons/box1.gif Binary files differnew file mode 100644 index 0000000000..65dcd002ea --- /dev/null +++ b/lib/inets/examples/server_root/icons/box1.gif diff --git a/lib/inets/examples/server_root/icons/box2.gif b/lib/inets/examples/server_root/icons/box2.gif Binary files differnew file mode 100644 index 0000000000..c43bc4faec --- /dev/null +++ b/lib/inets/examples/server_root/icons/box2.gif diff --git a/lib/inets/examples/server_root/icons/broken.gif b/lib/inets/examples/server_root/icons/broken.gif Binary files differnew file mode 100644 index 0000000000..9f8cbe9f76 --- /dev/null +++ b/lib/inets/examples/server_root/icons/broken.gif diff --git a/lib/inets/examples/server_root/icons/burst.gif b/lib/inets/examples/server_root/icons/burst.gif Binary files differnew file mode 100644 index 0000000000..fbdcf575f7 --- /dev/null +++ b/lib/inets/examples/server_root/icons/burst.gif diff --git a/lib/inets/examples/server_root/icons/button1.gif b/lib/inets/examples/server_root/icons/button1.gif Binary files differnew file mode 100644 index 0000000000..eb97cb7333 --- /dev/null +++ b/lib/inets/examples/server_root/icons/button1.gif diff --git a/lib/inets/examples/server_root/icons/button10.gif b/lib/inets/examples/server_root/icons/button10.gif Binary files differnew file mode 100644 index 0000000000..fe0c97998c --- /dev/null +++ b/lib/inets/examples/server_root/icons/button10.gif diff --git a/lib/inets/examples/server_root/icons/button2.gif b/lib/inets/examples/server_root/icons/button2.gif Binary files differnew file mode 100644 index 0000000000..7698455bf9 --- /dev/null +++ b/lib/inets/examples/server_root/icons/button2.gif diff --git a/lib/inets/examples/server_root/icons/button3.gif b/lib/inets/examples/server_root/icons/button3.gif Binary files differnew file mode 100644 index 0000000000..a8b8319232 --- /dev/null +++ b/lib/inets/examples/server_root/icons/button3.gif diff --git a/lib/inets/examples/server_root/icons/button4.gif b/lib/inets/examples/server_root/icons/button4.gif Binary files differnew file mode 100644 index 0000000000..0fd15a0d7f --- /dev/null +++ b/lib/inets/examples/server_root/icons/button4.gif diff --git a/lib/inets/examples/server_root/icons/button5.gif b/lib/inets/examples/server_root/icons/button5.gif Binary files differnew file mode 100644 index 0000000000..64241e5c5d --- /dev/null +++ b/lib/inets/examples/server_root/icons/button5.gif diff --git a/lib/inets/examples/server_root/icons/button6.gif b/lib/inets/examples/server_root/icons/button6.gif Binary files differnew file mode 100644 index 0000000000..867cfd1212 --- /dev/null +++ b/lib/inets/examples/server_root/icons/button6.gif diff --git a/lib/inets/examples/server_root/icons/button7.gif b/lib/inets/examples/server_root/icons/button7.gif Binary files differnew file mode 100644 index 0000000000..b3f5fb248f --- /dev/null +++ b/lib/inets/examples/server_root/icons/button7.gif diff --git a/lib/inets/examples/server_root/icons/button8.gif b/lib/inets/examples/server_root/icons/button8.gif Binary files differnew file mode 100644 index 0000000000..7a308be8f6 --- /dev/null +++ b/lib/inets/examples/server_root/icons/button8.gif diff --git a/lib/inets/examples/server_root/icons/button9.gif b/lib/inets/examples/server_root/icons/button9.gif Binary files differnew file mode 100644 index 0000000000..9acba576c0 --- /dev/null +++ b/lib/inets/examples/server_root/icons/button9.gif diff --git a/lib/inets/examples/server_root/icons/buttonl.gif b/lib/inets/examples/server_root/icons/buttonl.gif Binary files differnew file mode 100644 index 0000000000..3883088e7a --- /dev/null +++ b/lib/inets/examples/server_root/icons/buttonl.gif diff --git a/lib/inets/examples/server_root/icons/buttonr.gif b/lib/inets/examples/server_root/icons/buttonr.gif Binary files differnew file mode 100644 index 0000000000..c4dc3887db --- /dev/null +++ b/lib/inets/examples/server_root/icons/buttonr.gif diff --git a/lib/inets/examples/server_root/icons/c.gif b/lib/inets/examples/server_root/icons/c.gif Binary files differnew file mode 100644 index 0000000000..7555b6c164 --- /dev/null +++ b/lib/inets/examples/server_root/icons/c.gif diff --git a/lib/inets/examples/server_root/icons/comp.blue.gif b/lib/inets/examples/server_root/icons/comp.blue.gif Binary files differnew file mode 100644 index 0000000000..f8d76a8c23 --- /dev/null +++ b/lib/inets/examples/server_root/icons/comp.blue.gif diff --git a/lib/inets/examples/server_root/icons/comp.gray.gif b/lib/inets/examples/server_root/icons/comp.gray.gif Binary files differnew file mode 100644 index 0000000000..7664cd0364 --- /dev/null +++ b/lib/inets/examples/server_root/icons/comp.gray.gif diff --git a/lib/inets/examples/server_root/icons/compressed.gif b/lib/inets/examples/server_root/icons/compressed.gif Binary files differnew file mode 100644 index 0000000000..39e732739f --- /dev/null +++ b/lib/inets/examples/server_root/icons/compressed.gif diff --git a/lib/inets/examples/server_root/icons/continued.gif b/lib/inets/examples/server_root/icons/continued.gif Binary files differnew file mode 100644 index 0000000000..b0ffb7e0cc --- /dev/null +++ b/lib/inets/examples/server_root/icons/continued.gif diff --git a/lib/inets/examples/server_root/icons/dir.gif b/lib/inets/examples/server_root/icons/dir.gif Binary files differnew file mode 100644 index 0000000000..48264601ae --- /dev/null +++ b/lib/inets/examples/server_root/icons/dir.gif diff --git a/lib/inets/examples/server_root/icons/down.gif b/lib/inets/examples/server_root/icons/down.gif Binary files differnew file mode 100644 index 0000000000..a354c871cd --- /dev/null +++ b/lib/inets/examples/server_root/icons/down.gif diff --git a/lib/inets/examples/server_root/icons/dvi.gif b/lib/inets/examples/server_root/icons/dvi.gif Binary files differnew file mode 100644 index 0000000000..791be33105 --- /dev/null +++ b/lib/inets/examples/server_root/icons/dvi.gif diff --git a/lib/inets/examples/server_root/icons/f.gif b/lib/inets/examples/server_root/icons/f.gif Binary files differnew file mode 100644 index 0000000000..fbe353c282 --- /dev/null +++ b/lib/inets/examples/server_root/icons/f.gif diff --git a/lib/inets/examples/server_root/icons/folder.gif b/lib/inets/examples/server_root/icons/folder.gif Binary files differnew file mode 100644 index 0000000000..48264601ae --- /dev/null +++ b/lib/inets/examples/server_root/icons/folder.gif diff --git a/lib/inets/examples/server_root/icons/folder.open.gif b/lib/inets/examples/server_root/icons/folder.open.gif Binary files differnew file mode 100644 index 0000000000..30979cb528 --- /dev/null +++ b/lib/inets/examples/server_root/icons/folder.open.gif diff --git a/lib/inets/examples/server_root/icons/folder.sec.gif b/lib/inets/examples/server_root/icons/folder.sec.gif Binary files differnew file mode 100644 index 0000000000..75332d9e59 --- /dev/null +++ b/lib/inets/examples/server_root/icons/folder.sec.gif diff --git a/lib/inets/examples/server_root/icons/forward.gif b/lib/inets/examples/server_root/icons/forward.gif Binary files differnew file mode 100644 index 0000000000..b2959b4c85 --- /dev/null +++ b/lib/inets/examples/server_root/icons/forward.gif diff --git a/lib/inets/examples/server_root/icons/generic.gif b/lib/inets/examples/server_root/icons/generic.gif Binary files differnew file mode 100644 index 0000000000..de60b2940f --- /dev/null +++ b/lib/inets/examples/server_root/icons/generic.gif diff --git a/lib/inets/examples/server_root/icons/generic.red.gif b/lib/inets/examples/server_root/icons/generic.red.gif Binary files differnew file mode 100644 index 0000000000..94743981d9 --- /dev/null +++ b/lib/inets/examples/server_root/icons/generic.red.gif diff --git a/lib/inets/examples/server_root/icons/generic.sec.gif b/lib/inets/examples/server_root/icons/generic.sec.gif Binary files differnew file mode 100644 index 0000000000..88d5240c3c --- /dev/null +++ b/lib/inets/examples/server_root/icons/generic.sec.gif diff --git a/lib/inets/examples/server_root/icons/hand.right.gif b/lib/inets/examples/server_root/icons/hand.right.gif Binary files differnew file mode 100644 index 0000000000..5cdbc7206d --- /dev/null +++ b/lib/inets/examples/server_root/icons/hand.right.gif diff --git a/lib/inets/examples/server_root/icons/hand.up.gif b/lib/inets/examples/server_root/icons/hand.up.gif Binary files differnew file mode 100644 index 0000000000..85a5d68317 --- /dev/null +++ b/lib/inets/examples/server_root/icons/hand.up.gif diff --git a/lib/inets/examples/server_root/icons/htdig.gif b/lib/inets/examples/server_root/icons/htdig.gif Binary files differnew file mode 100644 index 0000000000..35443fb63a --- /dev/null +++ b/lib/inets/examples/server_root/icons/htdig.gif diff --git a/lib/inets/examples/server_root/icons/icon.sheet.gif b/lib/inets/examples/server_root/icons/icon.sheet.gif Binary files differnew file mode 100644 index 0000000000..ad1686e448 --- /dev/null +++ b/lib/inets/examples/server_root/icons/icon.sheet.gif diff --git a/lib/inets/examples/server_root/icons/image1.gif b/lib/inets/examples/server_root/icons/image1.gif Binary files differnew file mode 100644 index 0000000000..01e442bfa9 --- /dev/null +++ b/lib/inets/examples/server_root/icons/image1.gif diff --git a/lib/inets/examples/server_root/icons/image2.gif b/lib/inets/examples/server_root/icons/image2.gif Binary files differnew file mode 100644 index 0000000000..751faeea36 --- /dev/null +++ b/lib/inets/examples/server_root/icons/image2.gif diff --git a/lib/inets/examples/server_root/icons/image3.gif b/lib/inets/examples/server_root/icons/image3.gif Binary files differnew file mode 100644 index 0000000000..4f30484ff6 --- /dev/null +++ b/lib/inets/examples/server_root/icons/image3.gif diff --git a/lib/inets/examples/server_root/icons/index.gif b/lib/inets/examples/server_root/icons/index.gif Binary files differnew file mode 100644 index 0000000000..162478fb3a --- /dev/null +++ b/lib/inets/examples/server_root/icons/index.gif diff --git a/lib/inets/examples/server_root/icons/layout.gif b/lib/inets/examples/server_root/icons/layout.gif Binary files differnew file mode 100644 index 0000000000..c96338a152 --- /dev/null +++ b/lib/inets/examples/server_root/icons/layout.gif diff --git a/lib/inets/examples/server_root/icons/left.gif b/lib/inets/examples/server_root/icons/left.gif Binary files differnew file mode 100644 index 0000000000..279e6710d4 --- /dev/null +++ b/lib/inets/examples/server_root/icons/left.gif diff --git a/lib/inets/examples/server_root/icons/link.gif b/lib/inets/examples/server_root/icons/link.gif Binary files differnew file mode 100644 index 0000000000..c5b6889a76 --- /dev/null +++ b/lib/inets/examples/server_root/icons/link.gif diff --git a/lib/inets/examples/server_root/icons/movie.gif b/lib/inets/examples/server_root/icons/movie.gif Binary files differnew file mode 100644 index 0000000000..0035183774 --- /dev/null +++ b/lib/inets/examples/server_root/icons/movie.gif diff --git a/lib/inets/examples/server_root/icons/p.gif b/lib/inets/examples/server_root/icons/p.gif Binary files differnew file mode 100644 index 0000000000..7b917b4e91 --- /dev/null +++ b/lib/inets/examples/server_root/icons/p.gif diff --git a/lib/inets/examples/server_root/icons/patch.gif b/lib/inets/examples/server_root/icons/patch.gif Binary files differnew file mode 100644 index 0000000000..39bc90e795 --- /dev/null +++ b/lib/inets/examples/server_root/icons/patch.gif diff --git a/lib/inets/examples/server_root/icons/pdf.gif b/lib/inets/examples/server_root/icons/pdf.gif Binary files differnew file mode 100644 index 0000000000..c88fd777c4 --- /dev/null +++ b/lib/inets/examples/server_root/icons/pdf.gif diff --git a/lib/inets/examples/server_root/icons/pie0.gif b/lib/inets/examples/server_root/icons/pie0.gif Binary files differnew file mode 100644 index 0000000000..6f7a0ae7a7 --- /dev/null +++ b/lib/inets/examples/server_root/icons/pie0.gif diff --git a/lib/inets/examples/server_root/icons/pie1.gif b/lib/inets/examples/server_root/icons/pie1.gif Binary files differnew file mode 100644 index 0000000000..03aa6be71e --- /dev/null +++ b/lib/inets/examples/server_root/icons/pie1.gif diff --git a/lib/inets/examples/server_root/icons/pie2.gif b/lib/inets/examples/server_root/icons/pie2.gif Binary files differnew file mode 100644 index 0000000000..b04c5e0908 --- /dev/null +++ b/lib/inets/examples/server_root/icons/pie2.gif diff --git a/lib/inets/examples/server_root/icons/pie3.gif b/lib/inets/examples/server_root/icons/pie3.gif Binary files differnew file mode 100644 index 0000000000..4db9d023ed --- /dev/null +++ b/lib/inets/examples/server_root/icons/pie3.gif diff --git a/lib/inets/examples/server_root/icons/pie4.gif b/lib/inets/examples/server_root/icons/pie4.gif Binary files differnew file mode 100644 index 0000000000..93471fdd88 --- /dev/null +++ b/lib/inets/examples/server_root/icons/pie4.gif diff --git a/lib/inets/examples/server_root/icons/pie5.gif b/lib/inets/examples/server_root/icons/pie5.gif Binary files differnew file mode 100644 index 0000000000..57aee93f07 --- /dev/null +++ b/lib/inets/examples/server_root/icons/pie5.gif diff --git a/lib/inets/examples/server_root/icons/pie6.gif b/lib/inets/examples/server_root/icons/pie6.gif Binary files differnew file mode 100644 index 0000000000..0dc327b569 --- /dev/null +++ b/lib/inets/examples/server_root/icons/pie6.gif diff --git a/lib/inets/examples/server_root/icons/pie7.gif b/lib/inets/examples/server_root/icons/pie7.gif Binary files differnew file mode 100644 index 0000000000..8661337f06 --- /dev/null +++ b/lib/inets/examples/server_root/icons/pie7.gif diff --git a/lib/inets/examples/server_root/icons/pie8.gif b/lib/inets/examples/server_root/icons/pie8.gif Binary files differnew file mode 100644 index 0000000000..59ddb34ce0 --- /dev/null +++ b/lib/inets/examples/server_root/icons/pie8.gif diff --git a/lib/inets/examples/server_root/icons/portal.gif b/lib/inets/examples/server_root/icons/portal.gif Binary files differnew file mode 100644 index 0000000000..0e6e506e00 --- /dev/null +++ b/lib/inets/examples/server_root/icons/portal.gif diff --git a/lib/inets/examples/server_root/icons/poweredby.gif b/lib/inets/examples/server_root/icons/poweredby.gif Binary files differnew file mode 100644 index 0000000000..d324ab80ea --- /dev/null +++ b/lib/inets/examples/server_root/icons/poweredby.gif diff --git a/lib/inets/examples/server_root/icons/ps.gif b/lib/inets/examples/server_root/icons/ps.gif Binary files differnew file mode 100644 index 0000000000..0f565bc1db --- /dev/null +++ b/lib/inets/examples/server_root/icons/ps.gif diff --git a/lib/inets/examples/server_root/icons/quill.gif b/lib/inets/examples/server_root/icons/quill.gif Binary files differnew file mode 100644 index 0000000000..818a5cdc7e --- /dev/null +++ b/lib/inets/examples/server_root/icons/quill.gif diff --git a/lib/inets/examples/server_root/icons/right.gif b/lib/inets/examples/server_root/icons/right.gif Binary files differnew file mode 100644 index 0000000000..b256e5f75f --- /dev/null +++ b/lib/inets/examples/server_root/icons/right.gif diff --git a/lib/inets/examples/server_root/icons/screw1.gif b/lib/inets/examples/server_root/icons/screw1.gif Binary files differnew file mode 100644 index 0000000000..af6ba2b097 --- /dev/null +++ b/lib/inets/examples/server_root/icons/screw1.gif diff --git a/lib/inets/examples/server_root/icons/screw2.gif b/lib/inets/examples/server_root/icons/screw2.gif Binary files differnew file mode 100644 index 0000000000..06dccb3e44 --- /dev/null +++ b/lib/inets/examples/server_root/icons/screw2.gif diff --git a/lib/inets/examples/server_root/icons/script.gif b/lib/inets/examples/server_root/icons/script.gif Binary files differnew file mode 100644 index 0000000000..d8a853bc58 --- /dev/null +++ b/lib/inets/examples/server_root/icons/script.gif diff --git a/lib/inets/examples/server_root/icons/sound1.gif b/lib/inets/examples/server_root/icons/sound1.gif Binary files differnew file mode 100644 index 0000000000..8efb49f55d --- /dev/null +++ b/lib/inets/examples/server_root/icons/sound1.gif diff --git a/lib/inets/examples/server_root/icons/sound2.gif b/lib/inets/examples/server_root/icons/sound2.gif Binary files differnew file mode 100644 index 0000000000..48e6a7fb2f --- /dev/null +++ b/lib/inets/examples/server_root/icons/sound2.gif diff --git a/lib/inets/examples/server_root/icons/sphere1.gif b/lib/inets/examples/server_root/icons/sphere1.gif Binary files differnew file mode 100644 index 0000000000..7067070da2 --- /dev/null +++ b/lib/inets/examples/server_root/icons/sphere1.gif diff --git a/lib/inets/examples/server_root/icons/sphere2.gif b/lib/inets/examples/server_root/icons/sphere2.gif Binary files differnew file mode 100644 index 0000000000..a9e462a377 --- /dev/null +++ b/lib/inets/examples/server_root/icons/sphere2.gif diff --git a/lib/inets/examples/server_root/icons/star.gif b/lib/inets/examples/server_root/icons/star.gif Binary files differnew file mode 100644 index 0000000000..4cfe0a5e0f --- /dev/null +++ b/lib/inets/examples/server_root/icons/star.gif diff --git a/lib/inets/examples/server_root/icons/star_blank.gif b/lib/inets/examples/server_root/icons/star_blank.gif Binary files differnew file mode 100644 index 0000000000..a0c83cb85b --- /dev/null +++ b/lib/inets/examples/server_root/icons/star_blank.gif diff --git a/lib/inets/examples/server_root/icons/tar.gif b/lib/inets/examples/server_root/icons/tar.gif Binary files differnew file mode 100644 index 0000000000..617e779efa --- /dev/null +++ b/lib/inets/examples/server_root/icons/tar.gif diff --git a/lib/inets/examples/server_root/icons/tex.gif b/lib/inets/examples/server_root/icons/tex.gif Binary files differnew file mode 100644 index 0000000000..45e43233b8 --- /dev/null +++ b/lib/inets/examples/server_root/icons/tex.gif diff --git a/lib/inets/examples/server_root/icons/text.gif b/lib/inets/examples/server_root/icons/text.gif Binary files differnew file mode 100644 index 0000000000..4c623909fb --- /dev/null +++ b/lib/inets/examples/server_root/icons/text.gif diff --git a/lib/inets/examples/server_root/icons/transfer.gif b/lib/inets/examples/server_root/icons/transfer.gif Binary files differnew file mode 100644 index 0000000000..33697dbb66 --- /dev/null +++ b/lib/inets/examples/server_root/icons/transfer.gif diff --git a/lib/inets/examples/server_root/icons/unknown.gif b/lib/inets/examples/server_root/icons/unknown.gif Binary files differnew file mode 100644 index 0000000000..32b1ea23fb --- /dev/null +++ b/lib/inets/examples/server_root/icons/unknown.gif diff --git a/lib/inets/examples/server_root/icons/up.gif b/lib/inets/examples/server_root/icons/up.gif Binary files differnew file mode 100644 index 0000000000..6d6d6d1ebf --- /dev/null +++ b/lib/inets/examples/server_root/icons/up.gif diff --git a/lib/inets/examples/server_root/icons/uu.gif b/lib/inets/examples/server_root/icons/uu.gif Binary files differnew file mode 100644 index 0000000000..4387d529f6 --- /dev/null +++ b/lib/inets/examples/server_root/icons/uu.gif diff --git a/lib/inets/examples/server_root/icons/uuencoded.gif b/lib/inets/examples/server_root/icons/uuencoded.gif Binary files differnew file mode 100644 index 0000000000..4387d529f6 --- /dev/null +++ b/lib/inets/examples/server_root/icons/uuencoded.gif diff --git a/lib/inets/examples/server_root/icons/world1.gif b/lib/inets/examples/server_root/icons/world1.gif Binary files differnew file mode 100644 index 0000000000..05b4ec2058 --- /dev/null +++ b/lib/inets/examples/server_root/icons/world1.gif diff --git a/lib/inets/examples/server_root/icons/world2.gif b/lib/inets/examples/server_root/icons/world2.gif Binary files differnew file mode 100644 index 0000000000..e3203f7a88 --- /dev/null +++ b/lib/inets/examples/server_root/icons/world2.gif diff --git a/lib/inets/examples/server_root/logs/Dummy_File_Needed_By_WinZip b/lib/inets/examples/server_root/logs/Dummy_File_Needed_By_WinZip new file mode 100644 index 0000000000..8d1c8b69c3 --- /dev/null +++ b/lib/inets/examples/server_root/logs/Dummy_File_Needed_By_WinZip @@ -0,0 +1 @@ + diff --git a/lib/inets/examples/server_root/ssl/ssl_client.pem b/lib/inets/examples/server_root/ssl/ssl_client.pem new file mode 100644 index 0000000000..8221139eb4 --- /dev/null +++ b/lib/inets/examples/server_root/ssl/ssl_client.pem @@ -0,0 +1,22 @@ +-----BEGIN RSA PRIVATE KEY----- +MIIBPAIBAAJBAL6Ym/bgUvhhnPkw08sggGg8Tnp759ThGMEjkmDzhuJ3w3PfnF65 +mgHcgunku4G6LxAQfEUougJWf9Phmjj3oRUCAwEAAQJBAKMjvVvzZxFzfAlP4flc +OI0AEayFokp04dtvtzuFN09f+aBo2dP18xHmKLCZvxrBOaRAROoQYscALiIVpN07 +GAECIQDfi+sSfAFaDlT3vzpL3xE5UEH6IzY8jWpaZfM1QaToJQIhANpEF50H4wGO +8Sbh7dUutNd+s+NYUjsMySW2DjLKMsoxAiEAzzb2ftrdsempD0F+O0gZwiPIFKLB +Kp33YLYyHEKuJtUCIDGi+pvDh2R7VWw6RRQOIyI+tjolg83aAoSI+oGiahqBAiEA +xzmNNajwoaokvWvlaz0na8rhxu45grOvDrflBT9XvSQ= +-----END RSA PRIVATE KEY----- +-----BEGIN CERTIFICATE----- +MIICDDCCAbYCAQAwDQYJKoZIhvcNAQEEBQAwgZAxCzAJBgNVBAYTAlNFMRIwEAYD +VQQIEwlTdG9ja2hvbG0xDzANBgNVBAcTBkFsdnNqbzEMMAoGA1UEChMDRVRYMQ4w +DAYDVQQLEwVETi9TUDEXMBUGA1UEAxMOSm9ha2ltIEdyZWJlbm8xJTAjBgkqhkiG +9w0BCQEWFmpvY2tlQGVyaXguZXJpY3Nzb24uc2UwHhcNOTcwNzE1MTUzNDM2WhcN +MDMwMjIyMTUzNDM2WjCBkDELMAkGA1UEBhMCU0UxEjAQBgNVBAgTCVN0b2NraG9s +bTEPMA0GA1UEBxMGQWx2c2pvMQwwCgYDVQQKEwNFVFgxDjAMBgNVBAsTBUROL1NQ +MRcwFQYDVQQDEw5Kb2FraW0gR3JlYmVubzElMCMGCSqGSIb3DQEJARYWam9ja2VA +ZXJpeC5lcmljc3Nvbi5zZTBcMA0GCSqGSIb3DQEBAQUAA0sAMEgCQQC+mJv24FL4 +YZz5MNPLIIBoPE56e+fU4RjBI5Jg84bid8Nz35xeuZoB3ILp5LuBui8QEHxFKLoC +Vn/T4Zo496EVAgMBAAEwDQYJKoZIhvcNAQEEBQADQQBYxQVfTydyZCE0UXvZd7Ei +josNsAaWJk9fFIJaG9uyXCEfg2dVgoT2eBk3D9DI+7OB+78isM5CVlFbL7hilvP8 +-----END CERTIFICATE----- diff --git a/lib/inets/examples/server_root/ssl/ssl_server.pem b/lib/inets/examples/server_root/ssl/ssl_server.pem new file mode 100644 index 0000000000..fe739c15f7 --- /dev/null +++ b/lib/inets/examples/server_root/ssl/ssl_server.pem @@ -0,0 +1,22 @@ +-----BEGIN RSA PRIVATE KEY----- +MIIBOwIBAAJBAL9Bozj3BIjL5Cy8b3rjMT2kPZRychX4wz9bHoIIiKnKo1xXHYjw +g3N9zWM1f1ZzMADwVry1uAInA8q09+7hL20CAwEAAQJACwu2ao7RozjrV64WXimK +6X131P/7GMvCMwGHNIlbozqoOqmZcYrbKaF61l+XuwA2QvTo3ywW1Ivxcyr6TeAr +PQIhAOX+WXT6yiqqwjt08kjBCJyMgfZtdAO6pc/6pKjNWiZfAiEA1OH1iPW/OQe5 +tlQXpiRVdLyneNsPygPRJc4Bdwu3hbMCIQDbI5pA56QxOzqOREOGJsb5wrciAfAE +jZbnr72sSN2YqQIgAWFpvzagw9Tp/mWzNY+cwkIK7/yzsIKv04fveH8p9IMCIQCr +td4IiukeUwXmPSvYM4uCE/+J89wEL9qU8Mlc3gDLXA== +-----END RSA PRIVATE KEY----- +-----BEGIN CERTIFICATE----- +MIICDDCCAbYCAQAwDQYJKoZIhvcNAQEEBQAwgZAxCzAJBgNVBAYTAlNFMRIwEAYD +VQQIEwlTdG9ja2hvbG0xDzANBgNVBAcTBkFsdnNqbzEMMAoGA1UEChMDRVRYMQ4w +DAYDVQQLEwVETi9TUDEXMBUGA1UEAxMOSm9ha2ltIEdyZWJlbm8xJTAjBgkqhkiG +9w0BCQEWFmpvY2tlQGVyaXguZXJpY3Nzb24uc2UwHhcNOTcwNzE1MTUzMzQxWhcN +MDMwMjIyMTUzMzQxWjCBkDELMAkGA1UEBhMCU0UxEjAQBgNVBAgTCVN0b2NraG9s +bTEPMA0GA1UEBxMGQWx2c2pvMQwwCgYDVQQKEwNFVFgxDjAMBgNVBAsTBUROL1NQ +MRcwFQYDVQQDEw5Kb2FraW0gR3JlYmVubzElMCMGCSqGSIb3DQEJARYWam9ja2VA +ZXJpeC5lcmljc3Nvbi5zZTBcMA0GCSqGSIb3DQEBAQUAA0sAMEgCQQC/QaM49wSI +y+QsvG964zE9pD2UcnIV+MM/Wx6CCIipyqNcVx2I8INzfc1jNX9WczAA8Fa8tbgC +JwPKtPfu4S9tAgMBAAEwDQYJKoZIhvcNAQEEBQADQQAmXDY1CyJjzvQZX442kkHG +ic9QFY1UuVfzokzNMwlHYl1Qx9zaodx0cJCrcH5GF9O9LJbhhV77LzoxT1Q5wZp5 +-----END CERTIFICATE----- diff --git a/lib/inets/info b/lib/inets/info new file mode 100644 index 0000000000..079a271fe2 --- /dev/null +++ b/lib/inets/info @@ -0,0 +1,2 @@ +group: comm +short: A set of services such as a Web server and a ftp client etc. diff --git a/lib/inets/priv/Makefile b/lib/inets/priv/Makefile new file mode 100644 index 0000000000..85da409e55 --- /dev/null +++ b/lib/inets/priv/Makefile @@ -0,0 +1,64 @@ +# +# %CopyrightBegin% +# +# Copyright Ericsson AB 1997-2009. All Rights Reserved. +# +# The contents of this file are subject to the Erlang Public License, +# Version 1.1, (the "License"); you may not use this file except in +# compliance with the License. You should have received a copy of the +# Erlang Public License along with this software. If not, it can be +# retrieved online at http://www.erlang.org/. +# +# Software distributed under the License is distributed on an "AS IS" +# basis, WITHOUT WARRANTY OF ANY KIND, either express or implied. See +# the License for the specific language governing rights and limitations +# under the License. +# +# %CopyrightEnd% +# +# +include $(ERL_TOP)/make/target.mk +include $(ERL_TOP)/make/$(TARGET)/otp.mk + +# ---------------------------------------------------- +# Application version +# ---------------------------------------------------- +include ../vsn.mk +VSN=$(INETS_VSN) + +# ---------------------------------------------------- +# Release directory specification +# ---------------------------------------------------- +RELSYSDIR = $(RELEASE_PATH)/lib/inets-$(VSN) + +# ---------------------------------------------------- +# Target Specs +# ---------------------------------------------------- +EXECUTABLES=bin/runcgi.sh + +# ---------------------------------------------------- +# FLAGS +# ---------------------------------------------------- +ERL_COMPILE_FLAGS += + +# ---------------------------------------------------- +# Targets +# ---------------------------------------------------- + +debug opt: + +clean: + +docs: + +# ---------------------------------------------------- +# Release Target +# ---------------------------------------------------- +include $(ERL_TOP)/make/otp_release_targets.mk + +release_spec: opt + $(INSTALL_DIR) $(RELSYSDIR)/priv/bin + $(INSTALL_SCRIPT) $(EXECUTABLES) $(RELSYSDIR)/priv/bin + +release_docs_spec: + diff --git a/lib/inets/priv/bin/runcgi.sh b/lib/inets/priv/bin/runcgi.sh new file mode 100755 index 0000000000..8c71ab9802 --- /dev/null +++ b/lib/inets/priv/bin/runcgi.sh @@ -0,0 +1,4 @@ +#!/bin/sh +cd $1 +shift +exec env "$@" diff --git a/lib/inets/src/Makefile b/lib/inets/src/Makefile new file mode 100644 index 0000000000..dd18e92107 --- /dev/null +++ b/lib/inets/src/Makefile @@ -0,0 +1,35 @@ +# +# %CopyrightBegin% +# +# Copyright Ericsson AB 1996-2009. All Rights Reserved. +# +# The contents of this file are subject to the Erlang Public License, +# Version 1.1, (the "License"); you may not use this file except in +# compliance with the License. You should have received a copy of the +# Erlang Public License along with this software. If not, it can be +# retrieved online at http://www.erlang.org/. +# +# Software distributed under the License is distributed on an "AS IS" +# basis, WITHOUT WARRANTY OF ANY KIND, either express or implied. See +# the License for the specific language governing rights and limitations +# under the License. +# +# %CopyrightEnd% +# +# +include $(ERL_TOP)/make/target.mk +include $(ERL_TOP)/make/$(TARGET)/otp.mk + +# ---------------------------------------------------- +# Common Macros +# ---------------------------------------------------- + +include subdirs.mk + +SPECIAL_TARGETS = + +# ---------------------------------------------------- +# Default Subdir Targets +# ---------------------------------------------------- +include $(ERL_TOP)/make/otp_subdir.mk + diff --git a/lib/inets/src/ftp/Makefile b/lib/inets/src/ftp/Makefile new file mode 100644 index 0000000000..70d51115e6 --- /dev/null +++ b/lib/inets/src/ftp/Makefile @@ -0,0 +1,99 @@ +# +# %CopyrightBegin% +# +# Copyright Ericsson AB 2005-2009. All Rights Reserved. +# +# The contents of this file are subject to the Erlang Public License, +# Version 1.1, (the "License"); you may not use this file except in +# compliance with the License. You should have received a copy of the +# Erlang Public License along with this software. If not, it can be +# retrieved online at http://www.erlang.org/. +# +# Software distributed under the License is distributed on an "AS IS" +# basis, WITHOUT WARRANTY OF ANY KIND, either express or implied. See +# the License for the specific language governing rights and limitations +# under the License. +# +# %CopyrightEnd% +# +# + +include $(ERL_TOP)/make/target.mk +EBIN = ../../ebin +include $(ERL_TOP)/make/$(TARGET)/otp.mk + +# ---------------------------------------------------- +# Application version +# ---------------------------------------------------- +include ../../vsn.mk + +VSN = $(INETS_VSN) + +# ---------------------------------------------------- +# Release directory specification +# ---------------------------------------------------- +RELSYSDIR = $(RELEASE_PATH)/lib/inets-$(VSN) + +# ---------------------------------------------------- +# Target Specs +# ---------------------------------------------------- +MODULES = \ + ftp \ + ftp_progress \ + ftp_response \ + ftp_sup + +HRL_FILES = ftp_internal.hrl + +ERL_FILES = $(MODULES:%=%.erl) + +TARGET_FILES= $(MODULES:%=$(EBIN)/%.$(EMULATOR)) + +# ---------------------------------------------------- +# INETS FLAGS +# ---------------------------------------------------- +INETS_FLAGS = -D'SERVER_SOFTWARE="inets/$(VSN)"' + +ifeq ($(FTP_DEBUG),true) + INETS_FLAGS += -Dftp_debug +endif + +# ---------------------------------------------------- +# FLAGS +# ---------------------------------------------------- +INETS_ERL_FLAGS += -I ../inets_app -pa ../../ebin + +ERL_COMPILE_FLAGS += $(INETS_ERL_FLAGS) \ + $(INETS_FLAGS) \ + +'{parse_transform,sys_pre_attributes}' \ + +'{attribute,insert,app_vsn,$(APP_VSN)}' + +# ---------------------------------------------------- +# Targets +# ---------------------------------------------------- + +debug opt: $(TARGET_FILES) + +clean: + rm -f $(TARGET_FILES) + rm -f core + +docs: + +# ---------------------------------------------------- +# Release Target +# ---------------------------------------------------- +include $(ERL_TOP)/make/otp_release_targets.mk + +release_spec: opt + $(INSTALL_DIR) $(RELSYSDIR)/src + $(INSTALL_DATA) $(HRL_FILES) $(ERL_FILES) $(RELSYSDIR)/src + $(INSTALL_DIR) $(RELSYSDIR)/ebin + $(INSTALL_DATA) $(TARGET_FILES) $(RELSYSDIR)/ebin + +release_docs_spec: + +info: + @echo "INETS_DEBUG = $(INETS_DEBUG)" + @echo "INETS_FLAGS = $(INETS_FLAGS)" + @echo "ERL_COMPILE_FLAGS = $(ERL_COMPILE_FLAGS)" diff --git a/lib/inets/src/ftp/ftp.erl b/lib/inets/src/ftp/ftp.erl new file mode 100644 index 0000000000..534fcae675 --- /dev/null +++ b/lib/inets/src/ftp/ftp.erl @@ -0,0 +1,2009 @@ +%% +%% %CopyrightBegin% +%% +%% Copyright Ericsson AB 1997-2009. All Rights Reserved. +%% +%% The contents of this file are subject to the Erlang Public License, +%% Version 1.1, (the "License"); you may not use this file except in +%% compliance with the License. You should have received a copy of the +%% Erlang Public License along with this software. If not, it can be +%% retrieved online at http://www.erlang.org/. +%% +%% Software distributed under the License is distributed on an "AS IS" +%% basis, WITHOUT WARRANTY OF ANY KIND, either express or implied. See +%% the License for the specific language governing rights and limitations +%% under the License. +%% +%% %CopyrightEnd% +%% +%% +%% Description: This module implements an ftp client, RFC 959. +%% It also supports ipv6 RFC 2428. + +-module(ftp). + +-behaviour(gen_server). +-behaviour(inets_service). + +-deprecated({open, 3, next_major_release}). +-deprecated({force_active, 1, next_major_release}). + +%% API - Client interface +-export([cd/2, close/1, delete/2, formaterror/1, + lcd/2, lpwd/1, ls/1, ls/2, + mkdir/2, nlist/1, nlist/2, + open/1, open/2, open/3, force_active/1, + pwd/1, quote/2, + recv/2, recv/3, recv_bin/2, + recv_chunk_start/2, recv_chunk/1, + rename/3, rmdir/2, + send/2, send/3, send_bin/3, + send_chunk_start/2, send_chunk/2, send_chunk_end/1, + type/2, user/3, user/4, account/2, + append/3, append/2, append_bin/3, + append_chunk/2, append_chunk_end/1, append_chunk_start/2, info/1]). + +%% gen_server callbacks +-export([init/1, handle_call/3, handle_cast/2, + handle_info/2, terminate/2, code_change/3]). + +%% supervisor callbacks +-export([start_link/1, start_link/2]). + +%% Behavior callbacks +-export([start_standalone/1, start_service/1, + stop_service/1, services/0, service_info/1]). + +-include("ftp_internal.hrl"). + +%% Constante used in internal state definition +-define(CONNECTION_TIMEOUT, 60*1000). +-define(DEFAULT_MODE, passive). +-define(PROGRESS_DEFAULT, ignore). + +%% Internal Constants +-define(FTP_PORT, 21). +-define(FILE_BUFSIZE, 4096). + +%% Internal state +-record(state, { + csock = undefined, % socket() - Control connection socket + dsock = undefined, % socket() - Data connection socket + verbose = false, % boolean() + ldir = undefined, % string() - Current local directory + type = ftp_server_default, % atom() - binary | ascii + chunk = false, % boolean() - Receiving data chunks + mode = ?DEFAULT_MODE, % passive | active + timeout = ?CONNECTION_TIMEOUT, % integer() + %% Data received so far on the data connection + data = <<>>, % binary() + %% Data received so far on the control connection + %% {BinStream, AccLines}. If a binary sequence + %% ends with ?CR then keep it in the binary to + %% be able to detect if the next received byte is ?LF + %% and hence the end of the response is reached! + ctrl_data = {<<>>, [], start}, % {binary(), [bytes()], LineStatus} + %% pid() - Client pid (note not the same as "From") + owner = undefined, + client = undefined, % "From" to be used in gen_server:reply/2 + %% Function that activated a connection and maybe some + %% data needed further on. + caller = undefined, % term() + ipfamily, % inet | inet6 | inet6fb4 + progress = ignore % ignore | pid() + }). + + +%%%========================================================================= +%%% API - CLIENT FUNCTIONS +%%%========================================================================= + +%%-------------------------------------------------------------------------- +%% open(HostOrOtpList, <Port>, <Flags>) -> {ok, Pid} | {error, ehost} +%% HostOrOtpList = string() | [{option_list, Options}] +%% Port = integer(), +%% Flags = [Flag], +%% Flag = verbose | debug | trace +%% +%% Description: Start an ftp client and connect to a host. +%%-------------------------------------------------------------------------- + +%% <BACKWARD-COMPATIBILLITY> +open({option_list, Options}) when is_list(Options) -> + try + {ok, StartOptions} = start_options(Options), + {ok, OpenOptions} = open_options(Options), + case ftp_sup:start_child([[[{client, self()} | StartOptions], []]]) of + {ok, Pid} -> + call(Pid, {open, ip_comm, OpenOptions}, plain); + Error1 -> + Error1 + end + catch + throw:Error2 -> + Error2 + end; +%% </BACKWARD-COMPATIBILLITY> + +open(Host) -> + open(Host, []). + +%% <BACKWARD-COMPATIBILLITY> +open(Host, Port) when is_integer(Port) -> + open(Host, [{port, Port}]); +%% </BACKWARD-COMPATIBILLITY> + +%% <BACKWARD-COMPATIBILLITY> +open(Host, [H|_] = Flags) when is_atom(H) -> + open(Host, ?FTP_PORT, Flags); +%% </BACKWARD-COMPATIBILLITY> + +open(Host, Opts) when is_list(Opts) -> + ?fcrt("open", [{host, Host}, {opts, Opts}]), + try + {ok, StartOptions} = start_options(Opts), + ?fcrt("open", [{start_options, StartOptions}]), + {ok, OpenOptions} = open_options([{host, Host}|Opts]), + ?fcrt("open", [{open_options, OpenOptions}]), + case start_link(StartOptions, []) of + {ok, Pid} -> + ?fcrt("open - ok", [{pid, Pid}]), + call(Pid, {open, ip_comm, OpenOptions}, plain); + Error1 -> + ?fcrt("open - error", [{error1, Error1}]), + Error1 + end + catch + throw:Error2 -> + ?fcrt("open - error", [{error2, Error2}]), + Error2 + end. + + +%% <BACKWARD-COMPATIBILLITY> +open(Host, Port, Flags) when is_integer(Port) andalso is_list(Flags) -> + ?fcrt("open", [{host, Host}, {port, Port}, {flags, Flags}]), + try + {ok, StartOptions} = start_options([{flags, Flags}]), + ?fcrt("open", [{start_options, StartOptions}]), + {ok, OpenOptions} = open_options([{host, Host}, {port, Port}|Flags]), + ?fcrt("open", [{open_options, OpenOptions}]), + case ftp_sup:start_child([[{client, self()} | StartOptions], []]) of + {ok, Pid} -> + ?fcrt("open - ok", [{pid, Pid}]), + call(Pid, {open, ip_comm, OpenOptions}, plain); + Error1 -> + ?fcrt("open - error", [{error1, Error1}]), + Error1 + end + catch + throw:Error2 -> + Error2 + end. +%% </BACKWARD-COMPATIBILLITY> + + + + + +%%-------------------------------------------------------------------------- +%% user(Pid, User, Pass, <Acc>) -> ok | {error, euser} | {error, econn} +%% | {error, eacct} +%% Pid = pid(), +%% User = Pass = Acc = string() +%% +%% Description: Login with or without a supplied account name. +%%-------------------------------------------------------------------------- +user(Pid, User, Pass) -> + call(Pid, {user, User, Pass}, atom). + +user(Pid, User, Pass, Acc) -> + call(Pid, {user, User, Pass, Acc}, atom). + +%%-------------------------------------------------------------------------- +%% account(Pid, Acc) -> ok | {error, eacct} +%% Pid = pid() +%% Acc= string() +%% +%% Description: Set a user Account. +%%-------------------------------------------------------------------------- +account(Pid, Acc) -> + call(Pid, {account, Acc}, atom). + +%%-------------------------------------------------------------------------- +%% pwd(Pid) -> {ok, Dir} | {error, elogin} | {error, econn} +%% Pid = pid() +%% Dir = string() +%% +%% Description: Get the current working directory at remote server. +%%-------------------------------------------------------------------------- +pwd(Pid) -> + call(Pid, pwd, ctrl). + +%%-------------------------------------------------------------------------- +%% lpwd(Pid) -> {ok, Dir} | {error, elogin} +%% Pid = pid() +%% Dir = string() +%% +%% Description: Get the current working directory at local server. +%%-------------------------------------------------------------------------- +lpwd(Pid) -> + call(Pid, lpwd, string). + +%%-------------------------------------------------------------------------- +%% cd(Pid, Dir) -> ok | {error, epath} | {error, elogin} | {error, econn} +%% Pid = pid() +%% Dir = string() +%% +%% Description: Change current working directory at remote server. +%%-------------------------------------------------------------------------- +cd(Pid, Dir) -> + call(Pid, {cd, Dir}, atom). + +%%-------------------------------------------------------------------------- +%% lcd(Pid, Dir) -> ok | {error, epath} +%% Pid = pid() +%% Dir = string() +%% +%% Description: Change current working directory for the local client. +%%-------------------------------------------------------------------------- +lcd(Pid, Dir) -> + call(Pid, {lcd, Dir}, string). + +%%-------------------------------------------------------------------------- +%% ls(Pid) -> Result +%% ls(Pid, <Dir>) -> Result +%% +%% Pid = pid() +%% Dir = string() +%% Result = {ok, Listing} | {error, Reason} +%% Listing = string() +%% Reason = epath | elogin | econn +%% +%% Description: Returns a list of files in long format. +%%-------------------------------------------------------------------------- +ls(Pid) -> + ls(Pid, ""). +ls(Pid, Dir) -> + call(Pid, {dir, long, Dir}, string). + +%%-------------------------------------------------------------------------- +%% nlist(Pid) -> Result +%% nlist(Pid, Pathname) -> Result +%% +%% Pid = pid() +%% Pathname = string() +%% Result = {ok, Listing} | {error, Reason} +%% Listing = string() +%% Reason = epath | elogin | econn +%% +%% Description: Returns a list of files in short format +%%-------------------------------------------------------------------------- +nlist(Pid) -> + nlist(Pid, ""). +nlist(Pid, Dir) -> + call(Pid, {dir, short, Dir}, string). + +%%-------------------------------------------------------------------------- +%% rename(Pid, CurrFile, NewFile) -> ok | {error, epath} | {error, elogin} +%% | {error, econn} +%% Pid = pid() +%% CurrFile = NewFile = string() +%% +%% Description: Rename a file at remote server. +%%-------------------------------------------------------------------------- +rename(Pid, CurrFile, NewFile) -> + call(Pid, {rename, CurrFile, NewFile}, string). + +%%-------------------------------------------------------------------------- +%% delete(Pid, File) -> ok | {error, epath} | {error, elogin} | +%% {error, econn} +%% Pid = pid() +%% File = string() +%% +%% Description: Remove file at remote server. +%%-------------------------------------------------------------------------- +delete(Pid, File) -> + call(Pid, {delete, File}, string). + +%%-------------------------------------------------------------------------- +%% mkdir(Pid, Dir) -> ok | {error, epath} | {error, elogin} | {error, econn} +%% Pid = pid(), +%% Dir = string() +%% +%% Description: Make directory at remote server. +%%-------------------------------------------------------------------------- +mkdir(Pid, Dir) -> + call(Pid, {mkdir, Dir}, atom). + +%%-------------------------------------------------------------------------- +%% rmdir(Pid, Dir) -> ok | {error, epath} | {error, elogin} | {error, econn} +%% Pid = pid(), +%% Dir = string() +%% +%% Description: Remove directory at remote server. +%%-------------------------------------------------------------------------- +rmdir(Pid, Dir) -> + call(Pid, {rmdir, Dir}, atom). + +%%-------------------------------------------------------------------------- +%% type(Pid, Type) -> ok | {error, etype} | {error, elogin} | {error, econn} +%% Pid = pid() +%% Type = ascii | binary +%% +%% Description: Set transfer type. +%%-------------------------------------------------------------------------- +type(Pid, Type) -> + call(Pid, {type, Type}, atom). + +%%-------------------------------------------------------------------------- +%% recv(Pid, RemoteFileName <LocalFileName>) -> ok | {error, epath} | +%% {error, elogin} | {error, econn} +%% Pid = pid() +%% RemoteFileName = LocalFileName = string() +%% +%% Description: Transfer file from remote server. +%%-------------------------------------------------------------------------- +recv(Pid, RemotFileName) -> + recv(Pid, RemotFileName, RemotFileName). + +recv(Pid, RemotFileName, LocalFileName) -> + call(Pid, {recv, RemotFileName, LocalFileName}, atom). + +%%-------------------------------------------------------------------------- +%% recv_bin(Pid, RemoteFile) -> {ok, Bin} | {error, epath} | {error, elogin} +%% | {error, econn} +%% Pid = pid() +%% RemoteFile = string() +%% Bin = binary() +%% +%% Description: Transfer file from remote server into binary. +%%-------------------------------------------------------------------------- +recv_bin(Pid, RemoteFile) -> + call(Pid, {recv_bin, RemoteFile}, bin). + +%%-------------------------------------------------------------------------- +%% recv_chunk_start(Pid, RemoteFile) -> ok | {error, elogin} | {error, epath} +%% | {error, econn} +%% Pid = pid() +%% RemoteFile = string() +%% +%% Description: Start receive of chunks of remote file. +%%-------------------------------------------------------------------------- +recv_chunk_start(Pid, RemoteFile) -> + call(Pid, {recv_chunk_start, RemoteFile}, atom). + +%%-------------------------------------------------------------------------- +%% recv_chunk(Pid, RemoteFile) -> ok | {ok, Bin} | {error, Reason} +%% Pid = pid() +%% RemoteFile = string() +%% +%% Description: Transfer file from remote server into binary in chunks +%%-------------------------------------------------------------------------- +recv_chunk(Pid) -> + call(Pid, recv_chunk, atom). + +%%-------------------------------------------------------------------------- +%% send(Pid, LocalFileName <RemotFileName>) -> ok | {error, epath} +%% | {error, elogin} +%% | {error, econn} +%% Pid = pid() +%% LocalFileName = RemotFileName = string() +%% +%% Description: Transfer file to remote server. +%%-------------------------------------------------------------------------- +send(Pid, LocalFileName) -> + send(Pid, LocalFileName, LocalFileName). + +send(Pid, LocalFileName, RemotFileName) -> + call(Pid, {send, LocalFileName, RemotFileName}, atom). + +%%-------------------------------------------------------------------------- +%% send_bin(Pid, Bin, RemoteFile) -> ok | {error, epath} | {error, elogin} +%% | {error, enotbinary} | {error, econn} +%% Pid = pid() +%% Bin = binary() +%% RemoteFile = string() +%% +%% Description: Transfer a binary to a remote file. +%%-------------------------------------------------------------------------- +send_bin(Pid, Bin, RemoteFile) when is_binary(Bin) -> + call(Pid, {send_bin, Bin, RemoteFile}, atom); +send_bin(_Pid, _Bin, _RemoteFile) -> + {error, enotbinary}. + +%%-------------------------------------------------------------------------- +%% send_chunk_start(Pid, RemoteFile) -> ok | {error, elogin} | {error, epath} +%% | {error, econn} +%% Pid = pid() +%% RemoteFile = string() +%% +%% Description: Start transfer of chunks to remote file. +%%-------------------------------------------------------------------------- +send_chunk_start(Pid, RemoteFile) -> + call(Pid, {send_chunk_start, RemoteFile}, atom). + +%%-------------------------------------------------------------------------- +%% append_chunk_start(Pid, RemoteFile) -> ok | {error, elogin} | +%% {error, epath} | {error, econn} +%% Pid = pid() +%% RemoteFile = string() +%% +%% Description: Start append chunks of data to remote file. +%%-------------------------------------------------------------------------- +append_chunk_start(Pid, RemoteFile) -> + call(Pid, {append_chunk_start, RemoteFile}, atom). + +%%-------------------------------------------------------------------------- +%% send_chunk(Pid, Bin) -> ok | {error, elogin} | {error, enotbinary} +%% | {error, echunk} | {error, econn} +%% Pid = pid() +%% Bin = binary(). +%% +%% Purpose: Send chunk to remote file. +%%-------------------------------------------------------------------------- +send_chunk(Pid, Bin) when is_binary(Bin) -> + call(Pid, {transfer_chunk, Bin}, atom); +send_chunk(_Pid, _Bin) -> + {error, enotbinary}. + +%%-------------------------------------------------------------------------- +%% append_chunk(Pid, Bin) -> ok | {error, elogin} | {error, enotbinary} +%% | {error, echunk} | {error, econn} +%% Pid = pid() +%% Bin = binary() +%% +%% Description: Append chunk to remote file. +%%-------------------------------------------------------------------------- +append_chunk(Pid, Bin) when is_binary(Bin) -> + call(Pid, {transfer_chunk, Bin}, atom); +append_chunk(_Pid, _Bin) -> + {error, enotbinary}. + +%%-------------------------------------------------------------------------- +%% send_chunk_end(Pid) -> ok | {error, elogin} | {error, echunk} +%% | {error, econn} +%% Pid = pid() +%% +%% Description: End sending of chunks to remote file. +%%-------------------------------------------------------------------------- +send_chunk_end(Pid) -> + call(Pid, chunk_end, atom). + +%%-------------------------------------------------------------------------- +%% append_chunk_end(Pid) -> ok | {error, elogin} | {error, echunk} +%% | {error, econn} +%% Pid = pid() +%% +%% Description: End appending of chunks to remote file. +%%-------------------------------------------------------------------------- +append_chunk_end(Pid) -> + call(Pid, chunk_end, atom). + +%%-------------------------------------------------------------------------- +%% append(Pid, LocalFileName, RemotFileName) -> ok | {error, epath} +%% | {error, elogin} | {error, econn} +%% Pid = pid() +%% LocalFileName = RemotFileName = string() +%% +%% Description: Append the local file to the remote file +%%-------------------------------------------------------------------------- +append(Pid, LocalFileName) -> + append(Pid, LocalFileName, LocalFileName). + +append(Pid, LocalFileName, RemotFileName) -> + call(Pid, {append, LocalFileName, RemotFileName}, atom). + +%%-------------------------------------------------------------------------- +%% append_bin(Pid, Bin, RemoteFile) -> ok | {error, epath} | {error, elogin} +%% | {error, enotbinary} | {error, econn} +%% Pid = pid() +%% Bin = binary() +%% RemoteFile = string() +%% +%% Purpose: Append a binary to a remote file. +%%-------------------------------------------------------------------------- +append_bin(Pid, Bin, RemoteFile) when is_binary(Bin) -> + call(Pid, {append_bin, Bin, RemoteFile}, atom); +append_bin(_Pid, _Bin, _RemoteFile) -> + {error, enotbinary}. + +%%-------------------------------------------------------------------------- +%% quote(Pid, Cmd) -> ok +%% Pid = pid() +%% Cmd = string() +%% +%% Description: Send arbitrary ftp command. +%%-------------------------------------------------------------------------- +quote(Pid, Cmd) when is_list(Cmd) -> + call(Pid, {quote, Cmd}, atom). + +%%-------------------------------------------------------------------------- +%% close(Pid) -> ok +%% Pid = pid() +%% +%% Description: End the ftp session. +%%-------------------------------------------------------------------------- +close(Pid) -> + cast(Pid, close), + ok. + +%%-------------------------------------------------------------------------- +%% force_active(Pid) -> ok +%% Pid = pid() +%% +%% Description: Force connection to use active mode. +%%-------------------------------------------------------------------------- +force_active(Pid) -> + error_logger:info_report("This function is deprecated use the mode flag " + "instead"), + call(Pid, force_active, atom). + +%%-------------------------------------------------------------------------- +%% formaterror(Tag) -> string() +%% Tag = atom() | {error, atom()} +%% +%% Description: Return diagnostics. +%%-------------------------------------------------------------------------- +formaterror(Tag) -> + ftp_response:error_string(Tag). + +info(Pid) -> + call(Pid, info, list). + + +%%%======================================================================== +%%% Behavior callbacks +%%%======================================================================== +start_standalone(Options) -> + try + {ok, StartOptions} = start_options(Options), + {ok, OpenOptions} = open_options(Options), + case start_link(StartOptions, []) of + {ok, Pid} -> + call(Pid, {open, ip_comm, OpenOptions}, plain); + Error1 -> + Error1 + end + catch + throw:Error2 -> + Error2 + end. + +start_service(Options) -> + try + {ok, StartOptions} = start_options(Options), + {ok, OpenOptions} = open_options(Options), + case ftp_sup:start_child([[[{client, self()} | StartOptions], []]]) of + {ok, Pid} -> + call(Pid, {open, ip_comm, OpenOptions}, plain); + Error1 -> + Error1 + end + catch + throw:Error2 -> + Error2 + end. + +stop_service(Pid) -> + close(Pid). + +services() -> + [{ftpc, Pid} || {_, Pid, _, _} <- + supervisor:which_children(ftp_sup)]. +service_info(Pid) -> + {ok, Info} = call(Pid, info, list), + {ok, [proplists:lookup(mode, Info), + proplists:lookup(local_port, Info), + proplists:lookup(peer, Info), + proplists:lookup(peer_port, Info)]}. + + +%% This function extracts the start options from the +%% Valid options: +%% debug, +%% verbose +%% ipfamily +%% priority +%% flags (for backward compatibillity) +start_options(Options) -> + ?fcrt("start_options", [{options, Options}]), + case lists:keysearch(flags, 1, Options) of + {value, {flags, Flags}} -> + Verbose = lists:member(verbose, Flags), + IsTrace = lists:member(trace, Flags), + IsDebug = lists:member(debug, Flags), + DebugLevel = + if + (IsTrace =:= true) -> + trace; + IsDebug =:= true -> + debug; + true -> + disable + end, + {ok, [{verbose, Verbose}, + {debug, DebugLevel}, + {priority, low}]}; + false -> + ValidateVerbose = + fun(true) -> true; + (false) -> true; + (_) -> false + end, + ValidateDebug = + fun(trace) -> true; + (debug) -> true; + (disable) -> true; + (_) -> false + end, + ValidatePriority = + fun(low) -> true; + (normal) -> true; + (high) -> true; + (_) -> false + end, + ValidOptions = + [{verbose, ValidateVerbose, false, false}, + {debug, ValidateDebug, false, disable}, + {priority, ValidatePriority, false, low}], + validate_options(Options, ValidOptions, []) + end. + + +%% This function extracts and validates the open options from the +%% Valid options: +%% mode +%% host +%% port +%% timeout +%% progress +open_options(Options) -> + ?fcrt("open_options", [{options, Options}]), + ValidateMode = + fun(active) -> true; + (passive) -> true; + (_) -> false + end, + ValidateHost = + fun(Host) when is_list(Host) -> + true; + (Host) when is_tuple(Host) andalso + ((size(Host) =:= 4) orelse (size(Host) =:= 8)) -> + true; + (_) -> + false + end, + ValidatePort = + fun(Port) when is_integer(Port) andalso (Port > 0) -> true; + (_) -> false + end, + ValidateIpFamily = + fun(inet) -> true; + (inet6) -> true; + (inet6fb4) -> true; + (_) -> false + end, + ValidateTimeout = + fun(Timeout) when is_integer(Timeout) andalso (Timeout > 0) -> true; + (_) -> false + end, + ValidateProgress = + fun(ignore) -> + true; + ({Mod, Func, _InitProgress}) when is_atom(Mod) andalso + is_atom(Func) -> + true; + (_) -> + false + end, + ValidOptions = + [{mode, ValidateMode, false, ?DEFAULT_MODE}, + {host, ValidateHost, true, ehost}, + {port, ValidatePort, false, ?FTP_PORT}, + {ipfamily, ValidateIpFamily, false, inet}, + {timeout, ValidateTimeout, false, ?CONNECTION_TIMEOUT}, + {progress, ValidateProgress, false, ?PROGRESS_DEFAULT}], + validate_options(Options, ValidOptions, []). + +validate_options([], [], Acc) -> + ?fcrt("validate_options -> done", [{acc, Acc}]), + {ok, lists:reverse(Acc)}; +validate_options([], ValidOptions, Acc) -> + ?fcrt("validate_options -> done", + [{valid_options, ValidOptions}, {acc, Acc}]), + %% Check if any mandatory options are missing! + case [{Key, Reason} || {Key, _, true, Reason} <- ValidOptions] of + [] -> + Defaults = + [{Key, Default} || {Key, _, _, Default} <- ValidOptions], + {ok, lists:reverse(Defaults ++ Acc)}; + [{_, Reason}|_Missing] -> + throw({error, Reason}) + end; +validate_options([{Key, Value}|Options], ValidOptions, Acc) -> + ?fcrt("validate_options -> check", + [{key, Key}, {value, Value}, {acc, Acc}]), + case lists:keysearch(Key, 1, ValidOptions) of + {value, {Key, Validate, _, Default}} -> + case (catch Validate(Value)) of + true -> + ?fcrt("validate_options -> check - accept", []), + NewValidOptions = lists:keydelete(Key, 1, ValidOptions), + validate_options(Options, NewValidOptions, + [{Key, Value} | Acc]); + _ -> + ?fcrt("validate_options -> check - reject", + [{default, Default}]), + NewValidOptions = lists:keydelete(Key, 1, ValidOptions), + validate_options(Options, NewValidOptions, + [{Key, Default} | Acc]) + end; + false -> + validate_options(Options, ValidOptions, Acc) + end; +validate_options([_|Options], ValidOptions, Acc) -> + validate_options(Options, ValidOptions, Acc). + + + +%%%======================================================================== +%%% gen_server callback functions +%%%======================================================================== + +%%------------------------------------------------------------------------- +%% init(Args) -> {ok, State} | {ok, State, Timeout} | {stop, Reason} +%% Description: Initiates the erlang process that manages a ftp connection. +%%------------------------------------------------------------------------- +init(Options) -> + process_flag(trap_exit, true), + + %% Keep track of the client + {value, {client, Client}} = lists:keysearch(client, 1, Options), + erlang:monitor(process, Client), + + %% Make sure inet is started + inet_db:start(), + + %% Where are we + {ok, Dir} = file:get_cwd(), + + %% Maybe activate dbg + case key_search(debug, Options, disable) of + trace -> + dbg:tracer(), + dbg:p(all, [call]), + dbg:tpl(ftp, [{'_', [], [{return_trace}]}]), + dbg:tpl(ftp_response, [{'_', [], [{return_trace}]}]), + dbg:tpl(ftp_progress, [{'_', [], [{return_trace}]}]); + debug -> + dbg:tracer(), + dbg:p(all, [call]), + dbg:tp(ftp, [{'_', [], [{return_trace}]}]), + dbg:tp(ftp_response, [{'_', [], [{return_trace}]}]), + dbg:tp(ftp_progress, [{'_', [], [{return_trace}]}]); + _ -> + %% Keep silent + ok + end, + + %% Verbose? + Verbose = key_search(verbose, Options, false), + + %% IpFamily? + IpFamily = key_search(ipfamily, Options, inet), + + State = #state{owner = Client, + verbose = Verbose, + ipfamily = IpFamily, + ldir = Dir}, + + %% Set process prio + Priority = key_search(priority, Options, low), + process_flag(priority, Priority), + + %% And we are done + {ok, State}. + + +%%-------------------------------------------------------------------------- +%% handle_call(Request, From, State) -> {reply, Reply, State} | +%% {reply, Reply, State, Timeout} | +%% {noreply, State} | +%% {noreply, State, Timeout} | +%% {stop, Reason, Reply, State} | +%% Description: Handle incoming requests. +%%------------------------------------------------------------------------- + +%% Anyone can ask this question +handle_call({_, info}, _, #state{verbose = Verbose, + mode = Mode, + timeout = Timeout, + ipfamily = IpFamily, + csock = Socket, + progress = Progress} = State) -> + {ok, {_, LocalPort}} = inet:sockname(Socket), + {ok, {Address, Port}} = inet:peername(Socket), + Options = [{verbose, Verbose}, + {ipfamily, IpFamily}, + {mode, Mode}, + {peer, Address}, + {peer_port, Port}, + {local_port, LocalPort}, + {timeout, Timeout}, + {progress, Progress}], + {reply, {ok, Options}, State}; + +%% But everything else must come from the owner +handle_call({Pid, _}, _, #state{owner = Owner} = State) when Owner =/= Pid -> + {reply, {error, not_connection_owner}, State}; + +handle_call({_, {open, ip_comm, Opts}}, From, State) -> + ?fcrd("handle_call(open)", [{opts, Opts}]), + case key_search(host, Opts, undefined) of + undefined -> + {stop, normal, {error, ehost}, State}; + Host -> + Mode = key_search(mode, Opts, ?DEFAULT_MODE), + Port = key_search(port, Opts, ?FTP_PORT), + Timeout = key_search(timeout, Opts, ?CONNECTION_TIMEOUT), + Progress = key_search(progress, Opts, ignore), + + State2 = State#state{client = From, + mode = Mode, + progress = progress(Progress)}, + + ?fcrd("handle_call(open) -> setup ctrl connection with", + [{host, Host}, {port, Port}, {timeout, Timeout}]), + case setup_ctrl_connection(Host, Port, Timeout, State2) of + {ok, State3, WaitTimeout} -> + ?fcrd("handle_call(open) -> ctrl connection setup done", + [{waittimeout, WaitTimeout}]), + {noreply, State3, WaitTimeout}; + {error, Reason} -> + ?fcrd("handle_call(open) -> ctrl connection setup failed", + [{reason, Reason}]), + gen_server:reply(From, {error, ehost}), + {stop, normal, State2#state{client = undefined}} + end + end; + +handle_call({_, {open, ip_comm, Host, Opts}}, From, State) -> + Mode = key_search(mode, Opts, ?DEFAULT_MODE), + Port = key_search(port, Opts, ?FTP_PORT), + Timeout = key_search(timeout, Opts, ?CONNECTION_TIMEOUT), + Progress = key_search(progress, Opts, ignore), + + State2 = State#state{client = From, + mode = Mode, + progress = progress(Progress)}, + + case setup_ctrl_connection(Host, Port, Timeout, State2) of + {ok, State3, WaitTimeout} -> + {noreply, State3, WaitTimeout}; + {error, _Reason} -> + gen_server:reply(From, {error, ehost}), + {stop, normal, State2#state{client = undefined}} + end; + +handle_call({_, force_active}, _, State) -> + {reply, ok, State#state{mode = active}}; + +handle_call({_, {user, User, Password}}, From, + #state{csock = CSock} = State) when (CSock =/= undefined) -> + handle_user(User, Password, "", State#state{client = From}); + +handle_call({_, {user, User, Password, Acc}}, From, + #state{csock = CSock} = State) when (CSock =/= undefined) -> + handle_user(User, Password, Acc, State#state{client = From}); + +handle_call({_, {account, Acc}}, From, State)-> + handle_user_account(Acc, State#state{client = From}); + +handle_call({_, pwd}, From, #state{chunk = false} = State) -> + send_ctrl_message(State, mk_cmd("PWD", [])), + activate_ctrl_connection(State), + {noreply, State#state{client = From, caller = pwd}}; + +handle_call({_, lpwd}, From, #state{ldir = LDir} = State) -> + {reply, {ok, LDir}, State#state{client = From}}; + +handle_call({_, {cd, Dir}}, From, #state{chunk = false} = State) -> + send_ctrl_message(State, mk_cmd("CWD ~s", [Dir])), + activate_ctrl_connection(State), + {noreply, State#state{client = From, caller = cd}}; + +handle_call({_,{lcd, Dir}}, _From, #state{ldir = LDir0} = State) -> + LDir = filename:absname(Dir, LDir0), + case file:read_file_info(LDir) of %% FIX better check that LDir is a dir. + {ok, _ } -> + {reply, ok, State#state{ldir = LDir}}; + _ -> + {reply, {error, epath}, State} + end; + +handle_call({_, {dir, Len, Dir}}, {_Pid, _} = From, + #state{chunk = false} = State) -> + setup_data_connection(State#state{caller = {dir, Dir, Len}, + client = From}); +handle_call({_, {rename, CurrFile, NewFile}}, From, + #state{chunk = false} = State) -> + send_ctrl_message(State, mk_cmd("RNFR ~s", [CurrFile])), + activate_ctrl_connection(State), + {noreply, State#state{caller = {rename, NewFile}, client = From}}; + +handle_call({_, {delete, File}}, {_Pid, _} = From, + #state{chunk = false} = State) -> + send_ctrl_message(State, mk_cmd("DELE ~s", [File])), + activate_ctrl_connection(State), + {noreply, State#state{client = From}}; + +handle_call({_, {mkdir, Dir}}, From, #state{chunk = false} = State) -> + send_ctrl_message(State, mk_cmd("MKD ~s", [Dir])), + activate_ctrl_connection(State), + {noreply, State#state{client = From}}; + +handle_call({_,{rmdir, Dir}}, From, #state{chunk = false} = State) -> + send_ctrl_message(State, mk_cmd("RMD ~s", [Dir])), + activate_ctrl_connection(State), + {noreply, State#state{client = From}}; + +handle_call({_,{type, Type}}, From, #state{chunk = false} + = State) -> + case Type of + ascii -> + send_ctrl_message(State, mk_cmd("TYPE A", [])), + activate_ctrl_connection(State), + {noreply, State#state{caller = type, type = ascii, + client = From}}; + binary -> + send_ctrl_message(State, mk_cmd("TYPE I", [])), + activate_ctrl_connection(State), + {noreply, State#state{caller = type, type = binary, + client = From}}; + _ -> + {reply, {error, etype}, State} + end; + +handle_call({_,{recv, RemoteFile, LocalFile}}, From, + #state{chunk = false, ldir = LocalDir} = State) -> + progress_report({remote_file, RemoteFile}, State), + NewLocalFile = filename:absname(LocalFile, LocalDir), + + case file_open(NewLocalFile, write) of + {ok, Fd} -> + setup_data_connection(State#state{client = From, + caller = + {recv_file, + RemoteFile, Fd}}); + {error, _What} -> + {reply, {error, epath}, State} + end; + +handle_call({_, {recv_bin, RemoteFile}}, From, #state{chunk = false} = + State) -> + setup_data_connection(State#state{caller = {recv_bin, RemoteFile}, + client = From}); + +handle_call({_,{recv_chunk_start, RemoteFile}}, From, #state{chunk = false} + = State) -> + setup_data_connection(State#state{caller = {start_chunk_transfer, + "RETR", RemoteFile}, + client = From}); + +handle_call({_, recv_chunk}, _, #state{chunk = false} = State) -> + {reply, {error, "ftp:recv_chunk_start/2 not called"}, State}; + +handle_call({_, recv_chunk}, From, #state{chunk = true} = State) -> + activate_data_connection(State), + {noreply, State#state{client = From, caller = recv_chunk}}; + +handle_call({_, {send, LocalFile, RemoteFile}}, From, + #state{chunk = false, ldir = LocalDir} = State) -> + progress_report({local_file, filename:absname(LocalFile, LocalDir)}, + State), + setup_data_connection(State#state{caller = {transfer_file, + {"STOR", + LocalFile, RemoteFile}}, + client = From}); +handle_call({_, {append, LocalFile, RemoteFile}}, From, + #state{chunk = false} = State) -> + setup_data_connection(State#state{caller = {transfer_file, + {"APPE", + LocalFile, RemoteFile}}, + client = From}); +handle_call({_, {send_bin, Bin, RemoteFile}}, From, + #state{chunk = false} = State) -> + setup_data_connection(State#state{caller = {transfer_data, + {"STOR", Bin, RemoteFile}}, + client = From}); +handle_call({_,{append_bin, Bin, RemoteFile}}, From, + #state{chunk = false} = State) -> + setup_data_connection(State#state{caller = {transfer_data, + {"APPE", Bin, RemoteFile}}, + client = From}); +handle_call({_, {send_chunk_start, RemoteFile}}, From, #state{chunk = false} + = State) -> + setup_data_connection(State#state{caller = {start_chunk_transfer, + "STOR", RemoteFile}, + client = From}); +handle_call({_, {append_chunk_start, RemoteFile}}, From, #state{chunk = false} + = State) -> + setup_data_connection(State#state{caller = {start_chunk_transfer, + "APPE", RemoteFile}, + client = From}); +handle_call({_, {transfer_chunk, Bin}}, _, #state{chunk = true} = State) -> + send_data_message(State, Bin), + {reply, ok, State}; + +handle_call({_, chunk_end}, From, #state{chunk = true} = State) -> + close_data_connection(State), + activate_ctrl_connection(State), + {noreply, State#state{client = From, dsock = undefined, + caller = end_chunk_transfer, chunk = false}}; + +handle_call({_, {quote, Cmd}}, From, #state{chunk = false} = State) -> + send_ctrl_message(State, mk_cmd(Cmd, [])), + activate_ctrl_connection(State), + {noreply, State#state{client = From, caller = quote}}; + +handle_call({_, _Req}, _From, #state{csock = CSock} = State) + when (CSock =:= undefined) -> + {reply, {error, not_connected}, State}; + +handle_call(_, _, #state{chunk = true} = State) -> + {reply, {error, echunk}, State}; + +%% Catch all - This can only happen if the application programmer writes +%% really bad code that violates the API. +handle_call(Request, _Timeout, State) -> + {stop, {'API_violation_connection_closed', Request}, + {error, {connection_terminated, 'API_violation'}}, State}. + +%%-------------------------------------------------------------------------- +%% handle_cast(Request, State) -> {noreply, State} | +%% {noreply, State, Timeout} | +%% {stop, Reason, State} +%% Description: Handles cast messages. +%%------------------------------------------------------------------------- +handle_cast({Pid, close}, #state{owner = Pid} = State) -> + send_ctrl_message(State, mk_cmd("QUIT", [])), + close_ctrl_connection(State), + close_data_connection(State), + {stop, normal, State#state{csock = undefined, dsock = undefined}}; + +handle_cast({Pid, close}, State) -> + Report = io_lib:format("A none owner process ~p tried to close an " + "ftp connection: ~n", [Pid]), + error_logger:info_report(Report), + {noreply, State}; + +%% Catch all - This can oly happen if the application programmer writes +%% really bad code that violates the API. +handle_cast(Msg, State) -> + {stop, {'API_violation_connection_closed', Msg}, State}. + +%%-------------------------------------------------------------------------- +%% handle_info(Msg, State) -> {noreply, State} | {noreply, State, Timeout} | +%% {stop, Reason, State} +%% Description: Handles tcp messages from the ftp-server. +%% Note: The order of the function clauses is significant. +%%-------------------------------------------------------------------------- + +handle_info(timeout, #state{caller = open} = State) -> + {stop, timeout, State}; + +handle_info(timeout, State) -> + {noreply, State}; + +%%% Data socket messages %%% +handle_info({tcp, Socket, Data}, + #state{dsock = Socket, + caller = {recv_file, Fd}} = State) -> + file_write(binary_to_list(Data), Fd), + progress_report({binary, Data}, State), + activate_data_connection(State), + {noreply, State}; + +handle_info({tcp, Socket, Data}, #state{dsock = Socket, client = From, + caller = recv_chunk} + = State) -> + gen_server:reply(From, {ok, Data}), + {noreply, State#state{client = undefined, data = <<>>}}; + +handle_info({tcp, Socket, Data}, #state{dsock = Socket} = State) -> + activate_data_connection(State), + {noreply, State#state{data = <<(State#state.data)/binary, + Data/binary>>}}; + +handle_info({tcp_closed, Socket}, #state{dsock = Socket, + caller = {recv_file, Fd}} + = State) -> + file_close(Fd), + progress_report({transfer_size, 0}, State), + activate_ctrl_connection(State), + {noreply, State#state{dsock = undefined, data = <<>>}}; + +handle_info({tcp_closed, Socket}, #state{dsock = Socket, client = From, + caller = recv_chunk} + = State) -> + gen_server:reply(From, ok), + {noreply, State#state{dsock = undefined, client = undefined, + data = <<>>, caller = undefined, + chunk = false}}; + +handle_info({tcp_closed, Socket}, #state{dsock = Socket, caller = recv_bin, + data = Data} = State) -> + activate_ctrl_connection(State), + {noreply, State#state{dsock = undefined, data = <<>>, + caller = {recv_bin, Data}}}; + +handle_info({tcp_closed, Socket}, #state{dsock = Socket, data = Data, + caller = {handle_dir_result, Dir}} + = State) -> + activate_ctrl_connection(State), + {noreply, State#state{dsock = undefined, + caller = {handle_dir_result, Dir, Data}, +% data = <<?CR,?LF>>}}; + data = <<>>}}; + +handle_info({tcp_error, Socket, Reason}, #state{dsock = Socket, + client = From} = State) -> + gen_server:reply(From, {error, Reason}), + close_data_connection(State), + {noreply, State#state{dsock = undefined, client = undefined, + data = <<>>, caller = undefined, chunk = false}}; + +%%% Ctrl socket messages %%% +handle_info({tcp, Socket, Data}, #state{csock = Socket, + verbose = Verbose, + caller = Caller, + client = From, + ctrl_data = {CtrlData, AccLines, + LineStatus}} + = State) -> + case ftp_response:parse_lines(<<CtrlData/binary, Data/binary>>, + AccLines, LineStatus) of + {ok, Lines, NextMsgData} -> + verbose(Lines, Verbose, 'receive'), + CtrlResult = ftp_response:interpret(Lines), + case Caller of + quote -> + gen_server:reply(From, string:tokens(Lines, [?CR, ?LF])), + {noreply, State#state{client = undefined, + caller = undefined, + ctrl_data = {NextMsgData, [], + start}}}; + _ -> + handle_ctrl_result(CtrlResult, + State#state{ctrl_data = + {NextMsgData, [], start}}) + end; + {continue, NewCtrlData} -> + activate_ctrl_connection(State), + {noreply, State#state{ctrl_data = NewCtrlData}} + end; + +handle_info({tcp_closed, Socket}, #state{csock = Socket}) -> + %% If the server closes the control channel it is + %% the expected behavior that connection process terminates. + exit(normal); %% User will get error message from terminate/2 + +handle_info({tcp_error, Socket, Reason}, _) -> + Report = + io_lib:format("tcp_error on socket: ~p for reason: ~p~n", + [Socket, Reason]), + error_logger:error_report(Report), + %% If tcp does not work the only option is to terminate, + %% this is the expected behavior under these circumstances. + exit(normal); %% User will get error message from terminate/2 + +%% Monitor messages - if the process owning the ftp connection goes +%% down there is no point in continuing. +handle_info({'DOWN', _Ref, _Type, _Process, normal}, State) -> + {stop, normal, State#state{client = undefined}}; + +handle_info({'DOWN', _Ref, _Type, _Process, shutdown}, State) -> + {stop, normal, State#state{client = undefined}}; + +handle_info({'DOWN', _Ref, _Type, _Process, timeout}, State) -> + {stop, normal, State#state{client = undefined}}; + +handle_info({'DOWN', _Ref, _Type, Process, Reason}, State) -> + {stop, {stopped, {'EXIT', Process, Reason}}, + State#state{client = undefined}}; + +handle_info({'EXIT', Pid, Reason}, #state{progress = Pid} = State) -> + Report = io_lib:format("Progress reporting stopped for reason ~p~n", + Reason), + error_logger:info_report(Report), + {noreply, State#state{progress = ignore}}; + +%% Catch all - throws away unknown messages (This could happen by "accident" +%% so we do not want to crash, but we make a log entry as it is an +%% unwanted behaviour.) +handle_info(Info, State) -> + Report = io_lib:format("ftp : ~p : Unexpected message: ~p\n", + [self(), Info]), + error_logger:info_report(Report), + {noreply, State}. + +%%-------------------------------------------------------------------------- +%% terminate/2 and code_change/3 +%%-------------------------------------------------------------------------- +terminate(normal, State) -> + %% If terminate reason =/= normal the progress reporting process will + %% be killed by the exit signal. + progress_report(stop, State), + do_termiante({error, econn}, State); +terminate(Reason, State) -> + Report = io_lib:format("Ftp connection closed due to: ~p~n", [Reason]), + error_logger:error_report(Report), + do_termiante({error, eclosed}, State). + +do_termiante(ErrorMsg, State) -> + close_data_connection(State), + close_ctrl_connection(State), + case State#state.client of + undefined -> + ok; + From -> + gen_server:reply(From, ErrorMsg) + end, + ok. + +code_change(_Vsn, State1, upgrade_from_pre_5_12) -> + {state, CSock, DSock, Verbose, LDir, Type, Chunk, Mode, Timeout, + Data, CtrlData, Owner, Client, Caller, IPv6Disable, Progress} = State1, + IpFamily = + if + (IPv6Disable =:= true) -> + inet; + true -> + inet6fb4 + end, + State2 = #state{csock = CSock, + dsock = DSock, + verbose = Verbose, + ldir = LDir, + type = Type, + chunk = Chunk, + mode = Mode, + timeout = Timeout, + data = Data, + ctrl_data = CtrlData, + owner = Owner, + client = Client, + caller = Caller, + ipfamily = IpFamily, + progress = Progress}, + {ok, State2}; + +code_change(_Vsn, State1, downgrade_to_pre_5_12) -> + #state{csock = CSock, + dsock = DSock, + verbose = Verbose, + ldir = LDir, + type = Type, + chunk = Chunk, + mode = Mode, + timeout = Timeout, + data = Data, + ctrl_data = CtrlData, + owner = Owner, + client = Client, + caller = Caller, + ipfamily = IpFamily, + progress = Progress} = State1, + IPv6Disable = + if + (IpFamily =:= inet) -> + true; + true -> + false + end, + State2 = + {state, CSock, DSock, Verbose, LDir, Type, Chunk, Mode, Timeout, + Data, CtrlData, Owner, Client, Caller, IPv6Disable, Progress}, + {ok, State2}; + +code_change(_Vsn, State, _Extra) -> + {ok, State}. + + +%%%========================================================================= +%% Start/stop +%%%========================================================================= +%%-------------------------------------------------------------------------- +%% start_link([Opts, GenServerOptions]) -> {ok, Pid} | {error, Reason} +%% +%% Description: Callback function for the ftp supervisor. It is called +%% : when start_service/1 calls ftp_sup:start_child/1 to start an +%% : instance of the ftp process. Also called by start_standalone/1 +%%-------------------------------------------------------------------------- +start_link([Opts, GenServerOptions]) -> + start_link(Opts, GenServerOptions). + +start_link(Opts, GenServerOptions) -> + case lists:keysearch(client, 1, Opts) of + {value, _} -> + %% Via the supervisor + gen_server:start_link(?MODULE, Opts, GenServerOptions); + false -> + Opts2 = [{client, self()} | Opts], + gen_server:start_link(?MODULE, Opts2, GenServerOptions) + end. + + +%%% Stop functionality is handled by close/1 + +%%%======================================================================== +%%% Internal functions +%%%======================================================================== + +%%-------------------------------------------------------------------------- +%%% Help functions to handle_call and/or handle_ctrl_result +%%-------------------------------------------------------------------------- +%% User handling +handle_user(User, Password, Acc, State) -> + send_ctrl_message(State, mk_cmd("USER ~s", [User])), + activate_ctrl_connection(State), + {noreply, State#state{caller = {handle_user, Password, Acc}}}. + +handle_user_passwd(Password, Acc, State) -> + send_ctrl_message(State, mk_cmd("PASS ~s", [Password])), + activate_ctrl_connection(State), + {noreply, State#state{caller = {handle_user_passwd, Acc}}}. + +handle_user_account(Acc, State) -> + send_ctrl_message(State, mk_cmd("ACCT ~s", [Acc])), + activate_ctrl_connection(State), + {noreply, State#state{caller = handle_user_account}}. + + +%%-------------------------------------------------------------------------- +%% handle_ctrl_result +%%-------------------------------------------------------------------------- +%%-------------------------------------------------------------------------- +%% Handling of control connection setup +handle_ctrl_result({pos_compl, _}, #state{caller = open, client = From} + = State) -> + gen_server:reply(From, {ok, self()}), + {noreply, State#state{client = undefined, + caller = undefined }}; +handle_ctrl_result({_, Lines}, #state{caller = open} = State) -> + ctrl_result_response(econn, State, {error, Lines}); + +%%-------------------------------------------------------------------------- +%% Data connection setup active mode +handle_ctrl_result({pos_compl, _Lines}, + #state{mode = active, + caller = {setup_data_connection, + {LSock, Caller}}} = State) -> + handle_caller(State#state{caller = Caller, dsock = {lsock, LSock}}); + +handle_ctrl_result({Status, Lines}, + #state{mode = active, + caller = {setup_data_connection, {LSock, _}}} + = State) -> + close_connection(LSock), + ctrl_result_response(Status, State, {error, Lines}); + +%% Data connection setup passive mode +handle_ctrl_result({pos_compl, Lines}, + #state{mode = passive, + ipfamily = inet6, + client = From, + caller = {setup_data_connection, Caller}, + csock = CSock, + timeout = Timeout} + = State) -> + [_, PortStr | _] = lists:reverse(string:tokens(Lines, "|")), + {ok, {IP, _}} = inet:peername(CSock), + case connect(IP, list_to_integer(PortStr), Timeout, State) of + {ok, _, Socket} -> + handle_caller(State#state{caller = Caller, dsock = Socket}); + {error, _Reason} = Error -> + gen_server:reply(From, Error), + {noreply, State#state{client = undefined, caller = undefined}} + end; + +handle_ctrl_result({pos_compl, Lines}, + #state{mode = passive, + ipfamily = inet, + client = From, + caller = {setup_data_connection, Caller}, + timeout = Timeout} = State) -> + + {_, [?LEFT_PAREN | Rest]} = + lists:splitwith(fun(?LEFT_PAREN) -> false; (_) -> true end, Lines), + {NewPortAddr, _} = + lists:splitwith(fun(?RIGHT_PAREN) -> false; (_) -> true end, Rest), + [A1, A2, A3, A4, P1, P2] = + lists:map(fun(X) -> list_to_integer(X) end, + string:tokens(NewPortAddr, [$,])), + IP = {A1, A2, A3, A4}, + Port = (P1 * 256) + P2, + case connect(IP, Port, Timeout, State) of + {ok, _, Socket} -> + handle_caller(State#state{caller = Caller, dsock = Socket}); + {error, _Reason} = Error -> + gen_server:reply(From, Error), + {noreply,State#state{client = undefined, caller = undefined}} + end; + +%% FTP server does not support passive mode: try to fallback on active mode +handle_ctrl_result(_, + #state{mode = passive, + caller = {setup_data_connection, Caller}} = State) -> + setup_data_connection(State#state{mode = active, caller = Caller}); + + +%%-------------------------------------------------------------------------- +%% User handling +handle_ctrl_result({pos_interm, _}, + #state{caller = {handle_user, PassWord, Acc}} = State) -> + handle_user_passwd(PassWord, Acc, State); +handle_ctrl_result({Status, _}, + #state{caller = {handle_user, _, _}} = State) -> + ctrl_result_response(Status, State, {error, euser}); + +%% Accounts +handle_ctrl_result({pos_interm_acct, _}, + #state{caller = {handle_user_passwd, Acc}} = State) + when Acc =/= "" -> + handle_user_account(Acc, State); +handle_ctrl_result({Status, _}, + #state{caller = {handle_user_passwd, _}} = State) -> + ctrl_result_response(Status, State, {error, euser}); + +%%-------------------------------------------------------------------------- +%% Print current working directory +handle_ctrl_result({pos_compl, Lines}, + #state{caller = pwd, client = From} = State) -> + Dir = pwd_result(Lines), + gen_server:reply(From, {ok, Dir}), + {noreply, State#state{client = undefined, caller = undefined}}; + +%%-------------------------------------------------------------------------- +%% Directory listing +handle_ctrl_result({pos_prel, _}, #state{caller = {dir, Dir}} = State) -> + NewState = accept_data_connection(State), + activate_data_connection(NewState), + {noreply, NewState#state{caller = {handle_dir_result, Dir}}}; + +handle_ctrl_result({pos_compl, _}, #state{caller = {handle_dir_result, Dir, + Data}, client = From} + = State) -> + case Dir of + "" -> % Current directory + gen_server:reply(From, {ok, Data}), + {noreply, State#state{client = undefined, + caller = undefined}}; + _ -> + %% <WTF> + %% Dir cannot be assumed to be a dir. It is a string that + %% could be a dir, but could also be a file or even a string + %% containing wildcards (*). + %% + %% %% If there is only one line it might be a directory with one + %% %% file but it might be an error message that the directory + %% %% was not found. So in this case we have to endure a little + %% %% overhead to be able to give a good return value. Alas not + %% %% all ftp implementations behave the same and returning + %% %% an error string is allowed by the FTP RFC. + %% case lists:dropwhile(fun(?CR) -> false;(_) -> true end, + %% binary_to_list(Data)) of + %% L when (L =:= [?CR, ?LF]) orelse (L =:= []) -> + %% send_ctrl_message(State, mk_cmd("PWD", [])), + %% activate_ctrl_connection(State), + %% {noreply, + %% State#state{caller = {handle_dir_data, Dir, Data}}}; + %% _ -> + %% gen_server:reply(From, {ok, Data}), + %% {noreply, State#state{client = undefined, + %% caller = undefined}} + %% end + %% </WTF> + gen_server:reply(From, {ok, Data}), + {noreply, State#state{client = undefined, + caller = undefined}} + end; + +handle_ctrl_result({pos_compl, Lines}, + #state{caller = {handle_dir_data, Dir, DirData}} = + State) -> + OldDir = pwd_result(Lines), + send_ctrl_message(State, mk_cmd("CWD ~s", [Dir])), + activate_ctrl_connection(State), + {noreply, State#state{caller = {handle_dir_data_second_phase, OldDir, + DirData}}}; +handle_ctrl_result({Status, _}, + #state{caller = {handle_dir_data, _, _}} = State) -> + ctrl_result_response(Status, State, {error, epath}); + +handle_ctrl_result(S={_Status, _}, + #state{caller = {handle_dir_result, _, _}} = State) -> + %% OTP-5731, macosx + ctrl_result_response(S, State, {error, epath}); + +handle_ctrl_result({pos_compl, _}, + #state{caller = {handle_dir_data_second_phase, OldDir, + DirData}} = State) -> + send_ctrl_message(State, mk_cmd("CWD ~s", [OldDir])), + activate_ctrl_connection(State), + {noreply, State#state{caller = {handle_dir_data_third_phase, DirData}}}; +handle_ctrl_result({Status, _}, + #state{caller = {handle_dir_data_second_phase, _, _}} + = State) -> + ctrl_result_response(Status, State, {error, epath}); +handle_ctrl_result(_, #state{caller = {handle_dir_data_third_phase, DirData}, + client = From} = State) -> + gen_server:reply(From, {ok, DirData}), + {noreply, State#state{client = undefined, caller = undefined}}; + +handle_ctrl_result({Status, _}, #state{caller = cd} = State) -> + ctrl_result_response(Status, State, {error, epath}); + +handle_ctrl_result(Status={epath, _}, #state{caller = {dir,_}} = State) -> + ctrl_result_response(Status, State, {error, epath}); + +%%-------------------------------------------------------------------------- +%% File renaming +handle_ctrl_result({pos_interm, _}, #state{caller = {rename, NewFile}} + = State) -> + send_ctrl_message(State, mk_cmd("RNTO ~s", [NewFile])), + activate_ctrl_connection(State), + {noreply, State#state{caller = rename_second_phase}}; + +handle_ctrl_result({Status, _}, + #state{caller = {rename, _}} = State) -> + ctrl_result_response(Status, State, {error, epath}); + +handle_ctrl_result({Status, _}, + #state{caller = rename_second_phase} = State) -> + ctrl_result_response(Status, State, {error, epath}); + +%%-------------------------------------------------------------------------- +%% File handling - recv_bin +handle_ctrl_result({pos_prel, _}, #state{caller = recv_bin} = State) -> + NewState = accept_data_connection(State), + activate_data_connection(NewState), + {noreply, NewState}; + +handle_ctrl_result({pos_compl, _}, #state{caller = {recv_bin, Data}, + client = From} = State) -> + gen_server:reply(From, {ok, Data}), + close_data_connection(State), + {noreply, State#state{client = undefined, caller = undefined}}; + +handle_ctrl_result({Status, _}, #state{caller = recv_bin} = State) -> + close_data_connection(State), + ctrl_result_response(Status, State#state{dsock = undefined}, + {error, epath}); + +handle_ctrl_result({Status, _}, #state{caller = {recv_bin, _}} = State) -> + close_data_connection(State), + ctrl_result_response(Status, State#state{dsock = undefined}, + {error, epath}); +%%-------------------------------------------------------------------------- +%% File handling - start_chunk_transfer +handle_ctrl_result({pos_prel, _}, #state{client = From, + caller = start_chunk_transfer} + = State) -> + NewState = accept_data_connection(State), + gen_server:reply(From, ok), + {noreply, NewState#state{chunk = true, client = undefined, + caller = undefined}}; +%%-------------------------------------------------------------------------- +%% File handling - recv_file +handle_ctrl_result({pos_prel, _}, #state{caller = {recv_file, _}} = State) -> + NewState = accept_data_connection(State), + activate_data_connection(NewState), + {noreply, NewState}; + +handle_ctrl_result({Status, _}, #state{caller = {recv_file, Fd}} = State) -> + file_close(Fd), + close_data_connection(State), + ctrl_result_response(Status, State#state{dsock = undefined}, + {error, epath}); +%%-------------------------------------------------------------------------- +%% File handling - transfer_* +handle_ctrl_result({pos_prel, _}, #state{caller = {transfer_file, Fd}} + = State) -> + NewState = accept_data_connection(State), + send_file(Fd, NewState); + +handle_ctrl_result({pos_prel, _}, #state{caller = {transfer_data, Bin}} + = State) -> + NewState = accept_data_connection(State), + send_data_message(NewState, Bin), + close_data_connection(NewState), + activate_ctrl_connection(NewState), + {noreply, NewState#state{caller = transfer_data_second_phase, + dsock = undefined}}; +%%-------------------------------------------------------------------------- +%% Default +handle_ctrl_result({Status, Lines}, #state{client = From} = State) + when From =/= undefined -> + ctrl_result_response(Status, State, {error, Lines}). + +%%-------------------------------------------------------------------------- +%% Help functions to handle_ctrl_result +%%-------------------------------------------------------------------------- +ctrl_result_response(pos_compl, #state{client = From} = State, _) -> + gen_server:reply(From, ok), + {noreply, State#state{client = undefined, caller = undefined}}; + +ctrl_result_response(Status, #state{client = From} = State, _) + when (Status =:= etnospc) orelse + (Status =:= epnospc) orelse + (Status =:= efnamena) orelse + (Status =:= econn) -> +%Status == etnospc; Status == epnospc; Status == econn -> + gen_server:reply(From, {error, Status}), +%% {stop, normal, {error, Status}, State#state{client = undefined}}; + {stop, normal, State#state{client = undefined}}; + +ctrl_result_response(_, #state{client = From} = State, ErrorMsg) -> + gen_server:reply(From, ErrorMsg), + {noreply, State#state{client = undefined, caller = undefined}}. + +%%-------------------------------------------------------------------------- +handle_caller(#state{caller = {dir, Dir, Len}} = State) -> + Cmd = case Len of + short -> "NLST"; + long -> "LIST" + end, + case Dir of + "" -> + send_ctrl_message(State, mk_cmd(Cmd, "")); + _ -> + send_ctrl_message(State, mk_cmd(Cmd ++ " ~s", [Dir])) + end, + activate_ctrl_connection(State), + {noreply, State#state{caller = {dir, Dir}}}; + +handle_caller(#state{caller = {recv_bin, RemoteFile}} = State) -> + send_ctrl_message(State, mk_cmd("RETR ~s", [RemoteFile])), + activate_ctrl_connection(State), + {noreply, State#state{caller = recv_bin}}; + +handle_caller(#state{caller = {start_chunk_transfer, Cmd, RemoteFile}} = + State) -> + send_ctrl_message(State, mk_cmd("~s ~s", [Cmd, RemoteFile])), + activate_ctrl_connection(State), + {noreply, State#state{caller = start_chunk_transfer}}; + +handle_caller(#state{caller = {recv_file, RemoteFile, Fd}} = State) -> + send_ctrl_message(State, mk_cmd("RETR ~s", [RemoteFile])), + activate_ctrl_connection(State), + {noreply, State#state{caller = {recv_file, Fd}}}; + +handle_caller(#state{caller = {transfer_file, {Cmd, LocalFile, RemoteFile}}, + ldir = LocalDir, client = From} = State) -> + case file_open(filename:absname(LocalFile, LocalDir), read) of + {ok, Fd} -> + send_ctrl_message(State, mk_cmd("~s ~s", [Cmd, RemoteFile])), + activate_ctrl_connection(State), + {noreply, State#state{caller = {transfer_file, Fd}}}; + {error, _} -> + gen_server:reply(From, {error, epath}), + {noreply, State#state{client = undefined, caller = undefined, + dsock = undefined}} + end; + +handle_caller(#state{caller = {transfer_data, {Cmd, Bin, RemoteFile}}} = + State) -> + send_ctrl_message(State, mk_cmd("~s ~s", [Cmd, RemoteFile])), + activate_ctrl_connection(State), + {noreply, State#state{caller = {transfer_data, Bin}}}. + +%% ----------- FTP SERVER COMMUNICATION ------------------------- + +%% Connect to FTP server at Host (default is TCP port 21) +%% in order to establish a control connection. +setup_ctrl_connection(Host, Port, Timeout, State) -> + MsTime = millisec_time(), + case connect(Host, Port, Timeout, State) of + {ok, IpFam, CSock} -> + NewState = State#state{csock = CSock, ipfamily = IpFam}, + activate_ctrl_connection(NewState), + case Timeout - (millisec_time() - MsTime) of + Timeout2 when (Timeout2 >= 0) -> + {ok, NewState#state{caller = open}, Timeout2}; + _ -> + %% Oups: Simulate timeout + {ok, NewState#state{caller = open}, 0} + end; + Error -> + Error + end. + +setup_data_connection(#state{mode = active, + caller = Caller, + csock = CSock} = State) -> + IntToString = fun(Element) -> integer_to_list(Element) end, + + case (catch inet:sockname(CSock)) of + {ok, {{_, _, _, _, _, _, _, _} = IP, _}} -> + {ok, LSock} = + gen_tcp:listen(0, [{ip, IP}, {active, false}, + inet6, binary, {packet, 0}]), + {ok, Port} = inet:port(LSock), + Cmd = mk_cmd("EPRT |2|~s:~s:~s:~s:~s:~s:~s:~s|~s|", + lists:map(IntToString, + tuple_to_list(IP) ++ [Port])), + send_ctrl_message(State, Cmd), + activate_ctrl_connection(State), + {noreply, State#state{caller = {setup_data_connection, + {LSock, Caller}}}}; + {ok, {{_,_,_,_} = IP, _}} -> + {ok, LSock} = gen_tcp:listen(0, [{ip, IP}, {active, false}, + binary, {packet, 0}]), + {ok, Port} = inet:port(LSock), + {IP1, IP2, IP3, IP4} = IP, + {Port1, Port2} = {Port div 256, Port rem 256}, + send_ctrl_message(State, + mk_cmd("PORT ~w,~w,~w,~w,~w,~w", + [IP1, IP2, IP3, IP4, Port1, Port2])), + activate_ctrl_connection(State), + {noreply, State#state{caller = {setup_data_connection, + {LSock, Caller}}}} + end; + +setup_data_connection(#state{mode = passive, ipfamily = inet6, + caller = Caller} = State) -> + send_ctrl_message(State, mk_cmd("EPSV", [])), + activate_ctrl_connection(State), + {noreply, State#state{caller = {setup_data_connection, Caller}}}; + +setup_data_connection(#state{mode = passive, ipfamily = inet, + caller = Caller} = State) -> + send_ctrl_message(State, mk_cmd("PASV", [])), + activate_ctrl_connection(State), + {noreply, State#state{caller = {setup_data_connection, Caller}}}. + + +%% setup_data_connection(#state{mode = passive, ip_v6_disabled = false, +%% caller = Caller} = State) -> +%% send_ctrl_message(State, mk_cmd("EPSV", [])), +%% activate_ctrl_connection(State), +%% {noreply, State#state{caller = {setup_data_connection, Caller}}}; + +%% setup_data_connection(#state{mode = passive, ip_v6_disabled = true, +%% caller = Caller} = State) -> +%% send_ctrl_message(State, mk_cmd("PASV", [])), +%% activate_ctrl_connection(State), +%% {noreply, State#state{caller = {setup_data_connection, Caller}}}. + + +connect(Host, Port, Timeout, #state{ipfamily = inet = IpFam}) -> + connect2(Host, Port, IpFam, Timeout); + +connect(Host, Port, Timeout, #state{ipfamily = inet6 = IpFam}) -> + connect2(Host, Port, IpFam, Timeout); + +connect(Host, Port, Timeout, #state{ipfamily = inet6fb4}) -> + case inet:getaddr(Host, inet6) of + {ok, {0, 0, 0, 0, 0, 16#ffff, _, _} = IPv6} -> + case inet:getaddr(Host, inet) of + {ok, IPv4} -> + IpFam = inet, + connect2(IPv4, Port, IpFam, Timeout); + + _ -> + IpFam = inet6, + connect2(IPv6, Port, IpFam, Timeout) + end; + + {ok, IPv6} -> + IpFam = inet6, + connect2(IPv6, Port, IpFam, Timeout); + + _ -> + case inet:getaddr(Host, inet) of + {ok, IPv4} -> + IpFam = inet, + connect2(IPv4, Port, IpFam, Timeout); + Error -> + Error + end + end. + +connect2(Host, Port, IpFam, Timeout) -> + Opts = [IpFam, binary, {packet, 0}, {active, false}], + case gen_tcp:connect(Host, Port, Opts, Timeout) of + {ok, Sock} -> + {ok, IpFam, Sock}; + Error -> + Error + end. + + + +accept_data_connection(#state{mode = active, + dsock = {lsock, LSock}} = State) -> + {ok, Socket} = gen_tcp:accept(LSock), + gen_tcp:close(LSock), + State#state{dsock = Socket}; + +accept_data_connection(#state{mode = passive} = State) -> + State. + +send_ctrl_message(#state{csock = Socket, verbose = Verbose}, Message) -> + %% io:format("send control message: ~n~p~n", [lists:flatten(Message)]), + verbose(lists:flatten(Message),Verbose,send), + send_message(Socket, Message). + +send_data_message(#state{dsock = Socket}, Message) -> + send_message(Socket, Message). + +send_message(Socket, Message) -> + case gen_tcp:send(Socket, Message) of + ok -> + ok; + {error, Reason} -> + Report = io_lib:format("gen_tcp:send/2 failed for " + "reason ~p~n", [Reason]), + error_logger:error_report(Report), + %% If tcp does not work the only option is to terminate, + %% this is the expected behavior under these circumstances. + exit(normal) %% User will get error message from terminate/2 + end. + +activate_ctrl_connection(#state{csock = Socket, ctrl_data = {<<>>, _, _}}) -> + activate_connection(Socket); +activate_ctrl_connection(#state{csock = Socket}) -> + %% We have already received at least part of the next control message, + %% that has been saved in ctrl_data, process this first. + self() ! {tcp, Socket, <<>>}. + +activate_data_connection(#state{dsock = Socket}) -> + activate_connection(Socket). + +activate_connection(Socket) -> + inet:setopts(Socket, [{active, once}]). + +close_ctrl_connection(#state{csock = undefined}) -> + ok; +close_ctrl_connection(#state{csock = Socket}) -> + close_connection(Socket). + +close_data_connection(#state{dsock = undefined}) -> + ok; +close_data_connection(#state{dsock = {lsock, Socket}}) -> + close_connection(Socket); +close_data_connection(#state{dsock = Socket}) -> + close_connection(Socket). + +close_connection(Socket) -> + gen_tcp:close(Socket). + +%% ------------ FILE HANDELING ---------------------------------------- + +send_file(Fd, State) -> + case file_read(Fd) of + {ok, N, Bin} when N > 0-> + send_data_message(State, Bin), + progress_report({binary, Bin}, State), + send_file(Fd, State); + {ok, _, _} -> + file_close(Fd), + close_data_connection(State), + progress_report({transfer_size, 0}, State), + activate_ctrl_connection(State), + {noreply, State#state{caller = transfer_file_second_phase, + dsock = undefined}}; + {error, Reason} -> + gen_server:reply(State#state.client, {error, Reason}), + {stop, normal, State#state{client = undefined}} + end. + +file_open(File, Option) -> + file:open(File, [raw, binary, Option]). + +file_close(Fd) -> + file:close(Fd). + +file_read(Fd) -> + case file:read(Fd, ?FILE_BUFSIZE) of + {ok, Bytes} -> + {ok, size(Bytes), Bytes}; + eof -> + {ok, 0, []}; + Other -> + Other + end. + +file_write(Bytes, Fd) -> + file:write(Fd, Bytes). + +%% -------------- MISC ---------------------------------------------- + +call(GenServer, Msg, Format) -> + call(GenServer, Msg, Format, infinity). +call(GenServer, Msg, Format, Timeout) -> + Req = {self(), Msg}, + case (catch gen_server:call(GenServer, Req, Timeout)) of + {ok, Bin} when is_binary(Bin) andalso (Format =:= string) -> + {ok, binary_to_list(Bin)}; + {'EXIT', _} -> + {error, eclosed}; + Result -> + Result + end. + +cast(GenServer, Msg) -> + gen_server:cast(GenServer, {self(), Msg}). + +mk_cmd(Fmt, Args) -> + [io_lib:format(Fmt, Args)| [?CR, ?LF]]. % Deep list ok. + +pwd_result(Lines) -> + {_, [?DOUBLE_QUOTE | Rest]} = + lists:splitwith(fun(?DOUBLE_QUOTE) -> false; (_) -> true end, Lines), + {Dir, _} = + lists:splitwith(fun(?DOUBLE_QUOTE) -> false; (_) -> true end, Rest), + Dir. + +%% is_verbose(Params) -> +%% check_param(verbose, Params). + +%% is_debug(Flags) -> +%% check_param(debug, Flags). + +%% is_trace(Flags) -> +%% check_param(trace, Flags). + +%% is_ipv6_disabled(Flags) -> +%% check_param(ip_v6_disabled, Flags). + +%% check_param(Param, Params) -> +%% lists:member(Param, Params). + +key_search(Key, List, Default) -> + case lists:keysearch(Key, 1, List) of + {value, {_,Val}} -> + Val; + false -> + Default + end. + +%% check_option(Pred, Value, Default) -> +%% case Pred(Value) of +%% true -> +%% Value; +%% false -> +%% Default +%% end. + +verbose(Lines, true, Direction) -> + DirStr = + case Direction of + send -> + "Sending: "; + _ -> + "Receiving: " + end, + Str = string:strip(string:strip(Lines, right, ?LF), right, ?CR), + erlang:display(DirStr++Str); +verbose(_, false,_) -> + ok. + +progress(Options) -> + ftp_progress:start_link(Options). + +progress_report(_, #state{progress = ignore}) -> + ok; +progress_report(stop, #state{progress = ProgressPid}) -> + ftp_progress:stop(ProgressPid); +progress_report({binary, Data}, #state{progress = ProgressPid}) -> + ftp_progress:report(ProgressPid, {transfer_size, size(Data)}); +progress_report(Report, #state{progress = ProgressPid}) -> + ftp_progress:report(ProgressPid, Report). + + +millisec_time() -> + {A,B,C} = erlang:now(), + A*1000000000+B*1000+(C div 1000). diff --git a/lib/inets/src/ftp/ftp_internal.hrl b/lib/inets/src/ftp/ftp_internal.hrl new file mode 100644 index 0000000000..c3fa1e611d --- /dev/null +++ b/lib/inets/src/ftp/ftp_internal.hrl @@ -0,0 +1,31 @@ +%% +%% %CopyrightBegin% +%% +%% Copyright Ericsson AB 2005-2009. All Rights Reserved. +%% +%% The contents of this file are subject to the Erlang Public License, +%% Version 1.1, (the "License"); you may not use this file except in +%% compliance with the License. You should have received a copy of the +%% Erlang Public License along with this software. If not, it can be +%% retrieved online at http://www.erlang.org/. +%% +%% Software distributed under the License is distributed on an "AS IS" +%% basis, WITHOUT WARRANTY OF ANY KIND, either express or implied. See +%% the License for the specific language governing rights and limitations +%% under the License. +%% +%% %CopyrightEnd% +%% +%% + +-ifndef(ftp_internal_hrl). +-define(ftp_internal_hrl, true). + +-include("inets_internal.hrl"). +-define(SERVICE, ftpc). +-define(fcri(Label, Content), ?report_important(Label, ?SERVICE, Content)). +-define(fcrv(Label, Content), ?report_verbose(Label, ?SERVICE, Content)). +-define(fcrd(Label, Content), ?report_debug(Label, ?SERVICE, Content)). +-define(fcrt(Label, Content), ?report_trace(Label, ?SERVICE, Content)). + +-endif. % -ifdef(ftp_internal_hrl). diff --git a/lib/inets/src/ftp/ftp_progress.erl b/lib/inets/src/ftp/ftp_progress.erl new file mode 100644 index 0000000000..39f4d05bc2 --- /dev/null +++ b/lib/inets/src/ftp/ftp_progress.erl @@ -0,0 +1,127 @@ +%% +%% %CopyrightBegin% +%% +%% Copyright Ericsson AB 2005-2009. All Rights Reserved. +%% +%% The contents of this file are subject to the Erlang Public License, +%% Version 1.1, (the "License"); you may not use this file except in +%% compliance with the License. You should have received a copy of the +%% Erlang Public License along with this software. If not, it can be +%% retrieved online at http://www.erlang.org/. +%% +%% Software distributed under the License is distributed on an "AS IS" +%% basis, WITHOUT WARRANTY OF ANY KIND, either express or implied. See +%% the License for the specific language governing rights and limitations +%% under the License. +%% +%% %CopyrightEnd% +%% +%% +%% Description: This module impements a temporary process that +%% performes progress reporting during file transfer calling a user +%% defined callback function. Its life span is as long as the ftp connection +%% processes that spawned it lives. The purpose of this process is to +%% shild the ftp connection process from errors and time consuming operations +%% in the user defined callback function. + +-module(ftp_progress). + +%% Internal API +-export([start_link/1, report/2, stop/1]). + +%% Spawn export +-export([init/1]). + +-include_lib("kernel/include/file.hrl"). + +-record(progress, { + file, % string() + cb_module, % atom() + cb_function, % atom() + init_progress_term, % term() + current_progress_term % term() + }). + +%%%========================================================================= +%%% Internal application API +%%%========================================================================= +%%-------------------------------------------------------------------------- +%% start_link(Options) -> ignore | pid() +%% Options = ignore | {CBModule, CBFunction, InitProgressTerm} +%% +%% Description: Starts the progress report process unless progress reporting +%% should not be performed. +%%-------------------------------------------------------------------------- +start_link(ignore) -> + ignore; +start_link(Options) -> + spawn_link(?MODULE, init, [Options]). + +%%-------------------------------------------------------------------------- +%% report_progress(Pid, Report) -> _ +%% Pid = pid() +%% Report = {local_file, File} | {remote_file, File} | +%% {transfer_size, Size} +%% Size = integer() +%% +%% Description: Reports progress to the reporting process that calls the +%% user defined callback function. +%%-------------------------------------------------------------------------- +report(Pid, Report) -> + Pid ! {progress_report, Report}. + +%%-------------------------------------------------------------------------- +%% stop(Pid) -> _ +%% Pid = pid() +%% +%% Description: +%%-------------------------------------------------------------------------- +stop(Pid) -> + Pid ! stop. + +%%%========================================================================= +%%% Internal functions +%%%========================================================================= +init(Options) -> + loop(progress(Options)). + +loop(Progress) -> + receive + {progress_report, Report} -> + NewProgress = report_progress(Report, Progress), + loop(NewProgress); + stop -> + ok + end. + +progress({CBModule, CBFunction, InitProgressTerm}) when is_atom(CBModule), + is_atom(CBFunction) -> + #progress{cb_module = CBModule, + cb_function = CBFunction, + init_progress_term = InitProgressTerm, + current_progress_term = InitProgressTerm}. + +report_progress({local_file, File}, Progress) -> + {ok, FileInfo} = file:read_file_info(File), + report_progress({file_size, FileInfo#file_info.size}, + Progress#progress{file = File}); + +report_progress({remote_file, File}, Progress) -> + report_progress({file_size, unknown}, Progress#progress{file = File}); + +report_progress(Size, #progress{file = File, + cb_module = CBModule, + cb_function = CBFunction, + current_progress_term = Term, + init_progress_term = InitTerm} = Progress) -> + + NewProgressTerm = CBModule:CBFunction(Term, File, Size), + + case Size of + {transfer_size, 0} -> + %% Transfer is compleat reset initial values + Progress#progress{current_progress_term = InitTerm, + file = undefined}; + _ -> + Progress#progress{current_progress_term = NewProgressTerm} + end. diff --git a/lib/inets/src/ftp/ftp_response.erl b/lib/inets/src/ftp/ftp_response.erl new file mode 100644 index 0000000000..faeacb31ab --- /dev/null +++ b/lib/inets/src/ftp/ftp_response.erl @@ -0,0 +1,192 @@ +%% +%% %CopyrightBegin% +%% +%% Copyright Ericsson AB 2005-2009. All Rights Reserved. +%% +%% The contents of this file are subject to the Erlang Public License, +%% Version 1.1, (the "License"); you may not use this file except in +%% compliance with the License. You should have received a copy of the +%% Erlang Public License along with this software. If not, it can be +%% retrieved online at http://www.erlang.org/. +%% +%% Software distributed under the License is distributed on an "AS IS" +%% basis, WITHOUT WARRANTY OF ANY KIND, either express or implied. See +%% the License for the specific language governing rights and limitations +%% under the License. +%% +%% %CopyrightEnd% +%% +%% +%% Description: This module impements handling of ftp server responses. + +-module(ftp_response). + +%% Internal API +-export([parse_lines/3, interpret/1, error_string/1]). + +-include("ftp_internal.hrl"). + +%% First group of reply code digits +-define(POS_PREL, 1). +-define(POS_COMPL, 2). +-define(POS_INTERM, 3). +-define(TRANS_NEG_COMPL, 4). +-define(PERM_NEG_COMPL, 5). +%% Second group of reply code digits +-define(SYNTAX,0). +-define(INFORMATION,1). +-define(CONNECTION,2). +-define(AUTH_ACC,3). +-define(UNSPEC,4). +-define(FILE_SYSTEM,5). + +%%%========================================================================= +%%% INTERNAL API +%%%========================================================================= + +%%-------------------------------------------------------------------------- +%% parse_lines(Data, AccLines, StatusCode) -> {ok, Lines} | +%% {continue, {Data, +%% AccLines, StatusCode}} +%% +%% Data = binary() - data recived on the control connection from the +%% ftp-server. +%% AccLines = [string()] +%% StatusCode = start | {byte(), byte(), byte()} | finish - +%% Indicates where in the parsing process we are. +%% start - (looking for the status code of the message) +%% {byte(), byte(), byte()} - status code found, now +%% looking for the last line indication. +%% finish - now on the last line. +%% Description: Parses a ftp control response message. +%% "A reply is defined to contain the 3-digit code, followed by Space +%% <SP>, followed by one line of text (where some maximum line length +%% has been specified), and terminated by the Telnet end-of-line +%% code (CRLF), or a so called multilined reply for example: +%% +%% 123-First line +%% Second line +%% 234 A line beginning with numbers +%% 123 The last line +%% +%% The user-process then simply needs to search for the second +%% occurrence of the same reply code, followed by <SP> (Space), at +%% the beginning of a line, and ignore all intermediary lines. If +%% an intermediary line begins with a 3-digit number, the Server +%% will pad the front to avoid confusion. +%%-------------------------------------------------------------------------- + +%% Make sure we received the first 4 bytes so we know how to parse +%% the FTP server response e.i. is the response composed of one +%% or multiple lines. +parse_lines(Bin, Lines, start) when size(Bin) < 4 -> + {continue, {Bin, Lines, start}}; +%% Multiple lines exist +parse_lines(<<C1, C2, C3, $-, Rest/binary>>, Lines, start) -> + parse_lines(Rest, [$-, C3, C2, C1 | Lines], {C1, C2, C3}); +%% Only one line exists +parse_lines(<<C1, C2, C3, ?WHITE_SPACE, Bin/binary>>, Lines, start) -> + parse_lines(Bin, [?WHITE_SPACE, C3, C2, C1 | Lines], finish); + +%% Last line found +parse_lines(<<C1, C2, C3, ?WHITE_SPACE, Rest/binary>>, Lines, {C1, C2, C3}) -> + parse_lines(Rest, [?WHITE_SPACE, C3, C2, C1 | Lines], finish); +%% Potential end found wait for more data +parse_lines(<<C1, C2, C3>> = Bin, Lines, {C1, C2, C3}) -> + {continue, {Bin, Lines, {C1, C2, C3}}}; +%% Intermidate line begining with status code +parse_lines(<<C1, C2, C3, Rest/binary>>, Lines, {C1, C2, C3}) -> + parse_lines(Rest, [C3, C2, C1 | Lines], {C1, C2, C3}); + +%% Potential last line wait for more data +parse_lines(<<C1, C2>> = Data, Lines, {C1, C2, _} = StatusCode) -> + {continue, {Data, Lines, StatusCode}}; +parse_lines(<<C1>> = Data, Lines, {C1, _, _} = StatusCode) -> + {continue, {Data, Lines, StatusCode}}; +parse_lines(<<>> = Data, Lines, {_,_,_} = StatusCode) -> + {continue, {Data, Lines, StatusCode}}; +%% Part of the multiple lines +parse_lines(<<Octet, Rest/binary>>, Lines, {_,_, _} = StatusCode) -> + parse_lines(Rest, [Octet | Lines], StatusCode); + +%% End of FTP server response found +parse_lines(<<?CR, ?LF>>, Lines, finish) -> + {ok, lists:reverse([?LF, ?CR | Lines]), <<>>}; +parse_lines(<<?CR, ?LF, Rest/binary>>, Lines, finish) -> + {ok, lists:reverse([?LF, ?CR | Lines]), Rest}; + +%% Potential end found wait for more data +parse_lines(<<?CR>> = Data, Lines, finish) -> + {continue, {Data, Lines, finish}}; +parse_lines(<<>> = Data, Lines, finish) -> + {continue, {Data, Lines, finish}}; +%% Part of last line +parse_lines(<<Octet, Rest/binary>>, Lines, finish) -> + parse_lines(Rest, [Octet | Lines], finish). + +%%-------------------------------------------------------------------------- +%% interpret(Lines) -> {Status, Text} +%% Lines = [byte(), byte(), byte() | Text] - ftp server response as +%% returned by parse_lines/3 +%% Stauts = atom() (see interpret_status/3) +%% Text = [string()] +%% +%% Description: Create nicer data to match on. +%%-------------------------------------------------------------------------- +interpret([Didgit1, Didgit2, Didgit3 | Data]) -> + Code1 = Didgit1 - $0, + Code2 = Didgit2 - $0, + Code3 = Didgit3 - $0, + {interpret_status(Code1, Code2, Code3), Data}. + +%%-------------------------------------------------------------------------- +%% error_string(Error) -> string() +%% Error = {error, term()} | term() +%% +%% Description: Translates error codes into strings intended for +%% human interpretation. +%%-------------------------------------------------------------------------- +error_string({error, Reason}) -> + error_string(Reason); + +error_string(echunk) -> "Synchronisation error during chunk sending."; +error_string(eclosed) -> "Session has been closed."; +error_string(econn) -> "Connection to remote server prematurely closed."; +error_string(eexists) ->"File or directory already exists."; +error_string(ehost) -> "Host not found, FTP server not found, " + "or connection rejected."; +error_string(elogin) -> "User not logged in."; +error_string(enotbinary) -> "Term is not a binary."; +error_string(epath) -> "No such file or directory, already exists, " + "or permission denied."; +error_string(etype) -> "No such type."; +error_string(euser) -> "User name or password not valid."; +error_string(etnospc) -> "Insufficient storage space in system."; +error_string(epnospc) -> "Exceeded storage allocation " + "(for current directory or dataset)."; +error_string(efnamena) -> "File name not allowed."; +error_string(Reason) -> + lists:flatten(io_lib:format("Unknown error: ~w", [Reason])). + +%%%======================================================================== +%%% Internal functions +%%%======================================================================== + +%% Positive Preleminary Reply +interpret_status(?POS_PREL,_,_) -> pos_prel; +%% Positive Completion Reply +interpret_status(?POS_COMPL,_,_) -> pos_compl; +%% Positive Intermediate Reply nedd account +interpret_status(?POS_INTERM,?AUTH_ACC,2) -> pos_interm_acct; +%% Positive Intermediate Reply +interpret_status(?POS_INTERM,_,_) -> pos_interm; +%% No storage area no action taken +interpret_status(?TRANS_NEG_COMPL,?FILE_SYSTEM,2) -> etnospc; +%% Temporary Error, no action taken +interpret_status(?TRANS_NEG_COMPL,_,_) -> trans_neg_compl; +%% Permanent disk space error, the user shall not try again +interpret_status(?PERM_NEG_COMPL,?FILE_SYSTEM,0) -> epath; +interpret_status(?PERM_NEG_COMPL,?FILE_SYSTEM,2) -> epnospc; +interpret_status(?PERM_NEG_COMPL,?FILE_SYSTEM,3) -> efnamena; +interpret_status(?PERM_NEG_COMPL,_,_) -> perm_neg_compl. + diff --git a/lib/inets/src/ftp/ftp_sup.erl b/lib/inets/src/ftp/ftp_sup.erl new file mode 100644 index 0000000000..547170b671 --- /dev/null +++ b/lib/inets/src/ftp/ftp_sup.erl @@ -0,0 +1,59 @@ +%% +%% %CopyrightBegin% +%% +%% Copyright Ericsson AB 2004-2009. All Rights Reserved. +%% +%% The contents of this file are subject to the Erlang Public License, +%% Version 1.1, (the "License"); you may not use this file except in +%% compliance with the License. You should have received a copy of the +%% Erlang Public License along with this software. If not, it can be +%% retrieved online at http://www.erlang.org/. +%% +%% Software distributed under the License is distributed on an "AS IS" +%% basis, WITHOUT WARRANTY OF ANY KIND, either express or implied. See +%% the License for the specific language governing rights and limitations +%% under the License. +%% +%% %CopyrightEnd% +%% +%% +%%---------------------------------------------------------------------- +%% Purpose: The top supervisor for the ftp hangs under inets_sup. +%%---------------------------------------------------------------------- +-module(ftp_sup). + +-behaviour(supervisor). + +%% API +-export([start_link/0]). +-export([start_child/1]). + +%% Supervisor callback +-export([init/1]). + +%%%========================================================================= +%%% API +%%%========================================================================= +start_link() -> + supervisor:start_link({local, ?MODULE}, ?MODULE, []). + +start_child(Args) -> + supervisor:start_child(?MODULE, Args). + +%%%========================================================================= +%%% Supervisor callback +%%%========================================================================= +init(_) -> + RestartStrategy = simple_one_for_one, + MaxR = 0, + MaxT = 3600, + + Name = undefined, % As simple_one_for_one is used. + StartFunc = {ftp, start_link, []}, + Restart = temporary, % E.g. should not be restarted + Shutdown = 4000, + Modules = [ftp], + Type = worker, + + ChildSpec = {Name, StartFunc, Restart, Shutdown, Type, Modules}, + {ok, {{RestartStrategy, MaxR, MaxT}, [ChildSpec]}}. diff --git a/lib/inets/src/http_client/Makefile b/lib/inets/src/http_client/Makefile new file mode 100644 index 0000000000..23170f439f --- /dev/null +++ b/lib/inets/src/http_client/Makefile @@ -0,0 +1,102 @@ +# +# %CopyrightBegin% +# +# Copyright Ericsson AB 2005-2009. All Rights Reserved. +# +# The contents of this file are subject to the Erlang Public License, +# Version 1.1, (the "License"); you may not use this file except in +# compliance with the License. You should have received a copy of the +# Erlang Public License along with this software. If not, it can be +# retrieved online at http://www.erlang.org/. +# +# Software distributed under the License is distributed on an "AS IS" +# basis, WITHOUT WARRANTY OF ANY KIND, either express or implied. See +# the License for the specific language governing rights and limitations +# under the License. +# +# %CopyrightEnd% +# +# + +include $(ERL_TOP)/make/target.mk +EBIN = ../../ebin +include $(ERL_TOP)/make/$(TARGET)/otp.mk + +# ---------------------------------------------------- +# Application version +# ---------------------------------------------------- +include ../../vsn.mk + +VSN = $(INETS_VSN) + +# ---------------------------------------------------- +# Release directory specification +# ---------------------------------------------------- +RELSYSDIR = $(RELEASE_PATH)/lib/inets-$(VSN) + +# ---------------------------------------------------- +# Target Specs +# ---------------------------------------------------- +MODULES = \ + http \ + http_cookie \ + httpc_handler \ + httpc_manager \ + httpc_sup \ + httpc_handler_sup \ + httpc_profile_sup \ + httpc_response \ + httpc_request \ + http_uri \ + +HRL_FILES = httpc_internal.hrl + +ERL_FILES = $(MODULES:%=%.erl) + +TARGET_FILES= $(MODULES:%=$(EBIN)/%.$(EMULATOR)) + +# ---------------------------------------------------- +# INETS FLAGS +# ---------------------------------------------------- +INETS_FLAGS = -D'SERVER_SOFTWARE="inets/$(VSN)"' \ + +# ---------------------------------------------------- +# FLAGS +# ---------------------------------------------------- +INETS_ERL_FLAGS += -I ../http_lib -I ../inets_app -pa ../../ebin + +ERL_COMPILE_FLAGS += $(INETS_ERL_FLAGS)\ + $(INETS_FLAGS) \ + +'{parse_transform,sys_pre_attributes}' \ + +'{attribute,insert,app_vsn,$(APP_VSN)}' +# ---------------------------------------------------- +# Targets +# ---------------------------------------------------- + +debug opt: $(TARGET_FILES) + +clean: + rm -f $(TARGET_FILES) + rm -f core + +docs: + +# Release Target +# ---------------------------------------------------- +include $(ERL_TOP)/make/otp_release_targets.mk + +release_spec: opt + $(INSTALL_DIR) $(RELSYSDIR)/src + $(INSTALL_DATA) $(HRL_FILES) $(ERL_FILES) $(RELSYSDIR)/src + $(INSTALL_DIR) $(RELSYSDIR)/ebin + $(INSTALL_DATA) $(TARGET_FILES) $(RELSYSDIR)/ebin + +release_docs_spec: + +info: + @echo "INETS_DEBUG = $(INETS_DEBUG)" + @echo "INETS_FLAGS = $(INETS_FLAGS)" + @echo "ERL_COMPILE_FLAGS = $(ERL_COMPILE_FLAGS)" + + + diff --git a/lib/inets/src/http_client/http.erl b/lib/inets/src/http_client/http.erl new file mode 100644 index 0000000000..ce5d7723f0 --- /dev/null +++ b/lib/inets/src/http_client/http.erl @@ -0,0 +1,801 @@ +%% +%% %CopyrightBegin% +%% +%% Copyright Ericsson AB 2002-2009. All Rights Reserved. +%% +%% The contents of this file are subject to the Erlang Public License, +%% Version 1.1, (the "License"); you may not use this file except in +%% compliance with the License. You should have received a copy of the +%% Erlang Public License along with this software. If not, it can be +%% retrieved online at http://www.erlang.org/. +%% +%% Software distributed under the License is distributed on an "AS IS" +%% basis, WITHOUT WARRANTY OF ANY KIND, either express or implied. See +%% the License for the specific language governing rights and limitations +%% under the License. +%% +%% %CopyrightEnd% +%% +%% + +%% Description: +%%% This version of the HTTP/1.1 client supports: +%%% - RFC 2616 HTTP 1.1 client part +%%% - RFC 2818 HTTP Over TLS + +-module(http). +-behaviour(inets_service). + +%% API +-export([request/1, request/2, request/4, request/5, + cancel_request/1, cancel_request/2, + set_option/2, set_option/3, + set_options/1, set_options/2, + verify_cookies/2, verify_cookies/3, cookie_header/1, + cookie_header/2, stream_next/1, + default_profile/0]). + +%% Behavior callbacks +-export([start_standalone/1, start_service/1, + stop_service/1, services/0, service_info/1]). + +-include("http_internal.hrl"). +-include("httpc_internal.hrl"). + +-define(DEFAULT_PROFILE, default). + + +%%%========================================================================= +%%% API +%%%========================================================================= + +%%-------------------------------------------------------------------------- +%% request(Url [, Profile]) -> +%% {ok, {StatusLine, Headers, Body}} | {error,Reason} +%% +%% Url - string() +%% Description: Calls request/4 with default values. +%%-------------------------------------------------------------------------- + +request(Url) -> + request(Url, default_profile()). + +request(Url, Profile) -> + request(get, {Url, []}, [], [], Profile). + + +%%-------------------------------------------------------------------------- +%% request(Method, Request, HTTPOptions, Options [, Profile]) -> +%% {ok, {StatusLine, Headers, Body}} | {ok, {Status, Body}} | +%% {ok, RequestId} | {error,Reason} | {ok, {saved_as, FilePath} +%% +%% Method - atom() = head | get | put | post | trace | options| delete +%% Request - {Url, Headers} | {Url, Headers, ContentType, Body} +%% Url - string() +%% HTTPOptions - [HttpOption] +%% HTTPOption - {timeout, Time} | {connect_timeout, Time} | +%% {ssl, SSLOptions} | {proxy_auth, {User, Password}} +%% Ssloptions = [SSLOption] +%% SSLOption = {verify, code()} | {depth, depth()} | {certfile, path()} | +%% {keyfile, path()} | {password, string()} | {cacertfile, path()} | +%% {ciphers, string()} +%% Options - [Option] +%% Option - {sync, Boolean} | {body_format, BodyFormat} | +%% {full_result, Boolean} | {stream, To} | +%% {headers_as_is, Boolean} +%% StatusLine = {HTTPVersion, StatusCode, ReasonPhrase}</v> +%% HTTPVersion = string() +%% StatusCode = integer() +%% ReasonPhrase = string() +%% Headers = [Header] +%% Header = {Field, Value} +%% Field = string() +%% Value = string() +%% Body = string() | binary() - HTLM-code +%% +%% Description: Sends a HTTP-request. The function can be both +%% syncronus and asynchronous in the later case the function will +%% return {ok, RequestId} and later on a message will be sent to the +%% calling process on the format {http, {RequestId, {StatusLine, +%% Headers, Body}}} or {http, {RequestId, {error, Reason}}} +%%-------------------------------------------------------------------------- + +request(Method, Request, HttpOptions, Options) -> + request(Method, Request, HttpOptions, Options, default_profile()). + +request(Method, {Url, Headers}, HTTPOptions, Options, Profile) + when (Method =:= options) orelse + (Method =:= get) orelse + (Method =:= head) orelse + (Method =:= delete) orelse + (Method =:= trace) -> + case http_uri:parse(Url) of + {error, Reason} -> + {error, Reason}; + ParsedUrl -> + handle_request(Method, Url, ParsedUrl, Headers, [], [], + HTTPOptions, Options, Profile) + end; + +request(Method, {Url,Headers,ContentType,Body}, HTTPOptions, Options, Profile) + when (Method =:= post) orelse (Method =:= put) -> + case http_uri:parse(Url) of + {error, Reason} -> + {error, Reason}; + ParsedUrl -> + handle_request(Method, Url, + ParsedUrl, Headers, ContentType, Body, + HTTPOptions, Options, Profile) + end. + +%%-------------------------------------------------------------------------- +%% request(RequestId) -> ok +%% RequestId - As returned by request/4 +%% +%% Description: Cancels a HTTP-request. +%%------------------------------------------------------------------------- +cancel_request(RequestId) -> + cancel_request(RequestId, default_profile()). + +cancel_request(RequestId, Profile) -> + ok = httpc_manager:cancel_request(RequestId, profile_name(Profile)), + receive + %% If the request was allready fullfilled throw away the + %% answer as the request has been canceled. + {http, {RequestId, _}} -> + ok + after 0 -> + ok + end. + + +set_option(Key, Value) -> + set_option(Key, Value, default_profile()). + +set_option(Key, Value, Profile) -> + set_options([{Key, Value}], Profile). + + +%%-------------------------------------------------------------------------- +%% set_options(Options [, Profile]) -> ok | {error, Reason} +%% Options - [Option] +%% Profile - atom() +%% Option - {proxy, {Proxy, NoProxy}} | {max_sessions, MaxSessions} | +%% {max_pipeline_length, MaxPipeline} | +%% {pipeline_timeout, PipelineTimeout} | {cookies, CookieMode} | +%% {ipfamily, IpFamily} +%% Proxy - {Host, Port} +%% NoProxy - [Domain | HostName | IPAddress] +%% MaxSessions, MaxPipeline, PipelineTimeout = integer() +%% CookieMode - enabled | disabled | verify +%% IpFamily - inet | inet6 | inet6fb4 +%% Description: Informs the httpc_manager of the new settings. +%%------------------------------------------------------------------------- +set_options(Options) -> + set_options(Options, default_profile()). +set_options(Options, Profile) -> + case validate_options(Options) of + {ok, Opts} -> + try httpc_manager:set_options(Opts, profile_name(Profile)) of + Result -> + Result + catch + exit:{noproc, _} -> + {error, inets_not_started} + end; + {error, Reason} -> + {error, Reason} + end. + + +%%-------------------------------------------------------------------------- +%% verify_cookies(SetCookieHeaders, Url [, Profile]) -> ok | {error, reason} +%% +%% +%% Description: Store the cookies from <SetCookieHeaders> +%% in the cookie database +%% for the profile <Profile>. This function shall be used when the option +%% cookie is set to verify. +%%------------------------------------------------------------------------- +verify_cookies(SetCookieHeaders, Url) -> + verify_cookies(SetCookieHeaders, Url, default_profile()). + +verify_cookies(SetCookieHeaders, Url, Profile) -> + {_, _, Host, Port, Path, _} = http_uri:parse(Url), + ProfileName = profile_name(Profile), + Cookies = http_cookie:cookies(SetCookieHeaders, Path, Host), + try httpc_manager:store_cookies(Cookies, {Host, Port}, ProfileName) of + _ -> + ok + catch + exit:{noproc, _} -> + {error, {not_started, Profile}} + end. + +%%-------------------------------------------------------------------------- +%% cookie_header(Url [, Profile]) -> Header | {error, Reason} +%% +%% Description: Returns the cookie header that would be sent when making +%% a request to <Url>. +%%------------------------------------------------------------------------- +cookie_header(Url) -> + cookie_header(Url, default_profile()). + +cookie_header(Url, Profile) -> + try httpc_manager:cookies(Url, profile_name(Profile)) of + Header -> + Header + catch + exit:{noproc, _} -> + {error, {not_started, Profile}} + end. + + +stream_next(Pid) -> + httpc_handler:stream_next(Pid). + +%%%======================================================================== +%%% Behavior callbacks +%%%======================================================================== +start_standalone(PropList) -> + case proplists:get_value(profile, PropList) of + undefined -> + {error, no_profile}; + Profile -> + Dir = + proplists:get_value(data_dir, PropList, only_session_cookies), + httpc_manager:start_link({Profile, Dir}, stand_alone) + end. + +start_service(Config) -> + httpc_profile_sup:start_child(Config). + +stop_service(Profile) when is_atom(Profile) -> + httpc_profile_sup:stop_child(Profile); +stop_service(Pid) when is_pid(Pid) -> + case service_info(Pid) of + {ok, [{profile, Profile}]} -> + stop_service(Profile); + Error -> + Error + end. + +services() -> + [{httpc, Pid} || {_, Pid, _, _} <- + supervisor:which_children(httpc_profile_sup)]. +service_info(Pid) -> + try [{ChildName, ChildPid} || + {ChildName, ChildPid, _, _} <- + supervisor:which_children(httpc_profile_sup)] of + Children -> + child_name2info(child_name(Pid, Children)) + catch + exit:{noproc, _} -> + {error, service_not_available} + end. + + +%%%======================================================================== +%%% Internal functions +%%%======================================================================== +handle_request(Method, Url, + {Scheme, UserInfo, Host, Port, Path, Query}, + Headers, ContentType, Body, + HTTPOptions0, Options, Profile) -> + + HTTPOptions = http_options(HTTPOptions0), + Sync = proplists:get_value(sync, Options, true), + NewHeaders = lists:map(fun({Key, Val}) -> + {http_util:to_lower(Key), Val} end, + Headers), + Stream = proplists:get_value(stream, Options, none), + case {Sync, Stream} of + {true, self} -> + {error, streaming_error}; + _ -> + RecordHeaders = header_record(NewHeaders, + #http_request_h{}, + Host, + HTTPOptions#http_options.version), + Request = #request{from = self(), + scheme = Scheme, + address = {Host,Port}, + path = Path, + pquery = Query, + method = Method, + headers = RecordHeaders, + content = {ContentType,Body}, + settings = HTTPOptions, + abs_uri = Url, + userinfo = UserInfo, + stream = Stream, + headers_as_is = headers_as_is(Headers, Options)}, + try httpc_manager:request(Request, profile_name(Profile)) of + {ok, RequestId} -> + handle_answer(RequestId, Sync, Options); + {error, Reason} -> + {error, Reason} + catch + error:{noproc, _} -> + {error, {not_started, Profile}} + end + end. + + +handle_answer(RequestId, false, _) -> + {ok, RequestId}; +handle_answer(RequestId, true, Options) -> + receive + {http, {RequestId, saved_to_file}} -> + {ok, saved_to_file}; + {http, {RequestId, Result = {_,_,_}}} -> + return_answer(Options, Result); + {http, {RequestId, {error, Reason}}} -> + {error, Reason} + end. + +return_answer(Options, {{"HTTP/0.9",_,_}, _, BinBody}) -> + Body = format_body(BinBody, Options), + {ok, Body}; + +return_answer(Options, {StatusLine, Headers, BinBody}) -> + + Body = format_body(BinBody, Options), + + case proplists:get_value(full_result, Options, true) of + true -> + {ok, {StatusLine, Headers, Body}}; + false -> + {_, Status, _} = StatusLine, + {ok, {Status, Body}} + end. + +format_body(BinBody, Options) -> + case proplists:get_value(body_format, Options, string) of + string -> + binary_to_list(BinBody); + _ -> + BinBody + end. + +%% This options is a workaround for http servers that do not follow the +%% http standard and have case sensative header parsing. Should only be +%% used if there is no other way to communicate with the server or for +%% testing purpose. +headers_as_is(Headers, Options) -> + case proplists:get_value(headers_as_is, Options, false) of + false -> + []; + true -> + Headers + end. + + +http_options(HttpOptions) -> + HttpOptionsDefault = http_options_default(), + http_options(HttpOptionsDefault, HttpOptions, #http_options{}). + +http_options([], [], Acc) -> + Acc; +http_options([], HttpOptions, Acc) -> + Fun = fun(BadOption) -> + Report = io_lib:format("Invalid option ~p ignored ~n", + [BadOption]), + error_logger:info_report(Report) + end, + lists:foreach(Fun, HttpOptions), + Acc; +http_options([{Tag, Default, Idx, Post} | Defaults], HttpOptions, Acc) -> + case lists:keysearch(Tag, 1, HttpOptions) of + {value, {Tag, Val0}} -> + case Post(Val0) of + {ok, Val} -> + Acc2 = setelement(Idx, Acc, Val), + HttpOptions2 = lists:keydelete(Tag, 1, HttpOptions), + http_options(Defaults, HttpOptions2, Acc2); + error -> + Report = io_lib:format("Invalid option ~p:~p ignored ~n", + [Tag, Val0]), + error_logger:info_report(Report), + HttpOptions2 = lists:keydelete(Tag, 1, HttpOptions), + http_options(Defaults, HttpOptions2, Acc) + end; + false -> + DefaultVal = + case Default of + {value, Val} -> + Val; + {field, DefaultIdx} -> + element(DefaultIdx, Acc) + end, + Acc2 = setelement(Idx, Acc, DefaultVal), + http_options(Defaults, HttpOptions, Acc2) + end. + +http_options_default() -> + VersionPost = + fun(Value) when is_atom(Value) -> + {ok, http_util:to_upper(atom_to_list(Value))}; + (Value) when is_list(Value) -> + {ok, http_util:to_upper(Value)}; + (_) -> + error + end, + TimeoutPost = fun(Value) when is_integer(Value) andalso (Value >= 0) -> + {ok, Value}; + (infinity = Value) -> + {ok, Value}; + (_) -> + error + end, + AutoRedirectPost = fun(Value) when (Value =:= true) orelse + (Value =:= false) -> + {ok, Value}; + (_) -> + error + end, + SslPost = fun(Value) when is_list(Value) -> + {ok, Value}; + (_) -> + error + end, + ProxyAuthPost = fun({User, Passwd} = Value) when is_list(User) andalso + is_list(Passwd) -> + {ok, Value}; + (_) -> + error + end, + RelaxedPost = fun(Value) when (Value =:= true) orelse + (Value =:= false) -> + {ok, Value}; + (_) -> + error + end, + ConnTimeoutPost = + fun(Value) when is_integer(Value) andalso (Value >= 0) -> + {ok, Value}; + (infinity = Value) -> + {ok, Value}; + (_) -> + error + end, + [ + {version, {value, "HTTP/1.1"}, #http_options.version, VersionPost}, + {timeout, {value, ?HTTP_REQUEST_TIMEOUT}, #http_options.timeout, TimeoutPost}, + {autoredirect, {value, true}, #http_options.autoredirect, AutoRedirectPost}, + {ssl, {value, []}, #http_options.ssl, SslPost}, + {proxy_auth, {value, undefined}, #http_options.proxy_auth, ProxyAuthPost}, + {relaxed, {value, false}, #http_options.relaxed, RelaxedPost}, + %% this field has to be *after* the timeout field (as that field is used for the default value) + {connect_timeout, {field, #http_options.timeout}, #http_options.connect_timeout, ConnTimeoutPost} + ]. + +validate_options(Options) -> + (catch validate_options(Options, [])). + +validate_options([], ValidateOptions) -> + {ok, lists:reverse(ValidateOptions)}; + +validate_options([{proxy, Proxy} = Opt| Tail], Acc) -> + validate_proxy(Proxy), + validate_options(Tail, [Opt | Acc]); + +validate_options([{max_sessions, Value} = Opt| Tail], Acc) -> + validate_max_sessions(Value), + validate_options(Tail, [Opt | Acc]); + +validate_options([{keep_alive_timeout, Value} = Opt| Tail], Acc) -> + validate_keep_alive_timeout(Value), + validate_options(Tail, [Opt | Acc]); + +validate_options([{max_keep_alive_length, Value} = Opt| Tail], Acc) -> + validate_max_keep_alive_length(Value), + validate_options(Tail, [Opt | Acc]); + +validate_options([{pipeline_timeout, Value} = Opt| Tail], Acc) -> + validate_pipeline_timeout(Value), + validate_options(Tail, [Opt | Acc]); + +validate_options([{max_pipeline_length, Value} = Opt| Tail], Acc) -> + validate_max_pipeline_length(Value), + validate_options(Tail, [Opt | Acc]); + +validate_options([{cookies, Value} = Opt| Tail], Acc) -> + validate_cookies(Value), + validate_options(Tail, [Opt | Acc]); + +validate_options([{ipfamily, Value} = Opt| Tail], Acc) -> + validate_ipfamily(Value), + validate_options(Tail, [Opt | Acc]); + +%% For backward compatibillity +validate_options([{ipv6, Value}| Tail], Acc) -> + NewValue = validate_ipv6(Value), + Opt = {ipfamily, NewValue}, + validate_options(Tail, [Opt | Acc]); + +validate_options([{ip, Value} = Opt| Tail], Acc) -> + validate_ip(Value), + validate_options(Tail, [Opt | Acc]); + +validate_options([{port, Value} = Opt| Tail], Acc) -> + validate_port(Value), + validate_options(Tail, [Opt | Acc]); + +validate_options([{verbose, Value} = Opt| Tail], Acc) -> + validate_verbose(Value), + validate_options(Tail, [Opt | Acc]); + +validate_options([{_, _} = Opt| _], _Acc) -> + {error, {not_an_option, Opt}}. + + +validate_proxy({{ProxyHost, ProxyPort}, NoProxy} = Proxy) + when is_list(ProxyHost) andalso + is_integer(ProxyPort) andalso + is_list(NoProxy) -> + Proxy; +validate_proxy(BadProxy) -> + bad_option(proxy, BadProxy). + +validate_max_sessions(Value) when is_integer(Value) andalso (Value >= 0) -> + Value; +validate_max_sessions(BadValue) -> + bad_option(max_sessions, BadValue). + +validate_keep_alive_timeout(Value) when is_integer(Value) andalso (Value >= 0) -> + Value; +validate_keep_alive_timeout(infinity = Value) -> + Value; +validate_keep_alive_timeout(BadValue) -> + bad_option(keep_alive_timeout, BadValue). + +validate_max_keep_alive_length(Value) when is_integer(Value) andalso (Value >= 0) -> + Value; +validate_max_keep_alive_length(BadValue) -> + bad_option(max_keep_alive_length, BadValue). + +validate_pipeline_timeout(Value) when is_integer(Value) -> + Value; +validate_pipeline_timeout(infinity = Value) -> + Value; +validate_pipeline_timeout(BadValue) -> + bad_option(pipeline_timeout, BadValue). + +validate_max_pipeline_length(Value) when is_integer(Value) -> + Value; +validate_max_pipeline_length(BadValue) -> + bad_option(max_pipeline_length, BadValue). + +validate_cookies(Value) + when ((Value =:= enabled) orelse + (Value =:= disabled) orelse + (Value =:= verify)) -> + Value; +validate_cookies(BadValue) -> + bad_option(cookies, BadValue). + +validate_ipv6(Value) when (Value =:= enabled) orelse (Value =:= disabled) -> + case Value of + enabled -> + inet6fb4; + disabled -> + inet + end; +validate_ipv6(BadValue) -> + bad_option(ipv6, BadValue). + +validate_ipfamily(Value) + when (Value =:= inet) orelse (Value =:= inet6) orelse (Value =:= inet6fb4) -> + Value; +validate_ipfamily(BadValue) -> + bad_option(ipfamily, BadValue). + +validate_ip(Value) + when is_tuple(Value) andalso ((size(Value) =:= 4) orelse (size(Value) =:= 8)) -> + Value; +validate_ip(BadValue) -> + bad_option(ip, BadValue). + +validate_port(Value) when is_integer(Value) -> + Value; +validate_port(BadValue) -> + bad_option(port, BadValue). + +validate_verbose(Value) + when ((Value =:= false) orelse + (Value =:= verbose) orelse + (Value =:= debug) orelse + (Value =:= trace)) -> + ok; +validate_verbose(BadValue) -> + bad_option(verbose, BadValue). + +bad_option(Option, BadValue) -> + throw({error, {bad_option, Option, BadValue}}). + + + +header_record([], RequestHeaders, Host, Version) -> + validate_headers(RequestHeaders, Host, Version); +header_record([{"cache-control", Val} | Rest], RequestHeaders, Host, Version) -> + header_record(Rest, RequestHeaders#http_request_h{'cache-control' = Val}, + Host, Version); +header_record([{"connection", Val} | Rest], RequestHeaders, Host, Version) -> + header_record(Rest, RequestHeaders#http_request_h{connection = Val}, Host, + Version); +header_record([{"date", Val} | Rest], RequestHeaders, Host, Version) -> + header_record(Rest, RequestHeaders#http_request_h{date = Val}, Host, + Version); +header_record([{"pragma", Val} | Rest], RequestHeaders, Host, Version) -> + header_record(Rest, RequestHeaders#http_request_h{pragma = Val}, Host, + Version); +header_record([{"trailer", Val} | Rest], RequestHeaders, Host, Version) -> + header_record(Rest, RequestHeaders#http_request_h{trailer = Val}, Host, + Version); +header_record([{"transfer-encoding", Val} | Rest], RequestHeaders, Host, + Version) -> + header_record(Rest, + RequestHeaders#http_request_h{'transfer-encoding' = Val}, + Host, Version); +header_record([{"upgrade", Val} | Rest], RequestHeaders, Host, Version) -> + header_record(Rest, RequestHeaders#http_request_h{upgrade = Val}, Host, + Version); +header_record([{"via", Val} | Rest], RequestHeaders, Host, Version) -> + header_record(Rest, RequestHeaders#http_request_h{via = Val}, Host, + Version); +header_record([{"warning", Val} | Rest], RequestHeaders, Host, Version) -> + header_record(Rest, RequestHeaders#http_request_h{warning = Val}, Host, + Version); +header_record([{"accept", Val} | Rest], RequestHeaders, Host, Version) -> + header_record(Rest, RequestHeaders#http_request_h{accept = Val}, Host, + Version); +header_record([{"accept-charset", Val} | Rest], RequestHeaders, Host, Version) -> + header_record(Rest, RequestHeaders#http_request_h{'accept-charset' = Val}, + Host, Version); +header_record([{"accept-encoding", Val} | Rest], RequestHeaders, Host, + Version) -> + header_record(Rest, RequestHeaders#http_request_h{'accept-encoding' = Val}, + Host, Version); +header_record([{"accept-language", Val} | Rest], RequestHeaders, Host, + Version) -> + header_record(Rest, RequestHeaders#http_request_h{'accept-language' = Val}, + Host, Version); +header_record([{"authorization", Val} | Rest], RequestHeaders, Host, Version) -> + header_record(Rest, RequestHeaders#http_request_h{authorization = Val}, + Host, Version); +header_record([{"expect", Val} | Rest], RequestHeaders, Host, Version) -> + header_record(Rest, RequestHeaders#http_request_h{expect = Val}, Host, + Version); +header_record([{"from", Val} | Rest], RequestHeaders, Host, Version) -> + header_record(Rest, RequestHeaders#http_request_h{from = Val}, Host, + Version); +header_record([{"host", Val} | Rest], RequestHeaders, Host, Version) -> + header_record(Rest, RequestHeaders#http_request_h{host = Val}, Host, + Version); +header_record([{"if-match", Val} | Rest], RequestHeaders, Host, Version) -> + header_record(Rest, RequestHeaders#http_request_h{'if-match' = Val}, + Host, Version); +header_record([{"if-modified-since", Val} | Rest], RequestHeaders, Host, + Version) -> + header_record(Rest, + RequestHeaders#http_request_h{'if-modified-since' = Val}, + Host, Version); +header_record([{"if-none-match", Val} | Rest], RequestHeaders, Host, Version) -> + header_record(Rest, RequestHeaders#http_request_h{'if-none-match' = Val}, + Host, Version); +header_record([{"if-range", Val} | Rest], RequestHeaders, Host, Version) -> + header_record(Rest, RequestHeaders#http_request_h{'if-range' = Val}, + Host, Version); + +header_record([{"if-unmodified-since", Val} | Rest], RequestHeaders, Host, + Version) -> + header_record(Rest, RequestHeaders#http_request_h{'if-unmodified-since' + = Val}, Host, Version); +header_record([{"max-forwards", Val} | Rest], RequestHeaders, Host, Version) -> + header_record(Rest, RequestHeaders#http_request_h{'max-forwards' = Val}, + Host, Version); +header_record([{"proxy-authorization", Val} | Rest], RequestHeaders, Host, + Version) -> + header_record(Rest, RequestHeaders#http_request_h{'proxy-authorization' + = Val}, Host, Version); +header_record([{"range", Val} | Rest], RequestHeaders, Host, Version) -> + header_record(Rest, RequestHeaders#http_request_h{range = Val}, Host, + Version); +header_record([{"referer", Val} | Rest], RequestHeaders, Host, Version) -> + header_record(Rest, RequestHeaders#http_request_h{referer = Val}, Host, + Version); +header_record([{"te", Val} | Rest], RequestHeaders, Host, Version) -> + header_record(Rest, RequestHeaders#http_request_h{te = Val}, Host, + Version); +header_record([{"user-agent", Val} | Rest], RequestHeaders, Host, Version) -> + header_record(Rest, RequestHeaders#http_request_h{'user-agent' = Val}, + Host, Version); +header_record([{"allow", Val} | Rest], RequestHeaders, Host, Version) -> + header_record(Rest, RequestHeaders#http_request_h{allow = Val}, Host, + Version); +header_record([{"content-encoding", Val} | Rest], RequestHeaders, Host, + Version) -> + header_record(Rest, + RequestHeaders#http_request_h{'content-encoding' = Val}, + Host, Version); +header_record([{"content-language", Val} | Rest], RequestHeaders, + Host, Version) -> + header_record(Rest, + RequestHeaders#http_request_h{'content-language' = Val}, + Host, Version); +header_record([{"content-length", Val} | Rest], RequestHeaders, Host, Version) -> + header_record(Rest, RequestHeaders#http_request_h{'content-length' = Val}, + Host, Version); +header_record([{"content-location", Val} | Rest], RequestHeaders, + Host, Version) -> + header_record(Rest, + RequestHeaders#http_request_h{'content-location' = Val}, + Host, Version); +header_record([{"content-md5", Val} | Rest], RequestHeaders, Host, Version) -> + header_record(Rest, RequestHeaders#http_request_h{'content-md5' = Val}, + Host, Version); +header_record([{"content-range", Val} | Rest], RequestHeaders, Host, Version) -> + header_record(Rest, RequestHeaders#http_request_h{'content-range' = Val}, + Host, Version); +header_record([{"content-type", Val} | Rest], RequestHeaders, Host, Version) -> + header_record(Rest, RequestHeaders#http_request_h{'content-type' = Val}, + Host, Version); +header_record([{"expires", Val} | Rest], RequestHeaders, Host, Version) -> + header_record(Rest, RequestHeaders#http_request_h{expires = Val}, Host, + Version); +header_record([{"last-modified", Val} | Rest], RequestHeaders, Host, Version) -> + header_record(Rest, RequestHeaders#http_request_h{'last-modified' = Val}, + Host, Version); +header_record([{Key, Val} | Rest], RequestHeaders, Host, Version) -> + header_record(Rest, RequestHeaders#http_request_h{ + other = [{Key, Val} | + RequestHeaders#http_request_h.other]}, + Host, Version). + +validate_headers(RequestHeaders = #http_request_h{te = undefined}, Host, + "HTTP/1.1" = Version) -> + validate_headers(RequestHeaders#http_request_h{te = ""}, Host, + "HTTP/1.1" = Version); +validate_headers(RequestHeaders = #http_request_h{host = undefined}, + Host, "HTTP/1.1" = Version) -> + validate_headers(RequestHeaders#http_request_h{host = Host}, Host, Version); +validate_headers(RequestHeaders, _, _) -> + RequestHeaders. + + +default_profile() -> + ?DEFAULT_PROFILE. + +profile_name(?DEFAULT_PROFILE) -> + httpc_manager; +profile_name(Pid) when is_pid(Pid) -> + Pid; +profile_name(Profile) -> + list_to_atom("httpc_manager_" ++ atom_to_list(Profile)). + +child_name2info(undefined) -> + {error, no_such_service}; +child_name2info(httpc_manager) -> + {ok, [{profile, default}]}; +child_name2info({http, Profile}) -> + {ok, [{profile, Profile}]}. + +child_name(_, []) -> + undefined; +child_name(Pid, [{Name, Pid} | _]) -> + Name; +child_name(Pid, [_ | Children]) -> + child_name(Pid, Children). + +%% d(F) -> +%% d(F, []). + +%% d(F, A) -> +%% d(get(dbg), F, A). + +%% d(true, F, A) -> +%% io:format(user, "~w:~w:" ++ F ++ "~n", [self(), ?MODULE | A]); +%% d(_, _, _) -> +%% ok. + diff --git a/lib/inets/src/http_client/http_cookie.erl b/lib/inets/src/http_client/http_cookie.erl new file mode 100644 index 0000000000..e091070f72 --- /dev/null +++ b/lib/inets/src/http_client/http_cookie.erl @@ -0,0 +1,391 @@ +%% +%% %CopyrightBegin% +%% +%% Copyright Ericsson AB 2004-2009. All Rights Reserved. +%% +%% The contents of this file are subject to the Erlang Public License, +%% Version 1.1, (the "License"); you may not use this file except in +%% compliance with the License. You should have received a copy of the +%% Erlang Public License along with this software. If not, it can be +%% retrieved online at http://www.erlang.org/. +%% +%% Software distributed under the License is distributed on an "AS IS" +%% basis, WITHOUT WARRANTY OF ANY KIND, either express or implied. See +%% the License for the specific language governing rights and limitations +%% under the License. +%% +%% %CopyrightEnd% +%% +%% Description: Cookie handling according to RFC 2109 + +-module(http_cookie). + +-include("httpc_internal.hrl"). + +-export([header/4, cookies/3, open_cookie_db/1, close_cookie_db/1, insert/2]). + +%%%========================================================================= +%%% API +%%%========================================================================= +header(Scheme, {Host, _}, Path, CookieDb) -> + case lookup_cookies(Host, Path, CookieDb) of + [] -> + {"cookie", ""}; + Cookies -> + {"cookie", cookies_to_string(Scheme, Cookies)} + end. + +cookies(Headers, RequestPath, RequestHost) -> + Cookies = parse_set_cookies(Headers, {RequestPath, RequestHost}), + accept_cookies(Cookies, RequestPath, RequestHost). + +open_cookie_db({{_, only_session_cookies}, SessionDbName}) -> + EtsDb = ets:new(SessionDbName, [protected, bag, + {keypos, #http_cookie.domain}]), + {undefined, EtsDb}; + +open_cookie_db({{DbName, Dbdir}, SessionDbName}) -> + File = filename:join(Dbdir, atom_to_list(DbName)), + {ok, DetsDb} = dets:open_file(DbName, [{keypos, #http_cookie.domain}, + {type, bag}, + {file, File}, + {ram_file, true}]), + EtsDb = ets:new(SessionDbName, [protected, bag, + {keypos, #http_cookie.domain}]), + {DetsDb, EtsDb}. + +close_cookie_db({undefined, EtsDb}) -> + ets:delete(EtsDb); + +close_cookie_db({DetsDb, EtsDb}) -> + dets:close(DetsDb), + ets:delete(EtsDb). + +%% If no persistent cookie database is defined we +%% treat all cookies as if they where session cookies. +insert(Cookie = #http_cookie{max_age = Int}, + Dbs = {undefined, _}) when is_integer(Int) -> + insert(Cookie#http_cookie{max_age = session}, Dbs); + +insert(Cookie = #http_cookie{domain = Key, name = Name, + path = Path, max_age = session}, + Db = {_, CookieDb}) -> + case ets:match_object(CookieDb, #http_cookie{domain = Key, + name = Name, + path = Path, + _ = '_'}) of + [] -> + ets:insert(CookieDb, Cookie); + [NewCookie] -> + delete(NewCookie, Db), + ets:insert(CookieDb, Cookie) + end, + ok; +insert(#http_cookie{domain = Key, name = Name, + path = Path, max_age = 0}, + Db = {CookieDb, _}) -> + case dets:match_object(CookieDb, #http_cookie{domain = Key, + name = Name, + path = Path, + _ = '_'}) of + [] -> + ok; + [NewCookie] -> + delete(NewCookie, Db) + end, + ok; +insert(Cookie = #http_cookie{domain = Key, name = Name, path = Path}, + Db = {CookieDb, _}) -> + case dets:match_object(CookieDb, #http_cookie{domain = Key, + name = Name, + path = Path, + _ = '_'}) of + [] -> + dets:insert(CookieDb, Cookie); + [NewCookie] -> + delete(NewCookie, Db), + dets:insert(CookieDb, Cookie) + end, + ok. + +%%%======================================================================== +%%% Internal functions +%%%======================================================================== +lookup_cookies(Key, {undefined, Ets}) -> + ets:match_object(Ets, #http_cookie{domain = Key, + _ = '_'}); +lookup_cookies(Key, {Dets,Ets}) -> + SessionCookies = ets:match_object(Ets, #http_cookie{domain = Key, + _ = '_'}), + Cookies = dets:match_object(Dets, #http_cookie{domain = Key, + _ = '_'}), + Cookies ++ SessionCookies. + +delete(Cookie = #http_cookie{max_age = session}, {_, CookieDb}) -> + ets:delete_object(CookieDb, Cookie); +delete(Cookie, {CookieDb, _}) -> + dets:delete_object(CookieDb, Cookie). + +lookup_cookies(Host, Path, Db) -> + Cookies = + case http_util:is_hostname(Host) of + true -> + HostCookies = lookup_cookies(Host, Db), + [_| DomainParts] = string:tokens(Host, "."), + lookup_domain_cookies(DomainParts, Db, HostCookies); + false -> % IP-adress + lookup_cookies(Host, Db) + end, + ValidCookies = valid_cookies(Cookies, [], Db), + lists:filter(fun(Cookie) -> + lists:prefix(Cookie#http_cookie.path, Path) + end, ValidCookies). + +%% For instance if Host=localhost +lookup_domain_cookies([], _, AccCookies) -> + lists:flatten(AccCookies); +%% Top domains can not have cookies +lookup_domain_cookies([_], _, AccCookies) -> + lists:flatten(AccCookies); +lookup_domain_cookies([Next | DomainParts], CookieDb, AccCookies) -> + Domain = merge_domain_parts(DomainParts, [Next ++ "."]), + lookup_domain_cookies(DomainParts, CookieDb, + [lookup_cookies(Domain, CookieDb) + | AccCookies]). + +merge_domain_parts([Part], Merged) -> + lists:flatten(["." | lists:reverse([Part | Merged])]); +merge_domain_parts([Part| Rest], Merged) -> + merge_domain_parts(Rest, [".", Part | Merged]). + +cookies_to_string(Scheme, Cookies = [Cookie | _]) -> + Version = "$Version=" ++ Cookie#http_cookie.version ++ "; ", + cookies_to_string(Scheme, path_sort(Cookies), [Version]). + +cookies_to_string(_, [], CookieStrs) -> + case length(CookieStrs) of + 1 -> + ""; + _ -> + lists:flatten(lists:reverse(CookieStrs)) + end; + +cookies_to_string(https, [Cookie = #http_cookie{secure = true}| Cookies], + CookieStrs) -> + Str = case Cookies of + [] -> + cookie_to_string(Cookie); + _ -> + cookie_to_string(Cookie) ++ "; " + end, + cookies_to_string(https, Cookies, [Str | CookieStrs]); + +cookies_to_string(Scheme, [#http_cookie{secure = true}| Cookies], + CookieStrs) -> + cookies_to_string(Scheme, Cookies, CookieStrs); + +cookies_to_string(Scheme, [Cookie | Cookies], CookieStrs) -> + Str = case Cookies of + [] -> + cookie_to_string(Cookie); + _ -> + cookie_to_string(Cookie) ++ "; " + end, + cookies_to_string(Scheme, Cookies, [Str | CookieStrs]). + +cookie_to_string(Cookie = #http_cookie{name = Name, value = Value}) -> + Str = Name ++ "=" ++ Value, + add_domain(add_path(Str, Cookie), Cookie). + +add_path(Str, #http_cookie{path_default = true}) -> + Str; +add_path(Str, #http_cookie{path = Path}) -> + Str ++ "; $Path=" ++ Path. + +add_domain(Str, #http_cookie{domain_default = true}) -> + Str; +add_domain(Str, #http_cookie{domain = Domain}) -> + Str ++ "; $Domain=" ++ Domain. + +parse_set_cookies(OtherHeaders, DefaultPathDomain) -> + SetCookieHeaders = lists:foldl(fun({"set-cookie", Value}, Acc) -> + [string:tokens(Value, ",")| Acc]; + (_, Acc) -> + Acc + end, [], OtherHeaders), + + lists:flatten(lists:map(fun(CookieHeader) -> + NewHeader = + fix_netscape_cookie(CookieHeader, + []), + parse_set_cookie(NewHeader, [], + DefaultPathDomain) end, + SetCookieHeaders)). + +parse_set_cookie([], AccCookies, _) -> + AccCookies; +parse_set_cookie([CookieHeader | CookieHeaders], AccCookies, + Defaults = {DefaultPath, DefaultDomain}) -> + [CookieStr | Attributes] = case string:tokens(CookieHeader, ";") of + [CStr] -> + [CStr, ""]; + [CStr | Attr] -> + [CStr, Attr] + end, + Pos = string:chr(CookieStr, $=), + Name = string:substr(CookieStr, 1, Pos - 1), + Value = string:substr(CookieStr, Pos + 1), + Cookie = #http_cookie{name = string:strip(Name), + value = string:strip(Value)}, + NewAttributes = parse_set_cookie_attributes(Attributes), + TmpCookie = cookie_attributes(NewAttributes, Cookie), + %% Add runtime defult values if necessary + NewCookie = domain_default(path_default(TmpCookie, DefaultPath), + DefaultDomain), + parse_set_cookie(CookieHeaders, [NewCookie | AccCookies], Defaults). + +parse_set_cookie_attributes([]) -> + []; +parse_set_cookie_attributes([Attributes]) -> + lists:map(fun(Attr) -> + [AttrName, AttrValue] = + case string:tokens(Attr, "=") of + %% All attributes have the form + %% Name=Value except "secure"! + [Name] -> + [Name, ""]; + [Name, Value] -> + [Name, Value]; + %% Anything not expected will be + %% disregarded + _ -> + ["Dummy",""] + end, + {http_util:to_lower(string:strip(AttrName)), + string:strip(AttrValue)} + end, Attributes). + +cookie_attributes([], Cookie) -> + Cookie; +cookie_attributes([{"comment", Value}| Attributes], Cookie) -> + cookie_attributes(Attributes, + Cookie#http_cookie{comment = Value}); +cookie_attributes([{"domain", Value}| Attributes], Cookie) -> + cookie_attributes(Attributes, + Cookie#http_cookie{domain = Value}); +cookie_attributes([{"max-age", Value}| Attributes], Cookie) -> + ExpireTime = cookie_expires(list_to_integer(Value)), + cookie_attributes(Attributes, + Cookie#http_cookie{max_age = ExpireTime}); +%% Backwards compatibility with netscape cookies +cookie_attributes([{"expires", Value}| Attributes], Cookie) -> + Time = http_util:convert_netscapecookie_date(Value), + ExpireTime = calendar:datetime_to_gregorian_seconds(Time), + cookie_attributes(Attributes, + Cookie#http_cookie{max_age = ExpireTime}); +cookie_attributes([{"path", Value}| Attributes], Cookie) -> + cookie_attributes(Attributes, + Cookie#http_cookie{path = Value}); +cookie_attributes([{"secure", _}| Attributes], Cookie) -> + cookie_attributes(Attributes, + Cookie#http_cookie{secure = true}); +cookie_attributes([{"version", Value}| Attributes], Cookie) -> + cookie_attributes(Attributes, + Cookie#http_cookie{version = Value}); +%% Disregard unknown attributes. +cookie_attributes([_| Attributes], Cookie) -> + cookie_attributes(Attributes, Cookie). + +domain_default(Cookie = #http_cookie{domain = undefined}, + DefaultDomain) -> + Cookie#http_cookie{domain = DefaultDomain, domain_default = true}; +domain_default(Cookie, _) -> + Cookie. + +path_default(Cookie = #http_cookie{path = undefined}, + DefaultPath) -> + Cookie#http_cookie{path = skip_right_most_slash(DefaultPath), + path_default = true}; +path_default(Cookie, _) -> + Cookie. + +%% Note: if the path is only / that / will be keept +skip_right_most_slash("/") -> + "/"; +skip_right_most_slash(Str) -> + string:strip(Str, right, $/). + +accept_cookies(Cookies, RequestPath, RequestHost) -> + lists:filter(fun(Cookie) -> + accept_cookie(Cookie, RequestPath, RequestHost) + end, Cookies). + +accept_cookie(Cookie, RequestPath, RequestHost) -> + accept_path(Cookie, RequestPath) and accept_domain(Cookie, RequestHost). + +accept_path(#http_cookie{path = Path}, RequestPath) -> + lists:prefix(Path, RequestPath). + +accept_domain(#http_cookie{domain = RequestHost}, RequestHost) -> + true; + +accept_domain(#http_cookie{domain = Domain}, RequestHost) -> + HostCheck = case http_util:is_hostname(RequestHost) of + true -> + (lists:suffix(Domain, RequestHost) andalso + (not + lists:member($., + string:substr(RequestHost, 1, + (length(RequestHost) - + length(Domain)))))); + false -> + false + end, + HostCheck andalso (hd(Domain) == $.) + andalso (length(string:tokens(Domain, ".")) > 1). + +cookie_expires(0) -> + 0; +cookie_expires(DeltaSec) -> + NowSec = calendar:datetime_to_gregorian_seconds({date(), time()}), + NowSec + DeltaSec. + +is_cookie_expired(#http_cookie{max_age = session}) -> + false; +is_cookie_expired(#http_cookie{max_age = ExpireTime}) -> + NowSec = calendar:datetime_to_gregorian_seconds({date(), time()}), + ExpireTime - NowSec =< 0. + +valid_cookies([], Valid, _) -> + Valid; + +valid_cookies([Cookie | Cookies], Valid, Db) -> + case is_cookie_expired(Cookie) of + true -> + delete(Cookie, Db), + valid_cookies(Cookies, Valid, Db); + false -> + valid_cookies(Cookies, [Cookie | Valid], Db) + end. + +path_sort(Cookies)-> + lists:reverse(lists:keysort(#http_cookie.path, Cookies)). + + +%% Informally, the Set-Cookie response header comprises the token +%% Set-Cookie:, followed by a comma-separated list of one or more +%% cookies. Netscape cookies expires attribute may also have a +%% , in this case the header list will have been incorrectly split +%% in parse_set_cookies/2 this functions fixs that problem. +fix_netscape_cookie([Cookie1, Cookie2 | Rest], Acc) -> + case inets_regexp:match(Cookie1, "expires=") of + {_, _, _} -> + fix_netscape_cookie(Rest, [Cookie1 ++ Cookie2 | Acc]); + nomatch -> + fix_netscape_cookie([Cookie2 |Rest], [Cookie1| Acc]) + end; +fix_netscape_cookie([Cookie | Rest], Acc) -> + fix_netscape_cookie(Rest, [Cookie | Acc]); + +fix_netscape_cookie([], Acc) -> + Acc. diff --git a/lib/inets/src/http_client/http_uri.erl b/lib/inets/src/http_client/http_uri.erl new file mode 100644 index 0000000000..615a0d8ec4 --- /dev/null +++ b/lib/inets/src/http_client/http_uri.erl @@ -0,0 +1,116 @@ +%% +%% %CopyrightBegin% +%% +%% Copyright Ericsson AB 2006-2009. All Rights Reserved. +%% +%% The contents of this file are subject to the Erlang Public License, +%% Version 1.1, (the "License"); you may not use this file except in +%% compliance with the License. You should have received a copy of the +%% Erlang Public License along with this software. If not, it can be +%% retrieved online at http://www.erlang.org/. +%% +%% Software distributed under the License is distributed on an "AS IS" +%% basis, WITHOUT WARRANTY OF ANY KIND, either express or implied. See +%% the License for the specific language governing rights and limitations +%% under the License. +%% +%% %CopyrightEnd% +%% +%% + +-module(http_uri). + +-export([parse/1]). + +%%%========================================================================= +%%% API +%%%========================================================================= +parse(AbsURI) -> + case parse_scheme(AbsURI) of + {error, Reason} -> + {error, Reason}; + {Scheme, Rest} -> + case (catch parse_uri_rest(Scheme, Rest)) of + {UserInfo, Host, Port, Path, Query} -> + {Scheme, UserInfo, Host, Port, Path, Query}; + _ -> + {error, {malformed_url, AbsURI}} + end + end. + +%%%======================================================================== +%%% Internal functions +%%%======================================================================== +parse_scheme(AbsURI) -> + case split_uri(AbsURI, ":", {error, no_scheme}, 1, 1) of + {error, no_scheme} -> + {error, no_scheme}; + {StrScheme, Rest} -> + case list_to_atom(http_util:to_lower(StrScheme)) of + Scheme when Scheme == http; Scheme == https -> + {Scheme, Rest}; + Scheme -> + {error, {not_supported_scheme, Scheme}} + end + end. + +parse_uri_rest(Scheme, "//" ++ URIPart) -> + + {Authority, PathQuery} = + case split_uri(URIPart, "/", URIPart, 1, 0) of + Split = {_, _} -> + Split; + URIPart -> + case split_uri(URIPart, "\\?", URIPart, 1, 0) of + Split = {_, _} -> + Split; + URIPart -> + {URIPart,""} + end + end, + + {UserInfo, HostPort} = split_uri(Authority, "@", {"", Authority}, 1, 1), + {Host, Port} = parse_host_port(Scheme, HostPort), + {Path, Query} = parse_path_query(PathQuery), + {UserInfo, Host, Port, Path, Query}. + + +parse_path_query(PathQuery) -> + {Path, Query} = split_uri(PathQuery, "\\?", {PathQuery, ""}, 1, 0), + {path(Path), Query}. + + +parse_host_port(Scheme,"[" ++ HostPort) -> %ipv6 + DefaultPort = default_port(Scheme), + {Host, ColonPort} = split_uri(HostPort, "\\]", {HostPort, ""}, 1, 1), + {_, Port} = split_uri(ColonPort, ":", {"", DefaultPort}, 0, 1), + {Host, int_port(Port)}; + +parse_host_port(Scheme, HostPort) -> + DefaultPort = default_port(Scheme), + {Host, Port} = split_uri(HostPort, ":", {HostPort, DefaultPort}, 1, 1), + {Host, int_port(Port)}. + +split_uri(UriPart, SplitChar, NoMatchResult, SkipLeft, SkipRight) -> + case inets_regexp:first_match(UriPart, SplitChar) of + {match, Match, _} -> + {string:substr(UriPart, 1, Match - SkipLeft), + string:substr(UriPart, Match + SkipRight, length(UriPart))}; + nomatch -> + NoMatchResult + end. + +default_port(http) -> + 80; +default_port(https) -> + 443. + +int_port(Port) when is_integer(Port) -> + Port; +int_port(Port) when is_list(Port) -> + list_to_integer(Port). + +path("") -> + "/"; +path(Path) -> + Path. diff --git a/lib/inets/src/http_client/httpc_handler.erl b/lib/inets/src/http_client/httpc_handler.erl new file mode 100644 index 0000000000..7b737c2f86 --- /dev/null +++ b/lib/inets/src/http_client/httpc_handler.erl @@ -0,0 +1,1499 @@ +%% +%% %CopyrightBegin% +%% +%% Copyright Ericsson AB 2002-2009. All Rights Reserved. +%% +%% The contents of this file are subject to the Erlang Public License, +%% Version 1.1, (the "License"); you may not use this file except in +%% compliance with the License. You should have received a copy of the +%% Erlang Public License along with this software. If not, it can be +%% retrieved online at http://www.erlang.org/. +%% +%% Software distributed under the License is distributed on an "AS IS" +%% basis, WITHOUT WARRANTY OF ANY KIND, either express or implied. See +%% the License for the specific language governing rights and limitations +%% under the License. +%% +%% %CopyrightEnd% +%% +%% + +-module(httpc_handler). + +-behaviour(gen_server). + +-include("httpc_internal.hrl"). +-include("http_internal.hrl"). + + +%%-------------------------------------------------------------------- +%% Internal Application API +-export([start_link/3, send/2, cancel/2, stream/3, stream_next/1]). + +%% gen_server callbacks +-export([init/1, handle_call/3, handle_cast/2, handle_info/2, + terminate/2, code_change/3]). + +-record(timers, + { + request_timers = [], % [ref()] + queue_timer % ref() + }). + +-record(state, + { + request, % #request{} + session, % #tcp_session{} + status_line, % {Version, StatusCode, ReasonPharse} + headers, % #http_response_h{} + body, % binary() + mfa, % {Moduel, Function, Args} + pipeline = queue:new(), % queue() + keep_alive = queue:new(), % queue() + status = new, % new | pipeline | keep_alive | close | ssl_tunnel + canceled = [], % [RequestId] + max_header_size = nolimit, % nolimit | integer() + max_body_size = nolimit, % nolimit | integer() + options, % #options{} + timers = #timers{}, % #timers{} + profile_name, % atom() - id of httpc_manager process. + once % send | undefined + }). + + +%%==================================================================== +%% External functions +%%==================================================================== +%%-------------------------------------------------------------------- +%% Function: start_link(Request, Options, ProfileName) -> {ok, Pid} +%% +%% Request = #request{} +%% Options = #options{} +%% ProfileName = atom() - id of httpc manager process +%% +%% Description: Starts a http-request handler process. Intended to be +%% called by the httpc profile supervisor or the http manager process +%% if the client is started stand alone form inets. +%% +%% Note: Uses proc_lib and gen_server:enter_loop so that waiting +%% for gen_tcp:connect to timeout in init/1 will not +%% block the httpc manager process in odd cases such as trying to call +%% a server that does not exist. (See OTP-6735) The only API function +%% sending messages to the handler process that can be called before +%% init has compleated is cancel and that is not a problem! (Send and +%% stream will not be called before the first request has been sent and +%% the reply or part of it has arrived.) +%%-------------------------------------------------------------------- +%%-------------------------------------------------------------------- +start_link(Request, Options, ProfileName) -> + {ok, proc_lib:spawn_link(?MODULE, init, [[Request, Options, + ProfileName]])}. + + +%%-------------------------------------------------------------------- +%% Function: send(Request, Pid) -> ok +%% Request = #request{} +%% Pid = pid() - the pid of the http-request handler process. +%% +%% Description: Uses this handlers session to send a request. Intended +%% to be called by the httpc manager process. +%%-------------------------------------------------------------------- +send(Request, Pid) -> + call(Request, Pid, 5000). + + +%%-------------------------------------------------------------------- +%% Function: cancel(RequestId, Pid) -> ok +%% RequestId = ref() +%% Pid = pid() - the pid of the http-request handler process. +%% +%% Description: Cancels a request. Intended to be called by the httpc +%% manager process. +%%-------------------------------------------------------------------- +cancel(RequestId, Pid) -> + cast({cancel, RequestId}, Pid). + + +%%-------------------------------------------------------------------- +%% Function: stream_next(Pid) -> ok +%% Pid = pid() - the pid of the http-request handler process. +%% +%% Description: Works as inets:setopts(active, once) but for +%% body chunks sent to the user. +%%-------------------------------------------------------------------- +stream_next(Pid) -> + cast(stream_next, Pid). + + +%%-------------------------------------------------------------------- +%% Function: stream(BodyPart, Request, Code) -> _ +%% BodyPart = binary() +%% Request = #request{} +%% Code = integer() +%% +%% Description: Stream the HTTP body to the caller process (client) +%% or to a file. Note that the data that has been stream +%% does not have to be saved. (We do not want to use up +%% memory in vain.) +%%-------------------------------------------------------------------- +%% Request should not be streamed +stream(BodyPart, Request = #request{stream = none}, _) -> + ?hcrt("stream - none", [{body_part, BodyPart}]), + {BodyPart, Request}; + +%% Stream to caller +stream(BodyPart, Request = #request{stream = Self}, Code) + when ((Code =:= 200) orelse (Code =:= 206)) andalso + ((Self =:= self) orelse (Self =:= {self, once})) -> + ?hcrt("stream - self", [{stream, Self}, {code, Code}, {body_part, BodyPart}]), + httpc_response:send(Request#request.from, + {Request#request.id, stream, BodyPart}), + {<<>>, Request}; + +stream(BodyPart, Request = #request{stream = Self}, 404) + when (Self =:= self) orelse (Self =:= {self, once}) -> + ?hcrt("stream - self with 404", [{stream, Self}, {body_part, BodyPart}]), + httpc_response:send(Request#request.from, + {Request#request.id, stream, BodyPart}), + {<<>>, Request}; + +%% Stream to file +%% This has been moved to start_stream/3 +%% We keep this for backward compatibillity... +stream(BodyPart, Request = #request{stream = Filename}, Code) + when ((Code =:= 200) orelse (Code =:= 206)) andalso is_list(Filename) -> + ?hcrt("stream - filename", [{stream, Filename}, {code, Code}, {body_part, BodyPart}]), + case file:open(Filename, [write, raw, append, delayed_write]) of + {ok, Fd} -> + ?hcrt("stream - file open ok", [{fd, Fd}]), + stream(BodyPart, Request#request{stream = Fd}, 200); + {error, Reason} -> + exit({stream_to_file_failed, Reason}) + end; + +%% Stream to file +stream(BodyPart, Request = #request{stream = Fd}, Code) + when ((Code =:= 200) orelse (Code =:= 206)) -> + ?hcrt("stream to file", [{stream, Fd}, {code, Code}, {body_part, BodyPart}]), + case file:write(Fd, BodyPart) of + ok -> + {<<>>, Request}; + {error, Reason} -> + exit({stream_to_file_failed, Reason}) + end; + +stream(BodyPart, Request,_) -> % only 200 and 206 responses can be streamed + ?hcrt("stream - ignore", [{request, Request}, {body_part, BodyPart}]), + {BodyPart, Request}. + + +%%==================================================================== +%% Server functions +%%==================================================================== + +%%-------------------------------------------------------------------- +%% Function: init([Request, Options, ProfileName]) -> {ok, State} | +%% {ok, State, Timeout} | ignore |{stop, Reason} +%% +%% Request = #request{} +%% Options = #options{} +%% ProfileName = atom() - id of httpc manager process +%% +%% Description: Initiates the httpc_handler process +%% +%% Note: The init function may not fail, that will kill the +%% httpc_manager process. We could make the httpc_manager more comlex +%% but we do not want that so errors will be handled by the process +%% sending an init_error message to itself. +%%-------------------------------------------------------------------- +init([Request, Options, ProfileName]) -> + process_flag(trap_exit, true), + + handle_verbose(Options#options.verbose), + Address = handle_proxy(Request#request.address, Options#options.proxy), + {ok, State} = + case {Address /= Request#request.address, Request#request.scheme} of + {true, https} -> + Error = https_through_proxy_is_not_currently_supported, + self() ! {init_error, + Error, httpc_response:error(Request, Error)}, + {ok, #state{request = Request, options = Options, + status = ssl_tunnel}}; + %% This is what we should do if and when ssl supports + %% "socket upgrading" + %%send_ssl_tunnel_request(Address, Request, + %% #state{options = Options, + %% status = ssl_tunnel}); + {_, _} -> + send_first_request(Address, Request, + #state{options = Options, + profile_name = ProfileName}) + end, + gen_server:enter_loop(?MODULE, [], State). + +%%-------------------------------------------------------------------- +%% Function: handle_call(Request, From, State) -> {reply, Reply, State} | +%% {reply, Reply, State, Timeout} | +%% {noreply, State} | +%% {noreply, State, Timeout} | +%% {stop, Reason, Reply, State} | (terminate/2 is called) +%% {stop, Reason, State} (terminate/2 is called) +%% Description: Handling call messages +%%-------------------------------------------------------------------- +handle_call(Request, _, State = #state{session = Session = + #tcp_session{socket = Socket, + type = pipeline}, + timers = Timers, + options = Options, + profile_name = ProfileName}) -> + Address = handle_proxy(Request#request.address, Options#options.proxy), + + case httpc_request:send(Address, Request, Socket) of + ok -> + %% Activate the request time out for the new request + NewState = activate_request_timeout(State#state{request = + Request}), + + ClientClose = httpc_request:is_client_closing( + Request#request.headers), + case State#state.request of + #request{} -> %% Old request no yet finished + %% Make sure to use the new value of timers in state + NewTimers = NewState#state.timers, + NewPipeline = queue:in(Request, State#state.pipeline), + NewSession = + Session#tcp_session{queue_length = + %% Queue + current + queue:len(NewPipeline) + 1, + client_close = ClientClose}, + httpc_manager:insert_session(NewSession, ProfileName), + {reply, ok, State#state{pipeline = NewPipeline, + session = NewSession, + timers = NewTimers}}; + undefined -> + %% Note: tcp-message reciving has already been + %% activated by handle_pipeline/2. + cancel_timer(Timers#timers.queue_timer, + timeout_queue), + NewSession = + Session#tcp_session{queue_length = 1, + client_close = ClientClose}, + httpc_manager:insert_session(NewSession, ProfileName), + Relaxed = + (Request#request.settings)#http_options.relaxed, + {reply, ok, + NewState#state{request = Request, + session = NewSession, + mfa = {httpc_response, parse, + [State#state.max_header_size, + Relaxed]}, + timers = + Timers#timers{queue_timer = + undefined}}} + end; + {error, Reason} -> + {reply, {pipeline_failed, Reason}, State} + end; + +handle_call(Request, _, #state{session = Session = + #tcp_session{type = keep_alive, + socket = Socket}, + timers = Timers, + options = Options, + profile_name = ProfileName} = State) -> + + ClientClose = httpc_request:is_client_closing(Request#request.headers), + + Address = handle_proxy(Request#request.address, + Options#options.proxy), + case httpc_request:send(Address, Request, Socket) of + ok -> + NewState = + activate_request_timeout(State#state{request = + Request}), + + case State#state.request of + #request{} -> %% Old request not yet finished + %% Make sure to use the new value of timers in state + NewTimers = NewState#state.timers, + NewKeepAlive = queue:in(Request, State#state.keep_alive), + NewSession = + Session#tcp_session{queue_length = + %% Queue + current + queue:len(NewKeepAlive) + 1, + client_close = ClientClose}, + httpc_manager:insert_session(NewSession, ProfileName), + {reply, ok, State#state{keep_alive = NewKeepAlive, + session = NewSession, + timers = NewTimers}}; + undefined -> + %% Note: tcp-message reciving has already been + %% activated by handle_pipeline/2. + cancel_timer(Timers#timers.queue_timer, + timeout_queue), + NewSession = + Session#tcp_session{queue_length = 1, + client_close = ClientClose}, + httpc_manager:insert_session(NewSession, ProfileName), + Relaxed = + (Request#request.settings)#http_options.relaxed, + {reply, ok, + NewState#state{request = Request, + session = NewSession, + mfa = {httpc_response, parse, + [State#state.max_header_size, + Relaxed]}}} + end; + {error, Reason} -> + {reply, {request_failed, Reason}, State} + end. + +%%-------------------------------------------------------------------- +%% Function: handle_cast(Msg, State) -> {noreply, State} | +%% {noreply, State, Timeout} | +%% {stop, Reason, State} (terminate/2 is called) +%% Description: Handling cast messages +%%-------------------------------------------------------------------- + +%% When the request in process has been canceled the handler process is +%% stopped and the pipelined requests will be reissued or remaining +%% requests will be sent on a new connection. This is is +%% based on the assumption that it is proably cheaper to reissue the +%% requests than to wait for a potentiall large response that we then +%% only throw away. This of course is not always true maybe we could +%% do something smarter here?! If the request canceled is not +%% the one handled right now the same effect will take place in +%% handle_pipeline/2 when the canceled request is on turn, +%% handle_keep_alive_queue/2 on the other hand will just skip the +%% request as if it was never issued as in this case the request will +%% not have been sent. +handle_cast({cancel, RequestId}, State = #state{request = Request = + #request{id = RequestId}, + profile_name = ProfileName}) -> + httpc_manager:request_canceled(RequestId, ProfileName), + {stop, normal, + State#state{canceled = [RequestId | State#state.canceled], + request = Request#request{from = answer_sent}}}; +handle_cast({cancel, RequestId}, State = #state{profile_name = ProfileName}) -> + httpc_manager:request_canceled(RequestId, ProfileName), + {noreply, State#state{canceled = [RequestId | State#state.canceled]}}; +handle_cast(stream_next, #state{session = Session} = State) -> + http_transport:setopts(socket_type(Session#tcp_session.scheme), + Session#tcp_session.socket, [{active, once}]), + {noreply, State#state{once = once}}. + + +%%-------------------------------------------------------------------- +%% Function: handle_info(Info, State) -> {noreply, State} | +%% {noreply, State, Timeout} | +%% {stop, Reason, State} (terminate/2 is called) +%% Description: Handling all non call/cast messages +%%-------------------------------------------------------------------- +handle_info({Proto, _Socket, Data}, + #state{mfa = {Module, Function, Args} = MFA, + request = #request{method = Method, + stream = Stream} = Request, + session = Session, + status_line = StatusLine} = State) + when (Proto =:= tcp) orelse + (Proto =:= ssl) orelse + (Proto =:= httpc_handler) -> + + ?hcri("received data", [{proto, Proto}, {data, Data}, {mfa, MFA}, {method, Method}, {stream, Stream}, {session, Session}, {status_line, StatusLine}]), + + FinalResult = + try Module:Function([Data | Args]) of + {ok, Result} -> + ?hcrd("data processed - ok", [{result, Result}]), + handle_http_msg(Result, State); + {_, whole_body, _} when Method =:= head -> + ?hcrd("data processed - whole body", []), + handle_response(State#state{body = <<>>}); + {Module, whole_body, [Body, Length]} -> + ?hcrd("data processed - whole body", [{module, Module}, {body, Body}, {length, Length}]), + {_, Code, _} = StatusLine, + {NewBody, NewRequest} = stream(Body, Request, Code), + %% When we stream we will not keep the already + %% streamed data, that would be a waste of memory. + NewLength = case Stream of + none -> + Length; + _ -> + Length - size(Body) + end, + + NewState = next_body_chunk(State), + + {noreply, NewState#state{mfa = {Module, whole_body, + [NewBody, NewLength]}, + request = NewRequest}}; + NewMFA -> + ?hcrd("data processed", [{new_mfa, NewMFA}]), + http_transport:setopts(socket_type(Session#tcp_session.scheme), + Session#tcp_session.socket, + [{active, once}]), + {noreply, State#state{mfa = NewMFA}} + catch + exit:_ -> + ClientErrMsg = httpc_response:error(Request, + {could_not_parse_as_http, + Data}), + NewState = answer_request(Request, ClientErrMsg, State), + {stop, normal, NewState}; + error:_ -> + ClientErrMsg = httpc_response:error(Request, + {could_not_parse_as_http, + Data}), + NewState = answer_request(Request, ClientErrMsg, State), + {stop, normal, NewState} + + end, + ?hcri("data processed", [{result, FinalResult}]), + FinalResult; + + +handle_info({Proto, Socket, Data}, + #state{mfa = MFA, + request = Request, + session = Session, + status = Status, + status_line = StatusLine, + profile_name = Profile} = State) + when (Proto =:= tcp) orelse + (Proto =:= ssl) orelse + (Proto =:= httpc_handler) -> + + error_logger:warning_msg("Received unexpected ~p data on ~p" + "~n Data: ~p" + "~n MFA: ~p" + "~n Request: ~p" + "~n Session: ~p" + "~n Status: ~p" + "~n StatusLine: ~p" + "~n Profile: ~p" + "~n", + [Proto, Socket, Data, MFA, + Request, Session, Status, StatusLine, Profile]), + {noreply, State}; + + +%% The Server may close the connection to indicate that the +%% whole body is now sent instead of sending an length +%% indicator. +handle_info({tcp_closed, _}, State = #state{mfa = {_, whole_body, Args}}) -> + handle_response(State#state{body = hd(Args)}); +handle_info({ssl_closed, _}, State = #state{mfa = {_, whole_body, Args}}) -> + handle_response(State#state{body = hd(Args)}); + +%%% Server closes idle pipeline +handle_info({tcp_closed, _}, State = #state{request = undefined}) -> + {stop, normal, State}; +handle_info({ssl_closed, _}, State = #state{request = undefined}) -> + {stop, normal, State}; + +%%% Error cases +handle_info({tcp_closed, _}, #state{session = Session0} = State) -> + Socket = Session0#tcp_session.socket, + Session = Session0#tcp_session{socket = {remote_close, Socket}}, + %% {stop, session_remotly_closed, State}; + {stop, normal, State#state{session = Session}}; +handle_info({ssl_closed, _}, #state{session = Session0} = State) -> + Socket = Session0#tcp_session.socket, + Session = Session0#tcp_session{socket = {remote_close, Socket}}, + %% {stop, session_remotly_closed, State}; + {stop, normal, State#state{session = Session}}; +handle_info({tcp_error, _, _} = Reason, State) -> + {stop, Reason, State}; +handle_info({ssl_error, _, _} = Reason, State) -> + {stop, Reason, State}; + +%% Timeouts +%% Internally, to a request handling process, a request timeout is +%% seen as a canceled request. +handle_info({timeout, RequestId}, + #state{request = #request{id = RequestId} = Request, + canceled = Canceled} = State) -> + httpc_response:send(Request#request.from, + httpc_response:error(Request,timeout)), + {stop, normal, + State#state{request = Request#request{from = answer_sent}, + canceled = [RequestId | Canceled]}}; + +handle_info({timeout, RequestId}, #state{canceled = Canceled} = State) -> + Filter = + fun(#request{id = Id, from = From} = Request) when Id =:= RequestId -> + %% Notify the owner + Response = httpc_response:error(Request, timeout), + httpc_response:send(From, Response), + [Request#request{from = answer_sent}]; + (_) -> + true + end, + case State#state.status of + pipeline -> + Pipeline = queue:filter(Filter, State#state.pipeline), + {noreply, State#state{canceled = [RequestId | Canceled], + pipeline = Pipeline}}; + keep_alive -> + KeepAlive = queue:filter(Filter, State#state.keep_alive), + {noreply, State#state{canceled = [RequestId | Canceled], + keep_alive = KeepAlive}} + end; + +handle_info(timeout_queue, State = #state{request = undefined}) -> + {stop, normal, State}; + +%% Timing was such as the pipeline_timout was not canceled! +handle_info(timeout_queue, #state{timers = Timers} = State) -> + {noreply, State#state{timers = + Timers#timers{queue_timer = undefined}}}; + +%% Setting up the connection to the server somehow failed. +handle_info({init_error, _, ClientErrMsg}, + State = #state{request = Request}) -> + NewState = answer_request(Request, ClientErrMsg, State), + {stop, normal, NewState}; + + +%%% httpc_manager process dies. +handle_info({'EXIT', _, _}, State = #state{request = undefined}) -> + {stop, normal, State}; +%%Try to finish the current request anyway, +%% there is a fairly high probability that it can be done successfully. +%% Then close the connection, hopefully a new manager is started that +%% can retry requests in the pipeline. +handle_info({'EXIT', _, _}, State) -> + {noreply, State#state{status = close}}. + + +%%-------------------------------------------------------------------- +%% Function: terminate(Reason, State) -> _ (ignored by gen_server) +%% Description: Shutdown the httpc_handler +%%-------------------------------------------------------------------- + +%% Init error there is no socket to be closed. +terminate(normal, #state{session = undefined}) -> + ok; + +%% Init error sending, no session information has been setup but +%% there is a socket that needs closing. +terminate(normal, #state{request = Request, + session = #tcp_session{id = undefined, + socket = Socket}}) -> + http_transport:close(socket_type(Request), Socket); + +%% Socket closed remotely +terminate(normal, + #state{session = #tcp_session{socket = {remote_close, Socket}, + id = Id}, + profile_name = ProfileName, + request = Request, + timers = Timers, + pipeline = Pipeline}) -> + %% Clobber session + (catch httpc_manager:delete_session(Id, ProfileName)), + + %% Cancel timers + #timers{request_timers = ReqTmrs, queue_timer = QTmr} = Timers, + cancel_timer(QTmr, timeout_queue), + lists:foreach(fun({_, Timer}) -> cancel_timer(Timer, timeout) end, + ReqTmrs), + + %% Maybe deliver answers to requests + deliver_answers([Request | queue:to_list(Pipeline)]), + + %% And, just in case, close our side (**really** overkill) + http_transport:close(socket_type(Request), Socket); + +terminate(_, State = #state{session = Session, + request = undefined, + profile_name = ProfileName, + timers = Timers, + pipeline = Pipeline, + keep_alive = KeepAlive}) -> + catch httpc_manager:delete_session(Session#tcp_session.id, + ProfileName), + + maybe_retry_queue(Pipeline, State), + maybe_retry_queue(KeepAlive, State), + + cancel_timer(Timers#timers.queue_timer, timeout_queue), + Socket = Session#tcp_session.socket, + http_transport:close(socket_type(Session#tcp_session.scheme), Socket); + +terminate(Reason, State = #state{request = Request}) -> + NewState = maybe_send_answer(Request, + httpc_response:error(Request, Reason), + State), + terminate(Reason, NewState#state{request = undefined}). + +maybe_retry_queue(Q, State) -> + case queue:is_empty(Q) of + false -> + retry_pipeline(queue:to_list(Q), State); + true -> + ok + end. + +maybe_send_answer(#request{from = answer_sent}, _Reason, State) -> + State; +maybe_send_answer(Request, Answer, State) -> + answer_request(Request, Answer, State). + +deliver_answers([]) -> + ok; +deliver_answers([#request{from = From} = Request | Requests]) + when is_pid(From) -> + Response = httpc_response:error(Request, socket_closed_remotely), + httpc_response:send(From, Response), + deliver_answers(Requests); +deliver_answers([_|Requests]) -> + deliver_answers(Requests). + + +%%-------------------------------------------------------------------- +%% Func: code_change(_OldVsn, State, Extra) -> {ok, NewState} +%% Purpose: Convert process state when code is changed +%%-------------------------------------------------------------------- +code_change(_, #state{request = Request, pipeline = Queue} = State, + [{from, '5.0.1'}, {to, '5.0.2'}]) -> + Settings = new_http_options(Request#request.settings), + NewRequest = Request#request{settings = Settings}, + NewQueue = new_queue(Queue, fun new_http_options/1), + {ok, State#state{request = NewRequest, pipeline = NewQueue}}; + +code_change(_, #state{request = Request, pipeline = Queue} = State, + [{from, '5.0.2'}, {to, '5.0.1'}]) -> + Settings = old_http_options(Request#request.settings), + NewRequest = Request#request{settings = Settings}, + NewQueue = new_queue(Queue, fun old_http_options/1), + {ok, State#state{request = NewRequest, pipeline = NewQueue}}; + +code_change(_, State, _) -> + {ok, State}. + +new_http_options({http_options, TimeOut, AutoRedirect, SslOpts, + Auth, Relaxed}) -> + {http_options, "HTTP/1.1", TimeOut, AutoRedirect, SslOpts, + Auth, Relaxed}. + +old_http_options({http_options, _, TimeOut, AutoRedirect, + SslOpts, Auth, Relaxed}) -> + {http_options, TimeOut, AutoRedirect, SslOpts, Auth, Relaxed}. + +new_queue(Queue, Fun) -> + List = queue:to_list(Queue), + NewList = + lists:map(fun(Request) -> + Settings = + Fun(Request#request.settings), + Request#request{settings = Settings} + end, List), + queue:from_list(NewList). + +%%-------------------------------------------------------------------- +%%% Internal functions +%%-------------------------------------------------------------------- + +connect(SocketType, ToAddress, #options{ipfamily = IpFamily, + ip = FromAddress, + port = FromPort}, Timeout) -> + Opts1 = + case FromPort of + default -> + []; + _ -> + [{port, FromPort}] + end, + Opts2 = + case FromAddress of + default -> + Opts1; + _ -> + [{ip, FromAddress} | Opts1] + end, + case IpFamily of + inet6fb4 -> + Opts3 = [inet6 | Opts2], + case http_transport:connect(SocketType, ToAddress, Opts3, Timeout) of + {error, Reason} when ((Reason =:= nxdomain) orelse + (Reason =:= eafnosupport)) -> + Opts4 = [inet | Opts2], + http_transport:connect(SocketType, ToAddress, Opts4, Timeout); + Other -> + Other + end; + _ -> + Opts3 = [IpFamily | Opts2], + http_transport:connect(SocketType, ToAddress, Opts3, Timeout) + end. + + +send_first_request(Address, Request, #state{options = Options} = State) -> + SocketType = socket_type(Request), + ConnTimeout = (Request#request.settings)#http_options.connect_timeout, + ?hcri("connect", + [{address, Address}, {request, Request}, {options, Options}]), + case connect(SocketType, Address, Options, ConnTimeout) of + {ok, Socket} -> + ?hcri("connected - now send first request", [{socket, Socket}]), + case httpc_request:send(Address, Request, Socket) of + ok -> + ?hcri("first request sent", []), + ClientClose = + httpc_request:is_client_closing( + Request#request.headers), + SessionType = httpc_manager:session_type(Options), + Session = + #tcp_session{id = {Request#request.address, self()}, + scheme = Request#request.scheme, + socket = Socket, + client_close = ClientClose, + type = SessionType}, + TmpState = State#state{request = Request, + session = Session, + mfa = init_mfa(Request, State), + status_line = + init_status_line(Request), + headers = undefined, + body = undefined, + status = new}, + http_transport:setopts(SocketType, + Socket, [{active, once}]), + NewState = activate_request_timeout(TmpState), + {ok, NewState}; + + {error, Reason} -> + %% Commented out in wait of ssl support to avoid + %% dialyzer warning + %%case State#state.status of + %% new -> % Called from init/1 + self() ! {init_error, error_sending, + httpc_response:error(Request, Reason)}, + {ok, State#state{request = Request, + session = + #tcp_session{socket = Socket}}} + %%ssl_tunnel -> % Not called from init/1 + %% NewState = + %% answer_request(Request, + %%httpc_response:error(Request, + %%Reason), + %% State), + %% {stop, normal, NewState} + %% end + end; + + {error, Reason} -> + %% Commented out in wait of ssl support to avoid + %% dialyzer warning + %% case State#state.status of + %% new -> % Called from init/1 + self() ! {init_error, error_connecting, + httpc_response:error(Request, Reason)}, + {ok, State#state{request = Request}} + %% ssl_tunnel -> % Not called from init/1 + %% NewState = + %% answer_request(Request, + %% httpc_response:error(Request, + %% Reason), + %% State), + %% {stop, normal, NewState} + %%end + end. + +handle_http_msg({Version, StatusCode, ReasonPharse, Headers, Body}, + State = #state{request = Request}) -> + ?hcrt("handle_http_msg", [{body, Body}]), + case Headers#http_response_h.'content-type' of + "multipart/byteranges" ++ _Param -> + exit(not_yet_implemented); + _ -> + StatusLine = {Version, StatusCode, ReasonPharse}, + {ok, NewRequest} = start_stream(StatusLine, Headers, Request), + handle_http_body(Body, + State#state{request = NewRequest, + status_line = StatusLine, + headers = Headers}) + end; +handle_http_msg({ChunkedHeaders, Body}, + State = #state{headers = Headers}) -> + ?hcrt("handle_http_msg", [{chunked_headers, ChunkedHeaders}, {body, Body}]), + NewHeaders = http_chunk:handle_headers(Headers, ChunkedHeaders), + handle_response(State#state{headers = NewHeaders, body = Body}); +handle_http_msg(Body, State = #state{status_line = {_,Code, _}}) -> + ?hcrt("handle_http_msg", [{body, Body}, {code, Code}]), + {NewBody, NewRequest}= stream(Body, State#state.request, Code), + handle_response(State#state{body = NewBody, request = NewRequest}). + +handle_http_body(<<>>, State = #state{status_line = {_,304, _}}) -> + ?hcrt("handle_http_body - 304", []), + handle_response(State#state{body = <<>>}); + +handle_http_body(<<>>, State = #state{status_line = {_,204, _}}) -> + ?hcrt("handle_http_body - 204", []), + handle_response(State#state{body = <<>>}); + +handle_http_body(<<>>, State = #state{request = #request{method = head}}) -> + ?hcrt("handle_http_body - head", []), + handle_response(State#state{body = <<>>}); + +handle_http_body(Body, State = #state{headers = Headers, + max_body_size = MaxBodySize, + status_line = {_,Code, _}, + request = Request}) -> + ?hcrt("handle_http_body", [{body, Body}, {max_body_size, MaxBodySize}, {code, Code}]), + TransferEnc = Headers#http_response_h.'transfer-encoding', + case case_insensitive_header(TransferEnc) of + "chunked" -> + ?hcrt("handle_http_body - chunked", []), + case http_chunk:decode(Body, State#state.max_body_size, + State#state.max_header_size, + {Code, Request}) of + {Module, Function, Args} -> + ?hcrt("handle_http_body - new mfa", [{module, Module}, {function, Function}, {args, Args}]), + NewState = next_body_chunk(State), + {noreply, NewState#state{mfa = + {Module, Function, Args}}}; + {ok, {ChunkedHeaders, NewBody}} -> + ?hcrt("handle_http_body - nyew body", [{chunked_headers, ChunkedHeaders}, {new_body, NewBody}]), + NewHeaders = http_chunk:handle_headers(Headers, + ChunkedHeaders), + handle_response(State#state{headers = NewHeaders, + body = NewBody}) + end; + Encoding when is_list(Encoding) -> + ?hcrt("handle_http_body - encoding", [{encoding, Encoding}]), + NewState = answer_request(Request, + httpc_response:error(Request, + unknown_encoding), + State), + {stop, normal, NewState}; + _ -> + ?hcrt("handle_http_body - other", []), + Length = + list_to_integer(Headers#http_response_h.'content-length'), + case ((Length =< MaxBodySize) or (MaxBodySize == nolimit)) of + true -> + case httpc_response:whole_body(Body, Length) of + {ok, Body} -> + {NewBody, NewRequest}= stream(Body, Request, Code), + handle_response(State#state{body = NewBody, + request = NewRequest}); + MFA -> + NewState = next_body_chunk(State), + {noreply, NewState#state{mfa = MFA}} + end; + false -> + NewState = + answer_request(Request, + httpc_response:error(Request, + body_too_big), + State), + {stop, normal, NewState} + end + end. + +%%% Normaly I do not comment out code, I throw it away. But this might +%%% actually be used on day if ssl is improved. +%% handle_response(State = #state{status = ssl_tunnel, +%% request = Request, +%% options = Options, +%% session = #tcp_session{socket = Socket, +%% scheme = Scheme}, +%% status_line = {_, 200, _}}) -> +%% %%% Insert code for upgrading the socket if and when ssl supports this. +%% Address = handle_proxy(Request#request.address, Options#options.proxy), +%% send_first_request(Address, Request, State); +%% handle_response(State = #state{status = ssl_tunnel, +%% request = Request}) -> +%% NewState = answer_request(Request, +%% httpc_response:error(Request, +%% ssl_proxy_tunnel_failed), +%% State), +%% {stop, normal, NewState}; + +handle_response(State = #state{status = new}) -> + handle_response(try_to_enable_pipeline_or_keep_alive(State)); + +handle_response(State = + #state{request = Request, + status = Status, + session = Session, + status_line = StatusLine, + headers = Headers, + body = Body, + options = Options, + profile_name = ProfileName}) when Status =/= new -> + ?hcrt("handle response", [{status, Status}, {session, Session}, {status_line, StatusLine}, {profile_name, ProfileName}]), + handle_cookies(Headers, Request, Options, ProfileName), + case httpc_response:result({StatusLine, Headers, Body}, Request) of + %% 100-continue + continue -> + %% Send request body + {_, RequestBody} = Request#request.content, + http_transport:send(socket_type(Session#tcp_session.scheme), + Session#tcp_session.socket, + RequestBody), + %% Wait for next response + http_transport:setopts(socket_type(Session#tcp_session.scheme), + Session#tcp_session.socket, + [{active, once}]), + Relaxed = (Request#request.settings)#http_options.relaxed, + {noreply, + State#state{mfa = {httpc_response, parse, + [State#state.max_header_size, + Relaxed]}, + status_line = undefined, + headers = undefined, + body = undefined + }}; + %% Ignore unexpected 100-continue response and receive the + %% actual response that the server will send right away. + {ignore, Data} -> + Relaxed = (Request#request.settings)#http_options.relaxed, + NewState = State#state{mfa = + {httpc_response, parse, + [State#state.max_header_size, + Relaxed]}, + status_line = undefined, + headers = undefined, + body = undefined}, + handle_info({httpc_handler, dummy, Data}, NewState); + %% On a redirect or retry the current request becomes + %% obsolete and the manager will create a new request + %% with the same id as the current. + {redirect, NewRequest, Data} -> + ?hcrt("handle response - redirect", [{new_request, NewRequest}, {data, Data}]), + ok = httpc_manager:redirect_request(NewRequest, ProfileName), + handle_queue(State#state{request = undefined}, Data); + {retry, TimeNewRequest, Data} -> + ?hcrt("handle response - retry", [{time_new_request, TimeNewRequest}, {data, Data}]), + ok = httpc_manager:retry_request(TimeNewRequest, ProfileName), + handle_queue(State#state{request = undefined}, Data); + {ok, Msg, Data} -> + ?hcrt("handle response - result ok", [{msg, Msg}, {data, Data}]), + end_stream(StatusLine, Request), + NewState = answer_request(Request, Msg, State), + handle_queue(NewState, Data); + {stop, Msg} -> + ?hcrt("handle response - result stop", [{msg, Msg}]), + end_stream(StatusLine, Request), + NewState = answer_request(Request, Msg, State), + {stop, normal, NewState} + end. + +handle_cookies(_,_, #options{cookies = disabled}, _) -> + ok; +%% User wants to verify the cookies before they are stored, +%% so the user will have to call a store command. +handle_cookies(_,_, #options{cookies = verify}, _) -> + ok; +handle_cookies(Headers, Request, #options{cookies = enabled}, ProfileName) -> + {Host, _ } = Request#request.address, + Cookies = http_cookie:cookies(Headers#http_response_h.other, + Request#request.path, Host), + httpc_manager:store_cookies(Cookies, Request#request.address, + ProfileName). + +%% This request could not be pipelined or used as sequential keept alive +%% queue +handle_queue(State = #state{status = close}, _) -> + {stop, normal, State}; + +handle_queue(State = #state{status = keep_alive}, Data) -> + handle_keep_alive_queue(State, Data); + +handle_queue(State = #state{status = pipeline}, Data) -> + handle_pipeline(State, Data). + +handle_pipeline(State = + #state{status = pipeline, session = Session, + profile_name = ProfileName, + options = #options{pipeline_timeout = TimeOut}}, + Data) -> + case queue:out(State#state.pipeline) of + {empty, _} -> + %% The server may choose too teminate an idle pipeline + %% in this case we want to receive the close message + %% at once and not when trying to pipeline the next + %% request. + http_transport:setopts(socket_type(Session#tcp_session.scheme), + Session#tcp_session.socket, + [{active, once}]), + %% If a pipeline that has been idle for some time is not + %% closed by the server, the client may want to close it. + NewState = activate_queue_timeout(TimeOut, State), + NewSession = Session#tcp_session{queue_length = 0}, + httpc_manager:insert_session(NewSession, ProfileName), + %% Note mfa will be initilized when a new request + %% arrives. + {noreply, + NewState#state{request = undefined, + mfa = undefined, + status_line = undefined, + headers = undefined, + body = undefined + } + }; + {{value, NextRequest}, Pipeline} -> + case lists:member(NextRequest#request.id, + State#state.canceled) of + true -> + %% See comment for handle_cast({cancel, RequestId}) + {stop, normal, + State#state{request = + NextRequest#request{from = answer_sent}}}; + false -> + NewSession = + Session#tcp_session{queue_length = + %% Queue + current + queue:len(Pipeline) + 1}, + httpc_manager:insert_session(NewSession, ProfileName), + Relaxed = + (NextRequest#request.settings)#http_options.relaxed, + NewState = + State#state{pipeline = Pipeline, + request = NextRequest, + mfa = {httpc_response, parse, + [State#state.max_header_size, + Relaxed]}, + status_line = undefined, + headers = undefined, + body = undefined}, + case Data of + <<>> -> + http_transport:setopts( + socket_type(Session#tcp_session.scheme), + Session#tcp_session.socket, + [{active, once}]), + {noreply, NewState}; + _ -> + %% If we already received some bytes of + %% the next response + handle_info({httpc_handler, dummy, Data}, + NewState) + end + end + end. + +handle_keep_alive_queue(State = #state{status = keep_alive, + session = Session, + profile_name = ProfileName, + options = #options{keep_alive_timeout + = TimeOut} + }, + Data) -> + case queue:out(State#state.keep_alive) of + {empty, _} -> + %% The server may choose too terminate an idle keep_alive session + %% in this case we want to receive the close message + %% at once and not when trying to send the next + %% request. + http_transport:setopts(socket_type(Session#tcp_session.scheme), + Session#tcp_session.socket, + [{active, once}]), + %% If a keep_alive session has been idle for some time is not + %% closed by the server, the client may want to close it. + NewState = activate_queue_timeout(TimeOut, State), + NewSession = Session#tcp_session{queue_length = 0}, + httpc_manager:insert_session(NewSession, ProfileName), + %% Note mfa will be initilized when a new request + %% arrives. + {noreply, + NewState#state{request = undefined, + mfa = undefined, + status_line = undefined, + headers = undefined, + body = undefined + } + }; + {{value, NextRequest}, KeepAlive} -> + case lists:member(NextRequest#request.id, + State#state.canceled) of + true -> + handle_keep_alive_queue(State#state{keep_alive = + KeepAlive}, Data); + false -> + Relaxed = + (NextRequest#request.settings)#http_options.relaxed, + NewState = + State#state{request = NextRequest, + keep_alive = KeepAlive, + mfa = {httpc_response, parse, + [State#state.max_header_size, + Relaxed]}, + status_line = undefined, + headers = undefined, + body = undefined}, + case Data of + <<>> -> + http_transport:setopts( + socket_type(Session#tcp_session.scheme), + Session#tcp_session.socket, [{active, once}]), + {noreply, NewState}; + _ -> + %% If we already received some bytes of + %% the next response + handle_info({httpc_handler, dummy, Data}, + NewState) + end + end + end. + +call(Msg, Pid, Timeout) -> + gen_server:call(Pid, Msg, Timeout). + +cast(Msg, Pid) -> + gen_server:cast(Pid, Msg). + +case_insensitive_header(Str) when is_list(Str) -> + http_util:to_lower(Str); +%% Might be undefined if server does not send such a header +case_insensitive_header(Str) -> + Str. + +activate_request_timeout(State = #state{request = Request}) -> + Time = (Request#request.settings)#http_options.timeout, + case Time of + infinity -> + State; + _ -> + Ref = erlang:send_after(Time, self(), + {timeout, Request#request.id}), + State#state + {timers = + #timers{request_timers = + [{Request#request.id, Ref}| + (State#state.timers)#timers.request_timers]}} + end. + +activate_queue_timeout(infinity, State) -> + State; +activate_queue_timeout(Time, State) -> + Ref = erlang:send_after(Time, self(), timeout_queue), + State#state{timers = #timers{queue_timer = Ref}}. + + +is_pipeline_enabled_client(#tcp_session{type = pipeline}) -> + true; +is_pipeline_enabled_client(_) -> + false. + +is_keep_alive_enabled_server("HTTP/1." ++ N, _) when (hd(N) >= $1) -> + true; +is_keep_alive_enabled_server("HTTP/1.0", + #http_response_h{connection = "keep-alive"}) -> + true; +is_keep_alive_enabled_server(_,_) -> + false. + +is_keep_alive_connection(Headers, Session) -> + (not ((Session#tcp_session.client_close) or + httpc_response:is_server_closing(Headers))). + +try_to_enable_pipeline_or_keep_alive(State = + #state{session = Session, + request = #request{method = Method}, + status_line = {Version, _, _}, + headers = Headers, + profile_name = ProfileName}) -> + case (is_keep_alive_enabled_server(Version, Headers) andalso + is_keep_alive_connection(Headers, Session)) of + true -> + case (is_pipeline_enabled_client(Session) andalso + httpc_request:is_idempotent(Method)) of + true -> + httpc_manager:insert_session(Session, ProfileName), + State#state{status = pipeline}; + false -> + httpc_manager:insert_session(Session, ProfileName), + %% Make sure type is keep_alive in session + %% as it in this case might be pipeline + State#state{status = keep_alive, + session = + Session#tcp_session{type = keep_alive}} + end; + false -> + State#state{status = close} + end. + +answer_request(Request, Msg, #state{timers = Timers} = State) -> + httpc_response:send(Request#request.from, Msg), + RequestTimers = Timers#timers.request_timers, + TimerRef = + proplists:get_value(Request#request.id, RequestTimers, undefined), + Timer = {Request#request.id, TimerRef}, + cancel_timer(TimerRef, {timeout, Request#request.id}), + State#state{request = Request#request{from = answer_sent}, + timers = + Timers#timers{request_timers = + lists:delete(Timer, RequestTimers)}}. +cancel_timer(undefined, _) -> + ok; +cancel_timer(Timer, TimeoutMsg) -> + erlang:cancel_timer(Timer), + receive + TimeoutMsg -> + ok + after 0 -> + ok + end. + +retry_pipeline([], _) -> + ok; + +%% Skip requests when the answer has already been sent +retry_pipeline([#request{from = answer_sent}|PipeLine], State) -> + retry_pipeline(PipeLine, State); + +retry_pipeline([Request | PipeLine], + #state{timers = Timers, + profile_name = ProfileName} = State) -> + NewState = + case (catch httpc_manager:retry_request(Request, ProfileName)) of + ok -> + RequestTimers = Timers#timers.request_timers, + TimerRef = + proplists:get_value(Request#request.id, RequestTimers, + undefined), + cancel_timer(TimerRef, {timeout, Request#request.id}), + State#state{timers = Timers#timers{request_timers = + lists:delete({Request#request.id, + TimerRef}, + RequestTimers)}}; + Error -> + answer_request(Request#request.from, + httpc_response:error(Request, Error), State) + end, + retry_pipeline(PipeLine, NewState). + +%%% Check to see if the given {Host,Port} tuple is in the NoProxyList +%%% Returns an eventually updated {Host,Port} tuple, with the proxy address +handle_proxy(HostPort = {Host, _Port}, {Proxy, NoProxy}) -> + case Proxy of + undefined -> + HostPort; + Proxy -> + case is_no_proxy_dest(Host, NoProxy) of + true -> + HostPort; + false -> + Proxy + end + end. + +is_no_proxy_dest(_, []) -> + false; +is_no_proxy_dest(Host, [ "*." ++ NoProxyDomain | NoProxyDests]) -> + + case is_no_proxy_dest_domain(Host, NoProxyDomain) of + true -> + true; + false -> + is_no_proxy_dest(Host, NoProxyDests) + end; + +is_no_proxy_dest(Host, [NoProxyDest | NoProxyDests]) -> + IsNoProxyDest = case http_util:is_hostname(NoProxyDest) of + true -> + fun is_no_proxy_host_name/2; + false -> + fun is_no_proxy_dest_address/2 + end, + + case IsNoProxyDest(Host, NoProxyDest) of + true -> + true; + false -> + is_no_proxy_dest(Host, NoProxyDests) + end. + +is_no_proxy_host_name(Host, Host) -> + true; +is_no_proxy_host_name(_,_) -> + false. + +is_no_proxy_dest_domain(Dest, DomainPart) -> + lists:suffix(DomainPart, Dest). + +is_no_proxy_dest_address(Dest, Dest) -> + true; +is_no_proxy_dest_address(Dest, AddressPart) -> + lists:prefix(AddressPart, Dest). + +init_mfa(#request{settings = Settings}, State) -> + case Settings#http_options.version of + "HTTP/0.9" -> + {httpc_response, whole_body, [<<>>, -1]}; + _ -> + Relaxed = Settings#http_options.relaxed, + {httpc_response, parse, [State#state.max_header_size, Relaxed]} + end. + +init_status_line(#request{settings = Settings}) -> + case Settings#http_options.version of + "HTTP/0.9" -> + {"HTTP/0.9", 200, "OK"}; + _ -> + undefined + end. + +socket_type(#request{scheme = http}) -> + ip_comm; +socket_type(#request{scheme = https, settings = Settings}) -> + {ssl, Settings#http_options.ssl}; +socket_type(http) -> + ip_comm; +socket_type(https) -> + {ssl, []}. %% Dummy value ok for ex setops that does not use this value + +start_stream({_Version, _Code, _ReasonPhrase}, _Headers, #request{stream = none} = Request) -> + ?hcrt("start stream - none", []), + {ok, Request}; +start_stream({_Version, Code, _ReasonPhrase}, Headers, #request{stream = self} = Request) + when (Code =:= 200) orelse (Code =:= 206) -> + ?hcrt("start stream - self", [{code, Code}]), + Msg = httpc_response:stream_start(Headers, Request, ignore), + httpc_response:send(Request#request.from, Msg), + {ok, Request}; +start_stream({_Version, Code, _ReasonPhrase}, Headers, + #request{stream = {self, once}} = Request) + when (Code =:= 200) orelse (Code =:= 206) -> + ?hcrt("start stream - self:once", [{code, Code}]), + Msg = httpc_response:stream_start(Headers, Request, self()), + httpc_response:send(Request#request.from, Msg), + {ok, Request}; +start_stream({_Version, Code, _ReasonPhrase}, _Headers, #request{stream = Filename} = Request) + when ((Code =:= 200) orelse (Code =:= 206)) andalso is_list(Filename) -> + ?hcrt("start stream", [{code, Code}, {filename, Filename}]), + case file:open(Filename, [write, raw, append, delayed_write]) of + {ok, Fd} -> + ?hcri("start stream - file open ok", [{fd, Fd}]), + {ok, Request#request{stream = Fd}}; + {error, Reason} -> + exit({stream_to_file_failed, Reason}) + end; +start_stream(_StatusLine, _Headers, Request) -> + ?hcrt("start stream - no op", []), + {ok, Request}. + + +%% Note the end stream message is handled by httpc_response and will +%% be sent by answer_request +end_stream(_, #request{stream = none}) -> + ?hcrt("end stream - none", []), + ok; +end_stream(_, #request{stream = self}) -> + ?hcrt("end stream - self", []), + ok; +end_stream(_, #request{stream = {self, once}}) -> + ?hcrt("end stream - self:once", []), + ok; +end_stream({_,200,_}, #request{stream = Fd}) -> + ?hcrt("end stream - 200", [{stream, Fd}]), + case file:close(Fd) of + ok -> + ok; + {error, enospc} -> % Could be due to delayed_write + file:close(Fd) + end; +end_stream({_,206,_}, #request{stream = Fd}) -> + ?hcrt("end stream - 206", [{stream, Fd}]), + case file:close(Fd) of + ok -> + ok; + {error, enospc} -> % Could be due to delayed_write + file:close(Fd) + end; +end_stream(SL, R) -> + ?hcrt("end stream", [{status_line, SL}, {request, R}]), + ok. + + +next_body_chunk(#state{request = #request{stream = {self, once}}, + once = once, session = Session} = State) -> + http_transport:setopts(socket_type(Session#tcp_session.scheme), + Session#tcp_session.socket, + [{active, once}]), + State#state{once = inactive}; +next_body_chunk(#state{request = #request{stream = {self, once}}, + once = inactive} = State) -> + State; %% Wait for user to call stream_next +next_body_chunk(#state{session = Session} = State) -> + http_transport:setopts(socket_type(Session#tcp_session.scheme), + Session#tcp_session.socket, + [{active, once}]), + State. + +handle_verbose(verbose) -> + dbg:p(self(), [r]); +handle_verbose(debug) -> + dbg:p(self(), [call]), + dbg:tp(?MODULE, [{'_', [], [{return_trace}]}]); +handle_verbose(trace) -> + dbg:p(self(), [call]), + dbg:tpl(?MODULE, [{'_', [], [{return_trace}]}]); +handle_verbose(_) -> + ok. + +%%% Normaly I do not comment out code, I throw it away. But this might +%%% actually be used one day if ssl is improved. +%% send_ssl_tunnel_request(Address, Request = #request{address = {Host, Port}}, +%% State) -> +%% %% A ssl tunnel request is a special http request that looks like +%% %% CONNECT host:port HTTP/1.1 +%% SslTunnelRequest = #request{method = connect, scheme = http, +%% headers = +%% #http_request_h{ +%% host = Host, +%% address = Address, +%% path = Host ++ ":", +%% pquery = integer_to_list(Port), +%% other = [{ "Proxy-Connection", "keep-alive"}]}, +%% Ipv6 = (State#state.options)#options.ipv6, +%% SocketType = socket_type(SslTunnelRequest), +%% case http_transport:connect(SocketType, +%% SslTunnelRequest#request.address, Ipv6) of +%% {ok, Socket} -> +%% case httpc_request:send(Address, SslTunnelRequest, Socket) of +%% ok -> +%% Session = #tcp_session{id = +%% {SslTunnelRequest#request.address, +%% self()}, +%% scheme = +%% SslTunnelRequest#request.scheme, +%% socket = Socket}, +%% NewState = State#state{mfa = +%% {httpc_response, parse, +%% [State#state.max_header_size]}, +%% request = Request, +%% session = Session}, +%% http_transport:setopts(socket_type( +%% SslTunnelRequest#request.scheme), +%% Socket, +%% [{active, once}]), +%% {ok, NewState}; +%% {error, Reason} -> +%% self() ! {init_error, error_sending, +%% httpc_response:error(Request, Reason)}, +%% {ok, State#state{request = Request, +%% session = #tcp_session{socket = +%% Socket}}} +%% end; +%% {error, Reason} -> +%% self() ! {init_error, error_connecting, +%% httpc_response:error(Request, Reason)}, +%% {ok, State#state{request = Request}} +%% end. + +%% d(F) -> +%% d(F, []). + +%% d(F, A) -> +%% d(get(dbg), F, A). + +%% d(true, F, A) -> +%% io:format(user, "~w:~w:" ++ F ++ "~n", [self(), ?MODULE | A]); +%% d(_, _, _) -> +%% ok. + diff --git a/lib/inets/src/http_client/httpc_handler_sup.erl b/lib/inets/src/http_client/httpc_handler_sup.erl new file mode 100644 index 0000000000..d9edaa0599 --- /dev/null +++ b/lib/inets/src/http_client/httpc_handler_sup.erl @@ -0,0 +1,66 @@ +%% +%% %CopyrightBegin% +%% +%% Copyright Ericsson AB 2007-2009. All Rights Reserved. +%% +%% The contents of this file are subject to the Erlang Public License, +%% Version 1.1, (the "License"); you may not use this file except in +%% compliance with the License. You should have received a copy of the +%% Erlang Public License along with this software. If not, it can be +%% retrieved online at http://www.erlang.org/. +%% +%% Software distributed under the License is distributed on an "AS IS" +%% basis, WITHOUT WARRANTY OF ANY KIND, either express or implied. See +%% the License for the specific language governing rights and limitations +%% under the License. +%% +%% %CopyrightEnd% +%% +%% +-module(httpc_handler_sup). + +-behaviour(supervisor). + +%% API +-export([start_link/0]). +-export([start_child/1]). + +%% Supervisor callback +-export([init/1]). + +%%%========================================================================= +%%% API +%%%========================================================================= +start_link() -> + supervisor:start_link({local, ?MODULE}, ?MODULE, []). + +start_child(Args) -> + supervisor:start_child(?MODULE, Args). + +%%%========================================================================= +%%% Supervisor callback +%%%========================================================================= +init(Args) -> + RestartStrategy = simple_one_for_one, + MaxR = 0, + MaxT = 3600, + + Name = undefined, % As simple_one_for_one is used. + StartFunc = {httpc_handler, start_link, Args}, + Restart = temporary, % E.g. should not be restarted + Shutdown = 4000, + Modules = [httpc_handler], + Type = worker, + + ChildSpec = {Name, StartFunc, Restart, Shutdown, Type, Modules}, + {ok, {{RestartStrategy, MaxR, MaxT}, [ChildSpec]}}. + + + + + + + + + + diff --git a/lib/inets/src/http_client/httpc_internal.hrl b/lib/inets/src/http_client/httpc_internal.hrl new file mode 100644 index 0000000000..ec709b9860 --- /dev/null +++ b/lib/inets/src/http_client/httpc_internal.hrl @@ -0,0 +1,136 @@ +%% +%% %CopyrightBegin% +%% +%% Copyright Ericsson AB 2005-2009. All Rights Reserved. +%% +%% The contents of this file are subject to the Erlang Public License, +%% Version 1.1, (the "License"); you may not use this file except in +%% compliance with the License. You should have received a copy of the +%% Erlang Public License along with this software. If not, it can be +%% retrieved online at http://www.erlang.org/. +%% +%% Software distributed under the License is distributed on an "AS IS" +%% basis, WITHOUT WARRANTY OF ANY KIND, either express or implied. See +%% the License for the specific language governing rights and limitations +%% under the License. +%% +%% %CopyrightEnd% +%% +%% + +-include("inets_internal.hrl"). +-define(SERVICE, httpc). +-define(hcri(Label, Data), ?report_important(Label, ?SERVICE, Data)). +-define(hcrv(Label, Data), ?report_verbose(Label, ?SERVICE, Data)). +-define(hcrd(Label, Data), ?report_debug(Label, ?SERVICE, Data)). +-define(hcrt(Label, Data), ?report_trace(Label, ?SERVICE, Data)). + +-define(HTTP_REQUEST_TIMEOUT, infinity). +-define(HTTP_REQUEST_CTIMEOUT, ?HTTP_REQUEST_TIMEOUT). +-define(HTTP_PIPELINE_TIMEOUT, 0). +-define(HTTP_PIPELINE_LENGTH, 2). +-define(HTTP_MAX_TCP_SESSIONS, 2). +-define(HTTP_MAX_REDIRECTS, 4). +-define(HTTP_KEEP_ALIVE_TIMEOUT, 120000). +-define(HTTP_KEEP_ALIVE_LENGTH, 5). + +%%% HTTP Client per request settings +-record(http_options, + { + %% string() - "HTTP/1.1" | "HTTP/1.0" | "HTTP/0.9" + version, + + %% integer() | infinity - ms before a request times out + timeout = ?HTTP_REQUEST_TIMEOUT, + + %% bool() - true if auto redirect on 30x response + autoredirect = true, + + %% Ssl socket options + ssl = [], + + %% {User, Password} = {string(), string()} + proxy_auth, + + %% bool() - true if not strictly std compliant + relaxed = false, + + %% integer() - ms before a connect times out + connect_timeout = ?HTTP_REQUEST_CTIMEOUT + } + ). + +%%% HTTP Client per profile setting. +-record(options, + { + proxy = {undefined, []}, % {{ProxyHost, ProxyPort}, [NoProxy]}, + %% 0 means persistent connections are used without pipelining + pipeline_timeout = ?HTTP_PIPELINE_TIMEOUT, + max_pipeline_length = ?HTTP_PIPELINE_LENGTH, + max_keep_alive_length = ?HTTP_KEEP_ALIVE_LENGTH, + keep_alive_timeout = ?HTTP_KEEP_ALIVE_TIMEOUT, % Used when pipeline_timeout = 0 + max_sessions = ?HTTP_MAX_TCP_SESSIONS, + cookies = disabled, % enabled | disabled | verify + verbose = false, + ipfamily = inet, % inet | inet6 | inet6fb4 + ip = default, % specify local interface + port = default % specify local port + } + ). + +%%% All data associated to a specific HTTP request +-record(request, + { + id, % ref() - Request Id + from, % pid() - Caller + redircount = 0,% Number of redirects made for this request + scheme, % http | https + address, % ({Host,Port}) Destination Host and Port + path, % string() - Path of parsed URL + pquery, % string() - Rest of parsed URL + method, % atom() - HTTP request Method + headers, % #http_request_h{} + content, % {ContentType, Body} - Current HTTP request + settings, % #http_options{} - User defined settings + abs_uri, % string() ex: "http://www.erlang.org" + userinfo, % string() - optinal "<userinfo>@<host>:<port>" + stream, % Boolean() - stream async reply? + headers_as_is % Boolean() - workaround for servers that does + %% not honor the http standard, can also be used for testing purposes. + } + ). + +-record(tcp_session, + { + id, % {{Host, Port}, HandlerPid} + client_close, % true | false + scheme, % http (HTTP/TCP) | https (HTTP/SSL/TCP) + socket, % Open socket, used by connection + queue_length = 1, % Current length of pipeline or keep alive queue + type % pipeline | keep_alive (wait for response before sending new request) + }). + +-record(http_cookie, + { + domain, + domain_default = false, + name, + value, + comment, + max_age = session, + path, + path_default = false, + secure = false, + version = "0" + }). + + +%% -record(parsed_uri, +%% { +%% scheme, % http | https +%% uinfo, % string() +%% host, % string() +%% port, % integer() +%% path, % string() +%% q % query: string() +%% }). diff --git a/lib/inets/src/http_client/httpc_manager.erl b/lib/inets/src/http_client/httpc_manager.erl new file mode 100644 index 0000000000..63b00c7dce --- /dev/null +++ b/lib/inets/src/http_client/httpc_manager.erl @@ -0,0 +1,634 @@ +%% +%% %CopyrightBegin% +%% +%% Copyright Ericsson AB 2002-2009. All Rights Reserved. +%% +%% The contents of this file are subject to the Erlang Public License, +%% Version 1.1, (the "License"); you may not use this file except in +%% compliance with the License. You should have received a copy of the +%% Erlang Public License along with this software. If not, it can be +%% retrieved online at http://www.erlang.org/. +%% +%% Software distributed under the License is distributed on an "AS IS" +%% basis, WITHOUT WARRANTY OF ANY KIND, either express or implied. See +%% the License for the specific language governing rights and limitations +%% under the License. +%% +%% %CopyrightEnd% +%% +%% +-module(httpc_manager). + +-behaviour(gen_server). + +-include("httpc_internal.hrl"). +-include("http_internal.hrl"). + +%% Internal Application API +-export([start_link/1, start_link/2, request/2, cancel_request/2, + request_canceled/2, retry_request/2, redirect_request/2, + insert_session/2, delete_session/2, set_options/2, store_cookies/3, + cookies/2, session_type/1]). + +%% gen_server callbacks +-export([init/1, handle_call/3, handle_cast/2, handle_info/2, terminate/2, + code_change/3]). + +-record(state, { + cancel = [], % [{RequestId, HandlerPid, ClientPid}] + handler_db, % ets() - Entry: {Requestid, HandlerPid, ClientPid} + cookie_db, % {ets(), dets()} - {session_cookie_db, cookie_db} + session_db, % ets() - Entry: #tcp_session{} + profile_name, % atom() + options = #options{} + }). + + +%%==================================================================== +%% Internal Application API +%%==================================================================== +%%-------------------------------------------------------------------- +%% Function: start_link({ProfileName, CookieDir}) -> {ok, Pid} +%% +%% ProfileName - httpc_manager_<Profile> +%% CookieDir - directory() +%% +%% Description: Starts the http request manger process. (Started by +%% the intes supervisor.) +%%-------------------------------------------------------------------- +start_link({default, CookieDir}) -> + gen_server:start_link({local, ?MODULE}, ?MODULE, + [?MODULE, {http_default_cookie_db, CookieDir}], + []); +start_link({Profile, CookieDir}) -> + ProfileName = list_to_atom("httpc_manager_" ++ atom_to_list(Profile)), + gen_server:start_link({local, ProfileName}, ?MODULE, + [ProfileName, + {http_default_cookie_db, CookieDir}], []). +start_link({Profile, CookieDir}, stand_alone) -> + ProfileName = list_to_atom("stand_alone_" ++ atom_to_list(Profile)), + gen_server:start_link(?MODULE, [ProfileName, + {http_default_cookie_db, CookieDir}], + []). +%%-------------------------------------------------------------------- +%% Function: request(Request, ProfileName) -> +%% {ok, Requestid} | {error, Reason} +%% Request = #request{} +%% ProfileName = atom() +%% +%% Description: Sends a request to the httpc manager process. +%%-------------------------------------------------------------------- +request(Request, ProfileName) -> + call(ProfileName, {request, Request}, infinity). + +%%-------------------------------------------------------------------- +%% Function: retry_request(Request, ProfileName) -> _ +%% Request = #request{} +%% ProfileName = atom() +%% +%% Description: Resends a request to the httpc manager process, intended +%% to be called by the httpc handler process if it has to terminate with +%% a non empty pipeline. +%%-------------------------------------------------------------------- +retry_request(Request, ProfileName) -> + cast(ProfileName, {retry_or_redirect_request, Request}). + +%%-------------------------------------------------------------------- +%% Function: redirect_request(Request, ProfileName) -> _ +%% Request = #request{} +%% ProfileName = atom() +%% +%% Description: Sends an atoumatic redirect request to the httpc +%% manager process, intended to be called by the httpc handler process +%% when the automatic redirect option is set. +%%-------------------------------------------------------------------- +redirect_request(Request, ProfileName) -> + cast(ProfileName, {retry_or_redirect_request, Request}). + +%%-------------------------------------------------------------------- +%% Function: cancel_request(RequestId, ProfileName) -> ok +%% RequestId - ref() +%% ProfileName = atom() +%% +%% Description: Cancels the request with <RequestId>. +%%-------------------------------------------------------------------- +cancel_request(RequestId, ProfileName) -> + call(ProfileName, {cancel_request, RequestId}, infinity). + +%%-------------------------------------------------------------------- +%% Function: request_canceled(RequestId, ProfileName) -> ok +%% RequestId - ref() +%% ProfileName = atom() +%% +%% Description: Confirms that a request has been canceld. Intended to +%% be called by the httpc handler process. +%%-------------------------------------------------------------------- +request_canceled(RequestId, ProfileName) -> + cast(ProfileName, {request_canceled, RequestId}). + +%%-------------------------------------------------------------------- +%% Function: insert_session(Session, ProfileName) -> _ +%% Session - #tcp_session{} +%% ProfileName - atom() +%% +%% Description: Inserts session information into the httpc manager +%% table <ProfileName>_session_db. Intended to be called by +%% the httpc request handler process. +%%-------------------------------------------------------------------- +insert_session(Session, ProfileName) -> + Db = list_to_atom(atom_to_list(ProfileName) ++ "_session_db"), + ets:insert(Db, Session). + +%%-------------------------------------------------------------------- +%% Function: delete_session(SessionId, ProfileName) -> _ +%% SessionId - {{Host, Port}, HandlerPid} +%% ProfileName - atom() +%% +%% Description: Deletes session information from the httpc manager +%% table httpc_manager_session_db_<Profile>. Intended to be called by +%% the httpc request handler process. +%%-------------------------------------------------------------------- +delete_session(SessionId, ProfileName) -> + Db = list_to_atom(atom_to_list(ProfileName) ++ "_session_db"), + ets:delete(Db, SessionId). + +%%-------------------------------------------------------------------- +%% Function: set_options(Options, ProfileName) -> ok +%% +%% Options = [Option] +%% Option = {proxy, {Proxy, [NoProxy]}} +%% | {max_pipeline_length, integer()} | +%% {max_sessions, integer()} | {pipeline_timeout, integer()} +%% Proxy = {Host, Port} +%% NoProxy - [Domain | HostName | IPAddress] +%% Max - integer() +%% ProfileName = atom() +%% +%% Description: Sets the options to be used by the client. +%%-------------------------------------------------------------------- +set_options(Options, ProfileName) -> + cast(ProfileName, {set_options, Options}). + +%%-------------------------------------------------------------------- +%% Function: store_cookies(Cookies, Address, ProfileName) -> ok +%% +%% Cookies = [Cookie] +%% Cookie = #http_cookie{} +%% ProfileName = atom() +%% +%% Description: Stores cookies from the server. +%%-------------------------------------------------------------------- +store_cookies([], _, _) -> + ok; +store_cookies(Cookies, Address, ProfileName) -> + cast(ProfileName, {store_cookies, {Cookies, Address}}). + +%%-------------------------------------------------------------------- +%% Function: cookies(Url, ProfileName) -> ok +%% +%% Url = string() +%% ProfileName = atom() +%% +%% Description: Retrieves the cookies that would be sent when +%% requesting <Url>. +%%-------------------------------------------------------------------- +cookies(Url, ProfileName) -> + call(ProfileName, {cookies, Url}, infinity). + +%%-------------------------------------------------------------------- +%% Function: session_type(Options) -> ok +%% +%% Options = #options{} +%% +%% Description: Determines if to use pipelined sessions or not. +%%-------------------------------------------------------------------- +session_type(#options{pipeline_timeout = 0}) -> + keep_alive; +session_type(_) -> + pipeline. + +%%==================================================================== +%% gen_server callback functions +%%==================================================================== + +%%-------------------------------------------------------------------- +%% Function: init([ProfileName, CookiesConf]) -> {ok, State} | +%% {ok, State, Timeout} | ignore |{stop, Reason} +%% Description: Initiates the httpc_manger process +%%-------------------------------------------------------------------- +init([ProfileName, CookiesConf | _]) -> + process_flag(trap_exit, true), + SessionDb = list_to_atom(atom_to_list(ProfileName) ++ "_session_db"), + ets:new(SessionDb, + [public, set, named_table, {keypos, #tcp_session.id}]), + ?hcri("starting", [{profile, ProfileName}]), + {ok, #state{handler_db = ets:new(handler_db, [protected, set]), + cookie_db = + http_cookie:open_cookie_db({CookiesConf, + http_session_cookie_db}), + session_db = SessionDb, + profile_name = ProfileName + }}. + +%%-------------------------------------------------------------------- +%% Function: handle_call(Request, From, State) -> {reply, Reply, State} | +%% {reply, Reply, State, Timeout} | +%% {noreply, State} | +%% {noreply, State, Timeout} | +%% {stop, Reason, Reply, State} | (terminate/2 is called) +%% {stop, Reason, State} (terminate/2 is called) +%% Description: Handling call messages +%%-------------------------------------------------------------------- +handle_call({request, Request}, _, State) -> + ?hcri("request", [{request, Request}]), + case (catch handle_request(Request, State)) of + {reply, Msg, NewState} -> + {reply, Msg, NewState}; + Error -> + {stop, Error, httpc_response:error(Request, Error), State} + end; + +handle_call({cancel_request, RequestId}, From, State) -> + ?hcri("cancel_request", [{request_id, RequestId}]), + case ets:lookup(State#state.handler_db, RequestId) of + [] -> + ok, %% Nothing to cancel + {reply, ok, State}; + [{_, Pid, _}] -> + httpc_handler:cancel(RequestId, Pid), + {noreply, State#state{cancel = + [{RequestId, Pid, From} | + State#state.cancel]}} + end; + +handle_call({cookies, Url}, _, State) -> + case http_uri:parse(Url) of + {Scheme, _, Host, Port, Path, _} -> + CookieHeaders = + http_cookie:header(Scheme, {Host, Port}, + Path, State#state.cookie_db), + {reply, CookieHeaders, State}; + Msg -> + {reply, Msg, State} + end; + +handle_call(Msg, From, State) -> + Report = io_lib:format("HTTPC_MANAGER recived unkown call: ~p" + "from: ~p~n", [Msg, From]), + error_logger:error_report(Report), + {reply, {error, 'API_violation'}, State}. + +%%-------------------------------------------------------------------- +%% Function: handle_cast(Msg, State) -> {noreply, State} | +%% {noreply, State, Timeout} | +%% {stop, Reason, State} (terminate/2 is called) +%% Description: Handling cast messages +%%-------------------------------------------------------------------- +handle_cast({retry_or_redirect_request, {Time, Request}}, + #state{profile_name = ProfileName} = State) -> + {ok, _} = timer:apply_after(Time, ?MODULE, retry_request, [Request, ProfileName]), + {noreply, State}; + +handle_cast({retry_or_redirect_request, Request}, State) -> + case (catch handle_request(Request, State)) of + {reply, {ok, _}, NewState} -> + {noreply, NewState}; + Error -> + httpc_response:error(Request, Error), + {stop, Error, State} + end; + +handle_cast({request_canceled, RequestId}, State) -> + ets:delete(State#state.handler_db, RequestId), + case lists:keysearch(RequestId, 1, State#state.cancel) of + {value, Entry = {RequestId, _, From}} -> + gen_server:reply(From, ok), + {noreply, + State#state{cancel = lists:delete(Entry, State#state.cancel)}}; + _ -> + {noreply, State} + end; +handle_cast({set_options, Options}, State = #state{options = OldOptions}) -> + NewOptions = + #options{proxy = get_proxy(Options, OldOptions), + pipeline_timeout = get_pipeline_timeout(Options, OldOptions), + max_pipeline_length = get_max_pipeline_length(Options, OldOptions), + max_keep_alive_length = get_max_keep_alive_length(Options, OldOptions), + keep_alive_timeout = get_keep_alive_timeout(Options, OldOptions), + max_sessions = get_max_sessions(Options, OldOptions), + cookies = get_cookies(Options, OldOptions), + ipfamily = get_ipfamily(Options, OldOptions), + ip = get_ip(Options, OldOptions), + port = get_port(Options, OldOptions), + verbose = get_verbose(Options, OldOptions) + }, + case {OldOptions#options.verbose, NewOptions#options.verbose} of + {Same, Same} -> + ok; + {_, false} -> + dbg:stop(); + {false, Level} -> + dbg:tracer(), + handle_verbose(Level); + {_, Level} -> + dbg:stop(), + dbg:tracer(), + handle_verbose(Level) + end, + {noreply, State#state{options = NewOptions}}; + +handle_cast({store_cookies, _}, + State = #state{options = #options{cookies = disabled}}) -> + {noreply, State}; + +handle_cast({store_cookies, {Cookies, _}}, State) -> + ok = do_store_cookies(Cookies, State), + {noreply, State}; + +handle_cast(Msg, State) -> + Report = io_lib:format("HTTPC_MANAGER recived unkown cast: ~p", + [Msg]), + error_logger:error_report(Report), + {noreply, State}. + + + +%%-------------------------------------------------------------------- +%% Function: handle_info(Info, State) -> {noreply, State} | +%% {noreply, State, Timeout} | +%% {stop, Reason, State} (terminate/2 is called) +%% Description: Handling all non call/cast messages +%%--------------------------------------------------------- +handle_info({'EXIT', _, _}, State) -> + %% Handled in DOWN + {noreply, State}; +handle_info({'DOWN', _, _, Pid, _}, State) -> + ets:match_delete(State#state.handler_db, {'_', Pid, '_'}), + + %% If there where any canceled request, handled by the + %% the process that now has terminated, the + %% cancelation can be viewed as sucessfull! + NewCanceldList = + lists:foldl(fun(Entry = {_, HandlerPid, From}, Acc) -> + case HandlerPid of + Pid -> + gen_server:reply(From, ok), + lists:delete(Entry, Acc); + _ -> + Acc + end + end, State#state.cancel, State#state.cancel), + {noreply, State#state{cancel = NewCanceldList}}; +handle_info(Info, State) -> + Report = io_lib:format("Unknown message in " + "httpc_manager:handle_info ~p~n", [Info]), + error_logger:error_report(Report), + {noreply, State}. +%%-------------------------------------------------------------------- +%% Function: terminate(Reason, State) -> _ (ignored by gen_server) +%% Description: Shutdown the httpc_handler +%%-------------------------------------------------------------------- +terminate(_, State) -> + http_cookie:close_cookie_db(State#state.cookie_db), + ets:delete(State#state.session_db), + ets:delete(State#state.handler_db). + +%%-------------------------------------------------------------------- +%% Func: code_change(_OldVsn, State, Extra) -> {ok, NewState} +%% Purpose: Convert process state when code is changed +%%-------------------------------------------------------------------- +code_change(_OldVsn, State, _Extra) -> + {ok, State}. + +%%-------------------------------------------------------------------- +%% Internal functions +%%-------------------------------------------------------------------- +handle_request(#request{settings = + #http_options{version = "HTTP/0.9"}} = Request, + State) -> + %% Act as an HTTP/0.9 client that does not know anything + %% about persistent connections + + NewRequest = handle_cookies(generate_request_id(Request), State), + NewHeaders = + (NewRequest#request.headers)#http_request_h{connection + = undefined}, + start_handler(NewRequest#request{headers = NewHeaders}, State), + {reply, {ok, NewRequest#request.id}, State}; + +handle_request(#request{settings = + #http_options{version = "HTTP/1.0"}} = Request, + State) -> + %% Act as an HTTP/1.0 client that does not + %% use persistent connections + + NewRequest = handle_cookies(generate_request_id(Request), State), + NewHeaders = + (NewRequest#request.headers)#http_request_h{connection + = "close"}, + start_handler(NewRequest#request{headers = NewHeaders}, State), + {reply, {ok, NewRequest#request.id}, State}; + +handle_request(Request, State = #state{options = Options}) -> + + NewRequest = handle_cookies(generate_request_id(Request), State), + SessionType = session_type(Options), + case select_session(Request#request.method, + Request#request.address, + Request#request.scheme, SessionType, State) of + {ok, HandlerPid} -> + pipeline_or_keep_alive(NewRequest, HandlerPid, State); + no_connection -> + start_handler(NewRequest, State); + {no_session, OpenSessions} when OpenSessions + < Options#options.max_sessions -> + start_handler(NewRequest, State); + {no_session, _} -> + %% Do not start any more persistent connections + %% towards this server. + NewHeaders = + (NewRequest#request.headers)#http_request_h{connection + = "close"}, + start_handler(NewRequest#request{headers = NewHeaders}, State) + end, + {reply, {ok, NewRequest#request.id}, State}. + +select_session(Method, HostPort, Scheme, SessionTyp, + #state{options = #options{max_pipeline_length = + MaxPipe, + max_keep_alive_length = MaxKeepAlive}, + session_db = SessionDb}) -> + case httpc_request:is_idempotent(Method) or (SessionTyp == keep_alive) of + true -> + Candidates = ets:match(SessionDb, + {'_', {HostPort, '$1'}, + false, Scheme, '_', '$2', SessionTyp}), + select_session(Candidates, MaxKeepAlive, MaxPipe, SessionTyp); + false -> + no_connection + end. + +select_session(Candidates, Max, _, keep_alive) -> + select_session(Candidates, Max); +select_session(Candidates, _, Max, pipeline) -> + select_session(Candidates, Max). + +select_session(Candidates, Max) -> + case Candidates of + [] -> + no_connection; + _ -> + NewCandidates = + lists:foldl( + fun([Pid, Length], Acc) when Length =< Max -> + [{Pid, Length} | Acc]; + (_, Acc) -> + Acc + end, [], Candidates), + + case lists:keysort(2, NewCandidates) of + [] -> + {no_session, length(Candidates)}; + [{HandlerPid, _} | _] -> + {ok, HandlerPid} + end + end. + +pipeline_or_keep_alive(Request, HandlerPid, State) -> + case (catch httpc_handler:send(Request, HandlerPid)) of + ok -> + ets:insert(State#state.handler_db, {Request#request.id, + HandlerPid, + Request#request.from}); + _ -> %timeout pipelining failed + start_handler(Request, State) + end. + +start_handler(Request, State) -> + {ok, Pid} = + case is_inets_manager() of + true -> + httpc_handler_sup:start_child([Request, State#state.options, + State#state.profile_name]); + false -> + httpc_handler:start_link(Request, State#state.options, + State#state.profile_name) + end, + ets:insert(State#state.handler_db, {Request#request.id, + Pid, Request#request.from}), + erlang:monitor(process, Pid). + +is_inets_manager() -> + case get('$ancestors') of + [httpc_profile_sup | _] -> + true; + _ -> + false + end. + +generate_request_id(Request) -> + case Request#request.id of + undefined -> + RequestId = make_ref(), + Request#request{id = RequestId}; + _ -> + %% This is an automatic redirect or a retryed pipelined + %% request keep the old id. + Request + end. + +handle_cookies(Request, #state{options = #options{cookies = disabled}}) -> + Request; +handle_cookies(Request = #request{scheme = Scheme, address = Address, + path = Path, headers = + Headers = #http_request_h{other = Other}}, + #state{cookie_db = Db}) -> + case http_cookie:header(Scheme, Address, Path, Db) of + {"cookie", ""} -> + Request; + CookieHeader -> + NewHeaders = + Headers#http_request_h{other = [CookieHeader | Other]}, + Request#request{headers = NewHeaders} + end. + +do_store_cookies([], _) -> + ok; +do_store_cookies([Cookie | Cookies], State) -> + ok = http_cookie:insert(Cookie, State#state.cookie_db), + do_store_cookies(Cookies, State). + +call(ProfileName, Msg, Timeout) -> + gen_server:call(ProfileName, Msg, Timeout). + +cast(ProfileName, Msg) -> + gen_server:cast(ProfileName, Msg). + + + +get_proxy(Opts, #options{proxy = Default}) -> + proplists:get_value(proxy, Opts, Default). + +get_pipeline_timeout(Opts, #options{pipeline_timeout = Default}) -> + proplists:get_value(pipeline_timeout, Opts, Default). + +get_max_pipeline_length(Opts, #options{max_pipeline_length = Default}) -> + proplists:get_value(max_pipeline_length, Opts, Default). + +get_max_keep_alive_length(Opts, #options{max_keep_alive_length = Default}) -> + proplists:get_value(max_keep_alive_length, Opts, Default). + +get_keep_alive_timeout(Opts, #options{keep_alive_timeout = Default}) -> + proplists:get_value(keep_alive_timeout, Opts, Default). + +get_max_sessions(Opts, #options{max_sessions = Default}) -> + proplists:get_value(max_sessions, Opts, Default). + +get_cookies(Opts, #options{cookies = Default}) -> + proplists:get_value(cookies, Opts, Default). + +get_ipfamily(Opts, #options{ipfamily = IpFamily}) -> + case lists:keysearch(ipfamily, 1, Opts) of + false -> + case proplists:get_value(ipv6, Opts) of + enabled -> + inet6fb4; + disabled -> + inet; + _ -> + IpFamily + end; + {value, {_, Value}} -> + Value + end. + +get_ip(Opts, #options{ip = Default}) -> + proplists:get_value(ip, Opts, Default). + +get_port(Opts, #options{port = Default}) -> + proplists:get_value(port, Opts, Default). + +get_verbose(Opts, #options{verbose = Default}) -> + proplists:get_value(verbose, Opts, Default). + + +handle_verbose(debug) -> + dbg:p(self(), [call]), + dbg:tp(?MODULE, [{'_', [], [{return_trace}]}]); +handle_verbose(trace) -> + dbg:p(self(), [call]), + dbg:tpl(?MODULE, [{'_', [], [{return_trace}]}]); +handle_verbose(_) -> + ok. + +%% d(F) -> +%% d(F, []). + +%% d(F, A) -> +%% d(get(dbg), F, A). + +%% d(true, F, A) -> +%% io:format(user, "~w:~w:" ++ F ++ "~n", [self(), ?MODULE | A]); +%% d(_, _, _) -> +%% ok. + diff --git a/lib/inets/src/http_client/httpc_profile_sup.erl b/lib/inets/src/http_client/httpc_profile_sup.erl new file mode 100644 index 0000000000..2351083435 --- /dev/null +++ b/lib/inets/src/http_client/httpc_profile_sup.erl @@ -0,0 +1,107 @@ +%% +%% %CopyrightBegin% +%% +%% Copyright Ericsson AB 2007-2009. All Rights Reserved. +%% +%% The contents of this file are subject to the Erlang Public License, +%% Version 1.1, (the "License"); you may not use this file except in +%% compliance with the License. You should have received a copy of the +%% Erlang Public License along with this software. If not, it can be +%% retrieved online at http://www.erlang.org/. +%% +%% Software distributed under the License is distributed on an "AS IS" +%% basis, WITHOUT WARRANTY OF ANY KIND, either express or implied. See +%% the License for the specific language governing rights and limitations +%% under the License. +%% +%% %CopyrightEnd% +%% +%% +-module(httpc_profile_sup). + +-behaviour(supervisor). + +%% API +-export([start_link/1]). +-export([start_child/1, restart_child/1, stop_child/1]). + +%% Supervisor callback +-export([init/1]). + +%%%========================================================================= +%%% API +%%%========================================================================= +start_link(HttpcServices) -> + supervisor:start_link({local, ?MODULE}, ?MODULE, [HttpcServices]). + +start_child(PropList) -> + case proplists:get_value(profile, PropList) of + undefined -> + {error, no_profile}; + Profile -> + Dir = proplists:get_value(data_dir, PropList, only_session_cookies), + Spec = httpc_child_spec(Profile, Dir), + supervisor:start_child(?MODULE, Spec) + end. + +restart_child(Profile) -> + Name = id(Profile), + case supervisor:terminate_child(?MODULE, Name) of + ok -> + supervisor:restart_child(?MODULE, Name); + Error -> + Error + end. + +stop_child(Profile) -> + Name = id(Profile), + case supervisor:terminate_child(?MODULE, Name) of + ok -> + supervisor:delete_child(?MODULE, Name); + Error -> + Error + end. + +id(Profile) -> + DefaultProfile = http:default_profile(), + case Profile of + DefaultProfile -> + httpc_manager; + _ -> + {http, Profile} + end. + + +%%%========================================================================= +%%% Supervisor callback +%%%========================================================================= +init([]) -> + init([[]]); +init([HttpcServices]) -> + RestartStrategy = one_for_one, + MaxR = 10, + MaxT = 3600, + Children = child_spec(HttpcServices, []), + {ok, {{RestartStrategy, MaxR, MaxT}, Children}}. + +child_spec([], Acc) -> + Acc; +%% For backwards compatibility +child_spec([{httpc, {Profile, Dir}} | Rest], Acc) -> + Spec = httpc_child_spec(Profile, Dir), + child_spec(Rest, [Spec | Acc]); +child_spec([{httpc, PropList} | Rest], Acc) when is_list(PropList) -> + Profile = proplists:get_value(profile, PropList), + Dir = proplists:get_value(data_dir, PropList), + Spec = httpc_child_spec(Profile, Dir), + child_spec(Rest, [Spec | Acc]). + +httpc_child_spec(Profile, Dir) -> + Name = id(Profile), + StartFunc = {httpc_manager, start_link, [{Profile, Dir}]}, + Restart = permanent, + Shutdown = 4000, + Modules = [httpc_manager], + Type = worker, + {Name, StartFunc, Restart, Shutdown, Type, Modules}. + diff --git a/lib/inets/src/http_client/httpc_request.erl b/lib/inets/src/http_client/httpc_request.erl new file mode 100644 index 0000000000..3d66638d66 --- /dev/null +++ b/lib/inets/src/http_client/httpc_request.erl @@ -0,0 +1,209 @@ +%% +%% %CopyrightBegin% +%% +%% Copyright Ericsson AB 2004-2009. All Rights Reserved. +%% +%% The contents of this file are subject to the Erlang Public License, +%% Version 1.1, (the "License"); you may not use this file except in +%% compliance with the License. You should have received a copy of the +%% Erlang Public License along with this software. If not, it can be +%% retrieved online at http://www.erlang.org/. +%% +%% Software distributed under the License is distributed on an "AS IS" +%% basis, WITHOUT WARRANTY OF ANY KIND, either express or implied. See +%% the License for the specific language governing rights and limitations +%% under the License. +%% +%% %CopyrightEnd% +%% + +-module(httpc_request). + +-include("http_internal.hrl"). +-include("httpc_internal.hrl"). + +%%% Internal API +-export([send/3, is_idempotent/1, is_client_closing/1]). + +%%%========================================================================= +%%% Internal application API +%%%========================================================================= +%%------------------------------------------------------------------------- +%% send(MaybeProxy, Request) -> +%% MaybeProxy - {Host, Port} +%% Host = string() +%% Port = integer() +%% Request - #request{} +%% Socket - socket() +%% CookieSupport - enabled | disabled | verify +%% +%% Description: Composes and sends a HTTP-request. +%%------------------------------------------------------------------------- +send(SendAddr, #request{method = Method, scheme = Scheme, + path = Path, pquery = Query, headers = Headers, + content = Content, address = Address, + abs_uri = AbsUri, headers_as_is = HeadersAsIs, + settings = HttpOptions, + userinfo = UserInfo}, + Socket) -> + + TmpHeaders = handle_user_info(UserInfo, Headers), + + {TmpHeaders2, Body} = + post_data(Method, TmpHeaders, Content, HeadersAsIs), + + {NewHeaders, Uri} = case Address of + SendAddr -> + {TmpHeaders2, Path ++ Query}; + _Proxy -> + TmpHeaders3 = + handle_proxy(HttpOptions, TmpHeaders2), + {TmpHeaders3, AbsUri} + end, + + FinalHeaders = case NewHeaders of + HeaderList when is_list(HeaderList) -> + http_headers(HeaderList, []); + _ -> + http_request:http_headers(NewHeaders) + end, + Version = HttpOptions#http_options.version, + + Message = [method(Method), " ", Uri, " ", + version(Version), ?CRLF, headers(FinalHeaders, Version), ?CRLF, Body], + + http_transport:send(socket_type(Scheme), Socket, lists:append(Message)). + +%%------------------------------------------------------------------------- +%% is_idempotent(Method) -> +%% Method = atom() +%% +%% Description: Checks if Method is considered idempotent. +%%------------------------------------------------------------------------- + +%% In particular, the convention has been established that the GET and +%% HEAD methods SHOULD NOT have the significance of taking an action +%% other than retrieval. These methods ought to be considered "safe". +is_idempotent(head) -> + true; +is_idempotent(get) -> + true; +%% Methods can also have the property of "idempotence" in that (aside +%% from error or expiration issues) the side-effects of N > 0 +%% identical requests is the same as for a single request. +is_idempotent(put) -> + true; +is_idempotent(delete) -> + true; +%% Also, the methods OPTIONS and TRACE SHOULD NOT have side effects, +%% and so are inherently idempotent. +is_idempotent(trace) -> + true; +is_idempotent(options) -> + true; +is_idempotent(_) -> + false. + +%%------------------------------------------------------------------------- +%% is_client_closing(Headers) -> +%% Headers = #http_request_h{} +%% +%% Description: Checks if the client has supplied a "Connection: +%% close" header. +%%------------------------------------------------------------------------- +is_client_closing(Headers) -> + case Headers#http_request_h.connection of + "close" -> + true; + _ -> + false + end. + +%%%======================================================================== +%%% Internal functions +%%%======================================================================== +post_data(Method, Headers, {ContentType, Body}, HeadersAsIs) + when Method == post; Method == put -> + ContentLength = body_length(Body), + NewBody = case Headers#http_request_h.expect of + "100-continue" -> + ""; + _ -> + Body + end, + + NewHeaders = case HeadersAsIs of + [] -> + Headers#http_request_h{'content-type' = + ContentType, + 'content-length' = + ContentLength}; + _ -> + HeadersAsIs + end, + + {NewHeaders, NewBody}; + +post_data(_, Headers, _, []) -> + {Headers, ""}; +post_data(_, _, _, HeadersAsIs = [_|_]) -> + {HeadersAsIs, ""}. + +body_length(Body) when is_binary(Body) -> + integer_to_list(size(Body)); + +body_length(Body) when is_list(Body) -> + integer_to_list(length(Body)). + +method(Method) -> + http_util:to_upper(atom_to_list(Method)). + +version("HTTP/0.9") -> + ""; +version(Version) -> + Version. + +headers(_, "HTTP/0.9") -> + ""; +%% HTTP 1.1 headers not present in HTTP 1.0 should be +%% consider as unknown extension headers that should be +%% ignored. +headers(Headers, _) -> + Headers. + +socket_type(http) -> + ip_comm; +socket_type(https) -> + {ssl, []}. + +http_headers([], Headers) -> + lists:flatten(Headers); +http_headers([{Key,Value} | Rest], Headers) -> + Header = Key ++ ": " ++ Value ++ ?CRLF, + http_headers(Rest, [Header | Headers]). + +handle_proxy(_, Headers) when is_list(Headers) -> + Headers; %% Headers as is option was specified +handle_proxy(HttpOptions, Headers) -> + case HttpOptions#http_options.proxy_auth of + undefined -> + Headers; + {User, Password} -> + UserPasswd = base64:encode_to_string(User ++ ":" ++ Password), + Headers#http_request_h{'proxy-authorization' = + "Basic " ++ UserPasswd} + end. + +handle_user_info([], Headers) -> + Headers; +handle_user_info(UserInfo, Headers) -> + case string:tokens(UserInfo, ":") of + [User, Passwd] -> + UserPasswd = base64:encode_to_string(User ++ ":" ++ Passwd), + Headers#http_request_h{authorization = "Basic " ++ UserPasswd}; + [User] -> + UserPasswd = base64:encode_to_string(User ++ ":"), + Headers#http_request_h{authorization = "Basic " ++ UserPasswd}; + _ -> + Headers + end. diff --git a/lib/inets/src/http_client/httpc_response.erl b/lib/inets/src/http_client/httpc_response.erl new file mode 100644 index 0000000000..e2ba66f730 --- /dev/null +++ b/lib/inets/src/http_client/httpc_response.erl @@ -0,0 +1,431 @@ +%% +%% %CopyrightBegin% +%% +%% Copyright Ericsson AB 2004-2009. All Rights Reserved. +%% +%% The contents of this file are subject to the Erlang Public License, +%% Version 1.1, (the "License"); you may not use this file except in +%% compliance with the License. You should have received a copy of the +%% Erlang Public License along with this software. If not, it can be +%% retrieved online at http://www.erlang.org/. +%% +%% Software distributed under the License is distributed on an "AS IS" +%% basis, WITHOUT WARRANTY OF ANY KIND, either express or implied. See +%% the License for the specific language governing rights and limitations +%% under the License. +%% +%% %CopyrightEnd% +%% + +-module(httpc_response). + +-include("http_internal.hrl"). +-include("httpc_internal.hrl"). + +%% API +-export([parse/1, result/2, send/2, error/2, is_server_closing/1, + stream_start/3]). + +%% Callback API - used for example if the header/body is received a +%% little at a time on a socket. +-export([parse_version/1, parse_status_code/1, parse_reason_phrase/1, + parse_headers/1, whole_body/1, whole_body/2]). + +%%%========================================================================= +%%% API +%%%========================================================================= + +parse([Bin, MaxHeaderSize, Relaxed]) -> + parse_version(Bin, [], MaxHeaderSize, [], Relaxed). + +whole_body([Bin, Body, Length]) -> + whole_body(<<Body/binary, Bin/binary>>, Length). + +%% Functions that may be returned during the decoding process +%% if the input data is incompleate. +parse_version([Bin, Version, MaxHeaderSize, Result, Relaxed]) -> + parse_version(Bin, Version, MaxHeaderSize, Result, Relaxed). + +parse_status_code([Bin, Code, MaxHeaderSize, Result, Relaxed]) -> + parse_status_code(Bin, Code, MaxHeaderSize, Result, Relaxed). + +parse_reason_phrase([Bin, Rest, Phrase, MaxHeaderSize, Result, Relaxed]) -> + parse_reason_phrase(<<Rest/binary, Bin/binary>>, Phrase, + MaxHeaderSize, Result, Relaxed). + +parse_headers([Bin, Rest,Header, Headers, MaxHeaderSize, Result, Relaxed]) -> + parse_headers(<<Rest/binary, Bin/binary>>, Header, Headers, + MaxHeaderSize, Result, Relaxed). + +whole_body(Body, Length) -> + case size(Body) of + N when (N < Length) andalso (N > 0) -> + {?MODULE, whole_body, [Body, Length]}; + %% OBS! The Server may close the connection to indicate that the + %% whole body is now sent instead of sending a lengh + %% indicator.In this case the lengh indicator will be + %% -1. + N when (N >= Length) andalso (Length >= 0) -> + %% Potential trailing garbage will be thrown away in + %% format_response/1 Some servers may send a 100-continue + %% response without the client requesting it through an + %% expect header in this case the trailing bytes may be + %% part of the real response message. + {ok, Body}; + _ -> %% Length == -1 + {?MODULE, whole_body, [Body, Length]} + end. + +%%------------------------------------------------------------------------- +%% result(Response, Request) -> +%% Response - {StatusLine, Headers, Body} +%% Request - #request{} +%% Session - #tcp_session{} +%% +%% Description: Checks the status code ... +%%------------------------------------------------------------------------- +result(Response = {{_, Code,_}, _, _}, + Request = #request{stream = Stream}) + when ((Code =:= 200) orelse (Code =:= 206)) andalso (Stream =/= none) -> + stream_end(Response, Request); + +result(Response = {{_,100,_}, _, _}, Request) -> + status_continue(Response, Request); + +%% In redirect loop +result(Response = {{_, Code, _}, _, _}, Request = + #request{redircount = Redirects, + settings = #http_options{autoredirect = true}}) + when ((Code div 100) =:= 3) andalso (Redirects > ?HTTP_MAX_REDIRECTS) -> + transparent(Response, Request); + +%% multiple choices +result(Response = {{_, 300, _}, _, _}, + Request = #request{settings = + #http_options{autoredirect = + true}}) -> + redirect(Response, Request); + +result(Response = {{_, Code, _}, _, _}, + Request = #request{settings = + #http_options{autoredirect = true}, + method = head}) when (Code =:= 301) orelse + (Code =:= 302) orelse + (Code =:= 303) orelse + (Code =:= 307) -> + redirect(Response, Request); +result(Response = {{_, Code, _}, _, _}, + Request = #request{settings = + #http_options{autoredirect = true}, + method = get}) when (Code =:= 301) orelse + (Code =:= 302) orelse + (Code =:= 303) orelse + (Code =:= 307) -> + redirect(Response, Request); + + +result(Response = {{_,503,_}, _, _}, Request) -> + status_service_unavailable(Response, Request); +result(Response = {{_,Code,_}, _, _}, Request) when (Code div 100) =:= 5 -> + status_server_error_50x(Response, Request); + +result(Response, Request) -> + transparent(Response, Request). + +send(To, Msg) -> + To ! {http, Msg}. + +%%%======================================================================== +%%% Internal functions +%%%======================================================================== +parse_version(<<>>, Version, MaxHeaderSize, Result, Relaxed) -> + {?MODULE, parse_version, [Version, MaxHeaderSize,Result, Relaxed]}; +parse_version(<<?SP, Rest/binary>>, Version, + MaxHeaderSize, Result, Relaxed) -> + case lists:reverse(Version) of + "HTTP/" ++ _ = Newversion -> + parse_status_code(Rest, [], MaxHeaderSize, + [Newversion | Result], Relaxed); + NewVersion -> + throw({error, {invalid_version, NewVersion}}) + end; + +parse_version(<<Octet, Rest/binary>>, Version, + MaxHeaderSize, Result, Relaxed) -> + parse_version(Rest, [Octet | Version], MaxHeaderSize,Result, Relaxed). + +parse_status_code(<<>>, StatusCodeStr, MaxHeaderSize, Result, Relaxed) -> + {?MODULE, parse_status_code, + [StatusCodeStr, MaxHeaderSize, Result, Relaxed]}; + +%% Some Apache servers has been known to leave out the reason phrase, +%% in relaxed mode we will allow this. +parse_status_code(<<?CR>> = Data, StatusCodeStr, + MaxHeaderSize, Result, true) -> + {?MODULE, parse_status_code, + [Data, StatusCodeStr, MaxHeaderSize, Result, true]}; +parse_status_code(<<?LF>>, StatusCodeStr, + MaxHeaderSize, Result, true) -> + %% If ?CR is is missing RFC2616 section-19.3 + parse_status_code(<<?CR, ?LF>>, StatusCodeStr, + MaxHeaderSize, Result, true); + +parse_status_code(<<?CR, ?LF, Rest/binary>>, StatusCodeStr, + MaxHeaderSize, Result, true) -> + parse_headers(Rest, [], [], MaxHeaderSize, + [" ", list_to_integer(lists:reverse( + string:strip(StatusCodeStr))) + | Result], true); + +parse_status_code(<<?SP, Rest/binary>>, StatusCodeStr, + MaxHeaderSize, Result, Relaxed) -> + parse_reason_phrase(Rest, [], MaxHeaderSize, + [list_to_integer(lists:reverse(StatusCodeStr)) | + Result], Relaxed); + +parse_status_code(<<Octet, Rest/binary>>, StatusCodeStr, + MaxHeaderSize,Result, Relaxed) -> + parse_status_code(Rest, [Octet | StatusCodeStr], MaxHeaderSize, Result, + Relaxed). + +parse_reason_phrase(<<>>, Phrase, MaxHeaderSize, Result, Relaxed) -> + {?MODULE, parse_reason_phrase, + [<<>>, Phrase, MaxHeaderSize, Result, Relaxed]}; + +parse_reason_phrase(<<?CR, ?LF, ?LF, Body/binary>>, Phrase, + MaxHeaderSize, Result, Relaxed) -> + %% If ?CR is is missing RFC2616 section-19.3 + parse_reason_phrase(<<?CR, ?LF, ?CR, ?LF, Body/binary>>, Phrase, + MaxHeaderSize, Result, Relaxed); + +parse_reason_phrase(<<?CR, ?LF, ?CR, ?LF, Body/binary>>, Phrase, + _, Result, _) -> + ResponseHeaderRcord = + http_response:headers([], #http_response_h{}), + {ok, list_to_tuple( + lists:reverse([Body, ResponseHeaderRcord | + [lists:reverse(Phrase) | Result]]))}; + +parse_reason_phrase(<<?CR, ?LF, ?CR>> = Data, Phrase, MaxHeaderSize, Result, + Relaxed) -> + {?MODULE, parse_reason_phrase, [Data, Phrase, MaxHeaderSize, Result], + Relaxed}; + +parse_reason_phrase(<<?CR, ?LF>> = Data, Phrase, MaxHeaderSize, Result, + Relaxed) -> + {?MODULE, parse_reason_phrase, [Data, Phrase, MaxHeaderSize, Result, + Relaxed]}; +parse_reason_phrase(<<?LF, Rest/binary>>, Phrase, + MaxHeaderSize, Result, Relaxed) -> + %% If ?CR is is missing RFC2616 section-19.3 + parse_reason_phrase(<<?CR, ?LF, Rest/binary>>, Phrase, + MaxHeaderSize, Result, Relaxed); +parse_reason_phrase(<<?CR, ?LF, Rest/binary>>, Phrase, + MaxHeaderSize, Result, Relaxed) -> + parse_headers(Rest, [], [], MaxHeaderSize, + [lists:reverse(Phrase) | Result], Relaxed); +parse_reason_phrase(<<?LF>>, Phrase, MaxHeaderSize, Result, Relaxed) -> + %% If ?CR is is missing RFC2616 section-19.3 + parse_reason_phrase(<<?CR, ?LF>>, Phrase, MaxHeaderSize, Result, + Relaxed); +parse_reason_phrase(<<?CR>> = Data, Phrase, MaxHeaderSize, Result, Relaxed) -> + {?MODULE, parse_reason_phrase, + [Data, Phrase, MaxHeaderSize, Result, Relaxed]}; +parse_reason_phrase(<<Octet, Rest/binary>>, Phrase, MaxHeaderSize, Result, + Relaxed) -> + parse_reason_phrase(Rest, [Octet | Phrase], MaxHeaderSize, + Result, Relaxed). + +parse_headers(<<>>, Header, Headers, MaxHeaderSize, Result, Relaxed) -> + {?MODULE, parse_headers, [<<>>, Header, Headers, MaxHeaderSize, Result, + Relaxed]}; + +parse_headers(<<?CR,?LF,?LF,Body/binary>>, Header, Headers, + MaxHeaderSize, Result, Relaxed) -> + %% If ?CR is is missing RFC2616 section-19.3 + parse_headers(<<?CR,?LF,?CR,?LF,Body/binary>>, Header, Headers, + MaxHeaderSize, Result, Relaxed); + +parse_headers(<<?LF,?LF,Body/binary>>, Header, Headers, + MaxHeaderSize, Result, Relaxed) -> + %% If ?CR is is missing RFC2616 section-19.3 + parse_headers(<<?CR,?LF,?CR,?LF,Body/binary>>, Header, Headers, + MaxHeaderSize, Result, Relaxed); + +parse_headers(<<?CR,?LF,?CR,?LF,Body/binary>>, Header, Headers, + MaxHeaderSize, Result, _) -> + HTTPHeaders = [lists:reverse(Header) | Headers], + Length = lists:foldl(fun(H, Acc) -> length(H) + Acc end, + 0, HTTPHeaders), + case ((Length =< MaxHeaderSize) or (MaxHeaderSize == nolimit)) of + true -> + ResponseHeaderRcord = + http_response:headers(HTTPHeaders, #http_response_h{}), + {ok, list_to_tuple( + lists:reverse([Body, ResponseHeaderRcord | Result]))}; + false -> + throw({error, {header_too_long, MaxHeaderSize, + MaxHeaderSize-Length}}) + end; +parse_headers(<<?CR,?LF,?CR>> = Data, Header, Headers, + MaxHeaderSize, Result, Relaxed) -> + {?MODULE, parse_headers, [Data, Header, Headers, + MaxHeaderSize, Result, Relaxed]}; +parse_headers(<<?CR,?LF>> = Data, Header, Headers, + MaxHeaderSize, Result, Relaxed) -> + {?MODULE, parse_headers, [Data, Header, Headers, MaxHeaderSize, + Result, Relaxed]}; +parse_headers(<<?CR,?LF, Octet, Rest/binary>>, Header, Headers, + MaxHeaderSize, Result, Relaxed) -> + parse_headers(Rest, [Octet], + [lists:reverse(Header) | Headers], MaxHeaderSize, + Result, Relaxed); +parse_headers(<<?CR>> = Data, Header, Headers, + MaxHeaderSize, Result, Relaxed) -> + {?MODULE, parse_headers, [Data, Header, Headers, MaxHeaderSize, + Result, Relaxed]}; + +parse_headers(<<?LF>>, Header, Headers, + MaxHeaderSize, Result, Relaxed) -> + %% If ?CR is is missing RFC2616 section-19.3 + parse_headers(<<?CR, ?LF>>, Header, Headers, + MaxHeaderSize, Result, Relaxed); + +parse_headers(<<Octet, Rest/binary>>, Header, Headers, + MaxHeaderSize, Result, Relaxed) -> + parse_headers(Rest, [Octet | Header], Headers, MaxHeaderSize, + Result, Relaxed). + + +%% RFC2616, Section 10.1.1 +%% Note: +%% - Only act on the 100 status if the request included the +%% "Expect:100-continue" header, otherwise just ignore this response. +status_continue(_, #request{headers = + #http_request_h{expect = "100-continue"}}) -> + continue; + +status_continue({_,_, Data}, _) -> + %% The data in the body in this case is actually part of the real + %% response sent after the "fake" 100-continue. + {ignore, Data}. + +status_service_unavailable(Response = {_, Headers, _}, Request) -> + case Headers#http_response_h.'retry-after' of + undefined -> + status_server_error_50x(Response, Request); + Time when (length(Time) < 3) -> % Wait only 99 s or less + NewTime = list_to_integer(Time) * 100, % time in ms + {_, Data} = format_response(Response), + {retry, {NewTime, Request}, Data}; + _ -> + status_server_error_50x(Response, Request) + end. + +status_server_error_50x(Response, Request) -> + {Msg, _} = format_response(Response), + {stop, {Request#request.id, Msg}}. + + +redirect(Response = {StatusLine, Headers, Body}, Request) -> + {_, Data} = format_response(Response), + case Headers#http_response_h.location of + undefined -> + transparent(Response, Request); + RedirUrl -> + case http_uri:parse(RedirUrl) of + {error, no_scheme} when + (Request#request.settings)#http_options.relaxed -> + NewLocation = fix_relative_uri(Request, RedirUrl), + redirect({StatusLine, Headers#http_response_h{ + location = NewLocation}, + Body}, Request); + {error, Reason} -> + {ok, error(Request, Reason), Data}; + %% Automatic redirection + {Scheme, _, Host, Port, Path, Query} -> + NewHeaders = + (Request#request.headers)#http_request_h{host = + Host}, + NewRequest = + Request#request{redircount = + Request#request.redircount+1, + scheme = Scheme, + headers = NewHeaders, + address = {Host,Port}, + path = Path, + pquery = Query, + abs_uri = + atom_to_list(Scheme) ++ "://" ++ + Host ++ ":" ++ + integer_to_list(Port) ++ + Path ++ Query}, + {redirect, NewRequest, Data} + end + end. + +maybe_to_list(Port) when is_integer(Port) -> + integer_to_list(Port); +maybe_to_list(Port) when is_list(Port) -> + Port. + +%%% Guessing that we received a relative URI, fix it to become an absoluteURI +fix_relative_uri(Request, RedirUrl) -> + {Server, Port0} = Request#request.address, + Port = maybe_to_list(Port0), + Path = Request#request.path, + atom_to_list(Request#request.scheme) ++ "://" ++ Server ++ ":" ++ Port + ++ Path ++ RedirUrl. + +error(#request{id = Id}, Reason) -> + {Id, {error, Reason}}. + +transparent(Response, Request) -> + {Msg, Data} = format_response(Response), + {ok, {Request#request.id, Msg}, Data}. + +stream_start(Headers, Request, ignore) -> + {Request#request.id, stream_start, http_response:header_list(Headers)}; + +stream_start(Headers, Request, Pid) -> + {Request#request.id, stream_start, + http_response:header_list(Headers), Pid}. + +stream_end(Response, Request = #request{stream = Self}) + when (Self =:= self) orelse (Self =:= {self, once}) -> + {{_, Headers, _}, Data} = format_response(Response), + {ok, {Request#request.id, stream_end, Headers}, Data}; + +stream_end(Response, Request) -> + {_, Data} = format_response(Response), + {ok, {Request#request.id, saved_to_file}, Data}. + +is_server_closing(Headers) when is_record(Headers, http_response_h) -> + case Headers#http_response_h.connection of + "close" -> + true; + _ -> + false + end. + +format_response({{"HTTP/0.9", _, _} = StatusLine, _, Body}) -> + {{StatusLine, [], Body}, <<>>}; +format_response({StatusLine, Headers, Body = <<>>}) -> + {{StatusLine, http_response:header_list(Headers), Body}, <<>>}; + +format_response({StatusLine, Headers, Body}) -> + Length = list_to_integer(Headers#http_response_h.'content-length'), + {NewBody, Data} = + case Length of + 0 -> + {Body, <<>>}; + -1 -> % When no lenght indicator is provided + {Body, <<>>}; + Length when (Length =< size(Body)) -> + <<BodyThisReq:Length/binary, Next/binary>> = Body, + {BodyThisReq, Next}; + _ -> %% Connection prematurely ended. + {Body, <<>>} + end, + {{StatusLine, http_response:header_list(Headers), NewBody}, Data}. + diff --git a/lib/inets/src/http_client/httpc_sup.erl b/lib/inets/src/http_client/httpc_sup.erl new file mode 100644 index 0000000000..152a57d32d --- /dev/null +++ b/lib/inets/src/http_client/httpc_sup.erl @@ -0,0 +1,75 @@ +%% +%% %CopyrightBegin% +%% +%% Copyright Ericsson AB 2004-2009. All Rights Reserved. +%% +%% The contents of this file are subject to the Erlang Public License, +%% Version 1.1, (the "License"); you may not use this file except in +%% compliance with the License. You should have received a copy of the +%% Erlang Public License along with this software. If not, it can be +%% retrieved online at http://www.erlang.org/. +%% +%% Software distributed under the License is distributed on an "AS IS" +%% basis, WITHOUT WARRANTY OF ANY KIND, either express or implied. See +%% the License for the specific language governing rights and limitations +%% under the License. +%% +%% %CopyrightEnd% +%% +%% +%%---------------------------------------------------------------------- +%% Purpose: The top supervisor for the http client hangs under +%% inets_sup. +%%---------------------------------------------------------------------- + +-module(httpc_sup). + +-behaviour(supervisor). + +%% API +-export([start_link/1]). + +%% Supervisor callback +-export([init/1]). + +%%%========================================================================= +%%% API +%%%========================================================================= +start_link(HttpcServices) -> + supervisor:start_link({local, ?MODULE}, ?MODULE, [HttpcServices]). + +%%%========================================================================= +%%% Supervisor callback +%%%========================================================================= +init([HttpcServices]) -> + RestartStrategy = one_for_one, + MaxR = 10, + MaxT = 3600, + Children = child_specs(HttpcServices), + {ok, {{RestartStrategy, MaxR, MaxT}, Children}}. + +%%%========================================================================= +%%% Internal functions +%%%========================================================================= +child_specs(HttpcServices) -> + [httpc_profile_sup(HttpcServices), httpc_handler_sup()]. + +httpc_profile_sup(HttpcServices) -> + Name = httpc_profile_sup, + StartFunc = {httpc_profile_sup, start_link, [HttpcServices]}, + Restart = permanent, + Shutdown = infinity, + Modules = [httpc_profile_sup], + Type = supervisor, + {Name, StartFunc, Restart, Shutdown, Type, Modules}. + +httpc_handler_sup() -> + Name = httpc_handler_sup, + StartFunc = {httpc_handler_sup, start_link, []}, + Restart = permanent, + Shutdown = infinity, + Modules = [httpc_handler_sup], + Type = supervisor, + {Name, StartFunc, Restart, Shutdown, Type, Modules}. + + diff --git a/lib/inets/src/http_lib/Makefile b/lib/inets/src/http_lib/Makefile new file mode 100644 index 0000000000..27e7ee65c5 --- /dev/null +++ b/lib/inets/src/http_lib/Makefile @@ -0,0 +1,101 @@ +# +# %CopyrightBegin% +# +# Copyright Ericsson AB 2005-2009. All Rights Reserved. +# +# The contents of this file are subject to the Erlang Public License, +# Version 1.1, (the "License"); you may not use this file except in +# compliance with the License. You should have received a copy of the +# Erlang Public License along with this software. If not, it can be +# retrieved online at http://www.erlang.org/. +# +# Software distributed under the License is distributed on an "AS IS" +# basis, WITHOUT WARRANTY OF ANY KIND, either express or implied. See +# the License for the specific language governing rights and limitations +# under the License. +# +# %CopyrightEnd% +# +# + +include $(ERL_TOP)/make/target.mk +EBIN = ../../ebin +include $(ERL_TOP)/make/$(TARGET)/otp.mk + +# ---------------------------------------------------- +# Application version +# ---------------------------------------------------- +include ../../vsn.mk + +VSN = $(INETS_VSN) + +# ---------------------------------------------------- +# Release directory specification +# ---------------------------------------------------- +RELSYSDIR = $(RELEASE_PATH)/lib/inets-$(VSN) + +# ---------------------------------------------------- +# Target Specs +# ---------------------------------------------------- +MODULES = \ + http_chunk \ + http_transport\ + http_util \ + http_request \ + http_response + +HRL_FILES = http_internal.hrl + +ERL_FILES = $(MODULES:%=%.erl) + +TARGET_FILES= $(MODULES:%=$(EBIN)/%.$(EMULATOR)) + +# ---------------------------------------------------- +# INETS FLAGS +# ---------------------------------------------------- +INETS_FLAGS = -D'SERVER_SOFTWARE="inets/$(VSN)"' \ + +# ---------------------------------------------------- +# FLAGS +# ---------------------------------------------------- +INETS_ERL_FLAGS += -I ../inets_app + +ifeq ($(WARN_UNUSED_WARS),true) +ERL_COMPILE_FLAGS += +warn_unused_vars +endif + +ERL_COMPILE_FLAGS += $(INETS_ERL_FLAGS) \ + $(INETS_FLAGS) \ + +'{parse_transform,sys_pre_attributes}' \ + +'{attribute,insert,app_vsn,$(APP_VSN)}' + + +# ---------------------------------------------------- +# Targets +# ---------------------------------------------------- + +debug opt: $(TARGET_FILES) + +clean: + rm -f $(TARGET_FILES) + rm -f core + +docs: + +# ---------------------------------------------------- +# Release Target +# ---------------------------------------------------- +include $(ERL_TOP)/make/otp_release_targets.mk + +release_spec: opt + $(INSTALL_DIR) $(RELSYSDIR)/src + $(INSTALL_DATA) $(HRL_FILES) $(ERL_FILES) $(RELSYSDIR)/src + $(INSTALL_DIR) $(RELSYSDIR)/ebin + $(INSTALL_DATA) $(TARGET_FILES) $(RELSYSDIR)/ebin + +release_docs_spec: + +info: + @echo "INETS_DEBUG = $(INETS_DEBUG)" + @echo "INETS_FLAGS = $(INETS_FLAGS)" + @echo "ERL_COMPILE_FLAGS = $(ERL_COMPILE_FLAGS)" diff --git a/lib/inets/src/http_lib/http_chunk.erl b/lib/inets/src/http_lib/http_chunk.erl new file mode 100644 index 0000000000..cd20dce9d5 --- /dev/null +++ b/lib/inets/src/http_lib/http_chunk.erl @@ -0,0 +1,291 @@ +%% +%% %CopyrightBegin% +%% +%% Copyright Ericsson AB 2004-2009. All Rights Reserved. +%% +%% The contents of this file are subject to the Erlang Public License, +%% Version 1.1, (the "License"); you may not use this file except in +%% compliance with the License. You should have received a copy of the +%% Erlang Public License along with this software. If not, it can be +%% retrieved online at http://www.erlang.org/. +%% +%% Software distributed under the License is distributed on an "AS IS" +%% basis, WITHOUT WARRANTY OF ANY KIND, either express or implied. See +%% the License for the specific language governing rights and limitations +%% under the License. +%% +%% %CopyrightEnd% +%% +%% Description: Implements chunked transfer encoding see RFC2616 section +%% 3.6.1 +-module(http_chunk). + +-include("http_internal.hrl"). + +%% API +-export([decode/3, decode/4, encode/1, encode_last/0, handle_headers/2]). +%% Callback API - used for example if the chunkedbody is received a +%% little at a time on a socket. +-export([decode_size/1, ignore_extensions/1, decode_data/1, decode_trailer/1]). + +%%%========================================================================= +%%% API +%%%========================================================================= +%%------------------------------------------------------------------------- +%% decode(ChunkedBody, MaxBodySize, MaxHeaderSize, <Stream>) -> +%% {ok, {Headers, Body}} | {Module, Function, Args} +%% +%% Headers = ["Header:Value"] +%% ChunkedBody = binary() +%% MaxBodySize = integer() +%% MaxHeaderSize = integer() +%% Stream = {Code, Request} - if Request#request.stream =/= none +%% and Code == 200 the side effect of sending each decode chunk to the +%% client/file before the whole body is received will take place. +%% +%% Note: decode/4 should only be used from httpc_handler module. +%% Otherwhise use the side effect free decode/3. +%% +%% Description: Decodes a body encoded by the chunked transfer +%% encoding. If the ChunkedBody is not compleate it returns {Module, +%% Function, Args} so that decoding can be continued when more of the +%% data has been received by calling Module:Function([NewData | Args]). +%% +%% Note: In the case of pipelining a call to decode might contain data +%% that belongs to the next request/response and will be returned as +%% part of the body, hence functions calling http_chunk:decode must +%% look at the returned content-length header to make sure that they +%% split the actual body and data that possible should be passed along to +%% the next pass in the loop. +%%------------------------------------------------------------------------- +decode(ChunkedBody, MaxBodySize, MaxHeaderSize) -> + decode(ChunkedBody, MaxBodySize, MaxHeaderSize, false). + +decode(ChunkedBody, MaxBodySize, MaxHeaderSize, Stream) -> + %% Note decode_size will call decode_data. + decode_size([ChunkedBody, <<>>, [], + {MaxBodySize, <<>>, 0, MaxHeaderSize, Stream}]). + +%%------------------------------------------------------------------------- +%% encode(Chunk) -> EncodedChunk +%% +%% Chunked = binary() +%% EncodedChunk = binary() +%% +%% Description: Encodes a body part with the chunked transfer encoding. +%% Chunks are returned as lists or binaries depending on the +%% input format. When sending the data on the both formats +%% are accepted. +%%------------------------------------------------------------------------- +encode(Chunk) when is_binary(Chunk)-> + HEXSize = list_to_binary(http_util:integer_to_hexlist(size(Chunk))), + <<HEXSize/binary, ?CR, ?LF, Chunk/binary, ?CR, ?LF>>; + +encode(Chunk) when is_list(Chunk)-> + HEXSize = http_util:integer_to_hexlist(erlang:iolist_size(Chunk)), + [HEXSize, ?CR, ?LF, Chunk, ?CR, ?LF]. + +encode_last() -> + <<$0, ?CR, ?LF, ?CR, ?LF >>. + +%%------------------------------------------------------------------------- +%% handle_headers(HeaderRecord, ChunkedHeaders) -> NewHeaderRecord +%% +%% HeaderRecord = NewHeaderRecord = #http_request_h{} | #http_response_h{} +%% ChunkedHeaders = ["Header:Value"] as returnde by http_chunk:decode/3 +%% +%% Description: Removes chunked from the header as we now have decode +%% the body and adds a content-length header and any other headers +%% found in the chunked trail. +%%------------------------------------------------------------------------- +handle_headers(RequestHeaderRecord = #http_request_h{}, ChunkedHeaders) -> + NewHeaders = http_request:headers(ChunkedHeaders, RequestHeaderRecord), + TransferEncoding = + case NewHeaders#http_request_h.'transfer-encoding' -- "chunked" of + "" -> + undefined; + Other -> + Other + end, + NewHeaders#http_request_h{'transfer-encoding' = TransferEncoding}; + +handle_headers(ResponseHeaderRecord = #http_response_h{}, ChunkedHeaders) -> + NewHeaders = http_response:headers(ChunkedHeaders, ResponseHeaderRecord), + TransferEncoding = + case NewHeaders#http_response_h.'transfer-encoding' -- "chunked" of + "" -> + undefined; + Other -> + Other + end, + NewHeaders#http_response_h{'transfer-encoding' = TransferEncoding}. + +%% Functions that may be returned during the decoding process +%% if the input data is incompleate. +decode_size([Bin, Rest, HexList, Info]) -> + decode_size(<<Rest/binary, Bin/binary>>, HexList, Info). + +ignore_extensions([Bin, Rest, NextFunction]) -> + ignore_extensions(<<Rest/binary, Bin/binary>>, NextFunction). + +decode_data([Bin, ChunkSize, TotalChunk, Info]) -> + decode_data(ChunkSize, <<TotalChunk/binary, Bin/binary>>, Info). + +decode_trailer([Bin, Rest, Header, Headers, MaxHeaderSize, Body, + BodyLength]) -> + decode_trailer(<<Rest/binary, Bin/binary>>, + Header, Headers, MaxHeaderSize, Body, BodyLength). + +%%%======================================================================== +%%% Internal functions +%%%======================================================================== +decode_size(<<>>, HexList, Info) -> + {?MODULE, decode_size, [<<>>, HexList, Info]}; +decode_size(Data = <<?CR, ?LF, ChunkRest/binary>>, HexList, + {MaxBodySize, Body, + AccLength, + MaxHeaderSize, Stream}) -> + ChunkSize = http_util:hexlist_to_integer(lists:reverse(HexList)), + case ChunkSize of + 0 -> % Last chunk, there was no data + ignore_extensions(Data, {?MODULE, decode_trailer, + [<<>>, [],[], MaxHeaderSize, + Body, + integer_to_list(AccLength)]}); + _ -> + %% Note decode_data may call decode_size again if there + %% is more than one chunk, hence here is where the last parameter + %% to this function comes in. + decode_data(ChunkSize, ChunkRest, {MaxBodySize, Body, + ChunkSize + AccLength , + MaxHeaderSize, Stream}) + end; +decode_size(<<";", Rest/binary>>, HexList, Info) -> + %% Note ignore_extensions will call decode_size/1 again when + %% it ignored all extensions. + ignore_extensions(Rest, {?MODULE, decode_size, [<<>>, HexList, Info]}); +decode_size(<<?CR>> = Data, HexList, Info) -> + {?MODULE, decode_size, [Data, HexList, Info]}; +decode_size(<<Octet, Rest/binary>>, HexList, Info) -> + decode_size(Rest, [Octet | HexList], Info). + +%% "All applications MUST ignore chunk-extension extensions they +%% do not understand.", see RFC 2616 Section 3.6.1 We don't +%% understand any extension... +ignore_extensions(<<>>, NextFunction) -> + {?MODULE, ignore_extensions, [<<>>, NextFunction]}; +ignore_extensions(Data = <<?CR, ?LF, _ChunkRest/binary>>, + {Module, Function, Args}) -> + Module:Function([Data | Args]); +ignore_extensions(<<?CR>> = Data, NextFunction) -> + {?MODULE, ignore_extensions, [Data, NextFunction]}; +ignore_extensions(<<_Octet, Rest/binary>>, NextFunction) -> + ignore_extensions(Rest, NextFunction). + +decode_data(ChunkSize, TotalChunk, + Info = {MaxBodySize, BodySoFar, AccLength, MaxHeaderSize, Stream}) + when ChunkSize =< size(TotalChunk) -> + case TotalChunk of + %% Potential last chunk + <<_:ChunkSize/binary, ?CR, ?LF, "0">> -> + {?MODULE, decode_data, [ChunkSize, TotalChunk, Info]}; + <<_:ChunkSize/binary, ?CR, ?LF, "0", ?CR>> -> + {?MODULE, decode_data, [ChunkSize, TotalChunk, Info]}; + <<_:ChunkSize/binary, ?CR, ?LF>> -> + {?MODULE, decode_data, [ChunkSize, TotalChunk, Info]}; + %% Last chunk + <<Data:ChunkSize/binary, ?CR, ?LF, "0", ";">> -> + %% Note ignore_extensions will call decode_trailer/1 + %% once it ignored all extensions. + {NewBody, _} = + stream(<<BodySoFar/binary, Data/binary>>, Stream), + {?MODULE, ignore_extensions, + [<<>>, + {?MODULE, decode_trailer, [<<>>, [],[], MaxHeaderSize, + NewBody, + integer_to_list(AccLength)]}]}; + <<Data:ChunkSize/binary, ?CR, ?LF, "0", ";", Rest/binary>> -> + %% Note ignore_extensions will call decode_trailer/1 + %% once it ignored all extensions. + {NewBody, _} = stream(<<BodySoFar/binary, Data/binary>>, Stream), + ignore_extensions(Rest, {?MODULE, decode_trailer, + [<<>>, [],[], MaxHeaderSize, + NewBody, + integer_to_list(AccLength)]}); + <<Data:ChunkSize/binary, ?CR, ?LF, "0", ?CR, ?LF>> -> + {NewBody, _} = stream(<<BodySoFar/binary, Data/binary>>, Stream), + {?MODULE, decode_trailer, [<<?CR, ?LF>>, [],[], MaxHeaderSize, + NewBody, + integer_to_list(AccLength)]}; + <<Data:ChunkSize/binary, ?CR, ?LF, "0", ?CR, ?LF, Rest/binary>> -> + {NewBody,_}= stream(<<BodySoFar/binary, Data/binary>>, Stream), + decode_trailer(<<?CR, ?LF, Rest/binary>>, [],[], MaxHeaderSize, + NewBody, + integer_to_list(AccLength)); + %% There are more chunks, so here we go agin... + <<Data:ChunkSize/binary, ?CR, ?LF, Rest/binary>> + when (AccLength < MaxBodySize) or (MaxBodySize == nolimit) -> + {NewBody, NewStream} = + stream(<<BodySoFar/binary, Data/binary>>, Stream), + decode_size(Rest, [], + {MaxBodySize, NewBody, + AccLength, MaxHeaderSize, NewStream}); + <<_:ChunkSize/binary, ?CR, ?LF, _/binary>> -> + throw({error, body_too_big}); + _ -> + {?MODULE, decode_data, [ChunkSize, TotalChunk, Info]} + end; +decode_data(ChunkSize, TotalChunk, Info) -> + {?MODULE, decode_data, [ChunkSize, TotalChunk, Info]}. + +decode_trailer(<<>>, Header, Headers, MaxHeaderSize, Body, BodyLength) -> + {?MODULE, decode_trailer, [<<>>, Header, Headers, MaxHeaderSize, Body, + BodyLength]}; + +%% Note: If Bin is not empty it is part of a pipelined request/response. +decode_trailer(<<?CR,?LF,?CR,?LF, Bin/binary>>, [], [], _, Body, BodyLength) -> + {ok, {["content-length:" ++ BodyLength], <<Body/binary, Bin/binary>>}}; +decode_trailer(<<?CR,?LF,?CR,?LF, Bin/binary>>, + Header, Headers, MaxHeaderSize, Body, BodyLength) -> + NewHeaders = case Header of + [] -> + Headers; + _ -> + [lists:reverse(Header) | Headers] + end, + Length = length(NewHeaders), + case Length > MaxHeaderSize of + true -> + throw({error, {header_too_long, MaxHeaderSize, + MaxHeaderSize-Length}}); + false -> + {ok, {["content-length:" ++ BodyLength | NewHeaders], + <<Body/binary, Bin/binary>>}} + end; +decode_trailer(<<?CR,?LF,?CR>> = Data, Header, Headers, MaxHeaderSize, + Body, BodyLength) -> + {?MODULE, decode_trailer, [Data, Header, Headers, MaxHeaderSize, Body, + BodyLength]}; +decode_trailer(<<?CR,?LF>> = Data, Header, Headers, MaxHeaderSize, + Body, BodyLength) -> + {?MODULE, decode_trailer, [Data, Header, Headers, MaxHeaderSize, Body, + BodyLength]}; +decode_trailer(<<?CR>> = Data, Header, Headers, MaxHeaderSize, + Body, BodyLength) -> + {?MODULE, decode_trailer, [Data, Header, Headers, MaxHeaderSize, Body, + BodyLength]}; +decode_trailer(<<?CR, ?LF, Rest/binary>>, Header, Headers, + MaxHeaderSize, Body, BodyLength) -> + decode_trailer(Rest, [], [lists:reverse(Header) | Headers], + MaxHeaderSize, Body, BodyLength); + +decode_trailer(<<Octet, Rest/binary>>, Header, Headers, MaxHeaderSize, Body, + BodyLength) -> + decode_trailer(Rest, [Octet | Header], Headers, MaxHeaderSize, + Body, BodyLength). + +stream(BodyPart, false) -> + {BodyPart, false}; +stream(BodyPart, {Code, Request}) -> + {NewBody, NewRequest} = httpc_handler:stream(BodyPart, Request, Code), + {NewBody, {Code, NewRequest}}. diff --git a/lib/inets/src/http_lib/http_internal.hrl b/lib/inets/src/http_lib/http_internal.hrl new file mode 100644 index 0000000000..bb2e831727 --- /dev/null +++ b/lib/inets/src/http_lib/http_internal.hrl @@ -0,0 +1,108 @@ +%% +%% %CopyrightBegin% +%% +%% Copyright Ericsson AB 2002-2009. All Rights Reserved. +%% +%% The contents of this file are subject to the Erlang Public License, +%% Version 1.1, (the "License"); you may not use this file except in +%% compliance with the License. You should have received a copy of the +%% Erlang Public License along with this software. If not, it can be +%% retrieved online at http://www.erlang.org/. +%% +%% Software distributed under the License is distributed on an "AS IS" +%% basis, WITHOUT WARRANTY OF ANY KIND, either express or implied. See +%% the License for the specific language governing rights and limitations +%% under the License. +%% +%% %CopyrightEnd% +%% +%% + +-include("inets_internal.hrl"). + +-define(HTTP_MAX_BODY_SIZE, nolimit). +-define(HTTP_MAX_HEADER_SIZE, 10240). +-define(HTTP_MAX_URI_SIZE, nolimit). + +%%% Response headers +-record(http_response_h,{ +%%% --- Standard "General" headers + 'cache-control', + connection, + date, + pragma, + trailer, + 'transfer-encoding', + upgrade, + via, + warning, +%%% --- Standard "Response" headers + 'accept-ranges', + age, + etag, + location, + 'proxy-authenticate', + 'retry-after', + server, + vary, + 'www-authenticate', +%%% --- Standard "Entity" headers + allow, + 'content-encoding', + 'content-language', + 'content-length' = "-1", + 'content-location', + 'content-md5', + 'content-range', + 'content-type', + expires, + 'last-modified', + other=[] % list() - Key/Value list with other headers + }). + + +%%% Request headers +-record(http_request_h,{ +%%% --- Standard "General" headers + 'cache-control', + connection = "keep-alive", + date, + pragma, + trailer, + 'transfer-encoding', + upgrade, + via, + warning, +%%% --- Standard "Request" headers + accept, + 'accept-charset', + 'accept-encoding', + 'accept-language', + authorization, + expect, + from, + host, + 'if-match', + 'if-modified-since', + 'if-none-match', + 'if-range', + 'if-unmodified-since', + 'max-forwards', + 'proxy-authorization', + range, + referer, + te, + 'user-agent', +%%% --- Standard "Entity" headers + allow, + 'content-encoding', + 'content-language', + 'content-length' = "0", + 'content-location', + 'content-md5', + 'content-range', + 'content-type', + expires, + 'last-modified', + other=[] % list() - Key/Value list with other headers + }). diff --git a/lib/inets/src/http_lib/http_request.erl b/lib/inets/src/http_lib/http_request.erl new file mode 100644 index 0000000000..c214aca4a4 --- /dev/null +++ b/lib/inets/src/http_lib/http_request.erl @@ -0,0 +1,281 @@ +%% +%% %CopyrightBegin% +%% +%% Copyright Ericsson AB 2005-2009. All Rights Reserved. +%% +%% The contents of this file are subject to the Erlang Public License, +%% Version 1.1, (the "License"); you may not use this file except in +%% compliance with the License. You should have received a copy of the +%% Erlang Public License along with this software. If not, it can be +%% retrieved online at http://www.erlang.org/. +%% +%% Software distributed under the License is distributed on an "AS IS" +%% basis, WITHOUT WARRANTY OF ANY KIND, either express or implied. See +%% the License for the specific language governing rights and limitations +%% under the License. +%% +%% %CopyrightEnd% +%% + +-module(http_request). + +-include("http_internal.hrl"). + +-export([headers/2, http_headers/1, is_absolut_uri/1]). + +%%------------------------------------------------------------------------- +%% headers(HeaderList, #http_request_h{}) -> #http_request_h{} +%% HeaderList - ["HeaderField:Value"] +%% HeaderField - string() +%% Value - string() +%% +%% Description: Creates a http_request_h-record used internally to +%% handle http-headers. +%%------------------------------------------------------------------------- +headers([], Headers) -> + Headers; +headers([Header | Tail], Headers) -> + case lists:splitwith(fun($:) -> false; (_) -> true end, Header) of + {Key, [$: | Value]} -> + headers(Tail, headers(http_util:to_lower(string:strip(Key)), + string:strip(Value), Headers)); + {_, []} -> + Report = io_lib:format("Ignored invalid HTTP-header: ~p~n", + [Header]), + error_logger:error_report(Report), + headers(Tail, Headers) + end. + +%%------------------------------------------------------------------------- +%% headers(#http_request_h{}) -> HeaderList +%% HeaderList - ["HeaderField:Value"] +%% HeaderField - string() +%% Value - string() +%% +%% Description: Creates a HTTP header string. +%%------------------------------------------------------------------------- +http_headers(Headers = #http_request_h{other = Other}) -> + HeaderFields = record_info(fields, http_request_h) -- [other], + HeaderStr = lists:foldl(fun(Key, Acc) -> + case key_value_str(Key, Headers) of + undefined -> + Acc; + Str -> + [Str | Acc] + end + end, + [], HeaderFields), + + lists:flatten([HeaderStr | headers_other(Other, [])]). + +%%------------------------------------------------------------------------- +%% is_absolut_uri(URI) -> true | false +%% URI - string() +%% +%% Description: Checks if an URI is absolute or relative +%%------------------------------------------------------------------------- +is_absolut_uri("http://" ++ _) -> + true; +is_absolut_uri("https://" ++ _) -> + true; +is_absolut_uri(_) -> + false. + +%%%======================================================================== +%%% Internal functions +%%%======================================================================== + +%%% --- Request headers +headers("accept", Value, Headers) -> + Headers#http_request_h{accept = Value}; +headers("accept-charset", Value, Headers) -> + Headers#http_request_h{'accept-charset' = Value}; +headers("accept-encoding", Value, Headers) -> + Headers#http_request_h{'accept-encoding' = Value}; +headers("accept-language", Value, Headers) -> + Headers#http_request_h{'accept-language' = Value}; +headers("authorization", Value, Headers) -> + Headers#http_request_h{authorization = Value}; +headers("expect", Value, Headers) -> + Headers#http_request_h{expect = Value}; +headers("from", Value, Headers) -> + Headers#http_request_h{from = Value}; +headers("host", Value, Headers) -> + Headers#http_request_h{host = Value}; +headers("if-match", Value, Headers) -> + Headers#http_request_h{'if-match' = Value}; +headers("if-modified-since", Value, Headers) -> + Headers#http_request_h{'if-modified-since' = Value}; +headers("if-none-match", Value, Headers) -> + Headers#http_request_h{'if-none-match' = Value}; +headers("if-range", Value, Headers) -> + Headers#http_request_h{'if-range' = Value}; +headers("if-unmodified-since", Value, Headers) -> + Headers#http_request_h{'if-unmodified-since' = Value}; +headers("max-forwards", Value, Headers) -> + Headers#http_request_h{'max-forwards' = Value}; +headers("proxy-authorization", Value, Headers) -> + Headers#http_request_h{'proxy-authorization' = Value}; +headers("range", Value, Headers) -> + Headers#http_request_h{range = Value}; +headers("referer", Value, Headers) -> + Headers#http_request_h{referer = Value}; +headers("te", Value, Headers) -> + Headers#http_request_h{te = Value}; +headers("user-agent", Value, Headers) -> + Headers#http_request_h{'user-agent' = Value}; + +%% General-Headers +headers("cache-control", Value, Headers) -> + Headers#http_request_h{'cache-control' = Value}; +headers("connection", Value, Headers) -> + Headers#http_request_h{connection = Value}; +headers("date", Value, Headers) -> + Headers#http_request_h{date = Value}; +headers("pragma", Value, Headers) -> + Headers#http_request_h{pragma = Value}; +headers("trailer", Value, Headers) -> + Headers#http_request_h{trailer = Value}; +headers("transfer-encoding", Value, Headers) -> + Headers#http_request_h{'transfer-encoding' = Value}; +headers("upgrade", Value, Headers) -> + Headers#http_request_h{upgrade = Value}; +headers("via", Value, Headers) -> + Headers#http_request_h{via = Value}; +headers("warning", Value, Headers) -> + Headers#http_request_h{warning = Value}; + +%% Entity header +headers("allow", Value, Headers) -> + Headers#http_request_h{allow = Value}; +headers("content-encoding", Value, Headers) -> + Headers#http_request_h{'content-encoding' = Value}; +headers("content-language", Value, Headers) -> + Headers#http_request_h{'content-language' = Value}; +headers("content-length", Value, Headers) -> + Headers#http_request_h{'content-length' = Value}; +headers("content-location", Value, Headers) -> + Headers#http_request_h{'content-location' = Value}; +headers("content-md5", Value, Headers) -> + Headers#http_request_h{'content-md5' = Value}; +headers("content-range", Value, Headers) -> + Headers#http_request_h{'content-range' = Value}; +headers("content-type", Value, Headers) -> + Headers#http_request_h{'content-type' = Value}; +headers("expires", Value, Headers) -> + Headers#http_request_h{expires = Value}; +headers("last-modified", Value, Headers) -> + Headers#http_request_h{'last-modified' = Value}; +headers(Key, Value, Headers) -> + Headers#http_request_h{other= + [{Key, Value} | Headers#http_request_h.other]}. + +key_value_str(Key = 'cache-control', Headers) -> + key_value_str(atom_to_list(Key), Headers#http_request_h.'cache-control'); +key_value_str(Key = connection, Headers) -> + key_value_str(atom_to_list(Key), Headers#http_request_h.connection); +key_value_str(Key = date, Headers) -> + key_value_str(atom_to_list(Key), Headers#http_request_h.date); +key_value_str(Key = pragma, Headers) -> + key_value_str(atom_to_list(Key), Headers#http_request_h.pragma); +key_value_str(Key = trailer, Headers) -> + key_value_str(atom_to_list(Key), Headers#http_request_h.trailer); +key_value_str(Key = 'transfer-encoding', Headers) -> + key_value_str(atom_to_list(Key), + Headers#http_request_h.'transfer-encoding'); +key_value_str(Key = upgrade, Headers) -> + key_value_str(atom_to_list(Key), Headers#http_request_h.upgrade); +key_value_str(Key = via, Headers) -> + key_value_str(atom_to_list(Key), Headers#http_request_h.via); +key_value_str(Key = warning, Headers) -> + key_value_str(atom_to_list(Key), Headers#http_request_h.warning); +key_value_str(Key = accept, Headers) -> + key_value_str(atom_to_list(Key), Headers#http_request_h.accept); +key_value_str(Key = 'accept-charset', Headers) -> + key_value_str(atom_to_list(Key), Headers#http_request_h.'accept-charset'); +key_value_str(Key = 'accept-encoding', Headers) -> + key_value_str(atom_to_list(Key), Headers#http_request_h.'accept-encoding'); +key_value_str(Key = 'accept-language', Headers) -> + key_value_str(atom_to_list(Key), Headers#http_request_h.'accept-language'); +key_value_str(Key = authorization, Headers) -> + key_value_str(atom_to_list(Key), + Headers#http_request_h.authorization); +key_value_str(Key = expect, Headers) -> + key_value_str(atom_to_list(Key), Headers#http_request_h.expect); +key_value_str(Key = from, Headers) -> + key_value_str(atom_to_list(Key), Headers#http_request_h.from); +key_value_str(Key = host, Headers) -> + key_value_str(atom_to_list(Key), Headers#http_request_h.host); +key_value_str(Key = 'if-match', Headers) -> + key_value_str(atom_to_list(Key), + Headers#http_request_h.'if-match'); +key_value_str(Key = 'if-modified-since', Headers) -> + key_value_str(atom_to_list(Key), + Headers#http_request_h.'if-modified-since'); +key_value_str(Key = 'if-none-match', Headers) -> + key_value_str(atom_to_list(Key), + Headers#http_request_h.'if-none-match'); +key_value_str(Key = 'if-range', Headers) -> + key_value_str(atom_to_list(Key), + Headers#http_request_h.'if-range'); +key_value_str(Key = 'if-unmodified-since', Headers) -> + key_value_str(atom_to_list(Key), + Headers#http_request_h.'if-unmodified-since'); +key_value_str(Key = 'max-forwards', Headers) -> + key_value_str(atom_to_list(Key), + Headers#http_request_h.'max-forwards'); +key_value_str(Key = 'proxy-authorization', Headers) -> + key_value_str(atom_to_list(Key), + Headers#http_request_h.'proxy-authorization'); +key_value_str(Key = range, Headers) -> + key_value_str(atom_to_list(Key), + Headers#http_request_h.range); +key_value_str(Key = referer, Headers) -> + key_value_str(atom_to_list(Key), + Headers#http_request_h.referer); +key_value_str(Key = te, Headers) -> + key_value_str(atom_to_list(Key), + Headers#http_request_h.te); +key_value_str(Key = 'user-agent', Headers) -> + key_value_str(atom_to_list(Key), + Headers#http_request_h.'user-agent'); +key_value_str(Key = allow, Headers) -> + key_value_str(atom_to_list(Key), Headers#http_request_h.allow); +key_value_str(Key = 'content-encoding', Headers) -> + key_value_str(atom_to_list(Key), + Headers#http_request_h.'content-encoding'); +key_value_str(Key = 'content-language', Headers) -> + key_value_str(atom_to_list(Key), + Headers#http_request_h.'content-language'); +key_value_str(Key = 'content-length', Headers) -> + case Headers#http_request_h.'content-length' of + "0" -> + undefined; + _ -> + key_value_str(atom_to_list(Key), + Headers#http_request_h.'content-length') + end; +key_value_str(Key = 'content-location', Headers) -> + key_value_str(atom_to_list(Key), + Headers#http_request_h.'content-location'); +key_value_str(Key = 'content-md5', Headers) -> + key_value_str(atom_to_list(Key), + Headers#http_request_h.'content-md5'); +key_value_str(Key = 'content-range', Headers) -> + key_value_str(atom_to_list(Key), Headers#http_request_h.'content-range'); +key_value_str(Key = 'content-type', Headers) -> + key_value_str(atom_to_list(Key), Headers#http_request_h.'content-type'); +key_value_str(Key = expires, Headers) -> + key_value_str(atom_to_list(Key), Headers#http_request_h.expires); +key_value_str(Key = 'last-modified', Headers) -> + key_value_str(atom_to_list(Key), Headers#http_request_h.'last-modified'); +key_value_str(_, undefined) -> + undefined; +key_value_str(Key, Value) -> + Key ++ ": " ++ Value ++ ?CRLF. + +headers_other([], Headers) -> + Headers; +headers_other([{Key,Value} | Rest], Headers) -> + Header = Key ++ ": " ++ Value ++ ?CRLF, + headers_other(Rest, [Header | Headers]). diff --git a/lib/inets/src/http_lib/http_response.erl b/lib/inets/src/http_lib/http_response.erl new file mode 100644 index 0000000000..b1e7f1e647 --- /dev/null +++ b/lib/inets/src/http_lib/http_response.erl @@ -0,0 +1,208 @@ +%% +%% %CopyrightBegin% +%% +%% Copyright Ericsson AB 2005-2009. All Rights Reserved. +%% +%% The contents of this file are subject to the Erlang Public License, +%% Version 1.1, (the "License"); you may not use this file except in +%% compliance with the License. You should have received a copy of the +%% Erlang Public License along with this software. If not, it can be +%% retrieved online at http://www.erlang.org/. +%% +%% Software distributed under the License is distributed on an "AS IS" +%% basis, WITHOUT WARRANTY OF ANY KIND, either express or implied. See +%% the License for the specific language governing rights and limitations +%% under the License. +%% +%% %CopyrightEnd% +%% + +-module(http_response). + +-include("http_internal.hrl"). + +-export([headers/2, header_list/1]). + +%%------------------------------------------------------------------------- +%% headers(HeaderList, #http_response_h{}) -> #http_response_h{} +%% HeaderList - ["HeaderField:Value"] +%% HeaderField - string() +%% Value - string() +%% +%% Description: Creates a http_response_h-record used internally to +%% handle http-headers. +%%------------------------------------------------------------------------- +headers([], Headers) -> + Headers; + +headers([Header | Tail], Headers) -> + {Key, [$: | Value]} = + lists:splitwith(fun($:) -> false; (_) -> true end, Header), + headers(Tail, headers(http_util:to_lower(string:strip(Key)), + string:strip(Value), Headers)). + +%%------------------------------------------------------------------------- +%% headers(#http_response_h{}) -> HeaderList +%% HeaderList - [{"HeaderField", Value"}] +%% HeaderField - string() +%% Value - string() +%% +%% Description: Creates a list of key value tuples from the #http_response_h +%% record, to be returned to the application programmer. We +%% do not wish to make the application programmer dependent on +%% our records. +%%------------------------------------------------------------------------- +header_list(Headers) -> + HeaderFields = record_info(fields, http_response_h) -- [other], + HeaderList = lists:foldl(fun(Key, Acc) -> + case key_value_tuple(Key, Headers) of + undefined -> + Acc; + Tuple -> + [Tuple | Acc] + end + end, + [], HeaderFields), + lists:reverse(HeaderList) ++ Headers#http_response_h.other. +%%%======================================================================== +%%% Internal functions +%%%======================================================================== +headers("cache-control", Value, Headers) -> + Headers#http_response_h{'cache-control'= Value}; +headers("connection", Value, Headers) -> + Headers#http_response_h{connection = Value}; +headers("date", Value, Headers) -> + Headers#http_response_h{date = Value}; +headers("pragma", Value, Headers) -> + Headers#http_response_h{pragma = Value}; +headers("trailer", Value, Headers) -> + Headers#http_response_h{trailer = Value}; +headers("transfer-encoding", Value, Headers) -> + Headers#http_response_h{'transfer-encoding' = Value}; +headers("upgrade", Value, Headers) -> + Headers#http_response_h{upgrade = Value}; +headers("via", Value, Headers) -> + Headers#http_response_h{via = Value}; +headers("warning", Value, Headers) -> + Headers#http_response_h{warning = Value}; +headers("accept-ranges", Value, Headers) -> + Headers#http_response_h{'accept-ranges' = Value}; +headers("age", Value, Headers) -> + Headers#http_response_h{age = Value}; +headers("etag", Value, Headers) -> + Headers#http_response_h{etag = Value}; +headers("location", Value, Headers) -> + Headers#http_response_h{location = Value}; +headers("proxy-authenticate", Value, Headers) -> + Headers#http_response_h{'proxy-authenticate' = Value}; +headers("retry-after", Value, Headers) -> + Headers#http_response_h{'retry-after' = Value}; +headers("server", Value, Headers) -> + Headers#http_response_h{server = Value}; +headers("vary", Value, Headers) -> + Headers#http_response_h{vary = Value}; +headers("www-authenticate", Value, Headers) -> + Headers#http_response_h{'www-authenticate' = Value}; +headers("allow", Value, Headers) -> + Headers#http_response_h{allow = Value}; +headers("content-encoding", Value, Headers) -> + Headers#http_response_h{'content-encoding' = Value}; +headers("content-language", Value, Headers) -> + Headers#http_response_h{'content-language' = Value}; +headers("content-length", Value, Headers) -> + Headers#http_response_h{'content-length' = Value}; +headers("content-location", Value, Headers) -> + Headers#http_response_h{'content-location' = Value}; +headers("content-md5", Value, Headers) -> + Headers#http_response_h{'content-md5' = Value}; +headers("content-range", Value, Headers) -> + Headers#http_response_h{'content-range' = Value}; +headers("content-type", Value, Headers) -> + Headers#http_response_h{'content-type' = Value}; +headers("expires", Value, Headers) -> + Headers#http_response_h{expires = Value}; +headers("last-modified", Value, Headers) -> + Headers#http_response_h{'last-modified' = Value}; +headers(Key, Value, Headers) -> + Headers#http_response_h{other= + [{Key, Value} | Headers#http_response_h.other]}. + + +key_value_tuple(Key = 'cache-control', Headers) -> + key_value_tuple(atom_to_list(Key), + Headers#http_response_h.'cache-control'); +key_value_tuple(Key = connection, Headers) -> + key_value_tuple(atom_to_list(Key), Headers#http_response_h.connection); +key_value_tuple(Key = date, Headers) -> + key_value_tuple(atom_to_list(Key), Headers#http_response_h.date); +key_value_tuple(Key = pragma, Headers) -> + key_value_tuple(atom_to_list(Key), Headers#http_response_h.pragma); +key_value_tuple(Key = trailer, Headers) -> + key_value_tuple(atom_to_list(Key), Headers#http_response_h.trailer); +key_value_tuple(Key ='transfer-encoding', Headers) -> + key_value_tuple(atom_to_list(Key), + Headers#http_response_h.'transfer-encoding'); +key_value_tuple(Key = upgrade, Headers) -> + key_value_tuple(atom_to_list(Key), Headers#http_response_h.upgrade); +key_value_tuple(Key = via, Headers) -> + key_value_tuple(atom_to_list(Key), Headers#http_response_h.via); +key_value_tuple(Key = warning, Headers) -> + key_value_tuple(atom_to_list(Key), Headers#http_response_h.warning); +key_value_tuple(Key = 'accept-ranges', Headers) -> + key_value_tuple(atom_to_list(Key), + Headers#http_response_h.'accept-ranges'); +key_value_tuple(Key = age, Headers) -> + key_value_tuple(atom_to_list(Key), Headers#http_response_h.age); +key_value_tuple(Key = etag, Headers) -> + key_value_tuple(atom_to_list(Key), Headers#http_response_h.etag); +key_value_tuple(Key = location, Headers) -> + key_value_tuple(atom_to_list(Key), Headers#http_response_h.location); +key_value_tuple(Key = 'proxy-authenticate', Headers) -> + key_value_tuple(atom_to_list(Key), + Headers#http_response_h.'proxy-authenticate'); +key_value_tuple(Key = 'retry-after', Headers) -> + key_value_tuple(atom_to_list(Key), Headers#http_response_h.'retry-after'); +key_value_tuple(Key = server, Headers) -> + key_value_tuple(atom_to_list(Key), Headers#http_response_h.server); +key_value_tuple(Key = vary, Headers) -> + key_value_tuple(atom_to_list(Key), Headers#http_response_h.vary); +key_value_tuple(Key = 'www-authenticate', Headers) -> + key_value_tuple(atom_to_list(Key), + Headers#http_response_h.'www-authenticate'); +key_value_tuple(Key = allow, Headers) -> + key_value_tuple(atom_to_list(Key), Headers#http_response_h.allow); +key_value_tuple(Key = 'content-encoding', Headers) -> + key_value_tuple(atom_to_list(Key), + Headers#http_response_h.'content-encoding'); +key_value_tuple(Key = 'content-language', Headers) -> + key_value_tuple(atom_to_list(Key), + Headers#http_response_h.'content-language'); +key_value_tuple(Key = 'content-length', Headers) -> + case Headers#http_response_h.'content-length' of + "-1" -> + undefined; + _ -> + key_value_tuple(atom_to_list(Key), + Headers#http_response_h.'content-length') + end; +key_value_tuple(Key = 'content-location', Headers) -> + key_value_tuple(atom_to_list(Key), + Headers#http_response_h.'content-location'); +key_value_tuple(Key = 'content-md5', Headers) -> + key_value_tuple(atom_to_list(Key), + Headers#http_response_h.'content-md5'); +key_value_tuple(Key = 'content-range', Headers) -> + key_value_tuple(atom_to_list(Key), + Headers#http_response_h.'content-range'); +key_value_tuple(Key = 'content-type', Headers) -> + key_value_tuple(atom_to_list(Key), + Headers#http_response_h.'content-type'); +key_value_tuple(Key = expires, Headers) -> + key_value_tuple(atom_to_list(Key), Headers#http_response_h.expires); +key_value_tuple(Key = 'last-modified', Headers) -> + key_value_tuple(atom_to_list(Key), + Headers#http_response_h.'last-modified'); +key_value_tuple(_, undefined) -> + undefined; +key_value_tuple(Key, Value) -> + {Key, Value}. diff --git a/lib/inets/src/http_lib/http_transport.erl b/lib/inets/src/http_lib/http_transport.erl new file mode 100644 index 0000000000..8100d7183a --- /dev/null +++ b/lib/inets/src/http_lib/http_transport.erl @@ -0,0 +1,353 @@ +%% +%% %CopyrightBegin% +%% +%% Copyright Ericsson AB 2004-2009. All Rights Reserved. +%% +%% The contents of this file are subject to the Erlang Public License, +%% Version 1.1, (the "License"); you may not use this file except in +%% compliance with the License. You should have received a copy of the +%% Erlang Public License along with this software. If not, it can be +%% retrieved online at http://www.erlang.org/. +%% +%% Software distributed under the License is distributed on an "AS IS" +%% basis, WITHOUT WARRANTY OF ANY KIND, either express or implied. See +%% the License for the specific language governing rights and limitations +%% under the License. +%% +%% %CopyrightEnd% +%% +% +-module(http_transport). + +% Internal application API +-export([start/1, connect/3, connect/4, listen/2, listen/3, + accept/2, accept/3, close/2, + send/3, controlling_process/3, setopts/3, + peername/2, resolve/0]). + +-export([negotiate/3]). + + +%%%========================================================================= +%%% Internal application API +%%%========================================================================= + +%%------------------------------------------------------------------------- +%% start(SocketType) -> ok | {error, Reason} +%% SocketType = ip_comm | {ssl, _} +%% +%% Description: Makes sure inet_db or ssl is started. +%%------------------------------------------------------------------------- +start(ip_comm) -> + case inet_db:start() of + {ok, _} -> + ok; + {error, {already_started, _}} -> + ok; + Error -> + Error + end; +start({ssl, _}) -> + case ssl:start() of + ok -> + ok; + {error, {already_started,_}} -> + ok; + Error -> + Error + end. + + +%%------------------------------------------------------------------------- +%% connect(SocketType, Address, Options, Timeout) -> +%% {ok, Socket} | {error, Reason} +%% SocketType = ip_comm | {ssl, SslConfig} +%% Address = {Host, Port} +%% Options = [option()] +%% Socket = socket() +%% option() = ipfamily() | {ip, ip_address()} | {port, integer()} +%% ipfamily() = inet | inet6 +%% +%% Description: Connects to the Host and Port specified in HTTPRequest. +%%------------------------------------------------------------------------- + +connect(SocketType, Address, Opts) -> + connect(SocketType, Address, Opts, infinity). + +connect(ip_comm = _SocketType, {Host, Port}, Opts0, Timeout) + when is_list(Opts0) -> + Opts = [binary, {packet, 0}, {active, false}, {reuseaddr, true} | Opts0], + gen_tcp:connect(Host, Port, Opts, Timeout); + +connect({ssl, SslConfig}, {Host, Port}, _, Timeout) -> + Opts = [binary, {active, false}] ++ SslConfig, + ssl:connect(Host, Port, Opts, Timeout); + +connect({erl_ssl, SslConfig}, {Host, Port}, _, Timeout) -> + Opts = [binary, {active, false}, {ssl_imp, new}] ++ SslConfig, + ssl:connect(Host, Port, Opts, Timeout). + + +%%------------------------------------------------------------------------- +%% listen(SocketType, Port) -> {ok, Socket} | {error, Reason} +%% SocketType = ip_comm | {ssl, SSLConfig} +%% Port = integer() +%% Socket = socket() +%% +%% Description: Sets up socket to listen on the port Port on the local +%% host using either gen_tcp or ssl. In the gen_tcp case the port +%% might allready have been initiated by a wrapper-program and is +%% given as an Fd that can be retrieved by init:get_argument. The +%% reason for this to enable a HTTP-server not running as root to use +%% port 80. +%%------------------------------------------------------------------------- +listen(SocketType, Port) -> + listen(SocketType, undefined, Port). + +listen(ip_comm, Addr, Port) -> + case (catch listen_ip_comm(Addr, Port)) of + {'EXIT', Reason} -> + {error, {exit, Reason}}; + Else -> + Else + end; + +listen({ssl, SSLConfig} = Ssl, Addr, Port) -> + Opt = sock_opt(Ssl, Addr, SSLConfig), + ssl:listen(Port, Opt); + +listen({erl_ssl, SSLConfig} = Ssl, Addr, Port) -> + Opt = sock_opt(Ssl, Addr, SSLConfig), + ssl:listen(Port, [{ssl_imp, new} | Opt]). + + +listen_ip_comm(Addr, Port) -> + {NewPort, Opts, IpFamily} = get_socket_info(Addr, Port), + case IpFamily of + inet6fb4 -> + Opts2 = [inet6 | Opts], + case (catch gen_tcp:listen(NewPort, Opts2)) of + {error, Reason} when ((Reason =:= nxdomain) orelse + (Reason =:= eafnosupport)) -> + Opts3 = [inet | Opts], + gen_tcp:listen(NewPort, Opts3); + + %% This is when a given hostname has resolved to a + %% IPv4-address. The inet6-option together with a + %% {ip, IPv4} option results in badarg + {'EXIT', _} -> + Opts3 = [inet | Opts], + gen_tcp:listen(NewPort, Opts3); + + Other -> + Other + end; + _ -> + Opts2 = [IpFamily | Opts], + gen_tcp:listen(NewPort, Opts2) + end. + +ipfamily_default(Addr, Port) -> + httpd_conf:lookup(Addr, Port, ipfamily, inet6fb4). + +get_socket_info(Addr, Port) -> + Key = list_to_atom("httpd_" ++ integer_to_list(Port)), + BaseOpts = [{backlog, 128}, {reuseaddr, true}], + IpFamilyDefault = ipfamily_default(Addr, Port), + case init:get_argument(Key) of + {ok, [[Value]]} -> + {Fd, IpFamily} = + case string:tokens(Value, [$|]) of + [FdStr, IpFamilyStr] -> + Fd0 = fd_of(FdStr), + IpFamily0 = ip_family_of(IpFamilyStr), + {Fd0, IpFamily0}; + [FdStr] -> + {fd_of(FdStr), IpFamilyDefault}; + _ -> + throw({error, {bad_descriptor, Value}}) + end, + {0, sock_opt(ip_comm, Addr, [{fd, Fd} | BaseOpts]), IpFamily}; + error -> + {Port, sock_opt(ip_comm, Addr, BaseOpts), IpFamilyDefault} + end. + + +fd_of(FdStr) -> + case (catch list_to_integer(FdStr)) of + Fd when is_integer(Fd) -> + Fd; + _ -> + throw({error, {bad_descriptor, FdStr}}) + end. + +ip_family_of(IpFamilyStr) -> + IpFamily = list_to_atom(IpFamilyStr), + case lists:member(IpFamily, [inet, inet6, inet6fb4]) of + true -> + IpFamily; + false -> + throw({error, {bad_ipfamily, IpFamilyStr}}) + end. + + +%%------------------------------------------------------------------------- +%% accept(SocketType, ListenSocket) -> {ok, Socket} | {error, Reason} +%% accept(SocketType, ListenSocket, Timeout) -> ok | {error, Reason} +%% SocketType = ip_comm | {ssl, SSLConfig} +%% ListenSocket = socket() +%% Timeout = infinity | integer() >= 0 +%% Socket = socket() +%% +%% Description: Accepts an incoming connection request on a listen socket, +%% using either gen_tcp or ssl. +%%------------------------------------------------------------------------- +accept(SocketType, ListenSocket) -> + accept(SocketType, ListenSocket, infinity). +accept(ip_comm, ListenSocket, Timeout) -> + gen_tcp:accept(ListenSocket, Timeout); +accept({ssl,_SSLConfig}, ListenSocket, Timeout) -> + ssl:transport_accept(ListenSocket, Timeout). + +%%------------------------------------------------------------------------- +%% controlling_process(SocketType, Socket, NewOwner) -> ok | {error, Reason} +%% SocketType = ip_comm | {ssl, _} +%% Socket = socket() +%% NewOwner = pid() +%% +%% Description: Assigns a new controlling process to Socket. +%%------------------------------------------------------------------------- +controlling_process(ip_comm, Socket, NewOwner) -> + gen_tcp:controlling_process(Socket, NewOwner); +controlling_process({ssl, _}, Socket, NewOwner) -> + ssl:controlling_process(Socket, NewOwner). + +%%------------------------------------------------------------------------- +%% setopts(SocketType, Socket, Options) -> ok | {error, Reason} +%% SocketType = ip_comm | {ssl, _} +%% Socket = socket() +%% Options = list() +%% Description: Sets one or more options for a socket, using either +%% gen_tcp or ssl. +%%------------------------------------------------------------------------- +setopts(ip_comm, Socket, Options) -> + inet:setopts(Socket,Options); +setopts({ssl, _}, Socket, Options) -> + ssl:setopts(Socket, Options). + +%%------------------------------------------------------------------------- +%% send(RequestOrSocketType, Socket, Message) -> ok | {error, Reason} +%% SocketType = ip_comm | {ssl, _} +%% Socket = socket() +%% Message = list() | binary() +%% Description: Sends a packet on a socket, using either gen_tcp or ssl. +%%------------------------------------------------------------------------- +send(ip_comm, Socket, Message) -> + gen_tcp:send(Socket, Message); +send({ssl, _}, Socket, Message) -> + ssl:send(Socket, Message). + +%%------------------------------------------------------------------------- +%% close(SocketType, Socket) -> ok | {error, Reason} +%% SocketType = ip_comm | {ssl, _} +%% Socket = socket() +%% +%% Description: Closes a socket, using either gen_tcp or ssl. +%%------------------------------------------------------------------------- +close(ip_comm, Socket) -> + gen_tcp:close(Socket); +close({ssl, _}, Socket) -> + ssl:close(Socket). + +%%------------------------------------------------------------------------- +%% peername(SocketType, Socket) -> ok | {error, Reason} +%% SocketType = ip_comm | {ssl, _} +%% Socket = socket() +%% +%% Description: Returns the address and port for the other end of a +%% connection, usning either gen_tcp or ssl. +%%------------------------------------------------------------------------- +peername(ip_comm, Socket) -> + case inet:peername(Socket) of + {ok,{{A, B, C, D}, Port}} -> + PeerName = integer_to_list(A)++"."++integer_to_list(B)++"."++ + integer_to_list(C)++"."++integer_to_list(D), + {Port, PeerName}; + {ok,{{A, B, C, D, E, F, G, H}, Port}} -> + PeerName = http_util:integer_to_hexlist(A) ++ ":"++ + http_util:integer_to_hexlist(B) ++ ":" ++ + http_util:integer_to_hexlist(C) ++ ":" ++ + http_util:integer_to_hexlist(D) ++ ":" ++ + http_util:integer_to_hexlist(E) ++ ":" ++ + http_util:integer_to_hexlist(F) ++ ":" ++ + http_util:integer_to_hexlist(G) ++":"++ + http_util:integer_to_hexlist(H), + {Port, PeerName}; + {error, _} -> + {-1, "unknown"} + end; + +peername({ssl, _}, Socket) -> + case ssl:peername(Socket) of + {ok,{{A, B, C, D}, Port}} -> + PeerName = integer_to_list(A)++"."++integer_to_list(B)++"."++ + integer_to_list(C)++"."++integer_to_list(D), + {Port, PeerName}; + {error, _} -> + {-1, "unknown"} + end. + +%%------------------------------------------------------------------------- +%% resolve() -> HostName +%% HostName = string() +%% +%% Description: Returns the local hostname. +%%------------------------------------------------------------------------- +resolve() -> + {ok, Name} = inet:gethostname(), + Name. + + +%%%======================================================================== +%%% Internal functions +%%%======================================================================== + +%% Address any comes from directive: BindAddress "*" +sock_opt(ip_comm, any = Addr, Opts) -> + sock_opt2([{ip, Addr} | Opts]); +sock_opt(ip_comm, undefined, Opts) -> + sock_opt2(Opts); +sock_opt(_, any = _Addr, Opts) -> + sock_opt2(Opts); +sock_opt(_, undefined = _Addr, Opts) -> + sock_opt2(Opts); +sock_opt(_, {_,_,_,_} = Addr, Opts) -> + sock_opt2([{ip, Addr} | Opts]); +sock_opt(ip_comm, Addr, Opts) -> + sock_opt2([{ip, Addr} | Opts]); +sock_opt(_, Addr, Opts) -> + sock_opt2([{ip, Addr} | Opts]). + +sock_opt2(Opts) -> + [{packet, 0}, {active, false} | Opts]. + +negotiate(ip_comm,_,_) -> + ok; +negotiate({ssl,_},Socket,Timeout) -> + negotiate(Socket, Timeout); +negotiate({erl_ssl, _}, Socket, Timeout) -> + negotiate(Socket, Timeout). + +negotiate(Socket, Timeout) -> + case ssl:ssl_accept(Socket, Timeout) of + ok -> + ok; + {error, Error} -> + case lists:member(Error, + [timeout,econnreset,esslaccept,esslerrssl]) of + true -> + {error,normal}; + false -> + {error, Error} + end + end. diff --git a/lib/inets/src/http_lib/http_util.erl b/lib/inets/src/http_lib/http_util.erl new file mode 100644 index 0000000000..b03b780cf8 --- /dev/null +++ b/lib/inets/src/http_lib/http_util.erl @@ -0,0 +1,148 @@ +%% +%% %CopyrightBegin% +%% +%% Copyright Ericsson AB 2005-2009. All Rights Reserved. +%% +%% The contents of this file are subject to the Erlang Public License, +%% Version 1.1, (the "License"); you may not use this file except in +%% compliance with the License. You should have received a copy of the +%% Erlang Public License along with this software. If not, it can be +%% retrieved online at http://www.erlang.org/. +%% +%% Software distributed under the License is distributed on an "AS IS" +%% basis, WITHOUT WARRANTY OF ANY KIND, either express or implied. See +%% the License for the specific language governing rights and limitations +%% under the License. +%% +%% %CopyrightEnd% +%% +%% +-module(http_util). + +-export([to_upper/1, to_lower/1, convert_netscapecookie_date/1, + hexlist_to_integer/1, integer_to_hexlist/1, + convert_month/1, is_hostname/1]). + +%%%========================================================================= +%%% Internal application API +%%%========================================================================= +to_upper(Str) -> + string:to_upper(Str). + +to_lower(Str) -> + string:to_lower(Str). + +convert_netscapecookie_date([_D,_A,_Y, $,, _SP, + D1,D2,_DA, + M,O,N,_DA, + Y1,Y2,Y3,Y4,_SP, + H1,H2,_Col, + M1,M2,_Col, + S1,S2|_Rest]) -> + Year=list_to_integer([Y1,Y2,Y3,Y4]), + Day=list_to_integer([D1,D2]), + Month=convert_month([M,O,N]), + Hour=list_to_integer([H1,H2]), + Min=list_to_integer([M1,M2]), + Sec=list_to_integer([S1,S2]), + {{Year,Month,Day},{Hour,Min,Sec}}; + +convert_netscapecookie_date([_D,_A,_Y, _SP, + D1,D2,_DA, + M,O,N,_DA, + Y1,Y2,Y3,Y4,_SP, + H1,H2,_Col, + M1,M2,_Col, + S1,S2|_Rest]) -> + Year=list_to_integer([Y1,Y2,Y3,Y4]), + Day=list_to_integer([D1,D2]), + Month=convert_month([M,O,N]), + Hour=list_to_integer([H1,H2]), + Min=list_to_integer([M1,M2]), + Sec=list_to_integer([S1,S2]), + {{Year,Month,Day},{Hour,Min,Sec}}. + +hexlist_to_integer([])-> + empty; +%%When the string only contains one value its eaasy done. +%% 0-9 +hexlist_to_integer([Size]) when Size >= 48 , Size =< 57 -> + Size - 48; +%% A-F +hexlist_to_integer([Size]) when Size >= 65 , Size =< 70 -> + Size - 55; +%% a-f +hexlist_to_integer([Size]) when Size >= 97 , Size =< 102 -> + Size - 87; +hexlist_to_integer([_Size]) -> + not_a_num; + +hexlist_to_integer(Size) -> + Len = string:span(Size, "1234567890abcdefABCDEF"), + hexlist_to_integer2(Size, 16 bsl (4 *(Len-2)),0). + +integer_to_hexlist(Num)-> + integer_to_hexlist(Num, get_size(Num), []). + +convert_month("Jan") -> 1; +convert_month("Feb") -> 2; +convert_month("Mar") -> 3; +convert_month("Apr") -> 4; +convert_month("May") -> 5; +convert_month("Jun") -> 6; +convert_month("Jul") -> 7; +convert_month("Aug") -> 8; +convert_month("Sep") -> 9; +convert_month("Oct") -> 10; +convert_month("Nov") -> 11; +convert_month("Dec") -> 12. + +is_hostname(Dest) -> + inet_parse:domain(Dest). + +%%%======================================================================== +%%% Internal functions +%%%======================================================================== +hexlist_to_integer2([],_Pos,Sum)-> + Sum; +hexlist_to_integer2([HexVal | HexString], Pos, Sum) + when HexVal >= 48, HexVal =< 57 -> + hexlist_to_integer2(HexString, Pos bsr 4, Sum + ((HexVal-48) * Pos)); + +hexlist_to_integer2([HexVal | HexString], Pos, Sum) + when HexVal >= 65, HexVal =<70 -> + hexlist_to_integer2(HexString, Pos bsr 4, Sum + ((HexVal-55) * Pos)); + +hexlist_to_integer2([HexVal | HexString], Pos, Sum) + when HexVal>=97, HexVal=<102 -> + hexlist_to_integer2(HexString, Pos bsr 4, Sum + ((HexVal-87) * Pos)); + +hexlist_to_integer2(_AfterHexString, _Pos, Sum)-> + Sum. + +integer_to_hexlist(Num, Pot, Res) when Pot<0 -> + convert_to_ascii([Num | Res]); + +integer_to_hexlist(Num,Pot,Res) -> + Position = (16 bsl (Pot*4)), + PosVal = Num div Position, + integer_to_hexlist(Num - (PosVal*Position), Pot-1, [PosVal | Res]). + +get_size(Num)-> + get_size(Num, 0). + +get_size(Num, Pot) when Num < (16 bsl(Pot *4)) -> + Pot-1; + +get_size(Num, Pot) -> + get_size(Num, Pot+1). + +convert_to_ascii(RevesedNum) -> + convert_to_ascii(RevesedNum, []). + +convert_to_ascii([], Num)-> + Num; +convert_to_ascii([Num | Reversed], Number) when Num > -1, Num < 10 -> + convert_to_ascii(Reversed, [Num + 48 | Number]); +convert_to_ascii([Num | Reversed], Number) when Num > 9, Num < 16 -> + convert_to_ascii(Reversed, [Num + 55 | Number]). diff --git a/lib/inets/src/http_server/Makefile b/lib/inets/src/http_server/Makefile new file mode 100644 index 0000000000..4bbd23df3f --- /dev/null +++ b/lib/inets/src/http_server/Makefile @@ -0,0 +1,139 @@ +# +# %CopyrightBegin% +# +# Copyright Ericsson AB 2005-2009. All Rights Reserved. +# +# The contents of this file are subject to the Erlang Public License, +# Version 1.1, (the "License"); you may not use this file except in +# compliance with the License. You should have received a copy of the +# Erlang Public License along with this software. If not, it can be +# retrieved online at http://www.erlang.org/. +# +# Software distributed under the License is distributed on an "AS IS" +# basis, WITHOUT WARRANTY OF ANY KIND, either express or implied. See +# the License for the specific language governing rights and limitations +# under the License. +# +# %CopyrightEnd% +# +# +include $(ERL_TOP)/make/target.mk +EBIN = ../../ebin +include $(ERL_TOP)/make/$(TARGET)/otp.mk + + +# ---------------------------------------------------- +# Application version +# ---------------------------------------------------- +include ../../vsn.mk + +VSN = $(INETS_VSN) + + +# ---------------------------------------------------- +# Release directory specification +# ---------------------------------------------------- +RELSYSDIR = $(RELEASE_PATH)/lib/inets-$(VSN) + + +# ---------------------------------------------------- +# Target Specs +# ---------------------------------------------------- +MODULES = \ + httpd \ + httpd_acceptor \ + httpd_acceptor_sup \ + httpd_cgi \ + httpd_conf \ + httpd_example \ + httpd_esi \ + httpd_file\ + httpd_instance_sup \ + httpd_log \ + httpd_manager \ + httpd_misc_sup \ + httpd_request \ + httpd_request_handler \ + httpd_response \ + httpd_script_env \ + httpd_socket \ + httpd_sup \ + httpd_util \ + mod_actions \ + mod_alias \ + mod_auth \ + mod_auth_plain \ + mod_auth_dets \ + mod_auth_mnesia \ + mod_auth_server \ + mod_browser \ + mod_cgi \ + mod_dir \ + mod_disk_log \ + mod_esi \ + mod_get \ + mod_head \ + mod_htaccess \ + mod_include \ + mod_log \ + mod_range \ + mod_responsecontrol \ + mod_trace \ + mod_security \ + mod_security_server + +HRL_FILES = httpd.hrl httpd_internal.hrl mod_auth.hrl + +ERL_FILES = $(MODULES:%=%.erl) + +TARGET_FILES= $(MODULES:%=$(EBIN)/%.$(EMULATOR)) + + +# ---------------------------------------------------- +# INETS FLAGS +# ---------------------------------------------------- +INETS_FLAGS = -D'SERVER_SOFTWARE="inets/$(VSN)"' + + +# ---------------------------------------------------- +# FLAGS +# ---------------------------------------------------- +INETS_ERL_FLAGS += -I ../http_lib -I ../inets_app -pa ../../ebin + +ERL_COMPILE_FLAGS += $(INETS_ERL_FLAGS) \ + $(INETS_FLAGS) \ + +'{parse_transform,sys_pre_attributes}' \ + +'{attribute,insert,app_vsn,$(APP_VSN)}' + + +# ---------------------------------------------------- +# Targets +# ---------------------------------------------------- + +debug opt: $(TARGET_FILES) + +clean: + rm -f $(TARGET_FILES) + rm -f core + +docs: + + +# ---------------------------------------------------- +# Release Target +# ---------------------------------------------------- +include $(ERL_TOP)/make/otp_release_targets.mk + +release_spec: opt + $(INSTALL_DIR) $(RELSYSDIR)/src + $(INSTALL_DATA) $(HRL_FILES) $(ERL_FILES) $(RELSYSDIR)/src + $(INSTALL_DIR) $(RELSYSDIR)/ebin + $(INSTALL_DATA) $(TARGET_FILES) $(RELSYSDIR)/ebin + +release_docs_spec: + +info: + @echo "INETS_DEBUG = $(INETS_DEBUG)" + @echo "INETS_FLAGS = $(INETS_FLAGS)" + @echo "ERL_COMPILE_FLAGS = $(ERL_COMPILE_FLAGS)" + diff --git a/lib/inets/src/http_server/httpd.erl b/lib/inets/src/http_server/httpd.erl new file mode 100644 index 0000000000..554f162fc5 --- /dev/null +++ b/lib/inets/src/http_server/httpd.erl @@ -0,0 +1,600 @@ +%% +%% %CopyrightBegin% +%% +%% Copyright Ericsson AB 1997-2009. All Rights Reserved. +%% +%% The contents of this file are subject to the Erlang Public License, +%% Version 1.1, (the "License"); you may not use this file except in +%% compliance with the License. You should have received a copy of the +%% Erlang Public License along with this software. If not, it can be +%% retrieved online at http://www.erlang.org/. +%% +%% Software distributed under the License is distributed on an "AS IS" +%% basis, WITHOUT WARRANTY OF ANY KIND, either express or implied. See +%% the License for the specific language governing rights and limitations +%% under the License. +%% +%% %CopyrightEnd% +%% +%% + +-module(httpd). + +-behaviour(inets_service). + +-include("httpd.hrl"). + +-deprecated({start, 0, next_major_release}). +-deprecated({start, 1, next_major_release}). +-deprecated({start_link, 1, next_major_release}). +-deprecated({start_child, 0, next_major_release}). +-deprecated({start_child, 1, next_major_release}). +-deprecated({stop, 0, next_major_release}). +-deprecated({stop, 1, next_major_release}). +-deprecated({stop, 2, next_major_release}). +-deprecated({stop_child, 0, next_major_release}). +-deprecated({stop_child, 1, next_major_release}). +-deprecated({stop_child, 2, next_major_release}). +-deprecated({restart, 0, next_major_release}). +-deprecated({restart, 1, next_major_release}). +-deprecated({restart, 2, next_major_release}). +-deprecated({block, 0, next_major_release}). +-deprecated({block, 1, next_major_release}). +-deprecated({block, 2, next_major_release}). +-deprecated({block, 3, next_major_release}). +-deprecated({block, 4, next_major_release}). +-deprecated({unblock, 0, next_major_release}). +-deprecated({unblock, 1, next_major_release}). +-deprecated({unblock, 2, next_major_release}). + +%% Behavior callbacks +-export([start_standalone/1, start_service/1, stop_service/1, services/0, + service_info/1]). + +%% API +-export([parse_query/1, reload_config/2, info/1, info/2, info/3]). + +%% Deprecated +-export([start/0, start/1, + start_link/0, start_link/1, + start_child/0,start_child/1, + stop/0,stop/1,stop/2, + stop_child/0,stop_child/1,stop_child/2, + restart/0,restart/1,restart/2]). + +%% Management stuff should be internal functions +%% Will be from r13 +-export([block/0,block/1,block/2,block/3,block/4, + unblock/0,unblock/1,unblock/2]). + +%% Internal Debugging and status info stuff... +%% Keep for now should probably be moved to test catalog +-export([get_status/1,get_status/2,get_status/3, + get_admin_state/0,get_admin_state/1,get_admin_state/2, + get_usage_state/0,get_usage_state/1,get_usage_state/2]). + +%%%======================================================================== +%%% API +%%%======================================================================== + +parse_query(String) -> + {ok, SplitString} = inets_regexp:split(String,"[&;]"), + foreach(SplitString). + +reload_config(Config = [Value| _], Mode) when is_tuple(Value) -> + do_reload_config(Config, Mode); +reload_config(ConfigFile, Mode) -> + case httpd_conf:load(ConfigFile) of + {ok, ConfigList} -> + do_reload_config(ConfigList, Mode); + Error -> + Error + end. + +info(Pid) when is_pid(Pid) -> + info(Pid, []). + +info(Pid, Properties) when is_pid(Pid) andalso is_list(Properties) -> + {ok, ServiceInfo} = service_info(Pid), + Address = proplists:get_value(bind_address, ServiceInfo), + Port = proplists:get_value(port, ServiceInfo), + case Properties of + [] -> + info(Address, Port); + _ -> + info(Address, Port, Properties) + end; +info(Address, Port) when is_integer(Port) -> + httpd_conf:get_config(Address, Port). + +info(Address, Port, Properties) when is_integer(Port) andalso + is_list(Properties) -> + httpd_conf:get_config(Address, Port, Properties). + +%%%======================================================================== +%%% Behavior callbacks +%%%======================================================================== + +start_standalone(Config) -> + httpd_sup:start_link([{httpd, Config}], stand_alone). + +start_service(Conf) -> + httpd_sup:start_child(Conf). + +stop_service({Address, Port}) -> + httpd_sup:stop_child(Address, Port); + +stop_service(Pid) when is_pid(Pid) -> + case service_info(Pid) of + {ok, Info} -> + Address = proplists:get_value(bind_address, Info), + Port = proplists:get_value(port, Info), + stop_service({Address, Port}); + Error -> + Error + end. + +services() -> + [{httpd, ChildPid} || {_, ChildPid, _, _} <- + supervisor:which_children(httpd_sup)]. + +service_info(Pid) -> + try + [{ChildName, ChildPid} || + {ChildName, ChildPid, _, _} <- + supervisor:which_children(httpd_sup)] of + Children -> + child_name2info(child_name(Pid, Children)) + catch + exit:{noproc, _} -> + {error, service_not_available} + end. +%%%-------------------------------------------------------------- +%%% Internal functions +%%%-------------------------------------------------------------------- + +child_name(_, []) -> + undefined; +child_name(Pid, [{Name, Pid} | _]) -> + Name; +child_name(Pid, [_ | Children]) -> + child_name(Pid, Children). + +child_name2info(undefined) -> + {error, no_such_service}; +child_name2info({httpd_instance_sup, any, Port}) -> + {ok, Host} = inet:gethostname(), + Info = info(any, Port, [server_name]), + {ok, [{bind_address, any}, {host, Host}, {port, Port} | Info]}; +child_name2info({httpd_instance_sup, Address, Port}) -> + Info = info(Address, Port, [server_name]), + case inet:gethostbyaddr(Address) of + {ok, {_, Host, _, _,_, _}} -> + {ok, [{bind_address, Address}, + {host, Host}, {port, Port} | Info]}; + _ -> + {ok, [{bind_address, Address}, {port, Port} | Info]} + end. + +reload(Config, Address, Port) -> + Name = make_name(Address,Port), + case whereis(Name) of + Pid when is_pid(Pid) -> + httpd_manager:reload(Pid, Config); + _ -> + {error,not_started} + end. + +reload(Addr, Port) when is_integer(Port) -> + Name = make_name(Addr,Port), + case whereis(Name) of + Pid when is_pid(Pid) -> + httpd_manager:reload(Pid, undefined); + _ -> + {error,not_started} + end. + +%%% ========================================================= +%%% Function: block/0, block/1, block/2, block/3, block/4 +%%% block() +%%% block(Port) +%%% block(ConfigFile) +%%% block(Addr,Port) +%%% block(Port,Mode) +%%% block(ConfigFile,Mode) +%%% block(Addr,Port,Mode) +%%% block(ConfigFile,Mode,Timeout) +%%% block(Addr,Port,Mode,Timeout) +%%% +%%% Returns: ok | {error,Reason} +%%% +%%% Description: This function is used to block an HTTP server. +%%% The blocking can be done in two ways, +%%% disturbing or non-disturbing. Default is disturbing. +%%% When a HTTP server is blocked, all requests are rejected +%%% (status code 503). +%%% +%%% disturbing: +%%% By performing a disturbing block, the server +%%% is blocked forcefully and all ongoing requests +%%% are terminated. No new connections are accepted. +%%% If a timeout time is given then, on-going requests +%%% are given this much time to complete before the +%%% server is forcefully blocked. In this case no new +%%% connections is accepted. +%%% +%%% non-disturbing: +%%% A non-disturbing block is more gracefull. No +%%% new connections are accepted, but the ongoing +%%% requests are allowed to complete. +%%% If a timeout time is given, it waits this long before +%%% giving up (the block operation is aborted and the +%%% server state is once more not-blocked). +%%% +%%% Types: Port -> integer() +%%% Addr -> {A,B,C,D} | string() | undefined +%%% ConfigFile -> string() +%%% Mode -> disturbing | non_disturbing +%%% Timeout -> integer() +%%% +block() -> block(undefined,8888,disturbing). + +block(Port) when is_integer(Port) -> + block(undefined,Port,disturbing); + +block(ConfigFile) when is_list(ConfigFile) -> + case get_addr_and_port(ConfigFile) of + {ok,Addr,Port} -> + block(Addr,Port,disturbing); + Error -> + Error + end. + +block(Addr,Port) when is_integer(Port) -> + block(Addr,Port,disturbing); + +block(Port,Mode) when is_integer(Port) andalso is_atom(Mode) -> + block(undefined,Port,Mode); + +block(ConfigFile,Mode) when is_list(ConfigFile) andalso is_atom(Mode) -> + case get_addr_and_port(ConfigFile) of + {ok,Addr,Port} -> + block(Addr,Port,Mode); + Error -> + Error + end. + + +block(Addr,Port,disturbing) when is_integer(Port) -> + do_block(Addr,Port,disturbing); +block(Addr,Port,non_disturbing) when is_integer(Port) -> + do_block(Addr,Port,non_disturbing); + +block(ConfigFile,Mode,Timeout) when is_list(ConfigFile) andalso + is_atom(Mode) andalso + is_integer(Timeout) -> + case get_addr_and_port(ConfigFile) of + {ok,Addr,Port} -> + block(Addr,Port,Mode,Timeout); + Error -> + Error + end. + + +block(Addr,Port,non_disturbing,Timeout) + when is_integer(Port) andalso is_integer(Timeout) -> + do_block(Addr,Port,non_disturbing,Timeout); +block(Addr,Port,disturbing,Timeout) when is_integer(Port) andalso + is_integer(Timeout) -> + do_block(Addr,Port,disturbing,Timeout). + +do_block(Addr,Port,Mode) when is_integer(Port) andalso is_atom(Mode) -> + Name = make_name(Addr,Port), + case whereis(Name) of + Pid when is_pid(Pid) -> + httpd_manager:block(Pid,Mode); + _ -> + {error,not_started} + end. + + +do_block(Addr,Port,Mode,Timeout) + when is_integer(Port) andalso is_atom(Mode) -> + Name = make_name(Addr,Port), + case whereis(Name) of + Pid when is_pid(Pid) -> + httpd_manager:block(Pid,Mode,Timeout); + _ -> + {error,not_started} + end. + + +%%% ========================================================= +%%% Function: unblock/0, unblock/1, unblock/2 +%%% unblock() +%%% unblock(Port) +%%% unblock(ConfigFile) +%%% unblock(Addr,Port) +%%% +%%% Description: This function is used to reverse a previous block +%%% operation on the HTTP server. +%%% +%%% Types: Port -> integer() +%%% Addr -> {A,B,C,D} | string() | undefined +%%% ConfigFile -> string() +%%% +unblock() -> unblock(undefined,8888). +unblock(Port) when is_integer(Port) -> unblock(undefined,Port); + +unblock(ConfigFile) when is_list(ConfigFile) -> + case get_addr_and_port(ConfigFile) of + {ok,Addr,Port} -> + unblock(Addr,Port); + Error -> + Error + end. + +unblock(Addr, Port) when is_integer(Port) -> + Name = make_name(Addr,Port), + case whereis(Name) of + Pid when is_pid(Pid) -> + httpd_manager:unblock(Pid); + _ -> + {error,not_started} + end. + +foreach([]) -> + []; +foreach([KeyValue|Rest]) -> + {ok, Plus2Space, _} = inets_regexp:gsub(KeyValue,"[\+]"," "), + case inets_regexp:split(Plus2Space,"=") of + {ok,[Key|Value]} -> + [{httpd_util:decode_hex(Key), + httpd_util:decode_hex(lists:flatten(Value))}|foreach(Rest)]; + {ok,_} -> + foreach(Rest) + end. + +get_addr_and_port(ConfigFile) -> + case httpd_conf:load(ConfigFile) of + {ok, ConfigList} -> + case httpd_conf:validate_properties(ConfigList) of + {ok, Config} -> + Address = proplists:get_value(bind_address, Config, any), + Port = proplists:get_value(port, Config, 80), + {ok, Address, Port}; + Error -> + Error + end; + Error -> + Error + end. + + +make_name(Addr, Port) -> + httpd_util:make_name("httpd", Addr, Port). + + +%%%-------------------------------------------------------------- +%%% Internal debug functions - Do we want these functions here!? +%%%-------------------------------------------------------------------- + +%%% ========================================================= +%%% Function: get_admin_state/0, get_admin_state/1, get_admin_state/2 +%%% get_admin_state() +%%% get_admin_state(Port) +%%% get_admin_state(Addr,Port) +%%% +%%% Returns: {ok,State} | {error,Reason} +%%% +%%% Description: This function is used to retrieve the administrative +%%% state of the HTTP server. +%%% +%%% Types: Port -> integer() +%%% Addr -> {A,B,C,D} | string() | undefined +%%% State -> unblocked | shutting_down | blocked +%%% Reason -> term() +%%% +get_admin_state() -> get_admin_state(undefined,8888). +get_admin_state(Port) when is_integer(Port) -> get_admin_state(undefined,Port); + +get_admin_state(ConfigFile) when is_list(ConfigFile) -> + case get_addr_and_port(ConfigFile) of + {ok,Addr,Port} -> + unblock(Addr,Port); + Error -> + Error + end. + +get_admin_state(Addr,Port) when is_integer(Port) -> + Name = make_name(Addr,Port), + case whereis(Name) of + Pid when is_pid(Pid) -> + httpd_manager:get_admin_state(Pid); + _ -> + {error,not_started} + end. + + + +%%% ========================================================= +%%% Function: get_usage_state/0, get_usage_state/1, get_usage_state/2 +%%% get_usage_state() +%%% get_usage_state(Port) +%%% get_usage_state(Addr,Port) +%%% +%%% Returns: {ok,State} | {error,Reason} +%%% +%%% Description: This function is used to retrieve the usage +%%% state of the HTTP server. +%%% +%%% Types: Port -> integer() +%%% Addr -> {A,B,C,D} | string() | undefined +%%% State -> idle | active | busy +%%% Reason -> term() +%%% +get_usage_state() -> get_usage_state(undefined,8888). +get_usage_state(Port) when is_integer(Port) -> get_usage_state(undefined,Port); + +get_usage_state(ConfigFile) when is_list(ConfigFile) -> + case get_addr_and_port(ConfigFile) of + {ok,Addr,Port} -> + unblock(Addr,Port); + Error -> + Error + end. + +get_usage_state(Addr,Port) when is_integer(Port) -> + Name = make_name(Addr,Port), + case whereis(Name) of + Pid when is_pid(Pid) -> + httpd_manager:get_usage_state(Pid); + _ -> + {error,not_started} + end. + + + +%%% ========================================================= +%% Function: get_status(ConfigFile) -> Status +%% get_status(Port) -> Status +%% get_status(Addr,Port) -> Status +%% get_status(Port,Timeout) -> Status +%% get_status(Addr,Port,Timeout) -> Status +%% +%% Arguments: ConfigFile -> string() +%% Configuration file from which Port and +%% BindAddress will be extracted. +%% Addr -> {A,B,C,D} | string() +%% Bind Address of the http server +%% Port -> integer() +%% Port number of the http server +%% Timeout -> integer() +%% Timeout time for the call +%% +%% Returns: Status -> list() +%% +%% Description: This function is used when the caller runs in the +%% same node as the http server or if calling with a +%% program such as erl_call (see erl_interface). +%% + +get_status(ConfigFile) when is_list(ConfigFile) -> + case get_addr_and_port(ConfigFile) of + {ok,Addr,Port} -> + get_status(Addr,Port); + Error -> + Error + end; + +get_status(Port) when is_integer(Port) -> + get_status(undefined,Port,5000). + +get_status(Port,Timeout) when is_integer(Port) andalso is_integer(Timeout) -> + get_status(undefined,Port,Timeout); + +get_status(Addr,Port) -> + get_status(Addr,Port,5000). + +get_status(Addr,Port,Timeout) when is_integer(Port) -> + Name = make_name(Addr,Port), + case whereis(Name) of + Pid when is_pid(Pid) -> + httpd_manager:get_status(Pid,Timeout); + _ -> + not_started + end. + +do_reload_config(ConfigList, Mode) -> + case httpd_conf:validate_properties(ConfigList) of + {ok, Config} -> + Address = proplists:get_value(bind_address, Config, any), + Port = proplists:get_value(port, Config, 80), + block(Address, Port, Mode), + reload(Config, Address, Port), + unblock(Address, Port); + Error -> + Error + end. + + +%%%-------------------------------------------------------------- +%%% Deprecated +%%%-------------------------------------------------------------- +start() -> + start("/var/tmp/server_root/conf/8888.conf"). + +start(ConfigFile) -> + {ok, Pid} = inets:start(httpd, ConfigFile, stand_alone), + unlink(Pid), + {ok, Pid}. + +start_link() -> + start("/var/tmp/server_root/conf/8888.conf"). + +start_link(ConfigFile) when is_list(ConfigFile) -> + inets:start(httpd, ConfigFile, stand_alone). + +stop() -> + stop(8888). + +stop(Port) when is_integer(Port) -> + stop(undefined, Port); +stop(Pid) when is_pid(Pid) -> + old_stop(Pid); +stop(ConfigFile) when is_list(ConfigFile) -> + old_stop(ConfigFile). + +stop(Addr, Port) when is_integer(Port) -> + old_stop(Addr, Port). + +start_child() -> + start_child("/var/tmp/server_root/conf/8888.conf"). + +start_child(ConfigFile) -> + httpd_sup:start_child(ConfigFile). + +stop_child() -> + stop_child(8888). + +stop_child(Port) -> + stop_child(undefined, Port). + +stop_child(Addr, Port) when is_integer(Port) -> + httpd_sup:stop_child(Addr, Port). + +restart() -> reload(undefined, 8888). + +restart(Port) when is_integer(Port) -> + reload(undefined, Port). +restart(Addr, Port) -> + reload(Addr, Port). + +old_stop(Pid) when is_pid(Pid) -> + do_stop(Pid); +old_stop(ConfigFile) when is_list(ConfigFile) -> + case get_addr_and_port(ConfigFile) of + {ok, Addr, Port} -> + old_stop(Addr, Port); + + Error -> + Error + end; +old_stop(_StartArgs) -> + ok. + +old_stop(Addr, Port) when is_integer(Port) -> + Name = old_make_name(Addr, Port), + case whereis(Name) of + Pid when is_pid(Pid) -> + do_stop(Pid), + ok; + _ -> + not_started + end. + +do_stop(Pid) -> + exit(Pid, shutdown). + +old_make_name(Addr,Port) -> + httpd_util:make_name("httpd_instance_sup",Addr,Port). diff --git a/lib/inets/src/http_server/httpd.hrl b/lib/inets/src/http_server/httpd.hrl new file mode 100644 index 0000000000..0db8a029bb --- /dev/null +++ b/lib/inets/src/http_server/httpd.hrl @@ -0,0 +1,82 @@ +%% +%% %CopyrightBegin% +%% +%% Copyright Ericsson AB 1997-2009. All Rights Reserved. +%% +%% The contents of this file are subject to the Erlang Public License, +%% Version 1.1, (the "License"); you may not use this file except in +%% compliance with the License. You should have received a copy of the +%% Erlang Public License along with this software. If not, it can be +%% retrieved online at http://www.erlang.org/. +%% +%% Software distributed under the License is distributed on an "AS IS" +%% basis, WITHOUT WARRANTY OF ANY KIND, either express or implied. See +%% the License for the specific language governing rights and limitations +%% under the License. +%% +%% %CopyrightEnd% +%% +%% + +-include_lib("kernel/include/file.hrl"). + +-ifndef(SERVER_SOFTWARE). +-define(SERVER_SOFTWARE,"inets/develop"). % Define in Makefile! +-endif. +-define(SERVER_PROTOCOL,"HTTP/1.1"). +-define(DEFAULT_MODS, [mod_alias, mod_auth, mod_esi, mod_actions, mod_cgi, + mod_dir, mod_get, mod_head, mod_log, mod_disk_log]). +-define(SOCKET_CHUNK_SIZE,8192). +-define(SOCKET_MAX_POLL,25). +-define(FILE_CHUNK_SIZE,64*1024). +-define(GATEWAY_INTERFACE,"CGI/1.1"). +-define(NICE(Reason),lists:flatten(atom_to_list(?MODULE)++": "++Reason)). +-define(DEFAULT_CONTEXT, + [{errmsg,"[an error occurred while processing this directive]"}, + {timefmt,"%A, %d-%b-%y %T %Z"}, + {sizefmt,"abbrev"}]). + + +-ifdef(inets_error). +-define(ERROR(Format, Args), io:format("E(~p:~p:~p) : "++Format++"~n", + [self(),?MODULE,?LINE]++Args)). +-else. +-define(ERROR(F,A),[]). +-endif. + +-ifdef(inets_log). +-define(LOG(Format, Args), io:format("L(~p:~p:~p) : "++Format++"~n", + [self(),?MODULE,?LINE]++Args)). +-else. +-define(LOG(F,A),[]). +-endif. + +-ifdef(inets_debug). +-define(DEBUG(Format, Args), io:format("D(~p:~p:~p) : "++Format++"~n", + [self(),?MODULE,?LINE]++Args)). +-else. +-define(DEBUG(F,A),[]). +-endif. + +-ifdef(inets_cdebug). +-define(CDEBUG(Format, Args), io:format("C(~p:~p:~p) : "++Format++"~n", + [self(),?MODULE,?LINE]++Args)). +-else. +-define(CDEBUG(F,A),[]). +-endif. + + +-record(init_data,{peername,resolve}). +-record(mod,{init_data, + data=[], + socket_type=ip_comm, + socket, + config_db, + method, + absolute_uri=[], + request_uri, + http_version, + request_line, + parsed_header=[], + entity_body, + connection}). diff --git a/lib/inets/src/http_server/httpd_acceptor.erl b/lib/inets/src/http_server/httpd_acceptor.erl new file mode 100644 index 0000000000..568fd3c610 --- /dev/null +++ b/lib/inets/src/http_server/httpd_acceptor.erl @@ -0,0 +1,211 @@ +%% +%% %CopyrightBegin% +%% +%% Copyright Ericsson AB 2001-2009. All Rights Reserved. +%% +%% The contents of this file are subject to the Erlang Public License, +%% Version 1.1, (the "License"); you may not use this file except in +%% compliance with the License. You should have received a copy of the +%% Erlang Public License along with this software. If not, it can be +%% retrieved online at http://www.erlang.org/. +%% +%% Software distributed under the License is distributed on an "AS IS" +%% basis, WITHOUT WARRANTY OF ANY KIND, either express or implied. See +%% the License for the specific language governing rights and limitations +%% under the License. +%% +%% %CopyrightEnd% +%% +%% +-module(httpd_acceptor). + +-include("httpd.hrl"). +-include("httpd_internal.hrl"). + +%% Internal application API +-export([start_link/5, start_link/6]). + +%% Other exports (for spawn's etc.) +-export([acceptor_init/6, acceptor_init/7, acceptor_loop/5]). + +%% +%% External API +%% + +%% start_link + +start_link(Manager, SocketType, Addr, Port, ConfigDb, AcceptTimeout) -> + ?hdrd("start link", + [{manager, Manager}, + {socket_type, SocketType}, + {address, Addr}, + {port, Port}, + {timeout, AcceptTimeout}]), + Args = [self(), Manager, SocketType, Addr, Port, ConfigDb, AcceptTimeout], + proc_lib:start_link(?MODULE, acceptor_init, Args). + +start_link(Manager, SocketType, ListenSocket, ConfigDb, AcceptTimeout) -> + ?hdrd("start link", + [{manager, Manager}, + {socket_type, SocketType}, + {listen_socket, ListenSocket}, + {timeout, AcceptTimeout}]), + Args = [self(), Manager, SocketType, ListenSocket, + ConfigDb, AcceptTimeout], + proc_lib:start_link(?MODULE, acceptor_init, Args). + +acceptor_init(Parent, Manager, SocketType, {ListenOwner, ListenSocket}, + ConfigDb, AcceptTimeout) -> + ?hdrd("acceptor init", + [{parent, Parent}, + {manager, Manager}, + {socket_type, SocketType}, + {listen_owner, ListenOwner}, + {listen_socket, ListenSocket}, + {timeout, AcceptTimeout}]), + link(ListenOwner), + proc_lib:init_ack(Parent, {ok, self()}), + acceptor_loop(Manager, SocketType, ListenSocket, ConfigDb, AcceptTimeout). + +acceptor_init(Parent, Manager, SocketType, Addr, Port, + ConfigDb, AcceptTimeout) -> + ?hdrd("acceptor init", + [{parent, Parent}, + {manager, Manager}, + {socket_type, SocketType}, + {address, Addr}, + {port, Port}, + {timeout, AcceptTimeout}]), + case (catch do_init(SocketType, Addr, Port)) of + {ok, ListenSocket} -> + proc_lib:init_ack(Parent, {ok, self()}), + acceptor_loop(Manager, SocketType, + ListenSocket, ConfigDb, AcceptTimeout); + Error -> + proc_lib:init_ack(Parent, Error), + error + end. + +do_init(SocketType, Addr, Port) -> + ?hdrt("do init", []), + do_socket_start(SocketType), + ListenSocket = do_socket_listen(SocketType, Addr, Port), + {ok, ListenSocket}. + + +do_socket_start(SocketType) -> + ?hdrt("do socket start", []), + case http_transport:start(SocketType) of + ok -> + ok; + {error, Reason} -> + ?hdrv("failed starting transport", [{reason, Reason}]), + throw({error, {socket_start_failed, Reason}}) + end. + + +do_socket_listen(SocketType, Addr, Port) -> + ?hdrt("do socket listen", []), + case http_transport:listen(SocketType, Addr, Port) of + {ok, ListenSocket} -> + ListenSocket; + {error, Reason} -> + ?hdrv("listen failed", [{reason, Reason}, + {socket_type, SocketType}, + {addr, Addr}, + {port, Port}]), + throw({error, {listen, Reason}}) + end. + + +%% acceptor + +acceptor_loop(Manager, SocketType, ListenSocket, ConfigDb, AcceptTimeout) -> + ?hdrd("awaiting accept", + [{manager, Manager}, + {socket_type, SocketType}, + {listen_socket, ListenSocket}, + {timeout, AcceptTimeout}]), + case (catch http_transport:accept(SocketType, ListenSocket, 50000)) of + {ok, Socket} -> + ?hdrv("accepted", [{socket, Socket}]), + handle_connection(Manager, ConfigDb, AcceptTimeout, + SocketType, Socket), + ?MODULE:acceptor_loop(Manager, SocketType, + ListenSocket, ConfigDb,AcceptTimeout); + {error, Reason} -> + ?hdri("accept failed", [{reason, Reason}]), + handle_error(Reason, ConfigDb), + ?MODULE:acceptor_loop(Manager, SocketType, ListenSocket, + ConfigDb, AcceptTimeout); + {'EXIT', Reason} -> + ?hdri("accept exited", [{reason, Reason}]), + handle_error({'EXIT', Reason}, ConfigDb), + ?MODULE:acceptor_loop(Manager, SocketType, ListenSocket, + ConfigDb, AcceptTimeout) + end. + + +handle_connection(Manager, ConfigDb, AcceptTimeout, SocketType, Socket) -> + {ok, Pid} = httpd_request_handler:start(Manager, ConfigDb, AcceptTimeout), + http_transport:controlling_process(SocketType, Socket, Pid), + httpd_request_handler:socket_ownership_transfered(Pid, SocketType, Socket). + +handle_error(timeout, _) -> + ok; + +handle_error({enfile, _}, _) -> + %% Out of sockets... + sleep(200); + +handle_error(emfile, _) -> + %% Too many open files -> Out of sockets... + sleep(200); + +handle_error(closed, _) -> + error_logger:info_report("The httpd accept socket was closed by " + "a third party. " + "This will not have an impact on inets " + "that will open a new accept socket and " + "go on as nothing happened. It does however " + "indicate that some other software is behaving " + "badly."), + exit(normal); + +%% This will only happen when the client is terminated abnormaly +%% and is not a problem for the server, so we want +%% to terminate normal so that we can restart without any +%% error messages. +handle_error(econnreset,_) -> + exit(normal); + +handle_error(econnaborted, _) -> + ok; + +handle_error(esslaccept, _) -> + %% The user has selected to cancel the installation of + %% the certifikate, This is not a real error, so we do + %% not write an error message. + ok; + +handle_error({'EXIT', Reason}, ConfigDb) -> + String = lists:flatten(io_lib:format("Accept exit: ~p", [Reason])), + accept_failed(ConfigDb, String); + +handle_error(Reason, ConfigDb) -> + String = lists:flatten(io_lib:format("Accept error: ~p", [Reason])), + accept_failed(ConfigDb, String). + +-spec accept_failed(_, string()) -> no_return(). + +accept_failed(ConfigDb, String) -> + error_logger:error_report(String), + InitData = #init_data{peername = {0, "unknown"}}, + Info = #mod{config_db = ConfigDb, init_data = InitData}, + mod_log:error_log(Info, String), + mod_disk_log:error_log(Info, String), + exit({accept_failed, String}). + +sleep(T) -> receive after T -> ok end. + + diff --git a/lib/inets/src/http_server/httpd_acceptor_sup.erl b/lib/inets/src/http_server/httpd_acceptor_sup.erl new file mode 100644 index 0000000000..8b1e4b6c4f --- /dev/null +++ b/lib/inets/src/http_server/httpd_acceptor_sup.erl @@ -0,0 +1,97 @@ +%% +%% %CopyrightBegin% +%% +%% Copyright Ericsson AB 2001-2009. All Rights Reserved. +%% +%% The contents of this file are subject to the Erlang Public License, +%% Version 1.1, (the "License"); you may not use this file except in +%% compliance with the License. You should have received a copy of the +%% Erlang Public License along with this software. If not, it can be +%% retrieved online at http://www.erlang.org/. +%% +%% Software distributed under the License is distributed on an "AS IS" +%% basis, WITHOUT WARRANTY OF ANY KIND, either express or implied. See +%% the License for the specific language governing rights and limitations +%% under the License. +%% +%% %CopyrightEnd% +%% +%% +%%---------------------------------------------------------------------- +%% Purpose: The supervisor for acceptor processes in the http server, +%% hangs under the httpd_instance_sup_<Addr>_<Port> supervisor. +%%---------------------------------------------------------------------- + +-module(httpd_acceptor_sup). + +-behaviour(supervisor). + +%% API +-export([start_link/2, start_acceptor/5, start_acceptor/6, stop_acceptor/2]). + +%% Supervisor callback +-export([init/1]). + +%%%========================================================================= +%%% API +%%%========================================================================= +start_link(Addr, Port) -> + SupName = make_name(Addr, Port), + supervisor:start_link({local, SupName}, ?MODULE, []). + +%%---------------------------------------------------------------------- +%% Function: [start|stop]_acceptor/5 +%% Description: Starts/stops an [auth | security] worker (child) process +%%---------------------------------------------------------------------- +start_acceptor(SocketType, Addr, Port, ConfigDb, AcceptTimeout) -> + start_worker(httpd_acceptor, SocketType, Addr, Port, + ConfigDb, AcceptTimeout, self(), []). +start_acceptor(SocketType, Addr, Port, ConfigDb, AcceptTimeout, ListenSocket) -> + start_worker(httpd_acceptor, SocketType, Addr, Port, + ConfigDb, AcceptTimeout, ListenSocket, self(), []). + + +stop_acceptor(Addr, Port) -> + stop_worker(httpd_acceptor, Addr, Port). + +%%%========================================================================= +%%% Supervisor callback +%%%========================================================================= +init(_) -> + Flags = {one_for_one, 500, 100}, + Workers = [], + {ok, {Flags, Workers}}. + +%%%========================================================================= +%%% Internal functions +%%%========================================================================= + +make_name(Addr,Port) -> + httpd_util:make_name("httpd_acc_sup", Addr, Port). + +start_worker(M, SocketType, Addr, Port, ConfigDB, AcceptTimeout, Manager, Modules) -> + SupName = make_name(Addr, Port), + Args = [Manager, SocketType, Addr, Port, ConfigDB, AcceptTimeout], + Spec = {{M, Addr, Port}, + {M, start_link, Args}, + permanent, timer:seconds(1), worker, [M] ++ Modules}, + supervisor:start_child(SupName, Spec). + +start_worker(M, SocketType, Addr, Port, ConfigDB, AcceptTimeout, ListenSocket, + Manager, Modules) -> + SupName = make_name(Addr, Port), + Args = [Manager, SocketType, ListenSocket, ConfigDB, AcceptTimeout], + Spec = {{M, Addr, Port}, + {M, start_link, Args}, + permanent, timer:seconds(1), worker, [M] ++ Modules}, + supervisor:start_child(SupName, Spec). + +stop_worker(M, Addr, Port) -> + SupName = make_name(Addr, Port), + Name = {M, Addr, Port}, + case supervisor:terminate_child(SupName, Name) of + ok -> + supervisor:delete_child(SupName, Name); + Error -> + Error + end. diff --git a/lib/inets/src/http_server/httpd_cgi.erl b/lib/inets/src/http_server/httpd_cgi.erl new file mode 100644 index 0000000000..0532d7d100 --- /dev/null +++ b/lib/inets/src/http_server/httpd_cgi.erl @@ -0,0 +1,124 @@ +%% +%% %CopyrightBegin% +%% +%% Copyright Ericsson AB 2005-2009. All Rights Reserved. +%% +%% The contents of this file are subject to the Erlang Public License, +%% Version 1.1, (the "License"); you may not use this file except in +%% compliance with the License. You should have received a copy of the +%% Erlang Public License along with this software. If not, it can be +%% retrieved online at http://www.erlang.org/. +%% +%% Software distributed under the License is distributed on an "AS IS" +%% basis, WITHOUT WARRANTY OF ANY KIND, either express or implied. See +%% the License for the specific language governing rights and limitations +%% under the License. +%% +%% %CopyrightEnd% +%% +%% +-module(httpd_cgi). + +-export([parse_headers/1, handle_headers/1]). + +-include("inets_internal.hrl"). + +%%%========================================================================= +%%% Internal application API +%%%========================================================================= + +%%-------------------------------------------------------------------------- +%% parse_headers([Bin, Data, Header, Headers]) -> {RevHeaders, Body} | +%% {Module, Function, Args} +%% Bin = Data = binary() +%% Header = string() - Accumulator should be [] in first call +%% Headers = [Header] - Accumulator should be [] in first call +%% Body = string() +%% RevHeaders = string() - Note CGI-headers not HTTP-headers +%% +%% Description: Parses "<<Bin/binary, Data/binary>>" returned from the +%% CGI-script until it findes the end of the CGI-headers (at least one +%% CGI-HeaderField must be supplied) then it returns the CGI-headers +%% and maybe some body data. If {Module, Function, Args} is +%% returned it means that more data needs to be collected from the +%% cgi-script as the end of the headers was not yet found. When more +%% data has been collected call Module:Function([NewData | Args]). +%% +%% NOTE: The headers are backwards and should +%% be so, devide_and_reverse_headers will reverse them back after +%% taking advantage of the fact that they where backwards. +%%-------------------------------------------------------------------------- +parse_headers([Data, Bin, Header, Headers]) -> + parse_headers(<<Bin/binary, Data/binary>>, Header, Headers). + +%%-------------------------------------------------------------------------- +%% handle_headers(CGIHeaders) -> {ok, HTTPHeaders, StatusCode} | +%% {proceed, AbsPath} +%% CGIHeaders = [string()] +%% HTTPHeaders = [{HeaderField, HeaderValue}] +%% HeaderField = string() +%% HeaderValue = string() +%% StatusCode = integer() +%% +%% Description: Interprets CGI headers and creates HTTP headers and a +%% appropriate HTTP status code. Note if a CGI location header is present +%% the return value will be {proceed, AbsPath} +%%-------------------------------------------------------------------------- +handle_headers(CGIHeaders) -> + handle_headers(CGIHeaders, [], {200, "ok"}). + +%%%======================================================================== +%%% Internal functions +%%%======================================================================== +parse_headers(<<>>, Header, Headers) -> + {?MODULE, parse_headers, [<<>>, Header, Headers]}; +parse_headers(<<?CR,?LF>>, Header, Headers) -> + {?MODULE, parse_headers, [<<?CR,?LF>>, Header, Headers]}; +parse_headers(<<?LF>>, Header, Headers) -> + {?MODULE, parse_headers, [<<?LF>>, Header, Headers]}; +parse_headers(<<?CR, ?LF, ?CR, ?LF, Rest/binary>>, Header, Headers) -> + {ok, {[lists:reverse([?LF, ?CR | Header]) | Headers], Rest}}; +parse_headers(<<?LF, ?LF, Rest/binary>>, Header, Headers) -> + {ok, {[lists:reverse([?LF | Header]) | Headers], Rest}}; +parse_headers(<<?CR, ?LF, Rest/binary>>, Header, Headers) -> + parse_headers(Rest, [], [lists:reverse([?LF, ?CR | Header]) | Headers]); +parse_headers(<<?LF, Rest/binary>>, Header, Headers) -> + parse_headers(Rest, [], [lists:reverse([?LF | Header]) | Headers]); +parse_headers(<<Octet, Rest/binary>>, Header, Headers) -> + parse_headers(Rest, [Octet | Header], Headers). + +handle_headers([], HTTPHeaders, Status) -> + {ok, HTTPHeaders, Status}; + +handle_headers([CGIHeader | CGIHeaders], HTTPHeaders, Status) -> + + {FieldName, FieldValue} = httpd_response:split_header(CGIHeader, []), + + case FieldName of + "content-type" -> + handle_headers(CGIHeaders, + [{FieldName, FieldValue} | HTTPHeaders], + Status); + "location" -> + case http_request:is_absolut_uri(FieldValue) of + true -> + handle_headers(CGIHeaders, + [{FieldName, FieldValue} | + HTTPHeaders], {302, "Redirect"}); + false -> + {proceed, FieldValue} + end; + "status" -> + CodePhrase = + case httpd_util:split(FieldValue," ",2) of + {ok,[Code, Phrase]} -> + {list_to_integer(Code), Phrase}; + _ -> + {200, "OK"} + end, + handle_headers(CGIHeaders, HTTPHeaders, CodePhrase); + _ -> %% Extension headers + handle_headers(CGIHeaders, + [{FieldName, FieldValue} | HTTPHeaders], Status) + end. + diff --git a/lib/inets/src/http_server/httpd_conf.erl b/lib/inets/src/http_server/httpd_conf.erl new file mode 100644 index 0000000000..9c93e2c5fe --- /dev/null +++ b/lib/inets/src/http_server/httpd_conf.erl @@ -0,0 +1,1119 @@ +%% +%% %CopyrightBegin% +%% +%% Copyright Ericsson AB 1997-2009. All Rights Reserved. +%% +%% The contents of this file are subject to the Erlang Public License, +%% Version 1.1, (the "License"); you may not use this file except in +%% compliance with the License. You should have received a copy of the +%% Erlang Public License along with this software. If not, it can be +%% retrieved online at http://www.erlang.org/. +%% +%% Software distributed under the License is distributed on an "AS IS" +%% basis, WITHOUT WARRANTY OF ANY KIND, either express or implied. See +%% the License for the specific language governing rights and limitations +%% under the License. +%% +%% %CopyrightEnd% +%% +%% +-module(httpd_conf). + +%% EWSAPI +-export([is_directory/1, is_file/1, make_integer/1, clean/1, + custom_clean/3, check_enum/2]). + +%% Application internal API +-export([load/1, load/2, load_mime_types/1, store/1, store/2, + remove/1, remove_all/1, config/1, get_config/2, get_config/3, + lookup/2, lookup/3, lookup/4, + validate_properties/1]). + +-define(VMODULE,"CONF"). +-include("httpd.hrl"). +-include("httpd_internal.hrl"). + + +%%%========================================================================= +%%% EWSAPI +%%%========================================================================= +%%------------------------------------------------------------------------- +%% is_directory(FilePath) -> Result +%% FilePath = string() +%% Result = {ok,Directory} | {error,Reason} +%% Directory = string() +%% Reason = string() | enoent | eaccess | enotdir | FileInfo +%% FileInfo = File info record +%% +%% Description: Checks if FilePath is a directory in which case it is +%% returned. +%%------------------------------------------------------------------------- +is_directory(Directory) -> + case file:read_file_info(Directory) of + {ok,FileInfo} -> + #file_info{type = Type, access = Access} = FileInfo, + is_directory(Type,Access,FileInfo,Directory); + {error,Reason} -> + {error,Reason} + end. +is_directory(directory,read,_FileInfo,Directory) -> + {ok,Directory}; +is_directory(directory,read_write,_FileInfo,Directory) -> + {ok,Directory}; +is_directory(_Type,_Access,FileInfo,_Directory) -> + {error,FileInfo}. + + +%%------------------------------------------------------------------------- +%% is_file(FilePath) -> Result +%% FilePath = string() +%% Result = {ok,File} | {error,Reason} +%% File = string() +%% Reason = string() | enoent | eaccess | enotdir | FileInfo +%% FileInfo = File info record +%% +%% Description: Checks if FilePath is a regular file in which case it +%% is returned. +%%------------------------------------------------------------------------- +is_file(File) -> + case file:read_file_info(File) of + {ok,FileInfo} -> + #file_info{type = Type, access = Access} = FileInfo, + is_file(Type,Access,FileInfo,File); + {error,Reason} -> + {error,Reason} + end. +is_file(regular,read,_FileInfo,File) -> + {ok,File}; +is_file(regular,read_write,_FileInfo,File) -> + {ok,File}; +is_file(_Type,_Access,FileInfo,_File) -> + {error,FileInfo}. + + +%%------------------------------------------------------------------------- +%% make_integer(String) -> Result +%% String = string() +%% Result = {ok,integer()} | {error,nomatch} +%% +%% Description: make_integer/1 returns an integer representation of String. +%%------------------------------------------------------------------------- +make_integer(String) -> + case inets_regexp:match(clean(String),"[0-9]+") of + {match, _, _} -> + {ok, list_to_integer(clean(String))}; + nomatch -> + {error, nomatch} + end. + + +%%------------------------------------------------------------------------- +%% clean(String) -> Stripped +%% String = Stripped = string() +%% +%% Description:clean/1 removes leading and/or trailing white spaces +%% from String. +%%------------------------------------------------------------------------- +clean(String) -> + {ok,CleanedString,_} = + inets_regexp:gsub(String, "^[ \t\n\r\f]*|[ \t\n\r\f]*\$",""), + CleanedString. + + +%%------------------------------------------------------------------------- +%% custom_clean(String,Before,After) -> Stripped +%% Before = After = regexp() +%% String = Stripped = string() +%% +%% Description: custom_clean/3 removes leading and/or trailing white +%% spaces and custom characters from String. +%%------------------------------------------------------------------------- +custom_clean(String,MoreBefore,MoreAfter) -> + {ok,CleanedString,_} = inets_regexp:gsub(String,"^[ \t\n\r\f"++MoreBefore++ + "]*|[ \t\n\r\f"++MoreAfter++"]*\$",""), + CleanedString. + + +%%------------------------------------------------------------------------- +%% check_enum(EnumString,ValidEnumStrings) -> Result +%% EnumString = string() +%% ValidEnumStrings = [string()] +%% Result = {ok,atom()} | {error,not_valid} +%% +%% Description: check_enum/2 checks if EnumString is a valid +%% enumeration of ValidEnumStrings in which case it is returned as an +%% atom. +%%------------------------------------------------------------------------- +check_enum(_Enum,[]) -> + {error, not_valid}; +check_enum(Enum,[Enum|_Rest]) -> + {ok, list_to_atom(Enum)}; +check_enum(Enum, [_NotValid|Rest]) -> + check_enum(Enum, Rest). + + +%%%========================================================================= +%%% Application internal API +%%%========================================================================= +%% The configuration data is handled in three (3) phases: +%% 1. Parse the config file and put all directives into a key-vale +%% tuple list (load/1). +%% 2. Traverse the key-value tuple list store it into an ETS table. +%% Directives depending on other directives are taken care of here +%% (store/1). +%% 3. Traverse the ETS table and do a complete clean-up (remove/1). + +%% Phase 1: Load +load(ConfigFile) -> + ?hdrv("load config", [{config_file, ConfigFile}]), + case read_config_file(ConfigFile) of + {ok, Config} -> + ?hdrt("config read", []), + case bootstrap(Config) of + {error, Reason} -> + ?hdri("bootstrap failed", [{reason, Reason}]), + {error, Reason}; + {ok, Modules} -> + ?hdrd("config bootstrapped", [{modules, Modules}]), + load_config(Config, lists:append(Modules, [?MODULE])) + end; + {error, Reason} -> + ?hdri("failed reading config file", [{reason, Reason}]), + {error, ?NICE("Error while reading config file: "++Reason)} + end. + +load(eof, []) -> + eof; + +load("MaxHeaderSize " ++ MaxHeaderSize, []) -> + case make_integer(MaxHeaderSize) of + {ok, Integer} -> + {ok, [], {max_header_size,Integer}}; + {error, _} -> + {error, ?NICE(clean(MaxHeaderSize)++ + " is an invalid number of MaxHeaderSize")} + end; + +load("MaxURISize " ++ MaxHeaderSize, []) -> + case make_integer(MaxHeaderSize) of + {ok, Integer} -> + {ok, [], {max_uri_size, Integer}}; + {error, _} -> + {error, ?NICE(clean(MaxHeaderSize)++ + " is an invalid number of MaxHeaderSize")} + end; + +load("MaxBodySize " ++ MaxBodySize, []) -> + case make_integer(MaxBodySize) of + {ok, Integer} -> + {ok, [], {max_body_size,Integer}}; + {error, _} -> + {error, ?NICE(clean(MaxBodySize)++ + " is an invalid number of MaxBodySize")} + end; + +load("ServerName " ++ ServerName, []) -> + {ok,[],{server_name,clean(ServerName)}}; + +load("SocketType " ++ SocketType, []) -> + case check_enum(clean(SocketType),["ssl","ip_comm"]) of + {ok, ValidSocketType} -> + {ok, [], {socket_type,ValidSocketType}}; + {error,_} -> + {error, ?NICE(clean(SocketType) ++ " is an invalid SocketType")} + end; + +load("Port " ++ Port, []) -> + case make_integer(Port) of + {ok, Integer} -> + {ok, [], {port,Integer}}; + {error, _} -> + {error, ?NICE(clean(Port)++" is an invalid Port")} + end; + +load("BindAddress " ++ Address0, []) -> + %% If an ipv6 address is provided in URL-syntax strip the + %% url specific part e.i. "[FEDC:BA98:7654:3210:FEDC:BA98:7654:3210]" + %% -> "FEDC:BA98:7654:3210:FEDC:BA98:7654:3210" + + try + begin + ?hdrv("load BindAddress", [{address0, Address0}]), + {Address, IpFamily} = + case string:tokens(Address0, [$|]) of + [Address1] -> + ?hdrv("load BindAddress", [{address1, Address1}]), + {clean_address(Address1), inet6fb4}; + [Address1, IpFamilyStr] -> + ?hdrv("load BindAddress", + [{address1, Address1}, + {ipfamily_str, IpFamilyStr}]), + {clean_address(Address1), make_ipfamily(IpFamilyStr)}; + _Bad -> + ?hdrv("load BindAddress - bad address", + [{bad_address, _Bad}]), + throw({error, {bad_bind_address, Address0}}) + end, + + ?hdrv("load BindAddress - address and ipfamily separated", + [{address, Address}, {ipfamily, IpFamily}]), + + case Address of + "*" -> + {ok, [], [{bind_address, any}, {ipfamily, IpFamily}]}; + _ -> + case httpd_util:ip_address(Address, IpFamily) of + {ok, IPAddr} -> + ?hdrv("load BindAddress - checked", + [{ip_address, IPAddr}]), + Entries = [{bind_address, IPAddr}, + {ipfamily, IpFamily}], + {ok, [], Entries}; + {error, _} -> + {error, ?NICE(Address ++ " is an invalid address")} + end + end + end + catch + throw:{error, {bad_bind_address, _}} -> + ?hdrv("load BindAddress - bad bind address", []), + {error, ?NICE(Address0 ++ " is an invalid address")}; + throw:{error, {bad_ipfamily, _}} -> + ?hdrv("load BindAddress - bad ipfamily", []), + {error, ?NICE(Address0 ++ " has an invalid ipfamily")} + end; + +load("KeepAlive " ++ OnorOff, []) -> + case list_to_atom(clean(OnorOff)) of + off -> + {ok, [], {keep_alive, false}}; + _ -> + {ok, [], {keep_alive, true}} + end; + +load("MaxKeepAliveRequests " ++ MaxRequests, []) -> + case make_integer(MaxRequests) of + {ok, Integer} -> + {ok, [], {max_keep_alive_request, Integer}}; + {error, _} -> + {error, ?NICE(clean(MaxRequests) ++ + " is an invalid MaxKeepAliveRequests")} + end; + +%% This clause is keept for backwards compability +load("MaxKeepAliveRequest " ++ MaxRequests, []) -> + case make_integer(MaxRequests) of + {ok, Integer} -> + {ok, [], {max_keep_alive_request, Integer}}; + {error, _} -> + {error, ?NICE(clean(MaxRequests) ++ + " is an invalid MaxKeepAliveRequest")} + end; + +load("KeepAliveTimeout " ++ Timeout, []) -> + case make_integer(Timeout) of + {ok, Integer} -> + {ok, [], {keep_alive_timeout, Integer*1000}}; + {error, _} -> + {error, ?NICE(clean(Timeout)++" is an invalid KeepAliveTimeout")} + end; + +load("Modules " ++ Modules, []) -> + {ok, ModuleList} = inets_regexp:split(Modules," "), + {ok, [], {modules,[list_to_atom(X) || X <- ModuleList]}}; + +load("ServerAdmin " ++ ServerAdmin, []) -> + {ok, [], {server_admin,clean(ServerAdmin)}}; + +load("ServerRoot " ++ ServerRoot, []) -> + case is_directory(clean(ServerRoot)) of + {ok, Directory} -> + {ok, [], [{server_root,string:strip(Directory,right,$/)}]}; + {error, _} -> + {error, ?NICE(clean(ServerRoot)++" is an invalid ServerRoot")} + end; + +load("MimeTypes " ++ MimeTypes, []) -> + case load_mime_types(clean(MimeTypes)) of + {ok, MimeTypesList} -> + {ok, [], [{mime_types, MimeTypesList}]}; + {error, Reason} -> + {error, Reason} + end; + +load("MaxClients " ++ MaxClients, []) -> + case make_integer(MaxClients) of + {ok, Integer} -> + {ok, [], {max_clients,Integer}}; + {error, _} -> + {error, ?NICE(clean(MaxClients) ++ + " is an invalid number of MaxClients")} + end; +load("DocumentRoot " ++ DocumentRoot,[]) -> + case is_directory(clean(DocumentRoot)) of + {ok, Directory} -> + {ok, [], {document_root,string:strip(Directory,right,$/)}}; + {error, _} -> + {error, ?NICE(clean(DocumentRoot)++"is an invalid DocumentRoot")} + end; +load("DefaultType " ++ DefaultType, []) -> + {ok, [], {default_type,clean(DefaultType)}}; +load("SSLCertificateFile " ++ SSLCertificateFile, []) -> + case is_file(clean(SSLCertificateFile)) of + {ok, File} -> + {ok, [], {ssl_certificate_file,File}}; + {error, _} -> + {error, ?NICE(clean(SSLCertificateFile)++ + " is an invalid SSLCertificateFile")} + end; +load("SSLCertificateKeyFile " ++ SSLCertificateKeyFile, []) -> + case is_file(clean(SSLCertificateKeyFile)) of + {ok, File} -> + {ok, [], {ssl_certificate_key_file,File}}; + {error, _} -> + {error, ?NICE(clean(SSLCertificateKeyFile)++ + " is an invalid SSLCertificateKeyFile")} + end; +load("SSLVerifyClient " ++ SSLVerifyClient, []) -> + case make_integer(clean(SSLVerifyClient)) of + {ok, Integer} when (Integer >=0) andalso (Integer =< 2) -> + {ok, [], {ssl_verify_client,Integer}}; + {ok, _Integer} -> + {error,?NICE(clean(SSLVerifyClient) ++ + " is an invalid SSLVerifyClient")}; + {error, nomatch} -> + {error,?NICE(clean(SSLVerifyClient) ++ + " is an invalid SSLVerifyClient")} + end; +load("SSLVerifyDepth " ++ SSLVerifyDepth, []) -> + case make_integer(clean(SSLVerifyDepth)) of + {ok, Integer} when Integer > 0 -> + {ok, [], {ssl_verify_client_depth,Integer}}; + {ok, _Integer} -> + {error,?NICE(clean(SSLVerifyDepth) ++ + " is an invalid SSLVerifyDepth")}; + {error, nomatch} -> + {error,?NICE(clean(SSLVerifyDepth) ++ + " is an invalid SSLVerifyDepth")} + end; +load("SSLCiphers " ++ SSLCiphers, []) -> + {ok, [], {ssl_ciphers, clean(SSLCiphers)}}; +load("SSLCACertificateFile " ++ SSLCACertificateFile, []) -> + case is_file(clean(SSLCACertificateFile)) of + {ok, File} -> + {ok, [], {ssl_ca_certificate_file,File}}; + {error, _} -> + {error, ?NICE(clean(SSLCACertificateFile)++ + " is an invalid SSLCACertificateFile")} + end; +load("SSLPasswordCallbackModule " ++ SSLPasswordCallbackModule, []) -> + {ok, [], {ssl_password_callback_module, + list_to_atom(clean(SSLPasswordCallbackModule))}}; +load("SSLPasswordCallbackFunction " ++ SSLPasswordCallbackFunction, []) -> + {ok, [], {ssl_password_callback_function, + list_to_atom(clean(SSLPasswordCallbackFunction))}}; +load("SSLPasswordCallbackArguments " ++ SSLPasswordCallbackArguments, []) -> + {ok, [], {ssl_password_callback_arguments, + SSLPasswordCallbackArguments}}; +load("DisableChunkedTransferEncodingSend " ++ TrueOrFalse, []) -> + case list_to_atom(clean(TrueOrFalse)) of + true -> + {ok, [], {disable_chunked_transfer_encoding_send, true}}; + _ -> + {ok, [], {disable_chunked_transfer_encoding_send, false}} + end; +load("LogFormat " ++ LogFormat, []) -> + {ok,[],{log_format, list_to_atom(httpd_conf:clean(LogFormat))}}; +load("ErrorLogFormat " ++ LogFormat, []) -> + {ok,[],{error_log_format, list_to_atom(httpd_conf:clean(LogFormat))}}. + + +clean_address(Addr) -> + string:strip(string:strip(clean(Addr), left, $[), right, $]). + + +make_ipfamily(IpFamilyStr) -> + IpFamily = list_to_atom(IpFamilyStr), + case lists:member(IpFamily, [inet, inet6, inet6fb4]) of + true -> + IpFamily; + false -> + throw({error, {bad_ipfamily, IpFamilyStr}}) + end. + + +%% +%% load_mime_types/1 -> {ok, MimeTypes} | {error, Reason} +%% +load_mime_types(MimeTypesFile) -> + case file:open(MimeTypesFile, [read]) of + {ok, Stream} -> + parse_mime_types(Stream, []); + {error, _} -> + {error, ?NICE("Can't open " ++ MimeTypesFile)} + end. + + +validate_properties(Properties) -> + %% First, check that all mandatory properties are present + case mandatory_properties(Properties) of + ok -> + %% Second, check that property dependency are ok + {ok, validate_properties2(Properties)}; + Error -> + throw(Error) + end. + +%% This function is used to validate inter-property dependencies. +%% That is, if property A depends on property B. +%% The only sunch preperty at this time is bind_address that depends +%% on ipfamily. +validate_properties2(Properties) -> + case proplists:get_value(bind_address, Properties) of + undefined -> + case proplists:get_value(sock_type, Properties, ip_comm) of + ip_comm -> + case proplists:get_value(ipfamily, Properties) of + undefined -> + [{bind_address, any}, + {ipfamily, inet6fb4} | Properties]; + _ -> + [{bind_address, any} | Properties] + end; + _ -> + [{bind_address, any} | Properties] + end; + any -> + Properties; + Address0 -> + IpFamily = proplists:get_value(ipfamily, Properties, inet6fb4), + case httpd_util:ip_address(Address0, IpFamily) of + {ok, Address} -> + Properties1 = proplists:delete(bind_address, Properties), + [{bind_address, Address} | Properties1]; + {error, Reason} -> + Error = {error, + {failed_determine_ip_address, + Address0, IpFamily, Reason}}, + throw(Error) + end + end. + +mandatory_properties(ConfigList) -> + a_must(ConfigList, [server_name, port, server_root, document_root]). + +a_must(_ConfigList, []) -> + ok; +a_must(ConfigList, [Prop | Rest]) -> + case proplists:get_value(Prop, ConfigList) of + undefined -> + {error, {missing_property, Prop}}; + _ -> + a_must(ConfigList, Rest) + end. + + +validate_config_params([]) -> + ok; +validate_config_params([{max_header_size, Value} | Rest]) + when is_integer(Value) andalso (Value > 0) -> + validate_config_params(Rest); +validate_config_params([{max_header_size, Value} | _]) -> + throw({max_header_size, Value}); + +validate_config_params([{max_body_size, Value} | Rest]) + when is_integer(Value) andalso (Value > 0) -> + validate_config_params(Rest); +validate_config_params([{max_body_size, Value} | _]) -> + throw({max_body_size, Value}); + +validate_config_params([{server_name, Value} | Rest]) + when is_list(Value) -> + validate_config_params(Rest); +validate_config_params([{server_name, Value} | _]) -> + throw({server_name, Value}); + +validate_config_params([{socket_type, Value} | Rest]) + when (Value =:= ip_comm) orelse (Value =:= ssl) -> + validate_config_params(Rest); +validate_config_params([{socket_type, Value} | _]) -> + throw({socket_type, Value}); + +validate_config_params([{port, Value} | Rest]) + when is_integer(Value) andalso (Value >= 0) -> + validate_config_params(Rest); +validate_config_params([{port, Value} | _]) -> + throw({port, Value}); + +validate_config_params([{bind_address, Value} | Rest]) -> + case is_bind_address(Value) of + true -> + validate_config_params(Rest); + false -> + throw({bind_address, Value}) + end; + +validate_config_params([{ipfamily, Value} | Rest]) + when ((Value =:= inet) orelse + (Value =:= inet6) orelse + (Value =:= inet6fb4)) -> + validate_config_params(Rest); +validate_config_params([{ipfamily, Value} | _]) -> + throw({ipfamily, Value}); + +validate_config_params([{keep_alive, Value} | Rest]) + when (Value =:= true) orelse (Value =:= false) -> + validate_config_params(Rest); +validate_config_params([{keep_alive, Value} | _]) -> + throw({keep_alive, Value}); + +validate_config_params([{max_keep_alive_request, Value} | Rest]) + when is_integer(Value) andalso (Value > 0) -> + validate_config_params(Rest); +validate_config_params([{max_keep_alive_request, Value} | _]) -> + throw({max_header_size, Value}); + +validate_config_params([{keep_alive_timeout, Value} | Rest]) + when is_integer(Value) andalso (Value >= 0) -> + validate_config_params(Rest); +validate_config_params([{keep_alive_timeout, Value} | _]) -> + throw({keep_alive_timeout, Value}); + +validate_config_params([{modules, Value} | Rest]) -> + ok = httpd_util:modules_validate(Value), + validate_config_params(Rest); + +validate_config_params([{server_admin, Value} | Rest]) when is_list(Value) -> + validate_config_params(Rest); +validate_config_params([{server_admin, Value} | _]) -> + throw({server_admin, Value}); + +validate_config_params([{server_root, Value} | Rest]) -> + ok = httpd_util:dir_validate(server_root, Value), + validate_config_params(Rest); + +validate_config_params([{mime_types, Value} | Rest]) -> + ok = httpd_util:mime_types_validate(Value), + validate_config_params(Rest); + +validate_config_params([{max_clients, Value} | Rest]) + when is_integer(Value) andalso (Value > 0) -> + validate_config_params(Rest); +validate_config_params([{max_clients, Value} | _]) -> + throw({max_clients, Value}); + +validate_config_params([{document_root, Value} | Rest]) -> + ok = httpd_util:dir_validate(document_root, Value), + validate_config_params(Rest); + +validate_config_params([{default_type, Value} | Rest]) when is_list(Value) -> + validate_config_params(Rest); +validate_config_params([{default_type, Value} | _]) -> + throw({default_type, Value}); + +validate_config_params([{ssl_certificate_file = Key, Value} | Rest]) -> + ok = httpd_util:file_validate(Key, Value), + validate_config_params(Rest); + +validate_config_params([{ssl_certificate_key_file = Key, Value} | Rest]) -> + ok = httpd_util:file_validate(Key, Value), + validate_config_params(Rest); + +validate_config_params([{ssl_verify_client, Value} | Rest]) + when (Value =:= 0) orelse (Value =:= 1) orelse (Value =:= 2) -> + validate_config_params(Rest); + +validate_config_params([{ssl_verify_client_depth, Value} | Rest]) + when is_integer(Value) andalso (Value >= 0) -> + validate_config_params(Rest); +validate_config_params([{ssl_verify_client_depth, Value} | _]) -> + throw({ssl_verify_client_depth, Value}); + +validate_config_params([{ssl_ciphers, Value} | Rest]) when is_list(Value) -> + validate_config_params(Rest); +validate_config_params([{ssl_ciphers, Value} | _]) -> + throw({ssl_ciphers, Value}); + +validate_config_params([{ssl_ca_certificate_file = Key, Value} | Rest]) -> + ok = httpd_util:file_validate(Key, Value), + validate_config_params(Rest); + +validate_config_params([{ssl_password_callback_module, Value} | Rest]) + when is_atom(Value) -> + validate_config_params(Rest); +validate_config_params([{ssl_password_callback_module, Value} | _]) -> + throw({ssl_password_callback_module, Value}); + +validate_config_params([{ssl_password_callback_function, Value} | Rest]) + when is_atom(Value) -> + validate_config_params(Rest); +validate_config_params([{ssl_password_callback_function, Value} | _]) -> + throw({ssl_password_callback_function, Value}); + +validate_config_params([{ssl_password_callback_arguments, Value} | Rest]) + when is_list(Value) -> + validate_config_params(Rest); +validate_config_params([{ssl_password_callback_arguments, Value} | _]) -> + throw({ssl_password_callback_arguments, Value}); + +validate_config_params([{disable_chunked_transfer_encoding_send, Value} | + Rest]) + when (Value =:= true) orelse (Value =:= false) -> + validate_config_params(Rest); +validate_config_params([{disable_chunked_transfer_encoding_send, Value} | + _ ]) -> + throw({disable_chunked_transfer_encoding_send, Value}); +validate_config_params([_| Rest]) -> + validate_config_params(Rest). + +%% It is actually pointless to check bind_address in this way since +%% we need ipfamily to do it properly... +is_bind_address(any) -> + true; +is_bind_address(Value) -> + case httpd_util:ip_address(Value, inet6fb4) of + {ok, _} -> + true; + _ -> + false + end. + +store(ConfigList0) -> + ?hdrd("store", []), + try validate_config_params(ConfigList0) of + ok -> + Modules = + proplists:get_value(modules, ConfigList0, ?DEFAULT_MODS), + ?hdrt("store", [{modules, Modules}]), + Port = proplists:get_value(port, ConfigList0), + Addr = proplists:get_value(bind_address, ConfigList0, any), + ConfigList = fix_mime_types(ConfigList0), + Name = httpd_util:make_name("httpd_conf", Addr, Port), + ConfigDB = ets:new(Name, [named_table, bag, protected]), + store(ConfigDB, ConfigList, + lists:append(Modules, [?MODULE]), + ConfigList) + catch + throw:Error -> + {error, {invalid_option, Error}} + end. + +fix_mime_types(ConfigList0) -> + case proplists:get_value(mime_types, ConfigList0) of + undefined -> + ServerRoot = proplists:get_value(server_root, ConfigList0), + MimeTypesFile = + filename:join([ServerRoot,"conf", "mime.types"]), + case filelib:is_file(MimeTypesFile) of + true -> + {ok, MimeTypesList} = load_mime_types(MimeTypesFile), + [{mime_types, MimeTypesList} | ConfigList0]; + false -> + [{mime_types, + [{"html","text/html"},{"htm","text/html"}]} + | ConfigList0] + end; + _ -> + ConfigList0 + end. + +store({mime_types,MimeTypesList},ConfigList) -> + Port = proplists:get_value(port, ConfigList), + Addr = proplists:get_value(bind_address, ConfigList), + Name = httpd_util:make_name("httpd_mime",Addr,Port), + {ok, MimeTypesDB} = store_mime_types(Name,MimeTypesList), + {ok, {mime_types,MimeTypesDB}}; +store({log_format, LogFormat}, _ConfigList) + when (LogFormat =:= common) orelse (LogFormat =:= combined) -> + {ok,{log_format, LogFormat}}; +store({log_format, LogFormat}, _ConfigList) + when (LogFormat =:= compact) orelse (LogFormat =:= pretty) -> + {ok, {log_format, LogFormat}}; +store(ConfigListEntry, _ConfigList) -> + {ok, ConfigListEntry}. + +%% Phase 3: Remove +remove_all(ConfigDB) -> + Modules = httpd_util:lookup(ConfigDB,modules,[]), + remove_traverse(ConfigDB, lists:append(Modules,[?MODULE])). + +remove(ConfigDB) -> + ets:delete(ConfigDB), + ok. + +config(ConfigDB) -> + case httpd_util:lookup(ConfigDB, socket_type,ip_comm) of + ssl -> + case ssl_certificate_file(ConfigDB) of + undefined -> + {error, + "Directive SSLCertificateFile " + "not found in the config file"}; + SSLCertificateFile -> + {ssl, + SSLCertificateFile++ + ssl_certificate_key_file(ConfigDB)++ + ssl_verify_client(ConfigDB)++ + ssl_ciphers(ConfigDB)++ + ssl_password(ConfigDB)++ + ssl_verify_depth(ConfigDB)++ + ssl_ca_certificate_file(ConfigDB)} + end; + ip_comm -> + ip_comm + end. + + +get_config(Address, Port) -> + Tab = httpd_util:make_name("httpd_conf", Address, Port), + Properties = ets:tab2list(Tab), + MimeTab = proplists:get_value(mime_types, Properties), + NewProperties = proplists:delete(mime_types, Properties), + [{mime_types, ets:tab2list(MimeTab)} | NewProperties]. + +get_config(Address, Port, Properties) -> + Tab = httpd_util:make_name("httpd_conf", Address, Port), + Config = + lists:map(fun(Prop) -> {Prop, httpd_util:lookup(Tab, Prop)} end, + Properties), + [{Proporty, Value} || {Proporty, Value} <- Config, Value =/= undefined]. + + +lookup(Tab, Key) -> + httpd_util:lookup(Tab, Key). + +lookup(Tab, Key, Default) when is_atom(Key) -> + httpd_util:lookup(Tab, Key, Default); + +lookup(Address, Port, Key) when is_integer(Port) -> + Tab = table(Address, Port), + lookup(Tab, Key). + +lookup(Address, Port, Key, Default) when is_integer(Port) -> + Tab = table(Address, Port), + lookup(Tab, Key, Default). + +table(Address, Port) -> + httpd_util:make_name("httpd_conf", Address, Port). + + +%%%======================================================================== +%%% Internal functions +%%%======================================================================== +%%% Phase 1 Load: +bootstrap([]) -> + {ok, ?DEFAULT_MODS}; +bootstrap([Line|Config]) -> + case Line of + "Modules " ++ Modules -> + {ok, ModuleList} = inets_regexp:split(Modules," "), + TheMods = [list_to_atom(X) || X <- ModuleList], + case verify_modules(TheMods) of + ok -> + {ok, TheMods}; + {error, Reason} -> + {error, Reason} + end; + _ -> + bootstrap(Config) + end. + +load_config(Config, Modules) -> + %% Create default contexts for all modules + Contexts = lists:duplicate(length(Modules), []), + load_config(Config, Modules, Contexts, []). + +load_config([], _Modules, _Contexts, ConfigList) -> + ?hdrv("config loaded", []), + {ok, ConfigList}; + +load_config([Line|Config], Modules, Contexts, ConfigList) -> + ?hdrt("load config", [{config_line, Line}]), + case load_traverse(Line, Contexts, Modules, [], ConfigList, no) of + {ok, NewContexts, NewConfigList} -> + load_config(Config, Modules, NewContexts, NewConfigList); + {error, Reason} -> + {error, Reason} + end. + + +%% This loads the config file into each module specified by Modules +%% Each module has its own context that is passed to and (optionally) +%% returned by the modules load function. The module can also return +%% a ConfigEntry, which will be added to the global configuration +%% list. +%% All configuration directives are guaranteed to be passed to all +%% modules. Each module only implements the function clauses of +%% the load function for the configuration directives it supports, +%% it's ok if an apply returns {'EXIT', {function_clause, ..}}. +load_traverse(Line, [], [], _NewContexts, _ConfigList, no) -> + {error, ?NICE("Configuration directive not recognized: "++Line)}; +load_traverse(_Line, [], [], NewContexts, ConfigList, yes) -> + {ok, lists:reverse(NewContexts), ConfigList}; +load_traverse(Line, [Context|Contexts], [Module|Modules], NewContexts, + ConfigList, State) -> + ?hdrt("load config traverse", + [{context, Context}, {httpd_module, Module}, {state, State}]), + case catch apply(Module, load, [Line, Context]) of + {'EXIT', {function_clause, _FC}} -> + ?hdrt("does not handle load config", + [{config_line, Line}, {fc, _FC}]), + load_traverse(Line, Contexts, Modules, + [Context|NewContexts], ConfigList, State); + + {'EXIT', {undef, _}} -> + ?hdrt("does not implement load", []), + load_traverse(Line, Contexts, Modules, + [Context|NewContexts], ConfigList,yes); + + {'EXIT', Reason} -> + error_logger:error_report({'EXIT', Reason}), + load_traverse(Line, Contexts, Modules, + [Context|NewContexts], ConfigList, State); + + {ok, NewContext} -> + ?hdrt("line processed", [{new_context, NewContext}]), + load_traverse(Line, Contexts, Modules, + [NewContext|NewContexts], ConfigList,yes); + + {ok, NewContext, ConfigEntry} when is_tuple(ConfigEntry) -> + ?hdrt("line processed", + [{new_context, NewContext}, {config_entry, ConfigEntry}]), + load_traverse(Line, Contexts, + Modules, [NewContext|NewContexts], + [ConfigEntry|ConfigList], yes); + + {ok, NewContext, ConfigEntry} when is_list(ConfigEntry) -> + ?hdrt("line processed", + [{new_context, NewContext}, {config_entry, ConfigEntry}]), + load_traverse(Line, Contexts, Modules, [NewContext|NewContexts], + lists:append(ConfigEntry, ConfigList), yes); + + {error, Reason} -> + ?hdrv("line processing failed", [{reason, Reason}]), + {error, Reason} + end. + +%% Verifies that all specified modules are available. +verify_modules([]) -> + ok; +verify_modules([Mod|Rest]) -> + case code:which(Mod) of + non_existing -> + {error, ?NICE(atom_to_list(Mod)++" does not exist")}; + _Path -> + verify_modules(Rest) + end. + +%% Reads the entire configuration file and returns list of strings or +%% and error. +read_config_file(FileName) -> + case file:open(FileName, [read]) of + {ok, Stream} -> + read_config_file(Stream, []); + {error, _Reason} -> + {error, ?NICE("Cannot open "++FileName)} + end. +read_config_file(Stream, SoFar) -> + case io:get_line(Stream, []) of + eof -> + file:close(Stream), + {ok, lists:reverse(SoFar)}; + {error, Reason} -> + file:close(Stream), + {error, Reason}; + [$#|_Rest] -> + %% Ignore commented lines for efficiency later .. + read_config_file(Stream, SoFar); + Line -> + {ok, NewLine, _}=inets_regexp:sub(clean(Line),"[\t\r\f ]"," "), + case NewLine of + [] -> + %% Also ignore empty lines .. + read_config_file(Stream, SoFar); + _Other -> + read_config_file(Stream, [NewLine|SoFar]) + end + end. + +parse_mime_types(Stream,MimeTypesList) -> + Line= + case io:get_line(Stream,'') of + eof -> + eof; + String -> + clean(String) + end, + parse_mime_types(Stream, MimeTypesList, Line). +parse_mime_types(Stream, MimeTypesList, eof) -> + file:close(Stream), + {ok, MimeTypesList}; +parse_mime_types(Stream, MimeTypesList, "") -> + parse_mime_types(Stream, MimeTypesList); +parse_mime_types(Stream, MimeTypesList, [$#|_]) -> + parse_mime_types(Stream, MimeTypesList); +parse_mime_types(Stream, MimeTypesList, Line) -> + case inets_regexp:split(Line, " ") of + {ok, [NewMimeType|Suffixes]} -> + parse_mime_types(Stream, + lists:append(suffixes(NewMimeType,Suffixes), + MimeTypesList)); + {ok, _} -> + {error, ?NICE(Line)} + end. + +suffixes(_MimeType,[]) -> + []; +suffixes(MimeType,[Suffix|Rest]) -> + [{Suffix,MimeType}|suffixes(MimeType,Rest)]. + + +%% Phase 2: store +store(ConfigDB, _ConfigList, _Modules, []) -> + {ok, ConfigDB}; +store(ConfigDB, ConfigList, Modules, [ConfigListEntry|Rest]) -> + ?hdrt("store", [{entry, ConfigListEntry}]), + case store_traverse(ConfigListEntry, ConfigList, Modules) of + {ok, ConfigDBEntry} when is_tuple(ConfigDBEntry) -> + ets:insert(ConfigDB, ConfigDBEntry), + store(ConfigDB, ConfigList, Modules, Rest); + {ok, ConfigDBEntry} when is_list(ConfigDBEntry) -> + lists:foreach(fun(Entry) -> + ets:insert(ConfigDB,Entry) + end,ConfigDBEntry), + store(ConfigDB, ConfigList, Modules, Rest); + {error, Reason} -> + {error,Reason} + end. + +store_traverse(_ConfigListEntry, _ConfigList,[]) -> + {error, ?NICE("Unable to store configuration...")}; +store_traverse(ConfigListEntry, ConfigList, [Module|Rest]) -> + ?hdrt("store traverse", + [{httpd_module, Module}, {entry, ConfigListEntry}]), + case catch apply(Module, store, [ConfigListEntry, ConfigList]) of + {'EXIT',{function_clause,_}} -> + ?hdrt("does not handle store config", []), + store_traverse(ConfigListEntry,ConfigList,Rest); + {'EXIT',{undef, _}} -> + ?hdrt("does not implement store", []), + store_traverse(ConfigListEntry,ConfigList,Rest); + {'EXIT', Reason} -> + error_logger:error_report({'EXIT',Reason}), + store_traverse(ConfigListEntry,ConfigList,Rest); + Result -> + ?hdrt("config entry processed", [{result, Result}]), + Result + end. + +store_mime_types(Name,MimeTypesList) -> + %% Make sure that the ets table is not duplicated + %% when reloading configuration + catch ets:delete(Name), + MimeTypesDB = ets:new(Name, [named_table, set, protected]), + store_mime_types1(MimeTypesDB, MimeTypesList). +store_mime_types1(MimeTypesDB,[]) -> + {ok, MimeTypesDB}; +store_mime_types1(MimeTypesDB,[Type|Rest]) -> + ets:insert(MimeTypesDB, Type), + store_mime_types1(MimeTypesDB, Rest). + + +%% Phase 3: remove +remove_traverse(_ConfigDB,[]) -> + ok; +remove_traverse(ConfigDB,[Module|Rest]) -> + case (catch apply(Module,remove,[ConfigDB])) of + {'EXIT',{undef,_}} -> + remove_traverse(ConfigDB,Rest); + {'EXIT',{function_clause,_}} -> + remove_traverse(ConfigDB,Rest); + {'EXIT',Reason} -> + error_logger:error_report({'EXIT',Reason}), + remove_traverse(ConfigDB,Rest); + {error,Reason} -> + error_logger:error_report(Reason), + remove_traverse(ConfigDB,Rest); + _ -> + remove_traverse(ConfigDB,Rest) + end. + +ssl_certificate_file(ConfigDB) -> + case httpd_util:lookup(ConfigDB,ssl_certificate_file) of + undefined -> + undefined; + SSLCertificateFile -> + [{certfile,SSLCertificateFile}] + end. + +ssl_certificate_key_file(ConfigDB) -> + case httpd_util:lookup(ConfigDB,ssl_certificate_key_file) of + undefined -> + []; + SSLCertificateKeyFile -> + [{keyfile,SSLCertificateKeyFile}] + end. + +ssl_verify_client(ConfigDB) -> + case httpd_util:lookup(ConfigDB,ssl_verify_client) of + undefined -> + []; + SSLVerifyClient -> + [{verify,SSLVerifyClient}] + end. + +ssl_ciphers(ConfigDB) -> + case httpd_util:lookup(ConfigDB,ssl_ciphers) of + undefined -> + []; + Ciphers -> + [{ciphers, Ciphers}] + end. + +ssl_password(ConfigDB) -> + case httpd_util:lookup(ConfigDB,ssl_password_callback_module) of + undefined -> + []; + Module -> + case httpd_util:lookup(ConfigDB, + ssl_password_callback_function) of + undefined -> + []; + Function -> + Args = case httpd_util:lookup(ConfigDB, + ssl_password_callback_arguments) of + undefined -> + []; + Arguments -> + [Arguments] + end, + + case catch apply(Module, Function, Args) of + Password when is_list(Password) -> + [{password, Password}]; + Error -> + error_report(ssl_password,Module,Function,Error), + [] + end + end + end. + +ssl_verify_depth(ConfigDB) -> + case httpd_util:lookup(ConfigDB, ssl_verify_client_depth) of + undefined -> + []; + Depth -> + [{depth, Depth}] + end. + +ssl_ca_certificate_file(ConfigDB) -> + case httpd_util:lookup(ConfigDB, ssl_ca_certificate_file) of + undefined -> + []; + File -> + [{cacertfile, File}] + end. + +error_report(Where,M,F,Error) -> + error_logger:error_report([{?MODULE, Where}, + {apply, {M, F, []}}, Error]). diff --git a/lib/inets/src/http_server/httpd_esi.erl b/lib/inets/src/http_server/httpd_esi.erl new file mode 100644 index 0000000000..b1a75fda52 --- /dev/null +++ b/lib/inets/src/http_server/httpd_esi.erl @@ -0,0 +1,108 @@ +%% +%% %CopyrightBegin% +%% +%% Copyright Ericsson AB 2005-2009. All Rights Reserved. +%% +%% The contents of this file are subject to the Erlang Public License, +%% Version 1.1, (the "License"); you may not use this file except in +%% compliance with the License. You should have received a copy of the +%% Erlang Public License along with this software. If not, it can be +%% retrieved online at http://www.erlang.org/. +%% +%% Software distributed under the License is distributed on an "AS IS" +%% basis, WITHOUT WARRANTY OF ANY KIND, either express or implied. See +%% the License for the specific language governing rights and limitations +%% under the License. +%% +%% %CopyrightEnd% +%% +%% +-module(httpd_esi). + +-export([parse_headers/1, handle_headers/1]). + +-include("inets_internal.hrl"). + +%%%========================================================================= +%%% Internal application API +%%%========================================================================= + +%%-------------------------------------------------------------------------- +%% parse_headers(Data) -> {Headers, Body} +%% +%% Data = string() | io_list() +%% Headers = string() +%% Body = io_list() +%% +%% Description: Parses <Data> and divides it to a header part and a +%% body part. Note that it is presumed that <Data> starts with a +%% string including "\r\n\r\n" if there is any header information +%% present. The returned headers will not contain the HTTP header body +%% delimiter \r\n. (All header, header delimiters are keept.) +%% Ex: ["Content-Type : text/html\r\n Connection : closing \r\n\r\n" | +%% io_list()] --> {"Content-Type : text/html\r\n Connection : closing \r\n", +%% io_list()} +%%-------------------------------------------------------------------------- +parse_headers(Data) -> + parse_headers(Data, []). + +%%-------------------------------------------------------------------------- +%% handle_headers(Headers) -> {ok, HTTPHeaders, StatusCode} | +%% {proceed, AbsPath} +%% Headers = string() +%% HTTPHeaders = [{HeaderField, HeaderValue}] +%% HeaderField = string() +%% HeaderValue = string() +%% StatusCode = integer() +%% +%% Description: Transforms the plain HTTP header string data received +%% from the ESI program into a list of header values and an +%% appropriate HTTP status code. Note if a location header is present +%% the return value will be {proceed, AbsPath} +%%-------------------------------------------------------------------------- +handle_headers("") -> + {ok, [], 200}; +handle_headers(Headers) -> + NewHeaders = string:tokens(Headers, ?CRLF), + handle_headers(NewHeaders, [], 200). + +%%%======================================================================== +%%% Internal functions +%%%======================================================================== +parse_headers([], Acc) -> + {[], lists:reverse(Acc)}; +parse_headers([?CR, ?LF, ?CR, ?LF], Acc) -> + {lists:reverse(Acc) ++ [?CR, ?LF], []}; +parse_headers([?CR, ?LF, ?CR, ?LF | Rest], Acc) -> + {lists:reverse(Acc) ++ [?CR, ?LF], Rest}; +parse_headers([Char | Rest], Acc) -> + parse_headers(Rest, [Char | Acc]). + +handle_headers([], NewHeaders, StatusCode) -> + {ok, NewHeaders, StatusCode}; + +handle_headers([Header | Headers], NewHeaders, StatusCode) -> + {FieldName, FieldValue} = httpd_response:split_header(Header, []), + case FieldName of + "location" -> + case http_request:is_absolut_uri(FieldValue) of + true -> + handle_headers(Headers, + [{FieldName, FieldValue} | NewHeaders], + 302); + false -> + {proceed, FieldValue} + end; + "status" -> + NewStatusCode = + case httpd_util:split(FieldValue," ",2) of + {ok,[Code,_]} -> + list_to_integer(Code); + _ -> + 200 + end, + handle_headers(Headers, NewHeaders, NewStatusCode); + _ -> + handle_headers(Headers, + [{FieldName, FieldValue}| NewHeaders], StatusCode) + end. diff --git a/lib/inets/src/http_server/httpd_example.erl b/lib/inets/src/http_server/httpd_example.erl new file mode 100644 index 0000000000..16a080f8e2 --- /dev/null +++ b/lib/inets/src/http_server/httpd_example.erl @@ -0,0 +1,145 @@ +%% +%% %CopyrightBegin% +%% +%% Copyright Ericsson AB 1997-2009. All Rights Reserved. +%% +%% The contents of this file are subject to the Erlang Public License, +%% Version 1.1, (the "License"); you may not use this file except in +%% compliance with the License. You should have received a copy of the +%% Erlang Public License along with this software. If not, it can be +%% retrieved online at http://www.erlang.org/. +%% +%% Software distributed under the License is distributed on an "AS IS" +%% basis, WITHOUT WARRANTY OF ANY KIND, either express or implied. See +%% the License for the specific language governing rights and limitations +%% under the License. +%% +%% %CopyrightEnd% +%% +%% +-module(httpd_example). +-export([print/1]). +-export([get/2, post/2, yahoo/2, test1/2, get_bin/2]). + +-export([newformat/3]). +%% These are used by the inets test-suite +-export([delay/1]). + + +print(String) -> + [header(), + top("Print"), + String++"\n", + footer()]. + +test1(Env, []) -> + io:format("Env:~p~n",[Env]), + ["<html>", + "<head>", + "<title>Test1</title>", + "</head>", + "<body>", + "<h1>Erlang Body</h1>", + "<h2>Stuff</h2>", + "</body>", + "</html>"]. + + +get(_Env,[]) -> + [header(), + top("GET Example"), + "<FORM ACTION=\"/cgi-bin/erl/httpd_example:get\" METHOD=GET> +<B>Input:</B> <INPUT TYPE=\"text\" NAME=\"input1\"> +<INPUT TYPE=\"text\" NAME=\"input2\"> +<INPUT TYPE=\"submit\"><BR> +</FORM>" ++ "\n", + footer()]; + +get(Env,Input) -> + default(Env,Input). + +get_bin(_Env,_Input) -> + [list_to_binary(header()), + list_to_binary(top("GET Example")), + list_to_binary("<FORM ACTION=\"/cgi-bin/erl/httpd_example:get\" METHOD=GET> +<B>Input:</B> <INPUT TYPE=\"text\" NAME=\"input1\"> +<INPUT TYPE=\"text\" NAME=\"input2\"> +<INPUT TYPE=\"submit\"><BR> +</FORM>" ++ "\n"), + footer()]. + +post(_Env,[]) -> + [header(), + top("POST Example"), + "<FORM ACTION=\"/cgi-bin/erl/httpd_example:post\" METHOD=POST> +<B>Input:</B> <INPUT TYPE=\"text\" NAME=\"input1\"> +<INPUT TYPE=\"text\" NAME=\"input2\"> +<INPUT TYPE=\"submit\"><BR> +</FORM>" ++ "\n", + footer()]; + +post(Env,Input) -> + default(Env,Input). + +yahoo(_Env,_Input) -> + "Location: http://www.yahoo.com\r\n\r\n". + +default(Env,Input) -> + [header(), + top("Default Example"), + "<B>Environment:</B> ",io_lib:format("~p",[Env]),"<BR>\n", + "<B>Input:</B> ",Input,"<BR>\n", + "<B>Parsed Input:</B> ", + io_lib:format("~p",[httpd:parse_query(Input)]),"\n", + footer()]. + +header() -> + header("text/html"). +header(MimeType) -> + "Content-type: " ++ MimeType ++ "\r\n\r\n". + +top(Title) -> + "<HTML> +<HEAD> +<TITLE>" ++ Title ++ "</TITLE> +</HEAD> +<BODY>\n". + +footer() -> + "</BODY> +</HTML>\n". + + +newformat(SessionID, _Env, _Input)-> + mod_esi:deliver(SessionID, "Content-Type:text/html\r\n\r\n"), + mod_esi:deliver(SessionID, top("new esi format test")), + mod_esi:deliver(SessionID, "This new format is nice<BR>"), + mod_esi:deliver(SessionID, "This new format is nice<BR>"), + mod_esi:deliver(SessionID, "This new format is nice<BR>"), + mod_esi:deliver(SessionID, footer()). + +%% ------------------------------------------------------ + +delay(Time) when is_integer(Time) -> + i("httpd_example:delay(~p) -> do the delay",[Time]), + sleep(Time), + i("httpd_example:delay(~p) -> done, now reply",[Time]), + delay_reply("delay ok"); +delay(Time) when is_list(Time) -> + delay(httpd_conf:make_integer(Time)); +delay({ok,Time}) when is_integer(Time) -> + delay(Time); +delay({error,_Reason}) -> + i("delay -> called with invalid time"), + delay_reply("delay failed: invalid delay time"). + +delay_reply(Reply) -> + [header(), + top("delay"), + Reply, + footer()]. + +i(F) -> i(F,[]). +i(F,A) -> io:format(F ++ "~n",A). + +sleep(T) -> receive after T -> ok end. diff --git a/lib/inets/src/http_server/httpd_file.erl b/lib/inets/src/http_server/httpd_file.erl new file mode 100644 index 0000000000..5fd529100e --- /dev/null +++ b/lib/inets/src/http_server/httpd_file.erl @@ -0,0 +1,45 @@ +%% +%% %CopyrightBegin% +%% +%% Copyright Ericsson AB 2006-2009. All Rights Reserved. +%% +%% The contents of this file are subject to the Erlang Public License, +%% Version 1.1, (the "License"); you may not use this file except in +%% compliance with the License. You should have received a copy of the +%% Erlang Public License along with this software. If not, it can be +%% retrieved online at http://www.erlang.org/. +%% +%% Software distributed under the License is distributed on an "AS IS" +%% basis, WITHOUT WARRANTY OF ANY KIND, either express or implied. See +%% the License for the specific language governing rights and limitations +%% under the License. +%% +%% %CopyrightEnd% +%% +%% +-module(httpd_file). + +-export([handle_error/4]). + +-include("httpd.hrl"). + +handle_error(eacces, Op, ModData, Path) -> + handle_error(403, Op, ModData, Path,""); +handle_error(enoent, Op, ModData, Path) -> + handle_error(404, Op, ModData, Path,""); +handle_error(enotdir, Op, ModData, Path) -> + handle_error(404, Op, ModData, Path, + ": A component of the file name is not a directory"); +handle_error(emfile, Op, _ModData, Path) -> + handle_error(500, Op, none, Path, ": To many open files"); +handle_error({enfile,_}, Op, _ModData, Path) -> + handle_error(500, Op, none, Path, ": File table overflow"); +handle_error(_Reason, Op, _ModData, Path) -> + handle_error(500, Op, none, Path, ""). + +handle_error(StatusCode, Op, none, Path, Reason) -> + {StatusCode, none, ?NICE("Can't " ++ Op ++ Path ++ Reason)}; + +handle_error(StatusCode, Op, ModData, Path, Reason) -> + {StatusCode, ModData#mod.request_uri, + ?NICE("Can't " ++ Op ++ Path ++ Reason)}. diff --git a/lib/inets/src/http_server/httpd_instance_sup.erl b/lib/inets/src/http_server/httpd_instance_sup.erl new file mode 100644 index 0000000000..3b5464132c --- /dev/null +++ b/lib/inets/src/http_server/httpd_instance_sup.erl @@ -0,0 +1,169 @@ +%% +%% %CopyrightBegin% +%% +%% Copyright Ericsson AB 2001-2009. All Rights Reserved. +%% +%% The contents of this file are subject to the Erlang Public License, +%% Version 1.1, (the "License"); you may not use this file except in +%% compliance with the License. You should have received a copy of the +%% Erlang Public License along with this software. If not, it can be +%% retrieved online at http://www.erlang.org/. +%% +%% Software distributed under the License is distributed on an "AS IS" +%% basis, WITHOUT WARRANTY OF ANY KIND, either express or implied. See +%% the License for the specific language governing rights and limitations +%% under the License. +%% +%% %CopyrightEnd% +%% +%% +%%---------------------------------------------------------------------- +%% Purpose: The top supervisor for an instance of the http server. (You may +%% have several instances running on the same machine.) Hangs under +%% httpd_sup. +%%---------------------------------------------------------------------- + +-module(httpd_instance_sup). + +-behaviour(supervisor). + +%% Internal application API +-export([start_link/3, start_link/4]). + +%% Supervisor callbacks +-export([init/1]). + +%%%========================================================================= +%%% Internal Application API +%%%========================================================================= +start_link([{_, _}| _] = Config, AcceptTimeout, Debug) -> + case httpd_conf:validate_properties(Config) of + {ok, Config2} -> + Address = proplists:get_value(bind_address, Config2), + Port = proplists:get_value(port, Config2), + Name = make_name(Address, Port), + SupName = {local, Name}, + supervisor:start_link(SupName, ?MODULE, + [undefined, Config2, AcceptTimeout, + Debug, Address, Port]); + {error, Reason} -> + error_logger:error_report(Reason), + {stop, Reason} + end; + +start_link(ConfigFile, AcceptTimeout, Debug) -> + case file_2_config(ConfigFile) of + {ok, ConfigList, Address, Port} -> + Name = make_name(Address, Port), + SupName = {local, Name}, + supervisor:start_link(SupName, ?MODULE, + [ConfigFile, ConfigList, AcceptTimeout, + Debug, Address, Port]); + {error, Reason} -> + error_logger:error_report(Reason), + {stop, Reason} + end. + + +start_link([{_, _}| _] = Config, AcceptTimeout, ListenInfo, Debug) -> + case httpd_conf:validate_properties(Config) of + {ok, Config2} -> + Address = proplists:get_value(bind_address, Config2), + Port = proplists:get_value(port, Config2), + Name = make_name(Address, Port), + SupName = {local, Name}, + supervisor:start_link(SupName, ?MODULE, + [undefined, Config2, AcceptTimeout, + Debug, Address, Port, ListenInfo]); + {error, Reason} -> + error_logger:error_report(Reason), + {stop, Reason} + end; + +start_link(ConfigFile, AcceptTimeout, ListenInfo, Debug) -> + case file_2_config(ConfigFile) of + {ok, ConfigList, Address, Port} -> + Name = make_name(Address, Port), + SupName = {local, Name}, + supervisor:start_link(SupName, ?MODULE, + [ConfigFile, ConfigList, AcceptTimeout, + Debug, Address, Port, ListenInfo]); + {error, Reason} -> + error_logger:error_report(Reason), + {stop, Reason} + end. + + +%%%========================================================================= +%%% Supervisor callback +%%%========================================================================= +init([ConfigFile, ConfigList, AcceptTimeout, _Debug, Address, Port]) -> + Flags = {one_for_one, 0, 1}, + Children = [sup_spec(httpd_acceptor_sup, Address, Port), + sup_spec(httpd_misc_sup, Address, Port), + worker_spec(httpd_manager, Address, Port, + ConfigFile, ConfigList,AcceptTimeout)], + {ok, {Flags, Children}}; +init([ConfigFile, ConfigList, AcceptTimeout, _Debug, Address, Port, ListenInfo]) -> + Flags = {one_for_one, 0, 1}, + Children = [sup_spec(httpd_acceptor_sup, Address, Port), + sup_spec(httpd_misc_sup, Address, Port), + worker_spec(httpd_manager, Address, Port, ListenInfo, + ConfigFile, ConfigList, AcceptTimeout)], + {ok, {Flags, Children}}. + + +%%%========================================================================= +%%% Internal functions +%%%========================================================================= +sup_spec(SupModule, Address, Port) -> + Name = {SupModule, Address, Port}, + StartFunc = {SupModule, start_link, [Address, Port]}, + Restart = permanent, + Shutdown = infinity, + Modules = [SupModule], + Type = supervisor, + {Name, StartFunc, Restart, Shutdown, Type, Modules}. + +worker_spec(WorkerModule, Address, Port, ConfigFile, + ConfigList, AcceptTimeout) -> + Name = {WorkerModule, Address, Port}, + StartFunc = {WorkerModule, start_link, + [ConfigFile, ConfigList, AcceptTimeout]}, + Restart = permanent, + Shutdown = 4000, + Modules = [WorkerModule], + Type = worker, + {Name, StartFunc, Restart, Shutdown, Type, Modules}. + +worker_spec(WorkerModule, Address, Port, ListenInfo, ConfigFile, + ConfigList, AcceptTimeout) -> + Name = {WorkerModule, Address, Port}, + StartFunc = {WorkerModule, start_link, + [ConfigFile, ConfigList, AcceptTimeout, ListenInfo]}, + Restart = permanent, + Shutdown = 4000, + Modules = [WorkerModule], + Type = worker, + {Name, StartFunc, Restart, Shutdown, Type, Modules}. + +make_name(Address,Port) -> + httpd_util:make_name("httpd_instance_sup", Address, Port). + + +file_2_config(ConfigFile) -> + case httpd_conf:load(ConfigFile) of + {ok, ConfigList} -> + case httpd_conf:validate_properties(ConfigList) of + {ok, Config} -> + Address = proplists:get_value(bind_address, ConfigList), + Port = proplists:get_value(port, ConfigList), + {ok, Config, Address, Port}; + Error -> + Error + end; + Error -> + Error + end. + + diff --git a/lib/inets/src/http_server/httpd_internal.hrl b/lib/inets/src/http_server/httpd_internal.hrl new file mode 100644 index 0000000000..7795ab6c18 --- /dev/null +++ b/lib/inets/src/http_server/httpd_internal.hrl @@ -0,0 +1,31 @@ +%% +%% %CopyrightBegin% +%% +%% Copyright Ericsson AB 2009. All Rights Reserved. +%% +%% The contents of this file are subject to the Erlang Public License, +%% Version 1.1, (the "License"); you may not use this file except in +%% compliance with the License. You should have received a copy of the +%% Erlang Public License along with this software. If not, it can be +%% retrieved online at http://www.erlang.org/. +%% +%% Software distributed under the License is distributed on an "AS IS" +%% basis, WITHOUT WARRANTY OF ANY KIND, either express or implied. See +%% the License for the specific language governing rights and limitations +%% under the License. +%% +%% %CopyrightEnd% +%% +%% + +-ifndef(httpd_internal_hrl). +-define(httpd_internal_hrl, true). + +-include("inets_internal.hrl"). +-define(SERVICE, httpd). +-define(hdri(Label, Content), ?report_important(Label, ?SERVICE, Content)). +-define(hdrv(Label, Content), ?report_verbose(Label, ?SERVICE, Content)). +-define(hdrd(Label, Content), ?report_debug(Label, ?SERVICE, Content)). +-define(hdrt(Label, Content), ?report_trace(Label, ?SERVICE, Content)). + +-endif. % -ifdef(httpd_internal_hrl). diff --git a/lib/inets/src/http_server/httpd_log.erl b/lib/inets/src/http_server/httpd_log.erl new file mode 100644 index 0000000000..f3ea3aa0e2 --- /dev/null +++ b/lib/inets/src/http_server/httpd_log.erl @@ -0,0 +1,121 @@ +%% +%% %CopyrightBegin% +%% +%% Copyright Ericsson AB 2008-2009. All Rights Reserved. +%% +%% The contents of this file are subject to the Erlang Public License, +%% Version 1.1, (the "License"); you may not use this file except in +%% compliance with the License. You should have received a copy of the +%% Erlang Public License along with this software. If not, it can be +%% retrieved online at http://www.erlang.org/. +%% +%% Software distributed under the License is distributed on an "AS IS" +%% basis, WITHOUT WARRANTY OF ANY KIND, either express or implied. See +%% the License for the specific language governing rights and limitations +%% under the License. +%% +%% %CopyrightEnd% +%% +%% +-module(httpd_log). + +-include("httpd.hrl"). + +-export([access_entry/8, error_entry/5, error_report_entry/5, + security_entry/5]). + +%%%========================================================================= +%%% Internal Application API +%%%========================================================================= +access_entry(Log, NoLog, Info, RFC931, AuthUser, Date, StatusCode, Bytes) -> + ConfigDB = Info#mod.config_db, + case httpd_util:lookup(ConfigDB, Log) of + undefined -> + NoLog; + LogRef -> + {_, RemoteHost} + = (Info#mod.init_data)#init_data.peername, + RequestLine = Info#mod.request_line, + Headers = Info#mod.parsed_header, + Entry = do_access_entry(ConfigDB, Headers, RequestLine, + RemoteHost, RFC931, AuthUser, + Date, StatusCode, Bytes), + {LogRef, Entry} + end. + +error_entry(Log, NoLog, Info, Date, Reason) -> + ConfigDB = Info#mod.config_db, + case httpd_util:lookup(ConfigDB, Log) of + undefined -> + NoLog; + LogRef -> + {_, RemoteHost} = + (Info#mod.init_data)#init_data.peername, + URI = Info#mod.request_uri, + Entry = do_error_entry(ConfigDB, RemoteHost, URI, Date, Reason), + {LogRef, Entry} + end. + +error_report_entry(Log, NoLog, ConfigDb, Date, ErrorStr) -> + case httpd_util:lookup(ConfigDb, Log) of + undefined -> + NoLog; + LogRef -> + Entry = io_lib:format("[~s], ~s~n", [Date, ErrorStr]), + {LogRef, Entry} + end. + +security_entry(Log, NoLog, #mod{config_db = ConfigDb}, Date, Reason) -> + case httpd_util:lookup(ConfigDb, Log) of + undefined -> + NoLog; + LogRef -> + Entry = io_lib:format("[~s] ~s~n", [Date, Reason]), + {LogRef, Entry} + end. + +%%%======================================================================== +%%% Internal functions +%%%======================================================================== +do_access_entry(ConfigDB, Headers, RequestLine, + RemoteHost, RFC931, AuthUser, Date, StatusCode, + Bytes) -> + case httpd_util:lookup(ConfigDB, log_format, common) of + common -> + lists:flatten(io_lib:format("~s ~s ~s [~s] \"~s\" ~w ~w~n", + [RemoteHost, RFC931, AuthUser, Date, + RequestLine, + StatusCode, Bytes])); + combined -> + Referer = + proplists:get_value("referer", Headers, "-"), + UserAgent = + proplists:get_value("user-agent", + Headers, "-"), + io_lib:format("~s ~s ~s [~s] \"~s\" ~w ~w ~s ~s~n", + [RemoteHost, RFC931, AuthUser, Date, + RequestLine, StatusCode, Bytes, + Referer, UserAgent]) + end. + + +do_error_entry(ConfigDB, RemoteHost, undefined, Date, Reason) -> + case httpd_util:lookup(ConfigDB, error_log_format, pretty) of + pretty -> + io_lib:format("[~s] server crash for ~s, reason: ~n~p~n~n", + [Date, RemoteHost, Reason]); + compact -> + io_lib:format("[~s] server crash for ~s, reason: ~w~n", + [Date, RemoteHost, Reason]) + + end; + +do_error_entry(ConfigDB, RemoteHost, URI, Date, Reason) -> + case httpd_util:lookup(ConfigDB, error_log_format, pretty) of + pretty -> + io_lib:format("[~s] access to ~s failed for ~s reason: ~n~p~n", + [Date, URI, RemoteHost, Reason]); + compact -> + io_lib:format( "[~s] access to ~s failed for ~s, reason: ~w~n", + [Date, URI, RemoteHost, Reason]) + end. diff --git a/lib/inets/src/http_server/httpd_manager.erl b/lib/inets/src/http_server/httpd_manager.erl new file mode 100644 index 0000000000..f2e8763907 --- /dev/null +++ b/lib/inets/src/http_server/httpd_manager.erl @@ -0,0 +1,890 @@ +%% +%% %CopyrightBegin% +%% +%% Copyright Ericsson AB 2000-2009. All Rights Reserved. +%% +%% The contents of this file are subject to the Erlang Public License, +%% Version 1.1, (the "License"); you may not use this file except in +%% compliance with the License. You should have received a copy of the +%% Erlang Public License along with this software. If not, it can be +%% retrieved online at http://www.erlang.org/. +%% +%% Software distributed under the License is distributed on an "AS IS" +%% basis, WITHOUT WARRANTY OF ANY KIND, either express or implied. See +%% the License for the specific language governing rights and limitations +%% under the License. +%% +%% %CopyrightEnd% +%% +%% + +-module(httpd_manager). + +-include("httpd.hrl"). + +-behaviour(gen_server). + +%% Application internal API +-export([start/2, start_link/2, start_link/3, start_link/4, stop/1, reload/2]). +-export([new_connection/1, done_connection/1]). +-export([config_lookup/2, config_lookup/3, + config_multi_lookup/2, config_multi_lookup/3, + config_match/2, config_match/3]). + +%% gen_server exports +-export([init/1, + handle_call/3, handle_cast/2, handle_info/2, + terminate/2, + code_change/3]). + + +%% Management exports +-export([block/2, block/3, unblock/1]). +-export([get_admin_state/1, get_usage_state/1]). +-export([is_busy/1,is_busy/2,is_busy_or_blocked/1,is_blocked/1]). %% ??????? +-export([get_status/1, get_status/2]). + +-export([c/1]). + +-record(state,{socket_type = ip_comm, + config_file, + config_db = null, + connections, %% Current request handlers + admin_state = unblocked, + blocker_ref = undefined, + blocking_tmr = undefined, + status = []}). + + +%%TODO: Clean up this module! + +c(Port) -> + Ref = httpd_util:make_name("httpd",undefined,Port), + call(Ref, fake_close). + +%% +%% External API +%% +%% Deprecated +start(ConfigFile, ConfigList) -> + Port = proplists:get_value(port,ConfigList,80), + Addr = proplists:get_value(bind_address, ConfigList), + Name = make_name(Addr,Port), + gen_server:start({local,Name},?MODULE, + [ConfigFile, ConfigList, 15000, Addr, Port],[]). + +%% Deprecated +start_link(ConfigFile, ConfigList) -> + start_link(ConfigFile, ConfigList, 15000). + +start_link(ConfigFile, ConfigList, AcceptTimeout) -> + Port = proplists:get_value(port, ConfigList, 80), + Addr = proplists:get_value(bind_address, ConfigList), + Name = make_name(Addr, Port), + + gen_server:start_link({local, Name},?MODULE, + [ConfigFile, ConfigList, AcceptTimeout, Addr, Port],[]). + +start_link(ConfigFile, ConfigList, AcceptTimeout, ListenSocket) -> + Port = proplists:get_value(port, ConfigList, 80), + Addr = proplists:get_value(bind_address, ConfigList), + Name = make_name(Addr, Port), + + gen_server:start_link({local, Name},?MODULE, + [ConfigFile, ConfigList, AcceptTimeout, Addr, + Port, ListenSocket],[]). + +stop(ServerRef) -> + call(ServerRef, stop). + +reload(ServerRef, Conf) -> + call(ServerRef, {reload, Conf}). + + +%%%---------------------------------------------------------------- + +block(ServerRef, disturbing) -> + call(ServerRef,block); + +block(ServerRef, non_disturbing) -> + do_block(ServerRef, non_disturbing, infinity). + +block(ServerRef, Method, Timeout) -> + do_block(ServerRef, Method, Timeout). + + +%% The reason for not using call here, is that the manager cannot +%% _wait_ for completion of the requests. It must be able to do +%% do other things at the same time as the blocking goes on. +do_block(ServerRef, Method, infinity) -> + Ref = make_ref(), + cast(ServerRef, {block, Method, infinity, self(), Ref}), + receive + {block_reply, Reply, Ref} -> + Reply + end; +do_block(ServerRef,Method,Timeout) when Timeout > 0 -> + Ref = make_ref(), + cast(ServerRef,{block,Method,Timeout,self(),Ref}), + receive + {block_reply,Reply,Ref} -> + Reply + end. + + +%%%---------------------------------------------------------------- + +%% unblock + +unblock(ServerRef) -> + call(ServerRef,unblock). + +%% get admin/usage state + +get_admin_state(ServerRef) -> + call(ServerRef,get_admin_state). + +get_usage_state(ServerRef) -> + call(ServerRef,get_usage_state). + + +%% get_status + +get_status(ServerRef) -> + gen_server:call(ServerRef,get_status). + +get_status(ServerRef,Timeout) -> + gen_server:call(ServerRef,get_status,Timeout). + +%% +%% Internal API +%% + + +%% new_connection + +new_connection(Manager) -> + gen_server:call(Manager, {new_connection, self()}, infinity). + +%% done + +done_connection(Manager) -> + gen_server:cast(Manager, {done_connection, self()}). + + +%% is_busy(ServerRef) -> true | false +%% +%% Tests if the server is (in usage state) busy, +%% i.e. has rached the heavy load limit. +%% + +is_busy(ServerRef) -> + gen_server:call(ServerRef,is_busy). + +is_busy(ServerRef,Timeout) -> + gen_server:call(ServerRef,is_busy,Timeout). + + +%% is_busy_or_blocked(ServerRef) -> busy | blocked | false +%% +%% Tests if the server is busy (usage state), i.e. has rached, +%% the heavy load limit, or blocked (admin state) . +%% + +is_busy_or_blocked(ServerRef) -> + gen_server:call(ServerRef,is_busy_or_blocked). + + +%% is_blocked(ServerRef) -> true | false +%% +%% Tests if the server is blocked (admin state) . +%% + +is_blocked(ServerRef) -> + gen_server:call(ServerRef,is_blocked). + + +%% +%% Module API. Theese functions are intended for use from modules only. +%% + +config_lookup(Port, Query) -> + config_lookup(undefined, Port, Query). +config_lookup(Addr, Port, Query) -> + Name = httpd_util:make_name("httpd",Addr,Port), + gen_server:call(whereis(Name), {config_lookup, Query}). + +config_multi_lookup(Port, Query) -> + config_multi_lookup(undefined,Port,Query). +config_multi_lookup(Addr,Port, Query) -> + Name = httpd_util:make_name("httpd",Addr,Port), + gen_server:call(whereis(Name), {config_multi_lookup, Query}). + +config_match(Port, Pattern) -> + config_match(undefined,Port,Pattern). +config_match(Addr, Port, Pattern) -> + Name = httpd_util:make_name("httpd",Addr,Port), + gen_server:call(whereis(Name), {config_match, Pattern}). + + +%% +%% Server call-back functions +%% + +%% init + +init([ConfigFile, ConfigList, AcceptTimeout, Addr, Port]) -> + process_flag(trap_exit, true), + case (catch do_init(ConfigFile, ConfigList, AcceptTimeout, Addr, Port)) of + {error, Reason} -> + String = lists:flatten( + io_lib:format("Failed initiating " + "web server: ~n~p~n~p~n", + [ConfigFile,Reason])), + error_logger:error_report(String), + {stop, {error, Reason}}; + {ok, State} -> + {ok, State} + end; +init([ConfigFile, ConfigList, AcceptTimeout, Addr, Port, + ListenInfo]) -> + process_flag(trap_exit, true), + case (catch do_init(ConfigFile, ConfigList, AcceptTimeout, + Addr, Port, ListenInfo)) of + {error, Reason} -> + String = lists:flatten( + io_lib:format("Failed initiating " + "web server: ~n~p~n~p~n", + [ConfigFile,Reason])), + error_logger:error_report(String), + {stop, {error, Reason}}; + {ok, State} -> + {ok, State} + end. + +do_init(ConfigFile, ConfigList, AcceptTimeout, Addr, Port) -> + NewConfigFile = proplists:get_value(file, ConfigList, ConfigFile), + ConfigDB = do_initial_store(ConfigList), + SocketType = httpd_conf:config(ConfigDB), + case httpd_acceptor_sup:start_acceptor(SocketType, Addr, + Port, ConfigDB, AcceptTimeout) of + {ok, _Pid} -> + Status = [{max_conn,0}, {last_heavy_load,never}, + {last_connection,never}], + State = #state{socket_type = SocketType, + config_file = NewConfigFile, + config_db = ConfigDB, + connections = [], + status = Status}, + {ok, State}; + Else -> + Else + end. + +do_init(ConfigFile, ConfigList, AcceptTimeout, Addr, Port, ListenInfo) -> + NewConfigFile = proplists:get_value(file, ConfigList, ConfigFile), + ConfigDB = do_initial_store(ConfigList), + SocketType = httpd_conf:config(ConfigDB), + case httpd_acceptor_sup:start_acceptor(SocketType, Addr, + Port, ConfigDB, + AcceptTimeout, ListenInfo) of + {ok, _Pid} -> + Status = [{max_conn,0}, {last_heavy_load,never}, + {last_connection,never}], + State = #state{socket_type = SocketType, + config_file = NewConfigFile, + config_db = ConfigDB, + connections = [], + status = Status}, + {ok, State}; + Else -> + Else + end. + +do_initial_store(ConfigList) -> + case httpd_conf:store(ConfigList) of + {ok, ConfigDB} -> + ConfigDB; + {error, Reason} -> + throw({error, Reason}) + end. + + + +%% handle_call + +handle_call(stop, _From, State) -> + {stop, normal, ok, State}; + +handle_call({config_lookup, Query}, _From, State) -> + Res = httpd_util:lookup(State#state.config_db, Query), + {reply, Res, State}; + +handle_call({config_multi_lookup, Query}, _From, State) -> + Res = httpd_util:multi_lookup(State#state.config_db, Query), + {reply, Res, State}; + +handle_call({config_match, Query}, _From, State) -> + Res = ets:match_object(State#state.config_db, Query), + {reply, Res, State}; + +handle_call(get_status, _From, State) -> + ManagerStatus = manager_status(self()), + S1 = [{current_conn,length(State#state.connections)}|State#state.status]++ + [ManagerStatus], + {reply,S1,State}; + +handle_call(is_busy, _From, State) -> + Reply = case get_ustate(State) of + busy -> + true; + _ -> + false + end, + {reply,Reply,State}; + +handle_call(is_busy_or_blocked, _From, State) -> + Reply = + case get_astate(State) of + unblocked -> + case get_ustate(State) of + busy -> + busy; + _ -> + false + end; + _ -> + blocked + end, + {reply,Reply,State}; + +handle_call(is_blocked, _From, State) -> + Reply = + case get_astate(State) of + unblocked -> + false; + _ -> + true + end, + {reply,Reply,State}; + +handle_call(get_admin_state, _From, State) -> + Reply = get_astate(State), + {reply,Reply,State}; + +handle_call(get_usage_state, _From, State) -> + Reply = get_ustate(State), + {reply,Reply,State}; + +handle_call({reload, Conf}, _From, State) + when State#state.admin_state =:= blocked -> + case handle_reload(Conf, State) of + {stop, Reply,S1} -> + {stop, Reply, S1}; + {_, Reply, S1} -> + {reply,Reply,S1} + end; + +handle_call({reload, _}, _From, State) -> + {reply,{error,{invalid_admin_state,State#state.admin_state}},State}; + +handle_call(block, _From, State) -> + {Reply,S1} = handle_block(State), + {reply,Reply,S1}; + +handle_call(unblock, {From,_Tag}, State) -> + {Reply,S1} = handle_unblock(State,From), + {reply, Reply, S1}; + +handle_call({new_connection, Pid}, _From, State) -> + {Status, NewState} = handle_new_connection(State, Pid), + {reply, Status, NewState}; + +handle_call(Request, From, State) -> + String = + lists:flatten( + io_lib:format("Unknown request " + "~n ~p" + "~nto manager (~p)" + "~nfrom ~p", + [Request, self(), From])), + report_error(State,String), + {reply, ok, State}. + + +%% handle_cast + +handle_cast({done_connection, Pid}, State) -> + S1 = handle_done_connection(State, Pid), + {noreply, S1}; + +handle_cast({block, disturbing, Timeout, From, Ref}, State) -> + S1 = handle_block(State, Timeout, From, Ref), + {noreply,S1}; + +handle_cast({block, non_disturbing, Timeout, From, Ref}, State) -> + S1 = handle_nd_block(State, Timeout, From, Ref), + {noreply,S1}; + +handle_cast(Message, State) -> + String = + lists:flatten( + io_lib:format("Unknown message " + "~n ~p" + "~nto manager (~p)", + [Message, self()])), + report_error(State, String), + {noreply, State}. + +%% handle_info + +handle_info({block_timeout, Method}, State) -> + S1 = handle_block_timeout(State,Method), + {noreply, S1}; + +handle_info({'DOWN', Ref, process, _Object, _Info}, State) -> + S1 = + case State#state.blocker_ref of + Ref -> + handle_blocker_exit(State); + _ -> + %% Not our blocker, so ignore + State + end, + {noreply, S1}; + +handle_info({'EXIT', _, normal}, State) -> + {noreply, State}; + +handle_info({'EXIT', _, blocked}, S) -> + {noreply, S}; + +handle_info({'EXIT', Pid, Reason}, State) -> + S1 = check_connections(State, Pid, Reason), + {noreply, S1}; + +handle_info(Info, State) -> + String = + lists:flatten( + io_lib:format("Unknown info " + "~n ~p" + "~nto manager (~p)", + [Info, self()])), + report_error(State, String), + {noreply, State}. + + +%% terminate + +terminate(_, #state{config_db = Db}) -> + httpd_conf:remove_all(Db), + ok. + + +%% code_change({down,ToVsn}, State, Extra) +%% + +code_change({down,_ToVsn}, State, _Extra) -> + {ok,State}; + +%% code_change(FromVsn, State, Extra) +%% +code_change(_FromVsn, State, _Extra) -> + {ok,State}. + + + +%% ------------------------------------------------------------------------- +%% check_connection +%% +%% +%% +%% + +check_connections(#state{connections = []} = State, _Pid, _Reason) -> + State; +check_connections(#state{admin_state = shutting_down, + connections = Connections} = State, Pid, Reason) -> + %% Could be a crashing request handler + case lists:delete(Pid, Connections) of + [] -> % Crashing request handler => block complete + String = + lists:flatten( + io_lib:format("request handler (~p) crashed:" + "~n ~p", [Pid, Reason])), + report_error(State, String), + demonitor_blocker(State#state.blocker_ref), + {Tmr,From,Ref} = State#state.blocking_tmr, + stop_block_tmr(Tmr), + From ! {block_reply,ok,Ref}, + State#state{admin_state = blocked, connections = [], + blocker_ref = undefined}; + Connections1 -> + State#state{connections = Connections1} + end; +check_connections(#state{connections = Connections} = State, Pid, Reason) -> + case lists:delete(Pid, Connections) of + Connections -> % Not a request handler, so ignore + State; + NewConnections -> + String = + lists:flatten( + io_lib:format("request handler (~p) crashed:" + "~n ~p", [Pid, Reason])), + report_error(State, String), + State#state{connections = NewConnections} + end. + + +%% ------------------------------------------------------------------------- +%% handle_[new | done]_connection +%% +%% +%% +%% + +handle_new_connection(State, Handler) -> + UsageState = get_ustate(State), + AdminState = get_astate(State), + handle_new_connection(UsageState, AdminState, State, Handler). + +handle_new_connection(busy, unblocked, State, _Handler) -> + Status = update_heavy_load_status(State#state.status), + {{reject, busy}, + State#state{status = Status}}; + +handle_new_connection(_UsageState, unblocked, State, Handler) -> + Connections = State#state.connections, + Status = update_connection_status(State#state.status, + length(Connections)+1), + link(Handler), + {{ok, accept}, + State#state{connections = [Handler|Connections], status = Status}}; + +handle_new_connection(_UsageState, _AdminState, State, _Handler) -> + {{reject, blocked}, + State}. + +handle_done_connection(#state{admin_state = shutting_down, + connections = Connections} = State, Handler) -> + unlink(Handler), + case lists:delete(Handler, Connections) of + [] -> % Ok, block complete + demonitor_blocker(State#state.blocker_ref), + {Tmr,From,Ref} = State#state.blocking_tmr, + stop_block_tmr(Tmr), + From ! {block_reply,ok,Ref}, + State#state{admin_state = blocked, connections = [], + blocker_ref = undefined}; + Connections1 -> + State#state{connections = Connections1} + end; + +handle_done_connection(#state{connections = Connections} = State, Handler) -> + State#state{connections = lists:delete(Handler, Connections)}. + + +%% ------------------------------------------------------------------------- +%% handle_block +%% +%% +%% +%% +handle_block(#state{admin_state = AdminState} = S) -> + handle_block(S, AdminState). + +handle_block(S,unblocked) -> + %% Kill all connections + [kill_handler(Pid) || Pid <- S#state.connections], + {ok,S#state{connections = [], admin_state = blocked}}; +handle_block(S,blocked) -> + {ok,S}; +handle_block(S,shutting_down) -> + {{error,shutting_down},S}. + + +kill_handler(Pid) -> + exit(Pid, blocked). + +handle_block(S,Timeout,From,Ref) when Timeout >= 0 -> + do_block(S,Timeout,From,Ref); + +handle_block(S,Timeout,From,Ref) -> + Reply = {error,{invalid_block_request,Timeout}}, + From ! {block_reply,Reply,Ref}, + S. + +do_block(S,Timeout,From,Ref) -> + case S#state.connections of + [] -> + %% Already in idle usage state => go directly to blocked + From ! {block_reply,ok,Ref}, + S#state{admin_state = blocked}; + _ -> + %% Active or Busy usage state => go to shutting_down + %% Make sure we get to know if blocker dies... + MonitorRef = monitor_blocker(From), + Tmr = {start_block_tmr(Timeout,disturbing),From,Ref}, + S#state{admin_state = shutting_down, + blocker_ref = MonitorRef, blocking_tmr = Tmr} + end. + +handle_nd_block(S,infinity,From,Ref) -> + do_nd_block(S,infinity,From,Ref); + +handle_nd_block(S,Timeout,From,Ref) when Timeout >= 0 -> + do_nd_block(S,Timeout,From,Ref); + +handle_nd_block(S,Timeout,From,Ref) -> + Reply = {error,{invalid_block_request,Timeout}}, + From ! {block_reply,Reply,Ref}, + S. + +do_nd_block(S,Timeout,From,Ref) -> + case S#state.connections of + [] -> + %% Already in idle usage state => go directly to blocked + From ! {block_reply,ok,Ref}, + S#state{admin_state = blocked}; + _ -> + %% Active or Busy usage state => go to shutting_down + %% Make sure we get to know if blocker dies... + MonitorRef = monitor_blocker(From), + Tmr = {start_block_tmr(Timeout,non_disturbing),From,Ref}, + S#state{admin_state = shutting_down, + blocker_ref = MonitorRef, blocking_tmr = Tmr} + end. + +handle_block_timeout(S,Method) -> + %% Time to take this to the road... + demonitor_blocker(S#state.blocker_ref), + handle_block_timeout1(S,Method,S#state.blocking_tmr). + +handle_block_timeout1(S,non_disturbing,{_,From,Ref}) -> + From ! {block_reply,{error,timeout},Ref}, + S#state{admin_state = unblocked, + blocker_ref = undefined, blocking_tmr = undefined}; + +handle_block_timeout1(S,disturbing,{_,From,Ref}) -> + [exit(Pid,blocked) || Pid <- S#state.connections], + + From ! {block_reply,ok,Ref}, + S#state{admin_state = blocked, connections = [], + blocker_ref = undefined, blocking_tmr = undefined}; + +handle_block_timeout1(S,Method,{_,From,Ref}) -> + From ! {block_reply,{error,{unknown_block_method,Method}},Ref}, + S#state{admin_state = blocked, connections = [], + blocker_ref = undefined, blocking_tmr = undefined}; + +handle_block_timeout1(S, _Method, _TmrInfo) -> + S#state{admin_state = unblocked, + blocker_ref = undefined, blocking_tmr = undefined}. + +handle_unblock(S, FromA) -> + handle_unblock(S, FromA, S#state.admin_state). + +handle_unblock(S, _FromA, unblocked) -> + {ok,S}; +handle_unblock(S, FromA, _AdminState) -> + stop_block_tmr(S#state.blocking_tmr), + case S#state.blocking_tmr of + {_Tmr,FromB,Ref} -> + %% Another process is trying to unblock + %% Inform the blocker + FromB ! {block_reply, {error,{unblocked,FromA}},Ref}; + _ -> + ok + end, + {ok,S#state{admin_state = unblocked, blocking_tmr = undefined}}. + +%% The blocker died so we give up on the block. +handle_blocker_exit(S) -> + {Tmr,_From,_Ref} = S#state.blocking_tmr, + stop_block_tmr(Tmr), + S#state{admin_state = unblocked, + blocker_ref = undefined, blocking_tmr = undefined}. + + + +%% ------------------------------------------------------------------------- +%% handle_reload +%% +%% +%% +%% +handle_reload(undefined, #state{config_file = undefined} = State) -> + {continue, {error, undefined_config_file}, State}; +handle_reload(undefined, #state{config_file = ConfigFile} = State) -> + case load_config(ConfigFile) of + {ok, Config} -> + do_reload(Config, State); + {error, Reason} -> + error_logger:error_msg("Bad config file: ~p~n", [Reason]), + {continue, {error, Reason}, State} + end; +handle_reload(Config, State) -> + do_reload(Config, State). + +load_config(ConfigFile) -> + case httpd_conf:load(ConfigFile) of + {ok, Config} -> + httpd_conf:validate_properties(Config); + Error -> + Error + end. + +do_reload(Config, #state{config_db = Db} = State) -> + case (catch check_constant_values(Db, Config)) of + ok -> + %% If something goes wrong between the remove + %% and the store where fu-ed + httpd_conf:remove_all(Db), + case httpd_conf:store(Config) of + {ok, NewConfigDB} -> + {continue, ok, State#state{config_db = NewConfigDB}}; + Error -> + {stop, Error, State} + end; + Error -> + {continue, Error, State} + end. + +check_constant_values(Db, Config) -> + %% Check port number + Port = httpd_util:lookup(Db,port), + case proplists:get_value(port,Config) of %% MUST be equal + Port -> + ok; + OtherPort -> + throw({error,{port_number_changed,Port,OtherPort}}) + end, + + %% Check bind address + Addr = httpd_util:lookup(Db,bind_address), + case proplists:get_value(bind_address, Config) of %% MUST be equal + Addr -> + ok; + OtherAddr -> + throw({error,{addr_changed,Addr,OtherAddr}}) + end, + + %% Check socket type + SockType = httpd_util:lookup(Db, socket_type), + case proplists:get_value(socket_type, Config) of %% MUST be equal + SockType -> + ok; + OtherSockType -> + throw({error,{sock_type_changed,SockType,OtherSockType}}) + end, + ok. + + +%% get_ustate(State) -> idle | active | busy +%% +%% Retrieve the usage state of the HTTP server: +%% 0 active connection -> idle +%% max_clients active connections -> busy +%% Otherwise -> active +%% +get_ustate(State) -> + get_ustate(length(State#state.connections),State). + +get_ustate(0,_State) -> + idle; +get_ustate(ConnectionCnt,State) -> + ConfigDB = State#state.config_db, + case httpd_util:lookup(ConfigDB, max_clients, 150) of + ConnectionCnt -> + busy; + _ -> + active + end. + + +get_astate(S) -> S#state.admin_state. + + +%% Timer handling functions +start_block_tmr(infinity,_) -> + undefined; +start_block_tmr(T,M) -> + erlang:send_after(T,self(),{block_timeout,M}). + +stop_block_tmr(undefined) -> + ok; +stop_block_tmr(Ref) -> + erlang:cancel_timer(Ref). + + +%% Monitor blocker functions +monitor_blocker(Pid) when is_pid(Pid) -> + case (catch erlang:monitor(process,Pid)) of + {'EXIT', _Reason} -> + undefined; + MonitorRef -> + MonitorRef + end; +monitor_blocker(_) -> + undefined. + +demonitor_blocker(undefined) -> + ok; +demonitor_blocker(Ref) -> + (catch erlang:demonitor(Ref)). + + +%% Some status utility functions + +update_heavy_load_status(Status) -> + update_status_with_time(Status,last_heavy_load). + +update_connection_status(Status,ConnCount) -> + S1 = case lists:keysearch(max_conn,1,Status) of + {value, {max_conn, C1}} when ConnCount > C1 -> + lists:keyreplace(max_conn,1,Status,{max_conn,ConnCount}); + {value, {max_conn, _C2}} -> + Status; + false -> + [{max_conn, ConnCount} | Status] + end, + update_status_with_time(S1,last_connection). + +update_status_with_time(Status,Key) -> + lists:keyreplace(Key,1,Status,{Key,universal_time()}). + +universal_time() -> calendar:universal_time(). + +manager_status(P) -> + Items = [status, message_queue_len, reductions, + heap_size, stack_size], + {manager_status, process_status(P,Items,[])}. + + +process_status(P,[],L) -> + [{pid,P}|lists:reverse(L)]; +process_status(P,[H|T],L) -> + case (catch process_info(P,H)) of + {H, Value} -> + process_status(P,T,[{H,Value}|L]); + _ -> + process_status(P,T,[{H,undefined}|L]) + end. + +make_name(Addr,Port) -> + httpd_util:make_name("httpd",Addr,Port). + + +report_error(State,String) -> + Cdb = State#state.config_db, + error_logger:error_report(String), + mod_log:report_error(Cdb,String), + mod_disk_log:report_error(Cdb,String). + +%% +call(ServerRef,Request) -> + gen_server:call(ServerRef,Request). + +cast(ServerRef,Message) -> + gen_server:cast(ServerRef,Message). + diff --git a/lib/inets/src/http_server/httpd_misc_sup.erl b/lib/inets/src/http_server/httpd_misc_sup.erl new file mode 100644 index 0000000000..fd7c28bd7d --- /dev/null +++ b/lib/inets/src/http_server/httpd_misc_sup.erl @@ -0,0 +1,91 @@ +%% +%% %CopyrightBegin% +%% +%% Copyright Ericsson AB 2001-2009. All Rights Reserved. +%% +%% The contents of this file are subject to the Erlang Public License, +%% Version 1.1, (the "License"); you may not use this file except in +%% compliance with the License. You should have received a copy of the +%% Erlang Public License along with this software. If not, it can be +%% retrieved online at http://www.erlang.org/. +%% +%% Software distributed under the License is distributed on an "AS IS" +%% basis, WITHOUT WARRANTY OF ANY KIND, either express or implied. See +%% the License for the specific language governing rights and limitations +%% under the License. +%% +%% %CopyrightEnd% +%% +%% +%%---------------------------------------------------------------------- +%% Purpose: The supervisor for auth and sec processes in the http server, +%% hangs under the httpd_instance_sup_<Addr>_<Port> supervisor. +%%---------------------------------------------------------------------- + +-module(httpd_misc_sup). + +-behaviour(supervisor). + +%% API +-export([start_link/2, start_auth_server/2, stop_auth_server/2, + start_sec_server/2, stop_sec_server/2]). + +%% Supervisor callback +-export([init/1]). + +%%%========================================================================= +%%% API +%%%========================================================================= + +start_link(Addr, Port) -> + SupName = make_name(Addr, Port), + supervisor:start_link({local, SupName}, ?MODULE, []). + +%%---------------------------------------------------------------------- +%% Function: [start|stop]_[auth|sec]_server/3 +%% Description: Starts a [auth | security] worker (child) process +%%---------------------------------------------------------------------- +start_auth_server(Addr, Port) -> + start_permanent_worker(mod_auth_server, Addr, Port, [gen_server]). + +stop_auth_server(Addr, Port) -> + stop_permanent_worker(mod_auth_server, Addr, Port). + + +start_sec_server(Addr, Port) -> + start_permanent_worker(mod_security_server, Addr, Port, [gen_server]). + +stop_sec_server(Addr, Port) -> + stop_permanent_worker(mod_security_server, Addr, Port). + + +%%%========================================================================= +%%% Supervisor callback +%%%========================================================================= +init(_) -> + Flags = {one_for_one, 0, 1}, + Workers = [], + {ok, {Flags, Workers}}. + +%%%========================================================================= +%%% Internal functions +%%%========================================================================= +start_permanent_worker(Mod, Addr, Port, Modules) -> + SupName = make_name(Addr, Port), + Spec = {{Mod, Addr, Port}, + {Mod, start_link, [Addr, Port]}, + permanent, timer:seconds(1), worker, [Mod] ++ Modules}, + supervisor:start_child(SupName, Spec). + +stop_permanent_worker(Mod, Addr, Port) -> + SupName = make_name(Addr, Port), + Name = {Mod, Addr, Port}, + case supervisor:terminate_child(SupName, Name) of + ok -> + supervisor:delete_child(SupName, Name); + Error -> + Error + end. + +make_name(Addr,Port) -> + httpd_util:make_name("httpd_misc_sup",Addr,Port). diff --git a/lib/inets/src/http_server/httpd_request.erl b/lib/inets/src/http_server/httpd_request.erl new file mode 100644 index 0000000000..ad2cc4bda3 --- /dev/null +++ b/lib/inets/src/http_server/httpd_request.erl @@ -0,0 +1,379 @@ +%% +%% %CopyrightBegin% +%% +%% Copyright Ericsson AB 2005-2009. All Rights Reserved. +%% +%% The contents of this file are subject to the Erlang Public License, +%% Version 1.1, (the "License"); you may not use this file except in +%% compliance with the License. You should have received a copy of the +%% Erlang Public License along with this software. If not, it can be +%% retrieved online at http://www.erlang.org/. +%% +%% Software distributed under the License is distributed on an "AS IS" +%% basis, WITHOUT WARRANTY OF ANY KIND, either express or implied. See +%% the License for the specific language governing rights and limitations +%% under the License. +%% +%% %CopyrightEnd% +%% + +-module(httpd_request). + +-include("http_internal.hrl"). +-include("httpd.hrl"). + +-export([parse/1, whole_body/2, validate/3, update_mod_data/5, + body_data/2]). + +%% Callback API - used for example if the header/body is received a +%% little at a time on a socket. +-export([parse_method/1, parse_uri/1, parse_version/1, parse_headers/1, + whole_body/1]). + +%%%========================================================================= +%%% Internal application API +%%%========================================================================= +parse([Bin, MaxSizes]) -> + parse_method(Bin, [], MaxSizes, []). + +%% Functions that may be returned during the decoding process +%% if the input data is incompleate. +parse_method([Bin, Method, MaxSizes, Result]) -> + parse_method(Bin, Method, MaxSizes, Result). + +parse_uri([Bin, URI, CurrSize, MaxSizes, Result]) -> + parse_uri(Bin, URI, CurrSize, MaxSizes, Result). + +parse_version([Bin, Rest, Version, MaxSizes, Result]) -> + parse_version(<<Rest/binary, Bin/binary>>, Version, MaxSizes, + Result). + +parse_headers([Bin, Rest, Header, Headers, CurrSize, MaxSizes, Result]) -> + parse_headers(<<Rest/binary, Bin/binary>>, + Header, Headers, CurrSize, MaxSizes, Result). + +whole_body([Bin, Body, Length]) -> + whole_body(<<Body/binary, Bin/binary>>, Length). + + +%% Separate the body for this request from a possible piplined new +%% request and convert the body data to "string" format. +body_data(Headers, Body) -> + ContentLength = list_to_integer(Headers#http_request_h.'content-length'), + case size(Body) - ContentLength of + 0 -> + {binary_to_list(Body), <<>>}; + _ -> + <<BodyThisReq:ContentLength/binary, Next/binary>> = Body, + {binary_to_list(BodyThisReq), Next} + end. + +%%------------------------------------------------------------------------- +%% validate(Method, Uri, Version) -> ok | {error, {bad_request, Reason} | +%% {error, {not_supported, {Method, Uri, Version}} +%% Method = "HEAD" | "GET" | "POST" | "TRACE" +%% Uri = uri() +%% Version = "HTTP/N.M" +%% Description: Checks that HTTP-request-line is valid. +%%------------------------------------------------------------------------- +validate("HEAD", Uri, "HTTP/1." ++ _N) -> + validate_uri(Uri); +validate("GET", Uri, []) -> %% Simple HTTP/0.9 + validate_uri(Uri); +validate("GET", Uri, "HTTP/0.9") -> + validate_uri(Uri); +validate("GET", Uri, "HTTP/1." ++ _N) -> + validate_uri(Uri); +validate("POST", Uri, "HTTP/1." ++ _N) -> + validate_uri(Uri); +validate("TRACE", Uri, "HTTP/1." ++ N) when hd(N) >= $1 -> + validate_uri(Uri); +validate(Method, Uri, Version) -> + {error, {not_supported, {Method, Uri, Version}}}. + +%%---------------------------------------------------------------------- +%% The request is passed through the server as a record of type mod +%% create it. +%% ---------------------------------------------------------------------- +update_mod_data(ModData, Method, RequestURI, HTTPVersion, Headers)-> + ParsedHeaders = tagup_header(Headers), + PersistentConn = get_persistens(HTTPVersion, ParsedHeaders, + ModData#mod.config_db), + {ok, ModData#mod{data = [], + method = Method, + absolute_uri = format_absolute_uri(RequestURI, + ParsedHeaders), + request_uri = format_request_uri(RequestURI), + http_version = HTTPVersion, + request_line = Method ++ " " ++ RequestURI ++ + " " ++ HTTPVersion, + parsed_header = ParsedHeaders, + connection = PersistentConn}}. + +%%%======================================================================== +%%% Internal functions +%%%======================================================================== +parse_method(<<>>, Method, MaxSizes, Result) -> + {?MODULE, parse_method, [Method, MaxSizes, Result]}; +parse_method(<<?SP, Rest/binary>>, Method, MaxSizes, Result) -> + parse_uri(Rest, [], 0, MaxSizes, + [string:strip(lists:reverse(Method)) | Result]); +parse_method(<<Octet, Rest/binary>>, Method, MaxSizes, Result) -> + parse_method(Rest, [Octet | Method], MaxSizes, Result). + +parse_uri(_, _, CurrSize, {MaxURI, _}, _) when CurrSize > MaxURI, + MaxURI =/= nolimit -> + %% We do not know the version of the client as it comes after the + %% uri send the lowest version in the response so that the client + %% will be able to handle it. + HttpVersion = "HTTP/0.9", + {error, {uri_too_long, MaxURI}, HttpVersion}; +parse_uri(<<>>, URI, CurrSize, MaxSizes, Result) -> + {?MODULE, parse_uri, [URI, CurrSize, MaxSizes, Result]}; +parse_uri(<<?SP, Rest/binary>>, URI, _, MaxSizes, Result) -> + parse_version(Rest, [], MaxSizes, + [string:strip(lists:reverse(URI)) | Result]); +%% Can happen if it is a simple HTTP/0.9 request e.i "GET /\r\n\r\n" +parse_uri(<<?CR, _Rest/binary>> = Data, URI, _,MaxSizes, Result) -> + parse_version(Data, [], MaxSizes, + [string:strip(lists:reverse(URI)) | Result]); +parse_uri(<<Octet, Rest/binary>>, URI, CurrSize, MaxSizes, Result) -> + parse_uri(Rest, [Octet | URI], CurrSize + 1, MaxSizes, Result). + +parse_version(<<>>, Version, MaxSizes, Result) -> + {?MODULE, parse_version, [<<>>, Version, MaxSizes, Result]}; +parse_version(<<?LF, Rest/binary>>, Version, MaxSizes, Result) -> + %% If ?CR is is missing RFC2616 section-19.3 + parse_version(<<?CR, ?LF, Rest/binary>>, Version, MaxSizes, Result); +parse_version(<<?CR, ?LF, Rest/binary>>, Version, MaxSizes, Result) -> + parse_headers(Rest, [], [], 0, MaxSizes, + [string:strip(lists:reverse(Version)) | Result]); +parse_version(<<?CR>> = Data, Version, MaxSizes, Result) -> + {?MODULE, parse_version, [Data, Version, MaxSizes, Result]}; +parse_version(<<Octet, Rest/binary>>, Version, MaxSizes, Result) -> + parse_version(Rest, [Octet | Version], MaxSizes, Result). + +parse_headers(_, _, _, CurrSize, {_, MaxHeaderSize}, Result) + when CurrSize > MaxHeaderSize, MaxHeaderSize =/= nolimit -> + HttpVersion = lists:nth(3, lists:reverse(Result)), + {error, {header_too_long, MaxHeaderSize}, HttpVersion}; + +parse_headers(<<>>, Header, Headers, CurrSize, MaxSizes, Result) -> + {?MODULE, parse_headers, [<<>>, Header, Headers, CurrSize, + MaxSizes, Result]}; +parse_headers(<<?CR,?LF,?LF,Body/binary>>, [], [], CurrSize, MaxSizes, Result) -> + %% If ?CR is is missing RFC2616 section-19.3 + parse_headers(<<?CR,?LF,?CR,?LF,Body/binary>>, [], [], CurrSize, + MaxSizes, Result); + +parse_headers(<<?LF,?LF,Body/binary>>, [], [], CurrSize, MaxSizes, Result) -> + %% If ?CR is is missing RFC2616 section-19.3 + parse_headers(<<?CR,?LF,?CR,?LF,Body/binary>>, [], [], CurrSize, + MaxSizes, Result); + +parse_headers(<<?CR,?LF,?CR,?LF,Body/binary>>, [], [], _, _, Result) -> + NewResult = list_to_tuple(lists:reverse([Body, {#http_request_h{}, []} | + Result])), + {ok, NewResult}; +parse_headers(<<?CR,?LF,?CR,?LF,Body/binary>>, Header, Headers, _, + _, Result) -> + HTTPHeaders = [lists:reverse(Header) | Headers], + RequestHeaderRcord = + http_request:headers(HTTPHeaders, #http_request_h{}), + NewResult = + list_to_tuple(lists:reverse([Body, {RequestHeaderRcord, + HTTPHeaders} | Result])), + {ok, NewResult}; + +parse_headers(<<?CR,?LF,?CR>> = Data, Header, Headers, CurrSize, + MaxSizes, Result) -> + {?MODULE, parse_headers, [Data, Header, Headers, CurrSize, + MaxSizes, Result]}; +parse_headers(<<?LF>>, [], [], CurrSize, MaxSizes, Result) -> + %% If ?CR is is missing RFC2616 section-19.3 + parse_headers(<<?CR,?LF>>, [], [], CurrSize, MaxSizes, Result); + +%% There where no headers, which is unlikely to happen. +parse_headers(<<?CR,?LF>>, [], [], _, _, Result) -> + NewResult = list_to_tuple(lists:reverse([<<>>, {#http_request_h{}, []} | + Result])), + {ok, NewResult}; + +parse_headers(<<?LF>>, Header, Headers, CurrSize, + MaxSizes, Result) -> + %% If ?CR is is missing RFC2616 section-19.3 + parse_headers(<<?CR,?LF>>, Header, Headers, CurrSize, MaxSizes, Result); + +parse_headers(<<?CR,?LF>> = Data, Header, Headers, CurrSize, + MaxSizes, Result) -> + {?MODULE, parse_headers, [Data, Header, Headers, CurrSize, + MaxSizes, Result]}; +parse_headers(<<?LF, Octet, Rest/binary>>, Header, Headers, CurrSize, + MaxSizes, Result) -> + %% If ?CR is is missing RFC2616 section-19.3 + parse_headers(<<?CR,?LF, Octet, Rest/binary>>, Header, Headers, CurrSize, + MaxSizes, Result); +parse_headers(<<?CR,?LF, Octet, Rest/binary>>, Header, Headers, CurrSize, + MaxSizes, Result) -> + parse_headers(Rest, [Octet], [lists:reverse(Header) | Headers], + CurrSize + 1, MaxSizes, Result); + +parse_headers(<<?CR>> = Data, Header, Headers, CurrSize, + MaxSizes, Result) -> + {?MODULE, parse_headers, [Data, Header, Headers, CurrSize, + MaxSizes, Result]}; +parse_headers(<<?LF>>, Header, Headers, CurrSize, + MaxSizes, Result) -> + %% If ?CR is is missing RFC2616 section-19.3 + parse_headers(<<?CR, ?LF>>, Header, Headers, CurrSize, + MaxSizes, Result); + +parse_headers(<<Octet, Rest/binary>>, Header, Headers, + CurrSize, MaxSizes, Result) -> + parse_headers(Rest, [Octet | Header], Headers, CurrSize + 1, + MaxSizes, Result). + +whole_body(Body, Length) -> + case size(Body) of + N when N < Length, Length > 0 -> + {?MODULE, whole_body, [Body, Length]}; + N when N >= Length, Length >= 0 -> + %% When a client uses pipelining trailing data + %% may be part of the next request! + %% Trailing data will be separated from + %% the actual body in body_data/2. + {ok, Body} + end. + +%% Prevent people from trying to access directories/files +%% relative to the ServerRoot. +validate_uri(RequestURI) -> + UriNoQueryNoHex = + case string:str(RequestURI, "?") of + 0 -> + (catch httpd_util:decode_hex(RequestURI)); + Ndx -> + (catch httpd_util:decode_hex(string:left(RequestURI, Ndx))) + end, + case UriNoQueryNoHex of + {'EXIT',_Reason} -> + {error, {bad_request, {malformed_syntax, RequestURI}}}; + _ -> + Path = format_request_uri(UriNoQueryNoHex), + Path2=[X||X<-string:tokens(Path, "/"),X=/="."], %% OTP-5938 + validate_path( Path2,0, RequestURI) + end. + +validate_path([], _, _) -> + ok; +validate_path([".." | _], 0, RequestURI) -> + {error, {bad_request, {forbidden, RequestURI}}}; +validate_path([".." | Rest], N, RequestURI) -> + validate_path(Rest, N - 1, RequestURI); +validate_path([_ | Rest], N, RequestURI) -> + validate_path(Rest, N + 1, RequestURI). + +%%---------------------------------------------------------------------- +%% There are 3 possible forms of the reuqest URI +%% +%% 1. * When the request is not for a special assset. is is instead +%% to the server itself +%% +%% 2. absoluteURI the whole servername port and asset is in the request +%% +%% 3. The most common form that http/1.0 used abs path that is a path +%% to the requested asset. +%%---------------------------------------------------------------------- +format_request_uri("*")-> + "*"; +format_request_uri("http://" ++ ServerAndPath) -> + remove_server(ServerAndPath); + +format_request_uri("HTTP://" ++ ServerAndPath) -> + remove_server(ServerAndPath); + +format_request_uri(ABSPath) -> + ABSPath. + +remove_server([]) -> + "/"; +remove_server([$\/|Url])-> + case Url of + []-> + "/"; + _-> + [$\/|Url] + end; +remove_server([_|Url]) -> + remove_server(Url). + +format_absolute_uri("http://"++ Uri, _)-> + "HTTP://" ++ Uri; + +format_absolute_uri(OrigUri = "HTTP://" ++ _, _)-> + OrigUri; + +format_absolute_uri(Uri,ParsedHeader)-> + case proplists:get_value("host", ParsedHeader) of + undefined -> + nohost; + Host -> + Host++Uri + end. + +get_persistens(HTTPVersion,ParsedHeader,ConfigDB)-> + case httpd_util:lookup(ConfigDB, keep_alive, true) of + true-> + case HTTPVersion of + %%If it is version prio to 1.1 kill the conneciton + "HTTP/1." ++ NList -> + case proplists:get_value("connection", ParsedHeader, + "keep-alive") of + %%if the connection is not ordered to go down + %%let it live The keep-alive value is the + %%older http/1.1 might be older Clients that + %%use it. + "keep-alive" when hd(NList) >= 49 -> + ?DEBUG("CONNECTION MODE: ~p",[true]), + true; + "close" -> + ?DEBUG("CONNECTION MODE: ~p",[false]), + false; + _Connect -> + ?DEBUG("CONNECTION MODE: ~p VALUE: ~p", + [false, _Connect]), + false + end; + _ -> + ?DEBUG("CONNECTION MODE: ~p VERSION: ~p", + [false, HTTPVersion]), + false + end; + _ -> + false + end. + + +%%---------------------------------------------------------------------- +%% tagup_header +%% +%% Parses the header of a HTTP request and returns a key,value tuple +%% list containing Name and Value of each header directive as of: +%% +%% Content-Type: multipart/mixed -> {"Content-Type", "multipart/mixed"} +%% +%% But in http/1.1 the field-names are case insencitive so now it must be +%% Content-Type: multipart/mixed -> {"content-type", "multipart/mixed"} +%% The standard furthermore says that leading and traling white space +%% is not a part of the fieldvalue and shall therefore be removed. +%%---------------------------------------------------------------------- +tagup_header([]) -> []; +tagup_header([Line|Rest]) -> [tag(Line, [])|tagup_header(Rest)]. + +tag([], Tag) -> + {http_util:to_lower(lists:reverse(Tag)), ""}; +tag([$:|Rest], Tag) -> + {http_util:to_lower(lists:reverse(Tag)), string:strip(Rest)}; +tag([Chr|Rest], Tag) -> + tag(Rest, [Chr|Tag]). + diff --git a/lib/inets/src/http_server/httpd_request_handler.erl b/lib/inets/src/http_server/httpd_request_handler.erl new file mode 100644 index 0000000000..fa832cba3f --- /dev/null +++ b/lib/inets/src/http_server/httpd_request_handler.erl @@ -0,0 +1,611 @@ +%% +%% %CopyrightBegin% +%% +%% Copyright Ericsson AB 1997-2009. All Rights Reserved. +%% +%% The contents of this file are subject to the Erlang Public License, +%% Version 1.1, (the "License"); you may not use this file except in +%% compliance with the License. You should have received a copy of the +%% Erlang Public License along with this software. If not, it can be +%% retrieved online at http://www.erlang.org/. +%% +%% Software distributed under the License is distributed on an "AS IS" +%% basis, WITHOUT WARRANTY OF ANY KIND, either express or implied. See +%% the License for the specific language governing rights and limitations +%% under the License. +%% +%% %CopyrightEnd% +%% +%% +%% Description: Implements a request handler process for the HTTP server. +%% + +-module(httpd_request_handler). + +-behaviour(gen_server). + +%% Application internal API +-export([start/2, start/3, socket_ownership_transfered/3]). + +%% gen_server callbacks +-export([init/1, handle_call/3, handle_cast/2, handle_info/2, + terminate/2, code_change/3]). + +-include("httpd.hrl"). +-include("http_internal.hrl"). +-include("httpd_internal.hrl"). + +-record(state, {mod, %% #mod{} + manager, %% pid() + status, %% accept | busy | blocked + mfa, %% {Module, Function, Args} + max_keep_alive_request = infinity, %% integer() | infinity + response_sent = false, %% true | false + timeout, %% infinity | integer() > 0 + timer, %% ref() - Request timer + headers, %% #http_request_h{} + body %% binary() + }). + +%%==================================================================== +%% Application internal API +%%==================================================================== +%%-------------------------------------------------------------------- +%% Function: start() -> {ok, Pid} | ignore | {error,Error} +%% Description: Starts a httpd-request handler process. Intended to be +%% called by the httpd acceptor process. +%%-------------------------------------------------------------------- +start(Manager, ConfigDB) -> + start(Manager, ConfigDB, 15000). +start(Manager, ConfigDB, AcceptTimeout) -> + proc_lib:start(?MODULE, init, [[Manager, ConfigDB,AcceptTimeout]]). + + +%%-------------------------------------------------------------------- +%% socket_ownership_transfered(Pid, SocketType, Socket) -> void() +%% +%% Pid = pid() +%% SocketType = ip_comm | ssl +%% Socket = socket() +%% +%% Description: Send a message to the request handler process +%% confirming that the socket ownership has now sucssesfully been +%% transfered to it. Intended to be called by the httpd acceptor +%% process. +%%-------------------------------------------------------------------- +socket_ownership_transfered(Pid, SocketType, Socket) -> + Pid ! {socket_ownership_transfered, SocketType, Socket}. + +%%-------------------------------------------------------------------- +%% Function: init(Args) -> _ +%% +%% Description: Initiates the server. Obs special init that uses +%% gen_server:enter_loop/3. This is used instead of the normal +%% gen_server callback init, as a more complex init than the +%% gen_server provides is needed. +%%-------------------------------------------------------------------- +init([Manager, ConfigDB, AcceptTimeout]) -> + ?hdrd("initiate", + [{manager, Manager}, {cdb, ConfigDB}, {timeout, AcceptTimeout}]), + %% Make sure this process terminates if the httpd manager process + %% should die! + link(Manager), + %% At this point the function httpd_request_handler:start/2 will return. + proc_lib:init_ack({ok, self()}), + + {SocketType, Socket} = await_socket_ownership_transfer(AcceptTimeout), + ?hdrd("socket ownership transfered", + [{socket_type, SocketType}, {socket, Socket}]), + + TimeOut = httpd_util:lookup(ConfigDB, keep_alive_timeout, 150000), + + Then = erlang:now(), + + case http_transport:negotiate(SocketType, Socket, TimeOut) of + {error, Error} -> + exit(Error); %% Can be 'normal'. + ok -> + ?hdrt("negotiated", []), + NewTimeout = TimeOut - timer:now_diff(now(),Then) div 1000, + continue_init(Manager, ConfigDB, SocketType, Socket, NewTimeout) + end. + +continue_init(Manager, ConfigDB, SocketType, Socket, TimeOut) -> + ?hdrt("continue init", [{timeout, TimeOut}]), + Resolve = http_transport:resolve(), + + Peername = httpd_socket:peername(SocketType, Socket), + InitData = #init_data{peername = Peername, resolve = Resolve}, + Mod = #mod{config_db = ConfigDB, + socket_type = SocketType, + socket = Socket, + init_data = InitData}, + + MaxHeaderSize = httpd_util:lookup(ConfigDB, max_header_size, + ?HTTP_MAX_HEADER_SIZE), + MaxURISize = httpd_util:lookup(ConfigDB, max_uri_size, + ?HTTP_MAX_URI_SIZE), + NrOfRequest = httpd_util:lookup(ConfigDB, + max_keep_alive_request, infinity), + + {_, Status} = httpd_manager:new_connection(Manager), + + MFA = {httpd_request, parse, [{MaxURISize, MaxHeaderSize}]}, + + State = #state{mod = Mod, + manager = Manager, + status = Status, + timeout = TimeOut, + max_keep_alive_request = NrOfRequest, + mfa = MFA}, + + ?hdrt("activate request timeout", []), + NewState = activate_request_timeout(State), + + ?hdrt("update socket options", []), + http_transport:setopts(SocketType, Socket, [binary,{packet, 0}, + {active, once}]), + ?hdrt("init done", []), + gen_server:enter_loop(?MODULE, [], NewState). + + +%%==================================================================== +%% gen_server callbacks +%%==================================================================== + +%%-------------------------------------------------------------------- +%% handle_call(Request, From, State) -> {reply, Reply, State} | +%% {reply, Reply, State, Timeout} | +%% {noreply, State} | +%% {noreply, State, Timeout} | +%% {stop, Reason, Reply, State} | +%% {stop, Reason, State} +%% Description: Handling call messages +%%-------------------------------------------------------------------- +handle_call(Request, From, State) -> + {stop, {call_api_violation, Request, From}, State}. + +%%-------------------------------------------------------------------- +%% handle_cast(Msg, State) -> {noreply, State} | +%% {noreply, State, Timeout} | +%% {stop, Reason, State} +%% Description: Handling cast messages +%%-------------------------------------------------------------------- +handle_cast(Msg, State) -> + {reply, {cast_api_violation, Msg}, State}. + +%%-------------------------------------------------------------------- +%% handle_info(Info, State) -> {noreply, State} | +%% {noreply, State, Timeout} | +%% {stop, Reason, State} +%% Description: Handling all non call/cast messages +%%-------------------------------------------------------------------- +handle_info({Proto, Socket, Data}, State = + #state{mfa = {Module, Function, Args} = MFA, + mod = #mod{socket_type = SockType, + socket = Socket} = ModData} = State) + when (((Proto =:= tcp) orelse + (Proto =:= ssl) orelse + (Proto =:= dummy)) andalso is_binary(Data)) -> + ?hdrd("received data", + [{data, Data}, {proto, Proto}, + {socket, Socket}, {socket_type, SockType}, {mfa, MFA}]), + case Module:Function([Data | Args]) of + {ok, Result} -> + ?hdrd("data processed", [{result, Result}]), + NewState = cancel_request_timeout(State), + handle_http_msg(Result, NewState); + {error, {uri_too_long, MaxSize}, Version} -> + ?hdrv("uri too long", [{max_size, MaxSize}, {version, Version}]), + NewModData = ModData#mod{http_version = Version}, + httpd_response:send_status(NewModData, 414, "URI too long"), + Reason = io_lib:format("Uri too long, max size is ~p~n", + [MaxSize]), + error_log(Reason, NewModData), + {stop, normal, State#state{response_sent = true, + mod = NewModData}}; + {error, {header_too_long, MaxSize}, Version} -> + ?hdrv("header too long", [{max_size, MaxSize}, {version, Version}]), + NewModData = ModData#mod{http_version = Version}, + httpd_response:send_status(NewModData, 413, "Header too long"), + Reason = io_lib:format("Header too long, max size is ~p~n", + [MaxSize]), + error_log(Reason, NewModData), + {stop, normal, State#state{response_sent = true, + mod = NewModData}}; + NewMFA -> + ?hdrd("data processed - reactivate socket", [{new_mfa, NewMFA}]), + http_transport:setopts(SockType, Socket, [{active, once}]), + {noreply, State#state{mfa = NewMFA}} + end; + +%% Error cases +handle_info({tcp_closed, _}, State) -> + {stop, normal, State}; +handle_info({ssl_closed, _}, State) -> + {stop, normal, State}; +handle_info({tcp_error, _, _} = Reason, State) -> + {stop, Reason, State}; +handle_info({ssl_error, _, _} = Reason, State) -> + {stop, Reason, State}; + +%% Timeouts +handle_info(timeout, #state{mod = ModData, mfa = {_, parse, _}} = State) -> + error_log("No request received on keep-alive connection" + "before server side timeout", ModData), + %% No response should be sent! + {stop, normal, State#state{response_sent = true}}; +handle_info(timeout, #state{mod = ModData} = State) -> + httpd_response:send_status(ModData, 408, "Request timeout"), + error_log("The client did not send the whole request before the" + "server side timeout", ModData), + {stop, normal, State#state{response_sent = true}}; + +%% Default case +handle_info(Info, #state{mod = ModData} = State) -> + Error = lists:flatten( + io_lib:format("Unexpected message received: ~n~p~n", [Info])), + error_log(Error, ModData), + {noreply, State}. + + +%%-------------------------------------------------------------------- +%% terminate(Reason, State) -> void() +%% +%% Description: This function is called by a gen_server when it is about to +%% terminate. It should be the opposite of Module:init/1 and do any necessary +%% cleaning up. When it returns, the gen_server terminates with Reason. +%% The return value is ignored. +%%-------------------------------------------------------------------- +terminate(normal, State) -> + do_terminate(State); +terminate(Reason, #state{response_sent = false, mod = ModData} = State) -> + httpd_response:send_status(ModData, 500, none), + error_log(httpd_util:reason_phrase(500), ModData), + terminate(Reason, State#state{response_sent = true, mod = ModData}); +terminate(_, State) -> + do_terminate(State). + +do_terminate(#state{mod = ModData, manager = Manager} = State) -> + catch httpd_manager:done_connection(Manager), + cancel_request_timeout(State), + httpd_socket:close(ModData#mod.socket_type, ModData#mod.socket). + +%%-------------------------------------------------------------------- +%% code_change(OldVsn, State, Extra) -> {ok, NewState} +%% +%% Description: Convert process state when code is changed +%%-------------------------------------------------------------------- +code_change(_OldVsn, State, _Extra) -> + {ok, State}. + +%%-------------------------------------------------------------------- +%%% Internal functions +%%-------------------------------------------------------------------- +await_socket_ownership_transfer(AcceptTimeout) -> + receive + {socket_ownership_transfered, SocketType, Socket} -> + {SocketType, Socket} + after AcceptTimeout -> + exit(accept_socket_timeout) + end. + +handle_http_msg({_, _, Version, {_, _}, _}, + #state{status = busy, mod = ModData} = State) -> + ?hdrt("handle http msg when manager busy", [{mod, ModData}]), + handle_manager_busy(State#state{mod = + ModData#mod{http_version = Version}}), + {stop, normal, State}; + +handle_http_msg({_, _, Version, {_, _}, _}, + #state{status = blocked, mod = ModData} = State) -> + ?hdrt("handle http msg when manager blocket", [{mod, ModData}]), + handle_manager_blocked(State#state{mod = + ModData#mod{http_version = Version}}), + {stop, normal, State}; + +handle_http_msg({Method, Uri, Version, {RecordHeaders, Headers}, Body}, + #state{status = accept, mod = ModData} = State) -> + ?hdrt("handle http msg when manager accepting", + [{method, Method}, {mod, ModData}]), + case httpd_request:validate(Method, Uri, Version) of + ok -> + ?hdrt("request validated", []), + {ok, NewModData} = + httpd_request:update_mod_data(ModData, Method, Uri, + Version, Headers), + + ?hdrt("new mod data", [{mod, NewModData}]), + case is_host_specified_if_required(NewModData#mod.absolute_uri, + RecordHeaders, Version) of + true -> + handle_body(State#state{headers = RecordHeaders, + body = Body, + mod = NewModData}); + false -> + httpd_response:send_status(ModData#mod{http_version = + Version}, + 400, none), + {stop, normal, State#state{response_sent = true}} + end; + {error, {not_supported, What}} -> + ?hdrd("validation failed: not supported", [{what, What}]), + httpd_response:send_status(ModData#mod{http_version = Version}, + 501, {Method, Uri, Version}), + Reason = io_lib:format("Not supported: ~p~n", [What]), + error_log(Reason, ModData), + {stop, normal, State#state{response_sent = true}}; + {error, {bad_request, {forbidden, URI}}} -> + ?hdrd("validation failed: bad request - forbidden", + [{uri, URI}]), + httpd_response:send_status(ModData#mod{http_version = Version}, + 403, URI), + Reason = io_lib:format("Forbidden URI: ~p~n", [URI]), + error_log(Reason, ModData), + {stop, normal, State#state{response_sent = true}}; + {error,{bad_request, {malformed_syntax, URI}}} -> + ?hdrd("validation failed: bad request - malformed syntax", + [{uri, URI}]), + httpd_response:send_status(ModData#mod{http_version = Version}, + 400, URI), + Reason = io_lib:format("Malformed syntax in URI: ~p~n", [URI]), + error_log(Reason, ModData), + {stop, normal, State#state{response_sent = true}} + end; +handle_http_msg({ChunkedHeaders, Body}, + State = #state{headers = Headers}) -> + ?hdrt("handle http msg", + [{chunked_headers, ChunkedHeaders}, {body, Body}]), + NewHeaders = http_chunk:handle_headers(Headers, ChunkedHeaders), + handle_response(State#state{headers = NewHeaders, body = Body}); +handle_http_msg(Body, State) -> + ?hdrt("handle http msg", [{body, Body}]), + handle_response(State#state{body = Body}). + +handle_manager_busy(#state{mod = #mod{config_db = ConfigDB}} = State) -> + MaxClients = httpd_util:lookup(ConfigDB, max_clients, 150), + Reason = io_lib:format("heavy load (>~w processes)", [MaxClients]), + reject_connection(State, lists:flatten(Reason)). + +handle_manager_blocked(State) -> + Reason = "Server maintenance performed, try again later", + reject_connection(State, Reason). + +reject_connection(#state{mod = ModData} = State, Reason) -> + httpd_response:send_status(ModData, 503, Reason), + {stop, normal, State#state{response_sent = true}}. + +is_host_specified_if_required(nohost, #http_request_h{host = undefined}, + "HTTP/1.1") -> + false; +is_host_specified_if_required(_, _, _) -> + true. + +handle_body(#state{mod = #mod{config_db = ConfigDB}} = State) -> + ?hdrt("handle body", []), + MaxHeaderSize = + httpd_util:lookup(ConfigDB, max_header_size, ?HTTP_MAX_HEADER_SIZE), + MaxBodySize = httpd_util:lookup(ConfigDB, max_body_size, nolimit), + + case handle_expect(State, MaxBodySize) of + ok -> + handle_body(State, MaxHeaderSize, MaxBodySize); + Other -> + Other + + end. + +handle_body(#state{headers = Headers, body = Body, mod = ModData} = State, + MaxHeaderSize, MaxBodySize) -> + ?hdrt("handle body", [{headers, Headers}, {body, Body}]), + case Headers#http_request_h.'transfer-encoding' of + "chunked" -> + ?hdrt("chunked - attempt decode", []), + case http_chunk:decode(Body, MaxBodySize, MaxHeaderSize) of + {Module, Function, Args} -> + ?hdrt("chunk decoded", + [{module, Module}, + {function, Function}, + {args, Args}]), + http_transport:setopts(ModData#mod.socket_type, + ModData#mod.socket, + [{active, once}]), + {noreply, State#state{mfa = + {Module, Function, Args}}}; + {ok, {ChunkedHeaders, NewBody}} -> + ?hdrt("chunk decoded", + [{chunked_headers, ChunkedHeaders}, + {new_body, NewBody}]), + NewHeaders = + http_chunk:handle_headers(Headers, ChunkedHeaders), + ?hdrt("chunked - headers handled", + [{new_headers, NewHeaders}]), + handle_response(State#state{headers = NewHeaders, + body = NewBody}) + end; + Encoding when is_list(Encoding) -> + ?hdrt("not chunked - encoding", [{encoding, Encoding}]), + httpd_response:send_status(ModData, 501, + "Unknown Transfer-Encoding"), + Reason = io_lib:format("Unknown Transfer-Encoding: ~p~n", + [Encoding]), + error_log(Reason, ModData), + {stop, normal, State#state{response_sent = true}}; + _ -> + ?hdrt("not chunked", []), + Length = + list_to_integer(Headers#http_request_h.'content-length'), + case ((Length =< MaxBodySize) or (MaxBodySize == nolimit)) of + true -> + case httpd_request:whole_body(Body, Length) of + {Module, Function, Args} -> + ?hdrt("whole body", + [{module, Module}, + {function, Function}, + {args, Args}]), + http_transport:setopts(ModData#mod.socket_type, + ModData#mod.socket, + [{active, once}]), + {noreply, State#state{mfa = + {Module, Function, Args}}}; + + {ok, NewBody} -> + ?hdrt("whole body", + [{new_body, NewBody}]), + handle_response( + State#state{headers = Headers, + body = NewBody}) + end; + false -> + ?hdrd("body too long", + [{length, Length}, {max_body_size, MaxBodySize}]), + httpd_response:send_status(ModData, 413, "Body too long"), + error_log("Body too long", ModData), + {stop, normal, State#state{response_sent = true}} + end + end. + +handle_expect(#state{headers = Headers, mod = + #mod{config_db = ConfigDB} = ModData} = State, + MaxBodySize) -> + Length = Headers#http_request_h.'content-length', + case expect(Headers, ModData#mod.http_version, ConfigDB) of + continue when (MaxBodySize > Length) orelse (MaxBodySize =:= nolimit) -> + httpd_response:send_status(ModData, 100, ""), + ok; + continue when MaxBodySize < Length -> + httpd_response:send_status(ModData, 413, "Body too long"), + error_log("Body too long", ModData), + {stop, normal, State#state{response_sent = true}}; + {break, Value} -> + httpd_response:send_status(ModData, 417, + "Unexpected expect value"), + Reason = io_lib:format("Unexpected expect value: ~p~n", [Value]), + error_log(Reason, ModData), + {stop, normal, State#state{response_sent = true}}; + no_expect_header -> + ok; + http_1_0_expect_header -> + httpd_response:send_status(ModData, 400, + "Only HTTP/1.1 Clients " + "may use the Expect Header"), + error_log("Client with lower version than 1.1 tried to send" + "an expect header", ModData), + {stop, normal, State#state{response_sent = true}} + end. + +expect(Headers, "HTTP/1.1", _) -> + case Headers#http_request_h.expect of + "100-continue" -> + continue; + undefined -> + no_expect_header; + Other -> + {break, Other} + end; +expect(Headers, _, ConfigDB) -> + case Headers#http_request_h.expect of + undefined -> + no_expect_header; + _ -> + case httpd_util:lookup(ConfigDB, expect, continue) of + continue-> + no_expect_header; + _ -> + http_1_0_expect_header + end + end. + +handle_response(#state{body = Body, + mod = ModData, + headers = Headers, + max_keep_alive_request = Max} = State) when Max > 0 -> + ?hdrt("handle response", + [{body, Body}, {mod, ModData}, {headers, Headers}, {max, Max}]), + {NewBody, Data} = httpd_request:body_data(Headers, Body), + ok = httpd_response:generate_and_send_response( + ModData#mod{entity_body = NewBody}), + handle_next_request(State#state{response_sent = true}, Data); + +handle_response(#state{body = Body, + headers = Headers, + mod = ModData} = State) -> + ?hdrt("handle response", + [{body, Body}, {mod, ModData}, {headers, Headers}]), + {NewBody, _} = httpd_request:body_data(Headers, Body), + ok = httpd_response:generate_and_send_response( + ModData#mod{entity_body = NewBody}), + {stop, normal, State#state{response_sent = true}}. + +handle_next_request(#state{mod = #mod{connection = true} = ModData, + max_keep_alive_request = Max} = State, Data) -> + ?hdrt("handle next request", [{max, Max}]), + NewModData = #mod{socket_type = ModData#mod.socket_type, + socket = ModData#mod.socket, + config_db = ModData#mod.config_db, + init_data = ModData#mod.init_data}, + MaxHeaderSize = + httpd_util:lookup(ModData#mod.config_db, + max_header_size, ?HTTP_MAX_HEADER_SIZE), + MaxURISize = httpd_util:lookup(ModData#mod.config_db, max_uri_size, + ?HTTP_MAX_URI_SIZE), + TmpState = State#state{mod = NewModData, + mfa = {httpd_request, parse, [{MaxURISize, + MaxHeaderSize}]}, + max_keep_alive_request = decrease(Max), + headers = undefined, + body = undefined, + response_sent = false}, + + NewState = activate_request_timeout(TmpState), + + case Data of + <<>> -> + http_transport:setopts(ModData#mod.socket_type, + ModData#mod.socket, [{active, once}]), + {noreply, NewState}; + _ -> + handle_info({dummy, ModData#mod.socket, Data}, NewState) + end; + +handle_next_request(State, _) -> + ?hdrt("handle next request - stop", []), + {stop, normal, State}. + +activate_request_timeout(#state{timeout = Time} = State) -> + ?hdrt("activate request timeout", [{time, Time}]), + Ref = erlang:send_after(Time, self(), timeout), + State#state{timer = Ref}. + +cancel_request_timeout(#state{timer = undefined} = State) -> + State; +cancel_request_timeout(#state{timer = Timer} = State) -> + erlang:cancel_timer(Timer), + receive + timeout -> + ok + after 0 -> + ok + end, + State#state{timer = undefined}. + +decrease(N) when is_integer(N) -> + N-1; +decrease(N) -> + N. + +error_log(ReasonString, Info) -> + Error = lists:flatten( + io_lib:format("Error reading request:~s",[ReasonString])), + error_log(mod_log, Info, Error), + error_log(mod_disk_log, Info, Error). + +error_log(Mod, #mod{config_db = ConfigDB} = Info, String) -> + Modules = httpd_util:lookup(ConfigDB, modules, + [mod_get, mod_head, mod_log]), + case lists:member(Mod, Modules) of + true -> + Mod:error_log(Info, String); + _ -> + ok + end. diff --git a/lib/inets/src/http_server/httpd_response.erl b/lib/inets/src/http_server/httpd_response.erl new file mode 100644 index 0000000000..ea9cfbf4f2 --- /dev/null +++ b/lib/inets/src/http_server/httpd_response.erl @@ -0,0 +1,407 @@ +%% +%% %CopyrightBegin% +%% +%% Copyright Ericsson AB 1997-2009. All Rights Reserved. +%% +%% The contents of this file are subject to the Erlang Public License, +%% Version 1.1, (the "License"); you may not use this file except in +%% compliance with the License. You should have received a copy of the +%% Erlang Public License along with this software. If not, it can be +%% retrieved online at http://www.erlang.org/. +%% +%% Software distributed under the License is distributed on an "AS IS" +%% basis, WITHOUT WARRANTY OF ANY KIND, either express or implied. See +%% the License for the specific language governing rights and limitations +%% under the License. +%% +%% %CopyrightEnd% +%% +%% +-module(httpd_response). +-export([generate_and_send_response/1, send_status/3, send_header/3, + send_body/3, send_chunk/3, send_final_chunk/2, split_header/2, + is_disable_chunked_send/1, cache_headers/1]). +-export([map_status_code/2]). + +-include("httpd.hrl"). +-include("http_internal.hrl"). +-include("httpd_internal.hrl"). + +-define(VMODULE,"RESPONSE"). + +%% If peername does not exist the client already discarded the +%% request so we do not need to send a reply. +generate_and_send_response(#mod{init_data = + #init_data{peername = {_,"unknown"}}}) -> + ok; +generate_and_send_response(#mod{config_db = ConfigDB} = ModData) -> + Modules = httpd_util:lookup(ConfigDB,modules, + [mod_get, mod_head, mod_log]), + case traverse_modules(ModData, Modules) of + done -> + ok; + {proceed, Data} -> + case proplists:get_value(status, Data) of + {StatusCode, PhraseArgs, _Reason} -> + send_status(ModData, StatusCode, PhraseArgs), + ok; + undefined -> + case proplists:get_value(response, Data) of + {already_sent, _StatusCode, _Size} -> + ok; + {response, Header, Body} -> %% New way + send_response(ModData, Header, Body), + ok; + {StatusCode, Response} -> %% Old way + send_response_old(ModData, StatusCode, Response), + ok; + undefined -> + send_status(ModData, 500, none), + ok + end + end + end. + + +%% traverse_modules + +traverse_modules(ModData,[]) -> + {proceed,ModData#mod.data}; +traverse_modules(ModData,[Module|Rest]) -> + ?hdrd("traverse modules", [{callback_module, Module}]), + case (catch apply(Module, do, [ModData])) of + {'EXIT', Reason} -> + ?hdrd("traverse modules - exit", [{reason, Reason}]), + String = + lists:flatten( + io_lib:format("traverse exit from apply: ~p:do => ~n~p", + [Module, Reason])), + report_error(mod_log, ModData#mod.config_db, String), + report_error(mod_disk_log, ModData#mod.config_db, String), + done; + done -> + ?hdrt("traverse modules - done", []), + done; + {break, NewData} -> + ?hdrt("traverse modules - break", [{new_data, NewData}]), + {proceed, NewData}; + {proceed, NewData} -> + ?hdrt("traverse modules - proceed", [{new_data, NewData}]), + traverse_modules(ModData#mod{data = NewData}, Rest) + end. + +%% send_status %% + + +send_status(ModData, 100, _PhraseArgs) -> + send_header(ModData, 100, [{content_length, "0"}]); + +send_status(#mod{socket_type = SocketType, + socket = Socket, + config_db = ConfigDB} = ModData, StatusCode, PhraseArgs) -> + + ReasonPhrase = httpd_util:reason_phrase(StatusCode), + Message = httpd_util:message(StatusCode, PhraseArgs, ConfigDB), + Body = get_body(ReasonPhrase, Message), + + send_header(ModData, StatusCode, [{content_type, "text/html"}, + {content_length, integer_to_list(length(Body))}]), + httpd_socket:deliver(SocketType, Socket, Body). + + +get_body(ReasonPhrase, Message)-> + "<HTML> + <HEAD> + <TITLE>"++ReasonPhrase++"</TITLE> + </HEAD> + <BODY> + <H1>"++ReasonPhrase++"</H1>\n"++Message++"\n</BODY> + </HTML>\n". + + +send_response(ModData, Header, Body) -> + case proplists:get_value(code, Header) of + undefined -> + %% No status code + %% Ooops this must be very bad: + %% generate a 404 content not availible + send_status(ModData, 404, "The file is not availible"); + StatusCode -> + case send_header(ModData, StatusCode, lists:keydelete(code, 1, + Header)) of + ok -> + send_body(ModData, StatusCode, Body); + _ -> + done + end + end. + +send_header(#mod{socket_type = Type, socket = Sock, + http_version = Ver, connection = Conn} = _ModData, + StatusCode, KeyValueTupleHeaders) -> + Headers = create_header(lists:map(fun transform/1, KeyValueTupleHeaders)), + NewVer = case {Ver, StatusCode} of + {[], _} -> + %% May be implicit! + "HTTP/0.9"; + {unknown, 408} -> + %% This will proably never happen! It means the + %% server has timed out the request without + %% receiving a version for the request! Send the + %% lowest version so to ensure that the client + %% will be able to handle it, probably the + %% sensible thing to do! + "HTTP/0.9"; + {undefined,_} -> + "HTTP/1.0"; %% See rfc2145 2.3 last paragraph + _ -> + Ver + end, + NewStatusCode = map_status_code(NewVer, StatusCode), + StatusLine = [NewVer, " ", io_lib:write(NewStatusCode), " ", + httpd_util:reason_phrase(NewStatusCode), ?CRLF], + ConnectionHeader = get_connection(Conn, NewVer), + Head = list_to_binary([StatusLine, Headers, ConnectionHeader , ?CRLF]), + httpd_socket:deliver(Type, Sock, Head). + +map_status_code("HTTP/1.0", Code) + when ((Code div 100) =:= 2) andalso (Code > 204) -> + 403; +map_status_code("HTTP/1.0", Code) + when ((Code div 100) =:= 3) andalso (Code > 304) -> + 403; +map_status_code("HTTP/1.0", Code) + when ((Code div 100) =:= 4) andalso (Code > 404) -> + 403; +map_status_code("HTTP/1.0", Code) + when ((Code div 100) =:= 5) andalso (Code > 503) -> + 403; +map_status_code(_, Code) -> + Code. + +send_body(#mod{socket_type = Type, socket = Socket}, _, nobody) -> + httpd_socket:close(Type, Socket), + ok; + +send_body(#mod{socket_type = Type, socket = Sock}, + _StatusCode, Body) when is_list(Body) -> + case httpd_socket:deliver(Type, Sock, Body) of + socket_closed -> + done; + Else -> + Else + end; + +send_body(#mod{socket_type = Type, socket = Sock} = ModData, + StatusCode, {Fun, Args}) -> + case (catch apply(Fun, Args)) of + close -> + httpd_socket:close(Type, Sock), + done; + + sent -> + {proceed,[{response,{already_sent, StatusCode, + proplists:get_value(content_length, + ModData#mod.data)}}]}; + {ok, Body} -> + case httpd_socket:deliver(Type, Sock, Body) of + ok -> + {proceed,[{response, + {already_sent, StatusCode, + proplists:get_value(content_length, + ModData#mod.data)}}]}; + _ -> + done + end; + + _ -> + done + end. + +split_header([$: | Value], AccName) -> + Name = http_util:to_lower(string:strip(AccName)), + {lists:reverse(Name), + string:strip(string:strip(string:strip(Value, right, ?LF), right, ?CR))}; +split_header([Char | Rest], AccName) -> + split_header(Rest, [Char | AccName]). + +send_chunk(_, <<>>, _) -> + ok; +send_chunk(_, [], _) -> + ok; + +send_chunk(#mod{http_version = "HTTP/1.1", + socket_type = Type, socket = Sock}, Response0, false) -> + Response = http_chunk:encode(Response0), + httpd_socket:deliver(Type, Sock, Response); + +send_chunk(#mod{socket_type = Type, socket = Sock} = _ModData, Response, _) -> + httpd_socket:deliver(Type, Sock, Response). + +send_final_chunk(#mod{http_version = "HTTP/1.1", + socket_type = Type, socket = Sock}, false) -> + httpd_socket:deliver(Type, Sock, http_chunk:encode_last()); +send_final_chunk(#mod{socket_type = Type, socket = Sock}, _) -> + httpd_socket:close(Type, Sock). + +is_disable_chunked_send(Db) -> + httpd_util:lookup(Db, disable_chunked_transfer_encoding_send, false). + +%% Return a HTTP-header field that indicates that the +%% connection will be inpersistent +get_connection(true,"HTTP/1.0")-> + "Connection:close\r\n"; +get_connection(false,"HTTP/1.1") -> + "Connection:close\r\n"; +get_connection(_,_) -> + "". + +cache_headers(#mod{config_db = Db}) -> + case httpd_util:lookup(Db, script_nocache, false) of + true -> + Date = httpd_util:rfc1123_date(), + [{"cache-control", "no-cache"}, + {"pragma", "no-cache"}, + {"expires", Date}]; + false -> + [] + end. + +create_header(KeyValueTupleHeaders) -> + NewHeaders = add_default_headers([{"date", httpd_util:rfc1123_date()}, + {"content-type", "text/html"}, + {"server", ?SERVER_SOFTWARE}], + KeyValueTupleHeaders), + lists:map(fun fix_header/1, NewHeaders). + +fix_header({Key0, Value}) -> + %% make sure first letter is capital + Words1 = string:tokens(Key0, "-"), + Words2 = upify(Words1, []), + Key = new_key(Words2), + Key ++ ": " ++ Value ++ ?CRLF . + +new_key([]) -> + ""; +new_key([W]) -> + W; +new_key([W1,W2]) -> + W1 ++ "-" ++ W2; +new_key([W|R]) -> + W ++ "-" ++ new_key(R). + +upify([], Acc) -> + lists:reverse(Acc); +upify([Key|Rest], Acc) -> + upify(Rest, [upify2(Key)|Acc]). + +upify2([C|Rest]) when (C >= $a) andalso (C =< $z) -> + [C-($a-$A)|Rest]; +upify2(Str) -> + Str. + +add_default_headers([], Headers) -> + Headers; + +add_default_headers([Header = {Default, _} | Defaults], Headers) -> + case lists:keysearch(Default, 1, Headers) of + {value, _} -> + add_default_headers(Defaults, Headers); + _ -> + add_default_headers(Defaults, [Header | Headers]) + end. + +transform({content_type, Value}) -> + {"content-type", Value}; +transform({accept_ranges, Value}) -> + {"accept-ranges", Value}; +transform({cache_control, Value}) -> + {"cache-control",Value}; +transform({transfer_encoding, Value}) -> + {"transfer-encoding", Value}; +transform({content_encoding, Value}) -> + {"content-encoding", Value}; +transform({content_language, Value}) -> + {"content-language", Value}; +transform({retry_after, Value}) -> + {"retry-after", Value}; +transform({content_location, Value}) -> + {"Content-Location:", Value}; +transform({content_length, Value}) -> + {"content-length", Value}; +transform({content_MD5, Value}) -> + {"content-md5", Value}; +transform({content_range, Value}) -> + {"content-range", Value}; +transform({last_modified, Value}) -> + {"last-modified", Value}; +transform({Field, Value}) when is_atom(Field) -> + {atom_to_list(Field), Value}; +transform({Field, Value}) when is_list(Field) -> + {Field, Value}. + +%%---------------------------------------------------------------------- +%% This is the old way of sending data it is strongly encouraged to +%% Leave this method and go on to the newer form of response +%% OTP-4408 +%%---------------------------------------------------------------------- +send_response_old(#mod{method = "HEAD"} = ModData, + StatusCode, Response) -> + NewResponse = lists:flatten(Response), + + case httpd_util:split(NewResponse, [?CR, ?LF, ?CR, ?LF],2) of + {ok, [Head, Body]} -> + {ok, NewHead} = handle_headers(string:tokens(Head, [?CR,?LF]), []), + send_header(ModData, StatusCode, [{content_length, + content_length(Body)} | NewHead]); + {ok, [NewResponse]} -> + send_header(ModData, StatusCode, [{content_length, + content_length(NewResponse)}]); + _Error -> + send_status(ModData, 500, "Internal Server Error") + end; + +send_response_old(#mod{socket_type = Type, + socket = Sock} = ModData, + StatusCode, Response) -> + + NewResponse = lists:flatten(Response), + + case httpd_util:split(NewResponse, [?CR, ?LF, ?CR, ?LF], 2) of + {ok, [Head, Body]} -> + {ok, NewHead} = handle_headers(string:tokens(Head, + [?CR,?LF]), []), + send_header(ModData, StatusCode, [{content_length, + content_length(Body)} | + NewHead]), + httpd_socket:deliver(Type, Sock, Body); + {ok, [NewResponse]} -> + send_header(ModData, StatusCode, [{content_length, + content_length(NewResponse)}]), + httpd_socket:deliver(Type, Sock, NewResponse); + + {error, _Reason} -> + send_status(ModData, 500, "Internal Server Error") + end. + +content_length(Body)-> + integer_to_list(httpd_util:flatlength(Body)). + +report_error(Mod, ConfigDB, Error) -> + Modules = httpd_util:lookup(ConfigDB, modules, + [mod_get, mod_head, mod_log]), + case lists:member(Mod, Modules) of + true -> + Mod:report_error(ConfigDB, Error); + _ -> + ok + end. + +handle_headers([], NewHeaders) -> + {ok, NewHeaders}; + +handle_headers([Header | Headers], NewHeaders) -> + {FieldName, FieldValue} = split_header(Header, []), + handle_headers(Headers, + [{FieldName, FieldValue}| NewHeaders]). + diff --git a/lib/inets/src/http_server/httpd_script_env.erl b/lib/inets/src/http_server/httpd_script_env.erl new file mode 100644 index 0000000000..a742cbef76 --- /dev/null +++ b/lib/inets/src/http_server/httpd_script_env.erl @@ -0,0 +1,144 @@ +%% +%% %CopyrightBegin% +%% +%% Copyright Ericsson AB 2005-2009. All Rights Reserved. +%% +%% The contents of this file are subject to the Erlang Public License, +%% Version 1.1, (the "License"); you may not use this file except in +%% compliance with the License. You should have received a copy of the +%% Erlang Public License along with this software. If not, it can be +%% retrieved online at http://www.erlang.org/. +%% +%% Software distributed under the License is distributed on an "AS IS" +%% basis, WITHOUT WARRANTY OF ANY KIND, either express or implied. See +%% the License for the specific language governing rights and limitations +%% under the License. +%% +%% %CopyrightEnd% +%% +%% + +-module(httpd_script_env). + +-export([create_env/3]). + +-include("httpd.hrl"). + +%%%========================================================================= +%%% Internal application API +%%%========================================================================= +%%-------------------------------------------------------------------------- +%% create_env(ScriptType, ModData, ScriptElements) -> [{EnvVariable, Value}] +%% +%% ScriptType = cgi | esi +%% ModData = #mod{} +%% ScriptElements = [{Element, Value}] +%% Element = path_info | query_string | entity_body +%% Value = term() +%% EnvVariable = string() - cgi | atom() - esi +%% +%% Description: Creates a list of cgi/esi environment variables and +%% there values. +%%-------------------------------------------------------------------------- +create_env(ScriptType, ModData, ScriptElements) -> + create_basic_elements(ScriptType, ModData) + ++ create_http_header_elements(ScriptType, ModData#mod.parsed_header) + ++ create_script_elements(ScriptType, ModData, ScriptElements) + ++ create_mod_interaction_elements(ScriptType, ModData). + +%%%======================================================================== +%%% Internal functions +%%%======================================================================== +create_basic_elements(esi, ModData) -> + {_, RemoteAddr} = (ModData#mod.init_data)#init_data.peername, + [{server_software, ?SERVER_SOFTWARE}, + {server_name, (ModData#mod.init_data)#init_data.resolve}, + {gateway_interface,?GATEWAY_INTERFACE}, + {server_protocol, ?SERVER_PROTOCOL}, + {server_port, httpd_util:lookup(ModData#mod.config_db,port,80)}, + {request_method, ModData#mod.method}, + {remote_addr, RemoteAddr}, + {script_name, ModData#mod.request_uri}]; + +create_basic_elements(cgi, ModData) -> + {_, RemoteAddr} = (ModData#mod.init_data)#init_data.peername, + [{"SERVER_SOFTWARE",?SERVER_SOFTWARE}, + {"SERVER_NAME", (ModData#mod.init_data)#init_data.resolve}, + {"GATEWAY_INTERFACE",?GATEWAY_INTERFACE}, + {"SERVER_PROTOCOL",?SERVER_PROTOCOL}, + {"SERVER_PORT", + integer_to_list(httpd_util:lookup( + ModData#mod.config_db, port, 80))}, + {"REQUEST_METHOD", ModData#mod.method}, + {"REMOTE_ADDR", RemoteAddr}, + {"SCRIPT_NAME", ModData#mod.request_uri}]. + +create_http_header_elements(ScriptType, Headers) -> + create_http_header_elements(ScriptType, Headers, []). + +create_http_header_elements(_, [], Acc) -> + Acc; +create_http_header_elements(ScriptType, [{Name, [Value | _] = Values } | + Headers], Acc) + when is_list(Value) -> + NewName = lists:map(fun(X) -> if X == $- -> $_; true -> X end end, Name), + Element = http_env_element(ScriptType, NewName, multi_value(Values)), + create_http_header_elements(ScriptType, Headers, [Element | Acc]); + +create_http_header_elements(ScriptType, [{Name, Value} | Headers], Acc) + when is_list(Value) -> + {ok, NewName, _} = inets_regexp:gsub(Name,"-","_"), + Element = http_env_element(ScriptType, NewName, Value), + create_http_header_elements(ScriptType, Headers, [Element | Acc]). + +http_env_element(cgi, VarName, Value) -> + {"HTTP_"++ http_util:to_upper(VarName), Value}; +http_env_element(esi, VarName, Value) -> + {list_to_atom("http_"++ http_util:to_lower(VarName)), Value}. + +multi_value([]) -> + []; +multi_value([Value]) -> + Value; +multi_value([Value | Rest]) -> + Value ++ ", " ++ multi_value(Rest). + +create_script_elements(ScriptType, ModData, ScriptElements) -> + lists:flatmap(fun({Element, Data}) -> + create_script_elements(ScriptType, + Element, + Data, ModData) + end, ScriptElements). + +create_script_elements(esi, query_string, QueryString, _) -> + [{query_string, QueryString}]; +create_script_elements(cgi, query_string, QueryString, _) -> + [{"QUERY_STRING", QueryString}]; +create_script_elements(esi, path_info, PathInfo, ModData) -> + Aliases = httpd_util:multi_lookup(ModData#mod.config_db, alias), + {_,PathTranslated,_} = + mod_alias:real_name(ModData#mod.config_db, PathInfo, + Aliases), + [{path_info, PathInfo}, + {path_translated, PathTranslated}]; +create_script_elements(cgi, path_info, PathInfo, ModData) -> + Aliases = httpd_util:multi_lookup(ModData#mod.config_db, alias), + {_,PathTranslated,_} = + mod_alias:real_name(ModData#mod.config_db, PathInfo, + Aliases), + [{"PATH_INFO", PathInfo}, + {"PATH_TRANSLATED", PathTranslated}]; +create_script_elements(esi, entity_body, Body, _) -> + [{content_length, httpd_util:flatlength(Body)}]; +create_script_elements(cgi, entity_body, Body, _) -> + [{"CONTENT_LENGTH", httpd_util:flatlength(Body)}]; +create_script_elements(_, _, _, _) -> + []. + +create_mod_interaction_elements(_, ModData)-> + case proplists:get_value(remote_user, ModData#mod.data) of + undefined -> + []; + RemoteUser -> + [{remote_user, RemoteUser}] + end. diff --git a/lib/inets/src/http_server/httpd_socket.erl b/lib/inets/src/http_server/httpd_socket.erl new file mode 100644 index 0000000000..e9d3e06b38 --- /dev/null +++ b/lib/inets/src/http_server/httpd_socket.erl @@ -0,0 +1,64 @@ +%% +%% %CopyrightBegin% +%% +%% Copyright Ericsson AB 1997-2009. All Rights Reserved. +%% +%% The contents of this file are subject to the Erlang Public License, +%% Version 1.1, (the "License"); you may not use this file except in +%% compliance with the License. You should have received a copy of the +%% Erlang Public License along with this software. If not, it can be +%% retrieved online at http://www.erlang.org/. +%% +%% Software distributed under the License is distributed on an "AS IS" +%% basis, WITHOUT WARRANTY OF ANY KIND, either express or implied. See +%% the License for the specific language governing rights and limitations +%% under the License. +%% +%% %CopyrightEnd% +%% +%% +-module(httpd_socket). + +%% API (document close ?) +-export([deliver/3, peername/2, resolve/0, close/2]). + +-include("httpd.hrl"). + +-define(VMODULE,"SOCKET"). +-include_lib("kernel/include/inet.hrl"). + +deliver(SocketType, Socket, IOListOrBinary) -> + case http_transport:send(SocketType, Socket, IOListOrBinary) of + {error, _Reason} -> + (catch close(SocketType, Socket)), + socket_closed; + _ -> + ok + end. + +peername(SocketType, Socket) -> + http_transport:peername(SocketType, Socket). + +resolve() -> + http_transport:resolve(). + +close(SocketType, Socket) -> + close_sleep(SocketType, 1000), + Res = + case (catch http_transport:close(SocketType, Socket)) of + ok -> ok; + {error,Reason} -> {error,Reason}; + {'EXIT',{noproc,_}} -> {error,closed}; + {'EXIT',Reason} -> {error,Reason}; + Otherwise -> {error,Otherwise} + end, + Res. + +%% Workaround for ssl problem when ssl does not deliver the message +%% sent prior to the close before the close signal. +close_sleep({ssl, _}, Time) -> + sleep(Time); +close_sleep(_, _) -> + ok. + +sleep(T) -> receive after T -> ok end. diff --git a/lib/inets/src/http_server/httpd_sup.erl b/lib/inets/src/http_server/httpd_sup.erl new file mode 100644 index 0000000000..fc41994727 --- /dev/null +++ b/lib/inets/src/http_server/httpd_sup.erl @@ -0,0 +1,264 @@ +%% +%% %CopyrightBegin% +%% +%% Copyright Ericsson AB 2004-2009. All Rights Reserved. +%% +%% The contents of this file are subject to the Erlang Public License, +%% Version 1.1, (the "License"); you may not use this file except in +%% compliance with the License. You should have received a copy of the +%% Erlang Public License along with this software. If not, it can be +%% retrieved online at http://www.erlang.org/. +%% +%% Software distributed under the License is distributed on an "AS IS" +%% basis, WITHOUT WARRANTY OF ANY KIND, either express or implied. See +%% the License for the specific language governing rights and limitations +%% under the License. +%% +%% %CopyrightEnd% +%% +%% +%%---------------------------------------------------------------------- +%% Purpose: The top supervisor for the http server (httpd) hangs under +%% inets_sup. +%%---------------------------------------------------------------------- + +-module(httpd_sup). + +-behaviour(supervisor). + +%% Internal application API +-export([start_link/1, start_link/2]). +-export([start_child/1, restart_child/2, stop_child/2]). + +%% Supervisor callback +-export([init/1]). + +-export([listen_init/4]). + +-define(TIMEOUT, 15000). +-include("httpd_internal.hrl"). + + +%%%========================================================================= +%%% API +%%%========================================================================= +start_link(HttpdServices) -> + supervisor:start_link({local, ?MODULE}, ?MODULE, [HttpdServices]). + +start_link(HttpdServices, stand_alone) -> + supervisor:start_link(?MODULE, [HttpdServices]). + +start_child(Config) -> + try httpd_config(Config) of + {ok, NewConfig} -> + Spec = httpd_child_spec(NewConfig, ?TIMEOUT, []), + case supervisor:start_child(?MODULE, Spec) of + {error, {invalid_child_spec, Error}} -> + Error; + Other -> + Other + end + catch + throw:Error -> + Error + end. + + +restart_child(Address, Port) -> + Name = id(Address, Port), + case supervisor:terminate_child(?MODULE, Name) of + ok -> + supervisor:restart_child(?MODULE, Name); + Error -> + Error + end. + +stop_child(Address, Port) -> + Name = id(Address, Port), + case supervisor:terminate_child(?MODULE, Name) of + ok -> + supervisor:delete_child(?MODULE, Name); + Error -> + Error + end. + +id(Address, Port) -> + {httpd_instance_sup, Address, Port}. + + +%%%========================================================================= +%%% Supervisor callback +%%%========================================================================= +init([HttpdServices]) -> + ?hdrd("starting", []), + RestartStrategy = one_for_one, + MaxR = 10, + MaxT = 3600, + Children = child_specs(HttpdServices, []), + {ok, {{RestartStrategy, MaxR, MaxT}, Children}}. + + +%%%========================================================================= +%%% Internal functions +%%%========================================================================= + +%% The format of the httpd service is: +%% httpd_service() -> {httpd,httpd()} +%% httpd() -> [httpd_config()] | file() +%% httpd_config() -> {file,file()} | +%% {debug,debug()} | +%% {accept_timeout,integer()} +%% debug() -> disable | [debug_options()] +%% debug_options() -> {all_functions,modules()} | +%% {exported_functions,modules()} | +%% {disable,modules()} +%% modules() -> [atom()] + + +child_specs([], Acc) -> + Acc; +child_specs([{httpd, HttpdService} | Rest], Acc) -> + ?hdrd("child specs", [{httpd, HttpdService}]), + NewHttpdService = (catch mk_tuple_list(HttpdService)), + ?hdrd("child specs", [{new_httpd, NewHttpdService}]), + case catch child_spec(NewHttpdService) of + {error, Reason} -> + ?hdri("failed generating child spec", [{reason, Reason}]), + error_msg("Failed to start service: ~n~p ~n due to: ~p~n", + [HttpdService, Reason]), + child_specs(Rest, Acc); + Spec -> + ?hdrt("child spec", [{child_spec, Spec}]), + child_specs(Rest, [Spec | Acc]) + end. + +child_spec(HttpdService) -> + {ok, Config} = httpd_config(HttpdService), + ?hdrt("child spec", [{config, Config}]), + Debug = proplists:get_value(debug, Config, []), + AcceptTimeout = proplists:get_value(accept_timeout, Config, 15000), + httpd_util:valid_options(Debug, AcceptTimeout, Config), + httpd_child_spec(Config, AcceptTimeout, Debug). + +httpd_config([Value| _] = Config) when is_tuple(Value) -> + case proplists:get_value(file, Config) of + undefined -> + case proplists:get_value(proplist_file, Config) of + undefined -> + httpd_conf:validate_properties(Config); + File -> + try file:consult(File) of + {ok, [PropList]} -> + httpd_conf:validate_properties(PropList) + catch + exit:_ -> + throw({error, + {could_not_consult_proplist_file, File}}) + end + end; + File -> + {ok, File} + end. + +httpd_child_spec([Value| _] = Config, AcceptTimeout, Debug) + when is_tuple(Value) -> + Address = proplists:get_value(bind_address, Config, any), + Port = proplists:get_value(port, Config, 80), + httpd_child_spec(Config, AcceptTimeout, Debug, Address, Port); + +httpd_child_spec(ConfigFile, AcceptTimeout, Debug) -> + case httpd_conf:load(ConfigFile) of + {ok, ConfigList} -> + case httpd_conf:validate_properties(ConfigList) of + {ok, Config} -> + Address = proplists:get_value(bind_address, Config, any), + Port = proplists:get_value(port, Config, 80), + httpd_child_spec([{file, ConfigFile} | Config], + AcceptTimeout, Debug, Address, Port); + Error -> + Error + end; + Error -> + Error + end. + +httpd_child_spec(Config, AcceptTimeout, Debug, Addr, 0) -> + case start_listen(Addr, 0, Config) of + {Pid, {NewPort, NewConfig, ListenSocket}} -> + Name = {httpd_instance_sup, Addr, NewPort}, + StartFunc = {httpd_instance_sup, start_link, + [NewConfig, AcceptTimeout, + {Pid, ListenSocket}, Debug]}, + Restart = permanent, + Shutdown = infinity, + Modules = [httpd_instance_sup], + Type = supervisor, + {Name, StartFunc, Restart, Shutdown, Type, Modules}; + {Pid, {error, Reason}} -> + exit(Pid, normal), + {error, Reason} + end; + +httpd_child_spec(Config, AcceptTimeout, Debug, Addr, Port) -> + Name = {httpd_instance_sup, Addr, Port}, + StartFunc = {httpd_instance_sup, start_link, + [Config, AcceptTimeout, Debug]}, + Restart = permanent, + Shutdown = infinity, + Modules = [httpd_instance_sup], + Type = supervisor, + {Name, StartFunc, Restart, Shutdown, Type, Modules}. + + +mk_tuple_list([]) -> + []; +mk_tuple_list([H={_,_}|T]) -> + [H|mk_tuple_list(T)]; +mk_tuple_list(F) -> + [{file, F}]. + +error_msg(F, A) -> + error_logger:error_msg(F ++ "~n", A). + +listen(Address, Port, Config) -> + SocketType = proplists:get_value(socket_type, Config, ip_comm), + case http_transport:start(SocketType) of + ok -> + case http_transport:listen(SocketType, Address, Port) of + {ok, ListenSocket} -> + NewConfig = proplists:delete(port, Config), + {ok, NewPort} = inet:port(ListenSocket), + {NewPort, [{port, NewPort} | NewConfig], ListenSocket}; + {error, Reason} -> + {error, {listen, Reason}} + end; + {error, Reason} -> + {error, {socket_start_failed, Reason}} + end. + +start_listen(Address, Port, Config) -> + Pid = listen_owner(Address, Port, Config), + receive + {Pid, Result} -> + {Pid, Result} + end. + +listen_owner(Address, Port, Config) -> + spawn(?MODULE, listen_init, [self(), Address, Port, Config]). + +listen_init(From, Address, Port, Config) -> + process_flag(trap_exit, true), + Result = listen(Address, Port, Config), + From ! {self(), Result}, + listen_loop(). + +listen_loop() -> + receive + {'EXIT', _, _} -> + ok + end. + + + + + diff --git a/lib/inets/src/http_server/httpd_util.erl b/lib/inets/src/http_server/httpd_util.erl new file mode 100644 index 0000000000..b59fd861dc --- /dev/null +++ b/lib/inets/src/http_server/httpd_util.erl @@ -0,0 +1,780 @@ +%% +%% %CopyrightBegin% +%% +%% Copyright Ericsson AB 1997-2009. All Rights Reserved. +%% +%% The contents of this file are subject to the Erlang Public License, +%% Version 1.1, (the "License"); you may not use this file except in +%% compliance with the License. You should have received a copy of the +%% Erlang Public License along with this software. If not, it can be +%% retrieved online at http://www.erlang.org/. +%% +%% Software distributed under the License is distributed on an "AS IS" +%% basis, WITHOUT WARRANTY OF ANY KIND, either express or implied. See +%% the License for the specific language governing rights and limitations +%% under the License. +%% +%% %CopyrightEnd% +%% +%% +-module(httpd_util). +-export([ip_address/2, lookup/2, lookup/3, multi_lookup/2, + lookup_mime/2, lookup_mime/3, lookup_mime_default/2, + lookup_mime_default/3, reason_phrase/1, message/3, rfc1123_date/0, + rfc1123_date/1, day/1, month/1, decode_hex/1, + flatlength/1, split_path/1, split_script_path/1, + suffix/1, split/3, uniq/1, + make_name/2,make_name/3,make_name/4,strip/1, + hexlist_to_integer/1,integer_to_hexlist/1, + convert_request_date/1,create_etag/1,create_etag/2, + convert_netscapecookie_date/1, enable_debug/1, valid_options/3, + modules_validate/1, module_validate/1, + dir_validate/2, file_validate/2, mime_type_validate/1, + mime_types_validate/1, custom_date/0]). + +-export([encode_hex/1]). +-include_lib("kernel/include/file.hrl"). + +ip_address({_,_,_,_} = Address, _IpFamily) -> + {ok, Address}; +ip_address({_,_,_,_,_,_,_,_} = Address, _IpFamily) -> + {ok, Address}; +ip_address(Host, IpFamily) + when ((IpFamily =:= inet) orelse (IpFamily =:= inet6)) -> + inet:getaddr(Host, IpFamily); +ip_address(Host, inet6fb4 = _IpFamily) -> + Inet = case gen_tcp:listen(0, [inet6]) of + {ok, Dummyport} -> + gen_tcp:close(Dummyport), + inet6; + _ -> + inet + end, + inet:getaddr(Host, Inet). + + +%% lookup + +lookup(Table,Key) -> + lookup(Table,Key,undefined). + +lookup(Table,Key,Undefined) -> + case catch ets:lookup(Table,Key) of + [{Key,Value}|_] -> + Value; + _-> + Undefined + end. + +%% multi_lookup + +multi_lookup(Table,Key) -> + remove_key(ets:lookup(Table,Key)). + +remove_key([]) -> + []; +remove_key([{_Key, Value}| Rest]) -> + [Value | remove_key(Rest)]. + +%% lookup_mime + +lookup_mime(ConfigDB,Suffix) -> + lookup_mime(ConfigDB,Suffix,undefined). + +lookup_mime(ConfigDB,Suffix,Undefined) -> + [{mime_types,MimeTypesDB}|_]=ets:lookup(ConfigDB,mime_types), + case ets:lookup(MimeTypesDB,Suffix) of + [] -> + Undefined; + [{Suffix,MimeType}|_] -> + MimeType + end. + +%% lookup_mime_default + +lookup_mime_default(ConfigDB,Suffix) -> + lookup_mime_default(ConfigDB,Suffix,undefined). + +lookup_mime_default(ConfigDB,Suffix,Undefined) -> + [{mime_types,MimeTypesDB}|_]=ets:lookup(ConfigDB,mime_types), + case ets:lookup(MimeTypesDB,Suffix) of + [] -> + case ets:lookup(ConfigDB,default_type) of + [] -> + Undefined; + [{default_type,DefaultType}|_] -> + DefaultType + end; + [{Suffix,MimeType}|_] -> + MimeType + end. + +%%% RFC 2616, HTTP 1.1 Status codes +reason_phrase(100) -> "Continue"; +reason_phrase(101) -> "Switching Protocols" ; +reason_phrase(200) -> "OK" ; +reason_phrase(201) -> "Created" ; +reason_phrase(202) -> "Accepted" ; +reason_phrase(203) -> "Non-Authoritative Information" ; +reason_phrase(204) -> "No Content" ; +reason_phrase(205) -> "Reset Content" ; +reason_phrase(206) -> "Partial Content" ; +reason_phrase(300) -> "Multiple Choices" ; +reason_phrase(301) -> "Moved Permanently" ; +reason_phrase(302) -> "Moved Temporarily" ; +reason_phrase(303) -> "See Other" ; +reason_phrase(304) -> "Not Modified" ; +reason_phrase(305) -> "Use Proxy" ; +reason_phrase(306) -> "(unused)" ; +reason_phrase(307) -> "Temporary Redirect" ; +reason_phrase(400) -> "Bad Request"; +reason_phrase(401) -> "Unauthorized"; +reason_phrase(402) -> "Payment Required"; +reason_phrase(403) -> "Forbidden" ; +reason_phrase(404) -> "Object Not Found" ; +reason_phrase(405) -> "Method Not Allowed" ; +reason_phrase(406) -> "Not Acceptable" ; +reason_phrase(407) -> "Proxy Authentication Required" ; +reason_phrase(408) -> "Request Time-out" ; +reason_phrase(409) -> "Conflict" ; +reason_phrase(410) -> "Gone" ; +reason_phrase(411) -> "Length Required" ; +reason_phrase(412) -> "Precondition Failed" ; +reason_phrase(413) -> "Request Entity Too Large" ; +reason_phrase(414) -> "Request-URI Too Large" ; +reason_phrase(415) -> "Unsupported Media Type" ; +reason_phrase(416) -> "Requested Range Not Satisfiable" ; +reason_phrase(417) -> "Expectation Failed" ; +reason_phrase(500) -> "Internal Server Error" ; +reason_phrase(501) -> "Not Implemented" ; +reason_phrase(502) -> "Bad Gateway" ; +reason_phrase(503) -> "Service Unavailable" ; +reason_phrase(504) -> "Gateway Time-out" ; +reason_phrase(505) -> "HTTP Version not supported"; + +%%% RFC 2518, HTTP Extensions for Distributed Authoring -- WEBDAV +reason_phrase(102) -> "Processing"; +reason_phrase(207) -> "Multi-Status"; +reason_phrase(422) -> "Unprocessable Entity"; +reason_phrase(423) -> "Locked"; +reason_phrase(424) -> "Failed Dependency"; +reason_phrase(507) -> "Insufficient Storage"; + +%%% (Work in Progress) WebDAV Advanced Collections +reason_phrase(425) -> ""; + +%%% RFC 2817, HTTP Upgrade to TLS +reason_phrase(426) -> "Upgrade Required"; + +%%% RFC 3229, Delta encoding in HTTP +reason_phrase(226) -> "IM Used"; + +reason_phrase(_) -> "Internal Server Error". + + +%% message + +message(301,URL,_) -> + "The document has moved <A HREF=\""++URL++"\">here</A>."; +message(304, _URL,_) -> + "The document has not been changed."; +message(400,none,_) -> + "Your browser sent a query that this server could not understand."; +message(400,Msg,_) -> + "Your browser sent a query that this server could not understand. "++Msg; +message(401,none,_) -> + "This server could not verify that you +are authorized to access the document you + requested. Either you supplied the wrong +credentials (e.g., bad password), or your +browser doesn't understand how to supply +the credentials required."; +message(403,RequestURI,_) -> + "You don't have permission to access "++RequestURI++" on this server."; +message(404,RequestURI,_) -> + "The requested URL "++RequestURI++" was not found on this server."; +message(408, Timeout, _) -> + Timeout; +message(412,none,_) -> + "The requested preconditions where false"; +message(413, Reason,_) -> + "Entity: " ++ Reason; +message(414,ReasonPhrase,_) -> + "Message "++ReasonPhrase++"."; +message(416,ReasonPhrase,_) -> + ReasonPhrase; + +message(500,_,ConfigDB) -> + ServerAdmin=lookup(ConfigDB,server_admin,"unknown@unknown"), + "The server encountered an internal error or " + "misconfiguration and was unable to complete " + "your request.<P>Please contact the server administrator " + ++ ServerAdmin ++ ", and inform them of the time the error occurred " + "and anything you might have done that may have caused the error."; + +message(501,{Method, RequestURI, HTTPVersion}, _ConfigDB) -> + if + is_atom(Method) -> + atom_to_list(Method)++ + " to "++RequestURI++" ("++HTTPVersion++") not supported."; + is_list(Method) -> + Method++ + " to "++RequestURI++" ("++HTTPVersion++") not supported." + end; + +message(503, String, _ConfigDB) -> + "This service in unavailable due to: "++String. + +%%convert_rfc_date(Date)->{{YYYY,MM,DD},{HH,MIN,SEC}} + +convert_request_date([D,A,Y,DateType| Rest])-> + Func=case DateType of + $\, -> + fun convert_rfc1123_date/1; + $\ -> + fun convert_ascii_date/1; + _ -> + fun convert_rfc850_date/1 + end, + case catch Func([D,A,Y,DateType| Rest]) of + {ok,Date} -> + Date; + _Error-> + bad_date + end. +convert_rfc850_date(DateStr) -> + [_WeekDay,Date,Time,_TimeZone|_Rest] = string:tokens(DateStr," "), + convert_rfc850_date(Date,Time). + +convert_rfc850_date([D1,D2,_, + M,O,N,_, + Y1,Y2|_Rest],[H1,H2,_Col,M1,M2,_Col,S1,S2|_Rest2])-> + Year=list_to_integer([50,48,Y1,Y2]), + Day=list_to_integer([D1,D2]), + Month = http_util:convert_month([M,O,N]), + Hour=list_to_integer([H1,H2]), + Min=list_to_integer([M1,M2]), + Sec=list_to_integer([S1,S2]), + {ok,{{Year,Month,Day},{Hour,Min,Sec}}}. + +convert_ascii_date([_D,_A,_Y,_SP, + M,O,N,_SP, + D1,D2,_SP, + H1,H2,_Col, + M1,M2,_Col, + S1,S2,_SP, + Y1,Y2,Y3,Y4| _Rest])-> + Year=list_to_integer([Y1,Y2,Y3,Y4]), + Day=case D1 of + $\ -> + list_to_integer([D2]); + _-> + list_to_integer([D1,D2]) + end, + Month=http_util:convert_month([M,O,N]), + Hour=list_to_integer([H1,H2]), + Min=list_to_integer([M1,M2]), + Sec=list_to_integer([S1,S2]), + {ok,{{Year,Month,Day},{Hour,Min,Sec}}}. + +convert_rfc1123_date([_D,_A,_Y,_C,_SP, + D1,D2,_SP, + M,O,N,_SP, + Y1,Y2,Y3,Y4,_SP, + H1,H2,_Col, + M1,M2,_Col, + S1,S2|_Rest]) -> + Year=list_to_integer([Y1,Y2,Y3,Y4]), + Day=list_to_integer([D1,D2]), + Month=http_util:convert_month([M,O,N]), + Hour=list_to_integer([H1,H2]), + Min=list_to_integer([M1,M2]), + Sec=list_to_integer([S1,S2]), + {ok,{{Year,Month,Day},{Hour,Min,Sec}}}. + +convert_netscapecookie_date(Date)-> + case (catch http_util:convert_netscapecookie_date(Date)) of + Ret = {ok, _} -> + Ret; + _ -> + {error,bad_date} + end. + + +%% rfc1123_date + +rfc1123_date() -> + {{YYYY,MM,DD},{Hour,Min,Sec}} = calendar:universal_time(), + DayNumber = calendar:day_of_the_week({YYYY,MM,DD}), + lists:flatten( + io_lib:format("~s, ~2.2.0w ~3.s ~4.4.0w ~2.2.0w:~2.2.0w:~2.2.0w GMT", + [day(DayNumber),DD,month(MM),YYYY,Hour,Min,Sec])). + +rfc1123_date(undefined) -> + undefined; +rfc1123_date(LocalTime) -> + {{YYYY,MM,DD},{Hour,Min,Sec}} = + case calendar:local_time_to_universal_time_dst(LocalTime) of + [Gmt] -> Gmt; + [_,Gmt] -> Gmt + end, + DayNumber = calendar:day_of_the_week({YYYY,MM,DD}), + lists:flatten( + io_lib:format("~s, ~2.2.0w ~3.s ~4.4.0w ~2.2.0w:~2.2.0w:~2.2.0w GMT", + [day(DayNumber),DD,month(MM),YYYY,Hour,Min,Sec])). + +custom_date() -> + LocalTime = calendar:local_time(), + UniversalTime = calendar:universal_time(), + Minutes = round(diff_in_minutes(LocalTime,UniversalTime)), + {{YYYY,MM,DD},{Hour,Min,Sec}} = LocalTime, + Date = + io_lib:format("~.2.0w/~.3s/~.4w:~.2.0w:~.2.0w:~.2.0w ~c~.2.0w~.2.0w", + [DD,httpd_util:month(MM),YYYY,Hour,Min,Sec, + sign(Minutes), abs(Minutes) div 60, + abs(Minutes) rem 60]), + lists:flatten(Date). + +diff_in_minutes(L,U) -> + (calendar:datetime_to_gregorian_seconds(L) - + calendar:datetime_to_gregorian_seconds(U))/60. + +sign(Minutes) when Minutes > 0 -> + $+; +sign(_Minutes) -> + $-. + +%% uniq + +uniq([]) -> + []; +uniq([First,First|Rest]) -> + uniq([First|Rest]); +uniq([First|Rest]) -> + [First|uniq(Rest)]. + + +%% day + +day(1) -> "Mon"; +day(2) -> "Tue"; +day(3) -> "Wed"; +day(4) -> "Thu"; +day(5) -> "Fri"; +day(6) -> "Sat"; +day(7) -> "Sun". + +%% month + +month(1) -> "Jan"; +month(2) -> "Feb"; +month(3) -> "Mar"; +month(4) -> "Apr"; +month(5) -> "May"; +month(6) -> "Jun"; +month(7) -> "Jul"; +month(8) -> "Aug"; +month(9) -> "Sep"; +month(10) -> "Oct"; +month(11) -> "Nov"; +month(12) -> "Dec". + +%% decode_hex + +decode_hex([$%,Hex1,Hex2|Rest]) -> + [hex2dec(Hex1)*16+hex2dec(Hex2)|decode_hex(Rest)]; +decode_hex([First|Rest]) -> + [First|decode_hex(Rest)]; +decode_hex([]) -> + []. + +hex2dec(X) when (X>=$0) andalso (X=<$9) -> X-$0; +hex2dec(X) when (X>=$A) andalso (X=<$F) -> X-$A+10; +hex2dec(X) when (X>=$a) andalso (X=<$f) -> X-$a+10. + +%% flatlength +flatlength(List) -> + flatlength(List, 0). + +flatlength([H|T],L) when is_list(H) -> + flatlength(H,flatlength(T,L)); +flatlength([H|T],L) when is_binary(H) -> + flatlength(T,L+size(H)); +flatlength([_H|T],L) -> + flatlength(T,L+1); +flatlength([],L) -> + L. + +%% split_path + +split_path(Path) -> + case inets_regexp:match(Path,"[\?].*\$") of + %% A QUERY_STRING exists! + {match,Start,Length} -> + {httpd_util:decode_hex(string:substr(Path,1,Start-1)), + string:substr(Path,Start,Length)}; + %% A possible PATH_INFO exists! + nomatch -> + split_path(Path,[]) + end. + +split_path([],SoFar) -> + {httpd_util:decode_hex(lists:reverse(SoFar)),[]}; +split_path([$/|Rest],SoFar) -> + Path=httpd_util:decode_hex(lists:reverse(SoFar)), + case file:read_file_info(Path) of + {ok,FileInfo} when FileInfo#file_info.type =:= regular -> + {Path,[$/|Rest]}; + {ok, _FileInfo} -> + split_path(Rest,[$/|SoFar]); + {error, _Reason} -> + split_path(Rest,[$/|SoFar]) + end; +split_path([C|Rest],SoFar) -> + split_path(Rest,[C|SoFar]). + +%% split_script_path + +split_script_path(Path) -> + case split_script_path(Path, []) of + {Script, AfterPath} -> + {PathInfo, QueryString} = pathinfo_querystring(AfterPath), + {Script, {PathInfo, QueryString}}; + not_a_script -> + not_a_script + end. + +pathinfo_querystring(Str) -> + pathinfo_querystring(Str, []). +pathinfo_querystring([], SoFar) -> + {lists:reverse(SoFar), []}; +pathinfo_querystring([$?|Rest], SoFar) -> + {lists:reverse(SoFar), Rest}; +pathinfo_querystring([C|Rest], SoFar) -> + pathinfo_querystring(Rest, [C|SoFar]). + +split_script_path([$?|QueryString], SoFar) -> + Path = httpd_util:decode_hex(lists:reverse(SoFar)), + case file:read_file_info(Path) of + {ok,FileInfo} when FileInfo#file_info.type =:= regular -> + {Path, [$?|QueryString]}; + {ok, _FileInfo} -> + not_a_script; + {error, _Reason} -> + not_a_script + end; +split_script_path([], SoFar) -> + Path = httpd_util:decode_hex(lists:reverse(SoFar)), + case file:read_file_info(Path) of + {ok,FileInfo} when FileInfo#file_info.type =:= regular -> + {Path, []}; + {ok, _FileInfo} -> + not_a_script; + {error, _Reason} -> + not_a_script + end; +split_script_path([$/|Rest], SoFar) -> + Path = httpd_util:decode_hex(lists:reverse(SoFar)), + case file:read_file_info(Path) of + {ok, FileInfo} when FileInfo#file_info.type =:= regular -> + {Path, [$/|Rest]}; + {ok, _FileInfo} -> + split_script_path(Rest, [$/|SoFar]); + {error, _Reason} -> + split_script_path(Rest, [$/|SoFar]) + end; +split_script_path([C|Rest], SoFar) -> + split_script_path(Rest,[C|SoFar]). + +%% suffix + +suffix(Path) -> + case filename:extension(Path) of + [] -> + []; + Extension -> + tl(Extension) + end. + + +%% strip +strip(Value)-> + lists:reverse(remove_ws(lists:reverse(remove_ws(Value)))). + +remove_ws([$\s|Rest])-> + remove_ws(Rest); +remove_ws([$\t|Rest]) -> + remove_ws(Rest); +remove_ws(Rest) -> + Rest. + +%% split + +split(String,RegExp,Limit) -> + case inets_regexp:parse(RegExp) of + {error,Reason} -> + {error,Reason}; + {ok,_} -> + {ok,do_split(String,RegExp,Limit)} + end. + +do_split(String, _RegExp, 1) -> + [String]; + +do_split(String,RegExp,Limit) -> + case inets_regexp:first_match(String,RegExp) of + {match,Start,Length} -> + [string:substr(String,1,Start-1)| + do_split(lists:nthtail(Start+Length-1,String),RegExp,Limit-1)]; + nomatch -> + [String] + end. + +%% make_name/2, make_name/3 +%% Prefix -> string() +%% First part of the name, e.g. "httpd" +%% Addr -> {A,B,C,D} | string() | undefined +%% The address part of the name. +%% e.g. "123.234.55.66" or {123,234,55,66} or "otp.ericsson.se" +%% for a host address or undefined if local host. +%% Port -> integer() +%% Last part of the name, such as the HTTPD server port +%% number (80). +%% Postfix -> Any string that will be added last to the name +%% +%% Example: +%% make_name("httpd","otp.ericsson.se",80) => httpd__otp_ericsson_se__80 +%% make_name("httpd",undefined,8088) => httpd_8088 + +make_name(Prefix,Port) -> + make_name(Prefix,undefined,Port,""). + +make_name(Prefix,Addr,Port) -> + make_name(Prefix,Addr,Port,""). + +make_name(Prefix,"*",Port,Postfix) -> + make_name(Prefix,undefined,Port,Postfix); + +make_name(Prefix,any,Port,Postfix) -> + make_name1(io_lib:format("~s_~w~s",[Prefix,Port,Postfix])); + +make_name(Prefix,undefined,Port,Postfix) -> + make_name1(io_lib:format("~s_~w~s",[Prefix,Port,Postfix])); + +make_name(Prefix,Addr,Port,Postfix) -> + NameString = + Prefix ++ "__" ++ make_name2(Addr) ++ "__" ++ + integer_to_list(Port) ++ Postfix, + make_name1(NameString). + +make_name1(String) -> + list_to_atom(lists:flatten(String)). + +make_name2({A,B,C,D}) -> + io_lib:format("~w_~w_~w_~w", [A,B,C,D]); + +make_name2({A, B, C, D, E, F, G, H}) -> + io_lib:format("~s_~s_~s_~s_~s_~s_~s_~s", [integer_to_hexlist(A), + integer_to_hexlist(B), + integer_to_hexlist(C), + integer_to_hexlist(D), + integer_to_hexlist(E), + integer_to_hexlist(F), + integer_to_hexlist(G), + integer_to_hexlist(H) + ]); +make_name2(Addr) -> + search_and_replace(Addr,$.,$_). + +search_and_replace(S,A,B) -> + Fun = fun(What) -> + case What of + A -> B; + O -> O + end + end, + lists:map(Fun,S). + + + +%%---------------------------------------------------------------------- +%% Converts a string that constists of 0-9,A-F,a-f to a +%% integer +%%---------------------------------------------------------------------- + +hexlist_to_integer(List)-> + http_util:hexlist_to_integer(List). + +%%---------------------------------------------------------------------- +%%Converts an integer to an hexlist +%%---------------------------------------------------------------------- +encode_hex(Num)-> + integer_to_hexlist(Num). + +integer_to_hexlist(Num) when is_integer(Num) -> + http_util:integer_to_hexlist(Num). + +create_etag(FileInfo) -> + create_etag(FileInfo#file_info.mtime,FileInfo#file_info.size). + +create_etag({{Year,Month,Day},{Hour,Min,Sec}},Size)-> + create_part([Year,Month,Day,Hour,Min,Sec])++io_lib:write(Size); + +create_etag(FileInfo,Size)-> + create_etag(FileInfo#file_info.mtime,Size). + +create_part(Values)-> + lists:map(fun(Val0)-> + Val=Val0 rem 60, + if + Val=<25 -> + 65+Val; % A-Z + Val=<50 -> + 72+Val; % a-z + %%Since no date s + true -> + Val-3 + end + end,Values). + +%%---------------------------------------------------------------------- +%% Validate httpd options +%%---------------------------------------------------------------------- +modules_validate([]) -> + ok; +modules_validate([Head | Tail]) -> + ok = module_validate(Head), + modules_validate(Tail). + +module_validate(Module) when is_atom(Module) -> + case code:which(Module) of + non_existing -> + throw({module_does_not_exist, Module}); + _ -> + ok + end; + +module_validate(Module) -> + throw({module_name_not_atom, Module}). + +dir_validate(ConfDir, Dir) -> + case filelib:is_dir(Dir) of + true -> + ok; + false when is_list(Dir) -> + throw({non_existing, {ConfDir, Dir}}); + false -> + throw({ConfDir, Dir}) + end. + +file_validate(ConfFile, File) -> + case filelib:is_file(File) of + true -> + ok; + false when is_list(File) -> + throw({non_existing, {ConfFile, File}}); + false -> + throw({ConfFile, File}) + end. + +mime_type_validate({Value1, Value2}) + when is_list(Value1) andalso is_list(Value2) -> + ok; +mime_type_validate({_, _} = Value) -> + throw({mime_type, Value}); +mime_type_validate(MimeFile) -> + file_validate(mime_types, MimeFile). + +mime_types_validate([{_, _} = Value | Rest]) -> + ok = mime_types_validate(Value), + mime_types_validate(Rest); +mime_types_validate([]) -> + ok; +mime_types_validate(MimeFile) -> + mime_type_validate(MimeFile). + + +valid_options(Debug, AcceptTimeout, Config) -> + valid_debug(Debug), + valid_accept_timeout(AcceptTimeout), + valid_config(Config). + +valid_debug([]) -> + ok; +valid_debug(disable) -> + ok; +valid_debug(L) when is_list(L) -> + valid_debug2(L); +valid_debug(D) -> + throw({error, {bad_debug_option,D}}). + +valid_debug2([{all_functions,L}|Rest]) when is_list(L) -> + try modules_validate(L) of + ok -> + valid_debug2(Rest) + catch + throw:Error -> + throw({error, Error}) + end; +valid_debug2([{exported_functions,L}|Rest]) when is_list(L) -> + modules_validate(L), + valid_debug2(Rest); +valid_debug2([{disable,L}|Rest]) when is_list(L) -> + modules_validate(L), + valid_debug2(Rest); +valid_debug2([H|_T]) -> + throw({error,{bad_debug_option,H}}); +valid_debug2([]) -> + ok. + +valid_accept_timeout(I) when is_integer(I) -> + ok; +valid_accept_timeout(A) -> + throw({error,{bad_debug_option,A}}). + +valid_config(_) -> + ok. + + +%%---------------------------------------------------------------------- +%% Enable debugging, +%%---------------------------------------------------------------------- + +enable_debug([]) -> + ok; +enable_debug(Debug) -> + dbg:tracer(), + dbg:p(all, [call]), + do_enable_debug(Debug). + +do_enable_debug(disable) -> + dbg:stop(); +do_enable_debug([]) -> + ok; +do_enable_debug([{Level,Modules}|Rest]) + when is_atom(Level) andalso is_list(Modules) -> + case Level of + all_functions -> + io:format("Tracing on all functions set on modules: ~p~n", + [Modules]), + lists:foreach( + fun(X)-> + dbg:tpl(X, [{'_', [], [{return_trace}]}]) + end, Modules); + exported_functions -> + io:format("Tracing on exported functions set on " + "modules: ~p~n",[Modules]), + lists:foreach( + fun(X)-> + dbg:tp(X, [{'_', [], [{return_trace}]}]) + end, Modules); + disable -> + io:format("Tracing disabled on modules: ~p~n", [Modules]), + lists:foreach( + fun(X)-> + dbg:ctp(X) + end, Modules); + _ -> + ok + end, + do_enable_debug(Rest). diff --git a/lib/inets/src/http_server/mod_actions.erl b/lib/inets/src/http_server/mod_actions.erl new file mode 100644 index 0000000000..d50ed4b16c --- /dev/null +++ b/lib/inets/src/http_server/mod_actions.erl @@ -0,0 +1,117 @@ +%% +%% %CopyrightBegin% +%% +%% Copyright Ericsson AB 1997-2009. All Rights Reserved. +%% +%% The contents of this file are subject to the Erlang Public License, +%% Version 1.1, (the "License"); you may not use this file except in +%% compliance with the License. You should have received a copy of the +%% Erlang Public License along with this software. If not, it can be +%% retrieved online at http://www.erlang.org/. +%% +%% Software distributed under the License is distributed on an "AS IS" +%% basis, WITHOUT WARRANTY OF ANY KIND, either express or implied. See +%% the License for the specific language governing rights and limitations +%% under the License. +%% +%% %CopyrightEnd% +%% +%% +-module(mod_actions). +-export([do/1,load/2, store/2]). + +-include("httpd.hrl"). + +%% do + +do(Info) -> + case proplists:get_value(status, Info#mod.data) of + %% A status code has been generated! + {_StatusCode, _PhraseArgs, _Reason} -> + {proceed,Info#mod.data}; + %% No status code has been generated! + undefined -> + case proplists:get_value(response, Info#mod.data) of + %% No response has been generated! + undefined -> + Path = mod_alias:path(Info#mod.data,Info#mod.config_db, + Info#mod.request_uri), + Suffix = httpd_util:suffix(Path), + MimeType = httpd_util:lookup_mime(Info#mod.config_db,Suffix, + "text/plain"), + Actions = httpd_util:multi_lookup(Info#mod.config_db,action), + case action(Info#mod.request_uri,MimeType,Actions) of + {yes, RequestURI} -> + {proceed, [{new_request_uri, RequestURI} | Info#mod.data]}; + no -> + Scripts = httpd_util:multi_lookup(Info#mod.config_db, script), + case script(Info#mod.request_uri, Info#mod.method, Scripts) of + {yes, RequestURI} -> + {proceed,[{new_request_uri, RequestURI} | Info#mod.data]}; + no -> + {proceed, Info#mod.data} + end + end; + %% A response has been generated or sent! + _Response -> + {proceed, Info#mod.data} + end + end. + +action(_RequestURI, _MimeType, []) -> + no; +action(RequestURI, MimeType, [{MimeType, CGIScript} | _Rest]) -> + {yes, CGIScript ++ RequestURI}; +action(RequestURI, MimeType, [_ | Rest]) -> + action(RequestURI, MimeType, Rest). + +script(_RequestURI, _Method, []) -> + no; +script(RequestURI, Method, [{Method, CGIScript} | _Rest]) -> + {yes, CGIScript ++ RequestURI}; +script(RequestURI, Method, [_ | Rest]) -> + script(RequestURI, Method, Rest). + +%% +%% Configuration +%% + +%% load + +load("Action "++ Action, []) -> + case inets_regexp:split(Action, " ") of + {ok,[MimeType, CGIScript]} -> + {ok,[],{action, {MimeType, CGIScript}}}; + {ok,_} -> + {error,?NICE(httpd_conf:clean(Action)++" is an invalid Action")} + end; +load("Script " ++ Script,[]) -> + case inets_regexp:split(Script, " ") of + {ok,[Method, CGIScript]} -> + {ok,[],{script, {Method, CGIScript}}}; + {ok,_} -> + {error,?NICE(httpd_conf:clean(Script)++" is an invalid Script")} + end. + +store({action, {MimeType, CGIScript}} = Conf, _) when is_list(MimeType), + is_list(CGIScript) -> + {ok, Conf}; +store({action, Value}, _) -> + {error, {wrong_type, {action, Value}}}; + +store({script, {Method, CGIScript}} = Conf, _) when is_list(Method), + is_list(CGIScript) -> + case string:to_lower(Method) of + "get" -> + {ok, Conf}; + "post" -> + {ok, Conf}; + _ -> + {error, {wrong_type, Conf}} + end; + +store({script, Value}, _) -> + {error, {wrong_type, {script, Value}}}. + + + diff --git a/lib/inets/src/http_server/mod_alias.erl b/lib/inets/src/http_server/mod_alias.erl new file mode 100644 index 0000000000..7073f5405d --- /dev/null +++ b/lib/inets/src/http_server/mod_alias.erl @@ -0,0 +1,210 @@ +%% +%% %CopyrightBegin% +%% +%% Copyright Ericsson AB 1997-2009. All Rights Reserved. +%% +%% The contents of this file are subject to the Erlang Public License, +%% Version 1.1, (the "License"); you may not use this file except in +%% compliance with the License. You should have received a copy of the +%% Erlang Public License along with this software. If not, it can be +%% retrieved online at http://www.erlang.org/. +%% +%% Software distributed under the License is distributed on an "AS IS" +%% basis, WITHOUT WARRANTY OF ANY KIND, either express or implied. See +%% the License for the specific language governing rights and limitations +%% under the License. +%% +%% %CopyrightEnd% +%% +%% +-module(mod_alias). + +-export([do/1, + real_name/3, + real_script_name/3, + default_index/2, + load/2, + store/2, + path/3]). + +-include("httpd.hrl"). + +-define(VMODULE,"ALIAS"). + +%% do + +do(Info) -> + case proplists:get_value(status, Info#mod.data) of + %% A status code has been generated! + {_StatusCode, _PhraseArgs, _Reason} -> + {proceed,Info#mod.data}; + %% No status code has been generated! + undefined -> + case proplists:get_value(response, Info#mod.data) of + %% No response has been generated! + undefined -> + do_alias(Info); + %% A response has been generated or sent! + _Response -> + {proceed, Info#mod.data} + end + end. + +do_alias(Info) -> + {ShortPath, Path, AfterPath} = + real_name(Info#mod.config_db, + Info#mod.request_uri, + httpd_util:multi_lookup(Info#mod.config_db,alias)), + %% Relocate if a trailing slash is missing else proceed! + LastChar = lists:last(ShortPath), + case file:read_file_info(ShortPath) of + {ok, FileInfo} when FileInfo#file_info.type == directory, + LastChar /= $/ -> + ServerName = httpd_util:lookup(Info#mod.config_db, server_name), + Port = port_string(httpd_util:lookup(Info#mod.config_db,port, 80)), + URL = "http://" ++ ServerName ++ Port ++ + Info#mod.request_uri ++ "/", + ReasonPhrase = httpd_util:reason_phrase(301), + Message = httpd_util:message(301, URL, Info#mod.config_db), + {proceed, + [{response, + {301, ["Location: ", URL, "\r\n" + "Content-Type: text/html\r\n", + "\r\n", + "<HTML>\n<HEAD>\n<TITLE>",ReasonPhrase, + "</TITLE>\n</HEAD>\n" + "<BODY>\n<H1>",ReasonPhrase, + "</H1>\n", Message, + "\n</BODY>\n</HTML>\n"]}}| + [{real_name, {Path, AfterPath}} | Info#mod.data]]}; + _NoFile -> + {proceed,[{real_name, {Path, AfterPath}} | Info#mod.data]} + end. + +port_string(80) -> + ""; +port_string(Port) -> + ":"++integer_to_list(Port). + +%% real_name + +real_name(ConfigDB, RequestURI, []) -> + DocumentRoot = httpd_util:lookup(ConfigDB, document_root, ""), + RealName = DocumentRoot ++ RequestURI, + {ShortPath, _AfterPath} = httpd_util:split_path(RealName), + {Path, AfterPath} = httpd_util:split_path(default_index(ConfigDB, + RealName)), + {ShortPath, Path, AfterPath}; +real_name(ConfigDB, RequestURI, [{FakeName,RealName}|Rest]) -> + case inets_regexp:match(RequestURI, "^" ++ FakeName) of + {match, _, _} -> + {ok, ActualName, _} = inets_regexp:sub(RequestURI, + "^" ++ FakeName, RealName), + {ShortPath, _AfterPath} = httpd_util:split_path(ActualName), + {Path, AfterPath} = + httpd_util:split_path(default_index(ConfigDB, ActualName)), + {ShortPath, Path, AfterPath}; + nomatch -> + real_name(ConfigDB,RequestURI,Rest) + end. + +%% real_script_name + +real_script_name(_ConfigDB, _RequestURI, []) -> + not_a_script; +real_script_name(ConfigDB, RequestURI, [{FakeName,RealName} | Rest]) -> + case inets_regexp:match(RequestURI,"^"++FakeName) of + {match,_,_} -> + {ok,ActualName,_}=inets_regexp:sub(RequestURI,"^"++FakeName,RealName), + httpd_util:split_script_path(default_index(ConfigDB,ActualName)); + nomatch -> + real_script_name(ConfigDB,RequestURI,Rest) + end. + +%% default_index + +default_index(ConfigDB, Path) -> + case file:read_file_info(Path) of + {ok, FileInfo} when FileInfo#file_info.type == directory -> + DirectoryIndex = httpd_util:lookup(ConfigDB, directory_index, []), + append_index(Path, DirectoryIndex); + _ -> + Path + end. + +append_index(RealName, []) -> + RealName; +append_index(RealName, [Index | Rest]) -> + case file:read_file_info(filename:join(RealName, Index)) of + {error, _Reason} -> + append_index(RealName, Rest); + _ -> + filename:join(RealName, Index) + end. + +%% path + +path(Data, ConfigDB, RequestURI) -> + case proplists:get_value(real_name, Data) of + undefined -> + DocumentRoot = httpd_util:lookup(ConfigDB, document_root, ""), + {Path, _AfterPath} = + httpd_util:split_path(DocumentRoot++RequestURI), + Path; + {Path, _AfterPath} -> + Path + end. + +%% +%% Configuration +%% + +%% load + +load("DirectoryIndex " ++ DirectoryIndex, []) -> + {ok, DirectoryIndexes} = inets_regexp:split(DirectoryIndex," "), + {ok,[], {directory_index, DirectoryIndexes}}; +load("Alias " ++ Alias,[]) -> + case inets_regexp:split(Alias," ") of + {ok, [FakeName, RealName]} -> + {ok,[],{alias,{FakeName,RealName}}}; + {ok, _} -> + {error,?NICE(httpd_conf:clean(Alias)++" is an invalid Alias")} + end; +load("ScriptAlias " ++ ScriptAlias, []) -> + case inets_regexp:split(ScriptAlias, " ") of + {ok, [FakeName, RealName]} -> + %% Make sure the path always has a trailing slash.. + RealName1 = filename:join(filename:split(RealName)), + {ok, [], {script_alias, {FakeName, RealName1++"/"}}}; + {ok, _} -> + {error, ?NICE(httpd_conf:clean(ScriptAlias)++ + " is an invalid ScriptAlias")} + end. + +store({directory_index, Value} = Conf, _) when is_list(Value) -> + case is_directory_index_list(Value) of + true -> + {ok, Conf}; + false -> + {error, {wrong_type, {directory_index, Value}}} + end; +store({directory_index, Value}, _) -> + {error, {wrong_type, {directory_index, Value}}}; +store({alias, {Fake, Real}} = Conf, _) when is_list(Fake), + is_list(Real) -> + {ok, Conf}; +store({alias, Value}, _) -> + {error, {wrong_type, {alias, Value}}}; +store({script_alias, {Fake, Real}} = Conf, _) when is_list(Fake), + is_list(Real) -> + {ok, Conf}; +store({script_alias, Value}, _) -> + {error, {wrong_type, {script_alias, Value}}}. + +is_directory_index_list([]) -> + true; +is_directory_index_list([Head | Tail]) when is_list(Head) -> + is_directory_index_list(Tail); +is_directory_index_list(_) -> + false. diff --git a/lib/inets/src/http_server/mod_auth.erl b/lib/inets/src/http_server/mod_auth.erl new file mode 100644 index 0000000000..07cafb4726 --- /dev/null +++ b/lib/inets/src/http_server/mod_auth.erl @@ -0,0 +1,797 @@ +%% +%% %CopyrightBegin% +%% +%% Copyright Ericsson AB 1997-2009. All Rights Reserved. +%% +%% The contents of this file are subject to the Erlang Public License, +%% Version 1.1, (the "License"); you may not use this file except in +%% compliance with the License. You should have received a copy of the +%% Erlang Public License along with this software. If not, it can be +%% retrieved online at http://www.erlang.org/. +%% +%% Software distributed under the License is distributed on an "AS IS" +%% basis, WITHOUT WARRANTY OF ANY KIND, either express or implied. See +%% the License for the specific language governing rights and limitations +%% under the License. +%% +%% %CopyrightEnd% +%% +%% +-module(mod_auth). + +%% The functions that the webbserver call on startup stop +%% and when the server traverse the modules. +-export([do/1, load/2, store/2, remove/1]). + +%% User entries to the gen-server. +-export([add_user/2, add_user/5, add_user/6, + add_group_member/3, add_group_member/4, add_group_member/5, + list_users/1, list_users/2, list_users/3, + delete_user/2, delete_user/3, delete_user/4, + delete_group_member/3, delete_group_member/4, delete_group_member/5, + list_groups/1, list_groups/2, list_groups/3, + delete_group/2, delete_group/3, delete_group/4, + get_user/2, get_user/3, get_user/4, + list_group_members/2, list_group_members/3, list_group_members/4, + update_password/6, update_password/5]). + +-include("httpd.hrl"). +-include("mod_auth.hrl"). +-include("httpd_internal.hrl"). + +-define(VMODULE,"AUTH"). + +-define(NOPASSWORD,"NoPassword"). + +%% do +do(Info) -> + ?hdrt("do", [{info, Info}]), + case proplists:get_value(status,Info#mod.data) of + %% A status code has been generated! + {_StatusCode, _PhraseArgs, _Reason} -> + {proceed, Info#mod.data}; + %% No status code has been generated! + undefined -> + case proplists:get_value(response, Info#mod.data) of + %% No response has been generated! + undefined -> + Path = mod_alias:path(Info#mod.data,Info#mod.config_db, + Info#mod.request_uri), + %% Is it a secret area? + case secretp(Path,Info#mod.config_db) of + {yes, {Directory, DirectoryData}} -> + ?hdrt("secret area", + [{directory, Directory}, + {directory_data, DirectoryData}]), + + %% Authenticate (allow) + case allow((Info#mod.init_data)#init_data.peername, + Info#mod.socket_type,Info#mod.socket, + DirectoryData) of + allowed -> + ?hdrt("allowed", []), + case deny((Info#mod.init_data)#init_data.peername, + Info#mod.socket_type, + Info#mod.socket, + DirectoryData) of + not_denied -> + ?hdrt("not denied", []), + case proplists:get_value(auth_type, + DirectoryData) of + undefined -> + {proceed, Info#mod.data}; + none -> + {proceed, Info#mod.data}; + AuthType -> + do_auth(Info, + Directory, + DirectoryData, + AuthType) + end; + {denied, Reason} -> + ?hdrt("denied", [{reason, Reason}]), + {proceed, + [{status, {403, + Info#mod.request_uri, + Reason}}| + Info#mod.data]} + end; + {not_allowed, Reason} -> + ?hdrt("not allowed", [{reason, Reason}]), + {proceed,[{status,{403, + Info#mod.request_uri, + Reason}} | + Info#mod.data]} + end; + no -> + {proceed, Info#mod.data} + end; + %% A response has been generated or sent! + _Response -> + {proceed, Info#mod.data} + end + end. + + +do_auth(Info, Directory, DirectoryData, AuthType) -> + %% Authenticate (require) + ?hdrt("authenticate", [{auth_type, AuthType}]), + case require(Info, Directory, DirectoryData) of + authorized -> + ?hdrt("authorized", []), + {proceed,Info#mod.data}; + {authorized, User} -> + ?hdrt("authorized", [{user, User}]), + {proceed, [{remote_user,User}|Info#mod.data]}; + {authorization_required, Realm} -> + ?hdrt("authorization required", [{realm, Realm}]), + ReasonPhrase = httpd_util:reason_phrase(401), + Message = httpd_util:message(401,none,Info#mod.config_db), + {proceed, + [{response, + {401, + ["WWW-Authenticate: Basic realm=\"",Realm, + "\"\r\n\r\n","<HTML>\n<HEAD>\n<TITLE>", + ReasonPhrase,"</TITLE>\n", + "</HEAD>\n<BODY>\n<H1>",ReasonPhrase, + "</H1>\n",Message,"\n</BODY>\n</HTML>\n"]}}| + Info#mod.data]}; + {status, {StatusCode,PhraseArgs,Reason}} -> + {proceed, [{status,{StatusCode,PhraseArgs,Reason}}| + Info#mod.data]} + end. + +%% require + +require(Info, Directory, DirectoryData) -> + ParsedHeader = Info#mod.parsed_header, + ValidUsers = proplists:get_value(require_user, DirectoryData), + ValidGroups = proplists:get_value(require_group, DirectoryData), + %% Any user or group restrictions? + case ValidGroups of + undefined when ValidUsers =:= undefined -> + authorized; + _ -> + case proplists:get_value("authorization", ParsedHeader) of + undefined -> + authorization_required(DirectoryData); + %% Check credentials! + "Basic" ++ EncodedString = Credentials -> + case (catch base64:decode_to_string(EncodedString)) of + {'EXIT',{function_clause, _}} -> + {status, {401, none, ?NICE("Bad credentials "++ + Credentials)}}; + DecodedString -> + validate_user(Info, Directory, DirectoryData, + ValidUsers, ValidGroups, + DecodedString) + end; + %% Bad credentials! + BadCredentials -> + {status, {401, none, ?NICE("Bad credentials "++ + BadCredentials)}} + end + end. + +authorization_required(DirectoryData) -> + case proplists:get_value(auth_name, DirectoryData) of + undefined -> + {status,{500, none,?NICE("AuthName directive not specified")}}; + Realm -> + {authorization_required, Realm} + end. + + +validate_user(Info, Directory, DirectoryData, ValidUsers, + ValidGroups, DecodedString) -> + case a_valid_user(Info, DecodedString, + ValidUsers, ValidGroups, + Directory, DirectoryData) of + {yes, User} -> + {authorized, User}; + {no, _Reason} -> + authorization_required(DirectoryData); + {status, {StatusCode,PhraseArgs,Reason}} -> + {status,{StatusCode,PhraseArgs,Reason}} + end. + +a_valid_user(Info,DecodedString,ValidUsers,ValidGroups,Dir,DirData) -> + case httpd_util:split(DecodedString,":",2) of + {ok, [SupposedUser, Password]} -> + case user_accepted(SupposedUser, ValidUsers) of + true -> + check_password(SupposedUser, Password, Dir, DirData); + false -> + case group_accepted(Info,SupposedUser, + ValidGroups,Dir,DirData) of + true -> + check_password(SupposedUser,Password,Dir,DirData); + false -> + {no,?NICE("No such user exists")} + end + end; + {ok, BadCredentials} -> + {status,{401,none,?NICE("Bad credentials "++BadCredentials)}} + end. + +user_accepted(_SupposedUser, undefined) -> + false; +user_accepted(SupposedUser, ValidUsers) -> + lists:member(SupposedUser, ValidUsers). + + +group_accepted(_Info, _User, undefined, _Dir, _DirData) -> + false; +group_accepted(_Info, _User, [], _Dir, _DirData) -> + false; +group_accepted(Info, User, [Group|Rest], Dir, DirData) -> + Ret = int_list_group_members(Group, Dir, DirData), + case Ret of + {ok, UserList} -> + case lists:member(User, UserList) of + true -> + true; + false -> + group_accepted(Info, User, Rest, Dir, DirData) + end; + _ -> + false + end. + +check_password(User, Password, _Dir, DirData) -> + case int_get_user(DirData, User) of + {ok, UStruct} -> + case UStruct#httpd_user.password of + Password -> + %% FIXME + {yes, UStruct#httpd_user.username}; + _ -> + {no, "No such user"} % Don't say 'Bad Password' !!! + end; + _Other -> + {no, "No such user"} + end. + + +%% Middle API. Theese functions call the appropriate authentication module. +int_get_user(DirData, User) -> + AuthMod = auth_mod_name(DirData), + apply(AuthMod, get_user, [DirData, User]). + +int_list_group_members(Group, _Dir, DirData) -> + AuthMod = auth_mod_name(DirData), + apply(AuthMod, list_group_members, [DirData, Group]). + +auth_mod_name(DirData) -> + case proplists:get_value(auth_type, DirData, plain) of + plain -> mod_auth_plain; + mnesia -> mod_auth_mnesia; + dets -> mod_auth_dets + end. + + +%% +%% Is it a secret area? +%% + +%% secretp + +secretp(Path,ConfigDB) -> + Directories = ets:match(ConfigDB,{directory, {'$1','_'}}), + case secret_path(Path, Directories) of + {yes,Directory} -> + {yes, {Directory, + lists:flatten( + ets:match(ConfigDB,{directory, {Directory,'$1'}}))}}; + no -> + no + end. + +secret_path(Path, Directories) -> + secret_path(Path, httpd_util:uniq(lists:sort(Directories)),to_be_found). + +secret_path(_Path, [], to_be_found) -> + no; +secret_path(_Path, [], Directory) -> + {yes, Directory}; +secret_path(Path, [[NewDirectory] | Rest], Directory) -> + case inets_regexp:match(Path, NewDirectory) of + {match, _, _} when Directory =:= to_be_found -> + secret_path(Path, Rest, NewDirectory); + {match, _, Length} when Length > length(Directory)-> + secret_path(Path, Rest,NewDirectory); + {match, _, _Length} -> + secret_path(Path, Rest, Directory); + nomatch -> + secret_path(Path, Rest, Directory) + end. + +%% +%% Authenticate +%% + +%% allow + +allow({_,RemoteAddr}, _SocketType, _Socket, DirectoryData) -> + Hosts = proplists:get_value(allow_from, DirectoryData, all), + case validate_addr(RemoteAddr, Hosts) of + true -> + allowed; + false -> + {not_allowed, ?NICE("Connection from your host is not allowed")} + end. + +validate_addr(_RemoteAddr, all) -> % When called from 'allow' + true; +validate_addr(_RemoteAddr, none) -> % When called from 'deny' + false; +validate_addr(_RemoteAddr, []) -> + false; +validate_addr(RemoteAddr, [HostRegExp | Rest]) -> + case inets_regexp:match(RemoteAddr, HostRegExp) of + {match,_,_} -> + true; + nomatch -> + validate_addr(RemoteAddr,Rest) + end. + +%% deny + +deny({_,RemoteAddr}, _SocketType, _Socket,DirectoryData) -> + Hosts = proplists:get_value(deny_from, DirectoryData, none), + case validate_addr(RemoteAddr,Hosts) of + true -> + {denied, ?NICE("Connection from your host is not allowed")}; + false -> + not_denied + end. + +%% +%% Configuration +%% + +%% load/2 +%% + +%% mod_auth recognizes the following Configuration Directives: +%% <Directory /path/to/directory> +%% AuthDBType +%% AuthName +%% AuthUserFile +%% AuthGroupFile +%% AuthAccessPassword +%% require +%% allow +%% </Directory> + +%% When a <Directory> directive is found, a new context is set to +%% [{directory, Directory, DirData}|OtherContext] +%% DirData in this case is a key-value list of data belonging to the +%% directory in question. +%% +%% When the </Directory> statement is found, the Context created earlier +%% will be returned as a ConfigList and the context will return to the +%% state it was previously. + +load("<Directory " ++ Directory,[]) -> + Dir = httpd_conf:custom_clean(Directory,"",">"), + {ok,[{directory, {Dir, [{path, Dir}]}}]}; +load(eof,[{directory, {Directory, _DirData}}|_]) -> + {error, ?NICE("Premature end-of-file in "++ Directory)}; + +load("AuthName " ++ AuthName, [{directory, {Directory, DirData}}|Rest]) -> + {ok, [{directory, {Directory, + [{auth_name, httpd_conf:clean(AuthName)} | DirData]}} + | Rest ]}; +load("AuthUserFile " ++ AuthUserFile0, + [{directory, {Directory, DirData}}|Rest]) -> + AuthUserFile = httpd_conf:clean(AuthUserFile0), + {ok, [{directory, {Directory, + [{auth_user_file, AuthUserFile}|DirData]}} | Rest ]}; +load("AuthGroupFile " ++ AuthGroupFile0, + [{directory, {Directory, DirData}}|Rest]) -> + AuthGroupFile = httpd_conf:clean(AuthGroupFile0), + {ok,[{directory, {Directory, + [{auth_group_file, AuthGroupFile}|DirData]}} | Rest]}; + +%AuthAccessPassword +load("AuthAccessPassword " ++ AuthAccessPassword0, + [{directory, {Directory, DirData}}|Rest]) -> + AuthAccessPassword = httpd_conf:clean(AuthAccessPassword0), + {ok,[{directory, {Directory, + [{auth_access_password, AuthAccessPassword}|DirData]}} | Rest]}; + +load("AuthDBType " ++ Type, + [{directory, {Dir, DirData}}|Rest]) -> + case httpd_conf:clean(Type) of + "plain" -> + {ok, [{directory, {Dir, [{auth_type, plain}|DirData]}} | Rest ]}; + "mnesia" -> + {ok, [{directory, {Dir, [{auth_type, mnesia}|DirData]}} | Rest ]}; + "dets" -> + {ok, [{directory, {Dir, [{auth_type, dets}|DirData]}} | Rest ]}; + _ -> + {error, ?NICE(httpd_conf:clean(Type)++" is an invalid AuthDBType")} + end; + +load("require " ++ Require,[{directory, {Directory, DirData}}|Rest]) -> + case inets_regexp:split(Require," ") of + {ok,["user"|Users]} -> + {ok,[{directory, {Directory, + [{require_user,Users}|DirData]}} | Rest]}; + {ok,["group"|Groups]} -> + {ok,[{directory, {Directory, + [{require_group,Groups}|DirData]}} | Rest]}; + {ok,_} -> + {error,?NICE(httpd_conf:clean(Require) ++" is an invalid require")} + end; + +load("allow " ++ Allow,[{directory, {Directory, DirData}}|Rest]) -> + case inets_regexp:split(Allow," ") of + {ok,["from","all"]} -> + {ok,[{directory, {Directory, + [{allow_from,all}|DirData]}} | Rest]}; + {ok,["from"|Hosts]} -> + {ok,[{directory, {Directory, + [{allow_from,Hosts}|DirData]}} | Rest]}; + {ok,_} -> + {error,?NICE(httpd_conf:clean(Allow) ++" is an invalid allow")} + end; + +load("deny " ++ Deny,[{directory, {Directory, DirData}}|Rest]) -> + case inets_regexp:split(Deny," ") of + {ok, ["from", "all"]} -> + {ok,[{{directory, Directory, + [{deny_from, all}|DirData]}} | Rest]}; + {ok, ["from"|Hosts]} -> + {ok,[{{directory, Directory, + [{deny_from, Hosts}|DirData]}} | Rest]}; + {ok, _} -> + {error,?NICE(httpd_conf:clean(Deny) ++" is an invalid deny")} + end; + +load("</Directory>",[{directory, {Directory, DirData}}|Rest]) -> + {ok, Rest, {directory, {Directory, DirData}}}; + +load("AuthMnesiaDB " ++ AuthMnesiaDB, + [{directory, {Dir, DirData}}|Rest]) -> + case httpd_conf:clean(AuthMnesiaDB) of + "On" -> + {ok,[{directory, {Dir,[{auth_type,mnesia}|DirData]}}|Rest]}; + "Off" -> + {ok,[{directory, {Dir,[{auth_type,plain}|DirData]}}|Rest]}; + _ -> + {error, ?NICE(httpd_conf:clean(AuthMnesiaDB) ++ + " is an invalid AuthMnesiaDB")} + end. + +directory_config_check(Directory, DirData) -> + case proplists:get_value(auth_type, DirData) of + plain -> + check_filename_present(Directory,auth_user_file,DirData), + check_filename_present(Directory,auth_group_file,DirData); + _ -> + ok + end. +check_filename_present(Dir,AuthFile,DirData) -> + case proplists:get_value(AuthFile,DirData) of + Name when is_list(Name) -> + ok; + _ -> + throw({missing_auth_file, AuthFile, {directory, {Dir, DirData}}}) + end. + +%% store + +store({directory, {Directory, DirData}}, ConfigList) + when is_list(Directory) andalso is_list(DirData) -> + ?hdrt("store", + [{directory, Directory}, {dir_data, DirData}]), + try directory_config_check(Directory, DirData) of + ok -> + store_directory(Directory, DirData, ConfigList) + catch + throw:Error -> + {error, Error, {directory, Directory, DirData}} + end; +store({directory, {Directory, DirData}}, _) -> + {error, {wrong_type, {directory, {Directory, DirData}}}}. + +store_directory(Directory0, DirData0, ConfigList) -> + ?hdrt("store directory - entry", + [{directory, Directory0}, {dir_data, DirData0}]), + Port = proplists:get_value(port, ConfigList), + DirData = case proplists:get_value(bind_address, ConfigList) of + undefined -> + [{port, Port}|DirData0]; + Addr -> + [{port, Port},{bind_address,Addr}|DirData0] + end, + Directory = + case filename:pathtype(Directory0) of + relative -> + SR = proplists:get_value(server_root, ConfigList), + filename:join(SR, Directory0); + _ -> + Directory0 + end, + AuthMod = + case proplists:get_value(auth_type, DirData0) of + mnesia -> mod_auth_mnesia; + dets -> mod_auth_dets; + plain -> mod_auth_plain; + _ -> no_module_at_all + end, + ?hdrt("store directory", + [{directory, Directory}, {dir_data, DirData}, {auth_mod, AuthMod}]), + case AuthMod of + no_module_at_all -> + {ok, {directory, {Directory, DirData}}}; + _ -> + %% Check that there are a password or add a standard password: + %% "NoPassword" + %% In this way a user must select to use a noPassword + Passwd = + case proplists:get_value(auth_access_password, DirData) of + undefined -> + ?NOPASSWORD; + PassW -> + PassW + end, + DirDataLast = lists:keydelete(auth_access_password,1,DirData), + Server_root = proplists:get_value(server_root, ConfigList), + case catch AuthMod:store_directory_data(Directory, + DirDataLast, + Server_root) of + ok -> + add_auth_password(Directory, Passwd, ConfigList), + {ok, {directory, {Directory, DirDataLast}}}; + {ok, NewDirData} -> + add_auth_password(Directory, Passwd, ConfigList), + {ok, {directory, {Directory, NewDirData}}}; + {error, Reason} -> + {error, Reason}; + Other -> + {error, Other} + end + end. + +add_auth_password(Dir, Pwd0, ConfigList) -> + Addr = proplists:get_value(bind_address, ConfigList), + Port = proplists:get_value(port, ConfigList), + mod_auth_server:start(Addr, Port), + mod_auth_server:add_password(Addr, Port, Dir, Pwd0). + +%% remove + + +remove(ConfigDB) -> + lists:foreach(fun({directory, {_Dir, DirData}}) -> + AuthMod = auth_mod_name(DirData), + (catch apply(AuthMod, remove, [DirData])) + end, + ets:match_object(ConfigDB,{directory,{'_','_'}})), + Addr = case lookup(ConfigDB, bind_address) of + [] -> + undefined; + [{bind_address, Address}] -> + Address + end, + [{port, Port}] = lookup(ConfigDB, port), + mod_auth_server:stop(Addr, Port), + ok. + +%% -------------------------------------------------------------------- + +%% update_password + +update_password(Port, Dir, Old, New, New)-> + update_password(undefined, Port, Dir, Old, New, New). + +update_password(Addr, Port, Dir, Old, New, New) when is_list(New) -> + mod_auth_server:update_password(Addr, Port, Dir, Old, New); + +update_password(_Addr, _Port, _Dir, _Old, _New, _New) -> + {error, badtype}; +update_password(_Addr, _Port, _Dir, _Old, _New, _New1) -> + {error, notqeual}. + + +%% add_user + +add_user(UserName, Opt) -> + case get_options(Opt, mandatory) of + {Addr, Port, Dir, AuthPwd}-> + case get_options(Opt, userData) of + {error, Reason}-> + {error, Reason}; + {UserData, Password}-> + User = [#httpd_user{username = UserName, + password = Password, + user_data = UserData}], + mod_auth_server:add_user(Addr, Port, Dir, User, AuthPwd) + end + end. + + +add_user(UserName, Password, UserData, Port, Dir) -> + add_user(UserName, Password, UserData, undefined, Port, Dir). +add_user(UserName, Password, UserData, Addr, Port, Dir) -> + User = [#httpd_user{username = UserName, + password = Password, + user_data = UserData}], + mod_auth_server:add_user(Addr, Port, Dir, User, ?NOPASSWORD). + + +%% get_user + +get_user(UserName, Opt) -> + case get_options(Opt, mandatory) of + {Addr, Port, Dir, AuthPwd} -> + mod_auth_server:get_user(Addr, Port, Dir, UserName, AuthPwd); + {error, Reason} -> + {error, Reason} + end. + +get_user(UserName, Port, Dir) -> + get_user(UserName, undefined, Port, Dir). +get_user(UserName, Addr, Port, Dir) -> + mod_auth_server:get_user(Addr, Port, Dir, UserName, ?NOPASSWORD). + + +%% add_group_member + +add_group_member(GroupName, UserName, Opt)-> + case get_options(Opt, mandatory) of + {Addr, Port, Dir, AuthPwd}-> + mod_auth_server:add_group_member(Addr, Port, Dir, + GroupName, UserName, AuthPwd); + {error, Reason} -> + {error, Reason} + end. + +add_group_member(GroupName, UserName, Port, Dir) -> + add_group_member(GroupName, UserName, undefined, Port, Dir). + +add_group_member(GroupName, UserName, Addr, Port, Dir) -> + mod_auth_server:add_group_member(Addr, Port, Dir, + GroupName, UserName, ?NOPASSWORD). + + +%% delete_group_member + +delete_group_member(GroupName, UserName, Opt) -> + case get_options(Opt, mandatory) of + {Addr, Port, Dir, AuthPwd} -> + mod_auth_server:delete_group_member(Addr, Port, Dir, + GroupName, UserName, AuthPwd); + {error, Reason} -> + {error, Reason} + end. + +delete_group_member(GroupName, UserName, Port, Dir) -> + delete_group_member(GroupName, UserName, undefined, Port, Dir). +delete_group_member(GroupName, UserName, Addr, Port, Dir) -> + mod_auth_server:delete_group_member(Addr, Port, Dir, + GroupName, UserName, ?NOPASSWORD). + + +%% list_users + +list_users(Opt) -> + case get_options(Opt, mandatory) of + {Addr, Port, Dir, AuthPwd} -> + mod_auth_server:list_users(Addr, Port, Dir, AuthPwd); + {error, Reason} -> + {error, Reason} + end. + +list_users(Port, Dir) -> + list_users(undefined, Port, Dir). +list_users(Addr, Port, Dir) -> + mod_auth_server:list_users(Addr, Port, Dir, ?NOPASSWORD). + + +%% delete_user + +delete_user(UserName, Opt) -> + case get_options(Opt, mandatory) of + {Addr, Port, Dir, AuthPwd} -> + mod_auth_server:delete_user(Addr, Port, Dir, UserName, AuthPwd); + {error, Reason} -> + {error, Reason} + end. + +delete_user(UserName, Port, Dir) -> + delete_user(UserName, undefined, Port, Dir). +delete_user(UserName, Addr, Port, Dir) -> + mod_auth_server:delete_user(Addr, Port, Dir, UserName, ?NOPASSWORD). + + +%% delete_group + +delete_group(GroupName, Opt) -> + case get_options(Opt, mandatory) of + {Addr, Port, Dir, AuthPwd} -> + mod_auth_server:delete_group(Addr, Port, Dir, GroupName, AuthPwd); + {error, Reason} -> + {error, Reason} + end. + +delete_group(GroupName, Port, Dir) -> + delete_group(GroupName, undefined, Port, Dir). +delete_group(GroupName, Addr, Port, Dir) -> + mod_auth_server:delete_group(Addr, Port, Dir, GroupName, ?NOPASSWORD). + + +%% list_groups + +list_groups(Opt) -> + case get_options(Opt, mandatory) of + {Addr, Port, Dir, AuthPwd} -> + mod_auth_server:list_groups(Addr, Port, Dir, AuthPwd); + {error, Reason} -> + {error, Reason} + end. + +list_groups(Port, Dir) -> + list_groups(undefined, Port, Dir). +list_groups(Addr, Port, Dir) -> + mod_auth_server:list_groups(Addr, Port, Dir, ?NOPASSWORD). + + +%% list_group_members + +list_group_members(GroupName, Opt) -> + case get_options(Opt, mandatory) of + {Addr, Port, Dir, AuthPwd} -> + mod_auth_server:list_group_members(Addr, Port, Dir, GroupName, + AuthPwd); + {error, Reason} -> + {error, Reason} + end. + +list_group_members(GroupName, Port, Dir) -> + list_group_members(GroupName, undefined, Port, Dir). +list_group_members(GroupName, Addr, Port, Dir) -> + mod_auth_server:list_group_members(Addr, Port, Dir, + GroupName, ?NOPASSWORD). + +%% Opt = [{port, Port}, +%% {addr, Addr}, +%% {dir, Dir}, +%% {authPassword, AuthPassword} | FunctionSpecificData] +get_options(Opt, mandatory)-> + case proplists:get_value(port, Opt, undefined) of + Port when is_integer(Port) -> + case proplists:get_value(dir, Opt, undefined) of + Dir when is_list(Dir) -> + Addr = proplists:get_value(addr, Opt, + undefined), + AuthPwd = proplists:get_value(authPassword, Opt, + ?NOPASSWORD), + {Addr, Port, Dir, AuthPwd}; + _-> + {error, bad_dir} + end; + _ -> + {error, bad_dir} + end; + +%% FunctionSpecificData = {userData, UserData} | {password, Password} +get_options(Opt, userData)-> + case proplists:get_value(userData, Opt, undefined) of + undefined -> + {error, no_userdata}; + UserData -> + case proplists:get_value(password, Opt, undefined) of + undefined-> + {error, no_password}; + Pwd -> + {UserData, Pwd} + end + end. + + +lookup(Db, Key) -> + ets:lookup(Db, Key). diff --git a/lib/inets/src/http_server/mod_auth.hrl b/lib/inets/src/http_server/mod_auth.hrl new file mode 100644 index 0000000000..9b316cecc4 --- /dev/null +++ b/lib/inets/src/http_server/mod_auth.hrl @@ -0,0 +1,29 @@ +%% +%% %CopyrightBegin% +%% +%% Copyright Ericsson AB 1998-2009. All Rights Reserved. +%% +%% The contents of this file are subject to the Erlang Public License, +%% Version 1.1, (the "License"); you may not use this file except in +%% compliance with the License. You should have received a copy of the +%% Erlang Public License along with this software. If not, it can be +%% retrieved online at http://www.erlang.org/. +%% +%% Software distributed under the License is distributed on an "AS IS" +%% basis, WITHOUT WARRANTY OF ANY KIND, either express or implied. See +%% the License for the specific language governing rights and limitations +%% under the License. +%% +%% %CopyrightEnd% +%% +%% + +-record(httpd_user, + {username, + password, + user_data}). + +-record(httpd_group, + {name, + userlist}). + diff --git a/lib/inets/src/http_server/mod_auth_dets.erl b/lib/inets/src/http_server/mod_auth_dets.erl new file mode 100644 index 0000000000..bc6c2b70a0 --- /dev/null +++ b/lib/inets/src/http_server/mod_auth_dets.erl @@ -0,0 +1,254 @@ +%% +%% %CopyrightBegin% +%% +%% Copyright Ericsson AB 1998-2009. All Rights Reserved. +%% +%% The contents of this file are subject to the Erlang Public License, +%% Version 1.1, (the "License"); you may not use this file except in +%% compliance with the License. You should have received a copy of the +%% Erlang Public License along with this software. If not, it can be +%% retrieved online at http://www.erlang.org/. +%% +%% Software distributed under the License is distributed on an "AS IS" +%% basis, WITHOUT WARRANTY OF ANY KIND, either express or implied. See +%% the License for the specific language governing rights and limitations +%% under the License. +%% +%% %CopyrightEnd% +%% +%% +-module(mod_auth_dets). + +%% dets authentication storage + +-export([get_user/2, + list_group_members/2, + add_user/2, + add_group_member/3, + list_users/1, + delete_user/2, + list_groups/1, + delete_group_member/3, + delete_group/2, + remove/1]). + +-export([store_directory_data/3]). + +-include("httpd.hrl"). +-include("mod_auth.hrl"). + +store_directory_data(_Directory, DirData, Server_root) -> + ?CDEBUG("store_directory_data -> ~n" + " Directory: ~p~n" + " DirData: ~p", + [_Directory, DirData]), + + {PWFile, Absolute_pwdfile} = absolute_file_name(auth_user_file, DirData, + Server_root), + {GroupFile, Absolute_groupfile} = absolute_file_name(auth_group_file, + DirData, Server_root), + Addr = proplists:get_value(bind_address, DirData), + Port = proplists:get_value(port, DirData), + + PWName = httpd_util:make_name("httpd_dets_pwdb",Addr,Port), + case dets:open_file(PWName,[{type,set},{file,Absolute_pwdfile},{repair,true}]) of + {ok, PWDB} -> + GDBName = httpd_util:make_name("httpd_dets_groupdb",Addr,Port), + case dets:open_file(GDBName,[{type,set},{file,Absolute_groupfile},{repair,true}]) of + {ok, GDB} -> + NDD1 = lists:keyreplace(auth_user_file, 1, DirData, + {auth_user_file, PWDB}), + NDD2 = lists:keyreplace(auth_group_file, 1, NDD1, + {auth_group_file, GDB}), + {ok, NDD2}; + {error, Err}-> + {error, {{file, GroupFile},Err}} + end; + {error, Err2} -> + {error, {{file, PWFile},Err2}} + end. + +%% +%% Storage format of users in the dets table: +%% {{UserName, Addr, Port, Dir}, Password, UserData} +%% + +add_user(DirData, UStruct) -> + {Addr, Port, Dir} = lookup_common(DirData), + PWDB = proplists:get_value(auth_user_file, DirData), + Record = {{UStruct#httpd_user.username, Addr, Port, Dir}, + UStruct#httpd_user.password, UStruct#httpd_user.user_data}, + case dets:lookup(PWDB, UStruct#httpd_user.username) of + [Record] -> + {error, user_already_in_db}; + _ -> + dets:insert(PWDB, Record), + true + end. + +get_user(DirData, UserName) -> + {Addr, Port, Dir} = lookup_common(DirData), + PWDB = proplists:get_value(auth_user_file, DirData), + User = {UserName, Addr, Port, Dir}, + case dets:lookup(PWDB, User) of + [{User, Password, UserData}] -> + {ok, #httpd_user{username=UserName, password=Password, user_data=UserData}}; + _ -> + {error, no_such_user} + end. + +list_users(DirData) -> + ?DEBUG("list_users -> ~n" + " DirData: ~p", [DirData]), + {Addr, Port, Dir} = lookup_common(DirData), + PWDB = proplists:get_value(auth_user_file, DirData), + case dets:traverse(PWDB, fun(X) -> {continue, X} end) of %% SOOOO Ugly ! + Records when is_list(Records) -> + ?DEBUG("list_users -> ~n" + " Records: ~p", [Records]), + {ok, [UserName || {{UserName, AnyAddr, AnyPort, AnyDir}, + _Password, _Data} <- Records, + AnyAddr == Addr, AnyPort == Port, + AnyDir == Dir]}; + _O -> + ?DEBUG("list_users -> ~n" + " O: ~p", [_O]), + {ok, []} + end. + +delete_user(DirData, UserName) -> + {Addr, Port, Dir} = lookup_common(DirData), + PWDB = proplists:get_value(auth_user_file, DirData), + User = {UserName, Addr, Port, Dir}, + case dets:lookup(PWDB, User) of + [{User, _SomePassword, _UserData}] -> + dets:delete(PWDB, User), + {ok, Groups} = list_groups(DirData), + lists:foreach(fun(Group) -> + delete_group_member(DirData, + Group, UserName) end, + Groups), + true; + _ -> + {error, no_such_user} + end. + +%% +%% Storage of groups in the dets table: +%% {Group, UserList} where UserList is a list of strings. +%% +add_group_member(DirData, GroupName, UserName) -> + {Addr, Port, Dir} = lookup_common(DirData), + GDB = proplists:get_value(auth_group_file, DirData), + Group = {GroupName, Addr, Port, Dir}, + case dets:lookup(GDB, Group) of + [{Group, Users}] -> + case lists:member(UserName, Users) of + true -> + true; + false -> + dets:insert(GDB, {Group, [UserName|Users]}), + true + end; + [] -> + dets:insert(GDB, {Group, [UserName]}), + true; + Other -> + {error, Other} + end. + +list_group_members(DirData, GroupName) -> + {Addr, Port, Dir} = lookup_common(DirData), + GDB = proplists:get_value(auth_group_file, DirData), + Group = {GroupName, Addr, Port, Dir}, + case dets:lookup(GDB, Group) of + [{Group, Users}] -> + {ok, Users}; + _ -> + {error, no_such_group} + end. + +list_groups(DirData) -> + {Addr, Port, Dir} = lookup_common(DirData), + GDB = proplists:get_value(auth_group_file, DirData), + case dets:match(GDB, {'$1', '_'}) of + [] -> + {ok, []}; + List when is_list(List) -> + Groups = lists:flatten(List), + {ok, [GroupName || + {GroupName, AnyAddr, AnyPort, AnyDir} <- Groups, + AnyAddr == Addr, AnyPort == Port, AnyDir == Dir]}; + _ -> + {ok, []} + end. + +delete_group_member(DirData, GroupName, UserName) -> + {Addr, Port, Dir} = lookup_common(DirData), + GDB = proplists:get_value(auth_group_file, DirData), + Group = {GroupName, Addr, Port, Dir}, + case dets:lookup(GDB, GroupName) of + [{Group, Users}] -> + case lists:member(UserName, Users) of + true -> + dets:delete(GDB, Group), + dets:insert(GDB, {Group, + lists:delete(UserName, Users)}), + true; + false -> + {error, no_such_group_member} + end; + _ -> + {error, no_such_group} + end. + +delete_group(DirData, GroupName) -> + {Addr, Port, Dir} = lookup_common(DirData), + GDB = proplists:get_value(auth_group_file, DirData), + Group = {GroupName, Addr, Port, Dir}, + case dets:lookup(GDB, Group) of + [{Group, _Users}] -> + dets:delete(GDB, Group), + true; + _ -> + {error, no_such_group} + end. + +lookup_common(DirData) -> + Dir = proplists:get_value(path, DirData), + Port = proplists:get_value(port, DirData), + Addr = proplists:get_value(bind_address, DirData), + {Addr, Port, Dir}. + +%% remove/1 +%% +%% Closes dets tables used by this auth mod. +%% +remove(DirData) -> + PWDB = proplists:get_value(auth_user_file, DirData), + GDB = proplists:get_value(auth_group_file, DirData), + dets:close(GDB), + dets:close(PWDB), + ok. + +%% absolute_file_name/2 +%% +%% Return the absolute path name of File_type. +absolute_file_name(File_type, DirData, Server_root) -> + Path = proplists:get_value(File_type, DirData), + Absolute_path = case filename:pathtype(Path) of + relative -> + case Server_root of + undefined -> + {error, + ?NICE(Path++ + " is an invalid file name because " + "ServerRoot is not defined")}; + _ -> + filename:join(Server_root,Path) + end; + _ -> + Path + end, + {Path, Absolute_path}. + diff --git a/lib/inets/src/http_server/mod_auth_mnesia.erl b/lib/inets/src/http_server/mod_auth_mnesia.erl new file mode 100644 index 0000000000..ffe028617b --- /dev/null +++ b/lib/inets/src/http_server/mod_auth_mnesia.erl @@ -0,0 +1,284 @@ +%% +%% %CopyrightBegin% +%% +%% Copyright Ericsson AB 1997-2009. All Rights Reserved. +%% +%% The contents of this file are subject to the Erlang Public License, +%% Version 1.1, (the "License"); you may not use this file except in +%% compliance with the License. You should have received a copy of the +%% Erlang Public License along with this software. If not, it can be +%% retrieved online at http://www.erlang.org/. +%% +%% Software distributed under the License is distributed on an "AS IS" +%% basis, WITHOUT WARRANTY OF ANY KIND, either express or implied. See +%% the License for the specific language governing rights and limitations +%% under the License. +%% +%% %CopyrightEnd% +%% +%% +-module(mod_auth_mnesia). +-export([get_user/2, + list_group_members/2, + add_user/2, + add_group_member/3, + list_users/1, + delete_user/2, + list_groups/1, + delete_group_member/3, + delete_group/2]). + +-export([store_user/5, store_user/6, + store_group_member/5, store_group_member/6, + list_group_members/3, list_group_members/4, + list_groups/2, list_groups/3, + list_users/2, list_users/3, + remove_user/4, remove_user/5, + remove_group_member/5, remove_group_member/6, + remove_group/4, remove_group/5]). + +-export([store_directory_data/3]). + +-include("httpd.hrl"). +-include("mod_auth.hrl"). + + + +store_directory_data(_Directory, _DirData, _Server_root) -> + %% We don't need to do anything here, we could of course check that + %% the appropriate mnesia tables has been created prior to + %% starting the http server. + ok. + + +%% +%% API +%% + +%% Compability API + +store_user(UserName, Password, Port, Dir, _AccessPassword) -> + %% AccessPassword is ignored - was not used in previous version + DirData = [{path,Dir},{port,Port}], + UStruct = #httpd_user{username = UserName, + password = Password}, + add_user(DirData, UStruct). + +store_user(UserName, Password, Addr, Port, Dir, _AccessPassword) -> + %% AccessPassword is ignored - was not used in previous version + DirData = [{path,Dir},{bind_address,Addr},{port,Port}], + UStruct = #httpd_user{username = UserName, + password = Password}, + add_user(DirData, UStruct). + +store_group_member(GroupName, UserName, Port, Dir, _AccessPassword) -> + DirData = [{path,Dir},{port,Port}], + add_group_member(DirData, GroupName, UserName). + +store_group_member(GroupName, UserName, Addr, Port, Dir, _AccessPassword) -> + DirData = [{path,Dir},{bind_address,Addr},{port,Port}], + add_group_member(DirData, GroupName, UserName). + +list_group_members(GroupName, Port, Dir) -> + DirData = [{path,Dir},{port,Port}], + list_group_members(DirData, GroupName). + +list_group_members(GroupName, Addr, Port, Dir) -> + DirData = [{path,Dir},{bind_address,Addr},{port,Port}], + list_group_members(DirData, GroupName). + +list_groups(Port, Dir) -> + DirData = [{path,Dir},{port,Port}], + list_groups(DirData). + +list_groups(Addr, Port, Dir) -> + DirData = [{path,Dir},{bind_address,Addr},{port,Port}], + list_groups(DirData). + +list_users(Port, Dir) -> + DirData = [{path,Dir},{port,Port}], + list_users(DirData). + +list_users(Addr, Port, Dir) -> + DirData = [{path,Dir},{bind_address,Addr},{port,Port}], + list_users(DirData). + +remove_user(UserName, Port, Dir, _AccessPassword) -> + DirData = [{path,Dir},{port,Port}], + delete_user(DirData, UserName). + +remove_user(UserName, Addr, Port, Dir, _AccessPassword) -> + DirData = [{path,Dir},{bind_address,Addr},{port,Port}], + delete_user(DirData, UserName). + +remove_group_member(GroupName,UserName,Port,Dir,_AccessPassword) -> + DirData = [{path,Dir},{port,Port}], + delete_group_member(DirData, GroupName, UserName). + +remove_group_member(GroupName,UserName,Addr,Port,Dir,_AccessPassword) -> + DirData = [{path,Dir},{bind_address,Addr},{port,Port}], + delete_group_member(DirData, GroupName, UserName). + +remove_group(GroupName,Port,Dir,_AccessPassword) -> + DirData = [{path,Dir},{port,Port}], + delete_group(DirData, GroupName). + +remove_group(GroupName,Addr,Port,Dir,_AccessPassword) -> + DirData = [{path,Dir},{bind_address,Addr},{port,Port}], + delete_group(DirData, GroupName). + +%% +%% Storage format of users in the mnesia table: +%% httpd_user records +%% + +add_user(DirData, UStruct) -> + {Addr, Port, Dir} = lookup_common(DirData), + UserName = UStruct#httpd_user.username, + Password = UStruct#httpd_user.password, + Data = UStruct#httpd_user.user_data, + User=#httpd_user{username={UserName,Addr,Port,Dir}, + password=Password, + user_data=Data}, + case mnesia:transaction(fun() -> mnesia:write(User) end) of + {aborted,Reason} -> + {error,Reason}; + _ -> + true + end. + +get_user(DirData, UserName) -> + {Addr, Port, Dir} = lookup_common(DirData), + case mnesia:transaction(fun() -> + mnesia:read({httpd_user, + {UserName,Addr,Port,Dir}}) + end) of + {aborted,Reason} -> + {error, Reason}; + {atomic,[]} -> + {error, no_such_user}; + {atomic, [Record]} when is_record(Record, httpd_user) -> + {ok, Record#httpd_user{username=UserName}}; + _ -> + {error, no_such_user} + end. + +list_users(DirData) -> + {Addr, Port, Dir} = lookup_common(DirData), + case mnesia:transaction(fun() -> + mnesia:match_object({httpd_user, + {'_',Addr,Port,Dir},'_','_'}) + end) of + {aborted,Reason} -> + {error,Reason}; + {atomic,Users} -> + {ok, + lists:foldr(fun({httpd_user, + {UserName, _AnyAddr, _AnyPort, _AnyDir}, + _Password, _Data}, Acc) -> + [UserName|Acc] + end, + [], Users)} + end. + +delete_user(DirData, UserName) -> + {Addr, Port, Dir} = lookup_common(DirData), + case mnesia:transaction(fun() -> + mnesia:delete({httpd_user, + {UserName,Addr,Port,Dir}}) + end) of + {aborted,Reason} -> + {error,Reason}; + _ -> + true + end. + +%% +%% Storage of groups in the mnesia table: +%% Multiple instances of {#httpd_group, User} +%% + +add_group_member(DirData, GroupName, User) -> + {Addr, Port, Dir} = lookup_common(DirData), + Group=#httpd_group{name={GroupName, Addr, Port, Dir}, userlist=User}, + case mnesia:transaction(fun() -> mnesia:write(Group) end) of + {aborted,Reason} -> + {error,Reason}; + _ -> + true + end. + +list_group_members(DirData, GroupName) -> + {Addr, Port, Dir} = lookup_common(DirData), + case mnesia:transaction(fun() -> + mnesia:read({httpd_group, + {GroupName,Addr,Port,Dir}}) + end) of + {aborted, Reason} -> + {error,Reason}; + {atomic, Members} -> + {ok,[UserName || {httpd_group,{AnyGroupName,AnyAddr, + AnyPort,AnyDir},UserName} + <- Members, + AnyGroupName == GroupName, AnyAddr == Addr, + AnyPort == Port, AnyDir == Dir]} + end. + +list_groups(DirData) -> + {Addr, Port, Dir} = lookup_common(DirData), + case mnesia:transaction(fun() -> + mnesia:match_object({httpd_group, + {'_',Addr,Port,Dir}, + '_'}) + end) of + {aborted, Reason} -> + {error, Reason}; + {atomic, Groups} -> + GroupNames= + [GroupName || {httpd_group,{GroupName,AnyAddr,AnyPort,AnyDir}, + _UserName} <- Groups, + AnyAddr == Addr, AnyPort == AnyPort, + AnyDir == Dir], + {ok, httpd_util:uniq(lists:sort(GroupNames))} + end. + +delete_group_member(DirData, GroupName, UserName) -> + {Addr, Port, Dir} = lookup_common(DirData), + Group = #httpd_group{name={GroupName, Addr, Port, Dir}, userlist=UserName}, + case mnesia:transaction(fun() -> mnesia:delete_object(Group) end) of + {aborted,Reason} -> + {error,Reason}; + _ -> + true + end. + +%% THIS IS WRONG (?) ! +%% Should first match out all httpd_group records for this group and then +%% do mnesia:delete on those. Or ? + +delete_group(DirData, GroupName) -> + {Addr, Port, Dir} = lookup_common(DirData), + case mnesia:transaction(fun() -> + mnesia:delete({httpd_group, + {GroupName,Addr,Port,Dir}}) + end) of + {aborted,Reason} -> + {error,Reason}; + _ -> + true + end. + +%% Utility functions. + +lookup_common(DirData) -> + Dir = proplists:get_value(path, DirData), + Port = proplists:get_value(port, DirData), + Addr = proplists:get_value(bind_address, DirData), + {Addr, Port, Dir}. + + + + + + + diff --git a/lib/inets/src/http_server/mod_auth_plain.erl b/lib/inets/src/http_server/mod_auth_plain.erl new file mode 100644 index 0000000000..d88859d28a --- /dev/null +++ b/lib/inets/src/http_server/mod_auth_plain.erl @@ -0,0 +1,325 @@ +%% +%% %CopyrightBegin% +%% +%% Copyright Ericsson AB 1998-2009. All Rights Reserved. +%% +%% The contents of this file are subject to the Erlang Public License, +%% Version 1.1, (the "License"); you may not use this file except in +%% compliance with the License. You should have received a copy of the +%% Erlang Public License along with this software. If not, it can be +%% retrieved online at http://www.erlang.org/. +%% +%% Software distributed under the License is distributed on an "AS IS" +%% basis, WITHOUT WARRANTY OF ANY KIND, either express or implied. See +%% the License for the specific language governing rights and limitations +%% under the License. +%% +%% %CopyrightEnd% +%% +%% +-module(mod_auth_plain). + +-include("httpd.hrl"). +-include("mod_auth.hrl"). +-include("httpd_internal.hrl"). + +-define(VMODULE,"AUTH_PLAIN"). + +%% Internal API +-export([store_directory_data/3]). + + +-export([get_user/2, + list_group_members/2, + add_user/2, + add_group_member/3, + list_users/1, + delete_user/2, + list_groups/1, + delete_group_member/3, + delete_group/2, + remove/1]). + +%% +%% API +%% + +%% +%% Storage format of users in the ets table: +%% {UserName, Password, UserData} +%% + +add_user(DirData, #httpd_user{username = User} = UStruct) -> + ?hdrt("add user", [{user, UStruct}]), + PWDB = proplists:get_value(auth_user_file, DirData), + Record = {User, + UStruct#httpd_user.password, + UStruct#httpd_user.user_data}, + case ets:lookup(PWDB, User) of + [{User, _SomePassword, _SomeData}] -> + {error, user_already_in_db}; + _ -> + ets:insert(PWDB, Record), + true + end. + +get_user(DirData, User) -> + ?hdrt("get user", [{dir_data, DirData}, {user, User}]), + PWDB = proplists:get_value(auth_user_file, DirData), + case ets:lookup(PWDB, User) of + [{User, PassWd, Data}] -> + {ok, #httpd_user{username = User, + password = PassWd, + user_data = Data}}; + _Other -> + {error, no_such_user} + end. + +list_users(DirData) -> + PWDB = proplists:get_value(auth_user_file, DirData), + Records = ets:match(PWDB, '$1'), + {ok, lists:foldr(fun({User, _PassWd, _Data}, A) -> [User | A] end, + [], lists:flatten(Records))}. + +delete_user(DirData, UserName) -> + ?hdrt("delete user", [{dir_data, DirData}, {user, UserName}]), + PWDB = proplists:get_value(auth_user_file, DirData), + case ets:lookup(PWDB, UserName) of + [{UserName, _SomePassword, _SomeData}] -> + ets:delete(PWDB, UserName), + {ok, Groups} = list_groups(DirData), + lists:foreach(fun(Group) -> + delete_group_member(DirData, + Group, UserName) + end, Groups); + _ -> + {error, no_such_user} + end. + +%% +%% Storage of groups in the ets table: +%% {Group, UserList} where UserList is a list of strings. +%% + +add_group_member(DirData, Group, UserName) -> + GDB = proplists:get_value(auth_group_file, DirData), + case ets:lookup(GDB, Group) of + [{Group, Users}] -> + case lists:member(UserName, Users) of + true -> + true; + false -> + ets:insert(GDB, {Group, [UserName|Users]}), + true + end; + [] -> + ets:insert(GDB, {Group, [UserName]}), + true; + Other -> + {error, Other} + end. + +list_group_members(DirData, Group) -> + GDB = proplists:get_value(auth_group_file, DirData), + case ets:lookup(GDB, Group) of + [{Group, Users}] -> + {ok, Users}; + _ -> + {error, no_such_group} + end. + +list_groups(DirData) -> + GDB = proplists:get_value(auth_group_file, DirData), + Groups = ets:match(GDB, '$1'), + {ok, httpd_util:uniq(lists:foldr(fun({G, _}, A) -> [G|A] end, + [], lists:flatten(Groups)))}. + +delete_group_member(DirData, Group, User) -> + GDB = proplists:get_value(auth_group_file, DirData), + case ets:lookup(GDB, Group) of + [{Group, Users}] when is_list(Users) -> + case lists:member(User, Users) of + true -> + ets:delete(GDB, Group), + ets:insert(GDB, {Group, lists:delete(User, Users)}), + true; + false -> + {error, no_such_group_member} + end; + _ -> + {error, no_such_group} + end. + +delete_group(DirData, Group) -> + GDB = proplists:get_value(auth_group_file, DirData), + case ets:lookup(GDB, Group) of + [{Group, _Users}] -> + ets:delete(GDB, Group), + true; + _ -> + {error, no_such_group} + end. + +store_directory_data(_Directory, DirData, Server_root) -> + ?hdrt("store directory data", + [{dir_data, DirData}, {server_root, Server_root}]), + PWFile = absolute_file_name(auth_user_file, DirData, Server_root), + GroupFile = absolute_file_name(auth_group_file, DirData, Server_root), + case load_passwd(PWFile) of + {ok, PWDB} -> + ?hdrt("password file loaded", [{file, PWFile}, {pwdb, PWDB}]), + case load_group(GroupFile) of + {ok, GRDB} -> + ?hdrt("group file loaded", + [{file, GroupFile}, {grdb, GRDB}]), + %% Address and port is included in the file names... + Addr = proplists:get_value(bind_address, DirData), + Port = proplists:get_value(port, DirData), + {ok, PasswdDB} = store_passwd(Addr,Port,PWDB), + {ok, GroupDB} = store_group(Addr,Port,GRDB), + NDD1 = lists:keyreplace(auth_user_file, 1, DirData, + {auth_user_file, PasswdDB}), + NDD2 = lists:keyreplace(auth_group_file, 1, NDD1, + {auth_group_file, GroupDB}), + {ok, NDD2}; + Err -> + {error, Err} + end; + Err2 -> + {error, Err2} + end. + + + +%% load_passwd + +load_passwd(AuthUserFile) -> + case file:open(AuthUserFile, [read]) of + {ok,Stream} -> + parse_passwd(Stream, []); + {error, _} -> + {error, ?NICE("Can't open " ++ AuthUserFile)} + end. + +parse_passwd(Stream, PasswdList) -> + Line = + case io:get_line(Stream, '') of + eof -> + eof; + String -> + httpd_conf:clean(String) + end, + parse_passwd(Stream, PasswdList, Line). + +parse_passwd(Stream, PasswdList, eof) -> + file:close(Stream), + {ok, PasswdList}; +parse_passwd(Stream, PasswdList, "") -> + parse_passwd(Stream, PasswdList); +parse_passwd(Stream, PasswdList, [$#|_]) -> + parse_passwd(Stream, PasswdList); +parse_passwd(Stream, PasswdList, Line) -> + case inets_regexp:split(Line,":") of + {ok, [User,Password]} -> + parse_passwd(Stream, [{User,Password, []}|PasswdList]); + {ok,_} -> + {error, ?NICE(Line)} + end. + +%% load_group + +load_group(AuthGroupFile) -> + case file:open(AuthGroupFile, [read]) of + {ok, Stream} -> + parse_group(Stream,[]); + {error, _} -> + {error, ?NICE("Can't open " ++ AuthGroupFile)} + end. + +parse_group(Stream, GroupList) -> + Line = + case io:get_line(Stream,'') of + eof -> + eof; + String -> + httpd_conf:clean(String) + end, + parse_group(Stream, GroupList, Line). + +parse_group(Stream, GroupList, eof) -> + file:close(Stream), + {ok, GroupList}; +parse_group(Stream, GroupList, "") -> + parse_group(Stream, GroupList); +parse_group(Stream, GroupList, [$#|_]) -> + parse_group(Stream, GroupList); +parse_group(Stream, GroupList, Line) -> + case inets_regexp:split(Line, ":") of + {ok, [Group,Users]} -> + {ok, UserList} = inets_regexp:split(Users," "), + parse_group(Stream, [{Group,UserList}|GroupList]); + {ok, _} -> + {error, ?NICE(Line)} + end. + + +%% store_passwd + +store_passwd(Addr,Port,PasswdList) -> + Name = httpd_util:make_name("httpd_passwd",Addr,Port), + PasswdDB = ets:new(Name, [set, public]), + store_passwd(PasswdDB, PasswdList). + +store_passwd(PasswdDB, []) -> + {ok, PasswdDB}; +store_passwd(PasswdDB, [User|Rest]) -> + ets:insert(PasswdDB, User), + store_passwd(PasswdDB, Rest). + +%% store_group + +store_group(Addr,Port,GroupList) -> + Name = httpd_util:make_name("httpd_group",Addr,Port), + GroupDB = ets:new(Name, [set, public]), + store_group(GroupDB, GroupList). + + +store_group(GroupDB,[]) -> + {ok, GroupDB}; +store_group(GroupDB, [User|Rest]) -> + ets:insert(GroupDB, User), + store_group(GroupDB, Rest). + + +%% remove/1 +%% +%% Deletes ets tables used by this auth mod. +%% +remove(DirData) -> + PWDB = proplists:get_value(auth_user_file, DirData), + GDB = proplists:get_value(auth_group_file, DirData), + ets:delete(PWDB), + ets:delete(GDB). + + + +%% absolute_file_name/2 +%% +%% Return the absolute path name of File_type. +absolute_file_name(File_type, DirData, Server_root) -> + Path = proplists:get_value(File_type, DirData), + case filename:pathtype(Path) of + relative -> + case Server_root of + undefined -> + {error, + ?NICE(Path++ + " is an invalid file name because " + "ServerRoot is not defined")}; + _ -> + filename:join(Server_root,Path) + end; + _ -> + Path + end. + diff --git a/lib/inets/src/http_server/mod_auth_server.erl b/lib/inets/src/http_server/mod_auth_server.erl new file mode 100644 index 0000000000..5f9e59be9d --- /dev/null +++ b/lib/inets/src/http_server/mod_auth_server.erl @@ -0,0 +1,400 @@ +%% +%% %CopyrightBegin% +%% +%% Copyright Ericsson AB 2001-2009. All Rights Reserved. +%% +%% The contents of this file are subject to the Erlang Public License, +%% Version 1.1, (the "License"); you may not use this file except in +%% compliance with the License. You should have received a copy of the +%% Erlang Public License along with this software. If not, it can be +%% retrieved online at http://www.erlang.org/. +%% +%% Software distributed under the License is distributed on an "AS IS" +%% basis, WITHOUT WARRANTY OF ANY KIND, either express or implied. See +%% the License for the specific language governing rights and limitations +%% under the License. +%% +%% %CopyrightEnd% +%% +%% + +-module(mod_auth_server). + +-include("httpd.hrl"). +-include("httpd_internal.hrl"). + +-behaviour(gen_server). + + +%% mod_auth exports +-export([start/2, stop/2, + add_password/4, update_password/5, + add_user/5, delete_user/5, get_user/5, list_users/4, + add_group_member/6, delete_group_member/6, list_group_members/5, + delete_group/5, list_groups/4]). + +%% gen_server exports +-export([start_link/2, init/1, + handle_call/3, handle_cast/2, handle_info/2, + terminate/2, code_change/3]). + +-record(state, {tab}). + + +%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%% +%% %% +%% External API %% +%% %% +%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%% + +%% start_link/3 +%% +%% NOTE: This is called by httpd_misc_sup when the process is started +%% +start_link(Addr, Port) -> + ?hdrt("start_link", [{address, Addr}, {port, Port}]), + Name = make_name(Addr, Port), + gen_server:start_link({local, Name}, ?MODULE, [], [{timeout, infinity}]). + + +%% start/2 + +start(Addr, Port) -> + ?hdrd("start", [{address, Addr}, {port, Port}]), + Name = make_name(Addr, Port), + case whereis(Name) of + undefined -> + httpd_misc_sup:start_auth_server(Addr, Port); + _ -> %% Already started... + ok + end. + + +%% stop/2 + +stop(Addr, Port) -> + ?hdrd("stop", [{address, Addr}, {port, Port}]), + Name = make_name(Addr, Port), + case whereis(Name) of + undefined -> %% Already stopped + ok; + _ -> + (catch httpd_misc_sup:stop_auth_server(Addr, Port)) + end. + +%% add_password/4 + +add_password(Addr, Port, Dir, Password) -> + ?hdrt("add password", [{address, Addr}, {port, Port}]), + Name = make_name(Addr, Port), + Req = {add_password, Dir, Password}, + call(Name, Req). + + +%% update_password/6 + +update_password(Addr, Port, Dir, Old, New) when is_list(New) -> + ?hdrt("update password", + [{address, Addr}, {port, Port}, {dir, Dir}, {old, Old}, {new, New}]), + Name = make_name(Addr, Port), + Req = {update_password, Dir, Old, New}, + call(Name, Req). + + +%% add_user/5 + +add_user(Addr, Port, Dir, User, Password) -> + ?hdrt("add user", + [{address, Addr}, {port, Port}, + {dir, Dir}, {user, User}, {passwd, Password}]), + Name = make_name(Addr, Port), + Req = {add_user, Addr, Port, Dir, User, Password}, + call(Name, Req). + + +%% delete_user/5 + +delete_user(Addr, Port, Dir, UserName, Password) -> + ?hdrt("delete user", + [{address, Addr}, {port, Port}, + {dir, Dir}, {user, UserName}, {passwd, Password}]), + Name = make_name(Addr, Port), + Req = {delete_user, Addr, Port, Dir, UserName, Password}, + call(Name, Req). + + +%% get_user/5 + +get_user(Addr, Port, Dir, UserName, Password) -> + ?hdrt("get user", + [{address, Addr}, {port, Port}, + {dir, Dir}, {user, UserName}, {passwd, Password}]), + Name = make_name(Addr, Port), + Req = {get_user, Addr, Port, Dir, UserName, Password}, + call(Name, Req). + + +%% list_users/4 + +list_users(Addr, Port, Dir, Password) -> + ?hdrt("list users", + [{address, Addr}, {port, Port}, {dir, Dir}, {passwd, Password}]), + Name = make_name(Addr,Port), + Req = {list_users, Addr, Port, Dir, Password}, + call(Name, Req). + + +%% add_group_member/6 + +add_group_member(Addr, Port, Dir, GroupName, UserName, Password) -> + ?hdrt("add group member", + [{address, Addr}, {port, Port}, {dir, Dir}, + {group, GroupName}, {user, UserName}, {passwd, Password}]), + Name = make_name(Addr,Port), + Req = {add_group_member, Addr, Port, Dir, GroupName, UserName, Password}, + call(Name, Req). + + +%% delete_group_member/6 + +delete_group_member(Addr, Port, Dir, GroupName, UserName, Password) -> + ?hdrt("delete group member", + [{address, Addr}, {port, Port}, {dir, Dir}, + {group, GroupName}, {user, UserName}, {passwd, Password}]), + Name = make_name(Addr,Port), + Req = {delete_group_member, Addr, Port, Dir, GroupName, UserName, Password}, + call(Name, Req). + + +%% list_group_members/4 + +list_group_members(Addr, Port, Dir, Group, Password) -> + ?hdrt("list group members", + [{address, Addr}, {port, Port}, {dir, Dir}, + {group, Group}, {passwd, Password}]), + Name = make_name(Addr, Port), + Req = {list_group_members, Addr, Port, Dir, Group, Password}, + call(Name, Req). + + +%% delete_group/5 + +delete_group(Addr, Port, Dir, GroupName, Password) -> + ?hdrt("delete group", + [{address, Addr}, {port, Port}, {dir, Dir}, + {group, GroupName}, {passwd, Password}]), + Name = make_name(Addr, Port), + Req = {delete_group, Addr, Port, Dir, GroupName, Password}, + call(Name, Req). + + +%% list_groups/4 + +list_groups(Addr, Port, Dir, Password) -> + ?hdrt("list groups", + [{address, Addr}, {port, Port}, {dir, Dir}, {passwd, Password}]), + Name = make_name(Addr, Port), + Req = {list_groups, Addr, Port, Dir, Password}, + call(Name, Req). + + +%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%% +%% %% +%% Server call-back functions %% +%% %% +%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%% + +%% init + +init(_) -> + ?hdrv("initiating", []), + {ok,#state{tab = ets:new(auth_pwd,[set,protected])}}. + +%% handle_call + +%% Add a user +handle_call({add_user, Addr, Port, Dir, User, AuthPwd}, _From, State) -> + Reply = api_call(Addr, Port, Dir, add_user, User, AuthPwd, State), + ?hdrt("add user", [{reply, Reply}]), + {reply, Reply, State}; + +%% Get data about a user +handle_call({get_user, Addr, Port, Dir, User, AuthPwd}, _From, State) -> + Reply = api_call(Addr, Port, Dir, get_user, [User], AuthPwd, State), + {reply, Reply, State}; + +%% Add a group member +handle_call({add_group_member, Addr, Port, Dir, Group, User, AuthPwd}, + _From, State) -> + Reply = api_call(Addr, Port, Dir, add_group_member, [Group, User], + AuthPwd, State), + {reply, Reply, State}; + +%% delete a group +handle_call({delete_group_member, Addr, Port, Dir, Group, User, AuthPwd}, + _From, State) -> + Reply = api_call(Addr, Port, Dir, delete_group_member, [Group, User], + AuthPwd, State), + {reply, Reply, State}; + +%% List all users thats standalone users +handle_call({list_users, Addr, Port, Dir, AuthPwd}, _From, State) -> + Reply = api_call(Addr, Port, Dir, list_users, [], AuthPwd, State), + {reply, Reply, State}; + +%% Delete a user +handle_call({delete_user, Addr, Port, Dir, User, AuthPwd}, _From, State) -> + Reply = api_call(Addr, Port, Dir, delete_user, [User], AuthPwd, State), + {reply, Reply, State}; + +%% Delete a group +handle_call({delete_group, Addr, Port, Dir, Group, AuthPwd}, _From, State) -> + Reply = api_call(Addr, Port, Dir, delete_group, [Group], AuthPwd, State), + {reply, Reply, State}; + +%% List the current groups +handle_call({list_groups, Addr, Port, Dir, AuthPwd}, _From, State) -> + Reply = api_call(Addr, Port, Dir, list_groups, [], AuthPwd, State), + {reply, Reply, State}; + +%% List the members of the given group +handle_call({list_group_members, Addr, Port, Dir, Group, AuthPwd}, + _From, State) -> + Reply = api_call(Addr, Port, Dir, list_group_members, [Group], + AuthPwd, State), + {reply, Reply, State}; + + +%% Add password for a directory +handle_call({add_password, Dir, Password}, _From, State) -> + Reply = do_add_password(Dir, Password, State), + {reply, Reply, State}; + + +%% Update the password for a directory + +handle_call({update_password, Dir, Old, New},_From,State) -> + Reply = + case getPassword(State, Dir) of + OldPwd when is_binary(OldPwd) -> + case erlang:md5(Old) of + OldPwd -> + %% The old password is right => + %% update the password to the new + do_update_password(Dir,New,State), + ok; + _-> + {error, error_new} + end; + _-> + {error, error_old} + end, + {reply, Reply, State}; + +handle_call(stop, _From, State) -> + {stop, normal, State}. + +handle_info(_Info, State) -> + {noreply, State}. + +handle_cast(_Request, State) -> + {noreply, State}. + + +terminate(_Reason,State) -> + ets:delete(State#state.tab), + ok. + + +%% code_change(Vsn, State, Extra) +%% +code_change(_Vsn, State, _Extra) -> + {ok, State}. + + +%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%% +%% %% +%% The functions that really changes the data in the database %% +%% of users to different directories %% +%% %% +%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%% + +%% API gateway + +api_call(Addr, Port, Dir, Func, Args,Password,State) -> + case controlPassword(Password, State, Dir) of + ok-> + ConfigName = httpd_util:make_name("httpd_conf", Addr, Port), + case ets:match_object(ConfigName, {directory, {Dir, '$1'}}) of + [{directory, {Dir, DirData}}] -> + AuthMod = auth_mod_name(DirData), + (catch apply(AuthMod, Func, [DirData|Args])); + _ -> + {error, no_such_directory} + end; + bad_password -> + {error,bad_password} + end. + +controlPassword(Password, _State, _Dir) when Password =:= "DummyPassword" -> + bad_password; + +controlPassword(Password,State,Dir) -> + case getPassword(State,Dir) of + Pwd when is_binary(Pwd) -> + case erlang:md5(Password) of + Pwd -> + ok; + _-> + bad_password + end; + _ -> + bad_password + end. + + +getPassword(State, Dir) -> + case lookup(State#state.tab, Dir) of + [{_,Pwd}]-> + Pwd; + _ -> + {error,bad_password} + end. + +do_update_password(Dir, New, State) -> + ets:insert(State#state.tab, {Dir, erlang:md5(New)}). + +do_add_password(Dir, Password, State) -> + case getPassword(State,Dir) of + PwdExists when is_binary(PwdExists) -> + {error, dir_protected}; + {error, _} -> + do_update_password(Dir, Password, State) + end. + + +auth_mod_name(DirData) -> + case proplists:get_value(auth_type, DirData, plain) of + plain -> mod_auth_plain; + mnesia -> mod_auth_mnesia; + dets -> mod_auth_dets + end. + + +lookup(Db, Key) -> + ets:lookup(Db, Key). + + +make_name(Addr,Port) -> + httpd_util:make_name("httpd_auth",Addr,Port). + + +call(Name, Req) -> + case (catch gen_server:call(Name, Req)) of + {'EXIT', Reason} -> + {error, Reason}; + Reply -> + Reply + end. + + diff --git a/lib/inets/src/http_server/mod_browser.erl b/lib/inets/src/http_server/mod_browser.erl new file mode 100644 index 0000000000..1c9b33dffa --- /dev/null +++ b/lib/inets/src/http_server/mod_browser.erl @@ -0,0 +1,249 @@ +%% +%% %CopyrightBegin% +%% +%% Copyright Ericsson AB 2001-2009. All Rights Reserved. +%% +%% The contents of this file are subject to the Erlang Public License, +%% Version 1.1, (the "License"); you may not use this file except in +%% compliance with the License. You should have received a copy of the +%% Erlang Public License along with this software. If not, it can be +%% retrieved online at http://www.erlang.org/. +%% +%% Software distributed under the License is distributed on an "AS IS" +%% basis, WITHOUT WARRANTY OF ANY KIND, either express or implied. See +%% the License for the specific language governing rights and limitations +%% under the License. +%% +%% %CopyrightEnd% +%% +%% +%% ---------------------------------------------------------------------- +%% +%% Browsers sends a string to the webbserver +%% to identify themsevles. They are a bit nasty +%% since the only thing that the specification really +%% is strict about is that they shall be short +%% some axamples: +%% +%% Netscape Mozilla/4.75 [en] (X11; U; SunOS 5.8 sun4u) +%% Mozilla/5.0 (X11; U; Linux i686; en-US; rv:1.0.1) Gecko/20020823 Netscape/7.0 +%% Mozilla Mozilla/5.0 (X11; U; Linux i686; en-US; rv:1.1) Gecko/20020827 +%% Safari Mozilla/5.0 (Macintosh; U; PPC Mac OS X; en-us) AppleWebKit/85 (KHTML, like Gecko) Safari/85 +%% IE5 Mozilla/4.0 (compatible; MSIE 5.0; SP1B; SunOS 5.8 sun4u; X11) +%% Lynx Lynx/2.8.3rel.1 libwww-FM/2.142 +%% +%% ---------------------------------------------------------------------- + +-module(mod_browser). + +-export([do/1, test/0, getBrowser/1]). + +%% Remember that the order of the mozilla browsers are +%% important since some browsers include others to behave +%% as they were something else +-define(MOZILLA_BROWSERS,[{netscape, "netscape"}, + {opera, "opera"}, + {msie, "msie"}, + {safari, "safari"}, + {mozilla, "rv:"}]). % fallback, must be last + + +%% If your operatingsystem is not recognized add it to this list. +-define(OPERATIVE_SYSTEMS,[{win3x, ["win16", "windows 3", "windows 16-bit"]}, + {win95, ["win95", "windows 95"]}, + {win98, ["win98", "windows 98"]}, + {winnt, ["winnt", "windows nt"]}, + {win2k, ["nt 5"]}, + {sunos4, ["sunos 4"]}, + {sunos5, ["sunos 5"]}, + {sun, ["sunos"]}, + {aix, ["aix"]}, + {linux, ["linux"]}, + {sco, ["sco", "unix_sv"]}, + {freebsd,["freebsd"]}, + {bsd, ["bsd"]}, + {macosx, ["mac os x"]}]). + +-define(LYNX, lynx). +-define(MOZILLA, mozilla). +-define(EMACS, emacs). +-define(STAROFFICE, soffice). +-define(MOSAIC, mosaic). +-define(NETSCAPE, netscape). +-define(SAFARU, safari). +-define(UNKOWN, unknown). + +-include("httpd.hrl"). + +-define(VMODULE,"BROWSER"). + +do(Info) -> + case proplists:get_value(status, Info#mod.data) of + {_StatusCode, _PhraseArgs, _Reason} -> + {proceed,Info#mod.data}; + undefined -> + Browser = getBrowser1(Info), + {proceed,[{'user-agent', Browser}|Info#mod.data]} + end. + +getBrowser1(Info) -> + PHead = Info#mod.parsed_header, + case proplists:get_value("user-agent", PHead) of + undefined -> + undefined; + AgentString -> + getBrowser(AgentString) + end. + +getBrowser(AgentString) -> + LAgentString = http_util:to_lower(AgentString), + case inets_regexp:first_match(LAgentString,"^[^ ]*") of + {match,Start,Length} -> + Browser = lists:sublist(LAgentString,Start,Length), + case browserType(Browser) of + {mozilla,Vsn} -> + {getMozilla(LAgentString, + ?MOZILLA_BROWSERS,{?NETSCAPE,Vsn}), + operativeSystem(LAgentString)}; + AnyBrowser -> + {AnyBrowser,operativeSystem(LAgentString)} + end; + nomatch -> + browserType(LAgentString) + end. + +browserType([$l,$y,$n,$x|Version]) -> + {?LYNX,browserVersion(Version)}; +browserType([$m,$o,$z,$i,$l,$l,$a|Version]) -> + {?MOZILLA,browserVersion(Version)}; +browserType([$e,$m,$a,$c,$s|Version]) -> + {?EMACS,browserVersion(Version)}; +browserType([$s,$t,$a,$r,$o,$f,$f,$i,$c,$e|Version]) -> + {?STAROFFICE,browserVersion(Version)}; +browserType([$m,$o,$s,$a,$i,$c|Version]) -> + {?MOSAIC,browserVersion(Version)}; +browserType(_Unknown) -> + unknown. + + +browserVersion([$/|VsnString]) -> + case catch list_to_float(VsnString) of + Number when is_float(Number) -> + Number; + _Whatever -> + case string:span(VsnString,"1234567890.") of + 0 -> + unknown; + VLength -> + Vsn = string:substr(VsnString,1,VLength), + case string:tokens(Vsn,".") of + [Number] -> + list_to_float(Number++".0"); + [Major,Minor|_MinorMinor] -> + list_to_float(Major++"."++Minor) + end + end + end; +browserVersion(VsnString) -> + browserVersion([$/|VsnString]). + +operativeSystem(OpString) -> + operativeSystem(OpString, ?OPERATIVE_SYSTEMS). + +operativeSystem(_OpString,[]) -> + unknown; +operativeSystem(OpString,[{RetVal,RegExps}|Rest]) -> + case controlOperativeSystem(OpString,RegExps) of + true -> + RetVal; + _ -> + operativeSystem(OpString,Rest) + end. + +controlOperativeSystem(_OpString,[]) -> + false; +controlOperativeSystem(OpString,[Regexp|Regexps]) -> + case inets_regexp:match(OpString,Regexp) of + {match,_,_} -> + true; + nomatch -> + controlOperativeSystem(OpString,Regexps) + end. + + +%% OK this is ugly but thats the only way since +%% all browsers dont conform to the name/vsn standard +%% First we check if it is one of the browsers that +%% are not the default mozillaborwser against the regexp +%% for the different browsers. if no match, it is a mozilla +%% browser i.e opera, netscape, ie or safari + +getMozilla(_AgentString,[],Default) -> + Default; +getMozilla(AgentString,[{Agent,AgentRegExp}|Rest],Default) -> + case inets_regexp:match(AgentString,AgentRegExp) of + {match,_,_} -> + {Agent,getMozVersion(AgentString,AgentRegExp)}; + nomatch -> + getMozilla(AgentString,Rest,Default) + end. + +getMozVersion(AgentString, AgentRegExp) -> + case inets_regexp:match(AgentString,AgentRegExp++"[0-9\.\ \/]*") of + {match,Start,Length} when length(AgentRegExp) < Length -> + %% Ok we got the number split it out + RealStart = Start+length(AgentRegExp), + RealLength = Length-length(AgentRegExp), + VsnString = string:substr(AgentString,RealStart,RealLength), + %% case string:strip(VsnString,both,$\ ) of + case strip(VsnString) of + [] -> + unknown; + [Y1,Y2,Y3,Y4,M1,M2,D1,D2] = DateVsn when + Y1 =< $9, Y1 >= $0, + Y2 =< $9, Y2 >= $0, + Y3 =< $9, Y3 >= $0, + Y4 =< $9, Y4 >= $0, + M1 =< $9, M1 >= $0, + M2 =< $9, M2 >= $0, + D1 =< $9, D1 >= $0, + D2 =< $9, D2 >= $0 -> + list_to_integer(DateVsn); + Vsn -> + case string:tokens(Vsn,".") of + [Number]-> + list_to_float(Number++".0"); + [Major,Minor|Rev] -> + V = lists:flatten([Major,".",Minor,Rev]), + list_to_float(V) + end + end; + nomatch -> + unknown + end. + +strip(VsnString) -> + strip2(strip1(VsnString)). + +strip1(VsnString) -> + string:strip(VsnString,both,$\ ). + +strip2(VsnString) -> + string:strip(VsnString,both,$/ ). + +test()-> + test("Mozilla/4.75 [en] (X11; U; SunOS 5.8 sun4u)"), + test("Mozilla/5.0 (X11; U; Linux i686; en-US; rv:1.0.1) Gecko/20020823 Netscape/7.0"), + test("Mozilla/5.0 (X11; U; Linux i686; en-US; rv:1.1) Gecko/20020827"), + test("Mozilla/5.0 (Macintosh; U; PPC Mac OS X Mach-O; en-US; rv:1.4) Gecko/20020827"), + test("Mozilla/5.0 (Macintosh; U; PPC Mac OS X; en-us) AppleWebKit/85 (KHTML, like Gecko) Safari/85"), + test("Mozilla/4.0 (compatible; MSIE 5.0; SP1B; SunOS 5.8 sun4u; X11)"), + test("Lynx/2.8.3rel.1 libwww-FM/2.142"), + ok. + +test(Str) -> + Browser = getBrowser(Str), + io:format("~n--------------------------------------------------------~n"), + io:format("~p",[Browser]), + io:format("~n--------------------------------------------------------~n"). + diff --git a/lib/inets/src/http_server/mod_cgi.erl b/lib/inets/src/http_server/mod_cgi.erl new file mode 100644 index 0000000000..ab12a3b57b --- /dev/null +++ b/lib/inets/src/http_server/mod_cgi.erl @@ -0,0 +1,350 @@ +%% +%% %CopyrightBegin% +%% +%% Copyright Ericsson AB 1997-2009. All Rights Reserved. +%% +%% The contents of this file are subject to the Erlang Public License, +%% Version 1.1, (the "License"); you may not use this file except in +%% compliance with the License. You should have received a copy of the +%% Erlang Public License along with this software. If not, it can be +%% retrieved online at http://www.erlang.org/. +%% +%% Software distributed under the License is distributed on an "AS IS" +%% basis, WITHOUT WARRANTY OF ANY KIND, either express or implied. See +%% the License for the specific language governing rights and limitations +%% under the License. +%% +%% %CopyrightEnd% +%% +%% +%% Implements The WWW Common Gateway Interface Version 1.1 + +-module(mod_cgi). + +-export([env/3]). + +%%% Callback API +-export([do/1, load/2, store/2]). + +-include("http_internal.hrl"). +-include("httpd.hrl"). + +-define(VMODULE,"CGI"). + +-define(DEFAULT_CGI_TIMEOUT, 15000). + +%%%========================================================================= +%%% API +%%%========================================================================= +%%-------------------------------------------------------------------------- +%% do(ModData, _, AfterScript) -> [{EnvVariable, Value}] +%% +%% AfterScript = string() +%% ModData = #mod{} +%% EnvVariable = string() +%% Value = term() +%% Description: Keep for now as it is documented in the man page +%%------------------------------------------------------------------------- +env(ModData, _Script, AfterScript) -> + ScriptElements = script_elements(ModData, AfterScript), + httpd_script_env:create_env(cgi, ModData, ScriptElements). + +%%%========================================================================= +%%% Callback API +%%%========================================================================= + +%%-------------------------------------------------------------------------- +%% do(ModData) -> {proceed, OldData} | {proceed, NewData} | {break, NewData} +%% | done +%% ModData = #mod{} +%% +%% Description: See httpd(3) ESWAPI CALLBACK FUNCTIONS +%%------------------------------------------------------------------------- +do(ModData) -> + case proplists:get_value(status, ModData#mod.data) of + %% A status code has been generated! + {_StatusCode, _PhraseArgs, _Reason} -> + {proceed, ModData#mod.data}; + %% No status code has been generated! + undefined -> + case proplists:get_value(response, ModData#mod.data) of + undefined -> + generate_response(ModData); + _Response -> + {proceed, ModData#mod.data} + end + end. + +%%-------------------------------------------------------------------------- +%% load(Line, Context) -> eof | ok | {ok, NewContext} | +%% {ok, NewContext, Directive} | +%% {ok, NewContext, DirectiveList} | {error, Reason} +%% Line = string() +%% Context = NewContext = DirectiveList = [Directive] +%% Directive = {DirectiveKey , DirectiveValue} +%% DirectiveKey = DirectiveValue = term() +%% Reason = term() +%% +%% Description: See httpd(3) ESWAPI CALLBACK FUNCTIONS +%%------------------------------------------------------------------------- + +%% ScriptNoCache true|false, defines whether the server shall add +%% header fields to stop proxies and +%% clients from saving the page in history +%% or cache +%% +load("ScriptNoCache " ++ CacheArg, [])-> + case catch list_to_atom(httpd_conf:clean(CacheArg)) of + true -> + {ok, [], {script_nocache, true}}; + false -> + {ok, [], {script_nocache, false}}; + _ -> + {error, ?NICE(httpd_conf:clean(CacheArg)++ + " is an invalid ScriptNoCache directive")} + end; +%% ScriptTimeout Seconds, The number of seconds that the server +%% maximum will wait for the script to +%% generate a part of the document +load("ScriptTimeout " ++ Timeout, [])-> + case catch list_to_integer(httpd_conf:clean(Timeout)) of + TimeoutSec when is_integer(TimeoutSec) -> + {ok, [], {script_timeout,TimeoutSec*1000}}; + _ -> + {error, ?NICE(httpd_conf:clean(Timeout)++ + " is an invalid ScriptTimeout")} + end. + +%%-------------------------------------------------------------------------- +%% store(Directive, DirectiveList) -> {ok, NewDirective} | +%% {ok, [NewDirective]} | +%% {error, Reason} +%% Directive = {DirectiveKey , DirectiveValue} +%% DirectiveKey = DirectiveValue = term() +%% Reason = term() +%% +%% Description: See httpd(3) ESWAPI CALLBACK FUNCTIONS +%%------------------------------------------------------------------------- +store({script_nocache, Value} = Conf, _) + when Value == true; Value == false -> + {ok, Conf}; +store({script_nocache, Value}, _) -> + {error, {wrong_type, {script_nocache, Value}}}; +store({script_timeout, Value} = Conf, _) + when is_integer(Value), Value >= 0 -> + {ok, Conf}; +store({script_timeout, Value}, _) -> + {error, {wrong_type, {script_timeout, Value}}}. + +%%%======================================================================== +%%% Internal functions +%%%======================================================================== +generate_response(ModData) -> + RequestURI = + case proplists:get_value(new_request_uri, ModData#mod.data) of + undefined -> + ModData#mod.request_uri; + Value -> + Value + end, + ScriptAliases = + httpd_util:multi_lookup(ModData#mod.config_db, script_alias), + case mod_alias:real_script_name(ModData#mod.config_db, RequestURI, + ScriptAliases) of + {Script, AfterScript} -> + exec_script(ModData, Script, AfterScript, + RequestURI); + not_a_script -> + {proceed, ModData#mod.data} + end. + +is_executable(File) -> + Dir = filename:dirname(File), + FileName = filename:basename(File), + case os:type() of + {win32,_} -> + %% temporary (hopefully) fix for win32 OTP-3627 + is_win32_executable(Dir,FileName); + _ -> + is_executable(Dir, FileName) + end. + +is_executable(Dir, FilName) -> + case os:find_executable(FilName, Dir) of + false -> + false; + _ -> + true + end. + +%% Start temporary (hopefully) fix for win32 OTP-3627 +%% --------------------------------- +is_win32_executable(Dir, FileName) -> + NewFileName = strip_extention(FileName, [".bat",".exe",".com", ".cmd"]), + is_executable(Dir, NewFileName). + +strip_extention(FileName, []) -> + FileName; +strip_extention(FileName, [Extention | Extentions]) -> + case filename:basename(FileName, Extention) of + FileName -> + strip_extention(FileName, Extentions); + NewFileName -> + NewFileName + end. + +%% End fix +%% --------------------------------- + +exec_script(ModData, Script, AfterScript, RequestURI) -> + exec_script(is_executable(Script), ModData, Script, + AfterScript, RequestURI). + +exec_script(true, ModData, Script, AfterScript, _RequestURI) -> + process_flag(trap_exit,true), + Dir = filename:dirname(Script), + ScriptElements = script_elements(ModData, AfterScript), + Env = (catch httpd_script_env:create_env(cgi, ModData, ScriptElements)), + + %% Run script + Port = (catch open_port({spawn, Script},[binary, stream, + {cd, Dir}, {env, Env}])), + case Port of + Port when is_port(Port) -> + send_request_body_to_script(ModData, Port), + deliver_webpage(ModData, Port); % Take care of script output + Error -> + exit({open_port_failed, Error, + [{mod,?MODULE}, + {uri,ModData#mod.request_uri}, {script,Script}, + {env,Env},{dir,Dir}]}) + end; + +exec_script(false, ModData, _Script, _AfterScript, _RequestURI) -> + {proceed, + [{status, + {404,ModData#mod.request_uri, + ?NICE("You don't have permission to execute " ++ + ModData#mod.request_uri ++ " on this server")}}| + ModData#mod.data]}. + +send_request_body_to_script(ModData, Port) -> + case ModData#mod.entity_body of + [] -> + ok; + EntityBody -> + port_command(Port, EntityBody) + end. + +deliver_webpage(#mod{config_db = Db} = ModData, Port) -> + Timeout = cgi_timeout(Db), + case receive_headers(Port, httpd_cgi, parse_headers, + [<<>>, [], []], Timeout) of + {Headers, Body} -> + case httpd_cgi:handle_headers(Headers) of + {proceed, AbsPath} -> + {proceed, [{real_name, + httpd_util:split_path(AbsPath)} | + ModData#mod.data]}; + {ok, HTTPHeaders, Status} -> + IsDisableChunkedSend = + httpd_response:is_disable_chunked_send(Db), + case (ModData#mod.http_version =/= "HTTP/1.1") or + (IsDisableChunkedSend) of + true -> + send_headers(ModData, Status, + [{"connection", "close"} + | HTTPHeaders]); + false -> + send_headers(ModData, Status, + [{"transfer-encoding", + "chunked"} | HTTPHeaders]) + end, + handle_body(Port, ModData, Body, Timeout, size(Body), + IsDisableChunkedSend) + end; + {'EXIT', Port, Reason} -> + process_flag(trap_exit, false), + {proceed, [{status, {400, none, reason(Reason)}} | + ModData#mod.data]}; + timeout -> + (catch port_close(Port)), % KILL the port !!!! + send_headers(ModData, {504, "Timeout"}, []), + httpd_socket:close(ModData#mod.socket_type, ModData#mod.socket), + process_flag(trap_exit,false), + {proceed,[{response, {already_sent, 200, 0}} | ModData#mod.data]} + end. + +receive_headers(Port, Module, Function, Args, Timeout) -> + receive + {Port, {data, Response}} when is_port(Port) -> + case Module:Function([Response | Args]) of + {NewModule, NewFunction, NewArgs} -> + receive_headers(Port, NewModule, + NewFunction, NewArgs, Timeout); + {ok, {Headers, Body}} -> + {Headers, Body} + end; + {'EXIT', Port, Reason} when is_port(Port) -> + {'EXIT', Port, Reason}; + {'EXIT', Pid, Reason} when is_pid(Pid) -> + exit({linked_process_died, Pid, Reason}) + after Timeout -> + timeout + end. + +send_headers(ModData, {StatusCode, _}, HTTPHeaders) -> + ExtraHeaders = httpd_response:cache_headers(ModData), + httpd_response:send_header(ModData, StatusCode, + ExtraHeaders ++ HTTPHeaders). + +handle_body(Port, #mod{method = "HEAD"} = ModData, _, _, Size, _) -> + (catch port_close(Port)), % KILL the port !!!! + process_flag(trap_exit,false), + {proceed, [{response, {already_sent, 200, Size}} | ModData#mod.data]}; + +handle_body(Port, ModData, Body, Timeout, Size, IsDisableChunkedSend) -> + httpd_response:send_chunk(ModData, Body, IsDisableChunkedSend), + receive + {Port, {data, Data}} when is_port(Port) -> + handle_body(Port, ModData, Data, Timeout, Size + size(Data), + IsDisableChunkedSend); + {'EXIT', Port, normal} when is_port(Port) -> + httpd_response:send_final_chunk(ModData, IsDisableChunkedSend), + process_flag(trap_exit,false), + {proceed, [{response, {already_sent, 200, Size}} | + ModData#mod.data]}; + {'EXIT', Port, Reason} when is_port(Port) -> + process_flag(trap_exit, false), + {proceed, [{status, {400, none, reason(Reason)}} | + ModData#mod.data]}; + {'EXIT', Pid, Reason} when is_pid(Pid) -> + exit({mod_cgi_linked_process_died, Pid, Reason}) + after Timeout -> + (catch port_close(Port)), % KILL the port !!!! + process_flag(trap_exit,false), + {proceed,[{response, {already_sent, 200, Size}} | + ModData#mod.data]} + end. + +script_elements(#mod{method = "GET"}, {[], QueryString}) -> + [{query_string, QueryString}]; +script_elements(#mod{method = "GET"}, {PathInfo, []}) -> + [{path_info, PathInfo}]; +script_elements(#mod{method = "GET"}, {PathInfo, QueryString}) -> + [{query_string, QueryString}, {path_info, PathInfo}]; +script_elements(#mod{method = "POST", entity_body = Body}, _) -> + [{entity_body, Body}]; +script_elements(_, _) -> + []. + +cgi_timeout(Db) -> + httpd_util:lookup(Db, cgi_timeout, ?DEFAULT_CGI_TIMEOUT). + +%% Convert error to printable string +%% +reason({error,emfile}) -> ": To many open files"; +reason({error,{enfile,_}}) -> ": File/port table overflow"; +reason({error,enomem}) -> ": Not enough memory"; +reason({error,eagain}) -> ": No more available OS processes"; +reason(Reason) -> lists:flatten(io_lib:format("Reason: ~p~n", [Reason])). diff --git a/lib/inets/src/http_server/mod_dir.erl b/lib/inets/src/http_server/mod_dir.erl new file mode 100644 index 0000000000..cdc7cc01e4 --- /dev/null +++ b/lib/inets/src/http_server/mod_dir.erl @@ -0,0 +1,284 @@ +%% +%% %CopyrightBegin% +%% +%% Copyright Ericsson AB 1997-2009. All Rights Reserved. +%% +%% The contents of this file are subject to the Erlang Public License, +%% Version 1.1, (the "License"); you may not use this file except in +%% compliance with the License. You should have received a copy of the +%% Erlang Public License along with this software. If not, it can be +%% retrieved online at http://www.erlang.org/. +%% +%% Software distributed under the License is distributed on an "AS IS" +%% basis, WITHOUT WARRANTY OF ANY KIND, either express or implied. See +%% the License for the specific language governing rights and limitations +%% under the License. +%% +%% %CopyrightEnd% +%% +%% +-module(mod_dir). +-export([do/1]). + +-include("httpd.hrl"). + +%% do + +do(Info) -> + ?DEBUG("do -> entry",[]), + case Info#mod.method of + "GET" -> + case proplists:get_value(status, Info#mod.data) of + %% A status code has been generated! + {_StatusCode, _PhraseArgs, _Reason} -> + {proceed,Info#mod.data}; + %% No status code has been generated! + undefined -> + case proplists:get_value(response, Info#mod.data) of + %% No response has been generated! + undefined -> + do_dir(Info); + %% A response has been generated or sent! + _Response -> + {proceed,Info#mod.data} + end + end; + %% Not a GET method! + _ -> + {proceed,Info#mod.data} + end. + +do_dir(Info) -> + ?DEBUG("do_dir -> Request URI: ~p",[Info#mod.request_uri]), + Path = mod_alias:path(Info#mod.data,Info#mod.config_db, + Info#mod.request_uri), + DefaultPath = mod_alias:default_index(Info#mod.config_db,Path), + %% Is it a directory? + case file:read_file_info(DefaultPath) of + {ok,FileInfo} when FileInfo#file_info.type == directory -> + DecodedRequestURI = + httpd_util:decode_hex(Info#mod.request_uri), + ?DEBUG("do_dir -> ~n" + " Path: ~p~n" + " DefaultPath: ~p~n" + " DecodedRequestURI: ~p", + [Path,DefaultPath,DecodedRequestURI]), + case dir(DefaultPath,string:strip(DecodedRequestURI,right,$/), + Info#mod.config_db) of + {ok, Dir} -> + LastModified = + case (catch httpd_util:rfc1123_date( + FileInfo#file_info.mtime)) of + Date when is_list(Date) -> + [{"date", Date}]; + _ -> %% This will rarly happen, but could happen + %% if a computer is wrongly configured. + [] + end, + Head=[{content_type,"text/html"}, + {content_length, + integer_to_list(httpd_util:flatlength(Dir))}, + {code,200} | LastModified], + {proceed,[{response,{response, Head, Dir}}, + {mime_type,"text/html"} | Info#mod.data]}; + {error, Reason} -> + ?ERROR("do_dir -> dir operation failed: ~p",[Reason]), + {proceed, + [{status,{404,Info#mod.request_uri,Reason}}| + Info#mod.data]} + end; + {ok, _FileInfo} -> + ?DEBUG("do_dir -> ~n" + " Path: ~p~n" + " DefaultPath: ~p~n" + " FileInfo: ~p", + [Path,DefaultPath,FileInfo]), + {proceed,Info#mod.data}; + {error,Reason} -> + ?LOG("do_dir -> failed reading file info (~p) for: ~p", + [Reason,DefaultPath]), + Status = httpd_file:handle_error(Reason, "access", Info, + DefaultPath), + {proceed, [{status, Status}| Info#mod.data]} + end. + +dir(Path,RequestURI,ConfigDB) -> + case file:list_dir(Path) of + {ok,FileList} -> + SortedFileList=lists:sort(FileList), + {ok,[header(Path,RequestURI), + body(Path,RequestURI,ConfigDB,SortedFileList), + footer(Path,SortedFileList)]}; + {error,Reason} -> + {error,?NICE("Can't open directory "++Path++": "++ + file:format_error(Reason))} + end. + +%% header + +header(Path,RequestURI) -> + Header = "<HTML>\n<HEAD>\n<TITLE>Index of "++ RequestURI ++ + "</TITLE>\n</HEAD>\n<BODY>\n<H1>Index of "++ + RequestURI ++ "</H1>\n<PRE><IMG SRC=\"" ++ icon(blank) ++ + "\" ALT=" "> Name Last modified " + "Size Description <HR>\n", + case inets_regexp:sub(RequestURI,"[^/]*\$","") of + {ok,"/",_} -> + Header; + {ok,ParentRequestURI,_} -> + {ok,ParentPath,_} = + inets_regexp:sub(string:strip(Path,right,$/),"[^/]*\$",""), + Header++format(ParentPath,ParentRequestURI) + end. + +format(Path,RequestURI) -> + {ok,FileInfo}=file:read_file_info(Path), + {{Year, Month, Day},{Hour, Minute, _Second}} = FileInfo#file_info.mtime, + io_lib:format("<IMG SRC=\"~s\" ALT=\"[~s]\">" + " <A HREF=\"~s\">Parent directory</A> " + " ~2.2.0w-~s-~w ~2.2.0w:~2.2.0w -\n", + [icon(back),"DIR",RequestURI,Day, + httpd_util:month(Month),Year,Hour,Minute]). + +%% body + +body(_Path, _RequestURI, _ConfigDB, []) -> + []; +body(Path, RequestURI, ConfigDB, [Entry | Rest]) -> + [format(Path, RequestURI, ConfigDB, Entry)| + body(Path, RequestURI, ConfigDB, Rest)]. + +format(Path,RequestURI,ConfigDB,Entry) -> + case file:read_file_info(Path++"/"++Entry) of + {ok,FileInfo} when FileInfo#file_info.type == directory -> + {{Year, Month, Day},{Hour, Minute, _Second}} = + FileInfo#file_info.mtime, + EntryLength=length(Entry), + if + EntryLength > 21 -> + io_lib:format("<IMG SRC=\"~s\" ALT=\"[~s]\"> " + "<A HREF=\"~s\">~-21.s..</A>" + "~2.2.0w-~s-~w ~2.2.0w:~2.2.0w" + " -\n", [icon(folder),"DIR", + RequestURI++"/"++Entry++"/", + Entry, + Day, httpd_util:month(Month), + Year,Hour,Minute]); + true -> + io_lib:format("<IMG SRC=\"~s\" ALT=\"[~s]\">" + " <A HREF=\"~s\">~s</A>~*.*c~2.2.0" + "w-~s-~w ~2.2.0w:~2.2.0w -\n", + [icon(folder),"DIR",RequestURI ++ "/" ++ + Entry ++ "/",Entry, + 23-EntryLength,23-EntryLength,$ ,Day, + httpd_util:month(Month),Year,Hour,Minute]) + end; + {ok,FileInfo} -> + {{Year, Month, Day},{Hour, Minute,_Second}} = + FileInfo#file_info.mtime, + Suffix=httpd_util:suffix(Entry), + MimeType=httpd_util:lookup_mime(ConfigDB,Suffix,""), + EntryLength=length(Entry), + if + EntryLength > 21 -> + io_lib:format("<IMG SRC=\"~s\" ALT=\"[~s]\">" + " <A HREF=\"~s\">~-21.s..</A>~2.2.0" + "w-~s-~w ~2.2.0w:~2.2.0w~8wk ~s\n", + [icon(Suffix, MimeType), Suffix, RequestURI + ++"/"++Entry, Entry,Day, + httpd_util:month(Month),Year,Hour,Minute, + trunc(FileInfo#file_info.size/1024+1), + MimeType]); + true -> + io_lib:format("<IMG SRC=\"~s\" ALT=\"[~s]\"> " + "<A HREF=\"~s\">~s</A>~*.*c~2.2.0w-~s-~w" + " ~2.2.0w:~2.2.0w~8wk ~s\n", + [icon(Suffix, MimeType), Suffix, RequestURI + ++ "/" ++ Entry, Entry, 23-EntryLength, + 23-EntryLength, $ ,Day, + httpd_util:month(Month),Year,Hour,Minute, + trunc(FileInfo#file_info.size/1024+1), + MimeType]) + end; + {error, _Reason} -> + "" + end. + +%% footer + +footer(Path,FileList) -> + case lists:member("README",FileList) of + true -> + {ok,Body}=file:read_file(Path++"/README"), + "</PRE>\n<HR>\n<PRE>\n"++binary_to_list(Body)++ + "\n</PRE>\n</BODY>\n</HTML>\n"; + false -> + "</PRE>\n</BODY>\n</HTML>\n" + end. + +%% +%% Icon mappings are hard-wired ala default Apache (Ugly!) +%% + +icon(Suffix,MimeType) -> + case icon(Suffix) of + undefined -> + case MimeType of + [$t,$e,$x,$t,$/|_] -> + "/icons/text.gif"; + [$i,$m,$a,$g,$e,$/|_] -> + "/icons/image2.gif"; + [$a,$u,$d,$i,$o,$/|_] -> + "/icons/sound2.gif"; + [$v,$i,$d,$e,$o,$/|_] -> + "/icons/movie.gif"; + _ -> + "/icons/unknown.gif" + end; + Icon -> + Icon + end. + +icon(blank) -> "/icons/blank.gif"; +icon(back) -> "/icons/back.gif"; +icon(folder) -> "/icons/folder.gif"; +icon("bin") -> "/icons/binary.gif"; +icon("exe") -> "/icons/binary.gif"; +icon("hqx") -> "/icons/binhex.gif"; +icon("tar") -> "/icons/tar.gif"; +icon("wrl") -> "/icons/world2.gif"; +icon("wrl.gz") -> "/icons/world2.gif"; +icon("vrml") -> "/icons/world2.gif"; +icon("vrm") -> "/icons/world2.gif"; +icon("iv") -> "/icons/world2.gif"; +icon("Z") -> "/icons/compressed.gif"; +icon("z") -> "/icons/compressed.gif"; +icon("tgz") -> "/icons/compressed.gif"; +icon("gz") -> "/icons/compressed.gif"; +icon("zip") -> "/icons/compressed.gif"; +icon("ps") -> "/icons/a.gif"; +icon("ai") -> "/icons/a.gif"; +icon("eps") -> "/icons/a.gif"; +icon("html") -> "/icons/layout.gif"; +icon("shtml") -> "/icons/layout.gif"; +icon("htm") -> "/icons/layout.gif"; +icon("pdf") -> "/icons/layout.gif"; +icon("txt") -> "/icons/text.gif"; +icon("erl") -> "/icons/burst.gif"; +icon("c") -> "/icons/c.gif"; +icon("pl") -> "/icons/p.gif"; +icon("py") -> "/icons/p.gif"; +icon("for") -> "/icons/f.gif"; +icon("dvi") -> "/icons/dvi.gif"; +icon("uu") -> "/icons/uuencoded.gif"; +icon("conf") -> "/icons/script.gif"; +icon("sh") -> "/icons/script.gif"; +icon("shar") -> "/icons/script.gif"; +icon("csh") -> "/icons/script.gif"; +icon("ksh") -> "/icons/script.gif"; +icon("tcl") -> "/icons/script.gif"; +icon("tex") -> "/icons/tex.gif"; +icon("core") -> "/icons/tex.gif"; +icon(_) -> undefined. + + diff --git a/lib/inets/src/http_server/mod_disk_log.erl b/lib/inets/src/http_server/mod_disk_log.erl new file mode 100644 index 0000000000..95e0d00c70 --- /dev/null +++ b/lib/inets/src/http_server/mod_disk_log.erl @@ -0,0 +1,415 @@ +%% +%% %CopyrightBegin% +%% +%% Copyright Ericsson AB 1997-2009. All Rights Reserved. +%% +%% The contents of this file are subject to the Erlang Public License, +%% Version 1.1, (the "License"); you may not use this file except in +%% compliance with the License. You should have received a copy of the +%% Erlang Public License along with this software. If not, it can be +%% retrieved online at http://www.erlang.org/. +%% +%% Software distributed under the License is distributed on an "AS IS" +%% basis, WITHOUT WARRANTY OF ANY KIND, either express or implied. See +%% the License for the specific language governing rights and limitations +%% under the License. +%% +%% %CopyrightEnd% +%% +%% +-module(mod_disk_log). + +%% Application internal API +-export([error_log/2, report_error/2, security_log/2]). + +%% Callback API +-export([do/1, load/2, store/2, remove/1]). + +-define(VMODULE,"DISK_LOG"). + +-include("httpd.hrl"). + + +%%%========================================================================= +%%% API +%%%========================================================================= + +%% security_log +security_log(#mod{config_db = ConfigDb} = Info, Event) -> + Format = get_log_format(ConfigDb), + Date = httpd_util:custom_date(), + case httpd_log:security_entry(security_disk_log, no_security_log, + Info, Date, Event) of + no_security_log -> + ok; + {Log, Entry} -> + write(Log, Entry, Format) + end. + +report_error(ConfigDB, Error) -> + Format = get_log_format(ConfigDB), + Date = httpd_util:custom_date(), + case httpd_log:error_report_entry(error_disk_log, no_error_log, ConfigDB, + Date, Error) of + no_error_log -> + ok; + {Log, Entry} -> + write(Log, Entry, Format) + end. + +error_log(Info, Reason) -> + Date = httpd_util:custom_date(), + error_log(Info, Date, Reason). + +error_log(#mod{config_db = ConfigDB} = Info, Date, Reason) -> + Format = get_log_format(ConfigDB), + case httpd_log:error_entry(error_disk_log, no_error_log, + Info, Date, Reason) of + no_error_log -> + ok; + {Log, Entry} -> + write(Log, Entry, Format) + end. + +%%%========================================================================= +%%% CALLBACK API +%%%========================================================================= +%%-------------------------------------------------------------------------- +%% do(ModData) -> {proceed, OldData} | {proceed, NewData} | {break, NewData} +%% | done +%% ModData = #mod{} +%% +%% Description: See httpd(3) ESWAPI CALLBACK FUNCTIONS +%%------------------------------------------------------------------------- +do(Info) -> + AuthUser = auth_user(Info#mod.data), + Date = httpd_util:custom_date(), + log_internal_info(Info,Date,Info#mod.data), + LogFormat = get_log_format(Info#mod.config_db), + case proplists:get_value(status, Info#mod.data) of + %% A status code has been generated! + {StatusCode, _PhraseArgs, Reason} -> + transfer_log(Info, "-", AuthUser, Date, StatusCode, 0, + LogFormat), + if + StatusCode >= 400 -> + error_log(Info, Date, Reason); + true -> + not_an_error + end, + {proceed,Info#mod.data}; + %% No status code has been generated! + undefined -> + case proplists:get_value(response, Info#mod.data) of + {already_sent,StatusCode,Size} -> + transfer_log(Info, "-", AuthUser, Date, StatusCode, + Size, LogFormat), + {proceed,Info#mod.data}; + + {response, Head, _Body} -> + Size = proplists:get_value(content_length, Head, 0), + Code = proplists:get_value(code, Head, 200), + transfer_log(Info, "-", AuthUser, Date, Code, + Size, LogFormat), + {proceed,Info#mod.data}; + + {_StatusCode, Response} -> + transfer_log(Info, "-", AuthUser, Date, 200, + httpd_util:flatlength(Response), LogFormat), + {proceed,Info#mod.data}; + undefined -> + transfer_log(Info, "-", AuthUser, Date, 200, + 0, LogFormat), + {proceed,Info#mod.data} + end + end. + +%%-------------------------------------------------------------------------- +%% load(Line, Context) -> eof | ok | {ok, NewContext} | +%% {ok, NewContext, Directive} | +%% {ok, NewContext, DirectiveList} | {error, Reason} +%% Line = string() +%% Context = NewContext = DirectiveList = [Directive] +%% Directive = {DirectiveKey , DirectiveValue} +%% DirectiveKey = DirectiveValue = term() +%% Reason = term() +%% +%% Description: See httpd(3) ESWAPI CALLBACK FUNCTIONS +%%------------------------------------------------------------------------- +load("TransferDiskLogSize " ++ TransferDiskLogSize, []) -> + case inets_regexp:split(TransferDiskLogSize," ") of + {ok,[MaxBytes,MaxFiles]} -> + case httpd_conf:make_integer(MaxBytes) of + {ok,MaxBytesInteger} -> + case httpd_conf:make_integer(MaxFiles) of + {ok,MaxFilesInteger} -> + {ok,[],{transfer_disk_log_size, + {MaxBytesInteger,MaxFilesInteger}}}; + {error,_} -> + {error, + ?NICE(httpd_conf:clean(TransferDiskLogSize)++ + " is an invalid TransferDiskLogSize")} + end; + {error,_} -> + {error,?NICE(httpd_conf:clean(TransferDiskLogSize)++ + " is an invalid TransferDiskLogSize")} + end + end; +load("TransferDiskLog " ++ TransferDiskLog,[]) -> + {ok,[],{transfer_disk_log,httpd_conf:clean(TransferDiskLog)}}; + +load("ErrorDiskLogSize " ++ ErrorDiskLogSize, []) -> + case inets_regexp:split(ErrorDiskLogSize," ") of + {ok,[MaxBytes,MaxFiles]} -> + case httpd_conf:make_integer(MaxBytes) of + {ok,MaxBytesInteger} -> + case httpd_conf:make_integer(MaxFiles) of + {ok,MaxFilesInteger} -> + {ok,[],{error_disk_log_size, + {MaxBytesInteger,MaxFilesInteger}}}; + {error,_} -> + {error,?NICE(httpd_conf:clean(ErrorDiskLogSize)++ + " is an invalid ErrorDiskLogSize")} + end; + {error,_} -> + {error,?NICE(httpd_conf:clean(ErrorDiskLogSize)++ + " is an invalid ErrorDiskLogSize")} + end + end; +load("ErrorDiskLog " ++ ErrorDiskLog, []) -> + {ok, [], {error_disk_log, httpd_conf:clean(ErrorDiskLog)}}; + +load("SecurityDiskLogSize " ++ SecurityDiskLogSize, []) -> + case inets_regexp:split(SecurityDiskLogSize, " ") of + {ok, [MaxBytes, MaxFiles]} -> + case httpd_conf:make_integer(MaxBytes) of + {ok, MaxBytesInteger} -> + case httpd_conf:make_integer(MaxFiles) of + {ok, MaxFilesInteger} -> + {ok, [], {security_disk_log_size, + {MaxBytesInteger, MaxFilesInteger}}}; + {error,_} -> + {error, + ?NICE(httpd_conf:clean(SecurityDiskLogSize) ++ + " is an invalid SecurityDiskLogSize")} + end; + {error, _} -> + {error, ?NICE(httpd_conf:clean(SecurityDiskLogSize) ++ + " is an invalid SecurityDiskLogSize")} + end + end; +load("SecurityDiskLog " ++ SecurityDiskLog, []) -> + {ok, [], {security_disk_log, httpd_conf:clean(SecurityDiskLog)}}; + +load("DiskLogFormat " ++ Format, []) -> + case httpd_conf:clean(Format) of + "internal" -> + {ok, [], {disk_log_format,internal}}; + "external" -> + {ok, [], {disk_log_format,external}}; + _Default -> + {ok, [], {disk_log_format,external}} + end. + +%%-------------------------------------------------------------------------- +%% store(Directive, DirectiveList) -> {ok, NewDirective} | +%% {ok, [NewDirective]} | +%% {error, Reason} +%% Directive = {DirectiveKey , DirectiveValue} +%% DirectiveKey = DirectiveValue = term() +%% Reason = term() +%% +%% Description: See httpd(3) ESWAPI CALLBACK FUNCTIONS +%%------------------------------------------------------------------------- +store({transfer_disk_log,TransferDiskLog}, ConfigList) + when is_list(TransferDiskLog) -> + case create_disk_log(TransferDiskLog, + transfer_disk_log_size, ConfigList) of + {ok,TransferDB} -> + {ok,{transfer_disk_log,TransferDB}}; + {error,Reason} -> + {error,Reason} + end; +store({transfer_disk_log,TransferLog}, _) -> + {error, {wrong_type, {transfer_disk_log, TransferLog}}}; +store({security_disk_log,SecurityDiskLog},ConfigList) + when is_list(SecurityDiskLog) -> + case create_disk_log(SecurityDiskLog, + security_disk_log_size, ConfigList) of + {ok,SecurityDB} -> + {ok,{security_disk_log,SecurityDB}}; + {error,Reason} -> + {error,Reason} + end; +store({security_disk_log, SecurityLog}, _) -> + {error, {wrong_type, {security_disk_log, SecurityLog}}}; + +store({error_disk_log,ErrorDiskLog},ConfigList) when is_list(ErrorDiskLog) -> + case create_disk_log(ErrorDiskLog, error_disk_log_size, ConfigList) of + {ok,ErrorDB} -> + {ok,{error_disk_log,ErrorDB}}; + {error,Reason} -> + {error,Reason} + end; +store({error_disk_log,ErrorLog}, _) -> + {error, {wrong_type, {error_disk_log, ErrorLog}}}; +store({transfer_disk_log_size, {ByteInt, FileInt}} = Conf, _) + when is_integer(ByteInt), is_integer(FileInt)-> + {ok, Conf}; +store({transfer_disk_log_size, Value}, _) -> + {error, {wrong_type, {transfer_disk_log_size, Value}}}; +store({error_disk_log_size, {ByteInt, FileInt}} = Conf, _) + when is_integer(ByteInt), is_integer(FileInt)-> + {ok, Conf}; +store({error_disk_log_size, Value}, _) -> + {error, {wrong_type, {error_disk_log_size, Value}}}; +store({security_disk_log_size, {ByteInt, FileInt}} = Conf, _) + when is_integer(ByteInt), is_integer(FileInt)-> + {ok, Conf}; +store({security_disk_log_size, Value}, _) -> + {error, {wrong_type, {security_disk_log_size, Value}}}; +store({disk_log_format, Value} = Conf, _) when Value == internal; + Value == external -> + {ok, Conf}; +store({disk_log_format, Value}, _) -> + {error, {wrong_type, {disk_log_format, Value}}}. + +%%-------------------------------------------------------------------------- +%% remove(ConfigDb) -> _ +%% +%% Description: See httpd(3) ESWAPI CALLBACK FUNCTIONS +%%------------------------------------------------------------------------- +remove(ConfigDB) -> + lists:foreach(fun([DiskLog]) -> close(DiskLog) end, + ets:match(ConfigDB,{transfer_disk_log,'$1'})), + lists:foreach(fun([DiskLog]) -> close(DiskLog) end, + ets:match(ConfigDB,{error_disk_log,'$1'})), + lists:foreach(fun([DiskLog]) -> close(DiskLog) end, + ets:match(ConfigDB,{security_disk_log,'$1'})), + ok. + +%%%======================================================================== +%%% Internal functions +%%%======================================================================== + +%% transfer_log +transfer_log(Info, RFC931, AuthUser, Date, StatusCode, Bytes, Format) -> + case httpd_log:access_entry(transfer_disk_log, no_transfer_log, + Info, RFC931, AuthUser, Date, StatusCode, + Bytes) of + no_transfer_log -> + ok; + {Log, Entry} -> + write(Log, Entry, Format) + end. + + +get_log_format(ConfigDB)-> + httpd_util:lookup(ConfigDB,disk_log_format,external). + +%%---------------------------------------------------------------------- +%% Open or creates the disklogs +%%---------------------------------------------------------------------- +log_size(ConfigList, Tag) -> + proplists:get_value(Tag, ConfigList, {500*1024,8}). + +create_disk_log(LogFile, SizeTag, ConfigList) -> + Filename = httpd_conf:clean(LogFile), + {MaxBytes, MaxFiles} = log_size(ConfigList, SizeTag), + case filename:pathtype(Filename) of + absolute -> + create_disk_log(Filename, MaxBytes, MaxFiles, ConfigList); + volumerelative -> + create_disk_log(Filename, MaxBytes, MaxFiles, ConfigList); + relative -> + case proplists:get_value(server_root,ConfigList) of + undefined -> + {error, + ?NICE(Filename++ + " is an invalid ErrorLog beacuse ServerRoot " + "is not defined")}; + ServerRoot -> + AbsoluteFilename = filename:join(ServerRoot,Filename), + create_disk_log(AbsoluteFilename, MaxBytes, MaxFiles, + ConfigList) + end + end. + +create_disk_log(Filename, MaxBytes, MaxFiles, ConfigList) -> + Format = proplists:get_value(disk_log_format, ConfigList, external), + open(Filename, MaxBytes, MaxFiles, Format). + + +%%---------------------------------------------------------------------- +%% Function: open/4 +%% Description: Open a disk log file. +%% Control which format the disk log will be in. The external file +%% format is used as default since that format was used by older +%% implementations of inets. +%% +%% When the internal disk log format is used, we will do some extra +%% controls. If the files are valid, try to repair them and if +%% thats not possible, truncate. +%%---------------------------------------------------------------------- + +open(Filename, MaxBytes, MaxFiles, internal) -> + Opts = [{format, internal}, {repair, truncate}], + open1(Filename, MaxBytes, MaxFiles, Opts); +open(Filename, MaxBytes, MaxFiles, _) -> + Opts = [{format, external}], + open1(Filename, MaxBytes, MaxFiles, Opts). + +open1(Filename, MaxBytes, MaxFiles, Opts0) -> + Opts1 = [{name, Filename}, {file, Filename}, {type, wrap}] ++ Opts0, + case open2(Opts1, {MaxBytes, MaxFiles}) of + {ok, LogDB} -> + {ok, LogDB}; + {error, Reason} -> + {error, + ?NICE("Can't create " ++ Filename ++ + lists:flatten(io_lib:format(", ~p",[Reason])))}; + _ -> + {error, ?NICE("Can't create "++Filename)} + end. + +open2(Opts, Size) -> + case disk_log:open(Opts) of + {error, {badarg, size}} -> + %% File did not exist, add the size option and try again + disk_log:open([{size, Size} | Opts]); + Else -> + Else + end. + + +%%---------------------------------------------------------------------- +%% Actually writes the entry to the disk_log. If the log is an +%% internal disk_log write it with log otherwise with blog. +%%---------------------------------------------------------------------- +write(Log, Entry, internal) -> + disk_log:log(Log, list_to_binary(Entry)); + +write(Log, Entry, _) -> + disk_log:blog(Log, list_to_binary(Entry)). + +%% Close the log file +close(Log) -> + disk_log:close(Log). + +auth_user(Data) -> + case proplists:get_value(remote_user,Data) of + undefined -> + "-"; + RemoteUser -> + RemoteUser + end. +%% log_internal_info + +log_internal_info(_, _,[]) -> + ok; +log_internal_info(Info,Date,[{internal_info,Reason}|Rest]) -> + error_log(Info,Date,Reason), + log_internal_info(Info,Date,Rest); +log_internal_info(Info,Date,[_|Rest]) -> + log_internal_info(Info,Date,Rest). + diff --git a/lib/inets/src/http_server/mod_esi.erl b/lib/inets/src/http_server/mod_esi.erl new file mode 100644 index 0000000000..dd6f62ae2d --- /dev/null +++ b/lib/inets/src/http_server/mod_esi.erl @@ -0,0 +1,492 @@ +%% +%% %CopyrightBegin% +%% +%% Copyright Ericsson AB 1997-2009. All Rights Reserved. +%% +%% The contents of this file are subject to the Erlang Public License, +%% Version 1.1, (the "License"); you may not use this file except in +%% compliance with the License. You should have received a copy of the +%% Erlang Public License along with this software. If not, it can be +%% retrieved online at http://www.erlang.org/. +%% +%% Software distributed under the License is distributed on an "AS IS" +%% basis, WITHOUT WARRANTY OF ANY KIND, either express or implied. See +%% the License for the specific language governing rights and limitations +%% under the License. +%% +%% %CopyrightEnd% +%% +%% +-module(mod_esi). + +%% API +%% Functions provided to help erl scheme alias programmer to +%% Create dynamic webpages that are sent back to the user during +%% Generation +-export([deliver/2]). + +%% Callback API +-export([do/1, load/2, store/2]). + +-include("httpd.hrl"). + +-define(VMODULE,"ESI"). +-define(DEFAULT_ERL_TIMEOUT,15000). + +%%%========================================================================= +%%% API +%%%========================================================================= +%%-------------------------------------------------------------------------- +%% deliver(SessionID, Data) -> ok | {error, bad_sessionID} +%% SessionID = pid() +%% Data = string() | io_list() (first call must send a string that +%% contains all header information including "\r\n\r\n", unless there +%% is no header information at all.) +%% +%% Description: Send <Data> (Html page generated sofar) to the server +%% request handling process so it can forward it to the client. +%%------------------------------------------------------------------------- +deliver(SessionID, Data) when is_pid(SessionID) -> + SessionID ! {ok, Data}, + ok; +deliver(_SessionID, _Data) -> + {error, bad_sessionID}. + +%%%========================================================================= +%%% CALLBACK API +%%%========================================================================= +%%-------------------------------------------------------------------------- +%% do(ModData) -> {proceed, OldData} | {proceed, NewData} | {break, NewData} +%% | done +%% ModData = #mod{} +%% +%% Description: See httpd(3) ESWAPI CALLBACK FUNCTIONS +%%------------------------------------------------------------------------- +do(ModData) -> + case proplists:get_value(status, ModData#mod.data) of + {_StatusCode, _PhraseArgs, _Reason} -> + {proceed, ModData#mod.data}; + undefined -> + case proplists:get_value(response, ModData#mod.data) of + undefined -> + generate_response(ModData); + _Response -> + {proceed, ModData#mod.data} + end + end. +%%-------------------------------------------------------------------------- +%% load(Line, Context) -> eof | ok | {ok, NewContext} | +%% {ok, NewContext, Directive} | +%% {ok, NewContext, DirectiveList} | {error, Reason} +%% Line = string() +%% Context = NewContext = DirectiveList = [Directive] +%% Directive = {DirectiveKey , DirectiveValue} +%% DirectiveKey = DirectiveValue = term() +%% Reason = term() +%% +%% Description: See httpd(3) ESWAPI CALLBACK FUNCTIONS +%%------------------------------------------------------------------------- +load("ErlScriptAlias " ++ ErlScriptAlias, []) -> + case inets_regexp:split(ErlScriptAlias," ") of + {ok, [ErlName | StrModules]} -> + Modules = lists:map(fun(Str) -> + list_to_atom(httpd_conf:clean(Str)) + end, StrModules), + {ok, [], {erl_script_alias, {ErlName, Modules}}}; + {ok, _} -> + {error, ?NICE(httpd_conf:clean(ErlScriptAlias) ++ + " is an invalid ErlScriptAlias")} + end; +load("EvalScriptAlias " ++ EvalScriptAlias, []) -> + case inets_regexp:split(EvalScriptAlias, " ") of + {ok, [EvalName | StrModules]} -> + Modules = lists:map(fun(Str) -> + list_to_atom(httpd_conf:clean(Str)) + end, StrModules), + {ok, [], {eval_script_alias, {EvalName, Modules}}}; + {ok, _} -> + {error, ?NICE(httpd_conf:clean(EvalScriptAlias) ++ + " is an invalid EvalScriptAlias")} + end; +load("ErlScriptTimeout " ++ Timeout, [])-> + case catch list_to_integer(httpd_conf:clean(Timeout)) of + TimeoutSec when is_integer(TimeoutSec) -> + {ok, [], {erl_script_timeout, TimeoutSec * 1000}}; + _ -> + {error, ?NICE(httpd_conf:clean(Timeout) ++ + " is an invalid ErlScriptTimeout")} + end; +load("ErlScriptNoCache " ++ CacheArg, [])-> + case catch list_to_atom(httpd_conf:clean(CacheArg)) of + true -> + {ok, [], {erl_script_nocache, true}}; + false -> + {ok, [], {erl_script_nocache, false}}; + _ -> + {error, ?NICE(httpd_conf:clean(CacheArg)++ + " is an invalid ErlScriptNoCache directive")} + end. + +%%-------------------------------------------------------------------------- +%% store(Directive, DirectiveList) -> {ok, NewDirective} | +%% {ok, [NewDirective]} | +%% {error, Reason} +%% Directive = {DirectiveKey , DirectiveValue} +%% DirectiveKey = DirectiveValue = term() +%% Reason = term() +%% +%% Description: See httpd(3) ESWAPI CALLBACK FUNCTIONS +%%------------------------------------------------------------------------- +store({erl_script_alias, {Name, [all]}} = Conf, _) + when is_list(Name) -> + {ok, Conf}; + +store({erl_script_alias, {Name, Modules}} = Conf, _) + when is_list(Name) -> + try httpd_util:modules_validate(Modules) of + ok -> + {ok, Conf} + catch + throw:Error -> + {error, {wrong_type, {erl_script_alias, Error}}} + end; + +store({eval_script_alias, {Name, Modules}} = Conf, _) + when is_list(Name)-> + try httpd_util:modules_validate(Modules) of + ok -> + {ok, Conf} + catch + throw:Error -> + {error, {wrong_type, {eval_script_alias, Error}}} + end; + +store({erl_script_alias, Value}, _) -> + {error, {wrong_type, {erl_script_alias, Value}}}; +store({erl_script_timeout, Value} = Conf, _) + when is_integer(Value), Value >= 0 -> + {ok, Conf}; +store({erl_script_timeout, Value}, _) -> + {error, {wrong_type, {erl_script_timeout, Value}}}; +store({erl_script_nocache, Value} = Conf, _) when Value == true; + Value == false -> + {ok, Conf}; +store({erl_script_nocache, Value}, _) -> + {error, {wrong_type, {erl_script_nocache, Value}}}. +%%%======================================================================== +%%% Internal functions +%%%======================================================================== +generate_response(ModData) -> + case scheme(ModData#mod.request_uri, ModData#mod.config_db) of + {eval, ESIBody, Modules} -> + eval(ModData, ESIBody, Modules); + {erl, ESIBody, Modules} -> + erl(ModData, ESIBody, Modules); + no_scheme -> + {proceed, ModData#mod.data} + end. + +scheme(RequestURI, ConfigDB) -> + case match_script(RequestURI, ConfigDB, erl_script_alias) of + no_match -> + case match_script(RequestURI, ConfigDB, eval_script_alias) of + no_match -> + no_scheme; + {EsiBody, ScriptModules} -> + {eval, EsiBody, ScriptModules} + end; + {EsiBody, ScriptModules} -> + {erl, EsiBody, ScriptModules} + end. + +match_script(RequestURI, ConfigDB, AliasType) -> + case httpd_util:multi_lookup(ConfigDB, AliasType) of + [] -> + no_match; + AliasAndMods -> + match_esi_script(RequestURI, AliasAndMods, AliasType) + end. + +match_esi_script(_, [], _) -> + no_match; +match_esi_script(RequestURI, [{Alias,Modules} | Rest], AliasType) -> + AliasMatchStr = alias_match_str(Alias, AliasType), + case inets_regexp:first_match(RequestURI, AliasMatchStr) of + {match, 1, Length} -> + {string:substr(RequestURI, Length + 1), Modules}; + nomatch -> + match_esi_script(RequestURI, Rest, AliasType) + end. + +alias_match_str(Alias, erl_script_alias) -> + "^" ++ Alias ++ "/"; +alias_match_str(Alias, eval_script_alias) -> + "^" ++ Alias ++ "\\?". + + +%%------------------------ Erl mechanism -------------------------------- + +erl(#mod{method = Method} = ModData, ESIBody, Modules) + when Method == "GET"; Method == "HEAD"-> + case httpd_util:split(ESIBody,":|%3A|/",2) of + {ok, [ModuleName, FuncAndInput]} -> + case httpd_util:split(FuncAndInput,"[\?/]",2) of + {ok, [FunctionName, Input]} -> + generate_webpage(ModData, ESIBody, Modules, + list_to_atom(ModuleName), + FunctionName, Input, + script_elements(FuncAndInput, Input)); + {ok, [FunctionName]} -> + generate_webpage(ModData, ESIBody, Modules, + list_to_atom(ModuleName), + FunctionName, "", + script_elements(FuncAndInput, "")); + {ok, BadRequest} -> + {proceed,[{status,{400,none, BadRequest}} | + ModData#mod.data]} + end; + {ok, BadRequest} -> + {proceed, [{status,{400, none, BadRequest}} | ModData#mod.data]} + end; + +erl(#mod{method = "POST", entity_body = Body} = ModData, ESIBody, Modules) -> + case httpd_util:split(ESIBody,":|%3A|/",2) of + {ok,[ModuleName, Function]} -> + generate_webpage(ModData, ESIBody, Modules, + list_to_atom(ModuleName), + Function, Body, [{entity_body, Body}]); + {ok, BadRequest} -> + {proceed,[{status, {400, none, BadRequest}} | ModData#mod.data]} + end. + +generate_webpage(ModData, ESIBody, [all], Module, FunctionName, + Input, ScriptElements) -> + generate_webpage(ModData, ESIBody, [Module], Module, + FunctionName, Input, ScriptElements); +generate_webpage(ModData, ESIBody, Modules, Module, FunctionName, + Input, ScriptElements) -> + Function = list_to_atom(FunctionName), + case lists:member(Module, Modules) of + true -> + Env = httpd_script_env:create_env(esi, ModData, ScriptElements), + case erl_scheme_webpage_chunk(Module, Function, + Env, Input, ModData) of + {error, erl_scheme_webpage_chunk_undefined} -> + erl_scheme_webpage_whole(Module, Function, Env, Input, + ModData); + ResponseResult -> + ResponseResult + end; + false -> + {proceed, [{status, {403, ModData#mod.request_uri, + ?NICE("Client not authorized to evaluate: " + ++ ESIBody)}} | ModData#mod.data]} + end. + +%% Old API that waits for the dymnamic webpage to be totally generated +%% before anythig is sent back to the client. +erl_scheme_webpage_whole(Module, Function, Env, Input, ModData) -> + case (catch Module:Function(Env, Input)) of + {'EXIT',{undef, _}} -> + {proceed, [{status, {404, ModData#mod.request_uri, "Not found"}} + | ModData#mod.data]}; + {'EXIT',Reason} -> + {proceed, [{status, {500, none, Reason}} | + ModData#mod.data]}; + Response -> + {Headers, Body} = + httpd_esi:parse_headers(lists:flatten(Response)), + Length = httpd_util:flatlength(Body), + case httpd_esi:handle_headers(Headers) of + {proceed, AbsPath} -> + {proceed, [{real_name, httpd_util:split_path(AbsPath)} + | ModData#mod.data]}; + {ok, NewHeaders, StatusCode} -> + send_headers(ModData, StatusCode, + [{"content-length", + integer_to_list(Length)}| NewHeaders]), + case ModData#mod.method of + "HEAD" -> + {proceed, [{response, {already_sent, 200, 0}} | + ModData#mod.data]}; + _ -> + httpd_response:send_body(ModData, + StatusCode, Body), + {proceed, [{response, {already_sent, 200, + Length}} | + ModData#mod.data]} + end + end + end. + +%% New API that allows the dynamic wepage to be sent back to the client +%% in small chunks at the time during generation. +erl_scheme_webpage_chunk(Mod, Func, Env, Input, ModData) -> + process_flag(trap_exit, true), + Self = self(), + %% Spawn worker that generates the webpage. + %% It would be nicer to use erlang:function_exported/3 but if the + %% Module isn't loaded the function says that it is not loaded + Pid = spawn_link( + fun() -> + case catch Mod:Func(Self, Env, Input) of + {'EXIT',{undef,_}} -> + %% Will force fallback on the old API + exit(erl_scheme_webpage_chunk_undefined); + _ -> + ok + end + end), + + Response = deliver_webpage_chunk(ModData, Pid), + + process_flag(trap_exit,false), + Response. + +deliver_webpage_chunk(#mod{config_db = Db} = ModData, Pid) -> + Timeout = erl_script_timeout(Db), + deliver_webpage_chunk(ModData, Pid, Timeout). + +deliver_webpage_chunk(#mod{config_db = Db} = ModData, Pid, Timeout) -> + case receive_headers(Timeout) of + {error, Reason} -> + %% Happens when webpage generator callback/3 is undefined + {error, Reason}; + {Headers, Body} -> + case httpd_esi:handle_headers(Headers) of + {proceed, AbsPath} -> + {proceed, [{real_name, httpd_util:split_path(AbsPath)} + | ModData#mod.data]}; + {ok, NewHeaders, StatusCode} -> + IsDisableChunkedSend = + httpd_response:is_disable_chunked_send(Db), + case (ModData#mod.http_version =/= "HTTP/1.1") or + (IsDisableChunkedSend) of + true -> + send_headers(ModData, StatusCode, + [{"connection", "close"} | + NewHeaders]); + false -> + send_headers(ModData, StatusCode, + [{"transfer-encoding", + "chunked"} | NewHeaders]) + end, + handle_body(Pid, ModData, Body, Timeout, length(Body), + IsDisableChunkedSend) + end; + timeout -> + send_headers(ModData, {504, "Timeout"},[{"connection", "close"}]), + httpd_socket:close(ModData#mod.socket_type, ModData#mod.socket), + process_flag(trap_exit,false), + {proceed,[{response, {already_sent, 200, 0}} | ModData#mod.data]} + end. + +receive_headers(Timeout) -> + receive + {ok, Chunk} -> + httpd_esi:parse_headers(lists:flatten(Chunk)); + {'EXIT', Pid, erl_scheme_webpage_chunk_undefined} when is_pid(Pid) -> + {error, erl_scheme_webpage_chunk_undefined}; + {'EXIT', Pid, Reason} when is_pid(Pid) -> + exit({mod_esi_linked_process_died, Pid, Reason}) + after Timeout -> + timeout + end. + +send_headers(ModData, StatusCode, HTTPHeaders) -> + ExtraHeaders = httpd_response:cache_headers(ModData), + httpd_response:send_header(ModData, StatusCode, + ExtraHeaders ++ HTTPHeaders). + +handle_body(_, #mod{method = "HEAD"} = ModData, _, _, Size, _) -> + process_flag(trap_exit,false), + {proceed, [{response, {already_sent, 200, Size}} | ModData#mod.data]}; + +handle_body(Pid, ModData, Body, Timeout, Size, IsDisableChunkedSend) -> + httpd_response:send_chunk(ModData, Body, IsDisableChunkedSend), + receive + {ok, Data} -> + handle_body(Pid, ModData, Data, Timeout, Size + length(Data), + IsDisableChunkedSend); + {'EXIT', Pid, normal} when is_pid(Pid) -> + httpd_response:send_final_chunk(ModData, IsDisableChunkedSend), + {proceed, [{response, {already_sent, 200, Size}} | + ModData#mod.data]}; + {'EXIT', Pid, Reason} when is_pid(Pid) -> + exit({mod_esi_linked_process_died, Pid, Reason}) + after Timeout -> + process_flag(trap_exit,false), + {proceed,[{response, {already_sent, 200, Size}} | + ModData#mod.data]} + end. + +erl_script_timeout(Db) -> + httpd_util:lookup(Db, erl_script_timeout, ?DEFAULT_ERL_TIMEOUT). + +script_elements(FuncAndInput, Input) -> + case input_type(FuncAndInput) of + path_info -> + [{path_info, Input}]; + query_string -> + [{query_string, Input}]; + _ -> + [] + end. + +input_type([]) -> + no_input; +input_type([$/|_Rest]) -> + path_info; +input_type([$?|_Rest]) -> + query_string; +input_type([_First|Rest]) -> + input_type(Rest). + +%%------------------------ Eval mechanism -------------------------------- + +eval(#mod{request_uri = ReqUri, method = "POST", + http_version = Version, data = Data}, _ESIBody, _Modules) -> + {proceed,[{status,{501,{"POST", ReqUri, Version}, + ?NICE("Eval mechanism doesn't support method POST")}}| + Data]}; + +eval(#mod{method = Method} = ModData, ESIBody, Modules) + when Method == "GET"; Method == "HEAD" -> + case is_authorized(ESIBody, Modules) of + true -> + case generate_webpage(ESIBody) of + {error, Reason} -> + {proceed, [{status, {500, none, Reason}} | + ModData#mod.data]}; + {ok, Response} -> + {Headers, _} = + httpd_esi:parse_headers(lists:flatten(Response)), + case httpd_esi:handle_headers(Headers) of + {ok, _, StatusCode} -> + {proceed,[{response, {StatusCode, Response}} | + ModData#mod.data]}; + {proceed, AbsPath} -> + {proceed, [{real_name, AbsPath} | + ModData#mod.data]} + end + end; + false -> + {proceed,[{status, + {403, ModData#mod.request_uri, + ?NICE("Client not authorized to evaluate: " + ++ ESIBody)}} | ModData#mod.data]} + end. + +generate_webpage(ESIBody) -> + (catch lib:eval_str(string:concat(ESIBody,". "))). + +is_authorized(_ESIBody, [all]) -> + true; +is_authorized(ESIBody, Modules) -> + case inets_regexp:match(ESIBody, "^[^\:(%3A)]*") of + {match, Start, Length} -> + lists:member(list_to_atom(string:substr(ESIBody, Start, Length)), + Modules); + nomatch -> + false + end. diff --git a/lib/inets/src/http_server/mod_get.erl b/lib/inets/src/http_server/mod_get.erl new file mode 100644 index 0000000000..9fd1fcec47 --- /dev/null +++ b/lib/inets/src/http_server/mod_get.erl @@ -0,0 +1,126 @@ +%% +%% %CopyrightBegin% +%% +%% Copyright Ericsson AB 1997-2009. All Rights Reserved. +%% +%% The contents of this file are subject to the Erlang Public License, +%% Version 1.1, (the "License"); you may not use this file except in +%% compliance with the License. You should have received a copy of the +%% Erlang Public License along with this software. If not, it can be +%% retrieved online at http://www.erlang.org/. +%% +%% Software distributed under the License is distributed on an "AS IS" +%% basis, WITHOUT WARRANTY OF ANY KIND, either express or implied. See +%% the License for the specific language governing rights and limitations +%% under the License. +%% +%% %CopyrightEnd% +%% +%% +-module(mod_get). +-export([do/1]). +-include("httpd.hrl"). + +%% do + +do(Info) -> + ?DEBUG("do -> entry",[]), + case Info#mod.method of + "GET" -> + case proplists:get_value(status, Info#mod.data) of + %% A status code has been generated! + {_StatusCode, _PhraseArgs, _Reason} -> + {proceed,Info#mod.data}; + %% No status code has been generated! + undefined -> + case proplists:get_value(response, Info#mod.data) of + %% No response has been generated! + undefined -> + do_get(Info); + %% A response has been generated or sent! + _Response -> + {proceed,Info#mod.data} + end + end; + %% Not a GET method! + _ -> + {proceed,Info#mod.data} + end. + + +do_get(Info) -> + ?DEBUG("do_get -> Request URI: ~p",[Info#mod.request_uri]), + Path = mod_alias:path(Info#mod.data, Info#mod.config_db, + Info#mod.request_uri), + + send_response(Info#mod.socket,Info#mod.socket_type, Path, Info). + + +%% The common case when no range is specified +send_response(_Socket, _SocketType, Path, Info)-> + %% Send the file! + %% Find the modification date of the file + case file:open(Path,[raw,binary]) of + {ok, FileDescriptor} -> + {FileInfo, LastModified} = get_modification_date(Path), + ?DEBUG("do_get -> FileDescriptor: ~p",[FileDescriptor]), + Suffix = httpd_util:suffix(Path), + MimeType = httpd_util:lookup_mime_default(Info#mod.config_db, + Suffix,"text/plain"), + %% FileInfo = file:read_file_info(Path), + Size = integer_to_list(FileInfo#file_info.size), + Headers = case Info#mod.http_version of + "HTTP/1.1" -> + [{content_type, MimeType}, + {etag, httpd_util:create_etag(FileInfo)}, + {content_length, Size}|LastModified]; + %% OTP-4935 + _ -> + %% i.e http/1.0 and http/0.9 + [{content_type, MimeType}, + {content_length, Size}|LastModified] + end, + send(Info, 200, Headers, FileDescriptor), + file:close(FileDescriptor), + {proceed,[{response,{already_sent,200, + FileInfo#file_info.size}}, + {mime_type,MimeType}|Info#mod.data]}; + {error, Reason} -> + Status = httpd_file:handle_error(Reason, "open", Info, Path), + {proceed, + [{status, Status}| Info#mod.data]} + end. + +%% send + +send(#mod{socket = Socket, socket_type = SocketType} = Info, + StatusCode, Headers, FileDescriptor) -> + ?DEBUG("send -> send header",[]), + httpd_response:send_header(Info, StatusCode, Headers), + send_body(SocketType,Socket,FileDescriptor). + + +send_body(SocketType,Socket,FileDescriptor) -> + case file:read(FileDescriptor,?FILE_CHUNK_SIZE) of + {ok,Binary} -> + ?DEBUG("send_body -> send another chunk: ~p",[size(Binary)]), + case httpd_socket:deliver(SocketType,Socket,Binary) of + socket_closed -> + ?LOG("send_body -> socket closed while sending",[]), + socket_close; + _ -> + send_body(SocketType,Socket,FileDescriptor) + end; + eof -> + ?DEBUG("send_body -> done with this file",[]), + eof + end. + +get_modification_date(Path)-> + {ok, FileInfo0} = file:read_file_info(Path), + LastModified = + case catch httpd_util:rfc1123_date(FileInfo0#file_info.mtime) of + Date when is_list(Date) -> [{last_modified, Date}]; + _ -> [] + end, + {FileInfo0, LastModified}. diff --git a/lib/inets/src/http_server/mod_head.erl b/lib/inets/src/http_server/mod_head.erl new file mode 100644 index 0000000000..8b08d61651 --- /dev/null +++ b/lib/inets/src/http_server/mod_head.erl @@ -0,0 +1,75 @@ +%% +%% %CopyrightBegin% +%% +%% Copyright Ericsson AB 1997-2009. All Rights Reserved. +%% +%% The contents of this file are subject to the Erlang Public License, +%% Version 1.1, (the "License"); you may not use this file except in +%% compliance with the License. You should have received a copy of the +%% Erlang Public License along with this software. If not, it can be +%% retrieved online at http://www.erlang.org/. +%% +%% Software distributed under the License is distributed on an "AS IS" +%% basis, WITHOUT WARRANTY OF ANY KIND, either express or implied. See +%% the License for the specific language governing rights and limitations +%% under the License. +%% +%% %CopyrightEnd% +%% +%% +-module(mod_head). +-export([do/1]). + +-include("httpd.hrl"). + +-define(VMODULE,"HEAD"). + +%% do + +do(Info) -> + case Info#mod.method of + "HEAD" -> + case proplists:get_value(status, Info#mod.data) of + %% A status code has been generated! + {_StatusCode, _PhraseArgs, _Reason} -> + {proceed,Info#mod.data}; + %% No status code has been generated! + _undefined -> + case proplists:get_value(response, Info#mod.data) of + %% No response has been generated! + undefined -> + do_head(Info); + %% A response has been sent! Nothing to do about it! + {already_sent, _StatusCode, _Size} -> + {proceed,Info#mod.data}; + %% A response has been generated! + {_StatusCode, _Response} -> + {proceed,Info#mod.data} + end + end; + %% Not a HEAD method! + _ -> + {proceed,Info#mod.data} + end. + +do_head(Info) -> + Path = mod_alias:path(Info#mod.data, + Info#mod.config_db, + Info#mod.request_uri), + Suffix = httpd_util:suffix(Path), + %% Does the file exists? + case file:read_file_info(Path) of + {ok, FileInfo} -> + MimeType = + httpd_util:lookup_mime_default(Info#mod.config_db, + Suffix,"text/plain"), + Length = io_lib:write(FileInfo#file_info.size), + Head = + [{content_type, MimeType}, + {content_length, Length}, {code,200}], + {proceed,[{response, {response, Head, nobody}} | Info#mod.data]}; + {error, Reason} -> + Status = httpd_file:handle_error(Reason, "access", Info, Path), + {proceed, + [{status, Status} | Info#mod.data]} + end. diff --git a/lib/inets/src/http_server/mod_htaccess.erl b/lib/inets/src/http_server/mod_htaccess.erl new file mode 100644 index 0000000000..d8835198f5 --- /dev/null +++ b/lib/inets/src/http_server/mod_htaccess.erl @@ -0,0 +1,1078 @@ +%% +%% %CopyrightBegin% +%% +%% Copyright Ericsson AB 2001-2009. All Rights Reserved. +%% +%% The contents of this file are subject to the Erlang Public License, +%% Version 1.1, (the "License"); you may not use this file except in +%% compliance with the License. You should have received a copy of the +%% Erlang Public License along with this software. If not, it can be +%% retrieved online at http://www.erlang.org/. +%% +%% Software distributed under the License is distributed on an "AS IS" +%% basis, WITHOUT WARRANTY OF ANY KIND, either express or implied. See +%% the License for the specific language governing rights and limitations +%% under the License. +%% +%% %CopyrightEnd% +%% +%% + +-module(mod_htaccess). + +-export([do/1, load/2, store/2]). + +-include("httpd.hrl"). + +%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%% +%% Public methods that interface the eswapi %% +%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%% + +%---------------------------------------------------------------------- +% Public method called by the webbserver to insert the data about +% Names on accessfiles +%---------------------------------------------------------------------- +load("AccessFileName" ++ FileNames, _Context)-> + CleanFileNames=httpd_conf:clean(FileNames), + {ok,[],{access_files,string:tokens(CleanFileNames," ")}}. + +store({access_files, Files} = Conf, _) when is_list(Files)-> + {ok, Conf}; +store({access_files, Value}, _) -> + {error, {wrong_type, {access_files, Value}}}. + +%---------------------------------------------------------------------- +% Public method that the webbserver calls to control the page +%---------------------------------------------------------------------- +do(Info)-> + case proplists:get_value(status, Info#mod.data) of + {_Status_code, _PhraseArgs, _Reason}-> + {proceed,Info#mod.data}; + undefined -> + control_path(Info) + end. + + +%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%% +%% %% +%% The functions that start the control if there is a accessfile %% +%% and if so controls if the dir is allowed or not %% +%% %% +%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%% +%---------------------------------------------------------------------- +%Info = record mod as specified in httpd.hrl +%returns either {proceed,Info#mod.data} +%{proceed,[{status,403....}|Info#mod.data]} +%{proceed,[{status,401....}|Info#mod.data]} +%{proceed,[{status,500....}|Info#mod.data]} +%---------------------------------------------------------------------- +control_path(Info) -> + Path = mod_alias:path(Info#mod.data, + Info#mod.config_db, + Info#mod.request_uri), + case isErlScriptOrNotAccessibleFile(Path,Info) of + true-> + {proceed,Info#mod.data}; + false-> + case getHtAccessData(Path,Info)of + {ok,public}-> + %%There was no restrictions on the page continue + {proceed,Info#mod.data}; + {error, _Reason} -> + %%Something got wrong continue or quit??????????????????/ + {proceed,Info#mod.data}; + {accessData,AccessData}-> + controlAllowedMethod(Info,AccessData) + end + end. + + +%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%% +%% %% +%% These methods controls that the method the client used in the %% +%% request is one of the limited %% +%% %% +%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%% +%---------------------------------------------------------------------- +%Control that if the accessmethod used is in the list of modes to challenge +% +%Info is the mod record as specified in httpd.hrl +%AccessData is an ets table whit the data in the .htaccessfiles +%---------------------------------------------------------------------- +controlAllowedMethod(Info,AccessData)-> + case allowedRequestMethod(Info,AccessData) of + allow-> + %%The request didnt use one of the limited methods + ets:delete(AccessData), + {proceed,Info#mod.data}; + challenge-> + authenticateUser(Info,AccessData) + end. + +%---------------------------------------------------------------------- +%Check the specified access method in the .htaccessfile +%---------------------------------------------------------------------- +allowedRequestMethod(Info,AccessData)-> + case ets:lookup(AccessData,limit) of + [{limit,all}]-> + challenge; + [{limit,Methods}]-> + isLimitedRequestMethod(Info,Methods) + end. + + +%---------------------------------------------------------------------- +%Check the specified accessmethods in the .htaccesfile against the users +%accessmethod +% +%Info is the record from the do call +%Methods is a list of the methods specified in the .htaccessfile +%---------------------------------------------------------------------- +isLimitedRequestMethod(Info,Methods)-> + case lists:member(Info#mod.method,Methods) of + true-> + challenge; + false -> + allow + end. + + +%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%% +%% %% +%% These methods controls that the user comes from an allowwed net %% +%% and if so wheather its a valid user or a challenge shall be %% +%% generated %% +%% %% +%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%% +%---------------------------------------------------------------------- +%The first thing to control is that the user is from a network +%that has access to the page +%---------------------------------------------------------------------- +authenticateUser(Info,AccessData)-> + case controlNet(Info,AccessData) of + allow-> + %the network is ok control that it is an allowed user + authenticateUser2(Info,AccessData); + deny-> + %The user isnt allowed to access the pages from that network + ets:delete(AccessData), + {proceed,[{status,{403,Info#mod.request_uri, + "Restricted area not allowed from your network"}}|Info#mod.data]} + end. + + +%---------------------------------------------------------------------- +%The network the user comes from is allowed to view the resources +%control whether the user needsto supply a password or not +%---------------------------------------------------------------------- +authenticateUser2(Info,AccessData)-> + case ets:lookup(AccessData,require) of + [{require,AllowedUsers}]-> + case ets:lookup(AccessData,auth_name) of + [{auth_name,Realm}]-> + authenticateUser2(Info,AccessData,Realm,AllowedUsers); + _NoAuthName-> + ets:delete(AccessData), + {break,[{status,{500,none, + ?NICE("mod_htaccess:AuthName directive " + "not specified")}}]} + end; + [] -> + %%No special user is required the network is ok so let + %%the user in + ets:delete(AccessData), + {proceed,Info#mod.data} + end. + + +%---------------------------------------------------------------------- +%The user must send a userId and a password to get the resource +%Control if its already in the http-request +%if the file with users is bad send an 500 response +%---------------------------------------------------------------------- +authenticateUser2(Info,AccessData,Realm,AllowedUsers)-> + case authenticateUser(Info,AccessData,AllowedUsers) of + allow -> + ets:delete(AccessData), + {user,Name, _Pwd} = getAuthenticatingDataFromHeader(Info), + {proceed, [{remote_user_name,Name}|Info#mod.data]}; + challenge-> + ets:delete(AccessData), + ReasonPhrase = httpd_util:reason_phrase(401), + Message = httpd_util:message(401,none,Info#mod.config_db), + {proceed, + [{response, + {401, + ["WWW-Authenticate: Basic realm=\"",Realm, + "\"\r\n\r\n","<HTML>\n<HEAD>\n<TITLE>", + ReasonPhrase,"</TITLE>\n", + "</HEAD>\n<BODY>\n<H1>",ReasonPhrase, + "</H1>\n",Message,"\n</BODY>\n</HTML>\n"]}}| + Info#mod.data]}; + deny-> + ets:delete(AccessData), + {break,[{status,{500,none, + ?NICE("mod_htaccess:Bad path to user " + "or group file")}}]} + end. + + +%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%% +%% %% +%% Methods that validate the netwqork the user comes from %% +%% according to the allowed networks %% +%% %% +%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%% +%--------------------------------------------------------------------- +%Controls the users networkaddress agains the specifed networks to +%allow or deny +% +%returns either allow or deny +%---------------------------------------------------------------------- +controlNet(Info,AccessData)-> + UserNetwork=getUserNetworkAddress(Info), + case getAllowDenyOrder(AccessData) of + {_deny,[],_allow,[]}-> + allow; + {deny,[],allow,AllowedNetworks}-> + controlIfAllowed(AllowedNetworks,UserNetwork,allow,deny); + {allow,AllowedNetworks,deny,[]}-> + controlIfAllowed(AllowedNetworks,UserNetwork,allow,deny); + + {deny,DeniedNetworks,allow,[]}-> + controlIfAllowed(DeniedNetworks,UserNetwork,allow,deny); + {allow,[],deny,DeniedNetworks}-> + controlIfAllowed(DeniedNetworks,UserNetwork,allow,deny); + + {deny,DeniedNetworks,allow,AllowedNetworks}-> + controlDenyAllow(DeniedNetworks,AllowedNetworks,UserNetwork); + {allow,AllowedNetworks,deny,DeniedNetworks}-> + controlAllowDeny(AllowedNetworks,DeniedNetworks,UserNetwork) + end. + + +%---------------------------------------------------------------------- +%Returns the users IP-Number +%---------------------------------------------------------------------- +getUserNetworkAddress(Info)-> + {_Socket,Address}=(Info#mod.init_data)#init_data.peername, + Address. + + +%---------------------------------------------------------------------- +%Control the users Ip-number against the ip-numbers in the .htaccessfile +%---------------------------------------------------------------------- +controlIfAllowed(AllowedNetworks,UserNetwork,IfAllowed,IfDenied)-> + case AllowedNetworks of + [{allow,all}]-> + IfAllowed; + [{deny,all}]-> + IfDenied; + [{deny,Networks}]-> + memberNetwork(Networks,UserNetwork,IfDenied,IfAllowed); + [{allow,Networks}]-> + memberNetwork(Networks,UserNetwork,IfAllowed,IfDenied); + _Error-> + IfDenied + end. + + +%---------------------------------------------------------------------% +%The Denycontrol isn't neccessary to preform since the allow control % +%override the deny control % +%---------------------------------------------------------------------% +controlDenyAllow(_DeniedNetworks, AllowedNetworks, UserNetwork)-> + case AllowedNetworks of + [{allow, all}]-> + allow; + [{allow, Networks}]-> + case memberNetwork(Networks, UserNetwork) of + true-> + allow; + false-> + deny + end + end. + + +%----------------------------------------------------------------------% +%Control that the user is in the allowed list if so control that the % +%network is in the denied list +%----------------------------------------------------------------------% +controlAllowDeny(AllowedNetworks,DeniedNetworks,UserNetwork)-> + case controlIfAllowed(AllowedNetworks,UserNetwork,allow,deny) of + allow-> + controlIfAllowed(DeniedNetworks,UserNetwork,deny,allow); + deny -> + deny + end. + +%---------------------------------------------------------------------- +%Controls if the users Ipnumber is in the list of either denied or +%allowed networks +%---------------------------------------------------------------------- +memberNetwork(Networks,UserNetwork,IfTrue,IfFalse)-> + case memberNetwork(Networks,UserNetwork) of + true-> + IfTrue; + false-> + IfFalse + end. + + +%---------------------------------------------------------------------- +%regexp match the users ip-address against the networks in the list of +%ipadresses or subnet addresses. +memberNetwork(Networks,UserNetwork)-> + case lists:filter(fun(Net)-> + case inets_regexp:match(UserNetwork, + formatRegexp(Net)) of + {match,1,_}-> + true; + _NotSubNet -> + false + end + end,Networks) of + []-> + false; + _MemberNetWork -> + true + end. + + +%---------------------------------------------------------------------- +%Creates a regexp from an ip-number i.e "127.0.0-> "^127[.]0[.]0.*" +%"127.0.0.-> "^127[.]0[.]0[.].*" +%---------------------------------------------------------------------- +formatRegexp(Net)-> + [SubNet1|SubNets]=string:tokens(Net,"."), + NetRegexp=lists:foldl(fun(SubNet,Newnet)-> + Newnet ++ "[.]" ++SubNet + end,"^"++SubNet1,SubNets), + case string:len(Net)-string:rchr(Net,$.) of + 0-> + NetRegexp++"[.].*"; + _-> + NetRegexp++".*" + end. + +%---------------------------------------------------------------------- +%If the user has specified if the allow or deny check shall be preformed +%first get that order if no order is specified take +%allow - deny since its harder that deny - allow +%---------------------------------------------------------------------- +getAllowDenyOrder(AccessData)-> + case ets:lookup(AccessData,order) of + [{order,{deny,allow}}]-> + {deny,ets:lookup(AccessData,deny), + allow,ets:lookup(AccessData,allow)}; + _DefaultOrder-> + {allow,ets:lookup(AccessData,allow), + deny,ets:lookup(AccessData,deny)} + end. + + +%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%% +%% %% +%% The methods that validates the user %% +%% %% +%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%% + +%---------------------------------------------------------------------- +%Control if there is anyu autheticating data in threquest header +%if so it controls it against the users in the list Allowed Users +%---------------------------------------------------------------------- +authenticateUser(Info,AccessData,AllowedUsers)-> + case getAuthenticatingDataFromHeader(Info) of + {user,User,PassWord}-> + authenticateUser(Info,AccessData,AllowedUsers, + {user,User,PassWord}); + {error,nouser}-> + challenge; + {error, _BadData}-> + challenge + end. + + +%---------------------------------------------------------------------- +%Returns the Autheticating data in the http-request +%---------------------------------------------------------------------- +getAuthenticatingDataFromHeader(Info)-> + PrsedHeader=Info#mod.parsed_header, + case proplists:get_value("authorization", PrsedHeader) of + undefined-> + {error,nouser}; + [$B,$a,$s,$i,$c,$\ |EncodedString] = Credentials -> + case (catch base64:decode_to_string(EncodedString)) of + {'EXIT',{function_clause, _}} -> + {error, Credentials}; + UnCodedString -> + case httpd_util:split(UnCodedString,":",2) of + {ok,[User,PassWord]}-> + {user,User,PassWord}; + {error,Error}-> + {error,Error} + end + end; + BadCredentials -> + {error,BadCredentials} + end. + +%---------------------------------------------------------------------- +%Returns a list of all members of the allowed groups +%---------------------------------------------------------------------- +getGroupMembers(Groups,AllowedGroups)-> + Allowed=lists:foldl(fun({group,Name,Members},AllowedMembers)-> + case lists:member(Name,AllowedGroups) of + true-> + AllowedMembers++Members; + false -> + AllowedMembers + end + end,[],Groups), + {ok,Allowed}. + +authenticateUser(Info,AccessData,{{users,[]},{groups,Groups}},User)-> + authenticateUser(Info,AccessData,{groups,Groups},User); +authenticateUser(Info,AccessData,{{users,Users},{groups,[]}},User)-> + authenticateUser(Info,AccessData,{users,Users},User); + +authenticateUser(Info,AccessData,{{users,Users},{groups,Groups}},User)-> + AllowUser=authenticateUser(Info,AccessData,{users,Users},User), + AllowGroup=authenticateUser(Info,AccessData,{groups,Groups},User), + case {AllowGroup,AllowUser} of + {_,allow}-> + allow; + {allow,_}-> + allow; + {challenge,_}-> + challenge; + {_,challenge}-> + challenge; + {_deny,_deny}-> + deny + end; + + +%---------------------------------------------------------------------- +%Controls that the user is a member in one of the allowed group +%---------------------------------------------------------------------- +authenticateUser(Info,AccessData,{groups,AllowedGroups},{user,User,PassWord})-> + case getUsers(AccessData,group_file) of + {group_data,Groups}-> + {ok, Members } = getGroupMembers(Groups,AllowedGroups), + authenticateUser(Info,AccessData,{users,Members}, + {user,User,PassWord}); + {error, _BadData}-> + deny + end; + + +%---------------------------------------------------------------------- +%Control that the user is one of the allowed users and that the passwd is ok +%---------------------------------------------------------------------- +authenticateUser(_Info,AccessData,{users,AllowedUsers},{user,User,PassWord})-> + case lists:member(User,AllowedUsers) of + true-> + %Get the usernames and passwords from the file + case getUsers(AccessData,user_file) of + {error, _BadData}-> + deny; + {user_data,Users}-> + %Users is a list of the users in + %the userfile [{user,User,Passwd}] + checkPassWord(Users,{user,User,PassWord}) + end; + false -> + challenge + end. + + +%---------------------------------------------------------------------- +%Control that the user User={user,"UserName","PassWd"} is +%member of the list of Users +%---------------------------------------------------------------------- +checkPassWord(Users,User)-> + case lists:member(User,Users) of + true-> + allow; + false-> + challenge + end. + + +%---------------------------------------------------------------------- +%Get the users in the specified file +%UserOrGroup is an atom that specify if its a group file or a user file +%i.e. group_file or user_file +%---------------------------------------------------------------------- +getUsers({file,FileName},UserOrGroup)-> + case file:open(FileName,[read]) of + {ok,AccessFileHandle} -> + getUsers({stream,AccessFileHandle},[],UserOrGroup); + {error,Reason} -> + {error,{Reason,FileName}} + end; + + +%---------------------------------------------------------------------- +%The method that starts the lokkong for user files +%---------------------------------------------------------------------- + +getUsers(AccessData,UserOrGroup)-> + case ets:lookup(AccessData,UserOrGroup) of + [{UserOrGroup,File}]-> + getUsers({file,File},UserOrGroup); + _ -> + {error,noUsers} + end. + + +%---------------------------------------------------------------------- +%Reads data from the filehandle File to the list FileData and when its +%reach the end it returns the list in a tuple {user_file|group_file,FileData} +%---------------------------------------------------------------------- +getUsers({stream,File},FileData,UserOrGroup)-> + case io:get_line(File,[]) of + eof when UserOrGroup =:= user_file -> + {user_data,FileData}; + eof when UserOrGroup =:= group_file -> + {group_data,FileData}; + Line -> + getUsers({stream,File}, + formatUser(Line,FileData,UserOrGroup),UserOrGroup) + end. + + +%---------------------------------------------------------------------- +%If the line is a comment remove it +%---------------------------------------------------------------------- +formatUser([$#|_UserDataComment],FileData,_UserOrgroup)-> + FileData; + + +%---------------------------------------------------------------------- +%The user name in the file is Username:Passwd\n +%Remove the newline sign and split the user name in +%UserName and Password +%---------------------------------------------------------------------- +formatUser(UserData,FileData,UserOrGroup)-> + case string:tokens(UserData," \r\n")of + [User| _Whitespace] when UserOrGroup =:= user_file -> + case string:tokens(User,":") of + [Name,PassWord]-> + [{user,Name,PassWord}|FileData]; + _Error-> + FileData + end; + GroupData when UserOrGroup =:= group_file -> + parseGroupData(GroupData,FileData); + _Error -> + FileData + end. + + +%---------------------------------------------------------------------- +%if everything is right GroupData is on the form +% ["groupName:", "Member1", "Member2", "Member2" +%---------------------------------------------------------------------- +parseGroupData([GroupName|GroupData],FileData)-> + [{group,formatGroupName(GroupName),GroupData}|FileData]. + + +%---------------------------------------------------------------------- +%the line in the file is GroupName: Member1 Member2 .....MemberN +%Remove the : from the group name +%---------------------------------------------------------------------- +formatGroupName(GroupName)-> + string:strip(GroupName,right,$:). + + +%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%% +%% %% +%% Functions that parses the accessfiles %% +%% %% +%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%% +%---------------------------------------------------------------------- +%Control that the asset is a real file and not a request for an virtual +%asset +%---------------------------------------------------------------------- +isErlScriptOrNotAccessibleFile(Path, _Info)-> + case file:read_file_info(Path) of + {ok,_fileInfo}-> + false; + {error,_Reason} -> + true + end. + + +%---------------------------------------------------------------------- +%Path=PathToTheRequestedFile=String +%Innfo=record#mod +%---------------------------------------------------------------------- +getHtAccessData(Path,Info)-> + HtAccessFileNames=getHtAccessFileNames(Info), + case getData(Path,Info,HtAccessFileNames) of + {ok,public}-> + {ok,public}; + {accessData,AccessData}-> + {accessData,AccessData}; + {error,Reason} -> + {error,Reason} + end. + + +%---------------------------------------------------------------------- +%returns the names of the accessfiles +%---------------------------------------------------------------------- +getHtAccessFileNames(Info)-> + case httpd_util:lookup(Info#mod.config_db,access_files) of + undefined-> + [".htaccess"]; + Files-> + Files + end. +%---------------------------------------------------------------------- +%HtAccessFileNames=["accessfileName1",..."AccessFileName2"] +%---------------------------------------------------------------------- +getData(Path,Info,HtAccessFileNames)-> + case inets_regexp:split(Path,"/") of + {error,Error}-> + {error,Error}; + {ok,SplittedPath}-> + getData2(HtAccessFileNames,SplittedPath,Info) + end. + + +%---------------------------------------------------------------------- +%Add to together the data in the Splittedpath up to the path +%that is the alias or the document root +%Since we do not need to control after any accessfiles before here +%---------------------------------------------------------------------- +getData2(HtAccessFileNames,SplittedPath,Info)-> + case getRootPath(SplittedPath,Info) of + {error,Path}-> + {error,Path}; + {ok,StartPath,RestOfSplittedPath} -> + getData2(HtAccessFileNames,StartPath,RestOfSplittedPath,Info) + end. + + +%---------------------------------------------------------------------- +%HtAccessFilenames is a list the names the accesssfiles can have +%Path is the shortest match agains all alias and documentroot +%rest of splitted path is a list of the parts of the path +%Info is the mod recod from the server +%---------------------------------------------------------------------- +getData2(HtAccessFileNames, StartPath, RestOfSplittedPath, _Info)-> + case getHtAccessFiles(HtAccessFileNames,StartPath,RestOfSplittedPath) of + []-> + %No accessfile qiut its a public directory + {ok,public}; + Files -> + loadAccessFilesData(Files) + end. + + +%---------------------------------------------------------------------- +%Loads the data in the accessFiles specifiied by +% AccessFiles=["/hoem/public/html/accefile", +% "/home/public/html/priv/accessfile"] +%---------------------------------------------------------------------- +loadAccessFilesData(AccessFiles)-> + loadAccessFilesData(AccessFiles,ets:new(accessData,[])). + + +%---------------------------------------------------------------------- +%Returns the found data +%---------------------------------------------------------------------- +contextToValues(AccessData)-> + case ets:lookup(AccessData,context) of + [{context,Values}]-> + ets:delete(AccessData,context), + insertContext(AccessData,Values), + {accessData,AccessData}; + _Error-> + {error,errorInAccessFile} + end. + + +insertContext(_AccessData, [])-> + ok; + +insertContext(AccessData,[{allow,From}|Values])-> + insertDenyAllowContext(AccessData,{allow,From}), + insertContext(AccessData,Values); + +insertContext(AccessData,[{deny,From}|Values])-> + insertDenyAllowContext(AccessData,{deny,From}), + insertContext(AccessData,Values); + +insertContext(AccessData,[{require,{GrpOrUsr,Members}}|Values])-> + case ets:lookup(AccessData,require) of + [] when GrpOrUsr =:= users -> + ets:insert(AccessData,{require,{{users,Members},{groups,[]}}}); + + [{require,{{users,Users},{groups,Groups}}}] when GrpOrUsr =:= users -> + ets:insert(AccessData,{require,{{users,Users++Members}, + {groups,Groups}}}); + [] when GrpOrUsr =:= groups -> + ets:insert(AccessData,{require,{{users,[]},{groups,Members}}}); + + [{require,{{users,Users},{groups,Groups}}}] when GrpOrUsr =:= groups -> + ets:insert(AccessData,{require,{{users,Users}, + {groups,Groups++Members}}}) + end, + insertContext(AccessData,Values); + + + +%%limit and order directive need no transforming they areis just to insert +insertContext(AccessData,[Elem|Values])-> + ets:insert(AccessData,Elem), + insertContext(AccessData,Values). + + +insertDenyAllowContext(AccessData,{AllowDeny,From})-> + case From of + all -> + ets:insert(AccessData,{AllowDeny,all}); + _AllowedSubnets -> + case ets:lookup(AccessData,AllowDeny) of + []-> + ets:insert(AccessData,{AllowDeny,From}); + [{AllowDeny,all}]-> + ok; + [{AllowDeny,Networks}]-> + ets:insert(AccessData,{allow,Networks++From}) + end + end. + +loadAccessFilesData([],AccessData)-> + %preform context to limits + contextToValues(AccessData), + {accessData,AccessData}; + +%---------------------------------------------------------------------- +%Takes each file in the list and load the data to the ets table +%AccessData +%---------------------------------------------------------------------- +loadAccessFilesData([FileName|FileNames],AccessData)-> + case loadAccessFileData({file,FileName},AccessData) of + overRide-> + loadAccessFilesData(FileNames,AccessData); + noOverRide -> + {accessData,AccessData}; + error-> + ets:delete(AccessData), + {error,errorInAccessFile} + end. + +%---------------------------------------------------------------------- +%opens the filehandle to the specified file +%---------------------------------------------------------------------- +loadAccessFileData({file,FileName},AccessData)-> + case file:open(FileName,[read]) of + {ok,AccessFileHandle}-> + loadAccessFileData({stream,AccessFileHandle},AccessData,[]); + {error, _Reason} -> + overRide + end. + +%---------------------------------------------------------------------- +%%look att each line in the file and add them to the database +%%When end of file is reached control i overrride is allowed +%% if so return +%---------------------------------------------------------------------- +loadAccessFileData({stream,File},AccessData,FileData)-> + case io:get_line(File,[]) of + eof-> + insertData(AccessData,FileData), + case ets:match_object(AccessData,{'_',error}) of + []-> + %Case we got no error control that we can override a + %at least some of the values + case ets:match_object(AccessData, + {allow_over_ride,none}) of + []-> + overRide; + _NoOverride-> + noOverRide + end; + _ -> + error + end; + Line -> + loadAccessFileData({stream,File},AccessData, + insertLine(string:strip(Line,left),FileData)) + end. + +%---------------------------------------------------------------------- +%AccessData is a ets table where the previous found data is inserted +%FileData is a list of the directives in the last parsed file +%before insertion a control is done that the directive is allowed to +%override +%---------------------------------------------------------------------- +insertData(AccessData,{{context,Values},FileData})-> + insertData(AccessData,[{context,Values}|FileData]); + +insertData(AccessData,FileData)-> + case ets:lookup(AccessData,allow_over_ride) of + [{allow_over_ride,all}]-> + lists:foreach(fun(Elem)-> + ets:insert(AccessData,Elem) + end,FileData); + []-> + lists:foreach(fun(Elem)-> + ets:insert(AccessData,Elem) + end,FileData); + [{allow_over_ride,Directives}] when is_list(Directives)-> + lists:foreach(fun({Key,Value}) -> + case lists:member(Key,Directives) of + true-> + ok; + false -> + ets:insert(AccessData,{Key,Value}) + end + end,FileData); + [{allow_over_ride,_}]-> + %Will never appear if the user + %aint doing very strang econfig files + ok + end. +%---------------------------------------------------------------------- +%Take a line in the accessfile and transform it into a tuple that +%later can be inserted in to the ets:table +%---------------------------------------------------------------------- +%%%Here is the alternatives that resides inside the limit context + +insertLine("order"++ Order, {{context, Values}, FileData})-> + {{context,[{order,getOrder(Order)}|Values]},FileData}; +%%Let the user place a tab in the beginning +insertLine([$\t,$o,$r,$d,$e,$r|Order],{{context,Values},FileData})-> + {{context,[{order,getOrder(Order)}|Values]},FileData}; + +insertLine("allow" ++ Allow, {{context, Values}, FileData})-> + {{context,[{allow,getAllowDenyData(Allow)}|Values]},FileData}; +insertLine([$\t,$a,$l,$l,$o,$w|Allow],{{context,Values},FileData})-> + {{context,[{allow,getAllowDenyData(Allow)}|Values]},FileData}; + +insertLine("deny" ++ Deny, {{context,Values}, FileData})-> + {{context,[{deny,getAllowDenyData(Deny)}|Values]},FileData}; +insertLine([$\t, $d,$e,$n,$y|Deny],{{context,Values},FileData})-> + {{context,[{deny,getAllowDenyData(Deny)}|Values]},FileData}; + +insertLine("require" ++ Require, {{context, Values}, FileData})-> + {{context,[{require,getRequireData(Require)}|Values]},FileData}; +insertLine([$\t,$r,$e,$q,$u,$i,$r,$e|Require],{{context,Values},FileData})-> + {{context,[{require,getRequireData(Require)}|Values]},FileData}; + +insertLine("</Limit" ++ _EndLimit, {Context,FileData})-> + [Context | FileData]; +insertLine("<Limit" ++ Limit, FileData)-> + {{context,[{limit,getLimits(Limit)}]}, FileData}; + +insertLine([$A,$u,$t,$h,$U,$s,$e,$r,$F,$i,$l,$e,$\ |AuthUserFile],FileData)-> + [{user_file,string:strip(AuthUserFile,right,$\n)}|FileData]; + +insertLine([$A,$u,$t,$h,$G,$r,$o,$u,$p,$F,$i,$l,$e,$\ |AuthGroupFile], + FileData)-> + [{group_file,string:strip(AuthGroupFile,right,$\n)}|FileData]; + +insertLine("AllowOverRide" ++ AllowOverRide, FileData)-> + [{allow_over_ride,getAllowOverRideData(AllowOverRide)} + | FileData]; + +insertLine([$A,$u,$t,$h,$N,$a,$m,$e,$\ |AuthName],FileData)-> + [{auth_name,string:strip(AuthName,right,$\n)}|FileData]; + +insertLine("AuthType" ++ AuthType,FileData)-> + [{auth_type,getAuthorizationType(AuthType)}|FileData]; + +insertLine(_BadDirectiveOrComment,FileData)-> + FileData. + +%---------------------------------------------------------------------- +%transform the Data specified about override to a form that is ieasier +%handled later +%Override data="all"|"md5"|"Directive1 .... DirectioveN" +%---------------------------------------------------------------------- + +getAllowOverRideData(OverRideData)-> + case string:tokens(OverRideData," \r\n") of + ["all" ++ _] -> + all; + ["none" ++ _]-> + none; + Directives -> + getOverRideDirectives(Directives) + end. + +getOverRideDirectives(Directives)-> + lists:map(fun(Directive)-> + transformDirective(Directive) + end,Directives). +transformDirective("AuthUserFile" ++ _)-> + user_file; +transformDirective("AuthGroupFile" ++ _) -> + group_file; +transformDirective("AuthName" ++ _)-> + auth_name; +transformDirective("AuthType" ++ _)-> + auth_type; +transformDirective(_UnAllowedOverRideDirective) -> + unallowed. +%---------------------------------------------------------------------- +%Replace the string that specify which method to use for authentication +%and replace it with the atom for easier mathing +%---------------------------------------------------------------------- +getAuthorizationType(AuthType)-> + [Arg | _Crap] = string:tokens(AuthType,"\n\r\ "), + case Arg of + "Basic"-> + basic; + "MD5" -> + md5; + _What -> + error + end. +%---------------------------------------------------------------------- +%Returns a list of the specified methods to limit or the atom all +%---------------------------------------------------------------------- +getLimits(Limits)-> + case inets_regexp:split(Limits,">")of + {ok,[_NoEndOnLimit]}-> + error; + {ok, [Methods | _Crap]}-> + case inets_regexp:split(Methods," ") of + {ok,[]}-> + all; + {ok,SplittedMethods}-> + SplittedMethods; + {error, _Error}-> + error + end; + {error,_Error}-> + error + end. + + +%---------------------------------------------------------------------- +% Transform the order to prefrom deny allow control to a tuple of atoms +%---------------------------------------------------------------------- +getOrder(Order)-> + [First | _Rest]=lists:map(fun(Part)-> + list_to_atom(Part) + end,string:tokens(Order," \n\r")), + case First of + deny-> + {deny,allow}; + allow-> + {allow,deny}; + _Error-> + error + end. + +%---------------------------------------------------------------------- +% The string AllowDeny is "from all" or "from Subnet1 Subnet2...SubnetN" +%---------------------------------------------------------------------- +getAllowDenyData(AllowDeny)-> + case string:tokens(AllowDeny," \n\r") of + [_From|AllowDenyData] when length(AllowDenyData)>=1 -> + case lists:nth(1,AllowDenyData) of + "all" -> + all; + _Hosts-> + AllowDenyData + end; + _ -> + error + end. +%---------------------------------------------------------------------- +% Fix the string that describes who is allowed to se the page +%---------------------------------------------------------------------- +getRequireData(Require)-> + [UserOrGroup|UserData]=string:tokens(Require," \n\r"), + case UserOrGroup of + "user"-> + {users,UserData}; + "group" -> + {groups,UserData}; + _Whatever -> + error + end. + + +%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%% +%% %% +%% Methods that collects the searchways to the accessfiles %% +%% %% +%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%% + +%---------------------------------------------------------------------- +% Get the whole path to the different accessfiles +%---------------------------------------------------------------------- +getHtAccessFiles(HtAccessFileNames,Path,RestOfSplittedPath)-> + getHtAccessFiles(HtAccessFileNames,Path,RestOfSplittedPath,[]). + +getHtAccessFiles(HtAccessFileNames,Path,[[]],HtAccessFiles)-> + HtAccessFiles ++ accessFilesOfPath(HtAccessFileNames,Path++"/"); + +getHtAccessFiles(_HtAccessFileNames, _Path, [], HtAccessFiles)-> + HtAccessFiles; +getHtAccessFiles(HtAccessFileNames,Path,[NextDir|RestOfSplittedPath], + AccessFiles)-> + getHtAccessFiles(HtAccessFileNames,Path++"/"++NextDir,RestOfSplittedPath, + AccessFiles ++ + accessFilesOfPath(HtAccessFileNames,Path++"/")). + + +%---------------------------------------------------------------------- +%Control if therer are any accessfies in the path +%---------------------------------------------------------------------- +accessFilesOfPath(HtAccessFileNames,Path)-> + lists:foldl(fun(HtAccessFileName,Files)-> + case file:read_file_info(Path++HtAccessFileName) of + {ok, _}-> + [Path++HtAccessFileName|Files]; + {error,_Error} -> + Files + end + end,[],HtAccessFileNames). + + +%---------------------------------------------------------------------- +%Sake the splitted path and joins it up to the documentroot or the alias +%that match first +%---------------------------------------------------------------------- + +getRootPath(SplittedPath, Info)-> + DocRoot=httpd_util:lookup(Info#mod.config_db,document_root,"/"), + PresumtiveRootPath= + [DocRoot|lists:map(fun({_Alias,RealPath})-> + RealPath + end, + httpd_util:multi_lookup(Info#mod.config_db,alias))], + getRootPath(PresumtiveRootPath,SplittedPath,Info). + + +getRootPath(PresumtiveRootPath,[[],Splittedpath],Info)-> + getRootPath(PresumtiveRootPath,["/",Splittedpath],Info); + + +getRootPath(PresumtiveRootPath,[Part,NextPart|SplittedPath],Info)-> + case lists:member(Part,PresumtiveRootPath)of + true-> + {ok,Part,[NextPart|SplittedPath]}; + false -> + getRootPath(PresumtiveRootPath, + [Part++"/"++NextPart|SplittedPath],Info) + end; + +getRootPath(PresumtiveRootPath, [Part], _Info)-> + case lists:member(Part,PresumtiveRootPath)of + true-> + {ok,Part,[]}; + false -> + {error,Part} + end. diff --git a/lib/inets/src/http_server/mod_include.erl b/lib/inets/src/http_server/mod_include.erl new file mode 100644 index 0000000000..534eba8a36 --- /dev/null +++ b/lib/inets/src/http_server/mod_include.erl @@ -0,0 +1,597 @@ +%% +%% %CopyrightBegin% +%% +%% Copyright Ericsson AB 1997-2009. All Rights Reserved. +%% +%% The contents of this file are subject to the Erlang Public License, +%% Version 1.1, (the "License"); you may not use this file except in +%% compliance with the License. You should have received a copy of the +%% Erlang Public License along with this software. If not, it can be +%% retrieved online at http://www.erlang.org/. +%% +%% Software distributed under the License is distributed on an "AS IS" +%% basis, WITHOUT WARRANTY OF ANY KIND, either express or implied. See +%% the License for the specific language governing rights and limitations +%% under the License. +%% +%% %CopyrightEnd% +%% +%% +-module(mod_include). +-export([do/1,parse/2,config/6,include/6,echo/6,fsize/6,flastmod/6,exec/6]). + +-include("httpd.hrl"). + +-define(VMODULE,"INCLUDE"). + +%% do + +do(Info) -> + case Info#mod.method of + "GET" -> + case proplists:get_value(status, Info#mod.data) of + %% A status code has been generated! + {_StatusCode, _PhraseArgs, _Reason} -> + {proceed,Info#mod.data}; + %% No status code has been generated! + undefined -> + case proplists:get_value(response, Info#mod.data) of + %% No response has been generated! + undefined -> + do_include(Info); + %% A response has been generated or sent! + _Response -> + {proceed,Info#mod.data} + end + end; + %% Not a GET method! + _ -> + {proceed,Info#mod.data} + end. + +do_include(Info) -> + Path = mod_alias:path(Info#mod.data,Info#mod.config_db, + Info#mod.request_uri), + Suffix = httpd_util:suffix(Path), + case httpd_util:lookup_mime_default(Info#mod.config_db,Suffix) of + "text/x-server-parsed-html" -> + HeaderStart = [{content_type, "text/html"}], + case send_in(Info, Path, HeaderStart, file:read_file_info(Path)) of + {ok, ErrorLog, Size} -> + {proceed, [{response, {already_sent, 200, Size}}, + {mime_type, "text/html"} | + lists:append(ErrorLog, Info#mod.data)]}; + {error, Reason} -> + {proceed, + [{status,send_error(Reason,Info,Path)}|Info#mod.data]} + end; + _ -> %% Unknown mime type, ignore + {proceed,Info#mod.data} + end. + + +%% +%% config directive +%% + +config(_Info, Context, ErrorLog, TagList, ValueList, R) -> + case verify_tags("config",[errmsg,timefmt,sizefmt], + TagList,ValueList) of + ok -> + {ok,update_context(TagList,ValueList,Context),ErrorLog,"",R}; + {error,Reason} -> + {ok,Context,[{internal_info,Reason}|ErrorLog], + proplists:get_value(errmsg,Context,""),R} + end. + +update_context([],[],Context) -> + Context; +update_context([Tag|R1],[Value|R2],Context) -> + update_context(R1,R2,[{Tag,Value}|Context]). + +verify_tags(Command,ValidTags,TagList,ValueList) + when length(TagList) =:= length(ValueList) -> + verify_tags(Command, ValidTags, TagList); +verify_tags(Command, _ValidTags, _TagList, _ValueList) -> + {error, ?NICE(Command ++ " directive has spurious tags")}. + +verify_tags(_Command, _ValidTags, []) -> + ok; +verify_tags(Command, ValidTags, [Tag|Rest]) -> + case lists:member(Tag, ValidTags) of + true -> + verify_tags(Command, ValidTags, Rest); + false -> + {error, ?NICE(Command++" directive has a spurious tag ("++ + atom_to_list(Tag)++")")} + end. + +%% +%% include directive +%% + +include(Info,Context,ErrorLog,[virtual],[VirtualPath],R) -> + Aliases = httpd_util:multi_lookup(Info#mod.config_db,alias), + {_, Path, _AfterPath} = + mod_alias:real_name(Info#mod.config_db, VirtualPath, Aliases), + include(Info,Context,ErrorLog,R,Path); +include(Info, Context, ErrorLog, [file], [FileName], R) -> + Path = file(Info#mod.config_db, Info#mod.request_uri, FileName), + include(Info, Context, ErrorLog, R, Path); +include(_Info, Context, ErrorLog, _TagList, _ValueList, R) -> + {ok, Context, + [{internal_info,?NICE("include directive has a spurious tag")}| + ErrorLog], proplists:get_value(errmsg, Context, ""), R}. + +include(Info, Context, ErrorLog, R, Path) -> + case file:read_file(Path) of + {ok, Body} -> + {ok, NewContext, NewErrorLog, Result} = + parse(Info, binary_to_list(Body), Context, ErrorLog, []), + {ok, NewContext, NewErrorLog, Result, R}; + {error, _Reason} -> + {ok, Context, + [{internal_info, ?NICE("Can't open "++Path)}|ErrorLog], + proplists:get_value(errmsg, Context, ""), R} + end. + +file(ConfigDB, RequestURI, FileName) -> + Aliases = httpd_util:multi_lookup(ConfigDB, alias), + {_, Path, _AfterPath} + = mod_alias:real_name(ConfigDB, RequestURI, Aliases), + Pwd = filename:dirname(Path), + filename:join(Pwd, FileName). + +%% +%% echo directive +%% + +echo(Info,Context,ErrorLog,[var],["DOCUMENT_NAME"],R) -> + {ok,Context,ErrorLog,document_name(Info#mod.data,Info#mod.config_db, + Info#mod.request_uri),R}; +echo(Info,Context,ErrorLog,[var],["DOCUMENT_URI"],R) -> + {ok,Context,ErrorLog,document_uri(Info#mod.config_db, + Info#mod.request_uri),R}; +echo(Info,Context,ErrorLog,[var],["QUERY_STRING_UNESCAPED"],R) -> + {ok,Context,ErrorLog,query_string_unescaped(Info#mod.request_uri),R}; +echo(_Info,Context,ErrorLog,[var],["DATE_LOCAL"],R) -> + {ok,Context,ErrorLog,date_local(),R}; +echo(_Info,Context,ErrorLog,[var],["DATE_GMT"],R) -> + {ok,Context,ErrorLog,date_gmt(),R}; +echo(Info,Context,ErrorLog,[var],["LAST_MODIFIED"],R) -> + {ok,Context,ErrorLog,last_modified(Info#mod.data,Info#mod.config_db, + Info#mod.request_uri),R}; +echo(_Info, Context, ErrorLog, _TagList, _ValueList, R) -> + {ok,Context, + [{internal_info,?NICE("echo directive has a spurious tag")}| + ErrorLog],"(none)",R}. + +document_name(Data,ConfigDB,RequestURI) -> + Path = mod_alias:path(Data,ConfigDB,RequestURI), + case inets_regexp:match(Path,"[^/]*\$") of + {match,Start,Length} -> + string:substr(Path,Start,Length); + nomatch -> + "(none)" + end. + +document_uri(ConfigDB, RequestURI) -> + Aliases = httpd_util:multi_lookup(ConfigDB, alias), + + {_, Path, AfterPath} = mod_alias:real_name(ConfigDB, RequestURI, Aliases), + + VirtualPath = string:substr(RequestURI, 1, + length(RequestURI)-length(AfterPath)), + {match, Start, Length} = inets_regexp:match(Path,"[^/]*\$"), + FileName = string:substr(Path,Start,Length), + case inets_regexp:match(VirtualPath, FileName++"\$") of + {match, _, _} -> + httpd_util:decode_hex(VirtualPath)++AfterPath; + nomatch -> + string:strip(httpd_util:decode_hex(VirtualPath),right,$/)++ + "/"++FileName++AfterPath + end. + +query_string_unescaped(RequestURI) -> + case inets_regexp:match(RequestURI,"[\?].*\$") of + {match,Start,Length} -> + %% Escape all shell-special variables with \ + escape(string:substr(RequestURI,Start+1,Length-1)); + nomatch -> + "(none)" + end. + +escape([]) -> []; +escape([$;|R]) -> [$\\,$;|escape(R)]; +escape([$&|R]) -> [$\\,$&|escape(R)]; +escape([$(|R]) -> [$\\,$(|escape(R)]; +escape([$)|R]) -> [$\\,$)|escape(R)]; +escape([$||R]) -> [$\\,$||escape(R)]; +escape([$^|R]) -> [$\\,$^|escape(R)]; +escape([$<|R]) -> [$\\,$<|escape(R)]; +escape([$>|R]) -> [$\\,$>|escape(R)]; +escape([$\n|R]) -> [$\\,$\n|escape(R)]; +escape([$ |R]) -> [$\\,$ |escape(R)]; +escape([$\t|R]) -> [$\\,$\t|escape(R)]; +escape([C|R]) -> [C|escape(R)]. + +date_local() -> + {{Year,Month,Day},{Hour,Minute,Second}}=calendar:local_time(), + %% Time format hard-wired to: "%a %b %e %T %Y" according to strftime(3) + io_lib:format("~s ~s ~2w ~2.2.0w:~2.2.0w:~2.2.0w ~w", + [httpd_util:day(calendar:day_of_the_week(Year,Month,Day)), + httpd_util:month(Month),Day,Hour,Minute,Second,Year]). + +date_gmt() -> + {{Year,Month,Day},{Hour,Minute,Second}}=calendar:universal_time(), + %% Time format hard-wired to: "%a %b %e %T %Z %Y" according to strftime(3) + io_lib:format("~s ~s ~2w ~2.2.0w:~2.2.0w:~2.2.0w GMT ~w", + [httpd_util:day(calendar:day_of_the_week(Year,Month,Day)), + httpd_util:month(Month),Day,Hour,Minute,Second,Year]). + +last_modified(Data,ConfigDB,RequestURI) -> + {ok,FileInfo}=file:read_file_info(mod_alias:path(Data,ConfigDB,RequestURI)), + {{Year,Month,Day},{Hour,Minute,Second}}=FileInfo#file_info.mtime, + io_lib:format("~s ~s ~2w ~2.2.0w:~2.2.0w:~2.2.0w ~w", + [httpd_util:day(calendar:day_of_the_week(Year,Month,Day)), + httpd_util:month(Month),Day,Hour,Minute,Second,Year]). + +%% +%% fsize directive +%% + +fsize(Info,Context,ErrorLog,[virtual],[VirtualPath],R) -> + Aliases = httpd_util:multi_lookup(Info#mod.config_db,alias), + {_,Path, _AfterPath}= + mod_alias:real_name(Info#mod.config_db,VirtualPath,Aliases), + fsize(Info, Context, ErrorLog, R, Path); +fsize(Info,Context,ErrorLog,[file],[FileName],R) -> + Path = file(Info#mod.config_db,Info#mod.request_uri,FileName), + fsize(Info,Context,ErrorLog,R,Path); +fsize(_Info, Context, ErrorLog, _TagList, _ValueList, R) -> + {ok,Context,[{internal_info,?NICE("fsize directive has a spurious tag")}| + ErrorLog],proplists:get_value(errmsg,Context,""),R}. + +fsize(_Info, Context, ErrorLog, R, Path) -> + case file:read_file_info(Path) of + {ok,FileInfo} -> + case proplists:get_value(sizefmt,Context) of + "bytes" -> + {ok,Context,ErrorLog, + integer_to_list(FileInfo#file_info.size),R}; + "abbrev" -> + Size = integer_to_list(trunc(FileInfo#file_info.size/1024+1))++"k", + {ok,Context,ErrorLog,Size,R}; + Value-> + {ok,Context, + [{internal_info, + ?NICE("fsize directive has a spurious tag value ("++ + Value++")")}| + ErrorLog], + proplists:get_value(errmsg, Context, ""), R} + end; + {error, _Reason} -> + {ok,Context,[{internal_info,?NICE("Can't open "++Path)}|ErrorLog], + proplists:get_value(errmsg,Context,""),R} + end. + +%% +%% flastmod directive +%% + +flastmod(#mod{config_db = Db} = Info, + Context, ErrorLog, [virtual], [VirtualPath],R) -> + Aliases = httpd_util:multi_lookup(Db,alias), + {_,Path, _AfterPath} = mod_alias:real_name(Db, VirtualPath, Aliases), + flastmod(Info,Context,ErrorLog,R,Path); +flastmod(#mod{config_db = Db, request_uri = RequestUri} = Info, + Context, ErrorLog, [file], [FileName], R) -> + Path = file(Db, RequestUri, FileName), + flastmod(Info, Context, ErrorLog, R, Path); +flastmod(_Info, Context, ErrorLog, _TagList, _ValueList, R) -> + {ok,Context, + [{internal_info,?NICE("flastmod directive has a spurious tag")}| + ErrorLog],proplists:get_value(errmsg,Context,""),R}. + +flastmod(_Info, Context, ErrorLog, R, File) -> + case file:read_file_info(File) of + {ok, FileInfo} -> + {{Yr,Mon,Day},{Hour,Minute,Second}}=FileInfo#file_info.mtime, + Result = + io_lib:format("~s ~s ~2w ~w:~w:~w ~w", + [httpd_util:day( + calendar:day_of_the_week(Yr,Mon, Day)), + httpd_util:month(Mon),Day,Hour,Minute,Second, Yr]), + {ok, Context, ErrorLog, Result, R}; + {error, _Reason} -> + {ok,Context,[{internal_info,?NICE("Can't open "++File)}|ErrorLog], + proplists:get_value(errmsg,Context,""),R} + end. + +%% +%% exec directive +%% + +exec(Info,Context,ErrorLog,[cmd],[Command],R) -> + cmd(Info,Context,ErrorLog,R,Command); +exec(Info,Context,ErrorLog,[cgi],[RequestURI],R) -> + cgi(Info,Context,ErrorLog,R,RequestURI); +exec(_Info, Context, ErrorLog, _TagList, _ValueList, R) -> + {ok, Context, + [{internal_info,?NICE("exec directive has a spurious tag")}| + ErrorLog], proplists:get_value(errmsg,Context,""),R}. + +%% cmd + +cmd(Info, Context, ErrorLog, R, Command) -> + process_flag(trap_exit,true), + Env = env(Info), + Dir = filename:dirname(Command), + Port = (catch open_port({spawn,Command},[stream,{cd,Dir},{env,Env}])), + case Port of + P when is_port(P) -> + {NewErrorLog, Result} = proxy(Port, ErrorLog), + {ok, Context, NewErrorLog, Result, R}; + {'EXIT', Reason} -> + exit({open_port_failed,Reason, + [{uri,Info#mod.request_uri},{script,Command}, + {env,Env},{dir,Dir}]}); + O -> + exit({open_port_failed,O, + [{uri,Info#mod.request_uri},{script,Command}, + {env,Env},{dir,Dir}]}) + end. + +env(Info) -> + [{"DOCUMENT_NAME",document_name(Info#mod.data,Info#mod.config_db, + Info#mod.request_uri)}, + {"DOCUMENT_URI", document_uri(Info#mod.config_db, Info#mod.request_uri)}, + {"QUERY_STRING_UNESCAPED", query_string_unescaped(Info#mod.request_uri)}, + {"DATE_LOCAL", date_local()}, + {"DATE_GMT", date_gmt()}, + {"LAST_MODIFIED", last_modified(Info#mod.data, Info#mod.config_db, + Info#mod.request_uri)} + ]. + +%% cgi + +cgi(Info, Context, ErrorLog, R, RequestURI) -> + ScriptAliases = httpd_util:multi_lookup(Info#mod.config_db, script_alias), + case mod_alias:real_script_name(Info#mod.config_db, RequestURI, + ScriptAliases) of + {Script, AfterScript} -> + exec_script(Info,Script,AfterScript,ErrorLog,Context,R); + not_a_script -> + {ok, Context, + [{internal_info, ?NICE(RequestURI++" is not a script")}| + ErrorLog], proplists:get_value(errmsg, Context, ""),R} + end. + +remove_header([]) -> + []; +remove_header([$\n,$\n|Rest]) -> + Rest; +remove_header([_C|Rest]) -> + remove_header(Rest). + + +exec_script(#mod{config_db = Db, request_uri = RequestUri} = Info, + Script, _AfterScript, ErrorLog, Context, R) -> + process_flag(trap_exit,true), + Aliases = httpd_util:multi_lookup(Db, alias), + {_, Path, AfterPath} = mod_alias:real_name(Db, RequestUri, Aliases), + Env = env(Info) ++ mod_cgi:env(Info, Path, AfterPath), + Dir = filename:dirname(Path), + Port = (catch open_port({spawn,Script},[stream,{env, Env},{cd, Dir}])), + case Port of + P when is_port(P) -> + %% Send entity body to port. + Res = case Info#mod.entity_body of + [] -> + true; + EntityBody -> + (catch port_command(Port, EntityBody)) + end, + case Res of + {'EXIT', Reason} -> + exit({open_cmd_failed,Reason, + [{mod,?MODULE},{port,Port}, + {uri,RequestUri}, + {script,Script},{env,Env},{dir,Dir}, + {ebody_size,sz(Info#mod.entity_body)}]}); + true -> + {NewErrorLog, Result} = proxy(Port, ErrorLog), + {ok, Context, NewErrorLog, remove_header(Result), R} + end; + {'EXIT', Reason} -> + exit({open_port_failed,Reason, + [{mod,?MODULE},{uri,RequestUri},{script,Script}, + {env,Env},{dir,Dir}]}); + O -> + exit({open_port_failed,O, + [{mod,?MODULE},{uri,RequestUri},{script,Script}, + {env,Env},{dir,Dir}]}) + end. + + +%% +%% Port communication +%% + +proxy(Port, ErrorLog) -> + process_flag(trap_exit, true), + proxy(Port, ErrorLog, []). + +proxy(Port, ErrorLog, Result) -> + receive + {Port, {data, Response}} -> + proxy(Port, ErrorLog, lists:append(Result,Response)); + {'EXIT', Port, normal} when is_port(Port) -> + process_flag(trap_exit, false), + {ErrorLog, Result}; + {'EXIT', Port, _Reason} when is_port(Port) -> + process_flag(trap_exit, false), + {[{internal_info, + ?NICE("Scrambled output from CGI-script")}|ErrorLog], + Result}; + {'EXIT', Pid, Reason} when is_pid(Pid) -> + process_flag(trap_exit, false), + {'EXIT', Pid, Reason}; + %% This should not happen! + _WhatEver -> + process_flag(trap_exit, false), + {ErrorLog, Result} + end. + + +%% ------ +%% Temporary until I figure out a way to fix send_in_chunks +%% (comments and directives that start in one chunk but end +%% in another is not handled). +%% + +send_in(Info, Path, Head, {ok,FileInfo}) -> + case file:read_file(Path) of + {ok, Bin} -> + send_in1(Info, binary_to_list(Bin), Head, FileInfo); + {error, Reason} -> + {error, {read,Reason}} + end; +send_in(_Info , _Path, _Head,{error,Reason}) -> + {error, {open,Reason}}. + +send_in1(Info, Data, Head, FileInfo) -> + {ok, _Context, Err, ParsedBody} = parse(Info,Data,?DEFAULT_CONTEXT,[],[]), + Size = length(ParsedBody), + LastModified = case catch httpd_util:rfc1123_date(FileInfo#file_info.mtime) of + Date when is_list(Date) -> [{last_modified,Date}]; + _ -> [] + end, + Head1 = case Info#mod.http_version of + "HTTP/1.1"-> + Head ++ [{content_length, integer_to_list(Size)}, + {etag, httpd_util:create_etag(FileInfo,Size)}| + LastModified]; + _-> + %% i.e http/1.0 and http/0.9 + Head ++ [{content_length, integer_to_list(Size)}| + LastModified] + end, + httpd_response:send_header(Info, 200, Head1), + httpd_socket:deliver(Info#mod.socket_type,Info#mod.socket, ParsedBody), + {ok, Err, Size}. + + +parse(Info,Body) -> + parse(Info, Body, ?DEFAULT_CONTEXT, [], []). + +parse(_Info, [], Context, ErrorLog, Result) -> + {ok, Context, lists:reverse(ErrorLog), lists:reverse(Result)}; +parse(Info,[$<,$!,$-,$-,$#|R1],Context,ErrorLog,Result) -> + case catch parse0(R1,Context) of + {parse_error,Reason} -> + parse(Info,R1,Context,[{internal_info,?NICE(Reason)}|ErrorLog], + [$#,$-,$-,$!,$<|Result]); + {ok,Context,Command,TagList,ValueList,R2} -> + {ok,NewContext,NewErrorLog,MoreResult,R3}= + handle(Info,Context,ErrorLog,Command,TagList,ValueList,R2), + parse(Info,R3,NewContext,NewErrorLog, + lists:reverse(MoreResult)++Result) + end; +parse(Info,[$<,$!,$-,$-|R1],Context,ErrorLog,Result) -> + case catch parse5(R1,[],0) of + {parse_error,Reason} -> + parse(Info,R1,Context, + [{internal_info,?NICE(Reason)}|ErrorLog],Result); + {Comment,R2} -> + parse(Info,R2,Context,ErrorLog,Comment++Result) + end; +parse(Info,[C|R],Context,ErrorLog,Result) -> + parse(Info,R,Context,ErrorLog,[C|Result]). + +handle(Info,Context,ErrorLog,Command,TagList,ValueList,R) -> + case catch apply(?MODULE,Command,[Info,Context,ErrorLog,TagList,ValueList, + R]) of + {'EXIT',{undef,_}} -> + throw({parse_error,"Unknown command "++atom_to_list(Command)++ + " in parsed doc"}); + Result -> + Result + end. + +parse0([], _Context) -> + throw({parse_error,"Premature EOF in parsed file"}); +parse0([$-,$-,$>|_R], _Context) -> + throw({parse_error,"Premature EOF in parsed file"}); +parse0([$ |R], Context) -> + parse0(R,Context); +parse0(String, Context) -> + parse1(String, Context,""). + +parse1([], _Context, _Command) -> + throw({parse_error,"Premature EOF in parsed file"}); +parse1([$-,$-,$>|_R], _Context, _Command) -> + throw({parse_error,"Premature EOF in parsed file"}); +parse1([$ |R], Context, Command) -> + parse2(R,Context,list_to_atom(lists:reverse(Command)),[],[],""); +parse1([C|R], Context, Command) -> + parse1(R,Context,[C|Command]). + +parse2([], _Context, _Command, _TagList, _ValueList, _Tag) -> + throw({parse_error,"Premature EOF in parsed file"}); +parse2([$-,$-,$>|R], Context, Command, TagList, ValueList, _Tag) -> + {ok,Context,Command,TagList,ValueList,R}; +parse2([$ |R],Context,Command,TagList,ValueList,Tag) -> + parse2(R,Context,Command,TagList,ValueList,Tag); +parse2([$=|R],Context,Command,TagList,ValueList,Tag) -> + parse3(R,Context,Command,[list_to_atom(lists:reverse(Tag))|TagList], + ValueList); +parse2([C|R],Context,Command,TagList,ValueList,Tag) -> + parse2(R,Context,Command,TagList,ValueList,[C|Tag]). + +parse3([], _Context, _Command, _TagList, _ValueList) -> + throw({parse_error,"Premature EOF in parsed file"}); +parse3([$-,$-,$>|_R], _Context, _Command, _TagList, _ValueList) -> + throw({parse_error,"Premature EOF in parsed file"}); +parse3([$ |R], Context, Command, TagList, ValueList) -> + parse3(R, Context, Command, TagList, ValueList); +parse3([$"|R], Context, Command, TagList, ValueList) -> + parse4(R,Context,Command,TagList,ValueList,""); +parse3(_String, _Context, _Command, _TagList, _ValueList) -> + throw({parse_error,"Premature EOF in parsed file"}). + +parse4([], _Context, _Command, _TagList, _ValueList, _Value) -> + throw({parse_error,"Premature EOF in parsed file"}); +parse4([$-,$-,$>|_R], _Context, _Command, _TagList, _ValueList, _Value) -> + throw({parse_error,"Premature EOF in parsed file"}); +parse4([$"|R],Context,Command,TagList,ValueList,Value) -> + parse2(R,Context,Command,TagList,[lists:reverse(Value)|ValueList],""); +parse4([C|R],Context,Command,TagList,ValueList,Value) -> + parse4(R,Context,Command,TagList,ValueList,[C|Value]). + +parse5([], _Comment, _Depth) -> + throw({parse_error,"Premature EOF in parsed file"}); +parse5([$<,$!,$-,$-|R],Comment,Depth) -> + parse5(R,[$-,$-,$!,$<|Comment],Depth+1); +parse5([$-,$-,$>|R],Comment,0) -> + {">--"++Comment++"--!<",R}; +parse5([$-,$-,$>|R],Comment,Depth) -> + parse5(R,[$>,$-,$-|Comment],Depth-1); +parse5([C|R],Comment,Depth) -> + parse5(R,[C|Comment],Depth). + + +sz(B) when is_binary(B) -> {binary,size(B)}; +sz(L) when is_list(L) -> {list,length(L)}; +sz(_) -> undefined. + +%% send_error - Handle failure to send the file +%% +send_error({open,Reason},Info,Path) -> + httpd_file:handle_error(Reason, "open", Info, Path); +send_error({read,Reason},Info,Path) -> + httpd_file:handle_error(Reason, "read", Info, Path). + + + + diff --git a/lib/inets/src/http_server/mod_log.erl b/lib/inets/src/http_server/mod_log.erl new file mode 100644 index 0000000000..de24d5a569 --- /dev/null +++ b/lib/inets/src/http_server/mod_log.erl @@ -0,0 +1,256 @@ +%% +%% %CopyrightBegin% +%% +%% Copyright Ericsson AB 1997-2009. All Rights Reserved. +%% +%% The contents of this file are subject to the Erlang Public License, +%% Version 1.1, (the "License"); you may not use this file except in +%% compliance with the License. You should have received a copy of the +%% Erlang Public License along with this software. If not, it can be +%% retrieved online at http://www.erlang.org/. +%% +%% Software distributed under the License is distributed on an "AS IS" +%% basis, WITHOUT WARRANTY OF ANY KIND, either express or implied. See +%% the License for the specific language governing rights and limitations +%% under the License. +%% +%% %CopyrightEnd% +%% +%% +-module(mod_log). + +%% Application internal API +-export([error_log/2, security_log/2, report_error/2]). + +%% Callback API +-export([do/1, load/2, store/2, remove/1]). + +-include("httpd.hrl"). +-define(VMODULE,"LOG"). + +%%%========================================================================= +%%% API +%%%========================================================================= + +%% security log +security_log(Info, ReasonStr) -> + Date = httpd_util:custom_date(), + case httpd_log:security_entry(security_log, no_security_log, Info, + Date, ReasonStr) of + no_security_log -> + ok; + {Log, Entry} -> + io:format(Log, "~s", [Entry]) + end. + +%% error_log +error_log(Info, Reason) -> + Date = httpd_util:custom_date(), + error_log(Info, Date, Reason). + +error_log(Info, Date, Reason) -> + case httpd_log:error_entry(error_log, no_error_log, + Info, Date, Reason) of + no_error_log -> + ok; + {Log, Entry} -> + io:format(Log, "~s", [Entry]) + end. + +report_error(ConfigDB, Error) -> + Date = httpd_util:custom_date(), + case httpd_log:error_report_entry(error_log, no_error_log, ConfigDB, + Date, Error) of + no_error_log -> + ok; + {Log, Entry} -> + io:format(Log, "~s", [Entry]) + end. + +%%%========================================================================= +%%% CALLBACK API +%%%========================================================================= +%%-------------------------------------------------------------------------- +%% do(ModData) -> {proceed, OldData} | {proceed, NewData} | {break, NewData} +%% | done +%% ModData = #mod{} +%% +%% Description: See httpd(3) ESWAPI CALLBACK FUNCTIONS +%%------------------------------------------------------------------------- +do(Info) -> + AuthUser = auth_user(Info#mod.data), + Date = httpd_util:custom_date(), + log_internal_info(Info,Date,Info#mod.data), + case proplists:get_value(status, Info#mod.data) of + %% A status code has been generated! + {StatusCode, _PhraseArgs, Reason} -> + transfer_log(Info,"-",AuthUser,Date,StatusCode,0), + if + StatusCode >= 400 -> + error_log(Info,Date,Reason); + true -> + not_an_error + end, + {proceed,Info#mod.data}; + %% No status code has been generated! + undefined -> + case proplists:get_value(response, Info#mod.data) of + {already_sent,StatusCode,Size} -> + transfer_log(Info,"-",AuthUser,Date,StatusCode,Size), + {proceed,Info#mod.data}; + {response, Head, _Body} -> + Size = proplists:get_value(content_length,Head,unknown), + Code = proplists:get_value(code,Head,unknown), + transfer_log(Info, "-", AuthUser, Date, Code, Size), + {proceed, Info#mod.data}; + {_StatusCode, Response} -> + transfer_log(Info,"-",AuthUser,Date,200, + httpd_util:flatlength(Response)), + {proceed,Info#mod.data}; + undefined -> + transfer_log(Info,"-",AuthUser,Date,200,0), + {proceed,Info#mod.data} + end + end. + +%%-------------------------------------------------------------------------- +%% load(Line, Context) -> eof | ok | {ok, NewContext} | +%% {ok, NewContext, Directive} | +%% {ok, NewContext, DirectiveList} | {error, Reason} +%% Line = string() +%% Context = NewContext = DirectiveList = [Directive] +%% Directive = {DirectiveKey , DirectiveValue} +%% DirectiveKey = DirectiveValue = term() +%% Reason = term() +%% +%% Description: See httpd(3) ESWAPI CALLBACK FUNCTIONS +%%------------------------------------------------------------------------- +load("TransferLog " ++ TransferLog, []) -> + {ok,[],{transfer_log,httpd_conf:clean(TransferLog)}}; +load("ErrorLog " ++ ErrorLog, []) -> + {ok,[],{error_log,httpd_conf:clean(ErrorLog)}}; +load("SecurityLog " ++ SecurityLog, []) -> + {ok, [], {security_log, httpd_conf:clean(SecurityLog)}}. + +%%-------------------------------------------------------------------------- +%% store(Directive, DirectiveList) -> {ok, NewDirective} | +%% {ok, [NewDirective]} | +%% {error, Reason} +%% Directive = {DirectiveKey , DirectiveValue} +%% DirectiveKey = DirectiveValue = term() +%% Reason = term() +%% +%% Description: See httpd(3) ESWAPI CALLBACK FUNCTIONS +%%------------------------------------------------------------------------- +store({transfer_log,TransferLog}, ConfigList) when is_list(TransferLog)-> + case create_log(TransferLog,ConfigList) of + {ok,TransferLogStream} -> + {ok,{transfer_log,TransferLogStream}}; + {error,Reason} -> + {error,Reason} + end; +store({transfer_log,TransferLog}, _) -> + {error, {wrong_type, {transfer_log, TransferLog}}}; +store({error_log,ErrorLog}, ConfigList) when is_list(ErrorLog) -> + case create_log(ErrorLog,ConfigList) of + {ok,ErrorLogStream} -> + {ok,{error_log,ErrorLogStream}}; + {error,Reason} -> + {error,Reason} + end; +store({error_log,ErrorLog}, _) -> + {error, {wrong_type, {error_log, ErrorLog}}}; +store({security_log, SecurityLog}, ConfigList) when is_list(SecurityLog) -> + case create_log(SecurityLog, ConfigList) of + {ok, SecurityLogStream} -> + {ok, {security_log, SecurityLogStream}}; + {error, Reason} -> + {error, Reason} + end; +store({security_log, SecurityLog}, _) -> + {error, {wrong_type, {security_log, SecurityLog}}}. + +%%-------------------------------------------------------------------------- +%% remove(ConfigDb) -> _ +%% +%% Description: See httpd(3) ESWAPI CALLBACK FUNCTIONS +%%------------------------------------------------------------------------- +remove(ConfigDB) -> + lists:foreach(fun([Stream]) -> file:close(Stream) end, + ets:match(ConfigDB,{transfer_log,'$1'})), + lists:foreach(fun([Stream]) -> file:close(Stream) end, + ets:match(ConfigDB,{error_log,'$1'})), + lists:foreach(fun([Stream]) -> file:close(Stream) end, + ets:match(ConfigDB,{security_log,'$1'})), + ok. + +%%%======================================================================== +%%% Internal functions +%%%======================================================================== +%% transfer_log +transfer_log(Info,RFC931,AuthUser,Date,StatusCode,Bytes) -> + case httpd_log:access_entry(transfer_log, no_transfer_log, + Info, RFC931, AuthUser, Date, + StatusCode, Bytes) of + no_transfer_log -> + ok; + {Log, Entry} -> + io:format(Log, "~s", [Entry]) + end. + +create_log(LogFile, ConfigList) -> + Filename = httpd_conf:clean(LogFile), + case filename:pathtype(Filename) of + absolute -> + case file:open(Filename, [read, write]) of + {ok,LogStream} -> + file:position(LogStream,{eof,0}), + {ok,LogStream}; + {error,_} -> + {error,?NICE("Can't create "++Filename)} + end; + volumerelative -> + case file:open(Filename, [read, write]) of + {ok,LogStream} -> + file:position(LogStream,{eof,0}), + {ok,LogStream}; + {error,_} -> + {error,?NICE("Can't create "++Filename)} + end; + relative -> + case proplists:get_value(server_root,ConfigList) of + undefined -> + {error, + ?NICE(Filename++ + " is an invalid logfile name beacuse " + "ServerRoot is not defined")}; + ServerRoot -> + AbsoluteFilename=filename:join(ServerRoot,Filename), + case file:open(AbsoluteFilename, [read, write]) of + {ok,LogStream} -> + file:position(LogStream,{eof,0}), + {ok,LogStream}; + {error, _Reason} -> + {error,?NICE("Can't create "++AbsoluteFilename)} + end + end + end. + +%% log_internal_info +log_internal_info(_Info, _Date, []) -> + ok; +log_internal_info(Info,Date,[{internal_info,Reason}|Rest]) -> + error_log(Info, Date, Reason), + log_internal_info(Info,Date,Rest); +log_internal_info(Info,Date,[_|Rest]) -> + log_internal_info(Info,Date,Rest). + +auth_user(Data) -> + case proplists:get_value(remote_user, Data) of + undefined -> + "-"; + RemoteUser -> + RemoteUser + end. + + diff --git a/lib/inets/src/http_server/mod_range.erl b/lib/inets/src/http_server/mod_range.erl new file mode 100644 index 0000000000..0698fb9099 --- /dev/null +++ b/lib/inets/src/http_server/mod_range.erl @@ -0,0 +1,419 @@ +%% +%% %CopyrightBegin% +%% +%% Copyright Ericsson AB 2001-2009. All Rights Reserved. +%% +%% The contents of this file are subject to the Erlang Public License, +%% Version 1.1, (the "License"); you may not use this file except in +%% compliance with the License. You should have received a copy of the +%% Erlang Public License along with this software. If not, it can be +%% retrieved online at http://www.erlang.org/. +%% +%% Software distributed under the License is distributed on an "AS IS" +%% basis, WITHOUT WARRANTY OF ANY KIND, either express or implied. See +%% the License for the specific language governing rights and limitations +%% under the License. +%% +%% %CopyrightEnd% +%% +%% +-module(mod_range). +-export([do/1]). +-include("httpd.hrl"). + +%% do + +do(Info) -> + ?DEBUG("do -> entry",[]), + case Info#mod.method of + "GET" -> + case proplists:get_value(status, Info#mod.data) of + %% A status code has been generated! + {_StatusCode, _PhraseArgs, _Reason} -> + {proceed,Info#mod.data}; + %% No status code has been generated! + undefined -> + case proplists:get_value(response, Info#mod.data) of + %% No response has been generated! + undefined -> + case proplists:get_value("range", + Info#mod.parsed_header) of + undefined -> + %Not a range response + {proceed,Info#mod.data}; + Range -> + %%Control that there weren't a + %%if-range field that stopped The + %%range request in favor for the + %%whole file + case proplists:get_value(if_range, + Info#mod.data) of + send_file -> + {proceed,Info#mod.data}; + _undefined -> + do_get_range(Info,Range) + end + end; + %% A response has been generated or sent! + _Response -> + {proceed, Info#mod.data} + end + end; + %% Not a GET method! + _ -> + {proceed,Info#mod.data} + end. + +do_get_range(Info,Ranges) -> + ?DEBUG("do_get_range -> Request URI: ~p",[Info#mod.request_uri]), + Path = mod_alias:path(Info#mod.data, Info#mod.config_db, + Info#mod.request_uri), + {FileInfo, LastModified} = get_modification_date(Path), + send_range_response(Path, Info, Ranges, FileInfo, LastModified). + + +send_range_response(Path, Info, Ranges, FileInfo, LastModified)-> + case parse_ranges(Ranges) of + error-> + ?ERROR("send_range_response-> Unparsable range request",[]), + {proceed,Info#mod.data}; + {multipart,RangeList}-> + send_multi_range_response(Path, Info, RangeList); + {Start,Stop}-> + send_range_response(Path, Info, Start, Stop, FileInfo, + LastModified) + end. +%%More than one range specified +%%Send a multipart reponse to the user +% +%%An example of an multipart range response + +% HTTP/1.1 206 Partial Content +% Date:Wed 15 Nov 1995 04:08:23 GMT +% Last-modified:Wed 14 Nov 1995 04:08:23 GMT +% Content-type: multipart/byteranges; boundary="SeparatorString" +% +% --"SeparatorString" +% Content-Type: application/pdf +% Content-Range: bytes 500-600/1010 +% .... The data..... 101 bytes +% +% --"SeparatorString" +% Content-Type: application/pdf +% Content-Range: bytes 700-1009/1010 +% .... The data..... + + + +send_multi_range_response(Path,Info,RangeList)-> + case file:open(Path, [raw,binary]) of + {ok, FileDescriptor} -> + file:close(FileDescriptor), + ?DEBUG("send_multi_range_response -> FileDescriptor: ~p", + [FileDescriptor]), + Suffix = httpd_util:suffix(Path), + PartMimeType = httpd_util:lookup_mime_default(Info#mod.config_db, + Suffix,"text/plain"), + {FileInfo, LastModified} = get_modification_date(Path), + case valid_ranges(RangeList,Path,FileInfo) of + {ValidRanges,true}-> + ?DEBUG("send_multi_range_response ->Ranges are valid:",[]), + %Apache breaks the standard by sending the size + %field in the Header. + Header = + [{code,206}, + {content_type, "multipart/byteranges;boundary" + "=RangeBoundarySeparator"}, + {etag, httpd_util:create_etag(FileInfo)} | + LastModified], + ?DEBUG("send_multi_range_response -> Valid Ranges: ~p", + [RagneList]), + Body = {fun send_multiranges/4, + [ValidRanges, Info, PartMimeType, Path]}, + {proceed,[{response, + {response, Header, Body}} | Info#mod.data]}; + _ -> + {proceed, [{status, {416, "Range not valid", + bad_range_boundaries }}]} + end; + {error, _Reason} -> + ?ERROR("do_get -> failed open file: ~p",[_Reason]), + {proceed,Info#mod.data} + end. + +send_multiranges(ValidRanges,Info,PartMimeType,Path)-> + ?DEBUG("send_multiranges -> Start sending the ranges",[]), + case file:open(Path, [raw,binary]) of + {ok,FileDescriptor} -> + lists:foreach(fun(Range)-> + send_multipart_start(Range, + Info, + PartMimeType, + FileDescriptor) + end,ValidRanges), + file:close(FileDescriptor), + %%Sends an end of the multipart + httpd_socket:deliver(Info#mod.socket_type,Info#mod.socket, + "\r\n--RangeBoundarySeparator--"), + sent; + _ -> + close + end. + +send_multipart_start({{Start,End},{StartByte,EndByte,Size}},Info, + PartMimeType,FileDescriptor) + when StartByte < Size -> + PartHeader=["\r\n--RangeBoundarySeparator\r\n","Content-type: ", + PartMimeType,"\r\n", + "Content-Range:bytes=",integer_to_list(StartByte),"-", + integer_to_list(EndByte),"/", + integer_to_list(Size),"\r\n\r\n"], + send_part_start(Info#mod.socket_type, Info#mod.socket, PartHeader, + FileDescriptor, Start, End); + + +send_multipart_start({{Start,End},{StartByte,EndByte,Size}}, Info, + PartMimeType, FileDescriptor)-> + PartHeader=["\r\n--RangeBoundarySeparator\r\n","Content-type: ", + PartMimeType,"\r\n", + "Content-Range:bytes=",integer_to_list(Size-(StartByte-Size)), + "-",integer_to_list(EndByte),"/", + integer_to_list(Size),"\r\n\r\n"], + send_part_start(Info#mod.socket_type, Info#mod.socket, PartHeader, + FileDescriptor, Start, End). + +send_part_start(SocketType, Socket, PartHeader, FileDescriptor, Start, End)-> + case httpd_socket:deliver(SocketType, Socket, PartHeader) of + ok -> + send_part_start(SocketType,Socket,FileDescriptor,Start,End); + _ -> + close + end. + +send_range_response(Path, Info, Start, Stop, FileInfo, LastModified)-> + case file:open(Path, [raw,binary]) of + {ok, FileDescriptor} -> + file:close(FileDescriptor), + ?DEBUG("send_range_response -> FileDescriptor: ~p", + [FileDescriptor]), + Suffix = httpd_util:suffix(Path), + MimeType = httpd_util:lookup_mime_default(Info#mod.config_db, + Suffix,"text/plain"), + Size = get_range_size(Start,Stop,FileInfo), + case valid_range(Start,Stop,FileInfo) of + {true,StartByte,EndByte,TotByte}-> + Head =[{code,206},{content_type, MimeType}, + {etag, httpd_util:create_etag(FileInfo)}, + {content_range,["bytes=", + integer_to_list(StartByte),"-", + integer_to_list(EndByte),"/", + integer_to_list(TotByte)]}, + {content_length, Size} | LastModified], + BodyFunc = fun send_range_body/5, + Arg = [Info#mod.socket_type, + Info#mod.socket, Path, Start, Stop], + {proceed,[{response,{response ,Head, {BodyFunc,Arg}}}| + Info#mod.data]}; + {false,Reason} -> + {proceed, [{status, {416, Reason, bad_range_boundaries }}]} + end; + {error, _Reason} -> + ?ERROR("send_range_response -> failed open file: ~p",[_Reason]), + {proceed,Info#mod.data} + end. + + +send_range_body(SocketType,Socket,Path,Start,End) -> + ?DEBUG("mod_range -> send_range_body",[]), + case file:open(Path, [raw,binary]) of + {ok,FileDescriptor} -> + send_part_start(SocketType,Socket,FileDescriptor,Start,End), + file:close(FileDescriptor); + _ -> + close + end. + +send_part_start(SocketType,Socket,FileDescriptor,Start,End) -> + case Start of + from_end -> + file:position(FileDescriptor,{eof,End}), + send_body(SocketType,Socket,FileDescriptor); + from_start -> + file:position(FileDescriptor,{bof,End}), + send_body(SocketType,Socket,FileDescriptor); + Byte when is_integer(Byte) -> + file:position(FileDescriptor,{bof,Start}), + send_part(SocketType,Socket,FileDescriptor,End) + end, + sent. + + +%%This function could replace send_body by calling it with Start=0 end +%%=FileSize But i gues it would be stupid when we look at performance +send_part(SocketType,Socket,FileDescriptor,End)-> + case file:position(FileDescriptor,{cur,0}) of + {ok,NewPos} -> + if + NewPos > End -> + ok; + true -> + Size = get_file_chunk_size(NewPos,End,?FILE_CHUNK_SIZE), + case file:read(FileDescriptor,Size) of + eof -> + ok; + {error, _Reason} -> + ok; + {ok,Binary} -> + case httpd_socket:deliver(SocketType,Socket, + Binary) of + socket_closed -> + ?LOG("send_range of body -> socket " + "closed while sending",[]), + socket_close; + _ -> + send_part(SocketType,Socket, + FileDescriptor,End) + end + end + end; + _-> + ok + end. + +%% validate that the range is in the limits of the file +valid_ranges(RangeList, _Path, FileInfo)-> + lists:mapfoldl(fun({Start,End},Acc)-> + case Acc of + true -> + case valid_range(Start,End,FileInfo) of + {true,StartB,EndB,Size}-> + {{{Start,End}, + {StartB,EndB,Size}},true}; + _ -> + false + end; + _ -> + {false,false} + end + end,true,RangeList). + + + +valid_range(from_end,End,FileInfo)-> + Size=FileInfo#file_info.size, + if + End < Size -> + {true,(Size+End),Size-1,Size}; + true -> + false + end; +valid_range(from_start,End,FileInfo)-> + Size=FileInfo#file_info.size, + if + End < Size -> + {true,End,Size-1,Size}; + true -> + false + end; + +valid_range(Start,End,FileInfo) when Start =< End -> + case FileInfo#file_info.size of + FileSize when Start< FileSize -> + case FileInfo#file_info.size of + Size when End<Size -> + {true,Start,End,FileInfo#file_info.size}; + Size -> + {true,Start,Size-1,Size} + end; + _-> + {false,"The size of the range is negative"} + end; + +valid_range(_Start,_End,_FileInfo)-> + {false,"Range starts out of file boundaries"}. +%% Find the modification date of the file +get_modification_date(Path)-> + case file:read_file_info(Path) of + {ok, FileInfo0} -> + case (catch httpd_util:rfc1123_date(FileInfo0#file_info.mtime)) of + Date when is_list(Date) -> + {FileInfo0, [{last_modified, Date}]}; + _ -> + {FileInfo0, []} + end; + _ -> + {#file_info{}, []} + end. + +%Calculate the size of the chunk to read + +get_file_chunk_size(Position, End, DefaultChunkSize) + when (Position+DefaultChunkSize) =< End -> + DefaultChunkSize; +get_file_chunk_size(Position, End, _DefaultChunkSize) -> + (End-Position) +1. + + + +%Get the size of the range to send. Remember that +%A range is from startbyte up to endbyte which means that +%the nuber of byte in a range is (StartByte-EndByte)+1 + +get_range_size(from_end, Stop, _FileInfo)-> + integer_to_list(-1*Stop); + +get_range_size(from_start, StartByte, FileInfo) -> + integer_to_list((((FileInfo#file_info.size)-StartByte))); + +get_range_size(StartByte, EndByte, _FileInfo) -> + integer_to_list((EndByte-StartByte)+1). + +parse_ranges("\bytes\=" ++ Ranges)-> + parse_ranges("bytes\=" ++ Ranges); +parse_ranges("bytes\=" ++ Ranges)-> + case string:tokens(Ranges,", ") of + [Range] -> + parse_range(Range); + [Range1|SplittedRanges]-> + {multipart,lists:map(fun parse_range/1,[Range1|SplittedRanges])} + end; +%Bad unit +parse_ranges(Ranges)-> + io:format("Bad Ranges : ~p",[Ranges]), + error. +%Parse the range specification from the request to {Start,End} +%Start=End : Numreric string | [] + +parse_range(Range)-> + format_range(split_range(Range,[],[])). +format_range({[],BytesFromEnd})-> + {from_end,-1*(list_to_integer(BytesFromEnd))}; +format_range({StartByte,[]})-> + {from_start,list_to_integer(StartByte)}; +format_range({StartByte,EndByte})-> + {list_to_integer(StartByte),list_to_integer(EndByte)}. +%Last case return the splitted range +split_range([],Current,Other)-> + {lists:reverse(Other),lists:reverse(Current)}; + +split_range([$-|Rest],Current,Other)-> + split_range(Rest,Other,Current); + +split_range([N|Rest],Current,End) -> + split_range(Rest,[N|Current],End). + +send_body(SocketType,Socket,FileDescriptor) -> + case file:read(FileDescriptor,?FILE_CHUNK_SIZE) of + {ok,Binary} -> + ?DEBUG("send_body -> send another chunk: ~p",[size(Binary)]), + case httpd_socket:deliver(SocketType,Socket,Binary) of + socket_closed -> + ?LOG("send_body -> socket closed while sending",[]), + socket_close; + _ -> + send_body(SocketType,Socket,FileDescriptor) + end; + eof -> + ?DEBUG("send_body -> done with this file",[]), + eof + end. diff --git a/lib/inets/src/http_server/mod_responsecontrol.erl b/lib/inets/src/http_server/mod_responsecontrol.erl new file mode 100644 index 0000000000..79e2e1bdba --- /dev/null +++ b/lib/inets/src/http_server/mod_responsecontrol.erl @@ -0,0 +1,303 @@ +%% +%% %CopyrightBegin% +%% +%% Copyright Ericsson AB 2001-2009. All Rights Reserved. +%% +%% The contents of this file are subject to the Erlang Public License, +%% Version 1.1, (the "License"); you may not use this file except in +%% compliance with the License. You should have received a copy of the +%% Erlang Public License along with this software. If not, it can be +%% retrieved online at http://www.erlang.org/. +%% +%% Software distributed under the License is distributed on an "AS IS" +%% basis, WITHOUT WARRANTY OF ANY KIND, either express or implied. See +%% the License for the specific language governing rights and limitations +%% under the License. +%% +%% %CopyrightEnd% +%% +%% + +-module(mod_responsecontrol). +-export([do/1]). + +-include("httpd.hrl"). + +do(Info) -> + ?DEBUG("do -> response_control",[]), + case proplists:get_value(status, Info#mod.data) of + %% A status code has been generated! + {_StatusCode, _PhraseArgs, _Reason} -> + {proceed, Info#mod.data}; + %% No status code has been generated! + undefined -> + case proplists:get_value(response, Info#mod.data) of + %% No response has been generated! + undefined -> + case do_responsecontrol(Info) of + continue -> + {proceed, Info#mod.data}; + Response -> + {proceed,[Response | Info#mod.data]} + end; + %% A response has been generated or sent! + _Response -> + {proceed, Info#mod.data} + end + end. + +%%---------------------------------------------------------------------- +%%Control that the request header did not contians any limitations +%%wheather a response shall be createed or not +%%---------------------------------------------------------------------- +do_responsecontrol(Info) -> + ?DEBUG("do_response_control -> Request URI: ~p",[Info#mod.request_uri]), + Path = mod_alias:path(Info#mod.data, Info#mod.config_db, + Info#mod.request_uri), + case file:read_file_info(Path) of + {ok, FileInfo} -> + control(Path, Info, FileInfo); + _ -> + %% The requested asset is not a plain file and then it must + %% be generated everytime its requested + continue + end. + +%%---------------------------------------------------------------------- +%%Control the If-Match, If-None-Match, and If-Modified-Since +%%---------------------------------------------------------------------- + + +%% If a client sends more then one of the if-XXXX fields in a request +%% The standard says it does not specify the behaviuor so I specified it :-) +%% The priority between the fields is +%% 1.If-modified +%% 2.If-Unmodified +%% 3.If-Match +%% 4.If-Nomatch + +%% This means if more than one of the fields are in the request the +%% field with highest priority will be used + +%%If the request is a range request the If-Range field will be the winner. + +control(Path, Info, FileInfo) -> + case control_range(Path, Info, FileInfo) of + undefined -> + case control_Etag(Path, Info, FileInfo) of + undefined -> + case control_modification(Path, Info, FileInfo) of + continue -> + continue; + ReturnValue -> + send_return_value(ReturnValue, FileInfo) + end; + continue -> + continue; + ReturnValue -> + send_return_value(ReturnValue, FileInfo) + end; + Response-> + Response + end. + +%%---------------------------------------------------------------------- +%%If there are both a range and an if-range field control if +%%---------------------------------------------------------------------- +control_range(Path,Info,FileInfo) -> + case proplists:get_value("range", Info#mod.parsed_header) of + undefined-> + undefined; + _Range -> + case proplists:get_value("if-range", Info#mod.parsed_header) of + undefined -> + undefined; + EtagOrDate -> + control_if_range(Path,Info,FileInfo,EtagOrDate) + end + end. + +control_if_range(_Path, Info, FileInfo, EtagOrDate) -> + case httpd_util:convert_request_date(strip_date(EtagOrDate)) of + bad_date -> + FileEtag=httpd_util:create_etag(FileInfo), + case FileEtag of + EtagOrDate -> + continue; + _ -> + {if_range,send_file} + end; + _ErlDate -> + %%We got the date in the request if it is + case control_modification_data(Info, FileInfo#file_info.mtime, + "if-range") of + modified -> + {if_range,send_file}; + _UnmodifiedOrUndefined-> + continue + end + end. + +%%---------------------------------------------------------------------- +%%Controls the values of the If-Match and I-None-Mtch +%%---------------------------------------------------------------------- +control_Etag(Path, Info, FileInfo)-> + FileEtag = httpd_util:create_etag(FileInfo), + %%Control if the E-Tag for the resource matches one of the Etags in + %%the -if-match header field + case control_match(Info, FileInfo, "if-match", FileEtag) of + nomatch -> + %%None of the Etags in the if-match field matched the current + %%Etag for the resource return a 304 + {412, Info, Path}; + match -> + continue; + undefined -> + case control_match(Info, FileInfo, "if-none-match", FileEtag) of + nomatch -> + continue; + match -> + case Info#mod.method of + "GET" -> + {304, Info, Path}; + "HEAD" -> + {304, Info, Path}; + _OtherrequestMethod -> + {412, Info, Path} + end; + undefined -> + undefined + end + end. + +%%---------------------------------------------------------------------- +%%Control if there are any Etags for HeaderField in the request if so +%%Control if they match the Etag for the requested file +%%---------------------------------------------------------------------- +control_match(Info, _FileInfo, HeaderField, FileEtag)-> + case split_etags(proplists:get_value(HeaderField, + Info#mod.parsed_header)) of + undefined-> + undefined; + Etags-> + %%Control that the match any star not is availible + case lists:member("*",Etags) of + true-> + match; + false-> + compare_etags(FileEtag, Etags) + end + end. + +%%---------------------------------------------------------------------- +%%Split the etags from the request +%%---------------------------------------------------------------------- +split_etags(undefined)-> + undefined; +split_etags(Tags) -> + string:tokens(Tags,", "). + +%%---------------------------------------------------------------------- +%%Control if the etag for the file is in the list +%%---------------------------------------------------------------------- +compare_etags(Tag,Etags) -> + case lists:member(Tag,Etags) of + true -> + match; + _ -> + nomatch + end. + +%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%% +%% %% +%%Control if the file is modificated %% +%% %% +%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%% + +%%---------------------------------------------------------------------- +%%Control the If-Modified-Since and If-Not-Modified-Since header fields +%%---------------------------------------------------------------------- +control_modification(Path,Info,FileInfo)-> + ?DEBUG("control_modification() -> entry",[]), + case control_modification_data(Info, + FileInfo#file_info.mtime, + "if-modified-since") of + modified-> + continue; + unmodified-> + {304, Info, Path}; + undefined -> + case control_modification_data(Info, + FileInfo#file_info.mtime, + "if-unmodified-since") of + modified -> + {412, Info, Path}; + _ContinueUndefined -> + continue + end + end. + +%%---------------------------------------------------------------------- +%%Controls the date from the http-request if-modified-since and +%%if-not-modified-since against the modification data of the +%%File +%%---------------------------------------------------------------------- +%%Info is the record about the request +%%ModificationTime is the time the file was edited last +%%Header Field is the name of the field to control + +control_modification_data(Info, ModificationTime, HeaderField)-> + case strip_date(proplists:get_value(HeaderField, + Info#mod.parsed_header)) of + undefined-> + undefined; + LastModified0 -> + LastModified = calendar:universal_time_to_local_time( + httpd_util:convert_request_date(LastModified0)), + ?DEBUG("control_modification_data() -> " + "~n Request-Field: ~s" + "~n FileLastModified: ~p" + "~n FieldValue: ~p", + [HeaderField, ModificationTime, LastModified]), + FileTime = + calendar:datetime_to_gregorian_seconds(ModificationTime), + FieldTime = calendar:datetime_to_gregorian_seconds(LastModified), + if + FileTime =< FieldTime -> + ?DEBUG("File unmodified~n", []), unmodified; + FileTime >= FieldTime -> + ?DEBUG("File modified~n", []), modified + end + end. + +%% IE4 & NS4 sends an extra '; length=xxxx' string at the end of the If-Modified-Since +%% header, we detect this and ignore it (the RFCs does not mention this). +strip_date(undefined) -> + undefined; +strip_date([]) -> + []; +strip_date([$;,$ | _]) -> + []; +strip_date([C | Rest]) -> + [C | strip_date(Rest)]. + +send_return_value({412,_,_}, _FileInfo)-> + {status,{412,none,"Precondition Failed"}}; + +send_return_value({304,Info,Path}, FileInfo)-> + Suffix = httpd_util:suffix(Path), + MimeType = httpd_util:lookup_mime_default(Info#mod.config_db,Suffix, + "text/plain"), + LastModified = + case (catch httpd_util:rfc1123_date(FileInfo#file_info.mtime)) of + Date when is_list(Date) -> + [{last_modified, Date}]; + _ -> %% This will rarly happen, but could happen + %% if a computer is wrongly configured. + [] + end, + + Header = [{code,304}, + {etag, httpd_util:create_etag(FileInfo)}, + {content_length,"0"}, {mime_type, MimeType} | LastModified], + {response, {response, Header, nobody}}. diff --git a/lib/inets/src/http_server/mod_security.erl b/lib/inets/src/http_server/mod_security.erl new file mode 100644 index 0000000000..95793e1cfb --- /dev/null +++ b/lib/inets/src/http_server/mod_security.erl @@ -0,0 +1,325 @@ +%% +%% %CopyrightBegin% +%% +%% Copyright Ericsson AB 1998-2009. All Rights Reserved. +%% +%% The contents of this file are subject to the Erlang Public License, +%% Version 1.1, (the "License"); you may not use this file except in +%% compliance with the License. You should have received a copy of the +%% Erlang Public License along with this software. If not, it can be +%% retrieved online at http://www.erlang.org/. +%% +%% Software distributed under the License is distributed on an "AS IS" +%% basis, WITHOUT WARRANTY OF ANY KIND, either express or implied. See +%% the License for the specific language governing rights and limitations +%% under the License. +%% +%% %CopyrightEnd% +%% +%% +-module(mod_security). + +%% Security Audit Functionality + +%% User API exports +-export([list_blocked_users/1, list_blocked_users/2, list_blocked_users/3, + block_user/4, block_user/5, + unblock_user/2, unblock_user/3, unblock_user/4, + list_auth_users/1, list_auth_users/2, list_auth_users/3]). + +%% module API exports +-export([do/1, load/2, store/2, remove/1]). + +-include("httpd.hrl"). +-include("httpd_internal.hrl"). + +-define(VMODULE,"SEC"). + + +%% do/1 +do(Info) -> + ?hdrt("do", [{info, Info}]), + %% Check and see if any user has been authorized. + case proplists:get_value(remote_user, Info#mod.data,not_defined_user) of + not_defined_user -> + %% No user has been authorized. + case proplists:get_value(response, Info#mod.data) of + %% A status code has been generated! + {401, _Response} -> + case proplists:get_value("authorization", + Info#mod.parsed_header) of + undefined -> + %% Not an authorization attempt (server + %% just replied to challenge for + %% authentication) + {proceed, Info#mod.data}; + [$B,$a,$s,$i,$c,$ |EncodedString] -> + %% Someone tried to authenticate, and + %% obviously failed! + DecodedString = + case (catch + base64:decode_to_string( + EncodedString)) of + %% Decode failed + {'EXIT',{function_clause, _}} -> + EncodedString; + String -> + String + end, + + report_failed(Info, DecodedString, + "Failed authentication"), + take_failed_action(Info, DecodedString), + {proceed, Info#mod.data} + end; + _ -> + {proceed, Info#mod.data} + end; + User -> + %% A user has been authenticated, now is he blocked ? + Path = mod_alias:path(Info#mod.data, + Info#mod.config_db, + Info#mod.request_uri), + {_Dir, SDirData} = secretp(Path, Info#mod.config_db), + Addr = httpd_util:lookup(Info#mod.config_db, bind_address), + Port = httpd_util:lookup(Info#mod.config_db, port), + case mod_security_server:check_blocked_user(Info, User, + SDirData, + Addr, Port) of + true -> + report_failed(Info, User ,"User Blocked"), + {proceed, [{status, {403, Info#mod.request_uri, ""}} | + Info#mod.data]}; + false -> + report_failed(Info, User,"Authentication Succedded"), + mod_security_server:store_successful_auth(Addr, Port, + User, + SDirData), + {proceed, Info#mod.data} + end + end. + +report_failed(Info, Auth, Event) -> + Request = Info#mod.request_line, + {_PortNumber,RemoteHost}=(Info#mod.init_data)#init_data.peername, + String = RemoteHost ++ " : " ++ Event ++ " : " ++ Request ++ + " : " ++ Auth, + mod_disk_log:security_log(Info,String), + mod_log:security_log(Info, String). + +take_failed_action(Info, Auth) -> + ?hdrd("take failed action", [{auth, Auth}]), + Path = mod_alias:path(Info#mod.data, Info#mod.config_db, + Info#mod.request_uri), + {_Dir, SDirData} = secretp(Path, Info#mod.config_db), + Addr = httpd_util:lookup(Info#mod.config_db, bind_address), + Port = httpd_util:lookup(Info#mod.config_db, port), + mod_security_server:store_failed_auth(Info, Addr, Port, + Auth, SDirData). + +secretp(Path, ConfigDB) -> + Directories = ets:match(ConfigDB,{directory,{'$1','_'}}), + case secret_path(Path, Directories) of + {yes, Directory} -> + ?hdrd("secretp - yes", [{dir, Directory}]), + SDirs0 = httpd_util:multi_lookup(ConfigDB, security_directory), + [SDir] = lists:filter(fun({Directory0, _}) + when Directory0 == Directory -> + true; + (_) -> + false + end, SDirs0), + SDir; + no -> + {[], []} + end. + +secret_path(Path,Directories) -> + secret_path(Path, httpd_util:uniq(lists:sort(Directories)), to_be_found). + +secret_path(_Path, [], to_be_found) -> + no; +secret_path(_Path, [], Dir) -> + {yes, Dir}; +secret_path(Path, [[NewDir]|Rest], Dir) -> + case inets_regexp:match(Path, NewDir) of + {match, _, _} when Dir =:= to_be_found -> + secret_path(Path, Rest, NewDir); + {match, _, Length} when Length > length(Dir) -> + secret_path(Path, Rest, NewDir); + {match, _, _} -> + secret_path(Path, Rest, Dir); + nomatch -> + secret_path(Path, Rest, Dir) + end. + + +load("<Directory " ++ Directory, []) -> + ?hdrt("load security directory - begin", [{directory, Directory}]), + Dir = httpd_conf:custom_clean(Directory,"",">"), + {ok, [{security_directory, {Dir, [{path, Dir}]}}]}; +load(eof,[{security_directory, {Directory, _DirData}}|_]) -> + {error, ?NICE("Premature end-of-file in "++Directory)}; +load("SecurityDataFile " ++ FileName, + [{security_directory, {Dir, DirData}}]) -> + ?hdrt("load security directory", + [{file, FileName}, {dir, Dir}, {dir_data, DirData}]), + File = httpd_conf:clean(FileName), + {ok, [{security_directory, {Dir, [{data_file, File}|DirData]}}]}; +load("SecurityCallbackModule " ++ ModuleName, + [{security_directory, {Dir, DirData}}]) -> + ?hdrt("load security directory", + [{module, ModuleName}, {dir, Dir}, {dir_data, DirData}]), + Mod = list_to_atom(httpd_conf:clean(ModuleName)), + {ok, [{security_directory, {Dir, [{callback_module, Mod}|DirData]}}]}; +load("SecurityMaxRetries " ++ Retries, + [{security_directory, {Dir, DirData}}]) -> + ?hdrt("load security directory", + [{max_retries, Retries}, {dir, Dir}, {dir_data, DirData}]), + load_return_int_tag("SecurityMaxRetries", max_retries, + httpd_conf:clean(Retries), Dir, DirData); +load("SecurityBlockTime " ++ Time, + [{security_directory, {Dir, DirData}}]) -> + ?hdrt("load security directory", + [{block_time, Time}, {dir, Dir}, {dir_data, DirData}]), + load_return_int_tag("SecurityBlockTime", block_time, + httpd_conf:clean(Time), Dir, DirData); +load("SecurityFailExpireTime " ++ Time, + [{security_directory, {Dir, DirData}}]) -> + ?hdrt("load security directory", + [{expire_time, Time}, {dir, Dir}, {dir_data, DirData}]), + load_return_int_tag("SecurityFailExpireTime", fail_expire_time, + httpd_conf:clean(Time), Dir, DirData); +load("SecurityAuthTimeout " ++ Time0, + [{security_directory, {Dir, DirData}}]) -> + ?hdrt("load security directory", + [{auth_timeout, Time0}, {dir, Dir}, {dir_data, DirData}]), + Time = httpd_conf:clean(Time0), + load_return_int_tag("SecurityAuthTimeout", auth_timeout, + httpd_conf:clean(Time), Dir, DirData); +load("AuthName " ++ Name0, + [{security_directory, {Dir, DirData}}]) -> + ?hdrt("load security directory", + [{name, Name0}, {dir, Dir}, {dir_data, DirData}]), + Name = httpd_conf:clean(Name0), + {ok, [{security_directory, {Dir, [{auth_name, Name}|DirData]}}]}; +load("</Directory>",[{security_directory, {Dir, DirData}}]) -> + ?hdrt("load security directory - end", + [{dir, Dir}, {dir_data, DirData}]), + {ok, [], {security_directory, {Dir, DirData}}}. + +load_return_int_tag(Name, Atom, Time, Dir, DirData) -> + case Time of + "infinity" -> + {ok, [{security_directory, {Dir, + [{Atom, 99999999999999999999999999999} | DirData]}}]}; + _Int -> + case catch list_to_integer(Time) of + {'EXIT', _} -> + {error, Time++" is an invalid "++Name}; + Val -> + {ok, [{security_directory, {Dir, [{Atom, Val}|DirData]}}]} + end + end. + +store({security_directory, {Dir, DirData}}, ConfigList) + when is_list(Dir) andalso is_list(DirData) -> + ?hdrt("store security directory", [{dir, Dir}, {dir_data, DirData}]), + Addr = proplists:get_value(bind_address, ConfigList), + Port = proplists:get_value(port, ConfigList), + mod_security_server:start(Addr, Port), + SR = proplists:get_value(server_root, ConfigList), + case proplists:get_value(data_file, DirData, no_data_file) of + no_data_file -> + {error, {missing_security_data_file, {security_directory, {Dir, DirData}}}}; + DataFile0 -> + DataFile = + case filename:pathtype(DataFile0) of + relative -> + filename:join(SR, DataFile0); + _ -> + DataFile0 + end, + case mod_security_server:new_table(Addr, Port, DataFile) of + {ok, TwoTables} -> + NewDirData0 = lists:keyreplace(data_file, 1, DirData, + {data_file, TwoTables}), + NewDirData1 = case Addr of + undefined -> + [{port,Port}|NewDirData0]; + _ -> + [{port,Port},{bind_address,Addr}| + NewDirData0] + end, + {ok, {security_directory, {Dir, NewDirData1}}}; + {error, Err} -> + {error, {{open_data_file, DataFile}, Err}} + end + end; +store({directory, {Directory, DirData}}, _) -> + {error, {wrong_type, {security_directory, {Directory, DirData}}}}. + +remove(ConfigDB) -> + Addr = case ets:lookup(ConfigDB, bind_address) of + [] -> + undefined; + [{bind_address, Address}] -> + Address + end, + [{port, Port}] = ets:lookup(ConfigDB, port), + mod_security_server:delete_tables(Addr, Port), + mod_security_server:stop(Addr, Port). + + +%% +%% User API +%% + +%% list_blocked_users + +list_blocked_users(Port) -> + list_blocked_users(undefined, Port). + +list_blocked_users(Port, Dir) when is_integer(Port) -> + list_blocked_users(undefined,Port,Dir); +list_blocked_users(Addr, Port) when is_integer(Port) -> + mod_security_server:list_blocked_users(Addr, Port). + +list_blocked_users(Addr, Port, Dir) -> + mod_security_server:list_blocked_users(Addr, Port, Dir). + + +%% block_user + +block_user(User, Port, Dir, Time) -> + block_user(User, undefined, Port, Dir, Time). +block_user(User, Addr, Port, Dir, Time) -> + mod_security_server:block_user(User, Addr, Port, Dir, Time). + + +%% unblock_user + +unblock_user(User, Port) -> + unblock_user(User, undefined, Port). + +unblock_user(User, Port, Dir) when is_integer(Port) -> + unblock_user(User, undefined, Port, Dir); +unblock_user(User, Addr, Port) when is_integer(Port) -> + mod_security_server:unblock_user(User, Addr, Port). + +unblock_user(User, Addr, Port, Dir) -> + mod_security_server:unblock_user(User, Addr, Port, Dir). + + +%% list_auth_users + +list_auth_users(Port) -> + list_auth_users(undefined,Port). + +list_auth_users(Port, Dir) when is_integer(Port) -> + list_auth_users(undefined, Port, Dir); +list_auth_users(Addr, Port) when is_integer(Port) -> + mod_security_server:list_auth_users(Addr, Port). + +list_auth_users(Addr, Port, Dir) -> + mod_security_server:list_auth_users(Addr, Port, Dir). diff --git a/lib/inets/src/http_server/mod_security_server.erl b/lib/inets/src/http_server/mod_security_server.erl new file mode 100644 index 0000000000..58060686b3 --- /dev/null +++ b/lib/inets/src/http_server/mod_security_server.erl @@ -0,0 +1,665 @@ +%% +%% %CopyrightBegin% +%% +%% Copyright Ericsson AB 2001-2009. All Rights Reserved. +%% +%% The contents of this file are subject to the Erlang Public License, +%% Version 1.1, (the "License"); you may not use this file except in +%% compliance with the License. You should have received a copy of the +%% Erlang Public License along with this software. If not, it can be +%% retrieved online at http://www.erlang.org/. +%% +%% Software distributed under the License is distributed on an "AS IS" +%% basis, WITHOUT WARRANTY OF ANY KIND, either express or implied. See +%% the License for the specific language governing rights and limitations +%% under the License. +%% +%% %CopyrightEnd% +%% +%% +%% Security Audit Functionality + +%% +%% The gen_server code. +%% +%% A gen_server is needed in this module to take care of shared access to the +%% data file used to store failed and successful authentications aswell as +%% user blocks. +%% +%% The storage model is a write-through model with both an ets and a dets +%% table. Writes are done to both the ets and then the dets table, but reads +%% are only done from the ets table. +%% +%% This approach also enables parallelism when using dets by returning the +%% same dets table identifier when opening several files with the same +%% physical location. +%% +%% NOTE: This could be implemented using a single dets table, as it is +%% possible to open a dets file with the ram_file flag, but this +%% would require periodical sync's to disk, and it would be hard +%% to decide when such an operation should occur. +%% + + +-module(mod_security_server). + +-include("httpd.hrl"). +-include("httpd_internal.hrl"). + +-behaviour(gen_server). + + +%% User API exports (called via mod_security) +-export([list_blocked_users/2, list_blocked_users/3, + block_user/5, + unblock_user/3, unblock_user/4, + list_auth_users/2, list_auth_users/3]). + +%% Internal exports (for mod_security only) +-export([start/2, stop/1, stop/2, + new_table/3, delete_tables/2, + store_failed_auth/5, store_successful_auth/4, + check_blocked_user/5]). + +%% gen_server exports +-export([start_link/2, init/1, + handle_info/2, handle_call/3, handle_cast/2, + terminate/2, + code_change/3]). + + +%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%% +%% %% +%% External API %% +%% %% +%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%% + +%% start_link/3 +%% +%% NOTE: This is called by httpd_misc_sup when the process is started +%% + +start_link(Addr, Port) -> + ?hdrt("start_link", [{address, Addr}, {port, Port}]), + Name = make_name(Addr, Port), + gen_server:start_link({local, Name}, ?MODULE, [], [{timeout, infinity}]). + + +%% start/2 +%% Called by the mod_security module. + +start(Addr, Port) -> + ?hdrt("start", [{address, Addr}, {port, Port}]), + Name = make_name(Addr, Port), + case whereis(Name) of + undefined -> + httpd_misc_sup:start_sec_server(Addr, Port); + _ -> %% Already started... + ok + end. + + +%% stop + +stop(Port) -> + stop(undefined, Port). +stop(Addr, Port) -> + ?hdrt("stop", [{address, Addr}, {port, Port}]), + Name = make_name(Addr, Port), + case whereis(Name) of + undefined -> + ok; + _ -> + httpd_misc_sup:stop_sec_server(Addr, Port) + end. + + +addr(undefined) -> + any; +addr(Addr) -> + Addr. + + +%% list_blocked_users + +list_blocked_users(Addr, Port) -> + Name = make_name(Addr, Port), + Req = {list_blocked_users, addr(Addr), Port, '_'}, + call(Name, Req). + +list_blocked_users(Addr, Port, Dir) -> + Name = make_name(Addr, Port), + Req = {list_blocked_users, addr(Addr), Port, Dir}, + call(Name, Req). + + +%% block_user + +block_user(User, Addr, Port, Dir, Time) -> + Name = make_name(Addr, Port), + Req = {block_user, User, addr(Addr), Port, Dir, Time}, + call(Name, Req). + + +%% unblock_user + +unblock_user(User, Addr, Port) -> + Name = make_name(Addr, Port), + Req = {unblock_user, User, addr(Addr), Port, '_'}, + call(Name, Req). + +unblock_user(User, Addr, Port, Dir) -> + Name = make_name(Addr, Port), + Req = {unblock_user, User, addr(Addr), Port, Dir}, + call(Name, Req). + + +%% list_auth_users + +list_auth_users(Addr, Port) -> + Name = make_name(Addr, Port), + Req = {list_auth_users, addr(Addr), Port, '_'}, + call(Name, Req). + +list_auth_users(Addr, Port, Dir) -> + Name = make_name(Addr,Port), + Req = {list_auth_users, addr(Addr), Port, Dir}, + call(Name, Req). + + +%% new_table + +new_table(Addr, Port, TabName) -> + Name = make_name(Addr,Port), + Req = {new_table, addr(Addr), Port, TabName}, + call(Name, Req). + + +%% delete_tables + +delete_tables(Addr, Port) -> + Name = make_name(Addr, Port), + case whereis(Name) of + undefined -> + ok; + _ -> + call(Name, delete_tables) + end. + + +%% store_failed_auth + +store_failed_auth(Info, Addr, Port, DecodedString, SDirData) -> + ?hdrv("store failed auth", + [{addr, Addr}, {port, Port}, + {decoded_string, DecodedString}, {sdir_data, SDirData}]), + Name = make_name(Addr,Port), + Msg = {store_failed_auth,[Info,DecodedString,SDirData]}, + cast(Name, Msg). + + +%% store_successful_auth + +store_successful_auth(Addr, Port, User, SDirData) -> + Name = make_name(Addr,Port), + Msg = {store_successful_auth, [User,Addr,Port,SDirData]}, + cast(Name, Msg). + + +%% check_blocked_user + +check_blocked_user(Info, User, SDirData, Addr, Port) -> + Name = make_name(Addr, Port), + Req = {check_blocked_user, [Info, User, SDirData]}, + call(Name, Req). + + +%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%% +%% %% +%% Server call-back functions %% +%% %% +%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%% + +init(_) -> + ?hdrv("initiating", []), + process_flag(trap_exit, true), + {ok, []}. + +handle_call(stop, _From, _Tables) -> + {stop, normal, ok, []}; + +handle_call({block_user, User, Addr, Port, Dir, Time}, _From, Tables) -> + ?hdrv("block user", + [{user, User}, {addr, Addr}, {port, Port}, {dir, Dir}, + {time, Time}]), + Ret = block_user_int(User, Addr, Port, Dir, Time), + {reply, Ret, Tables}; + +handle_call({list_blocked_users, Addr, Port, Dir}, _From, Tables) -> + ?hdrv("list blocked users", + [{addr, Addr}, {port, Port}, {dir, Dir}]), + Blocked = list_blocked(Tables, Addr, Port, Dir, []), + {reply, Blocked, Tables}; + +handle_call({unblock_user, User, Addr, Port, Dir}, _From, Tables) -> + ?hdrv("block user", + [{user, User}, {addr, Addr}, {port, Port}, {dir, Dir}]), + Ret = unblock_user_int(User, Addr, Port, Dir), + {reply, Ret, Tables}; + +handle_call({list_auth_users, Addr, Port, Dir}, _From, Tables) -> + ?hdrv("list auth users", + [{addr, Addr}, {port, Port}, {dir, Dir}]), + Auth = list_auth(Tables, Addr, Port, Dir, []), + {reply, Auth, Tables}; + +handle_call({new_table, Addr, Port, Name}, _From, Tables) -> + case lists:keysearch(Name, 1, Tables) of + {value, {Name, {Ets, Dets}}} -> + {reply, {ok, {Ets, Dets}}, Tables}; + false -> + TName = make_name(Addr,Port,length(Tables)), + case dets:open_file(TName, [{type, bag}, {file, Name}, + {repair, true}, + {access, read_write}]) of + {ok, DFile} -> + ETS = ets:new(TName, [bag, private]), + sync_dets_to_ets(DFile, ETS), + NewTables = [{Name, {ETS, DFile}}|Tables], + {reply, {ok, {ETS, DFile}}, NewTables}; + {error, Err} -> + {reply, {error, {create_dets, Err}}, Tables} + end + end; + +handle_call(delete_tables, _From, Tables) -> + lists:foreach(fun({_Name, {ETS, DETS}}) -> + dets:close(DETS), + ets:delete(ETS) + end, Tables), + {reply, ok, []}; + +handle_call({check_blocked_user, [Info, User, SDirData]}, _From, Tables) -> + {ETS, DETS} = proplists:get_value(data_file, SDirData), + Dir = proplists:get_value(path, SDirData), + Addr = proplists:get_value(bind_address, SDirData), + Port = proplists:get_value(port, SDirData), + CBModule = + proplists:get_value(callback_module, SDirData, no_module_at_all), + Ret = + check_blocked_user(Info, User, Dir, Addr, Port, ETS, DETS, CBModule), + {reply, Ret, Tables}; + +handle_call(_Request,_From,Tables) -> + {reply,ok,Tables}. + + +%% handle_cast + +handle_cast({store_failed_auth, [_, _, []]}, Tables) -> + %% Some other authentication scheme than mod_auth (example mod_htacess) + %% was the source for the authentication failure so we should ignor it! + {noreply, Tables}; +handle_cast({store_failed_auth, [Info, DecodedString, SDirData]}, Tables) -> + {ETS, DETS} = proplists:get_value(data_file, SDirData), + Dir = proplists:get_value(path, SDirData), + Addr = proplists:get_value(bind_address, SDirData), + Port = proplists:get_value(port, SDirData), + {ok, [User,Password]} = httpd_util:split(DecodedString,":",2), + Seconds = universal_time(), + Key = {User, Dir, Addr, Port}, + %% Event + CBModule = proplists:get_value(callback_module, + SDirData, no_module_at_all), + auth_fail_event(CBModule,Addr,Port,Dir,User,Password), + + %% Find out if any of this user's other failed logins are too old to keep.. + case ets:match_object(ETS, {failed, {Key, '_', '_'}}) of + [] -> + no; + List -> + ExpireTime = proplists:get_value(fail_expire_time, + SDirData, 30)*60, + lists:map(fun({failed, {TheKey, LS, Gen}}) -> + Diff = Seconds-LS, + if + Diff > ExpireTime -> + ets:match_delete(ETS, + {failed, + {TheKey, LS, Gen}}), + dets:match_delete(DETS, + {failed, + {TheKey, LS, Gen}}); + true -> + ok + end + end, + List) + end, + + %% Insert the new failure.. + Generation = length(ets:match_object(ETS, {failed, {Key, '_', '_'}})), + ets:insert(ETS, {failed, {Key, Seconds, Generation}}), + dets:insert(DETS, {failed, {Key, Seconds, Generation}}), + + %% See if we should block this user.. + MaxRetries = proplists:get_value(max_retries, SDirData, 3), + BlockTime = proplists:get_value(block_time, SDirData, 60), + case ets:match_object(ETS, {failed, {Key, '_', '_'}}) of + List1 when length(List1) >= MaxRetries -> + %% Block this user until Future + Future = Seconds+BlockTime*60, + Reason = io_lib:format("Blocking user ~s from dir ~s " + "for ~p minutes", + [User, Dir, BlockTime]), + mod_log:security_log(Info, lists:flatten(Reason)), + + %% Event + user_block_event(CBModule,Addr,Port,Dir,User), + + ets:match_delete(ETS,{blocked_user, + {User, Addr, Port, Dir, '$1'}}), + dets:match_delete(DETS, {blocked_user, + {User, Addr, Port, Dir, '$1'}}), + BlockRecord = {blocked_user, + {User, Addr, Port, Dir, Future}}, + ets:insert(ETS, BlockRecord), + dets:insert(DETS, BlockRecord), + %% Remove previous failed requests. + ets:match_delete(ETS, {failed, {Key, '_', '_'}}), + dets:match_delete(DETS, {failed, {Key, '_', '_'}}); + _ -> + no + end, + {noreply, Tables}; + +handle_cast({store_successful_auth, [User, Addr, Port, SDirData]}, Tables) -> + {ETS, DETS} = proplists:get_value(data_file, SDirData), + AuthTimeOut = proplists:get_value(auth_timeout, SDirData, 30), + Dir = proplists:get_value(path, SDirData), + Key = {User, Dir, Addr, Port}, + + %% Remove failed entries for this Key + dets:match_delete(DETS, {failed, {Key, '_', '_'}}), + ets:match_delete(ETS, {failed, {Key, '_', '_'}}), + + %% Keep track of when the last successful login took place. + Seconds = universal_time()+AuthTimeOut, + ets:match_delete(ETS, {success, {Key, '_'}}), + dets:match_delete(DETS, {success, {Key, '_'}}), + ets:insert(ETS, {success, {Key, Seconds}}), + dets:insert(DETS, {success, {Key, Seconds}}), + {noreply, Tables}; + +handle_cast(Req, Tables) -> + error_msg("security server got unknown cast: ~p",[Req]), + {noreply, Tables}. + + +%% handle_info + +handle_info(_Info, State) -> + {noreply, State}. + + +%% terminate + +terminate(_Reason, _Tables) -> + ok. + + +%% code_change({down, ToVsn}, State, Extra) +%% +code_change({down, _}, State, _Extra) -> + {ok, State}; + + +%% code_change(FromVsn, State, Extra) +%% +code_change(_, State, _Extra) -> + {ok, State}. + +%% block_user_int/5 +block_user_int(User, Addr, Port, Dir, Time) -> + Dirs = httpd_manager:config_match(Addr, Port, + {security_directory, {'_', '_'}}), + case find_dirdata(Dirs, Dir) of + {ok, DirData, {ETS, DETS}} -> + Time1 = + case Time of + infinity -> + 99999999999999999999999999999; + _ -> + Time + end, + Future = universal_time()+Time1, + ets:match_delete(ETS, {blocked_user, {User,Addr,Port,Dir,'_'}}), + dets:match_delete(DETS, {blocked_user, + {User,Addr,Port,Dir,'_'}}), + ets:insert(ETS, {blocked_user, {User,Addr,Port,Dir,Future}}), + dets:insert(DETS, {blocked_user, {User,Addr,Port,Dir,Future}}), + CBModule = proplists:get_value(callback_module, DirData, + no_module_at_all), + user_block_event(CBModule,Addr,Port,Dir,User), + true; + _ -> + {error, no_such_directory} + end. + + +find_dirdata([], _Dir) -> + false; +find_dirdata([{security_directory, {_, DirData}}|SDirs], Dir) -> + case lists:keysearch(path, 1, DirData) of + {value, {path, Dir}} -> + {value, {data_file, {ETS, DETS}}} = + lists:keysearch(data_file, 1, DirData), + {ok, DirData, {ETS, DETS}}; + _ -> + find_dirdata(SDirs, Dir) + end. + +%% unblock_user_int/4 +unblock_user_int(User, Addr, Port, Dir) -> + Dirs = httpd_manager:config_match(Addr, Port, + {security_directory, {'_', '_'}}), + case find_dirdata(Dirs, Dir) of + {ok, DirData, {ETS, DETS}} -> + case ets:match_object(ETS, + {blocked_user,{User,Addr,Port,Dir,'_'}}) of + [] -> + {error, not_blocked}; + _Objects -> + ets:match_delete(ETS, {blocked_user, + {User, Addr, Port, Dir, '_'}}), + dets:match_delete(DETS, {blocked_user, + {User, Addr, Port, Dir, '_'}}), + CBModule = proplists:get_value(callback_module, + DirData, + no_module_at_all), + user_unblock_event(CBModule,Addr,Port,Dir,User), + true + end; + _ -> + {error, no_such_directory} + end. + + + +%% list_auth/2 + +list_auth([], _Addr, _Port, _Dir, Acc) -> + Acc; +list_auth([{_Name, {ETS, DETS}}|Tables], Addr, Port, Dir, Acc) -> + case ets:match_object(ETS, {success, {{'_', Dir, Addr, Port}, '_'}}) of + [] -> + list_auth(Tables, Addr, Port, Dir, Acc); + List -> + TN = universal_time(), + NewAcc = lists:foldr(fun({success,{{U,Ad,P,D},T}},Ac) -> + if + T-TN > 0 -> + [U|Ac]; + true -> + Rec = {success, + {{U,Ad,P,D},T}}, + ets:match_delete(ETS,Rec), + dets:match_delete(DETS,Rec), + Ac + end + end, + Acc, List), + list_auth(Tables, Addr, Port, Dir, NewAcc) + end. + + +%% list_blocked/2 + +list_blocked([], _Addr, _Port, _Dir, Acc) -> + ?hdrv("list blocked", [{acc, Acc}]), + TN = universal_time(), + lists:foldl(fun({U,Ad,P,D,T}, Ac) -> + if + T-TN > 0 -> + [{U,Ad,P,D,local_time(T)}|Ac]; + true -> + Ac + end + end, + [], Acc); +list_blocked([{_Name, {ETS, _DETS}}|Tables], Addr, Port, Dir, Acc) -> + ?hdrv("list blocked", [{ets, ETS}, {tab2list, ets:tab2list(ETS)}]), + List = ets:match_object(ETS, {blocked_user, + {'_',Addr,Port,Dir,'_'}}), + + NewBlocked = lists:foldl(fun({blocked_user, X}, A) -> + [X|A] end, Acc, List), + + list_blocked(Tables, Addr, Port, Dir, NewBlocked). + + +%% +%% sync_dets_to_ets/2 +%% +%% Reads dets-table DETS and syncronizes it with the ets-table ETS. +%% +sync_dets_to_ets(DETS, ETS) -> + dets:traverse(DETS, fun(X) -> + ets:insert(ETS, X), + continue + end). + +%% +%% check_blocked_user/7 -> true | false +%% +%% Check if a specific user is blocked from access. +%% +%% The sideeffect of this routine is that it unblocks also other users +%% whos blocking time has expired. This to keep the tables as small +%% as possible. +%% +check_blocked_user(Info, User, Dir, Addr, Port, ETS, DETS, CBModule) -> + TN = universal_time(), + BlockList = ets:match_object(ETS, {blocked_user, {User, '_', '_', '_', '_'}}), + Blocked = lists:foldl(fun({blocked_user, X}, A) -> + [X|A] end, [], BlockList), + check_blocked_user(Info,User,Dir, + Addr,Port,ETS,DETS,TN,Blocked,CBModule). + +check_blocked_user(_Info, _User, _Dir, _Addr, _Port, _ETS, _DETS, _TN, + [], _CBModule) -> + false; +check_blocked_user(Info, User, Dir, Addr, Port, ETS, DETS, TN, + [{User,Addr,Port,Dir,T}| _], CBModule) -> + TD = T-TN, + if + TD =< 0 -> + %% Blocking has expired, remove and grant access. + unblock_user(Info, User, Dir, Addr, Port, ETS, DETS, CBModule), + false; + true -> + true + end; +check_blocked_user(Info, User, Dir, Addr, Port, ETS, DETS, TN, + [{OUser,ODir,OAddr,OPort,T}|Ls], CBModule) -> + TD = T-TN, + if + TD =< 0 -> + %% Blocking has expired, remove. + unblock_user(Info, OUser, ODir, OAddr, OPort, + ETS, DETS, CBModule); + true -> + true + end, + check_blocked_user(Info, User, Dir, Addr, Port, ETS, DETS, + TN, Ls, CBModule). + +unblock_user(Info, User, Dir, Addr, Port, ETS, DETS, CBModule) -> + Reason = + io_lib:format("User ~s was removed from the block list for dir ~s", + [User, Dir]), + mod_log:security_log(Info, lists:flatten(Reason)), + user_unblock_event(CBModule,Addr,Port,Dir,User), + dets:match_delete(DETS, {blocked_user, {User, Addr, Port, Dir, '_'}}), + ets:match_delete(ETS, {blocked_user, {User, Addr, Port, Dir, '_'}}). + + +make_name(Addr,Port) -> + httpd_util:make_name("httpd_security",Addr,Port). + +make_name(Addr,Port,Num) -> + httpd_util:make_name("httpd_security",Addr,Port, + "__" ++ integer_to_list(Num)). + + +auth_fail_event(Mod,Addr,Port,Dir,User,Passwd) -> + event(auth_fail,Mod,Addr,Port,Dir,[{user,User},{password,Passwd}]). + +user_block_event(Mod,Addr,Port,Dir,User) -> + event(user_block,Mod,Addr,Port,Dir,[{user,User}]). + +user_unblock_event(Mod,Addr,Port,Dir,User) -> + event(user_unblock,Mod,Addr,Port,Dir,[{user,User}]). + +event(Event, Mod, undefined, Port, Dir, Info) -> + ?hdrt("event", + [{event, Event}, {mod, Mod}, {port, Port}, {dir, Dir}]), + (catch Mod:event(Event,Port,Dir,Info)); +event(Event, Mod, any, Port, Dir, Info) -> + ?hdrt("event", + [{event, Event}, {mod, Mod}, {port, Port}, {dir, Dir}]), + (catch Mod:event(Event,Port,Dir,Info)); +event(Event, Mod, Addr, Port, Dir, Info) -> + ?hdrt("event", + [{event, Event}, {mod, Mod}, + {addr, Addr}, {port, Port}, {dir, Dir}]), + (catch Mod:event(Event,Addr,Port,Dir,Info)). + +universal_time() -> + calendar:datetime_to_gregorian_seconds(calendar:universal_time()). + +local_time(T) -> + calendar:universal_time_to_local_time( + calendar:gregorian_seconds_to_datetime(T)). + + +error_msg(F, A) -> + error_logger:error_msg(F, A). + + +call(Name, Req) -> + case (catch gen_server:call(Name, Req)) of + {'EXIT', Reason} -> + {error, Reason}; + Reply -> + Reply + end. + + +cast(Name, Msg) -> + case (catch gen_server:cast(Name, Msg)) of + {'EXIT', Reason} -> + {error, Reason}; + Result -> + Result + end. diff --git a/lib/inets/src/http_server/mod_trace.erl b/lib/inets/src/http_server/mod_trace.erl new file mode 100644 index 0000000000..df482228d8 --- /dev/null +++ b/lib/inets/src/http_server/mod_trace.erl @@ -0,0 +1,89 @@ +%% +%% %CopyrightBegin% +%% +%% Copyright Ericsson AB 2001-2009. All Rights Reserved. +%% +%% The contents of this file are subject to the Erlang Public License, +%% Version 1.1, (the "License"); you may not use this file except in +%% compliance with the License. You should have received a copy of the +%% Erlang Public License along with this software. If not, it can be +%% retrieved online at http://www.erlang.org/. +%% +%% Software distributed under the License is distributed on an "AS IS" +%% basis, WITHOUT WARRANTY OF ANY KIND, either express or implied. See +%% the License for the specific language governing rights and limitations +%% under the License. +%% +%% %CopyrightEnd% +%% +%% +-module(mod_trace). + +-export([do/1]). + +-include("httpd.hrl"). + + +do(Info) -> + %%?vtrace("do",[]), + case Info#mod.method of + "TRACE" -> + case response_generated(Info) of + false-> + generate_trace_response(Info); + true-> + {proceed,Info#mod.data} + end; + _ -> + {proceed,Info#mod.data} + end. + + +%%--------------------------------------------------------------------- +%%Generate the trace response the trace response consists of a +%%http-header and the body will be the request. +%5---------------------------------------------------------------------- + +generate_trace_response(Info)-> + RequestHead=Info#mod.parsed_header, + Body=generate_trace_response_body(RequestHead), + Len = length(Info#mod.request_line ++ Body), + Response=["HTTP/1.1 200 OK\r\n", + "Content-Type:message/http\r\n", + "Content-Length:",integer_to_list(Len),"\r\n\r\n", + Info#mod.request_line,Body], + httpd_socket:deliver(Info#mod.socket_type,Info#mod.socket,Response), + {proceed,[{response,{already_sent,200,Len}}|Info#mod.data]}. + +generate_trace_response_body(Parsed_header)-> + generate_trace_response_body(Parsed_header,[]). + +generate_trace_response_body([],Head)-> + lists:flatten(Head); +generate_trace_response_body([{[],[]}|Rest],Head) -> + generate_trace_response_body(Rest,Head); +generate_trace_response_body([{Field,Value}|Rest],Head) -> + generate_trace_response_body(Rest,[Field ++ ":" ++ Value ++ "\r\n"|Head]). + + + +%%---------------------------------------------------------------------- +%%Function that controls whether a response is generated or not +%%---------------------------------------------------------------------- +response_generated(Info)-> + case proplists:get_value(status, Info#mod.data) of + %% A status code has been generated! + {_StatusCode,_PhraseArgs,_Reason}-> + true; + %%No status code control repsonsxe + undefined -> + case proplists:get_value(response, Info#mod.data) of + %% No response has been generated! + undefined -> + false; + %% A response has been generated or sent! + _Response -> + true + end + end. + diff --git a/lib/inets/src/inets_app/Makefile b/lib/inets/src/inets_app/Makefile new file mode 100644 index 0000000000..2dab99386a --- /dev/null +++ b/lib/inets/src/inets_app/Makefile @@ -0,0 +1,121 @@ +# +# %CopyrightBegin% +# +# Copyright Ericsson AB 2005-2009. All Rights Reserved. +# +# The contents of this file are subject to the Erlang Public License, +# Version 1.1, (the "License"); you may not use this file except in +# compliance with the License. You should have received a copy of the +# Erlang Public License along with this software. If not, it can be +# retrieved online at http://www.erlang.org/. +# +# Software distributed under the License is distributed on an "AS IS" +# basis, WITHOUT WARRANTY OF ANY KIND, either express or implied. See +# the License for the specific language governing rights and limitations +# under the License. +# +# %CopyrightEnd% +# +# + +include $(ERL_TOP)/make/target.mk +EBIN = ../../ebin +include $(ERL_TOP)/make/$(TARGET)/otp.mk + +# ---------------------------------------------------- +# Application version +# ---------------------------------------------------- +include ../../vsn.mk + +VSN = $(INETS_VSN) + +# ---------------------------------------------------- +# Release directory specification +# ---------------------------------------------------- +RELSYSDIR = $(RELEASE_PATH)/lib/inets-$(VSN) + + +# ---------------------------------------------------- +# Target Specs +# ---------------------------------------------------- +MODULES = \ + inets_service \ + inets \ + inets_app \ + inets_sup \ + inets_regexp + +HRL_FILES = inets_internal.hrl + +ERL_FILES = $(MODULES:%=%.erl) + +TARGET_FILES= $(MODULES:%=$(EBIN)/%.$(EMULATOR)) \ + $(APP_TARGET) \ + $(APPUP_TARGET) + +APP_FILE = inets.app +APPUP_FILE = inets.appup + +APP_SRC = $(APP_FILE).src +APP_TARGET = $(EBIN)/$(APP_FILE) + +APPUP_SRC = $(APPUP_FILE).src +APPUP_TARGET = $(EBIN)/$(APPUP_FILE) + + +# ---------------------------------------------------- +# INETS FLAGS +# ---------------------------------------------------- +INETS_FLAGS = -D'SERVER_SOFTWARE="inets/$(VSN)"' \ + + +# ---------------------------------------------------- +# FLAGS +# ---------------------------------------------------- + +ERL_COMPILE_FLAGS += $(INETS_FLAGS) \ + +'{parse_transform,sys_pre_attributes}' \ + +'{attribute,insert,app_vsn,$(APP_VSN)}' + + +# ---------------------------------------------------- +# Targets +# ---------------------------------------------------- + +debug opt: $(TARGET_FILES) + +clean: + rm -f $(TARGET_FILES) + rm -f core + +docs: + + +# ---------------------------------------------------- +# Special Build Targets +# ---------------------------------------------------- + +$(APP_TARGET): $(APP_SRC) ../../vsn.mk + sed -e 's;%VSN%;$(VSN);' $< > $@ + +$(APPUP_TARGET): $(APPUP_SRC) ../../vsn.mk + sed -e 's;%VSN%;$(VSN);' $< > $@ + + +# ---------------------------------------------------- +# Release Target +# ---------------------------------------------------- +include $(ERL_TOP)/make/otp_release_targets.mk + +release_spec: opt + $(INSTALL_DIR) $(RELSYSDIR)/src + $(INSTALL_DATA) $(HRL_FILES) $(ERL_FILES) $(RELSYSDIR)/src + $(INSTALL_DIR) $(RELSYSDIR)/ebin + $(INSTALL_DATA) $(TARGET_FILES) $(RELSYSDIR)/ebin + +release_docs_spec: + +info: + @echo "INETS_DEBUG = $(INETS_DEBUG)" + @echo "INETS_FLAGS = $(INETS_FLAGS)" + @echo "ERL_COMPILE_FLAGS = $(ERL_COMPILE_FLAGS)" diff --git a/lib/inets/src/inets_app/inets.app.src b/lib/inets/src/inets_app/inets.app.src new file mode 100644 index 0000000000..6524c3b19b --- /dev/null +++ b/lib/inets/src/inets_app/inets.app.src @@ -0,0 +1,110 @@ +%% This is an -*- erlang -*- file. +%% %CopyrightBegin% +%% +%% Copyright Ericsson AB 1997-2009. All Rights Reserved. +%% +%% The contents of this file are subject to the Erlang Public License, +%% Version 1.1, (the "License"); you may not use this file except in +%% compliance with the License. You should have received a copy of the +%% Erlang Public License along with this software. If not, it can be +%% retrieved online at http://www.erlang.org/. +%% +%% Software distributed under the License is distributed on an "AS IS" +%% basis, WITHOUT WARRANTY OF ANY KIND, either express or implied. See +%% the License for the specific language governing rights and limitations +%% under the License. +%% +%% %CopyrightEnd% +%% + +{application,inets, + [{description,"INETS CXC 138 49"}, + {vsn,"%VSN%"}, + {modules,[ + inets, + inets_sup, + inets_app, + inets_service, + inets_regexp, + + %% FTP + ftp, + ftp_progress, + ftp_response, + ftp_sup, + + %% HTTP client: + http, + httpc_handler, + httpc_handler_sup, + httpc_manager, + httpc_profile_sup, + httpc_request, + httpc_response, + httpc_sup, + http_cookie, + + http_uri, %% Proably will by used by server also in the future + + %% HTTP used by both client and server + http_chunk, + http_request, + http_response, + http_transport, + http_util, + + %% HTTP server: + httpd, + httpd_acceptor, + httpd_acceptor_sup, + httpd_cgi, + httpd_conf, + httpd_esi, + httpd_example, + httpd_file, + httpd_instance_sup, + httpd_log, + httpd_manager, + httpd_misc_sup, + httpd_request, + httpd_request_handler, + httpd_response, + httpd_script_env, + httpd_socket, + httpd_sup, + httpd_util, + mod_actions, + mod_alias, + mod_auth, + mod_auth_dets, + mod_auth_mnesia, + mod_auth_plain, + mod_auth_server, + mod_browser, + mod_cgi, + mod_dir, + mod_disk_log, + mod_esi, + mod_get, + mod_head, + mod_htaccess, + mod_include, + mod_log, + mod_range, + mod_responsecontrol, + mod_security, + mod_security_server, + mod_trace, + + %% TFTP + tftp, + tftp_binary, + tftp_engine, + tftp_file, + tftp_lib, + tftp_logger, + tftp_sup + ]}, + {registered,[inets_sup, httpc_manager]}, + {applications,[kernel,stdlib]}, + {mod,{inets_app,[]}}]}. diff --git a/lib/inets/src/inets_app/inets.appup.src b/lib/inets/src/inets_app/inets.appup.src new file mode 100644 index 0000000000..59ee1ba03d --- /dev/null +++ b/lib/inets/src/inets_app/inets.appup.src @@ -0,0 +1,50 @@ +%% This is an -*- erlang -*- file. +%% %CopyrightBegin% +%% +%% Copyright Ericsson AB 1999-2009. All Rights Reserved. +%% +%% The contents of this file are subject to the Erlang Public License, +%% Version 1.1, (the "License"); you may not use this file except in +%% compliance with the License. You should have received a copy of the +%% Erlang Public License along with this software. If not, it can be +%% retrieved online at http://www.erlang.org/. +%% +%% Software distributed under the License is distributed on an "AS IS" +%% basis, WITHOUT WARRANTY OF ANY KIND, either express or implied. See +%% the License for the specific language governing rights and limitations +%% under the License. +%% +%% %CopyrightEnd% + +{"%VSN%", + [ + {"5.1.3", + [ + {load_module, httpd_response, soft_purge, soft_purge, []}, + {update, ftp, {advanced, upgrade_from_pre_5_12}, + soft_purge, soft_purge, []}, + {update, httpc_handler, soft, soft_purge, soft_purge, []} + ] + }, + {"5.1.2", + [ + {restart_application, inets} + ] + } + ], + [ + {"5.1.3", + [ + {load_module, httpd_response, soft_purge, soft_purge, []}, + {update, ftp, {advanced, downgrade_to_pre_5_12}, + soft_purge, soft_purge, []}, + {update, httpc_handler, soft, soft_purge, soft_purge, []} + ] + }, + {"5.1.2", + [ + {restart_application, inets} + ] + } + ] +}. diff --git a/lib/inets/src/inets_app/inets.config b/lib/inets/src/inets_app/inets.config new file mode 100644 index 0000000000..adf0e3ecf1 --- /dev/null +++ b/lib/inets/src/inets_app/inets.config @@ -0,0 +1,2 @@ +[{inets,[{services,[{httpd,"/var/tmp/server_root/conf/8888.conf"}, + {httpd,"/var/tmp/server_root/conf/8080.conf"}]}]}]. diff --git a/lib/inets/src/inets_app/inets.erl b/lib/inets/src/inets_app/inets.erl new file mode 100644 index 0000000000..7977a3dc2a --- /dev/null +++ b/lib/inets/src/inets_app/inets.erl @@ -0,0 +1,740 @@ +%% +%% %CopyrightBegin% +%% +%% Copyright Ericsson AB 2006-2009. All Rights Reserved. +%% +%% The contents of this file are subject to the Erlang Public License, +%% Version 1.1, (the "License"); you may not use this file except in +%% compliance with the License. You should have received a copy of the +%% Erlang Public License along with this software. If not, it can be +%% retrieved online at http://www.erlang.org/. +%% +%% Software distributed under the License is distributed on an "AS IS" +%% basis, WITHOUT WARRANTY OF ANY KIND, either express or implied. See +%% the License for the specific language governing rights and limitations +%% under the License. +%% +%% %CopyrightEnd% +%% +%% +%%---------------------------------------------------------------------- +%% Purpose: The main interface module of the inets application +%%---------------------------------------------------------------------- + +-module(inets). + +%% API +-export([start/0, start/1, start/2, start/3, + stop/0, stop/2, + services/0, services_info/0, + service_names/0]). +-export([enable_trace/2, enable_trace/3, disable_trace/0, set_trace/1, + report_event/4]). +-export([versions/0, + print_version_info/0, print_version_info/1]). + + +%%==================================================================== +%% API +%%==================================================================== + +%%-------------------------------------------------------------------- +%% Function: start([, Type]) -> ok +%% +%% Type = permanent | transient | temporary +%% +%% Description: Starts the inets application. Default type +%% is temporary. see application(3) +%%-------------------------------------------------------------------- +start() -> + application:start(inets). + +start(Type) -> + application:start(inets, Type). + + +%%-------------------------------------------------------------------- +%% Function: start(Service, ServiceConfig [, How]) -> {ok, Pid} | +%% {error, Reason} +%% +%% Service = - ftpc | tftpd | httpc | httpd +%% ServiceConfig = ConfPropList | ConfFile +%% ConfPropList = [{Property, Value}] according to service +%% ConfFile = Path - when service is httpd +%% How = inets | stand_alone +%% +%% Description: Dynamically starts an inets service after the inets +%% application has been started. +%% +%% Note: Dynamically started services will not be handled by +%% application takeover and failover behavior when inets is run as a +%% distributed application. Nor will they be automaticly restarted +%% when the inets application is restarted, but as long as the inets +%% application is up and running they will be supervised and may be +%% soft code upgraded. Services started with the option stand alone, +%% e.i. the service is not started as part of the inets application, +%% will lose all OTP application benefits such as soft upgrade. The +%% stand alone service will be linked to the process that started it. +%% In most cases some of the supervison functionallity will still be +%% in place and in some sense the calling process has now become the +%% top supervisor. +%% -------------------------------------------------------------------- +start(Service, ServiceConfig) -> + Module = service_module(Service), + start_service(Module, ServiceConfig, inets). + +start(Service, ServiceConfig, How) -> + Module = service_module(Service), + start_service(Module, ServiceConfig, How). + +%%-------------------------------------------------------------------- +%% Function: stop() -> ok +%% +%% Description: Stops the inets application. +%%-------------------------------------------------------------------- +stop() -> + application:stop(inets). + + +%%-------------------------------------------------------------------- +%% Function: stop(Service, Pid) -> ok +%% +%% Service - ftp | tftpd | http | httpd | stand_alone +%% +%% Description: Stops a started service of the inets application or takes +%% down a stand alone "service" gracefully. +%%-------------------------------------------------------------------- +stop(stand_alone, Pid) -> + true = exit(Pid, shutdown), + ok; + +stop(Service, Pid) -> + Module = service_module(Service), + call_service(Module, stop_service, Pid). + + +%%-------------------------------------------------------------------- +%% Function: services() -> [{Service, Pid}] +%% +%% Description: Returns a list of currently running services. +%% Note: Services started with the stand alone option will not be listed +%%-------------------------------------------------------------------- +services() -> + Modules = [service_module(Service) || Service <- + service_names()], + try lists:flatten(lists:map(fun(Module) -> + Module:services() + end, Modules)) of + Result -> + Result + catch + exit:{noproc, _} -> + {error, inets_not_started} + end. + + +%%-------------------------------------------------------------------- +%% Function: services_info() -> [{Service, Pid, Info}] +%% +%% Description: Returns a list of currently running services where +%% each service is described by a [{Property, Value}] list. +%%-------------------------------------------------------------------- +services_info() -> + case services() of + {error, inets_not_started} -> + {error, inets_not_started}; + Services -> + Fun = fun({Service, Pid}) -> + Module = service_module(Service), + Info = + case Module:service_info(Pid) of + {ok, PropList} -> + PropList; + {error, Reason} -> + Reason + end, + {Service, Pid, Info} + end, + lists:flatten(lists:map(Fun, Services)) + end. + + + +%%-------------------------------------------------------------------- +%% Function: print_version_info() +%% +%% Description: Simple utility function to print information +%% about versions (system, OS and modules). +%%-------------------------------------------------------------------- + +print_version_info() -> + {ok, Versions} = inets:versions(), + print_version_info(Versions). + +print_version_info(Versions) when is_list(Versions) -> + print_sys_info(Versions), + print_os_info(Versions), + print_mods_info(Versions). + +print_sys_info(Versions) -> + case key1search(sys_info, Versions) of + {value, SysInfo} when is_list(SysInfo) -> + {value, Arch} = key1search(arch, SysInfo, "Not found"), + {value, Ver} = key1search(ver, SysInfo, "Not found"), + io:format("System info: " + "~n Arch: ~s" + "~n Ver: ~s" + "~n", [Arch, Ver]), + ok; + _ -> + io:format("System info: Not found~n", []), + not_found + end. + +print_os_info(Versions) -> + case key1search(os_info, Versions) of + {value, OsInfo} when is_list(OsInfo) -> + Fam = + case key1search(fam, OsInfo, "Not found") of + {value, F} when is_atom(F) -> + atom_to_list(F); + {value, LF} when is_list(LF) -> + LF; + {value, XF} -> + lists:flatten(io_lib:format("~p", [XF])) + end, + Name = + case key1search(name, OsInfo) of + {value, N} when is_atom(N) -> + "[" ++ atom_to_list(N) ++ "]"; + {value, LN} when is_list(LN) -> + "[" ++ LN ++ "]"; + not_found -> + "" + end, + Ver = + case key1search(ver, OsInfo, "Not found") of + {value, T} when is_tuple(T) -> + tversion(T); + {value, LV} when is_list(LV) -> + LV; + {value, XV} -> + lists:flatten(io_lib:format("~p", [XV])) + end, + io:format("OS info: " + "~n Family: ~s ~s" + "~n Ver: ~s" + "~n", [Fam, Name, Ver]), + ok; + _ -> + io:format("OS info: Not found~n", []), + not_found + end. + +versions() -> + App = inets, + LibDir = code:lib_dir(App), + File = filename:join([LibDir, "ebin", atom_to_list(App) ++ ".app"]), + case file:consult(File) of + {ok, [{application, App, AppFile}]} -> + case lists:keysearch(modules, 1, AppFile) of + {value, {modules, Mods}} -> + {ok, version_info(Mods)}; + _ -> + {error, {invalid_format, modules}} + end; + Error -> + {error, {invalid_format, Error}} + end. + +version_info(Mods) -> + SysInfo = sys_info(), + OsInfo = os_info(), + ModInfo = [mod_version_info(Mod) || Mod <- Mods], + [{sys_info, SysInfo}, {os_info, OsInfo}, {mod_info, ModInfo}]. + +mod_version_info(Mod) -> + Info = Mod:module_info(), + {value, {attributes, Attr}} = lists:keysearch(attributes, 1, Info), + {value, {vsn, [Vsn]}} = lists:keysearch(vsn, 1, Attr), + {value, {app_vsn, AppVsn}} = lists:keysearch(app_vsn, 1, Attr), + {value, {compile, Comp}} = lists:keysearch(compile, 1, Info), + {value, {version, Ver}} = lists:keysearch(version, 1, Comp), + {value, {time, Time}} = lists:keysearch(time, 1, Comp), + {Mod, [{vsn, Vsn}, + {app_vsn, AppVsn}, + {compiler_version, Ver}, + {compile_time, Time}]}. + +sys_info() -> + SysArch = string:strip(erlang:system_info(system_architecture),right,$\n), + SysVer = string:strip(erlang:system_info(system_version),right,$\n), + [{arch, SysArch}, {ver, SysVer}]. + +os_info() -> + V = os:version(), + case os:type() of + {OsFam, OsName} -> + [{fam, OsFam}, {name, OsName}, {ver, V}]; + OsFam -> + [{fam, OsFam}, {ver, V}] + end. + + +print_mods_info(Versions) -> + case key1search(mod_info, Versions) of + {value, ModsInfo} when is_list(ModsInfo) -> + io:format("Module info: ~n", []), + lists:foreach(fun print_mod_info/1, ModsInfo); + _ -> + io:format("Module info: Not found~n", []), + not_found + end. + +tversion(T) -> + L = tuple_to_list(T), + lversion(L). + +lversion([]) -> + ""; +lversion([A]) -> + integer_to_list(A); +lversion([A|R]) -> + integer_to_list(A) ++ "." ++ lversion(R). + +print_mod_info({Module, Info}) -> + % Maybe a asn1 generated module + Asn1Vsn = + case (catch Module:info()) of + AI when is_list(AI) -> + case (catch key1search(vsn, AI)) of + {value, V} when is_atom(V) -> + atom_to_list(V); + _ -> + "-" + end; + _ -> + "-" + end, + Vsn = + case key1search(vsn, Info) of + {value, I} when is_integer(I) -> + integer_to_list(I); + _ -> + "Not found" + end, + AppVsn = + case key1search(app_vsn, Info) of + {value, S1} when is_list(S1) -> + S1; + _ -> + "Not found" + end, + CompVer = + case key1search(compiler_version, Info) of + {value, S2} when is_list(S2) -> + S2; + _ -> + "Not found" + end, + CompDate = + case key1search(compile_time, Info) of + {value, {Year, Month, Day, Hour, Min, Sec}} -> + lists:flatten( + io_lib:format("~w-~2..0w-~2..0w ~2..0w:~2..0w:~2..0w", + [Year, Month, Day, Hour, Min, Sec])); + _ -> + "Not found" + end, + io:format(" ~w:~n" + " Vsn: ~s~n" + " App vsn: ~s~n" + " ASN.1 vsn: ~s~n" + " Compiler ver: ~s~n" + " Compile time: ~s~n", + [Module, Vsn, AppVsn, Asn1Vsn, CompVer, CompDate]), + ok. + + +key1search(Key, Vals) -> + case lists:keysearch(Key, 1, Vals) of + {value, {Key, Val}} -> + {value, Val}; + false -> + not_found + end. + +key1search(Key, Vals, Def) -> + case key1search(Key, Vals) of + not_found -> + {value, Def}; + Value -> + Value + end. + + +%%-------------------------------------------------------------------- +%% Function: service_names() -> [ServiceName] +%% +%% ServiceName = atom() +%% +%% Description: Returns a list of supported services +%%------------------------------------------------------------------- +service_names() -> + [ftpc, tftpd, httpc, httpd]. + + +%%----------------------------------------------------------------- +%% enable_trace(Level, Destination) -> void() +%% enable_trace(Level, Destination, Service) -> void() +%% +%% Parameters: +%% Level -> max | min | integer() +%% Destination -> File | Port | io | HandlerSpec +%% Service -> httpc | httpd | ftpc | tftp | all +%% File -> string() +%% Port -> integer() +%% Verbosity -> true | false +%% HandlerSpec = {function(), Data} +%% Data = term() +%% +%% Description: +%% This function is used to start tracing at level Level and send +%% the result either to the file File, the port Port or to a +%% trace handler. +%% Note that it starts a tracer server. +%% When Destination is the atom io (or the tuple {io, Verbosity}), +%% all (printable) inets trace events (trace_ts events which has +%% Severity withing Limit) will be written to stdout using io:format. +%% +%%----------------------------------------------------------------- +enable_trace(Level, Dest) -> + enable_trace(Level, Dest, all). + +enable_trace(Level, Dest, Service) -> + case valid_trace_service(Service) of + true -> + enable_trace2(Level, Dest, Service); + false -> + {error, {invalid_service, Service}} + end. + +enable_trace2(Level, File, Service) + when is_list(File) -> + case file:open(File, [write]) of + {ok, Fd} -> + HandleSpec = {fun handle_trace/2, {Service, Fd}}, + do_enable_trace(Level, process, HandleSpec); + Err -> + Err + end; +enable_trace2(Level, Port, _) when is_integer(Port) -> + do_enable_trace(Level, port, dbg:trace_port(ip, Port)); +enable_trace2(Level, io, Service) -> + HandleSpec = {fun handle_trace/2, {Service, standard_io}}, + do_enable_trace(Level, process, HandleSpec); +enable_trace2(Level, {Fun, _Data} = HandleSpec, _) when is_function(Fun) -> + do_enable_trace(Level, process, HandleSpec). + +do_enable_trace(Level, Type, HandleSpec) -> + case dbg:tracer(Type, HandleSpec) of + {ok, _} -> + set_trace(Level), + ok; + Error -> + Error + end. + +valid_trace_service(all) -> + true; +valid_trace_service(Service) -> + lists:member(Service, [httpc, httpd, ftpc, tftp]). + + +%%----------------------------------------------------------------- +%% disable_trace() -> void() +%% +%% Description: +%% This function is used to stop tracing. +%%----------------------------------------------------------------- +disable_trace() -> + %% This is to make handle_trace/2 close the output file (if the + %% event gets there before dbg closes) + inets:report_event(100, "stop trace", stop_trace, [stop_trace]), + dbg:stop(). + + + +%%----------------------------------------------------------------- +%% set_trace(Level) -> void() +%% +%% Parameters: +%% Level -> max | min | integer() +%% +%% Description: +%% This function is used to change the trace level when tracing has +%% already been started. +%%----------------------------------------------------------------- +set_trace(Level) -> + set_trace(Level, all). + +set_trace(Level, Service) -> + Pat = make_pattern(?MODULE, Service, Level), + change_pattern(Pat). + +make_pattern(Mod, Service, Level) + when is_atom(Mod) andalso is_atom(Service) -> + case Level of + min -> + {Mod, Service, []}; + max -> + Head = ['$1', '_', '_', '_'], + Body = [], + Cond = [], + {Mod, Service, [{Head, Cond, Body}]}; + DetailLevel when is_integer(DetailLevel) -> + Head = ['$1', '_', '_', '_'], + Body = [], + Cond = [{ '=<', '$1', DetailLevel}], + {Mod, Service, [{Head, Cond, Body}]}; + _ -> + exit({bad_level, Level}) + end. + +change_pattern({Mod, Service, Pattern}) + when is_atom(Mod) andalso is_atom(Service) -> + MFA = {Mod, report_event, 4}, + case Pattern of + [] -> + try + error_to_exit(ctp, dbg:ctp(MFA)), + error_to_exit(p, dbg:p(all, clear)) + catch + exit:{Where, Reason} -> + {error, {Where, Reason}} + end; + List when is_list(List) -> + try + error_to_exit(ctp, dbg:ctp(MFA)), + error_to_exit(tp, dbg:tp(MFA, Pattern)), + error_to_exit(p, dbg:p(all, [call, timestamp])) + catch + exit:{Where, Reason} -> + {error, {Where, Reason}} + end; + _ -> + exit({bad_pattern, Pattern}) + end, + ok. + +error_to_exit(_Where, {ok, _} = OK) -> + OK; +error_to_exit(Where, {error, Reason}) -> + exit({Where, Reason}). + + +%%----------------------------------------------------------------- +%% report_event(Serverity, Label, Service, Content) +%% +%% Parameters: +%% Severity -> 0 =< integer() =< 100 +%% Label -> string() +%% Service -> httpd | httpc | ftp | tftp +%% Content -> [{tag, term()}] +%% +%% Description: +%% This function is used to generate trace events, that is, +%% put trace on this function. +%%----------------------------------------------------------------- + +report_event(Severity, Label, Service, Content) + when (is_integer(Severity) andalso + (Severity >= 0) andalso (100 >= Severity)) andalso + is_list(Label) andalso + is_atom(Service) andalso + is_list(Content) -> + hopefully_traced. + + +%% ---------------------------------------------------------------------- +%% handle_trace(Event, Fd) -> Verbosity +%% +%% Parameters: +%% Event -> The trace event (only megaco 'trace_ts' events are printed) +%% Fd -> standard_io | file_descriptor() | trace_port() +%% +%% Description: +%% This function is used to "receive" and print the trace events. +%% Events are printed if: +%% - Verbosity is max +%% - Severity is =< Verbosity (e.g. Severity = 30, and Verbosity = 40) +%% Events are not printed if: +%% - Verbosity is min +%% - Severity is > Verbosity +%%----------------------------------------------------------------- + +handle_trace(_, closed_file = Fd) -> + Fd; +handle_trace({trace_ts, _Who, call, + {?MODULE, report_event, + [_Sev, "stop trace", stop_trace, [stop_trace]]}, + Timestamp}, + {standard_io, _} = Fd) -> + (catch io:format(Fd, "stop trace at ~s~n", [format_timestamp(Timestamp)])), + Fd; +handle_trace({trace_ts, _Who, call, + {?MODULE, report_event, + [_Sev, "stop trace", stop_trace, [stop_trace]]}, + Timestamp}, + standard_io = Fd) -> + (catch io:format(Fd, "stop trace at ~s~n", [format_timestamp(Timestamp)])), + Fd; +handle_trace({trace_ts, _Who, call, + {?MODULE, report_event, + [_Sev, "stop trace", stop_trace, [stop_trace]]}, + Timestamp}, + {_Service, Fd}) -> + (catch io:format(Fd, "stop trace at ~s~n", [format_timestamp(Timestamp)])), + (catch file:close(Fd)), + closed_file; +handle_trace({trace_ts, _Who, call, + {?MODULE, report_event, + [_Sev, "stop trace", stop_trace, [stop_trace]]}, + Timestamp}, + Fd) -> + (catch io:format(Fd, "stop trace at ~s~n", [format_timestamp(Timestamp)])), + (catch file:close(Fd)), + closed_file; +handle_trace({trace_ts, Who, call, + {?MODULE, report_event, + [Sev, Label, Service, Content]}, Timestamp}, + Fd) -> + (catch print_inets_trace(Fd, Sev, Timestamp, Who, + Label, Service, Content)), + Fd; +handle_trace(Event, Fd) -> + (catch print_trace(Fd, Event)), + Fd. + + +print_inets_trace({Service, Fd}, + Sev, Timestamp, Who, Label, Service, Content) -> + do_print_inets_trace(Fd, Sev, Timestamp, Who, Label, Service, Content); +print_inets_trace({ServiceA, Fd}, + Sev, Timestamp, Who, Label, ServiceB, Content) + when (ServiceA =:= all) -> + do_print_inets_trace(Fd, Sev, Timestamp, Who, Label, ServiceB, Content); +print_inets_trace({ServiceA, _Fd}, + _Sev, _Timestamp, _Who, _Label, ServiceB, _Content) + when ServiceA =/= ServiceB -> + ok; +print_inets_trace(Fd, Sev, Timestamp, Who, Label, Service, Content) -> + do_print_inets_trace(Fd, Sev, Timestamp, Who, Label, Service, Content). + +do_print_inets_trace(Fd, Sev, Timestamp, Who, Label, Service, Content) -> + Ts = format_timestamp(Timestamp), + io:format(Fd, "[inets ~w trace ~w ~w ~s] ~s " + "~n Content: ~p" + "~n", + [Service, Sev, Who, Ts, Label, Content]). + +print_trace({_, Fd}, Event) -> + do_print_trace(Fd, Event); +print_trace(Fd, Event) -> + do_print_trace(Fd, Event). + +do_print_trace(Fd, {trace, Who, What, Where}) -> + io:format(Fd, "[trace]" + "~n Who: ~p" + "~n What: ~p" + "~n Where: ~p" + "~n", [Who, What, Where]); + +do_print_trace(Fd, {trace, Who, What, Where, Extra}) -> + io:format(Fd, "[trace]" + "~n Who: ~p" + "~n What: ~p" + "~n Where: ~p" + "~n Extra: ~p" + "~n", [Who, What, Where, Extra]); + +do_print_trace(Fd, {trace_ts, Who, What, Where, When}) -> + Ts = format_timestamp(When), + io:format(Fd, "[trace ~s]" + "~n Who: ~p" + "~n What: ~p" + "~n Where: ~p" + "~n", [Ts, Who, What, Where]); + +do_print_trace(Fd, {trace_ts, Who, What, Where, Extra, When}) -> + Ts = format_timestamp(When), + io:format(Fd, "[trace ~s]" + "~n Who: ~p" + "~n What: ~p" + "~n Where: ~p" + "~n Extra: ~p" + "~n", [Ts, Who, What, Where, Extra]); + +do_print_trace(Fd, {seq_trace, What, Where}) -> + io:format(Fd, "[seq trace]" + "~n What: ~p" + "~n Where: ~p" + "~n", [What, Where]); + +do_print_trace(Fd, {seq_trace, What, Where, When}) -> + Ts = format_timestamp(When), + io:format(Fd, "[seq trace ~s]" + "~n What: ~p" + "~n Where: ~p" + "~n", [Ts, What, Where]); + +do_print_trace(Fd, {drop, Num}) -> + io:format(Fd, "[drop trace] ~p~n", [Num]); + +do_print_trace(Fd, Trace) -> + io:format(Fd, "[trace] " + "~n ~p" + "~n", [Trace]). + + +format_timestamp({_N1, _N2, N3} = Now) -> + {Date, Time} = calendar:now_to_datetime(Now), + {YYYY,MM,DD} = Date, + {Hour,Min,Sec} = Time, + FormatDate = + io_lib:format("~.4w:~.2.0w:~.2.0w ~.2.0w:~.2.0w:~.2.0w 4~w", + [YYYY,MM,DD,Hour,Min,Sec,round(N3/1000)]), + lists:flatten(FormatDate). + + +%%-------------------------------------------------------------------- +%%% Internal functions +%%-------------------------------------------------------------------- +start_service(Service, Args, stand_alone) -> + Service:start_standalone(Args); +start_service(Service, Args, inets) -> + call_service(Service, start_service, Args). + +call_service(Service, Call, Args) -> + try Service:Call(Args) of + Result -> + Result + catch + exit:{noproc, _} -> + {error, inets_not_started} + end. + +service_module(tftpd) -> + tftp; +service_module(httpc) -> + http; +service_module(ftpc) -> + ftp; +service_module(Service) -> + Service. + + + + + + diff --git a/lib/inets/src/inets_app/inets_app.erl b/lib/inets/src/inets_app/inets_app.erl new file mode 100644 index 0000000000..cae79a6767 --- /dev/null +++ b/lib/inets/src/inets_app/inets_app.erl @@ -0,0 +1,30 @@ +%% +%% %CopyrightBegin% +%% +%% Copyright Ericsson AB 2004-2009. All Rights Reserved. +%% +%% The contents of this file are subject to the Erlang Public License, +%% Version 1.1, (the "License"); you may not use this file except in +%% compliance with the License. You should have received a copy of the +%% Erlang Public License along with this software. If not, it can be +%% retrieved online at http://www.erlang.org/. +%% +%% Software distributed under the License is distributed on an "AS IS" +%% basis, WITHOUT WARRANTY OF ANY KIND, either express or implied. See +%% the License for the specific language governing rights and limitations +%% under the License. +%% +%% %CopyrightEnd% +%% +%% +-module(inets_app). + +-behaviour(application). + +-export([start/2, stop/1]). + +start(_Type, _State) -> + supervisor:start_link({local, inets_sup}, inets_sup, []). + +stop(_State) -> + ok. diff --git a/lib/inets/src/inets_app/inets_internal.hrl b/lib/inets/src/inets_app/inets_internal.hrl new file mode 100644 index 0000000000..55c3669e4a --- /dev/null +++ b/lib/inets/src/inets_app/inets_internal.hrl @@ -0,0 +1,49 @@ +%% +%% %CopyrightBegin% +%% +%% Copyright Ericsson AB 2005-2009. All Rights Reserved. +%% +%% The contents of this file are subject to the Erlang Public License, +%% Version 1.1, (the "License"); you may not use this file except in +%% compliance with the License. You should have received a copy of the +%% Erlang Public License along with this software. If not, it can be +%% retrieved online at http://www.erlang.org/. +%% +%% Software distributed under the License is distributed on an "AS IS" +%% basis, WITHOUT WARRANTY OF ANY KIND, either express or implied. See +%% the License for the specific language governing rights and limitations +%% under the License. +%% +%% %CopyrightEnd% +%% +%% + +-ifndef(inets_internal_hrl). +-define(inets_internal_hrl, true). + +%% Various trace macros + +-define(report(Severity, Label, Service, Content), + inets:report_event(Severity, Label, Service, + [{module, ?MODULE}, {line, ?LINE} | Content])). +-define(report_important(Label, Service, Content), + ?report(20, Label, Service, Content)). +-define(report_verbose(Label, Service, Content), + ?report(40, Label, Service, Content)). +-define(report_debug(Label, Service, Content), + ?report(60, Label, Service, Content)). +-define(report_trace(Label, Service, Content), + ?report(80, Label, Service, Content)). + + +-define(CR, $\r). +-define(LF, $\n). +-define(CRLF, [$\r,$\n]). +-define(SP, $\s). +-define(TAB, $\t). +-define(LEFT_PAREN, $(). +-define(RIGHT_PAREN, $)). +-define(WHITE_SPACE, $ ). +-define(DOUBLE_QUOTE, $"). + +-endif. % -ifdef(inets_internal_hrl). diff --git a/lib/inets/src/inets_app/inets_regexp.erl b/lib/inets/src/inets_app/inets_regexp.erl new file mode 100644 index 0000000000..a065533236 --- /dev/null +++ b/lib/inets/src/inets_app/inets_regexp.erl @@ -0,0 +1,413 @@ +%% +%% %CopyrightBegin% +%% +%% Copyright Ericsson AB 2009. All Rights Reserved. +%% +%% The contents of this file are subject to the Erlang Public License, +%% Version 1.1, (the "License"); you may not use this file except in +%% compliance with the License. You should have received a copy of the +%% Erlang Public License along with this software. If not, it can be +%% retrieved online at http://www.erlang.org/. +%% +%% Software distributed under the License is distributed on an "AS IS" +%% basis, WITHOUT WARRANTY OF ANY KIND, either express or implied. See +%% the License for the specific language governing rights and limitations +%% under the License. +%% +%% %CopyrightEnd% +%% + +-module(inets_regexp). + +-export([parse/1, match/2, first_match/2, split/2, sub/3, gsub/3]). + + +%%%========================================================================= +%%% API +%%%========================================================================= + +%% parse(RegExp) -> {ok, RE} | {error, E}. +%% Parse the regexp described in the string RegExp. + +parse(S) -> + case (catch reg(S)) of + {R, []} -> + {ok, R}; + {_R, [C|_]} -> + {error, {illegal, [C]}}; + {error, E} -> + {error, E} + end. + + +%% Find the longest match of RegExp in String. + +match(S, RegExp) when is_list(RegExp) -> + case parse(RegExp) of + {ok,RE} -> match(S, RE); + {error,E} -> {error,E} + end; +match(S, RE) -> + case match(RE, S, 1, 0, -1) of + {Start,Len} when Len >= 0 -> + {match, Start, Len}; + {_Start,_Len} -> + nomatch + end. + +%% Find the first match of RegExp in String. + +first_match(S, RegExp) when is_list(RegExp) -> + case parse(RegExp) of + {ok, RE} -> + first_match(S, RE); + {error, E} -> + {error, E} + end; +first_match(S, RE) -> + case first_match(RE, S, 1) of + {Start,Len} when Len >= 0 -> + {match, Start,Len}; + nomatch -> + nomatch + end. + +first_match(RE, S, St) when S =/= [] -> + case re_apply(S, St, RE) of + {match, P, _Rest} -> + {St, P-St}; + nomatch -> + first_match(RE, tl(S), St+1) + end; +first_match(_RE, [], _St) -> + nomatch. + + +match(RE, S, St, Pos, L) -> + case first_match(RE, S, St) of + {St1, L1} -> + Nst = St1 + 1, + if L1 > L -> + match(RE, lists:nthtail(Nst-St, S), Nst, St1, L1); + true -> + match(RE, lists:nthtail(Nst-St, S), Nst, Pos, L) + end; + nomatch -> + {Pos, L} + end. + + +%% Split a string into substrings where the RegExp describes the +%% field seperator. The RegExp " " is specially treated. + +split(String, " ") -> %This is really special + {ok, RE} = parse("[ \t]+"), + case split_apply(String, RE, true) of + [[]|Ss] -> + {ok,Ss}; + Ss -> + {ok,Ss} + end; +split(String, RegExp) when is_list(RegExp) -> + case parse(RegExp) of + {ok, RE} -> + {ok, split_apply(String, RE, false)}; + {error, E} -> + {error,E} + end; +split(String, RE) -> + {ok, split_apply(String, RE, false)}. + + +%% Substitute the first match of the regular expression RegExp +%% with the string Replace in String. Accept pre-parsed regular +%% expressions. + +sub(String, RegExp, Rep) when is_list(RegExp) -> + case parse(RegExp) of + {ok, RE} -> + sub(String, RE, Rep); + {error, E} -> + {error, E} + end; +sub(String, RE, Rep) -> + Ss = sub_match(String, RE, 1), + {ok, sub_repl(Ss, Rep, String, 1), length(Ss)}. + + +%% Substitute every match of the regular expression RegExp with +%% the string New in String. Accept pre-parsed regular expressions. + +gsub(String, RegExp, Rep) when is_list(RegExp) -> + case parse(RegExp) of + {ok, RE} -> + gsub(String, RE, Rep); + {error, E} -> + {error, E} + end; +gsub(String, RE, Rep) -> + Ss = matches(String, RE, 1), + {ok, sub_repl(Ss, Rep, String, 1), length(Ss)}. + + +%%%======================================================================== +%%% Internal functions +%%%======================================================================== + +%% This is the regular expression grammar used. It is equivalent to the +%% one used in AWK, except that we allow ^ $ to be used anywhere and fail +%% in the matching. +%% +%% reg -> reg1 : '$1'. +%% reg1 -> reg1 "|" reg2 : {'or','$1','$2'}. +%% reg1 -> reg2 : '$1'. +%% reg2 -> reg2 reg3 : {concat,'$1','$2'}. +%% reg2 -> reg3 : '$1'. +%% reg3 -> reg3 "*" : {kclosure,'$1'}. +%% reg3 -> reg3 "+" : {pclosure,'$1'}. +%% reg3 -> reg3 "?" : {optional,'$1'}. +%% reg3 -> reg4 : '$1'. +%% reg4 -> "(" reg ")" : '$2'. +%% reg4 -> "\\" char : '$2'. +%% reg4 -> "^" : bos. +%% reg4 -> "$" : eos. +%% reg4 -> "." : char. +%% reg4 -> "[" class "]" : {char_class,char_class('$2')} +%% reg4 -> "[" "^" class "]" : {comp_class,char_class('$3')} +%% reg4 -> "\"" chars "\"" : char_string('$2') +%% reg4 -> char : '$1'. +%% reg4 -> empty : epsilon. +%% The grammar of the current regular expressions. The actual parser +%% is a recursive descent implementation of the grammar. + +reg(S) -> reg1(S). + +%% reg1 -> reg2 reg1' +%% reg1' -> "|" reg2 +%% reg1' -> empty + +reg1(S0) -> + {L,S1} = reg2(S0), + reg1p(S1, L). + +reg1p([$||S0], L) -> + {R,S1} = reg2(S0), + reg1p(S1, {'or',L,R}); +reg1p(S, L) -> {L,S}. + +%% reg2 -> reg3 reg2' +%% reg2' -> reg3 +%% reg2' -> empty + +reg2(S0) -> + {L,S1} = reg3(S0), + reg2p(S1, L). + +reg2p([C|S0], L) when (C =/= $|) andalso (C =/= $)) -> + {R,S1} = reg3([C|S0]), + reg2p(S1, {concat,L,R}); +reg2p(S, L) -> {L,S}. + +%% reg3 -> reg4 reg3' +%% reg3' -> "*" reg3' +%% reg3' -> "+" reg3' +%% reg3' -> "?" reg3' +%% reg3' -> empty + +reg3(S0) -> + {L,S1} = reg4(S0), + reg3p(S1, L). + +reg3p([$*|S], L) -> reg3p(S, {kclosure,L}); +reg3p([$+|S], L) -> reg3p(S, {pclosure,L}); +reg3p([$?|S], L) -> reg3p(S, {optional,L}); +reg3p(S, L) -> {L,S}. + +reg4([$(|S0]) -> + case reg(S0) of + {R,[$)|S1]} -> {R,S1}; + {_R,_S} -> throw({error,{unterminated,"("}}) + end; +reg4([$\\,O1,O2,O3|S]) + when ((O1 >= $0) andalso + (O1 =< $7) andalso + (O2 >= $0) andalso + (O2 =< $7) andalso + (O3 >= $0) andalso + (O3 =< $7)) -> + {(O1*8 + O2)*8 + O3 - 73*$0,S}; +reg4([$\\,C|S]) -> + {escape_char(C),S}; +reg4([$\\]) -> + throw({error, {unterminated,"\\"}}); +reg4([$^|S]) -> + {bos,S}; +reg4([$$|S]) -> + {eos,S}; +reg4([$.|S]) -> + {{comp_class,"\n"},S}; +reg4("[^" ++ S0) -> + case char_class(S0) of + {Cc,[$]|S1]} -> {{comp_class,Cc},S1}; + {_Cc,_S} -> throw({error,{unterminated,"["}}) + end; +reg4([$[|S0]) -> + case char_class(S0) of + {Cc,[$]|S1]} -> {{char_class,Cc},S1}; + {_Cc,_S1} -> throw({error,{unterminated,"["}}) + end; +reg4([C|S]) + when (C =/= $*) andalso (C =/= $+) andalso (C =/= $?) andalso (C =/= $]) -> + {C, S}; +reg4([C|_S]) -> + throw({error,{illegal,[C]}}); +reg4([]) -> + {epsilon,[]}. + +escape_char($n) -> $\n; %\n = LF +escape_char($r) -> $\r; %\r = CR +escape_char($t) -> $\t; %\t = TAB +escape_char($v) -> $\v; %\v = VT +escape_char($b) -> $\b; %\b = BS +escape_char($f) -> $\f; %\f = FF +escape_char($e) -> $\e; %\e = ESC +escape_char($s) -> $\s; %\s = SPACE +escape_char($d) -> $\d; %\d = DEL +escape_char(C) -> C. + +char_class([$]|S]) -> char_class(S, [$]]); +char_class(S) -> char_class(S, []). + +char($\\, [O1,O2,O3|S]) when + O1 >= $0, O1 =< $7, O2 >= $0, O2 =< $7, O3 >= $0, O3 =< $7 -> + {(O1*8 + O2)*8 + O3 - 73*$0,S}; +char($\\, [C|S]) -> {escape_char(C),S}; +char(C, S) -> {C,S}. + +char_class([C1|S0], Cc) when C1 =/= $] -> + case char(C1, S0) of + {Cf,[$-,C2|S1]} when C2 =/= $] -> + case char(C2, S1) of + {Cl,S2} when Cf < Cl -> char_class(S2, [{Cf,Cl}|Cc]); + {Cl,_S2} -> throw({error,{char_class,[Cf,$-,Cl]}}) + end; + {C,S1} -> char_class(S1, [C|Cc]) + end; +char_class(S, Cc) -> {Cc,S}. + + +%% re_apply(String, StartPos, RegExp) -> re_app_res(). +%% +%% Apply the (parse of the) regular expression RegExp to String. If +%% there is a match return the position of the remaining string and +%% the string if else return 'nomatch'. BestMatch specifies if we want +%% the longest match, or just a match. +%% +%% StartPos should be the real start position as it is used to decide +%% if we ae at the beginning of the string. +%% +%% Pass two functions to re_apply_or so it can decide, on the basis +%% of BestMatch, whether to just any take any match or try both to +%% find the longest. This is slower but saves duplicatng code. + +re_apply(S, St, RE) -> re_apply(RE, [], S, St). + +re_apply(epsilon, More, S, P) -> %This always matches + re_apply_more(More, S, P); +re_apply({'or',RE1,RE2}, More, S, P) -> + re_apply_or(re_apply(RE1, More, S, P), + re_apply(RE2, More, S, P)); +re_apply({concat,RE1,RE2}, More, S0, P) -> + re_apply(RE1, [RE2|More], S0, P); +re_apply({kclosure,CE}, More, S, P) -> + %% Be careful with the recursion, explicitly do one call before + %% looping. + re_apply_or(re_apply_more(More, S, P), + re_apply(CE, [{kclosure,CE}|More], S, P)); +re_apply({pclosure,CE}, More, S, P) -> + re_apply(CE, [{kclosure,CE}|More], S, P); +re_apply({optional,CE}, More, S, P) -> + re_apply_or(re_apply_more(More, S, P), + re_apply(CE, More, S, P)); +re_apply(bos, More, S, 1) -> re_apply_more(More, S, 1); +re_apply(eos, More, [$\n|S], P) -> re_apply_more(More, S, P); +re_apply(eos, More, [], P) -> re_apply_more(More, [], P); +re_apply({char_class,Cc}, More, [C|S], P) -> + case in_char_class(C, Cc) of + true -> re_apply_more(More, S, P+1); + false -> nomatch + end; +re_apply({comp_class,Cc}, More, [C|S], P) -> + case in_char_class(C, Cc) of + true -> nomatch; + false -> re_apply_more(More, S, P+1) + end; +re_apply(C, More, [C|S], P) when is_integer(C) -> + re_apply_more(More, S, P+1); +re_apply(_RE, _More, _S, _P) -> nomatch. + +%% re_apply_more([RegExp], String, Length) -> re_app_res(). + +re_apply_more([RE|More], S, P) -> re_apply(RE, More, S, P); +re_apply_more([], S, P) -> {match,P,S}. + +%% in_char_class(Char, Class) -> bool(). + +in_char_class(C, [{C1,C2}|_Cc]) when C >= C1, C =< C2 -> true; +in_char_class(C, [C|_Cc]) -> true; +in_char_class(C, [_|Cc]) -> in_char_class(C, Cc); +in_char_class(_C, []) -> false. + +%% re_apply_or(Match1, Match2) -> re_app_res(). +%% If we want the best match then choose the longest match, else just +%% choose one by trying sequentially. + +re_apply_or({match,P1,S1}, {match,P2,_S2}) when P1 >= P2 -> {match,P1,S1}; +re_apply_or({match,_P1,_S1}, {match,P2,S2}) -> {match,P2,S2}; +re_apply_or(nomatch, R2) -> R2; +re_apply_or(R1, nomatch) -> R1. + + +matches(S, RE, St) -> + case first_match(RE, S, St) of + {St1,0} -> + [{St1,0}|matches(string:substr(S, St1+2-St), RE, St1+1)]; + {St1,L1} -> + [{St1,L1}|matches(string:substr(S, St1+L1+1-St), RE, St1+L1)]; + nomatch -> + [] + end. + +sub_match(S, RE, St) -> + case first_match(RE, S, St) of + {St1,L1} -> [{St1,L1}]; + nomatch -> [] + end. + +sub_repl([{St,L}|Ss], Rep, S, Pos) -> + Rs = sub_repl(Ss, Rep, S, St+L), + string:substr(S, Pos, St-Pos) ++ + sub_repl(Rep, string:substr(S, St, L), Rs); +sub_repl([], _Rep, S, Pos) -> + string:substr(S, Pos). + +sub_repl([$&|Rep], M, Rest) -> M ++ sub_repl(Rep, M, Rest); +sub_repl("\\&" ++ Rep, M, Rest) -> [$&|sub_repl(Rep, M, Rest)]; +sub_repl([C|Rep], M, Rest) -> [C|sub_repl(Rep, M, Rest)]; +sub_repl([], _M, Rest) -> Rest. + +split_apply(S, RE, Trim) -> split_apply(S, 1, RE, Trim, []). + +split_apply([], _P, _RE, true, []) -> + []; +split_apply([], _P, _RE, _T, Sub) -> + [lists:reverse(Sub)]; +split_apply(S, P, RE, T, Sub) -> + case re_apply(S, P, RE) of + {match,P,_Rest} -> + split_apply(tl(S), P+1, RE, T, [hd(S)|Sub]); + {match,P1,Rest} -> + [lists:reverse(Sub)|split_apply(Rest, P1, RE, T, [])]; + nomatch -> + split_apply(tl(S), P+1, RE, T, [hd(S)|Sub]) + end. diff --git a/lib/inets/src/inets_app/inets_service.erl b/lib/inets/src/inets_app/inets_service.erl new file mode 100644 index 0000000000..3499314d54 --- /dev/null +++ b/lib/inets/src/inets_app/inets_service.erl @@ -0,0 +1,65 @@ +%% +%% %CopyrightBegin% +%% +%% Copyright Ericsson AB 2007-2009. All Rights Reserved. +%% +%% The contents of this file are subject to the Erlang Public License, +%% Version 1.1, (the "License"); you may not use this file except in +%% compliance with the License. You should have received a copy of the +%% Erlang Public License along with this software. If not, it can be +%% retrieved online at http://www.erlang.org/. +%% +%% Software distributed under the License is distributed on an "AS IS" +%% basis, WITHOUT WARRANTY OF ANY KIND, either express or implied. See +%% the License for the specific language governing rights and limitations +%% under the License. +%% +%% %CopyrightEnd% +%% +%% + +-module(inets_service). + +-export([behaviour_info/1]). + +behaviour_info(callbacks) -> + [{start_standalone, 1}, + {start_service, 1}, + {stop_service, 1}, + {services, 0}, + {service_info, 1}]; +behaviour_info(_) -> + undefined. + +%% Starts service stand-alone +%% start_standalone(Config) -> % {ok, Pid} | {error, Reason} +%% <service>:start_link(Config). + +%% Starts service as part of inets +%% start_service(Config) -> % {ok, Pid} | {error, Reason} +%% <service_sup>:start_child(Config). +%% Stop service +%% stop_service(Pid) -> % ok | {error, Reason} +%% <service_sup>:stop_child(maybe_map_pid_to_other_ref(Pid)). +%% +%% <service_sup>:stop_child(Ref) -> +%% Id = id(Ref), +%% case supervisor:terminate_child(?MODULE, Id) of +%% ok -> +%% supervisor:delete_child(?MODULE, Id); +%% Error -> +%% Error +%% end. + +%% Returns list of running services. Services started as stand alone +%% are not listed +%% services() -> % [{Service, Pid}] +%% Exampel: +%% services() -> +%% [{httpc, Pid} || {_, Pid, _, _} <- +%% supervisor:which_children(httpc_profile_sup)]. + + +%% service_info() -> [{Property, Value}] | {error, Reason} +%% ex: http:service_info() -> [{profile, ProfileName}] +%% httpd:service_info() -> [{host, Host}, {port, Port}] diff --git a/lib/inets/src/inets_app/inets_sup.erl b/lib/inets/src/inets_app/inets_sup.erl new file mode 100644 index 0000000000..20d5ef343e --- /dev/null +++ b/lib/inets/src/inets_app/inets_sup.erl @@ -0,0 +1,117 @@ +%% +%% %CopyrightBegin% +%% +%% Copyright Ericsson AB 1997-2009. All Rights Reserved. +%% +%% The contents of this file are subject to the Erlang Public License, +%% Version 1.1, (the "License"); you may not use this file except in +%% compliance with the License. You should have received a copy of the +%% Erlang Public License along with this software. If not, it can be +%% retrieved online at http://www.erlang.org/. +%% +%% Software distributed under the License is distributed on an "AS IS" +%% basis, WITHOUT WARRANTY OF ANY KIND, either express or implied. See +%% the License for the specific language governing rights and limitations +%% under the License. +%% +%% %CopyrightEnd% +%% +%% +%%---------------------------------------------------------------------- +%% Purpose: The top supervisor for the inets application +%%---------------------------------------------------------------------- + +-module(inets_sup). + +-behaviour(supervisor). + +-export([init/1]). + +%%%========================================================================= +%%% Supervisor callback +%%%========================================================================= +init([]) -> + SupFlags = {one_for_one, 10, 3600}, + Children = children(), + {ok, {SupFlags, Children}}. + +%%%========================================================================= +%%% Internal functions +%%%========================================================================= +get_services() -> + case (catch application:get_env(inets, services)) of + {ok, Services} -> + Services; + _ -> + [] + end. + +children() -> + Services = get_services(), + HttpdServices = [Service || Service <- Services, is_httpd(Service)], + HttpcServices = [Service || Service <- Services, is_httpc(Service)], + TftpdServices = [Service || Service <- Services, is_tftpd(Service)], + [ftp_child_spec(), httpc_child_spec(HttpcServices), + httpd_child_spec(HttpdServices), tftpd_child_spec(TftpdServices)]. + +ftp_child_spec() -> + Name = ftp_sup, + StartFunc = {ftp_sup, start_link, []}, + Restart = permanent, + Shutdown = infinity, + Modules = [ftp_sup], + Type = supervisor, + {Name, StartFunc, Restart, Shutdown, Type, Modules}. + + +httpc_child_spec(HttpcServices0) -> + HttpcServices = default_profile(HttpcServices0, []), + Name = httpc_sup, + StartFunc = {httpc_sup, start_link, [HttpcServices]}, + Restart = permanent, + Shutdown = infinity, + Modules = [httpc_sup], + Type = supervisor, + {Name, StartFunc, Restart, Shutdown, Type, Modules}. + +httpd_child_spec(HttpdServices) -> + Name = httpd_sup, + StartFunc = {httpd_sup, start_link, [HttpdServices]}, + Restart = permanent, + Shutdown = infinity, + Modules = [httpd_sup], + Type = supervisor, + {Name, StartFunc, Restart, Shutdown, Type, Modules}. + +tftpd_child_spec(TftpServices) -> + Name = tftp_sup, + StartFunc = {tftp_sup, start_link, [TftpServices]}, + Restart = permanent, + Shutdown = infinity, + Modules = [tftp_sup], + Type = supervisor, + {Name, StartFunc, Restart, Shutdown, Type, Modules}. + +is_httpd({httpd, _}) -> + true; +is_httpd({httpd, _, _}) -> + true; +is_httpd(_) -> + false. + +is_httpc({httpc, _}) -> + true; +is_httpc(_) -> + false. + +is_tftpd({tftpd, _}) -> + true; +is_tftpd(_) -> + false. + +default_profile([], Acc) -> + [{httpc, {default, only_session_cookies}} | Acc]; +default_profile([{httpc, {default, _}} | _] = Profiles, Acc) -> + Profiles ++ Acc; +default_profile([Profile | Profiles], Acc) -> + default_profile(Profiles, [Profile | Acc]). diff --git a/lib/inets/src/subdirs.mk b/lib/inets/src/subdirs.mk new file mode 100644 index 0000000000..9f2a0079f2 --- /dev/null +++ b/lib/inets/src/subdirs.mk @@ -0,0 +1,3 @@ +#-*-makefile-*- ; force emacs to enter makefile-mode + +SUB_DIRECTORIES = inets_app http_lib http_client http_server ftp tftp diff --git a/lib/inets/src/tftp/Makefile b/lib/inets/src/tftp/Makefile new file mode 100644 index 0000000000..63f70f7943 --- /dev/null +++ b/lib/inets/src/tftp/Makefile @@ -0,0 +1,95 @@ +# +# %CopyrightBegin% +# +# Copyright Ericsson AB 2005-2009. All Rights Reserved. +# +# The contents of this file are subject to the Erlang Public License, +# Version 1.1, (the "License"); you may not use this file except in +# compliance with the License. You should have received a copy of the +# Erlang Public License along with this software. If not, it can be +# retrieved online at http://www.erlang.org/. +# +# Software distributed under the License is distributed on an "AS IS" +# basis, WITHOUT WARRANTY OF ANY KIND, either express or implied. See +# the License for the specific language governing rights and limitations +# under the License. +# +# %CopyrightEnd% +# +# + +include $(ERL_TOP)/make/target.mk +EBIN = ../../ebin +include $(ERL_TOP)/make/$(TARGET)/otp.mk + +# ---------------------------------------------------- +# Application version +# ---------------------------------------------------- +include ../../vsn.mk + +VSN = $(INETS_VSN) + + +# ---------------------------------------------------- +# Release directory specification +# ---------------------------------------------------- +RELSYSDIR = $(RELEASE_PATH)/lib/inets-$(VSN) + +# ---------------------------------------------------- +# Target Specs +# ---------------------------------------------------- +MODULES = \ + tftp \ + tftp_binary \ + tftp_engine \ + tftp_file \ + tftp_lib \ + tftp_logger \ + tftp_sup + +HRL_FILES = tftp.hrl + +ERL_FILES = $(MODULES:%=%.erl) + +TARGET_FILES= $(MODULES:%=$(EBIN)/%.$(EMULATOR)) + +# ---------------------------------------------------- +# INETS FLAGS +# ---------------------------------------------------- +INETS_FLAGS = -D'SERVER_SOFTWARE="inets/$(VSN)"' \ + +# ---------------------------------------------------- +# FLAGS +# ---------------------------------------------------- +ERL_COMPILE_FLAGS += $(INETS_FLAGS) \ + +'{parse_transform,sys_pre_attributes}' \ + +'{attribute,insert,app_vsn,$(APP_VSN)}' +# ---------------------------------------------------- +# Targets +# ---------------------------------------------------- + +debug opt: $(TARGET_FILES) + +clean: + rm -f $(TARGET_FILES) + rm -f core + +docs: + +# ---------------------------------------------------- +# Release Target +# ---------------------------------------------------- +include $(ERL_TOP)/make/otp_release_targets.mk + +release_spec: opt + $(INSTALL_DIR) $(RELSYSDIR)/src + $(INSTALL_DATA) $(HRL_FILES) $(ERL_FILES) $(RELSYSDIR)/src + $(INSTALL_DIR) $(RELSYSDIR)/ebin + $(INSTALL_DATA) $(TARGET_FILES) $(RELSYSDIR)/ebin + +release_docs_spec: + +info: + @echo "INETS_DEBUG = $(INETS_DEBUG)" + @echo "INETS_FLAGS = $(INETS_FLAGS)" + @echo "ERL_COMPILE_FLAGS = $(ERL_COMPILE_FLAGS)" diff --git a/lib/inets/src/tftp/tftp.erl b/lib/inets/src/tftp/tftp.erl new file mode 100644 index 0000000000..bfdb4c0030 --- /dev/null +++ b/lib/inets/src/tftp/tftp.erl @@ -0,0 +1,362 @@ +%% +%% %CopyrightBegin% +%% +%% Copyright Ericsson AB 2005-2009. All Rights Reserved. +%% +%% The contents of this file are subject to the Erlang Public License, +%% Version 1.1, (the "License"); you may not use this file except in +%% compliance with the License. You should have received a copy of the +%% Erlang Public License along with this software. If not, it can be +%% retrieved online at http://www.erlang.org/. +%% +%% Software distributed under the License is distributed on an "AS IS" +%% basis, WITHOUT WARRANTY OF ANY KIND, either express or implied. See +%% the License for the specific language governing rights and limitations +%% under the License. +%% +%% %CopyrightEnd% +%% +%% + +%%%------------------------------------------------------------------- +%%% File : tftp.erl +%%% Author : Hakan Mattsson <[email protected]> +%%% Description : Trivial FTP +%%% Created : 18 May 2004 by Hakan Mattsson <[email protected]> +%%%------------------------------------------------------------------- +%%% +%%% This is a complete implementation of the following IETF standards: +%%% +%%% RFC 1350, The TFTP Protocol (revision 2). +%%% RFC 2347, TFTP Option Extension. +%%% RFC 2348, TFTP Blocksize Option. +%%% RFC 2349, TFTP Timeout Interval and Transfer Size Options. +%%% +%%% The only feature that not is implemented in this release is +%%% the "netascii" transfer mode. +%%% +%%% The start/1 function starts a daemon process which, listens for +%%% UDP packets on a port. When it receives a request for read or +%%% write it spawns a temporary server process which handles the +%%% actual transfer of the file. On the client side the read_file/3 +%%% and write_file/3 functions spawns a temporary client process which +%%% establishes contact with a TFTP daemon and performs the actual +%%% transfer of the file. +%%% +%%% Most of the options are common for both the client and the server +%%% side, but some of them differs a little. Here are the available +%%% options: +%%% +%%% {debug, Level} +%%% +%%% Level = none | error | warning brief | normal | verbose | all +%%% +%%% Controls the level of debug printouts. The default is none. +%%% +%%% {host, Host} +%%% +%%% The name or IP address of the host where the TFTP daemon +%%% resides. This option is only used by the client. See +%%% 'inet' about valid host names. +%%% +%%% {port, Port} +%%% +%%% Port = integer() +%%% +%%% The TFTP port where the daemon listens. It defaults to the +%%% standardized number 69. On the server side it may sometimes +%%% make sense to set it to 0, which means that the daemon just +%%% will pick a free port (which is returned by the start/1 +%%% function). +%%% +%%% If a socket has somehow already has been connected, the +%%% {udp, [{fd, integer()}]} option can be used to pass the +%%% open file descriptor to gen_udp. This can be automated +%%% a bit by using a command line argument stating the +%%% prebound file descriptor number. For example, if the +%%% Port is 69 and the file descriptor 22 has been opened by +%%% setuid_socket_wrap. Then the command line argument +%%% "-tftpd_69 22" will trigger the prebound file +%%% descriptor 22 to be used instead of opening port 69. +%%% The UDP option {udp, [{fd, 22}]} autmatically be added. +%%% See init:get_argument/ about command line arguments and +%%% gen_udp:open/2 about UDP options. +%%% +%%% {port_policy, Policy} +%%% +%%% Policy = random | Port | {range, MinPort, MaxPort} +%%% Port = MinPort = MaxPort = integer() +%%% +%%% Policy for the selection of the temporary port which is used +%%% by the server/client during the file transfer. It defaults to +%%% 'random' which is the standardized policy. With this policy a +%%% randomized free port used. A single port or a range of ports +%%% can be useful if the protocol should pass thru a firewall. +%%% +%%% {prebound_fd, InitArgFlag} +%%% +%%% InitArgFlag = atom() +%%% +%%% If a socket has somehow already has been connected, the +%%% {udp, [{fd, integer()}]} option can be used to pass the +%%% open file descriptor to gen_udp. +%%% +%%% The prebound_fd option makes it possible to pass give the +%%% file descriptor as a command line argument. The typical +%%% usage is when used in conjunction with setuid_socket_wrap +%%% to be able to open privileged sockets. For example if the +%%% file descriptor 22 has been opened by setuid_socket_wrap +%%% and you have choosen my_tftp_fd as init argument, the +%%% command line should like this "erl -my_tftp_fd 22" and +%%% FileDesc should be set to my_tftpd_fd. This would +%%% automatically imply {fd, 22} to be set as UDP option. +%%% +%%% {udp, UdpOptions} +%%% +%%% Options to gen_udp:open/2. +%%% +%%% {use_tsize, Bool} +%%% +%%% Bool = boolean() +%%% +%%% Flag for automated usage of the "tsize" option. With this set +%%% to true, the write_file/3 client will determine the filesize +%%% and send it to the server as the standardized "tsize" option. +%%% A read_file/3 client will just acquire filesize from the +%%% server by sending a zero "tsize". +%%% +%%% {max_tsize, MaxTsize} +%%% +%%% MaxTsize = integer() | infinity +%%% +%%% Threshold for the maximal filesize in bytes. The transfer will +%%% be aborted if the limit is exceeded. It defaults to +%%% 'infinity'. +%%% +%%% {max_conn, MaxConn} +%%% +%%% MaxConn = integer() | infinity +%%% +%%% Threshold for the maximal number of active connections. The +%%% daemon will reject the setup of new connections if the limit +%%% is exceeded. It defaults to 'infinity'. +%%% +%%% {TftpKey, TftpVal} +%%% +%%% TftpKey = string() +%%% TftpVal = string() +%%% +%%% The name and value of a TFTP option. +%%% +%%% {reject, Feature} +%%% +%%% Feature = Mode | TftpKey +%%% Mode = read | write +%%% TftpKey = string() +%%% +%%% Control which features that should be rejected. +%%% This is mostly useful for the server as it may restrict +%%% usage of certain TFTP options or read/write access. +%%% +%%% {callback, {RegExp, Module, State}} +%%% +%%% RegExp = string() +%%% Module = atom() +%%% State = term() +%%% +%%% Registration of a callback module. When a file is to be +%%% transferred, its local filename will be matched to the +%%% regular expressions of the registered callbacks. The first +%%% matching callback will be used the during the transfer.The +%%% callback module must implement the 'tftp' behaviour. +%%% +%%% On the server side the callback interaction starts with a +%%% call to open/5 with the registered initial callback +%%% state. open/5 is expected to open the (virtual) file. Then +%%% either the read/1 or write/2 functions are invoked +%%% repeatedly, once per transfererred block. At each function +%%% call the state returned from the previous call is +%%% obtained. When the last block has been encountered the read/1 +%%% or write/2 functions is expected to close the (virtual) +%%% file.and return its last state. The abort/3 function is only +%%% used in error situations. prepare/5 is not used on the server +%%% side. +%%% +%%% On the client side the callback interaction is the same, but +%%% it starts and ends a bit differently. It starts with a call +%%% to prepare/5 with the same arguments as open/5 +%%% takes. prepare/5 is expected to validate the TFTP options, +%%% suggested by the user and return the subset of them that it +%%% accepts. Then the options is sent to the server which will +%%% perform the same TFTP option negotiation procedure. The +%%% options that are accepted by the server is forwarded to the +%%% open/5 function on the client side. On the client side the +%%% open/5 function must accept all option as is or reject the +%%% transfer. Then the callback interaction follows the same +%%% pattern as described above for the server side. When the last +%%% block is encountered in read/1 or write/2 the returned stated +%%% is forwarded to the user and returned from read_file/3 or +%%% write_file/3. +%%%------------------------------------------------------------------- + +-module(tftp). + +%%------------------------------------------------------------------- +%% Interface +%%------------------------------------------------------------------- + +%% Public functions +-export([ + read_file/3, + write_file/3, + start/1, + info/1, + change_config/2, + start/0 + ]). + +-export([behaviour_info/1]). + +%% Application local functions +-export([ + start_standalone/1, + start_service/1, + stop_service/1, + services/0, + service_info/1 + ]). + + +behaviour_info(callbacks) -> + [{prepare, 6}, {open, 6}, {read, 1}, {write, 2}, {abort, 3}]; +behaviour_info(_) -> + undefined. + +-include("tftp.hrl"). + +%%------------------------------------------------------------------- +%% read_file(RemoteFilename, LocalFilename, Options) -> +%% {ok, LastCallbackState} | {error, Reason} +%% +%% RemoteFilename = string() +%% LocalFilename = binary | string() +%% Options = [option()] +%% LastCallbackState = term() +%% Reason = term() +%% +%% Reads a (virtual) file from a TFTP server +%% +%% If LocalFilename is the atom 'binary', tftp_binary will be used as +%% callback module. It will concatenate all transferred blocks and +%% return them as one single binary in the CallbackState. +%% +%% When LocalFilename is a string, it will be matched to the +%% registered callback modules and hopefully one of them will be +%% selected. By default, tftp_file will be used as callback module. It +%% will write each transferred block to the file named +%% LocalFilename. The number of transferred bytes will be returned as +%% LastCallbackState. +%%------------------------------------------------------------------- + +read_file(RemoteFilename, LocalFilename, Options) -> + tftp_engine:client_start(read, RemoteFilename, LocalFilename, Options). + +%%------------------------------------------------------------------- +%% write(RemoteFilename, LocalFilename, Options) -> +%% {ok, LastCallbackState} | {error, Reason} +%% +%% RemoteFilename = string() +%% LocalFilename = binary() | string() +%% Options = [option()] +%% LastCallbackState = term() +%% Reason = term() +%% +%% Writes a (virtual) file to a TFTP server +%% +%% If LocalFilename is a binary, tftp_binary will be used as callback +%% module. The binary will be transferred block by block and the number +%% of transferred bytes will be returned as LastCallbackState. +%% +%% When LocalFilename is a string, it will be matched to the +%% registered callback modules and hopefully one of them will be +%% selected. By default, tftp_file will be used as callback module. It +%% will read the file named LocalFilename block by block. The number +%% of transferred bytes will be returned as LastCallbackState. +%%------------------------------------------------------------------- + +write_file(RemoteFilename, LocalFilename, Options) -> + tftp_engine:client_start(write, RemoteFilename, LocalFilename, Options). + +%%------------------------------------------------------------------- +%% start(Options) -> {ok, Pid} | {error, Reason} +%% +%% Options = [option()] +%% Pid = pid() +%% Reason = term() +%% +%% Starts a daemon process which listens for udp packets on a +%% port. When it receives a request for read or write it spawns +%% a temporary server process which handles the actual transfer +%% of the (virtual) file. +%%------------------------------------------------------------------- + +start(Options) -> + tftp_engine:daemon_start(Options). + +%%------------------------------------------------------------------- +%% info(Pid) -> {ok, Options} | {error, Reason} +%% +%% Options = [option()] +%% Reason = term() +%% +%% Returns info about a tftp daemon, server or client process +%%------------------------------------------------------------------- + +info(Pid) -> + tftp_engine:info(Pid). + +%%------------------------------------------------------------------- +%% change_config(Pid, Options) -> ok | {error, Reason} +%% +%% Options = [option()] +%% Reason = term() +%% +%% Changes config for a tftp daemon, server or client process +%% Must be used with care. +%%------------------------------------------------------------------- + +change_config(Pid, Options) -> + tftp_engine:change_config(Pid, Options). + +%%------------------------------------------------------------------- +%% start() -> ok | {error, Reason} +%% +%% Reason = term() +%% +%% Start the application +%%------------------------------------------------------------------- + +start() -> + application:start(inets). + +%%------------------------------------------------------------------- +%% Inets service behavior +%%------------------------------------------------------------------- + +start_standalone(Options) -> + start(Options). + +start_service(Options) -> + tftp_sup:start_child(Options). + +stop_service(Pid) -> + tftp_sup:stop_child(Pid). + +services() -> + tftp_sup:which_children(). + +service_info(Pid) -> + info(Pid). + + + diff --git a/lib/inets/src/tftp/tftp.hrl b/lib/inets/src/tftp/tftp.hrl new file mode 100644 index 0000000000..6846b07690 --- /dev/null +++ b/lib/inets/src/tftp/tftp.hrl @@ -0,0 +1,68 @@ +%% +%% %CopyrightBegin% +%% +%% Copyright Ericsson AB 2005-2009. All Rights Reserved. +%% +%% The contents of this file are subject to the Erlang Public License, +%% Version 1.1, (the "License"); you may not use this file except in +%% compliance with the License. You should have received a copy of the +%% Erlang Public License along with this software. If not, it can be +%% retrieved online at http://www.erlang.org/. +%% +%% Software distributed under the License is distributed on an "AS IS" +%% basis, WITHOUT WARRANTY OF ANY KIND, either express or implied. See +%% the License for the specific language governing rights and limitations +%% under the License. +%% +%% %CopyrightEnd% +%% +%% + +%%%------------------------------------------------------------------- +%%% Defines +%%%------------------------------------------------------------------- + +-define(TFTP_DEFAULT_PORT, 69).% Default server port + +-define(TFTP_OPCODE_RRQ, 1). % Read request +-define(TFTP_OPCODE_WRQ, 2). % Write request +-define(TFTP_OPCODE_DATA, 3). % Data +-define(TFTP_OPCODE_ACK, 4). % Acknowledgement +-define(TFTP_OPCODE_ERROR, 5). % Error +-define(TFTP_OPCODE_OACK, 6). % Option acknowledgment + +-define(TFTP_ERROR_UNDEF, 0). % Not defined, see error message (if any) +-define(TFTP_ERROR_ENOENT, 1). % File not found. +-define(TFTP_ERROR_EACCES, 2). % Access violation. +-define(TFTP_ERROR_ENOSPC, 3). % Disk full or allocation exceeded. +-define(TFTP_ERROR_BADOP, 4). % Illegal TFTP operation. +-define(TFTP_ERROR_BADBLK, 5). % Unknown transfer ID. +-define(TFTP_ERROR_EEXIST, 6). % File already exists. +-define(TFTP_ERROR_BADUSER, 7). % No such user. +-define(TFTP_ERROR_BADOPT, 8). % Unrequested or illegal option. + +-record(tftp_msg_req, {access, filename, mode, options, local_filename}). +-record(tftp_msg_data, {block_no, data}). +-record(tftp_msg_ack, {block_no}). +-record(tftp_msg_error, {code, text, details}). +-record(tftp_msg_oack, {options}). + +-record(config, {parent_pid = self(), + udp_socket, + udp_options = [binary, {reuseaddr, true}, {active, once}], + udp_host = "localhost", + udp_port = ?TFTP_DEFAULT_PORT, + port_policy = random, + use_tsize = false, + max_tsize = infinity, % Filesize + max_conn = infinity, + rejected = [], + polite_ack = false, + debug_level = none, + timeout, + user_options = [], + callbacks = [], + logger = tftp_logger, + max_retries = 5}). + +-record(callback, {regexp, internal, module, state, block_no, count}). diff --git a/lib/inets/src/tftp/tftp_binary.erl b/lib/inets/src/tftp/tftp_binary.erl new file mode 100644 index 0000000000..9efa79105d --- /dev/null +++ b/lib/inets/src/tftp/tftp_binary.erl @@ -0,0 +1,238 @@ +%% +%% %CopyrightBegin% +%% +%% Copyright Ericsson AB 2005-2009. All Rights Reserved. +%% +%% The contents of this file are subject to the Erlang Public License, +%% Version 1.1, (the "License"); you may not use this file except in +%% compliance with the License. You should have received a copy of the +%% Erlang Public License along with this software. If not, it can be +%% retrieved online at http://www.erlang.org/. +%% +%% Software distributed under the License is distributed on an "AS IS" +%% basis, WITHOUT WARRANTY OF ANY KIND, either express or implied. See +%% the License for the specific language governing rights and limitations +%% under the License. +%% +%% %CopyrightEnd% +%% +%% + +%%%------------------------------------------------------------------- +%%% File : tft_binary.erl +%%% Author : Hakan Mattsson <[email protected]> +%%% Description : +%%% +%%% Created : 24 May 2004 by Hakan Mattsson <[email protected]> +%%%------------------------------------------------------------------- + +-module(tftp_binary). + +%%%------------------------------------------------------------------- +%%% Interface +%%%------------------------------------------------------------------- + +-behaviour(tftp). + +-export([prepare/6, open/6, read/1, write/2, abort/3]). + +-record(read_state, {options, blksize, bin, is_native_ascii, is_network_ascii, count}). +-record(write_state, {options, blksize, list, is_native_ascii, is_network_ascii}). + +%%------------------------------------------------------------------- +%% Prepare +%%------------------------------------------------------------------- + +prepare(_Peer, Access, Filename, Mode, SuggestedOptions, Initial) when is_list(Initial) -> + %% Client side + IsNativeAscii = is_native_ascii(Initial), + case catch handle_options(Access, Filename, Mode, SuggestedOptions, IsNativeAscii) of + {ok, IsNetworkAscii, AcceptedOptions} when Access =:= read, is_binary(Filename) -> + State = #read_state{options = AcceptedOptions, + blksize = lookup_blksize(AcceptedOptions), + bin = Filename, + is_network_ascii = IsNetworkAscii, + count = size(Filename), + is_native_ascii = IsNativeAscii}, + {ok, AcceptedOptions, State}; + {ok, IsNetworkAscii, AcceptedOptions} when Access =:= write, Filename =:= binary -> + State = #write_state{options = AcceptedOptions, + blksize = lookup_blksize(AcceptedOptions), + list = [], + is_network_ascii = IsNetworkAscii, + is_native_ascii = IsNativeAscii}, + {ok, AcceptedOptions, State}; + {ok, _, _} -> + {error, {undef, "Illegal callback usage. Mode and filename is incompatible."}}; + {error, {Code, Text}} -> + {error, {Code, Text}} + end; +prepare(_Peer, _Access, _Bin, _Mode, _SuggestedOptions, _Initial) -> + {error, {undef, "Illegal callback options."}}. + +%%------------------------------------------------------------------- +%% Open +%%------------------------------------------------------------------- + +open(Peer, Access, Filename, Mode, SuggestedOptions, Initial) when is_list(Initial) -> + %% Server side + case prepare(Peer, Access, Filename, Mode, SuggestedOptions, Initial) of + {ok, AcceptedOptions, State} -> + open(Peer, Access, Filename, Mode, AcceptedOptions, State); + {error, {Code, Text}} -> + {error, {Code, Text}} + end; +open(_Peer, Access, Filename, Mode, NegotiatedOptions, State) when is_record(State, read_state) -> + %% Both sides + case catch handle_options(Access, Filename, Mode, NegotiatedOptions, State#read_state.is_native_ascii) of + {ok, IsNetworkAscii, Options} + when Options =:= NegotiatedOptions, + IsNetworkAscii =:= State#read_state.is_network_ascii -> + {ok, NegotiatedOptions, State}; + {error, {Code, Text}} -> + {error, {Code, Text}} + end; +open(_Peer, Access, Filename, Mode, NegotiatedOptions, State) when is_record(State, write_state) -> + %% Both sides + case catch handle_options(Access, Filename, Mode, NegotiatedOptions, State#write_state.is_native_ascii) of + {ok, IsNetworkAscii, Options} + when Options =:= NegotiatedOptions, + IsNetworkAscii =:= State#write_state.is_network_ascii -> + {ok, NegotiatedOptions, State}; + {error, {Code, Text}} -> + {error, {Code, Text}} + end; +open(Peer, Access, Filename, Mode, NegotiatedOptions, State) -> + %% Handle upgrade from old releases. Please, remove this clause in next release. + State2 = upgrade_state(State), + open(Peer, Access, Filename, Mode, NegotiatedOptions, State2). + +%%------------------------------------------------------------------- +%% Read +%%------------------------------------------------------------------- + +read(#read_state{bin = Bin} = State) when is_binary(Bin) -> + BlkSize = State#read_state.blksize, + if + size(Bin) >= BlkSize -> + <<Block:BlkSize/binary, Bin2/binary>> = Bin, + State2 = State#read_state{bin = Bin2}, + {more, Block, State2}; + size(Bin) < BlkSize -> + {last, Bin, State#read_state.count} + end; +read(State) -> + %% Handle upgrade from old releases. Please, remove this clause in next release. + State2 = upgrade_state(State), + read(State2). + +%%------------------------------------------------------------------- +%% Write +%%------------------------------------------------------------------- + +write(Bin, #write_state{list = List} = State) when is_binary(Bin), is_list(List) -> + Size = size(Bin), + BlkSize = State#write_state.blksize, + if + Size =:= BlkSize -> + {more, State#write_state{list = [Bin | List]}}; + Size < BlkSize -> + Bin2 = list_to_binary(lists:reverse([Bin | List])), + {last, Bin2} + end; +write(Bin, State) -> + %% Handle upgrade from old releases. Please, remove this clause in next release. + State2 = upgrade_state(State), + write(Bin, State2). + +%%------------------------------------------------------------------- +%% Abort +%%------------------------------------------------------------------- + +abort(_Code, _Text, #read_state{bin = Bin} = State) + when is_record(State, read_state), is_binary(Bin) -> + ok; +abort(_Code, _Text, #write_state{list = List} = State) + when is_record(State, write_state), is_list(List) -> + ok; +abort(Code, Text, State) -> + %% Handle upgrade from old releases. Please, remove this clause in next release. + State2 = upgrade_state(State), + abort(Code, Text, State2). + +%%------------------------------------------------------------------- +%% Process options +%%------------------------------------------------------------------- + +handle_options(Access, Bin, Mode, Options, IsNativeAscii) -> + IsNetworkAscii = handle_mode(Mode, IsNativeAscii), + Options2 = do_handle_options(Access, Bin, Options), + {ok, IsNetworkAscii, Options2}. + +handle_mode(Mode, IsNativeAscii) -> + case Mode of + "netascii" when IsNativeAscii =:= true -> true; + "octet" -> false; + _ -> throw({error, {badop, "Illegal mode " ++ Mode}}) + end. + +do_handle_options(Access, Bin, [{Key, Val} | T]) -> + case Key of + "tsize" -> + case Access of + read when Val =:= "0", is_binary(Bin) -> + Tsize = integer_to_list(size(Bin)), + [{Key, Tsize} | do_handle_options(Access, Bin, T)]; + _ -> + handle_integer(Access, Bin, Key, Val, T, 0, infinity) + end; + "blksize" -> + handle_integer(Access, Bin, Key, Val, T, 8, 65464); + "timeout" -> + handle_integer(Access, Bin, Key, Val, T, 1, 255); + _ -> + do_handle_options(Access, Bin, T) + end; +do_handle_options(_Access, _Bin, []) -> + []. + + +handle_integer(Access, Bin, Key, Val, Options, Min, Max) -> + case catch list_to_integer(Val) of + {'EXIT', _} -> + do_handle_options(Access, Bin, Options); + Int when Int >= Min, Int =< Max -> + [{Key, Val} | do_handle_options(Access, Bin, Options)]; + Int when Int >= Min, Max =:= infinity -> + [{Key, Val} | do_handle_options(Access, Bin, Options)]; + _Int -> + throw({error, {badopt, "Illegal " ++ Key ++ " value " ++ Val}}) + end. + +lookup_blksize(Options) -> + case lists:keysearch("blksize", 1, Options) of + {value, {_, Val}} -> + list_to_integer(Val); + false -> + 512 + end. + +is_native_ascii([]) -> + is_native_ascii(); +is_native_ascii([{native_ascii, Bool}]) -> + case Bool of + true -> true; + false -> false + end. + +is_native_ascii() -> + case os:type() of + {win32, _} -> true; + _ -> false + end. + +%% Handle upgrade from old releases. Please, remove this function in next release. +upgrade_state({read_state, Options, Blksize, Bin, IsNetworkAscii, Count}) -> + {read_state, Options, Blksize, Bin, false, IsNetworkAscii, Count}; +upgrade_state({write_state, Options, Blksize, List, IsNetworkAscii}) -> + {write_state, Options, Blksize, List, false, IsNetworkAscii}. diff --git a/lib/inets/src/tftp/tftp_engine.erl b/lib/inets/src/tftp/tftp_engine.erl new file mode 100644 index 0000000000..81c53bbe40 --- /dev/null +++ b/lib/inets/src/tftp/tftp_engine.erl @@ -0,0 +1,1442 @@ +%% +%% %CopyrightBegin% +%% +%% Copyright Ericsson AB 2005-2009. All Rights Reserved. +%% +%% The contents of this file are subject to the Erlang Public License, +%% Version 1.1, (the "License"); you may not use this file except in +%% compliance with the License. You should have received a copy of the +%% Erlang Public License along with this software. If not, it can be +%% retrieved online at http://www.erlang.org/. +%% +%% Software distributed under the License is distributed on an "AS IS" +%% basis, WITHOUT WARRANTY OF ANY KIND, either express or implied. See +%% the License for the specific language governing rights and limitations +%% under the License. +%% +%% %CopyrightEnd% +%% +%%------------------------------------------------------------------- +%% Protocol engine for trivial FTP +%%------------------------------------------------------------------- + +-module(tftp_engine). + +%%%------------------------------------------------------------------- +%%% Interface +%%%------------------------------------------------------------------- + +%% application internal functions +-export([ + daemon_start/1, + daemon_loop/1, + daemon_loop/3, %% Handle upgrade from old releases. Please, remove this function in next release. + client_start/4, + common_loop/6, + info/1, + change_config/2 + ]). + +%% module internal +-export([ + daemon_init/1, + server_init/2, + client_init/2, + wait_for_msg/3, + callback/4 + ]). + +%% sys callback functions +-export([ + system_continue/3, + system_terminate/4, + system_code_change/4 + ]). + +-include("tftp.hrl"). + +-type prep_status() :: 'error' | 'last' | 'more' | 'terminate'. + +-record(daemon_state, {config, n_servers, server_tab, file_tab}). +-record(server_info, {pid, req, peer}). +-record(file_info, {peer_req, pid}). +-record(sys_misc, {module, function, arguments}). +-record(error, {where, code, text, filename}). +-record(prepared, {status :: prep_status(), result, block_no, next_data, prev_data}). +-record(transfer_res, {status, decoded_msg, prepared}). +-define(ERROR(Where, Code, Text, Filename), + #error{where = Where, code = Code, text = Text, filename = Filename}). + +%%%------------------------------------------------------------------- +%%% Info +%%%------------------------------------------------------------------- + +info(daemons) -> + Daemons = supervisor:which_children(tftp_sup), + [{Pid, info(Pid)} || {_, Pid, _, _} <- Daemons]; +info(servers) -> + [{Pid, info(Pid)} || {_, {ok, DeamonInfo}} <- info(daemons), + {server, Pid} <- DeamonInfo]; +info(ToPid) when is_pid(ToPid) -> + call(info, ToPid, timer:seconds(10)). + +change_config(daemons, Options) -> + Daemons = supervisor:which_children(tftp_sup), + [{Pid, change_config(Pid, Options)} || {_, Pid, _, _} <- Daemons]; +change_config(servers, Options) -> + [{Pid, change_config(Pid, Options)} || {_, {ok, DeamonInfo}} <- info(daemons), + {server, Pid} <- DeamonInfo]; +change_config(ToPid, Options) when is_pid(ToPid) -> + BadKeys = [host, port, udp], + BadOptions = [{Key, Val} || {Key, Val} <- Options, + BadKey <- BadKeys, + Key =:= BadKey], + case BadOptions of + [] -> + call({change_config, Options}, ToPid, timer:seconds(10)); + [{Key, Val} | _] -> + {error, {badarg, {Key, Val}}} + end. + +call(Req, ToPid, Timeout) when is_pid(ToPid) -> + Type = process, + Ref = erlang:monitor(Type, ToPid), + ToPid ! {Req, Ref, self()}, + receive + {Reply, Ref, FromPid} when FromPid =:= ToPid -> + erlang:demonitor(Ref, [flush]), + Reply; + {'DOWN', Ref, Type, FromPid, _Reason} when FromPid =:= ToPid -> + {error, timeout} + after Timeout -> + {error, timeout} + end. + +reply(Reply, Ref, ToPid) -> + ToPid ! {Reply, Ref, self()}. + +%%%------------------------------------------------------------------- +%%% Daemon +%%%------------------------------------------------------------------- + +%% Returns {ok, Port} +daemon_start(Options) when is_list(Options) -> + Config = tftp_lib:parse_config(Options), + proc_lib:start_link(?MODULE, daemon_init, [Config], infinity). + +daemon_init(Config) when is_record(Config, config), + is_pid(Config#config.parent_pid) -> + process_flag(trap_exit, true), + UdpOptions = prepare_daemon_udp(Config), + case catch gen_udp:open(Config#config.udp_port, UdpOptions) of + {ok, Socket} -> + {ok, ActualPort} = inet:port(Socket), + proc_lib:init_ack({ok, self()}), + Config2 = Config#config{udp_socket = Socket, + udp_port = ActualPort}, + print_debug_info(Config2, daemon, open, #tftp_msg_req{filename = ""}), + ServerTab = ets:new(tftp_daemon_servers, [{keypos, 2}]), + FileTab = ets:new(tftp_daemon_files, [{keypos, 2}]), + State = #daemon_state{config = Config2, + n_servers = 0, + server_tab = ServerTab, + file_tab = FileTab}, + daemon_loop(State); + {error, Reason} -> + Text = lists:flatten(io_lib:format("UDP open ~p -> ~p", [UdpOptions, Reason])), + print_debug_info(Config, daemon, open, ?ERROR(open, undef, Text, "")), + exit({gen_udp_open, UdpOptions, Reason}); + Reason -> + Text = lists:flatten(io_lib:format("UDP open ~p -> ~p", [UdpOptions, Reason])), + print_debug_info(Config, daemon, open, ?ERROR(open, undef, Text, "")), + exit({gen_udp_open, UdpOptions, Reason}) + end. + +prepare_daemon_udp(#config{udp_port = Port, udp_options = UdpOptions} = Config) -> + case lists:keymember(fd, 1, UdpOptions) of + true -> + %% Use explicit fd + UdpOptions; + false -> + %% Use fd from setuid_socket_wrap, such as -tftpd_69 + InitArg = list_to_atom("tftpd_" ++ integer_to_list(Port)), + case init:get_argument(InitArg) of + {ok, [[FdStr]] = Badarg} when is_list(FdStr) -> + case catch list_to_integer(FdStr) of + Fd when is_integer(Fd) -> + [{fd, Fd} | UdpOptions]; + {'EXIT', _} -> + Text = lists:flatten(io_lib:format("Illegal prebound fd ~p: ~p", [InitArg, Badarg])), + print_debug_info(Config, daemon, open, ?ERROR(open, undef, Text, "")), + exit({badarg, {prebound_fd, InitArg, Badarg}}) + end; + {ok, Badarg} -> + Text = lists:flatten(io_lib:format("Illegal prebound fd ~p: ~p", [InitArg, Badarg])), + print_debug_info(Config, daemon, open, ?ERROR(open, undef, Text, "")), + exit({badarg, {prebound_fd, InitArg, Badarg}}); + error -> + UdpOptions + end + end. + +daemon_loop(DaemonConfig, N, Servers) when is_list(Servers) -> + %% Handle upgrade from old releases. Please, remove this function in next release. + ServerTab = ets:new(tftp_daemon_servers, [{keypos, 2}]), + FileTab = ets:new(tftp_daemon_files, [{keypos, 2}]), + State = #daemon_state{config = DaemonConfig, + n_servers = N, + server_tab = ServerTab, + file_tab = FileTab}, + Req = #tftp_msg_req{filename = dummy}, + [ets:insert(ServerTab, #server_info{pid = Pid, req = Req, peer = dummy}) || Pid <- Servers], + daemon_loop(State). + +daemon_loop(#daemon_state{config = DaemonConfig, + n_servers = N, + server_tab = ServerTab, + file_tab = FileTab} = State) when is_record(DaemonConfig, config) -> + %% info_msg(DaemonConfig, "=====> TFTP: Daemon #~p\n", [N]), %% XXX + receive + {info, Ref, FromPid} when is_pid(FromPid) -> + Fun = fun(#server_info{pid = Pid}, Acc) -> [{server, Pid} | Acc] end, + ServerInfo = ets:foldl(Fun, [], ServerTab), + Info = internal_info(DaemonConfig, daemon) ++ [{n_conn, N}] ++ ServerInfo, + reply({ok, Info}, Ref, FromPid), + ?MODULE:daemon_loop(State); + {{change_config, Options}, Ref, FromPid} when is_pid(FromPid) -> + case catch tftp_lib:parse_config(Options, DaemonConfig) of + {'EXIT', Reason} -> + reply({error, Reason}, Ref, FromPid), + ?MODULE:daemon_loop(State); + DaemonConfig2 when is_record(DaemonConfig2, config) -> + reply(ok, Ref, FromPid), + ?MODULE:daemon_loop(State#daemon_state{config = DaemonConfig2}) + end; + {udp, Socket, RemoteHost, RemotePort, Bin} when is_binary(Bin) -> + inet:setopts(Socket, [{active, once}]), + ServerConfig = DaemonConfig#config{parent_pid = self(), + udp_host = RemoteHost, + udp_port = RemotePort}, + Msg = (catch tftp_lib:decode_msg(Bin)), + print_debug_info(ServerConfig, daemon, recv, Msg), + case Msg of + Req when is_record(Req, tftp_msg_req), + N =< DaemonConfig#config.max_conn -> + Peer = peer_info(ServerConfig), + PeerReq = {Peer, Req}, + PeerInfo = lists:flatten(io_lib:format("~p", [Peer])), + case ets:lookup(FileTab, PeerReq) of + [] -> + Args = [ServerConfig, Req], + Pid = proc_lib:spawn_link(?MODULE, server_init, Args), + ets:insert(ServerTab, #server_info{pid = Pid, req = Req, peer = Peer}), + ets:insert(FileTab, #file_info{peer_req = PeerReq, pid = Pid}), + ?MODULE:daemon_loop(State#daemon_state{n_servers = N + 1}); + [#file_info{pid = Pid}] -> + %% Yet another request of the file from same peer + warning_msg(DaemonConfig, "~p Reuse connection for ~s\n\t~p\n", + [Pid, PeerInfo, Req#tftp_msg_req.filename]), + ?MODULE:daemon_loop(State) + end; + Req when is_record(Req, tftp_msg_req) -> + Reply = #tftp_msg_error{code = enospc, text = "Too many connections"}, + Peer = peer_info(ServerConfig), + PeerInfo = lists:flatten(io_lib:format("~p", [Peer])), + warning_msg(DaemonConfig, + "Daemon has too many connections (~p)." + "\n\tRejecting request from ~s\n", + [N, PeerInfo]), + send_msg(ServerConfig, daemon, Reply), + ?MODULE:daemon_loop(State); + {'EXIT', Reply} when is_record(Reply, tftp_msg_error) -> + send_msg(ServerConfig, daemon, Reply), + ?MODULE:daemon_loop(State); + Req -> + Reply = #tftp_msg_error{code = badop, + text = "Illegal TFTP operation"}, + warning_msg(DaemonConfig, "Daemon received: ~p.\n\tfrom ~p:~p", + [Req, RemoteHost, RemotePort]), + send_msg(ServerConfig, daemon, Reply), + ?MODULE:daemon_loop(State) + end; + {system, From, Msg} -> + Misc = #sys_misc{module = ?MODULE, function = daemon_loop, arguments = [State]}, + sys:handle_system_msg(Msg, From, DaemonConfig#config.parent_pid, ?MODULE, [], Misc); + {'EXIT', Pid, Reason} when DaemonConfig#config.parent_pid =:= Pid -> + close_port(DaemonConfig, daemon, #tftp_msg_req{filename = ""}), + exit(Reason); + {'EXIT', Pid, _Reason} = Info -> + case ets:lookup(ServerTab, Pid) of + [] -> + warning_msg(DaemonConfig, "Daemon received: ~p", [Info]), + ?MODULE:daemon_loop(State); + [#server_info{req = Req, peer = Peer}] -> + PeerReq = {Peer, Req}, + ets:delete(FileTab, PeerReq), + ets:delete(ServerTab, Pid), + ?MODULE:daemon_loop(State#daemon_state{n_servers = N - 1}) + end; + Info -> + warning_msg(DaemonConfig, "Daemon received: ~p", [Info]), + ?MODULE:daemon_loop(State) + end; +daemon_loop(#daemon_state{config = Config} = State) -> + %% Handle upgrade from old releases. Please, remove this clause in next release. + Config2 = upgrade_config(Config), + daemon_loop(State#daemon_state{config = Config2}). + +upgrade_config({config, ParentPid, UdpSocket, UdpOptions, UdpHost, UdpPort, PortPolicy, + UseTsize, MaxTsize, MaxConn, Rejected, PoliteAck, DebugLevel, + Timeout, UserOptions, Callbacks}) -> + Callbacks2 = tftp_lib:add_default_callbacks(Callbacks), + Logger = tftp_logger, + MaxRetries = 5, + {config, ParentPid, UdpSocket, UdpOptions, UdpHost, UdpPort, PortPolicy, + UseTsize, MaxTsize, MaxConn, Rejected, PoliteAck, DebugLevel, + Timeout, UserOptions, Callbacks2, Logger, MaxRetries}. + +%%%------------------------------------------------------------------- +%%% Server +%%%------------------------------------------------------------------- + +server_init(Config, Req) when is_record(Config, config), + is_pid(Config#config.parent_pid), + is_record(Req, tftp_msg_req) -> + process_flag(trap_exit, true), + %% Config = + %% case os:getenv("TFTPDEBUG") of + %% false -> + %% Config0; + %% DebugLevel -> + %% Config0#config{debug_level = list_to_atom(DebugLevel)} + %% end, + SuggestedOptions = Req#tftp_msg_req.options, + UdpOptions = Config#config.udp_options, + UdpOptions2 = lists:keydelete(fd, 1, UdpOptions), + Config1 = Config#config{udp_options = UdpOptions2}, + Config2 = tftp_lib:parse_config(SuggestedOptions, Config1), + SuggestedOptions2 = Config2#config.user_options, + Req2 = Req#tftp_msg_req{options = SuggestedOptions2}, + case open_free_port(Config2, server, Req2) of + {ok, Config3} -> + Filename = Req#tftp_msg_req.filename, + case match_callback(Filename, Config3#config.callbacks) of + {ok, Callback} -> + print_debug_info(Config3, server, match, Callback), + case pre_verify_options(Config3, Req2) of + ok -> + case callback({open, server_open}, Config3, Callback, Req2) of + {Callback2, {ok, AcceptedOptions}} -> + {LocalAccess, _} = local_file_access(Req2), + OptText = "Internal error. Not allowed to add new options.", + case post_verify_options(Config3, Req2, AcceptedOptions, OptText) of + {ok, Config4, Req3} when AcceptedOptions =/= [] -> + Reply = #tftp_msg_oack{options = AcceptedOptions}, + BlockNo = + case LocalAccess of + read -> 0; + write -> 1 + end, + {Config5, Callback3, TransferRes} = + transfer(Config4, Callback2, Req3, Reply, LocalAccess, BlockNo, #prepared{}), + common_loop(Config5, Callback3, Req3, TransferRes, LocalAccess, BlockNo); + {ok, Config4, Req3} when LocalAccess =:= write -> + BlockNo = 0, + common_ack(Config4, Callback2, Req3, LocalAccess, BlockNo, #prepared{}); + {ok, Config4, Req3} when LocalAccess =:= read -> + BlockNo = 0, + common_read(Config4, Callback2, Req3, LocalAccess, BlockNo, BlockNo, #prepared{}); + {error, {Code, Text}} -> + {undefined, Error} = + callback({abort, {Code, Text}}, Config3, Callback2, Req2), + send_msg(Config3, Req, Error), + terminate(Config3, Req2, ?ERROR(post_verify_options, Code, Text, Req2#tftp_msg_req.filename)) + end; + {undefined, #tftp_msg_error{code = Code, text = Text} = Error} -> + send_msg(Config3, Req, Error), + terminate(Config3, Req, ?ERROR(server_open, Code, Text, Req2#tftp_msg_req.filename)) + end; + {error, {Code, Text}} -> + {undefined, Error} = + callback({abort, {Code, Text}}, Config2, Callback, Req2), + send_msg(Config2, Req, Error), + terminate(Config2, Req2, ?ERROR(pre_verify_options, Code, Text, Req2#tftp_msg_req.filename)) + end; + {error, #tftp_msg_error{code = Code, text = Text} = Error} -> + send_msg(Config3, Req, Error), + terminate(Config3, Req, ?ERROR(match_callback, Code, Text, Req2#tftp_msg_req.filename)) + end; + #error{} = Error -> + terminate(Config2, Req, Error) + end; +server_init(Config, Req) when is_record(Req, tftp_msg_req) -> + Config2 = upgrade_config(Config), + server_init(Config2, Req). + +%%%------------------------------------------------------------------- +%%% Client +%%%------------------------------------------------------------------- + +%% LocalFilename = filename() | 'binary' | binary() +%% Returns {ok, LastCallbackState} | {error, Reason} +client_start(Access, RemoteFilename, LocalFilename, Options) -> + Config = tftp_lib:parse_config(Options), + Config2 = Config#config{parent_pid = self(), + udp_socket = undefined}, + Req = #tftp_msg_req{access = Access, + filename = RemoteFilename, + mode = lookup_mode(Config2#config.user_options), + options = Config2#config.user_options, + local_filename = LocalFilename}, + Args = [Config2, Req], + case proc_lib:start_link(?MODULE, client_init, Args, infinity) of + {ok, LastCallbackState} -> + {ok, LastCallbackState}; + {error, Error} -> + {error, Error} + end. + +client_init(Config, Req) when is_record(Config, config), + is_pid(Config#config.parent_pid), + is_record(Req, tftp_msg_req) -> + process_flag(trap_exit, true), + %% Config = + %% case os:getenv("TFTPDEBUG") of + %% false -> + %% Config0; + %% "none" -> + %% Config0; + %% DebugLevel -> + %% info_msg(Config, "TFTPDEBUG: ~s\n", [DebugLevel]), + %% Config0#config{debug_level = list_to_atom(DebugLevel)} + %% end, + case open_free_port(Config, client, Req) of + {ok, Config2} -> + Req2 = + case Config2#config.use_tsize of + true -> + SuggestedOptions = Req#tftp_msg_req.options, + SuggestedOptions2 = tftp_lib:replace_val("tsize", "0", SuggestedOptions), + Req#tftp_msg_req{options = SuggestedOptions2}; + false -> + Req + end, + LocalFilename = Req2#tftp_msg_req.local_filename, + case match_callback(LocalFilename, Config2#config.callbacks) of + {ok, Callback} -> + print_debug_info(Config2, client, match, Callback), + client_prepare(Config2, Callback, Req2); + {error, #tftp_msg_error{code = Code, text = Text}} -> + terminate(Config, Req, ?ERROR(match, Code, Text, Req#tftp_msg_req.filename)) + end; + #error{} = Error -> + terminate(Config, Req, Error) + end. + +client_prepare(Config, Callback, Req) when is_record(Req, tftp_msg_req) -> + case pre_verify_options(Config, Req) of + ok -> + case callback({open, client_prepare}, Config, Callback, Req) of + {Callback2, {ok, AcceptedOptions}} -> + OptText = "Internal error. Not allowed to add new options.", + case post_verify_options(Config, Req, AcceptedOptions, OptText) of + {ok, Config2, Req2} -> + {LocalAccess, _} = local_file_access(Req2), + BlockNo = 0, + {Config3, Callback3, TransferRes} = + transfer(Config2, Callback2, Req2, Req2, LocalAccess, BlockNo, #prepared{}), + client_open(Config3, Callback3, Req2, BlockNo, TransferRes); + {error, {Code, Text}} -> + callback({abort, {Code, Text}}, Config, Callback2, Req), + terminate(Config, Req, ?ERROR(post_verify_options, Code, Text, Req#tftp_msg_req.filename)) + end; + {undefined, #tftp_msg_error{code = Code, text = Text}} -> + terminate(Config, Req, ?ERROR(client_prepare, Code, Text, Req#tftp_msg_req.filename)) + end; + {error, {Code, Text}} -> + callback({abort, {Code, Text}}, Config, Callback, Req), + terminate(Config, Req, ?ERROR(pre_verify_options, Code, Text, Req#tftp_msg_req.filename)) + end. + +client_open(Config, Callback, Req, BlockNo, #transfer_res{status = Status, decoded_msg = DecodedMsg, prepared = Prepared}) -> + {LocalAccess, _} = local_file_access(Req), + case Status of + ok when is_record(Prepared, prepared) -> + case DecodedMsg of + Msg when is_record(Msg, tftp_msg_oack) -> + ServerOptions = Msg#tftp_msg_oack.options, + OptText = "Protocol violation. Server is not allowed new options", + case post_verify_options(Config, Req, ServerOptions, OptText) of + {ok, Config2, Req2} -> + {Config3, Callback2, Req3} = + do_client_open(Config2, Callback, Req2), + case LocalAccess of + read -> + common_read(Config3, Callback2, Req3, LocalAccess, BlockNo, BlockNo, Prepared); + write -> + common_ack(Config3, Callback2, Req3, LocalAccess, BlockNo, Prepared) + end; + {error, {Code, Text}} -> + {undefined, Error} = + callback({abort, {Code, Text}}, Config, Callback, Req), + send_msg(Config, Req, Error), + terminate(Config, Req, ?ERROR(verify_server_options, Code, Text, Req#tftp_msg_req.filename)) + end; + #tftp_msg_ack{block_no = ActualBlockNo} when LocalAccess =:= read -> + Req2 = Req#tftp_msg_req{options = []}, + {Config2, Callback2, Req2} = do_client_open(Config, Callback, Req2), + ExpectedBlockNo = 0, + common_read(Config2, Callback2, Req2, LocalAccess, ExpectedBlockNo, ActualBlockNo, Prepared); + #tftp_msg_data{block_no = ActualBlockNo, data = Data} when LocalAccess =:= write -> + Req2 = Req#tftp_msg_req{options = []}, + {Config2, Callback2, Req2} = do_client_open(Config, Callback, Req2), + ExpectedBlockNo = 1, + common_write(Config2, Callback2, Req2, LocalAccess, ExpectedBlockNo, ActualBlockNo, Data, Prepared); + %% #tftp_msg_error{code = Code, text = Text} when Req#tftp_msg_req.options =/= [] -> + %% %% Retry without options + %% callback({abort, {Code, Text}}, Config, Callback, Req), + %% Req2 = Req#tftp_msg_req{options = []}, + %% client_prepare(Config, Callback, Req2); + #tftp_msg_error{code = Code, text = Text} -> + callback({abort, {Code, Text}}, Config, Callback, Req), + terminate(Config, Req, ?ERROR(client_open, Code, Text, Req#tftp_msg_req.filename)); + {'EXIT', #tftp_msg_error{code = Code, text = Text}} -> + callback({abort, {Code, Text}}, Config, Callback, Req), + terminate(Config, Req, ?ERROR(client_open, Code, Text, Req#tftp_msg_req.filename)); + Msg when is_tuple(Msg) -> + Code = badop, + Text = "Illegal TFTP operation", + {undefined, Error} = + callback({abort, {Code, Text}}, Config, Callback, Req), + send_msg(Config, Req, Error), + Text2 = lists:flatten([Text, ". ", io_lib:format("~p", [element(1, Msg)])]), + terminate(Config, Req, ?ERROR(client_open, Code, Text2, Req#tftp_msg_req.filename)) + end; + error when is_record(Prepared, tftp_msg_error) -> + #tftp_msg_error{code = Code, text = Text} = Prepared, + callback({abort, {Code, Text}}, Config, Callback, Req), + terminate(Config, Req, ?ERROR(client_open, Code, Text, Req#tftp_msg_req.filename)) + end. + +do_client_open(Config, Callback, Req) -> + case callback({open, client_open}, Config, Callback, Req) of + {Callback2, {ok, FinalOptions}} -> + OptText = "Internal error. Not allowed to change options.", + case post_verify_options(Config, Req, FinalOptions, OptText) of + {ok, Config2, Req2} -> + {Config2, Callback2, Req2}; + {error, {Code, Text}} -> + {undefined, Error} = + callback({abort, {Code, Text}}, Config, Callback2, Req), + send_msg(Config, Req, Error), + terminate(Config, Req, ?ERROR(post_verify_options, Code, Text, Req#tftp_msg_req.filename)) + end; + {undefined, #tftp_msg_error{code = Code, text = Text} = Error} -> + send_msg(Config, Req, Error), + terminate(Config, Req, ?ERROR(client_open, Code, Text, Req#tftp_msg_req.filename)) + end. + +%%%------------------------------------------------------------------- +%%% Common loop for both client and server +%%%------------------------------------------------------------------- + +common_loop(Config, Callback, Req, #transfer_res{status = Status, decoded_msg = DecodedMsg, prepared = Prepared}, LocalAccess, ExpectedBlockNo) + when is_record(Config, config)-> + %% Config = + %% case os:getenv("TFTPMAX") of + %% false -> + %% Config0; + %% MaxBlockNoStr when Config0#config.debug_level =/= none -> + %% case list_to_integer(MaxBlockNoStr) of + %% MaxBlockNo when ExpectedBlockNo > MaxBlockNo -> + %% info_msg(Config, "TFTPMAX: ~p\n", [MaxBlockNo]), + %% info_msg(Config, "TFTPDEBUG: none\n", []), + %% Config0#config{debug_level = none}; + %% _ -> + %% Config0 + %% end; + %% _MaxBlockNoStr -> + %% Config0 + %% end, + case Status of + ok when is_record(Prepared, prepared) -> + case DecodedMsg of + #tftp_msg_ack{block_no = ActualBlockNo} when LocalAccess =:= read -> + common_read(Config, Callback, Req, LocalAccess, ExpectedBlockNo, ActualBlockNo, Prepared); + #tftp_msg_data{block_no = ActualBlockNo, data = Data} when LocalAccess =:= write -> + common_write(Config, Callback, Req, LocalAccess, ExpectedBlockNo, ActualBlockNo, Data, Prepared); + #tftp_msg_error{code = Code, text = Text} -> + callback({abort, {Code, Text}}, Config, Callback, Req), + terminate(Config, Req, ?ERROR(common_loop, Code, Text, Req#tftp_msg_req.filename)); + {'EXIT', #tftp_msg_error{code = Code, text = Text} = Error} -> + callback({abort, {Code, Text}}, Config, Callback, Req), + send_msg(Config, Req, Error), + terminate(Config, Req, ?ERROR(common_loop, Code, Text, Req#tftp_msg_req.filename)); + Msg when is_tuple(Msg) -> + Code = badop, + Text = "Illegal TFTP operation", + {undefined, Error} = + callback({abort, {Code, Text}}, Config, Callback, Req), + send_msg(Config, Req, Error), + Text2 = lists:flatten([Text, ". ", io_lib:format("~p", [element(1, Msg)])]), + terminate(Config, Req, ?ERROR(common_loop, Code, Text2, Req#tftp_msg_req.filename)) + end; + error when is_record(Prepared, tftp_msg_error) -> + #tftp_msg_error{code = Code, text = Text} = Prepared, + send_msg(Config, Req, Prepared), + terminate(Config, Req, ?ERROR(transfer, Code, Text, Req#tftp_msg_req.filename)) + end; +common_loop(Config, Callback, Req, TransferRes, LocalAccess, ExpectedBlockNo) -> + %% Handle upgrade from old releases. Please, remove this clause in next release. + Config2 = upgrade_config(Config), + common_loop(Config2, Callback, Req, TransferRes, LocalAccess, ExpectedBlockNo). + +-spec common_read(#config{}, #callback{}, _, 'read', _, _, #prepared{}) -> no_return(). + +common_read(Config, _, Req, _, _, _, #prepared{status = terminate, result = Result}) -> + terminate(Config, Req, {ok, Result}); +common_read(Config, Callback, Req, LocalAccess, ExpectedBlockNo, ActualBlockNo, Prepared) + when ActualBlockNo =:= ExpectedBlockNo, is_record(Prepared, prepared) -> + case early_read(Config, Callback, Req, LocalAccess, ActualBlockNo, Prepared) of + {Callback2, #prepared{status = more, next_data = Data} = Prepared2} when is_binary(Data) -> + Prepared3 = Prepared2#prepared{prev_data = Data, next_data = undefined}, + do_common_read(Config, Callback2, Req, LocalAccess, ActualBlockNo, Data, Prepared3); + {undefined, #prepared{status = last, next_data = Data} = Prepared2} when is_binary(Data) -> + Prepared3 = Prepared2#prepared{status = terminate}, + do_common_read(Config, undefined, Req, LocalAccess, ActualBlockNo, Data, Prepared3); + {undefined, #prepared{status = error, result = Error}} -> + #tftp_msg_error{code = Code, text = Text} = Error, + send_msg(Config, Req, Error), + terminate(Config, Req, ?ERROR(read, Code, Text, Req#tftp_msg_req.filename)) + end; +common_read(Config, Callback, Req, LocalAccess, ExpectedBlockNo, ActualBlockNo, Prepared) + when ActualBlockNo =:= (ExpectedBlockNo - 1), is_record(Prepared, prepared) -> + case Prepared of + #prepared{status = more, prev_data = Data} when is_binary(Data) -> + do_common_read(Config, Callback, Req, LocalAccess, ActualBlockNo, Data, Prepared); + #prepared{status = last, prev_data = Data} when is_binary(Data) -> + do_common_read(Config, Callback, Req, LocalAccess, ActualBlockNo, Data, Prepared); + #prepared{status = error, result = Error} -> + #tftp_msg_error{code = Code, text = Text} = Error, + send_msg(Config, Req, Error), + terminate(Config, Req, ?ERROR(read, Code, Text, Req#tftp_msg_req.filename)) + end; +common_read(Config, Callback, Req, LocalAccess, ExpectedBlockNo, ActualBlockNo, Prepared) + when ActualBlockNo =< ExpectedBlockNo, is_record(Prepared, prepared) -> + %% error_logger:error_msg("TFTP READ ~s: Expected block ~p but got block ~p - IGNORED\n", + %% [Req#tftp_msg_req.filename, ExpectedBlockNo, ActualBlockNo]), + case Prepared of + #prepared{status = more, prev_data = Data} when is_binary(Data) -> + Reply = #tftp_msg_data{block_no = ExpectedBlockNo, data = Data}, + {Config2, Callback2, TransferRes} = + wait_for_msg_and_handle_timeout(Config, Callback, Req, Reply, LocalAccess, ExpectedBlockNo, Prepared), + ?MODULE:common_loop(Config2, Callback2, Req, TransferRes, LocalAccess, ExpectedBlockNo); + #prepared{status = last, prev_data = Data} when is_binary(Data) -> + Reply = #tftp_msg_data{block_no = ExpectedBlockNo, data = Data}, + {Config2, Callback2, TransferRes} = + wait_for_msg_and_handle_timeout(Config, Callback, Req, Reply, LocalAccess, ExpectedBlockNo, Prepared), + ?MODULE:common_loop(Config2, Callback2, Req, TransferRes, LocalAccess, ExpectedBlockNo); + #prepared{status = error, result = Error} -> + #tftp_msg_error{code = Code, text = Text} = Error, + send_msg(Config, Req, Error), + terminate(Config, Req, ?ERROR(read, Code, Text, Req#tftp_msg_req.filename)) + end; +common_read(Config, Callback, Req, _LocalAccess, ExpectedBlockNo, ActualBlockNo, Prepared) + when is_record(Prepared, prepared) -> + Code = badblk, + Text = "Unknown transfer ID = " ++ + integer_to_list(ActualBlockNo) ++ " (" ++ integer_to_list(ExpectedBlockNo) ++ ")", + {undefined, Error} = + callback({abort, {Code, Text}}, Config, Callback, Req), + send_msg(Config, Req, Error), + terminate(Config, Req, ?ERROR(read, Code, Text, Req#tftp_msg_req.filename)). + +-spec do_common_read(#config{}, #callback{} | undefined, _, 'read', integer(), binary(), #prepared{}) -> no_return(). + +do_common_read(Config, Callback, Req, LocalAccess, BlockNo, Data, Prepared) + when is_binary(Data), is_record(Prepared, prepared) -> + NextBlockNo = BlockNo + 1, + case NextBlockNo =< 65535 of + true -> + Reply = #tftp_msg_data{block_no = NextBlockNo, data = Data}, + {Config2, Callback2, TransferRes} = + transfer(Config, Callback, Req, Reply, LocalAccess, NextBlockNo, Prepared), + ?MODULE:common_loop(Config2, Callback2, Req, TransferRes, LocalAccess, NextBlockNo); + false -> + Code = badblk, + Text = "Too big transfer ID = " ++ + integer_to_list(NextBlockNo) ++ " > 65535", + {undefined, Error} = + callback({abort, {Code, Text}}, Config, Callback, Req), + send_msg(Config, Req, Error), + terminate(Config, Req, ?ERROR(read, Code, Text, Req#tftp_msg_req.filename)) + end. + +-spec common_write(#config{}, #callback{}, _, 'write', integer(), integer(), _, #prepared{}) -> no_return(). + +common_write(Config, _, Req, _, _, _, _, #prepared{status = terminate, result = Result}) -> + terminate(Config, Req, {ok, Result}); +common_write(Config, Callback, Req, LocalAccess, ExpectedBlockNo, ActualBlockNo, Data, Prepared) + when ActualBlockNo =:= ExpectedBlockNo, is_binary(Data), is_record(Prepared, prepared) -> + case callback({write, Data}, Config, Callback, Req) of + {Callback2, #prepared{status = more} = Prepared2} -> + common_ack(Config, Callback2, Req, LocalAccess, ActualBlockNo, Prepared2); + {undefined, #prepared{status = last, result = Result} = Prepared2} -> + Config2 = pre_terminate(Config, Req, {ok, Result}), + Prepared3 = Prepared2#prepared{status = terminate}, + common_ack(Config2, undefined, Req, LocalAccess, ActualBlockNo, Prepared3); + {undefined, #prepared{status = error, result = Error}} -> + #tftp_msg_error{code = Code, text = Text} = Error, + send_msg(Config, Req, Error), + terminate(Config, Req, ?ERROR(write, Code, Text, Req#tftp_msg_req.filename)) + end; +common_write(Config, Callback, Req, LocalAccess, ExpectedBlockNo, ActualBlockNo, Data, Prepared) + when ActualBlockNo =:= (ExpectedBlockNo - 1), is_binary(Data), is_record(Prepared, prepared) -> + common_ack(Config, Callback, Req, LocalAccess, ExpectedBlockNo - 1, Prepared); +common_write(Config, Callback, Req, LocalAccess, ExpectedBlockNo, ActualBlockNo, Data, Prepared) + when ActualBlockNo =< ExpectedBlockNo, is_binary(Data), is_record(Prepared, prepared) -> + %% error_logger:error_msg("TFTP WRITE ~s: Expected block ~p but got block ~p - IGNORED\n", + %% [Req#tftp_msg_req.filename, ExpectedBlockNo, ActualBlockNo]), + Reply = #tftp_msg_ack{block_no = ExpectedBlockNo}, + {Config2, Callback2, TransferRes} = + wait_for_msg_and_handle_timeout(Config, Callback, Req, Reply, LocalAccess, ExpectedBlockNo, Prepared), + ?MODULE:common_loop(Config2, Callback2, Req, TransferRes, LocalAccess, ExpectedBlockNo); +common_write(Config, Callback, Req, _, ExpectedBlockNo, ActualBlockNo, Data, Prepared) + when is_binary(Data), is_record(Prepared, prepared) -> + Code = badblk, + Text = "Unknown transfer ID = " ++ + integer_to_list(ActualBlockNo) ++ " (" ++ integer_to_list(ExpectedBlockNo) ++ ")", + {undefined, Error} = + callback({abort, {Code, Text}}, Config, Callback, Req), + send_msg(Config, Req, Error), + terminate(Config, Req, ?ERROR(write, Code, Text, Req#tftp_msg_req.filename)). + +common_ack(Config, Callback, Req, LocalAccess, BlockNo, Prepared) + when is_record(Prepared, prepared) -> + Reply = #tftp_msg_ack{block_no = BlockNo}, + NextBlockNo = BlockNo + 1, + {Config2, Callback2, TransferRes} = + transfer(Config, Callback, Req, Reply, LocalAccess, NextBlockNo, Prepared), + case NextBlockNo =< 65535 of + true -> + ?MODULE:common_loop(Config2, Callback2, Req, TransferRes, LocalAccess, NextBlockNo); + false -> + Code = badblk, + Text = "Too big transfer ID = " ++ + integer_to_list(NextBlockNo) ++ " > 65535", + {undefined, Error} = + callback({abort, {Code, Text}}, Config, Callback2, Req), + send_msg(Config, Req, Error), + terminate(Config, Req, ?ERROR(read, Code, Text, Req#tftp_msg_req.filename)) + end. + +pre_terminate(Config, Req, Result) -> + if + Req#tftp_msg_req.local_filename =/= undefined, + Config#config.parent_pid =/= undefined -> + proc_lib:init_ack(Result), + unlink(Config#config.parent_pid), + Config#config{parent_pid = undefined, polite_ack = true}; + true -> + Config#config{polite_ack = true} + end. + +-spec terminate(#config{}, #tftp_msg_req{}, {'ok', _} | #error{}) -> no_return(). + +terminate(Config, Req, Result) -> + Result2 = + case Result of + {ok, _} -> + Result; + #error{where = Where, code = Code, text = Text} = Error -> + print_debug_info(Config, Req, Where, Error#error{filename = Req#tftp_msg_req.filename}), + {error, {Where, Code, Text}} + end, + if + Config#config.parent_pid =:= undefined -> + close_port(Config, client, Req), + exit(normal); + Req#tftp_msg_req.local_filename =/= undefined -> + %% Client + close_port(Config, client, Req), + proc_lib:init_ack(Result2), + unlink(Config#config.parent_pid), + exit(normal); + true -> + %% Server + close_port(Config, server, Req), + exit(shutdown) + end. + +close_port(Config, Who, Req) when is_record(Req, tftp_msg_req) -> + case Config#config.udp_socket of + undefined -> + ignore; + Socket -> + print_debug_info(Config, Who, close, Req), + gen_udp:close(Socket) + end. + +open_free_port(Config, Who, Req) when is_record(Config, config), is_record(Req, tftp_msg_req) -> + UdpOptions = Config#config.udp_options, + case Config#config.port_policy of + random -> + %% BUGBUG: Should be a random port + case catch gen_udp:open(0, UdpOptions) of + {ok, Socket} -> + Config2 = Config#config{udp_socket = Socket}, + print_debug_info(Config2, Who, open, Req), + {ok, Config2}; + {error, Reason} -> + Text = lists:flatten(io_lib:format("UDP open ~p -> ~p", [[0 | UdpOptions], Reason])), + ?ERROR(open, undef, Text, Req#tftp_msg_req.filename); + {'EXIT', _} = Reason -> + Text = lists:flatten(io_lib:format("UDP open ~p -> ~p", [[0 | UdpOptions], Reason])), + ?ERROR(open, undef, Text, Req#tftp_msg_req.filename) + end; + {range, Port, Max} when Port =< Max -> + case catch gen_udp:open(Port, UdpOptions) of + {ok, Socket} -> + Config2 = Config#config{udp_socket = Socket}, + print_debug_info(Config2, Who, open, Req), + {ok, Config2}; + {error, eaddrinuse} -> + PortPolicy = {range, Port + 1, Max}, + Config2 = Config#config{port_policy = PortPolicy}, + open_free_port(Config2, Who, Req); + {error, Reason} -> + Text = lists:flatten(io_lib:format("UDP open ~p -> ~p", [[Port | UdpOptions], Reason])), + ?ERROR(open, undef, Text, Req#tftp_msg_req.filename); + {'EXIT', _} = Reason-> + Text = lists:flatten(io_lib:format("UDP open ~p -> ~p", [[Port | UdpOptions], Reason])), + ?ERROR(open, undef, Text, Req#tftp_msg_req.filename) + end; + {range, Port, _Max} -> + Reason = "Port range exhausted", + Text = lists:flatten(io_lib:format("UDP open ~p -> ~p", [[Port | UdpOptions], Reason])), + ?ERROR(Who, undef, Text, Req#tftp_msg_req.filename) + end. + +%%------------------------------------------------------------------- +%% Transfer +%%------------------------------------------------------------------- + +%% Returns {Config, Callback, #transfer_res{}} +transfer(Config, Callback, Req, Msg, LocalAccess, NextBlockNo, Prepared) + when is_record(Prepared, prepared) -> + IoList = tftp_lib:encode_msg(Msg), + Retries = Config#config.max_retries + 1, + do_transfer(Config, Callback, Req, Msg, IoList, LocalAccess, NextBlockNo, Prepared, Retries). + +do_transfer(Config, Callback, Req, Msg, IoList, LocalAccess, NextBlockNo, Prepared, Retries) + when is_record(Prepared, prepared), is_integer(Retries), Retries >= 0 -> + case do_send_msg(Config, Req, Msg, IoList) of + ok -> + {Callback2, Prepared2} = + early_read(Config, Callback, Req, LocalAccess, NextBlockNo, Prepared), + do_wait_for_msg_and_handle_timeout(Config, Callback2, Req, Msg, IoList, LocalAccess, NextBlockNo, Prepared2, Retries); + {error, _Reason} when Retries > 0 -> + Retries2 = 0, % Just retry once when send fails + do_transfer(Config, Callback, Req, Msg, IoList, LocalAccess, NextBlockNo, Prepared, Retries2); + {error, Reason} -> + Code = undef, + Text = lists:flatten(io_lib:format("Transfer failed - giving up -> ~p", [Reason])), + Error = #tftp_msg_error{code = Code, text = Text}, + {Config, Callback, #transfer_res{status = error, prepared = Error}} + end. + +wait_for_msg_and_handle_timeout(Config, Callback, Req, Msg, LocalAccess, NextBlockNo, Prepared) -> + IoList = tftp_lib:encode_msg(Msg), + Retries = Config#config.max_retries + 1, + do_wait_for_msg_and_handle_timeout(Config, Callback, Req, Msg, IoList, LocalAccess, NextBlockNo, Prepared, Retries). + +do_wait_for_msg_and_handle_timeout(Config, Callback, Req, Msg, IoList, LocalAccess, NextBlockNo, Prepared, Retries) -> + Code = undef, + Text = "Transfer timed out.", + case wait_for_msg(Config, Callback, Req) of + timeout when Config#config.polite_ack =:= true -> + do_send_msg(Config, Req, Msg, IoList), + case Prepared of + #prepared{status = terminate, result = Result} -> + terminate(Config, Req, {ok, Result}); + #prepared{} -> + terminate(Config, Req, ?ERROR(transfer, Code, Text, Req#tftp_msg_req.filename)) + end; + timeout when Retries > 0 -> + Retries2 = Retries - 1, + do_transfer(Config, Callback, Req, Msg, IoList, LocalAccess, NextBlockNo, Prepared, Retries2); + timeout -> + Error = #tftp_msg_error{code = Code, text = Text}, + {Config, Callback, #transfer_res{status = error, prepared = Error}}; + {Config2, DecodedMsg} -> + {Config2, Callback, #transfer_res{status = ok, decoded_msg = DecodedMsg, prepared = Prepared}} + end. + +send_msg(Config, Req, Msg) -> + case catch tftp_lib:encode_msg(Msg) of + {'EXIT', Reason} -> + Code = undef, + Text = "Internal error. Encode failed", + Msg2 = #tftp_msg_error{code = Code, text = Text, details = Reason}, + send_msg(Config, Req, Msg2); + IoList -> + do_send_msg(Config, Req, Msg, IoList) + end. + +do_send_msg(#config{udp_socket = Socket, udp_host = RemoteHost, udp_port = RemotePort} = Config, Req, Msg, IoList) -> + %% {ok, LocalPort} = inet:port(Socket), + %% if + %% LocalPort =/= ?TFTP_DEFAULT_PORT -> + %% ok; + %% true -> + %% print_debug_info(Config#config{debug_level = all}, Req, send, Msg), + %% error(Config, + %% "Daemon replies from the default port (~p)\n\t to ~p:~p\n\t¨~p\n", + %% [LocalPort, RemoteHost, RemotePort, Msg]) + %% end, + + print_debug_info(Config, Req, send, Msg), + + %% case os:getenv("TFTPDUMP") of + %% false -> + %% ignore; + %% DumpPath -> + %% trace_udp_send(Req, Msg, IoList, DumpPath) + %% end, + Res = gen_udp:send(Socket, RemoteHost, RemotePort, IoList), + case Res of + ok -> + ok; + {error, einval = Reason} -> + error_msg(Config, + "Stacktrace; ~p\n gen_udp:send(~p, ~p, ~p, ~p) -> ~p\n", + [erlang:get_stacktrace(), Socket, RemoteHost, RemotePort, IoList, {error, Reason}]); + {error, Reason} -> + {error, Reason} + end. + +%% trace_udp_send(#tftp_msg_req{filename = [$/ | RelFile]} = Req, Msg, IoList, DumpPath) -> +%% trace_udp_send(Req#tftp_msg_req{filename = RelFile}, Msg, IoList, DumpPath); +%% trace_udp_send(#tftp_msg_req{filename = RelFile}, +%% #tftp_msg_data{block_no = BlockNo, data = Data}, +%% _IoList, +%% DumpPath) -> +%% File = filename:join([DumpPath, RelFile, "block" ++ string:right(integer_to_list(BlockNo), 5, $0) ++ ".dump"]), +%% if +%% (BlockNo rem 1000) =:= 1 -> +%% info_msg(Config, "TFTPDUMP: Data ~s\n", [File]); +%% true -> +%% ignore +%% end, +%% ok = filelib:ensure_dir(File), +%% ok = file:write_file(File, Data); +%% trace_udp_send(#tftp_msg_req{filename = RelFile}, Msg, _IoList, _DumpPath) -> +%% info_msg(Config, "TFTPDUMP: No data ~s -> ~p\n", [RelFile, element(1, Msg)]). + +wait_for_msg(Config, Callback, Req) -> + receive + {udp, Socket, RemoteHost, RemotePort, Bin} + when is_binary(Bin), Callback#callback.block_no =:= undefined -> + %% Client prepare + inet:setopts(Socket, [{active, once}]), + Config2 = Config#config{udp_host = RemoteHost, + udp_port = RemotePort}, + DecodedMsg = (catch tftp_lib:decode_msg(Bin)), + print_debug_info(Config2, Req, recv, DecodedMsg), + {Config2, DecodedMsg}; + {udp, Socket, Host, Port, Bin} when is_binary(Bin), + Config#config.udp_host =:= Host, + Config#config.udp_port =:= Port -> + inet:setopts(Socket, [{active, once}]), + DecodedMsg = (catch tftp_lib:decode_msg(Bin)), + print_debug_info(Config, Req, recv, DecodedMsg), + {Config, DecodedMsg}; + {info, Ref, FromPid} when is_pid(FromPid) -> + Type = + case Req#tftp_msg_req.local_filename =/= undefined of + true -> client; + false -> server + end, + Info = internal_info(Config, Type), + reply({ok, Info}, Ref, FromPid), + wait_for_msg(Config, Callback, Req); + {{change_config, Options}, Ref, FromPid} when is_pid(FromPid) -> + case catch tftp_lib:parse_config(Options, Config) of + {'EXIT', Reason} -> + reply({error, Reason}, Ref, FromPid), + wait_for_msg(Config, Callback, Req); + Config2 when is_record(Config2, config) -> + reply(ok, Ref, FromPid), + wait_for_msg(Config2, Callback, Req) + end; + {system, From, Msg} -> + Misc = #sys_misc{module = ?MODULE, function = wait_for_msg, arguments = [Config, Callback, Req]}, + sys:handle_system_msg(Msg, From, Config#config.parent_pid, ?MODULE, [], Misc); + {'EXIT', Pid, _Reason} when Config#config.parent_pid =:= Pid -> + Code = undef, + Text = "Parent exited.", + terminate(Config, Req, ?ERROR(wait_for_msg, Code, Text, Req#tftp_msg_req.filename)); + Msg when Req#tftp_msg_req.local_filename =/= undefined -> + warning_msg(Config, "Client received : ~p", [Msg]), + wait_for_msg(Config, Callback, Req); + Msg when Req#tftp_msg_req.local_filename =:= undefined -> + warning_msg(Config, "Server received : ~p", [Msg]), + wait_for_msg(Config, Callback, Req) + after Config#config.timeout * 1000 -> + print_debug_info(Config, Req, recv, timeout), + timeout + end. + +early_read(Config, Callback, Req, LocalAccess, _NextBlockNo, + #prepared{status = Status, next_data = NextData, prev_data = PrevData} = Prepared) -> + if + Status =/= terminate, + LocalAccess =:= read, + Callback#callback.block_no =/= undefined, + NextData =:= undefined -> + case callback(read, Config, Callback, Req) of + {undefined, Error} when is_record(Error, tftp_msg_error) -> + {undefined, Error}; + {Callback2, Prepared2} when is_record(Prepared2, prepared)-> + {Callback2, Prepared2#prepared{prev_data = PrevData}} + end; + true -> + {Callback, Prepared} + end. + +%%------------------------------------------------------------------- +%% Callback +%%------------------------------------------------------------------- + +callback(Access, Config, Callback, Req) -> + {Callback2, Result} = + do_callback(Access, Config, Callback, Req), + print_debug_info(Config, Req, call, {Callback2, Result}), + {Callback2, Result}. + +do_callback(read = Fun, Config, Callback, Req) + when is_record(Config, config), + is_record(Callback, callback), + is_record(Req, tftp_msg_req) -> + Args = [Callback#callback.state], + NextBlockNo = Callback#callback.block_no + 1, + case catch safe_apply(Callback#callback.module, Fun, Args) of + {more, Bin, NewState} when is_binary(Bin) -> + Count = Callback#callback.count + size(Bin), + Callback2 = Callback#callback{state = NewState, + block_no = NextBlockNo, + count = Count}, + Prepared = #prepared{status = more, + result = undefined, + block_no = NextBlockNo, + next_data = Bin}, + verify_count(Config, Callback2, Req, Prepared); + {last, Bin, Result} when is_binary(Bin) -> + Prepared = #prepared{status = last, + result = Result, + block_no = NextBlockNo, + next_data = Bin}, + {undefined, Prepared}; + {error, {Code, Text}} -> + Error = #tftp_msg_error{code = Code, text = Text}, + Prepared = #prepared{status = error, + result = Error}, + {undefined, Prepared}; + Illegal -> + Code = undef, + Text = "Internal error. File handler error.", + callback({abort, {Code, Text, Illegal}}, Config, Callback, Req) + end; +do_callback({write = Fun, Bin}, Config, Callback, Req) + when is_record(Config, config), + is_record(Callback, callback), + is_record(Req, tftp_msg_req), + is_binary(Bin) -> + Args = [Bin, Callback#callback.state], + NextBlockNo = Callback#callback.block_no + 1, + case catch safe_apply(Callback#callback.module, Fun, Args) of + {more, NewState} -> + Count = Callback#callback.count + size(Bin), + Callback2 = Callback#callback{state = NewState, + block_no = NextBlockNo, + count = Count}, + Prepared = #prepared{status = more, + block_no = NextBlockNo}, + verify_count(Config, Callback2, Req, Prepared); + {last, Result} -> + Prepared = #prepared{status = last, + result = Result, + block_no = NextBlockNo}, + {undefined, Prepared}; + {error, {Code, Text}} -> + Error = #tftp_msg_error{code = Code, text = Text}, + Prepared = #prepared{status = error, + result = Error}, + {undefined, Prepared}; + Illegal -> + Code = undef, + Text = "Internal error. File handler error.", + callback({abort, {Code, Text, Illegal}}, Config, Callback, Req) + end; +do_callback({open, Type}, Config, Callback, Req) + when is_record(Config, config), + is_record(Callback, callback), + is_record(Req, tftp_msg_req) -> + {Access, Filename} = local_file_access(Req), + {Fun, BlockNo} = + case Type of + client_prepare -> {prepare, undefined}; + client_open -> {open, 0}; + server_open -> {open, 0} + end, + Mod = Callback#callback.module, + Args = [Access, + Filename, + Req#tftp_msg_req.mode, + Req#tftp_msg_req.options, + Callback#callback.state], + PeerInfo = peer_info(Config), + fast_ensure_loaded(Mod), + Args2 = + case erlang:function_exported(Mod, Fun, length(Args)) of + true -> Args; + false -> [PeerInfo | Args] + end, + case catch safe_apply(Mod, Fun, Args2) of + {ok, AcceptedOptions, NewState} -> + Callback2 = Callback#callback{state = NewState, + block_no = BlockNo, + count = 0}, + {Callback2, {ok, AcceptedOptions}}; + {error, {Code, Text}} -> + {undefined, #tftp_msg_error{code = Code, text = Text}}; + Illegal -> + Code = undef, + Text = "Internal error. File handler error.", + callback({abort, {Code, Text, Illegal}}, Config, Callback, Req) + end; +do_callback({abort, {Code, Text}}, Config, Callback, Req) -> + Error = #tftp_msg_error{code = Code, text = Text}, + do_callback({abort, Error}, Config, Callback, Req); +do_callback({abort, {Code, Text, Details}}, Config, Callback, Req) -> + Error = #tftp_msg_error{code = Code, text = Text, details = Details}, + do_callback({abort, Error}, Config, Callback, Req); +do_callback({abort = Fun, #tftp_msg_error{code = Code, text = Text} = Error}, Config, Callback, Req) + when is_record(Config, config), + is_record(Callback, callback), + is_record(Req, tftp_msg_req) -> + Args = [Code, Text, Callback#callback.state], + catch safe_apply(Callback#callback.module, Fun, Args), + {undefined, Error}; +do_callback({abort, Error}, _Config, undefined, _Req) when is_record(Error, tftp_msg_error) -> + {undefined, Error}. + +peer_info(#config{udp_host = Host, udp_port = Port}) -> + if + is_tuple(Host), size(Host) =:= 4 -> + {inet, tftp_lib:host_to_string(Host), Port}; + is_tuple(Host), size(Host) =:= 8 -> + {inet6, tftp_lib:host_to_string(Host), Port}; + true -> + {undefined, Host, Port} + end. + +match_callback(Filename, Callbacks) -> + if + Filename =:= binary -> + lookup_callback_mod(tftp_binary, Callbacks); + is_binary(Filename) -> + lookup_callback_mod(tftp_binary, Callbacks); + true -> + do_match_callback(Filename, Callbacks) + end. + +do_match_callback(Filename, [C | Tail]) when is_record(C, callback) -> + case catch inets_regexp:match(Filename, C#callback.internal) of + {match, _, _} -> + {ok, C}; + nomatch -> + do_match_callback(Filename, Tail); + Details -> + Code = baduser, + Text = "Internal error. File handler not found", + {error, #tftp_msg_error{code = Code, text = Text, details = Details}} + end; +do_match_callback(Filename, []) -> + Code = baduser, + Text = "Internal error. File handler not found", + {error, #tftp_msg_error{code = Code, text = Text, details = Filename}}. + +lookup_callback_mod(Mod, Callbacks) -> + {value, C} = lists:keysearch(Mod, #callback.module, Callbacks), + {ok, C}. + +verify_count(Config, Callback, Req, Result) -> + case Config#config.max_tsize of + infinity -> + {Callback, Result}; + Max when Callback#callback.count =< Max -> + {Callback, Result}; + _Max -> + Code = enospc, + Text = "Too large file.", + callback({abort, {Code, Text}}, Config, Callback, Req) + end. + +%%------------------------------------------------------------------- +%% Miscellaneous +%%------------------------------------------------------------------- + +internal_info(Config, Type) when is_record(Config, config) -> + {ok, ActualPort} = inet:port(Config#config.udp_socket), + [ + {type, Type}, + {host, tftp_lib:host_to_string(Config#config.udp_host)}, + {port, Config#config.udp_port}, + {local_port, ActualPort}, + {port_policy, Config#config.port_policy}, + {udp, Config#config.udp_options}, + {use_tsize, Config#config.use_tsize}, + {max_tsize, Config#config.max_tsize}, + {max_conn, Config#config.max_conn}, + {rejected, Config#config.rejected}, + {timeout, Config#config.timeout}, + {polite_ack, Config#config.polite_ack}, + {debug, Config#config.debug_level}, + {parent_pid, Config#config.parent_pid} + ] ++ Config#config.user_options ++ Config#config.callbacks. + +local_file_access(#tftp_msg_req{access = Access, + local_filename = Local, + filename = Filename}) -> + case Local =:= undefined of + true -> + %% Server side + {Access, Filename}; + false -> + %% Client side + case Access of + read -> {write, Local}; + write -> {read, Local} + end + end. + +pre_verify_options(Config, Req) -> + Options = Req#tftp_msg_req.options, + case catch verify_reject(Config, Req, Options) of + ok -> + case verify_integer("tsize", 0, Config#config.max_tsize, Options) of + true -> + case verify_integer("blksize", 0, 65464, Options) of + true -> + ok; + false -> + {error, {badopt, "Too large blksize"}} + end; + false -> + {error, {badopt, "Too large tsize"}} + end; + {error, Reason} -> + {error, Reason} + end. + +post_verify_options(Config, Req, NewOptions, Text) -> + OldOptions = Req#tftp_msg_req.options, + BadOptions = + [Key || {Key, _Val} <- NewOptions, + not lists:keymember(Key, 1, OldOptions)], + case BadOptions =:= [] of + true -> + Config2 = Config#config{timeout = lookup_timeout(NewOptions)}, + Req2 = Req#tftp_msg_req{options = NewOptions}, + {ok, Config2, Req2}; + false -> + {error, {badopt, Text}} + end. + +verify_reject(Config, Req, Options) -> + Access = Req#tftp_msg_req.access, + Rejected = Config#config.rejected, + case lists:member(Access, Rejected) of + true -> + {error, {eacces, atom_to_list(Access) ++ " mode not allowed"}}; + false -> + [throw({error, {badopt, Key ++ " not allowed"}}) || + {Key, _} <- Options, lists:member(Key, Rejected)], + ok + end. + +lookup_timeout(Options) -> + case lists:keysearch("timeout", 1, Options) of + {value, {_, Val}} -> + list_to_integer(Val); + false -> + 3 + end. + +lookup_mode(Options) -> + case lists:keysearch("mode", 1, Options) of + {value, {_, Val}} -> + Val; + false -> + "octet" + end. + +verify_integer(Key, Min, Max, Options) -> + case lists:keysearch(Key, 1, Options) of + {value, {_, Val}} when is_list(Val) -> + case catch list_to_integer(Val) of + {'EXIT', _} -> + false; + Int when Int >= Min, is_integer(Min), + Max =:= infinity -> + true; + Int when Int >= Min, is_integer(Min), + Int =< Max, is_integer(Max) -> + true; + _ -> + false + end; + false -> + true + end. + +error_msg(#config{logger = Logger, debug_level = _Level}, F, A) -> + safe_apply(Logger, error_msg, [F, A]). + +warning_msg(#config{logger = Logger, debug_level = Level}, F, A) -> + case Level of + none -> ok; + error -> ok; + _ -> safe_apply(Logger, warning_msg, [F, A]) + end. + +info_msg(#config{logger = Logger}, F, A) -> + safe_apply(Logger, info_msg, [F, A]). + +safe_apply(Mod, Fun, Args) -> + fast_ensure_loaded(Mod), + apply(Mod, Fun, Args). + +fast_ensure_loaded(Mod) -> + case erlang:function_exported(Mod, module_info, 0) of + true -> + ok; + false -> + Res = code:load_file(Mod), + %% io:format("tftp: code:load_file(~p) -> ~p\n", [Mod, Res]), %% XXX + Res + end. + +print_debug_info(#config{debug_level = Level} = Config, Who, Where, Data) -> + if + Level =:= none -> + ok; + is_record(Data, error) -> + do_print_debug_info(Config, Who, Where, Data); + Level =:= warning -> + ok; + Level =:= error -> + ok; + Level =:= all -> + do_print_debug_info(Config, Who, Where, Data); + Where =:= open -> + do_print_debug_info(Config, Who, Where, Data); + Where =:= close -> + do_print_debug_info(Config, Who, Where, Data); + Level =:= brief -> + ok; + Where =/= recv, Where =/= send -> + ok; + is_record(Data, tftp_msg_data), Level =:= normal -> + ok; + is_record(Data, tftp_msg_ack), Level =:= normal -> + ok; + true -> + do_print_debug_info(Config, Who, Where, Data) + end. + +do_print_debug_info(Config, Who, Where, #tftp_msg_data{data = Bin} = Msg) when is_binary(Bin) -> + Msg2 = Msg#tftp_msg_data{data = {bytes, size(Bin)}}, + do_print_debug_info(Config, Who, Where, Msg2); +do_print_debug_info(Config, Who, Where, #tftp_msg_req{local_filename = Filename} = Msg) when is_binary(Filename) -> + Msg2 = Msg#tftp_msg_req{local_filename = binary}, + do_print_debug_info(Config, Who, Where, Msg2); +do_print_debug_info(Config, Who, Where, Data) -> + Local = + case catch inet:port(Config#config.udp_socket) of + {'EXIT', _Reason} -> + 0; + {ok, Port} -> + Port + end, + %% Remote = Config#config.udp_port, + PeerInfo = lists:flatten(io_lib:format("~p", [peer_info(Config)])), + Side = + if + is_record(Who, tftp_msg_req), + Who#tftp_msg_req.local_filename =/= undefined -> + client; + is_record(Who, tftp_msg_req), + Who#tftp_msg_req.local_filename =:= undefined -> + server; + is_atom(Who) -> + Who + end, + case {Where, Data} of + {_, #error{where = Where, code = Code, text = Text, filename = Filename}} -> + do_format(Config, Side, Local, "error ~s ->\n\t~p ~p\n\t~p ~p: ~s\n", + [PeerInfo, self(), Filename, Where, Code, Text]); + {open, #tftp_msg_req{filename = Filename}} -> + do_format(Config, Side, Local, "open ~s ->\n\t~p ~p\n", + [PeerInfo, self(), Filename]); + {close, #tftp_msg_req{filename = Filename}} -> + do_format(Config, Side, Local, "close ~s ->\n\t~p ~p\n", + [PeerInfo, self(), Filename]); + {recv, _} -> + do_format(Config, Side, Local, "recv ~s <-\n\t~p\n", + [PeerInfo, Data]); + {send, _} -> + do_format(Config, Side, Local, "send ~s ->\n\t~p\n", + [PeerInfo, Data]); + {match, _} when is_record(Data, callback) -> + Mod = Data#callback.module, + State = Data#callback.state, + do_format(Config, Side, Local, "match ~s ~p =>\n\t~p\n", + [PeerInfo, Mod, State]); + {call, _} -> + case Data of + {Callback, _Result} when is_record(Callback, callback) -> + Mod = Callback#callback.module, + State = Callback#callback.state, + do_format(Config, Side, Local, "call ~s ~p =>\n\t~p\n", + [PeerInfo, Mod, State]); + {undefined, Result} -> + do_format(Config, Side, Local, "call ~s result =>\n\t~p\n", + [PeerInfo, Result]) + end + end. + +do_format(Config, Side, Local, Format, Args) -> + info_msg(Config, "~p(~p): " ++ Format, [Side, Local | Args]). + +%%------------------------------------------------------------------- +%% System upgrade +%%------------------------------------------------------------------- + +system_continue(_Parent, _Debug, #sys_misc{module = Mod, function = Fun, arguments = Args}) -> + apply(Mod, Fun, Args); +system_continue(Parent, Debug, {Fun, Args}) -> + %% Handle upgrade from old releases. Please, remove this clause in next release. + system_continue(Parent, Debug, #sys_misc{module = ?MODULE, function = Fun, arguments = Args}). + +-spec system_terminate(_, _, _, #sys_misc{} | {_, _}) -> no_return(). + +system_terminate(Reason, _Parent, _Debug, #sys_misc{}) -> + exit(Reason); +system_terminate(Reason, Parent, Debug, {Fun, Args}) -> + %% Handle upgrade from old releases. Please, remove this clause in next release. + system_terminate(Reason, Parent, Debug, #sys_misc{module = ?MODULE, function = Fun, arguments = Args}). + +system_code_change({Fun, Args}, _Module, _OldVsn, _Extra) -> + {ok, {Fun, Args}}. diff --git a/lib/inets/src/tftp/tftp_file.erl b/lib/inets/src/tftp/tftp_file.erl new file mode 100644 index 0000000000..e0cbb49330 --- /dev/null +++ b/lib/inets/src/tftp/tftp_file.erl @@ -0,0 +1,389 @@ +%% +%% %CopyrightBegin% +%% +%% Copyright Ericsson AB 2005-2009. All Rights Reserved. +%% +%% The contents of this file are subject to the Erlang Public License, +%% Version 1.1, (the "License"); you may not use this file except in +%% compliance with the License. You should have received a copy of the +%% Erlang Public License along with this software. If not, it can be +%% retrieved online at http://www.erlang.org/. +%% +%% Software distributed under the License is distributed on an "AS IS" +%% basis, WITHOUT WARRANTY OF ANY KIND, either express or implied. See +%% the License for the specific language governing rights and limitations +%% under the License. +%% +%% %CopyrightEnd% +%% +%% + +%%%------------------------------------------------------------------- +%%% File : tft_file.erl +%%% Author : Hakan Mattsson <[email protected]> +%%% Description : +%%% +%%% Created : 24 May 2004 by Hakan Mattsson <[email protected]> +%%%------------------------------------------------------------------- + +-module(tftp_file). + +%%%------------------------------------------------------------------- +%%% Interface +%%%------------------------------------------------------------------- + +-behaviour(tftp). + +-export([prepare/6, open/6, read/1, write/2, abort/3]). + +%%%------------------------------------------------------------------- +%%% Defines +%%%------------------------------------------------------------------- + +-include_lib("kernel/include/file.hrl"). + +-record(initial, + {filename, + is_native_ascii}). + +-record(state, + {access, + filename, + is_native_ascii, + is_network_ascii, + root_dir, + options, + blksize, + fd, + count, + buffer}). + +%%------------------------------------------------------------------- +%% prepare(Peer, Access, Filename, Mode, SuggestedOptions, InitialState) -> +%% {ok, AcceptedOptions, NewState} | {error, Code, Text} +%% +%% Peer = {PeerType, PeerHost, PeerPort} +%% PeerType = inet | inet6 +%% PeerHost = ip_address() +%% PeerPort = integer() +%% Acess = read | write +%% Filename = string() +%% Mode = string() +%% SuggestedOptions = [{Key, Value}] +%% AcceptedOptions = [{Key, Value}] +%% Key = string() +%% Value = string() +%% InitialState = [] | [{root_dir, string()}] +%% NewState = term() +%% Code = undef | enoent | eacces | enospc | +%% badop | eexist | baduser | badopt | +%% integer() +%% Text = string() +%% +%% Prepares open of a file on the client side. +%% +%% Will be followed by a call to open/4 before any read/write access +%% is performed. The AcceptedOptions will be sent to the server which +%% will reply with those options that it accepts. The options that are +%% accepted by the server will be forwarded to open/4 as SuggestedOptions. +%% +%% No new options may be added, but the ones that are present as +%% SuggestedOptions may be omitted or replaced with new values +%% in the AcceptedOptions. +%%------------------------------------------------------------------- + +prepare(_Peer, Access, Filename, Mode, SuggestedOptions, Initial) when is_list(Initial) -> + %% Client side + case catch handle_options(Access, Filename, Mode, SuggestedOptions, Initial) of + {ok, Filename2, IsNativeAscii, IsNetworkAscii, AcceptedOptions} -> + State = #state{access = Access, + filename = Filename2, + is_native_ascii = IsNativeAscii, + is_network_ascii = IsNetworkAscii, + options = AcceptedOptions, + blksize = lookup_blksize(AcceptedOptions), + count = 0, + buffer = []}, + {ok, AcceptedOptions, State}; + {error, {Code, Text}} -> + {error, {Code, Text}} + end. + +%% --------------------------------------------------------- +%% open(Peer, Access, Filename, Mode, SuggestedOptions, State) -> +%% {ok, AcceptedOptions, NewState} | {error, Code, Text} +%% +%% Peer = {PeerType, PeerHost, PeerPort} +%% PeerType = inet | inet6 +%% PeerHost = ip_address() +%% PeerPort = integer() +%% Acess = read | write +%% Filename = string() +%% Mode = string() +%% SuggestedOptions = [{Key, Value}] +%% AcceptedOptions = [{Key, Value}] +%% Key = string() +%% Value = string() +%% State = InitialState | #state{} +%% InitialState = [] | [{root_dir, string()}] +%% NewState = term() +%% Code = undef | enoent | eacces | enospc | +%% badop | eexist | baduser | badopt | +%% integer() +%% Text = string() +%% +%% Opens a file for read or write access. +%% +%% On the client side where the open/4 call has been preceeded by a +%% call to prepare/4, all options must be accepted or rejected. +%% On the server side, where there are no preceeding prepare/4 call, +%% noo new options may be added, but the ones that are present as +%% SuggestedOptions may be omitted or replaced with new values +%% in the AcceptedOptions. +%%------------------------------------------------------------------- + +open(Peer, Access, Filename, Mode, SuggestedOptions, Initial) when is_list(Initial) -> + %% Server side + case prepare(Peer, Access, Filename, Mode, SuggestedOptions, Initial) of + {ok, AcceptedOptions, State} -> + open(Peer, Access, Filename, Mode, AcceptedOptions, State); + {error, {Code, Text}} -> + {error, {Code, Text}} + end; +open(_Peer, Access, Filename, Mode, NegotiatedOptions, State) when is_record(State, state) -> + %% Both sides + case catch handle_options(Access, Filename, Mode, NegotiatedOptions, State) of + {ok, _Filename2, _IsNativeAscii, _IsNetworkAscii, Options} + when Options =:= NegotiatedOptions -> + do_open(State); + {error, {Code, Text}} -> + {error, {Code, Text}} + end; +open(Peer, Access, Filename, Mode, NegotiatedOptions, State) -> + %% Handle upgrade from old releases. Please, remove this clause in next release. + State2 = upgrade_state(State), + open(Peer, Access, Filename, Mode, NegotiatedOptions, State2). + +do_open(State) when is_record(State, state) -> + case file:open(State#state.filename, file_options(State)) of + {ok, Fd} -> + {ok, State#state.options, State#state{fd = Fd}}; + {error, Reason} when is_atom(Reason) -> + {error, file_error(Reason)} + end. + +file_options(State) -> + case State#state.access of + read -> [read, read_ahead, raw, binary]; + write -> [write, delayed_write, raw, binary] + end. + +file_error(Reason) when is_atom(Reason) -> + Details = file:format_error(Reason), + case Reason of + eexist -> {Reason, Details}; + enoent -> {Reason, Details}; + eacces -> {Reason, Details}; + eperm -> {eacces, Details}; + enospc -> {Reason, Details}; + _ -> {undef, Details ++ " (" ++ atom_to_list(Reason) ++ ")"} + end. + +%%------------------------------------------------------------------- +%% read(State) -> +%% {more, Bin, NewState} | {last, Bin, FileSize} | {error, {Code, Text}} +%% +%% State = term() +%% NewState = term() +%% Bin = binary() +%% FileSize = integer() +%% Code = undef | enoent | eacces | enospc | +%% badop | eexist | baduser | badopt | +%% integer() +%% Text = string() +%% +%% Reads a chunk from the file +%% +%% The file is automatically closed when the last chunk is read. +%%------------------------------------------------------------------- + +read(#state{access = read} = State) -> + BlkSize = State#state.blksize, + case file:read(State#state.fd, BlkSize) of + {ok, Bin} when is_binary(Bin), size(Bin) =:= BlkSize -> + Count = State#state.count + size(Bin), + {more, Bin, State#state{count = Count}}; + {ok, Bin} when is_binary(Bin), size(Bin) < BlkSize -> + file:close(State#state.fd), + Count = State#state.count + size(Bin), + {last, Bin, Count}; + eof -> + {last, <<>>, State#state.count}; + {error, Reason} -> + file:close(State#state.fd), + {error, file_error(Reason)} + end; +read(State) -> + %% Handle upgrade from old releases. Please, remove this clause in next release. + State2 = upgrade_state(State), + read(State2). + +%%------------------------------------------------------------------- +%% write(Bin, State) -> +%% {more, NewState} | {last, FileSize} | {error, {Code, Text}} +%% +%% State = term() +%% NewState = term() +%% Bin = binary() +%% FileSize = integer() +%% Code = undef | enoent | eacces | enospc | +%% badop | eexist | baduser | badopt | +%% integer() +%% Text = string() +%% +%% Writes a chunk to the file +%% +%% The file is automatically closed when the last chunk is written +%%------------------------------------------------------------------- + +write(Bin, #state{access = write} = State) when is_binary(Bin) -> + Size = size(Bin), + BlkSize = State#state.blksize, + case file:write(State#state.fd, Bin) of + ok when Size =:= BlkSize-> + Count = State#state.count + Size, + {more, State#state{count = Count}}; + ok when Size < BlkSize-> + file:close(State#state.fd), + Count = State#state.count + Size, + {last, Count}; + {error, Reason} -> + file:close(State#state.fd), + file:delete(State#state.filename), + {error, file_error(Reason)} + end; +write(Bin, State) -> + %% Handle upgrade from old releases. Please, remove this clause in next release. + State2 = upgrade_state(State), + write(Bin, State2). + +%%------------------------------------------------------------------- +%% abort(Code, Text, State) -> ok +%% +%% State = term() +%% Code = undef | enoent | eacces | enospc | +%% badop | eexist | baduser | badopt | +%% badblk | integer() +%% Text = string() +%% +%% Aborts the file transfer +%%------------------------------------------------------------------- + +abort(_Code, _Text, #state{fd = Fd, access = Access} = State) -> + file:close(Fd), + case Access of + write -> + ok = file:delete(State#state.filename); + read -> + ok + end. + +%%------------------------------------------------------------------- +%% Process options +%%------------------------------------------------------------------- + +handle_options(Access, Filename, Mode, Options, Initial) -> + I = #initial{filename = Filename, is_native_ascii = is_native_ascii()}, + {Filename2, IsNativeAscii} = handle_initial(Initial, I), + IsNetworkAscii = handle_mode(Mode, IsNativeAscii), + Options2 = do_handle_options(Access, Filename2, Options), + {ok, Filename2, IsNativeAscii, IsNetworkAscii, Options2}. + +handle_mode(Mode, IsNativeAscii) -> + case Mode of + "netascii" when IsNativeAscii =:= true -> true; + "octet" -> false; + _ -> throw({error, {badop, "Illegal mode " ++ Mode}}) + end. + +handle_initial([{root_dir, Dir} | Initial], I) -> + case catch filename_join(Dir, I#initial.filename) of + {'EXIT', _} -> + throw({error, {badop, "Internal error. root_dir is not a string"}}); + Filename2 -> + handle_initial(Initial, I#initial{filename = Filename2}) + end; +handle_initial([{native_ascii, Bool} | Initial], I) -> + case Bool of + true -> handle_initial(Initial, I#initial{is_native_ascii = true}); + false -> handle_initial(Initial, I#initial{is_native_ascii = false}) + end; +handle_initial([], I) when is_record(I, initial) -> + {I#initial.filename, I#initial.is_native_ascii}; +handle_initial(State, _) when is_record(State, state) -> + {State#state.filename, State#state.is_native_ascii}. + +filename_join(Dir, Filename) -> + case filename:pathtype(Filename) of + absolute -> + [_ | RelFilename] = filename:split(Filename), + filename:join([Dir, RelFilename]); + _ -> + filename:join([Dir, Filename]) + end. + +do_handle_options(Access, Filename, [{Key, Val} | T]) -> + case Key of + "tsize" -> + case Access of + read when Val =:= "0" -> + case file:read_file_info(Filename) of + {ok, FI} -> + Tsize = integer_to_list(FI#file_info.size), + [{Key, Tsize} | do_handle_options(Access, Filename, T)]; + {error, _} -> + do_handle_options(Access, Filename, T) + end; + _ -> + handle_integer(Access, Filename, Key, Val, T, 0, infinity) + end; + "blksize" -> + handle_integer(Access, Filename, Key, Val, T, 8, 65464); + "timeout" -> + handle_integer(Access, Filename, Key, Val, T, 1, 255); + _ -> + do_handle_options(Access, Filename, T) + end; +do_handle_options(_Access, _Filename, []) -> + []. + + +handle_integer(Access, Filename, Key, Val, Options, Min, Max) -> + case catch list_to_integer(Val) of + {'EXIT', _} -> + do_handle_options(Access, Filename, Options); + Int when Int >= Min, Int =< Max -> + [{Key, Val} | do_handle_options(Access, Filename, Options)]; + Int when Int >= Min, Max =:= infinity -> + [{Key, Val} | do_handle_options(Access, Filename, Options)]; + _Int -> + throw({error, {badopt, "Illegal " ++ Key ++ " value " ++ Val}}) + end. + +lookup_blksize(Options) -> + case lists:keysearch("blksize", 1, Options) of + {value, {_, Val}} -> + list_to_integer(Val); + false -> + 512 + end. + +is_native_ascii() -> + case os:type() of + {win32, _} -> true; + _ -> false + end. + +%% Handle upgrade from old releases. Please, remove this function in next release. +upgrade_state({state, Access, Filename, RootDir, Options, BlkSize, Fd, Count, Buffer}) -> + {state, Access, Filename, false, false, RootDir, Options, BlkSize, Fd, Count, Buffer}. diff --git a/lib/inets/src/tftp/tftp_lib.erl b/lib/inets/src/tftp/tftp_lib.erl new file mode 100644 index 0000000000..ffb7b9a797 --- /dev/null +++ b/lib/inets/src/tftp/tftp_lib.erl @@ -0,0 +1,473 @@ +%% +%% %CopyrightBegin% +%% +%% Copyright Ericsson AB 2005-2009. All Rights Reserved. +%% +%% The contents of this file are subject to the Erlang Public License, +%% Version 1.1, (the "License"); you may not use this file except in +%% compliance with the License. You should have received a copy of the +%% Erlang Public License along with this software. If not, it can be +%% retrieved online at http://www.erlang.org/. +%% +%% Software distributed under the License is distributed on an "AS IS" +%% basis, WITHOUT WARRANTY OF ANY KIND, either express or implied. See +%% the License for the specific language governing rights and limitations +%% under the License. +%% +%% %CopyrightEnd% +%% +%% + +%%%------------------------------------------------------------------- +%%% File : tftp_lib.erl +%%% Author : Hakan Mattsson <[email protected]> +%%% Description : Option parsing, decode, encode etc. +%%% +%%% Created : 18 May 2004 by Hakan Mattsson <[email protected]> +%%%------------------------------------------------------------------- + +-module(tftp_lib). + +%%------------------------------------------------------------------- +%% Interface +%%------------------------------------------------------------------- + +%% application internal functions +-export([ + parse_config/1, + parse_config/2, + decode_msg/1, + encode_msg/1, + replace_val/3, + to_lower/1, + host_to_string/1, + add_default_callbacks/1 + ]). + +%%------------------------------------------------------------------- +%% Defines +%%------------------------------------------------------------------- + +-include("tftp.hrl"). + +-define(LOWER(Char), + if + Char >= $A, Char =< $Z -> + Char - ($A - $a); + true -> + Char + end). + +%%------------------------------------------------------------------- +%% Config +%%------------------------------------------------------------------- + +parse_config(Options) -> + parse_config(Options, #config{}). + +parse_config(Options, Config) -> + do_parse_config(Options, Config). + +do_parse_config([{Key, Val} | Tail], Config) when is_record(Config, config) -> + case Key of + debug -> + if + Val =:= 0; Val =:= none -> + do_parse_config(Tail, Config#config{debug_level = none}); + Val =:= 1; Val =:= error -> + do_parse_config(Tail, Config#config{debug_level = error}); + Val =:= 2; Val =:= warning -> + do_parse_config(Tail, Config#config{debug_level = warning}); + Val =:= 3; Val =:= brief -> + do_parse_config(Tail, Config#config{debug_level = brief}); + Val =:= 4; Val =:= normal -> + do_parse_config(Tail, Config#config{debug_level = normal}); + Val =:= 5; Val =:= verbose -> + do_parse_config(Tail, Config#config{debug_level = verbose}); + Val =:= 6; Val =:= all -> + do_parse_config(Tail, Config#config{debug_level = all}); + true -> + exit({badarg, {Key, Val}}) + end; + host -> + if + is_list(Val) -> + do_parse_config(Tail, Config#config{udp_host = Val}); + is_tuple(Val), size(Val) =:= 4 -> + do_parse_config(Tail, Config#config{udp_host = Val}); + is_tuple(Val), size(Val) =:= 8 -> + do_parse_config(Tail, Config#config{udp_host = Val}); + true -> + exit({badarg, {Key, Val}}) + end; + port -> + if + is_integer(Val), Val >= 0 -> + Config2 = Config#config{udp_port = Val, udp_options = Config#config.udp_options}, + do_parse_config(Tail, Config2); + true -> + exit({badarg, {Key, Val}}) + end; + port_policy -> + case Val of + random -> + do_parse_config(Tail, Config#config{port_policy = Val}); + 0 -> + do_parse_config(Tail, Config#config{port_policy = random}); + MinMax when is_integer(MinMax), MinMax > 0 -> + do_parse_config(Tail, Config#config{port_policy = {range, MinMax, MinMax}}); + {range, Min, Max} when Max >= Min, + is_integer(Min), Min > 0, + is_integer(Max), Max > 0 -> + do_parse_config(Tail, Config#config{port_policy = Val}); + true -> + exit({badarg, {Key, Val}}) + end; + udp when is_list(Val) -> + Fun = + fun({K, V}, List) when K /= active -> + replace_val(K, V, List); + (V, List) when V /= list, V /= binary -> + List ++ [V]; + (V, _List) -> + exit({badarg, {udp, [V]}}) + end, + UdpOptions = lists:foldl(Fun, Config#config.udp_options, Val), + do_parse_config(Tail, Config#config{udp_options = UdpOptions}); + use_tsize -> + case Val of + true -> + do_parse_config(Tail, Config#config{use_tsize = Val}); + false -> + do_parse_config(Tail, Config#config{use_tsize = Val}); + _ -> + exit({badarg, {Key, Val}}) + end; + max_tsize -> + if + Val =:= infinity -> + do_parse_config(Tail, Config#config{max_tsize = Val}); + is_integer(Val), Val >= 0 -> + do_parse_config(Tail, Config#config{max_tsize = Val}); + true -> + exit({badarg, {Key, Val}}) + end; + max_conn -> + if + Val =:= infinity -> + do_parse_config(Tail, Config#config{max_conn = Val}); + is_integer(Val), Val > 0 -> + do_parse_config(Tail, Config#config{max_conn = Val}); + true -> + exit({badarg, {Key, Val}}) + end; + _ when is_list(Key), is_list(Val) -> + Key2 = to_lower(Key), + Val2 = to_lower(Val), + TftpOptions = replace_val(Key2, Val2, Config#config.user_options), + do_parse_config(Tail, Config#config{user_options = TftpOptions}); + reject -> + case Val of + read -> + Rejected = [Val | Config#config.rejected], + do_parse_config(Tail, Config#config{rejected = Rejected}); + write -> + Rejected = [Val | Config#config.rejected], + do_parse_config(Tail, Config#config{rejected = Rejected}); + _ when is_list(Val) -> + Rejected = [Val | Config#config.rejected], + do_parse_config(Tail, Config#config{rejected = Rejected}); + _ -> + exit({badarg, {Key, Val}}) + end; + callback -> + case Val of + {RegExp, Mod, State} when is_list(RegExp), is_atom(Mod) -> + case inets_regexp:parse(RegExp) of + {ok, Internal} -> + Callback = #callback{regexp = RegExp, + internal = Internal, + module = Mod, + state = State}, + Callbacks = Config#config.callbacks ++ [Callback], + do_parse_config(Tail, Config#config{callbacks = Callbacks}); + {error, Reason} -> + exit({badarg, {Key, Val}, Reason}) + end; + _ -> + exit({badarg, {Key, Val}}) + end; + logger -> + if + is_atom(Val) -> + do_parse_config(Tail, Config#config{logger = Val}); + true -> + exit({badarg, {Key, Val}}) + end; + max_retries -> + if + is_integer(Val), Val > 0 -> + do_parse_config(Tail, Config#config{max_retries = Val}); + true -> + exit({badarg, {Key, Val}}) + end; + _ -> + exit({badarg, {Key, Val}}) + end; +do_parse_config([], #config{udp_host = Host, + udp_options = UdpOptions, + user_options = UserOptions, + callbacks = Callbacks} = Config) -> + IsInet6 = lists:member(inet6, UdpOptions), + IsInet = lists:member(inet, UdpOptions), + Host2 = + if + (IsInet and not IsInet6); (not IsInet and not IsInet6) -> + case inet:getaddr(Host, inet) of + {ok, Addr} -> + Addr; + {error, Reason} -> + exit({badarg, {host, Reason}}) + end; + (IsInet6 and not IsInet) -> + case inet:getaddr(Host, inet6) of + {ok, Addr} -> + Addr; + {error, Reason} -> + exit({badarg, {host, Reason}}) + end; + true -> + %% Conflicting options + exit({badarg, {udp, [inet]}}) + end, + UdpOptions2 = lists:reverse(UdpOptions), + TftpOptions = lists:reverse(UserOptions), + Callbacks2 = add_default_callbacks(Callbacks), + Config#config{udp_host = Host2, + udp_options = UdpOptions2, + user_options = TftpOptions, + callbacks = Callbacks2}; +do_parse_config(Options, Config) when is_record(Config, config) -> + exit({badarg, Options}). + +add_default_callbacks(Callbacks) -> + RegExp = "", + {ok, Internal} = inets_regexp:parse(RegExp), + File = #callback{regexp = RegExp, + internal = Internal, + module = tftp_file, + state = []}, + Bin = #callback{regexp = RegExp, + internal = Internal, + module = tftp_binary, + state = []}, + Callbacks ++ [File, Bin]. + +host_to_string(Host) -> + case Host of + String when is_list(String) -> + String; + {A1, A2, A3, A4} -> % inet + lists:concat([A1, ".", A2, ".", A3, ".",A4]); + {A1, A2, A3, A4, A5, A6, A7, A8} -> % inet6 + lists:concat([ + int16_to_hex(A1), "::", + int16_to_hex(A2), "::", + int16_to_hex(A3), "::", + int16_to_hex(A4), "::", + int16_to_hex(A5), "::", + int16_to_hex(A6), "::", + int16_to_hex(A7), "::", + int16_to_hex(A8) + ]) + end. + +int16_to_hex(0) -> + [$0]; +int16_to_hex(I) -> + N1 = ((I bsr 8) band 16#ff), + N2 = (I band 16#ff), + [code_character(N1 div 16), code_character(N1 rem 16), + code_character(N2 div 16), code_character(N2 rem 16)]. + +code_character(N) when N < 10 -> + $0 + N; +code_character(N) -> + $A + (N - 10). + +%%------------------------------------------------------------------- +%% Decode +%%------------------------------------------------------------------- + +decode_msg(Bin) when is_binary(Bin) -> + case Bin of + <<?TFTP_OPCODE_RRQ:16/integer, Tail/binary>> -> + case decode_strings(Tail, [keep_case, lower_case]) of + [Filename, Mode | Strings] -> + Options = decode_options(Strings), + #tftp_msg_req{access = read, + filename = Filename, + mode = to_lower(Mode), + options = Options}; + [_Filename | _Strings] -> + exit(#tftp_msg_error{code = undef, + text = "Missing mode"}); + _ -> + exit(#tftp_msg_error{code = undef, + text = "Missing filename"}) + end; + <<?TFTP_OPCODE_WRQ:16/integer, Tail/binary>> -> + case decode_strings(Tail, [keep_case, lower_case]) of + [Filename, Mode | Strings] -> + Options = decode_options(Strings), + #tftp_msg_req{access = write, + filename = Filename, + mode = to_lower(Mode), + options = Options}; + [_Filename | _Strings] -> + exit(#tftp_msg_error{code = undef, + text = "Missing mode"}); + _ -> + exit(#tftp_msg_error{code = undef, + text = "Missing filename"}) + end; + <<?TFTP_OPCODE_DATA:16/integer, SeqNo:16/integer, Data/binary>> -> + #tftp_msg_data{block_no = SeqNo, data = Data}; + <<?TFTP_OPCODE_ACK:16/integer, SeqNo:16/integer>> -> + #tftp_msg_ack{block_no = SeqNo}; + <<?TFTP_OPCODE_ERROR:16/integer, ErrorCode:16/integer, Tail/binary>> -> + case decode_strings(Tail, [keep_case]) of + [ErrorText] -> + ErrorCode2 = decode_error_code(ErrorCode), + #tftp_msg_error{code = ErrorCode2, + text = ErrorText}; + _ -> + exit(#tftp_msg_error{code = undef, + text = "Trailing garbage"}) + end; + <<?TFTP_OPCODE_OACK:16/integer, Tail/binary>> -> + Strings = decode_strings(Tail, [lower_case]), + Options = decode_options(Strings), + #tftp_msg_oack{options = Options}; + _ -> + exit(#tftp_msg_error{code = undef, + text = "Invalid syntax"}) + end. + +decode_strings(Bin, Cases) when is_binary(Bin), is_list(Cases) -> + do_decode_strings(Bin, Cases, []). + +do_decode_strings(<<>>, _Cases, Strings) -> + lists:reverse(Strings); +do_decode_strings(Bin, [Case | Cases], Strings) -> + {String, Tail} = decode_string(Bin, Case, []), + if + Cases =:= [] -> + do_decode_strings(Tail, [Case], [String | Strings]); + true -> + do_decode_strings(Tail, Cases, [String | Strings]) + end. + +decode_string(<<Char:8/integer, Tail/binary>>, Case, String) -> + if + Char =:= 0 -> + {lists:reverse(String), Tail}; + Case =:= keep_case -> + decode_string(Tail, Case, [Char | String]); + Case =:= lower_case -> + Char2 = ?LOWER(Char), + decode_string(Tail, Case, [Char2 | String]) + end; +decode_string(<<>>, _Case, _String) -> + exit(#tftp_msg_error{code = undef, text = "Trailing null missing"}). + +decode_options([Key, Value | Strings]) -> + [{to_lower(Key), Value} | decode_options(Strings)]; +decode_options([]) -> + []. + +decode_error_code(Int) -> + case Int of + ?TFTP_ERROR_UNDEF -> undef; + ?TFTP_ERROR_ENOENT -> enoent; + ?TFTP_ERROR_EACCES -> eacces; + ?TFTP_ERROR_ENOSPC -> enospc; + ?TFTP_ERROR_BADOP -> badop; + ?TFTP_ERROR_BADBLK -> badblk; + ?TFTP_ERROR_EEXIST -> eexist; + ?TFTP_ERROR_BADUSER -> baduser; + ?TFTP_ERROR_BADOPT -> badopt; + Int when is_integer(Int), Int >= 0, Int =< 65535 -> Int; + _ -> exit(#tftp_msg_error{code = undef, text = "Error code outside range."}) + end. + +%%------------------------------------------------------------------- +%% Encode +%%------------------------------------------------------------------- + +encode_msg(#tftp_msg_req{access = Access, + filename = Filename, + mode = Mode, + options = Options}) -> + OpCode = case Access of + read -> ?TFTP_OPCODE_RRQ; + write -> ?TFTP_OPCODE_WRQ + end, + [ + <<OpCode:16/integer>>, + Filename, + 0, + Mode, + 0, + [[Key, 0, Val, 0] || {Key, Val} <- Options] + ]; +encode_msg(#tftp_msg_data{block_no = BlockNo, data = Data}) when BlockNo =< 65535 -> + [ + <<?TFTP_OPCODE_DATA:16/integer, BlockNo:16/integer>>, + Data + ]; +encode_msg(#tftp_msg_ack{block_no = BlockNo}) when BlockNo =< 65535 -> + <<?TFTP_OPCODE_ACK:16/integer, BlockNo:16/integer>>; +encode_msg(#tftp_msg_error{code = Code, text = Text}) -> + IntCode = encode_error_code(Code), + [ + <<?TFTP_OPCODE_ERROR:16/integer, IntCode:16/integer>>, + Text, + 0 + ]; +encode_msg(#tftp_msg_oack{options = Options}) -> + [ + <<?TFTP_OPCODE_OACK:16/integer>>, + [[Key, 0, Val, 0] || {Key, Val} <- Options] + ]. + +encode_error_code(Code) -> + case Code of + undef -> ?TFTP_ERROR_UNDEF; + enoent -> ?TFTP_ERROR_ENOENT; + eacces -> ?TFTP_ERROR_EACCES; + enospc -> ?TFTP_ERROR_ENOSPC; + badop -> ?TFTP_ERROR_BADOP; + badblk -> ?TFTP_ERROR_BADBLK; + eexist -> ?TFTP_ERROR_EEXIST; + baduser -> ?TFTP_ERROR_BADUSER; + badopt -> ?TFTP_ERROR_BADOPT; + Int when is_integer(Int), Int >= 0, Int =< 65535 -> Int + end. + +%%------------------------------------------------------------------- +%% Miscellaneous +%%------------------------------------------------------------------- + +replace_val(Key, Val, List) -> + case lists:keysearch(Key, 1, List) of + false -> + List ++ [{Key, Val}]; + {value, {_, OldVal}} when OldVal =:= Val -> + List; + {value, {_, _}} -> + lists:keyreplace(Key, 1, List, {Key, Val}) + end. + +to_lower(Chars) -> + [?LOWER(Char) || Char <- Chars]. diff --git a/lib/inets/src/tftp/tftp_logger.erl b/lib/inets/src/tftp/tftp_logger.erl new file mode 100644 index 0000000000..0c3620e665 --- /dev/null +++ b/lib/inets/src/tftp/tftp_logger.erl @@ -0,0 +1,98 @@ +%% +%% %CopyrightBegin% +%% +%% Copyright Ericsson AB 2008-2009. All Rights Reserved. +%% +%% The contents of this file are subject to the Erlang Public License, +%% Version 1.1, (the "License"); you may not use this file except in +%% compliance with the License. You should have received a copy of the +%% Erlang Public License along with this software. If not, it can be +%% retrieved online at http://www.erlang.org/. +%% +%% Software distributed under the License is distributed on an "AS IS" +%% basis, WITHOUT WARRANTY OF ANY KIND, either express or implied. See +%% the License for the specific language governing rights and limitations +%% under the License. +%% +%% %CopyrightEnd% +%% +%% +-module(tftp_logger). + +%%------------------------------------------------------------------- +%% Interface +%%------------------------------------------------------------------- + +%% public functions +-export([ + error_msg/2, + warning_msg/2, + info_msg/2 + ]). + +-export([behaviour_info/1]). + +behaviour_info(callbacks) -> + [{error_msg, 2}, {warning_msg, 2}, {info_msg, 2}]; +behaviour_info(_) -> + undefined. + +%%------------------------------------------------------------------- +%% error_msg(Format, Data) -> ok | exit(Reason) +%% +%% Format = string() +%% Data = [term()] +%% Reason = term() +%% +%% Log an error message +%%------------------------------------------------------------------- + +error_msg(Format, Data) -> + {Format2, Data2} = add_timestamp(Format, Data), + error_logger:error_msg(Format2, Data2). + +%%------------------------------------------------------------------- +%% warning_msg(Format, Data) -> ok | exit(Reason) +%% +%% Format = string() +%% Data = [term()] +%% Reason = term() +%% +%% Log a warning message +%%------------------------------------------------------------------- + +warning_msg(Format, Data) -> + {Format2, Data2} = add_timestamp(Format, Data), + error_logger:warning_msg(Format2, Data2). + +%%------------------------------------------------------------------- +%% info_msg(Format, Data) -> ok | exit(Reason) +%% +%% Format = string() +%% Data = [term()] +%% Reason = term() +%% +%% Log an info message +%%------------------------------------------------------------------- + +info_msg(Format, Data) -> + {Format2, Data2} = add_timestamp(Format, Data), + io:format(Format2, Data2). + +%%------------------------------------------------------------------- +%% Add timestamp to log message +%%------------------------------------------------------------------- + +add_timestamp(Format, Data) -> + Now = {_MegaSecs, _Secs, _MicroSecs} = erlang:now(), + {{_Y, _Mo, _D}, {H, Mi, S}} = calendar:now_to_universal_time(Now), + %% {"~p-~s-~sT~s:~s:~sZ,~6.6.0w tftp: " ++ Format ++ "\n", + %% [Y, t(Mo), t(D), t(H), t(Mi), t(S), MicroSecs | Data]}. + {"~s:~s:~s tftp: " ++ Format, [t(H), t(Mi), t(S) | Data]}. + +%% Convert 9 to "09". +t(Int) -> + case integer_to_list(Int) of + [Single] -> [$0, Single]; + Multi -> Multi + end. diff --git a/lib/inets/src/tftp/tftp_sup.erl b/lib/inets/src/tftp/tftp_sup.erl new file mode 100644 index 0000000000..1cafcc1069 --- /dev/null +++ b/lib/inets/src/tftp/tftp_sup.erl @@ -0,0 +1,110 @@ +%% +%% %CopyrightBegin% +%% +%% Copyright Ericsson AB 2005-2009. All Rights Reserved. +%% +%% The contents of this file are subject to the Erlang Public License, +%% Version 1.1, (the "License"); you may not use this file except in +%% compliance with the License. You should have received a copy of the +%% Erlang Public License along with this software. If not, it can be +%% retrieved online at http://www.erlang.org/. +%% +%% Software distributed under the License is distributed on an "AS IS" +%% basis, WITHOUT WARRANTY OF ANY KIND, either express or implied. See +%% the License for the specific language governing rights and limitations +%% under the License. +%% +%% %CopyrightEnd% +%% +%% +%%---------------------------------------------------------------------- +%% Purpose: The top supervisor for tftp hangs under inets_sup. +%%---------------------------------------------------------------------- + +-module(tftp_sup). + +-behaviour(supervisor). + +%% API +-export([start_link/1, + start_child/1, + stop_child/1, + which_children/0]). + +%% Supervisor callback +-export([init/1]). + +%%%========================================================================= +%%% API +%%%========================================================================= + +start_link(TftpServices) -> + supervisor:start_link({local, ?MODULE}, ?MODULE, [TftpServices]). + +start_child(Options) -> + KillAfter = default_kill_after(), + ChildSpec = worker_spec(KillAfter, Options), + supervisor:start_child(?MODULE, ChildSpec). + +stop_child(Pid) when is_pid(Pid) -> + Children = supervisor:which_children(?MODULE), + case [Id || {Id, P, _Type, _Modules} <- Children, P =:= Pid] of + [] -> + {error, not_found}; + [Id] -> + case supervisor:terminate_child(?MODULE, Id) of + ok -> + supervisor:delete_child(?MODULE, Id); + {error, not_found} -> + supervisor:delete_child(?MODULE, Id); + {error, Reason} -> + {error, Reason} + end + end. + +which_children() -> + Children = supervisor:which_children(?MODULE), + [{tftpd, Pid} || {_Id, Pid, _Type, _Modules} <- Children, Pid =/= undefined]. + +%%%========================================================================= +%%% Supervisor callback +%%%========================================================================= + +init([Services]) when is_list(Services) -> + RestartStrategy = one_for_one, + MaxR = 10, + MaxT = 3600, + KillAfter = default_kill_after(), + Children = [worker_spec(KillAfter, Options) || {tftpd, Options} <- Services], + {ok, {{RestartStrategy, MaxR, MaxT}, Children}}. + +%%%========================================================================= +%%% Internal functions +%%%========================================================================= + +worker_spec(KillAfter, Options) -> + Modules = [proc_lib, tftp, tftp_engine], + KA = supervisor_timeout(KillAfter), + Name = unique_name(Options), + {Name, {tftp, start, [Options]}, permanent, KA, worker, Modules}. + +unique_name(Options) -> + case lists:keysearch(port, 1, Options) of + {value, {_, Port}} when is_integer(Port), Port > 0 -> + {tftpd, Port}; + _ -> + {tftpd, erlang:now()} + end. + +default_kill_after() -> + timer:seconds(3). + +%% supervisor_spec(Name) -> +%% {Name, {Name, start, []}, permanent, infinity, supervisor, +%% [Name, supervisor]}. + +-ifdef(debug_shutdown). +supervisor_timeout(_KillAfter) -> timer:hours(24). +-else. +supervisor_timeout(KillAfter) -> KillAfter. +-endif. diff --git a/lib/inets/vsn.mk b/lib/inets/vsn.mk new file mode 100644 index 0000000000..f80524e7e7 --- /dev/null +++ b/lib/inets/vsn.mk @@ -0,0 +1,81 @@ +#-*-makefile-*- ; force emacs to enter makefile-mode + +# %CopyrightBegin% +# +# Copyright Ericsson AB 1997-2009. All Rights Reserved. +# +# The contents of this file are subject to the Erlang Public License, +# Version 1.1, (the "License"); you may not use this file except in +# compliance with the License. You should have received a copy of the +# Erlang Public License along with this software. If not, it can be +# retrieved online at http://www.erlang.org/. +# +# Software distributed under the License is distributed on an "AS IS" +# basis, WITHOUT WARRANTY OF ANY KIND, either express or implied. See +# the License for the specific language governing rights and limitations +# under the License. +# +# %CopyrightEnd% + +INETS_VSN = 5.2 +PRE_VSN = +APP_VSN = "inets-$(INETS_VSN)$(PRE_VSN)" + +TICKETS = OTP-8204 OTP-8206 OTP-8247 OTP-8248 OTP-8249 OTP-8258 OTP-8280 + +TICKETS_5_1_3 = OTP-8154 + +TICKETS_5_1_2 = OTP-7298 OTP-8101 OTP-8118 + +TICKETS_5_1_1 = OTP-8052 OTP-8069 + +TICKETS_5_1 = OTP-7994 OTP-7998 OTP-8001 OTP-8004 OTP-8005 + +TICKETS_5_0_14 = OTP-7882 OTP-7883 OTP-7888 OTP-7950 OTP-7976 + +TICKETS_5.0.13 = \ + OTP-7723 \ + OTP-7724 \ + OTP-7726 \ + OTP-7463 \ + OTP-7815 \ + OTP-7857 + +# TICKETS_5.0.12 = \ +# OTP-7636 +# +# TICKETS_5.0.11 = \ +# OTP-7574 \ +# OTP-7597 \ +# OTP-7598 \ +# OTP-7605 +# +# TICKETS_5.0.10 = \ +# OTP-7450 \ +# OTP-7454 \ +# OTP-7490 \ +# OTP-7512 +# +# TICKETS_5.0.9 = \ +# OTP-7257 \ +# OTP-7323 \ +# OTP-7341 +# +# TICKETS_5.0.8 = \ +# OTP-7315 \ +# OTP-7321 +# +# TICKETS_5.0.7 = \ +# OTP-7304 +# +# TICKETS_5.0.6 = \ +# OTP-7266 +# +# TICKETS_5.0.5 = \ +# OTP-7220 \ +# OTP-7221 +# +# TICKETS_5.0.4 = \ +# OTP-7173 +# + |
