aboutsummaryrefslogtreecommitdiffstats
path: root/lib/stdlib
diff options
context:
space:
mode:
Diffstat (limited to 'lib/stdlib')
-rw-r--r--lib/stdlib/doc/src/uri_string.xml115
1 files changed, 83 insertions, 32 deletions
diff --git a/lib/stdlib/doc/src/uri_string.xml b/lib/stdlib/doc/src/uri_string.xml
index 8322eecb24..55d8690b98 100644
--- a/lib/stdlib/doc/src/uri_string.xml
+++ b/lib/stdlib/doc/src/uri_string.xml
@@ -69,6 +69,9 @@
<item>Changing inbound binary and percent-encoding of URIs<br></br>
<seealso marker="#transcode/2"><c>transcode/2</c></seealso>
</item>
+ <item>Transforming URIs into a normalized form<br></br>
+ <seealso marker="#normalize/1"><c>normalize/1</c></seealso>
+ </item>
<item>Composing form-urlencoded query strings from a list of key-value pairs<br></br>
<seealso marker="#compose_query/1"><c>compose_query/1</c></seealso><br></br>
<seealso marker="#compose_query/2"><c>compose_query/2</c></seealso>
@@ -84,12 +87,21 @@
<item>Outbound binary encoding in binaries</item>
<item>Outbound percent-encoding in lists and binaries</item>
</list>
+ <p>Functions with <c>uri_string()</c> argument accept lists, binaries and
+ mixed lists (lists with binary elements) as input type. All of the functions but
+ <c>transcode/2</c> expects input as lists of unicode codepoints, UTF-8 encoded binaries
+ and UTF-8 percent-encoded URI parts ("%C3%B6" corresponds to the unicode character "ö").</p>
<p>Unless otherwise specified the return value type and encoding are the same as the input
type and encoding. That is, binary input returns binary output, list input returns a list
output but mixed input returns list output.</p>
- <p>All of the functions but <c>transcode/2</c> expects input as unicode codepoints in
- lists, UTF-8 encoding in binaries and UTF-8 encoding in percent-encoded URI parts.
- <c>transcode/2</c> provides the means to convert between the supported URI encodings.</p>
+ <p>In case of lists there is only percent-encoding. In binaries, however, both binary encoding
+ and percent-encoding shall be considered. <c>transcode/2</c> provides the means to convert
+ between the supported encodings, it takes a <c>uri_string()</c> and a list of options
+ specifying inbound and outbound encodings.</p>
+ <p><url href="https://www.ietf.org/rfc/rfc3986.txt">RFC 3986</url> does not mandate any specific
+ character encoding and it is usually defined by the protocol or surrounding text. This library
+ takes the same assumption, binary and percent-encoding are handled as one configuration unit,
+ they cannot be set to different values.</p>
</description>
<datatypes>
@@ -97,28 +109,30 @@
<name name="error"/>
<desc>
<p>Error tuple indicating the type of error. Possible values of the second component:</p>
- <list type="bulleted">
- <item><c>invalid_character</c></item>
- <item><c>invalid_input</c></item>
- <item><c>invalid_map</c></item>
- <item><c>invalid_percent_encoding</c></item>
- <item><c>invalid_scheme</c></item>
- <item><c>invalid_uri</c></item>
- <item><c>invalid_utf8</c></item>
- <item><c>missing_value</c></item>
- </list>
+ <list type="bulleted">
+ <item><c>invalid_character</c></item>
+ <item><c>invalid_input</c></item>
+ <item><c>invalid_map</c></item>
+ <item><c>invalid_percent_encoding</c></item>
+ <item><c>invalid_scheme</c></item>
+ <item><c>invalid_uri</c></item>
+ <item><c>invalid_utf8</c></item>
+ <item><c>missing_value</c></item>
+ </list>
+ <p>The third component is a list or binary providing additional information about the
+ cause of the error.</p>
</desc>
</datatype>
<datatype>
<name name="uri_map"/>
<desc>
- <p>URI map holding the main components of a URI.</p>
+ <p>Map holding the main components of a URI.</p>
</desc>
</datatype>
<datatype>
<name name="uri_string"/>
<desc>
- <p>List of unicode codepoints, UTF-8 encoded binary, or a mix of the two,
+ <p>List of unicode codepoints, a UTF-8 encoded binary, or a mix of the two,
representing an <url href="https://www.ietf.org/rfc/rfc3986.txt">RFC 3986</url>
compliant URI (<em>percent-encoded form</em>).
A URI is a sequence of characters from a very limited set: the letters of
@@ -134,10 +148,11 @@
<fsummary>Compose urlencoded query string.</fsummary>
<desc>
<p>Composes a form-urlencoded <c><anno>QueryString</anno></c> based on a
- <c><anno>QueryList</anno></c>, a list of unescaped key-value pairs.
- Media type <c>application/x-www-form-urlencoded</c> is defined in section
+ <c><anno>QueryList</anno></c>, a list of non-percent-encoded key-value pairs.
+ Form-urlencoding is defined in section
8.2.1 of <url href="https://www.ietf.org/rfc/rfc1866.txt">RFC 1866</url>
- (HTML 2.0). Reserved and unsafe characters, as
+ (HTML 2.0) for media type <c>application/x-www-form-urlencoded</c>.
+ Reserved and unsafe characters, as
defined by <url href="https://www.ietf.org/rfc/rfc1738.txt">RFC 1738</url>
(Uniform Resource Locators), are percent-encoded.</p>
<p>See also the opposite operation <seealso marker="#dissect_query/1">
@@ -145,9 +160,8 @@
</p>
<p><em>Example:</em></p>
<pre>
-1> <input>uri_string:compose_query([{"foo bar","1"},{"city","örebro"}],</input>
-1> [{separator, semicolon}]).
-"foo+bar=1;city=%C3%B6rebro"
+1> <input>uri_string:compose_query([{"foo bar","1"},{"city","örebro"}]).</input>
+<![CDATA["foo+bar=1&amp;city=%C3%B6rebro"]]>
2> <![CDATA[uri_string:compose_query([{<<"foo bar">>,<<"1">>},
2> {<<"city">>,<<"örebro"/utf8>>}]).]]>
<![CDATA[<<"foo+bar=1&amp;city=%C3%B6rebro">>]]>
@@ -169,7 +183,10 @@
<pre>
1> <input>uri_string:compose_query([{"foo bar","1"},{"city","örebro"}],</input>
1> [{separator, amp}]).
-<![CDATA["foo+bar=1&city=%C3%B6rebro"]]>
+<![CDATA["foo+bar=1&city=%C3%B6rebro"
+2> uri_string:compose_query([{<<"foo bar">>,<<"1">>},
+2> {<<"city">>,<<"örebro"/utf8>>}], [{separator, escaped_amp}]).]]>
+<![CDATA[<<"foo+bar=1&amp;city=%C3%B6rebro">>]]>
</pre>
</desc>
</func>
@@ -179,12 +196,15 @@
<fsummary>Dissect query string.</fsummary>
<desc>
<p>Dissects an urlencoded <c><anno>QueryString</anno></c> and returns a
- <c><anno>QueryList</anno></c>, a list of unescaped key-value pairs.
- Media type <c>application/x-www-form-urlencoded</c> is defined in section
+ <c><anno>QueryList</anno></c>, a list of non-percent-encoded key-value pairs.
+ Form-urlencoding is defined in section
8.2.1 of <url href="https://www.ietf.org/rfc/rfc1866.txt">RFC 1866</url>
- (HTML 2.0). Percent-encoded segments are decoded
- as defined by <url href="https://www.ietf.org/rfc/rfc1738.txt">RFC 1738</url>
+ (HTML 2.0) for media type <c>application/x-www-form-urlencoded</c>.
+ Percent-encoded segments are decoded as defined by
+ <url href="https://www.ietf.org/rfc/rfc1738.txt">RFC 1738</url>
(Uniform Resource Locators).</p>
+ <p>Supported separator types: <c>amp</c> (<![CDATA[&]]>), <c>escaped_amp</c>
+ (<![CDATA[&amp;]]>) and <c>semicolon</c> (;).</p>
<p>See also the opposite operation <seealso marker="#compose_query/1">
<c>compose_query/1</c></seealso>.
</p>
@@ -192,18 +212,42 @@
<pre>
1> <input>uri_string:dissect_query("foo+bar=1;city=%C3%B6rebro").</input>
[{"foo bar","1"},{"city","örebro"}]
-2> <![CDATA[uri_string:dissect_query(<<"foo+bar=1;city=%C3%B6rebro">>).]]>
+2> <![CDATA[uri_string:dissect_query(<<"foo+bar=1&city=%C3%B6rebro">>).]]>
<![CDATA[[{<<"foo bar">>,<<"1">>},{<<"city">>,<<"örebro"/utf8>>}] ]]>
</pre>
</desc>
</func>
<func>
+ <name name="normalize" arity="1"/>
+ <fsummary>Syntax-based normalization.</fsummary>
+ <desc>
+ <p>Transforms <c><anno>URIString</anno></c> into a normalized form
+ using Syntax-Based Normalization as defined by
+ <url href="https://www.ietf.org/rfc/rfc3986.txt">RFC 3986</url>.</p>
+ <p>This function implements case normalization, percent-encoding
+ normalization, path segment normalization and scheme based normalization
+ for HTTP(S) with basic support for FTP, SSH, SFTP and TFTP.</p>
+ <p><em>Example:</em></p>
+ <pre>
+1> <input>uri_string:normalize("/a/b/c/./../../g").</input>
+"/a/g"
+2> <![CDATA[uri_string:normalize(<<"mid/content=5/../6">>).]]>
+<![CDATA[<<"mid/6">>]]>
+3> uri_string:normalize("http://localhost:80").
+"https://localhost/"
+ </pre>
+ </desc>
+ </func>
+
+ <func>
<name name="parse" arity="1"/>
<fsummary>Parse URI into a map.</fsummary>
<desc>
- <p>Returns a <c>URIMap</c>, that is a <em>uri_map()</em> with the parsed components
- of the <c><anno>URIString</anno></c>. If parsing fails, an error tuple is returned.</p>
+ <p>Parses an <url href="https://www.ietf.org/rfc/rfc3986.txt">RFC 3986</url>
+ compliant <c>uri_string()</c> into a <c>uri_map()</c>, that holds the parsed
+ components of the <c>URI</c>.
+ If parsing fails, an error tuple is returned.</p>
<p>See also the opposite operation <seealso marker="#recompose/1">
<c>recompose/1</c></seealso>.</p>
<p><em>Example:</em></p>
@@ -224,8 +268,9 @@
<name name="recompose" arity="1"/>
<fsummary>Recompose URI.</fsummary>
<desc>
- <p>Returns an <url href="https://www.ietf.org/rfc/rfc3986.txt">RFC 3986</url> compliant
- <c><anno>URIString</anno></c> (percent-encoded).
+ <p>Creates an <url href="https://www.ietf.org/rfc/rfc3986.txt">RFC 3986</url> compliant
+ <c><anno>URIString</anno></c> (percent-encoded), based on the components of
+ <c><anno>URIMap</anno></c>.
If the <c><anno>URIMap</anno></c> is invalid, an error tuple is returned.</p>
<p>See also the opposite operation <seealso marker="#parse/1">
<c>parse/1</c></seealso>.</p>
@@ -249,13 +294,19 @@
<p>Transcodes an <url href="https://www.ietf.org/rfc/rfc3986.txt">RFC 3986</url>
compliant <c><anno>URIString</anno></c>,
where <c><anno>Options</anno></c> is a list of tagged tuples, specifying the inbound
- (<c>in_encoding</c>) and outbound (<c>out_encoding</c>) encodings.
+ (<c>in_encoding</c>) and outbound (<c>out_encoding</c>) encodings. <c>in_encoding</c>
+ and <c>out_encoding</c> specifies both binary encoding and percent-encoding for the
+ input and output data. Mixed encoding, where binary encoding is not the same as
+ percent-encoding, is not supported.
If an argument is invalid, an error tuple is returned.</p>
<p><em>Example:</em></p>
<pre>
1> <input><![CDATA[uri_string:transcode(<<"foo%00%00%00%F6bar"/utf32>>,]]></input>
1> [{in_encoding, utf32},{out_encoding, utf8}]).
<![CDATA[<<"foo%C3%B6bar"/utf8>>]]>
+2> uri_string:transcode("foo%F6bar", [{in_encoding, latin1},
+2> {out_encoding, utf8}]).
+"foo%C3%B6bar"
</pre>
</desc>
</func>