path: root/lib/stdlib/doc/src/uri_string.xml

<?xml version="1.0" encoding="utf-8" ?>
<!DOCTYPE erlref SYSTEM "erlref.dtd">

<erlref>
  <header>
    <copyright>
      <year>2017</year><year>2017</year>
      <holder>Ericsson AB. All Rights Reserved.</holder>
    </copyright>
    <legalnotice>
      Licensed under the Apache License, Version 2.0 (the "License");
      you may not use this file except in compliance with the License.
      You may obtain a copy of the License at

          http://www.apache.org/licenses/LICENSE-2.0

      Unless required by applicable law or agreed to in writing, software
      distributed under the License is distributed on an "AS IS" BASIS,
      WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
      See the License for the specific language governing permissions and
      limitations under the License.
    </legalnotice>

    <title>uri_string</title>
    <prepared>Péter Dimitrov</prepared>
    <docno>1</docno>
    <date>2017-10-24</date>
    <rev>A</rev>
  </header>
  <module>uri_string</module>
  <modulesummary>URI processing functions.</modulesummary>
  <description>
    <p>This module contains functions for parsing and handling URIs
    (<url href="https://www.ietf.org/rfc/rfc3986.txt">RFC 3986</url>) and
    form-urlencoded query strings (<url href="https://www.ietf.org/rfc/rfc1866.txt">RFC 1866</url>).
    </p>
    <p>A URI is an identifier consisting of a sequence of characters   matching the syntax
    rule named <em>URI</em> in <url href="https://www.ietf.org/rfc/rfc3986.txt">RFC 3986</url>.
    </p>
    <p>  The generic URI syntax consists of a hierarchical sequence of components referred
    to as the scheme, authority, path, query, and fragment:</p>
    <pre>
    URI         = scheme ":" hier-part [ "?" query ] [ "#" fragment ]
    hier-part   = "//" authority path-abempty
                   / path-absolute
                   / path-rootless
                   / path-empty
    scheme      = ALPHA *( ALPHA / DIGIT / "+" / "-" / "." )
    authority   = [ userinfo "@" ] host [ ":" port ]
    userinfo    = *( unreserved / pct-encoded / sub-delims / ":" )

    reserved    = gen-delims / sub-delims
    gen-delims  = ":" / "/" / "?" / "#" / "[" / "]" / "@"
    sub-delims  = "!" / "$" / "&amp;" / "'" / "(" / ")"
                / "*" / "+" / "," / ";" / "="

    unreserved  = ALPHA / DIGIT / "-" / "." / "_" / "~"
    </pre><br></br>
    <p>The interpretation of a URI depends only on the characters used and not on how those
    characters are represented in a network protocol.</p>
    <p>The functions implemented by this module cover the following use cases:</p>
    <list type="bulleted">
      <item>Parsing URIs into its components and returing a map<br></br>
      <seealso marker="#parse/1"><c>parse/1</c></seealso>
      </item>
      <item>Recomposing a map of URI components into a URI string<br></br>
      <seealso marker="#recompose/1"><c>recompose/1</c></seealso>
      </item>
      <item>Changing inbound binary and percent-encoding of URIs<br></br>
      <seealso marker="#transcode/2"><c>transcode/2</c></seealso>
      </item>
      <item>Composing form-urlencoded query strings from a list of key-value pairs<br></br>
      <seealso marker="#compose_query/1"><c>compose_query/1</c></seealso><br></br>
      <seealso marker="#compose_query/2"><c>compose_query/2</c></seealso>
      </item>
      <item>Dissecting form-urlencoded query strings into a list of key-value pairs<br></br>
      <seealso marker="#dissect_query/1"><c>dissect_query/1</c></seealso>
      </item>
    </list>
    <p>There are four different encodings present during the handling of URIs:</p>
    <list type="bulleted">
      <item>Inbound binary encoding in binaries</item>
      <item>Inbound percent-encoding in lists and binaries</item>
      <item>Outbound binary encoding in binaries</item>
      <item>Outbound percent-encoding in lists and binaries</item>
    </list>
    <p>Unless otherwise specified the return value type and encoding are the same as the input
    type and encoding. That is, binary input returns binary output, list input returns a list
    output but mixed input returns list output.</p>
    <p>All of the functions but <c>transcode/2</c> expects input as unicode codepoints in
    lists, UTF-8 encoding in binaries and UTF-8 encoding in percent-encoded URI parts.
    <c>transcode/2</c> provides the means to convert between the supported URI encodings.</p>
  </description>

  <datatypes>
    <datatype>
      <name name="error"/>
      <desc>
        <p>Error tuple indicating the type of error. Possible values of the second component:</p>
	    <list type="bulleted">
	      <item><c>invalid_character</c></item>
	      <item><c>invalid_input</c></item>
	      <item><c>invalid_map</c></item>
	      <item><c>invalid_percent_encoding</c></item>
	      <item><c>invalid_scheme</c></item>
	      <item><c>invalid_uri</c></item>
	      <item><c>invalid_utf8</c></item>
	      <item><c>missing_value</c></item>
	    </list>
      </desc>
    </datatype>
    <datatype>
      <name name="uri_map"/>
      <desc>
        <p>URI map holding the main components of a URI.</p>
      </desc>
    </datatype>
    <datatype>
      <name name="uri_string"/>
      <desc>
        <p>List of unicode codepoints, UTF-8 encoded binary, or a mix of the two,
        representing an <url href="https://www.ietf.org/rfc/rfc3986.txt">RFC 3986</url>
	compliant URI (<em>percent-encoded form</em>).
        A URI is a sequence of characters from a very limited set: the letters of
        the basic Latin alphabet, digits, and a few special characters.</p>
      </desc>
    </datatype>
  </datatypes>

  <funcs>

    <func>
      <name name="compose_query" arity="1"/>
      <fsummary>Compose urlencoded query string.</fsummary>
      <desc>
        <p>Composes a form-urlencoded <c><anno>QueryString</anno></c> based on a
	<c><anno>QueryList</anno></c>, a list of unescaped key-value pairs.
	Media type <c>application/x-www-form-urlencoded</c> is defined in section
	8.2.1 of <url href="https://www.ietf.org/rfc/rfc1866.txt">RFC 1866</url>
	(HTML 2.0). Reserved and unsafe characters, as
	defined by <url href="https://www.ietf.org/rfc/rfc1738.txt">RFC 1738</url>
	(Uniform Resource Locators), are percent-encoded.</p>
	<p>See also the opposite operation <seealso marker="#dissect_query/1">
	<c>dissect_query/1</c></seealso>.
	</p>
        <p><em>Example:</em></p>
        <pre>
1> <input>uri_string:compose_query([{"foo bar","1"},{"city","örebro"}],</input>
1> [{separator, semicolon}]).
"foo+bar=1;city=%C3%B6rebro"
2> <![CDATA[uri_string:compose_query([{<<"foo bar">>,<<"1">>},
2> {<<"city">>,<<"örebro"/utf8>>}]).]]>
<![CDATA[<<"foo+bar=1&amp;city=%C3%B6rebro">>]]>
	</pre>
      </desc>
    </func>

    <func>
      <name name="compose_query" arity="2"/>
      <fsummary>Compose urlencoded query string.</fsummary>
      <desc>
        <p>Same as <c>compose_query/1</c> but with an additional
	<c><anno>Options</anno></c> parameter, that controls the type of separator used
	between key-value pairs. There are three supported separator types: <c>amp</c> (<![CDATA[&]]>), <c>escaped_amp</c> (<![CDATA[&amp;]]>) and <c>semicolon</c> (;). If the parameter <c><anno>Options</anno></c> is empty, separator takes the default value (<c>escaped_amp</c>).</p>
	<p>See also the opposite operation <seealso marker="#dissect_query/1">
	<c>dissect_query/1</c></seealso>.
	</p>
        <p><em>Example:</em></p>
        <pre>
1> <input>uri_string:compose_query([{"foo bar","1"},{"city","örebro"}],</input>
1> [{separator, amp}]).
<![CDATA["foo+bar=1&city=%C3%B6rebro"]]>
	</pre>
      </desc>
    </func>

    <func>
      <name name="dissect_query" arity="1"/>
      <fsummary>Dissect query string.</fsummary>
      <desc>
        <p>Dissects an urlencoded <c><anno>QueryString</anno></c> and returns a
	<c><anno>QueryList</anno></c>, a list of unescaped key-value pairs.
	Media type <c>application/x-www-form-urlencoded</c> is defined in section
	8.2.1 of <url href="https://www.ietf.org/rfc/rfc1866.txt">RFC 1866</url>
	(HTML 2.0). Percent-encoded segments are decoded
	as defined by <url href="https://www.ietf.org/rfc/rfc1738.txt">RFC 1738</url>
	(Uniform Resource Locators).</p>
	<p>See also the opposite operation <seealso marker="#compose_query/1">
	<c>compose_query/1</c></seealso>.
	</p>
        <p><em>Example:</em></p>
        <pre>
1> <input>uri_string:dissect_query("foo+bar=1;city=%C3%B6rebro").</input>
[{"foo bar","1"},{"city","örebro"}]
2> <![CDATA[uri_string:dissect_query(<<"foo+bar=1;city=%C3%B6rebro">>).]]>
<![CDATA[[{<<"foo bar">>,<<"1">>},{<<"city">>,<<"örebro"/utf8>>}] ]]>
	</pre>
      </desc>
    </func>

    <func>
      <name name="parse" arity="1"/>
      <fsummary>Parse URI into a map.</fsummary>
      <desc>
        <p>Returns a <c>URIMap</c>, that is a <em>uri_map()</em> with the parsed components
	of the <c><anno>URIString</anno></c>. If parsing fails, an error tuple is returned.</p>
	<p>See also the opposite operation <seealso marker="#recompose/1">
	<c>recompose/1</c></seealso>.</p>
        <p><em>Example:</em></p>
        <pre>
1> <input>uri_string:parse("foo://[email protected]:8042/over/there?name=ferret#nose").</input>
#{fragment => "nose",host => "example.com",
  path => "/over/there",port => 8042,query => "name=ferret",
  scheme => foo,userinfo => "user"}
2> <![CDATA[uri_string:parse(<<"foo://[email protected]:8042/over/there?name=ferret">>).]]>
<![CDATA[#{host => <<"example.com">>,path => <<"/over/there">>,
  port => 8042,query => <<"name=ferret">>,scheme => <<"foo">>,
  userinfo => <<"user">>}]]>
	</pre>
      </desc>
    </func>

    <func>
      <name name="recompose" arity="1"/>
      <fsummary>Recompose URI.</fsummary>
      <desc>
        <p>Returns an <url href="https://www.ietf.org/rfc/rfc3986.txt">RFC 3986</url> compliant
	<c><anno>URIString</anno></c> (percent-encoded).
	If the <c><anno>URIMap</anno></c> is invalid, an error tuple is returned.</p>
	<p>See also the opposite operation <seealso marker="#parse/1">
	<c>parse/1</c></seealso>.</p>
        <p><em>Example:</em></p>
        <pre>
1> <input>URIMap = #{fragment => "nose", host => "example.com", path => "/over/there",</input>
1> port => 8042, query => "name=ferret", scheme => "foo", userinfo => "user"}.
#{fragment => "top",host => "example.com",
  path => "/over/there",port => 8042,query => "?name=ferret",
  scheme => foo,userinfo => "user"}

2> <input>uri_string:recompose(URIMap).</input>
"foo://example.com:8042/over/there?name=ferret#nose"</pre>
      </desc>
    </func>

    <func>
      <name name="transcode" arity="2"/>
      <fsummary>Transcode URI.</fsummary>
      <desc>
        <p>Transcodes an <url href="https://www.ietf.org/rfc/rfc3986.txt">RFC 3986</url>
	compliant <c><anno>URIString</anno></c>,
	where <c><anno>Options</anno></c> is a list of tagged tuples, specifying the inbound
	(<c>in_encoding</c>) and outbound (<c>out_encoding</c>) encodings.
	If an argument is invalid, an error tuple is returned.</p>
        <p><em>Example:</em></p>
        <pre>
1> <input><![CDATA[uri_string:transcode(<<"foo%00%00%00%F6bar"/utf32>>,]]></input>
1> [{in_encoding, utf32},{out_encoding, utf8}]).
<![CDATA[<<"foo%C3%B6bar"/utf8>>]]>
	</pre>
      </desc>
    </func>

  </funcs>
</erlref>