aboutsummaryrefslogtreecommitdiffstats
path: root/lib/stdlib/doc/src/erl_scan.xml
diff options
context:
space:
mode:
Diffstat (limited to 'lib/stdlib/doc/src/erl_scan.xml')
-rw-r--r--lib/stdlib/doc/src/erl_scan.xml337
1 files changed, 174 insertions, 163 deletions
diff --git a/lib/stdlib/doc/src/erl_scan.xml b/lib/stdlib/doc/src/erl_scan.xml
index ee0d6b6033..137ccd3416 100644
--- a/lib/stdlib/doc/src/erl_scan.xml
+++ b/lib/stdlib/doc/src/erl_scan.xml
@@ -4,7 +4,7 @@
<erlref>
<header>
<copyright>
- <year>1996</year><year>2015</year>
+ <year>1996</year><year>2016</year>
<holder>Ericsson AB. All Rights Reserved.</holder>
</copyright>
<legalnotice>
@@ -28,16 +28,17 @@
<docno>1</docno>
<approved>Bjarne D&auml;cker</approved>
<checked></checked>
- <date>97-01-24</date>
+ <date>1997-01-24</date>
<rev>B</rev>
- <file>erl_scan.sgml</file>
+ <file>erl_scan.xml</file>
</header>
<module>erl_scan</module>
- <modulesummary>The Erlang Token Scanner</modulesummary>
+ <modulesummary>The Erlang token scanner.</modulesummary>
<description>
- <p>This module contains functions for tokenizing characters into
+ <p>This module contains functions for tokenizing (scanning) characters into
Erlang tokens.</p>
</description>
+
<datatypes>
<datatype>
<name name="category"></name>
@@ -70,23 +71,96 @@
<name name="tokens_result"></name>
</datatype>
</datatypes>
+
<funcs>
<func>
+ <name name="category" arity="1"/>
+ <fsummary>Return the category.</fsummary>
+ <desc>
+ <p>Returns the category of <c><anno>Token</anno></c>.</p>
+ </desc>
+ </func>
+
+ <func>
+ <name name="column" arity="1"/>
+ <fsummary>Return the column.</fsummary>
+ <desc>
+ <p>Returns the column of <c><anno>Token</anno></c>'s
+ collection of annotations.</p>
+ </desc>
+ </func>
+
+ <func>
+ <name name="end_location" arity="1"/>
+ <fsummary>Return the end location of the text.</fsummary>
+ <desc>
+ <p>Returns the end location of the text of
+ <c><anno>Token</anno></c>'s collection of annotations. If
+ there is no text, <c>undefined</c> is returned.</p>
+ </desc>
+ </func>
+
+ <func>
+ <name name="format_error" arity="1"/>
+ <fsummary>Format an error descriptor.</fsummary>
+ <desc>
+ <p>Uses an <c><anno>ErrorDescriptor</anno></c> and returns a string
+ that describes the error or warning. This function is usually
+ called implicitly when an <c>ErrorInfo</c> structure is
+ processed (see section
+ <seealso marker="#errorinfo">Error Information</seealso>).</p>
+ </desc>
+ </func>
+
+ <func>
+ <name name="line" arity="1"/>
+ <fsummary>Return the line.</fsummary>
+ <desc>
+ <p>Returns the line of <c><anno>Token</anno></c>'s collection
+ of annotations.</p>
+ </desc>
+ </func>
+
+ <func>
+ <name name="location" arity="1"/>
+ <fsummary>Return the location.</fsummary>
+ <desc>
+ <p>Returns the location of <c><anno>Token</anno></c>'s
+ collection of annotations.</p>
+ </desc>
+ </func>
+
+ <func>
+ <name name="reserved_word" arity="1"/>
+ <fsummary>Test for a reserved word.</fsummary>
+ <desc>
+ <p>Returns <c>true</c> if <c><anno>Atom</anno></c> is an
+ Erlang reserved word, otherwise <c>false</c>.</p>
+ </desc>
+ </func>
+
+ <func>
<name name="string" arity="1"/>
<name name="string" arity="2"/>
<name name="string" arity="3"/>
- <fsummary>Scan a string and return the Erlang tokens</fsummary>
+ <fsummary>Scan a string and return the Erlang tokens.</fsummary>
<desc>
<p>Takes the list of characters <c><anno>String</anno></c> and tries to
- scan (tokenize) them. Returns <c>{ok, <anno>Tokens</anno>,
- <anno>EndLocation</anno>}</c>,
- where <c><anno>Tokens</anno></c> are the Erlang tokens from
- <c><anno>String</anno></c>. <c><anno>EndLocation</anno></c>
- is the first location after the last token.</p>
- <p><c>{error, <anno>ErrorInfo</anno>, <anno>ErrorLocation</anno>}</c>
- is returned if an error occurs.
- <c><anno>ErrorLocation</anno></c> is the first location after
- the erroneous token.</p>
+ scan (tokenize) them. Returns one of the following:</p>
+ <taglist>
+ <tag><c>{ok, <anno>Tokens</anno>, <anno>EndLocation</anno>}</c></tag>
+ <item>
+ <p><c><anno>Tokens</anno></c> are the Erlang tokens from
+ <c><anno>String</anno></c>. <c><anno>EndLocation</anno></c>
+ is the first location after the last token.</p>
+ </item>
+ <tag><c>{error, <anno>ErrorInfo</anno>,
+ <anno>ErrorLocation</anno>}</c></tag>
+ <item>
+ <p>An error occurred. <c><anno>ErrorLocation</anno></c> is the
+ first location after the erroneous token.</p>
+ </item>
+ </taglist>
<p><c>string(<anno>String</anno>)</c> is equivalent to
<c>string(<anno>String</anno>, 1)</c>, and
<c>string(<anno>String</anno>,
@@ -95,80 +169,102 @@
<anno>StartLocation</anno>, [])</c>.</p>
<p><c><anno>StartLocation</anno></c> indicates the initial location
when scanning starts. If <c><anno>StartLocation</anno></c> is a line,
- <c>Anno</c> as well as <c><anno>EndLocation</anno></c> and
- <c><anno>ErrorLocation</anno></c> will be lines. If
- <c><anno>StartLocation</anno></c> is a pair of a line and a column
+ <c>Anno</c>, <c><anno>EndLocation</anno></c>, and
+ <c><anno>ErrorLocation</anno></c> are lines. If
+ <c><anno>StartLocation</anno></c> is a pair of a line and a column,
<c>Anno</c> takes the form of an opaque compound
data type, and <c><anno>EndLocation</anno></c> and
<c><anno>ErrorLocation</anno></c>
- will be pairs of a line and a column. The <em>token
+ are pairs of a line and a column. The <em>token
annotations</em> contain information about the column and the
line where the token begins, as well as the text of the
- token (if the <c>text</c> option is given), all of which can
+ token (if option <c>text</c> is specified), all of which can
be accessed by calling
- <seealso marker="#column/1">column/1</seealso>,
- <seealso marker="#line/1">line/1</seealso>,
- <seealso marker="#location/1">location/1</seealso>, and
- <seealso marker="#text/1">text/1</seealso>.</p>
+ <seealso marker="#column/1"><c>column/1</c></seealso>,
+ <seealso marker="#line/1"><c>line/1</c></seealso>,
+ <seealso marker="#location/1"><c>location/1</c></seealso>, and
+ <seealso marker="#text/1"><c>text/1</c></seealso>.</p>
<p>A <em>token</em> is a tuple containing information about
- syntactic category, the token annotations, and the actual
- terminal symbol. For punctuation characters (e.g. <c>;</c>,
+ syntactic category, the token annotations, and the
+ terminal symbol. For punctuation characters (such as <c>;</c> and
<c>|</c>) and reserved words, the category and the symbol
coincide, and the token is represented by a two-tuple.
- Three-tuples have one of the following forms: <c>{atom,
- Info, atom()}</c>,
- <c>{char, Info, integer()}</c>, <c>{comment, Info,
- string()}</c>, <c>{float, Info, float()}</c>, <c>{integer,
- Info, integer()}</c>, <c>{var, Info, atom()}</c>,
- and <c>{white_space, Info, string()}</c>.</p>
- <p>The valid options are:</p>
+ Three-tuples have one of the following forms:</p>
+ <list type="bulleted">
+ <item><c>{atom, Anno, atom()}</c></item>
+ <item><c>{char, Anno, char()}</c></item>
+ <item><c>{comment, Anno, string()}</c></item>
+ <item><c>{float, Anno, float()}</c></item>
+ <item><c>{integer, Anno, integer()}</c></item>
+ <item><c>{var, Anno, atom()}</c></item>
+ <item><c>{white_space, Anno, string()}</c></item>
+ </list>
+ <p>Valid options:</p>
<taglist>
- <tag><c>{reserved_word_fun, reserved_word_fun()}</c></tag>
- <item><p>A callback function that is called when the scanner
- has found an unquoted atom. If the function returns
- <c>true</c>, the unquoted atom itself will be the category
- of the token; if the function returns <c>false</c>,
- <c>atom</c> will be the category of the unquoted atom.</p>
- </item>
- <tag><c>return_comments</c></tag>
- <item><p>Return comment tokens.</p>
- </item>
- <tag><c>return_white_spaces</c></tag>
- <item><p>Return white space tokens. By convention, if there is
- a newline character, it is always the first character of the
- text (there cannot be more than one newline in a white space
- token).</p>
- </item>
- <tag><c>return</c></tag>
- <item><p>Short for <c>[return_comments, return_white_spaces]</c>.</p>
- </item>
- <tag><c>text</c></tag>
- <item><p>Include the token's text in the token annotation. The
- text is the part of the input corresponding to the token.</p>
- </item>
+ <tag><c>{reserved_word_fun, reserved_word_fun()}</c></tag>
+ <item><p>A callback function that is called when the scanner
+ has found an unquoted atom. If the function returns
+ <c>true</c>, the unquoted atom itself becomes the category
+ of the token. If the function returns <c>false</c>,
+ <c>atom</c> becomes the category of the unquoted atom.</p>
+ </item>
+ <tag><c>return_comments</c></tag>
+ <item><p>Return comment tokens.</p>
+ </item>
+ <tag><c>return_white_spaces</c></tag>
+ <item><p>Return white space tokens. By convention, a newline
+ character, if present, is always the first character of the
+ text (there cannot be more than one newline in a white space
+ token).</p>
+ </item>
+ <tag><c>return</c></tag>
+ <item><p>Short for <c>[return_comments, return_white_spaces]</c>.</p>
+ </item>
+ <tag><c>text</c></tag>
+ <item><p>Include the token text in the token annotation. The
+ text is the part of the input corresponding to the token.</p>
+ </item>
</taglist>
</desc>
</func>
+
+ <func>
+ <name name="symbol" arity="1"/>
+ <fsummary>Return the symbol.</fsummary>
+ <desc>
+ <p>Returns the symbol of <c><anno>Token</anno></c>.</p>
+ </desc>
+ </func>
+
+ <func>
+ <name name="text" arity="1"/>
+ <fsummary>Return the text.</fsummary>
+ <desc>
+ <p>Returns the text of <c><anno>Token</anno></c>'s collection
+ of annotations. If there is no text, <c>undefined</c> is
+ returned.</p>
+ </desc>
+ </func>
+
<func>
<name name="tokens" arity="3"/>
<name name="tokens" arity="4"/>
- <fsummary>Re-entrant scanner</fsummary>
+ <fsummary>Re-entrant scanner.</fsummary>
<type name="char_spec"/>
<type name="return_cont"/>
- <type_desc name="return_cont">An opaque continuation</type_desc>
+ <type_desc name="return_cont">An opaque continuation.</type_desc>
<desc>
- <p>This is the re-entrant scanner which scans characters until
- a <em>dot</em> ('.' followed by a white space) or
- <c>eof</c> has been reached. It returns:</p>
+ <p>This is the re-entrant scanner, which scans characters until
+ either a <em>dot</em> ('.' followed by a white space) or
+ <c>eof</c> is reached. It returns:</p>
<taglist>
<tag><c>{done, <anno>Result</anno>, <anno>LeftOverChars</anno>}</c>
</tag>
<item>
- <p>This return indicates that there is sufficient input
+ <p>Indicates that there is sufficient input
data to get a result. <c><anno>Result</anno></c> is:</p>
<taglist>
- <tag><c>{ok, Tokens, EndLocation}</c>
- </tag>
+ <tag><c>{ok, Tokens, EndLocation}</c></tag>
<item>
<p>The scanning was successful. <c>Tokens</c>
is the list of tokens including <em>dot</em>.</p>
@@ -177,8 +273,7 @@
<item>
<p>End of file was encountered before any more tokens.</p>
</item>
- <tag><c>{error, ErrorInfo, EndLocation}</c>
- </tag>
+ <tag><c>{error, ErrorInfo, EndLocation}</c></tag>
<item>
<p>An error occurred. <c><anno>LeftOverChars</anno></c>
is the remaining characters of the input data,
@@ -194,110 +289,26 @@
</item>
</taglist>
<p>The <c><anno>CharSpec</anno></c> <c>eof</c> signals end of file.
- <c><anno>LeftOverChars</anno></c> will then take the value <c>eof</c>
+ <c><anno>LeftOverChars</anno></c> then takes the value <c>eof</c>
as well.</p>
<p><c>tokens(<anno>Continuation</anno>, <anno>CharSpec</anno>,
<anno>StartLocation</anno>)</c> is equivalent to
<c>tokens(<anno>Continuation</anno>, <anno>CharSpec</anno>,
<anno>StartLocation</anno>, [])</c>.</p>
- <p>See <seealso marker="#string/3">string/3</seealso> for a
- description of the various options.</p>
- </desc>
- </func>
- <func>
- <name name="reserved_word" arity="1"/>
- <fsummary>Test for a reserved word</fsummary>
- <desc>
- <p>Returns <c>true</c> if <c><anno>Atom</anno></c> is an Erlang
- reserved word, otherwise <c>false</c>.</p>
- </desc>
- </func>
- <func>
- <name name="category" arity="1"/>
- <fsummary>Return the category</fsummary>
- <desc>
- <p>Returns the category of <c><anno>Token</anno></c>.
- </p>
- </desc>
- </func>
- <func>
- <name name="symbol" arity="1"/>
- <fsummary>Return the symbol</fsummary>
- <desc>
- <p>Returns the symbol of <c><anno>Token</anno></c>.
- </p>
- </desc>
- </func>
- <func>
- <name name="column" arity="1"/>
- <fsummary>Return the column</fsummary>
- <desc>
- <p>Returns the column of <c><anno>Token</anno></c>'s
- collection of annotations.
- </p>
- </desc>
- </func>
- <func>
- <name name="end_location" arity="1"/>
- <fsummary>Return the end location of the text</fsummary>
- <desc>
- <p>Returns the end location of the text of
- <c><anno>Token</anno></c>'s collection of annotations. If
- there is no text,
- <c>undefined</c> is returned.
- </p>
- </desc>
- </func>
- <func>
- <name name="line" arity="1"/>
- <fsummary>Return the line</fsummary>
- <desc>
- <p>Returns the line of <c><anno>Token</anno></c>'s collection
- of annotations.
- </p>
- </desc>
- </func>
- <func>
- <name name="location" arity="1"/>
- <fsummary>Return the location</fsummary>
- <desc>
- <p>Returns the location of <c><anno>Token</anno></c>'s
- collection of annotations.
- </p>
- </desc>
- </func>
- <func>
- <name name="text" arity="1"/>
- <fsummary>Return the text</fsummary>
- <desc>
- <p>Returns the text of <c><anno>Token</anno></c>'s collection
- of annotations. If there is no text, <c>undefined</c> is
- returned.
- </p>
- </desc>
- </func>
- <func>
- <name name="format_error" arity="1"/>
- <fsummary>Format an error descriptor</fsummary>
- <desc>
- <p>Takes an <c><anno>ErrorDescriptor</anno></c> and returns
- a string which
- describes the error or warning. This function is usually
- called implicitly when processing an <c>ErrorInfo</c>
- structure (see below).</p>
+ <p>For a description of the options, see
+ <seealso marker="#string/3"><c>string/3</c></seealso>.</p>
</desc>
</func>
</funcs>
<section>
+ <marker id="errorinfo"/>
<title>Error Information</title>
- <p>The <c>ErrorInfo</c> mentioned above is the standard
- <c>ErrorInfo</c> structure which is returned from all IO
- modules. It has the following format:</p>
+ <p><c>ErrorInfo</c> is the standard <c>ErrorInfo</c> structure that is
+ returned from all I/O modules. The format is as follows:</p>
<code type="none">
{ErrorLocation, Module, ErrorDescriptor}</code>
- <p>A string which describes the error is obtained with the
- following call:</p>
+ <p>A string describing the error is obtained with the following call:</p>
<code type="none">
Module:format_error(ErrorDescriptor)</code>
</section>
@@ -305,15 +316,15 @@ Module:format_error(ErrorDescriptor)</code>
<section>
<title>Notes</title>
<p>The continuation of the first call to the re-entrant input
- functions must be <c>[]</c>. Refer to Armstrong, Virding and
- Williams, 'Concurrent Programming in Erlang', Chapter 13, for a
- complete description of how the re-entrant input scheme works.</p>
+ functions must be <c>[]</c>. For a complete description of how the
+ re-entrant input scheme works, see Armstrong, Virding and
+ Williams: 'Concurrent Programming in Erlang', Chapter 13.</p>
</section>
<section>
<title>See Also</title>
- <p><seealso marker="io">io(3)</seealso>,
- <seealso marker="erl_anno">erl_anno(3)</seealso>,
- <seealso marker="erl_parse">erl_parse(3)</seealso></p>
+ <p><seealso marker="erl_anno"><c>erl_anno(3)</c></seealso>,
+ <seealso marker="erl_parse"><c>erl_parse(3)</c></seealso>,
+ <seealso marker="io"><c>io(3)</c></seealso></p>
</section>
</erlref>