From 68d53c01b0b8e9a007a6a30158c19e34b2d2a34e Mon Sep 17 00:00:00 2001 From: =?UTF-8?q?Bj=C3=B6rn=20Gustavsson?= Date: Wed, 18 May 2016 15:53:35 +0200 Subject: Update STDLIB documentation MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Language cleaned up by the technical writers xsipewe and tmanevik from Combitech. Proofreading and corrections by Björn Gustavsson and Hans Bolinder. --- lib/stdlib/doc/src/erl_scan.xml | 337 +++++++++++++++++++++------------------- 1 file changed, 174 insertions(+), 163 deletions(-) (limited to 'lib/stdlib/doc/src/erl_scan.xml') diff --git a/lib/stdlib/doc/src/erl_scan.xml b/lib/stdlib/doc/src/erl_scan.xml index ee0d6b6033..137ccd3416 100644 --- a/lib/stdlib/doc/src/erl_scan.xml +++ b/lib/stdlib/doc/src/erl_scan.xml @@ -4,7 +4,7 @@
- 19962015 + 19962016 Ericsson AB. All Rights Reserved. @@ -28,16 +28,17 @@ 1 Bjarne Däcker - 97-01-24 + 1997-01-24 B - erl_scan.sgml + erl_scan.xml
erl_scan - The Erlang Token Scanner + The Erlang token scanner. -

This module contains functions for tokenizing characters into +

This module contains functions for tokenizing (scanning) characters into Erlang tokens.

 + @@ -70,23 +71,96 @@ + + + + Return the category. + +

Returns the category of Token.

+ + + + + + Return the column. + +

Returns the column of Token's + collection of annotations.

+ + + + + + Return the end location of the text. + +

Returns the end location of the text of + Token's collection of annotations. If + there is no text, undefined is returned.

+ + + + + + Format an error descriptor. + +

Uses an ErrorDescriptor and returns a string + that describes the error or warning. This function is usually + called implicitly when an ErrorInfo structure is + processed (see section + Error Information).

+ + + + + + Return the line. + +

Returns the line of Token's collection + of annotations.

+ + + + + + Return the location. + +

Returns the location of Token's + collection of annotations.

+ + + + + + Test for a reserved word. + +

Returns true if Atom is an + Erlang reserved word, otherwise false.

+ + + - Scan a string and return the Erlang tokens + Scan a string and return the Erlang tokens.

Takes the list of characters String and tries to - scan (tokenize) them. Returns {ok, Tokens, - EndLocation}, - where Tokens are the Erlang tokens from - String. EndLocation - is the first location after the last token.

-

{error, ErrorInfo, ErrorLocation} - is returned if an error occurs. - ErrorLocation is the first location after - the erroneous token.

+ scan (tokenize) them. Returns one of the following:

+ + {ok, Tokens, EndLocation} + +

Tokens are the Erlang tokens from + String. EndLocation + is the first location after the last token.

+ + {error, ErrorInfo, + ErrorLocation} + +

An error occurred. ErrorLocation is the + first location after the erroneous token.

+ +

string(String) is equivalent to string(String, 1), and string(String, @@ -95,80 +169,102 @@ StartLocation, []).

StartLocation indicates the initial location when scanning starts. If StartLocation is a line, - Anno as well as EndLocation and - ErrorLocation will be lines. If - StartLocation is a pair of a line and a column + Anno, EndLocation, and + ErrorLocation are lines. If + StartLocation is a pair of a line and a column, Anno takes the form of an opaque compound data type, and EndLocation and ErrorLocation - will be pairs of a line and a column. The token + are pairs of a line and a column. The token annotations contain information about the column and the line where the token begins, as well as the text of the - token (if the text option is given), all of which can + token (if option text is specified), all of which can be accessed by calling - column/1, - line/1, - location/1, and - text/1.

+ column/1, + line/1, + location/1, and + text/1.

A token is a tuple containing information about - syntactic category, the token annotations, and the actual - terminal symbol. For punctuation characters (e.g. ;, + syntactic category, the token annotations, and the + terminal symbol. For punctuation characters (such as ; and |) and reserved words, the category and the symbol coincide, and the token is represented by a two-tuple. - Three-tuples have one of the following forms: {atom, - Info, atom()}, - {char, Info, integer()}, {comment, Info, - string()}, {float, Info, float()}, {integer, - Info, integer()}, {var, Info, atom()}, - and {white_space, Info, string()}.

-

The valid options are:

+ Three-tuples have one of the following forms:

+ + {atom, Anno, atom()} + {char, Anno, char()} + {comment, Anno, string()} + {float, Anno, float()} + {integer, Anno, integer()} + {var, Anno, atom()} + {white_space, Anno, string()} + +

Valid options:

- {reserved_word_fun, reserved_word_fun()} -

A callback function that is called when the scanner - has found an unquoted atom. If the function returns - true, the unquoted atom itself will be the category - of the token; if the function returns false, - atom will be the category of the unquoted atom.

- - return_comments -

Return comment tokens.

- - return_white_spaces -

Return white space tokens. By convention, if there is - a newline character, it is always the first character of the - text (there cannot be more than one newline in a white space - token).

- - return -

Short for [return_comments, return_white_spaces].

- - text -

Include the token's text in the token annotation. The - text is the part of the input corresponding to the token.

- + {reserved_word_fun, reserved_word_fun()} +

A callback function that is called when the scanner + has found an unquoted atom. If the function returns + true, the unquoted atom itself becomes the category + of the token. If the function returns false, + atom becomes the category of the unquoted atom.

+ + return_comments +

Return comment tokens.

+ + return_white_spaces +

Return white space tokens. By convention, a newline + character, if present, is always the first character of the + text (there cannot be more than one newline in a white space + token).

+ + return +

Short for [return_comments, return_white_spaces].

+ + text +

Include the token text in the token annotation. The + text is the part of the input corresponding to the token.

+ + + + + Return the symbol. + +

Returns the symbol of Token.

+ + + + + + Return the text. + +

Returns the text of Token's collection + of annotations. If there is no text, undefined is + returned.

+ + + - Re-entrant scanner + Re-entrant scanner. - An opaque continuation + An opaque continuation. -

This is the re-entrant scanner which scans characters until - a dot ('.' followed by a white space) or - eof has been reached. It returns:

+

This is the re-entrant scanner, which scans characters until + either a dot ('.' followed by a white space) or + eof is reached. It returns:

{done, Result, LeftOverChars} -

This return indicates that there is sufficient input +

Indicates that there is sufficient input data to get a result. Result is:

- {ok, Tokens, EndLocation} - + {ok, Tokens, EndLocation}

The scanning was successful. Tokens is the list of tokens including dot.

@@ -177,8 +273,7 @@

End of file was encountered before any more tokens.

 - {error, ErrorInfo, EndLocation} - + {error, ErrorInfo, EndLocation}

An error occurred. LeftOverChars is the remaining characters of the input data, @@ -194,110 +289,26 @@

The CharSpec eof signals end of file. - LeftOverChars will then take the value eof + LeftOverChars then takes the value eof as well.

tokens(Continuation, CharSpec, StartLocation) is equivalent to tokens(Continuation, CharSpec, StartLocation, []).

-

See string/3 for a - description of the various options.

- - - - - Test for a reserved word - -

Returns true if Atom is an Erlang - reserved word, otherwise false.

- - - - - Return the category - -

Returns the category of Token. -

- - - - - Return the symbol - -

Returns the symbol of Token. -

- - - - - Return the column - -

Returns the column of Token's - collection of annotations. -

- - - - - Return the end location of the text - -

Returns the end location of the text of - Token's collection of annotations. If - there is no text, - undefined is returned. -

- - - - - Return the line - -

Returns the line of Token's collection - of annotations. -

- - - - - Return the location - -

Returns the location of Token's - collection of annotations. -

- - - - - Return the text - -

Returns the text of Token's collection - of annotations. If there is no text, undefined is - returned. -

- - - - - Format an error descriptor - -

Takes an ErrorDescriptor and returns - a string which - describes the error or warning. This function is usually - called implicitly when processing an ErrorInfo - structure (see below).

+

For a description of the options, see + string/3.
+ Error Information -

The ErrorInfo mentioned above is the standard - ErrorInfo structure which is returned from all IO - modules. It has the following format:

+

ErrorInfo is the standard ErrorInfo structure that is + returned from all I/O modules. The format is as follows:

{ErrorLocation, Module, ErrorDescriptor} -

A string which describes the error is obtained with the - following call:

+

A string describing the error is obtained with the following call:

Module:format_error(ErrorDescriptor)
@@ -305,15 +316,15 @@ Module:format_error(ErrorDescriptor)
Notes

The continuation of the first call to the re-entrant input - functions must be []. Refer to Armstrong, Virding and - Williams, 'Concurrent Programming in Erlang', Chapter 13, for a - complete description of how the re-entrant input scheme works.

+ functions must be []. For a complete description of how the + re-entrant input scheme works, see Armstrong, Virding and + Williams: 'Concurrent Programming in Erlang', Chapter 13.

See Also -

io(3), - erl_anno(3), - erl_parse(3)

+

erl_anno(3), + erl_parse(3), + io(3)

 -- cgit v1.2.3