From 4b42bf9358eca2c4597837e87dd10e49c1b60bc7 Mon Sep 17 00:00:00 2001
From: Hans Bolinder This module contains functions for converting between different character representations. Basically it converts between iso-latin-1 characters and Unicode ditto, but it can also convert between different Unicode encodings (like UTF-8, UTF-16 and UTF-32). This module contains functions for converting between different character representations. Basically it converts between ISO-latin-1 characters and Unicode ditto, but it can also convert between different Unicode encodings (like UTF-8, UTF-16 and UTF-32). The default Unicode encoding in Erlang is in binaries UTF-8, which is also the format in which built in functions and libraries in OTP expect to find binary Unicode data. In lists, Unicode data is encoded as integers, each integer representing one character and encoded simply as the Unicode codepoint for the character. Other Unicode encodings than integers representing codepoints or UTF-8 in binaries are referred to as "external encodings". The iso-latin-1 encoding is in binaries and lists referred to as latin1-encoding. Other Unicode encodings than integers representing codepoints or UTF-8 in binaries are referred to as "external encodings". The ISO-latin-1 encoding is in binaries and lists referred to as latin1-encoding. It is recommended to only use external encodings for communication with external entities where this is required. When working inside the Erlang/OTP environment, it is recommended to keep binaries in UTF-8 when representing Unicode characters. Latin1 encoding is supported both for backward compatibility and for communication with external entities not supporting Unicode character sets. A binary() with characters encoded in the UTF-8 coding standard. A An integer() representing a valid unicode codepoint. An A unicode_binary is allowed as the tail of the list. A A A Check for a UTF byte order mark (BOM) in the beginning of a
@@ -126,7 +128,7 @@
Same as characters_to_list( Same as This function converts a possibly deep list of integers and
- binaries into a list of integers representing unicode
+ Converts a possibly deep list of integers and
+ binaries into a list of integers representing Unicode
characters. The binaries in the input may have characters
encoded as latin1 (0 - 255, one character per byte), in which
case the If The purpose of the function is mainly to be able to convert
- combinations of unicode characters into a pure unicode
+ combinations of Unicode characters into a pure Unicode
string in list representation for further processing. For
writing the data to an external entity, the reverse function
The option The option If for some reason, the data cannot be converted, either
- because of illegal unicode/latin1 characters in the list, or
+ because of illegal Unicode/latin1 characters in the list, or
because of invalid UTF encoding in any binaries, an error
tuple is returned. The error tuple contains the tag
However, if the input If any parameters are of the wrong type, the list structure
- is invalid (a number as tail) or the binaries does not contain
+ is invalid (a number as tail) or the binaries do not contain
whole bytes (bit-strings), a Same as characters_to_binary(Data, unicode, unicode). Same as Same as characters_to_binary( Same as This function behaves as Behaves as Errors and exceptions occur as in Create an UTF byte order mark (BOM) as a binary from the
+ Create a UTF byte order mark (BOM) as a binary from the
supplied The function returns
@@ -205,14 +207,14 @@
(like the upper
bits of the bytes being wrong), the bytes are decoded to a
too large number, the bytes are decoded to a code-point in the
- invalid unicode
- range or encoding is "overlong", meaning that a
+ invalid Unicode
+ range, or encoding is "overlong", meaning that a
number should have been encoded in fewer bytes. The
case of a truncated UTF is handled specially, see the
paragraph about incomplete binaries below. If
It can be noted that the BOM for UTF-8 is seldom used, and it is really not a byte order mark. There are obviously no -- cgit v1.2.3