From b6196f5d47c06e6991cfc7e76db3d588a7797302 Mon Sep 17 00:00:00 2001 From: Patrik Nyblom Date: Tue, 14 Aug 2012 15:02:43 +0200 Subject: Add documetation about Unicode in environment --- erts/doc/src/erl.xml | 2 +- erts/doc/src/erlang.xml | 7 +++++-- lib/kernel/doc/src/os.xml | 15 +++++++++++++++ lib/stdlib/doc/src/unicode_usage.xml | 8 +++++++- 4 files changed, 28 insertions(+), 4 deletions(-) diff --git a/erts/doc/src/erl.xml b/erts/doc/src/erl.xml index f8a92e1ec4..8c028e1d14 100644 --- a/erts/doc/src/erl.xml +++ b/erts/doc/src/erl.xml @@ -533,7 +533,7 @@

Force the compressed option on all ETS tables. Only intended for test and evaluation.

- +

The VM works with file names as if they are encoded using the ISO-latin-1 encoding, disallowing Unicode characters with codepoints beyond 255. This is default on operating systems that have transparent file naming, i.e. all Unixes except MacOSX.

 diff --git a/erts/doc/src/erlang.xml b/erts/doc/src/erlang.xml index e69f223897..248b755969 100644 --- a/erts/doc/src/erlang.xml +++ b/erts/doc/src/erlang.xml @@ -3039,8 +3039,11 @@ os_prompt% port process. Both Name and Val must be strings. The one exception is Val being the atom false (in analogy with os:getenv/1), which - removes the environment variable. Not available on - VxWorks.

+ removes the environment variable.

+

If Unicode filename encoding is in effect (see the erl manual + page), the strings (both Name and + Value) may contain characters with codepoints > 255.

{args, [ string() ]} diff --git a/lib/kernel/doc/src/os.xml b/lib/kernel/doc/src/os.xml index e94119845a..09c525b376 100644 --- a/lib/kernel/doc/src/os.xml +++ b/lib/kernel/doc/src/os.xml @@ -80,6 +80,10 @@ DirOut = os:cmd("dir"), % on Win32 platform Each environment variable is given as a single string on the format "VarName=Value", where VarName is the name of the variable and Value its value.

+

If Unicode file name encoding is in effect (see the erl manual + page), the strings may contain characters with + codepoints > 255.

@@ -93,6 +97,10 @@ DirOut = os:cmd("dir"), % on Win32 platform

Returns the Value of the environment variable VarName, or false if the environment variable is undefined.

+

If Unicode file name encoding is in effect (see the erl manual + page), the strings (both VarName and + Value) may contain characters with codepoints > 255.

 @@ -123,6 +131,13 @@ DirOut = os:cmd("dir"), % on Win32 platform

Sets a new Value for the environment variable VarName.

+

If Unicode filename encoding is in effect (see the erl manual + page), the strings (both VarName and + Value) may contain characters with codepoints > 255.

+

On Unix platforms, the environment will be set using UTF-8 encoding + if Unicode file name translation is in effect. On Windows the + environment is set using wide character interfaces.

 diff --git a/lib/stdlib/doc/src/unicode_usage.xml b/lib/stdlib/doc/src/unicode_usage.xml index a7e010a05f..b7b5d497d0 100644 --- a/lib/stdlib/doc/src/unicode_usage.xml +++ b/lib/stdlib/doc/src/unicode_usage.xml @@ -53,7 +53,7 @@ Basically the same as UTF-32, but without some Unicode semantics, defined by IEEE and has little use as a separate encoding standard. For all normal (and possibly abnormal) usages, UTF-32 and UCS-4 are interchangeable.

Certain ranges of characters are left unused and certain ranges are even deemed invalid. The most notable invalid range is 16#D800 - 16#DFFF, as the UTF-16 encoding does not allow for encoding of these numbers. It can be speculated that the UTF-16 encoding standard was, from the beginning, expected to be able to hold all Unicode characters in one 16-bit entity, but then had to be extended, leaving a hole in the Unicode range to cope with backward compatibility.

-

Additionally, the codepoint 16#FEFF is used for byte order marks (BOM's) and use of that character is not encouraged in other contexts than that. It actually is valid though, as the character "ZWNBS" (Zero Width Non Breaking Space). BOM's are used to identify encodings and byte order for programs where such parameters are not known in advance. Byte order marks are more seldom used than one could expect, put their use is becoming more widely spread as they provide the means for programs to make educated guesses about the Unicode format of a certain file.

+

Additionally, the codepoint 16#FEFF is used for byte order marks (BOM's) and use of that character is not encouraged in other contexts than that. It actually is valid though, as the character "ZWNBS" (Zero Width Non Breaking Space). BOM's are used to identify encodings and byte order for programs where such parameters are not known in advance. Byte order marks are more seldom used than one could expect, but their use is becoming more widely spread as they provide the means for programs to make educated guesses about the Unicode format of a certain file.

Standard Unicode representation in Erlang @@ -210,6 +210,12 @@ Eshell V5.7 (abort with ^G)
+Unicode in environment variables and parameters +

Environment variables and their interpretation is handled much in the same way as file names. If Unicode file names are enabled, environment variables as well as parameters to the Erlang VM are expected to be in Unicode.

+

If Unicode file names are enabled, the calls to os:getenv/0, os:getenv/1 and os:putenv/2 will handle Unicode strings. On Unix-like platforms, the built-in functions will translate environment variables in UTF-8 to/from Unicode strings, possibly with codepoints > 255. On Windows the Unicode versions of the environment system API will be used, also allowing for codepoints > 255.

+

On Unix-like operating systems, parameters are expected to be UTF-8 without translation if Unicode file names are enabled.

+
+
Unicode-aware modules

Most of the modules in Erlang/OTP are of course Unicode-unaware in the sense that they have no notion of Unicode and really shouldn't have. Typically they handle non-textual or byte-oriented data (like gen_tcp etc).

Modules that actually handle textual data (like io_lib, string etc) are sometimes subject to conversion or extension to be able to handle Unicode characters.

-- cgit v1.2.3