aboutsummaryrefslogtreecommitdiffstats
path: root/system
diff options
context:
space:
mode:
authorBjörn Gustavsson <[email protected]>2011-08-30 11:51:11 +0200
committerBjörn Gustavsson <[email protected]>2011-10-13 14:16:00 +0200
commit34db76765561487e526fe66d3d19ecf3b3fb9dc8 (patch)
tree9141e3c5729e46d03c8b27b14da3b29b1e54abca /system
parent6ca6dd3c670fb8185ebb9a20c2a731a7375c1cac (diff)
downloadotp-34db76765561487e526fe66d3d19ecf3b3fb9dc8.tar.gz
otp-34db76765561487e526fe66d3d19ecf3b3fb9dc8.tar.bz2
otp-34db76765561487e526fe66d3d19ecf3b3fb9dc8.zip
Allow noncharacter code points in unicode encoding and decoding
The two noncharacter code points 16#FFFE and 16#FFFF were not allowed to be encoded or decoded using the unicode module or bit syntax. That causes an inconsistency, since the noncharacters 16#FDD0 to 16#FDEF could be encoded/decoded. There is two ways to fix that inconsistency. We have chosen to allow 16#FFFE and 16#FFFF to be encoded and decoded, because the noncharacters could be useful internally within an application and it will make encoding and decoding slightly faster. Reported-by: Alisdair Sullivan
Diffstat (limited to 'system')
-rw-r--r--system/doc/reference_manual/expressions.xml12
1 files changed, 5 insertions, 7 deletions
diff --git a/system/doc/reference_manual/expressions.xml b/system/doc/reference_manual/expressions.xml
index 497d7eb464..644896cd7f 100644
--- a/system/doc/reference_manual/expressions.xml
+++ b/system/doc/reference_manual/expressions.xml
@@ -879,9 +879,8 @@ Ei = Value |
and UTF-32, respectively.</p>
<p>When constructing a segment of a <c>utf</c> type, <c>Value</c>
- must be an integer in one of the ranges 0..16#D7FF,
- 16#E000..16#FFFD, or 16#10000..16#10FFFF
- (i.e. a valid Unicode code point). Construction
+ must be an integer in the range 0..16#D7FF or
+ 16#E000....16#10FFFF. Construction
will fail with a <c>badarg</c> exception if <c>Value</c> is
outside the allowed ranges. The size of the resulting binary
segment depends on the type and/or <c>Value</c>. For <c>utf8</c>,
@@ -896,14 +895,13 @@ Ei = Value |
<c><![CDATA[<<$a/utf8,$b/utf8,$c/utf8>>]]></c>.</p>
<p>A successful match of a segment of a <c>utf</c> type results
- in an integer in one of the ranges 0..16#D7FF, 16#E000..16#FFFD,
- or 16#10000..16#10FFFF
- (i.e. a valid Unicode code point). The match will fail if returned value
+ in an integer in the range 0..16#D7FF or 16#E000..16#10FFFF.
+ The match will fail if returned value
would fall outside those ranges.</p>
<p>A segment of type <c>utf8</c> will match 1 to 4 bytes in the binary,
if the binary at the match position contains a valid UTF-8 sequence.
- (See RFC-2279 or the Unicode standard.)</p>
+ (See RFC-3629 or the Unicode standard.)</p>
<p>A segment of type <c>utf16</c> may match 2 or 4 bytes in the binary.
The match will fail if the binary at the match position does not contain