Xmerl Release Notes

20042011 Ericsson AB. All Rights Reserved. The contents of this file are subject to the Erlang Public License, Version 1.1, (the "License"); you may not use this file except in compliance with the License. You should have received a copy of the Erlang Public License along with this software. If not, it can be retrieved online at http://www.erlang.org/. Software distributed under the License is distributed on an "AS IS" basis, WITHOUT WARRANTY OF ANY KIND, either express or implied. See the License for the specific language governing rights and limitations under the License. Xmerl Release Notes otp_appnotes nil nil nil notes.xml

This document describes the changes made to the Xmerl application.

Xmerl 1.3

Fixed Bugs and Malfunctions

Fix character check of non-characters due to change in unicode module.

Own Id: OTP-9670

Treat , as special in xmerl_xpath_scan. (Thanks to Anneli Cuss)

Own Id: OTP-9753

Improvements and New Features

Updates to the xml scanner

xmerl_scan is now returning xmlComment records in the output.

Functions xmerl_scan:file/2 and xmerl_scan:string/2 now accepts a new option {comments, Flag} for filtering of comments.
Default (true) is that #xmlComment records are returned from the scanner and this flag should be set to false if one don't want comments in the output.

Add default_attrs option

When default_attrs is true any attribute with a default value defined in the doctype but not in the attribute axis of the currently scanned element is added to it.

Allow whole documents to be returned

Functions xmerl_scan:file/2 and xmerl_scan:string/2 now accepts a new option {document, true} to produce a whole document as a xmlDocument record instead of just the root element node.
This option is the only way to get to the top-level comments and processing instructions without hooking through the customization functions. Those nodes are needed to implement [Canonical XML][c14n-xml] support.
[c14n-xml]: http://www.w3.org/TR/2008/PR-xml-c14n11-20080129/ Canonical XML

Parents and namespace are tracked in #xmlAttribute nodes

Parents are tracked in #xmlPI nodes

Set vsn field in #xmlDecl record

Fix namespace-conformance constraints

See [Namespaces in XML 1.0 (Third Edition)][1]: The prefix xml is by definition bound to the namespace name http://www.w3.org/XML/1998/namespace. It MAY, but need not, be declared, and MUST NOT be bound to any other namespace name. Other prefixes MUST NOT be bound to this namespace name, and it MUST NOT be declared as the default namespace.
The prefix xmlns is used only to declare namespace bindings and is by definition bound to the namespace name http://www.w3.org/2000/xmlns/. It MUST NOT be declared . Other prefixes MUST NOT be bound to this namespace name, and it MUST NOT be declared as the default namespace. Element names MUST NOT have the prefix xmlns.
In XML documents conforming to this specification, no tag may contain two attributes which have identical names, or have qualified names with the same local part and with prefixes which have been bound to namespace names that are identical.
[1] http://www.w3.org/TR/REC-xml-names/

Updates of xmerl's Xpath functionality.

Add #xmlPI support to xmerl_xpath:write_node/1

Fix processing-instruction(name?)

Fix path filters, support more top-level primary expressions

Accumulate comments in element nodes

Implement namespace axis

Namespace nodes are represented as #xmlNsNode records. Now that the namespace axis is correctly implemented, attributes nodes corresponding to attributes that declare namespaces are ignored.
See [5.3 Attribute Nodes][xpath-5.3]:
There are no attribute nodes corresponding to attributes that declare namespaces.
[xpath-5.3]: http://www.w3.org/TR/xpath/#attribute-nodes

(Thanks to Anthony Ramine)

*** POTENTIAL INCOMPATIBILITY ***

Own Id: OTP-9664

Eliminate use of deprecated regexp module

Own Id: OTP-9810

Xmerl 1.2.10

Fixed Bugs and Malfunctions

Fixed a schema search bug in xmerl_xsd.

A new flag was needed in the xsd_state record so if the state is saved there is an incompatibility and a state conversion is needed.

*** INCOMPATIBILITY with R14B03 ***

Own Id: OTP-9410

Fixed xmerl_scan problems with entities in attribute values.

Own Id: OTP-9411

Streaming bug in xmerl_scan.

If the continuation_fun runs out of input at the end of an attribute value then it crashed. (Thanks to Simon Cornish)

Own Id: OTP-9457

Fixed xmerl_ucs UCS2 little endian en/decoding

Corrected number of shift bytes in xmerl_ucs:char_to_ucs2le and recursive call from from_ucs2le to from_ucs4le. (Thanks to Michal Ptaszek)

Own Id: OTP-9548

Add latin9 (iso-8859-15) support in xmerl_ucs (Thanks to David Julien)

Own Id: OTP-9552

Improve spelling throughout documentation, code comments and error messages

Own Id: OTP-9555

Xmerl 1.2.9

Fixed Bugs and Malfunctions

Fix minor typos and improve punctuation in the xmerl_xpath @doc comment (Thanks to Marcus Marinelli)

Own Id: OTP-9187

Prevent xmerl from over-normalizing character references in attributes

Section 3.3.3 of the XML Recommendation gives the rules for attribute-value normalization. One of those rules requires that character references not be re-normalized after being replaced with the referenced characters. (Thanks to Tom Moertel)

Own Id: OTP-9274

Fixed the default encoding option in SAX parser.

Own Id: OTP-9288

Improvements and New Features

Added the xmerl test suites and examples to the open source distribution.

Own Id: OTP-9228

Xmerl 1.2.8

Fixed Bugs and Malfunctions

The function xmerl_lib:expand_content/1 is mainly for expanding Simple XML, but can also handle xmerl records. This patch fixes an omission that caused expand_content/1 to not maintain the parents list when expanding #xmlElement{} records. (Thanks to Ulf Wiger)

Own Id: OTP-9034

Improvements and New Features

Removed some dialyzer warnings.

Own Id: OTP-9074

Xmerl 1.2.7

Fixed Bugs and Malfunctions

An empty element declared as simpleContent was not properly validated.

Own Id: OTP-8599

Fix format_man_pages so it handles all man sections and remove warnings/errors in various man pages.

Own Id: OTP-8600

Improvements and New Features

Fix entity checking so there are no fatal errors for undefined entities when option skip_external_dtd is used.

Own Id: OTP-8947

Xmerl 1.2.6

Fixed Bugs and Malfunctions

Fixed problem with hex entities in UTF-8 documents: When a document was in UTF-8 encoding, xmerl_scan improperly replaced hex entities by the UTF-8 bytes instead of returning the character, as it does with inline UTF-8 text and decimal entities. (Thanks to Paul Guyot.)

Own Id: OTP-8697

Xmerl 1.2.5

Improvements and New Features

All Erlang files are now built by the test server instead of the test directory Makefile.

Erlang files in data directories are now built by the test suites instead of using prebuilt versions under version control.

Removed a number of obsolete guards.

Own Id: OTP-8537

An empty element declared as a simpleContent was not properly validated.

Own Id: OTP-8599

Xmerl 1.2.4

Improvements and New Features

Updated the documentation Makefile to work with the new documentation build process.

Own Id: OTP-8343

Xmerl 1.2.3

Fixed Bugs and Malfunctions

A continuation clause of parse_reference/3 had its parameters in wrong order.

Own Id: OTP-8251 Aux Id: seq11429

Improvements and New Features

A new option to turn off the parsing of an external DTD is added to xmerl_sax_parser:file/2 and xmerl_sax_parser:stream/2 (skip_external_dtd).

Own Id: OTP-8252 Aux Id: seq11432

The documentation is now built with open source tools (xsltproc and fop) that exists on most platforms. One visible change is that the frames are removed.

Own Id: OTP-8253

Xmerl 1.2.2

Fixed Bugs and Malfunctions

xmerl_sax_parse:stream/2 failed with {fatal_error,_, "Continuation function undefined, and more data needed",_,_} when no continuation function was defined even though it was a complete document as input.

Own Id: OTP-8213

The namespace URI supplied on unprefixed attributes in startElement tuples is the same as the URI for the default namespace. According to the standard the namespace for an unprefixed attribute should always has no value.

Own Id: OTP-8214

Xmerl 1.2.1

Fixed Bugs and Malfunctions

xmerl/include/xmerl.hrl contained internal debug macros (dbg/2 and DBG/0) which now is moved to xmerl_internal.hrl.

Own Id: OTP-8084

The function xmerl_uri:parse/1 couldn't handle FTP URIs containing username and password. The default FTP port constant was also wrong. (Thanks to Steve Vinoski)

Own Id: OTP-8156

Improvements and New Features

The SAX parser couldn't handle consecutive documents on the same stream. The return values are now changed so they return a rest value instead of giving an error about "erranous information after the end tag".

This means that the functions file/2 and stream/2 now returns {ok, EventState, Rest} when the parsing is correct. The rest can then be used as input to a new call to xmerl_sax_parse:stream/2. If one know that it's just one document the rest value in the result tuple can be matched against <<>> or [] depending on if the input is in binary form or not.

Own Id: OTP-8153 Aux Id: seq11388

Xmerl 1.2

Improvements and New Features

In xmerl-1.2 we have added the first Beta version of the new SAX parser (module: xmerl_sax_parser), it supports XML 1.0. We call it Beta due to that the validation part is not ready yet and that the parser still has some known limitations (mostly in the DTD area).

Known limitations:

the external DTD in the DOCTYPE declaration is handled but other external entities are not supported. the general entity values are just checked in the structure after replacement. parsed entities are supported on markup declaration level (e.g. partly replacement of markup declaration with PEReference is not supported). conditionalSect in external DTD's are not supported. recursive loops in entity declarations are not detected.

The version is increased from 1.1.12 to 1.2 is due to that the new parser is dependent on the Unicode support that was added in OTP R13B. The old xmerl functionality is not changed.

Own Id: OTP-6635

Xmerl 1.1.12

Improvements and New Features

Updated copyright notice in source files

Own Id: OTP-7847

Xmerl 1.1.11

Fixed Bugs and Malfunctions

An empty element with a complexType and simpleContent was not properly validated. This error is now corrected.

Own Id: OTP-7736

Xmerl 1.1.10

Fixed Bugs and Malfunctions

Changed the examples in Customization Functions Tutorial to correct Erlang code.

Own Id: OTP-6053

Some XPath errors solved, typo in compare function '!=', error in id() function.

Own Id: OTP-6792 Aux Id: seq10570

The XPath function contains() now implemented. See XPath 1.0 section 4.2.

Own Id: OTP-6873

Fixed that xmerl_xsd:process_schema/2 with {xsdbase, Dirname} failed with enoent and a number of inor documentation bugs in xmerl_xsd reference manual.

Own Id: OTP-7165

Fixed xmerl_scan's problem with numeric character references followed by UTF-8 characters in the contents.

Own Id: OTP-7430

Fixed an incorrect guard for xmerl_scan:to_ucs/2.

Own Id: OTP-7473

Some bug corrections of xmerl XPath implementation, most provided by Matthew Dempsky.

Own Id: OTP-7496

Now with string() and name() all XPath functions are implemented. The string representation of QName by name() is "{Namespace URI}local-name".

Own Id: OTP-7510

Xmerl 1.1.9

Fixed Bugs and Malfunctions

A number of minor scanner faults have got more clear error messages.

Own Id: OTP-5998, Aux Id: seq9803

An example error in the Xmerl Users Guide is corrected.

Own Id: OTP-6947

When xmerl_xsd:validate was executed the schema table in the state was deleted and next execution would fail. This is now corrected.

Own Id: OTP-7288

Xmerl 1.1.8

Fixed Bugs and Malfunctions

A Kleene Closure child in a sequence consumed all following children. This problem has been fixed.

Own Id: OTP-7211

Now validating xhtml1-transitional.dtd. A certain contentspec with a succeeding choice, that didn't match all content, followed by other child elements caused a failure. This is now corrected.

Own Id: OTP-7214

Xmerl 1.1.7

Improvements and New Features

xmerl's schema validation now takes default facets into account

Own Id: OTP-7190

Xmerl 1.1.6

Fixed Bugs and Malfunctions

Parsing XML with option {validation,schema} is now corrected.

Own Id: OTP-6773

union type is now supported

Own Id: OTP-6877 Aux Id: seq10755

Now xmerl validates as expected when a sequence has a present group member and a following element.

Own Id: OTP-6910

Xmerl 1.1.5

Fixed Bugs and Malfunctions

The head of a substitutionGroup may have type anyType and thus allow members of any type. This was an oversight, but is now corrected.

Own Id: OTP-6720

A recursive group reference in a redefine refers to the definition in the redefined schema. See 4.2.2 in XMLSchema part1 "Schema Representation Constraint: Individual Component Redefinition" bullet 2.

Own Id: OTP-6739

Solved some content model problems, for instance in some cases failed when more than one choice.

Own Id: OTP-6752

Xmerl 1.1.4

Improvements and New Features

An additional format is possible for the simple syntax: {Fun, State}. The fun should retrieve the replacement in simple syntax format. The semantics of fun: fun(State) -> code that creates replacement, then returns {SimpleSyntax,NewState} | done

Own Id: OTP-6679

Xmerl 1.1.3

Improvements and New Features

Memory consumption decreased: moved transforming from utf-8 to unicode from an extra pass of the document to the occasion when a character is parsed. Removed use of lists:subtract. Those changes also speeds up parsing in some scenarios.

Own Id: OTP-6599 Aux Id: seq10552

Xmerl 1.1.2

Fixed Bugs and Malfunctions

Schema processor reprocessed schemas that already were processed, using process_schemas on a system of schemas with circular dependencies.

Own Id: OTP-6460 Aux Id: seq10564

Improvements and New Features

Dialyzer warnings now removed, i.e. dead code have been removed.

Own Id: OTP-6507

Xmerl 1.1.1

Fixed Bugs and Malfunctions

Bug in xmerl removed so that simple syntax element content is exported correctly.

Own Id: OTP-6402 Aux Id: OTP-6099

Xmerl 1.1

Fixed Bugs and Malfunctions

Xmerl failed to parse and export with the sax_file front-end. Therefore hook function calls were added in the parser and handling of text content were changed.

Own Id: OTP-6043

Bug in xmerl removed so that simple syntax element content is exported correctly.

Own Id: OTP-6099

Improvements and New Features

xmerl now supports XMLSchema validation. Documentation in reference manual for xmerl. The release of XMLSchema validation should be considered as a beta release. The user interface may still be adjusted in a coming release. Opinions and evaluations are welcome.

Own Id: OTP-6401

xmerl 1.0.5

Fixed Bugs and Malfunctions

Code that caused compiler warnings has been reviewed.

xmerl 1.0.4

Fixed Bugs and Malfunctions

xmerl behaved strange parsing a XML-document with a copyright sign in a comment.

Own Id: OTP-5599

Line count for error messages in DTD improved, still problem because of ENTITY expansions. Didn't delete digraphs after recursion test. Now correctly parsing of declaration separators [28a-b].

Own Id: OTP-5718

Failed to validate a XML file with a content spec that had a choice of which one element was a sequence with optional elements, and all elements of that sequence were missing

Own Id: OTP-5734

Location paths for document root and attributes is now working as expected.

Own Id: OTP-5895

Now has the last() predicate in the XPATH modules the properties specified in ch 2.4 in the XPATH spec, i.e. if last() evaluates to a number other than the context position it is false, otherwise true.

Own Id: OTP-5902

The location path of a single wildcard now only selects element nodes.

Own Id: OTP-5905

Xmerl 1.0.3

Fixed Bugs and Malfunctions

Removed call of undefined function in xmerl_lib.

Own Id: OTP-5587

Xmerl 1.0.2

Fixed Bugs and Malfunctions

Better identification of errors in xml code.

Own Id: OTP-5498 Aux Id: seq9803

Some minor bugs fixed.

Own Id: OTP-5500

Parser failed on PE reference as EnumeratedType AttType, now corrected.

Own Id: OTP-5531

Xmerl 1.0.1

Fixed Bugs and Malfunctions

Fixed bug in xmerl_xpath. Xpath expressions that select nodes of type text() didn't work, like "context/text()", "child::text()", "descendant::text()".

Own Id: OTP-5268 Aux Id: seq9656

Minor bugs fixed.

Own Id: OTP-5301

Xmerl 1.0

Improvements and New Features

The OTP release of xmerl 1.0 is mainly the same as xmerl-0.20 of http://sowap.sourceforge.net/. It is capable of parsing XML 1.0. There have only been minor improvements: Some bugs that caused an unexpected crash when parsing bad XML. Failure report that also tells which file that caused an error.

Own Id: OTP-5174