aboutsummaryrefslogtreecommitdiffstats
path: root/lib/xmerl/doc/examples/xml/xmerl.xml
blob: f02282dbeff9bffbc0378b1246baa017fcc2e02a (plain) (blame)
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
157
158
159
160
161
162
163
164
165
166
167
168
169
170
171
172
173
174
175
176
177
178
179
180
181
182
183
184
185
186
187
188
189
190
191
192
193
194
195
196
197
198
199
200
201
202
203
204
205
206
207
208
209
210
211
212
213
214
215
216
217
218
219
220
221
222
223
224
225
226
227
228
229
230
231
232
233
234
235
236
237
238
239
240
241
242
243
244
245
246
247
248
249
250
251
252
253
254
255
256
257
258
259
260
261
262
263
264
265
266
267
268
269
270
271
272
273
274
275
276
277
278
279
280
281
282
283
284
285
286
287
288
289
290
291
292
293
294
295
296
297
298
299
300
301
302
303
304
305
306
307
308
309
310
311
312
313
314
315
316
317
318
319
320
321
322
323
324
325
326
327
328
329
330
331
332
333
334
335
336
337
338
339
340
341
342
343
344
345
346
347
348
349
350
351
352
353
354
355
356
357
358
359
360
361
362
363
364
365
366
367
368
369
370
371
372
373
374
375
376
377
378
379
380
381
382
383
384
385
386
387
388
389
390
391
392
393
394
395
396
397
398
399
400
401
402
403
404
405
406
407
408
409
410
411
412
413
414
415
416
417
418
419
420
421
422
423
424
425
426
427
428
429
430
431
432
433
434
435
436
437
438
439
440
441
442
443
444
445
446
447
448
449
450
451
452
453
454
455
456
457
458
459
460
461
462
463
464
465
466
467
468
469
470
471
472
473
474
475
476
477
478
479
480
481
482
483
484
485
486
487
488
489
490
491
492
493
494
495
496
497
498
499
500
501
502
503
504
505
506
507
508
509
510
511
512
513
514
515
516
517
518
519
520
521
522
523
<?xml version="1.0" encoding="iso-8859-1"?>
<!DOCTYPE article
      PUBLIC "-//OASIS//DTD Simplified DocBook XML V1.0//EN"
      "http://www.oasis-open.org/docbook/xml/simple/1.0/sdocbook.dtd">

<article  lang="en" xml:lang="en" >
  <articleinfo>
    <title>XMerL - XML processing tools for Erlang</title>
    <subtitle>Reference Manual</subtitle>
    <authorgroup>
      <author>
	<firstname>Ulf</firstname>
	<surname>Wiger</surname>
      </author>
    </authorgroup>
    <revhistory>
      <revision>
      <revnumber>1.0</revnumber><date>2003-02-04</date>
      <revremark>Converted xml from html</revremark>
      </revision>
    </revhistory>
    <abstract>
      <para>XMerL tools contains xmerl_scan; a non-validating XML
      processor, xmerl_xpath; a XPath implementation, xmerl for export
      of XML trees to HTML, XML or text and xmerl_xs for XSLT like
      transforms in erlang. 
      </para>
    </abstract>
  </articleinfo>
  
  <section>
    <title>xmerl_scan - the XML processor</title>
    <para>The (non-validating) XML processor is activated through 
    <computeroutput>xmerl_scan:string/[1,2]</computeroutput> or 
    <computeroutput>xmerl_scan:file/[1,2]</computeroutput>.
    It returns records of the type defined in xmerl.hrl.
    </para>
  
    <para>As far as I can tell, xmerl_scan implements the complete XML
    1.0 spec, including:</para>
    <itemizedlist>
      <listitem><para>entity expansion</para></listitem>
      <listitem><para>fetching and parsing external DTDs</para></listitem>
      <listitem><para>contitional processing</para></listitem>
      <listitem><para>UniCode</para></listitem>
      <listitem><para>XML Names</para></listitem>
    </itemizedlist>
    <programlisting>
xmerl_scan:string(Text [ , Options ]) -> #xmlElement{}.
xmerl_scan:file(Filename [ , Options ]) -> #xmlElement{}. </programlisting>
    
    <para>The Options are basically to specify the behaviour of the
    scanner.  See the source code for details, but you can specify
    funs to handle scanner events (event_fun), process the document
    entities once identified (hook_fun), and decide what to do if the
    scanner runs into eof before the document is complete
    (continuation_fun).</para>

    <para>You can also specify a path (fetch_path) as a list of
    directories to search when fetching files. If the file in question
    is not in the fetch_path, the URI will be used as a file
    name.</para>


    <section>
      <title>Customization functions</title>
      <para>The XML processor offers a number of hooks for
      customization. These hooks are defined as function objects, and
      can be provided by the caller.</para>
      
      <para>The following customization functions are available. If
      they also have access to their own state variable, the access
      function for this state is identified within parentheses:</para>

      <itemizedlist>

	<listitem><para>event function (<computeroutput>
	xmerl_scan:event_state/[1,2]
	</computeroutput>)</para></listitem>

	<listitem><para>hook function (<computeroutput>
	xmerl_scan:hook_state/[1,2]
	</computeroutput>)</para></listitem>

	<listitem><para>fetch function (<computeroutput>
	xmerl_scan:fetch_state/[1,2] </computeroutput>)
	</para></listitem>

	<listitem><para>continuation function (<computeroutput>
	xmerl_scan:cont_state/[1,2] </computeroutput>)
	</para></listitem>

	<listitem><para>rules function (<computeroutput>
      xmerl_scan:rules_state/[1,2] </computeroutput>)
      </para></listitem>

	<listitem><para>accumulator function</para></listitem>

	<listitem><para>close function</para></listitem>

      </itemizedlist>

      <para>For all of the above state access functions, the function
      with one argument
      (e.g. <computeroutput>event_fun(GlobalState)</computeroutput>)
      will read the state variable, while the function with two
      arguments (e.g.: <computeroutput>event_fun(NewStateData,
      GlobalState)</computeroutput>) will modify it.</para>

      <para>For each function, the description starts with the syntax
      for specifying the function in the
      <computeroutput>Options</computeroutput> list. The general forms
      are <computeroutput>{Tag, Fun}</computeroutput>, or
      <computeroutput>{Tag, Fun, LocalState}</computeroutput>. The
      second form can be used to initialize the state variable in
      question.</para>

      <section>
	<title>User State</title>

	<para>All customization functions are free to access a
	&quot;User state&quot; variable. Care must of course be taken
	to coordinate the use of this state. It is recommended that
	functions, which do not really have anything to contribute to
	the &quot;global&quot; user state, use their own state
	variable instead. Another option (used in
	e.g. <computeroutput>xmerl_eventp.erl</computeroutput>) is for
	customization functions to share one of the local states (in
	<computeroutput>xmerl_eventp.erl</computeroutput>, the
	continuation function and the fetch function both acces the
	<computeroutput>cont_state</computeroutput>.)</para>

	<para>Functions to access user state:</para>

	<itemizedlist>

	  <listitem><para><computeroutput>
	  xmerl_scan:user_state(GlobalState) </computeroutput>
	  </para></listitem>

	  <listitem><para><computeroutput>xmerl_scan:user_state(UserState',
	  GlobalState) </computeroutput></para></listitem>

	</itemizedlist>

      </section>
      <section>
	<title>Event Function</title>

	<para><computeroutput>{event_fun, fun()} | {event_fun, fun(),
	LocalState}</computeroutput></para>

	<para>The event function is called at the beginning and at the
	end of a parsed entity. It has the following format and
	semantics:</para>

<programlisting>
<![CDATA[
fun(Event, GlobalState) ->
   EventState = xmerl_scan:event_state(GlobalState),
   EventState' = foo(Event, EventState),
   GlobalState' = xmerl_scan:event_state(EventState', GlobalState)
end.
]]></programlisting>

      </section>
      <section>
	<title>Hook Function</title>
	<para> <computeroutput>{hook_fun, fun()} | {hook_fun, fun(),
	LocalState}</computeroutput></para>



<para>The hook function is called when the processor has parsed a complete
entity. Format and semantics:</para>

<programlisting>
<![CDATA[
fun(Entity, GlobalState) ->
   HookState = xmerl_scan:hook_state(GlobalState),
   {TransformedEntity, HookState'} = foo(Entity, HookState),
   GlobalState' = xmerl_scan:hook_state(HookState', GlobalState),
   {TransformedEntity, GlobalState'}
end.
]]></programlisting>

	<para>The relationship between the event function, the hook
	function and the accumulator function is as follows:</para>

	<orderedlist>
	  <listitem><para>The event function is first called with an
	  'ended' event for the parsed entity.</para></listitem>

	  <listitem><para>The hook function is called, possibly
	  re-formatting the entity.</para></listitem>

	  <listitem><para>The acc function is called in order to
	  (optionally) add the re-formatted entity to the contents of
	  its parent element.</para></listitem>

    </orderedlist>

      </section>
      <section>
	<title>Fetch Function</title>
<para>
<computeroutput>{fetch_fun, fun()} | {fetch_fun, fun(), LocalState}</computeroutput>
</para>
<para>The fetch function is called in order to fetch an external resource
(e.g. a DTD).</para>

<para>The fetch function can respond with three different return values:</para>

    <programlisting>
<![CDATA[
    Result ::=
      {ok, GlobalState'} |
      {ok, {file, Filename}, GlobalState'} |
      {ok, {string, String}, GlobalState'}
]]></programlisting>

<para>Format and semantics:</para>

    <programlisting>
<![CDATA[
fun(URI, GlobalState) ->
   FetchState = xmerl_scan:fetch_state(GlobalState),
   Result = foo(URI, FetchState).  % Result being one of the above
end.
]]></programlisting>

      </section>
      <section>
	<title>Continuation Function</title>
<para>
<computeroutput>{continuation_fun, fun()} | {continuation_fun, fun(), LocalState}</computeroutput>
</para>
<para>The continuation function is called when the parser encounters the end
of the byte stream. Format and semantics:</para>

    <programlisting>
<![CDATA[
fun(Continue, Exception, GlobalState) ->
   ContState = xmerl_scan:cont_state(GlobalState),
   {Result, ContState'} = get_more_bytes(ContState),
   GlobalState' = xmerl_scan:cont_state(ContState', GlobalState),
   case Result of
      [] ->
         GlobalState' = xmerl_scan:cont_state(ContState', GlobalState),
         Exception(GlobalState');
      MoreBytes ->
         {MoreBytes', Rest} = end_on_whitespace_char(MoreBytes),
         ContState'' = update_cont_state(Rest, ContState'),
         GlobalState' = xmerl_scan:cont_state(ContState'', GlobalState),
         Continue(MoreBytes', GlobalState')
   end
end.
]]></programlisting>
      </section>
      <section>
	<title>Rules Functions</title>
	<para>
<computeroutput>
{rules, ReadFun : fun(), WriteFun : fun(), LocalState} |
{rules, Table : ets()}</computeroutput>
</para>
	<para>The rules functions take care of storing scanner
	information in a rules database. User-provided rules functions
	may opt to store the information in mnesia, or perhaps in the
	user_state(LocalState).</para>

	<para>The following modes exist:</para>

	<itemizedlist>

	  <listitem><para>If the user doesn't specify an option, the
	  scanner creates an ets table, and uses built-in functions to
	  read and write data to it. When the scanner is done, the ets
	  table is deleted.</para></listitem>

	  <listitem><para>If the user specifies an ets table via the 
	<computeroutput>{rules, Table}</computeroutput> option, the
	scanner uses this table. When the scanner is done, it does
	<emphasis>not</emphasis> delete the table.</para></listitem>
	  
	  <listitem><para>If the user specifies read and write
	  functions, the scanner will use them instead.</para></listitem>

	</itemizedlist>
	
	<para>The format for the read and write functions are as
	follows:</para>


<programlisting>
<![CDATA[
WriteFun(Context, Name, Definition, ScannerState) -> NewScannerState.
ReadFun(Context, Name, ScannerState) -> Definition | undefined.
]]></programlisting>

	<para>Here is a summary of the data objects currently being
	written by the scanner:</para>
	
	<table>
	  <title>Scanner data objects</title>
	  <tgroup cols="3">
	    <thead>
	      <row>
		<entry>Context</entry>
		<entry>Key Value</entry>
		<entry>Definition</entry>
	      </row>
	    </thead>
	    <tbody>
	      <row>
		<entry>notation</entry>
		<entry>NotationName</entry>
		<entry><computeroutput>{system, SL} | {public, PIDL, SL}</computeroutput></entry>
	      </row>
	      <row>
		<entry>elem_def</entry>
		<entry>ElementName</entry>
		<entry><computeroutput>#xmlElement{content = ContentSpec}</computeroutput></entry>
	      </row>
	      <row>
		<entry>parameter_entity</entry>
		<entry>PEName</entry>
		<entry><computeroutput>PEDef</computeroutput></entry>
	      </row>
	      <row>
		<entry>entity</entry>
		<entry>EntityName</entry>
	  <entry><computeroutput>EntityDef</computeroutput></entry>
	      </row>
	    </tbody>
	  </tgroup>
	</table>
      
	
<programlisting>
<![CDATA[
ContentSpec ::= empty | any | ElemContent
ElemContent ::= {Mode, Elems}
Mode        ::= seq | choice
Elems       ::= [Elem]
Elem        ::= '#PCDATA' | Name | ElemContent | {Occurrence, Elems}
Occurrence  ::= '*' | '?' | '+'
]]></programlisting>
	<note><para>When &lt;Elem&gt; is not wrapped with
&lt;Occurrence&gt;, (Occurrence = once) is implied.</para></note>

      </section>
      <section>
	<title>Accumulator Function</title>
	<para><computeroutput>{acc_fun, fun()} | {acc_fun, fun(),
	LocalState}</computeroutput></para>

	<para>The accumulator function is called to accumulate the
	contents of an entity.When parsing very large files, it may
	not be desireable to do so.In this case, an acc function can
	be provided that simply doesn't accumulate.</para>

	<para>Note that it is possible to even modify the parsed
	entity before accumulating it, but this must be done with
	care. <computeroutput>xmerl_scan</computeroutput> performs
	post-processing of the element for namespace management. Thus,
	the element must keep its original structure for this to
	work.</para>

	<para>The acc function has the following format and
	semantics:</para>

	<programlisting>
<![CDATA[
%% default accumulating acc fun
fun(ParsedEntity, Acc, GlobalState) ->
   {[X|Acc], GlobalState}.

%% non-accumulating acc fun
fun(ParsedEntity, Acc, GlobalState) ->
   {Acc, GlobalState}.
]]></programlisting>
      </section>
      <section>
	<title>Close Function</title>

	<para>The close function is called when a document (either the
	main document or an external DTD) has been completely
	parsed. When xmerl_scan was started using
	<computeroutput>xmerl_scan:file/[1,2]</computeroutput>, the
	file will be read in full, and closed immediately, before the
	parsing starts, so when the close function is called, it will
	not need to actually close the file. In this case, the close
	function will be a good place to modify the state
	variables.</para>

	<para>Format and semantics:</para>

	<programlisting>
<![CDATA[
fun(GlobalState) ->
   GlobalState' = ....  % state variables may be altered
]]></programlisting>
      </section>

    </section>

  </section>

  <section>
    <title>XPATH</title>

    <programlisting>
<![CDATA[
xmerl_xpath:string(QueryString, #xmlElement{}) ->
	[DocEntity]

DocEntity :	#xmlElement{} 
		| #xmlAttribute{} 
		| #xmlText{} 
		| #xmlPI{}
		| #xmlComment{}
]]></programlisting>

    <para>The xmerl_xpath module does seem to handle the entire XPATH
    1.0 spec, but I haven't tested that much yet. The grammar is
    defined in
    <computeroutput>xmerl_xpath_parse.yrl</computeroutput>.  The core
    functions are defined in
    <computeroutput>xmerl_xpath_pred.erl</computeroutput>.</para>
  </section>
  <section>
    <title>Some useful shell commands for debugging the XPath parser</title>
<para>
    <command>
<![CDATA[
c(xmerl_xpath_scan).
yecc:yecc("xmerl_xpath_parse.yrl", "xmerl_xpath_parse", true, []).
c(xmerl_xpath_parse).

xmerl_xpath_parse:parse(xmerl_xpath_scan:tokens("position() > -1")).
xmerl_xpath_parse:parse(xmerl_xpath_scan:tokens("5 * 6 div 2")).
xmerl_xpath_parse:parse(xmerl_xpath_scan:tokens("5 + 6 mod 2")).
xmerl_xpath_parse:parse(xmerl_xpath_scan:tokens("5 * 6")).
xmerl_xpath_parse:parse(xmerl_xpath_scan:tokens("5 * 6")).
xmerl_xpath_parse:parse(xmerl_xpath_scan:tokens("-----6")).
xmerl_xpath_parse:parse(xmerl_xpath_scan:tokens("parent::node()")).
xmerl_xpath_parse:parse(xmerl_xpath_scan:tokens("descendant-or-self::node()")).
xmerl_xpath_parse:parse(xmerl_xpath_scan:tokens("parent::processing-instruction('foo')")).]]></command></para>
  </section>
  <section>
    <title>Erlang Data Structure Export</title>

    <para>The idea as follows:</para>

    <para>The Erlang data structure should look like this:</para>
    <programlisting>
<![CDATA[
Element:	{Tag, Attributes, Content}
Tag :		atom()
Attributes:	[{Key, Value}]
Content:	[String | Element]
String:		[char() | binary() | String]
]]></programlisting>

    <para>Some short forms are allowed:</para>
    <programlisting>
<![CDATA[
{Tag, Content}	-> {Tag, [], Content}
Tag		-> {Tag, [], []}
]]></programlisting>

    <para>Note that content lists must be flat, but strings can be
    deep.</para>

    <para>It is also allowed to include normal
    <computeroutput>#xml...</computeroutput> elements in the simple
    format.</para>

    <para><computeroutput>xmerl:export_simple(Data,
    Callback)</computeroutput> takes the above data structure and
    exports it, using the callback module
    <computeroutput>Callback</computeroutput>.</para>

    <para>The callback module should contain hook functions for all
    tags present in the data structure. The hook function must have
    the format:</para>
    <para><computeroutput> Tag(Data, Attrs, Parents, E)
    </computeroutput></para>

    <para>where E is an <computeroutput>#xmlElement{}</computeroutput>
    record  (see <computeroutput>xmerl.hrl</computeroutput>).</para>

    <para>Attrs is converted from the simple <computeroutput>[{Key,
    Value}]</computeroutput> to
    <computeroutput>[#xmlAttribute{}]</computeroutput></para>

    <para>Parents is a list of <computeroutput>[{ParentTag,
    ParentTagPosition}]</computeroutput>.</para>

    <para>The hook function should return either the Data to be
    exported, or the tuple <computeroutput>{'#xml-redefine#',
    NewStructure}</computeroutput>, where
    <computeroutput>NewStructure</computeroutput> is an element (which
    can be simple), or a (simple-) content list wrapped in a 1-tuple
    as <computeroutput>{NewContent}</computeroutput>.</para>

    <para>The callback module can inherit definitions from other
    callback modules, through the required function
    <computeroutput>'#xml-interitance#() ->
    [ModuleName]</computeroutput>. </para>

    <para>As long as a tag is represented in one of the callback
    modules, things will work. It is of course also possible to
    redefine a tag.</para>
      <section>
      <title>XSLT like transforms</title>
	<para>See separate document <ulink url="xmerl_xs.html" >xmerl_xs.html
	</ulink></para>.
      </section>
  </section>

</article>