aboutsummaryrefslogblamecommitdiffstats
path: root/system/doc/design_principles/release_handling.xml
blob: 1d62c242c06e438fb4862e73b320d2dfe7c76a6e (plain) (tree)
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
157
158
159
160
161
162
163
164
165
166
167
168
169
170
171
172
173
174
175
176
177
178
179
180
181
182
183
184
185
186
187
188
189
190
191
192
193
194
195
196
197
198
199
200
201
202
203
204
205
206
207
208
209
210
211
212
213
214
215
216
217
218
219
220
221
222
223
224
225
226
227
228
229
230
231
232
233
234
235
236
237
238
239
240
241
242
243
244
245
246
247
248
249
250
251
252
253
254
255
256
257
258
259
260
261
262
263
264
265
266
267
268
269
270
271
272
273
274
275
276
277
278
279
280
281
282
283
284
285
286
287
288
289
290
291
292
293
294
295
296
297
298
299
300
301
302
303
304
305
306
307
308
309
310
311
312
313
314
315
316
317
318
319
320
321
322
323
324
325
326
327
328
329
330
331
332
333
334
335
336
337
338
339
340
341
342
343
344
345
346
347
348
349
350
351
352
353
354
355
356
357
358
359
360
361
362
363
364
365
366
367
368
369
370
371
372
373
374
375
376
377
378
379
380
381
382
383
384
385
386
387
388
389
390
391
392
393
394
395
396
397
398
399
400
401
402
403
404
405
406
407
408
409
410
411
412
413
414
415
416
417
418
419
420
421
422
423
424
425
426
427
428
429
430
431
432
433
434
435
436
437
438
439
440
441
442
443
444
445
446
447
448
449
450
451
452
453
454
455
456
457
458
459
460
461
462
463
464
465
466
467
468
469
470
471
472
473
474
475
476
477
478
479
480
481
482
483
484
485
486
487
488
489
490
491
492
493
494
495
496
497
498
499
500
501
502
503
504
505
506
507
508
509
510
511
512
513
514
515
516
517
518
519
520
521
522
523
524
525
526
527
528
529
530
531
532
533
534
535
536
537
538
539
540
541
542
543
544
545
546
547
548
549
550
551
552
553
554
555
556
557
558
559
560
561
562
563
564
565
566
567
568
569
570
571
572
573
574
575
576
577
578
579
580
581
582
583
584
585
586
587
588
589
590
591
592
593
594
595
596
597
598
599
600
601
602
603
604
605
606
607
608
609
610
611
612
613
614
615
616
617
618
619
620
621
622
623
624
625
626
627
628
629
630
631
632
633
634
635
636
637
638
639
640
641
642
643
644
645
646
647
648
649
650
651
652
653
654
655
656
657
658
659
660
661
662
663
664
665
666
667

























































































































































































































































































































































































































































































































































































































































































                                                                                                                                     
<?xml version="1.0" encoding="latin1" ?>
<!DOCTYPE chapter SYSTEM "chapter.dtd">

<chapter>
  <header>
    <copyright>
      <year>2003</year><year>2009</year>
      <holder>Ericsson AB. All Rights Reserved.</holder>
    </copyright>
    <legalnotice>
      The contents of this file are subject to the Erlang Public License,
      Version 1.1, (the "License"); you may not use this file except in
      compliance with the License. You should have received a copy of the
      Erlang Public License along with this software. If not, it can be
      retrieved online at http://www.erlang.org/.
    
      Software distributed under the License is distributed on an "AS IS"
      basis, WITHOUT WARRANTY OF ANY KIND, either express or implied. See
      the License for the specific language governing rights and limitations
      under the License.
    
    </legalnotice>

    <title>Release Handling</title>
    <prepared></prepared>
    <docno></docno>
    <date></date>
    <rev></rev>
    <file>release_handling.xml</file>
  </header>

  <section>
    <title>Release Handling Principles</title>
    <p>An important feature of the Erlang programming language is
      the ability to change module code in run-time, <em>code replacement</em>, as described in <em>Erlang Reference Manual</em>.</p>
    <p>Based on this feature, the OTP application SASL provides a
      framework for upgrading and downgrading between different
      versions of an entire release in run-time. This is what we call
      <em>release handling</em>.</p>
    <p>The framework consists of off-line support (<c>systools</c>) for
      generating scripts and building release packages, and on-line
      support (<c>release_handler</c>) for unpacking and installing
      release packages.</p>
    <p>Note that the minimal system based on Erlang/OTP, enabling
      release handling, thus consists of Kernel, STDLIB and SASL.</p>
    <list type="ordered">
      <item>
        <p>A release is created as described in the previous chapter
          <seealso marker="release_structure">Releases</seealso>.
          The release is transferred to and installed at target
          environment. Refer to <em>System Principles</em> for
          information of how to install the first target system.</p>
      </item>
      <item>
        <p>Modifications, for example error corrections, are made to
          the code in the development environment.</p>
      </item>
      <item>
        <p>At some point it is time to make a new version of release.
          The relevant <c>.app</c> files are updated and a new
          <c>.rel</c> file is written.</p>
      </item>
      <item>
        <p>For each modified application, an
          <seealso marker="#appup">application upgrade file</seealso>,
          <c>.appup</c>, is created. In this file, it is described how
          to upgrade and/or downgrade between the old and new version of
          the application.</p>
      </item>
      <item>
        <p>Based on the <c>.appup</c> files, a
          <seealso marker="#relup">release upgrade file</seealso> called
          <c>relup</c>, is created. This file describes how to upgrade
          and/or downgrade between the old and new version of
          the entire release.</p>
      </item>
      <item>
        <p>A new release package is made and transferred to
          the target system.</p>
      </item>
      <item>
        <p>The new release package is unpacked using the release
          handler.</p>
      </item>
      <item>
        <p>The new version of the release is installed, also using
          the release handler. This is done by evaluating
          the instructions in <c>relup</c>. Modules may be added,
          deleted or re-loaded, applications may be started, stopped or
          re-started etc. In some cases, it is even necessary to restart
          the entire emulator.</p>
        <p>If the installation fails, the system may be rebooted.
          The old release version is then automatically used.</p>
      </item>
      <item>
        <p>If the installation succeeds, the new version is made
          the default version, which should now be used in case of a
          system reboot.</p>
      </item>
    </list>
    <p>The next chapter, <seealso marker="appup_cookbook">Appup Cookbook</seealso>, contains examples of <c>.appup</c> files
      for typical cases of upgrades/downgrades that are normally easy
      to handle in run-time. However, there are a many aspects that can
      make release handling complicated. To name a few examples:</p>
    <list type="bulleted">
      <item>
        <p>Complicated or circular dependencies can make it difficult
          or even impossible to decide in which order things must be
          done without risking run-time errors during an upgrade or
          downgrade. Dependencies may be:</p>
        <list type="bulleted">
          <item>between nodes,</item>
          <item>between processes, and</item>
          <item>between modules.</item>
        </list>
      </item>
      <item>
        <p>During release handling, non-affected processes continue
          normal execution. This may lead to timeouts or other problems.
          For example, new processes created in the time window between
          suspending processes using a certain module and loading a new
          version of this module, may execute old code.</p>
      </item>
    </list>
    <p>It is therefore recommended that code is changed in as small
      steps as possible, and always kept backwards compatible.</p>
  </section>

  <section>
    <marker id="req"></marker>
    <title>Requirements</title>
    <p>For release handling to work properly, the runtime system needs
      to have knowledge about which release it is currently running. It
      must also be able to change (in run-time) which boot script and
      system configuration file should be used if the system is
      rebooted, for example by <c>heart</c> after a failure.
      Therefore, Erlang must be started as an embedded system, see
      <em>Embedded System</em> for information on how to do this.</p>
    <p>For system reboots to work properly, it is also required that
      the system is started with heart beat monitoring, see
      <c>erl(1)</c> and <c>heart(3)</c>.</p>
    <p>Other requirements:</p>
    <list type="bulleted">
      <item>
        <p>The boot script included in a release package must be
          generated from the same <c>.rel</c> file as the release
          package itself.</p>
        <p>Information about applications are fetched from the script
          when an upgrade or downgrade is performed.</p>
      </item>
      <item>
        <p>The system must be configured using one and only one system
          configuration file, called <c>sys.config</c>.</p>
        <p>If found, this file is automatically included when a release
          package is created.</p>
      </item>
      <item>
        <p>All versions of a release, except the first one, must
          contain a <c>relup</c> file.</p>
        <p>If found, this file is automatically included when a release
          package is created.</p>
      </item>
    </list>
  </section>

  <section>
    <title>Distributed Systems</title>
    <p>If the system consists of several Erlang nodes, each node may use
      its own version of the release. The release handler is a locally
      registered process and must be called at each node where an
      upgrade or downgrade is required. There is a release handling
      instruction that can be used to synchronize the release handler
      processes at a number of nodes: <c>sync_nodes</c>. See
      <c>appup(4)</c>.</p>
  </section>

  <section>
    <marker id="instr"></marker>
    <title>Release Handling Instructions</title>
    <p>OTP supports a set of <em>release handling instructions</em>
      that is used when creating <c>.appup</c> files. The release
      handler understands a subset of these, the <em>low-level</em>
      instructions. To make it easier for the user, there are also a
      number of <em>high-level</em> instructions, which are translated
      to low-level instructions by <c>systools:make_relup</c>.</p>
    <p>Here, some of the most frequently used instructions are
      described. The complete list of instructions is found in
      <c>appup(4)</c>.</p>
    <p>First, some definitions:</p>
    <taglist>
      <tag><em>Residence module</em></tag>
      <item>
        <p>The module where a process has its tail-recursive loop
          function(s). If the tail-recursive loop functions are
          implemented in several modules, all those modules are residence
          modules for the process.</p>
      </item>
      <tag><em>Functional module</em></tag>
      <item>
        <p>A module which is not a residence module for any process.</p>
      </item>
    </taglist>
    <p>Note that for a process implemented using an OTP behaviour,
      the behaviour module is the residence module for that process.
      The callback module is a functional module.</p>

    <section>
      <title>load_module</title>
      <p>If a simple extension has been made to a functional module, it
        is sufficient to simply load the new version of the module into
        the system, and remove the old version. This is called
        <em>simple code replacement</em> and for this the following
        instruction is used:</p>
      <code type="none">
{load_module, Module}</code>
    </section>

    <section>
      <title>update</title>
      <p>If a more complex change has been made, for example a change
        to the format of the internal state of a gen_server, simple code
        replacement is not sufficient. Instead it is necessary to
        suspend the processes using the module (to avoid that they try
        to handle any requests before the code replacement is
        completed), ask them to transform the internal state format and
        switch to the new version of the module, remove the old version
        and last, resume the processes. This is called <em>synchronized code replacement</em> and for this the following instructions
        are used:</p>
      <code type="none">
{update, Module, {advanced, Extra}}
{update, Module, supervisor}</code>
      <p><c>update</c> with argument <c>{advanced,Extra}</c> is used
        when changing the internal state of a behaviour as described
        above. It will cause behaviour processes to call the callback
        function <c>code_change</c>, passing the term <c>Extra</c> and
        some other information as arguments. See the man pages for
        the respective behaviours and
        <seealso marker="appup_cookbook#int_state">Appup Cookbook</seealso>.</p>
      <p><c>update</c> with argument <c>supervisor</c> is used when
        changing the start specification of a supervisor. See
        <seealso marker="appup_cookbook#sup">Appup Cookbook</seealso>.</p>
      <p>The release handler finds the processes <em>using</em> a module
        to update by traversing the supervision tree of each running
        application and checking all the child specifications:</p>
      <code type="none">
{Id, StartFunc, Restart, Shutdown, Type, Modules}</code>
      <p>A process is using a module if the name is listed in
        <c>Modules</c> in the child specification for the process.</p>
      <p>If <c>Modules=dynamic</c>, which is the case for event
        managers, the event manager process informs the release handler
        about the list of currently installed event handlers (gen_fsm)
        and it is checked if the module name is in this list instead.</p>
      <p>The release handler suspends, asks for code change, and
        resumes processes by calling the functions
        <c>sys:suspend/1,2</c>, <c>sys:change_code/4,5</c> and
        <c>sys:resume/1,2</c> respectively.</p>
    </section>

    <section>
      <title>add_module and delete_module</title>
      <p>If a new module is introduced, the following instruction is
        used:</p>
      <code type="none">
{add_module, Module}</code>
      <p>The instruction loads the module and is absolutely necessary
        when running Erlang in embedded mode. It is not strictly
        required when running Erlang in interactive (default) mode,
        since the code server automatically searches for and loads
        unloaded modules.</p>
      <p>The opposite of <c>add_module</c> is <c>delete_module</c> which
        unloads a module:</p>
      <code type="none">
{delete_module, Module}</code>
      <p>Note that any process, in any application, with <c>Module</c>
        as residence module, is killed when the instruction is
        evaluated. The user should therefore ensure that all such
        processes are terminated before deleting the module, to avoid
        a possible situation with failing supervisor restarts.</p>
    </section>

    <section>
      <title>Application Instructions</title>
      <p>Instruction for adding an application:</p>
      <code type="none">
{add_application, Application}</code>
      <p>Adding an application means that the modules defined by
        the <c>modules</c> key in the <c>.app</c> file are loaded using
        a number of <c>add_module</c> instructions, then the application
        is started.</p>
      <p>Instruction for removing an application:</p>
      <code type="none">
{remove_application, Application}</code>
      <p>Removing an application means that the application is stopped,
        the modules are unloaded using a number of <c>delete_module</c>
        instructions and then the application specification is unloaded
        from the application controller.</p>
      <p>Instruction for removing an application:</p>
      <code type="none">
{restart_application, Application}</code>
      <p>Restarting an application means that the application is stopped
        and then started again similar to using the instructions
        <c>remove_application</c> and <c>add_application</c> in
        sequence.</p>
    </section>

    <section>
      <title>apply (low-level)</title>
      <p>To call an arbitrary function from the release handler,
        the following instruction is used:</p>
      <code type="none">
{apply, {M, F, A}}</code>
      <p>The release handler will evalute <c>apply(M, F, A)</c>.</p>
    </section>

    <section>
      <title>restart_new_emulator (low-level)</title>
      <p>This instruction is used when changing to a new emulator
        version, or if a system reboot is needed for some other reason.
        Requires that the system is started with heart beat
        monitoring, see <c>erl(1)</c> and <c>heart(3)</c>.</p>
      <p>When the release handler encounters the instruction, it shuts
        down the current emulator by calling <c>init:reboot()</c>, see
        <c>init(3)</c>. All processes are terminated gracefully and
        the system can then be rebooted by the heart program, using
        the new release version. This new version must still be made
        permanent when the new emulator is up and running. Otherwise,
        the old version is used in case of a new system reboot.</p>
      <p>On UNIX, the release handler tells the heart program which
        command to use to reboot the system. Note that the environment
        variable <c>HEART_COMMAND</c>, normally used by the heart
        program, in this case is ignored. The command instead defaults
        to <c>$ROOT/bin/start</c>. Another command can be set
        by using the SASL configuration parameter <c>start_prg</c>, see
        <c>sasl(6)</c>.</p>
    </section>
  </section>

  <section>
    <marker id="appup"></marker>
    <title>Application Upgrade File</title>
    <p>To define how to upgrade/downgrade between the current version
      and previous versions of an application, we create an
      <em>application upgrade file</em>, or in short <c>.appup</c> file.
      The file should be called <c>Application.appup</c>, where
      <c>Application</c> is the name of the application:</p>
    <code type="none">
{Vsn,
 [{UpFromVsn1, InstructionsU1},
  ...,
  {UpFromVsnK, InstructionsUK}],
 [{DownToVsn1, InstructionsD1},
  ...,
  {DownToVsnK, InstructionsDK}]}.</code>
    <p><c>Vsn</c>, a string, is the current version of the application,
      as defined in the <c>.app</c> file. Each <c>UpFromVsn</c>
      is a previous version of the application to upgrade from, and each
      <c>DownToVsn</c> is a previous version of the application to
      downgrade to. Each <c>Instructions</c> is a list of release
      handling instructions.</p>
    <p>The syntax and contents of the <c>appup</c> file are described
      in detail in <c>appup(4)</c>.</p>
    <p>In the chapter <seealso marker="appup_cookbook">Appup Cookbook</seealso>, examples of <c>.appup</c> files for typical
      upgrade/downgrade cases are given.</p>
    <p>Example: Consider the release <c>ch_rel-1</c> from
      the <seealso marker="release_structure#ch_rel">Releases</seealso>
      chapter. Assume we want to add a function <c>available/0</c> to
      the server <c>ch3</c> which returns the number of available
      channels:</p>
    <p>(Hint: When trying out the example, make the changes in a copy of
      the original directory, so that the first versions are still
      available.)</p>
    <code type="none">
-module(ch3).
-behaviour(gen_server).

-export([start_link/0]).
-export([alloc/0, free/1]).
-export([available/0]).
-export([init/1, handle_call/3, handle_cast/2]).

start_link() ->
    gen_server:start_link({local, ch3}, ch3, [], []).

alloc() ->
    gen_server:call(ch3, alloc).

free(Ch) ->
    gen_server:cast(ch3, {free, Ch}).

available() ->
    gen_server:call(ch3, available).

init(_Args) ->
    {ok, channels()}.

handle_call(alloc, _From, Chs) ->
    {Ch, Chs2} = alloc(Chs),
    {reply, Ch, Chs2};
handle_call(available, _From, Chs) ->
    N = available(Chs),
    {reply, N, Chs}.

handle_cast({free, Ch}, Chs) ->
    Chs2 = free(Ch, Chs),
    {noreply, Chs2}.</code>
    <p>A new version of the <c>ch_app.app</c> file must now be created,
      where the version is updated:</p>
    <code type="none">
{application, ch_app,
 [{description, "Channel allocator"},
  {vsn, "2"},
  {modules, [ch_app, ch_sup, ch3]},
  {registered, [ch3]},
  {applications, [kernel, stdlib, sasl]},
  {mod, {ch_app,[]}}
 ]}.</code>
    <p>To upgrade <c>ch_app</c> from <c>"1"</c> to <c>"2"</c> (and
      to downgrade from <c>"2"</c> to <c>"1"</c>), we simply need to
      load the new (old) version of the <c>ch3</c> callback module.
      We create the application upgrade file <c>ch_app.appup</c> in
      the <c>ebin</c> directory:</p>
    <code type="none">
{"2",
 [{"1", [{load_module, ch3}]}],
 [{"1", [{load_module, ch3}]}]
}.</code>
  </section>

  <section>
    <marker id="relup"></marker>
    <title>Release Upgrade File</title>
    <p>To define how to upgrade/downgrade between the new version and
      previous versions of a release, we create a <em>release upgrade file</em>, or in short <c>relup</c> file.</p>
    <p>This file does not need to be created manually, it can be
      generated by <c>systools:make_relup/3,4</c>. The relevant versions
      of the <c>.rel</c> file, <c>.app</c> files and <c>.appup</c> files
      are used as input. It is deducted which applications should be
      added and deleted, and which applications that need to be upgraded
      and/or downgraded. The instructions for this is fetched from
      the <c>.appup</c> files and transformed into a single list of
      low-level instructions in the right order.</p>
    <p>If the <c>relup</c> file is relatively simple, it can be created
      manually. Remember that it should only contain low-level
      instructions.</p>
    <p>The syntax and contents of the release upgrade file are
      described in detail in <c>relup(4)</c>.</p>
    <p>Example, continued from the previous section. We have a new
      version "2" of <c>ch_app</c> and an <c>.appup</c> file. We also
      need a new version of the <c>.rel</c> file. This time the file is
      called <c>ch_rel-2.rel</c> and the release version string is
      changed changed from "A" to "B":</p>
    <code type="none">
{release,
 {"ch_rel", "B"},
 {erts, "5.3"},
 [{kernel, "2.9"},
  {stdlib, "1.12"},
  {sasl, "1.10"},
  {ch_app, "2"}]
}.</code>
    <p>Now the <c>relup</c> file can be generated:</p>
    <pre>
1> <input>systools:make_relup("ch_rel-2", ["ch_rel-1"], ["ch_rel-1"]).</input>
ok</pre>
    <p>This will generate a <c>relup</c> file with instructions for
      how to upgrade from version "A" ("ch_rel-1") to version "B"
      ("ch_rel-2") and how to downgrade from version "B" to version "A".</p>
    <p>Note that both the old and new versions of the <c>.app</c> and
      <c>.rel</c> files must be in the code path, as well as
      the <c>.appup</c> and (new) <c>.beam</c> files. It is possible
      to extend the code path by using the option <c>path</c>:</p>
    <pre>
1> <input>systools:make_relup("ch_rel-2", ["ch_rel-1"], ["ch_rel-1"],</input>
<input>[{path,["../ch_rel-1",</input>
<input>"../ch_rel-1/lib/ch_app-1/ebin"]}]).</input>
ok</pre>
  </section>

  <section>
    <marker id="rel_handler"></marker>
    <title>Installing a Release</title>
    <p>When we have made a new version of a release, a release package
      can be created with this new version and transferred to the target
      environment.</p>
    <p>To install the new version of the release in run-time,
      the <em>release handler</em> is used. This is a process belonging
      to the SASL application, that handles unpacking, installation,
      and removal of release packages. It is interfaced through
      the module <c>release_handler</c>, which is described in detail in
      <c>release_handler(3)</c>.</p>
    <p>Assuming there is a target system up and running with
      installation root directory <c>$ROOT</c>, the release package with
      the new version of the release should be copied to
      <c>$ROOT/releases</c>.</p>
    <p>The first action is to <em>unpack</em> the release package,
      the files are then extracted from the package:</p>
    <code type="none">
release_handler:unpack_release(ReleaseName) => {ok, Vsn}</code>
    <p><c>ReleaseName</c> is the name of the release package except
      the <c>.tar.gz</c> extension. <c>Vsn</c> is the version of
      the unpacked release, as defined in its <c>.rel</c> file.</p>
    <p>A directory <c>$ROOT/lib/releases/Vsn</c> will be created, where
      the <c>.rel</c> file, the boot script <c>start.boot</c>,
      the system configuration file <c>sys.config</c> and <c>relup</c>
      are placed. For applications with new version numbers,
      the application directories will be placed under <c>$ROOT/lib</c>.
      Unchanged applications are not affected.</p>
    <p>An unpacked release can be <em>installed</em>. The release
      handler then evaluates the instructions in <c>relup</c>, step by
      step:</p>
    <code type="none">
release_handler:install_release(Vsn) => {ok, FromVsn, []}</code>
    <p>If an error occurs during the installation, the system is
      rebooted using the old version of the release. If installation
      succeeds, the system is afterwards using the new version of
      the release, but should anything happen and the system is
      rebooted, it would start using the previous version again. To be
      made the default version, the newly installed release must be made
      <em>permanent</em>, which means the previous version becomes
      <em>old</em>:</p>
    <code type="none">
release_handler:make_permanent(Vsn) => ok</code>
    <p>The system keeps information about which versions are old and
      permanent in the files <c>$ROOT/releases/RELEASES</c> and
      <c>$ROOT/releases/start_erl.data</c>.</p>
    <p>To downgrade from <c>Vsn</c> to <c>FromVsn</c>,
      <c>install_release</c> must be called again:</p>
    <code type="none">
release_handler:install_release(FromVsn) => {ok, Vsn, []}</code>
    <p>An installed, but not permanent, release can be <em>removed</em>.
      Information about the release is then deleted from
      <c>$ROOT/releases/RELEASES</c> and the release specific code,
      that is the new application directories and
      the <c>$ROOT/releases/Vsn</c> directory, are removed.</p>
    <code type="none">
release_handler:remove_release(Vsn) => ok</code>
    <p>Example, continued from the previous sections:</p>
    <p>1) Create a target system as described in <em>System Principles</em> of the first version <c>"A"</c> of <c>ch_rel</c>
      from
      the <seealso marker="release_structure#ch_rel">Releases</seealso>
      chapter. This time <c>sys.config</c> must be included in
      the release package. If no configuration is needed, the file
      should contain the empty list:</p>
    <code type="none">
[].</code>
    <p>2) Start the system as a simple target system. Note that in
      reality, it should be started as an embedded system. However,
      using <c>erl</c> with the correct boot script and <c>.config</c>
      file is enough for illustration purposes:</p>
    <pre>
% <input>cd $ROOT</input>
% <input>bin/erl -boot $ROOT/releases/A/start -config $ROOT/releases/A/sys</input>
...</pre>
    <p><c>$ROOT</c> is the installation directory of the target system.</p>
    <p>3) In another Erlang shell, generate start scripts and create a
      release package for the new version <c>"B"</c>. Remember to
      include (a possible updated) <c>sys.config</c> and
      the <c>relup</c> file, see <seealso marker="#relup">Release Upgrade File</seealso> above.</p>
    <pre>
1> <input>systools:make_script("ch_rel-2").</input>
ok
2> <input>systools:make_tar("ch_rel-2").</input>
ok</pre>
    <p>The new release package now contains version "2" of <c>ch_app</c>
      and the <c>relup</c> file as well:</p>
    <code type="none">
% tar tf ch_rel-2.tar 
lib/kernel-2.9/ebin/kernel.app
lib/kernel-2.9/ebin/application.beam
...
lib/stdlib-1.12/ebin/stdlib.app
lib/stdlib-1.12/ebin/beam_lib.beam
...      
lib/sasl-1.10/ebin/sasl.app
lib/sasl-1.10/ebin/sasl.beam
...
lib/ch_app-2/ebin/ch_app.app
lib/ch_app-2/ebin/ch_app.beam
lib/ch_app-2/ebin/ch_sup.beam
lib/ch_app-2/ebin/ch3.beam
releases/B/start.boot
releases/B/relup
releases/B/sys.config
releases/ch_rel-2.rel</code>
    <p>4) Copy the release package <c>ch_rel-2.tar.gz</c> to
      the <c>$ROOT/releases</c> directory.</p>
    <p>5) In the running target system, unpack the release package:</p>
    <pre>
1> <input>release_handler:unpack_release("ch_rel-2").</input>
{ok,"B"}</pre>
    <p>The new application version <c>ch_app-2</c> is installed under
      <c>$ROOT/lib</c> next to <c>ch_app-1</c>. The <c>kernel</c>,
      <c>stdlib</c> and <c>sasl</c> directories are not affected, as
      they have not changed.</p>
    <p>Under <c>$ROOT/releases</c>, a new directory <c>B</c> is created,
      containing <c>ch_rel-2.rel</c>, <c>start.boot</c>,
      <c>sys.config</c> and <c>relup</c>.</p>
    <p>6) Check if the function <c>ch3:available/0</c> is available:</p>
    <pre>
2> <input>ch3:available().</input>
** exception error: undefined function ch3:available/0</pre>
    <p>7) Install the new release. The instructions in
      <c>$ROOT/releases/B/relup</c> are executed one by one, resulting
      in the new version of <c>ch3</c> being loaded. The function
      <c>ch3:available/0</c> is now available:</p>
    <pre>
3> <input>release_handler:install_release("B").</input>
{ok,"A",[]}
4> <input>ch3:available().</input>
3
5> <input>code:which(ch3).</input>
".../lib/ch_app-2/ebin/ch3.beam"
6> <input>code:which(ch_sup).</input>
".../lib/ch_app-1/ebin/ch_sup.beam"</pre>
    <p>Note that processes in <c>ch_app</c> for which code have not
      been updated, for example the supervisor, are still evaluating
      code from <c>ch_app-1</c>.</p>
    <p>8) If the target system is now rebooted, it will use version "A"
      again. The "B" version must be made permanent, in order to be
      used when the system is rebooted.</p>
    <pre>
7> <input>release_handler:make_permanent("B").</input>
ok</pre>
  </section>

  <section>
    <marker id="sys"></marker>
    <title>Updating Application Specifications</title>
    <p>When a new version of a release is installed, the application
      specifications are automatically updated for all loaded
      applications.</p>
    <note>
      <p>The information about the new application specifications are
        fetched from the boot script included in the release package.
        It is therefore important that the boot script is generated from
        the same <c>.rel</c> file as is used to build the release
        package itself.</p>
    </note>
    <p>Specifically, the application configuration parameters are
      automatically updated according to (in increasing priority
      order):</p>
    <list type="ordered">
      <item>The data in the boot script, fetched from the new
       application resource file <c>App.app</c></item>
      <item>The new <c>sys.config</c></item>
      <item>Command line arguments <c>-App Par Val</c></item>
    </list>
    <p>This means that parameter values set in the other system
      configuration files, as well as values set using
      <c>application:set_env/3</c>, are disregarded.</p>
    <p>When an installed release is made permanent, the system process
      <c>init</c> is set to point out the new <c>sys.config</c>.</p>
    <p>After the installation, the application controller will compare
      the old and new configuration parameters for all running
      applications and call the callback function:</p>
    <code type="none">
Module:config_change(Changed, New, Removed)</code>
    <p><c>Module</c> is the application callback module as defined by
      the <c>mod</c> key in the <c>.app</c> file. <c>Changed</c> and
      <c>New</c> are lists of <c>{Par,Val}</c> for all changed and
      added configuration parameters, respectively. <c>Removed</c> is
      a list of all parameters <c>Par</c> that have been removed.</p>
    <p>The function is optional and may be omitted when implementing an
      application callback module.</p>
  </section>
</chapter>