diff options
733 files changed, 33445 insertions, 35008 deletions
diff --git a/HOWTO/BENCHMARKS.md b/HOWTO/BENCHMARKS.md index b3df4638ea..28a590dfd2 100644 --- a/HOWTO/BENCHMARKS.md +++ b/HOWTO/BENCHMARKS.md @@ -18,13 +18,13 @@ benchmarks you have to [install the tests][]. To get a listing of all benchmarks you have available call `ts:benchmarks()`. To run all benchmarks call `ts:bench()`. This will run all benchmarks using -the emulator which is in you `$PATH` (Note that this does not have to be the +the emulator which is in your `$PATH` (Note that this does not have to be the same as from which the benchmarks were built from). All the results of the benchmarks are put in a folder in `$TESTROOT/test_server/` called `YYYY_MO_DDTHH_MI_SS`. Each benchmark is run multiple times and the data for all runs is collected in -the files within the benchmark folder. All benchmarks are written so that a +the files within the benchmark folder. All benchmarks are written so that higher values are better. Writing benchmarks diff --git a/HOWTO/DTRACE.md b/HOWTO/DTRACE.md index b719c68c59..8fa2fd9d50 100644 --- a/HOWTO/DTRACE.md +++ b/HOWTO/DTRACE.md @@ -69,7 +69,8 @@ following may be executed in a different Pthread: Example output from `lib/runtime_tools/examples/efile_drv.d` while executing `file:rename("old-name", "new-name")`: - efile_drv enter tag={3,84} user tag some-user-tag | RENAME (12) | args: old-name new-name , 0 0 (port #Port<0.59>) + efile_drv enter tag={3,84} user tag some-user-tag | RENAME (12) | args: old-name new-name ,\ + 0 0 (port #Port<0.59>) async I/O worker tag={3,83} | RENAME (12) | efile_drv-int_entry async I/O worker tag={3,83} | RENAME (12) | efile_drv-int_return efile_drv return tag={3,83} user tag | RENAME (12) | errno 2 diff --git a/aclocal.m4 b/aclocal.m4 index b1cf1fe404..9578cd35c4 100644 --- a/aclocal.m4 +++ b/aclocal.m4 @@ -740,11 +740,16 @@ dnl Try to find POSIX threads dnl The usual pthread lib... AC_CHECK_LIB(pthread, pthread_create, THR_LIBS="-lpthread") -dnl FreeBSD has pthreads in special c library, c_r... +dnl Very old versions of FreeBSD have pthreads in special c library, c_r... if test "x$THR_LIBS" = "x"; then AC_CHECK_LIB(c_r, pthread_create, THR_LIBS="-lc_r") fi +dnl QNX has pthreads in standard C library + if test "x$THR_LIBS" = "x"; then + AC_CHECK_FUNC(pthread_create, THR_LIBS="none_needed") + fi + dnl On ofs1 the '-pthread' switch should be used if test "x$THR_LIBS" = "x"; then AC_MSG_CHECKING([if the '-pthread' switch can be used]) @@ -765,6 +770,9 @@ dnl On ofs1 the '-pthread' switch should be used if test "x$THR_LIBS" != "x"; then THR_DEFS="$THR_DEFS -D_THREAD_SAFE -D_REENTRANT -DPOSIX_THREADS" THR_LIB_NAME=pthread + if test "x$THR_LIBS" = "xnone_needed"; then + THR_LIBS= + fi case $host_os in solaris*) THR_DEFS="$THR_DEFS -D_POSIX_PTHREAD_SEMANTICS" ;; diff --git a/bootstrap/bin/start.boot b/bootstrap/bin/start.boot Binary files differindex c6ebc852a2..a98b0eda7d 100644 --- a/bootstrap/bin/start.boot +++ b/bootstrap/bin/start.boot diff --git a/bootstrap/bin/start_clean.boot b/bootstrap/bin/start_clean.boot Binary files differindex c6ebc852a2..a98b0eda7d 100644 --- a/bootstrap/bin/start_clean.boot +++ b/bootstrap/bin/start_clean.boot diff --git a/bootstrap/lib/compiler/ebin/beam_asm.beam b/bootstrap/lib/compiler/ebin/beam_asm.beam Binary files differindex e09584fdeb..fd36250580 100644 --- a/bootstrap/lib/compiler/ebin/beam_asm.beam +++ b/bootstrap/lib/compiler/ebin/beam_asm.beam diff --git a/bootstrap/lib/compiler/ebin/beam_block.beam b/bootstrap/lib/compiler/ebin/beam_block.beam Binary files differindex 0b93439386..72cce89b9d 100644 --- a/bootstrap/lib/compiler/ebin/beam_block.beam +++ b/bootstrap/lib/compiler/ebin/beam_block.beam diff --git a/bootstrap/lib/compiler/ebin/beam_bsm.beam b/bootstrap/lib/compiler/ebin/beam_bsm.beam Binary files differindex e0ec059999..e0d595bea0 100644 --- a/bootstrap/lib/compiler/ebin/beam_bsm.beam +++ b/bootstrap/lib/compiler/ebin/beam_bsm.beam diff --git a/bootstrap/lib/compiler/ebin/beam_disasm.beam b/bootstrap/lib/compiler/ebin/beam_disasm.beam Binary files differindex b80b543679..2ecba86d56 100644 --- a/bootstrap/lib/compiler/ebin/beam_disasm.beam +++ b/bootstrap/lib/compiler/ebin/beam_disasm.beam diff --git a/bootstrap/lib/compiler/ebin/beam_jump.beam b/bootstrap/lib/compiler/ebin/beam_jump.beam Binary files differindex 07963ae01e..c296bdd18d 100644 --- a/bootstrap/lib/compiler/ebin/beam_jump.beam +++ b/bootstrap/lib/compiler/ebin/beam_jump.beam diff --git a/bootstrap/lib/compiler/ebin/beam_utils.beam b/bootstrap/lib/compiler/ebin/beam_utils.beam Binary files differindex 0c3ce5b710..1aa160b998 100644 --- a/bootstrap/lib/compiler/ebin/beam_utils.beam +++ b/bootstrap/lib/compiler/ebin/beam_utils.beam diff --git a/bootstrap/lib/compiler/ebin/compile.beam b/bootstrap/lib/compiler/ebin/compile.beam Binary files differindex ae8f3ee903..d2e93214d1 100644 --- a/bootstrap/lib/compiler/ebin/compile.beam +++ b/bootstrap/lib/compiler/ebin/compile.beam diff --git a/bootstrap/lib/compiler/ebin/core_parse.beam b/bootstrap/lib/compiler/ebin/core_parse.beam Binary files differindex 0993838799..ac2a161ced 100644 --- a/bootstrap/lib/compiler/ebin/core_parse.beam +++ b/bootstrap/lib/compiler/ebin/core_parse.beam diff --git a/bootstrap/lib/compiler/ebin/sys_pre_expand.beam b/bootstrap/lib/compiler/ebin/sys_pre_expand.beam Binary files differindex c08dd86a80..4c8547ea27 100644 --- a/bootstrap/lib/compiler/ebin/sys_pre_expand.beam +++ b/bootstrap/lib/compiler/ebin/sys_pre_expand.beam diff --git a/bootstrap/lib/compiler/ebin/v3_codegen.beam b/bootstrap/lib/compiler/ebin/v3_codegen.beam Binary files differindex 8782fb01a8..8071b9b616 100644 --- a/bootstrap/lib/compiler/ebin/v3_codegen.beam +++ b/bootstrap/lib/compiler/ebin/v3_codegen.beam diff --git a/bootstrap/lib/compiler/ebin/v3_kernel.beam b/bootstrap/lib/compiler/ebin/v3_kernel.beam Binary files differindex 3801b77852..97a9a79260 100644 --- a/bootstrap/lib/compiler/ebin/v3_kernel.beam +++ b/bootstrap/lib/compiler/ebin/v3_kernel.beam diff --git a/bootstrap/lib/kernel/ebin/application_controller.beam b/bootstrap/lib/kernel/ebin/application_controller.beam Binary files differindex 7dc72beba6..28512de9f1 100644 --- a/bootstrap/lib/kernel/ebin/application_controller.beam +++ b/bootstrap/lib/kernel/ebin/application_controller.beam diff --git a/bootstrap/lib/kernel/ebin/code.beam b/bootstrap/lib/kernel/ebin/code.beam Binary files differindex f1fd519f50..04150eee56 100644 --- a/bootstrap/lib/kernel/ebin/code.beam +++ b/bootstrap/lib/kernel/ebin/code.beam diff --git a/bootstrap/lib/kernel/ebin/code_server.beam b/bootstrap/lib/kernel/ebin/code_server.beam Binary files differindex 258a7b9181..cd96c87f99 100644 --- a/bootstrap/lib/kernel/ebin/code_server.beam +++ b/bootstrap/lib/kernel/ebin/code_server.beam diff --git a/bootstrap/lib/kernel/ebin/disk_log_1.beam b/bootstrap/lib/kernel/ebin/disk_log_1.beam Binary files differindex aa2e21382b..d851460d38 100644 --- a/bootstrap/lib/kernel/ebin/disk_log_1.beam +++ b/bootstrap/lib/kernel/ebin/disk_log_1.beam diff --git a/bootstrap/lib/kernel/ebin/dist_util.beam b/bootstrap/lib/kernel/ebin/dist_util.beam Binary files differindex 98d73eef55..8e07296a49 100644 --- a/bootstrap/lib/kernel/ebin/dist_util.beam +++ b/bootstrap/lib/kernel/ebin/dist_util.beam diff --git a/bootstrap/lib/kernel/ebin/file.beam b/bootstrap/lib/kernel/ebin/file.beam Binary files differindex 45efb31e93..3d3da9c344 100644 --- a/bootstrap/lib/kernel/ebin/file.beam +++ b/bootstrap/lib/kernel/ebin/file.beam diff --git a/bootstrap/lib/kernel/ebin/file_io_server.beam b/bootstrap/lib/kernel/ebin/file_io_server.beam Binary files differindex 9ab1228354..b5380a69cf 100644 --- a/bootstrap/lib/kernel/ebin/file_io_server.beam +++ b/bootstrap/lib/kernel/ebin/file_io_server.beam diff --git a/bootstrap/lib/kernel/ebin/group.beam b/bootstrap/lib/kernel/ebin/group.beam Binary files differindex bc2ef712aa..2a7e46faf5 100644 --- a/bootstrap/lib/kernel/ebin/group.beam +++ b/bootstrap/lib/kernel/ebin/group.beam diff --git a/bootstrap/lib/kernel/ebin/heart.beam b/bootstrap/lib/kernel/ebin/heart.beam Binary files differindex 80f0a60c9f..7e81d1610a 100644 --- a/bootstrap/lib/kernel/ebin/heart.beam +++ b/bootstrap/lib/kernel/ebin/heart.beam diff --git a/bootstrap/lib/kernel/ebin/hipe_unified_loader.beam b/bootstrap/lib/kernel/ebin/hipe_unified_loader.beam Binary files differindex f8d3e5847a..5b41832dec 100644 --- a/bootstrap/lib/kernel/ebin/hipe_unified_loader.beam +++ b/bootstrap/lib/kernel/ebin/hipe_unified_loader.beam diff --git a/bootstrap/lib/kernel/ebin/inet.beam b/bootstrap/lib/kernel/ebin/inet.beam Binary files differindex b2eccbb475..88a97e0ec2 100644 --- a/bootstrap/lib/kernel/ebin/inet.beam +++ b/bootstrap/lib/kernel/ebin/inet.beam diff --git a/bootstrap/lib/kernel/ebin/inet_dns.beam b/bootstrap/lib/kernel/ebin/inet_dns.beam Binary files differindex 4b82d77ccc..07b4356f7b 100644 --- a/bootstrap/lib/kernel/ebin/inet_dns.beam +++ b/bootstrap/lib/kernel/ebin/inet_dns.beam diff --git a/bootstrap/lib/kernel/ebin/inet_parse.beam b/bootstrap/lib/kernel/ebin/inet_parse.beam Binary files differindex ee4f541641..1b92542df2 100644 --- a/bootstrap/lib/kernel/ebin/inet_parse.beam +++ b/bootstrap/lib/kernel/ebin/inet_parse.beam diff --git a/bootstrap/lib/kernel/ebin/kernel.app b/bootstrap/lib/kernel/ebin/kernel.app index 48f6250828..ca2dc4f7c5 100644 --- a/bootstrap/lib/kernel/ebin/kernel.app +++ b/bootstrap/lib/kernel/ebin/kernel.app @@ -28,7 +28,6 @@ application_starter, auth, code, - packages, code_server, dist_util, erl_boot_server, diff --git a/bootstrap/lib/kernel/ebin/packages.beam b/bootstrap/lib/kernel/ebin/packages.beam Binary files differdeleted file mode 100644 index 86fb5b6094..0000000000 --- a/bootstrap/lib/kernel/ebin/packages.beam +++ /dev/null diff --git a/bootstrap/lib/kernel/ebin/user.beam b/bootstrap/lib/kernel/ebin/user.beam Binary files differindex d678631703..7b761283a0 100644 --- a/bootstrap/lib/kernel/ebin/user.beam +++ b/bootstrap/lib/kernel/ebin/user.beam diff --git a/bootstrap/lib/stdlib/ebin/base64.beam b/bootstrap/lib/stdlib/ebin/base64.beam Binary files differindex 1e9589e945..2ca9a4d0b8 100644 --- a/bootstrap/lib/stdlib/ebin/base64.beam +++ b/bootstrap/lib/stdlib/ebin/base64.beam diff --git a/bootstrap/lib/stdlib/ebin/beam_lib.beam b/bootstrap/lib/stdlib/ebin/beam_lib.beam Binary files differindex 250be4dcb1..c7ebc3b440 100644 --- a/bootstrap/lib/stdlib/ebin/beam_lib.beam +++ b/bootstrap/lib/stdlib/ebin/beam_lib.beam diff --git a/bootstrap/lib/stdlib/ebin/c.beam b/bootstrap/lib/stdlib/ebin/c.beam Binary files differindex d627009e03..d919fa0a2d 100644 --- a/bootstrap/lib/stdlib/ebin/c.beam +++ b/bootstrap/lib/stdlib/ebin/c.beam diff --git a/bootstrap/lib/stdlib/ebin/dets_utils.beam b/bootstrap/lib/stdlib/ebin/dets_utils.beam Binary files differindex 882228c5d8..06ace1f82d 100644 --- a/bootstrap/lib/stdlib/ebin/dets_utils.beam +++ b/bootstrap/lib/stdlib/ebin/dets_utils.beam diff --git a/bootstrap/lib/stdlib/ebin/dets_v9.beam b/bootstrap/lib/stdlib/ebin/dets_v9.beam Binary files differindex f504c8e9d4..4d383902cd 100644 --- a/bootstrap/lib/stdlib/ebin/dets_v9.beam +++ b/bootstrap/lib/stdlib/ebin/dets_v9.beam diff --git a/bootstrap/lib/stdlib/ebin/epp.beam b/bootstrap/lib/stdlib/ebin/epp.beam Binary files differindex e48ebd1d89..a3a86afce9 100644 --- a/bootstrap/lib/stdlib/ebin/epp.beam +++ b/bootstrap/lib/stdlib/ebin/epp.beam diff --git a/bootstrap/lib/stdlib/ebin/erl_eval.beam b/bootstrap/lib/stdlib/ebin/erl_eval.beam Binary files differindex d097eaf1f1..1445449c6e 100644 --- a/bootstrap/lib/stdlib/ebin/erl_eval.beam +++ b/bootstrap/lib/stdlib/ebin/erl_eval.beam diff --git a/bootstrap/lib/stdlib/ebin/erl_expand_records.beam b/bootstrap/lib/stdlib/ebin/erl_expand_records.beam Binary files differindex 093bd0f501..764f624cab 100644 --- a/bootstrap/lib/stdlib/ebin/erl_expand_records.beam +++ b/bootstrap/lib/stdlib/ebin/erl_expand_records.beam diff --git a/bootstrap/lib/stdlib/ebin/erl_lint.beam b/bootstrap/lib/stdlib/ebin/erl_lint.beam Binary files differindex 9dc80f67b9..3b330266f3 100644 --- a/bootstrap/lib/stdlib/ebin/erl_lint.beam +++ b/bootstrap/lib/stdlib/ebin/erl_lint.beam diff --git a/bootstrap/lib/stdlib/ebin/erl_parse.beam b/bootstrap/lib/stdlib/ebin/erl_parse.beam Binary files differindex d7a5103030..144d485b7f 100644 --- a/bootstrap/lib/stdlib/ebin/erl_parse.beam +++ b/bootstrap/lib/stdlib/ebin/erl_parse.beam diff --git a/bootstrap/lib/stdlib/ebin/erl_pp.beam b/bootstrap/lib/stdlib/ebin/erl_pp.beam Binary files differindex 32e33ad130..33b1962d20 100644 --- a/bootstrap/lib/stdlib/ebin/erl_pp.beam +++ b/bootstrap/lib/stdlib/ebin/erl_pp.beam diff --git a/bootstrap/lib/stdlib/ebin/erl_scan.beam b/bootstrap/lib/stdlib/ebin/erl_scan.beam Binary files differindex 3abab91ce1..7614fe72d8 100644 --- a/bootstrap/lib/stdlib/ebin/erl_scan.beam +++ b/bootstrap/lib/stdlib/ebin/erl_scan.beam diff --git a/bootstrap/lib/stdlib/ebin/erl_tar.beam b/bootstrap/lib/stdlib/ebin/erl_tar.beam Binary files differindex 90ed044a66..89ee92fddb 100644 --- a/bootstrap/lib/stdlib/ebin/erl_tar.beam +++ b/bootstrap/lib/stdlib/ebin/erl_tar.beam diff --git a/bootstrap/lib/stdlib/ebin/escript.beam b/bootstrap/lib/stdlib/ebin/escript.beam Binary files differindex 2a29ae551f..f8f3145066 100644 --- a/bootstrap/lib/stdlib/ebin/escript.beam +++ b/bootstrap/lib/stdlib/ebin/escript.beam diff --git a/bootstrap/lib/stdlib/ebin/eval_bits.beam b/bootstrap/lib/stdlib/ebin/eval_bits.beam Binary files differindex dc2ea0f521..379fc3655f 100644 --- a/bootstrap/lib/stdlib/ebin/eval_bits.beam +++ b/bootstrap/lib/stdlib/ebin/eval_bits.beam diff --git a/bootstrap/lib/stdlib/ebin/file_sorter.beam b/bootstrap/lib/stdlib/ebin/file_sorter.beam Binary files differindex e118a6ec85..180054c091 100644 --- a/bootstrap/lib/stdlib/ebin/file_sorter.beam +++ b/bootstrap/lib/stdlib/ebin/file_sorter.beam diff --git a/bootstrap/lib/stdlib/ebin/filelib.beam b/bootstrap/lib/stdlib/ebin/filelib.beam Binary files differindex 82dfeb7ca8..65f45632ac 100644 --- a/bootstrap/lib/stdlib/ebin/filelib.beam +++ b/bootstrap/lib/stdlib/ebin/filelib.beam diff --git a/bootstrap/lib/stdlib/ebin/filename.beam b/bootstrap/lib/stdlib/ebin/filename.beam Binary files differindex f8a42129c1..de676f2bed 100644 --- a/bootstrap/lib/stdlib/ebin/filename.beam +++ b/bootstrap/lib/stdlib/ebin/filename.beam diff --git a/bootstrap/lib/stdlib/ebin/io.beam b/bootstrap/lib/stdlib/ebin/io.beam Binary files differindex cade9d5ac6..ddfcca41e8 100644 --- a/bootstrap/lib/stdlib/ebin/io.beam +++ b/bootstrap/lib/stdlib/ebin/io.beam diff --git a/bootstrap/lib/stdlib/ebin/io_lib.beam b/bootstrap/lib/stdlib/ebin/io_lib.beam Binary files differindex 151fcd9d99..9420ea0f53 100644 --- a/bootstrap/lib/stdlib/ebin/io_lib.beam +++ b/bootstrap/lib/stdlib/ebin/io_lib.beam diff --git a/bootstrap/lib/stdlib/ebin/io_lib_format.beam b/bootstrap/lib/stdlib/ebin/io_lib_format.beam Binary files differindex 226f8216be..1a7c3393f8 100644 --- a/bootstrap/lib/stdlib/ebin/io_lib_format.beam +++ b/bootstrap/lib/stdlib/ebin/io_lib_format.beam diff --git a/bootstrap/lib/stdlib/ebin/io_lib_pretty.beam b/bootstrap/lib/stdlib/ebin/io_lib_pretty.beam Binary files differindex 16fad6af64..b14eb23e7c 100644 --- a/bootstrap/lib/stdlib/ebin/io_lib_pretty.beam +++ b/bootstrap/lib/stdlib/ebin/io_lib_pretty.beam diff --git a/bootstrap/lib/stdlib/ebin/lib.beam b/bootstrap/lib/stdlib/ebin/lib.beam Binary files differindex 3865cfb227..668ea7eb29 100644 --- a/bootstrap/lib/stdlib/ebin/lib.beam +++ b/bootstrap/lib/stdlib/ebin/lib.beam diff --git a/bootstrap/lib/stdlib/ebin/otp_internal.beam b/bootstrap/lib/stdlib/ebin/otp_internal.beam Binary files differindex 270d35ef33..cff55cb641 100644 --- a/bootstrap/lib/stdlib/ebin/otp_internal.beam +++ b/bootstrap/lib/stdlib/ebin/otp_internal.beam diff --git a/bootstrap/lib/stdlib/ebin/proc_lib.beam b/bootstrap/lib/stdlib/ebin/proc_lib.beam Binary files differindex 25843dfe45..e22b3ad758 100644 --- a/bootstrap/lib/stdlib/ebin/proc_lib.beam +++ b/bootstrap/lib/stdlib/ebin/proc_lib.beam diff --git a/bootstrap/lib/stdlib/ebin/qlc.beam b/bootstrap/lib/stdlib/ebin/qlc.beam Binary files differindex eef4cef042..302dda040f 100644 --- a/bootstrap/lib/stdlib/ebin/qlc.beam +++ b/bootstrap/lib/stdlib/ebin/qlc.beam diff --git a/bootstrap/lib/stdlib/ebin/qlc_pt.beam b/bootstrap/lib/stdlib/ebin/qlc_pt.beam Binary files differindex e76d49052d..458684586b 100644 --- a/bootstrap/lib/stdlib/ebin/qlc_pt.beam +++ b/bootstrap/lib/stdlib/ebin/qlc_pt.beam diff --git a/bootstrap/lib/stdlib/ebin/re.beam b/bootstrap/lib/stdlib/ebin/re.beam Binary files differindex f0508c73b3..8001c49672 100644 --- a/bootstrap/lib/stdlib/ebin/re.beam +++ b/bootstrap/lib/stdlib/ebin/re.beam diff --git a/bootstrap/lib/stdlib/ebin/shell.beam b/bootstrap/lib/stdlib/ebin/shell.beam Binary files differindex 8705c85853..5614d31ac8 100644 --- a/bootstrap/lib/stdlib/ebin/shell.beam +++ b/bootstrap/lib/stdlib/ebin/shell.beam diff --git a/bootstrap/lib/stdlib/ebin/supervisor.beam b/bootstrap/lib/stdlib/ebin/supervisor.beam Binary files differindex 0e16688bc4..aae89985c8 100644 --- a/bootstrap/lib/stdlib/ebin/supervisor.beam +++ b/bootstrap/lib/stdlib/ebin/supervisor.beam diff --git a/bootstrap/lib/stdlib/ebin/unicode.beam b/bootstrap/lib/stdlib/ebin/unicode.beam Binary files differindex 4e91cf2466..572cb2b6fa 100644 --- a/bootstrap/lib/stdlib/ebin/unicode.beam +++ b/bootstrap/lib/stdlib/ebin/unicode.beam diff --git a/erts/aclocal.m4 b/erts/aclocal.m4 index b1cf1fe404..9578cd35c4 100644 --- a/erts/aclocal.m4 +++ b/erts/aclocal.m4 @@ -740,11 +740,16 @@ dnl Try to find POSIX threads dnl The usual pthread lib... AC_CHECK_LIB(pthread, pthread_create, THR_LIBS="-lpthread") -dnl FreeBSD has pthreads in special c library, c_r... +dnl Very old versions of FreeBSD have pthreads in special c library, c_r... if test "x$THR_LIBS" = "x"; then AC_CHECK_LIB(c_r, pthread_create, THR_LIBS="-lc_r") fi +dnl QNX has pthreads in standard C library + if test "x$THR_LIBS" = "x"; then + AC_CHECK_FUNC(pthread_create, THR_LIBS="none_needed") + fi + dnl On ofs1 the '-pthread' switch should be used if test "x$THR_LIBS" = "x"; then AC_MSG_CHECKING([if the '-pthread' switch can be used]) @@ -765,6 +770,9 @@ dnl On ofs1 the '-pthread' switch should be used if test "x$THR_LIBS" != "x"; then THR_DEFS="$THR_DEFS -D_THREAD_SAFE -D_REENTRANT -DPOSIX_THREADS" THR_LIB_NAME=pthread + if test "x$THR_LIBS" = "xnone_needed"; then + THR_LIBS= + fi case $host_os in solaris*) THR_DEFS="$THR_DEFS -D_POSIX_PTHREAD_SEMANTICS" ;; diff --git a/erts/doc/src/Makefile b/erts/doc/src/Makefile index 91e823fdb1..89d7c85a86 100644 --- a/erts/doc/src/Makefile +++ b/erts/doc/src/Makefile @@ -77,6 +77,7 @@ XML_CHAPTER_FILES = \ inet_cfg.xml \ erl_ext_dist.xml \ erl_dist_protocol.xml \ + communication.xml \ notes.xml \ notes_history.xml diff --git a/erts/doc/src/communication.xml b/erts/doc/src/communication.xml new file mode 100644 index 0000000000..6049123f6a --- /dev/null +++ b/erts/doc/src/communication.xml @@ -0,0 +1,89 @@ +<?xml version="1.0" encoding="latin1" ?> +<!DOCTYPE chapter SYSTEM "chapter.dtd"> + +<chapter> + <header> + <copyright> + <year>2012</year><year>2012</year> + <holder>Ericsson AB. All Rights Reserved.</holder> + </copyright> + <legalnotice> + The contents of this file are subject to the Erlang Public License, + Version 1.1, (the "License"); you may not use this file except in + compliance with the License. You should have received a copy of the + Erlang Public License along with this software. If not, it can be + retrieved online at http://www.erlang.org/. + + Software distributed under the License is distributed on an "AS IS" + basis, WITHOUT WARRANTY OF ANY KIND, either express or implied. See + the License for the specific language governing rights and limitations + under the License. + + </legalnotice> + + <title>Communication in Erlang</title> + <prepared>Rickard Green</prepared> + <responsible></responsible> + <docno></docno> + <approved></approved> + <checked></checked> + <date>2012-12-03</date> + <rev>PA1</rev> + <file>communication.xml</file> + </header> + <p>Communication in Erlang is conceptually performed using + asynchronous signaling. All different executing entities + such as processes, and ports communicate via asynchronous + signals. The most commonly used signal is a message. Other + common signals are exit, link, unlink, monitor, demonitor + signals.</p> + <section> + <title>Passing of Signals</title> + <p>The amount of time that passes between a signal being sent + and the arrival of the signal at the destination is unspecified + but positive. If the receiver has terminated, the signal will + not arrive, but it is possible that it triggers another signal. + For example, a link signal sent to a non-existing process will + trigger an exit signal which will be sent back to where the link + signal originated from. When communicating over the distribution, + signals may be lost if the distribution channel goes down.</p> + <p>The only signal ordering guarantee given is the following. If + an entity sends multiple signals to the same destination entity, + the order will be preserved. That is, if <c>A</c> send + a signal <c>S1</c> to <c>B</c>, and later sends + the signal <c>S2</c> to <c>B</c>, <c>S1</c> is guaranteed not to + arrive after <c>S2</c>.</p> + </section> + <section> + <title>Synchronous Communication</title> + <p>Some communication is synchronous. If broken down into pieces, + a synchronous communication operation, consists of two asynchronous + signals. One request signal and one reply signal. An example of + such a synchronous communication is a call to <c>process_info/2</c> + when the first argument is not <c>self()</c>. The caller will send + an asynchronous signal requesting information, and will then + wait for the reply signal containing the requested information. When + the request signal reaches its destination the destination process + replies with the requested information.</p> + </section> + <section> + <title>Implementation</title> + <p>The implementation of different asynchronous signals in the + VM may vary over time, but the behavior will always respect this + concept of asynchronous signals being passed between entities + as described above.</p> + <p>By inspecting the implementation you might notice that some + specific signal actually gives a stricter guarantee than described + above. It is of vital importance that such knowledge about the + implementation is <em>not</em> used by Erlang code, since the + implementation might change at any time without prior notice.</p> + <p>Some example of major implementation changes:</p> + <list type="bulleted"> + <item>As of ERTS version 5.5.2 exit signals to processes are truly + asynchronously delivered.</item> + <item>As of ERTS version 5.10 all signals from processes to ports + are truly asynchronously delivered.</item> + </list> + </section> +</chapter> + diff --git a/erts/doc/src/driver_entry.xml b/erts/doc/src/driver_entry.xml index 929c485c36..c37138e7b1 100644 --- a/erts/doc/src/driver_entry.xml +++ b/erts/doc/src/driver_entry.xml @@ -4,7 +4,7 @@ <cref> <header> <copyright> - <year>2001</year><year>2011</year> + <year>2001</year><year>2012</year> <holder>Ericsson AB. All Rights Reserved.</holder> </copyright> <legalnotice> @@ -400,11 +400,11 @@ typedef struct erl_drv_entry { <tag><marker id="driver_flags"/>int driver_flags</tag> <item> - <p>This field is used to pass driver capability information to the - runtime system. If the <c>extended_marker</c> field equals - <c>ERL_DRV_EXTENDED_MARKER</c>, it should contain <c>0</c> or - driver flags (<c>ERL_DRV_FLAG_*</c>) ored bitwise. Currently - the following driver flags exist: + <p>This field is used to pass driver capability and other + information to the runtime system. If the + <c>extended_marker</c> field equals <c>ERL_DRV_EXTENDED_MARKER</c>, + it should contain <c>0</c> or driver flags (<c>ERL_DRV_FLAG_*</c>) + ored bitwise. Currently the following driver flags exist: </p> <taglist> <tag><c>ERL_DRV_FLAG_USE_PORT_LOCKING</c></tag> @@ -427,6 +427,12 @@ typedef struct erl_drv_entry { by the Erlang distribution (the behaviour has always been required by drivers used by the distribution). </item> + <tag><c>ERL_DRV_FLAG_NO_BUSY_MSGQ</c></tag> + <item>Disable busy port message queue functionality. For + more information, see the documentation of the + <seealso marker="erl_driver#erl_drv_busy_msgq_limits">erl_drv_busy_msgq_limits()</seealso> + function. + </item> </taglist> </item> <tag>void *handle2</tag> diff --git a/erts/doc/src/erl.xml b/erts/doc/src/erl.xml index 7cdb3a4dfe..99f2466d79 100644 --- a/erts/doc/src/erl.xml +++ b/erts/doc/src/erl.xml @@ -582,13 +582,71 @@ <seealso marker="erts_alloc">erts_alloc(3)</seealso> for further information.</p> </item> + <tag><c><![CDATA[+n Behavior]]></c></tag> + <item> + <p>Control behavior of signals to ports.</p> + <p>As of OTP-R16 signals to ports are truly asynchronously + delivered. Note that signals always have been documented as + asynchronous. The underlying implementation has, however, + previously delivered these signals synchronously. Correctly + written Erlang programs should be able to handle this without + any issues. Bugs in existing Erlang programs that make false + assumptions about signals to ports may, however, be tricky to + find. This switch has been introduced in order to at least + make it easier to compare behaviors during a transition period. + Note that <em>this flag is deprecated</em> as of its + introduction, and is scheduled for removal in OTP-R17. + <c>Behavior</c> should be one of the following characters:</p> + <taglist> + <tag><c>d</c></tag> + <item>The default. Asynchronous signals. A process that sends + a signal to a port may continue execution before the signal + has been delivered to the port.</item> + <tag><c>s</c></tag> + <item>Synchronous signals. A processes that sends a signal + to a port will not continue execution until the signal has + been delivered. Should <em>only</em> be used for testing and + debugging.</item> + <tag><c>a</c></tag> + <item>Asynchronous signals. As the default, but a processes + that sends a signal will even more frequently continue + execution before the signal has been delivered to the + port. Should <em>only</em> be used for testing and + debugging.</item> + </taglist> + </item> <tag><marker id="max_processes"><c><![CDATA[+P Number]]></c></marker></tag> <item> - <p>Sets the maximum number of concurrent processes for this - system. <c><![CDATA[Number]]></c> must be in the range 16..134217727. - Default is 32768.</p> - <p><em>NOTE</em>: It is possible to choose any value in the range - but powers of 2 perform best.</p> + <p>Sets the maximum number of simultaneously existing processes for this + system. Valid range for <c>Number</c> is <c>[1024-134217727]</c></p> + <p><em>NOTE</em>: The actual maximum chosen may be much larger than + the <c>Number</c> passed. Currently the runtime system often, + but not always, chooses a value that is a power of 2. This might, + however, be changed in the future. The actual value chosen can be + checked by calling + <seealso marker="erlang#system_info_process_limit">erlang:system_info(process_limit)</seealso>.</p> + <p>The default value is <c>262144</c></p> + </item> + <tag><marker id="max_ports"><c><![CDATA[+Q Number]]></c></marker></tag> + <item> + <p>Sets the maximum number of simultaneously existing ports for this + system. Valid range for <c>Number</c> is <c>[1024-134217727]</c></p> + <p><em>NOTE</em>: The actual maximum chosen may be much larger than + the actual <c>Number</c> passed. Currently the runtime system often, + but not always, chooses a value that is a power of 2. This might, + however, be changed in the future. The actual value chosen can be + checked by calling + <seealso marker="erlang#system_info_port_limit">erlang:system_info(port_limit)</seealso>.</p> + <p>The default value used is normally <c>65536</c>. However, if + the runtime system is able to determine maximum amount of file + descriptors that it is allowed to open and this value is larger + than <c>65536</c>, the chosen value will increased to a value + larger or equal to the maximum amount of file descriptors that + can be opened.</p> + <p>Previously the environment variable <c>ERL_MAX_PORTS</c> was used + for setting the maximum number of simultaneously existing ports. This + environment variable is deprecated, and scheduled for removal in + OTP-R17, but can still be used.</p> </item> <tag><marker id="compat_rel"><c><![CDATA[+R ReleaseNumber]]></c></marker></tag> <item> @@ -597,21 +655,14 @@ default. This flags sets the emulator in compatibility mode with an earlier Erlang/OTP release <c><![CDATA[ReleaseNumber]]></c>. The release number must be in the range - <c><![CDATA[7..<current release>]]></c>. This limits the emulator, - making it possible for it to communicate with Erlang nodes - (as well as C- and Java nodes) running that earlier release.</p> - <p>For example, an R10 node is not automatically compatible - with an R9 node, but R10 nodes started with the <c><![CDATA[+R 9]]></c> - flag can co-exist with R9 nodes in the same distributed - Erlang system, they are R9-compatible.</p> + <c><![CDATA[<current release>-2..<current release>]]></c>. This + limits the emulator, making it possible for it to communicate + with Erlang nodes (as well as C- and Java nodes) running that + earlier release.</p> <p>Note: Make sure all nodes (Erlang-, C-, and Java nodes) of a distributed Erlang system is of the same Erlang/OTP release, or from two different Erlang/OTP releases X and Y, where <em>all</em> Y nodes have compatibility mode X.</p> - <p>For example: A distributed Erlang system can consist of - R10 nodes, or of R9 nodes and R9-compatible R10 nodes, but - not of R9 nodes, R9-compatible R10 nodes and "regular" R10 - nodes, as R9 and "regular" R10 nodes are not compatible.</p> </item> <tag><c><![CDATA[+r]]></c></tag> <item> @@ -934,6 +985,22 @@ without prior notice. </p> </item> + <tag><marker id="+spp"><c>+spp Bool</c></marker></tag> + <item> + <p>Set default scheduler hint for port parallelism. If set to + <c>true</c>, the VM will schedule port tasks when it by this can + improve the parallelism in the system. If set to <c>false</c>, + the VM will try to perform port tasks immediately and by this + improve latency at the expense of parallelism. If this + flag has not been passed, the default scheduler hint for port + parallelism is currently <c>false</c>. The default used can be + inspected in runtime by calling + <seealso marker="erlang#system_info_port_parallelism">erlang:system_info(port_parallelism)</seealso>. + The default can be overriden on port creation by passing the + <seealso marker="erlang#open_port_parallelism">parallelism</seealso> + option to + <seealso marker="erlang#open_port/2">open_port/2</seealso></p>. + </item> <tag><marker id="sched_thread_stack_size"><c><![CDATA[+sss size]]></c></marker></tag> <item> <p>Suggested stack size, in kilowords, for scheduler threads. @@ -1074,7 +1141,7 @@ </item> </taglist> </item> - <tag><c><![CDATA[ERL_AFLAGS]]></c></tag> + <tag><marker id="ERL_AFLAGS"><c><![CDATA[ERL_AFLAGS]]></c></marker></tag> <item> <p>The content of this environment variable will be added to the beginning of the command line for <c><![CDATA[erl]]></c>.</p> @@ -1084,7 +1151,7 @@ the <c><![CDATA[-extra]]></c> section, i.e. the end of the command line following after an <c><![CDATA[-extra]]></c> flag.</p> </item> - <tag><c><![CDATA[ERL_ZFLAGS]]></c> and <c><![CDATA[ERL_FLAGS]]></c></tag> + <tag><marker id="ERL_ZFLAGS"><c><![CDATA[ERL_ZFLAGS]]></c></marker> and <marker id="ERL_FLAGS"><c><![CDATA[ERL_FLAGS]]></c></marker></tag> <item> <p>The content of these environment variables will be added to the end of the command line for <c><![CDATA[erl]]></c>.</p> diff --git a/erts/doc/src/erl_driver.xml b/erts/doc/src/erl_driver.xml index e16fd744c0..13f42a74a7 100644 --- a/erts/doc/src/erl_driver.xml +++ b/erts/doc/src/erl_driver.xml @@ -145,12 +145,20 @@ different threads. This, however, is not a problem for any function in this API, since the emulator has control over these threads.</p> - <note> - <p>Functions not explicitly documented as thread-safe are - <em>not</em> thread-safe. Also note that some functions + <warning> + <p>Functions not explicitly documented as thread safe are + <em>not</em> thread safe. Also note that some functions are <em>only</em> thread safe when used in a runtime system with SMP support.</p> - </note> + <p>A function not explicitly documented as thread safe may at + some point in time have a thread safe implementation in the + runtime system. Such an implementation may however change to + a thread <em>unsafe</em> implementation at any time <em>without + any notice</em> at all. + </p> + <p><em>Only use functions explicitly documented as thread safe + from arbitrary threads.</em></p> + </warning> <p><marker id="lengthy_work"/> As mentioned in the <seealso marker="#WARNING">warning</seealso> text at the beginning of this document it is of vital importance that a driver callback @@ -1529,16 +1537,81 @@ typedef struct ErlIOVec { </desc> </func> <func> + <name><ret>void</ret><nametext>erl_drv_busy_msgq_limits(ErlDrvPort port, ErlDrvSizeT *low, ErlDrvSizeT *high)</nametext></name> + <fsummary>Set and get limits for busy port message queue</fsummary> + <desc> + <marker id="erl_drv_busy_msgq_limits"></marker> + <p>Sets and gets limits that will be used for controling the + busy state of the port message queue.</p> + <p>The port message queue will be set into a busy + state when the amount of command data queued on the + message queue reaches the <c>high</c> limit. The port + message queue will be set into a not busy state when the + amount of command data queued on the message queue falls + below the <c>low</c> limit. Command data is in this + context data passed to the port using either + <c>Port ! {Owner, {command, Data}}</c>, or + <c>port_command/[2,3]</c>. Note that these limits + only concerns command data that have not yet reached the + port. The <seealso marker="#set_busy_port">busy port</seealso> + feature can be used for data that has reached the port.</p> + + <p>Valid limits are values in the range + <c>[ERL_DRV_BUSY_MSGQ_LIM_MIN, ERL_DRV_BUSY_MSGQ_LIM_MAX]</c>. + Limits will be automatically adjusted to be sane. That is, + the system will adjust values so that the low limit used is + lower or equal to the high limit used. By default the high + limit will be 8 kB and the low limit will be 4 kB.</p> + + <p>By passing a pointer to an integer variable containing + the value <c>ERL_DRV_BUSY_MSGQ_READ_ONLY</c>, currently used + limit will be read and written back to the integer variable. + A new limit can be set by passing a pointer to an integer + variable containing a valid limit. The passed value will be + written to the internal limit. The internal limit will then + be adjusted. After this the adjusted limit will be written + back to the integer variable from which the new value was + read. Values are in bytes.</p> + + <p>The busy message queue feature can be disabled either + by setting the <c>ERL_DRV_FLAG_NO_BUSY_MSGQ</c> + <seealso marker="driver_entry#driver_flags">driver flag</seealso> + in the <seealso marker="driver_entry">driver_entry</seealso> + used by the driver, or by calling this function with + <c>ERL_DRV_BUSY_MSGQ_DISABLED</c> as a limit (either low or + high). When this feature has been disabled it cannot be + enabled again. When reading the limits both of them + will be <c>ERL_DRV_BUSY_MSGQ_DISABLED</c>, if this + feature has been disabled.</p> + + <p>Processes sending command data to the port will be suspended + if either the port is busy or if the port message queue is + busy. Suspended processes will be resumed when neither the + port is busy, nor the port message queue is busy.</p> + + <p>For information about busy port functionality + see the documentation of the + <seealso marker="#set_busy_port">set_busy_port()</seealso> + function.</p> + </desc> + </func> + <func> <name><ret>void</ret><nametext>set_busy_port(ErlDrvPort port, int on)</nametext></name> <fsummary>Signal or unsignal port as busy</fsummary> <desc> <marker id="set_busy_port"></marker> - <p>This function set and resets the busy status of the port. If - <c>on</c> is 1, the port is set to busy, if it's 0 the port - is set to not busy.</p> - <p>When the port is busy, sending to it with <c>Port ! Data</c> - or <c>port_command/2</c>, will block the port owner process, - until the port is signaled as not busy.</p> + <p>This function set and unset the busy state of the port. If + <c>on</c> is non-zero, the port is set to busy, if it's zero the port + is set to not busy. You typically want to combine + this feature with the <seealso marker="#erl_drv_busy_msgq_limits">busy + port message queue</seealso> functionality.</p> + <p>Processes sending command data to the port will be suspended + if either the port is busy or if the port message queue + is busy. Suspended processes will be resumed when neither the + port is busy, nor the port message queue is busy. Command data + is in this context data passed to the port using either + <c>Port ! {Owner, {command, Data}}</c>, or + <c>port_command/[2,3]</c>.</p> <p>If the <seealso marker="driver_entry#driver_flags"><![CDATA[ERL_DRV_FLAG_SOFT_BUSY]]></seealso> has been set in the @@ -1547,6 +1620,10 @@ typedef struct ErlIOVec { <seealso marker="erlang#port_command/3">port_command(Port, Data, [force])</seealso> even though the driver has signaled that it is busy. </p> + <p>For information about busy port message queue functionality + see the documentation of the + <seealso marker="#erl_drv_busy_msgq_limits">erl_drv_busy_msgq_limits()</seealso> + function.</p> </desc> </func> <func> @@ -1607,6 +1684,8 @@ typedef struct ErlIOVec { <desc> <marker id="driver_connected"></marker> <p>This function returns the port owner process.</p> + <p>Note that this function is <em>not</em> thread-safe, not + even when the emulator with SMP support is used.</p> </desc> </func> <func> @@ -1634,22 +1713,32 @@ typedef struct ErlIOVec { <tag><seealso marker="driver_entry#call">call</seealso></tag> <item>Called from <c>erlang:port_call/3</c></item> </taglist> + <p>Note that this function is <em>not</em> thread-safe, not + even when the emulator with SMP support is used.</p> </desc> </func> <func> - <name><ret>int</ret><nametext>driver_output_term(ErlDrvPort port, ErlDrvTermData* term, int n)</nametext></name> + <name><ret>int</ret><nametext>erl_drv_output_term(ErlDrvTermData port, ErlDrvTermData* term, int n)</nametext></name> <fsummary>Send term data from driver to port owner</fsummary> <desc> - <marker id="driver_output_term"></marker> + <marker id="erl_drv_output_term"></marker> <p>This functions sends data in the special driver term - format. This is a fast way to deliver term data from a - driver. It also needs no binary conversion, so the port - owner process receives data as normal Erlang terms.</p> + format to the port owner process. This is a fast way to + deliver term data from a driver. It also needs no binary + conversion, so the port owner process receives data as + normal Erlang terms. The + <seealso marker="#erl_drv_send_term">erl_drv_send_term()</seealso> + functions can be used for sending to any arbitrary process + on the local node.</p> + <note><p>Note that the <c>port</c> parameter is <em>not</em> + an ordinary port handle, but a port handle converted using + <c>driver_mk_port()</c>.</p></note> <p>The <c>term</c> parameter points to an array of <c>ErlDrvTermData</c>, with <c>n</c> elements. This array contains terms described in the driver term format. Every term consists of one to four elements in the array. The - term first has a term type, and then arguments.</p> + term first has a term type, and then arguments. The + <c>port</c> parameter specifies the sending port.</p> <p>Tuple and lists (with the exception of strings, see below), are built in reverse polish notation, so that to build a tuple, the elements are given first, and then the tuple @@ -1701,17 +1790,17 @@ ERL_DRV_EXT2TERM char *buf, ErlDrvUInt len ErlDrvPort port = ... ErlDrvTermData spec[] = { ERL_DRV_ATOM, driver_mk_atom("tcp"), - ERL_DRV_PORT, driver_mk_port(port), + ERL_DRV_PORT, driver_mk_port(drvport), ERL_DRV_INT, 100, ERL_DRV_BINARY, bin, 50, 0, ERL_DRV_LIST, 2, ERL_DRV_TUPLE, 3, }; - driver_output_term(port, spec, sizeof(spec) / sizeof(spec[0])); + erl_drv_output_term(driver_mk_port(drvport), spec, sizeof(spec) / sizeof(spec[0])); ]]> </code> <p>Where <c>bin</c> is a driver binary of length at least 50 - and <c>port</c> is a port handle. Note that the <c>ERL_DRV_LIST</c> + and <c>drvport</c> is a port handle. Note that the <c>ERL_DRV_LIST</c> comes after the elements of the list, likewise the <c>ERL_DRV_TUPLE</c>.</p> <p>The term <c>ERL_DRV_STRING_CONS</c> is a way to construct @@ -1732,7 +1821,7 @@ ERL_DRV_EXT2TERM char *buf, ErlDrvUInt len ERL_DRV_NIL, ERL_DRV_LIST, 4 }; - driver_output_term(port, spec, sizeof(spec) / sizeof(spec[0])); + erl_drv_output_term(driver_mk_port(drvport), spec, sizeof(spec) / sizeof(spec[0])); ]]></code> <p></p> <code type="none"><![CDATA[ @@ -1742,7 +1831,7 @@ ERL_DRV_EXT2TERM char *buf, ErlDrvUInt len ERL_DRV_STRING_CONS, (ErlDrvTermData)"123", 3, ERL_DRV_STRING_CONS, (ErlDrvTermData)"abc", 3, }; - driver_output_term(port, spec, sizeof(spec) / sizeof(spec[0])); + erl_drv_output_term(driver_mk_port(drvport), spec, sizeof(spec) / sizeof(spec[0])); ]]></code> <p>The <c>ERL_DRV_EXT2TERM</c> term type is used for passing a term encoded with the @@ -1762,7 +1851,7 @@ ERL_DRV_EXT2TERM char *buf, ErlDrvUInt len ERL_DRV_EXT2TERM, (ErlDrvTermData) binp->orig_bytes, binp->orig_size ERL_DRV_TUPLE, 2, }; - driver_output_term(port, spec, sizeof(spec) / sizeof(spec[0])); + erl_drv_output_term(driver_mk_port(drvport), spec, sizeof(spec) / sizeof(spec[0])); ]]></code> <p>If you want to pass a binary and don't already have the content of the binary in an <c>ErlDrvBinary</c>, you can benefit from using @@ -1778,6 +1867,22 @@ ERL_DRV_EXT2TERM char *buf, ErlDrvUInt len <c>ERL_DRV_EXT2TERM</c> term types were introduced in the 5.6 version of erts. </p> + <p>This function is only thread-safe when the emulator with SMP + support is used.</p> + </desc> + </func> + <func> + <name><ret>int</ret><nametext>driver_output_term(ErlDrvPort port, ErlDrvTermData* term, int n)</nametext></name> + <fsummary>Send term data from driver to port owner</fsummary> + <desc> + <marker id="driver_output_term"></marker> + <warning><p><c>driver_output_term()</c> is deferred and will + be removed in the OTP-R17 release. Use + <seealso marker="#erl_drv_send_term">erl_drv_output_term()</seealso> + instead.</p> + </warning> + <p>The parameters <c>term</c> and <c>n</c> do the same thing + as in <seealso marker="#erl_drv_output_term">erl_drv_output_term()</seealso>.</p> <p>Note that this function is <em>not</em> thread-safe, not even when the emulator with SMP support is used.</p> </desc> @@ -1791,6 +1896,8 @@ ERL_DRV_EXT2TERM char *buf, ErlDrvUInt len <c>string</c>. The atom is created and won't change, so the return value may be saved and reused, which is faster than looking up the atom several times.</p> + <p>Note that this function is <em>not</em> thread-safe, not + even when the emulator with SMP support is used.</p> </desc> </func> <func> @@ -1799,20 +1906,46 @@ ERL_DRV_EXT2TERM char *buf, ErlDrvUInt len <desc> <marker id="driver_mk_port"></marker> <p>This function converts a port handle to the erlang term - format, usable in the <c>driver_output_send</c> function.</p> + format, usable in the <seealso marker="#erl_drv_output_term">erl_drv_output_term()</seealso>, and <seealso marker="#erl_drv_send_term">erl_drv_send_term()</seealso> functions.</p> + <p>Note that this function is <em>not</em> thread-safe, not + even when the emulator with SMP support is used.</p> </desc> </func> <func> - <name><ret>int</ret><nametext>driver_send_term(ErlDrvPort port, ErlDrvTermData receiver, ErlDrvTermData* term, int n)</nametext></name> + <name><ret>int</ret><nametext>erl_drv_send_term(ErlDrvTermData port, ErlDrvTermData receiver, ErlDrvTermData* term, int n)</nametext></name> <fsummary>Send term data to other process than port owner process</fsummary> <desc> - <marker id="driver_send_term"></marker> + <marker id="erl_drv_send_term"></marker> <p>This function is the only way for a driver to send data to <em>other</em> processes than the port owner process. The <c>receiver</c> parameter specifies the process to receive the data.</p> + <note><p>Note that the <c>port</c> parameter is <em>not</em> + an ordinary port handle, but a port handle converted using + <c>driver_mk_port()</c>.</p></note> + <p>The parameters <c>port</c>, <c>term</c> and <c>n</c> do the same thing + as in <seealso marker="#erl_drv_output_term">erl_drv_output_term()</seealso>.</p> + <p>This function is only thread-safe when the emulator with SMP + support is used.</p> + </desc> + </func> + <func> + <name><ret>int</ret><nametext>driver_send_term(ErlDrvPort port, ErlDrvTermData receiver, ErlDrvTermData* term, int n)</nametext></name> + <fsummary>Send term data to other process than port owner process</fsummary> + <desc> + <marker id="driver_send_term"></marker> + <warning><p><c>driver_send_term()</c> is deferred and will + be removed in the OTP-R17 release. Use + <seealso marker="#erl_drv_send_term">erl_drv_send_term()</seealso> + instead.</p> + <p>Also note that parameters of <c>driver_send_term()</c> + cannot be properly checked by the runtime system when + executed by arbitrary threads. This may cause the + <c>driver_send_term()</c> function not to fail when + it should.</p> + </warning> <p>The parameters <c>term</c> and <c>n</c> do the same thing - as in <seealso marker="#driver_output_term">driver_output_term</seealso>.</p> + as in <seealso marker="#erl_drv_output_term">erl_drv_output_term()</seealso>.</p> <p>This function is only thread-safe when the emulator with SMP support is used.</p> </desc> diff --git a/erts/doc/src/erlang.xml b/erts/doc/src/erlang.xml index ef0e6fea84..1d67be2e52 100644 --- a/erts/doc/src/erlang.xml +++ b/erts/doc/src/erlang.xml @@ -626,6 +626,22 @@ false</pre> {more,6}</pre> </desc> </func> + + <func> + <name name="delete_element" arity="2"/> + <fsummary>Delete element at index in a tuple</fsummary> + <type_desc variable="Index">1..tuple_size(<anno>Tuple1</anno>)</type_desc> + <desc> + <p> + Returns a new tuple with element at <c><anno>Index</anno></c> removed from + tuple <c><anno>Tuple1</anno></c>. + </p> + <pre> +> <input>erlang:delete_element(2, {one, two, three}).</input> +{one,three}</pre> + </desc> + </func> + <func> <name name="delete_module" arity="1"/> <fsummary>Make the current code for a module old</fsummary> @@ -854,10 +870,10 @@ b</pre> </func> <func> <name name="exit" arity="2"/> - <fsummary>Send an exit signal to a process</fsummary> + <fsummary>Send an exit signal to a process or a port</fsummary> <desc> <p>Sends an exit signal with exit reason <c><anno>Reason</anno></c> to - the process <c><anno>Pid</anno></c>.</p> + the process or port identified by <c><anno>Pid</anno></c>.</p> <p>The following behavior apply if <c><anno>Reason</anno></c> is any term except <c>normal</c> or <c>kill</c>:</p> <p>If <c><anno>Pid</anno></c> is not trapping exits, <c><anno>Pid</anno></c> itself will @@ -1361,6 +1377,24 @@ os_prompt% </pre> when the process wakes up.</p> </desc> </func> + + <func> + <name name="insert_element" arity="3"/> + <fsummary>Insert an element at index in a tuple</fsummary> + <type_desc variable="Index">1..tuple_size(<anno>Tuple1</anno>) + 1</type_desc> + <desc> + <p> + Returns a new tuple with element <c><anno>Term</anno></c> insert at position + <c><anno>Index</anno></c> in tuple <c><anno>Tuple1</anno></c>. + All elements from position <c><anno>Index</anno></c> and upwards are subsequently + pushed one step higher in the new tuple <c><anno>Tuple2</anno></c>. + </p> + <pre> +> <input>erlang:insert_element(2, {one, two, three}, new).</input> +{one,new,two,three}</pre> + </desc> + </func> + <func> <name name="integer_to_list" arity="1"/> <fsummary>Text representation of an integer</fsummary> @@ -1497,13 +1531,6 @@ os_prompt% </pre> applied with <c><anno>Arity</anno></c> number of arguments; otherwise returns <c>false</c>.</p> <p>Allowed in guard tests.</p> - <warning> - <p>Currently, <c>is_function/2</c> will also return - <c>true</c> if the first argument is a tuple fun (a tuple - containing two atoms). In a future release, tuple funs will - no longer be supported and <c>is_function/2</c> will return - <c>false</c> if given a tuple fun.</p> - </warning> </desc> </func> <func> @@ -2767,6 +2794,18 @@ os_prompt% </pre> console window when spawning the port program. (This option has no effect on other platforms.)</p> </item> + <tag><marker id="open_port_parallelism"><c>{parallelism, Boolean}</c></marker></tag> + <item> + <p>Set scheduler hint for port parallelism. If set to <c>true</c>, + the VM will schedule port tasks when it by this can improve the + parallelism in the system. If set to <c>false</c>, the VM will + try to perform port tasks immediately and by this improving the + latency at the expense of parallelism. The default can be set on + system startup by passing the + <seealso marker="erl#+spp">+spp</seealso> command line argument + to <seealso marker="erl">erl(1)</seealso>. + </p> + </item> </taglist> <p>The default is <c>stream</c> for all types of port and <c>use_stdio</c> for spawned ports.</p> @@ -2819,10 +2858,11 @@ os_prompt% </pre> the owning process using signals of the form <c>{'EXIT', Port, PosixCode}</c>. See <c>file(3)</c> for possible values of <c>PosixCode</c>.</p> - <p><marker id="ERL_MAX_PORTS"></marker> - The maximum number of ports that can be open at the same - time is 1024 by default, but can be configured by - the environment variable <c>ERL_MAX_PORTS</c>.</p> + <p>The maximum number of ports that can be open at the same + time can be configured by passing the + <seealso marker="erl#max_ports"><c>+Q</c></seealso> + command line flag to + <seealso marker="erl"><c>erl(1)</c></seealso>.</p> </desc> </func> <func> @@ -2880,10 +2920,10 @@ os_prompt% </pre> <desc> <p>Closes an open port. Roughly the same as <c><anno>Port</anno> ! {self(), close}</c> except for the error behaviour - (see below), and that the port does <em>not</em> reply with - <c>{Port, closed}</c>. Any process may close a port with - <c>port_close/1</c>, not only the port owner (the connected - process).</p> + (see below), being synchronous, and that the port does + <em>not</em> reply with <c>{Port, closed}</c>. Any process may + close a port with <c>port_close/1</c>, not only the port owner + (the connected process).</p> <p>For comparison: <c><anno>Port</anno> ! {self(), close}</c> fails with <c>badarg</c> if <c><anno>Port</anno></c> cannot be sent to (i.e., <c><anno>Port</anno></c> refers neither to a port nor to a process). If @@ -2895,8 +2935,12 @@ os_prompt% </pre> <p>Note that any process can close a port using <c><anno>Port</anno> ! {PortOwner, close}</c> just as if it itself was the port owner, but the reply always goes to the port owner.</p> - <p>In short: <c>port_close(Port)</c> has a cleaner and more - logical behaviour than <c><anno>Port</anno> ! {self(), close}</c>.</p> + <p>As of OTP-R16 <c><anno>Port</anno> ! {PortOwner, close}</c> is truly + asynchronous. Note that this operation has always been + documented as an asynchronous operation, while the underlying + implementation has been synchronous. <c>port_close/1</c> is + however still fully synchronous. This due to its error + behavior.</p> <p>Failure: <c>badarg</c> if <c><anno>Port</anno></c> is not an open port or the registered name of an open port.</p> </desc> @@ -2906,11 +2950,11 @@ os_prompt% </pre> <fsummary>Send data to a port</fsummary> <desc> <p>Sends data to a port. Same as - <c><anno>Port</anno> ! {self(), {command, Data}}</c> except for the error - behaviour (see below). Any process may send data to a port - with <c>port_command/2</c>, not only the port owner - (the connected process).</p> - <p>For comparison: <c><anno>Port</anno> ! {self(), {command, Data}}</c> + <c><anno>Port</anno> ! {PortOwner, {command, Data}}</c> except for the error + behaviour and being synchronous (see below). Any process may + send data to a port with <c>port_command/2</c>, not only the + port owner (the connected process).</p> + <p>For comparison: <c><anno>Port</anno> ! {PortOwner, {command, Data}}</c> fails with <c>badarg</c> if <c><anno>Port</anno></c> cannot be sent to (i.e., <c><anno>Port</anno></c> refers neither to a port nor to a process). If <c><anno>Port</anno></c> is a closed port the data message disappears @@ -2921,11 +2965,14 @@ os_prompt% </pre> <p>Note that any process can send to a port using <c><anno>Port</anno> ! {PortOwner, {command, <anno>Data</anno>}}</c> just as if it itself was the port owner.</p> - <p>In short: <c>port_command(<anno>Port</anno>, <anno>Data</anno>)</c> has a cleaner and - more logical behaviour than - <c><anno>Port</anno> ! {self(), {command, Data}}</c>.</p> <p>If the port is busy, the calling process will be suspended until the port is not busy anymore.</p> + <p>As of OTP-R16 <c><anno>Port</anno> ! {PortOwner, {command, Data}}</c> is + truly asynchronous. Note that this operation has always been + documented as an asynchronous operation, while the underlying + implementation has been synchronous. <c>port_command/2</c> is + however still fully synchronous. This due to its error + behavior.</p> <p>Failures:</p> <taglist> <tag><c>badarg</c></tag> @@ -2999,7 +3046,7 @@ os_prompt% </pre> <fsummary>Set the owner of a port</fsummary> <desc> <p>Sets the port owner (the connected port) to <c><anno>Pid</anno></c>. - Roughly the same as <c><anno>Port</anno> ! {self(), {connect, <anno>Pid</anno>}}</c> + Roughly the same as <c><anno>Port</anno> ! {Owner, {connect, <anno>Pid</anno>}}</c> except for the following:</p> <list type="bulleted"> <item> @@ -3010,6 +3057,9 @@ os_prompt% </pre> <c>{Port,connected}</c>.</p> </item> <item> + <p><c>port_connect/1</c> is synchronous, see below.</p> + </item> + <item> <p>The new port owner gets linked to the port.</p> </item> </list> @@ -3033,11 +3083,14 @@ os_prompt% </pre> <c><anno>Port</anno> ! {PortOwner, {connect, <anno>Pid</anno>}}</c> just as if it itself was the port owner, but the reply always goes to the port owner.</p> - <p>In short: <c>port_connect(<anno>Port</anno>, <anno>Pid</anno>)</c> has a cleaner and - more logical behaviour than - <c><anno>Port</anno> ! {self(),{connect,<anno>Pid</anno>}}</c>.</p> + <p>As of OTP-R16 <c><anno>Port</anno> ! {PortOwner, {connect, <anno>Pid</anno>}}</c> is + truly asynchronous. Note that this operation has always been + documented as an asynchronous operation, while the underlying + implementation has been synchronous. <c>port_connect/2</c> is + however still fully synchronous. This due to its error + behavior.</p> <p>Failure: <c>badarg</c> if <c><anno>Port</anno></c> is not an open port - or the registered name of an open port, or if <c><anno>Pid</anno></c> is + or the registered name of an open port, or if <c>Pid</c> is not an existing local pid.</p> </desc> </func> @@ -3084,71 +3137,187 @@ os_prompt% </pre> </func> <func> <name name="port_info" arity="1"/> - <type name="port_info_result_item"/> <fsummary>Information about a port</fsummary> <desc> <p>Returns a list containing tuples with information about the <c><anno>Port</anno></c>, or <c>undefined</c> if the port is not open. The order of the tuples is not defined, nor are all the tuples mandatory.</p> - <taglist> - <tag><c>{registered_name, <anno>RegName</anno>}</c></tag> - <item> - <p><c><anno>RegName</anno></c> (an atom) is the registered name of - the port. If the port has no registered name, this tuple - is not present in the list.</p> - </item> - <tag><c>{id, <anno>Index</anno>}</c></tag> - <item> - <p><c><anno>Index</anno></c> (an integer) is the internal index of the - port. This index may be used to separate ports.</p> - </item> - <tag><c>{connected, <anno>Pid</anno>}</c></tag> - <item> - <p><c><anno>Pid</anno></c> is the process connected to the port.</p> - </item> - <tag><c>{links, <anno>Pids</anno>}</c></tag> - <item> - <p><c><anno>Pids</anno></c> is a list of pids to which processes the - port is linked.</p> - </item> - <tag><c>{name, <anno>String</anno>}</c></tag> - <item> - <p><c><anno>String</anno></c> is the command name set by - <c>open_port</c>.</p> - </item> - <tag><c>{input, <anno>Bytes</anno>}</c></tag> - <item> - <p><c><anno>Bytes</anno></c> is the total number of bytes read from - the port.</p> - </item> - <tag><c>{output, <anno>Bytes</anno>}</c></tag> - <item> - <p><c><anno>Bytes</anno></c> is the total number of bytes written to - the port.</p> - </item> - <tag><c>{os_pid, <anno>OsPid</anno> | undefined}</c></tag> - <item> - <p><c> <anno>OsPid</anno></c> is the process identifier (or equivalent) of an OS process created with <c>open_port({spawn | spawn_executable, Command}, Options)</c>. If the port is not the result of spawning an OS process, the value is <c>undefined</c>.</p> - </item> - </taglist> - <p>Failure: <c>badarg</c> if <c><anno>Port</anno></c> is not a local port.</p> + <p>Currently the result will containt information about the + following <c>Item</c>s: <c>registered_name</c> (if the port has + a registered name), <c>id</c>, <c>connected</c>, <c>links</c>, + <c>name</c>, <c>input</c>, and <c>output</c>. For more information + about the different <c>Item</c>s, see + <seealso marker="#port_info/2">port_info/2</seealso>.</p> + <p>Failure: <c>badarg</c> if <c>Port</c> is not a local port + identifier, or an atom.</p> </desc> </func> <func> - <name name="port_info" arity="2"/> - <type name="port_info_item"/> - <type name="port_info_result_item"/> - <fsummary>Information about a port</fsummary> + <name name="port_info" arity="2" clause_i="1"/> + <fsummary>Information about the connected process of a port</fsummary> <desc> - <p>Returns information about <c><anno>Port</anno></c> as specified - by <c><anno>Item</anno></c>, or <c>undefined</c> if the port is not open. - Also, if <c>Item =:= registered_name</c> and the port has no - registered name, <c>[]</c> is returned.</p> - <p>For valid values of <c><anno>Item</anno></c>, and corresponding - values of <c><anno>Result</anno></c>, see - <seealso marker="#port_info/1">erlang:port_info/1</seealso>.</p> - <p>Failure: <c>badarg</c> if <c><anno>Port</anno></c> is not a local port.</p> + <p><c><anno>Pid</anno></c> is the process identifier of the process + connected to the port.</p> + <p>If the port identified by <c><anno>Port</anno></c> is not open, + <c>undefined</c> is returned.</p> + <p>Failure: <c>badarg</c> if <c><anno>Port</anno></c> is not a local + port identifier, or an atom.</p> + </desc> + </func> + <func> + <name name="port_info" arity="2" clause_i="2"/> + <fsummary>Information about the internal index of a port</fsummary> + <desc> + <p><c><anno>Index</anno></c> is the internal index of the port. This + index may be used to separate ports.</p> + <p>If the port identified by <c><anno>Port</anno></c> is not open, + <c>undefined</c> is returned.</p> + <p>Failure: <c>badarg</c> if <c><anno>Port</anno></c> is not a local + port identifier, or an atom.</p> + </desc> + </func> + <func> + <name name="port_info" arity="2" clause_i="3"/> + <fsummary>Information about the input of a port</fsummary> + <desc> + <p><c><anno>Bytes</anno></c> is the total number of bytes + read from the port.</p> + <p>If the port identified by <c><anno>Port</anno></c> is not open, + <c>undefined</c> is returned.</p> + <p>Failure: <c>badarg</c> if <c><anno>Port</anno></c> is not a local + port identifier, or an atom.</p> + </desc> + </func> + <func> + <name name="port_info" arity="2" clause_i="4"/> + <fsummary>Information about the links of a port</fsummary> + <desc> + <p><c><anno>Pids</anno></c> is a list of the process identifiers + of the processes that the port is linked to.</p> + <p>If the port identified by <c><anno>Port</anno></c> is not open, + <c>undefined</c> is returned.</p> + <p>Failure: <c>badarg</c> if <c><anno>Port</anno></c> is not a local + port identifier, or an atom.</p> + </desc> + </func> + <func> + <name name="port_info" arity="2" clause_i="5"/> + <fsummary>Information about the locking of a port</fsummary> + <desc> + <p><c><anno>Locking</anno></c> is currently either <c>false</c> + (emulator without SMP support), <c>port_level</c> (port specific + locking), or <c>driver_level</c> (driver specific locking). Note + that these results are highly implementation specific and might + change in the future.</p> + <p>If the port identified by <c><anno>Port</anno></c> is not open, + <c>undefined</c> is returned.</p> + <p>Failure: <c>badarg</c> if <c><anno>Port</anno></c> is not a local + port identifier, or an atom.</p> + </desc> + </func> + <func> + <name name="port_info" arity="2" clause_i="6"/> + <fsummary>Information about the memory size of a port</fsummary> + <desc> + <p><c><anno>Bytes</anno></c> is the total amount of memory, + in bytes, allocated for this port by the runtime system. Note + that the port itself might have allocated memory which is not + included in <c><anno>Bytes</anno></c>.</p> + <p>If the port identified by <c><anno>Port</anno></c> is not open, + <c>undefined</c> is returned.</p> + <p>Failure: <c>badarg</c> if <c><anno>Port</anno></c> is not a local + port identifier, or an atom.</p> + </desc> + </func> + <func> + <name name="port_info" arity="2" clause_i="7"/> + <fsummary>Information about the monitors of a port</fsummary> + <desc> + <p><c><anno>Monitors</anno></c> represent processes that this port + is monitoring.</p> + <p>If the port identified by <c><anno>Port</anno></c> is not open, + <c>undefined</c> is returned.</p> + <p>Failure: <c>badarg</c> if <c><anno>Port</anno></c> is not a local + port identifier, or an atom.</p> + </desc> + </func> + <func> + <name name="port_info" arity="2" clause_i="8"/> + <fsummary>Information about the name of a port</fsummary> + <desc> + <p><c><anno>Name</anno></c> is the command name set by + <seealso marker="#open_port/2">open_port/2</seealso>.</p> + <p>If the port identified by <c><anno>Port</anno></c> is not open, + <c>undefined</c> is returned.</p> + <p>Failure: <c>badarg</c> if <c><anno>Port</anno></c> is not a local + port identifier, or an atom.</p> + </desc> + </func> + <func> + <name name="port_info" arity="2" clause_i="9"/> + <fsummary>Information about the OS pid of a port</fsummary> + <desc> + <p><c><anno>OsPid</anno></c> is the process identifier (or equivalent) + of an OS process created with + <seealso marker="#open_port/2">open_port({spawn | spawn_executable, + Command}, Options)</seealso>. If the port is not the result of spawning + an OS process, the value is <c>undefined</c>.</p> + <p>If the port identified by <c><anno>Port</anno></c> is not open, + <c>undefined</c> is returned.</p> + <p>Failure: <c>badarg</c> if <c><anno>Port</anno></c> is not a local + port identifier, or an atom.</p> + </desc> + </func> + <func> + <name name="port_info" arity="2" clause_i="10"/> + <fsummary>Information about the output of a port</fsummary> + <desc> + <p><c><anno>Bytes</anno></c> is the total number of bytes written + to the port from Erlang processes using either + <seealso marker="#port_command/2">port_command/2</seealso>, + <seealso marker="#port_command/3">port_command/3</seealso>, + or <c><anno>Port</anno> ! {Owner, {command, Data}</c>. + </p> + <p>If the port identified by <c><anno>Port</anno></c> is not open, + <c>undefined</c> is returned.</p> + <p>Failure: <c>badarg</c> if <c><anno>Port</anno></c> is not a local + port identifier, or an atom.</p> + </desc> + </func> + <func> + <name name="port_info" arity="2" clause_i="11"/> + <fsummary>Information about the parallelism hint of a port</fsummary> + <desc> + <p><c><anno>Boolean</anno></c> corresponds to the port parallelism + hint being used by this port. For more information see + the <seealso marker="#open_port_parallelism">parallelism</seealso> + option of <seealso marker="#open_port/2">open_port/2</seealso>.</p> + </desc> + </func> + <func> + <name name="port_info" arity="2" clause_i="12"/> + <fsummary>Information about the queue size of a port</fsummary> + <desc> + <p><c><anno>Bytes</anno></c> is the total amount of data, + in bytes, queued by the port using the ERTS driver queue + implementation.</p> + <p>If the port identified by <c><anno>Port</anno></c> is not open, + <c>undefined</c> is returned.</p> + <p>Failure: <c>badarg</c> if <c><anno>Port</anno></c> is not a local + port identifier, or an atom.</p> + </desc> + </func> + <func> + <name name="port_info" arity="2" clause_i="13"/> + <fsummary>Information about the registered name of a port</fsummary> + <desc> + <p><c><anno>RegisteredName</anno></c> is the registered name of + the port. If the port has no registered name, <c>[]</c> is returned.</p> + <p>If the port identified by <c><anno>Port</anno></c> is not open, + <c>undefined</c> is returned.</p> + <p>Failure: <c>badarg</c> if <c><anno>Port</anno></c> is not a local + port identifier, or an atom.</p> </desc> </func> <func> @@ -3168,7 +3337,10 @@ os_prompt% </pre> <name name="ports" arity="0"/> <fsummary>All open ports</fsummary> <desc> - <p>Returns a list of all ports on the local node.</p> + <p>Returns a list of port identifiers corresponding to all the + ports currently existing on the local node.</p> + + <p>Note that a port that is exiting, exists but is not open.</p> </desc> </func> <func> @@ -3534,10 +3706,11 @@ os_prompt% </pre> the initial function call with which the process was spawned.</p> </item> - <tag><c>{links, <anno>Pids</anno>}</c></tag> + <tag><c>{links, <anno>PidsAndPorts</anno>}</c></tag> <item> - <p><c><anno>Pids</anno></c> is a list of pids, with processes to - which the process has a link.</p> + <p><c><anno>PidsAndPorts</anno></c> is a list of pids and + port identifiers, with processes or ports to which the process + has a link.</p> </item> <tag><c>{last_calls, false|Calls}</c></tag> <item> @@ -5501,19 +5674,40 @@ ok <item> <p>Returns a string containing the OTP release number.</p> </item> + <tag><marker id="system_info_port_parallelism"><c>port_parallelism</c></marker></tag> + <item><p>Returns the default port parallelism scheduling hint used. + For more information see the + <seealso marker="erl#+spp">+spp</seealso> command line argument + of <seealso marker="erl">erl(1)</seealso>.</p></item> + <tag><c>process_count</c></tag> + <item> + <p>Returns the number of ports currently existing at + the local node as an integer. The same value as + <c>length(erlang:ports())</c> returns.</p> + </item> + <tag><marker id="system_info_port_limit"><c>port_limit</c></marker></tag> + <item> + <p>Returns the maximum number of simultaneously existing + ports at the local node as an integer. This limit + can be configured at startup by using the + <seealso marker="erl#max_ports"><c>+Q</c></seealso> + command line flag of + <seealso marker="erl"><c>erl(1)</c></seealso>.</p> + </item> <tag><c>process_count</c></tag> <item> <p>Returns the number of processes currently existing at the local node as an integer. The same value as <c>length(processes())</c> returns.</p> </item> - <tag><c>process_limit</c></tag> + <tag><marker id="system_info_process_limit"><c>process_limit</c></marker></tag> <item> - <p>Returns the maximum number of concurrently existing + <p>Returns the maximum number of simultaneously existing processes at the local node as an integer. This limit - can be configured at startup by using the command line - flag <c>+P</c>, see - <seealso marker="erts:erl#max_processes">erl(1)</seealso>.</p> + can be configured at startup by using the + <seealso marker="erl#max_processes"><c>+P</c></seealso> + command line flag of + <seealso marker="erl"><c>erl(1)</c></seealso>.</p> </item> <tag><c>procs</c></tag> <item> diff --git a/erts/doc/src/erts_alloc.xml b/erts/doc/src/erts_alloc.xml index ec5e7d9b74..87d6682328 100644 --- a/erts/doc/src/erts_alloc.xml +++ b/erts/doc/src/erts_alloc.xml @@ -341,7 +341,8 @@ Largest (<c>mseg_alloc</c>) multiblock carrier size (in kilobytes). See <seealso marker="#mseg_mbc_sizes">the description on how sizes for mseg_alloc multiblock carriers are decided</seealso> - in "the <c>alloc_util</c> framework" section.</item> + in "the <c>alloc_util</c> framework" section. On 32-bit Unix style OS + this limit can not be set higher than 128 megabyte.</item> <tag><marker id="M_mbcgs"><c><![CDATA[+M<S>mbcgs <ratio>]]></c></marker></tag> <item> (<c>mseg_alloc</c>) multiblock carrier growth stages. See @@ -413,7 +414,8 @@ Singleblock carrier threshold. Blocks larger than this threshold will be placed in singleblock carriers. Blocks smaller than this threshold will be placed in multiblock - carriers.</item> + carriers. On 32-bit Unix style OS this threshold can not be set higher + than 8 megabytes.</item> <tag><marker id="M_sbmbcs"><c><![CDATA[+M<S>sbmbcs <size>]]></c></marker></tag> <item> Small block multiblock carrier size (in bytes). Memory blocks smaller diff --git a/erts/doc/src/notes.xml b/erts/doc/src/notes.xml index cdb72b2b98..e996d3e8e3 100644 --- a/erts/doc/src/notes.xml +++ b/erts/doc/src/notes.xml @@ -30,6 +30,179 @@ </header> <p>This document describes the changes made to the ERTS application.</p> +<section><title>Erts 5.9.3.1</title> + + <section><title>Known Bugs and Problems</title> + <list> + <item> + <p> + Create an erl_crash.dump if no heart exists and no + ERL_CRASH_DUMP_SECONDS is set (behaviour changed).</p> + <p> + Don't create an erl_crash.dump if heart do exists and no + ERL_CRASH_DUMP_SECONDS is set (behaviour not changed).</p> + <p> + This changes the behaviour back to the R15B02 default + considering if a beam was running with no heart.</p> + <p> + Own Id: OTP-10602</p> + </item> + </list> + </section> + +</section> + +<section><title>Erts 5.9.3</title> + + <section><title>Fixed Bugs and Malfunctions</title> + <list> + <item> + <p> + Fix linking in OpenBSD. (Thanks to Matthew Dempsky)</p> + <p> + Own Id: OTP-10395</p> + </item> + <item> + <p> + Fix bug causing fallback atomics to be used even though + healthy gcc atomics or libatomic_ops was detected.</p> + <p> + Own Id: OTP-10418</p> + </item> + <item> + <p> + Ensure 'erl_crash.dump' when asked for it. This will + change erl_crash.dump behaviour.</p> + <p> + * Not setting ERL_CRASH_DUMP_SECONDS will now terminate + beam immediately on a crash without writing a crash dump + file.</p> + <p> + * Setting ERL_CRASH_DUMP_SECONDS to 0 will also terminate + beam immediately on a crash without writing a crash dump + file, i.e. same as not setting ERL_CRASH_DUMP_SECONDS + environment variable.</p> + <p> + * Setting ERL_CRASH_DUMP_SECONDS to a negative value will + let the beam wait indefinitely on the crash dump file + being written.</p> + <p> + * Setting ERL_CRASH_DUMP_SECONDS to a positive value will + let the beam wait that many seconds on the crash dump + file being written.</p> + <p> + A positive value will set an alarm/timeout for restart + both in beam and in heart if heart is running.</p> + <p> + *** POTENTIAL INCOMPATIBILITY ***</p> + <p> + Own Id: OTP-10422 Aux Id: kunagi-250 [161] </p> + </item> + <item> + <p> + Fix bug where MSVRT100.dll was not included in the + windows installer.</p> + <p> + Own Id: OTP-10481</p> + </item> + <item> + <p>In the expression + <c><<Bin/binary,...>></c>, if <c>Bin</c> was + a bitstring with a size not a multiple of 8, either no + exception was generated or an incorrect exception was + generated. (Thanks to Adam Rutkowski for reporting this + bug.)</p> + <p> + Own Id: OTP-10524</p> + </item> + <item> + <p> + The runtime system could crash while scheduling a port + task. The port task was scheduled either due to an + external I/O event being triggered, a driver timeout + being triggered, or data being sent over a distribution + channel.</p> + <p> + Own Id: OTP-10556</p> + </item> + <item> + <p> + <c>erlang:memory(ets)</c> erroneously included the size + of each ETS-table main structure twice.</p> + <p> + Own Id: OTP-10558</p> + </item> + <item> + <p> + Fix compile error in generated file hipe_amd64_bifs.S for + Solaris.</p> + <p> + Own Id: OTP-10577</p> + </item> + <item> + <p> + A faulty spec for process_info/2 could cause false + dialyzer warnings. The spec is corrected.</p> + <p> + Own Id: OTP-10584</p> + </item> + <item> + <p> + In very rare cases, the VM could crash if a garbage + collector was called while executing an appending bit + syntax instruction. The symptom was a core when + reallocating memory in the function erts_bs_append. The + garbage collector bug is now corrected.</p> + <p> + Own Id: OTP-10590</p> + </item> + </list> + </section> + + + <section><title>Improvements and New Features</title> + <list> + <item> + <p> + Improve support for building and testing in embedded ppc + environments.</p> + <p> + Own Id: OTP-10265 Aux Id: kunagi-159 + [daf97f67-5724-4812-a5b6-7e86990133d2-1] </p> + </item> + <item> + <p> + Due to a race condition on Windows, sometimes when + printing to standard output and then immediately + terminating erlang all data would not be printed. The + emulator now waits for all data to be printed before + exiting.</p> + <p> + Own Id: OTP-10325 Aux Id: kunagi-166 + [dd72d0e2-3e76-4a51-8b56-7564e24eecae] </p> + </item> + <item> + <p> + The frequency with which sleeping schedulers are woken + due to outstanding memory deallocation jobs has been + reduced.</p> + <p> + Own Id: OTP-10476 Aux Id: OTP-10162 </p> + </item> + <item> + <p> + Clearer warnings about the dangers of misuse of <seealso + marker="erl_nif#WARNING">native functions</seealso> and + <seealso marker="erl_driver#WARNING">drivers</seealso> + have been added to the documentation.</p> + <p> + Own Id: OTP-10557</p> + </item> + </list> + </section> + +</section> + <section><title>Erts 5.9.2</title> <section><title>Fixed Bugs and Malfunctions</title> diff --git a/erts/doc/src/part.xml b/erts/doc/src/part.xml index e27b722721..2c178556d4 100644 --- a/erts/doc/src/part.xml +++ b/erts/doc/src/part.xml @@ -31,6 +31,7 @@ <description> <p>The Erlang Runtime System Application <em>ERTS</em>.</p> </description> + <xi:include href="communication.xml"/> <xi:include href="match_spec.xml"/> <xi:include href="crash_dump.xml"/> <xi:include href="alt_dist.xml"/> diff --git a/erts/emulator/Makefile.in b/erts/emulator/Makefile.in index 7e966c81bb..89c948cc00 100644 --- a/erts/emulator/Makefile.in +++ b/erts/emulator/Makefile.in @@ -61,7 +61,7 @@ else ifeq ($(TYPE),purify) PURIFY = purify $(PURIFY_BUILD_OPTIONS) TYPEMARKER = .purify -TYPE_FLAGS = $(DEBUG_CFLAGS) -DPURIFY -DNO_JUMP_TABLE -DERTS_MSEG_FAKE_SEGMENTS +TYPE_FLAGS = $(DEBUG_CFLAGS) -DPURIFY -DNO_JUMP_TABLE ENABLE_ALLOC_TYPE_VARS += purify else @@ -92,7 +92,7 @@ else ifeq ($(TYPE),valgrind) PURIFY = TYPEMARKER = .valgrind -TYPE_FLAGS = $(DEBUG_CFLAGS) -DVALGRIND -DNO_JUMP_TABLE -DERTS_MSEG_FAKE_SEGMENTS +TYPE_FLAGS = $(DEBUG_CFLAGS) -DVALGRIND -DNO_JUMP_TABLE ENABLE_ALLOC_TYPE_VARS += valgrind else @@ -560,7 +560,8 @@ $(PRELOAD_SRC): $(ERL_TOP)/erts/preloaded/ebin/otp_ring0.beam \ $(ERL_TOP)/erts/preloaded/ebin/zlib.beam \ $(ERL_TOP)/erts/preloaded/ebin/prim_zip.beam \ $(ERL_TOP)/erts/preloaded/ebin/erl_prim_loader.beam \ - $(ERL_TOP)/erts/preloaded/ebin/erlang.beam + $(ERL_TOP)/erts/preloaded/ebin/erlang.beam \ + $(ERL_TOP)/erts/preloaded/ebin/erts_internal.beam LANG=C $(PERL) utils/make_preload $(MAKE_PRELOAD_EXTRA) -rc $^ > $@ else PRELOAD_OBJ = $(OBJDIR)/preload.o @@ -572,7 +573,8 @@ $(PRELOAD_SRC): $(ERL_TOP)/erts/preloaded/ebin/otp_ring0.beam \ $(ERL_TOP)/erts/preloaded/ebin/zlib.beam \ $(ERL_TOP)/erts/preloaded/ebin/prim_zip.beam \ $(ERL_TOP)/erts/preloaded/ebin/erl_prim_loader.beam \ - $(ERL_TOP)/erts/preloaded/ebin/erlang.beam + $(ERL_TOP)/erts/preloaded/ebin/erlang.beam \ + $(ERL_TOP)/erts/preloaded/ebin/erts_internal.beam LANG=C $(PERL) utils/make_preload -old $^ > $@ endif @@ -735,7 +737,8 @@ RUN_OBJS = \ $(OBJDIR)/packet_parser.o $(OBJDIR)/safe_hash.o \ $(OBJDIR)/erl_zlib.o $(OBJDIR)/erl_nif.o \ $(OBJDIR)/erl_bif_binary.o $(OBJDIR)/erl_ao_firstfit_alloc.o \ - $(OBJDIR)/erl_thr_queue.o $(OBJDIR)/erl_sched_spec_pre_alloc.o + $(OBJDIR)/erl_thr_queue.o $(OBJDIR)/erl_sched_spec_pre_alloc.o \ + $(OBJDIR)/erl_ptab.o ifeq ($(TARGET),win32) DRV_OBJS = \ diff --git a/erts/emulator/beam/atom.names b/erts/emulator/beam/atom.names index afcbd732df..c47a608215 100644 --- a/erts/emulator/beam/atom.names +++ b/erts/emulator/beam/atom.names @@ -94,6 +94,7 @@ atom asynchronous atom atom atom atom_used atom attributes +atom await_port_send_result atom await_proc_exit atom await_sched_wall_time_modifications atom awaiting_load @@ -152,6 +153,7 @@ atom connection_closed atom cons atom const atom context_switches +atom control atom copy atom cpu atom cpu_timestamp @@ -204,6 +206,7 @@ atom erlang atom ERROR='ERROR' atom error_handler atom error_logger +atom erts_internal atom ets atom ETS_TRANSFER='ETS-TRANSFER' atom event @@ -237,6 +240,7 @@ atom gc_end atom gc_start atom Ge='>=' atom generational +atom get_data atom get_seq_token atom get_tcw atom getenv @@ -408,6 +412,7 @@ atom overlapped_io atom owner atom packet atom packet_size +atom parallelism atom Plus='+' atom pause atom pending @@ -419,12 +424,12 @@ atom pid atom port atom ports atom port_count +atom port_limit atom print atom priority atom private atom process atom processes -atom processes_trap atom processes_used atom process_count atom process_display @@ -434,6 +439,7 @@ atom procs atom profile atom protected atom protection +atom ptab_list_continue atom public atom purify atom quantify @@ -481,6 +487,7 @@ atom sequential_trace_token atom serial atom set atom set_cpu_topology +atom set_data atom set_on_first_link atom set_on_first_spawn atom set_on_link diff --git a/erts/emulator/beam/beam_bif_load.c b/erts/emulator/beam/beam_bif_load.c index 9e4add823d..e0a4f86d2d 100644 --- a/erts/emulator/beam/beam_bif_load.c +++ b/erts/emulator/beam/beam_bif_load.c @@ -438,8 +438,7 @@ check_process_code_2(BIF_ALIST_2) if (is_internal_pid(BIF_ARG_1)) { Eterm res; ErtsCodeIndex code_ix; - if (internal_pid_index(BIF_ARG_1) >= erts_max_processes) - goto error; + code_ix = erts_active_code_ix(); modp = erts_get_module(BIF_ARG_2, code_ix); if (modp == NULL) { /* Doesn't exist. */ diff --git a/erts/emulator/beam/beam_bp.c b/erts/emulator/beam/beam_bp.c index 58e0090a76..c1e11f6448 100644 --- a/erts/emulator/beam/beam_bp.c +++ b/erts/emulator/beam/beam_bp.c @@ -65,10 +65,10 @@ #define ERTS_BPF_ALL 0xFF -extern Eterm beam_return_to_trace[1]; /* OpCode(i_return_to_trace) */ -extern Eterm beam_return_trace[1]; /* OpCode(i_return_trace) */ -extern Eterm beam_exception_trace[1]; /* OpCode(i_exception_trace) */ -extern Eterm beam_return_time_trace[1]; /* OpCode(i_return_time_trace) */ +extern BeamInstr beam_return_to_trace[1]; /* OpCode(i_return_to_trace) */ +extern BeamInstr beam_return_trace[1]; /* OpCode(i_return_trace) */ +extern BeamInstr beam_exception_trace[1]; /* OpCode(i_exception_trace) */ +extern BeamInstr beam_return_time_trace[1]; /* OpCode(i_return_time_trace) */ erts_smp_atomic32_t erts_active_bp_index; erts_smp_atomic32_t erts_staging_bp_index; @@ -161,7 +161,7 @@ erts_bp_match_functions(BpFunctions* f, Eterm mfa[3], int specified) for (current = 0; current < num_modules; current++) { BeamInstr** code_base = (BeamInstr **) module[current]->curr.code; BeamInstr* code; - Uint num_functions = (Uint) code_base[MI_NUM_FUNCTIONS]; + Uint num_functions = (Uint)(UWord) code_base[MI_NUM_FUNCTIONS]; Uint fi; if (specified > 0) { @@ -172,7 +172,7 @@ erts_bp_match_functions(BpFunctions* f, Eterm mfa[3], int specified) } for (fi = 0; fi < num_functions; fi++) { - Eterm* pc; + BeamInstr* pc; int wi; code = code_base[MI_FUNCTIONS+fi]; @@ -555,7 +555,7 @@ erts_clear_module_break(Module *modp) { if (code_base == NULL) { return 0; } - n = (Uint) code_base[MI_NUM_FUNCTIONS]; + n = (Uint)(UWord) code_base[MI_NUM_FUNCTIONS]; for (i = 0; i < n; ++i) { BeamInstr* pc; @@ -714,7 +714,7 @@ erts_bif_trace(int bif_index, Process* p, Eterm* args, BeamInstr* I) IS_TRACED_FL(p, F_TRACE_CALLS)) { int local = !!(bp_flags & ERTS_BPF_LOCAL_TRACE); flags = erts_call_trace(p, ep->code, bp->local_ms, args, - local, &p->tracer_proc); + local, &ERTS_TRACER_PROC(p)); } if (bp_flags & ERTS_BPF_META_TRACE) { Eterm tpid1, tpid2; @@ -800,7 +800,7 @@ erts_bif_trace(int bif_index, Process* p, Eterm* args, BeamInstr* I) } if (flags & MATCH_SET_EXCEPTION_TRACE) { erts_trace_exception(p, ep->code, class, value, - &p->tracer_proc); + &ERTS_TRACER_PROC(p)); } if ((flags & MATCH_SET_RETURN_TO_TRACE) && p->catches > 0) { /* can only happen if(local)*/ @@ -825,7 +825,7 @@ erts_bif_trace(int bif_index, Process* p, Eterm* args, BeamInstr* I) UnUseTmpHeapNoproc(3); if ((flags_meta|flags) & MATCH_SET_EXCEPTION_TRACE) { erts_smp_proc_lock(p, ERTS_PROC_LOCKS_ALL_MINOR); - p->trace_flags |= F_EXCEPTION_TRACE; + ERTS_TRACE_FLAGS(p) |= F_EXCEPTION_TRACE; erts_smp_proc_unlock(p, ERTS_PROC_LOCKS_ALL_MINOR); } } @@ -835,7 +835,7 @@ erts_bif_trace(int bif_index, Process* p, Eterm* args, BeamInstr* I) } /* MATCH_SET_RETURN_TO_TRACE cannot occur if(meta) */ if (flags & MATCH_SET_RX_TRACE) { - erts_trace_return(p, ep->code, result, &p->tracer_proc); + erts_trace_return(p, ep->code, result, &ERTS_TRACER_PROC(p)); } if (flags & MATCH_SET_RETURN_TO_TRACE) { /* can only happen if(local)*/ @@ -919,7 +919,7 @@ do_call_trace(Process* c_p, BeamInstr* I, Eterm* reg, E -= 1; ASSERT(c_p->htop <= E && E <= c_p->hend); E[0] = make_cp(c_p->cp); - c_p->cp = (BeamInstr *) beam_return_to_trace; + c_p->cp = beam_return_to_trace; } if (flags & MATCH_SET_RX_TRACE) { E -= 3; @@ -935,7 +935,7 @@ do_call_trace(Process* c_p, BeamInstr* I, Eterm* reg, c_p->cp = (flags & MATCH_SET_EXCEPTION_TRACE) ? beam_exception_trace : beam_return_trace; erts_smp_proc_lock(c_p, ERTS_PROC_LOCKS_ALL_MINOR); - c_p->trace_flags |= F_EXCEPTION_TRACE; + ERTS_TRACE_FLAGS(c_p) |= F_EXCEPTION_TRACE; erts_smp_proc_unlock(c_p, ERTS_PROC_LOCKS_ALL_MINOR); } c_p->stop = E; @@ -974,7 +974,7 @@ erts_trace_time_call(Process* c_p, BeamInstr* I, BpDataTime* bdt) ASSERT(pbt->pc); /* add time to previous code */ bp_time_diff(&sitem, pbt, ms, s, us); - sitem.pid = c_p->id; + sitem.pid = c_p->common.id; sitem.count = 0; /* previous breakpoint */ @@ -997,7 +997,7 @@ erts_trace_time_call(Process* c_p, BeamInstr* I, BpDataTime* bdt) } /* Add count to this code */ - sitem.pid = c_p->id; + sitem.pid = c_p->common.id; sitem.count = 1; sitem.s_time = 0; sitem.us_time = 0; @@ -1055,7 +1055,7 @@ erts_trace_time_return(Process *p, BeamInstr *pc) ASSERT(pbt->pc); bp_time_diff(&sitem, pbt, ms, s, us); - sitem.pid = p->id; + sitem.pid = p->common.id; sitem.count = 0; /* previous breakpoint */ @@ -1386,7 +1386,7 @@ void erts_schedule_time_break(Process *p, Uint schedule) { if (pbdt) { get_sys_now(&ms,&s,&us); bp_time_diff(&sitem, pbt, ms, s, us); - sitem.pid = p->id; + sitem.pid = p->common.id; sitem.count = 0; h = &(pbdt->hash[bp_sched2ix_proc(p)]); diff --git a/erts/emulator/beam/beam_emu.c b/erts/emulator/beam/beam_emu.c index 7b1ae624ce..0e9d140908 100644 --- a/erts/emulator/beam/beam_emu.c +++ b/erts/emulator/beam/beam_emu.c @@ -516,7 +516,7 @@ extern int count_instructions; # define Dispatchfun() DispatchMacroFun() #endif -#define Self(R) R = c_p->id +#define Self(R) R = c_p->common.id #define Node(R) R = erts_this_node->sysname #define Arg(N) I[(N)+1] @@ -1074,11 +1074,11 @@ init_emulator(void) void dtrace_drvport_str(ErlDrvPort drvport, char *port_buf) { - Port *port = erts_drvport2port(drvport); + Port *port = erts_drvport2port(drvport, NULL); erts_snprintf(port_buf, DTRACE_TERM_BUF_SIZE, "#Port<%lu.%lu>", - port_channel_no(port->id), - port_number(port->id)); + port_channel_no(port->common.id), + port_number(port->common.id)); } #endif /* @@ -1195,7 +1195,7 @@ void process_main(void) c_p = schedule(c_p, reds_used); ERTS_VERIFY_UNUSED_TEMP_ALLOC(c_p); #ifdef DEBUG - pid = c_p->id; /* Save for debugging purpouses */ + pid = c_p->common.id; /* Save for debugging purpouses */ #endif ERTS_SMP_REQ_PROC_MAIN_LOCK(c_p); PROCESS_MAIN_CHK_LOCKS(c_p); @@ -1227,7 +1227,7 @@ void process_main(void) reds = c_p->fcalls; if (ERTS_PROC_GET_SAVED_CALLS_BUF(c_p) - && (c_p->trace_flags & F_SENSITIVE) == 0) { + && (ERTS_TRACE_FLAGS(c_p) & F_SENSITIVE) == 0) { neg_o_reds = -reds; FCALLS = REDS_IN(c_p) = 0; } else { @@ -1591,6 +1591,7 @@ void process_main(void) reg[0] = r(0); result = erl_send(c_p, r(0), x(1)); PreFetch(0, next); + ERTS_VERIFY_UNUSED_TEMP_ALLOC(c_p); ERTS_SMP_REQ_PROC_MAIN_LOCK(c_p); PROCESS_MAIN_CHK_LOCKS(c_p); if (c_p->mbuf || MSO(c_p).overhead >= BIN_VHEAP_SZ(c_p)) { @@ -1866,14 +1867,14 @@ void process_main(void) erts_fprintf(stderr, "Dtrace -> (%T) stop spreading " "tag %T with message %T\r\n", - c_p->id,DT_UTAG(c_p),ERL_MESSAGE_TERM(msgp)); + c_p->common.id,DT_UTAG(c_p),ERL_MESSAGE_TERM(msgp)); #endif } else { #ifdef DTRACE_TAG_HARDDEBUG erts_fprintf(stderr, "Dtrace -> (%T) kill tag %T with " "message %T\r\n", - c_p->id,DT_UTAG(c_p),ERL_MESSAGE_TERM(msgp)); + c_p->common.id,DT_UTAG(c_p),ERL_MESSAGE_TERM(msgp)); #endif DT_UTAG(c_p) = NIL; SEQ_TRACE_TOKEN(c_p) = NIL; @@ -1898,7 +1899,7 @@ void process_main(void) erts_fprintf(stderr, "Dtrace -> (%T) receive tag (%T) " "with message %T\r\n", - c_p->id, DT_UTAG(c_p), ERL_MESSAGE_TERM(msgp)); + c_p->common.id, DT_UTAG(c_p), ERL_MESSAGE_TERM(msgp)); #endif } else { #endif @@ -1914,7 +1915,7 @@ void process_main(void) } msg = ERL_MESSAGE_TERM(msgp); seq_trace_output(SEQ_TRACE_TOKEN(c_p), msg, SEQ_TRACE_RECEIVE, - c_p->id, c_p); + c_p->common.id, c_p); #ifdef USE_VM_PROBES } #endif @@ -2567,6 +2568,7 @@ void process_main(void) reg[0] = r(0); result = (*bf)(c_p, reg, I); ASSERT(!ERTS_PROC_IS_EXITING(c_p) || is_non_value(result)); + ERTS_VERIFY_UNUSED_TEMP_ALLOC(c_p); ERTS_HOLE_CHECK(c_p); ERTS_SMP_REQ_PROC_MAIN_LOCK(c_p); PROCESS_MAIN_CHK_LOCKS(c_p); @@ -3301,7 +3303,6 @@ void process_main(void) PROCESS_MAIN_CHK_LOCKS(c_p); bif_nif_arity = I[-1]; ERTS_SMP_UNREQ_PROC_MAIN_LOCK(c_p); - ERTS_VERIFY_UNUSED_TEMP_ALLOC(c_p); ASSERT(!ERTS_PROC_IS_EXITING(c_p)); { @@ -3346,7 +3347,6 @@ void process_main(void) bif_nif_arity = I[-1]; ASSERT(bif_nif_arity <= 3); ERTS_SMP_UNREQ_PROC_MAIN_LOCK(c_p); - ERTS_VERIFY_UNUSED_TEMP_ALLOC(c_p); reg[0] = r(0); { Eterm (*bf)(Process*, Eterm*, BeamInstr*) = vbf; @@ -5257,7 +5257,7 @@ terminate_proc(Process* c_p, Eterm Value) /* EXF_LOG is a primary exception flag */ if (c_p->freason & EXF_LOG) { erts_dsprintf_buf_t *dsbufp = erts_create_logger_dsbuf(); - erts_dsprintf(dsbufp, "Error in process %T ", c_p->id); + erts_dsprintf(dsbufp, "Error in process %T ", c_p->common.id); if (erts_is_alive) erts_dsprintf(dsbufp, "on node %T ", erts_this_node->sysname); erts_dsprintf(dsbufp,"with exit value: %0.*T\n", display_items, Value); @@ -6186,7 +6186,7 @@ new_fun(Process* p, Eterm* reg, ErlFunEntry* fe, int num_free) MSO(p).first = (struct erl_off_heap_header*) funp; funp->fe = fe; funp->num_free = num_free; - funp->creator = p->id; + funp->creator = p->common.id; #ifdef HIPE funp->native_address = fe->native_address; #endif diff --git a/erts/emulator/beam/bif.c b/erts/emulator/beam/bif.c index 1cdce49eef..97c8114437 100644 --- a/erts/emulator/beam/bif.c +++ b/erts/emulator/beam/bif.c @@ -37,10 +37,13 @@ #include "erl_db_util.h" #include "register.h" #include "erl_thr_progress.h" +#define ERTS_PTAB_WANT_BIF_IMPL__ +#include "erl_ptab.h" static Export* flush_monitor_message_trap = NULL; static Export* set_cpu_topology_trap = NULL; static Export* await_proc_exit_trap = NULL; +static Export* await_port_send_result_trap = NULL; Export* erts_format_cpu_topology_trap = NULL; static Export *await_sched_wall_time_mod_trap; @@ -83,8 +86,10 @@ static int insert_internal_link(Process* p, Eterm rpid) ASSERT(is_internal_pid(rpid)); #ifdef ERTS_SMP - if (IS_TRACED(p) && (p->trace_flags & (F_TRACE_SOL|F_TRACE_SOL1))) + if (IS_TRACED(p) + && (ERTS_TRACE_FLAGS(p) & (F_TRACE_SOL|F_TRACE_SOL1))) { rp_locks = ERTS_PROC_LOCKS_ALL; + } erts_smp_proc_lock(p, ERTS_PROC_LOCK_LINK); #endif @@ -100,27 +105,27 @@ static int insert_internal_link(Process* p, Eterm rpid) } if (p != rp) { - erts_add_link(&(p->nlinks), LINK_PID, rp->id); - erts_add_link(&(rp->nlinks), LINK_PID, p->id); + erts_add_link(&ERTS_P_LINKS(p), LINK_PID, rp->common.id); + erts_add_link(&ERTS_P_LINKS(rp), LINK_PID, p->common.id); - ASSERT(is_nil(p->tracer_proc) - || is_internal_pid(p->tracer_proc) - || is_internal_port(p->tracer_proc)); + ASSERT(is_nil(ERTS_TRACER_PROC(p)) + || is_internal_pid(ERTS_TRACER_PROC(p)) + || is_internal_port(ERTS_TRACER_PROC(p))); if (IS_TRACED(p)) { - if (p->trace_flags & (F_TRACE_SOL|F_TRACE_SOL1)) { - rp->trace_flags |= (p->trace_flags & TRACEE_FLAGS); - rp->tracer_proc = p->tracer_proc; /* maybe steal */ + if (ERTS_TRACE_FLAGS(p) & (F_TRACE_SOL|F_TRACE_SOL1)) { + ERTS_TRACE_FLAGS(rp) |= (ERTS_TRACE_FLAGS(p) & TRACEE_FLAGS); + ERTS_TRACER_PROC(rp) = ERTS_TRACER_PROC(p); /* maybe steal */ - if (p->trace_flags & F_TRACE_SOL1) { /* maybe override */ - rp->trace_flags &= ~(F_TRACE_SOL1 | F_TRACE_SOL); - p->trace_flags &= ~(F_TRACE_SOL1 | F_TRACE_SOL); + if (ERTS_TRACE_FLAGS(p) & F_TRACE_SOL1) { /* maybe override */ + ERTS_TRACE_FLAGS(rp) &= ~(F_TRACE_SOL1 | F_TRACE_SOL); + ERTS_TRACE_FLAGS(p) &= ~(F_TRACE_SOL1 | F_TRACE_SOL); } } } } if (IS_TRACED_FL(rp, F_TRACE_PROCS)) - trace_proc(p, rp, am_getting_linked, p->id); + trace_proc(p, rp, am_getting_linked, p->common.id); if (p == rp) erts_smp_proc_unlock(p, rp_locks & ~ERTS_PROC_LOCK_MAIN); @@ -144,10 +149,6 @@ BIF_RETTYPE link_1(BIF_ALIST_1) /* check that the pid or port which is our argument is OK */ if (is_internal_pid(BIF_ARG_1)) { - if (internal_pid_index(BIF_ARG_1) >= erts_max_processes) { - BIF_ERROR(BIF_P, BADARG); - } - if (insert_internal_link(BIF_P, BIF_ARG_1)) { BIF_RET(am_true); } @@ -157,19 +158,37 @@ BIF_RETTYPE link_1(BIF_ALIST_1) } if (is_internal_port(BIF_ARG_1)) { - Port *pt = erts_id2port(BIF_ARG_1, BIF_P, ERTS_PROC_LOCK_MAIN); - if (!pt) { + int send_link_signal = 0; + Port *prt = erts_port_lookup(BIF_ARG_1, ERTS_PORT_SFLGS_INVALID_LOOKUP); + if (!prt) { goto res_no_proc; } erts_smp_proc_lock(BIF_P, ERTS_PROC_LOCK_LINK); - if (erts_add_link(&(BIF_P->nlinks), LINK_PID, BIF_ARG_1) >= 0) - erts_add_link(&(pt->nlinks), LINK_PID, BIF_P->id); + if (erts_add_link(&ERTS_P_LINKS(BIF_P), LINK_PID, BIF_ARG_1) >= 0) + send_link_signal = 1; /* else: already linked */ erts_smp_proc_unlock(BIF_P, ERTS_PROC_LOCK_LINK); - erts_smp_port_unlock(pt); + + if (send_link_signal) { + Eterm ref; + Eterm *refp = erts_port_synchronous_ops ? &ref : NULL; + + switch (erts_port_link(BIF_P, prt, BIF_P->common.id, refp)) { + case ERTS_PORT_OP_DROPPED: + case ERTS_PORT_OP_BADARG: + goto res_no_proc; + case ERTS_PORT_OP_SCHEDULED: + if (refp) { + ASSERT(is_internal_ref(ref)); + BIF_TRAP3(await_port_send_result_trap, BIF_P, ref, am_true, am_true); + } + default: + break; + } + } BIF_RET(am_true); } else if (is_external_port(BIF_ARG_1) @@ -182,7 +201,7 @@ BIF_RETTYPE link_1(BIF_ALIST_1) erts_smp_proc_lock(BIF_P, ERTS_PROC_LOCK_LINK); /* We may earn time by checking first that we're not linked already */ - if (erts_lookup_link(BIF_P->nlinks, BIF_ARG_1) != NULL) { + if (erts_lookup_link(ERTS_P_LINKS(BIF_P), BIF_ARG_1) != NULL) { erts_smp_proc_unlock(BIF_P, ERTS_PROC_LOCK_LINK); BIF_RET(am_true); } @@ -209,10 +228,10 @@ BIF_RETTYPE link_1(BIF_ALIST_1) erts_smp_de_links_lock(dep); - erts_add_link(&(BIF_P->nlinks), LINK_PID, BIF_ARG_1); + erts_add_link(&ERTS_P_LINKS(BIF_P), LINK_PID, BIF_ARG_1); lnk = erts_add_or_lookup_link(&(dep->nlinks), LINK_PID, - BIF_P->id); + BIF_P->common.id); ASSERT(lnk != NULL); erts_add_link(&ERTS_LINK_ROOT(lnk), LINK_PID, BIF_ARG_1); @@ -220,7 +239,7 @@ BIF_RETTYPE link_1(BIF_ALIST_1) erts_smp_de_runlock(dep); erts_smp_proc_unlock(BIF_P, ERTS_PROC_LOCK_LINK); - code = erts_dsig_send_link(&dsd, BIF_P->id, BIF_ARG_1); + code = erts_dsig_send_link(&dsd, BIF_P->common.id, BIF_ARG_1); if (code == ERTS_DSIG_SEND_YIELD) ERTS_BIF_YIELD_RETURN(BIF_P, am_true); BIF_RET(am_true); @@ -289,7 +308,7 @@ remote_demonitor(Process *c_p, DistEntry *dep, Eterm ref, Eterm to) if (dmon) erts_destroy_monitor(dmon); } - mon = erts_remove_monitor(&c_p->monitors, ref); + mon = erts_remove_monitor(&ERTS_P_MONITORS(c_p), ref); erts_smp_proc_unlock(c_p, ERTS_PROC_LOCK_LINK); res = ERTS_DEMONITOR_TRUE; @@ -298,7 +317,7 @@ remote_demonitor(Process *c_p, DistEntry *dep, Eterm ref, Eterm to) case ERTS_DSIG_PREP_CONNECTED: erts_smp_de_links_lock(dep); - mon = erts_remove_monitor(&c_p->monitors, ref); + mon = erts_remove_monitor(&ERTS_P_MONITORS(c_p), ref); dmon = erts_remove_monitor(&dep->monitors, ref); erts_smp_de_links_unlock(dep); erts_smp_de_runlock(dep); @@ -325,7 +344,7 @@ remote_demonitor(Process *c_p, DistEntry *dep, Eterm ref, Eterm to) * the atom is stored there. Yield if necessary. */ code = erts_dsig_send_demonitor(&dsd, - c_p->id, + c_p->common.id, (mon->name != NIL ? mon->name : mon->pid), @@ -387,7 +406,7 @@ static int demonitor(Process *c_p, Eterm ref) goto done; /* Cannot be this monitor's ref */ } - mon = erts_lookup_monitor(c_p->monitors, ref); + mon = erts_lookup_monitor(ERTS_P_MONITORS(c_p), ref); if (!mon) { res = ERTS_DEMONITOR_FALSE; goto done; @@ -426,7 +445,7 @@ static int demonitor(Process *c_p, Eterm ref) to, ERTS_PROC_LOCK_LINK, ERTS_P2P_FLG_ALLOW_OTHER_X); - mon = erts_remove_monitor(&c_p->monitors, ref); + mon = erts_remove_monitor(&ERTS_P_MONITORS(c_p), ref); #ifndef ERTS_SMP ASSERT(mon); #else @@ -440,7 +459,7 @@ static int demonitor(Process *c_p, Eterm ref) } if (rp) { ErtsMonitor *rmon; - rmon = erts_remove_monitor(&(rp->monitors), ref); + rmon = erts_remove_monitor(&ERTS_P_MONITORS(rp), ref); if (rp != c_p) erts_smp_proc_unlock(rp, ERTS_PROC_LOCK_LINK); if (rmon != NULL) @@ -582,7 +601,7 @@ local_pid_monitor(Process *p, Eterm target) mon_ref = erts_make_ref(p); ERTS_BIF_PREP_RET(ret, mon_ref); - if (target == p->id) { + if (target == p->common.id) { return ret; } @@ -599,8 +618,8 @@ local_pid_monitor(Process *p, Eterm target) else { ASSERT(rp != p); - erts_add_monitor(&(p->monitors), MON_ORIGIN, mon_ref, target, NIL); - erts_add_monitor(&(rp->monitors), MON_TARGET, mon_ref, p->id, NIL); + erts_add_monitor(&ERTS_P_MONITORS(p), MON_ORIGIN, mon_ref, target, NIL); + erts_add_monitor(&ERTS_P_MONITORS(rp), MON_TARGET, mon_ref, p->common.id, NIL); erts_smp_proc_unlock(rp, ERTS_PROC_LOCK_LINK); } @@ -635,9 +654,9 @@ local_name_monitor(Process *p, Eterm target_name) UnUseTmpHeap(3,p); } else if (rp != p) { - erts_add_monitor(&(p->monitors), MON_ORIGIN, mon_ref, rp->id, + erts_add_monitor(&ERTS_P_MONITORS(p), MON_ORIGIN, mon_ref, rp->common.id, target_name); - erts_add_monitor(&(rp->monitors), MON_TARGET, mon_ref, p->id, + erts_add_monitor(&ERTS_P_MONITORS(rp), MON_TARGET, mon_ref, p->common.id, target_name); erts_smp_proc_unlock(rp, ERTS_PROC_LOCK_LINK); } @@ -689,16 +708,16 @@ remote_monitor(Process *p, Eterm bifarg1, Eterm bifarg2, erts_smp_de_links_lock(dep); - erts_add_monitor(&(p->monitors), MON_ORIGIN, mon_ref, p_trgt, + erts_add_monitor(&ERTS_P_MONITORS(p), MON_ORIGIN, mon_ref, p_trgt, p_name); - erts_add_monitor(&(dep->monitors), MON_TARGET, mon_ref, p->id, + erts_add_monitor(&(dep->monitors), MON_TARGET, mon_ref, p->common.id, d_name); erts_smp_de_links_unlock(dep); erts_smp_de_runlock(dep); erts_smp_proc_unlock(p, ERTS_PROC_LOCK_LINK); - code = erts_dsig_send_monitor(&dsd, p->id, target, mon_ref); + code = erts_dsig_send_monitor(&dsd, p->common.id, target, mon_ref); if (code == ERTS_DSIG_SEND_YIELD) ERTS_BIF_PREP_YIELD_RETURN(ret, p, mon_ref); else @@ -941,36 +960,39 @@ BIF_RETTYPE unlink_1(BIF_ALIST_1) } if (is_internal_port(BIF_ARG_1)) { - Port *pt = erts_id2port_sflgs(BIF_ARG_1, - BIF_P, - ERTS_PROC_LOCK_MAIN, - ERTS_PORT_SFLGS_DEAD); - erts_smp_proc_lock(BIF_P, ERTS_PROC_LOCK_LINK|ERTS_PROC_LOCK_STATUS); #ifdef ERTS_SMP - if (ERTS_PROC_PENDING_EXIT(BIF_P)) { - if (pt) - erts_smp_port_unlock(pt); + if (ERTS_PROC_PENDING_EXIT(BIF_P)) goto handle_pending_exit; - } #endif - l = erts_remove_link(&BIF_P->nlinks, BIF_ARG_1); - - ASSERT(pt || !l); - - if (pt) { - rl = erts_remove_link(&pt->nlinks, BIF_P->id); - erts_smp_port_unlock(pt); - if (rl) - erts_destroy_link(rl); - } + l = erts_remove_link(&ERTS_P_LINKS(BIF_P), BIF_ARG_1); erts_smp_proc_unlock(BIF_P, ERTS_PROC_LOCK_LINK|ERTS_PROC_LOCK_STATUS); - if (l) + if (l) { + Port *prt; + erts_destroy_link(l); + /* Send unlink signal */ + prt = erts_port_lookup(BIF_ARG_1, ERTS_PORT_SFLGS_DEAD); + if (prt) { + ErtsPortOpResult res; + Eterm ref; + Eterm *refp = erts_port_synchronous_ops ? &ref : NULL; +#ifdef DEBUG + ref = NIL; +#endif + res = erts_port_unlink(BIF_P, prt, BIF_P->common.id, refp); + + if (refp && res == ERTS_PORT_OP_SCHEDULED) { + ASSERT(is_internal_ref(ref)); + BIF_TRAP3(await_port_send_result_trap, BIF_P, ref, am_true, am_true); + } + } + } + BIF_RET(am_true); } else if (is_external_port(BIF_ARG_1) @@ -993,7 +1015,7 @@ BIF_RETTYPE unlink_1(BIF_ALIST_1) if (ERTS_PROC_PENDING_EXIT(BIF_P)) goto handle_pending_exit; #endif - l = erts_remove_link(&BIF_P->nlinks,BIF_ARG_1); + l = erts_remove_link(&ERTS_P_LINKS(BIF_P), BIF_ARG_1); erts_smp_proc_unlock(BIF_P, ERTS_PROC_LOCK_LINK|ERTS_PROC_LOCK_STATUS); @@ -1022,8 +1044,8 @@ BIF_RETTYPE unlink_1(BIF_ALIST_1) #endif case ERTS_DSIG_PREP_CONNECTED: - erts_remove_dist_link(&dld, BIF_P->id, BIF_ARG_1, dep); - code = erts_dsig_send_unlink(&dsd, BIF_P->id, BIF_ARG_1); + erts_remove_dist_link(&dld, BIF_P->common.id, BIF_ARG_1, dep); + code = erts_dsig_send_unlink(&dsd, BIF_P->common.id, BIF_ARG_1); erts_destroy_dist_link(&dld); if (code == ERTS_DSIG_SEND_YIELD) ERTS_BIF_YIELD_RETURN(BIF_P, am_true); @@ -1037,10 +1059,6 @@ BIF_RETTYPE unlink_1(BIF_ALIST_1) /* Internal pid... */ - /* process ok ? */ - if (internal_pid_index(BIF_ARG_1) >= erts_max_processes) - BIF_ERROR(BIF_P, BADARG); - erts_smp_proc_lock(BIF_P, ERTS_PROC_LOCK_LINK|ERTS_PROC_LOCK_STATUS); /* get process struct */ @@ -1059,7 +1077,7 @@ BIF_RETTYPE unlink_1(BIF_ALIST_1) #endif /* unlink and ignore errors */ - l = erts_remove_link(&BIF_P->nlinks,BIF_ARG_1); + l = erts_remove_link(&ERTS_P_LINKS(BIF_P), BIF_ARG_1); if (l != NULL) erts_destroy_link(l); @@ -1067,12 +1085,12 @@ BIF_RETTYPE unlink_1(BIF_ALIST_1) ERTS_SMP_ASSERT_IS_NOT_EXITING(BIF_P); } else { - rl = erts_remove_link(&(rp->nlinks),BIF_P->id); + rl = erts_remove_link(&ERTS_P_LINKS(rp), BIF_P->common.id); if (rl != NULL) erts_destroy_link(rl); if (IS_TRACED_FL(rp, F_TRACE_PROCS) && rl != NULL) { - trace_proc(BIF_P, rp, am_getting_unlinked, BIF_P->id); + trace_proc(BIF_P, rp, am_getting_unlinked, BIF_P->common.id); } if (rp != BIF_P) @@ -1345,15 +1363,28 @@ BIF_RETTYPE exit_2(BIF_ALIST_2) */ if (is_internal_port(BIF_ARG_1)) { - Port *prt; - erts_smp_proc_unlock(BIF_P, ERTS_PROC_LOCK_MAIN); - prt = erts_id2port(BIF_ARG_1, NULL, 0); + Port *prt = erts_port_lookup(BIF_ARG_1, ERTS_PORT_SFLGS_INVALID_LOOKUP); + if (prt) { - erts_do_exit_port(prt, BIF_P->id, BIF_ARG_2); - erts_port_release(prt); + Eterm ref; + Eterm *refp = erts_port_synchronous_ops ? &ref : NULL; + ErtsPortOpResult res; + +#ifdef DEBUG + ref = NIL; +#endif + + res = erts_port_exit(BIF_P, 0, prt, BIF_P->common.id, BIF_ARG_2, refp); + + ERTS_BIF_CHK_EXITED(BIF_P); + + if (refp && res == ERTS_PORT_OP_SCHEDULED) { + ASSERT(is_internal_ref(ref)); + BIF_TRAP3(await_port_send_result_trap, BIF_P, ref, am_true, am_true); + } + } - erts_smp_proc_lock(BIF_P, ERTS_PROC_LOCK_MAIN); - ERTS_BIF_CHK_EXITED(BIF_P); + BIF_RET(am_true); } else if(is_external_port(BIF_ARG_1) @@ -1379,7 +1410,7 @@ BIF_RETTYPE exit_2(BIF_ALIST_2) case ERTS_DSIG_PREP_NOT_CONNECTED: BIF_TRAP2(dexit_trap, BIF_P, BIF_ARG_1, BIF_ARG_2); case ERTS_DSIG_PREP_CONNECTED: - code = erts_dsig_send_exit2(&dsd, BIF_P->id, BIF_ARG_1, BIF_ARG_2); + code = erts_dsig_send_exit2(&dsd, BIF_P->common.id, BIF_ARG_1, BIF_ARG_2); if (code == ERTS_DSIG_SEND_YIELD) ERTS_BIF_YIELD_RETURN(BIF_P, am_true); BIF_RET(am_true); @@ -1397,9 +1428,7 @@ BIF_RETTYPE exit_2(BIF_ALIST_2) */ ErtsProcLocks rp_locks; - if (internal_pid_index(BIF_ARG_1) >= erts_max_processes) - BIF_ERROR(BIF_P, BADARG); - if (BIF_ARG_1 == BIF_P->id) { + if (BIF_ARG_1 == BIF_P->common.id) { rp_locks = ERTS_PROC_LOCKS_ALL; rp = BIF_P; erts_smp_proc_lock(rp, ERTS_PROC_LOCKS_ALL_MINOR); @@ -1417,7 +1446,7 @@ BIF_RETTYPE exit_2(BIF_ALIST_2) * Send an exit signal. */ erts_send_exit_signal(BIF_P, - BIF_P->id, + BIF_P->common.id, rp, &rp_locks, BIF_ARG_2, @@ -1519,22 +1548,24 @@ BIF_RETTYPE process_flag_2(BIF_ALIST_2) } /* * NOTE: It is important that we check for pending exit signals - * and handle them before flag trap_exit is set to true. - * For more info, see implementation of erts_send_exit_signal(). + * and handle them before returning if trap_exit is set to + * true. For more info, see implementation of + * erts_send_exit_signal(). */ - erts_smp_proc_lock(BIF_P, ERTS_PROC_LOCK_STATUS); - ERTS_SMP_LC_ASSERT((ERTS_PROC_LOCK_MAIN|ERTS_PROC_LOCK_STATUS) - & erts_proc_lc_my_proc_locks(BIF_P)); - ERTS_SMP_BIF_CHK_PENDING_EXIT(BIF_P, - ERTS_PROC_LOCK_MAIN|ERTS_PROC_LOCK_STATUS); if (trap_exit) - state = erts_smp_atomic32_read_bor_nob(&BIF_P->state, - ERTS_PSFLG_TRAP_EXIT); + state = erts_smp_atomic32_read_bor_mb(&BIF_P->state, + ERTS_PSFLG_TRAP_EXIT); else - state = erts_smp_atomic32_read_band_nob(&BIF_P->state, - ~ERTS_PSFLG_TRAP_EXIT); + state = erts_smp_atomic32_read_band_mb(&BIF_P->state, + ~ERTS_PSFLG_TRAP_EXIT); +#ifdef ERTS_SMP + if (ERTS_PROC_PENDING_EXIT(BIF_P)) { + erts_handle_pending_exit(BIF_P, ERTS_PROC_LOCK_MAIN); + ERTS_BIF_EXITED(BIF_P); + } +#endif + old_value = (state & ERTS_PSFLG_TRAP_EXIT) ? am_true : am_false; - erts_smp_proc_unlock(BIF_P, ERTS_PROC_LOCK_STATUS); BIF_RET(old_value); } else if (BIF_ARG_1 == am_scheduler) { @@ -1617,11 +1648,13 @@ BIF_RETTYPE process_flag_2(BIF_ALIST_2) goto error; } erts_smp_proc_lock(BIF_P, ERTS_PROC_LOCKS_ALL_MINOR); - old_value = BIF_P->trace_flags & F_SENSITIVE ? am_true : am_false; + old_value = (ERTS_TRACE_FLAGS(BIF_P) & F_SENSITIVE + ? am_true + : am_false); if (is_sensitive) { - BIF_P->trace_flags |= F_SENSITIVE; + ERTS_TRACE_FLAGS(BIF_P) |= F_SENSITIVE; } else { - BIF_P->trace_flags &= ~F_SENSITIVE; + ERTS_TRACE_FLAGS(BIF_P) &= ~F_SENSITIVE; } erts_smp_proc_unlock(BIF_P, ERTS_PROC_LOCKS_ALL_MINOR); BIF_RET(old_value); @@ -1747,8 +1780,9 @@ ebif_bang_2(BIF_ALIST_2) #define SEND_BADARG (-4) #define SEND_USER_ERROR (-5) #define SEND_INTERNAL_ERROR (-6) +#define SEND_AWAIT_RESULT (-7) -Sint do_send(Process *p, Eterm to, Eterm msg, int suspend); +Sint do_send(Process *p, Eterm to, Eterm msg, int suspend, Eterm *refp); static Sint remote_send(Process *p, DistEntry *dep, Eterm to, Eterm full_to, Eterm msg, int suspend) @@ -1802,7 +1836,7 @@ static Sint remote_send(Process *p, DistEntry *dep, } Sint -do_send(Process *p, Eterm to, Eterm msg, int suspend) { +do_send(Process *p, Eterm to, Eterm msg, int suspend, Eterm *refp) { Eterm portid; Port *pt; Process* rp; @@ -1814,16 +1848,10 @@ do_send(Process *p, Eterm to, Eterm msg, int suspend) { trace_send(p, to, msg); if (ERTS_PROC_GET_SAVED_CALLS_BUF(p)) save_calls(p, &exp_send); - - if (internal_pid_index(to) >= erts_max_processes) - return SEND_BADARG; - rp = erts_proc_lookup_raw(to); - - if (!rp) { - ERTS_SMP_ASSERT_IS_NOT_EXITING(p); + rp = erts_proc_lookup_raw(to); + if (!rp) return 0; - } } else if (is_external_pid(to)) { dep = external_pid_dist_entry(to); if(dep == erts_this_dist_entry) { @@ -1832,7 +1860,7 @@ do_send(Process *p, Eterm to, Eterm msg, int suspend) { "Discarding message %T from %T to %T in an old " "incarnation (%d) of this node (%d)\n", msg, - p->id, + p->common.id, to, external_pid_creation(to), erts_this_node->creation); @@ -1841,45 +1869,24 @@ do_send(Process *p, Eterm to, Eterm msg, int suspend) { } return remote_send(p, dep, to, to, msg, suspend); } else if (is_atom(to)) { - - /* Need to virtual schedule out sending process - * because of lock wait. This is only necessary - * for internal port calling but the lock is bundled - * with name lookup. - */ - - if (IS_TRACED_FL(p, F_TRACE_SCHED_PROCS)) { - trace_virtual_sched(p, am_out); - } - if (erts_system_profile_flags.runnable_procs && erts_system_profile_flags.exclusive) { - profile_runnable_proc(p, am_inactive); - } - erts_whereis_name(p, ERTS_PROC_LOCK_MAIN, - to, - &rp, 0, 0, - &pt); + Eterm id = erts_whereis_name_to_id(p, to); + rp = erts_proc_lookup(id); + if (rp) + goto send_message; + + pt = erts_port_lookup(id, ERTS_PORT_SFLGS_INVALID_LOOKUP); if (pt) { - portid = pt->id; + portid = id; goto port_common; } - - /* Not a port virtually schedule the process back in */ - if (IS_TRACED_FL(p, F_TRACE_SCHED_PROCS)) { - trace_virtual_sched(p, am_in); - } - if (erts_system_profile_flags.runnable_procs && erts_system_profile_flags.exclusive) { - profile_runnable_proc(p, am_active); - } if (IS_TRACED(p)) trace_send(p, to, msg); if (ERTS_PROC_GET_SAVED_CALLS_BUF(p)) save_calls(p, &exp_send); - if (!rp) { - return SEND_BADARG; - } + return SEND_BADARG; } else if (is_external_port(to) && (external_port_dist_entry(to) == erts_this_dist_entry)) { @@ -1888,50 +1895,56 @@ do_send(Process *p, Eterm to, Eterm msg, int suspend) { "Discarding message %T from %T to %T in an old " "incarnation (%d) of this node (%d)\n", msg, - p->id, + p->common.id, to, external_port_creation(to), erts_this_node->creation); erts_send_error_to_logger(p->group_leader, dsbufp); return 0; } else if (is_internal_port(to)) { + int ret_val; portid = to; - /* schedule out calling process, waiting for lock*/ - if (IS_TRACED_FL(p, F_TRACE_SCHED_PROCS)) { - trace_virtual_sched(p, am_out); - } - if (erts_system_profile_flags.runnable_procs && erts_system_profile_flags.exclusive) { - profile_runnable_proc(p, am_inactive); - } - pt = erts_id2port(to, p, ERTS_PROC_LOCK_MAIN); + + pt = erts_port_lookup(portid, ERTS_PORT_SFLGS_INVALID_LOOKUP); + port_common: - ERTS_SMP_LC_ASSERT(!pt || erts_lc_is_port_locked(pt)); + ret_val = 0; - /* We have waited for locks, trace schedule ports */ - if (pt && IS_TRACED_FL(pt, F_TRACE_SCHED_PORTS)) { - trace_sched_ports_where(pt, am_in, am_command); - } - if (pt && erts_system_profile_flags.runnable_ports && !erts_port_is_scheduled(pt)) { - profile_runnable_port(pt, am_active); - } - - /* XXX let port_command handle the busy stuff !!! */ - if (pt && (pt->status & ERTS_PORT_SFLG_PORT_BUSY)) { - if (suspend) { - erts_suspend(p, ERTS_PROC_LOCK_MAIN, pt); - if (erts_system_monitor_flags.busy_port) { - monitor_generic(p, am_busy_port, portid); + if (pt) { + int ps_flags = suspend ? 0 : ERTS_PORT_SIG_FLG_NOSUSPEND; + *refp = NIL; + + switch (erts_port_command(p, ps_flags, pt, msg, refp)) { + case ERTS_PORT_OP_CALLER_EXIT: + /* We are exiting... */ + return SEND_USER_ERROR; + case ERTS_PORT_OP_BUSY: + /* Nothing has been sent */ + if (suspend) + erts_suspend(p, ERTS_PROC_LOCK_MAIN, pt); + return SEND_YIELD; + case ERTS_PORT_OP_BUSY_SCHEDULED: + /* Message was sent */ + if (suspend) { + erts_suspend(p, ERTS_PROC_LOCK_MAIN, pt); + ret_val = SEND_YIELD_RETURN; + break; } + /* Fall through */ + case ERTS_PORT_OP_SCHEDULED: + if (is_not_nil(*refp)) { + ASSERT(is_internal_ref(*refp)); + ret_val = SEND_AWAIT_RESULT; + } + break; + case ERTS_PORT_OP_DROPPED: + case ERTS_PORT_OP_BADARG: + case ERTS_PORT_OP_DONE: + break; + default: + ERTS_INTERNAL_ERROR("Unexpected erts_port_command() result"); + break; } - /* Virtually schedule out the port before releasing */ - if (IS_TRACED_FL(pt, F_TRACE_SCHED_PORTS)) { - trace_sched_ports_where(pt, am_out, am_command); - } - if (erts_system_profile_flags.runnable_ports && !erts_port_is_scheduled(pt)) { - profile_runnable_port(pt, am_inactive); - } - erts_port_release(pt); - return SEND_YIELD; } if (IS_TRACED(p)) /* trace once only !! */ @@ -1949,30 +1962,11 @@ do_send(Process *p, Eterm to, Eterm msg, int suspend) { SEQ_TRACE_SEND, portid, p); } - /* XXX NO GC in port command */ - erts_port_command(p, p->id, pt, msg); - if (pt) { - /* Virtually schedule out the port before releasing */ - if (IS_TRACED_FL(pt, F_TRACE_SCHED_PORTS)) { - trace_sched_ports_where(pt, am_out, am_command); - } - if (erts_system_profile_flags.runnable_ports && !erts_port_is_scheduled(pt)) { - profile_runnable_port(pt, am_inactive); - } - erts_port_release(pt); - } - /* Virtually schedule in process */ - if (IS_TRACED_FL(p, F_TRACE_SCHED_PROCS)) { - trace_virtual_sched(p, am_in); - } - if (erts_system_profile_flags.runnable_procs && erts_system_profile_flags.exclusive) { - profile_runnable_proc(p, am_active); - } if (ERTS_PROC_IS_EXITING(p)) { KILL_CATCHES(p); /* Must exit */ return SEND_USER_ERROR; } - return 0; + return ret_val; } else if (is_tuple(to)) { /* Remote send */ int ret; tp = tuple_val(to); @@ -1988,47 +1982,24 @@ do_send(Process *p, Eterm to, Eterm msg, int suspend) { dep = erts_sysname_to_connected_dist_entry(tp[2]); if (dep == erts_this_dist_entry) { + Eterm id; erts_deref_dist_entry(dep); if (IS_TRACED(p)) trace_send(p, to, msg); if (ERTS_PROC_GET_SAVED_CALLS_BUF(p)) save_calls(p, &exp_send); - - /* Need to virtual schedule out sending process - * because of lock wait. This is only necessary - * for internal port calling but the lock is bundled. - */ - - if (IS_TRACED_FL(p, F_TRACE_SCHED_PROCS)) { - trace_virtual_sched(p, am_out); - } - if (erts_system_profile_flags.runnable_procs && erts_system_profile_flags.exclusive) { - profile_runnable_proc(p, am_inactive); - } - erts_whereis_name(p, ERTS_PROC_LOCK_MAIN, - tp[1], - &rp, 0, 0, - &pt); + id = erts_whereis_name_to_id(p, tp[1]); + + rp = erts_proc_lookup_raw(id); + if (rp) + goto send_message; + pt = erts_port_lookup(id, ERTS_PORT_SFLGS_INVALID_LOOKUP); if (pt) { - portid = pt->id; + portid = id; goto port_common; } - /* Port lookup failed, virtually schedule the process - * back in. - */ - - if (IS_TRACED_FL(p, F_TRACE_SCHED_PROCS)) { - trace_virtual_sched(p, am_in); - } - if (erts_system_profile_flags.runnable_procs && erts_system_profile_flags.exclusive) { - profile_runnable_proc(p, am_active); - } - - if (!rp) { - return 0; - } - goto send_message; + return 0; } ret = remote_send(p, dep, tp[1], to, msg, suspend); @@ -2067,6 +2038,7 @@ do_send(Process *p, Eterm to, Eterm msg, int suspend) { BIF_RETTYPE send_3(BIF_ALIST_3) { + Eterm ref; Process *p = BIF_P; Eterm to = BIF_ARG_1; Eterm msg = BIF_ARG_2; @@ -2090,12 +2062,18 @@ BIF_RETTYPE send_3(BIF_ALIST_3) if(!is_nil(l)) { BIF_ERROR(p, BADARG); } - - result = do_send(p, to, msg, suspend); + +#ifdef DEBUG + ref = NIL; +#endif + + result = do_send(p, to, msg, suspend, &ref); if (result > 0) { ERTS_VBUMP_REDS(p, result); BIF_RET(am_ok); - } else switch (result) { + } + + switch (result) { case 0: BIF_RET(am_ok); break; @@ -2118,6 +2096,9 @@ BIF_RETTYPE send_3(BIF_ALIST_3) ERTS_BIF_YIELD_RETURN(p, am_ok); else BIF_RET(am_nosuspend); + case SEND_AWAIT_RESULT: + ASSERT(is_internal_ref(ref)); + BIF_TRAP3(await_port_send_result_trap, p, ref, am_nosuspend, am_ok); case SEND_BADARG: BIF_ERROR(p, BADARG); break; @@ -2142,12 +2123,21 @@ BIF_RETTYPE send_2(BIF_ALIST_2) Eterm erl_send(Process *p, Eterm to, Eterm msg) { - Sint result = do_send(p, to, msg, !0); + Eterm ref; + Sint result; + +#ifdef DEBUG + ref = NIL; +#endif + + result = do_send(p, to, msg, !0, &ref); if (result > 0) { ERTS_VBUMP_REDS(p, result); BIF_RET(msg); - } else switch (result) { + } + + switch (result) { case 0: BIF_RET(msg); break; @@ -2159,6 +2149,9 @@ Eterm erl_send(Process *p, Eterm to, Eterm msg) break; case SEND_YIELD_RETURN: ERTS_BIF_YIELD_RETURN(p, msg); + case SEND_AWAIT_RESULT: + ASSERT(is_internal_ref(ref)); + BIF_TRAP3(await_port_send_result_trap, p, ref, msg, msg); case SEND_BADARG: BIF_ERROR(p, BADARG); break; @@ -2428,9 +2421,7 @@ BIF_RETTYPE setelement_3(BIF_ALIST_3) /* copy the tuple */ resp = hp; - while (size--) { /* XXX use memcpy? */ - *hp++ = *ptr++; - } + sys_memcpy(hp, ptr, sizeof(Eterm)*size); resp[ix] = BIF_ARG_3; BIF_RET(make_tuple(resp)); } @@ -2443,7 +2434,7 @@ BIF_RETTYPE make_tuple_2(BIF_ALIST_2) Eterm* hp; Eterm res; - if (is_not_small(BIF_ARG_1) || (n = signed_val(BIF_ARG_1)) < 0) { + if (is_not_small(BIF_ARG_1) || (n = signed_val(BIF_ARG_1)) < 0 || n > ERTS_MAX_TUPLE_SIZE) { BIF_ERROR(BIF_P, BADARG); } hp = HAlloc(BIF_P, n+1); @@ -2464,7 +2455,7 @@ BIF_RETTYPE make_tuple_3(BIF_ALIST_3) Eterm list = BIF_ARG_3; Eterm* tup; - if (is_not_small(BIF_ARG_1) || (n = signed_val(BIF_ARG_1)) < 0) { + if (is_not_small(BIF_ARG_1) || (n = signed_val(BIF_ARG_1)) < 0 || n > ERTS_MAX_TUPLE_SIZE) { error: BIF_ERROR(BIF_P, BADARG); } @@ -2516,11 +2507,16 @@ BIF_RETTYPE append_element_2(BIF_ALIST_2) Eterm res; if (is_not_tuple(BIF_ARG_1)) { + error: BIF_ERROR(BIF_P, BADARG); } - ptr = tuple_val(BIF_ARG_1); + ptr = tuple_val(BIF_ARG_1); arity = arityval(*ptr); - hp = HAlloc(BIF_P, arity + 2); + + if (arity + 1 > ERTS_MAX_TUPLE_SIZE) + goto error; + + hp = HAlloc(BIF_P, arity + 2); res = make_tuple(hp); *hp = make_arityval(arity+1); while (arity--) { @@ -2530,6 +2526,78 @@ BIF_RETTYPE append_element_2(BIF_ALIST_2) BIF_RET(res); } +BIF_RETTYPE insert_element_3(BIF_ALIST_3) +{ + Eterm* ptr; + Eterm* hp; + Uint arity; + Eterm res; + Sint ix; + + if (is_not_tuple(BIF_ARG_2) || is_not_small(BIF_ARG_1)) { + BIF_ERROR(BIF_P, BADARG); + } + + ptr = tuple_val(BIF_ARG_2); + arity = arityval(*ptr); + ix = signed_val(BIF_ARG_1); + + if ((ix < 1) || (ix > (arity + 1))) { + BIF_ERROR(BIF_P, BADARG); + } + + hp = HAlloc(BIF_P, arity + 1 + 1); + res = make_tuple(hp); + *hp = make_arityval(arity + 1); + + ix--; + arity -= ix; + + while (ix--) { *++hp = *++ptr; } + + *++hp = BIF_ARG_3; + + while(arity--) { *++hp = *++ptr; } + + BIF_RET(res); +} + +BIF_RETTYPE delete_element_2(BIF_ALIST_3) +{ + Eterm* ptr; + Eterm* hp; + Uint arity; + Eterm res; + Sint ix; + + if (is_not_tuple(BIF_ARG_2) || is_not_small(BIF_ARG_1)) { + BIF_ERROR(BIF_P, BADARG); + } + + ptr = tuple_val(BIF_ARG_2); + arity = arityval(*ptr); + ix = signed_val(BIF_ARG_1); + + if ((ix < 1) || (ix > arity) || (arity == 0)) { + BIF_ERROR(BIF_P, BADARG); + } + + hp = HAlloc(BIF_P, arity + 1 - 1); + res = make_tuple(hp); + *hp = make_arityval(arity - 1); + + ix--; + arity -= ix; + + while (ix--) { *++hp = *++ptr; } + + ++ptr; + + while(arity--) { *++hp = *++ptr; } + + BIF_RET(res); +} + /**********************************************************************/ /* convert an atom to a list of ascii integer */ @@ -3104,7 +3172,7 @@ BIF_RETTYPE list_to_tuple_1(BIF_ALIST_1) Eterm* hp; int len; - if ((len = list_length(list)) < 0) { + if ((len = list_length(list)) < 0 || len > ERTS_MAX_TUPLE_SIZE) { BIF_ERROR(BIF_P, BADARG); } @@ -3126,7 +3194,7 @@ BIF_RETTYPE list_to_tuple_1(BIF_ALIST_1) BIF_RETTYPE self_0(BIF_ALIST_0) { - BIF_RET(BIF_P->id); + BIF_RET(BIF_P->common.id); } /**********************************************************************/ @@ -3163,11 +3231,9 @@ static erts_smp_spinlock_t make_ref_lock; static erts_smp_mtx_t ports_snapshot_mtx; erts_smp_atomic_t erts_dead_ports_ptr; /* To store dying ports during snapshot */ -Eterm erts_make_ref_in_buffer(Eterm buffer[REF_THING_SIZE]) +void +erts_make_ref_in_array(Uint32 ref[ERTS_MAX_REF_NUMBERS]) { - Eterm* hp = buffer; - Uint32 ref0, ref1, ref2; - erts_smp_spin_lock(&make_ref_lock); reference0++; @@ -3179,24 +3245,36 @@ Eterm erts_make_ref_in_buffer(Eterm buffer[REF_THING_SIZE]) } } - ref0 = reference0; - ref1 = reference1; - ref2 = reference2; + ref[0] = reference0; + ref[1] = reference1; + ref[2] = reference2; erts_smp_spin_unlock(&make_ref_lock); +} - write_ref_thing(hp, ref0, ref1, ref2); +Eterm erts_make_ref_in_buffer(Eterm buffer[REF_THING_SIZE]) +{ + Eterm* hp = buffer; + Uint32 ref[ERTS_MAX_REF_NUMBERS]; + + erts_make_ref_in_array(ref); + write_ref_thing(hp, ref[0], ref[1], ref[2]); return make_internal_ref(hp); } Eterm erts_make_ref(Process *p) { Eterm* hp; + Uint32 ref[ERTS_MAX_REF_NUMBERS]; ERTS_SMP_LC_ASSERT(ERTS_PROC_LOCK_MAIN & erts_proc_lc_my_proc_locks(p)); hp = HAlloc(p, REF_THING_SIZE); - return erts_make_ref_in_buffer(hp); + + erts_make_ref_in_array(ref); + write_ref_thing(hp, ref[0], ref[1], ref[2]); + + return make_internal_ref(hp); } BIF_RETTYPE make_ref_0(BIF_ALIST_0) @@ -3460,7 +3538,7 @@ BIF_RETTYPE garbage_collect_1(BIF_ALIST_1) BIF_ERROR(BIF_P, BADARG); } - if (BIF_P->id == BIF_ARG_1) + if (BIF_P->common.id == BIF_ARG_1) rp = BIF_P; else { #ifdef ERTS_SMP @@ -3500,71 +3578,23 @@ BIF_RETTYPE garbage_collect_0(BIF_ALIST_0) } /**********************************************************************/ -/* Return a list of active ports */ +/* + * The erlang:processes/0 BIF. + */ -BIF_RETTYPE ports_0(BIF_ALIST_0) +BIF_RETTYPE processes_0(BIF_ALIST_0) { - Eterm res = NIL; - Eterm* port_buf = erts_alloc(ERTS_ALC_T_TMP, - sizeof(Eterm)*erts_max_ports); - Eterm* pp = port_buf; - Eterm* dead_ports; - int alive, dead; - Uint32 next_ss; - int i; - - /* To get a consistent snapshot... - * We add alive ports from start of the buffer - * while dying ports are added from the other end by the killing threads. - */ - - erts_smp_mtx_lock(&ports_snapshot_mtx); /* One snapshot at a time */ - - erts_smp_atomic_set_nob(&erts_dead_ports_ptr, - (erts_aint_t) (port_buf + erts_max_ports)); - - next_ss = erts_smp_atomic32_inc_read_relb(&erts_ports_snapshot); - - for (i = erts_max_ports-1; i >= 0; i--) { - Port* prt = &erts_port[i]; - erts_smp_port_state_lock(prt); - if (!(prt->status & ERTS_PORT_SFLGS_DEAD) - && prt->snapshot != next_ss) { - ASSERT(prt->snapshot == next_ss - 1); - *pp++ = prt->id; - prt->snapshot = next_ss; /* Consumed by this snapshot */ - } - erts_smp_port_state_unlock(prt); - } - - dead_ports = (Eterm*)erts_smp_atomic_xchg_nob(&erts_dead_ports_ptr, - (erts_aint_t) NULL); - erts_smp_mtx_unlock(&ports_snapshot_mtx); - - ASSERT(pp <= dead_ports); - - alive = pp - port_buf; - dead = port_buf + erts_max_ports - dead_ports; - - ASSERT((alive+dead) <= erts_max_ports); - - if (alive+dead > 0) { - erts_aint_t i; - Eterm *hp = HAlloc(BIF_P, (alive+dead)*2); - - for (i = 0; i < alive; i++) { - res = CONS(hp, port_buf[i], res); - hp += 2; - } - for (i = 0; i < dead; i++) { - res = CONS(hp, dead_ports[i], res); - hp += 2; - } - } + return erts_ptab_list(BIF_P, &erts_proc); +} - erts_free(ERTS_ALC_T_TMP, port_buf); +/**********************************************************************/ +/* + * The erlang:ports/0 BIF. + */ - BIF_RET(res); +BIF_RETTYPE ports_0(BIF_ALIST_0) +{ + return erts_ptab_list(BIF_P, &erts_port); } /**********************************************************************/ @@ -4194,12 +4224,13 @@ BIF_RETTYPE system_flag_2(BIF_ALIST_2) BIF_RET(old_value); } } else if (BIF_ARG_1 == make_small(1)) { - Uint i; + int i, max; ErlMessage* mp; erts_smp_proc_unlock(BIF_P, ERTS_PROC_LOCK_MAIN); erts_smp_thr_progress_block(); - for (i = 0; i < erts_max_processes; i++) { + max = erts_ptab_max(&erts_proc); + for (i = 0; i < max; i++) { Process *p = erts_pix2proc(i); if (p) { #ifdef USE_VM_PROBES @@ -4546,6 +4577,8 @@ void erts_init_bif(void) am_format_cpu_topology, 1); await_proc_exit_trap = erts_export_put(am_erlang,am_await_proc_exit,3); + await_port_send_result_trap + = erts_export_put(am_erts_internal, am_await_port_send_result, 3); await_sched_wall_time_mod_trap = erts_export_put(am_erlang, am_await_sched_wall_time_modifications, 2); erts_smp_atomic32_init_nob(&sched_wall_time, 0); @@ -4561,19 +4594,18 @@ bif erlang:send_to_logger/2 BIF_RETTYPE send_to_logger_2(BIF_ALIST_2) { byte *buf; - int len; + ErlDrvSizeT len; if (!is_atom(BIF_ARG_1) || !(is_list(BIF_ARG_2) || is_nil(BIF_ARG_1))) { BIF_ERROR(BIF_P,BADARG); } - len = io_list_len(BIF_ARG_2); - if (len < 0) + if (erts_iolist_size(BIF_ARG_2, &len) != 0) BIF_ERROR(BIF_P,BADARG); else if (len == 0) buf = ""; else { #ifdef DEBUG - int len2; + ErlDrvSizeT len2; #endif buf = (byte *) erts_alloc(ERTS_ALC_T_TMP, len+1); #ifdef DEBUG @@ -4581,7 +4613,7 @@ BIF_RETTYPE send_to_logger_2(BIF_ALIST_2) #else (void) #endif - io_list_to_buf(BIF_ARG_2, buf, len); + erts_iolist_to_buf(BIF_ARG_2, buf, len); ASSERT(len2 == len); buf[len] = '\0'; switch (BIF_ARG_1) { @@ -4675,7 +4707,6 @@ BIF_RETTYPE dt_prepend_vm_tag_data_1(BIF_ALIST_1) #ifdef USE_VM_PROBES Eterm b; Eterm *hp; - hp = HAlloc(BIF_P,2); if (is_binary((DT_UTAG(BIF_P)))) { Uint sz = binary_size(DT_UTAG(BIF_P)); int i; @@ -4692,6 +4723,7 @@ BIF_RETTYPE dt_prepend_vm_tag_data_1(BIF_ALIST_1) } else { b = new_binary(BIF_P,(byte *)"\0",1); } + hp = HAlloc(BIF_P,2); BIF_RET(CONS(hp,b,BIF_ARG_1)); #else BIF_RET(BIF_ARG_1); @@ -4702,7 +4734,6 @@ BIF_RETTYPE dt_append_vm_tag_data_1(BIF_ALIST_1) #ifdef USE_VM_PROBES Eterm b; Eterm *hp; - hp = HAlloc(BIF_P,2); if (is_binary((DT_UTAG(BIF_P)))) { Uint sz = binary_size(DT_UTAG(BIF_P)); int i; @@ -4719,6 +4750,7 @@ BIF_RETTYPE dt_append_vm_tag_data_1(BIF_ALIST_1) } else { b = new_binary(BIF_P,(byte *)"\0",1); } + hp = HAlloc(BIF_P,2); BIF_RET(CONS(hp,BIF_ARG_1,b)); #else BIF_RET(BIF_ARG_1); @@ -4742,14 +4774,14 @@ BIF_RETTYPE dt_spread_tag_1(BIF_ALIST_1) #ifdef DTRACE_TAG_HARDDEBUG erts_fprintf(stderr, "Dtrace -> (%T) start spreading tag %T\r\n", - BIF_P->id,DT_UTAG(BIF_P)); + BIF_P->common.id,DT_UTAG(BIF_P)); #endif } else { DT_UTAG_FLAGS(BIF_P) &= ~DT_UTAG_SPREADING; #ifdef DTRACE_TAG_HARDDEBUG erts_fprintf(stderr, "Dtrace -> (%T) stop spreading tag %T\r\n", - BIF_P->id,DT_UTAG(BIF_P)); + BIF_P->common.id,DT_UTAG(BIF_P)); #endif } } @@ -4775,7 +4807,7 @@ BIF_RETTYPE dt_restore_tag_1(BIF_ALIST_1) #ifdef DTRACE_TAG_HARDDEBUG erts_fprintf(stderr, "Dtrace -> (%T) restore Killing tag!\r\n", - BIF_P->id); + BIF_P->common.id); #endif } DT_UTAG(BIF_P) = NIL; @@ -4792,12 +4824,12 @@ BIF_RETTYPE dt_restore_tag_1(BIF_ALIST_1) erts_fprintf(stderr, "Dtrace -> (%T) restore stop spreading " "tag %T\r\n", - BIF_P->id, tpl[2]); + BIF_P->common.id, tpl[2]); } else if ((x & DT_UTAG_SPREADING) && !(DT_UTAG_FLAGS(BIF_P) & DT_UTAG_SPREADING)) { erts_fprintf(stderr, "Dtrace -> (%T) restore start spreading " - "tag %T\r\n",BIF_P->id,tpl[2]); + "tag %T\r\n",BIF_P->common.id,tpl[2]); } #endif DT_UTAG_FLAGS(BIF_P) = x; diff --git a/erts/emulator/beam/bif.h b/erts/emulator/beam/bif.h index 7cb2c78815..71f232035d 100644 --- a/erts/emulator/beam/bif.h +++ b/erts/emulator/beam/bif.h @@ -322,27 +322,6 @@ do { \ ERTS_BIF_EXITED((PROC)); \ } while (0) -#ifdef ERTS_SMP -#define ERTS_SMP_BIF_CHK_PENDING_EXIT(P, L) \ -do { \ - ERTS_SMP_LC_ASSERT((L) == erts_proc_lc_my_proc_locks((P))); \ - ERTS_SMP_LC_ASSERT(ERTS_PROC_LOCK_MAIN & (L)); \ - if (!((L) & ERTS_PROC_LOCK_STATUS)) \ - erts_smp_proc_lock((P), ERTS_PROC_LOCK_STATUS); \ - if (ERTS_PROC_PENDING_EXIT((P))) { \ - erts_handle_pending_exit((P), (L)|ERTS_PROC_LOCK_STATUS); \ - erts_smp_proc_unlock((P), \ - (((L)|ERTS_PROC_LOCK_STATUS) \ - & ~ERTS_PROC_LOCK_MAIN)); \ - ERTS_BIF_EXITED((P)); \ - } \ - if (!((L) & ERTS_PROC_LOCK_STATUS)) \ - erts_smp_proc_unlock((P), ERTS_PROC_LOCK_STATUS); \ -} while (0) -#else -#define ERTS_SMP_BIF_CHK_PENDING_EXIT(P, L) -#endif - /* * The ERTS_BIF_*_AWAIT_X_*_TRAP makros either exits the caller, or * sets up a trap to erlang:await_proc_exit/3. diff --git a/erts/emulator/beam/bif.tab b/erts/emulator/beam/bif.tab index f7dad2767f..59a91cd40c 100644 --- a/erts/emulator/beam/bif.tab +++ b/erts/emulator/beam/bif.tab @@ -31,432 +31,233 @@ # # Important: Use "ubif" for guard BIFs and operators; use "bif" for ordinary BIFs. # -# Add new BIFs to the end of the file. Do not bother adding a "packaged BIF name" -# (such as 'erl.lang.number'); if/when packages will be supported we will add -# all those names. +# Add new BIFs to the end of the file. # # Note: Guards BIFs require special support in the compiler (to be able to actually # call them from within a guard). # ubif erlang:abs/1 -ubif 'erl.lang.number':abs/1 ebif_abs_1 bif erlang:adler32/1 -bif 'erl.util.crypt.adler32':sum/1 ebif_adler32_1 bif erlang:adler32/2 -bif 'erl.util.crypt.adler32':sum/2 ebif_adler32_2 bif erlang:adler32_combine/3 -bif 'erl.util.crypt.adler32':combine/3 ebif_adler32_combine_3 bif erlang:apply/3 -bif 'erl.lang':apply/3 ebif_apply_3 bif erlang:atom_to_list/1 -bif 'erl.lang.atom':to_string/1 ebif_atom_to_string_1 atom_to_list_1 bif erlang:binary_to_list/1 -bif 'erl.lang.binary':to_list/1 ebif_binary_to_list_1 bif erlang:binary_to_list/3 -bif 'erl.lang.binary':to_list/3 ebif_binary_to_list_3 bif erlang:binary_to_term/1 -bif 'erl.lang.binary':to_term/1 ebif_binary_to_term_1 bif erlang:check_process_code/2 -bif 'erl.system.code':check_process/2 ebif_check_process_code_2 bif erlang:crc32/1 -bif 'erl.util.crypt.crc32':sum/1 ebif_crc32_1 bif erlang:crc32/2 -bif 'erl.util.crypt.crc32':sum/2 ebif_crc32_2 bif erlang:crc32_combine/3 -bif 'erl.util.crypt.crc32':combine/3 ebif_crc32_combine_3 bif erlang:date/0 -bif 'erl.util.date':today/0 ebif_date_0 bif erlang:delete_module/1 -bif 'erl.system.code':delete/1 ebif_delete_module_1 bif erlang:display/1 -bif 'erl.system.debug':display/1 ebif_display_1 bif erlang:display_string/1 -bif 'erl.system.debug':display_string/1 ebif_display_string_1 bif erlang:display_nl/0 -bif 'erl.system.debug':display_nl/0 ebif_display_nl_0 ubif erlang:element/2 -ubif 'erl.lang.tuple':element/2 ebif_element_2 bif erlang:erase/0 -bif 'erl.lang.proc.pdict':erase/0 ebif_erase_0 bif erlang:erase/1 -bif 'erl.lang.proc.pdict':erase/1 ebif_erase_1 bif erlang:exit/1 -bif 'erl.lang':exit/1 ebif_exit_1 bif erlang:exit/2 -bif 'erl.lang.proc':signal/2 ebif_signal_2 exit_2 bif erlang:external_size/1 -bif 'erl.lang.term':external_size/1 ebif_external_size_1 bif erlang:external_size/2 -bif 'erl.lang.term':external_size/2 ebif_external_size_2 ubif erlang:float/1 -ubif 'erl.lang.number':to_float/1 ebif_to_float_1 float_1 bif erlang:float_to_list/1 -bif 'erl.lang.float':to_string/1 ebif_float_to_string_1 float_to_list_1 bif erlang:fun_info/2 -bif 'erl.lang.function':info/2 ebif_fun_info_2 bif erlang:garbage_collect/0 -bif 'erl.system':garbage_collect/0 ebif_garbage_collect_0 bif erlang:garbage_collect/1 -bif 'erl.system':garbage_collect/1 ebif_garbage_collect_1 bif erlang:get/0 -bif 'erl.lang.proc.pdict':get/0 ebif_get_0 bif erlang:get/1 -bif 'erl.lang.proc.pdict':get/1 ebif_get_1 bif erlang:get_keys/1 -bif 'erl.lang.proc.pdict':get_keys/1 ebif_get_keys_1 bif erlang:group_leader/0 -bif 'erl.lang.proc':group_leader/0 ebif_group_leader_0 bif erlang:group_leader/2 -bif 'erl.lang.proc':set_group_leader/2 ebif_group_leader_2 bif erlang:halt/0 -bif 'erl.lang.system':halt/0 ebif_halt_0 bif erlang:halt/1 -bif 'erl.lang.system':halt/1 ebif_halt_1 bif erlang:halt/2 -bif 'erl.lang.system':halt/2 ebif_halt_2 bif erlang:phash/2 bif erlang:phash2/1 bif erlang:phash2/2 -bif 'erl.lang.term':hash/1 ebif_phash2_1 -bif 'erl.lang.term':hash/2 ebif_phash2_2 ubif erlang:hd/1 -ubif 'erl.lang.list':hd/1 ebif_hd_1 bif erlang:integer_to_list/1 -bif 'erl.lang.integer':to_string/1 ebif_integer_to_string_1 integer_to_list_1 bif erlang:is_alive/0 -bif 'erl.lang.node':is_alive/0 ebif_is_alive_0 ubif erlang:length/1 -ubif 'erl.lang.list':length/1 ebif_length_1 bif erlang:link/1 -bif 'erl.lang.proc':link/1 ebif_link_1 bif erlang:list_to_atom/1 -bif 'erl.lang.atom':from_string/1 ebif_string_to_atom_1 list_to_atom_1 bif erlang:list_to_binary/1 -bif 'erl.lang.binary':from_list/1 ebif_list_to_binary_1 bif erlang:list_to_float/1 -bif 'erl.lang.float':from_string/1 ebif_string_to_float_1 list_to_float_1 bif erlang:list_to_integer/1 -bif 'erl.lang.integer':from_string/1 ebif_string_to_integer_1 list_to_integer_1 bif erlang:list_to_pid/1 -bif 'erl.lang.proc':string_to_pid/1 ebif_string_to_pid_1 list_to_pid_1 bif erlang:list_to_tuple/1 -bif 'erl.lang.tuple':from_list/1 ebif_list_to_tuple_1 bif erlang:loaded/0 -bif 'erl.system.code':loaded/0 ebif_loaded_0 bif erlang:localtime/0 -bif 'erl.util.date':local/0 ebif_localtime_0 bif erlang:localtime_to_universaltime/2 -bif 'erl.util.date':local_to_utc/2 ebif_localtime_to_universaltime_2 bif erlang:make_ref/0 -bif 'erl.lang.ref':new/0 ebif_make_ref_0 bif erlang:md5/1 -bif 'erl.util.crypt.md5':digest/1 ebif_md5_1 bif erlang:md5_init/0 -bif 'erl.util.crypt.md5':init/0 ebif_md5_init_0 bif erlang:md5_update/2 -bif 'erl.util.crypt.md5':update/2 ebif_md5_update_2 bif erlang:md5_final/1 -bif 'erl.util.crypt.md5':final/1 ebif_md5_final_1 bif erlang:module_loaded/1 -bif 'erl.system.code':is_loaded/1 ebif_is_loaded_1 module_loaded_1 bif erlang:function_exported/3 -bif 'erl.system.code':is_loaded/3 ebif_is_loaded_3 function_exported_3 bif erlang:monitor_node/2 -bif 'erl.lang.node':monitor/2 ebif_monitor_node_2 bif erlang:monitor_node/3 -bif 'erl.lang.node':monitor/3 ebif_monitor_node_3 ubif erlang:node/1 -ubif 'erl.lang.node':node/1 ebif_node_1 ubif erlang:node/0 -ubif 'erl.lang.node':node/0 ebif_node_0 bif erlang:nodes/1 -bif 'erl.lang.node':nodes/1 ebif_nodes_1 bif erlang:now/0 -bif 'erl.system':now/0 ebif_now_0 bif erlang:open_port/2 -bif 'erl.lang.port':open/2 ebif_open_port_2 open_port_2 bif erlang:pid_to_list/1 -bif 'erl.lang.proc':pid_to_string/1 ebif_pid_to_string_1 pid_to_list_1 -bif erlang:port_info/1 -bif 'erl.lang.port':info/1 ebif_port_info_1 -bif erlang:port_info/2 -bif 'erl.lang.port':info/2 ebif_port_info_2 bif erlang:ports/0 -bif 'erl.lang.node':ports/0 ebif_ports_0 bif erlang:pre_loaded/0 -bif 'erl.system.code':preloaded/0 ebif_pre_loaded_0 bif erlang:process_flag/2 -bif 'erl.lang.proc':set_flag/2 ebif_process_flag_2 bif erlang:process_flag/3 -bif 'erl.lang.proc':set_flag/3 ebif_process_flag_3 bif erlang:process_info/1 -bif 'erl.lang.proc':info/1 ebif_process_info_1 bif erlang:process_info/2 -bif 'erl.lang.proc':info/2 ebif_process_info_2 bif erlang:processes/0 -bif 'erl.lang.node':processes/0 ebif_processes_0 bif erlang:purge_module/1 -bif 'erl.system.code':purge/1 ebif_purge_module_1 bif erlang:put/2 -bif 'erl.lang.proc.pdict':put/2 ebif_put_2 bif erlang:register/2 -bif 'erl.lang.node':register/2 ebif_register_2 bif erlang:registered/0 -bif 'erl.lang.node':registered/0 ebif_registered_0 ubif erlang:round/1 -ubif 'erl.lang.number':round/1 ebif_round_1 ubif erlang:self/0 -ubif 'erl.lang.proc':self/0 ebif_self_0 bif erlang:setelement/3 -bif 'erl.lang.tuple':setelement/3 ebif_setelement_3 ubif erlang:size/1 -ubif 'erl.lang.term':size/1 ebif_size_1 bif erlang:spawn/3 -bif 'erl.lang.proc':spawn/3 ebif_spawn_3 bif erlang:spawn_link/3 -bif 'erl.lang.proc':spawn_link/3 ebif_spawn_link_3 bif erlang:split_binary/2 -bif 'erl.lang.binary':split/2 ebif_split_binary_2 bif erlang:statistics/1 -bif 'erl.system':statistics/1 ebif_statistics_1 bif erlang:term_to_binary/1 -bif 'erl.lang.binary':from_term/1 ebif_term_to_binary_1 bif erlang:term_to_binary/2 -bif 'erl.lang.binary':from_term/2 ebif_term_to_binary_2 bif erlang:throw/1 -bif 'erl.lang':throw/1 ebif_throw_1 bif erlang:time/0 -bif 'erl.util.date':time_of_day/0 ebif_time_0 ubif erlang:tl/1 -ubif 'erl.lang.list':tl/1 ebif_tl_1 ubif erlang:trunc/1 -ubif 'erl.lang.number':trunc/1 ebif_trunc_1 bif erlang:tuple_to_list/1 -bif 'erl.lang.tuple':to_list/1 ebif_tuple_to_list_1 bif erlang:universaltime/0 -bif 'erl.util.date':utc/0 ebif_universaltime_0 bif erlang:universaltime_to_localtime/1 -bif 'erl.util.date':utc_to_local/1 ebif_universaltime_to_localtime_1 bif erlang:unlink/1 -bif 'erl.lang.proc':unlink/1 ebif_unlink_1 bif erlang:unregister/1 -bif 'erl.lang.node':unregister/1 ebif_unregister_1 bif erlang:whereis/1 -bif 'erl.lang.node':whereis/1 ebif_whereis_1 bif erlang:spawn_opt/1 -bif 'erl.lang.proc':spawn_opt/1 ebif_spawn_opt_1 bif erlang:setnode/2 bif erlang:setnode/3 bif erlang:dist_exit/3 -bif erlang:port_call/2 -bif 'erl.lang.port':call/2 ebif_port_call_2 -bif erlang:port_call/3 -bif 'erl.lang.port':call/3 ebif_port_call_3 -bif erlang:port_command/2 -bif 'erl.lang.port':command/2 ebif_port_command_2 -bif erlang:port_command/3 -bif 'erl.lang.port':command/3 ebif_port_command_3 -bif erlang:port_control/3 -bif 'erl.lang.port':control/3 ebif_port_control_3 -bif erlang:port_close/1 -bif 'erl.lang.port':close/1 ebif_port_close_1 -bif erlang:port_connect/2 -bif 'erl.lang.port':connect/2 ebif_port_connect_2 -bif erlang:port_set_data/2 -bif 'erl.lang.port':set_data/2 ebif_port_set_data_2 -bif erlang:port_get_data/1 -bif 'erl.lang.port':get_data/1 ebif_port_get_data_1 +# Static native functions in erts_internal +bif erts_internal:port_info/1 +bif erts_internal:port_info/2 +bif erts_internal:port_call/3 +bif erts_internal:port_command/3 +bif erts_internal:port_control/3 +bif erts_internal:port_close/1 +bif erts_internal:port_connect/2 +bif erts_internal:port_set_data/2 +bif erts_internal:port_get_data/1 # Tracing & debugging. bif erlang:trace_pattern/2 -bif 'erl.system.debug':trace_pattern/2 ebif_trace_pattern_2 bif erlang:trace_pattern/3 -bif 'erl.system.debug':trace_pattern/3 ebif_trace_pattern_3 bif erlang:trace/3 -bif 'erl.system.debug':trace/3 ebif_trace_3 bif erlang:trace_info/2 -bif 'erl.system.debug':trace_info/2 ebif_trace_info_2 bif erlang:trace_delivered/1 -bif 'erl.system.debug':trace_delivered/1 ebif_trace_delivered_1 bif erlang:seq_trace/2 -bif 'erl.system.debug':seq_trace/2 ebif_seq_trace_2 bif erlang:seq_trace_info/1 -bif 'erl.system.debug':seq_trace_info/1 ebif_seq_trace_info_1 bif erlang:seq_trace_print/1 -bif 'erl.system.debug':seq_trace_print/1 ebif_seq_trace_print_1 bif erlang:seq_trace_print/2 -bif 'erl.system.debug':seq_trace_print/2 ebif_seq_trace_print_2 bif erlang:suspend_process/2 -bif 'erl.system.debug':suspend_process/2 ebif_suspend_process_2 bif erlang:resume_process/1 -bif 'erl.system.debug':resume_process/1 ebif_resume_process_1 bif erlang:process_display/2 -bif 'erl.system.debug':process_display/2 ebif_process_display_2 bif erlang:bump_reductions/1 -bif 'erl.lang.proc':bump_reductions/1 ebif_bump_reductions_1 bif math:cos/1 -bif 'erl.lang.math':cos/1 ebif_math_cos_1 bif math:cosh/1 -bif 'erl.lang.math':cosh/1 ebif_math_cosh_1 bif math:sin/1 -bif 'erl.lang.math':sin/1 ebif_math_sin_1 bif math:sinh/1 -bif 'erl.lang.math':sinh/1 ebif_math_sinh_1 bif math:tan/1 -bif 'erl.lang.math':tan/1 ebif_math_tan_1 bif math:tanh/1 -bif 'erl.lang.math':tanh/1 ebif_math_tanh_1 bif math:acos/1 -bif 'erl.lang.math':acos/1 ebif_math_acos_1 bif math:acosh/1 -bif 'erl.lang.math':acosh/1 ebif_math_acosh_1 bif math:asin/1 -bif 'erl.lang.math':asin/1 ebif_math_asin_1 bif math:asinh/1 -bif 'erl.lang.math':asinh/1 ebif_math_asinh_1 bif math:atan/1 -bif 'erl.lang.math':atan/1 ebif_math_atan_1 bif math:atanh/1 -bif 'erl.lang.math':atanh/1 ebif_math_atanh_1 bif math:erf/1 -bif 'erl.lang.math':erf/1 ebif_math_erf_1 bif math:erfc/1 -bif 'erl.lang.math':erfc/1 ebif_math_erfc_1 bif math:exp/1 -bif 'erl.lang.math':exp/1 ebif_math_exp_1 bif math:log/1 -bif 'erl.lang.math':log/1 ebif_math_log_1 bif math:log10/1 -bif 'erl.lang.math':log10/1 ebif_math_log10_1 bif math:sqrt/1 -bif 'erl.lang.math':sqrt/1 ebif_math_sqrt_1 bif math:atan2/2 -bif 'erl.lang.math':atan2/2 ebif_math_atan2_2 bif math:pow/2 -bif 'erl.lang.math':pow/2 ebif_math_pow_2 bif erlang:start_timer/3 -bif 'erl.lang.timer':start/3 ebif_start_timer_3 bif erlang:send_after/3 -bif 'erl.lang.timer':send_after/3 ebif_send_after_3 bif erlang:cancel_timer/1 -bif 'erl.lang.timer':cancel/1 ebif_cancel_timer_1 bif erlang:read_timer/1 -bif 'erl.lang.timer':read/1 ebif_read_timer_1 bif erlang:make_tuple/2 -bif 'erl.lang.tuple':make/2 ebif_make_tuple_2 bif erlang:append_element/2 -bif 'erl.lang.tuple':append_element/2 ebif_append_element_2 bif erlang:make_tuple/3 bif erlang:system_flag/2 -bif 'erl.system':set_flag/2 ebif_system_flag_2 bif erlang:system_info/1 -bif 'erl.system':info/1 ebif_system_info_1 # New in R9C bif erlang:system_monitor/0 -bif 'erl.system':monitor/0 ebif_system_monitor_0 bif erlang:system_monitor/1 -bif 'erl.system':monitor/1 ebif_system_monitor_1 bif erlang:system_monitor/2 -bif 'erl.system':monitor/2 ebif_system_monitor_2 # Added 2006-11-07 bif erlang:system_profile/2 -bif 'erl.system':profile/2 ebif_system_profile_2 # End Added 2006-11-07 # Added 2007-01-17 bif erlang:system_profile/0 -bif 'erl.system':profile/0 ebif_system_profile_0 # End Added 2007-01-17 bif erlang:ref_to_list/1 -bif 'erl.lang.ref':to_string/1 ebif_ref_to_string_1 ref_to_list_1 bif erlang:port_to_list/1 -bif 'erl.lang.port':to_string/1 ebif_port_to_string_1 port_to_list_1 bif erlang:fun_to_list/1 -bif 'erl.lang.function':to_string/1 ebif_fun_to_string_1 fun_to_list_1 bif erlang:monitor/2 -bif 'erl.lang.proc':monitor/2 ebif_monitor_2 bif erlang:demonitor/1 -bif 'erl.lang.proc':demonitor/1 ebif_demonitor_1 bif erlang:demonitor/2 -bif 'erl.lang.proc':demonitor/2 ebif_demonitor_2 bif erlang:is_process_alive/1 -bif 'erl.lang.proc':is_alive/1 ebif_proc_is_alive_1 is_process_alive_1 bif erlang:error/1 error_1 -bif 'erl.lang':error/1 ebif_error_1 error_1 bif erlang:error/2 error_2 -bif 'erl.lang':error/2 ebif_error_2 error_2 bif erlang:raise/3 raise_3 -bif 'erl.lang':raise/3 ebif_raise_3 raise_3 bif erlang:get_stacktrace/0 -bif 'erl.lang.proc':get_stacktrace/0 ebif_get_stacktrace_0 bif erlang:is_builtin/3 -bif 'erl.system.code':is_builtin/3 ebif_is_builtin_3 ubif erlang:'and'/2 -ubif 'erl.lang.bool':'and'/2 ebif_and_2 ubif erlang:'or'/2 -ubif 'erl.lang.bool':'or'/2 ebif_or_2 ubif erlang:'xor'/2 -ubif 'erl.lang.bool':'xor'/2 ebif_xor_2 ubif erlang:'not'/1 -ubif 'erl.lang.bool':'not'/1 ebif_not_1 ubif erlang:'>'/2 sgt_2 -ubif 'erl.lang.term':greater/2 ebif_gt_2 sgt_2 ubif erlang:'>='/2 sge_2 -ubif 'erl.lang.term':greater_or_equal/2 ebif_ge_2 sge_2 ubif erlang:'<'/2 slt_2 -ubif 'erl.lang.term':less/2 ebif_lt_2 slt_2 ubif erlang:'=<'/2 sle_2 -ubif 'erl.lang.term':less_or_equal/2 ebif_le_2 sle_2 ubif erlang:'=:='/2 seq_2 -ubif 'erl.lang.term':equal/2 ebif_eq_2 seq_2 ubif erlang:'=='/2 seqeq_2 -ubif 'erl.lang.term':arith_equal/2 ebif_areq_2 seqeq_2 ubif erlang:'=/='/2 sneq_2 -ubif 'erl.lang.term':not_equal/2 ebif_neq_2 sneq_2 ubif erlang:'/='/2 sneqeq_2 -ubif 'erl.lang.term':not_arith_equal/2 ebif_nareq_2 sneqeq_2 ubif erlang:'+'/2 splus_2 -ubif 'erl.lang.number':plus/2 ebif_plus_2 splus_2 ubif erlang:'-'/2 sminus_2 -ubif 'erl.lang.number':minus/2 ebif_minus_2 sminus_2 ubif erlang:'*'/2 stimes_2 -ubif 'erl.lang.number':multiply/2 ebif_multiply_2 stimes_2 ubif erlang:'/'/2 div_2 -ubif 'erl.lang.number':divide/2 ebif_divide_2 div_2 ubif erlang:'div'/2 intdiv_2 -ubif 'erl.lang.integer':'div'/2 ebif_intdiv_2 ubif erlang:'rem'/2 -ubif 'erl.lang.integer':'rem'/2 ebif_rem_2 ubif erlang:'bor'/2 -ubif 'erl.lang.integer':'bor'/2 ebif_bor_2 ubif erlang:'band'/2 -ubif 'erl.lang.integer':'band'/2 ebif_band_2 ubif erlang:'bxor'/2 -ubif 'erl.lang.integer':'bxor'/2 ebif_bxor_2 ubif erlang:'bsl'/2 -ubif 'erl.lang.integer':'bsl'/2 ebif_bsl_2 ubif erlang:'bsr'/2 -ubif 'erl.lang.integer':'bsr'/2 ebif_bsr_2 ubif erlang:'bnot'/1 -ubif 'erl.lang.integer':'bnot'/1 ebif_bnot_1 ubif erlang:'-'/1 sminus_1 -ubif 'erl.lang.number':minus/1 ebif_minus_1 sminus_1 ubif erlang:'+'/1 splus_1 -ubif 'erl.lang.number':plus/1 ebif_plus_1 splus_1 # New operators in R8. These were the only operators missing. # erlang:send/2, erlang:append/2 and erlang:subtract/2 are now also @@ -464,45 +265,27 @@ ubif 'erl.lang.number':plus/1 ebif_plus_1 splus_1 # internal references have been updated to the new ebif_... entries. bif erlang:'!'/2 ebif_bang_2 -bif 'erl.lang.proc':send/2 ebif_send_2 send_2 bif erlang:send/2 -bif 'erl.lang':send/3 ebif_send_3 send_3 bif erlang:send/3 bif erlang:'++'/2 ebif_plusplus_2 -bif 'erl.lang.list':append/2 ebif_append_2 ebif_plusplus_2 bif erlang:append/2 bif erlang:'--'/2 ebif_minusminus_2 -bif 'erl.lang.list':subtract/2 ebif_list_subtract_2 ebif_minusminus_2 bif erlang:subtract/2 ubif erlang:is_atom/1 -ubif 'erl.lang.term':is_atom/1 ebif_is_atom_1 ubif erlang:is_list/1 -ubif 'erl.lang.term':is_list/1 ebif_is_list_1 ubif erlang:is_tuple/1 -ubif 'erl.lang.term':is_tuple/1 ebif_is_tuple_1 ubif erlang:is_float/1 -ubif 'erl.lang.term':is_float/1 ebif_is_float_1 ubif erlang:is_integer/1 -ubif 'erl.lang.term':is_integer/1 ebif_is_integer_1 ubif erlang:is_number/1 -ubif 'erl.lang.term':is_number/1 ebif_is_number_1 ubif erlang:is_pid/1 -ubif 'erl.lang.term':is_pid/1 ebif_is_pid_1 ubif erlang:is_port/1 -ubif 'erl.lang.term':is_port/1 ebif_is_port_1 ubif erlang:is_reference/1 -ubif 'erl.lang.term':is_reference/1 ebif_is_reference_1 ubif erlang:is_binary/1 -ubif 'erl.lang.term':is_binary/1 ebif_is_binary_1 ubif erlang:is_function/1 -ubif 'erl.lang.term':is_function/1 ebif_is_function_1 ubif erlang:is_function/2 -ubif 'erl.lang.term':is_function/2 ebif_is_function_2 ubif erlang:is_record/2 -ubif 'erl.lang.term':is_record/2 ebif_is_record_2 ubif erlang:is_record/3 -ubif 'erl.lang.term':is_record/3 ebif_is_record_3 bif erlang:match_spec_test/3 @@ -511,96 +294,53 @@ bif erlang:match_spec_test/3 # bif ets:all/0 -bif 'erl.lang.ets':all/0 ebif_ets_all_0 bif ets:new/2 -bif 'erl.lang.ets':new/2 ebif_ets_new_2 bif ets:delete/1 -bif 'erl.lang.ets':delete/1 ebif_ets_delete_1 bif ets:delete/2 -bif 'erl.lang.ets':delete/2 ebif_ets_delete_2 bif ets:delete_all_objects/1 -bif 'erl.lang.ets':delete_all_objects/1 ebif_ets_delete_all_objects_1 bif ets:delete_object/2 -bif 'erl.lang.ets':delete_object/2 ebif_ets_delete_object_2 bif ets:first/1 -bif 'erl.lang.ets':first/1 ebif_ets_first_1 bif ets:is_compiled_ms/1 -bif 'erl.lang.ets':is_compiled_ms/1 ebif_ets_is_compiled_ms_1 bif ets:lookup/2 -bif 'erl.lang.ets':lookup/2 ebif_ets_lookup_2 bif ets:lookup_element/3 -bif 'erl.lang.ets':lookup_element/3 ebif_ets_lookup_element_3 bif ets:info/1 -bif 'erl.lang.ets':info/1 ebif_ets_info_1 bif ets:info/2 -bif 'erl.lang.ets':info/2 ebif_ets_info_2 bif ets:last/1 -bif 'erl.lang.ets':last/1 ebif_ets_last_1 bif ets:match/1 -bif 'erl.lang.ets':match/1 ebif_ets_match_1 bif ets:match/2 -bif 'erl.lang.ets':match/2 ebif_ets_match_2 bif ets:match/3 -bif 'erl.lang.ets':match/3 ebif_ets_match_3 bif ets:match_object/1 -bif 'erl.lang.ets':match_object/1 ebif_ets_match_object_1 bif ets:match_object/2 -bif 'erl.lang.ets':match_object/2 ebif_ets_match_object_2 bif ets:match_object/3 -bif 'erl.lang.ets':match_object/3 ebif_ets_match_object_3 bif ets:member/2 -bif 'erl.lang.ets':is_key/2 ebif_ets_member_2 bif ets:next/2 -bif 'erl.lang.ets':next/2 ebif_ets_next_2 bif ets:prev/2 -bif 'erl.lang.ets':prev/2 ebif_ets_prev_2 bif ets:insert/2 -bif 'erl.lang.ets':insert/2 ebif_ets_insert_2 bif ets:insert_new/2 -bif 'erl.lang.ets':insert_new/2 ebif_ets_insert_new_2 bif ets:rename/2 -bif 'erl.lang.ets':rename/2 ebif_ets_rename_2 bif ets:safe_fixtable/2 -bif 'erl.lang.ets':fixtable/2 ebif_ets_safe_fixtable_2 bif ets:slot/2 -bif 'erl.lang.ets':slot/2 ebif_ets_slot_2 bif ets:update_counter/3 -bif 'erl.lang.ets':update_counter/3 ebif_ets_update_counter_3 bif ets:select/1 -bif 'erl.lang.ets':select/1 ebif_ets_select_1 bif ets:select/2 -bif 'erl.lang.ets':select/2 ebif_ets_select_2 bif ets:select/3 -bif 'erl.lang.ets':select/3 ebif_ets_select_3 bif ets:select_count/2 -bif 'erl.lang.ets':select/2 ebif_ets_select_count_2 bif ets:select_reverse/1 -bif 'erl.lang.ets':select_reverse/1 ebif_ets_select_reverse_1 bif ets:select_reverse/2 -bif 'erl.lang.ets':select_reverse/2 ebif_ets_select_reverse_2 bif ets:select_reverse/3 -bif 'erl.lang.ets':select_reverse/3 ebif_ets_select_reverse_3 bif ets:select_delete/2 -bif 'erl.lang.ets':select_delete/2 ebif_ets_select_delete_2 bif ets:match_spec_compile/1 -bif 'erl.lang.ets':match_spec_compile/1 ebif_ets_match_spec_compile_1 bif ets:match_spec_run_r/3 -bif 'erl.lang.ets':match_spec_run_r/3 ebif_ets_match_spec_run_r_3 # # Bifs in os module. # bif os:putenv/2 -bif 'erl.system.os':setenv/2 ebif_os_setenv_2 os_putenv_2 bif os:getenv/0 -bif 'erl.system.os':getenv/0 ebif_os_getenv_0 bif os:getenv/1 -bif 'erl.system.os':getenv/1 ebif_os_getenv_1 bif os:getpid/0 -bif 'erl.system.os':pid/0 ebif_os_pid_0 os_getpid_0 bif os:timestamp/0 -bif 'erl.system.os':timestamp/0 ebif_os_timestamp_0 os_timestamp_0 # # Bifs in the erl_ddll module (the module actually does not exist) @@ -627,13 +367,9 @@ bif re:run/3 # bif lists:member/2 -bif 'erl.lang.list':is_element/2 ebif_list_is_element_2 lists_member_2 bif lists:reverse/2 -bif 'erl.lang.list':reverse/2 ebif_list_reverse_2 lists_reverse_2 bif lists:keymember/3 -bif 'erl.lang.list.keylist':is_element/3 ebif_keylist_is_element_3 lists_keymember_3 bif lists:keysearch/3 -bif 'erl.lang.list.keylist':search/3 ebif_keylist_search_3 lists_keysearch_3 bif lists:keyfind/3 # @@ -641,21 +377,13 @@ bif lists:keyfind/3 # bif erts_debug:disassemble/1 -bif 'erl.system.debug':disassemble/1 ebif_erts_debug_disassemble_1 bif erts_debug:breakpoint/2 -bif 'erl.system.debug':breakpoint/2 ebif_erts_debug_breakpoint_2 bif erts_debug:same/2 -bif 'erl.system.debug':same/2 ebif_erts_debug_same_2 bif erts_debug:flat_size/1 -bif 'erl.system.debug':flat_size/1 ebif_erts_debug_flat_size_1 bif erts_debug:get_internal_state/1 -bif 'erl.system.debug':get_internal_state/1 ebif_erts_debug_get_internal_state_1 bif erts_debug:set_internal_state/2 -bif 'erl.system.debug':set_internal_state/2 ebif_erts_debug_set_internal_state_2 bif erts_debug:display/1 -bif 'erl.system.debug':display/1 ebif_erts_debug_display_1 bif erts_debug:dist_ext_to_term/2 -bif 'erl.system.debug':dist_ext_to_term/2 ebif_erts_debug_dist_ext_to_term_2 bif erts_debug:instructions/0 # @@ -675,13 +403,9 @@ bif erts_debug:lock_counters/1 # bif code:get_chunk/2 -bif 'erl.system.code':get_chunk/2 ebif_code_get_chunk_2 bif code:module_md5/1 -bif 'erl.system.code':module_md5/1 ebif_code_module_md5_1 bif code:make_stub_module/3 -bif 'erl.system.code':make_stub_module/3 ebif_code_make_stub_module_3 bif code:is_module_native/1 -bif 'erl.system.code':is_native/1 ebif_code_is_native_1 code_is_module_native_1 # # New Bifs in R9C. @@ -833,6 +557,8 @@ bif erlang:dt_append_vm_tag_data/1 # bif erlang:prepare_loading/2 bif erlang:finish_loading/1 +bif erlang:insert_element/3 +bif erlang:delete_element/2 # # Obsolete diff --git a/erts/emulator/beam/binary.c b/erts/emulator/beam/binary.c index 3d2725e239..4441dab181 100644 --- a/erts/emulator/beam/binary.c +++ b/erts/emulator/beam/binary.c @@ -355,10 +355,10 @@ BIF_RETTYPE bitstring_to_list_1(BIF_ALIST_1) BIF_RETTYPE erts_list_to_binary_bif(Process *p, Eterm arg) { Eterm bin; - Uint size; + ErlDrvSizeT size; byte* bytes; #ifdef DEBUG - int offset; + ErlDrvSizeT offset; #endif if (is_nil(arg)) { @@ -377,7 +377,7 @@ BIF_RETTYPE erts_list_to_binary_bif(Process *p, Eterm arg) #ifdef DEBUG offset = #endif - io_list_to_buf(arg, (char*) bytes, size); + erts_iolist_to_buf(arg, (char*) bytes, size); ASSERT(offset == 0); BIF_RET(bin); diff --git a/erts/emulator/beam/break.c b/erts/emulator/beam/break.c index 63136d86c9..9aa1e5f30d 100644 --- a/erts/emulator/beam/break.c +++ b/erts/emulator/beam/break.c @@ -61,16 +61,19 @@ extern char* erts_system_version[]; static void port_info(int to, void *to_arg) { - int i; - for (i = 0; i < erts_max_ports; i++) - print_port_info(to, to_arg, i); + int i, max = erts_ptab_max(&erts_port); + for (i = 0; i < max; i++) { + Port *p = erts_pix2port(i); + if (p) + print_port_info(p, to, to_arg); + } } void process_info(int to, void *to_arg) { - int i; - for (i = 0; i < erts_max_processes; i++) { + int i, max = erts_ptab_max(&erts_proc); + for (i = 0; i < max; i++) { Process *p = erts_pix2proc(i); if (p && p->i != ENULL) { if (!ERTS_PROC_IS_EXITING(p)) @@ -84,12 +87,12 @@ process_info(int to, void *to_arg) static void process_killer(void) { - int i, j; + int i, j, max = erts_ptab_max(&erts_proc); Process* rp; erts_printf("\n\nProcess Information\n\n"); erts_printf("--------------------------------------------------\n"); - for (i = erts_max_processes-1; i >= 0; i--) { + for (i = max-1; i >= 0; i--) { rp = erts_pix2proc(i); if (rp && rp->i != ENULL) { int br; @@ -196,7 +199,7 @@ print_process_info(int to, void *to_arg, Process *p) erts_aint32_t state; /* display the PID */ - erts_print(to, to_arg, "=proc:%T\n", p->id); + erts_print(to, to_arg, "=proc:%T\n", p->common.id); /* Display the state */ erts_print(to, to_arg, "State: "); @@ -226,8 +229,8 @@ print_process_info(int to, void *to_arg, Process *p) * If the process is registered as a global process, display the * registered name */ - if (p->reg != NULL) - erts_print(to, to_arg, "Name: %T\n", p->reg->name); + if (p->common.u.alive.reg) + erts_print(to, to_arg, "Name: %T\n", p->common.u.alive.reg->name); /* * Display the initial function name @@ -301,11 +304,11 @@ print_process_info(int to, void *to_arg, Process *p) } /* display the links only if there are any*/ - if (p->nlinks != NULL || p->monitors != NULL) { + if (ERTS_P_LINKS(p) || ERTS_P_MONITORS(p)) { PrintMonitorContext context = {1,to}; erts_print(to, to_arg,"Link list: ["); - erts_doforall_links(p->nlinks, &doit_print_link, &context); - erts_doforall_monitors(p->monitors, &doit_print_monitor, &context); + erts_doforall_links(ERTS_P_LINKS(p), &doit_print_link, &context); + erts_doforall_monitors(ERTS_P_MONITORS(p), &doit_print_monitor, &context); erts_print(to, to_arg,"]\n"); } @@ -625,9 +628,9 @@ bin_check(void) { Process *rp; struct erl_off_heap_header* hdr; - int i, printed = 0; + int i, printed = 0, max = erts_ptab_max(&erts_proc); - for (i=0; i < erts_max_processes; i++) { + for (i=0; i < max; i++) { rp = erts_pix2proc(i); if (!rp) continue; @@ -635,7 +638,7 @@ bin_check(void) if (hdr->thing_word == HEADER_PROC_BIN) { ProcBin *bp = (ProcBin*) hdr; if (!printed) { - erts_printf("Process %T holding binary data \n", rp->id); + erts_printf("Process %T holding binary data \n", rp->common.id); printed = 1; } erts_printf("%p orig_size: %bpd, norefs = %bpd\n", @@ -670,6 +673,7 @@ erl_crash_dump_v(char *file, int line, char* fmt, va_list args) char dumpnamebuf[MAXPATHLEN]; char* dumpname; int secs; + int env_erl_crash_dump_seconds_set = 1; if (ERTS_SOMEONE_IS_CRASH_DUMPING) return; @@ -694,6 +698,8 @@ erl_crash_dump_v(char *file, int line, char* fmt, va_list args) envsz = sizeof(env); /* ERL_CRASH_DUMP_SECONDS not set + * if we have a heart port, break immediately + * otherwise dump crash indefinitely (until crash is complete) * same as ERL_CRASH_DUMP_SECONDS = 0 * - do not write dump * - do not set an alarm @@ -715,8 +721,10 @@ erl_crash_dump_v(char *file, int line, char* fmt, va_list args) */ if (erts_sys_getenv__("ERL_CRASH_DUMP_SECONDS", env, &envsz) != 0) { - return; /* break immediately */ + env_erl_crash_dump_seconds_set = 0; + secs = -1; } else { + env_erl_crash_dump_seconds_set = 1; secs = atoi(env); } @@ -724,7 +732,16 @@ erl_crash_dump_v(char *file, int line, char* fmt, va_list args) return; } - erts_sys_prepare_crash_dump(secs); + /* erts_sys_prepare_crash_dump returns 1 if heart port is found, otherwise 0 + * If we don't find heart (0) and we don't have ERL_CRASH_DUMP_SECONDS set + * we should continue writing a dump + * + * beware: secs -1 means no alarm + */ + + if (erts_sys_prepare_crash_dump(secs) && !env_erl_crash_dump_seconds_set ) { + return; + } if (erts_sys_getenv__("ERL_CRASH_DUMP",&dumpnamebuf[0],&dumpnamebufsize) != 0) dumpname = "erl_crash.dump"; @@ -752,7 +769,7 @@ erl_crash_dump_v(char *file, int line, char* fmt, va_list args) erts_print_nif_taints(fd, NULL); erts_fdprintf(fd, "Atoms: %d\n", atom_table_size()); info(fd, NULL); /* General system info */ - if (erts_proc.tab) + if (erts_ptab_initialized(&erts_proc)) process_info(fd, NULL); /* Info about each process and port */ db_info(fd, NULL, 0); erts_print_bif_timer_info(fd, NULL); diff --git a/erts/emulator/beam/code_ix.h b/erts/emulator/beam/code_ix.h index bc10e956f5..e2bd0da112 100644 --- a/erts/emulator/beam/code_ix.h +++ b/erts/emulator/beam/code_ix.h @@ -18,7 +18,7 @@ */ /* Description: - * This is the interface that facilitate changing the beam code + * This is the interface that facilitates changing the beam code * (load,upgrade,delete) while allowing executing Erlang processes to * access the code without any locks or other expensive memory barriers. * @@ -35,7 +35,7 @@ * The third code index is not explicitly used. It can be thought of as * the "previous active" or the "next staging" index. It is needed to make * sure that we do not reuse a code index for staging until we are sure - * that no executing BIFs are still referring it. + * that no executing BIFs are still referencing it. * We could get by with only two (0 and 1), but that would require that we * must wait for all schedulers to re-schedule before each code change * operation can start staging. diff --git a/erts/emulator/beam/copy.c b/erts/emulator/beam/copy.c index 36eda04de2..23c0fca6aa 100644 --- a/erts/emulator/beam/copy.c +++ b/erts/emulator/beam/copy.c @@ -47,7 +47,7 @@ copy_object(Eterm obj, Process* to) if (DTRACE_ENABLED(copy_object)) { DTRACE_CHARBUF(proc_name, 64); - erts_snprintf(proc_name, sizeof(proc_name), "%T", to->id); + erts_snprintf(proc_name, sizeof(proc_name), "%T", to->common.id); DTRACE2(copy_object, proc_name, size); } #endif diff --git a/erts/emulator/beam/dist.c b/erts/emulator/beam/dist.c index 28c4621ff2..64cd93a100 100644 --- a/erts/emulator/beam/dist.c +++ b/erts/emulator/beam/dist.c @@ -67,7 +67,7 @@ dist_msg_dbg(ErtsDistExternal *edep, char *what, byte *buf, int sz) { byte *extp = edep->extp; Eterm msg; - Sint size = erts_decode_dist_ext_size(edep, 0); + Sint size = erts_decode_dist_ext_size(edep); if (size < 0) { erts_fprintf(stderr, "DIST MSG DEBUG: erts_decode_dist_ext_size(%s) failed:\n", @@ -124,6 +124,13 @@ static void send_nodes_mon_msgs(Process *, Eterm, Eterm, Eterm, Eterm); static void init_nodes_monitors(void); static erts_smp_atomic_t no_caches; +static erts_smp_atomic_t no_nodes; + +struct { + Eterm reason; + ErlHeapFragment *bp; +} nodedown; + static void delete_cache(ErtsAtomCache *cache) @@ -144,7 +151,7 @@ create_cache(DistEntry *dep) ERTS_SMP_LC_ASSERT( is_internal_port(dep->cid) - && erts_lc_is_port_locked(&erts_port[internal_port_index(dep->cid)])); + && erts_lc_is_port_locked(erts_port_lookup_raw(dep->cid))); ASSERT(!dep->cache); dep->cache = cp = (ErtsAtomCache*) erts_alloc(ERTS_ALC_T_DCACHE, @@ -171,11 +178,10 @@ get_suspended_on_de(DistEntry *dep, Uint32 unset_qflgs) return NULL; } else { - ErtsProcList *plp; - plp = dep->suspended.first; - dep->suspended.first = NULL; - dep->suspended.last = NULL; - return plp; + ErtsProcList *suspended = dep->suspended; + dep->suspended = NULL; + erts_proclist_fetch(&suspended, NULL); + return suspended; } } @@ -252,7 +258,7 @@ static void doit_monitor_net_exits(ErtsMonitor *mon, void *vnecp) if (mon->type == MON_ORIGIN) { /* local pid is beeing monitored */ - rmon = erts_remove_monitor(&(rp->monitors),mon->ref); + rmon = erts_remove_monitor(&ERTS_P_MONITORS(rp), mon->ref); /* ASSERT(rmon != NULL); nope, can happen during process exit */ if (rmon != NULL) { erts_destroy_monitor(rmon); @@ -262,7 +268,7 @@ static void doit_monitor_net_exits(ErtsMonitor *mon, void *vnecp) Eterm watched; UseTmpHeapNoproc(3); ASSERT(mon->type == MON_TARGET); - rmon = erts_remove_monitor(&(rp->monitors),mon->ref); + rmon = erts_remove_monitor(&ERTS_P_MONITORS(rp), mon->ref); /* ASSERT(rmon != NULL); can happen during process exit */ if (rmon != NULL) { ASSERT(is_atom(rmon->name) || is_nil(rmon->name)); @@ -311,7 +317,7 @@ static void doit_link_net_exits_sub(ErtsLink *sublnk, void *vlnecp) goto done; } - rlnk = erts_remove_link(&(rp->nlinks), sublnk->pid); + rlnk = erts_remove_link(&ERTS_P_LINKS(rp), sublnk->pid); xres = erts_send_exit_signal(NULL, sublnk->pid, rp, @@ -370,7 +376,7 @@ static void doit_node_link_net_exits(ErtsLink *lnk, void *vnecp) if (!rp) { goto done; } - rlnk = erts_remove_link(&(rp->nlinks), name); + rlnk = erts_remove_link(&ERTS_P_LINKS(rp), name); if (rlnk != NULL) { ASSERT(is_atom(rlnk->pid) && (rlnk->type == LINK_NODE)); erts_destroy_link(rlnk); @@ -394,6 +400,47 @@ static void doit_node_link_net_exits(ErtsLink *lnk, void *vnecp) erts_destroy_link(lnk); } +static void +set_node_not_alive(void *unused) +{ + ErlHeapFragment *bp; + Eterm nodename = erts_this_dist_entry->sysname; + + ASSERT(erts_smp_atomic_read_nob(&no_nodes) == 0); + + erts_smp_thr_progress_block(); + erts_set_this_node(am_Noname, 0); + erts_is_alive = 0; + send_nodes_mon_msgs(NULL, am_nodedown, nodename, am_visible, nodedown.reason); + nodedown.reason = NIL; + bp = nodedown.bp; + nodedown.bp = NULL; + erts_smp_thr_progress_unblock(); + if (bp) + free_message_buffer(bp); +} + +static ERTS_INLINE void +dec_no_nodes(void) +{ + erts_aint_t no = erts_smp_atomic_dec_read_mb(&no_nodes); + ASSERT(no >= 0); + ASSERT(erts_get_scheduler_id()); /* Need to be a scheduler */ + if (no == 0) + erts_schedule_misc_aux_work(erts_get_scheduler_id(), + set_node_not_alive, + NULL); +} + +static ERTS_INLINE void +inc_no_nodes(void) +{ +#ifdef DEBUG + erts_aint_t no = erts_smp_atomic_read_nob(&no_nodes); + ASSERT(erts_is_alive ? no > 0 : no == 0); +#endif + erts_smp_atomic_inc_mb(&no_nodes); +} /* * proc is currently running or exiting process. @@ -403,47 +450,76 @@ int erts_do_net_exits(DistEntry *dep, Eterm reason) Eterm nodename; if (dep == erts_this_dist_entry) { /* Net kernel has died (clean up!!) */ + DistEntry *tdep; + int no_dist_port = 0; Eterm nd_reason = (reason == am_no_network ? am_no_network : am_net_kernel_terminated); erts_smp_rwmtx_rwlock(&erts_dist_table_rwmtx); + for (tdep = erts_hidden_dist_entries; tdep; tdep = tdep->next) + no_dist_port++; + for (tdep = erts_visible_dist_entries; tdep; tdep = tdep->next) + no_dist_port++; + /* KILL all port controllers */ - while(erts_visible_dist_entries || erts_hidden_dist_entries) { - DistEntry *tdep; - Eterm prt_id; - Port *prt; - if(erts_hidden_dist_entries) - tdep = erts_hidden_dist_entries; - else - tdep = erts_visible_dist_entries; - prt_id = tdep->cid; - ASSERT(is_internal_port(prt_id)); + if (no_dist_port == 0) erts_smp_rwmtx_rwunlock(&erts_dist_table_rwmtx); + else { + Eterm def_buf[128]; + int i = 0; + Eterm *dist_port; - prt = erts_id2port(prt_id, NULL, 0); - if (prt) { - ASSERT(prt->status & ERTS_PORT_SFLG_DISTRIBUTION); - ASSERT(prt->dist_entry); - /* will call do_net_exists !!! */ - erts_do_exit_port(prt, prt_id, nd_reason); - erts_port_release(prt); + if (no_dist_port <= sizeof(def_buf)/sizeof(def_buf[0])) + dist_port = &def_buf[0]; + else + dist_port = erts_alloc(ERTS_ALC_T_TMP, + sizeof(Eterm)*no_dist_port); + for (tdep = erts_hidden_dist_entries; tdep; tdep = tdep->next) { + ASSERT(is_internal_port(tdep->cid)); + dist_port[i++] = tdep->cid; + } + for (tdep = erts_visible_dist_entries; tdep; tdep = tdep->next) { + ASSERT(is_internal_port(tdep->cid)); + dist_port[i++] = tdep->cid; } + erts_smp_rwmtx_rwunlock(&erts_dist_table_rwmtx); - erts_smp_rwmtx_rwlock(&erts_dist_table_rwmtx); - } + for (i = 0; i < no_dist_port; i++) { + Port *prt = erts_port_lookup(dist_port[i], + ERTS_PORT_SFLGS_INVALID_LOOKUP); + if (!prt) + continue; + ASSERT(erts_atomic32_read_nob(&prt->state) + & ERTS_PORT_SFLG_DISTRIBUTION); - erts_smp_rwmtx_rwunlock(&erts_dist_table_rwmtx); + erts_port_exit(NULL, ERTS_PORT_SIG_FLG_FORCE_SCHED, + prt, dist_port[i], nd_reason, NULL); + } - nodename = erts_this_dist_entry->sysname; - erts_smp_thr_progress_block(); - erts_set_this_node(am_Noname, 0); - erts_is_alive = 0; - send_nodes_mon_msgs(NULL, am_nodedown, nodename, am_visible, nd_reason); - erts_smp_thr_progress_unblock(); + if (dist_port != &def_buf[0]) + erts_free(ERTS_ALC_T_TMP, dist_port); + } + /* + * When last dist port exits, node will be taken + * from alive to not alive. + */ + ASSERT(is_nil(nodedown.reason) && !nodedown.bp); + if (is_immed(nd_reason)) + nodedown.reason = nd_reason; + else { + Eterm *hp; + Uint sz = size_object(nd_reason); + nodedown.bp = new_message_buffer(sz); + hp = nodedown.bp->mem; + nodedown.reason = copy_struct(nd_reason, + sz, + &hp, + &nodedown.bp->off_heap); + } } - else { /* recursive call via erts_do_exit_port() will end up here */ + else { /* Call from distribution port */ NetExitsContext nec = {dep}; ErtsLink *nlinks; ErtsLink *node_links; @@ -454,10 +530,10 @@ int erts_do_net_exits(DistEntry *dep, Eterm reason) erts_smp_de_rwlock(dep); ERTS_SMP_LC_ASSERT(is_internal_port(dep->cid) - && erts_lc_is_port_locked(&erts_port[internal_port_index(dep->cid)])); + && erts_lc_is_port_locked(erts_port_lookup_raw(dep->cid))); if (erts_port_task_is_scheduled(&dep->dist_cmd)) - erts_port_task_abort(dep->cid, &dep->dist_cmd); + erts_port_task_abort(&dep->dist_cmd); if (dep->status & ERTS_DE_SFLG_EXITING) { #ifdef DEBUG @@ -503,6 +579,9 @@ int erts_do_net_exits(DistEntry *dep, Eterm reason) clear_dist_entry(dep); } + + dec_no_nodes(); + return 1; } @@ -516,6 +595,10 @@ void init_dist(void) { init_nodes_monitors(); + nodedown.reason = NIL; + nodedown.bp = NULL; + + erts_smp_atomic_init_nob(&no_nodes, 0); erts_smp_atomic_init_nob(&no_caches, 0); /* Lookup/Install all references to trap functions */ @@ -769,7 +852,7 @@ erts_dsig_send_msg(ErtsDSigData *dsdp, Eterm remote, Eterm message) *node_name = *sender_name = *receiver_name = '\0'; if (DTRACE_ENABLED(message_send) || DTRACE_ENABLED(message_send_remote)) { erts_snprintf(node_name, sizeof(node_name), "%T", dsdp->dep->sysname); - erts_snprintf(sender_name, sizeof(sender_name), "%T", sender->id); + erts_snprintf(sender_name, sizeof(sender_name), "%T", sender->common.id); erts_snprintf(receiver_name, sizeof(receiver_name), "%T", remote); msize = size_object(message); if (token != NIL && token != am_have_dt_utag) { @@ -826,7 +909,7 @@ erts_dsig_send_reg_msg(ErtsDSigData *dsdp, Eterm remote_name, Eterm message) *node_name = *sender_name = *receiver_name = '\0'; if (DTRACE_ENABLED(message_send) || DTRACE_ENABLED(message_send_remote)) { erts_snprintf(node_name, sizeof(node_name), "%T", dsdp->dep->sysname); - erts_snprintf(sender_name, sizeof(sender_name), "%T", sender->id); + erts_snprintf(sender_name, sizeof(sender_name), "%T", sender->common.id); erts_snprintf(receiver_name, sizeof(receiver_name), "{%T,%s}", remote_name, node_name); msize = size_object(message); @@ -840,10 +923,10 @@ erts_dsig_send_reg_msg(ErtsDSigData *dsdp, Eterm remote_name, Eterm message) if (token != NIL) ctl = TUPLE5(&ctl_heap[0], make_small(DOP_REG_SEND_TT), - sender->id, am_Cookie, remote_name, token); + sender->common.id, am_Cookie, remote_name, token); else ctl = TUPLE4(&ctl_heap[0], make_small(DOP_REG_SEND), - sender->id, am_Cookie, remote_name); + sender->common.id, am_Cookie, remote_name); DTRACE6(message_send, sender_name, receiver_name, msize, tok_label, tok_lastcnt, tok_serial); DTRACE7(message_send_remote, sender_name, node_name, receiver_name, @@ -889,7 +972,7 @@ erts_dsig_send_exit_tt(ErtsDSigData *dsdp, Eterm local, Eterm remote, *node_name = *sender_name = *remote_name = '\0'; if (DTRACE_ENABLED(process_exit_signal_remote)) { erts_snprintf(node_name, sizeof(node_name), "%T", dsdp->dep->sysname); - erts_snprintf(sender_name, sizeof(sender_name), "%T", sender->id); + erts_snprintf(sender_name, sizeof(sender_name), "%T", sender->common.id); erts_snprintf(remote_name, sizeof(remote_name), "{%T,%s}", remote, node_name); erts_snprintf(reason_str, sizeof(reason), "%T", reason); @@ -1141,7 +1224,7 @@ int erts_net_message(Port *prt, } erts_smp_de_links_lock(dep); - res = erts_add_link(&(rp->nlinks), LINK_PID, from); + res = erts_add_link(&ERTS_P_LINKS(rp), LINK_PID, from); if (res < 0) { /* It was already there! Lets skip the rest... */ @@ -1149,7 +1232,7 @@ int erts_net_message(Port *prt, erts_smp_proc_unlock(rp, ERTS_PROC_LOCK_LINK); break; } - lnk = erts_add_or_lookup_link(&(dep->nlinks), LINK_PID, rp->id); + lnk = erts_add_or_lookup_link(&(dep->nlinks), LINK_PID, rp->common.id); erts_add_link(&(ERTS_LINK_ROOT(lnk)), LINK_PID, from); erts_smp_de_links_unlock(dep); @@ -1176,7 +1259,7 @@ int erts_net_message(Port *prt, if (!rp) break; - lnk = erts_remove_link(&(rp->nlinks), from); + lnk = erts_remove_link(&ERTS_P_LINKS(rp), from); if (IS_TRACED_FL(rp, F_TRACE_PROCS) && lnk != NULL) { trace_proc(NULL, rp, am_getting_unlinked, from); @@ -1233,10 +1316,10 @@ int erts_net_message(Port *prt, } else { if (is_atom(watched)) - watched = rp->id; + watched = rp->common.id; erts_smp_de_links_lock(dep); erts_add_monitor(&(dep->monitors), MON_ORIGIN, ref, watched, name); - erts_add_monitor(&(rp->monitors), MON_TARGET, ref, watcher, name); + erts_add_monitor(&ERTS_P_MONITORS(rp), MON_TARGET, ref, watcher, name); erts_smp_de_links_unlock(dep); erts_smp_proc_unlock(rp, ERTS_PROC_LOCK_LINK); } @@ -1275,7 +1358,7 @@ int erts_net_message(Port *prt, if (!rp) { break; } - mon = erts_remove_monitor(&(rp->monitors),ref); + mon = erts_remove_monitor(&ERTS_P_MONITORS(rp), ref); erts_smp_proc_unlock(rp, ERTS_PROC_LOCK_LINK); ASSERT(mon != NULL); if (mon == NULL) { @@ -1432,7 +1515,7 @@ int erts_net_message(Port *prt, erts_destroy_monitor(mon); - mon = erts_remove_monitor(&(rp->monitors),ref); + mon = erts_remove_monitor(&ERTS_P_MONITORS(rp), ref); if (mon == NULL) { erts_smp_proc_unlock(rp, rp_locks); @@ -1483,7 +1566,7 @@ int erts_net_message(Port *prt, if (!rp) lnk = NULL; else { - lnk = erts_remove_link(&(rp->nlinks), from); + lnk = erts_remove_link(&ERTS_P_LINKS(rp), from); /* If lnk == NULL, we have unlinked on this side, i.e. * ignore exit. @@ -1597,7 +1680,7 @@ int erts_net_message(Port *prt, erts_free(ERTS_ALC_T_DCTRL_BUF, (void *) ctl); } UnUseTmpHeapNoproc(DIST_CTL_DEFAULT_SIZE); - erts_do_exit_port(prt, dep->cid, am_killed); + erts_deliver_port_exit(prt, dep->cid, am_killed, 0); ERTS_SMP_CHK_NO_PROC_LOCKS; return -1; } @@ -1693,7 +1776,6 @@ dsig_send(ErtsDSigData *dsdp, Eterm ctl, Eterm msg, int force_busy) erts_smp_mtx_unlock(&dep->qlock); plp = erts_proclist_create(c_p); - plp->next = NULL; erts_suspend(c_p, ERTS_PROC_LOCK_MAIN, NULL); suspended = 1; erts_smp_mtx_lock(&dep->qlock); @@ -1726,11 +1808,7 @@ dsig_send(ErtsDSigData *dsdp, Eterm ctl, Eterm msg, int force_busy) else { /* Enqueue suspended process on dist entry */ ASSERT(plp); - if (dep->suspended.last) - dep->suspended.last->next = plp; - else - dep->suspended.first = plp; - dep->suspended.last = plp; + erts_proclist_store_last(&dep->suspended, plp); } } @@ -1779,7 +1857,7 @@ dsig_send(ErtsDSigData *dsdp, Eterm ctl, Eterm msg, int force_busy) erts_snprintf(port_str, sizeof(port_str), "%T", cid); erts_snprintf(remote_str, sizeof(remote_str), "%T", dep->sysname); - erts_snprintf(pid_str, sizeof(pid_str), "%T", c_p->id); + erts_snprintf(pid_str, sizeof(pid_str), "%T", c_p->common.id); DTRACE4(dist_port_busy, erts_this_node_sysname, port_str, remote_str, pid_str); } @@ -1812,7 +1890,7 @@ dist_port_command(Port *prt, ErtsDistOutputBuf *obuf) DTRACE_CHARBUF(port_str, 64); DTRACE_CHARBUF(remote_str, 64); - erts_snprintf(port_str, sizeof(port_str), "%T", prt->id); + erts_snprintf(port_str, sizeof(port_str), "%T", prt->common.id); erts_snprintf(remote_str, sizeof(remote_str), "%T", prt->dist_entry->sysname); DTRACE4(dist_output, erts_this_node_sysname, port_str, @@ -1866,7 +1944,7 @@ dist_port_commandv(Port *prt, ErtsDistOutputBuf *obuf) DTRACE_CHARBUF(port_str, 64); DTRACE_CHARBUF(remote_str, 64); - erts_snprintf(port_str, sizeof(port_str), "%T", prt->id); + erts_snprintf(port_str, sizeof(port_str), "%T", prt->common.id); erts_snprintf(remote_str, sizeof(remote_str), "%T", prt->dist_entry->sysname); DTRACE4(dist_outputv, erts_this_node_sysname, port_str, @@ -1903,13 +1981,13 @@ int erts_dist_command(Port *prt, int reds_limit) { Sint reds = ERTS_PORT_REDS_DIST_CMD_START; - int prt_busy; Uint32 status; Uint32 flags; Sint obufsize = 0; ErtsDistOutputQueue oq, foq; DistEntry *dep = prt->dist_entry; Uint (*send)(Port *prt, ErtsDistOutputBuf *obuf); + erts_aint32_t sched_flags; ERTS_SMP_LC_ASSERT(erts_lc_is_port_locked(prt)); @@ -1925,7 +2003,7 @@ erts_dist_command(Port *prt, int reds_limit) erts_smp_de_runlock(dep); if (status & ERTS_DE_SFLG_EXITING) { - erts_do_exit_port(prt, prt->id, am_killed); + erts_deliver_port_exit(prt, prt->common.id, am_killed, 0); erts_deref_dist_entry(dep); return reds + ERTS_PORT_REDS_DIST_CMD_EXIT; } @@ -1952,12 +2030,12 @@ erts_dist_command(Port *prt, int reds_limit) dep->finalized_out_queue.first = NULL; dep->finalized_out_queue.last = NULL; + sched_flags = erts_smp_atomic32_read_nob(&prt->sched.flags); + if (reds > reds_limit) goto preempted; - prt_busy = (int) (prt->status & ERTS_PORT_SFLG_PORT_BUSY); - - if (!prt_busy && foq.first) { + if (!(sched_flags & ERTS_PTS_FLG_BUSY_PORT) && foq.first) { int preempt = 0; do { Uint size; @@ -1974,11 +2052,10 @@ erts_dist_command(Port *prt, int reds_limit) obufsize += size_obuf(fob); foq.first = foq.first->next; free_dist_obuf(fob); - preempt = reds > reds_limit || (prt->status & ERTS_PORT_SFLGS_DEAD); - if (prt->status & ERTS_PORT_SFLG_PORT_BUSY) { - prt_busy = 1; + sched_flags = erts_smp_atomic32_read_nob(&prt->sched.flags); + preempt = reds > reds_limit || (sched_flags & ERTS_PTS_FLG_EXIT); + if (sched_flags & ERTS_PTS_FLG_BUSY_PORT) break; - } } while (foq.first && !preempt); if (!foq.first) foq.last = NULL; @@ -1986,7 +2063,7 @@ erts_dist_command(Port *prt, int reds_limit) goto preempted; } - if (prt_busy) { + if (sched_flags & ERTS_PTS_FLG_BUSY_PORT) { if (oq.first) { ErtsDistOutputBuf *ob; int preempt; @@ -2058,12 +2135,10 @@ erts_dist_command(Port *prt, int reds_limit) obufsize += size_obuf(fob); oq.first = oq.first->next; free_dist_obuf(fob); - preempt = reds > reds_limit || (prt->status & ERTS_PORT_SFLGS_DEAD); - if (prt->status & ERTS_PORT_SFLG_PORT_BUSY) { - prt_busy = 1; - if (oq.first && !preempt) - goto finalize_only; - } + sched_flags = erts_smp_atomic32_read_nob(&prt->sched.flags); + preempt = reds > reds_limit || (sched_flags & ERTS_PTS_FLG_EXIT); + if ((sched_flags & ERTS_PTS_FLG_BUSY_PORT) && oq.first && !preempt) + goto finalize_only; } ASSERT(!oq.first || preempt); @@ -2091,7 +2166,7 @@ erts_dist_command(Port *prt, int reds_limit) ASSERT(dep->qsize >= obufsize); dep->qsize -= obufsize; obufsize = 0; - if (!prt_busy + if (!(sched_flags & ERTS_PTS_FLG_BUSY_PORT) && (dep->qflgs & ERTS_DE_QFLG_BUSY) && dep->qsize < erts_dist_buf_busy_limit) { ErtsProcList *suspendees; @@ -2137,11 +2212,15 @@ erts_dist_command(Port *prt, int reds_limit) return reds; preempted: + /* + * Here we assume that state has been read + * since last call to driver. + */ ASSERT(oq.first || !oq.last); ASSERT(!oq.first || oq.last); - if (prt->status & ERTS_PORT_SFLGS_DEAD) { + if (sched_flags & ERTS_PTS_FLG_EXIT) { /* * Port died during port command; clean up 'oq' * and 'foq'. Things buffered in dist entry after @@ -2199,7 +2278,7 @@ erts_dist_port_not_busy(Port *prt) DTRACE_CHARBUF(port_str, 64); DTRACE_CHARBUF(remote_str, 64); - erts_snprintf(port_str, sizeof(port_str), "%T", prt->id); + erts_snprintf(port_str, sizeof(port_str), "%T", prt->common.id); erts_snprintf(remote_str, sizeof(remote_str), "%T", prt->dist_entry->sysname); DTRACE3(dist_port_not_busy, erts_this_node_sysname, @@ -2241,7 +2320,7 @@ static void doit_print_monitor_info(ErtsMonitor *mon, void *vptdp) Process *rp; ErtsMonitor *rmon; rp = erts_proc_lookup(mon->pid); - if (!rp || (rmon = erts_lookup_monitor(rp->monitors, mon->ref)) == NULL) { + if (!rp || (rmon = erts_lookup_monitor(ERTS_P_MONITORS(rp), mon->ref)) == NULL) { erts_print(to, arg, "Warning, stray monitor for: %T\n", mon->pid); } else if (mon->type == MON_ORIGIN) { /* Local pid is being monitored */ @@ -2486,6 +2565,7 @@ BIF_RETTYPE setnode_2(BIF_ALIST_2) erts_smp_proc_unlock(BIF_P, ERTS_PROC_LOCK_MAIN); erts_smp_thr_progress_block(); + inc_no_nodes(); erts_set_this_node(BIF_ARG_1, (Uint32) creation); erts_is_alive = 1; send_nodes_mon_msgs(NULL, am_nodeup, BIF_ARG_1, am_visible, NIL); @@ -2554,9 +2634,9 @@ BIF_RETTYPE setnode_3(BIF_ALIST_3) /* DFLAG_EXTENDED_REFERENCES is compulsory from R9 and forward */ if (!(DFLAG_EXTENDED_REFERENCES & flags)) { erts_dsprintf_buf_t *dsbufp = erts_create_logger_dsbuf(); - erts_dsprintf(dsbufp, "%T", BIF_P->id); - if (BIF_P->reg) - erts_dsprintf(dsbufp, " (%T)", BIF_P->reg->name); + erts_dsprintf(dsbufp, "%T", BIF_P->common.id); + if (BIF_P->common.u.alive.reg) + erts_dsprintf(dsbufp, " (%T)", BIF_P->common.u.alive.reg->name); erts_dsprintf(dsbufp, " attempted to enable connection to node %T " "which is not able to handle extended references.\n", @@ -2576,10 +2656,14 @@ BIF_RETTYPE setnode_3(BIF_ALIST_3) else if (!dep) goto system_limit; /* Should never happen!!! */ - pp = erts_id2port(BIF_ARG_2, BIF_P, ERTS_PROC_LOCK_MAIN); + pp = erts_id2port_sflgs(BIF_ARG_2, + BIF_P, + ERTS_PROC_LOCK_MAIN, + ERTS_PORT_SFLGS_INVALID_LOOKUP); erts_smp_de_rwlock(dep); - if (!pp || (pp->status & ERTS_PORT_SFLG_EXITING)) + if (!pp || (erts_atomic32_read_nob(&pp->state) + & ERTS_PORT_SFLG_EXITING)) goto badarg; if ((pp->drv_ptr->flags & ERL_DRV_FLAG_SOFT_BUSY) == 0) @@ -2594,11 +2678,7 @@ BIF_RETTYPE setnode_3(BIF_ALIST_3) plp->next = NULL; erts_suspend(BIF_P, ERTS_PROC_LOCK_MAIN, NULL); erts_smp_mtx_lock(&dep->qlock); - if (dep->suspended.last) - dep->suspended.last->next = plp; - else - dep->suspended.first = plp; - dep->suspended.last = plp; + erts_proclist_store_last(&dep->suspended, plp); erts_smp_mtx_unlock(&dep->qlock); goto yield; } @@ -2608,7 +2688,16 @@ BIF_RETTYPE setnode_3(BIF_ALIST_3) if (pp->dist_entry || is_not_nil(dep->cid)) goto badarg; - erts_port_status_bor_set(pp, ERTS_PORT_SFLG_DISTRIBUTION); + erts_atomic32_read_bor_nob(&pp->state, ERTS_PORT_SFLG_DISTRIBUTION); + + /* + * Dist-ports do not use the "busy port message queue" functionality, but + * instead use "busy dist entry" functionality. + */ + { + ErlDrvSizeT disable = ERL_DRV_BUSY_MSGQ_DISABLED; + erl_drv_busy_msgq_limits((ErlDrvPort) pp, &disable, NULL); + } pp->dist_entry = dep; @@ -2640,6 +2729,8 @@ BIF_RETTYPE setnode_3(BIF_ALIST_3) erts_smp_de_rwunlock(dep); dep = NULL; /* inc of refc transferred to port (dist_entry field) */ + inc_no_nodes(); + send_nodes_mon_msgs(BIF_P, am_nodeup, BIF_ARG_1, @@ -2653,7 +2744,7 @@ BIF_RETTYPE setnode_3(BIF_ALIST_3) } if (pp) - erts_smp_port_unlock(pp); + erts_port_release(pp); return ret; @@ -2697,7 +2788,7 @@ BIF_RETTYPE dist_exit_3(BIF_ALIST_3) if (is_internal_pid(local)) { Process *lp; ErtsProcLocks lp_locks; - if (BIF_P->id == local) { + if (BIF_P->common.id == local) { lp_locks = ERTS_PROC_LOCKS_ALL; lp = BIF_P; erts_smp_proc_lock(BIF_P, ERTS_PROC_LOCKS_ALL_MINOR); @@ -2725,11 +2816,17 @@ BIF_RETTYPE dist_exit_3(BIF_ALIST_3) #endif erts_smp_proc_unlock(lp, lp_locks); if (lp == BIF_P) { + erts_aint32_t state = erts_smp_atomic32_read_acqb(&BIF_P->state); /* * We may have exited current process and may have to take action. */ - ERTS_BIF_CHK_EXITED(BIF_P); - ERTS_SMP_BIF_CHK_PENDING_EXIT(BIF_P, ERTS_PROC_LOCK_MAIN); + if (state & (ERTS_PSFLG_EXITING|ERTS_PSFLG_PENDING_EXIT)) { +#ifdef ERTS_SMP + if (state & ERTS_PSFLG_PENDING_EXIT) + erts_handle_pending_exit(BIF_P, ERTS_PROC_LOCK_MAIN); +#endif + ERTS_BIF_EXITED(BIF_P); + } } } else if (is_external_pid(local) @@ -2927,23 +3024,23 @@ monitor_node(Process* p, Eterm Node, Eterm Bool, Eterm Options) if (Bool == am_true) { ASSERT(dep->cid != NIL); lnk = erts_add_or_lookup_link(&(dep->node_links), LINK_NODE, - p->id); + p->common.id); ++ERTS_LINK_REFC(lnk); - lnk = erts_add_or_lookup_link(&(p->nlinks), LINK_NODE, Node); + lnk = erts_add_or_lookup_link(&ERTS_P_LINKS(p), LINK_NODE, Node); ++ERTS_LINK_REFC(lnk); } else { - lnk = erts_lookup_link(dep->node_links, p->id); + lnk = erts_lookup_link(dep->node_links, p->common.id); if (lnk != NULL) { if ((--ERTS_LINK_REFC(lnk)) == 0) { erts_destroy_link(erts_remove_link(&(dep->node_links), - p->id)); + p->common.id)); } } - lnk = erts_lookup_link(p->nlinks, Node); + lnk = erts_lookup_link(ERTS_P_LINKS(p), Node); if (lnk != NULL) { if ((--ERTS_LINK_REFC(lnk)) == 0) { - erts_destroy_link(erts_remove_link(&(p->nlinks), + erts_destroy_link(erts_remove_link(&ERTS_P_LINKS(p), Node)); } } @@ -3505,7 +3602,7 @@ erts_processes_monitoring_nodes(Process *c_p) olist = erts_bld_cons(hpp, szp, am_nodedown_reason, olist); res = erts_bld_cons(hpp, szp, erts_bld_tuple(hpp, szp, 2, - nmp->proc->id, + nmp->proc->common.id, olist), res); } diff --git a/erts/emulator/beam/dist.h b/erts/emulator/beam/dist.h index 845151c895..2bc3d9c881 100644 --- a/erts/emulator/beam/dist.h +++ b/erts/emulator/beam/dist.h @@ -187,11 +187,12 @@ void erts_schedule_dist_command(Port *prt, DistEntry *dist_entry) if (prt) { ERTS_SMP_LC_ASSERT(erts_lc_is_port_locked(prt)); - ASSERT((erts_port_status_get(prt) & ERTS_PORT_SFLGS_DEAD) == 0); + ASSERT((erts_atomic32_read_nob(&prt->state) + & ERTS_PORT_SFLGS_DEAD) == 0); ASSERT(prt->dist_entry); dep = prt->dist_entry; - id = prt->id; + id = prt->common.id; } else { ASSERT(dist_entry); @@ -203,13 +204,8 @@ void erts_schedule_dist_command(Port *prt, DistEntry *dist_entry) id = dep->cid; } - if (!erts_smp_atomic_xchg_mb(&dep->dist_cmd_scheduled, 1)) { - (void) erts_port_task_schedule(id, - &dep->dist_cmd, - ERTS_PORT_TASK_DIST_CMD, - (ErlDrvEvent) -1, - NULL); - } + if (!erts_smp_atomic_xchg_mb(&dep->dist_cmd_scheduled, 1)) + erts_port_task_schedule(id, &dep->dist_cmd, ERTS_PORT_TASK_DIST_CMD); } #endif diff --git a/erts/emulator/beam/erl_afit_alloc.c b/erts/emulator/beam/erl_afit_alloc.c index 570cc59be2..306b32691c 100644 --- a/erts/emulator/beam/erl_afit_alloc.c +++ b/erts/emulator/beam/erl_afit_alloc.c @@ -37,6 +37,12 @@ #define GET_ERL_AF_ALLOC_IMPL #include "erl_afit_alloc.h" +struct AFFreeBlock_t_ { + Block_t block_head; + AFFreeBlock_t *prev; + AFFreeBlock_t *next; +}; +#define AF_BLK_SZ(B) MBC_FBLK_SZ(&(B)->block_head) #define MIN_MBC_SZ (16*1024) #define MIN_MBC_FIRST_FREE_SZ (4*1024) @@ -80,7 +86,6 @@ erts_afalc_start(AFAllctr_t *afallctr, init->sbmbct = 0; /* Small mbc not supported by afit */ - allctr->mbc_header_size = sizeof(Carrier_t); allctr->min_mbc_size = MIN_MBC_SZ; allctr->min_mbc_first_free_size = MIN_MBC_FIRST_FREE_SZ; allctr->min_block_size = sizeof(AFFreeBlock_t); @@ -118,7 +123,7 @@ get_free_block(Allctr_t *allctr, Uint size, Block_t *cand_blk, Uint cand_size, ASSERT(!cand_blk || cand_size >= size); - if (afallctr->free_list && BLK_SZ(afallctr->free_list) >= size) { + if (afallctr->free_list && AF_BLK_SZ(afallctr->free_list) >= size) { AFFreeBlock_t *res = afallctr->free_list; afallctr->free_list = res->next; if (res->next) @@ -135,7 +140,7 @@ link_free_block(Allctr_t *allctr, Block_t *block, Uint32 flags) AFFreeBlock_t *blk = (AFFreeBlock_t *) block; AFAllctr_t *afallctr = (AFAllctr_t *) allctr; - if (afallctr->free_list && BLK_SZ(afallctr->free_list) > BLK_SZ(blk)) { + if (afallctr->free_list && AF_BLK_SZ(afallctr->free_list) > AF_BLK_SZ(blk)) { blk->next = afallctr->free_list->next; blk->prev = afallctr->free_list; afallctr->free_list->next = blk; diff --git a/erts/emulator/beam/erl_afit_alloc.h b/erts/emulator/beam/erl_afit_alloc.h index ea408a7194..87caccac20 100644 --- a/erts/emulator/beam/erl_afit_alloc.h +++ b/erts/emulator/beam/erl_afit_alloc.h @@ -49,11 +49,6 @@ Allctr_t *erts_afalc_start(AFAllctr_t *, AFAllctrInit_t *, AllctrInit_t *); #include "erl_alloc_util.h" typedef struct AFFreeBlock_t_ AFFreeBlock_t; -struct AFFreeBlock_t_ { - Block_t block_head; - AFFreeBlock_t *prev; - AFFreeBlock_t *next; -}; struct AFAllctr_t_ { Allctr_t allctr; /* Has to be first! */ diff --git a/erts/emulator/beam/erl_alloc.c b/erts/emulator/beam/erl_alloc.c index 3eee53eba3..5da8c69c03 100644 --- a/erts/emulator/beam/erl_alloc.c +++ b/erts/emulator/beam/erl_alloc.c @@ -310,9 +310,9 @@ set_default_ll_alloc_opts(struct au_init *ip) ip->init.util.name_prefix = "ll_"; ip->init.util.alloc_no = ERTS_ALC_A_LONG_LIVED; #ifndef SMALL_MEMORY - ip->init.util.mmbcs = 2*1024*1024; /* Main carrier size */ + ip->init.util.mmbcs = 2*1024*1024 - 40; /* Main carrier size */ #else - ip->init.util.mmbcs = 1*1024*1024; /* Main carrier size */ + ip->init.util.mmbcs = 1*1024*1024 - 40; /* Main carrier size */ #endif ip->init.util.ts = ERTS_ALC_MTA_LONG_LIVED; ip->init.util.asbcst = 0; @@ -1173,6 +1173,11 @@ handle_au_arg(struct au_init *auip, break; case 'e': auip->enable = get_bool_value(sub_param+1, argv, ip); +#if !HAVE_ERTS_SBMBC + if (auip->init.util.alloc_no == ERTS_ALC_A_SBMBC) { + auip->enable = 0; + } +#endif break; case 'l': if (has_prefix("lmbcs", sub_param)) { @@ -1233,10 +1238,16 @@ handle_au_arg(struct au_init *auip, auip->init.util.sbct = get_kb_value(sub_param + 4, argv, ip); } else if (has_prefix("sbmbcs", sub_param)) { - auip->init.util.sbmbcs = get_byte_value(sub_param + 6, argv, ip); +#if HAVE_ERTS_SBMBC + auip->init.util.sbmbcs = +#endif + get_byte_value(sub_param + 6, argv, ip); } else if (has_prefix("sbmbct", sub_param)) { - auip->init.util.sbmbct = get_byte_value(sub_param + 6, argv, ip); +#if HAVE_ERTS_SBMBC + auip->init.util.sbmbct = +#endif + get_byte_value(sub_param + 6, argv, ip); } else if (has_prefix("smbcs", sub_param)) { auip->default_.smbcs = 0; @@ -1403,6 +1414,9 @@ handle_args(int *argc, char **argv, erts_alc_hndl_args_init_t *init) else if (strcmp("max", arg) == 0) { for (a = 0; a < aui_sz; a++) aui[a]->enable = 1; +#if !HAVE_ERTS_SBMBC + init->sbmbc_alloc.enable = 0; +#endif } else if (strcmp("config", arg) == 0) { init->erts_alloc_config = 1; @@ -2128,6 +2142,7 @@ erts_memory(int *print_to_p, void *print_to_arg, void *proc, Eterm earg) if (want_tot_or_sys || want.processes || want.processes_used) { + int max_processes = erts_ptab_max(&erts_proc); UWord tmp; if (ERTS_MEM_NEED_ALL_ALCU) @@ -2137,7 +2152,7 @@ erts_memory(int *print_to_p, void *print_to_arg, void *proc, Eterm earg) fi, ERTS_ALC_NO_FIXED_SIZES); tmp = alcu_size(ERTS_ALC_A_EHEAP, NULL, 0); } - tmp += erts_max_processes*sizeof(Process*); + tmp += max_processes*sizeof(erts_smp_atomic_t); tmp += erts_bif_timer_memory_size(); tmp += erts_tot_link_lh_size(); @@ -2268,6 +2283,8 @@ erts_allocated_areas(int *print_to_p, void *print_to_arg, void *proc) Eterm res = THE_NON_VALUE; int i, length; Uint reserved_atom_space, atom_space; + int max_processes = erts_ptab_max(&erts_proc); + int max_ports = erts_ptab_max(&erts_port); if (proc) { ERTS_SMP_LC_ASSERT(ERTS_PROC_LOCK_MAIN @@ -2299,7 +2316,7 @@ erts_allocated_areas(int *print_to_p, void *print_to_arg, void *proc) values[i].arity = 2; values[i].name = "static"; values[i].ui[0] = - erts_max_ports*sizeof(Port) /* Port table */ + max_ports*sizeof(erts_smp_atomic_t) /* Port table */ + erts_timer_wheel_memory_size(); /* Timer wheel */ i++; @@ -2378,7 +2395,7 @@ erts_allocated_areas(int *print_to_p, void *print_to_arg, void *proc) values[i].arity = 2; values[i].name = "process_table"; - values[i].ui[0] = erts_max_processes*sizeof(Process*); + values[i].ui[0] = max_processes*sizeof(Process*); i++; values[i].arity = 2; diff --git a/erts/emulator/beam/erl_alloc.h b/erts/emulator/beam/erl_alloc.h index e475f9d8a2..ba5ec9c367 100644 --- a/erts/emulator/beam/erl_alloc.h +++ b/erts/emulator/beam/erl_alloc.h @@ -267,6 +267,8 @@ typedef void (*erts_alloc_verify_func_t)(Allctr_t *); erts_alloc_verify_func_t erts_alloc_get_verify_unused_temp_alloc(Allctr_t **allctr); +#define ERTS_ALC_DATA_ALIGN_SIZE(SZ) \ + (((((SZ) - 1) / 8) + 1) * 8) #define ERTS_ALC_CACHE_LINE_ALIGN_SIZE(SZ) \ (((((SZ) - 1) / ERTS_CACHE_LINE_SIZE) + 1) * ERTS_CACHE_LINE_SIZE) diff --git a/erts/emulator/beam/erl_alloc.types b/erts/emulator/beam/erl_alloc.types index 0a4407f009..d4de0d076a 100644 --- a/erts/emulator/beam/erl_alloc.types +++ b/erts/emulator/beam/erl_alloc.types @@ -146,6 +146,7 @@ class SYSTEM system_data type SBMBC SBMBC SYSTEM small_block_mbc type PROC FIXED_SIZE PROCESSES proc +type PORT DRIVER SYSTEM port type ATOM LONG_LIVED ATOM atom_entry type MODULE LONG_LIVED CODE module_entry type REG_PROC STANDARD PROCESSES reg_proc @@ -189,7 +190,10 @@ type PORT_TABLE LONG_LIVED SYSTEM port_tab type TIMER_WHEEL LONG_LIVED SYSTEM timer_wheel type DRV DRIVER SYSTEM drv_internal type DRV_BINARY BINARY BINARIES drv_binary -type DRIVER STANDARD SYSTEM driver +type DRIVER DRIVER SYSTEM driver +type DRV_CMD_DATA DRIVER SYSTEM driver_command_data +type DRV_CTRL_DATA DRIVER SYSTEM driver_control_data +type DRV_CALL_DATA DRIVER SYSTEM driver_call_data type NIF DRIVER SYSTEM nif_internal type BINARY BINARY BINARIES binary type NBIF_TABLE SYSTEM SYSTEM nbif_tab @@ -197,14 +201,12 @@ type ARG_REG STANDARD PROCESSES arg_reg type PROC_DICT STANDARD PROCESSES proc_dict type CALLS_BUF STANDARD PROCESSES calls_buf type BPD STANDARD SYSTEM bpd -type PORT_NAME STANDARD SYSTEM port_name type LINEBUF STANDARD SYSTEM line_buf type IOQ STANDARD SYSTEM io_queue type BITS_BUF STANDARD SYSTEM bits_buf type TMP_DIST_BUF TEMPORARY SYSTEM tmp_dist_buf type ASYNC_DATA LONG_LIVED SYSTEM internal_async_data type ESTACK TEMPORARY SYSTEM estack -type PORT_CALL_BUF TEMPORARY SYSTEM port_call_buf type DB_TABLE ETS ETS db_tab type DB_FIXATION SHORT_LIVED ETS db_fixation type DB_FIX_DEL SHORT_LIVED ETS fixed_del @@ -234,14 +236,14 @@ type DDLL_HANDLE STANDARD SYSTEM ddll_handle type DDLL_ERRCODES LONG_LIVED SYSTEM ddll_errcodes type DDLL_TMP_BUF TEMPORARY SYSTEM ddll_tmp_buf type PORT_TASK SHORT_LIVED SYSTEM port_task -type PORT_TASKQ SHORT_LIVED SYSTEM port_task_queue +type PT_HNDL_LIST SHORT_LIVED SYSTEM port_task_handle_list type MISC_OP_LIST SHORT_LIVED SYSTEM misc_op_list type PORT_NAMES SHORT_LIVED SYSTEM port_names -type PORT_DATA_LOCK STANDARD SYSTEM port_data_lock +type PORT_DATA_LOCK DRIVER SYSTEM port_data_lock type NODES_MON STANDARD PROCESSES nodes_monitor -type PROCS_TPROC_EL SHORT_LIVED PROCESSES processes_term_proc_el -type PROCS_CNKINF SHORT_LIVED PROCESSES processes_chunk_info -type PROCS_PIDS SHORT_LIVED PROCESSES processes_pids +type PTAB_LIST_DEL SHORT_LIVED PROCESSES ptab_list_deleted_el +type PTAB_LIST_CNKI SHORT_LIVED PROCESSES ptab_list_chunk_info +type PTAB_LIST_PIDS SHORT_LIVED PROCESSES ptab_list_pids type RE_TMP_BUF TEMPORARY SYSTEM re_tmp_buf type RE_SUBJECT SHORT_LIVED SYSTEM re_subject type RE_HEAP STANDARD SYSTEM re_heap @@ -266,6 +268,8 @@ type AUX_WORK_TMO LONG_LIVED SYSTEM aux_work_timeouts type MISC_AUX_WORK_Q LONG_LIVED SYSTEM misc_aux_work_q type CODE_IX_LOCK_Q SHORT_LIVED SYSTEM code_ix_lock_q type PROC_INTERVAL LONG_LIVED SYSTEM process_interval +type BUSY_CALLER_TAB SHORT_LIVED SYSTEM busy_caller_table +type BUSY_CALLER SHORT_LIVED SYSTEM busy_caller +if threads_no_smp # Need thread safe allocs, but std_alloc and fix_alloc are not; diff --git a/erts/emulator/beam/erl_alloc_util.c b/erts/emulator/beam/erl_alloc_util.c index 97ba306a79..3d6761339b 100644 --- a/erts/emulator/beam/erl_alloc_util.c +++ b/erts/emulator/beam/erl_alloc_util.c @@ -78,10 +78,12 @@ int erts_have_sbmbc_alloc; #if HAVE_ERTS_MSEG -#define INV_MSEG_UNIT_MASK ((UWord) (mseg_unit_size - 1)) -#define MSEG_UNIT_MASK (~INV_MSEG_UNIT_MASK) +#define MSEG_UNIT_SHIFT MSEG_ALIGN_BITS +#define MSEG_UNIT_SZ (1 << MSEG_UNIT_SHIFT) +#define MSEG_UNIT_MASK ((~(UWord)0) << MSEG_UNIT_SHIFT) + #define MSEG_UNIT_FLOOR(X) ((X) & MSEG_UNIT_MASK) -#define MSEG_UNIT_CEILING(X) MSEG_UNIT_FLOOR((X) + INV_MSEG_UNIT_MASK) +#define MSEG_UNIT_CEILING(X) MSEG_UNIT_FLOOR((X) + ~MSEG_UNIT_MASK) #endif @@ -104,7 +106,6 @@ int erts_have_sbmbc_alloc; static Uint sys_alloc_carrier_size; #if HAVE_ERTS_MSEG static Uint max_mseg_carriers; -static Uint mseg_unit_size; #endif #define ONE_GIGA (1000000000) @@ -117,16 +118,47 @@ static Uint mseg_unit_size; ? ((CC).giga_no--, (CC).no = ONE_GIGA - 1) \ : (CC).no--) -/* ... */ +/* Multi block carrier (MBC) memory layout in R16: + +Empty MBC: +[Carrier_t|pad|Block_t L0T|fhdr| free... ] + +MBC after allocating first block: +[Carrier_t|pad|Block_t 000| udata |pad|Block_t L0T|fhdr| free... ] + +MBC after allocating second block: +[Carrier_t|pad|Block_t 000| udata |pad|Block_t 000| udata |pad|Block_t L0T|fhdr| free... ] + +MBC after deallocating first block: +[Carrier_t|pad|Block_t 00T|fhdr| free |FreeBlkFtr_t|Block_t 0P0| udata |pad|Block_t L0T|fhdr| free... ] + + + udata = Allocated user data + pad = Padding to ensure correct alignment for user data + fhdr = Allocator specific header to keep track of free block + free = Unused free memory + T = This block is free (THIS_FREE_BLK_HDR_FLG) + P = Previous block is free (PREV_FREE_BLK_HDR_FLG) + L = Last block in carrier (LAST_BLK_HDR_FLG) +*/ + +/* Single block carrier (SBC): +[Carrier_t|pad|Block_t 111| udata... ] +*/ + /* Blocks ... */ -#define SBC_BLK_FTR_FLG (((UWord) 1) << 0) +#define UNUSED0_BLK_FTR_FLG (((UWord) 1) << 0) #define UNUSED1_BLK_FTR_FLG (((UWord) 1) << 1) #define UNUSED2_BLK_FTR_FLG (((UWord) 1) << 2) -#define ABLK_HDR_SZ (sizeof(Block_t)) -#define FBLK_FTR_SZ (sizeof(UWord)) +#if MBC_ABLK_OFFSET_BITS +# define ABLK_HDR_SZ (offsetof(Block_t,u)) +#else +# define ABLK_HDR_SZ (sizeof(Block_t)) +#endif +#define FBLK_FTR_SZ (sizeof(FreeBlkFtr_t)) #define UMEMSZ2BLKSZ(AP, SZ) \ (ABLK_HDR_SZ + (SZ) <= (AP)->min_block_size \ @@ -136,88 +168,181 @@ static Uint mseg_unit_size; #define UMEM2BLK(P) ((Block_t *) (((char *) (P)) - ABLK_HDR_SZ)) #define BLK2UMEM(P) ((void *) (((char *) (P)) + ABLK_HDR_SZ)) -#define PREV_BLK_SZ(B) \ - ((UWord) (*(((UWord *) (B)) - 1) & SZ_MASK)) +#define PREV_BLK_SZ(B) ((UWord) (((FreeBlkFtr_t *)(B))[-1])) #define SET_BLK_SZ_FTR(B, SZ) \ - (*((UWord *) (((char *) (B)) + (SZ) - sizeof(UWord))) = (SZ)) + (((FreeBlkFtr_t *) (((char *) (B)) + (SZ)))[-1] = (SZ)) #define THIS_FREE_BLK_HDR_FLG (((UWord) 1) << 0) #define PREV_FREE_BLK_HDR_FLG (((UWord) 1) << 1) #define LAST_BLK_HDR_FLG (((UWord) 1) << 2) -#define SET_BLK_SZ(B, SZ) \ +/* Special flag combo for (allocated) SBC blocks +*/ +#define SBC_BLK_HDR_FLG (THIS_FREE_BLK_HDR_FLG | PREV_FREE_BLK_HDR_FLG | LAST_BLK_HDR_FLG) + +#define SET_MBC_ABLK_SZ(B, SZ) \ (ASSERT(((SZ) & FLG_MASK) == 0), \ - (*((Block_t *) (B)) = ((*((Block_t *) (B)) & FLG_MASK) | (SZ)))) -#define SET_BLK_FREE(B) \ - (*((Block_t *) (B)) |= THIS_FREE_BLK_HDR_FLG) -#define SET_BLK_ALLOCED(B) \ - (*((Block_t *) (B)) &= ~THIS_FREE_BLK_HDR_FLG) -#define SET_PREV_BLK_FREE(B) \ - (*((Block_t *) (B)) |= PREV_FREE_BLK_HDR_FLG) + (B)->bhdr = (((B)->bhdr) & ~MBC_ABLK_SZ_MASK) | (SZ)) +#define SET_MBC_FBLK_SZ(B, SZ) \ + (ASSERT(((SZ) & FLG_MASK) == 0), \ + (B)->bhdr = (((B)->bhdr) & ~MBC_FBLK_SZ_MASK) | (SZ)) +#define SET_SBC_BLK_SZ(B, SZ) \ + (ASSERT(((SZ) & FLG_MASK) == 0), \ + (B)->bhdr = (((B)->bhdr) & ~SBC_BLK_SZ_MASK) | (SZ)) +#define SET_PREV_BLK_FREE(AP,B) \ + (ASSERT(!IS_MBC_FIRST_BLK(AP,B)), \ + ASSERT(!IS_FREE_BLK(B)), \ + (B)->bhdr |= PREV_FREE_BLK_HDR_FLG) #define SET_PREV_BLK_ALLOCED(B) \ - (*((Block_t *) (B)) &= ~PREV_FREE_BLK_HDR_FLG) + ((B)->bhdr &= ~PREV_FREE_BLK_HDR_FLG) #define SET_LAST_BLK(B) \ - (*((Block_t *) (B)) |= LAST_BLK_HDR_FLG) + ((B)->bhdr |= LAST_BLK_HDR_FLG) #define SET_NOT_LAST_BLK(B) \ - (*((Block_t *) (B)) &= ~LAST_BLK_HDR_FLG) + ((B)->bhdr &= ~LAST_BLK_HDR_FLG) #define SBH_THIS_FREE THIS_FREE_BLK_HDR_FLG -#define SBH_THIS_ALLOCED ((UWord) 0) #define SBH_PREV_FREE PREV_FREE_BLK_HDR_FLG -#define SBH_PREV_ALLOCED ((UWord) 0) #define SBH_LAST_BLK LAST_BLK_HDR_FLG -#define SBH_NOT_LAST_BLK ((UWord) 0) -#define SET_BLK_HDR(B, Sz, F) \ - (ASSERT(((Sz) & FLG_MASK) == 0), *((Block_t *) (B)) = ((Sz) | (F))) + +#if MBC_ABLK_OFFSET_BITS + +# define MBC_SZ_MAX_LIMIT ((((UWord)1 << MBC_ABLK_OFFSET_BITS) - 1) << MSEG_ALIGN_BITS) + +# define BLK_CARRIER_OFFSET(B, C) (((char*)(B) - (char*)(C)) >> MSEG_UNIT_SHIFT) + +# define SET_MBC_ABLK_HDR(B, Sz, F, C) \ + (ASSERT(((Sz) & ~MBC_ABLK_SZ_MASK) == 0), \ + ASSERT(!((UWord)(F) & (~FLG_MASK|THIS_FREE_BLK_HDR_FLG))), \ + (B)->bhdr = ((Sz) | (F) | (BLK_CARRIER_OFFSET(B,C) << MBC_ABLK_OFFSET_SHIFT))) + +# define SET_MBC_FBLK_HDR(B, Sz, F, C) \ + (ASSERT(((Sz) & ~MBC_FBLK_SZ_MASK) == 0), \ + ASSERT(((UWord)(F) & (~FLG_MASK|THIS_FREE_BLK_HDR_FLG|PREV_FREE_BLK_HDR_FLG)) == THIS_FREE_BLK_HDR_FLG), \ + (B)->bhdr = ((Sz) | (F)), \ + (B)->u.carrier = (C)) + +# define ABLK_TO_MBC(B) \ + (ASSERT(IS_MBC_BLK(B) && IS_ALLOCED_BLK(B)), \ + (Carrier_t*)((MSEG_UNIT_FLOOR((UWord)(B)) - \ + (((B)->bhdr >> MBC_ABLK_OFFSET_SHIFT) << MSEG_UNIT_SHIFT)))) + +# define FBLK_TO_MBC(B) \ + (ASSERT(IS_MBC_BLK(B) && IS_FREE_BLK(B)), \ + (B)->u.carrier) + +# define BLK_TO_MBC(B) (IS_FREE_BLK(B) ? FBLK_TO_MBC(B) : ABLK_TO_MBC(B)) + +# define IS_MBC_FIRST_ABLK(AP,B) \ + ((((UWord)(B) & ~MSEG_UNIT_MASK) == MBC_HEADER_SIZE(AP)) \ + && ((B)->bhdr & MBC_ABLK_OFFSET_MASK) == 0) + +# define IS_MBC_FIRST_FBLK(AP,B) \ + ((char*)(B) == (char*)((B)->u.carrier) + MBC_HEADER_SIZE(AP)) + +# define IS_MBC_FIRST_BLK(AP,B) \ + (IS_FREE_BLK(B) ? IS_MBC_FIRST_FBLK(AP,B) : IS_MBC_FIRST_ABLK(AP,B)) + +# define SET_BLK_FREE(B) \ + (ASSERT(!IS_PREV_BLK_FREE(B)), \ + (B)->u.carrier = ABLK_TO_MBC(B), \ + (B)->bhdr |= THIS_FREE_BLK_HDR_FLG, \ + (B)->bhdr &= (MBC_ABLK_SZ_MASK|FLG_MASK)) + +# define SET_BLK_ALLOCED(B) \ + (ASSERT(((B)->bhdr & (MBC_ABLK_OFFSET_MASK|THIS_FREE_BLK_HDR_FLG)) == THIS_FREE_BLK_HDR_FLG), \ + (B)->bhdr &= ~THIS_FREE_BLK_HDR_FLG, \ + (B)->bhdr |= (BLK_CARRIER_OFFSET(B,(B)->u.carrier) << MBC_ABLK_OFFSET_SHIFT)) + +#else /* !MBC_ABLK_OFFSET_BITS */ + +# define MBC_SZ_MAX_LIMIT ((UWord)~0) + +# define SET_MBC_ABLK_HDR(B, Sz, F, C) \ + (ASSERT(((Sz) & FLG_MASK) == 0), \ + ASSERT(!((UWord)(F) & (~FLG_MASK|THIS_FREE_BLK_HDR_FLG))), \ + ASSERT((UWord)(F) < SBC_BLK_HDR_FLG), \ + (B)->bhdr = ((Sz) | (F)), \ + (B)->carrier = (C)) + +# define SET_MBC_FBLK_HDR(B, Sz, F, C) \ + (ASSERT(((Sz) & FLG_MASK) == 0), \ + ASSERT(((UWord)(F) & (~FLG_MASK|THIS_FREE_BLK_HDR_FLG|PREV_FREE_BLK_HDR_FLG)) == THIS_FREE_BLK_HDR_FLG), \ + (B)->bhdr = ((Sz) | (F)), \ + (B)->carrier = (C)) + +# define BLK_TO_MBC(B) ((B)->carrier) +# define ABLK_TO_MBC(B) BLK_TO_MBC(B) +# define FBLK_TO_MBC(B) BLK_TO_MBC(B) + +# define IS_MBC_FIRST_BLK(AP,B) \ + ((char*)(B) == (char*)((B)->carrier) + MBC_HEADER_SIZE(AP)) +# define IS_MBC_FIRST_ABLK(AP,B) IS_MBC_FIRST_BLK(AP,B) +# define IS_MBC_FIRST_FBLK(AP,B) IS_MBC_FIRST_BLK(AP,B) + +# define SET_BLK_FREE(B) \ + (ASSERT(!IS_PREV_BLK_FREE(B)), \ + (B)->bhdr |= THIS_FREE_BLK_HDR_FLG) + +# define SET_BLK_ALLOCED(B) \ + ((B)->bhdr &= ~THIS_FREE_BLK_HDR_FLG) + +#endif /* !MBC_ABLK_OFFSET_BITS */ + +#define SET_SBC_BLK_HDR(B, Sz) \ + (ASSERT(((Sz) & FLG_MASK) == 0), (B)->bhdr = ((Sz) | (SBC_BLK_HDR_FLG))) + #define BLK_UMEM_SZ(B) \ (BLK_SZ(B) - (ABLK_HDR_SZ)) #define IS_PREV_BLK_FREE(B) \ - (*((Block_t *) (B)) & PREV_FREE_BLK_HDR_FLG) + ((B)->bhdr & PREV_FREE_BLK_HDR_FLG) #define IS_PREV_BLK_ALLOCED(B) \ (!IS_PREV_BLK_FREE((B))) #define IS_FREE_BLK(B) \ - (*((Block_t *) (B)) & THIS_FREE_BLK_HDR_FLG) + (ASSERT(!IS_SBC_BLK(B)), (B)->bhdr & THIS_FREE_BLK_HDR_FLG) #define IS_ALLOCED_BLK(B) \ (!IS_FREE_BLK((B))) #define IS_LAST_BLK(B) \ - (*((Block_t *) (B)) & LAST_BLK_HDR_FLG) + ((B)->bhdr & LAST_BLK_HDR_FLG) #define IS_NOT_LAST_BLK(B) \ (!IS_LAST_BLK((B))) #define GET_LAST_BLK_HDR_FLG(B) \ - (*((Block_t*) (B)) & LAST_BLK_HDR_FLG) + ((B)->bhdr & LAST_BLK_HDR_FLG) #define GET_THIS_FREE_BLK_HDR_FLG(B) \ - (*((Block_t*) (B)) & THIS_FREE_BLK_HDR_FLG) + ((B)->bhdr & THIS_FREE_BLK_HDR_FLG) #define GET_PREV_FREE_BLK_HDR_FLG(B) \ - (*((Block_t*) (B)) & PREV_FREE_BLK_HDR_FLG) + ((B)->bhdr & PREV_FREE_BLK_HDR_FLG) #define GET_BLK_HDR_FLGS(B) \ - (*((Block_t*) (B)) & FLG_MASK) - -#define IS_FIRST_BLK(B) \ - (IS_PREV_BLK_FREE((B)) && (PREV_BLK_SZ((B)) == 0)) -#define IS_NOT_FIRST_BLK(B) \ - (!IS_FIRST_BLK((B))) - -#define SET_SBC_BLK_FTR(FTR) \ - ((FTR) = (0 | SBC_BLK_FTR_FLG)) -#define SET_MBC_BLK_FTR(FTR) \ - ((FTR) = 0) + ((B)->bhdr & FLG_MASK) #define IS_SBC_BLK(B) \ - (IS_PREV_BLK_FREE((B)) && (((UWord *) (B))[-1] & SBC_BLK_FTR_FLG)) + (((B)->bhdr & FLG_MASK) == SBC_BLK_HDR_FLG) #define IS_MBC_BLK(B) \ (!IS_SBC_BLK((B))) +#define MBC_BLK_SZ(B) (IS_FREE_BLK(B) ? MBC_FBLK_SZ(B) : MBC_ABLK_SZ(B)) + #define NXT_BLK(B) \ - ((Block_t *) (((char *) (B)) + BLK_SZ((B)))) + (ASSERT(IS_MBC_BLK(B)), \ + (Block_t *) (((char *) (B)) + MBC_BLK_SZ((B)))) #define PREV_BLK(B) \ ((Block_t *) (((char *) (B)) - PREV_BLK_SZ((B)))) +#define BLK_AFTER(B,Sz) \ + ((Block_t *) (((char *) (B)) + (Sz))) + +#define BLK_SZ(B) ((B)->bhdr & (((B)->bhdr & THIS_FREE_BLK_HDR_FLG) ? MBC_FBLK_SZ_MASK : MBC_ABLK_SZ_MASK)) + /* Carriers ... */ +#define SBC_HEADER_SIZE (UNIT_CEILING(sizeof(Carrier_t) + ABLK_HDR_SZ) \ + - ABLK_HDR_SZ) +#define MBC_HEADER_SIZE(AP) SBC_HEADER_SIZE + + #define MSEG_CARRIER_HDR_FLAG (((UWord) 1) << 0) #define SBC_CARRIER_HDR_FLAG (((UWord) 1) << 1) @@ -226,20 +351,20 @@ static Uint mseg_unit_size; #define SCH_MBC 0 #define SCH_SBC SBC_CARRIER_HDR_FLAG -#define SET_CARRIER_HDR(C, Sz, F) \ - (ASSERT(((Sz) & FLG_MASK) == 0), (C)->chdr = ((Sz) | (F))) +#define SET_CARRIER_HDR(C, Sz, F, AP) \ + (ASSERT(((Sz) & FLG_MASK) == 0), (C)->chdr = ((Sz) | (F)), (C)->allctr = (AP)) -#define BLK2SBC(AP, B) \ - ((Carrier_t *) (((char *) (B)) - (AP)->sbc_header_size)) -#define FBLK2MBC(AP, B) \ - ((Carrier_t *) (((char *) (B)) - (AP)->mbc_header_size)) +#define BLK_TO_SBC(B) \ + ((Carrier_t *) (((char *) (B)) - SBC_HEADER_SIZE)) +#define FIRST_BLK_TO_MBC(AP, B) \ + ((Carrier_t *) (((char *) (B)) - MBC_HEADER_SIZE(AP))) -#define MBC2FBLK(AP, P) \ - ((Block_t *) (((char *) (P)) + (AP)->mbc_header_size)) +#define MBC_TO_FIRST_BLK(AP, P) \ + ((Block_t *) (((char *) (P)) + MBC_HEADER_SIZE(AP))) #define SBC2BLK(AP, P) \ - ((Block_t *) (((char *) (P)) + (AP)->sbc_header_size)) + ((Block_t *) (((char *) (P)) + SBC_HEADER_SIZE)) #define SBC2UMEM(AP, P) \ - ((void *) (((char *) (P)) + ((AP)->sbc_header_size + ABLK_HDR_SZ))) + ((void *) (((char *) (P)) + (SBC_HEADER_SIZE + ABLK_HDR_SZ))) #define IS_MSEG_CARRIER(C) \ ((C)->chdr & MSEG_CARRIER_HDR_FLAG) @@ -250,15 +375,6 @@ static Uint mseg_unit_size; #define IS_MB_CARRIER(C) \ (!IS_SB_CARRIER((C))) -#define SET_MSEG_CARRIER(C) \ - ((C)->chdr |= MSEG_CARRIER_HDR_FLAG) -#define SET_SYS_ALLOC_CARRIER(C) \ - ((C)->chdr &= ~MSEG_CARRIER_HDR_FLAG) -#define SET_SB_CARRIER(C) \ - ((C)->chdr |= SBC_CARRIER_HDR_FLAG) -#define SET_MB_CARRIER(C) \ - ((C)->chdr &= ~SBC_CARRIER_HDR_FLAG) - #define SET_CARRIER_SZ(C, SZ) \ (ASSERT(((SZ) & FLG_MASK) == 0), \ ((C)->chdr = ((C)->chdr & FLG_MASK) | (SZ))) @@ -506,11 +622,11 @@ static void mbc_free(Allctr_t *allctr, void *p); #if HAVE_ERTS_MSEG static ERTS_INLINE void * -alcu_mseg_alloc(Allctr_t *allctr, Uint *size_p) +alcu_mseg_alloc(Allctr_t *allctr, Uint *size_p, Uint flags) { void *res; - res = erts_mseg_alloc_opt(allctr->alloc_no, size_p, &allctr->mseg_opt); + res = erts_mseg_alloc_opt(allctr->alloc_no, size_p, flags, &allctr->mseg_opt); INC_CC(allctr->calls.mseg_alloc); return res; } @@ -521,7 +637,7 @@ alcu_mseg_realloc(Allctr_t *allctr, void *seg, Uint old_size, Uint *new_size_p) void *res; res = erts_mseg_realloc_opt(allctr->alloc_no, seg, old_size, new_size_p, - &allctr->mseg_opt); + ERTS_MSEG_FLG_NONE, &allctr->mseg_opt); INC_CC(allctr->calls.mseg_realloc); return res; } @@ -765,13 +881,9 @@ erts_alcu_fix_alloc_shrink(Allctr_t *allctr, erts_aint32_t flgs) #define ERTS_ALCU_DD_FIX_TYPE_OFFS \ ((sizeof(ErtsAllctrDDBlock_t)-1)/sizeof(UWord) + 1) -#define ERTS_AU_PREF_ALLOC_IX_MASK \ - ((((UWord) 1) << ERTS_AU_PREF_ALLOC_BITS) - 1) -#define ERTS_AU_PREF_ALLOC_SIZE_MASK \ - ((((UWord) 1) << (sizeof(UWord)*8 - ERTS_AU_PREF_ALLOC_BITS)) - 1) -static ERTS_INLINE int -get_pref_allctr(void *extra, Allctr_t **allctr) +static ERTS_INLINE Allctr_t* +get_pref_allctr(void *extra) { ErtsAllocatorThrSpec_t *tspec = (ErtsAllocatorThrSpec_t *) extra; int pref_ix; @@ -781,34 +893,33 @@ get_pref_allctr(void *extra, Allctr_t **allctr) ASSERT(sizeof(UWord) == sizeof(Allctr_t *)); ASSERT(0 <= pref_ix && pref_ix < tspec->size); - *allctr = tspec->allctr[pref_ix]; - return pref_ix; + return tspec->allctr[pref_ix]; } -static ERTS_INLINE void * -get_used_allctr(void *extra, void *p, Allctr_t **allctr, UWord *sizep) +/* SMP note: + * get_used_allctr() must be safe WITHOUT locking the allocator while + * concurrent threads may be updating adjacent blocks. + * We rely on getting a consistent result (without atomic op) when reading + * the block header word even if a concurrent thread is updating + * the "PREV_FREE" flag bit. + */ +static ERTS_INLINE Allctr_t* +get_used_allctr(void *extra, void *p, UWord *sizep) { - ErtsAllocatorThrSpec_t *tspec = (ErtsAllocatorThrSpec_t *) extra; - void *ptr = (void *) (((char *) p) - sizeof(UWord)); - UWord ainfo = *((UWord *) ptr); - int aix = (int) (ainfo & ERTS_AU_PREF_ALLOC_IX_MASK); - *allctr = tspec->allctr[aix]; - if (sizep) - *sizep = ((ainfo >> ERTS_AU_PREF_ALLOC_BITS) - & ERTS_AU_PREF_ALLOC_SIZE_MASK); - return ptr; -} + Block_t* blk = UMEM2BLK(p); + Carrier_t* crr; -static ERTS_INLINE void * -put_used_allctr(void *p, int ix, UWord size) -{ - UWord ainfo = (size >= ERTS_AU_PREF_ALLOC_SIZE_MASK - ? ERTS_AU_PREF_ALLOC_SIZE_MASK - : size); - ainfo <<= ERTS_AU_PREF_ALLOC_BITS; - ainfo |= (UWord) ix; - *((UWord *) p) = ainfo; - return (void *) (((char *) p) + sizeof(UWord)); + if (IS_SBC_BLK(blk)) { + crr = BLK_TO_SBC(blk); + if (sizep) + *sizep = SBC_BLK_SZ(blk) - ABLK_HDR_SZ; + } + else { + crr = ABLK_TO_MBC(blk); + if (sizep) + *sizep = MBC_ABLK_SZ(blk) - ABLK_HDR_SZ; + } + return crr->allctr; } static void @@ -1209,10 +1320,8 @@ mbc_alloc_block(Allctr_t *allctr, Uint size, Uint *blk_szp, Uint32 *alcu_flgsp) if ((*alcu_flgsp) & ERTS_ALCU_FLG_SBMBC) blk = create_sbmbc(allctr, get_blk_sz); else { -#if HALFWORD_HEAP - blk = create_carrier(allctr, get_blk_sz, CFLG_MBC|CFLG_FORCE_MSEG); -#else blk = create_carrier(allctr, get_blk_sz, CFLG_MBC); +#if !HALFWORD_HEAP && !HAVE_SUPER_ALIGNED_MB_CARRIERS if (!blk) { /* Emergency! We couldn't create the carrier as we wanted. Try to place it in a sys_alloced sbc. */ @@ -1242,6 +1351,7 @@ mbc_alloc_finalize(Allctr_t *allctr, Block_t *blk, Uint org_blk_sz, UWord flags, + Carrier_t *crr, Uint want_blk_sz, int valid_blk_info, Uint32 alcu_flgs) @@ -1262,22 +1372,18 @@ mbc_alloc_finalize(Allctr_t *allctr, /* Shrink block... */ blk_sz = want_blk_sz; nxt_blk_sz = org_blk_sz - blk_sz; - SET_BLK_HDR(blk, - blk_sz, - SBH_THIS_ALLOCED|SBH_NOT_LAST_BLK|prev_free_flg); + SET_MBC_ABLK_HDR(blk, blk_sz, prev_free_flg, crr); - nxt_blk = NXT_BLK(blk); - SET_BLK_HDR(nxt_blk, - nxt_blk_sz, - (SBH_THIS_FREE - | SBH_PREV_ALLOCED - | (flags & LAST_BLK_HDR_FLG))); + nxt_blk = BLK_AFTER(blk, blk_sz); + SET_MBC_FBLK_HDR(nxt_blk, nxt_blk_sz, + SBH_THIS_FREE|(flags & LAST_BLK_HDR_FLG), + crr); if (!(flags & LAST_BLK_HDR_FLG)) { SET_BLK_SZ_FTR(nxt_blk, nxt_blk_sz); if (!valid_blk_info) { - Block_t *nxt_nxt_blk = NXT_BLK(nxt_blk); - SET_PREV_BLK_FREE(nxt_nxt_blk); + Block_t *nxt_nxt_blk = BLK_AFTER(nxt_blk, nxt_blk_sz); + SET_PREV_BLK_FREE(allctr, nxt_nxt_blk); } } (*allctr->link_free_block)(allctr, nxt_blk, alcu_flgs); @@ -1291,40 +1397,40 @@ mbc_alloc_finalize(Allctr_t *allctr, || nxt_blk == PREV_BLK(NXT_BLK(nxt_blk))); ASSERT((flags & LAST_BLK_HDR_FLG) || IS_PREV_BLK_FREE(NXT_BLK(nxt_blk))); - ASSERT(nxt_blk_sz == BLK_SZ(nxt_blk)); + ASSERT(nxt_blk_sz == MBC_BLK_SZ(nxt_blk)); ASSERT(nxt_blk_sz % sizeof(Unit_t) == 0); ASSERT(nxt_blk_sz >= allctr->min_block_size); + ASSERT(ABLK_TO_MBC(blk) == crr); + ASSERT(FBLK_TO_MBC(nxt_blk) == crr); } else { + ASSERT(org_blk_sz <= MBC_ABLK_SZ_MASK); blk_sz = org_blk_sz; if (flags & LAST_BLK_HDR_FLG) { if (valid_blk_info) SET_BLK_ALLOCED(blk); else - SET_BLK_HDR(blk, - blk_sz, - SBH_THIS_ALLOCED|SBH_LAST_BLK|prev_free_flg); + SET_MBC_ABLK_HDR(blk, blk_sz, SBH_LAST_BLK|prev_free_flg, crr); } else { if (valid_blk_info) SET_BLK_ALLOCED(blk); else - SET_BLK_HDR(blk, - blk_sz, - SBH_THIS_ALLOCED|SBH_NOT_LAST_BLK|prev_free_flg); - nxt_blk = NXT_BLK(blk); + SET_MBC_ABLK_HDR(blk, blk_sz, prev_free_flg, crr); + nxt_blk = BLK_AFTER(blk, blk_sz); SET_PREV_BLK_ALLOCED(nxt_blk); } ASSERT((flags & LAST_BLK_HDR_FLG) ? IS_LAST_BLK(blk) : IS_NOT_LAST_BLK(blk)); + ASSERT(ABLK_TO_MBC(blk) == crr); } STAT_MBC_BLK_ALLOC(allctr, blk_sz, alcu_flgs); ASSERT(IS_ALLOCED_BLK(blk)); - ASSERT(blk_sz == BLK_SZ(blk)); + ASSERT(blk_sz == MBC_BLK_SZ(blk)); ASSERT(blk_sz % sizeof(Unit_t) == 0); ASSERT(blk_sz >= allctr->min_block_size); ASSERT(blk_sz >= want_blk_sz); @@ -1348,8 +1454,9 @@ mbc_alloc(Allctr_t *allctr, Uint size) if (IS_MBC_BLK(blk)) mbc_alloc_finalize(allctr, blk, - BLK_SZ(blk), + MBC_FBLK_SZ(blk), GET_BLK_HDR_FLGS(blk), + FBLK_TO_MBC(blk), blk_sz, 1, alcu_flgs); @@ -1370,7 +1477,7 @@ mbc_free(Allctr_t *allctr, void *p) ASSERT(p); blk = UMEM2BLK(p); - blk_sz = BLK_SZ(blk); + blk_sz = MBC_ABLK_SZ(blk); if (blk_sz < allctr->sbmbc_threshold) alcu_flgs |= ERTS_ALCU_FLG_SBMBC; @@ -1381,17 +1488,18 @@ mbc_free(Allctr_t *allctr, void *p) STAT_MBC_BLK_FREE(allctr, blk_sz, alcu_flgs); - is_first_blk = IS_FIRST_BLK(blk); + is_first_blk = IS_MBC_FIRST_ABLK(allctr, blk); is_last_blk = IS_LAST_BLK(blk); - if (!is_first_blk && IS_PREV_BLK_FREE(blk)) { + if (IS_PREV_BLK_FREE(blk)) { + ASSERT(!is_first_blk); /* Coalesce with previous block... */ blk = PREV_BLK(blk); (*allctr->unlink_free_block)(allctr, blk, alcu_flgs); - blk_sz += BLK_SZ(blk); - is_first_blk = IS_FIRST_BLK(blk); - SET_BLK_SZ(blk, blk_sz); + blk_sz += MBC_FBLK_SZ(blk); + is_first_blk = IS_MBC_FIRST_FBLK(allctr, blk); + SET_MBC_FBLK_SZ(blk, blk_sz); } else { SET_BLK_FREE(blk); @@ -1400,12 +1508,12 @@ mbc_free(Allctr_t *allctr, void *p) if (is_last_blk) SET_LAST_BLK(blk); else { - nxt_blk = NXT_BLK(blk); + nxt_blk = BLK_AFTER(blk, blk_sz); if (IS_FREE_BLK(nxt_blk)) { /* Coalesce with next block... */ (*allctr->unlink_free_block)(allctr, nxt_blk, alcu_flgs); - blk_sz += BLK_SZ(nxt_blk); - SET_BLK_SZ(blk, blk_sz); + blk_sz += MBC_FBLK_SZ(nxt_blk); + SET_MBC_FBLK_SZ(blk, blk_sz); is_last_blk = IS_LAST_BLK(nxt_blk); if (is_last_blk) @@ -1416,26 +1524,26 @@ mbc_free(Allctr_t *allctr, void *p) } } else { - SET_PREV_BLK_FREE(nxt_blk); + SET_PREV_BLK_FREE(allctr, nxt_blk); SET_NOT_LAST_BLK(blk); SET_BLK_SZ_FTR(blk, blk_sz); } } - ASSERT(is_last_blk ? IS_LAST_BLK(blk) : IS_NOT_LAST_BLK(blk)); - ASSERT(is_first_blk ? IS_FIRST_BLK(blk) : IS_NOT_FIRST_BLK(blk)); ASSERT(IS_FREE_BLK(blk)); + ASSERT(!is_last_blk == !IS_LAST_BLK(blk)); + ASSERT(!is_first_blk == !IS_MBC_FIRST_FBLK(allctr, blk)); ASSERT(is_first_blk || IS_PREV_BLK_ALLOCED(blk)); ASSERT(is_last_blk || IS_PREV_BLK_FREE(NXT_BLK(blk))); - ASSERT(blk_sz == BLK_SZ(blk)); + ASSERT(blk_sz == MBC_BLK_SZ(blk)); ASSERT(is_last_blk || blk == PREV_BLK(NXT_BLK(blk))); ASSERT(blk_sz % sizeof(Unit_t) == 0); ASSERT(IS_MBC_BLK(blk)); if (is_first_blk && is_last_blk - && allctr->main_carrier != FBLK2MBC(allctr, blk)) { + && allctr->main_carrier != FIRST_BLK_TO_MBC(allctr, blk)) { if (alcu_flgs & ERTS_ALCU_FLG_SBMBC) destroy_sbmbc(allctr, blk); else @@ -1472,7 +1580,7 @@ mbc_realloc(Allctr_t *allctr, void *p, Uint size, Uint32 alcu_flgs) ASSERT(size < allctr->sbc_threshold); blk = (Block_t *) UMEM2BLK(p); - old_blk_sz = BLK_SZ(blk); + old_blk_sz = MBC_ABLK_SZ(blk); ASSERT(old_blk_sz >= allctr->min_block_size); @@ -1497,6 +1605,7 @@ mbc_realloc(Allctr_t *allctr, void *p, Uint size, Uint32 alcu_flgs) return p; else if (blk_sz < old_blk_sz) { /* Shrink block... */ + Carrier_t* crr; Block_t *nxt_nxt_blk; Uint diff_sz_val = old_blk_sz - blk_sz; Uint old_blk_sz_val = old_blk_sz; @@ -1516,16 +1625,18 @@ mbc_realloc(Allctr_t *allctr, void *p, Uint size, Uint32 alcu_flgs) return NULL; cand_blk_sz = old_blk_sz; - if (!IS_PREV_BLK_FREE(blk) || IS_FIRST_BLK(blk)) + if (!IS_PREV_BLK_FREE(blk)) { cand_blk = blk; + } else { + ASSERT(!IS_MBC_FIRST_ABLK(allctr, blk)); cand_blk = PREV_BLK(blk); cand_blk_sz += PREV_BLK_SZ(blk); } if (!is_last_blk) { - nxt_blk = NXT_BLK(blk); + nxt_blk = BLK_AFTER(blk, old_blk_sz); if (IS_FREE_BLK(nxt_blk)) - cand_blk_sz += BLK_SZ(nxt_blk); + cand_blk_sz += MBC_FBLK_SZ(nxt_blk); } new_blk = (*allctr->get_free_block)(allctr, @@ -1541,52 +1652,48 @@ mbc_realloc(Allctr_t *allctr, void *p, Uint size, Uint32 alcu_flgs) nxt_blk_sz = old_blk_sz - blk_sz; - if ((is_last_blk || IS_ALLOCED_BLK(NXT_BLK(blk))) + if ((is_last_blk || IS_ALLOCED_BLK(BLK_AFTER(blk,old_blk_sz))) && (nxt_blk_sz < allctr->min_block_size)) return p; HARD_CHECK_BLK_CARRIER(allctr, blk); - SET_BLK_SZ(blk, blk_sz); + nxt_nxt_blk = BLK_AFTER(blk, old_blk_sz); + + SET_MBC_ABLK_SZ(blk, blk_sz); SET_NOT_LAST_BLK(blk); - nxt_blk = NXT_BLK(blk); - SET_BLK_HDR(nxt_blk, - nxt_blk_sz, - SBH_THIS_FREE|SBH_PREV_ALLOCED|SBH_NOT_LAST_BLK); + nxt_blk = BLK_AFTER(blk, blk_sz); STAT_MBC_BLK_FREE(allctr, old_blk_sz, alcu_flgs); STAT_MBC_BLK_ALLOC(allctr, blk_sz, alcu_flgs); - ASSERT(BLK_SZ(blk) >= allctr->min_block_size); + ASSERT(MBC_BLK_SZ(blk) >= allctr->min_block_size); - if (is_last_blk) - SET_LAST_BLK(nxt_blk); - else { - nxt_nxt_blk = NXT_BLK(nxt_blk); + if (!is_last_blk) { if (IS_FREE_BLK(nxt_nxt_blk)) { /* Coalesce with next free block... */ - nxt_blk_sz += BLK_SZ(nxt_nxt_blk); + nxt_blk_sz += MBC_FBLK_SZ(nxt_nxt_blk); (*allctr->unlink_free_block)(allctr, nxt_nxt_blk, alcu_flgs); - SET_BLK_SZ(nxt_blk, nxt_blk_sz); - is_last_blk = IS_LAST_BLK(nxt_nxt_blk); - if (is_last_blk) - SET_LAST_BLK(nxt_blk); - else - SET_BLK_SZ_FTR(nxt_blk, nxt_blk_sz); + is_last_blk = GET_LAST_BLK_HDR_FLG(nxt_nxt_blk); } else { - SET_BLK_SZ_FTR(nxt_blk, nxt_blk_sz); - SET_PREV_BLK_FREE(nxt_nxt_blk); + SET_PREV_BLK_FREE(allctr, nxt_nxt_blk); } + SET_BLK_SZ_FTR(nxt_blk, nxt_blk_sz); } + crr = ABLK_TO_MBC(blk); + SET_MBC_FBLK_HDR(nxt_blk, nxt_blk_sz, + SBH_THIS_FREE | (is_last_blk ? SBH_LAST_BLK : 0), + crr); + (*allctr->link_free_block)(allctr, nxt_blk, alcu_flgs); ASSERT(IS_ALLOCED_BLK(blk)); - ASSERT(blk_sz == BLK_SZ(blk)); + ASSERT(blk_sz == MBC_BLK_SZ(blk)); ASSERT(blk_sz % sizeof(Unit_t) == 0); ASSERT(blk_sz >= allctr->min_block_size); ASSERT(blk_sz >= size + ABLK_HDR_SZ); @@ -1594,14 +1701,15 @@ mbc_realloc(Allctr_t *allctr, void *p, Uint size, Uint32 alcu_flgs) ASSERT(IS_FREE_BLK(nxt_blk)); ASSERT(IS_PREV_BLK_ALLOCED(nxt_blk)); - ASSERT(nxt_blk_sz == BLK_SZ(nxt_blk)); + ASSERT(nxt_blk_sz == MBC_BLK_SZ(nxt_blk)); ASSERT(nxt_blk_sz % sizeof(Unit_t) == 0); ASSERT(nxt_blk_sz >= allctr->min_block_size); ASSERT(IS_MBC_BLK(nxt_blk)); ASSERT(is_last_blk ? IS_LAST_BLK(nxt_blk) : IS_NOT_LAST_BLK(nxt_blk)); ASSERT(is_last_blk || nxt_blk == PREV_BLK(NXT_BLK(nxt_blk))); ASSERT(is_last_blk || IS_PREV_BLK_FREE(NXT_BLK(nxt_blk))); - + ASSERT(FBLK_TO_MBC(nxt_blk) == crr); + HARD_CHECK_BLK_CARRIER(allctr, blk); return p; @@ -1610,8 +1718,8 @@ mbc_realloc(Allctr_t *allctr, void *p, Uint size, Uint32 alcu_flgs) /* Need larger block... */ if (!is_last_blk) { - nxt_blk = NXT_BLK(blk); - nxt_blk_sz = BLK_SZ(nxt_blk); + nxt_blk = BLK_AFTER(blk, old_blk_sz); + nxt_blk_sz = MBC_BLK_SZ(nxt_blk); if (IS_FREE_BLK(nxt_blk) && get_blk_sz <= old_blk_sz + nxt_blk_sz) { /* Grow into next block... */ @@ -1624,7 +1732,7 @@ mbc_realloc(Allctr_t *allctr, void *p, Uint size, Uint32 alcu_flgs) if (nxt_blk_sz < allctr->min_block_size) { blk_sz += nxt_blk_sz; - SET_BLK_SZ(blk, blk_sz); + SET_MBC_ABLK_SZ(blk, blk_sz); if (is_last_blk) { SET_LAST_BLK(blk); @@ -1633,21 +1741,20 @@ mbc_realloc(Allctr_t *allctr, void *p, Uint size, Uint32 alcu_flgs) #endif } else { - nxt_blk = NXT_BLK(blk); + nxt_blk = BLK_AFTER(blk, blk_sz); SET_PREV_BLK_ALLOCED(nxt_blk); #ifdef DEBUG is_last_blk = IS_LAST_BLK(nxt_blk); - nxt_blk_sz = BLK_SZ(nxt_blk); + nxt_blk_sz = MBC_BLK_SZ(nxt_blk); #endif } } else { - SET_BLK_SZ(blk, blk_sz); + Carrier_t* crr = ABLK_TO_MBC(blk); + SET_MBC_ABLK_SZ(blk, blk_sz); - nxt_blk = NXT_BLK(blk); - SET_BLK_HDR(nxt_blk, - nxt_blk_sz, - SBH_THIS_FREE|SBH_PREV_ALLOCED|SBH_NOT_LAST_BLK); + nxt_blk = BLK_AFTER(blk, blk_sz); + SET_MBC_FBLK_HDR(nxt_blk, nxt_blk_sz, SBH_THIS_FREE, crr); if (is_last_blk) SET_LAST_BLK(nxt_blk); @@ -1657,6 +1764,7 @@ mbc_realloc(Allctr_t *allctr, void *p, Uint size, Uint32 alcu_flgs) (*allctr->link_free_block)(allctr, nxt_blk, alcu_flgs); ASSERT(IS_FREE_BLK(nxt_blk)); + ASSERT(FBLK_TO_MBC(nxt_blk) == crr); } STAT_MBC_BLK_FREE(allctr, old_blk_sz, alcu_flgs); @@ -1664,14 +1772,14 @@ mbc_realloc(Allctr_t *allctr, void *p, Uint size, Uint32 alcu_flgs) ASSERT(IS_ALLOCED_BLK(blk)); - ASSERT(blk_sz == BLK_SZ(blk)); + ASSERT(blk_sz == MBC_BLK_SZ(blk)); ASSERT(blk_sz % sizeof(Unit_t) == 0); ASSERT(blk_sz >= allctr->min_block_size); ASSERT(blk_sz >= size + ABLK_HDR_SZ); ASSERT(IS_MBC_BLK(blk)); ASSERT(!nxt_blk || IS_PREV_BLK_ALLOCED(nxt_blk)); - ASSERT(!nxt_blk || nxt_blk_sz == BLK_SZ(nxt_blk)); + ASSERT(!nxt_blk || nxt_blk_sz == MBC_BLK_SZ(nxt_blk)); ASSERT(!nxt_blk || nxt_blk_sz % sizeof(Unit_t) == 0); ASSERT(!nxt_blk || nxt_blk_sz >= allctr->min_block_size); ASSERT(!nxt_blk || IS_MBC_BLK(nxt_blk)); @@ -1696,18 +1804,19 @@ mbc_realloc(Allctr_t *allctr, void *p, Uint size, Uint32 alcu_flgs) /* Need to grow in another block */ - if (!IS_PREV_BLK_FREE(blk) || IS_FIRST_BLK(blk)) { + if (!IS_PREV_BLK_FREE(blk)) { cand_blk = NULL; cand_blk_sz = 0; } else { + ASSERT(!IS_MBC_FIRST_ABLK(allctr, blk)); cand_blk = PREV_BLK(blk); cand_blk_sz = old_blk_sz + PREV_BLK_SZ(blk); if (!is_last_blk) { - nxt_blk = NXT_BLK(blk); + nxt_blk = BLK_AFTER(blk, old_blk_sz); if (IS_FREE_BLK(nxt_blk)) - cand_blk_sz += BLK_SZ(nxt_blk); + cand_blk_sz += MBC_FBLK_SZ(nxt_blk); } } @@ -1743,8 +1852,9 @@ mbc_realloc(Allctr_t *allctr, void *p, Uint size, Uint32 alcu_flgs) if (new_blk) { mbc_alloc_finalize(allctr, new_blk, - BLK_SZ(new_blk), + MBC_FBLK_SZ(new_blk), GET_BLK_HDR_FLGS(new_blk), + FBLK_TO_MBC(new_blk), blk_sz, 1, alcu_flgs); @@ -1754,6 +1864,7 @@ mbc_realloc(Allctr_t *allctr, void *p, Uint size, Uint32 alcu_flgs) return new_p; } else { + Carrier_t* crr; Uint new_blk_sz; UWord new_blk_flgs; Uint prev_blk_sz; @@ -1774,10 +1885,10 @@ mbc_realloc(Allctr_t *allctr, void *p, Uint size, Uint32 alcu_flgs) if (is_last_blk) new_blk_flgs |= LAST_BLK_HDR_FLG; else { - nxt_blk = NXT_BLK(blk); + nxt_blk = BLK_AFTER(blk, old_blk_sz); if (IS_FREE_BLK(nxt_blk)) { new_blk_flgs |= GET_LAST_BLK_HDR_FLG(nxt_blk); - new_blk_sz += BLK_SZ(nxt_blk); + new_blk_sz += MBC_FBLK_SZ(nxt_blk); (*allctr->unlink_free_block)(allctr, nxt_blk, alcu_flgs); } } @@ -1790,6 +1901,7 @@ mbc_realloc(Allctr_t *allctr, void *p, Uint size, Uint32 alcu_flgs) new_p = BLK2UMEM(new_blk); blk_cpy_sz = MIN(blk_sz, old_blk_sz); + crr = FBLK_TO_MBC(new_blk); if (prev_blk_sz >= blk_cpy_sz) sys_memcpy(new_p, p, blk_cpy_sz - ABLK_HDR_SZ); @@ -1800,6 +1912,7 @@ mbc_realloc(Allctr_t *allctr, void *p, Uint size, Uint32 alcu_flgs) new_blk, new_blk_sz, new_blk_flgs, + crr, blk_sz, 0, alcu_flgs); @@ -1815,35 +1928,40 @@ mbc_realloc(Allctr_t *allctr, void *p, Uint size, Uint32 alcu_flgs) #ifdef DEBUG #if HAVE_ERTS_MSEG -#define ASSERT_MSEG_UNIT_SIZE_MULTIPLE(CSZ) ASSERT((CSZ) % mseg_unit_size == 0) +#define ASSERT_MSEG_UNIT_SIZE_MULTIPLE(CSZ) ASSERT((CSZ) % MSEG_UNIT_SZ == 0) #else #define ASSERT_MSEG_UNIT_SIZE_MULTIPLE(CSZ) #endif -#define CHECK_1BLK_CARRIER(A, SBC, MSEGED, C, CSZ, B, BSZ) \ -do { \ - ASSERT(IS_FIRST_BLK((B))); \ - ASSERT(IS_LAST_BLK((B))); \ - ASSERT((CSZ) == CARRIER_SZ((C))); \ - ASSERT((BSZ) == BLK_SZ((B))); \ - ASSERT((BSZ) % sizeof(Unit_t) == 0); \ - if ((SBC)) { \ - ASSERT(IS_SBC_BLK((B))); \ - ASSERT(IS_SB_CARRIER((C))); \ - } \ - else { \ - ASSERT(IS_MBC_BLK((B))); \ - ASSERT(IS_MB_CARRIER((C))); \ - } \ - if ((MSEGED)) { \ - ASSERT(IS_MSEG_CARRIER((C))); \ - ASSERT_MSEG_UNIT_SIZE_MULTIPLE((CSZ)); \ - } \ - else { \ - ASSERT(IS_SYS_ALLOC_CARRIER((C))); \ - ASSERT((CSZ) % sizeof(Unit_t) == 0); \ - } \ -} while (0) +static void CHECK_1BLK_CARRIER(Allctr_t* A, int SBC, int MSEGED, Carrier_t* C, + UWord CSZ, Block_t* B, UWord BSZ) +{ + ASSERT(IS_LAST_BLK((B))); + ASSERT((CSZ) == CARRIER_SZ((C))); + ASSERT((BSZ) % sizeof(Unit_t) == 0); + if ((SBC)) { + ASSERT((BSZ) == SBC_BLK_SZ((B))); + ASSERT((char*)B == (char*)C + SBC_HEADER_SIZE); + ASSERT(IS_SBC_BLK((B))); + ASSERT(IS_SB_CARRIER((C))); + } + else { + ASSERT(IS_FREE_BLK(B)); + ASSERT((BSZ) == MBC_FBLK_SZ((B))); + ASSERT(IS_MBC_FIRST_FBLK(A, (B))); + ASSERT(IS_MBC_BLK((B))); + ASSERT(IS_MB_CARRIER((C))); + ASSERT(FBLK_TO_MBC(B) == (C)); + } + if ((MSEGED)) { + ASSERT(IS_MSEG_CARRIER((C))); + ASSERT_MSEG_UNIT_SIZE_MULTIPLE((CSZ)); + } + else { + ASSERT(IS_SYS_ALLOC_CARRIER((C))); + ASSERT((CSZ) % sizeof(Unit_t) == 0); + } +} #else #define CHECK_1BLK_CARRIER(A, SBC, MSEGED, C, CSZ, B, BSZ) @@ -1865,37 +1983,18 @@ create_sbmbc(Allctr_t *allctr, Uint umem_sz) crr = erts_alloc(ERTS_ALC_T_SBMBC, crr_sz); INC_CC(allctr->calls.sbmbc_alloc); - SET_CARRIER_HDR(crr, crr_sz, SCH_SYS_ALLOC|SCH_MBC); - - blk = MBC2FBLK(allctr, crr); + SET_CARRIER_HDR(crr, crr_sz, SCH_SYS_ALLOC|SCH_MBC, allctr); -#ifdef ERTS_ALLOC_UTIL_HARD_DEBUG - if (allctr->mbc_header_size % sizeof(Unit_t) == 0) - crr_sz -= sizeof(UWord); -#endif + blk = MBC_TO_FIRST_BLK(allctr, crr); - blk_sz = UNIT_FLOOR(crr_sz - allctr->mbc_header_size); + blk_sz = UNIT_FLOOR(crr_sz - MBC_HEADER_SIZE(allctr)); - SET_MBC_BLK_FTR(((UWord *) blk)[-1]); - SET_BLK_HDR(blk, blk_sz, SBH_THIS_FREE|SBH_PREV_FREE|SBH_LAST_BLK); - -#ifdef ERTS_ALLOC_UTIL_HARD_DEBUG - *((Carrier_t **) NXT_BLK(blk)) = crr; -#endif + SET_MBC_FBLK_HDR(blk, blk_sz, SBH_THIS_FREE|SBH_LAST_BLK, crr); link_carrier(&allctr->sbmbc_list, crr); -#ifdef ERTS_ALLOC_UTIL_HARD_DEBUG - if (allctr->mbc_header_size % sizeof(Unit_t) == 0) - crr_sz += sizeof(UWord); -#endif - STAT_SBMBC_ALLOC(allctr, crr_sz); CHECK_1BLK_CARRIER(allctr, 0, 0, crr, crr_sz, blk, blk_sz); -#ifdef ERTS_ALLOC_UTIL_HARD_DEBUG - if (allctr->mbc_header_size % sizeof(Unit_t) == 0) - crr_sz -= sizeof(UWord); -#endif if (allctr->creating_mbc) (*allctr->creating_mbc)(allctr, crr, ERTS_ALCU_FLG_SBMBC); @@ -1909,11 +2008,10 @@ destroy_sbmbc(Allctr_t *allctr, Block_t *blk) Uint crr_sz; Carrier_t *crr; - ASSERT(IS_FIRST_BLK(blk)); - ASSERT(IS_MBC_BLK(blk)); + ASSERT(IS_MBC_FIRST_FBLK(allctr, blk)); - crr = FBLK2MBC(allctr, blk); + crr = FIRST_BLK_TO_MBC(allctr, blk); crr_sz = CARRIER_SZ(crr); #ifdef DEBUG @@ -1952,14 +2050,25 @@ create_carrier(Allctr_t *allctr, Uint umem_sz, UWord flags) Uint blk_sz, bcrr_sz, crr_sz; #if HAVE_ERTS_MSEG int have_tried_sys_alloc = 0, have_tried_mseg = 0; + Uint mseg_flags; #endif #ifdef DEBUG int is_mseg = 0; #endif +#if HALFWORD_HEAP + flags |= CFLG_FORCE_MSEG; +#elif HAVE_SUPER_ALIGNED_MB_CARRIERS + if (flags & CFLG_MBC) { + flags |= CFLG_FORCE_MSEG; + } +#endif + ASSERT((flags & CFLG_SBC && !(flags & CFLG_MBC)) || (flags & CFLG_MBC && !(flags & CFLG_SBC))); + ASSERT(!(flags & CFLG_FORCE_MSEG && flags & CFLG_FORCE_SYS_ALLOC)); + blk_sz = UMEMSZ2BLKSZ(allctr, umem_sz); #if HAVE_ERTS_MSEG @@ -1974,29 +2083,27 @@ create_carrier(Allctr_t *allctr, Uint umem_sz, UWord flags) if (allctr->sbcs.curr.norm.mseg.no >= allctr->max_mseg_sbcs) goto try_sys_alloc; } +#if !HAVE_SUPER_ALIGNED_MB_CARRIERS else { if (allctr->mbcs.curr.norm.mseg.no >= allctr->max_mseg_mbcs) goto try_sys_alloc; } +#endif try_mseg: if (flags & CFLG_SBC) { - crr_sz = blk_sz + allctr->sbc_header_size; + crr_sz = blk_sz + SBC_HEADER_SIZE; + mseg_flags = ERTS_MSEG_FLG_NONE; } else { crr_sz = (*allctr->get_next_mbc_size)(allctr); - if (crr_sz < allctr->mbc_header_size + blk_sz) - crr_sz = allctr->mbc_header_size + blk_sz; -#ifdef ERTS_ALLOC_UTIL_HARD_DEBUG - if (allctr->mbc_header_size % sizeof(Unit_t) == 0) - crr_sz += sizeof(UWord); -#endif + if (crr_sz < MBC_HEADER_SIZE(allctr) + blk_sz) + crr_sz = MBC_HEADER_SIZE(allctr) + blk_sz; + mseg_flags = ERTS_MSEG_FLG_2POW; } - crr_sz = MSEG_UNIT_CEILING(crr_sz); - ASSERT(crr_sz % mseg_unit_size == 0); - crr = (Carrier_t *) alcu_mseg_alloc(allctr, &crr_sz); + crr = (Carrier_t *) alcu_mseg_alloc(allctr, &crr_sz, mseg_flags); if (!crr) { have_tried_mseg = 1; if (!(have_tried_sys_alloc || flags & CFLG_FORCE_MSEG)) @@ -2008,32 +2115,31 @@ create_carrier(Allctr_t *allctr, Uint umem_sz, UWord flags) is_mseg = 1; #endif if (flags & CFLG_SBC) { - SET_CARRIER_HDR(crr, crr_sz, SCH_MSEG|SCH_SBC); + SET_CARRIER_HDR(crr, crr_sz, SCH_MSEG|SCH_SBC, allctr); STAT_MSEG_SBC_ALLOC(allctr, crr_sz, blk_sz); goto sbc_final_touch; } else { - SET_CARRIER_HDR(crr, crr_sz, SCH_MSEG|SCH_MBC); +#ifndef ARCH_64 + ASSERT(crr_sz <= MBC_SZ_MAX_LIMIT); +#endif + SET_CARRIER_HDR(crr, crr_sz, SCH_MSEG|SCH_MBC, allctr); STAT_MSEG_MBC_ALLOC(allctr, crr_sz); goto mbc_final_touch; } try_sys_alloc: + #endif /* #if HAVE_ERTS_MSEG */ if (flags & CFLG_SBC) { - bcrr_sz = blk_sz + allctr->sbc_header_size; + bcrr_sz = blk_sz + SBC_HEADER_SIZE; } else { - bcrr_sz = allctr->mbc_header_size + blk_sz; + bcrr_sz = MBC_HEADER_SIZE(allctr) + blk_sz; if (!(flags & CFLG_MAIN_CARRIER) && bcrr_sz < allctr->smallest_mbc_size) bcrr_sz = allctr->smallest_mbc_size; -#ifdef ERTS_ALLOC_UTIL_HARD_DEBUG - if (allctr->mbc_header_size % sizeof(Unit_t) == 0) - bcrr_sz += sizeof(UWord); -#endif - } crr_sz = (flags & CFLG_FORCE_SIZE @@ -2057,7 +2163,7 @@ create_carrier(Allctr_t *allctr, Uint umem_sz, UWord flags) } } if (flags & CFLG_SBC) { - SET_CARRIER_HDR(crr, crr_sz, SCH_SYS_ALLOC|SCH_SBC); + SET_CARRIER_HDR(crr, crr_sz, SCH_SYS_ALLOC|SCH_SBC, allctr); STAT_SYS_ALLOC_SBC_ALLOC(allctr, crr_sz, blk_sz); #if HAVE_ERTS_MSEG @@ -2066,8 +2172,7 @@ create_carrier(Allctr_t *allctr, Uint umem_sz, UWord flags) blk = SBC2BLK(allctr, crr); - SET_SBC_BLK_FTR(((UWord *) blk)[-1]); - SET_BLK_HDR(blk, blk_sz, SBH_THIS_ALLOCED|SBH_PREV_FREE|SBH_LAST_BLK); + SET_SBC_BLK_HDR(blk, blk_sz); link_carrier(&allctr->sbc_list, crr); @@ -2075,28 +2180,18 @@ create_carrier(Allctr_t *allctr, Uint umem_sz, UWord flags) } else { - SET_CARRIER_HDR(crr, crr_sz, SCH_SYS_ALLOC|SCH_MBC); + SET_CARRIER_HDR(crr, crr_sz, SCH_SYS_ALLOC|SCH_MBC, allctr); STAT_SYS_ALLOC_MBC_ALLOC(allctr, crr_sz); #if HAVE_ERTS_MSEG mbc_final_touch: #endif - blk = MBC2FBLK(allctr, crr); - -#ifdef ERTS_ALLOC_UTIL_HARD_DEBUG - if (allctr->mbc_header_size % sizeof(Unit_t) == 0) - crr_sz -= sizeof(UWord); -#endif - - blk_sz = UNIT_FLOOR(crr_sz - allctr->mbc_header_size); + blk = MBC_TO_FIRST_BLK(allctr, crr); - SET_MBC_BLK_FTR(((UWord *) blk)[-1]); - SET_BLK_HDR(blk, blk_sz, SBH_THIS_FREE|SBH_PREV_FREE|SBH_LAST_BLK); + blk_sz = UNIT_FLOOR(crr_sz - MBC_HEADER_SIZE(allctr)); -#ifdef ERTS_ALLOC_UTIL_HARD_DEBUG - *((Carrier_t **) NXT_BLK(blk)) = crr; -#endif + SET_MBC_FBLK_HDR(blk, blk_sz, SBH_THIS_FREE|SBH_LAST_BLK, crr); if (flags & CFLG_MAIN_CARRIER) { ASSERT(!allctr->main_carrier); @@ -2105,15 +2200,7 @@ create_carrier(Allctr_t *allctr, Uint umem_sz, UWord flags) link_carrier(&allctr->mbc_list, crr); -#ifdef ERTS_ALLOC_UTIL_HARD_DEBUG - if (allctr->mbc_header_size % sizeof(Unit_t) == 0) - crr_sz += sizeof(UWord); -#endif CHECK_1BLK_CARRIER(allctr, 0, is_mseg, crr, crr_sz, blk, blk_sz); -#ifdef ERTS_ALLOC_UTIL_HARD_DEBUG - if (allctr->mbc_header_size % sizeof(Unit_t) == 0) - crr_sz -= sizeof(UWord); -#endif if (allctr->creating_mbc) (*allctr->creating_mbc)(allctr, crr, 0); @@ -2142,8 +2229,8 @@ resize_carrier(Allctr_t *allctr, Block_t *old_blk, Uint umem_sz, UWord flags) HARD_CHECK_BLK_CARRIER(allctr, old_blk); - old_blk_sz = BLK_SZ(old_blk); - old_crr = BLK2SBC(allctr, old_blk); + old_blk_sz = SBC_BLK_SZ(old_blk); + old_crr = BLK_TO_SBC(old_blk); old_crr_sz = CARRIER_SZ(old_crr); ASSERT(IS_SB_CARRIER(old_crr)); ASSERT(IS_SBC_BLK(old_blk)); @@ -2157,7 +2244,7 @@ resize_carrier(Allctr_t *allctr, Block_t *old_blk, Uint umem_sz, UWord flags) if (!(flags & CFLG_FORCE_SYS_ALLOC)) { - new_crr_sz = new_blk_sz + allctr->sbc_header_size; + new_crr_sz = new_blk_sz + SBC_HEADER_SIZE; new_crr_sz = MSEG_UNIT_CEILING(new_crr_sz); new_crr = (Carrier_t *) alcu_mseg_realloc(allctr, old_crr, @@ -2166,7 +2253,7 @@ resize_carrier(Allctr_t *allctr, Block_t *old_blk, Uint umem_sz, UWord flags) if (new_crr) { SET_CARRIER_SZ(new_crr, new_crr_sz); new_blk = SBC2BLK(allctr, new_crr); - SET_BLK_SZ(new_blk, new_blk_sz); + SET_SBC_BLK_SZ(new_blk, new_blk_sz); STAT_MSEG_SBC_ALLOC(allctr, new_crr_sz, new_blk_sz); relink_carrier(&allctr->sbc_list, new_crr); CHECK_1BLK_CARRIER(allctr, 1, 1, new_crr, new_crr_sz, @@ -2174,6 +2261,11 @@ resize_carrier(Allctr_t *allctr, Block_t *old_blk, Uint umem_sz, UWord flags) DEBUG_SAVE_ALIGNMENT(new_crr); return new_blk; } +#if HALFWORD_HEAP + /* Old carrier unchanged; restore stat */ + STAT_MSEG_SBC_ALLOC(allctr, old_crr_sz, old_blk_sz); + return NULL; +#endif create_flags |= CFLG_FORCE_SYS_ALLOC; /* since mseg_realloc() failed */ } @@ -2196,7 +2288,7 @@ resize_carrier(Allctr_t *allctr, Block_t *old_blk, Uint umem_sz, UWord flags) else { if (!(flags & CFLG_FORCE_MSEG)) { #endif /* #if HAVE_ERTS_MSEG */ - new_bcrr_sz = new_blk_sz + allctr->sbc_header_size; + new_bcrr_sz = new_blk_sz + SBC_HEADER_SIZE; new_crr_sz = (flags & CFLG_FORCE_SIZE ? UNIT_CEILING(new_bcrr_sz) : SYS_ALLOC_CARRIER_CEILING(new_bcrr_sz)); @@ -2208,7 +2300,7 @@ resize_carrier(Allctr_t *allctr, Block_t *old_blk, Uint umem_sz, UWord flags) sys_realloc_success: SET_CARRIER_SZ(new_crr, new_crr_sz); new_blk = SBC2BLK(allctr, new_crr); - SET_BLK_SZ(new_blk, new_blk_sz); + SET_SBC_BLK_SZ(new_blk, new_blk_sz); STAT_SYS_ALLOC_SBC_FREE(allctr, old_crr_sz, old_blk_sz); STAT_SYS_ALLOC_SBC_ALLOC(allctr, new_crr_sz, new_blk_sz); relink_carrier(&allctr->sbc_list, new_crr); @@ -2218,7 +2310,7 @@ resize_carrier(Allctr_t *allctr, Block_t *old_blk, Uint umem_sz, UWord flags) return new_blk; } else if (new_crr_sz > UNIT_CEILING(new_bcrr_sz)) { - new_crr_sz = new_blk_sz + allctr->sbc_header_size; + new_crr_sz = new_blk_sz + SBC_HEADER_SIZE; new_crr_sz = UNIT_CEILING(new_crr_sz); new_crr = (Carrier_t *) alcu_sys_realloc(allctr, (void *) old_crr, @@ -2262,11 +2354,9 @@ destroy_carrier(Allctr_t *allctr, Block_t *blk) Uint is_mseg = 0; #endif - ASSERT(IS_FIRST_BLK(blk)); - if (IS_SBC_BLK(blk)) { - Uint blk_sz = BLK_SZ(blk); - crr = BLK2SBC(allctr, blk); + Uint blk_sz = SBC_BLK_SZ(blk); + crr = BLK_TO_SBC(blk); crr_sz = CARRIER_SZ(crr); ASSERT(IS_LAST_BLK(blk)); @@ -2276,7 +2366,7 @@ destroy_carrier(Allctr_t *allctr, Block_t *blk) #if HAVE_ERTS_MSEG if (IS_MSEG_CARRIER(crr)) { is_mseg++; - ASSERT(crr_sz % mseg_unit_size == 0); + ASSERT(crr_sz % MSEG_UNIT_SZ == 0); STAT_MSEG_SBC_FREE(allctr, crr_sz, blk_sz); } else @@ -2287,7 +2377,8 @@ destroy_carrier(Allctr_t *allctr, Block_t *blk) } else { - crr = FBLK2MBC(allctr, blk); + ASSERT(IS_MBC_FIRST_FBLK(allctr, blk)); + crr = FIRST_BLK_TO_MBC(allctr, blk); crr_sz = CARRIER_SZ(crr); #ifdef DEBUG @@ -2305,7 +2396,7 @@ destroy_carrier(Allctr_t *allctr, Block_t *blk) #if HAVE_ERTS_MSEG if (IS_MSEG_CARRIER(crr)) { is_mseg++; - ASSERT(crr_sz % mseg_unit_size == 0); + ASSERT(crr_sz % MSEG_UNIT_SZ == 0); STAT_MSEG_MBC_FREE(allctr, crr_sz); } else @@ -3430,12 +3521,7 @@ do_erts_alcu_alloc(ErtsAlcType_t type, void *extra, Uint size) if (allctr->dd.use) ERTS_ALCU_HANDLE_DD_IN_OP(allctr, 1); #endif -#if HALFWORD_HEAP - blk = create_carrier(allctr, size, - CFLG_SBC | CFLG_FORCE_MSEG); -#else blk = create_carrier(allctr, size, CFLG_SBC); -#endif res = blk ? BLK2UMEM(blk) : NULL; } else @@ -3506,24 +3592,20 @@ erts_alcu_alloc_thr_spec(ErtsAlcType_t type, void *extra, Uint size) void * erts_alcu_alloc_thr_pref(ErtsAlcType_t type, void *extra, Uint size) { - int pref_ix; Allctr_t *pref_allctr; void *res; - pref_ix = get_pref_allctr(extra, &pref_allctr); + pref_allctr = get_pref_allctr(extra); if (pref_allctr->thread_safe) erts_mtx_lock(&pref_allctr->mutex); ERTS_ALCU_DBG_CHK_THR_ACCESS(pref_allctr); - res = do_erts_alcu_alloc(type, pref_allctr, size + sizeof(UWord)); + res = do_erts_alcu_alloc(type, pref_allctr, size); if (pref_allctr->thread_safe) erts_mtx_unlock(&pref_allctr->mutex); - if (res) - res = put_used_allctr(res, pref_ix, size); - DEBUG_CHECK_ALIGNMENT(res); @@ -3644,21 +3726,20 @@ erts_alcu_free_thr_pref(ErtsAlcType_t type, void *extra, void *p) { if (p) { Allctr_t *pref_allctr, *used_allctr; - void *ptr; - get_pref_allctr(extra, &pref_allctr); - ptr = get_used_allctr(extra, p, &used_allctr, NULL); + pref_allctr = get_pref_allctr(extra); + used_allctr = get_used_allctr(extra, p, NULL); if (pref_allctr != used_allctr) enqueue_dealloc_other_instance(type, used_allctr, - ptr, + p, (used_allctr->dd.ix - pref_allctr->dd.ix)); else { if (used_allctr->thread_safe) erts_mtx_lock(&used_allctr->mutex); ERTS_ALCU_DBG_CHK_THR_ACCESS(used_allctr); - do_erts_alcu_free(type, used_allctr, ptr); + do_erts_alcu_free(type, used_allctr, p); if (used_allctr->thread_safe) erts_mtx_unlock(&used_allctr->mutex); } @@ -3739,13 +3820,13 @@ do_erts_alcu_realloc(ErtsAlcType_t type, if (IS_MBC_BLK(blk)) res = mbc_realloc(allctr, p, size, alcu_flgs); else { - Uint used_sz = allctr->sbc_header_size + ABLK_HDR_SZ + size; + Uint used_sz = SBC_HEADER_SIZE + ABLK_HDR_SZ + size; Uint crr_sz; Uint diff_sz_val; Uint crr_sz_val; #if HAVE_ERTS_MSEG - if (IS_SYS_ALLOC_CARRIER(BLK2SBC(allctr, blk))) + if (IS_SYS_ALLOC_CARRIER(BLK_TO_SBC(blk))) #endif crr_sz = SYS_ALLOC_CARRIER_CEILING(used_sz); #if HAVE_ERTS_MSEG @@ -3775,7 +3856,7 @@ do_erts_alcu_realloc(ErtsAlcType_t type, if (res) { sys_memcpy((void*) res, (void*) p, - MIN(BLK_SZ(blk) - ABLK_HDR_SZ, size)); + MIN(SBC_BLK_SZ(blk) - ABLK_HDR_SZ, size)); destroy_carrier(allctr, blk); } } @@ -3798,16 +3879,12 @@ do_erts_alcu_realloc(ErtsAlcType_t type, else if (alcu_flgs & ERTS_ALCU_FLG_FAIL_REALLOC_MOVE) return NULL; else { -#if HALFWORD_HEAP - new_blk = create_carrier(allctr, size, CFLG_SBC | CFLG_FORCE_MSEG); -#else new_blk = create_carrier(allctr, size, CFLG_SBC); -#endif if (new_blk) { res = BLK2UMEM(new_blk); sys_memcpy((void *) res, (void *) p, - MIN(BLK_SZ(blk) - ABLK_HDR_SZ, size)); + MIN(MBC_ABLK_SZ(blk) - ABLK_HDR_SZ, size)); mbc_free(allctr, p); } else @@ -3966,16 +4043,15 @@ static ERTS_INLINE void * realloc_thr_pref(ErtsAlcType_t type, void *extra, void *p, Uint size, int force_move) { - int pref_ix; - void *ptr, *res; + void *res; Allctr_t *pref_allctr, *used_allctr; UWord old_user_size; if (!p) return erts_alcu_alloc_thr_pref(type, extra, size); - pref_ix = get_pref_allctr(extra, &pref_allctr); - ptr = get_used_allctr(extra, p, &used_allctr, &old_user_size); + pref_allctr = get_pref_allctr(extra); + used_allctr = get_used_allctr(extra, p, &old_user_size); ASSERT(used_allctr && pref_allctr); @@ -3985,56 +4061,33 @@ realloc_thr_pref(ErtsAlcType_t type, void *extra, void *p, Uint size, ERTS_ALCU_DBG_CHK_THR_ACCESS(used_allctr); res = do_erts_alcu_realloc(type, used_allctr, - ptr, - size + sizeof(UWord), + p, + size, 0); if (used_allctr->thread_safe) erts_mtx_unlock(&used_allctr->mutex); - if (res) - res = put_used_allctr(res, pref_ix, size); } else { if (pref_allctr->thread_safe) erts_mtx_lock(&pref_allctr->mutex); - res = do_erts_alcu_alloc(type, pref_allctr, size + sizeof(UWord)); - if (pref_allctr->thread_safe && (!force_move - || used_allctr != pref_allctr)) + res = do_erts_alcu_alloc(type, pref_allctr, size); + if (pref_allctr->thread_safe && used_allctr != pref_allctr) { erts_mtx_unlock(&pref_allctr->mutex); + } if (res) { - Block_t *blk; - size_t cpy_size; - - res = put_used_allctr(res, pref_ix, size); - DEBUG_CHECK_ALIGNMENT(res); - blk = UMEM2BLK(ptr); - if (old_user_size != ERTS_AU_PREF_ALLOC_SIZE_MASK) - cpy_size = old_user_size; - else { - if (used_allctr->thread_safe && (!force_move - || used_allctr != pref_allctr)) - erts_mtx_lock(&used_allctr->mutex); - ERTS_SMP_LC_ASSERT(!used_allctr->thread_safe || - erts_lc_mtx_is_locked(&used_allctr->mutex)); - cpy_size = BLK_SZ(blk); - if (used_allctr->thread_safe && (!force_move - || used_allctr != pref_allctr)) - erts_mtx_unlock(&used_allctr->mutex); - cpy_size -= ABLK_HDR_SZ + sizeof(UWord); - } - if (cpy_size > size) - cpy_size = size; - sys_memcpy(res, p, cpy_size); + sys_memcpy(res, p, MIN(size,old_user_size)); - if (!force_move || used_allctr != pref_allctr) + if (used_allctr != pref_allctr) { enqueue_dealloc_other_instance(type, used_allctr, - ptr, + p, (used_allctr->dd.ix - pref_allctr->dd.ix)); + } else { - do_erts_alcu_free(type, used_allctr, ptr); + do_erts_alcu_free(type, used_allctr, p); ASSERT(pref_allctr == used_allctr); if (pref_allctr->thread_safe) erts_mtx_unlock(&pref_allctr->mutex); @@ -4111,7 +4164,7 @@ erts_alcu_start(Allctr_t *allctr, AllctrInit_t *init) allctr->ramv = init->ramv; allctr->main_carrier_size = init->mmbcs; - allctr->sbc_threshold = init->sbct; + #if HAVE_ERTS_MSEG allctr->mseg_opt.abs_shrink_th = init->asbcst; allctr->mseg_opt.rel_shrink_th = init->rsbcst; @@ -4120,20 +4173,29 @@ erts_alcu_start(Allctr_t *allctr, AllctrInit_t *init) allctr->mbc_move_threshold = init->rmbcmt; #if HAVE_ERTS_MSEG allctr->max_mseg_sbcs = init->mmsbc; +# if HAVE_SUPER_ALIGNED_MB_CARRIERS + allctr->max_mseg_mbcs = ~(Uint)0; +# else allctr->max_mseg_mbcs = init->mmmbc; +# endif #endif allctr->largest_mbc_size = MAX(init->lmbcs, init->smbcs); +#ifndef ARCH_64 + if (allctr->largest_mbc_size > MBC_SZ_MAX_LIMIT) { + allctr->largest_mbc_size = MBC_SZ_MAX_LIMIT; + } +#endif allctr->smallest_mbc_size = init->smbcs; allctr->mbc_growth_stages = MAX(1, init->mbcgs); if (allctr->min_block_size < ABLK_HDR_SZ) goto error; allctr->min_block_size = UNIT_CEILING(allctr->min_block_size - + sizeof(UWord)); + + sizeof(FreeBlkFtr_t)); #if ERTS_SMP if (init->tpref) { - Uint sz = sizeof(Block_t); + Uint sz = ABLK_HDR_SZ; sz += ERTS_ALCU_DD_FIX_TYPE_OFFS*sizeof(UWord); if (init->fix) sz += sizeof(UWord); @@ -4143,6 +4205,23 @@ erts_alcu_start(Allctr_t *allctr, AllctrInit_t *init) } #endif + allctr->sbc_threshold = init->sbct; +#ifndef ARCH_64 + if (allctr->sbc_threshold > 0) { + Uint max_mbc_block_sz = UNIT_CEILING(allctr->sbc_threshold - 1 + ABLK_HDR_SZ); + if (max_mbc_block_sz + UNIT_FLOOR(allctr->min_block_size - 1) > MBC_ABLK_SZ_MASK + || max_mbc_block_sz < allctr->sbc_threshold) { /* wrap around */ + /* + * By limiting sbc_threshold to (hard limit - min_block_size) + * we avoid having to split off free "residue blocks" + * smaller than min_block_size. + */ + max_mbc_block_sz = MBC_ABLK_SZ_MASK - UNIT_FLOOR(allctr->min_block_size - 1); + allctr->sbc_threshold = max_mbc_block_sz - ABLK_HDR_SZ + 1; + } + } +#endif + allctr->sbmbc_threshold = init->sbmbct; @@ -4158,7 +4237,7 @@ erts_alcu_start(Allctr_t *allctr, AllctrInit_t *init) allctr->sbmbc_size = init->sbmbcs; min_size = allctr->sbmbc_threshold; min_size += allctr->min_block_size; - min_size += allctr->mbc_header_size; + min_size += MBC_HEADER_SIZE(allctr); if (allctr->sbmbc_size < min_size) allctr->sbmbc_size = min_size; } @@ -4203,59 +4282,26 @@ erts_alcu_start(Allctr_t *allctr, AllctrInit_t *init) if (!allctr->get_next_mbc_size) allctr->get_next_mbc_size = get_next_mbc_size; - if (allctr->mbc_header_size < sizeof(Carrier_t)) - goto error; #ifdef ERTS_SMP allctr->dd.use = 0; if (init->tpref) { - allctr->mbc_header_size = (UNIT_CEILING(allctr->mbc_header_size - + FBLK_FTR_SZ - + ABLK_HDR_SZ - + sizeof(UWord)) - - ABLK_HDR_SZ - - sizeof(UWord)); - allctr->sbc_header_size = (UNIT_CEILING(sizeof(Carrier_t) - + FBLK_FTR_SZ - + ABLK_HDR_SZ - + sizeof(UWord)) - - ABLK_HDR_SZ - - sizeof(UWord)); - allctr->dd.use = 1; init_dd_queue(&allctr->dd.q); allctr->dd.ix = init->ix; } - else #endif - { - allctr->mbc_header_size = (UNIT_CEILING(allctr->mbc_header_size - + FBLK_FTR_SZ - + ABLK_HDR_SZ) - - ABLK_HDR_SZ); - allctr->sbc_header_size = (UNIT_CEILING(sizeof(Carrier_t) - + FBLK_FTR_SZ - + ABLK_HDR_SZ) - - ABLK_HDR_SZ); - } if (allctr->main_carrier_size) { Block_t *blk; -#if HALFWORD_HEAP - blk = create_carrier(allctr, - allctr->main_carrier_size, - CFLG_MBC - | CFLG_FORCE_SIZE - | CFLG_FORCE_MSEG - | CFLG_MAIN_CARRIER); -#else blk = create_carrier(allctr, allctr->main_carrier_size, CFLG_MBC | CFLG_FORCE_SIZE +#if !HALFWORD_HEAP && !HAVE_SUPER_ALIGNED_MB_CARRIERS | CFLG_FORCE_SYS_ALLOC - | CFLG_MAIN_CARRIER); #endif + | CFLG_MAIN_CARRIER); if (!blk) goto error; @@ -4303,9 +4349,9 @@ erts_alcu_stop(Allctr_t *allctr) while (allctr->sbc_list.first) destroy_carrier(allctr, SBC2BLK(allctr, allctr->sbc_list.first)); while (allctr->mbc_list.first) - destroy_carrier(allctr, MBC2FBLK(allctr, allctr->mbc_list.first)); + destroy_carrier(allctr, MBC_TO_FIRST_BLK(allctr, allctr->mbc_list.first)); while (allctr->sbmbc_list.first) - destroy_sbmbc(allctr, MBC2FBLK(allctr, allctr->sbmbc_list.first)); + destroy_sbmbc(allctr, MBC_TO_FIRST_BLK(allctr, allctr->sbmbc_list.first)); #ifdef USE_THREADS if (allctr->thread_safe) @@ -4319,17 +4365,9 @@ erts_alcu_stop(Allctr_t *allctr) void erts_alcu_init(AlcUInit_t *init) { - + ASSERT(SBC_BLK_SZ_MASK == MBC_FBLK_SZ_MASK); /* see BLK_SZ */ #if HAVE_ERTS_MSEG - mseg_unit_size = erts_mseg_unit_size(); - - if (mseg_unit_size % sizeof(Unit_t)) /* A little paranoid... */ - erl_exit(-1, - "Mseg unit size (%d) not evenly divideble by " - "internal unit size of alloc_util (%d)\n", - mseg_unit_size, - sizeof(Unit_t)); - + ASSERT(erts_mseg_unit_size() == MSEG_UNIT_SZ); max_mseg_carriers = init->mmc; sys_alloc_carrier_size = MSEG_UNIT_CEILING(init->ycs); #else /* #if HAVE_ERTS_MSEG */ @@ -4372,11 +4410,10 @@ erts_alcu_test(unsigned long op, unsigned long a1, unsigned long a2) case 0x00b: return (unsigned long) CARRIER_SZ((Carrier_t *) a1); case 0x00c: return (unsigned long) SBC2BLK((Allctr_t *) a1, (Carrier_t *) a2); - case 0x00d: return (unsigned long) BLK2SBC((Allctr_t *) a1, - (Block_t *) a2); - case 0x00e: return (unsigned long) MBC2FBLK((Allctr_t *) a1, + case 0x00d: return (unsigned long) BLK_TO_SBC((Block_t *) a2); + case 0x00e: return (unsigned long) MBC_TO_FIRST_BLK((Allctr_t *) a1, (Carrier_t *) a2); - case 0x00f: return (unsigned long) FBLK2MBC((Allctr_t *) a1, + case 0x00f: return (unsigned long) FIRST_BLK_TO_MBC((Allctr_t *) a1, (Block_t *) a2); case 0x010: return (unsigned long) ((Allctr_t *) a1)->mbc_list.first; case 0x011: return (unsigned long) ((Allctr_t *) a1)->mbc_list.last; @@ -4388,7 +4425,7 @@ erts_alcu_test(unsigned long op, unsigned long a1, unsigned long a2) case 0x017: return (unsigned long) ((Allctr_t *) a1)->min_block_size; case 0x018: return (unsigned long) NXT_BLK((Block_t *) a1); case 0x019: return (unsigned long) PREV_BLK((Block_t *) a1); - case 0x01a: return (unsigned long) IS_FIRST_BLK((Block_t *) a1); + case 0x01a: return (unsigned long) IS_MBC_FIRST_BLK((Allctr_t*)a1, (Block_t *) a2); case 0x01b: return (unsigned long) sizeof(Unit_t); default: ASSERT(0); return ~((unsigned long) 0); } @@ -4432,6 +4469,13 @@ erts_alcu_verify_unused_ts(Allctr_t *allctr) #endif } +#ifdef DEBUG +int is_sbc_blk(Block_t* blk) +{ + return IS_SBC_BLK(blk); +} +#endif + #ifdef ERTS_ALLOC_UTIL_HARD_DEBUG static void @@ -4441,34 +4485,37 @@ check_blk_carrier(Allctr_t *allctr, Block_t *iblk) CarrierList_t *cl; if (IS_SBC_BLK(iblk)) { - Carrier_t *sbc = BLK2SBC(allctr, iblk); + Carrier_t *sbc = BLK_TO_SBC(iblk); ASSERT(SBC2BLK(allctr, sbc) == iblk); - ASSERT(IS_ALLOCED_BLK(iblk)); - ASSERT(IS_FIRST_BLK(iblk)); - ASSERT(IS_LAST_BLK(iblk)); - ASSERT(CARRIER_SZ(sbc) - allctr->sbc_header_size >= BLK_SZ(iblk)); + ASSERT(CARRIER_SZ(sbc) - SBC_HEADER_SIZE >= SBC_BLK_SZ(iblk)); #if HAVE_ERTS_MSEG if (IS_MSEG_CARRIER(sbc)) { - ASSERT(CARRIER_SZ(sbc) % mseg_unit_size == 0); + ASSERT(CARRIER_SZ(sbc) % MSEG_UNIT_SZ == 0); } #endif crr = sbc; cl = &allctr->sbc_list; } else { - Carrier_t *mbc = NULL; Block_t *prev_blk = NULL; Block_t *blk; char *carrier_end; Uint is_free_blk; Uint tot_blk_sz; Uint blk_sz; + int has_wrapped_around = 0; blk = iblk; tot_blk_sz = 0; + crr = BLK_TO_MBC(blk); + ASSERT(IS_MB_CARRIER(crr)); + /* Step around the carrier one whole lap starting at 'iblk' + */ while (1) { + ASSERT(IS_MBC_BLK(blk)); + ASSERT(BLK_TO_MBC(blk) == crr); if (prev_blk) { ASSERT(NXT_BLK(prev_blk) == blk); @@ -4481,18 +4528,16 @@ check_blk_carrier(Allctr_t *allctr, Block_t *iblk) } } - if (mbc) { + if (has_wrapped_around) { + ASSERT(((Block_t *) crr) < blk); if (blk == iblk) break; - ASSERT(((Block_t *) mbc) < blk && blk < iblk); + ASSERT(blk < iblk); } else ASSERT(blk >= iblk); - - ASSERT(IS_MBC_BLK(blk)); - - blk_sz = BLK_SZ(blk); + blk_sz = MBC_BLK_SZ(blk); ASSERT(blk_sz % sizeof(Unit_t) == 0); ASSERT(blk_sz >= allctr->min_block_size); @@ -4500,44 +4545,40 @@ check_blk_carrier(Allctr_t *allctr, Block_t *iblk) tot_blk_sz += blk_sz; is_free_blk = (int) IS_FREE_BLK(blk); - if(is_free_blk) { - if (IS_NOT_LAST_BLK(blk)) - ASSERT(*((UWord *) (((char *) blk)+blk_sz-sizeof(UWord))) - == blk_sz); - } + ASSERT(!is_free_blk + || IS_LAST_BLK(blk) + || PREV_BLK_SZ(((char *) blk)+blk_sz) == blk_sz); if (allctr->check_block) (*allctr->check_block)(allctr, blk, (int) is_free_blk); if (IS_LAST_BLK(blk)) { carrier_end = ((char *) NXT_BLK(blk)); - mbc = *((Carrier_t **) NXT_BLK(blk)); + has_wrapped_around = 1; prev_blk = NULL; - blk = MBC2FBLK(allctr, mbc); - ASSERT(IS_FIRST_BLK(blk)); + blk = MBC_TO_FIRST_BLK(allctr, crr); + ASSERT(IS_MBC_FIRST_BLK(allctr,blk)); } else { prev_blk = blk; blk = NXT_BLK(blk); } } - - ASSERT(IS_MB_CARRIER(mbc)); - ASSERT((((char *) mbc) - + allctr->mbc_header_size + + ASSERT((((char *) crr) + + MBC_HEADER_SIZE(allctr) + tot_blk_sz) == carrier_end); - ASSERT(((char *) mbc) + CARRIER_SZ(mbc) - sizeof(Unit_t) <= carrier_end - && carrier_end <= ((char *) mbc) + CARRIER_SZ(mbc)); + ASSERT(((char *) crr) + CARRIER_SZ(crr) - sizeof(Unit_t) <= carrier_end + && carrier_end <= ((char *) crr) + CARRIER_SZ(crr)); if (allctr->check_mbc) - (*allctr->check_mbc)(allctr, mbc); + (*allctr->check_mbc)(allctr, crr); #if HAVE_ERTS_MSEG - if (IS_MSEG_CARRIER(mbc)) { - ASSERT(CARRIER_SZ(mbc) % mseg_unit_size == 0); + if (IS_MSEG_CARRIER(crr)) { + ASSERT(CARRIER_SZ(crr) % MSEG_UNIT_SZ == 0); } #endif - crr = mbc; cl = &allctr->mbc_list; } @@ -4559,4 +4600,5 @@ check_blk_carrier(Allctr_t *allctr, Block_t *iblk) #endif } -#endif +#endif /* ERTS_ALLOC_UTIL_HARD_DEBUG */ + diff --git a/erts/emulator/beam/erl_alloc_util.h b/erts/emulator/beam/erl_alloc_util.h index cedf4ccf85..e0754e7f69 100644 --- a/erts/emulator/beam/erl_alloc_util.h +++ b/erts/emulator/beam/erl_alloc_util.h @@ -216,16 +216,40 @@ erts_aint32_t erts_alcu_fix_alloc_shrink(Allctr_t *, erts_aint32_t); #define UNIT_FLOOR(X) ((X) & UNIT_MASK) #define UNIT_CEILING(X) UNIT_FLOOR((X) + INV_UNIT_MASK) +#define FLG_MASK INV_UNIT_MASK +#define SBC_BLK_SZ_MASK UNIT_MASK +#define MBC_FBLK_SZ_MASK UNIT_MASK +#define CARRIER_SZ_MASK UNIT_MASK -#define SZ_MASK (~((UWord) 0) << 3) -#define FLG_MASK (~(SZ_MASK)) +#if HAVE_ERTS_MSEG +# ifdef ARCH_64 +# define MBC_ABLK_OFFSET_BITS 24 +# elif HAVE_SUPER_ALIGNED_MB_CARRIERS +# define MBC_ABLK_OFFSET_BITS 9 + /* Affects hard limits for sbct and lmbcs documented in erts_alloc.xml */ +# endif +#endif +#ifndef MBC_ABLK_OFFSET_BITS +# define MBC_ABLK_OFFSET_BITS 0 /* no carrier offset in block header */ +#endif + +#if MBC_ABLK_OFFSET_BITS +# define MBC_ABLK_OFFSET_SHIFT (sizeof(UWord)*8 - MBC_ABLK_OFFSET_BITS) +# define MBC_ABLK_OFFSET_MASK (~((UWord)0) << MBC_ABLK_OFFSET_SHIFT) +# define MBC_ABLK_SZ_MASK (~MBC_ABLK_OFFSET_MASK & ~FLG_MASK) +# define HAVE_ERTS_SBMBC 0 +#else +# define MBC_ABLK_SZ_MASK (~FLG_MASK) +# define HAVE_ERTS_SBMBC 1 +#endif -#define BLK_SZ(B) \ - (*((Block_t *) (B)) & SZ_MASK) +#define MBC_ABLK_SZ(B) (ASSERT_EXPR(!is_sbc_blk(B)), (B)->bhdr & MBC_ABLK_SZ_MASK) +#define MBC_FBLK_SZ(B) (ASSERT_EXPR(!is_sbc_blk(B)), (B)->bhdr & MBC_FBLK_SZ_MASK) +#define SBC_BLK_SZ(B) (ASSERT_EXPR(is_sbc_blk(B)), (B)->bhdr & SBC_BLK_SZ_MASK) #define CARRIER_SZ(C) \ - ((C)->chdr & SZ_MASK) + ((C)->chdr & CARRIER_SZ_MASK) extern int erts_have_sbmbc_alloc; @@ -236,6 +260,7 @@ struct Carrier_t_ { UWord chdr; Carrier_t *next; Carrier_t *prev; + Allctr_t *allctr; }; typedef struct { @@ -243,8 +268,19 @@ typedef struct { Carrier_t *last; } CarrierList_t; -typedef UWord Block_t; -typedef UWord FreeBlkFtr_t; +typedef struct { + UWord bhdr; +#if !MBC_ABLK_OFFSET_BITS + Carrier_t *carrier; +#else + union { + Carrier_t *carrier; /* if free */ + char udata__[1]; /* if allocated */ + }u; +#endif +} Block_t; + +typedef UWord FreeBlkFtr_t; /* Footer of a free block */ typedef struct { UWord giga_no; @@ -381,8 +417,6 @@ struct Allctr_t_ { #endif /* */ - Uint mbc_header_size; - Uint sbc_header_size; Uint min_mbc_size; Uint min_mbc_first_free_size; Uint min_block_size; @@ -469,6 +503,9 @@ void erts_alcu_verify_unused_ts(Allctr_t *allctr); unsigned long erts_alcu_test(unsigned long, unsigned long, unsigned long); +#ifdef DEBUG +int is_sbc_blk(Block_t*); +#endif #endif /* #if defined(GET_ERL_ALLOC_UTIL_IMPL) diff --git a/erts/emulator/beam/erl_ao_firstfit_alloc.c b/erts/emulator/beam/erl_ao_firstfit_alloc.c index 5bdb752d3a..86b4696d8f 100644 --- a/erts/emulator/beam/erl_ao_firstfit_alloc.c +++ b/erts/emulator/beam/erl_ao_firstfit_alloc.c @@ -91,6 +91,7 @@ struct AOFF_RBTree_t_ { AOFF_RBTree_t *right; Uint max_sz; /* of all blocks in this sub-tree */ }; +#define AOFF_BLK_SZ(B) MBC_FBLK_SZ(&(B)->hdr) #ifdef HARD_DEBUG static AOFF_RBTree_t * check_tree(AOFF_RBTree_t* root, Uint); @@ -102,7 +103,7 @@ static AOFF_RBTree_t * check_tree(AOFF_RBTree_t* root, Uint); */ static ERTS_INLINE Uint node_max_size(AOFF_RBTree_t *x) { - Uint sz = BLK_SZ(x); + Uint sz = AOFF_BLK_SZ(x); if (x->left && x->left->max_sz > sz) { sz = x->left->max_sz; } @@ -183,7 +184,6 @@ erts_aoffalc_start(AOFFAllctr_t *alc, sys_memcpy((void *) alc, (void *) &zero.allctr, sizeof(AOFFAllctr_t)); - allctr->mbc_header_size = sizeof(Carrier_t); allctr->min_mbc_size = MIN_MBC_SZ; allctr->min_mbc_first_free_size = MIN_MBC_FIRST_FREE_SZ; allctr->min_block_size = sizeof(AOFF_RBTree_t); @@ -587,7 +587,7 @@ aoff_link_free_block(Allctr_t *allctr, Block_t *block, Uint32 flags) AOFF_RBTree_t *blk = (AOFF_RBTree_t *) block; AOFF_RBTree_t **root = ((flags & ERTS_ALCU_FLG_SBMBC) ? &alc->sbmbc_root : &alc->mbc_root); - Uint blk_sz = BLK_SZ(blk); + Uint blk_sz = AOFF_BLK_SZ(blk); #ifdef HARD_DEBUG check_tree(*root, 0); @@ -659,7 +659,7 @@ aoff_get_free_block(Allctr_t *allctr, Uint size, if (x->left && x->left->max_sz >= size) { x = x->left; } - else if (BLK_SZ(x) >= size) { + else if (AOFF_BLK_SZ(x) >= size) { blk = x; break; } @@ -910,12 +910,12 @@ check_tree(AOFF_RBTree_t* root, Uint size) ASSERT(x->right > x); ASSERT(x->right->max_sz <= x->max_sz); } - ASSERT(x->max_sz >= BLK_SZ(x)); - ASSERT(x->max_sz == BLK_SZ(x) + ASSERT(x->max_sz >= AOFF_BLK_SZ(x)); + ASSERT(x->max_sz == AOFF_BLK_SZ(x) || x->max_sz == (x->left ? x->left->max_sz : 0) || x->max_sz == (x->right ? x->right->max_sz : 0)); - if (size && BLK_SZ(x) >= size) { + if (size && AOFF_BLK_SZ(x) >= size) { if (!res || x < res) { res = x; } @@ -956,7 +956,7 @@ print_tree_aux(AOFF_RBTree_t *x, int indent) } fprintf(stderr, "%s: sz=%lu addr=0x%lx max_size=%lu\r\n", IS_BLACK(x) ? "BLACK" : "RED", - BLK_SZ(x), (Uint)x, x->max_sz); + AOFF_BLK_SZ(x), (Uint)x, x->max_sz); print_tree_aux(x->left, indent + INDENT_STEP); } } diff --git a/erts/emulator/beam/erl_async.c b/erts/emulator/beam/erl_async.c index c5f432bea1..f2ca193ace 100644 --- a/erts/emulator/beam/erl_async.c +++ b/erts/emulator/beam/erl_async.c @@ -382,10 +382,15 @@ static ERTS_INLINE ErtsAsync *async_get(ErtsThrQ_t *q, static ERTS_INLINE void call_async_ready(ErtsAsync *a) { +#if ERTS_USE_ASYNC_READY_Q Port *p = erts_id2port_sflgs(a->port, NULL, 0, ERTS_PORT_SFLGS_INVALID_DRIVER_LOOKUP); +#else + Port *p = erts_thr_id2port_sflgs(a->port, + ERTS_PORT_SFLGS_INVALID_DRIVER_LOOKUP); +#endif if (!p) { if (a->async_free) a->async_free(a->async_data); @@ -395,7 +400,11 @@ static ERTS_INLINE void call_async_ready(ErtsAsync *a) if (a->async_free) a->async_free(a->async_data); } +#if ERTS_USE_ASYNC_READY_Q erts_port_release(p); +#else + erts_thr_port_release(p); +#endif } if (a->pdl) driver_pdl_dec_refc(a->pdl); @@ -603,7 +612,7 @@ long driver_async(ErlDrvPort ix, unsigned int* key, sched_id = 1; #endif - prt = erts_drvport2port(ix); + prt = erts_drvport2port(ix, NULL); if (!prt) return -1; @@ -615,7 +624,7 @@ long driver_async(ErlDrvPort ix, unsigned int* key, a->sched_id = sched_id; #endif a->hndl = (DE_Handle*)prt->drv_ptr->handle; - a->port = prt->id; + a->port = prt->common.id; a->pdl = NULL; a->async_data = async_data; a->async_invoke = async_invoke; diff --git a/erts/emulator/beam/erl_bestfit_alloc.c b/erts/emulator/beam/erl_bestfit_alloc.c index c50fdeb4e8..743cbd93c9 100644 --- a/erts/emulator/beam/erl_bestfit_alloc.c +++ b/erts/emulator/beam/erl_bestfit_alloc.c @@ -73,6 +73,8 @@ #define SET_RED(N) (((RBTree_t *) (N))->flags |= RED_FLG) #define SET_BLACK(N) (((RBTree_t *) (N))->flags &= ~RED_FLG) +#define BF_BLK_SZ(B) MBC_FBLK_SZ(&(B)->hdr) + #undef ASSERT #define ASSERT ASSERT_EXPR @@ -177,7 +179,6 @@ erts_bfalc_start(BFAllctr_t *bfallctr, bfallctr->address_order = bfinit->ao; - allctr->mbc_header_size = sizeof(Carrier_t); allctr->min_mbc_size = MIN_MBC_SZ; allctr->min_mbc_first_free_size = MIN_MBC_FIRST_FREE_SZ; allctr->min_block_size = (bfinit->ao @@ -592,7 +593,7 @@ aobf_link_free_block(Allctr_t *allctr, Block_t *block, Uint32 flags) ? &bfallctr->sbmbc_root : &bfallctr->mbc_root); RBTree_t *blk = (RBTree_t *) block; - Uint blk_sz = BLK_SZ(blk); + Uint blk_sz = BF_BLK_SZ(blk); @@ -610,7 +611,7 @@ aobf_link_free_block(Allctr_t *allctr, Block_t *block, Uint32 flags) while (1) { Uint size; - size = BLK_SZ(x); + size = BF_BLK_SZ(x); if (blk_sz < size || (blk_sz == size && blk < x)) { if (!x->left) { @@ -668,7 +669,7 @@ aobf_get_free_block(Allctr_t *allctr, Uint size, ASSERT(!cand_blk || cand_size >= size); while (x) { - blk_sz = BLK_SZ(x); + blk_sz = BF_BLK_SZ(x); if (blk_sz < size) { x = x->right; } @@ -686,7 +687,7 @@ aobf_get_free_block(Allctr_t *allctr, Uint size, #endif if (cand_blk) { - blk_sz = BLK_SZ(blk); + blk_sz = BF_BLK_SZ(blk); if (cand_size < blk_sz) return NULL; /* cand_blk was better */ if (cand_size == blk_sz && ((void *) cand_blk) < ((void *) blk)) @@ -711,7 +712,7 @@ bf_link_free_block(Allctr_t *allctr, Block_t *block, Uint32 flags) ? &bfallctr->sbmbc_root : &bfallctr->mbc_root); RBTree_t *blk = (RBTree_t *) block; - Uint blk_sz = BLK_SZ(blk); + Uint blk_sz = BF_BLK_SZ(blk); SET_TREE_NODE(blk); @@ -730,7 +731,7 @@ bf_link_free_block(Allctr_t *allctr, Block_t *block, Uint32 flags) while (1) { Uint size; - size = BLK_SZ(x); + size = BF_BLK_SZ(x); if (blk_sz == size) { @@ -796,7 +797,7 @@ bf_unlink_free_block(Allctr_t *allctr, Block_t *block, Uint32 flags) else if (LIST_NEXT(x)) { /* Replace tree node by next element in list... */ - ASSERT(BLK_SZ(LIST_NEXT(x)) == BLK_SZ(x)); + ASSERT(BF_BLK_SZ(LIST_NEXT(x)) == BF_BLK_SZ(x)); ASSERT(IS_TREE_NODE(x)); ASSERT(IS_LIST_ELEM(LIST_NEXT(x))); @@ -834,7 +835,7 @@ bf_get_free_block(Allctr_t *allctr, Uint size, ASSERT(!cand_blk || cand_size >= size); while (x) { - blk_sz = BLK_SZ(x); + blk_sz = BF_BLK_SZ(x); if (blk_sz < size) { x = x->right; } @@ -855,11 +856,11 @@ bf_get_free_block(Allctr_t *allctr, Uint size, #ifdef HARD_DEBUG { RBTree_t *ct_blk = check_tree(root, 0, size); - ASSERT(BLK_SZ(ct_blk) == BLK_SZ(blk)); + ASSERT(BF_BLK_SZ(ct_blk) == BF_BLK_SZ(blk)); } #endif - if (cand_blk && cand_size <= BLK_SZ(blk)) + if (cand_blk && cand_size <= BF_BLK_SZ(blk)) return NULL; /* cand_blk was better */ /* Use next block if it exist in order to avoid replacing @@ -1093,36 +1094,36 @@ check_tree(RBTree_t *root, int ao, Uint size) if (x->left) { ASSERT(x->left->parent == x); if (ao) { - ASSERT(BLK_SZ(x->left) < BLK_SZ(x) - || (BLK_SZ(x->left) == BLK_SZ(x) && x->left < x)); + ASSERT(BF_BLK_SZ(x->left) < BF_BLK_SZ(x) + || (BF_BLK_SZ(x->left) == BF_BLK_SZ(x) && x->left < x)); } else { ASSERT(IS_TREE_NODE(x->left)); - ASSERT(BLK_SZ(x->left) < BLK_SZ(x)); + ASSERT(BF_BLK_SZ(x->left) < BF_BLK_SZ(x)); } } if (x->right) { ASSERT(x->right->parent == x); if (ao) { - ASSERT(BLK_SZ(x->right) > BLK_SZ(x) - || (BLK_SZ(x->right) == BLK_SZ(x) && x->right > x)); + ASSERT(BF_BLK_SZ(x->right) > BF_BLK_SZ(x) + || (BF_BLK_SZ(x->right) == BF_BLK_SZ(x) && x->right > x)); } else { ASSERT(IS_TREE_NODE(x->right)); - ASSERT(BLK_SZ(x->right) > BLK_SZ(x)); + ASSERT(BF_BLK_SZ(x->right) > BF_BLK_SZ(x)); } } - if (size && BLK_SZ(x) >= size) { + if (size && BF_BLK_SZ(x) >= size) { if (ao) { if (!res - || BLK_SZ(x) < BLK_SZ(res) - || (BLK_SZ(x) == BLK_SZ(res) && x < res)) + || BF_BLK_SZ(x) < BF_BLK_SZ(res) + || (BF_BLK_SZ(x) == BF_BLK_SZ(res) && x < res)) res = x; } else { - if (!res || BLK_SZ(x) < BLK_SZ(res)) + if (!res || BF_BLK_SZ(x) < BF_BLK_SZ(res)) res = x; } } @@ -1168,7 +1169,7 @@ print_tree_aux(RBTree_t *x, int indent) } fprintf(stderr, "%s: sz=%lu addr=0x%lx\r\n", IS_BLACK(x) ? "BLACK" : "RED", - BLK_SZ(x), + BF_BLK_SZ(x), (Uint) x); print_tree_aux(x->left, indent + INDENT_STEP); } diff --git a/erts/emulator/beam/erl_bif_ddll.c b/erts/emulator/beam/erl_bif_ddll.c index 7f7c975e78..7cbea55eac 100644 --- a/erts/emulator/beam/erl_bif_ddll.c +++ b/erts/emulator/beam/erl_bif_ddll.c @@ -104,16 +104,49 @@ static void dereference_all_processes(DE_Handle *dh); static void restore_process_references(DE_Handle *dh); static void ddll_no_more_references(void *vdh); -#define lock_drv_list() erts_smp_mtx_lock(&erts_driver_list_lock) -#define unlock_drv_list() erts_smp_mtx_unlock(&erts_driver_list_lock) +#define lock_drv_list() erts_smp_rwmtx_rwlock(&erts_driver_list_lock) +#define unlock_drv_list() erts_smp_rwmtx_rwunlock(&erts_driver_list_lock) #define assert_drv_list_locked() \ - ERTS_SMP_LC_ASSERT(erts_smp_lc_mtx_is_locked(&erts_driver_list_lock)) + ERTS_SMP_LC_ASSERT(erts_smp_lc_rwmtx_is_rwlocked(&erts_driver_list_lock) \ + || erts_smp_lc_rwmtx_is_rlocked(&erts_driver_list_lock)) +#define assert_drv_list_rwlocked() \ + ERTS_SMP_LC_ASSERT(erts_smp_lc_rwmtx_is_rwlocked(&erts_driver_list_lock)) +#define assert_drv_list_rlocked() \ + ERTS_SMP_LC_ASSERT(erts_smp_lc_rwmtx_is_rlocked(&erts_driver_list_lock)) #define assert_drv_list_not_locked() \ - ERTS_SMP_LC_ASSERT(!erts_smp_lc_mtx_is_locked(&erts_driver_list_lock)) + ERTS_SMP_LC_ASSERT(!erts_smp_lc_rwmtx_is_rwlocked(&erts_driver_list_lock) \ + && !erts_smp_lc_rwmtx_is_rlocked(&erts_driver_list_lock)) #define FREE_PORT_FLAGS (ERTS_PORT_SFLGS_DEAD & (~ERTS_PORT_SFLG_INITIALIZING)) +static void +kill_ports_driver_unloaded(DE_Handle *dh) +{ + int ix, max = erts_ptab_max(&erts_port); + + for (ix = 0; ix < max; ix++) { + erts_aint32_t state; + Port* prt = erts_pix2port(ix); + if (!prt) + continue; + + ERTS_SMP_DATA_DEPENDENCY_READ_MEMORY_BARRIER; + + state = erts_atomic32_read_nob(&prt->state); + if (state & FREE_PORT_FLAGS) + continue; + + erts_smp_port_lock(prt); + + state = erts_atomic32_read_nob(&prt->state); + if (!(state & ERTS_PORT_SFLGS_DEAD) && prt->drv_ptr->handle == dh) + driver_failure_atom((ErlDrvPort) prt, "driver_unloaded"); + + erts_port_release(prt); + } +} + /* * try_load(Path, Name, OptionList) -> {ok,Status} | * {ok, PendingStatus, Ref} | @@ -149,7 +182,7 @@ BIF_RETTYPE erl_ddll_try_load_3(BIF_ALIST_3) Eterm name_term = BIF_ARG_2; Eterm options = BIF_ARG_3; char *path = NULL; - Uint path_len; + ErlDrvSizeT path_len; char *name = NULL; DE_Handle *dh; erts_driver_t *drv; @@ -228,7 +261,7 @@ BIF_RETTYPE erl_ddll_try_load_3(BIF_ALIST_3) goto error; } path = erts_alloc(ERTS_ALC_T_DDLL_TMP_BUF, path_len + 1 /* might need path separator */ + sys_strlen(name) + 1); - if (io_list_to_buf(path_term, path, path_len) != 0) { + if (erts_iolist_to_buf(path_term, path, path_len) != 0) { goto error; } while (path_len > 0 && (path[path_len-1] == '\\' || path[path_len-1] == '/')) { @@ -356,40 +389,16 @@ BIF_RETTYPE erl_ddll_try_load_3(BIF_ALIST_3) ok_term = mkatom("loaded"); } } - assert_drv_list_locked(); + assert_drv_list_rwlocked(); if (kill_ports) { - int j; - /* Avoid closing the driver by referencing it */ + /* Avoid closing the driver by referencing it */ erts_ddll_reference_driver(dh); ASSERT(dh->status == ERL_DE_RELOAD); dh->status = ERL_DE_FORCE_RELOAD; #if DDLL_SMP unlock_drv_list(); #endif - for (j = 0; j < erts_max_ports; j++) { - Port* prt = &erts_port[j]; - erts_smp_port_state_lock(prt); - if (!(prt->status & FREE_PORT_FLAGS) && - prt->drv_ptr->handle == dh) { - erts_smp_atomic_inc_nob(&prt->refc); -#if DDLL_SMP - /* Extremely rare spinlock */ - while(prt->status & ERTS_PORT_SFLG_INITIALIZING) { - erts_smp_port_state_unlock(prt); - erts_smp_port_state_lock(prt); - } - erts_smp_port_state_unlock(prt); - erts_smp_mtx_lock(prt->lock); - if (!(prt->status & ERTS_PORT_SFLGS_DEAD)) { - driver_failure_atom(j, "driver_unloaded"); - } -#else - driver_failure_atom(j, "driver_unloaded"); -#endif - erts_port_release(prt); - } - else erts_smp_port_state_unlock(prt); - } + kill_ports_driver_unloaded(dh); /* Dereference, eventually causing driver destruction */ #if DDLL_SMP lock_drv_list(); @@ -579,45 +588,21 @@ Eterm erl_ddll_try_unload_2(BIF_ALIST_2) dh->reload_full_path = dh->reload_driver_name = NULL; dh->reload_flags = 0; } - if (dh->port_count > 0) { + if (erts_smp_atomic32_read_nob(&dh->port_count) > 0) { ++kill_ports; } dh->status = ERL_DE_UNLOAD; ok_term = am_pending_driver; done: - assert_drv_list_locked(); + assert_drv_list_rwlocked(); if (kill_ports > 1) { - int j; /* Avoid closing the driver by referencing it */ erts_ddll_reference_driver(dh); dh->status = ERL_DE_FORCE_UNLOAD; #if DDLL_SMP unlock_drv_list(); #endif - for (j = 0; j < erts_max_ports; j++) { - Port* prt = &erts_port[j]; - erts_smp_port_state_lock(prt); - if (!(prt->status & FREE_PORT_FLAGS) - && prt->drv_ptr->handle == dh) { - erts_smp_atomic_inc_nob(&prt->refc); -#if DDLL_SMP - /* Extremely rare spinlock */ - while(prt->status & ERTS_PORT_SFLG_INITIALIZING) { - erts_smp_port_state_unlock(prt); - erts_smp_port_state_lock(prt); - } - erts_smp_port_state_unlock(prt); - erts_smp_mtx_lock(prt->lock); - if (!(prt->status & ERTS_PORT_SFLGS_DEAD)) { - driver_failure_atom(j, "driver_unloaded"); - } -#else - driver_failure_atom(j, "driver_unloaded"); -#endif - erts_port_release(prt); - } - else erts_smp_port_state_unlock(prt); - } + kill_ports_driver_unloaded(dh); #if DDLL_SMP lock_drv_list(); #endif @@ -787,7 +772,7 @@ BIF_RETTYPE erl_ddll_info_2(BIF_ALIST_2) } else if (drv->handle->status == ERL_DE_PERMANENT) { res = am_permanent; } else { - res = make_small(drv->handle->port_count); + res = make_small(erts_smp_atomic32_read_nob(&drv->handle->port_count)); } goto done; case am_linked_in_driver: @@ -1045,38 +1030,16 @@ void erts_ddll_proc_dead(Process *p, ErtsProcLocks plocks) } dh->status = ERL_DE_UNLOAD; } - if (!left && drv->handle->port_count > 0) { + if (!left + && erts_smp_atomic32_read_nob(&drv->handle->port_count) > 0) { if (kill_ports) { - int j; DE_Handle *dh = drv->handle; erts_ddll_reference_driver(dh); dh->status = ERL_DE_FORCE_UNLOAD; #if DDLL_SMP unlock_drv_list(); #endif - for (j = 0; j < erts_max_ports; j++) { - Port* prt = &erts_port[j]; - erts_smp_port_state_lock(prt); - if (!(prt->status & FREE_PORT_FLAGS) && - prt->drv_ptr->handle == dh) { - erts_smp_atomic_inc_nob(&prt->refc); -#if DDLL_SMP - while(prt->status & ERTS_PORT_SFLG_INITIALIZING) { - erts_smp_port_state_unlock(prt); - erts_smp_port_state_lock(prt); - } - erts_smp_port_state_unlock(prt); - erts_smp_mtx_lock(prt->lock); - if (!(prt->status & ERTS_PORT_SFLGS_DEAD)) { - driver_failure_atom(j, "driver_unloaded"); - } -#else - driver_failure_atom(j, "driver_unloaded"); -#endif - erts_port_release(prt); - } - else erts_smp_port_state_unlock(prt); - } + kill_ports_driver_unloaded(dh); #if DDLL_SMP lock_drv_list(); /* Needed for future list operations */ #endif @@ -1098,7 +1061,7 @@ void erts_ddll_proc_dead(Process *p, ErtsProcLocks plocks) void erts_ddll_lock_driver(DE_Handle *dh, char *name) { DE_ProcEntry *p,*q; - assert_drv_list_locked(); + assert_drv_list_rwlocked(); notify_all(dh, name, ERL_DE_PROC_AWAIT_LOAD, am_UP, am_permanent); notify_all(dh, name, @@ -1121,19 +1084,22 @@ void erts_ddll_lock_driver(DE_Handle *dh, char *name) void erts_ddll_increment_port_count(DE_Handle *dh) { assert_drv_list_locked(); - dh->port_count++; + erts_smp_atomic32_inc_nob(&dh->port_count); } void erts_ddll_decrement_port_count(DE_Handle *dh) { assert_drv_list_locked(); - ASSERT(dh->port_count > 0); - dh->port_count--; +#if DEBUG + ASSERT(erts_smp_atomic32_dec_read_nob(&dh->port_count) >= 0); +#else + erts_smp_atomic32_dec_nob(&dh->port_count); +#endif } static void first_ddll_reference(DE_Handle *dh) { - assert_drv_list_locked(); + assert_drv_list_rwlocked(); erts_refc_init(&(dh->refc),1); } @@ -1161,7 +1127,7 @@ void erts_ddll_dereference_driver(DE_Handle *dh) static void dereference_all_processes(DE_Handle *dh) { DE_ProcEntry *p; - assert_drv_list_locked(); + assert_drv_list_rwlocked(); for(p = dh->procs;p != NULL; p = p->next) { if (p->awaiting_status == ERL_DE_PROC_LOADED) { ASSERT(!(p->flags & ERL_DE_FL_DEREFERENCED)); @@ -1174,7 +1140,7 @@ static void dereference_all_processes(DE_Handle *dh) static void restore_process_references(DE_Handle *dh) { DE_ProcEntry *p; - assert_drv_list_locked(); + assert_drv_list_rwlocked(); ASSERT(erts_refc_read(&(dh->refc),0) == 0); for(p = dh->procs;p != NULL; p = p->next) { if (p->awaiting_status == ERL_DE_PROC_LOADED) { @@ -1402,7 +1368,7 @@ static int is_last_user(DE_Handle *dh, Process *proc) { DE_ProcEntry *p = dh->procs; int found = 0; - assert_drv_list_locked(); + assert_drv_list_rwlocked(); while (p != NULL) { if (p->proc == proc && p->awaiting_status == ERL_DE_PROC_LOADED) { @@ -1423,7 +1389,7 @@ static DE_ProcEntry *find_proc_entry(DE_Handle *dh, Process *proc, Uint status) { DE_ProcEntry *p = dh->procs; - assert_drv_list_locked(); + assert_drv_list_rwlocked(); while (p != NULL) { if (p->proc == proc && p->awaiting_status == status) { @@ -1450,7 +1416,7 @@ static int num_procs(DE_Handle *dh, Uint status) { DE_ProcEntry *p = dh->procs; int i = 0; - assert_drv_list_locked(); + assert_drv_list_rwlocked(); while (p != NULL) { if (p->awaiting_status == status) { @@ -1465,7 +1431,7 @@ static int num_entries(DE_Handle *dh, Process *proc, Uint status) { DE_ProcEntry *p = dh->procs; int i = 0; - assert_drv_list_locked(); + assert_drv_list_rwlocked(); while (p != NULL) { if (p->awaiting_status == status && p->proc == proc) { ++i; @@ -1478,7 +1444,7 @@ static int num_entries(DE_Handle *dh, Process *proc, Uint status) { static void add_proc_loaded(DE_Handle *dh, Process *proc) { DE_ProcEntry *p; - assert_drv_list_locked(); + assert_drv_list_rwlocked(); p = erts_alloc(ERTS_ALC_T_DDLL_PROCESS, sizeof(DE_ProcEntry)); p->proc = proc; p->flags = 0; @@ -1490,7 +1456,7 @@ static void add_proc_loaded(DE_Handle *dh, Process *proc) static void add_proc_loaded_deref(DE_Handle *dh, Process *proc) { DE_ProcEntry *p; - assert_drv_list_locked(); + assert_drv_list_rwlocked(); p = erts_alloc(ERTS_ALC_T_DDLL_PROCESS, sizeof(DE_ProcEntry)); p->proc = proc; p->awaiting_status = ERL_DE_PROC_LOADED; @@ -1510,7 +1476,7 @@ static void add_proc_waiting(DE_Handle *dh, Process *proc, Uint status, Eterm ref) { DE_ProcEntry *p; - assert_drv_list_locked(); + assert_drv_list_rwlocked(); p = erts_alloc(ERTS_ALC_T_DDLL_PROCESS, sizeof(DE_ProcEntry)); p->proc = proc; p->flags = 0; @@ -1524,7 +1490,7 @@ static Eterm add_monitor(Process *p, DE_Handle *dh, Uint status) { Eterm r; - assert_drv_list_locked(); + assert_drv_list_rwlocked(); r = erts_make_ref(p); add_proc_waiting(dh, p, status, r); return r; @@ -1535,7 +1501,7 @@ static void set_driver_reloading(DE_Handle *dh, Process *proc, char *path, char { DE_ProcEntry *p; - assert_drv_list_locked(); + assert_drv_list_rwlocked(); p = erts_alloc(ERTS_ALC_T_DDLL_PROCESS, sizeof(DE_ProcEntry)); p->proc = proc; p->awaiting_status = ERL_DE_OK; @@ -1556,7 +1522,7 @@ static int do_load_driver_entry(DE_Handle *dh, char *path, char *name) int res; ErlDrvEntry *dp; - assert_drv_list_locked(); + assert_drv_list_rwlocked(); if ((res = erts_sys_ddll_open(path, &(dh->handle))) != ERL_DE_NO_ERROR) { return res; @@ -1594,7 +1560,7 @@ static int do_load_driver_entry(DE_Handle *dh, char *path, char *name) goto error; } erts_smp_atomic_init_nob(&(dh->refc), (erts_aint_t) 0); - dh->port_count = 0; + erts_smp_atomic32_init_nob(&dh->port_count, 0); dh->full_path = erts_alloc(ERTS_ALC_T_DDLL_HANDLE, sys_strlen(path) + 1); sys_strcpy(dh->full_path, path); dh->flags = 0; @@ -1620,7 +1586,7 @@ static int do_unload_driver_entry(DE_Handle *dh, Eterm *save_name) { erts_driver_t *q, *p = driver_list; - assert_drv_list_locked(); + assert_drv_list_rwlocked(); while (p != NULL) { if (p->handle == dh) { @@ -1660,11 +1626,11 @@ static int load_driver_entry(DE_Handle **dhp, char *path, char *name) int res; DE_Handle *dh = erts_alloc(ERTS_ALC_T_DDLL_HANDLE, sizeof(DE_Handle)); - assert_drv_list_locked(); + assert_drv_list_rwlocked(); dh->handle = NULL; dh->procs = NULL; - dh->port_count = 0; + erts_smp_atomic32_init_nob(&dh->port_count, 0); erts_refc_init(&(dh->refc), (erts_aint_t) 0); dh->status = -1; dh->reload_full_path = NULL; @@ -1698,7 +1664,7 @@ static int reload_driver_entry(DE_Handle *dh) int loadres; Uint flags = dh->reload_flags; - assert_drv_list_locked(); + assert_drv_list_rwlocked(); dh->reload_full_path = NULL; dh->reload_driver_name = NULL; @@ -1736,7 +1702,7 @@ static void notify_proc(Process *proc, Eterm ref, Eterm driver_name, Eterm type, ErtsProcLocks rp_locks = 0; ERTS_SMP_CHK_NO_PROC_LOCKS; - assert_drv_list_locked(); + assert_drv_list_rwlocked(); if (errcode != 0) { int need = load_error_need(errcode); Eterm e; @@ -1769,7 +1735,7 @@ static void notify_all(DE_Handle *dh, char *name, Uint awaiting, Eterm type, Ete { DE_ProcEntry **p; - assert_drv_list_locked(); + assert_drv_list_rwlocked(); p = &(dh->procs); while (*p != NULL) { @@ -1875,7 +1841,7 @@ static Eterm mkatom(char *str) static char *pick_list_or_atom(Eterm name_term) { char *name = NULL; - Uint name_len; + ErlDrvSizeT name_len; if (is_atom(name_term)) { Atom *ap = atom_tab(atom_val(name_term)); if (ap->len == 0) { @@ -1891,7 +1857,7 @@ static char *pick_list_or_atom(Eterm name_term) goto error; } name = erts_alloc(ERTS_ALC_T_DDLL_TMP_BUF, name_len + 1); - if (io_list_to_buf(name_term, name, name_len) != 0) { + if (erts_iolist_to_buf(name_term, name, name_len) != 0) { goto error; } name[name_len] = '\0'; @@ -1912,10 +1878,10 @@ static int build_proc_info(DE_Handle *dh, ProcEntryInfo **out_pei, Uint filter) int i; DE_ProcEntry *pe; - assert_drv_list_locked(); + assert_drv_list_rwlocked(); for (pe = dh->procs; pe != NULL; pe = pe->next) { - Eterm id = pe->proc->id; + Eterm id = pe->proc->common.id; Uint stat = pe->awaiting_status; if (stat == ERL_DE_PROC_AWAIT_UNLOAD_ONLY) { stat = ERL_DE_PROC_AWAIT_UNLOAD; diff --git a/erts/emulator/beam/erl_bif_info.c b/erts/emulator/beam/erl_bif_info.c index a3811ccdb0..fabddffc68 100755 --- a/erts/emulator/beam/erl_bif_info.c +++ b/erts/emulator/beam/erl_bif_info.c @@ -40,6 +40,8 @@ #include "erl_cpu_topology.h" #include "erl_async.h" #include "erl_thr_progress.h" +#define ERTS_PTAB_WANT_DEBUG_FUNCS__ +#include "erl_ptab.h" #ifdef HIPE #include "hipe_arch.h" #endif @@ -128,8 +130,6 @@ static char erts_system_version[] = ("Erlang " ERLANG_OTP_RELEASE static Eterm os_type_tuple; static Eterm os_version_tuple; -static BIF_RETTYPE port_info(Process* p, Eterm portid, Eterm item); - static Eterm current_function(Process* p, Process* rp, Eterm** hpp, int full_info); static Eterm current_stacktrace(Process* p, Process* rp, Eterm** hpp); @@ -873,8 +873,7 @@ BIF_RETTYPE process_info_1(BIF_ALIST_1) && external_pid_dist_entry(BIF_ARG_1) == erts_this_dist_entry) BIF_RET(am_undefined); - if (is_not_internal_pid(BIF_ARG_1) - || internal_pid_index(BIF_ARG_1) >= erts_max_processes) { + if (is_not_internal_pid(BIF_ARG_1)) { BIF_ERROR(BIF_P, BADARG); } @@ -909,8 +908,7 @@ BIF_RETTYPE process_info_2(BIF_ALIST_2) && external_pid_dist_entry(pid) == erts_this_dist_entry) BIF_RET(am_undefined); - if (is_not_internal_pid(pid) - || internal_pid_index(BIF_ARG_1) >= erts_max_processes) { + if (is_not_internal_pid(pid)) { BIF_ERROR(BIF_P, BADARG); } @@ -1002,9 +1000,9 @@ process_info_aux(Process *BIF_P, switch (item) { case am_registered_name: - if (rp->reg != NULL) { + if (rp->common.u.alive.reg) { hp = HAlloc(BIF_P, 3); - res = rp->reg->name; + res = rp->common.u.alive.reg->name; } else { if (always_wrap) { hp = HAlloc(BIF_P, 3); @@ -1050,7 +1048,7 @@ process_info_aux(Process *BIF_P, ERTS_SMP_MSGQ_MV_INQ2PRIVQ(rp); n = rp->msg.len; - if (n == 0 || rp->trace_flags & F_SENSITIVE) { + if (n == 0 || ERTS_TRACE_FLAGS(rp) & F_SENSITIVE) { hp = HAlloc(BIF_P, 3); } else { int remove_bad_messages = 0; @@ -1209,7 +1207,7 @@ process_info_aux(Process *BIF_P, INIT_MONITOR_INFOS(mic); - erts_doforall_links(rp->nlinks,&collect_one_link,&mic); + erts_doforall_links(ERTS_P_LINKS(rp),&collect_one_link,&mic); hp = HAlloc(BIF_P, 3 + mic.sz); res = NIL; @@ -1227,7 +1225,7 @@ process_info_aux(Process *BIF_P, int i; INIT_MONITOR_INFOS(mic); - erts_doforall_monitors(rp->monitors,&collect_one_origin_monitor,&mic); + erts_doforall_monitors(ERTS_P_MONITORS(rp),&collect_one_origin_monitor,&mic); hp = HAlloc(BIF_P, 3 + mic.sz); res = NIL; for (i = 0; i < mic.mi_i; i++) { @@ -1264,7 +1262,7 @@ process_info_aux(Process *BIF_P, Eterm item; INIT_MONITOR_INFOS(mic); - erts_doforall_monitors(rp->monitors,&collect_one_target_monitor,&mic); + erts_doforall_monitors(ERTS_P_MONITORS(rp),&collect_one_target_monitor,&mic); hp = HAlloc(BIF_P, 3 + mic.sz); res = NIL; @@ -1330,7 +1328,7 @@ process_info_aux(Process *BIF_P, } case am_dictionary: - if (rp->trace_flags & F_SENSITIVE) { + if (ERTS_TRACE_FLAGS(rp) & F_SENSITIVE) { res = NIL; } else { res = erts_dictionary_copy(BIF_P, rp->dictionary); @@ -1426,8 +1424,8 @@ process_info_aux(Process *BIF_P, ERTS_SMP_MSGQ_MV_INQ2PRIVQ(rp); - erts_doforall_links(rp->nlinks, &one_link_size, &size); - erts_doforall_monitors(rp->monitors, &one_mon_size, &size); + erts_doforall_links(ERTS_P_LINKS(rp), &one_link_size, &size); + erts_doforall_monitors(ERTS_P_MONITORS(rp), &one_mon_size, &size); size += (rp->heap_sz + rp->mbuf_sz) * sizeof(Eterm); if (rp->old_hend && rp->old_heap) size += (rp->old_hend - rp->old_heap) * sizeof(Eterm); @@ -1500,7 +1498,7 @@ process_info_aux(Process *BIF_P, case am_trace: hp = HAlloc(BIF_P, 3); - res = make_small(rp->trace_flags & TRACEE_FLAGS); + res = make_small(ERTS_TRACE_FLAGS(rp) & TRACEE_FLAGS); break; case am_binary: { @@ -1605,7 +1603,7 @@ current_function(Process* BIF_P, Process* rp, Eterm** hpp, int full_info) } } - if (BIF_P->id == rp->id) { + if (BIF_P == rp) { FunctionInfo fi2; /* @@ -1837,17 +1835,17 @@ info_1_tuple(Process* BIF_P, /* Pointer to current process. */ # define ERTS_ERROR_CHECKER_PRINTF_XML VALGRIND_PRINTF_XML # endif #endif - Uint buf_size = 8*1024; /* Try with 8KB first */ + ErlDrvSizeT buf_size = 8*1024; /* Try with 8KB first */ char *buf = erts_alloc(ERTS_ALC_T_TMP, buf_size); - int r = io_list_to_buf(*tp, (char*) buf, buf_size - 1); - if (r < 0) { + ErlDrvSizeT r = erts_iolist_to_buf(*tp, (char*) buf, buf_size - 1); + if (ERTS_IOLIST_TO_BUF_FAILED(r)) { erts_free(ERTS_ALC_T_TMP, (void *) buf); if (erts_iolist_size(*tp, &buf_size)) { goto badarg; } buf_size++; buf = erts_alloc(ERTS_ALC_T_TMP, buf_size); - r = io_list_to_buf(*tp, (char*) buf, buf_size - 1); + r = erts_iolist_to_buf(*tp, (char*) buf, buf_size - 1); ASSERT(r == buf_size - 1); } buf[buf_size - 1 - r] = '\0'; @@ -2159,9 +2157,13 @@ BIF_RETTYPE system_info_1(BIF_ALIST_1) res = TUPLE2(hp, am_min_bin_vheap_size,make_small(BIN_VH_MIN_SIZE)); BIF_RET(res); } else if (BIF_ARG_1 == am_process_count) { - BIF_RET(make_small(erts_process_count())); + BIF_RET(make_small(erts_ptab_count(&erts_proc))); } else if (BIF_ARG_1 == am_process_limit) { - BIF_RET(make_small(erts_max_processes)); + BIF_RET(make_small(erts_ptab_max(&erts_proc))); + } else if (BIF_ARG_1 == am_port_count) { + BIF_RET(make_small(erts_ptab_count(&erts_port))); + } else if (BIF_ARG_1 == am_port_limit) { + BIF_RET(make_small(erts_ptab_max(&erts_port))); } else if (BIF_ARG_1 == am_info || BIF_ARG_1 == am_procs || BIF_ARG_1 == am_loaded @@ -2534,6 +2536,9 @@ BIF_RETTYPE system_info_1(BIF_ALIST_1) } else if (ERTS_IS_ATOM_STR("run_queues", BIF_ARG_1)) { res = make_small(erts_no_run_queues); BIF_RET(res); + } else if (ERTS_IS_ATOM_STR("port_parallelism", BIF_ARG_1)) { + res = erts_port_parallelism ? am_true : am_false; + BIF_RET(res); } else if (ERTS_IS_ATOM_STR("c_compiler_used", BIF_ARG_1)) { Eterm *hp = NULL; Uint sz = 0; @@ -2701,66 +2706,6 @@ BIF_RETTYPE system_info_1(BIF_ALIST_1) BIF_ERROR(BIF_P, BADARG); } -BIF_RETTYPE -port_info_1(BIF_ALIST_1) -{ - Process* p = BIF_P; - Eterm pid = BIF_ARG_1; - static Eterm keys[] = { - am_name, - am_links, - am_id, - am_connected, - am_input, - am_output, - am_os_pid - }; - Eterm items[ASIZE(keys)]; - Eterm result = NIL; - Eterm reg_name; - Eterm* hp; - Uint need; - int i; - - /* - * Collect all information about the port. - */ - - for (i = 0; i < ASIZE(keys); i++) { - Eterm item; - - item = port_info(p, pid, keys[i]); - if (is_non_value(item)) { - return THE_NON_VALUE; - } - if (item == am_undefined) { - return am_undefined; - } - items[i] = item; - } - reg_name = port_info(p, pid, am_registered_name); - - /* - * Build the resulting list. - */ - - need = 2*ASIZE(keys); - if (is_tuple(reg_name)) { - need += 2; - } - hp = HAlloc(p, need); - for (i = ASIZE(keys) - 1; i >= 0; i--) { - result = CONS(hp, items[i], result); - hp += 2; - } - if (is_tuple(reg_name)) { - result = CONS(hp, reg_name, result); - } - - return result; -} - - /**********************************************************************/ /* Return information on ports */ /* Info: @@ -2773,38 +2718,20 @@ port_info_1(BIF_ALIST_1) ** os_pid The child's process ID */ -BIF_RETTYPE port_info_2(BIF_ALIST_2) -{ - return port_info(BIF_P, BIF_ARG_1, BIF_ARG_2); -} - -static BIF_RETTYPE port_info(Process* p, Eterm portid, Eterm item) +Eterm +erts_bld_port_info(Eterm **hpp, ErlOffHeap *ohp, Uint *szp, Port *prt, Eterm item) { - BIF_RETTYPE ret; - Port *prt; - Eterm res; - Eterm* hp; - int count; - - if (is_internal_port(portid)) - prt = erts_id2port(portid, p, ERTS_PROC_LOCK_MAIN); - else if (is_atom(portid)) - erts_whereis_name(p, ERTS_PROC_LOCK_MAIN, - portid, NULL, 0, 0, &prt); - else if (is_external_port(portid) - && external_port_dist_entry(portid) == erts_this_dist_entry) - BIF_RET(am_undefined); - else { - BIF_ERROR(p, BADARG); - } + Eterm res = THE_NON_VALUE; - if (!prt) { - BIF_RET(am_undefined); - } + ERTS_SMP_LC_ASSERT(erts_lc_is_port_locked(prt)); if (item == am_id) { - hp = HAlloc(p, 3); - res = make_small(internal_port_number(portid)); + if (hpp) + res = make_small(internal_port_index(prt->common.id)); + if (szp) { + res = am_true; + goto done; + } } else if (item == am_links) { MonitorInfoCollection mic; @@ -2813,17 +2740,26 @@ static BIF_RETTYPE port_info(Process* p, Eterm portid, Eterm item) INIT_MONITOR_INFOS(mic); - erts_doforall_links(prt->nlinks, &collect_one_link, &mic); + erts_doforall_links(ERTS_P_LINKS(prt), &collect_one_link, &mic); - hp = HAlloc(p, 3 + mic.sz); - res = NIL; - for (i = 0; i < mic.mi_i; i++) { - item = STORE_NC(&hp, &MSO(p), mic.mi[i].entity); - res = CONS(hp, item, res); - hp += 2; + if (szp) + *szp += mic.sz; + + if (hpp) { + res = NIL; + for (i = 0; i < mic.mi_i; i++) { + item = STORE_NC(hpp, ohp, mic.mi[i].entity); + res = CONS(*hpp, item, res); + *hpp += 2; + } } + DESTROY_MONITOR_INFOS(mic); + if (szp) { + res = am_true; + goto done; + } } else if (item == am_monitors) { MonitorInfoCollection mic; @@ -2832,79 +2768,96 @@ static BIF_RETTYPE port_info(Process* p, Eterm portid, Eterm item) INIT_MONITOR_INFOS(mic); - erts_doforall_monitors(prt->monitors, &collect_one_origin_monitor, &mic); + erts_doforall_monitors(ERTS_P_MONITORS(prt), &collect_one_origin_monitor, &mic); - hp = HAlloc(p, 3 + mic.sz); - res = NIL; - for (i = 0; i < mic.mi_i; i++) { - Eterm t; - item = STORE_NC(&hp, &MSO(p), mic.mi[i].entity); - t = TUPLE2(hp, am_process, item); - hp += 3; - res = CONS(hp, t, res); - hp += 2; + if (szp) + *szp += mic.sz; + + if (hpp) { + res = NIL; + for (i = 0; i < mic.mi_i; i++) { + Eterm t; + item = STORE_NC(hpp, ohp, mic.mi[i].entity); + t = TUPLE2(*hpp, am_process, item); + *hpp += 3; + res = CONS(*hpp, t, res); + *hpp += 2; + } } + DESTROY_MONITOR_INFOS(mic); + if (szp) { + res = am_true; + goto done; + } } else if (item == am_name) { - count = sys_strlen(prt->name); + int count = sys_strlen(prt->name); + + if (hpp) + res = buf_to_intlist(hpp, prt->name, count, NIL); - hp = HAlloc(p, 3 + 2*count); - res = buf_to_intlist(&hp, prt->name, count, NIL); + if (szp) { + *szp += 2*count; + res = am_true; + goto done; + } } else if (item == am_connected) { - hp = HAlloc(p, 3); - res = prt->connected; /* internal pid */ + if (hpp) + res = ERTS_PORT_GET_CONNECTED(prt); /* internal pid */ + if (szp) { + res = am_true; + goto done; + } } else if (item == am_input) { - Uint hsz = 3; - Uint n = prt->bytes_in; - (void) erts_bld_uint(NULL, &hsz, n); - hp = HAlloc(p, hsz); - res = erts_bld_uint(&hp, NULL, n); + res = erts_bld_uint(hpp, szp, prt->bytes_in); + if (szp) { + res = am_true; + goto done; + } } else if (item == am_output) { - Uint hsz = 3; - Uint n = prt->bytes_out; - (void) erts_bld_uint(NULL, &hsz, n); - hp = HAlloc(p, hsz); - res = erts_bld_uint(&hp, NULL, n); + res = erts_bld_uint(hpp, szp, prt->bytes_out); + if (szp) { + res = am_true; + goto done; + } } else if (item == am_os_pid) { - if (prt->os_pid >= 0) { - Uint hsz = 3; - UWord n = prt->os_pid; - (void) erts_bld_uword(NULL, &hsz, n); - hp = HAlloc(p, hsz); - res = erts_bld_uword(&hp, NULL, n); - } else { - hp = HAlloc(p, 3); - res = am_undefined; - } + res = (prt->os_pid < 0 + ? am_undefined + : erts_bld_uword(hpp, szp, (UWord) prt->os_pid)); + if (szp) { + res = am_true; + goto done; + } } else if (item == am_registered_name) { - RegProc *reg; - reg = prt->reg; - if (reg == NULL) { - ERTS_BIF_PREP_RET(ret, NIL); - goto done; - } else { - hp = HAlloc(p, 3); + RegProc *reg = prt->common.u.alive.reg; + if (reg) { res = reg->name; + if (szp) { + res = am_true; + goto done; + } + } + else { + if (szp) + return am_undefined; + return NIL; } } else if (item == am_memory) { /* All memory consumed in bytes (the Port struct should not be included though). */ - Uint hsz = 3; Uint size = 0; ErlHeapFragment* bp; - hp = HAlloc(p, 3); - - erts_doforall_links(prt->nlinks, &one_link_size, &size); + erts_doforall_links(ERTS_P_LINKS(prt), &one_link_size, &size); for (bp = prt->bp; bp; bp = bp->next) size += sizeof(ErlHeapFragment) + (bp->alloc_size - 1)*sizeof(Eterm); @@ -2918,51 +2871,72 @@ static BIF_RETTYPE port_info(Process* p, Eterm portid, Eterm item) /* All memory allocated by the driver should be included, but it is hard to retrieve... */ - (void) erts_bld_uint(NULL, &hsz, size); - hp = HAlloc(p, hsz); - res = erts_bld_uint(&hp, NULL, size); + res = erts_bld_uint(hpp, szp, size); + if (szp) { + res = am_true; + goto done; + } } else if (item == am_queue_size) { Uint ioq_size = erts_port_ioq_size(prt); - Uint hsz = 3; - (void) erts_bld_uint(NULL, &hsz, ioq_size); - hp = HAlloc(p, hsz); - res = erts_bld_uint(&hp, NULL, ioq_size); + res = erts_bld_uint(hpp, szp, ioq_size); + if (szp) { + res = am_true; + goto done; + } } else if (ERTS_IS_ATOM_STR("locking", item)) { - hp = HAlloc(p, 3); + if (hpp) { #ifndef ERTS_SMP - res = am_false; + res = am_false; #else - if (prt->status & ERTS_PORT_SFLG_PORT_SPECIFIC_LOCK) { - DECL_AM(port_level); - ASSERT(prt->drv_ptr->flags - & ERL_DRV_FLAG_USE_PORT_LOCKING); - res = AM_port_level; + if (erts_atomic32_read_nob(&prt->state) + & ERTS_PORT_SFLG_PORT_SPECIFIC_LOCK) { + DECL_AM(port_level); + ASSERT(prt->drv_ptr->flags + & ERL_DRV_FLAG_USE_PORT_LOCKING); + res = AM_port_level; + } + else { + DECL_AM(driver_level); + ASSERT(!(prt->drv_ptr->flags + & ERL_DRV_FLAG_USE_PORT_LOCKING)); + res = AM_driver_level; + } +#endif } - else { - DECL_AM(driver_level); - ASSERT(!(prt->drv_ptr->flags - & ERL_DRV_FLAG_USE_PORT_LOCKING)); - res = AM_driver_level; + if (szp) { + res = am_true; + goto done; } -#endif + } + else if (item == am_parallelism) { + if (szp) { + res = am_true; + goto done; + } + res = ((ERTS_PTS_FLG_PARALLELISM & + erts_smp_atomic32_read_nob(&prt->sched.flags)) + ? am_true + : am_false); } else { - ERTS_BIF_PREP_ERROR(ret, p, BADARG); - goto done; + if (szp) + return am_false; + return THE_NON_VALUE; } - ERTS_BIF_PREP_RET(ret, TUPLE2(hp, item, res)); - - done: - - erts_smp_port_unlock(prt); +done: + if (szp) + *szp += 3; + if (hpp) { + res = TUPLE2(*hpp, item, res); + *hpp += 3; + } - return ret; + return res; } - BIF_RETTYPE fun_info_2(BIF_ALIST_2) { @@ -3094,12 +3068,9 @@ BIF_RETTYPE is_process_alive_1(BIF_ALIST_1) if(is_internal_pid(BIF_ARG_1)) { Process *rp; - if (BIF_ARG_1 == BIF_P->id) + if (BIF_ARG_1 == BIF_P->common.id) BIF_RET(am_true); - if(internal_pid_index(BIF_ARG_1) >= erts_max_processes) - BIF_ERROR(BIF_P, BADARG); - rp = erts_proc_lookup(BIF_ARG_1); if (!rp) { BIF_RET(am_false); @@ -3317,10 +3288,9 @@ BIF_RETTYPE erts_debug_get_internal_state_1(BIF_ALIST_1) /* Used by node_container_SUITE (emulator) */ Eterm res; if (ERTS_IS_ATOM_STR("next_pid", BIF_ARG_1)) - res = erts_test_next_pid(0, 0); - else { - res = erts_test_next_port(0, 0); - } + res = erts_ptab_test_next_id(&erts_proc, 0, 0); + else + res = erts_ptab_test_next_id(&erts_port, 0, 0); if (res < 0) BIF_RET(am_false); BIF_RET(erts_make_integer(res, BIF_P)); @@ -3356,11 +3326,11 @@ BIF_RETTYPE erts_debug_get_internal_state_1(BIF_ALIST_1) } else if (ERTS_IS_ATOM_STR("processes", BIF_ARG_1)) { /* Used by process_SUITE (emulator) */ - BIF_RET(erts_debug_processes(BIF_P)); + BIF_RET(erts_debug_ptab_list(BIF_P, &erts_proc)); } else if (ERTS_IS_ATOM_STR("processes_bif_info", BIF_ARG_1)) { /* Used by process_SUITE (emulator) */ - BIF_RET(erts_debug_processes_bif_info(BIF_P)); + BIF_RET(erts_debug_ptab_list_bif_info(BIF_P, &erts_proc)); } else if (ERTS_IS_ATOM_STR("max_atom_out_cache_index", BIF_ARG_1)) { /* Used by distribution_SUITE (emulator) */ @@ -3421,17 +3391,20 @@ BIF_RETTYPE erts_debug_get_internal_state_1(BIF_ALIST_1) ERTS_SMP_ASSERT_IS_NOT_EXITING(BIF_P); BIF_RET(am_undefined); } - res = make_link_list(BIF_P, p->nlinks, NIL); + res = make_link_list(BIF_P, ERTS_P_LINKS(p), NIL); erts_smp_proc_unlock(p, ERTS_PROC_LOCK_LINK); BIF_RET(res); } else if(is_internal_port(tp[2])) { Eterm res; - Port *p = erts_id2port(tp[2], BIF_P, ERTS_PROC_LOCK_MAIN); + Port *p = erts_id2port_sflgs(tp[2], + BIF_P, + ERTS_PROC_LOCK_MAIN, + ERTS_PORT_SFLGS_INVALID_LOOKUP); if(!p) BIF_RET(am_undefined); - res = make_link_list(BIF_P, p->nlinks, NIL); - erts_smp_port_unlock(p); + res = make_link_list(BIF_P, ERTS_P_LINKS(p), NIL); + erts_port_release(p); BIF_RET(res); } else if(is_node_name_atom(tp[2])) { @@ -3463,7 +3436,7 @@ BIF_RETTYPE erts_debug_get_internal_state_1(BIF_ALIST_1) ERTS_SMP_ASSERT_IS_NOT_EXITING(BIF_P); BIF_RET(am_undefined); } - res = make_monitor_list(BIF_P, p->monitors); + res = make_monitor_list(BIF_P, ERTS_P_MONITORS(p)); erts_smp_proc_unlock(p, ERTS_PROC_LOCK_LINK); BIF_RET(res); } else if(is_node_name_atom(tp[2])) { @@ -3606,7 +3579,7 @@ BIF_RETTYPE erts_debug_set_internal_state_2(BIF_ALIST_2) erts_aint_t prev_on = erts_smp_atomic_xchg_nob(&available_internal_state, on); if (on) { erts_dsprintf_buf_t *dsbufp = erts_create_logger_dsbuf(); - erts_dsprintf(dsbufp, "Process %T ", BIF_P->id); + erts_dsprintf(dsbufp, "Process %T ", BIF_P->common.id); if (erts_is_alive) erts_dsprintf(dsbufp, "on node %T ", erts_this_node->sysname); erts_dsprintf(dsbufp, @@ -3674,10 +3647,9 @@ BIF_RETTYPE erts_debug_set_internal_state_2(BIF_ALIST_2) Eterm res; if (ERTS_IS_ATOM_STR("next_pid", BIF_ARG_1)) - res = erts_test_next_pid(1, next); - else { - res = erts_test_next_port(1, next); - } + res = erts_ptab_test_next_id(&erts_proc, 1, next); + else + res = erts_ptab_test_next_id(&erts_port, 1, next); if (res < 0) BIF_RET(am_false); BIF_RET(erts_make_integer(res, BIF_P)); @@ -3948,8 +3920,8 @@ static Eterm lcnt_build_lock_term(Eterm **hpp, Uint *szp, erts_lcnt_lock_t *lock } else if (lock->flag & ERTS_LCNT_LT_PROCLOCK) { /* use registered names as id's for process locks if available */ proc = erts_proc_lookup(lock->id); - if (proc && proc->reg) { - id = proc->reg->name; + if (proc && proc->common.u.alive.reg) { + id = proc->common.u.alive.reg->name; } else { /* otherwise use process id */ id = lock->id; diff --git a/erts/emulator/beam/erl_bif_op.c b/erts/emulator/beam/erl_bif_op.c index 13f8b1f63c..d4cd9c89b3 100644 --- a/erts/emulator/beam/erl_bif_op.c +++ b/erts/emulator/beam/erl_bif_op.c @@ -261,11 +261,6 @@ Eterm erl_is_function(Process* p, Eterm arg1, Eterm arg2) if (exp->code[2] == (Uint) arity) { BIF_RET(am_true); } - } else if (is_tuple(arg1)) { - Eterm* tp = tuple_val(arg1); - if (tp[0] == make_arityval(2) && is_atom(tp[1]) && is_atom(tp[2])) { - BIF_RET(am_true); - } } BIF_RET(am_false); } diff --git a/erts/emulator/beam/erl_bif_port.c b/erts/emulator/beam/erl_bif_port.c index f9009166c0..81146e38d7 100644 --- a/erts/emulator/beam/erl_bif_port.c +++ b/erts/emulator/beam/erl_bif_port.c @@ -42,28 +42,26 @@ #include "erl_bits.h" #include "dtrace-wrapper.h" -static int open_port(Process* p, Eterm name, Eterm settings, int *err_nump); +static Port *open_port(Process* p, Eterm name, Eterm settings, int *err_typep, int *err_nump); static byte* convert_environment(Process* p, Eterm env); static char **convert_args(Eterm); static void free_args(char **); char *erts_default_arg0 = "default"; -static BIF_RETTYPE -port_call(Process* p, Eterm arg1, Eterm arg2, Eterm arg3); - BIF_RETTYPE open_port_2(BIF_ALIST_2) { - int port_num; - Eterm port_val; + Port *port; + Eterm port_id; char *str; - int err_num; + int err_type, err_num; - if ((port_num = open_port(BIF_P, BIF_ARG_1, BIF_ARG_2, &err_num)) < 0) { - if (port_num == -3) { + port = open_port(BIF_P, BIF_ARG_1, BIF_ARG_2, &err_type, &err_num); + if (!port) { + if (err_type == -3) { ASSERT(err_num == BADARG || err_num == SYSTEM_LIMIT); BIF_ERROR(BIF_P, err_num); - } else if (port_num == -2) { + } else if (err_type == -2) { str = erl_errno_id(err_num); } else { str = "einval"; @@ -74,546 +72,408 @@ BIF_RETTYPE open_port_2(BIF_ALIST_2) erts_smp_proc_lock(BIF_P, ERTS_PROC_LOCK_LINK); - port_val = erts_port[port_num].id; - erts_add_link(&(erts_port[port_num].nlinks), LINK_PID, BIF_P->id); - erts_add_link(&(BIF_P->nlinks), LINK_PID, port_val); + port_id = port->common.id; + erts_add_link(&ERTS_P_LINKS(port), LINK_PID, BIF_P->common.id); + erts_add_link(&ERTS_P_LINKS(BIF_P), LINK_PID, port_id); erts_smp_proc_unlock(BIF_P, ERTS_PROC_LOCK_LINK); - erts_port_release(&erts_port[port_num]); + erts_port_release(port); - BIF_RET(port_val); + BIF_RET(port_id); } -/**************************************************************************** - - PORT BIFS: - - port_command/2 -- replace Port ! {..., {command, Data}} - port_command(Port, Data) -> true - when port(Port), io-list(Data) - - port_control/3 -- new port_control(Port, Ctl, Data) -> Reply - port_control(Port, Ctl, Data) -> Reply - where integer(Ctl), io-list(Data), io-list(Reply) - - port_close/1 -- replace Port ! {..., close} - port_close(Port) -> true - when port(Port) - - port_connect/2 -- replace Port ! {..., {connect, Pid}} - port_connect(Port, Pid) - when port(Port), pid(Pid) - - ***************************************************************************/ - -static Port* -id_or_name2port(Process *c_p, Eterm id) +static ERTS_INLINE Port * +lookup_port(Process *c_p, Eterm id_or_name) { - Port *port; - if (is_not_atom(id)) - port = erts_id2port(id, c_p, ERTS_PROC_LOCK_MAIN); + /* TODO: Implement nicer lookup in register... */ + Eterm id; + if (is_atom(id_or_name)) + id = erts_whereis_name_to_id(c_p, id_or_name); else - erts_whereis_name(c_p, ERTS_PROC_LOCK_MAIN, id, NULL, 0, 0, &port); - return port; + id = id_or_name; + return erts_port_lookup(id, ERTS_PORT_SFLGS_INVALID_LOOKUP); } -#define ERTS_PORT_COMMAND_FLAG_FORCE (((Uint32) 1) << 0) -#define ERTS_PORT_COMMAND_FLAG_NOSUSPEND (((Uint32) 1) << 1) +/* + * erts_internal:port_command/3 is used by the + * erlang:port_command/2 and erlang:port_command/3 + * BIFs. + */ -static BIF_RETTYPE -do_port_command(Process *BIF_P, Eterm arg1, Eterm arg2, Eterm arg3, - Uint32 flags) +BIF_RETTYPE erts_internal_port_command_3(BIF_ALIST_3) { BIF_RETTYPE res; - Port *p; - - /* Trace sched out before lock check wait */ - if (IS_TRACED_FL(BIF_P, F_TRACE_SCHED_PROCS)) { - trace_virtual_sched(BIF_P, am_out); - } - - if (erts_system_profile_flags.runnable_procs && erts_system_profile_flags.exclusive) { - profile_runnable_proc(BIF_P, am_inactive); - } - - p = id_or_name2port(BIF_P, arg1); - if (!p) { - if (IS_TRACED_FL(BIF_P, F_TRACE_SCHED_PROCS)) { - trace_virtual_sched(BIF_P, am_in); + Port *prt; + int flags = 0; + Eterm ref; + + if (is_not_nil(BIF_ARG_3)) { + Eterm l = BIF_ARG_3; + while (is_list(l)) { + Eterm* cons = list_val(l); + Eterm car = CAR(cons); + if (car == am_force) + flags |= ERTS_PORT_SIG_FLG_FORCE; + else if (car == am_nosuspend) + flags |= ERTS_PORT_SIG_FLG_NOSUSPEND; + else + BIF_RET(am_badarg); + l = CDR(cons); } - if (erts_system_profile_flags.runnable_procs && erts_system_profile_flags.exclusive) { - profile_runnable_proc(BIF_P, am_active); - } - BIF_ERROR(BIF_P, BADARG); + if (!is_nil(l)) + BIF_RET(am_badarg); } - - /* Trace port in, id_or_name2port causes wait */ - if (IS_TRACED_FL(p, F_TRACE_SCHED_PORTS)) { - trace_sched_ports_where(p, am_in, am_command); - } - if (erts_system_profile_flags.runnable_ports && !erts_port_is_scheduled(p)) { - profile_runnable_port(p, am_active); + prt = lookup_port(BIF_P, BIF_ARG_1); + if (!prt) + BIF_RET(am_badarg); + + if (flags & ERTS_PORT_SIG_FLG_FORCE) { + if (!(prt->drv_ptr->flags & ERL_DRV_FLAG_SOFT_BUSY)) + BIF_RET(am_notsup); } - ERTS_BIF_PREP_RET(res, am_true); +#ifdef DEBUG + ref = NIL; +#endif - if ((flags & ERTS_PORT_COMMAND_FLAG_FORCE) - && !(p->drv_ptr->flags & ERL_DRV_FLAG_SOFT_BUSY)) { - ERTS_BIF_PREP_ERROR(res, BIF_P, EXC_NOTSUP); - } - else if (!(flags & ERTS_PORT_COMMAND_FLAG_FORCE) - && p->status & ERTS_PORT_SFLG_PORT_BUSY) { - if (flags & ERTS_PORT_COMMAND_FLAG_NOSUSPEND) { + switch (erts_port_output(BIF_P, flags, prt, prt->common.id, BIF_ARG_2, &ref)) { + case ERTS_PORT_OP_CALLER_EXIT: + case ERTS_PORT_OP_BADARG: + case ERTS_PORT_OP_DROPPED: + ERTS_BIF_PREP_RET(res, am_badarg); + break; + case ERTS_PORT_OP_BUSY: + ASSERT(!(flags & ERTS_PORT_SIG_FLG_FORCE)); + if (flags & ERTS_PORT_SIG_FLG_NOSUSPEND) ERTS_BIF_PREP_RET(res, am_false); - } else { - erts_suspend(BIF_P, ERTS_PROC_LOCK_MAIN, p); - if (erts_system_monitor_flags.busy_port) { - monitor_generic(BIF_P, am_busy_port, p->id); - } - ERTS_BIF_PREP_YIELD3(res, bif_export[BIF_port_command_3], BIF_P, - arg1, arg2, arg3); - } - } else { - int wres; - erts_smp_proc_unlock(BIF_P, ERTS_PROC_LOCK_MAIN); - ERTS_SMP_CHK_NO_PROC_LOCKS; - wres = erts_write_to_port(BIF_P->id, p, arg2); - erts_smp_proc_lock(BIF_P, ERTS_PROC_LOCK_MAIN); - if (wres != 0) { - ERTS_BIF_PREP_ERROR(res, BIF_P, BADARG); + erts_suspend(BIF_P, ERTS_PROC_LOCK_MAIN, prt); + ERTS_BIF_PREP_YIELD3(res, bif_export[BIF_erts_internal_port_command_3], + BIF_P, BIF_ARG_1, BIF_ARG_2, BIF_ARG_3); } - } - - if (IS_TRACED_FL(p, F_TRACE_SCHED_PORTS)) { - trace_sched_ports_where(p, am_out, am_command); - } - if (erts_system_profile_flags.runnable_ports && !erts_port_is_scheduled(p)) { - profile_runnable_port(p, am_inactive); + break; + case ERTS_PORT_OP_BUSY_SCHEDULED: + ASSERT(!(flags & ERTS_PORT_SIG_FLG_FORCE)); + /* Fall through... */ + case ERTS_PORT_OP_SCHEDULED: + ASSERT(is_internal_ref(ref)); + ERTS_BIF_PREP_RET(res, ref); + break; + case ERTS_PORT_OP_DONE: + ERTS_BIF_PREP_RET(res, am_true); + break; + default: + ERTS_INTERNAL_ERROR("Unexpected erts_port_output() result"); + break; } - erts_port_release(p); - /* Trace sched in after port release */ - if (IS_TRACED_FL(BIF_P, F_TRACE_SCHED_PROCS)) { - trace_virtual_sched(BIF_P, am_in); - } - if (erts_system_profile_flags.runnable_procs && erts_system_profile_flags.exclusive) { - profile_runnable_proc(BIF_P, am_active); - } - if (ERTS_PROC_IS_EXITING(BIF_P)) { KILL_CATCHES(BIF_P); /* Must exit */ ERTS_BIF_PREP_ERROR(res, BIF_P, EXC_ERROR); } - return res; -} -BIF_RETTYPE port_command_2(BIF_ALIST_2) -{ - return do_port_command(BIF_P, BIF_ARG_1, BIF_ARG_2, NIL, 0); + return res; } -BIF_RETTYPE port_command_3(BIF_ALIST_3) +BIF_RETTYPE erts_internal_port_call_3(BIF_ALIST_3) { - Eterm l = BIF_ARG_3; - Uint32 flags = 0; - while (is_list(l)) { - Eterm* cons = list_val(l); - Eterm car = CAR(cons); - if (car == am_force) { - flags |= ERTS_PORT_COMMAND_FLAG_FORCE; - } else if (car == am_nosuspend) { - flags |= ERTS_PORT_COMMAND_FLAG_NOSUSPEND; - } else { - BIF_ERROR(BIF_P, BADARG); - } - l = CDR(cons); - } - if(!is_nil(l)) { - BIF_ERROR(BIF_P, BADARG); + Port* prt; + Eterm retval; + Uint uint_op; + unsigned int op; + erts_aint32_t state; + + prt = lookup_port(BIF_P, BIF_ARG_1); + if (!prt) + BIF_RET(am_badarg); + + if (!term_to_Uint(BIF_ARG_2, &uint_op)) + BIF_RET(am_badarg); + + if (uint_op > (Uint) UINT_MAX) + BIF_RET(am_badarg); + + op = (unsigned int) uint_op; + + switch (erts_port_call(BIF_P, prt, op, BIF_ARG_3, &retval)) { + case ERTS_PORT_OP_CALLER_EXIT: + case ERTS_PORT_OP_DROPPED: + case ERTS_PORT_OP_BADARG: + retval = am_badarg; + break; + case ERTS_PORT_OP_SCHEDULED: + ASSERT(is_internal_ref(retval)); + break; + case ERTS_PORT_OP_DONE: + ASSERT(is_not_internal_ref(retval)); + break; + default: + ERTS_INTERNAL_ERROR("Unexpected erts_port_call() result"); + retval = am_internal_error; + break; } - return do_port_command(BIF_P, BIF_ARG_1, BIF_ARG_2, BIF_ARG_3, flags); -} -BIF_RETTYPE port_call_2(BIF_ALIST_2) -{ - return port_call(BIF_P,BIF_ARG_1, make_small(0), BIF_ARG_2); -} + state = erts_smp_atomic32_read_acqb(&BIF_P->state); + if (state & (ERTS_PSFLG_EXITING|ERTS_PSFLG_PENDING_EXIT)) { +#ifdef ERTS_SMP + if (state & ERTS_PSFLG_PENDING_EXIT) + erts_handle_pending_exit(BIF_P, ERTS_PROC_LOCK_MAIN); +#endif + ERTS_BIF_EXITED(BIF_P); + } -BIF_RETTYPE port_call_3(BIF_ALIST_3) -{ - return port_call(BIF_P, BIF_ARG_1, BIF_ARG_2, BIF_ARG_3); + BIF_RET(retval); } -static BIF_RETTYPE -port_call(Process* c_p, Eterm arg1, Eterm arg2, Eterm arg3) +BIF_RETTYPE erts_internal_port_control_3(BIF_ALIST_3) { - Uint op; - Port *p; - Uint size; - byte *bytes; - byte *endp; - ErlDrvSizeT real_size; - erts_driver_t *drv; - byte port_input[256]; /* Default input buffer to encode in */ - byte port_result[256]; /* Buffer for result from port. */ - byte* port_resp; /* Pointer to result buffer. */ - char *prc; - ErlDrvSSizeT ret; - Eterm res; - Sint result_size; - Eterm *hp; - Eterm *hp_end; - unsigned ret_flags = 0U; - int fpe_was_unmasked; - - bytes = &port_input[0]; - port_resp = port_result; - /* trace of port scheduling with virtual process descheduling - * lock wait - */ - if (IS_TRACED_FL(c_p, F_TRACE_SCHED_PROCS)) { - trace_virtual_sched(c_p, am_out); - } - - if (erts_system_profile_flags.runnable_procs && erts_system_profile_flags.exclusive) { - profile_runnable_proc(c_p, am_inactive); + Port* prt; + Eterm retval; + Uint uint_op; + unsigned int op; + erts_aint32_t state; + + prt = lookup_port(BIF_P, BIF_ARG_1); + if (!prt) + BIF_RET(am_badarg); + + if (!term_to_Uint(BIF_ARG_2, &uint_op)) + BIF_RET(am_badarg); + + if (uint_op > (Uint) UINT_MAX) + BIF_RET(am_badarg); + + op = (unsigned int) uint_op; + + switch (erts_port_control(BIF_P, prt, op, BIF_ARG_3, &retval)) { + case ERTS_PORT_OP_CALLER_EXIT: + case ERTS_PORT_OP_BADARG: + case ERTS_PORT_OP_DROPPED: + retval = am_badarg; + break; + case ERTS_PORT_OP_SCHEDULED: + ASSERT(is_internal_ref(retval)); + break; + case ERTS_PORT_OP_DONE: + ASSERT(is_not_internal_ref(retval)); + break; + default: + ERTS_INTERNAL_ERROR("Unexpected erts_port_control() result"); + retval = am_internal_error; + break; } - p = id_or_name2port(c_p, arg1); - if (!p) { - error: - if (port_resp != port_result && - !(ret_flags & DRIVER_CALL_KEEP_BUFFER)) { - driver_free(port_resp); - } - if (bytes != &port_input[0]) - erts_free(ERTS_ALC_T_PORT_CALL_BUF, bytes); - /* Need to virtual schedule in the process if there - * was an error. - */ - if (IS_TRACED_FL(c_p, F_TRACE_SCHED_PROCS)) { - trace_virtual_sched(c_p, am_in); - } - - if (erts_system_profile_flags.runnable_procs && erts_system_profile_flags.exclusive) { - profile_runnable_proc(c_p, am_active); - } - - if (p) - erts_port_release(p); + state = erts_smp_atomic32_read_acqb(&BIF_P->state); + if (state & (ERTS_PSFLG_EXITING|ERTS_PSFLG_PENDING_EXIT)) { #ifdef ERTS_SMP - ERTS_SMP_BIF_CHK_PENDING_EXIT(c_p, ERTS_PROC_LOCK_MAIN); -#else - ERTS_BIF_CHK_EXITED(c_p); + if (state & ERTS_PSFLG_PENDING_EXIT) + erts_handle_pending_exit(BIF_P, ERTS_PROC_LOCK_MAIN); #endif - BIF_ERROR(c_p, BADARG); - } - - if ((drv = p->drv_ptr) == NULL) { - goto error; - } - if (drv->call == NULL) { - goto error; - } - if (!term_to_Uint(arg2, &op)) { - goto error; - } - p->caller = c_p->id; - - /* Lock taken, virtual schedule of port */ - if (IS_TRACED_FL(p, F_TRACE_SCHED_PORTS)) { - trace_sched_ports_where(p, am_in, am_call); - } - - if (erts_system_profile_flags.runnable_ports && !erts_port_is_scheduled(p)) { - profile_runnable_port(p, am_active); + ERTS_BIF_EXITED(BIF_P); } - size = erts_encode_ext_size(arg3); - if (size > sizeof(port_input)) - bytes = erts_alloc(ERTS_ALC_T_PORT_CALL_BUF, size); - endp = bytes; - erts_encode_ext(arg3, &endp); + BIF_RET(retval); +} - real_size = endp - bytes; - if (real_size > size) { - erl_exit(1, "%s, line %d: buffer overflow: %d word(s)\n", - __FILE__, __LINE__, endp - (bytes + size)); - } - erts_smp_proc_unlock(c_p, ERTS_PROC_LOCK_MAIN); -#ifdef USE_VM_PROBES - if (DTRACE_ENABLED(driver_call)) { - DTRACE_CHARBUF(process_str, DTRACE_TERM_BUF_SIZE); - DTRACE_CHARBUF(port_str, DTRACE_TERM_BUF_SIZE); +/* + * erts_internal:port_close/1 is used by the + * erlang:port_close/1 BIF. + */ +BIF_RETTYPE erts_internal_port_close_1(BIF_ALIST_1) +{ + Eterm ref; + Port *prt; - dtrace_pid_str(p->connected, process_str); - dtrace_port_str(p, port_str); - DTRACE5(driver_call, process_str, port_str, p->name, op, real_size); - } -#endif - prc = (char *) port_resp; - fpe_was_unmasked = erts_block_fpe(); - ret = drv->call((ErlDrvData)p->drv_data, - (unsigned) op, - (char *) bytes, - (int) real_size, - &prc, - (int) sizeof(port_result), - &ret_flags); - erts_unblock_fpe(fpe_was_unmasked); - if (IS_TRACED_FL(p, F_TRACE_SCHED_PORTS)) { - trace_sched_ports_where(p, am_out, am_call); - } - - if (erts_system_profile_flags.runnable_ports && !erts_port_is_scheduled(p)) { - profile_runnable_port(p, am_inactive); - } - - port_resp = (byte *) prc; - p->caller = NIL; - erts_smp_proc_lock(c_p, ERTS_PROC_LOCK_MAIN); -#ifdef HARDDEBUG - { - ErlDrvSizeT z; - printf("real_size = %ld,%d, ret = %ld,%d\r\n", (unsigned long) real_size, - (int) real_size, (unsigned long)ret, (int) ret); - printf("["); - for(z = 0; z < real_size; ++z) { - printf("%d, ",(int) bytes[z]); - } - printf("]\r\n"); - printf("["); - for(z = 0; z < ret; ++z) { - printf("%d, ",(int) port_resp[z]); - } - printf("]\r\n"); - } -#endif - if (ret <= 0 || port_resp[0] != VERSION_MAGIC) { - /* Error or a binary without magic/ with wrong magic */ - goto error; - } - result_size = erts_decode_ext_size(port_resp, ret); - if (result_size < 0) { - goto error; - } - hp = HAlloc(c_p, result_size); - hp_end = hp + result_size; - endp = port_resp; - res = erts_decode_ext(&hp, &MSO(c_p), &endp); - if (res == THE_NON_VALUE) { - goto error; - } - HRelease(c_p, hp_end, hp); - if (port_resp != port_result && !(ret_flags & DRIVER_CALL_KEEP_BUFFER)) { - driver_free(port_resp); - } - if (bytes != &port_input[0]) - erts_free(ERTS_ALC_T_PORT_CALL_BUF, bytes); - if (p) - erts_port_release(p); -#ifdef ERTS_SMP - ERTS_SMP_BIF_CHK_PENDING_EXIT(c_p, ERTS_PROC_LOCK_MAIN); -#else - ERTS_BIF_CHK_EXITED(c_p); +#ifdef DEBUG + ref = NIL; #endif - if (IS_TRACED_FL(c_p, F_TRACE_SCHED_PROCS)) { - trace_virtual_sched(c_p, am_in); - } - if (erts_system_profile_flags.runnable_procs && erts_system_profile_flags.exclusive) { - profile_runnable_proc(c_p, am_active); + prt = lookup_port(BIF_P, BIF_ARG_1); + if (!prt) + BIF_RET(am_badarg); + + + switch (erts_port_exit(BIF_P, 0, prt, prt->common.id, am_normal, &ref)) { + case ERTS_PORT_OP_CALLER_EXIT: + case ERTS_PORT_OP_BADARG: + case ERTS_PORT_OP_DROPPED: + BIF_RET(am_badarg); + case ERTS_PORT_OP_SCHEDULED: + ASSERT(is_internal_ref(ref)); + BIF_RET(ref); + case ERTS_PORT_OP_DONE: + BIF_RET(am_true); + default: + ERTS_INTERNAL_ERROR("Unexpected erts_port_exit() result"); + BIF_RET(am_internal_error); } - - return res; } - -BIF_RETTYPE port_control_3(BIF_ALIST_3) -{ - Port* p; - Uint op; - Eterm res = THE_NON_VALUE; - - /* Virtual schedule out calling process before lock wait */ - if (IS_TRACED_FL(BIF_P, F_TRACE_SCHED_PROCS)) { - trace_virtual_sched(BIF_P, am_out); - } - if (erts_system_profile_flags.runnable_procs && erts_system_profile_flags.exclusive) { - profile_runnable_proc(BIF_P, am_inactive); - } +/* + * erts_internal:port_connect/2 is used by the + * erlang:port_connect/2 BIF. + */ +BIF_RETTYPE erts_internal_port_connect_2(BIF_ALIST_2) +{ + Eterm ref; + Port* prt; - p = id_or_name2port(BIF_P, BIF_ARG_1); - if (!p) { - /* Schedule the process before exiting */ - if (IS_TRACED_FL(BIF_P, F_TRACE_SCHED_PROCS)) { - trace_virtual_sched(BIF_P, am_in); - } - - if (erts_system_profile_flags.runnable_procs && erts_system_profile_flags.exclusive) { - profile_runnable_proc(BIF_P, am_active); - } - - BIF_ERROR(BIF_P, BADARG); - } + prt = lookup_port(BIF_P, BIF_ARG_1); + if (!prt) + BIF_RET(am_badarg); - /* Trace the port for scheduling in */ - if (IS_TRACED_FL(p, F_TRACE_SCHED_PORTS)) { - trace_sched_ports_where(p, am_in, am_control); - } - - if (erts_system_profile_flags.runnable_ports && !erts_port_is_scheduled(p)) { - profile_runnable_port(p, am_active); - } +#ifdef DEBUG + ref = NIL; +#endif - if (term_to_Uint(BIF_ARG_2, &op)) - res = erts_port_control(BIF_P, p, op, BIF_ARG_3); - - /* Trace the port for scheduling out */ - if (IS_TRACED_FL(p, F_TRACE_SCHED_PORTS)) { - trace_sched_ports_where(p, am_out, am_control); + switch (erts_port_connect(BIF_P, 0, prt, prt->common.id, BIF_ARG_2, &ref)) { + case ERTS_PORT_OP_CALLER_EXIT: + case ERTS_PORT_OP_BADARG: + case ERTS_PORT_OP_DROPPED: + BIF_RET(am_badarg); + case ERTS_PORT_OP_SCHEDULED: + ASSERT(is_internal_ref(ref)); + BIF_RET(ref); + break; + case ERTS_PORT_OP_DONE: + BIF_RET(am_true); + break; + default: + ERTS_INTERNAL_ERROR("Unexpected erts_port_connect() result"); + BIF_RET(am_internal_error); } +} - if (erts_system_profile_flags.runnable_ports && !erts_port_is_scheduled(p)) { - profile_runnable_port(p, am_inactive); - } +BIF_RETTYPE erts_internal_port_info_1(BIF_ALIST_1) +{ + Eterm retval; + Port* prt; - erts_port_release(p); -#ifdef ERTS_SMP - ERTS_SMP_BIF_CHK_PENDING_EXIT(BIF_P, ERTS_PROC_LOCK_MAIN); -#else - ERTS_BIF_CHK_EXITED(BIF_P); -#endif - - if (IS_TRACED_FL(BIF_P, F_TRACE_SCHED_PROCS)) { - trace_virtual_sched(BIF_P, am_in); + if (is_internal_port(BIF_ARG_1) || is_atom(BIF_ARG_1)) { + prt = lookup_port(BIF_P, BIF_ARG_1); + if (!prt) + BIF_RET(am_undefined); } - - if (erts_system_profile_flags.runnable_procs && erts_system_profile_flags.exclusive) { - profile_runnable_proc(BIF_P, am_active); + else if (is_external_port(BIF_ARG_1)) { + if (external_port_dist_entry(BIF_ARG_1) == erts_this_dist_entry) + BIF_RET(am_undefined); + else + BIF_RET(am_badarg); } - - if (is_non_value(res)) { - BIF_ERROR(BIF_P, BADARG); + else { + BIF_RET(am_badarg); + } + + switch (erts_port_info(BIF_P, prt, THE_NON_VALUE, &retval)) { + case ERTS_PORT_OP_CALLER_EXIT: + case ERTS_PORT_OP_BADARG: + BIF_RET(am_badarg); + case ERTS_PORT_OP_DROPPED: + BIF_RET(am_undefined); + case ERTS_PORT_OP_SCHEDULED: + ASSERT(is_internal_ref(retval)); + BIF_RET(retval); + case ERTS_PORT_OP_DONE: + ASSERT(is_not_internal_ref(retval)); + BIF_RET(retval); + default: + ERTS_INTERNAL_ERROR("Unexpected erts_port_info() result"); + BIF_RET(am_internal_error); } - BIF_RET(res); } -BIF_RETTYPE port_close_1(BIF_ALIST_1) -{ - Port* p; - erts_smp_proc_unlock(BIF_P, ERTS_PROC_LOCK_MAIN); - p = id_or_name2port(NULL, BIF_ARG_1); - if (!p) { - erts_smp_proc_lock(BIF_P, ERTS_PROC_LOCK_MAIN); - BIF_ERROR(BIF_P, BADARG); - } - erts_do_exit_port(p, p->connected, am_normal); - /* if !ERTS_SMP: since we terminate port with reason normal - we SHOULD never get an exit signal ourselves - */ - erts_port_release(p); - erts_smp_proc_lock(BIF_P, ERTS_PROC_LOCK_MAIN); - BIF_RET(am_true); -} -BIF_RETTYPE port_connect_2(BIF_ALIST_2) +BIF_RETTYPE erts_internal_port_info_2(BIF_ALIST_2) { + Eterm retval; Port* prt; - Process* rp; - Eterm pid = BIF_ARG_2; - if (is_not_internal_pid(pid)) { - error: - BIF_ERROR(BIF_P, BADARG); + if (is_internal_port(BIF_ARG_1) || is_atom(BIF_ARG_1)) { + prt = lookup_port(BIF_P, BIF_ARG_1); + if (!prt) + BIF_RET(am_undefined); } - prt = id_or_name2port(BIF_P, BIF_ARG_1); - if (!prt) { - goto error; + else if (is_external_port(BIF_ARG_1)) { + if (external_port_dist_entry(BIF_ARG_1) == erts_this_dist_entry) + BIF_RET(am_undefined); + else + BIF_RET(am_badarg); } - - rp = erts_pid2proc(BIF_P, ERTS_PROC_LOCK_MAIN, - pid, ERTS_PROC_LOCK_LINK); - if (!rp) { - erts_smp_port_unlock(prt); - ERTS_SMP_ASSERT_IS_NOT_EXITING(BIF_P); - goto error; - } - - erts_add_link(&(rp->nlinks), LINK_PID, prt->id); - erts_add_link(&(prt->nlinks), LINK_PID, pid); - - erts_smp_proc_unlock(rp, ERTS_PROC_LOCK_LINK); - - prt->connected = pid; /* internal pid */ - erts_smp_port_unlock(prt); -#ifdef USE_VM_PROBES - if (DTRACE_ENABLED(port_connect)) { - DTRACE_CHARBUF(process_str, DTRACE_TERM_BUF_SIZE); - DTRACE_CHARBUF(port_str, DTRACE_TERM_BUF_SIZE); - DTRACE_CHARBUF(newprocess_str, DTRACE_TERM_BUF_SIZE); - - dtrace_pid_str(prt->connected, process_str); - erts_snprintf(port_str, sizeof(port_str), "%T", prt->id); - dtrace_proc_str(rp, newprocess_str); - DTRACE4(port_connect, process_str, port_str, prt->name, newprocess_str); + else { + BIF_RET(am_badarg); + } + + switch (erts_port_info(BIF_P, prt, BIF_ARG_2, &retval)) { + case ERTS_PORT_OP_CALLER_EXIT: + case ERTS_PORT_OP_BADARG: + BIF_RET(am_badarg); + case ERTS_PORT_OP_DROPPED: + BIF_RET(am_undefined); + case ERTS_PORT_OP_SCHEDULED: + ASSERT(is_internal_ref(retval)); + BIF_RET(retval); + case ERTS_PORT_OP_DONE: + ASSERT(is_not_internal_ref(retval)); + BIF_RET(retval); + default: + ERTS_INTERNAL_ERROR("Unexpected erts_port_info() result"); + BIF_RET(am_internal_error); } -#endif - BIF_RET(am_true); } -BIF_RETTYPE port_set_data_2(BIF_ALIST_2) + +BIF_RETTYPE erts_internal_port_set_data_2(BIF_ALIST_2) { + Eterm ref; Port* prt; - Eterm portid = BIF_ARG_1; - Eterm data = BIF_ARG_2; - prt = id_or_name2port(BIF_P, portid); - if (!prt) { - BIF_ERROR(BIF_P, BADARG); - } - if (prt->bp != NULL) { - free_message_buffer(prt->bp); - prt->bp = NULL; - } - if (IS_CONST(data)) { - prt->data = data; - } else { - Uint size; - ErlHeapFragment* bp; - Eterm* hp; - - size = size_object(data); - prt->bp = bp = new_message_buffer(size); - hp = bp->mem; - prt->data = copy_struct(data, size, &hp, &bp->off_heap); + prt = lookup_port(BIF_P, BIF_ARG_1); + if (!prt) + BIF_RET(am_badarg); + + switch (erts_port_set_data(BIF_P, prt, BIF_ARG_2, &ref)) { + case ERTS_PORT_OP_CALLER_EXIT: + case ERTS_PORT_OP_BADARG: + case ERTS_PORT_OP_DROPPED: + BIF_RET(am_badarg); + case ERTS_PORT_OP_SCHEDULED: + ASSERT(is_internal_ref(ref)); + BIF_RET(ref); + case ERTS_PORT_OP_DONE: + BIF_RET(am_true); + default: + ERTS_INTERNAL_ERROR("Unexpected erts_port_set_data() result"); + BIF_RET(am_internal_error); } - erts_smp_port_unlock(prt); - BIF_RET(am_true); } -BIF_RETTYPE port_get_data_1(BIF_ALIST_1) +BIF_RETTYPE erts_internal_port_get_data_1(BIF_ALIST_1) { - BIF_RETTYPE res; + Eterm retval; Port* prt; - Eterm portid = BIF_ARG_1; - prt = id_or_name2port(BIF_P, portid); - if (!prt) { - BIF_ERROR(BIF_P, BADARG); - } - if (prt->bp == NULL) { /* MUST be CONST! */ - res = prt->data; - } else { - Eterm* hp = HAlloc(BIF_P, prt->bp->used_size); - res = copy_struct(prt->data, prt->bp->used_size, &hp, &MSO(BIF_P)); + prt = lookup_port(BIF_P, BIF_ARG_1); + if (!prt) + BIF_RET(am_badarg); + + switch (erts_port_get_data(BIF_P, prt, &retval)) { + case ERTS_PORT_OP_CALLER_EXIT: + case ERTS_PORT_OP_BADARG: + case ERTS_PORT_OP_DROPPED: + BIF_RET(am_badarg); + case ERTS_PORT_OP_SCHEDULED: + ASSERT(is_internal_ref(retval)); + BIF_RET(retval); + case ERTS_PORT_OP_DONE: + ASSERT(is_not_internal_ref(retval)); + BIF_RET(retval); + default: + ERTS_INTERNAL_ERROR("Unexpected erts_port_get_data() result"); + BIF_RET(am_internal_error); } - erts_smp_port_unlock(prt); - BIF_RET(res); } /* @@ -625,11 +485,10 @@ BIF_RETTYPE port_get_data_1(BIF_ALIST_1) * either BADARG or SYSTEM_LIMIT). */ -static int -open_port(Process* p, Eterm name, Eterm settings, int *err_nump) +static Port * +open_port(Process* p, Eterm name, Eterm settings, int *err_typep, int *err_nump) { -#define OPEN_PORT_ERROR(VAL) do { port_num = (VAL); goto do_return; } while (0) - int i, port_num; + int i; Eterm option; Uint arity; Eterm* tp; @@ -637,11 +496,11 @@ open_port(Process* p, Eterm name, Eterm settings, int *err_nump) erts_driver_t* driver; char* name_buf = NULL; SysDriverOpts opts; - int binary_io; - int soft_eof; Sint linebuf; Eterm edir = NIL; byte dir[MAXPATHLEN]; + erts_aint32_t sflgs = 0; + Port *port; /* These are the defaults */ opts.packet_bytes = 0; @@ -655,8 +514,7 @@ open_port(Process* p, Eterm name, Eterm settings, int *err_nump) opts.overlapped_io = 0; opts.spawn_type = ERTS_SPAWN_ANY; opts.argv = NULL; - binary_io = 0; - soft_eof = 0; + opts.parallelism = erts_port_parallelism; linebuf = 0; *err_nump = 0; @@ -734,6 +592,13 @@ open_port(Process* p, Eterm name, Eterm settings, int *err_nump) } } else if (option == am_cd) { edir = *tp; + } else if (option == am_parallelism) { + if (*tp == am_true) + opts.parallelism = 1; + else if (*tp == am_false) + opts.parallelism = 0; + else + goto badarg; } else { goto badarg; } @@ -748,13 +613,13 @@ open_port(Process* p, Eterm name, Eterm settings, int *err_nump) } else if (*nargs == am_nouse_stdio) { opts.use_stdio = 0; } else if (*nargs == am_binary) { - binary_io = 1; + sflgs |= ERTS_PORT_SFLG_BINARY_IO; } else if (*nargs == am_in) { opts.read_write |= DO_READ; } else if (*nargs == am_out) { opts.read_write |= DO_WRITE; } else if (*nargs == am_eof) { - soft_eof = 1; + sflgs |= ERTS_PORT_SFLG_SOFT_EOF; } else if (*nargs == am_hide) { opts.hide_window = 1; } else if (*nargs == am_exit_status) { @@ -902,9 +767,9 @@ open_port(Process* p, Eterm name, Eterm settings, int *err_nump) heap[2] = make_small(0); heap[3] = NIL; iolist = make_list(heap); - r = io_list_to_buf(iolist, (char*) dir, MAXPATHLEN); + r = erts_iolist_to_buf(iolist, (char*) dir, MAXPATHLEN); UnUseTmpHeap(4,p); - if (r < 0) { + if (ERTS_IOLIST_TO_BUF_FAILED(r)) { goto badarg; } opts.wd = (char *) dir; @@ -926,44 +791,40 @@ open_port(Process* p, Eterm name, Eterm settings, int *err_nump) erts_smp_proc_unlock(p, ERTS_PROC_LOCK_MAIN); - port_num = erts_open_driver(driver, p->id, name_buf, &opts, err_nump); + port = erts_open_driver(driver, p->common.id, name_buf, &opts, err_typep, err_nump); #ifdef USE_VM_PROBES - if (port_num >= 0 && DTRACE_ENABLED(port_open)) { + if (port && DTRACE_ENABLED(port_open)) { DTRACE_CHARBUF(process_str, DTRACE_TERM_BUF_SIZE); DTRACE_CHARBUF(port_str, DTRACE_TERM_BUF_SIZE); dtrace_proc_str(p, process_str); - erts_snprintf(port_str, sizeof(port_str), "%T", erts_port[port_num].id); + erts_snprintf(port_str, sizeof(port_str), "%T", port->common.id); DTRACE3(port_open, process_str, name_buf, port_str); } #endif erts_smp_proc_lock(p, ERTS_PROC_LOCK_MAIN); - if (port_num < 0) { - DEBUGF(("open_driver returned %d(%d)\n", port_num, *err_nump)); + if (!port) { + DEBUGF(("open_driver returned (%d:%d)\n", + err_typep ? *err_typep : 4711, + err_nump ? *err_nump : 4711)); if (IS_TRACED_FL(p, F_TRACE_SCHED_PROCS)) { trace_virtual_sched(p, am_in); } - OPEN_PORT_ERROR(port_num); + goto do_return; } if (IS_TRACED_FL(p, F_TRACE_SCHED_PROCS)) { trace_virtual_sched(p, am_in); } - if (binary_io) { - erts_port_status_bor_set(&erts_port[port_num], - ERTS_PORT_SFLG_BINARY_IO); - } - if (soft_eof) { - erts_port_status_bor_set(&erts_port[port_num], - ERTS_PORT_SFLG_SOFT_EOF); - } - if (linebuf && erts_port[port_num].linebuf == NULL){ - erts_port[port_num].linebuf = allocate_linebuf(linebuf); - erts_port_status_bor_set(&erts_port[port_num], - ERTS_PORT_SFLG_LINEBUF_IO); + if (linebuf && port->linebuf == NULL){ + port->linebuf = allocate_linebuf(linebuf); + sflgs |= ERTS_PORT_SFLG_LINEBUF_IO; } + + if (sflgs) + erts_atomic32_read_bor_relb(&port->state, sflgs); do_return: if (name_buf) @@ -974,13 +835,15 @@ open_port(Process* p, Eterm name, Eterm settings, int *err_nump) if (opts.wd && opts.wd != ((char *)dir)) { erts_free(ERTS_ALC_T_TMP, (void *) opts.wd); } - return port_num; + return port; badarg: - *err_nump = BADARG; - OPEN_PORT_ERROR(-3); + if (err_typep) + *err_typep = -3; + if (err_nump) + *err_nump = BADARG; + port = NULL; goto do_return; -#undef OPEN_PORT_ERROR } /* Arguments can be given i unicode and as raw binaries, convert filename is used to convert */ diff --git a/erts/emulator/beam/erl_bif_re.c b/erts/emulator/beam/erl_bif_re.c index b036c5ef5c..88f980d19f 100644 --- a/erts/emulator/beam/erl_bif_re.c +++ b/erts/emulator/beam/erl_bif_re.c @@ -413,7 +413,7 @@ build_compile_result(Process *p, Eterm error_tag, pcre *result, int errcode, con static BIF_RETTYPE re_compile(Process* p, Eterm arg1, Eterm arg2) { - Uint slen; + ErlDrvSizeT slen; char *expr; pcre *result; int errcode = 0; @@ -444,7 +444,7 @@ re_compile(Process* p, Eterm arg1, Eterm arg2) BIF_ERROR(p,BADARG); } expr = erts_alloc(ERTS_ALC_T_RE_TMP_BUF, slen + 1); - if (io_list_to_buf(arg1, expr, slen) != 0) { + if (erts_iolist_to_buf(arg1, expr, slen) != 0) { erts_free(ERTS_ALC_T_RE_TMP_BUF, expr); BIF_ERROR(p,BADARG); } @@ -797,7 +797,7 @@ build_capture(Eterm capture_spec[CAPSPEC_SIZE], const pcre *code) memcpy(tmpb,ap->name,ap->len); tmpb[ap->len] = '\0'; } else { - Uint slen; + ErlDrvSizeT slen; if (erts_iolist_size(val, &slen)) { goto error; } @@ -809,7 +809,7 @@ build_capture(Eterm capture_spec[CAPSPEC_SIZE], const pcre *code) (tmpbsiz = slen + 1)); } } - if (io_list_to_buf(val, tmpb, slen) != 0) { + if (erts_iolist_to_buf(val, tmpb, slen) != 0) { goto error; } tmpb[slen] = '\0'; @@ -853,7 +853,7 @@ re_run(Process *p, Eterm arg1, Eterm arg2, Eterm arg3) const pcre *code_tmp; RestartContext restart; byte *temp_alloc = NULL; - Uint slength; + ErlDrvSizeT slength; int startoffset = 0; int options = 0, comp_options = 0; int ovsize; @@ -877,7 +877,7 @@ re_run(Process *p, Eterm arg1, Eterm arg2, Eterm arg3) if (is_not_tuple(arg2) || (arityval(*tuple_val(arg2)) != 4)) { if (is_binary(arg2) || is_list(arg2) || is_nil(arg2)) { /* Compile from textual RE */ - Uint slen; + ErlDrvSizeT slen; char *expr; pcre *result; int errcode = 0; @@ -896,7 +896,7 @@ re_run(Process *p, Eterm arg1, Eterm arg2, Eterm arg3) } expr = erts_alloc(ERTS_ALC_T_RE_TMP_BUF, slen + 1); - if (io_list_to_buf(arg2, expr, slen) != 0) { + if (erts_iolist_to_buf(arg2, expr, slen) != 0) { erts_free(ERTS_ALC_T_RE_TMP_BUF, expr); BIF_ERROR(p,BADARG); } @@ -1039,7 +1039,7 @@ handle_iolist: } restart.subject = erts_alloc(ERTS_ALC_T_RE_SUBJECT, slength); - if (io_list_to_buf(arg1, restart.subject, slength) != 0) { + if (erts_iolist_to_buf(arg1, restart.subject, slength) != 0) { erts_free(ERTS_ALC_T_RE_SUBJECT, restart.ovector); erts_free(ERTS_ALC_T_RE_SUBJECT, restart.code); erts_free(ERTS_ALC_T_RE_SUBJECT, restart.subject); diff --git a/erts/emulator/beam/erl_bif_timer.c b/erts/emulator/beam/erl_bif_timer.c index 525b11f61c..d67695e533 100644 --- a/erts/emulator/beam/erl_bif_timer.c +++ b/erts/emulator/beam/erl_bif_timer.c @@ -265,10 +265,10 @@ link_proc(Process *p, ErtsBifTimer* btm) { btm->receiver.proc.ess = p; btm->receiver.proc.prev = NULL; - btm->receiver.proc.next = p->bif_timers; - if (p->bif_timers) - p->bif_timers->receiver.proc.prev = btm; - p->bif_timers = btm; + btm->receiver.proc.next = p->u.bif_timers; + if (p->u.bif_timers) + p->u.bif_timers->receiver.proc.prev = btm; + p->u.bif_timers = btm; } static ERTS_INLINE void @@ -277,7 +277,7 @@ unlink_proc(ErtsBifTimer* btm) if (btm->receiver.proc.prev) btm->receiver.proc.prev->receiver.proc.next = btm->receiver.proc.next; else - btm->receiver.proc.ess->bif_timers = btm->receiver.proc.next; + btm->receiver.proc.ess->u.bif_timers = btm->receiver.proc.next; if (btm->receiver.proc.next) btm->receiver.proc.next->receiver.proc.prev = btm->receiver.proc.prev; } @@ -613,7 +613,7 @@ erts_print_bif_timer_info(int to, void *to_arg) for (btm = bif_timer_tab[i]; btm; btm = btm->tab.next) { Eterm receiver = (btm->flags & BTM_FLG_BYNAME ? btm->receiver.name - : btm->receiver.proc.ess->id); + : btm->receiver.proc.ess->common.id); erts_print(to, to_arg, "=timer:%T\n", receiver); erts_print(to, to_arg, "Message: %T\n", btm->message); erts_print(to, to_arg, "Time left: %u ms\n", @@ -637,7 +637,7 @@ erts_cancel_bif_timers(Process *p, ErtsProcLocks plocks) erts_smp_proc_lock(p, plocks); } - btm = p->bif_timers; + btm = p->u.bif_timers; while (btm) { ErtsBifTimer *tmp_btm; ASSERT(!(btm->flags & BTM_FLG_CANCELED)); @@ -647,7 +647,7 @@ erts_cancel_bif_timers(Process *p, ErtsProcLocks plocks) erts_cancel_timer(&tmp_btm->tm); } - p->bif_timers = NULL; + p->u.bif_timers = NULL; erts_smp_btm_rwunlock(); } @@ -696,7 +696,7 @@ erts_bif_timer_foreach(void (*func)(Eterm, Eterm, ErlHeapFragment *, void *), for (btm = bif_timer_tab[i]; btm; btm = btm->tab.next) { (*func)((btm->flags & BTM_FLG_BYNAME ? btm->receiver.name - : btm->receiver.proc.ess->id), + : btm->receiver.proc.ess->common.id), btm->message, btm->bp, arg); diff --git a/erts/emulator/beam/erl_bif_trace.c b/erts/emulator/beam/erl_bif_trace.c index 805d788177..99a4394666 100644 --- a/erts/emulator/beam/erl_bif_trace.c +++ b/erts/emulator/beam/erl_bif_trace.c @@ -124,7 +124,7 @@ trace_pattern(Process* p, Eterm MFA, Eterm Pattern, Eterm flaglist) struct trace_pattern_flags flags = erts_trace_pattern_flags_off; int is_global; Process *meta_tracer_proc = p; - Eterm meta_tracer_pid = p->id; + Eterm meta_tracer_pid = p->common.id; if (!erts_try_seize_code_write_permission(p)) { ERTS_BIF_YIELD3(bif_export[BIF_trace_pattern_3], p, MFA, Pattern, flaglist); @@ -171,14 +171,12 @@ trace_pattern(Process* p, Eterm MFA, Eterm Pattern, Eterm flaglist) } } else if (is_internal_port(meta_tracer_pid)) { Port *meta_tracer_port; - meta_tracer_proc = NULL; - if (internal_port_index(meta_tracer_pid) >= erts_max_ports) + meta_tracer_proc = NULL; + meta_tracer_port = (erts_port_lookup( + meta_tracer_pid, + ERTS_PORT_SFLGS_INVALID_TRACER_LOOKUP)); + if (!meta_tracer_port) goto error; - meta_tracer_port = - &erts_port[internal_port_index(meta_tracer_pid)]; - if (INVALID_TRACER_PORT(meta_tracer_port, meta_tracer_pid)) { - goto error; - } } else { goto error; } @@ -254,7 +252,7 @@ trace_pattern(Process* p, Eterm MFA, Eterm Pattern, Eterm flaglist) MatchSetRef(erts_default_meta_match_spec); erts_default_meta_tracer_pid = meta_tracer_pid; if (meta_tracer_proc) { - meta_tracer_proc->trace_flags |= F_TRACER; + ERTS_TRACE_FLAGS(meta_tracer_proc) |= F_TRACER; } } else if (! flags.breakpoint) { MatchSetUnref(erts_default_meta_match_spec); @@ -342,7 +340,7 @@ trace_pattern(Process* p, Eterm MFA, Eterm Pattern, Eterm flaglist) } if (meta_tracer_proc) { - meta_tracer_proc->trace_flags |= F_TRACER; + ERTS_TRACE_FLAGS(meta_tracer_proc) |= F_TRACER; } matches = erts_set_trace_pattern(p, mfa, specified, @@ -526,24 +524,24 @@ Eterm trace_3(BIF_ALIST_3) if (is_nil(tracer) || is_internal_pid(tracer)) { Process *tracer_proc = erts_pid2proc(p, ERTS_PROC_LOCK_MAIN, - is_nil(tracer) ? p->id : tracer, + is_nil(tracer) ? p->common.id : tracer, ERTS_PROC_LOCKS_ALL); if (!tracer_proc) goto error; - tracer_proc->trace_flags |= F_TRACER; + ERTS_TRACE_FLAGS(tracer_proc) |= F_TRACER; erts_smp_proc_unlock(tracer_proc, (tracer_proc == p ? ERTS_PROC_LOCKS_ALL_MINOR : ERTS_PROC_LOCKS_ALL)); } else if (is_internal_port(tracer)) { - Port *tracer_port = erts_id2port(tracer, p, ERTS_PROC_LOCK_MAIN); - if (!erts_is_valid_tracer_port(tracer)) { - if (tracer_port) - erts_smp_port_unlock(tracer_port); + Port *tracer_port = erts_id2port_sflgs(tracer, + p, + ERTS_PROC_LOCK_MAIN, + ERTS_PORT_SFLGS_INVALID_TRACER_LOOKUP); + if (!tracer_port) goto error; - } - tracer_port->trace_flags |= F_TRACER; - erts_smp_port_unlock(tracer_port); + ERTS_TRACE_FLAGS(tracer_port) |= F_TRACER; + erts_port_release(tracer_port); } else goto error; @@ -554,7 +552,7 @@ Eterm trace_3(BIF_ALIST_3) case am_true: on = 1; if (is_nil(tracer)) - tracer = p->id; + tracer = p->common.id; break; default: goto error; @@ -576,26 +574,29 @@ Eterm trace_3(BIF_ALIST_3) if (pid_spec == tracer) goto error; - tracee_port = erts_id2port(pid_spec, p, ERTS_PROC_LOCK_MAIN); + tracee_port = erts_id2port_sflgs(pid_spec, + p, + ERTS_PROC_LOCK_MAIN, + ERTS_PORT_SFLGS_INVALID_LOOKUP); if (!tracee_port) goto error; if (tracer != NIL && port_already_traced(p, tracee_port, tracer)) { - erts_smp_port_unlock(tracee_port); + erts_port_release(tracee_port); goto already_traced; } if (on) - tracee_port->trace_flags |= mask; + ERTS_TRACE_FLAGS(tracee_port) |= mask; else - tracee_port->trace_flags &= ~mask; + ERTS_TRACE_FLAGS(tracee_port) &= ~mask; - if (!tracee_port->trace_flags) - tracee_port->tracer_proc = NIL; + if (!ERTS_TRACE_FLAGS(tracee_port)) + ERTS_TRACER_PROC(tracee_port) = NIL; else if (tracer != NIL) - tracee_port->tracer_proc = tracer; + ERTS_TRACER_PROC(tracee_port) = tracer; - erts_smp_port_unlock(tracee_port); + erts_port_release(tracee_port); matches = 1; } else if (is_pid(pid_spec)) { @@ -627,14 +628,14 @@ Eterm trace_3(BIF_ALIST_3) } if (on) - tracee_p->trace_flags |= mask; + ERTS_TRACE_FLAGS(tracee_p) |= mask; else - tracee_p->trace_flags &= ~mask; + ERTS_TRACE_FLAGS(tracee_p) &= ~mask; - if ((tracee_p->trace_flags & TRACEE_FLAGS) == 0) - tracee_p->tracer_proc = NIL; + if ((ERTS_TRACE_FLAGS(tracee_p) & TRACEE_FLAGS) == 0) + ERTS_TRACER_PROC(tracee_p) = NIL; else if (tracer != NIL) - tracee_p->tracer_proc = tracer; + ERTS_TRACER_PROC(tracee_p) = tracer; erts_smp_proc_unlock(tracee_p, (tracee_p == p @@ -708,47 +709,56 @@ Eterm trace_3(BIF_ALIST_3) ok = 1; if (procs || mods) { + int max = erts_ptab_max(&erts_proc); /* tracing of processes */ - for (i = 0; i < erts_max_processes; i++) { + for (i = 0; i < max; i++) { Process* tracee_p = erts_pix2proc(i); if (! tracee_p) continue; if (tracer != NIL) { - if (tracee_p->id == tracer) + if (tracee_p->common.id == tracer) continue; if (already_traced(NULL, tracee_p, tracer)) continue; } if (on) { - tracee_p->trace_flags |= mask; + ERTS_TRACE_FLAGS(tracee_p) |= mask; } else { - tracee_p->trace_flags &= ~mask; + ERTS_TRACE_FLAGS(tracee_p) &= ~mask; } - if(!(tracee_p->trace_flags & TRACEE_FLAGS)) { - tracee_p->tracer_proc = NIL; + if(!(ERTS_TRACE_FLAGS(tracee_p) & TRACEE_FLAGS)) { + ERTS_TRACER_PROC(tracee_p) = NIL; } else if (tracer != NIL) { - tracee_p->tracer_proc = tracer; + ERTS_TRACER_PROC(tracee_p) = tracer; } matches++; } } if (ports || mods) { + int max = erts_ptab_max(&erts_port); /* tracing of ports */ - for (i = 0; i < erts_max_ports; i++) { - Port *tracee_port = &erts_port[i]; - if (tracee_port->status & ERTS_PORT_SFLGS_DEAD) continue; + for (i = 0; i < max; i++) { + erts_aint32_t state; + Port *tracee_port = erts_pix2port(i); + if (!tracee_port) + continue; + state = erts_atomic32_read_nob(&tracee_port->state); + if (state & ERTS_PORT_SFLGS_DEAD) + continue; if (tracer != NIL) { - if (tracee_port->id == tracer) continue; - if (port_already_traced(NULL, tracee_port, tracer)) continue; + if (tracee_port->common.id == tracer) + continue; + if (port_already_traced(NULL, tracee_port, tracer)) + continue; } - if (on) tracee_port->trace_flags |= mask; - else tracee_port->trace_flags &= ~mask; + if (on) ERTS_TRACE_FLAGS(tracee_port) |= mask; + else ERTS_TRACE_FLAGS(tracee_port) &= ~mask; - if (!(tracee_port->trace_flags & TRACEE_FLAGS)) { - tracee_port->tracer_proc = NIL; + if (!(ERTS_TRACE_FLAGS(tracee_port) & TRACEE_FLAGS)) { + ERTS_TRACER_PROC(tracee_port) = NIL; } else if (tracer != NIL) { - tracee_port->tracer_proc = tracer; + ERTS_TRACER_PROC(tracee_port) = tracer; } /* matches are not counted for ports since it would violate compatibility */ /* This could be a reason to modify this function or make a new one. */ @@ -817,20 +827,20 @@ static int port_already_traced(Process *c_p, Port *tracee_port, Eterm tracer) * * main lock is held on c_p * * all locks are held on port tracee_p */ - if ((tracee_port->trace_flags & TRACEE_FLAGS) - && tracee_port->tracer_proc != tracer) { + if ((ERTS_TRACE_FLAGS(tracee_port) & TRACEE_FLAGS) + && ERTS_TRACER_PROC(tracee_port) != tracer) { /* This tracee is already being traced, and not by the * tracer to be */ - if (is_internal_port(tracee_port->tracer_proc)) { - if (!erts_is_valid_tracer_port(tracee_port->tracer_proc)) { + if (is_internal_port(ERTS_TRACER_PROC(tracee_port))) { + if (!erts_is_valid_tracer_port(ERTS_TRACER_PROC(tracee_port))) { /* Current trace port now invalid * - discard it and approve the new. */ goto remove_tracer; } else return 1; } - else if(is_internal_pid(tracee_port->tracer_proc)) { - Process *tracer_p = erts_proc_lookup(tracee_port->tracer_proc); + else if(is_internal_pid(ERTS_TRACER_PROC(tracee_port))) { + Process *tracer_p = erts_proc_lookup(ERTS_TRACER_PROC(tracee_port)); if (!tracer_p) { /* Current trace process now invalid * - discard it and approve the new. */ @@ -840,8 +850,8 @@ static int port_already_traced(Process *c_p, Port *tracee_port, Eterm tracer) } else { remove_tracer: - tracee_port->trace_flags &= ~TRACEE_FLAGS; - tracee_port->tracer_proc = NIL; + ERTS_TRACE_FLAGS(tracee_port) &= ~TRACEE_FLAGS; + ERTS_TRACER_PROC(tracee_port) = NIL; } } return 0; @@ -857,20 +867,22 @@ static int already_traced(Process *c_p, Process *tracee_p, Eterm tracer) * * main lock is held on c_p * * all locks multiple are held on tracee_p */ - if ((tracee_p->trace_flags & TRACEE_FLAGS) - && tracee_p->tracer_proc != tracer) { + if ((ERTS_TRACE_FLAGS(tracee_p) & TRACEE_FLAGS) + && ERTS_TRACER_PROC(tracee_p) != tracer) { /* This tracee is already being traced, and not by the * tracer to be */ - if (is_internal_port(tracee_p->tracer_proc)) { - if (!erts_is_valid_tracer_port(tracee_p->tracer_proc)) { + if (is_internal_port(ERTS_TRACER_PROC(tracee_p))) { + if (!erts_is_valid_tracer_port(ERTS_TRACER_PROC(tracee_p))) { /* Current trace port now invalid * - discard it and approve the new. */ goto remove_tracer; } else return 1; } - else if(is_internal_pid(tracee_p->tracer_proc)) { - Process *tracer_p = erts_proc_lookup(tracee_p->tracer_proc); + else if(is_internal_pid(ERTS_TRACER_PROC(tracee_p))) { + Process *tracer_p; + + tracer_p = erts_proc_lookup(ERTS_TRACER_PROC(tracee_p)); if (!tracer_p) { /* Current trace process now invalid * - discard it and approve the new. */ @@ -880,8 +892,8 @@ static int already_traced(Process *c_p, Process *tracee_p, Eterm tracer) } else { remove_tracer: - tracee_p->trace_flags &= ~TRACEE_FLAGS; - tracee_p->tracer_proc = NIL; + ERTS_TRACE_FLAGS(tracee_p) &= ~TRACEE_FLAGS; + ERTS_TRACER_PROC(tracee_p) = NIL; } } return 0; @@ -925,8 +937,7 @@ trace_info_pid(Process* p, Eterm pid_spec, Eterm key) if (pid_spec == am_new) { erts_get_default_tracing(&trace_flags, &tracer); - } else if (is_internal_pid(pid_spec) - && internal_pid_index(pid_spec) < erts_max_processes) { + } else if (is_internal_pid(pid_spec)) { Process *tracee; tracee = erts_pid2proc(p, ERTS_PROC_LOCK_MAIN, pid_spec, ERTS_PROC_LOCKS_ALL); @@ -934,16 +945,16 @@ trace_info_pid(Process* p, Eterm pid_spec, Eterm key) if (!tracee) { return am_undefined; } else { - tracer = tracee->tracer_proc; - trace_flags = tracee->trace_flags; + tracer = ERTS_TRACER_PROC(tracee); + trace_flags = ERTS_TRACE_FLAGS(tracee); } if (is_internal_pid(tracer)) { if (!erts_proc_lookup(tracer)) { reset_tracer: - tracee->trace_flags &= ~TRACEE_FLAGS; - trace_flags = tracee->trace_flags; - tracer = tracee->tracer_proc = NIL; + ERTS_TRACE_FLAGS(tracee) &= ~TRACEE_FLAGS; + trace_flags = ERTS_TRACE_FLAGS(tracee); + tracer = ERTS_TRACER_PROC(tracee) = NIL; } } else if (is_internal_port(tracer)) { @@ -1877,7 +1888,7 @@ new_seq_trace_token(Process* p) SEQ_TRACE_TOKEN(p) = TUPLE5(hp, make_small(0), /* Flags */ make_small(0), /* Label */ make_small(0), /* Serial */ - p->id, /* Internal pid */ /* From */ + p->common.id, /* Internal pid */ /* From */ make_small(p->seq_trace_lastcnt)); } } @@ -2247,9 +2258,11 @@ BIF_RETTYPE system_profile_2(BIF_ALIST_2) if (!profiler_p) goto error; } else if (is_internal_port(profiler)) { - if (internal_port_index(profiler) >= erts_max_ports) goto error; - profiler_port = &erts_port[internal_port_index(profiler)]; - if (INVALID_TRACER_PORT(profiler_port, profiler)) goto error; + profiler_port = (erts_port_lookup( + profiler, + ERTS_PORT_SFLGS_INVALID_TRACER_LOOKUP)); + if (!profiler_port) + goto error; } else { goto error; } @@ -2313,8 +2326,7 @@ trace_delivered_1(BIF_ALIST_1) p = NULL; } else if (! (p = erts_pid2proc(BIF_P, ERTS_PROC_LOCK_MAIN, BIF_ARG_1, ERTS_PROC_LOCKS_ALL))) { - if (is_not_internal_pid(BIF_ARG_1) - || internal_pid_index(BIF_ARG_1) >= erts_max_processes) { + if (is_not_internal_pid(BIF_ARG_1)) { BIF_ERROR(BIF_P, BADARG); } } @@ -2333,7 +2345,7 @@ trace_delivered_1(BIF_ALIST_1) msg = TUPLE3(hp, AM_trace_delivered, BIF_ARG_1, msg_ref); #ifdef ERTS_SMP - erts_send_sys_msg_proc(BIF_P->id, BIF_P->id, msg, bp); + erts_send_sys_msg_proc(BIF_P->common.id, BIF_P->common.id, msg, bp); if (p) erts_smp_proc_unlock(p, (BIF_P == p diff --git a/erts/emulator/beam/erl_db.c b/erts/emulator/beam/erl_db.c index 1ba1048afa..48a95cdf32 100644 --- a/erts/emulator/beam/erl_db.c +++ b/erts/emulator/beam/erl_db.c @@ -425,7 +425,8 @@ DbTable* db_get_table_aux(Process *p, if (tb) { db_lock(tb, kind); if (tb->common.id != id - || ((tb->common.status & what) == 0 && p->id != tb->common.owner)) { + || ((tb->common.status & what) == 0 + && p->common.id != tb->common.owner)) { db_unlock(tb, kind); tb = NULL; } @@ -615,7 +616,7 @@ BIF_RETTYPE ets_safe_fixtable_2(BIF_ALIST_2) #ifdef HARDDEBUG erts_fprintf(stderr, "ets:safe_fixtable(%T,%T); Process: %T, initial: %T:%T/%bpu\n", - BIF_ARG_1, BIF_ARG_2, BIF_P->id, + BIF_ARG_1, BIF_ARG_2, BIF_P->common.id, BIF_P->initial[0], BIF_P->initial[1], BIF_P->initial[2]); #endif kind = (BIF_ARG_2 == am_true) ? LCK_READ : LCK_WRITE_REC; @@ -1194,7 +1195,7 @@ BIF_RETTYPE ets_rename_2(BIF_ALIST_2) #ifdef HARDDEBUG erts_fprintf(stderr, "ets:rename(%T,%T); Process: %T, initial: %T:%T/%bpu\n", - BIF_ARG_1, BIF_ARG_2, BIF_P->id, + BIF_ARG_1, BIF_ARG_2, BIF_P->common.id, BIF_P->initial[0], BIF_P->initial[1], BIF_P->initial[2]); #endif @@ -1437,7 +1438,7 @@ BIF_RETTYPE ets_new_2(BIF_ALIST_2) db_init_lock(tb, status & (DB_FINE_LOCKED|DB_FREQ_READ), "db_tab", "db_tab_fix"); tb->common.keypos = keypos; - tb->common.owner = BIF_P->id; + tb->common.owner = BIF_P->common.id; set_heir(BIF_P, tb, heir, heir_data); erts_smp_atomic_init_nob(&tb->common.nitems, 0); @@ -1506,7 +1507,7 @@ BIF_RETTYPE ets_new_2(BIF_ALIST_2) #ifdef HARDDEBUG erts_fprintf(stderr, "ets:new(%T,%T)=%T; Process: %T, initial: %T:%T/%bpu\n", - BIF_ARG_1, BIF_ARG_2, ret, BIF_P->id, + BIF_ARG_1, BIF_ARG_2, ret, BIF_P->common.id, BIF_P->initial[0], BIF_P->initial[1], BIF_P->initial[2]); erts_fprintf(stderr, "ets: new: meta_pid_to_tab common.memory_size = %ld\n", erts_smp_atomic_read_nob(&meta_pid_to_tab->common.memory_size)); @@ -1518,7 +1519,9 @@ BIF_RETTYPE ets_new_2(BIF_ALIST_2) db_meta_lock(meta_pid_to_tab, LCK_WRITE_REC); if (db_put_hash(meta_pid_to_tab, - TUPLE2(meta_tuple, BIF_P->id, make_small(slot)), + TUPLE2(meta_tuple, + BIF_P->common.id, + make_small(slot)), 0) != DB_ERROR_NONE) { erl_exit(1,"Could not update ets metadata."); } @@ -1637,7 +1640,7 @@ BIF_RETTYPE ets_delete_1(BIF_ALIST_1) #ifdef HARDDEBUG erts_fprintf(stderr, "ets:delete(%T); Process: %T, initial: %T:%T/%bpu\n", - BIF_ARG_1, BIF_P->id, + BIF_ARG_1, BIF_P->common.id, BIF_P->initial[0], BIF_P->initial[1], BIF_P->initial[2]); #endif @@ -1654,7 +1657,7 @@ BIF_RETTYPE ets_delete_1(BIF_ALIST_1) tb->common.status &= ~(DB_PROTECTED|DB_PUBLIC|DB_PRIVATE); tb->common.status |= DB_DELETE; - if (tb->common.owner != BIF_P->id) { + if (tb->common.owner != BIF_P->common.id) { DeclareTmpHeap(meta_tuple,3,BIF_P); /* @@ -1669,10 +1672,12 @@ BIF_RETTYPE ets_delete_1(BIF_ALIST_1) make_small(tb->common.slot)); BIF_P->flags |= F_USING_DB; - tb->common.owner = BIF_P->id; + tb->common.owner = BIF_P->common.id; db_put_hash(meta_pid_to_tab, - TUPLE2(meta_tuple,BIF_P->id,make_small(tb->common.slot)), + TUPLE2(meta_tuple, + BIF_P->common.id, + make_small(tb->common.slot)), 0); db_meta_unlock(meta_pid_to_tab, LCK_WRITE_REC); UnUseTmpHeap(3,BIF_P); @@ -1748,7 +1753,7 @@ BIF_RETTYPE ets_give_away_3(BIF_ALIST_3) } if ((tb = db_get_table(BIF_P, BIF_ARG_1, DB_WRITE, LCK_WRITE)) == NULL - || tb->common.owner != BIF_P->id) { + || tb->common.owner != BIF_P->common.id) { goto badarg; } from_pid = tb->common.owner; @@ -1771,7 +1776,10 @@ BIF_RETTYPE ets_give_away_3(BIF_ALIST_3) db_unlock(tb,LCK_WRITE); erts_send_message(BIF_P, to_proc, &to_locks, - TUPLE4(buf, am_ETS_TRANSFER, tb->common.id, from_pid, BIF_ARG_3), + TUPLE4(buf, am_ETS_TRANSFER, + tb->common.id, + from_pid, + BIF_ARG_3), 0); erts_smp_proc_unlock(to_proc, to_locks); UnUseTmpHeap(5,BIF_P); @@ -1833,7 +1841,7 @@ BIF_RETTYPE ets_setopts_2(BIF_ALIST_2) if (tail != NIL || (tb = db_get_table(BIF_P, BIF_ARG_1, DB_WRITE, LCK_WRITE)) == NULL - || tb->common.owner != BIF_P->id) { + || tb->common.owner != BIF_P->common.id) { goto badarg; } @@ -2647,7 +2655,7 @@ BIF_RETTYPE ets_info_1(BIF_ALIST_1) */ /* If/when we implement lockless private tables: - if ((tb->common.status & DB_PRIVATE) && owner != BIF_P->id) { + if ((tb->common.status & DB_PRIVATE) && owner != BIF_P->common.id) { db_unlock(tb, LCK_READ); rp = erts_pid2proc_not_running(BIF_P, ERTS_PROC_LOCK_MAIN, owner, ERTS_PROC_LOCK_MAIN); @@ -3039,9 +3047,9 @@ static int give_away_to_heir(Process* p, DbTable* tb) Eterm to_pid; UWord heir_data; - ASSERT(tb->common.owner == p->id); + ASSERT(tb->common.owner == p->common.id); ASSERT(is_internal_pid(tb->common.heir)); - ASSERT(tb->common.heir != p->id); + ASSERT(tb->common.heir != p->common.id); retry: to_pid = tb->common.heir; to_proc = erts_pid2proc_opt(p, ERTS_PROC_LOCK_MAIN, @@ -3054,7 +3062,7 @@ retry: db_lock(tb,LCK_WRITE); ASSERT(tb != NULL); - if (tb->common.owner != p->id) { + if (tb->common.owner != p->common.id) { if (to_proc != NULL ) { erts_smp_proc_unlock(to_proc, to_locks); } @@ -3065,7 +3073,7 @@ retry: if (to_proc != NULL ) { erts_smp_proc_unlock(to_proc, to_locks); } - if (to_pid == p->id || to_pid == am_none) { + if (to_pid == p->common.id || to_pid == am_none) { return 0; /* no real heir, table still mine */ } goto retry; @@ -3074,7 +3082,8 @@ retry: if (to_proc == NULL) { return 0; /* heir not alive, table still mine */ } - if (to_proc->started_interval != tb->common.heir_started_interval) { + if (to_proc->common.u.alive.started_interval + != tb->common.heir_started_interval) { erts_smp_proc_unlock(to_proc, to_locks); return 0; /* heir dead and pid reused, table still mine */ } @@ -3099,7 +3108,11 @@ retry: heir_data = tpv[1]; } erts_send_message(p, to_proc, &to_locks, - TUPLE4(buf, am_ETS_TRANSFER, tb->common.id, p->id, heir_data), + TUPLE4(buf, + am_ETS_TRANSFER, + tb->common.id, + p->common.id, + heir_data), 0); erts_smp_proc_unlock(to_proc, to_locks); return !0; @@ -3108,7 +3121,7 @@ retry: /* * erts_db_process_exiting() is called when a process terminates. * It returns 0 when completely done, and !0 when it wants to - * yield. c_p->u.exit_data can hold a pointer to a state while + * yield. c_p->u.terminate can hold a pointer to a state while * yielding. */ #define ERTS_DB_INTERNAL_ERROR(LSTR) \ @@ -3118,8 +3131,8 @@ retry: int erts_db_process_exiting(Process *c_p, ErtsProcLocks c_p_locks) { - ErtsDbProcCleanupState *state = (ErtsDbProcCleanupState *) c_p->u.exit_data; - Eterm pid = c_p->id; + ErtsDbProcCleanupState *state = (ErtsDbProcCleanupState *) c_p->u.terminate; + Eterm pid = c_p->common.id; ErtsDbProcCleanupState default_state; int ret; @@ -3300,7 +3313,7 @@ erts_db_process_exiting(Process *c_p, ErtsProcLocks c_p_locks) if (state != &default_state) erts_free(ERTS_ALC_T_DB_PROC_CLEANUP, state); - c_p->u.exit_data = NULL; + c_p->u.terminate = NULL; return 0; default: @@ -3321,13 +3334,13 @@ erts_db_process_exiting(Process *c_p, ErtsProcLocks c_p_locks) break; } - ASSERT(c_p->u.exit_data == (void *) state + ASSERT(c_p->u.terminate == (void *) state || state == &default_state); if (state == &default_state) { - c_p->u.exit_data = erts_alloc(ERTS_ALC_T_DB_PROC_CLEANUP, + c_p->u.terminate = erts_alloc(ERTS_ALC_T_DB_PROC_CLEANUP, sizeof(ErtsDbProcCleanupState)); - sys_memcpy(c_p->u.exit_data, + sys_memcpy(c_p->u.terminate, (void*) state, sizeof(ErtsDbProcCleanupState)); } @@ -3353,7 +3366,7 @@ static void fix_table_locked(Process* p, DbTable* tb) } else { for (; fix != NULL; fix = fix->next) { - if (fix->pid == p->id) { + if (fix->pid == p->common.id) { ++(fix->counter); #ifdef ERTS_SMP erts_smp_mtx_unlock(&tb->common.fixlock); @@ -3365,7 +3378,7 @@ static void fix_table_locked(Process* p, DbTable* tb) fix = (DbFixation *) erts_db_alloc(ERTS_ALC_T_DB_FIXATION, tb, sizeof(DbFixation)); ERTS_ETS_MISC_MEM_ADD(sizeof(DbFixation)); - fix->pid = p->id; + fix->pid = p->common.id; fix->counter = 1; fix->next = tb->common.fixations; tb->common.fixations = fix; @@ -3376,7 +3389,9 @@ static void fix_table_locked(Process* p, DbTable* tb) UseTmpHeap(3,p); db_meta_lock(meta_pid_to_fixed_tab, LCK_WRITE_REC); if (db_put_hash(meta_pid_to_fixed_tab, - TUPLE2(meta_tuple, p->id, make_small(tb->common.slot)), + TUPLE2(meta_tuple, + p->common.id, + make_small(tb->common.slot)), 0) != DB_ERROR_NONE) { UnUseTmpHeap(3,p); erl_exit(1,"Could not insert ets metadata in safe_fixtable."); @@ -3396,7 +3411,7 @@ static void unfix_table_locked(Process* p, DbTable* tb, erts_smp_mtx_lock(&tb->common.fixlock); #endif for (pp = &tb->common.fixations; *pp != NULL; pp = &(*pp)->next) { - if ((*pp)->pid == p->id) { + if ((*pp)->pid == p->common.id) { DbFixation* fix = *pp; erts_refc_dec(&tb->common.ref,0); --(fix->counter); @@ -3410,7 +3425,7 @@ static void unfix_table_locked(Process* p, DbTable* tb, #endif db_meta_lock(meta_pid_to_fixed_tab, LCK_WRITE_REC); db_erase_bag_exact2(meta_pid_to_fixed_tab, - p->id, make_small(tb->common.slot)); + p->common.id, make_small(tb->common.slot)); db_meta_unlock(meta_pid_to_fixed_tab, LCK_WRITE_REC); erts_db_free(ERTS_ALC_T_DB_FIXATION, tb, (void *) fix, sizeof(DbFixation)); @@ -3469,15 +3484,15 @@ static void set_heir(Process* me, DbTable* tb, Eterm heir, UWord heir_data) if (heir == am_none) { return; } - if (heir == me->id) { - erts_ensure_later_proc_interval(me->started_interval); - tb->common.heir_started_interval = me->started_interval; + if (heir == me->common.id) { + erts_ensure_later_proc_interval(me->common.u.alive.started_interval); + tb->common.heir_started_interval = me->common.u.alive.started_interval; } else { Process* heir_proc= erts_proc_lookup(heir); if (heir_proc != NULL) { - erts_ensure_later_proc_interval(heir_proc->started_interval); - tb->common.heir_started_interval = heir_proc->started_interval; + erts_ensure_later_proc_interval(heir_proc->common.u.alive.started_interval); + tb->common.heir_started_interval = heir_proc->common.u.alive.started_interval; } else { tb->common.heir = am_none; } diff --git a/erts/emulator/beam/erl_db_util.c b/erts/emulator/beam/erl_db_util.c index 0c9ca83ce4..bb08762b26 100644 --- a/erts/emulator/beam/erl_db_util.c +++ b/erts/emulator/beam/erl_db_util.c @@ -138,21 +138,23 @@ set_tracee_flags(Process *tracee_p, Eterm tracer, Uint d_flags, Uint e_flags) { Uint flags; if (tracer == NIL) { - flags = tracee_p->trace_flags & ~TRACEE_FLAGS; + flags = ERTS_TRACE_FLAGS(tracee_p) & ~TRACEE_FLAGS; } else { - flags = ((tracee_p->trace_flags & ~d_flags) | e_flags); + flags = ((ERTS_TRACE_FLAGS(tracee_p) & ~d_flags) | e_flags); if (! flags) tracer = NIL; } - ret = tracee_p->tracer_proc != tracer || tracee_p->trace_flags != flags - ? am_true : am_false; - tracee_p->tracer_proc = tracer; - tracee_p->trace_flags = flags; + ret = ((ERTS_TRACER_PROC(tracee_p) != tracer + || ERTS_TRACE_FLAGS(tracee_p) != flags) + ? am_true + : am_false); + ERTS_TRACER_PROC(tracee_p) = tracer; + ERTS_TRACE_FLAGS(tracee_p) = flags; return ret; } /* ** Assuming all locks on tracee_p on entry ** -** Changes tracee_p->trace_flags and tracee_p->tracer_proc +** Changes ERTS_TRACE_FLAGS(tracee_p) and ERTS_TRACER_PROC(tracee_p) ** according to input disable/enable flags and tracer. ** ** Returns am_true|am_false on success, am_true if value changed, @@ -173,17 +175,20 @@ set_match_trace(Process *tracee_p, Eterm fail_term, Eterm tracer, tracer, ERTS_PROC_LOCKS_ALL))) { if (tracee_p != tracer_p) { ret = set_tracee_flags(tracee_p, tracer, d_flags, e_flags); - tracer_p->trace_flags |= tracee_p->trace_flags ? F_TRACER : 0; + ERTS_TRACE_FLAGS(tracer_p) |= (ERTS_TRACE_FLAGS(tracee_p) + ? F_TRACER + : 0); erts_smp_proc_unlock(tracer_p, ERTS_PROC_LOCKS_ALL); } } else if (is_internal_port(tracer)) { Port *tracer_port = - erts_id2port(tracer, tracee_p, ERTS_PROC_LOCKS_ALL); + erts_id2port_sflgs(tracer, + tracee_p, + ERTS_PROC_LOCKS_ALL, + ERTS_PORT_SFLGS_INVALID_TRACER_LOOKUP); if (tracer_port) { - if (! INVALID_TRACER_PORT(tracer_port, tracer)) { - ret = set_tracee_flags(tracee_p, tracer, d_flags, e_flags); - } - erts_smp_port_unlock(tracer_port); + ret = set_tracee_flags(tracee_p, tracer, d_flags, e_flags); + erts_port_release(tracer_port); } } else { ASSERT(is_nil(tracer)); @@ -2174,7 +2179,7 @@ restart: pc += n; break; case matchSelf: - *esp++ = c_p->id; + *esp++ = c_p->common.id; break; case matchWaste: --esp; @@ -2261,7 +2266,7 @@ restart: case matchEnableTrace: if ( (n = erts_trace_flag2bit(esp[-1]))) { BEGIN_ATOMIC_TRACE(c_p); - set_tracee_flags(c_p, c_p->tracer_proc, 0, n); + set_tracee_flags(c_p, ERTS_TRACER_PROC(c_p), 0, n); esp[-1] = am_true; } else { esp[-1] = FAIL_TERM; @@ -2274,7 +2279,7 @@ restart: BEGIN_ATOMIC_TRACE(c_p); if ( (tmpp = get_proc(c_p, 0, esp[0], 0))) { /* Always take over the tracer of the current process */ - set_tracee_flags(tmpp, c_p->tracer_proc, 0, n); + set_tracee_flags(tmpp, ERTS_TRACER_PROC(c_p), 0, n); esp[-1] = am_true; } } @@ -2282,7 +2287,7 @@ restart: case matchDisableTrace: if ( (n = erts_trace_flag2bit(esp[-1]))) { BEGIN_ATOMIC_TRACE(c_p); - set_tracee_flags(c_p, c_p->tracer_proc, n, 0); + set_tracee_flags(c_p, ERTS_TRACER_PROC(c_p), n, 0); esp[-1] = am_true; } else { esp[-1] = FAIL_TERM; @@ -2295,7 +2300,7 @@ restart: BEGIN_ATOMIC_TRACE(c_p); if ( (tmpp = get_proc(c_p, 0, esp[0], 0))) { /* Always take over the tracer of the current process */ - set_tracee_flags(tmpp, c_p->tracer_proc, n, 0); + set_tracee_flags(tmpp, ERTS_TRACER_PROC(c_p), n, 0); esp[-1] = am_true; } } @@ -2316,12 +2321,12 @@ restart: --esp; if (*esp == am_true) { erts_smp_proc_lock(c_p, ERTS_PROC_LOCKS_ALL_MINOR); - c_p->trace_flags |= F_TRACE_SILENT; + ERTS_TRACE_FLAGS(c_p) |= F_TRACE_SILENT; erts_smp_proc_unlock(c_p, ERTS_PROC_LOCKS_ALL_MINOR); } else if (*esp == am_false) { erts_smp_proc_lock(c_p, ERTS_PROC_LOCKS_ALL_MINOR); - c_p->trace_flags &= ~F_TRACE_SILENT; + ERTS_TRACE_FLAGS(c_p) &= ~F_TRACE_SILENT; erts_smp_proc_unlock(c_p, ERTS_PROC_LOCKS_ALL_MINOR); } break; @@ -2329,11 +2334,11 @@ restart: { /* disable enable */ Uint d_flags = 0, e_flags = 0; /* process trace flags */ - Eterm tracer = c_p->tracer_proc; + Eterm tracer = ERTS_TRACER_PROC(c_p); /* XXX Atomicity note: Not fully atomic. Default tracer * is sampled from current process but applied to * tracee and tracer later after releasing main - * locks on current process, so c_p->tracer_proc + * locks on current process, so ERTS_TRACER_PROC(c_p) * may actually have changed when tracee and tracer * gets updated. I do not think nobody will notice. * It is just the default value that is not fully atomic. @@ -2358,7 +2363,7 @@ restart: { /* disable enable */ Uint d_flags = 0, e_flags = 0; /* process trace flags */ - Eterm tracer = c_p->tracer_proc; + Eterm tracer = ERTS_TRACER_PROC(c_p); /* XXX Atomicity note. Not fully atomic. See above. * Above it could possibly be solved, but not here. */ @@ -5004,7 +5009,7 @@ static Eterm match_spec_test(Process *p, Eterm against, Eterm spec, int trace) static Eterm seq_trace_fake(Process *p, Eterm arg1) { Eterm result = erl_seq_trace_info(p, arg1); - if (is_tuple(result) && *tuple_val(result) == 2) { + if (!is_non_value(result) && is_tuple(result) && *tuple_val(result) == 2) { return (tuple_val(result))[2]; } return result; diff --git a/erts/emulator/beam/erl_db_util.h b/erts/emulator/beam/erl_db_util.h index dcecc4251a..d8f6e40d2e 100644 --- a/erts/emulator/beam/erl_db_util.h +++ b/erts/emulator/beam/erl_db_util.h @@ -320,10 +320,10 @@ ERTS_GLB_INLINE int db_eq(DbTableCommon* tb, Eterm a, DbTerm* b) #define DB_INFO (DB_PROTECTED|DB_PUBLIC|DB_PRIVATE) #define ONLY_WRITER(P,T) (((T)->common.status & (DB_PRIVATE|DB_PROTECTED)) \ - && (T)->common.owner == (P)->id) + && (T)->common.owner == (P)->common.id) #define ONLY_READER(P,T) (((T)->common.status & DB_PRIVATE) && \ -(T)->common.owner == (P)->id) +(T)->common.owner == (P)->common.id) /* Function prototypes */ BIF_RETTYPE db_get_trace_control_word(Process* p); diff --git a/erts/emulator/beam/erl_debug.c b/erts/emulator/beam/erl_debug.c index 22e873afc6..b90d00f236 100644 --- a/erts/emulator/beam/erl_debug.c +++ b/erts/emulator/beam/erl_debug.c @@ -252,16 +252,16 @@ void erts_check_stack(Process *p) if (p->stop > stack_start) erl_exit(1, "<%lu.%lu.%lu>: Stack underflow\n", - internal_pid_channel_no(p->id), - internal_pid_number(p->id), - internal_pid_serial(p->id)); + internal_pid_channel_no(p->common.id), + internal_pid_number(p->common.id), + internal_pid_serial(p->common.id)); if (p->stop < stack_end) erl_exit(1, "<%lu.%lu.%lu>: Stack overflow\n", - internal_pid_channel_no(p->id), - internal_pid_number(p->id), - internal_pid_serial(p->id)); + internal_pid_channel_no(p->common.id), + internal_pid_number(p->common.id), + internal_pid_serial(p->common.id)); for (elemp = p->stop; elemp < stack_start; elemp++) { int in_mbuf = 0; @@ -284,9 +284,9 @@ void erts_check_stack(Process *p) erl_exit(1, "<%lu.%lu.%lu>: Wild stack pointer\n", - internal_pid_channel_no(p->id), - internal_pid_number(p->id), - internal_pid_serial(p->id)); + internal_pid_channel_no(p->common.id), + internal_pid_number(p->common.id), + internal_pid_serial(p->common.id)); } } @@ -387,16 +387,16 @@ void verify_process(Process *p) #define VERIFY_AREA(name,ptr,sz) { \ int n = (sz); \ while (n--) if(!verify_eterm(p,*(ptr+n))) \ - erl_exit(1,"Wild pointer found in " name " of %T!\n",p->id); } + erl_exit(1,"Wild pointer found in " name " of %T!\n",p->common.id); } #define VERIFY_ETERM(name,eterm) { \ if(!verify_eterm(p,eterm)) \ - erl_exit(1,"Wild pointer found in " name " of %T!\n",p->id); } + erl_exit(1,"Wild pointer found in " name " of %T!\n",p->common.id); } ErlMessage* mp = p->msg.first; - VERBOSE(DEBUG_MEMORY,("Verify process: %T...\n",p->id)); + VERBOSE(DEBUG_MEMORY,("Verify process: %T...\n",p->common.id)); while (mp != NULL) { VERIFY_ETERM("message term",ERL_MESSAGE_TERM(mp)); @@ -516,7 +516,7 @@ static void print_process_memory(Process *p) ErlHeapFragment* bp = MBUF(p); erts_printf("==============================\n"); - erts_printf("|| Memory info for %T ||\n",p->id); + erts_printf("|| Memory info for %T ||\n",p->common.id); erts_printf("==============================\n"); erts_printf("-- %-*s ---%s-%s-%s-%s--\n", @@ -601,7 +601,7 @@ void print_memory_info(Process *p) { if (p != NULL) { erts_printf("======================================\n"); - erts_printf("|| Memory info for %-12T ||\n",p->id); + erts_printf("|| Memory info for %-12T ||\n",p->common.id); erts_printf("======================================\n"); erts_printf("+- local heap ----%s-%s-%s-%s-+\n", dashes,dashes,dashes,dashes); diff --git a/erts/emulator/beam/erl_driver.h b/erts/emulator/beam/erl_driver.h index 771ee46d2b..046b46513f 100644 --- a/erts/emulator/beam/erl_driver.h +++ b/erts/emulator/beam/erl_driver.h @@ -133,7 +133,7 @@ typedef struct { #define ERL_DRV_EXTENDED_MARKER (0xfeeeeeed) #define ERL_DRV_EXTENDED_MAJOR_VERSION 2 -#define ERL_DRV_EXTENDED_MINOR_VERSION 0 +#define ERL_DRV_EXTENDED_MINOR_VERSION 1 /* * The emulator will refuse to load a driver with different major @@ -154,6 +154,7 @@ typedef struct { #define ERL_DRV_FLAG_USE_PORT_LOCKING (1 << 0) #define ERL_DRV_FLAG_SOFT_BUSY (1 << 1) +#define ERL_DRV_FLAG_NO_BUSY_MSGQ (1 << 2) /* * Integer types @@ -207,8 +208,8 @@ typedef struct erl_drv_binary { typedef struct _erl_drv_data* ErlDrvData; /* Data to be used by the driver itself. */ #ifndef ERL_SYS_DRV typedef struct _erl_drv_event* ErlDrvEvent; /* An event to be selected on. */ -typedef struct _erl_drv_port* ErlDrvPort; /* A port descriptor. */ #endif +typedef struct _erl_drv_port* ErlDrvPort; /* A port descriptor. */ typedef struct _erl_drv_port* ErlDrvThreadData; /* Thread data. */ #if !defined(__WIN32__) && !defined(_WIN32) && !defined(_WIN32_) && !defined(USE_SELECT) @@ -377,9 +378,18 @@ typedef struct erl_drv_entry { ErlDrvEntry* driver_init(void) #endif +#define ERL_DRV_BUSY_MSGQ_DISABLED (~((ErlDrvSizeT) 0)) +#define ERL_DRV_BUSY_MSGQ_READ_ONLY ((ErlDrvSizeT) 0) +#define ERL_DRV_BUSY_MSGQ_LIM_MAX (ERL_DRV_BUSY_MSGQ_DISABLED - 1) +#define ERL_DRV_BUSY_MSGQ_LIM_MIN ((ErlDrvSizeT) 1) + /* * These are the functions available for driver writers. */ +EXTERN void erl_drv_busy_msgq_limits(ErlDrvPort port, + ErlDrvSizeT *low, + ErlDrvSizeT *high); + EXTERN int driver_select(ErlDrvPort port, ErlDrvEvent event, int mode, int on); EXTERN int driver_event(ErlDrvPort port, ErlDrvEvent event, ErlDrvEventData event_data); @@ -594,11 +604,33 @@ EXTERN ErlDrvPort driver_create_port(ErlDrvPort creator_port, ErlDrvData drv_data); +/* + * driver_output_term() is deprecated, and scheduled for removal in + * OTP-R17. Use erl_drv_output_term() instead. For more information + * see the erl_driver(3) documentation. + */ +EXTERN int driver_output_term(ErlDrvPort ix, + ErlDrvTermData* data, + int len) ERL_DRV_DEPRECATED_FUNC; +/* + * driver_send_term() is deprecated, and scheduled for removal in + * OTP-R17. Use erl_drv_send_term() instead. For more information + * see the erl_driver(3) documentation. + */ +EXTERN int driver_send_term(ErlDrvPort ix, + ErlDrvTermData to, + ErlDrvTermData* data, + int len) ERL_DRV_DEPRECATED_FUNC; + /* output term data to the port owner */ -EXTERN int driver_output_term(ErlDrvPort ix, ErlDrvTermData* data, int len); +EXTERN int erl_drv_output_term(ErlDrvTermData port, + ErlDrvTermData* data, + int len); /* output term data to a specific process */ -EXTERN int driver_send_term(ErlDrvPort ix, ErlDrvTermData to, - ErlDrvTermData* data, int len); +EXTERN int erl_drv_send_term(ErlDrvTermData port, + ErlDrvTermData to, + ErlDrvTermData* data, + int len); /* Async IO functions */ EXTERN long driver_async(ErlDrvPort ix, diff --git a/erts/emulator/beam/erl_gc.c b/erts/emulator/beam/erl_gc.c index 6075a527c3..a33085315a 100644 --- a/erts/emulator/beam/erl_gc.c +++ b/erts/emulator/beam/erl_gc.c @@ -59,7 +59,7 @@ static Uint reclaimed; /* no of words reclaimed in GCs */ erts_fprintf(stderr, "htop=%p\n", (p)->htop); \ erts_fprintf(stderr, "heap=%p\n", (p)->heap); \ erl_exit(ERTS_ABORT_EXIT, "%s, line %d: %T: Overrun stack and heap\n", \ - __FILE__,__LINE__,(P)->id); \ + __FILE__,__LINE__,(P)->common.id); \ } #ifdef DEBUG @@ -129,7 +129,7 @@ static void disallow_heap_frag_ref(Process* p, Eterm* n_htop, Eterm* objv, int n #if defined(ARCH_64) && !HALFWORD_HEAP # define MAX_HEAP_SIZES 154 #else -# define MAX_HEAP_SIZES 55 +# define MAX_HEAP_SIZES 59 #endif static Sint heap_sizes[MAX_HEAP_SIZES]; /* Suitable heap sizes. */ @@ -144,6 +144,7 @@ void erts_init_gc(void) { int i = 0; + Sint max_heap_size = 0; ASSERT(offsetof(ProcBin,thing_word) == offsetof(struct erl_off_heap_header,thing_word)); ASSERT(offsetof(ProcBin,thing_word) == offsetof(ErlFunThing,thing_word)); @@ -168,16 +169,30 @@ erts_init_gc(void) * we really don't want that growth when the heaps are that big. */ - heap_sizes[0] = 34; - heap_sizes[1] = 55; - for (i = 2; i < 23; i++) { - heap_sizes[i] = heap_sizes[i-1] + heap_sizes[i-2]; + /* Growth stage 1 - Fibonacci + 1*/ + /* 12,38 will hit size 233, the old default */ + + heap_sizes[0] = 12; + heap_sizes[1] = 38; + + for(i = 2; i < 23; i++) { + /* one extra word for block header */ + heap_sizes[i] = heap_sizes[i-1] + heap_sizes[i-2] + 1; } + + /* for 32 bit we want max_heap_size to be MAX(32bit) / 4 [words] (and halfword) + * for 64 bit we want max_heap_size to be MAX(52bit) / 8 [words] + */ + + max_heap_size = sizeof(Eterm) < 8 ? (Sint)((~(Uint)0)/(sizeof(Eterm))) : + (Sint)(((Uint64)1 << 53)/sizeof(Eterm)); + + /* Growth stage 2 - 20% growth */ /* At 1.3 mega words heap, we start to slow down. */ for (i = 23; i < ALENGTH(heap_sizes); i++) { - heap_sizes[i] = 5*(heap_sizes[i-1]/4); - if (heap_sizes[i] < 0) { + heap_sizes[i] = heap_sizes[i-1] + heap_sizes[i-1]/5; + if ((heap_sizes[i] < 0) || heap_sizes[i] > max_heap_size) { /* Size turned negative. Discard this last size. */ i--; break; @@ -336,6 +351,19 @@ erts_gc_after_bif_call(Process* p, Eterm result, Eterm* regs, Uint arity) return result; } +static ERTS_INLINE void reset_active_writer(Process *p) +{ + struct erl_off_heap_header* ptr; + ptr = MSO(p).first; + while (ptr) { + if (ptr->thing_word == HEADER_PROC_BIN) { + ProcBin *pbp = (ProcBin*) ptr; + pbp->flags &= ~PB_ACTIVE_WRITER; + } + ptr = ptr->next; + } +} + /* * Garbage collect a process. * @@ -391,6 +419,7 @@ erts_garbage_collect(Process* p, int need, Eterm* objv, int nobj) DTRACE2(gc_minor_end, pidbuf, reclaimed_now); } } + reset_active_writer(p); /* * Finish. @@ -854,14 +883,12 @@ minor_collection(Process* p, int need, Eterm* objv, int nobj, Uint *recl) } } - if (wanted < MIN_HEAP_SIZE(p)) { - wanted = MIN_HEAP_SIZE(p); - } else { - wanted = next_heap_size(p, wanted, 0); - } + wanted = wanted < MIN_HEAP_SIZE(p) ? MIN_HEAP_SIZE(p) + : next_heap_size(p, wanted, 0); if (wanted < HEAP_SIZE(p)) { shrink_new_heap(p, wanted, objv, nobj); } + ASSERT(HEAP_SIZE(p) == next_heap_size(p, HEAP_SIZE(p), 0)); return 1; /* We are done. */ } @@ -1420,11 +1447,10 @@ adjust_after_fullsweep(Process *p, Uint size_before, int need, Eterm *objv, int I think this is better as fullsweep is used mainly on small memory systems, but I could be wrong... */ wanted = 2 * need_after; - if (wanted < p->min_heap_size) { - sz = p->min_heap_size; - } else { - sz = next_heap_size(p, wanted, 0); - } + + sz = wanted < p->min_heap_size ? p->min_heap_size + : next_heap_size(p, wanted, 0); + if (sz < HEAP_SIZE(p)) { shrink_new_heap(p, sz, objv, nobj); } @@ -1932,9 +1958,9 @@ setup_rootset(Process *p, Eterm *objv, int nobj, Rootset *rootset) n++; } #endif - ASSERT(is_nil(p->tracer_proc) || - is_internal_pid(p->tracer_proc) || - is_internal_port(p->tracer_proc)); + ASSERT(is_nil(ERTS_TRACER_PROC(p)) || + is_internal_pid(ERTS_TRACER_PROC(p)) || + is_internal_port(ERTS_TRACER_PROC(p))); ASSERT(is_pid(follow_moved(p->group_leader))); if (is_not_immed(p->group_leader)) { @@ -2166,7 +2192,6 @@ link_live_proc_bin(struct shrink_cand_data *shrink, if (pbp->flags & PB_ACTIVE_WRITER) { - pbp->flags &= ~PB_ACTIVE_WRITER; shrink->no_of_active++; } else { /* inactive */ diff --git a/erts/emulator/beam/erl_goodfit_alloc.c b/erts/emulator/beam/erl_goodfit_alloc.c index e7d4ac2b67..f98d377fd6 100644 --- a/erts/emulator/beam/erl_goodfit_alloc.c +++ b/erts/emulator/beam/erl_goodfit_alloc.c @@ -205,7 +205,6 @@ erts_gfalc_start(GFAllctr_t *gfallctr, init->sbmbct = 0; /* Small mbc not yet supported by goodfit */ - allctr->mbc_header_size = sizeof(Carrier_t); allctr->min_mbc_size = MIN_MBC_SZ; allctr->min_mbc_first_free_size = MIN_MBC_FIRST_FREE_SZ; allctr->min_block_size = sizeof(GFFreeBlock_t); @@ -363,7 +362,7 @@ search_bucket(Allctr_t *allctr, int ix, Uint size) blk && i < max_blk_search; blk = blk->next, i++) { - blk_sz = BLK_SZ(blk); + blk_sz = MBC_FBLK_SZ(&blk->block_head); blk_on_lambc = (((char *) blk) < gfallctr->last_aux_mbc_end && gfallctr->last_aux_mbc_start <= ((char *) blk)); @@ -402,7 +401,7 @@ get_free_block(Allctr_t *allctr, Uint size, if (min_bi == unsafe_bi) { blk = search_bucket(allctr, min_bi, size); if (blk) { - if (cand_blk && cand_size <= BLK_SZ(blk)) + if (cand_blk && cand_size <= MBC_FBLK_SZ(blk)) return NULL; /* cand_blk was better */ unlink_free_block(allctr, blk, flags); return blk; @@ -422,7 +421,7 @@ get_free_block(Allctr_t *allctr, Uint size, /* We are guaranteed to find a block that fits in this bucket */ blk = search_bucket(allctr, min_bi, size); ASSERT(blk); - if (cand_blk && cand_size <= BLK_SZ(blk)) + if (cand_blk && cand_size <= MBC_FBLK_SZ(blk)) return NULL; /* cand_blk was better */ unlink_free_block(allctr, blk, flags); return blk; @@ -435,7 +434,7 @@ link_free_block(Allctr_t *allctr, Block_t *block, Uint32 flags) { GFAllctr_t *gfallctr = (GFAllctr_t *) allctr; GFFreeBlock_t *blk = (GFFreeBlock_t *) block; - Uint sz = BLK_SZ(blk); + Uint sz = MBC_FBLK_SZ(&blk->block_head); int i = BKT_IX(gfallctr, sz); ASSERT(sz >= MIN_BLK_SZ); @@ -456,7 +455,7 @@ unlink_free_block(Allctr_t *allctr, Block_t *block, Uint32 flags) { GFAllctr_t *gfallctr = (GFAllctr_t *) allctr; GFFreeBlock_t *blk = (GFFreeBlock_t *) block; - Uint sz = BLK_SZ(blk); + Uint sz = MBC_FBLK_SZ(&blk->block_head); int i = BKT_IX(gfallctr, sz); if (!blk->prev) { @@ -618,7 +617,7 @@ check_block(Allctr_t *allctr, Block_t * blk, int free_block) GFFreeBlock_t *fblk; if(free_block) { - Uint blk_sz = BLK_SZ(blk); + Uint blk_sz = is_sbc_blk(blk) ? SBC_BLK_SZ(blk) : MBC_BLK_SZ(blk); bi = BKT_IX(gfallctr, blk_sz); ASSERT(gfallctr->bucket_mask.main & (((UWord) 1) << IX2SMIX(bi))); diff --git a/erts/emulator/beam/erl_init.c b/erts/emulator/beam/erl_init.c index 4b90e5394a..8cdf954dd2 100644 --- a/erts/emulator/beam/erl_init.c +++ b/erts/emulator/beam/erl_init.c @@ -44,6 +44,7 @@ #include "erl_thr_progress.h" #include "erl_thr_queue.h" #include "erl_async.h" +#include "erl_ptab.h" #ifdef HIPE #include "hipe_mode_switch.h" /* for hipe_mode_switch_init() */ @@ -109,6 +110,11 @@ const int etp_lock_check = 1; #else const int etp_lock_check = 0; #endif +#ifdef WORDS_BIGENDIAN +const int etp_big_endian = 1; +#else +const int etp_big_endian = 0; +#endif /* * Note about VxWorks: All variables must be initialized by executable code, * not by an initializer. Otherwise a new instance of the emulator will @@ -121,9 +127,10 @@ extern void ConNormalExit(void); extern void ConWaitForExit(void); #endif -static void erl_init(int ncpu); - -#define ERTS_MIN_COMPAT_REL 7 +static void erl_init(int ncpu, + int proc_tab_sz, + int port_tab_sz, + int port_tab_sz_ignore_files); static erts_atomic_t exiting; @@ -206,8 +213,6 @@ ErtsModifiedTimings erts_modified_timings[] = { Export *erts_delay_trap = NULL; -int erts_use_r9_pids_ports; - int ignore_break; int replace_intr; @@ -271,12 +276,18 @@ void erts_short_init(void) { int ncpu = early_init(NULL, NULL); - erl_init(ncpu); + erl_init(ncpu, + ERTS_DEFAULT_MAX_PROCESSES, + ERTS_DEFAULT_MAX_PORTS, + 0); erts_initialized = 1; } static void -erl_init(int ncpu) +erl_init(int ncpu, + int proc_tab_sz, + int port_tab_sz, + int port_tab_sz_ignore_files) { init_benchmarking(); @@ -284,7 +295,7 @@ erl_init(int ncpu) erts_init_gc(); erts_init_time(); erts_init_sys_common_misc(); - erts_init_process(ncpu); + erts_init_process(ncpu, proc_tab_sz); erts_init_scheduling(no_schedulers, no_schedulers_online); erts_init_cpu_topology(); /* Must be after init_scheduling */ @@ -306,6 +317,7 @@ erl_init(int ncpu) erts_bif_info_init(); erts_ddll_init(); init_emulator(); + erts_ptab_init(); /* Must be after init_emulator() */ erts_bp_init(); init_db(); /* Must be after init_emulator */ erts_bif_timer_init(); @@ -313,7 +325,7 @@ erl_init(int ncpu) init_dist(); erl_drv_thr_init(); erts_init_async(); - init_io(); + erts_init_io(port_tab_sz, port_tab_sz_ignore_files); init_load(); erts_init_bif(); erts_init_bif_chksum(); @@ -457,6 +469,7 @@ load_preloaded(void) /* be helpful (or maybe downright rude:-) */ void erts_usage(void) { + int this_rel = this_rel_num(); erts_fprintf(stderr, "Usage: %s [flags] [ -- [init_args] ]\n", progname(program)); erts_fprintf(stderr, "The flags are:\n\n"); @@ -490,16 +503,20 @@ void erts_usage(void) /* erts_fprintf(stderr, "-i module set the boot module (default init)\n"); */ erts_fprintf(stderr, "-K boolean enable or disable kernel poll\n"); - + erts_fprintf(stderr, "-n[s|a|d] Control behavior of signals to ports\n"); + erts_fprintf(stderr, " Note that this flag is deprecated!\n"); erts_fprintf(stderr, "-M<X> <Y> memory allocator switches,\n"); erts_fprintf(stderr, " see the erts_alloc(3) documentation for more info.\n"); erts_fprintf(stderr, "-P number set maximum number of processes on this node,\n"); erts_fprintf(stderr, " valid range is [%d-%d]\n", - ERTS_MIN_PROCESSES, ERTS_MAX_PROCESSES); + ERTS_MIN_PROCESSES, ERTS_MAX_PROCESSES); + erts_fprintf(stderr, "-Q number set maximum number of ports on this node,\n"); + erts_fprintf(stderr, " valid range is [%d-%d]\n", + ERTS_MIN_PORTS, ERTS_MAX_PORTS); erts_fprintf(stderr, "-R number set compatibility release number,\n"); erts_fprintf(stderr, " valid range [%d-%d]\n", - ERTS_MIN_COMPAT_REL, this_rel_num()); + this_rel-2, this_rel); erts_fprintf(stderr, "-r force ets memory block to be moved on realloc\n"); erts_fprintf(stderr, "-rg amount set reader groups limit\n"); @@ -519,6 +536,7 @@ void erts_usage(void) erts_fprintf(stderr, " valid range is [%d-%d]\n", ERTS_SCHED_THREAD_MIN_STACK_SIZE, ERTS_SCHED_THREAD_MAX_STACK_SIZE); + erts_fprintf(stderr, "-spp Bool set port parallelism scheduling hint\n"); erts_fprintf(stderr, "-S n1:n2 set number of schedulers (n1), and number of\n"); erts_fprintf(stderr, " schedulers online (n2), valid range for both\n"); erts_fprintf(stderr, " numbers are [1-%d]\n", @@ -612,7 +630,6 @@ early_init(int *argc, char **argv) /* erts_printf_eterm_func = erts_printf_term; erts_disable_tolerant_timeofday = 0; display_items = 200; - erts_proc.max = ERTS_DEFAULT_MAX_PROCESSES; erts_backtrace_depth = DEFAULT_BACKTRACE_SIZE; erts_async_max_threads = 0; erts_async_thread_suggested_stack_size = ERTS_ASYNC_THREAD_MIN_STACK_SIZE; @@ -641,8 +658,6 @@ early_init(int *argc, char **argv) /* erts_compat_rel = this_rel_num(); - erts_use_r9_pids_ports = 0; - erts_sys_pre_init(); erts_atomic_init_nob(&exiting, 0); #ifdef ERTS_SMP @@ -897,11 +912,13 @@ erl_start(int argc, char **argv) { int i = 1; char* arg=NULL; - char* Parg = NULL; int have_break_handler = 1; char envbuf[21]; /* enough for any 64-bit integer */ size_t envbufsz; int ncpu = early_init(&argc, argv); + int proc_tab_sz = ERTS_DEFAULT_MAX_PROCESSES; + int port_tab_sz = ERTS_DEFAULT_MAX_PORTS; + int port_tab_sz_ignore_files = 0; envbufsz = sizeof(envbuf); if (erts_sys_getenv_raw(ERL_MAX_ETS_TABLES_ENV, envbuf, &envbufsz) == 0) @@ -972,6 +989,7 @@ erl_start(int argc, char **argv) break; case 'a': erts_set_user_requested_filename_encoding(ERL_FILENAME_UNKNOWN); + break; default: erts_fprintf(stderr, "bad filename encoding %s, can be (l,u or a)\n", arg); erts_usage(); @@ -1152,12 +1170,53 @@ erl_start(int argc, char **argv) arg); break; - case 'P': - /* set maximum number of processes */ - Parg = get_arg(argv[i]+2, argv[i+1], &i); - erts_proc.max = atoi(Parg); - /* Check of result is delayed until later. This is because +R - may be given after +P. */ + case 'n': + arg = get_arg(argv[i]+2, argv[i+1], &i); + switch (arg[0]) { + case 's': /* synchronous */ + erts_port_synchronous_ops = 1; + erts_port_schedule_all_ops = 0; + break; + case 'a': /* asynchronous */ + erts_port_synchronous_ops = 0; + erts_port_schedule_all_ops = 1; + break; + case 'd': /* Default - schedule on conflict (asynchronous) */ + erts_port_synchronous_ops = 0; + erts_port_schedule_all_ops = 0; + break; + default: + bad_n_option: + erts_fprintf(stderr, "bad -n option %s\n", arg); + erts_usage(); + } + if (arg[1] != '\0') + goto bad_n_option; + break; + + case 'P': /* set maximum number of processes */ + arg = get_arg(argv[i]+2, argv[i+1], &i); + errno = 0; + proc_tab_sz = strtol(arg, NULL, 10); + if (errno != 0 + || proc_tab_sz < ERTS_MIN_PROCESSES + || ERTS_MAX_PROCESSES < proc_tab_sz) { + erts_fprintf(stderr, "bad number of processes %s\n", arg); + erts_usage(); + } + break; + + case 'Q': /* set maximum number of ports */ + arg = get_arg(argv[i]+2, argv[i+1], &i); + errno = 0; + port_tab_sz = strtol(arg, NULL, 10); + if (errno != 0 + || port_tab_sz < ERTS_MIN_PROCESSES + || ERTS_MAX_PROCESSES < port_tab_sz) { + erts_fprintf(stderr, "bad number of ports %s\n", arg); + erts_usage(); + } + port_tab_sz_ignore_files = 1; break; case 'S' : /* Was handled in early_init() just read past it */ @@ -1259,6 +1318,19 @@ erl_start(int argc, char **argv) erts_usage(); } } + else if (has_prefix("pp", sub_param)) { + arg = get_arg(sub_param+2, argv[i+1], &i); + if (sys_strcmp(arg, "true") == 0) + erts_port_parallelism = 1; + else if (sys_strcmp(arg, "false") == 0) + erts_port_parallelism = 0; + else { + erts_fprintf(stderr, + "bad port parallelism scheduling hint %s\n", + arg); + erts_usage(); + } + } else if (sys_strcmp("nsp", sub_param) == 0) erts_use_sender_punish = 0; else if (sys_strcmp("wt", sub_param) == 0) { @@ -1340,22 +1412,19 @@ erl_start(int argc, char **argv) case 'R': { /* set compatibility release */ + int this_rel; arg = get_arg(argv[i]+2, argv[i+1], &i); erts_compat_rel = atoi(arg); - if (erts_compat_rel < ERTS_MIN_COMPAT_REL - || erts_compat_rel > this_rel_num()) { + this_rel = this_rel_num(); + if (erts_compat_rel < this_rel - 2 || this_rel < erts_compat_rel) { erts_fprintf(stderr, "bad compatibility release number %s\n", arg); erts_usage(); } - ASSERT(ERTS_MIN_COMPAT_REL >= 7); switch (erts_compat_rel) { - case 7: - case 8: - case 9: - erts_use_r9_pids_ports = 1; + /* Currently no compat features... */ default: break; } @@ -1397,8 +1466,6 @@ erl_start(int argc, char **argv) } break; } - case 'n': /* XXX obsolete */ - break; case 'c': if (argv[i][2] == 0) { /* -c: documented option */ erts_disable_tolerant_timeofday = 1; @@ -1453,15 +1520,6 @@ erl_start(int argc, char **argv) i++; } - /* Delayed check of +P flag */ - if (erts_proc.max < ERTS_MIN_PROCESSES - || erts_proc.max > ERTS_MAX_PROCESSES - || (erts_use_r9_pids_ports - && erts_proc.max > ERTS_MAX_R9_PROCESSES)) { - erts_fprintf(stderr, "bad number of processes %s\n", Parg); - erts_usage(); - } - /* Restart will not reinstall the break handler */ #ifdef __WIN32__ if (ignore_break) @@ -1482,7 +1540,10 @@ erl_start(int argc, char **argv) boot_argc = argc - i; /* Number of arguments to init */ boot_argv = &argv[i]; - erl_init(ncpu); + erl_init(ncpu, + proc_tab_sz, + port_tab_sz, + port_tab_sz_ignore_files); load_preloaded(); erts_end_staging_code_ix(); diff --git a/erts/emulator/beam/erl_lock_check.c b/erts/emulator/beam/erl_lock_check.c index 314d2f6a9c..69bb4be717 100644 --- a/erts/emulator/beam/erl_lock_check.c +++ b/erts/emulator/beam/erl_lock_check.c @@ -95,7 +95,6 @@ static erts_lc_lock_order_t erts_lock_order[] = { { "dist_entry_links", "address" }, { "code_write_permission", NULL }, { "proc_status", "pid" }, - { "proc_tab", NULL }, { "ports_snapshot", NULL }, { "meta_name_tab", "address" }, { "meta_main_tab_slot", "address" }, @@ -115,9 +114,6 @@ static erts_lc_lock_order_t erts_lock_order[] = { #if defined(ENABLE_CHILD_WAITER_THREAD) || defined(ERTS_SMP) { "child_status", NULL }, #endif -#ifdef __WIN32__ - { "sys_driver_data_lock", NULL }, -#endif { "drv_ev_state_grow", NULL, }, { "drv_ev_state", "address" }, { "safe_hash", "address" }, @@ -127,6 +123,7 @@ static erts_lc_lock_order_t erts_lock_order[] = { { "schdlr_sspnd", NULL }, { "migration_info_update", NULL }, { "run_queue", "address" }, + { "process_table", NULL }, { "cpu_info", NULL }, { "pollset", "address" }, #ifdef __WIN32__ @@ -157,12 +154,10 @@ static erts_lc_lock_order_t erts_lock_order[] = { { "pmmap", NULL }, #endif #ifdef ERTS_SMP + { "port_sched_lock", "port_id" }, { "port_task_pre_alloc_lock", "address" }, - { "port_taskq_pre_alloc_lock", "address" }, { "proclist_pre_alloc_lock", "address" }, - { "port_tasks_lock", NULL }, - { "get_free_port", NULL }, - { "port_state", "address" }, + { "port_table", NULL }, { "xports_list_pre_alloc_lock", "address" }, { "inet_buffer_stack_lock", NULL }, { "gc_info", NULL }, @@ -247,6 +242,7 @@ typedef struct { typedef struct erts_lc_locked_locks_t_ erts_lc_locked_locks_t; struct erts_lc_locked_locks_t_ { char *thread_name; + int emu_thread; erts_tid_t tid; erts_lc_locked_locks_t *next; erts_lc_locked_locks_t *prev; @@ -364,6 +360,7 @@ create_locked_locks(char *thread_name) if (!l_lcks->thread_name) lc_abort(); + l_lcks->emu_thread = 0; l_lcks->tid = erts_thr_self(); l_lcks->required.first = NULL; l_lcks->required.last = NULL; @@ -671,7 +668,7 @@ erts_lc_set_thread_name(char *thread_name) { erts_lc_locked_locks_t *l_lcks = get_my_locked_locks(); if (!l_lcks) - (void) create_locked_locks(thread_name); + l_lcks = create_locked_locks(thread_name); else { ASSERT(l_lcks->thread_name); free((void *) l_lcks->thread_name); @@ -679,6 +676,14 @@ erts_lc_set_thread_name(char *thread_name) if (!l_lcks->thread_name) lc_abort(); } + l_lcks->emu_thread = 1; +} + +int +erts_lc_is_emu_thr(void) +{ + erts_lc_locked_locks_t *l_lcks = get_my_locked_locks(); + return l_lcks->emu_thread; } int diff --git a/erts/emulator/beam/erl_lock_check.h b/erts/emulator/beam/erl_lock_check.h index b67f36fa06..068340abe7 100644 --- a/erts/emulator/beam/erl_lock_check.h +++ b/erts/emulator/beam/erl_lock_check.h @@ -1,7 +1,7 @@ /* * %CopyrightBegin% * - * Copyright Ericsson AB 2005-2011. All Rights Reserved. + * Copyright Ericsson AB 2005-2012. All Rights Reserved. * * The contents of this file are subject to the Erlang Public License, * Version 1.1, (the "License"); you may not use this file except in @@ -102,9 +102,10 @@ void erts_lc_unrequire_lock_flg(erts_lc_lock_t *lck, Uint16 op_flags); void erts_lc_require_lock(erts_lc_lock_t *lck); void erts_lc_unrequire_lock(erts_lc_lock_t *lck); +int erts_lc_is_emu_thr(void); #define ERTS_LC_ASSERT(A) \ - ((void) ((A) ? 1 : erts_lc_assert_failed(__FILE__, __LINE__, #A))) + ((void) (((A) || ERTS_SOMEONE_IS_CRASH_DUMPING) ? 1 : erts_lc_assert_failed(__FILE__, __LINE__, #A))) #ifdef ERTS_SMP #define ERTS_SMP_LC_ASSERT(A) ERTS_LC_ASSERT(A) #else diff --git a/erts/emulator/beam/erl_message.c b/erts/emulator/beam/erl_message.c index d5b7d01048..325d77e911 100644 --- a/erts/emulator/beam/erl_message.c +++ b/erts/emulator/beam/erl_message.c @@ -495,7 +495,7 @@ queue_message(Process *c_p, #ifndef ERTS_SMP res = receiver->msg.len; #else - res = receiver->u.alive.msg_inq.len; + res = receiver->msg_inq.len; if (*receiver_locks & ERTS_PROC_LOCK_MAIN) { /* * We move 'in queue' to 'private queue' and place @@ -894,8 +894,8 @@ erts_send_message(Process* sender, #ifdef USE_VM_PROBES *sender_name = *receiver_name = '\0'; if (DTRACE_ENABLED(message_send)) { - erts_snprintf(sender_name, sizeof(DTRACE_CHARBUF_NAME(sender_name)), "%T", sender->id); - erts_snprintf(receiver_name, sizeof(DTRACE_CHARBUF_NAME(receiver_name)), "%T", receiver->id); + erts_snprintf(sender_name, sizeof(DTRACE_CHARBUF_NAME(sender_name)), "%T", sender->common.id); + erts_snprintf(receiver_name, sizeof(DTRACE_CHARBUF_NAME(receiver_name)), "%T", receiver->common.id); } #endif if (SEQ_TRACE_TOKEN(sender) != NIL && !(flags & ERTS_SND_FLG_NO_SEQ_TRACE)) { @@ -917,7 +917,7 @@ erts_send_message(Process* sender, seq_trace_update_send(sender); seq_trace_output(stoken, message, SEQ_TRACE_SEND, - receiver->id, sender); + receiver->common.id, sender); seq_trace_size = 6; /* TUPLE5 */ #ifdef USE_VM_PROBES } @@ -948,7 +948,7 @@ erts_send_message(Process* sender, #ifdef DTRACE_TAG_HARDDEBUG erts_fprintf(stderr, "Dtrace -> (%T) Spreading tag (%T) with " - "message %T!\r\n",sender->id, utag, message); + "message %T!\r\n",sender->common.id, utag, message); #endif } #endif @@ -1136,7 +1136,7 @@ erts_deliver_exit_message(Eterm from, Process *to, ErtsProcLocks *to_locksp, save = TUPLE3(hp, am_EXIT, from_copy, mess); hp += 4; /* the trace token must in this case be updated by the caller */ - seq_trace_output(token, save, SEQ_TRACE_SEND, to->id, NULL); + seq_trace_output(token, save, SEQ_TRACE_SEND, to->common.id, NULL); temptoken = copy_struct(token, sz_token, &hp, &bp->off_heap); erts_queue_message(to, to_locksp, bp, save, temptoken #ifdef USE_VM_PROBES diff --git a/erts/emulator/beam/erl_message.h b/erts/emulator/beam/erl_message.h index 2ae94965b1..771eba431f 100644 --- a/erts/emulator/beam/erl_message.h +++ b/erts/emulator/beam/erl_message.h @@ -127,21 +127,21 @@ typedef struct { /* Move in message queue to end of private message queue */ #define ERTS_SMP_MSGQ_MV_INQ2PRIVQ(P) \ do { \ - if ((P)->u.alive.msg_inq.first) { \ - *(P)->msg.last = (P)->u.alive.msg_inq.first; \ - (P)->msg.last = (P)->u.alive.msg_inq.last; \ - (P)->msg.len += (P)->u.alive.msg_inq.len; \ - (P)->u.alive.msg_inq.first = NULL; \ - (P)->u.alive.msg_inq.last = &(P)->u.alive.msg_inq.first; \ - (P)->u.alive.msg_inq.len = 0; \ + if ((P)->msg_inq.first) { \ + *(P)->msg.last = (P)->msg_inq.first; \ + (P)->msg.last = (P)->msg_inq.last; \ + (P)->msg.len += (P)->msg_inq.len; \ + (P)->msg_inq.first = NULL; \ + (P)->msg_inq.last = &(P)->msg_inq.first; \ + (P)->msg_inq.len = 0; \ } \ } while (0) /* Add message last in message queue */ #define LINK_MESSAGE(p, mp) do { \ - *(p)->u.alive.msg_inq.last = (mp); \ - (p)->u.alive.msg_inq.last = &(mp)->next; \ - (p)->u.alive.msg_inq.len++; \ + *(p)->msg_inq.last = (mp); \ + (p)->msg_inq.last = &(mp)->next; \ + (p)->msg_inq.len++; \ } while(0) #else @@ -245,6 +245,9 @@ void erts_move_msg_attached_data_to_heap(Eterm **, ErlOffHeap *, ErlMessage *); Eterm erts_msg_distext2heap(Process *, ErtsProcLocks *, ErlHeapFragment **, Eterm *, ErtsDistExternal *); +void erts_cleanup_offheap(ErlOffHeap *offheap); + + ERTS_GLB_INLINE Uint erts_msg_used_frag_sz(const ErlMessage *msg); ERTS_GLB_INLINE Uint erts_msg_attached_data_size(ErlMessage *msg); diff --git a/erts/emulator/beam/erl_monitors.c b/erts/emulator/beam/erl_monitors.c index 1a84950120..63175c44d6 100644 --- a/erts/emulator/beam/erl_monitors.c +++ b/erts/emulator/beam/erl_monitors.c @@ -971,7 +971,7 @@ Eterm erts_debug_dump_monitors_1(BIF_ALIST_1) } } else { erts_printf("Dumping pid monitors--------------------\n"); - erts_dump_monitors(rp->monitors,0); + erts_dump_monitors(ERTS_P_MONITORS(rp),0); erts_printf("Monitors dumped-------------------------\n"); erts_smp_proc_unlock(rp, ERTS_PROC_LOCK_LINK); BIF_RET(am_true); @@ -985,12 +985,15 @@ Eterm erts_debug_dump_links_1(BIF_ALIST_1) Process *rp; DistEntry *dep; if (is_internal_port(pid)) { - Port *rport = erts_id2port(pid, p, ERTS_PROC_LOCK_MAIN); + Port *rport = erts_id2port_sflgs(pid, + p, + ERTS_PROC_LOCK_MAIN, + ERTS_PORT_SFLGS_INVALID_LOOKUP); if (rport) { erts_printf("Dumping port links----------------------\n"); - erts_dump_links(rport->nlinks,0); + erts_dump_links(ERTS_P_LINKS(rport), 0); erts_printf("Links dumped----------------------------\n"); - erts_smp_port_unlock(rport); + erts_port_release(rport); BIF_RET(am_true); } else { BIF_ERROR(p,BADARG); @@ -1014,7 +1017,7 @@ Eterm erts_debug_dump_links_1(BIF_ALIST_1) } else { erts_printf("Dumping pid links-----------------------\n"); - erts_dump_links(rp->nlinks,0); + erts_dump_links(ERTS_P_LINKS(rp), 0); erts_printf("Links dumped----------------------------\n"); erts_smp_proc_unlock(rp, ERTS_PROC_LOCK_LINK); BIF_RET(am_true); diff --git a/erts/emulator/beam/erl_monitors.h b/erts/emulator/beam/erl_monitors.h index d3f6d410dd..a7fa4e0145 100644 --- a/erts/emulator/beam/erl_monitors.h +++ b/erts/emulator/beam/erl_monitors.h @@ -137,8 +137,6 @@ typedef struct erts_suspend_monitor { #define ERTS_LINK_ROOT(Linkp) ((Linkp)->shared.root) #define ERTS_LINK_REFC(Linkp) ((Linkp)->shared.refc) -#define ERTS_LINK_ROOT_AS_UINT(Linkp) (*((Uint *) &((Linkp)->root))) - Uint erts_tot_link_lh_size(void); diff --git a/erts/emulator/beam/erl_nif.c b/erts/emulator/beam/erl_nif.c index 0dca588847..1bd2d933b2 100644 --- a/erts/emulator/beam/erl_nif.c +++ b/erts/emulator/beam/erl_nif.c @@ -263,7 +263,7 @@ ErlNifEnv* enif_alloc_env(void) HEAP_LIMIT(&msg_env->phony_proc) = phony_heap; HEAP_END(&msg_env->phony_proc) = phony_heap; MBUF(&msg_env->phony_proc) = NULL; - msg_env->phony_proc.id = ERTS_INVALID_PID; + msg_env->phony_proc.common.id = ERTS_INVALID_PID; #ifdef FORCE_HEAP_FRAGS msg_env->phony_proc.space_verified = 0; msg_env->phony_proc.space_verified_from = NULL; @@ -287,7 +287,7 @@ void enif_clear_env(ErlNifEnv* env) struct enif_msg_environment_t* menv = (struct enif_msg_environment_t*)env; Process* p = &menv->phony_proc; ASSERT(p == menv->env.proc); - ASSERT(p->id == ERTS_INVALID_PID); + ASSERT(p->common.id == ERTS_INVALID_PID); ASSERT(MBUF(p) == menv->env.heap_frag); if (MBUF(p) != NULL) { erts_cleanup_offheap(&MSO(p)); @@ -319,7 +319,7 @@ int enif_send(ErlNifEnv* env, const ErlNifPid* to_pid, if (env != NULL) { c_p = env->proc; - if (receiver == c_p->id) { + if (receiver == c_p->common.id) { rp_locks = ERTS_PROC_LOCK_MAIN; flush_me = 1; } @@ -341,7 +341,7 @@ int enif_send(ErlNifEnv* env, const ErlNifPid* to_pid, : erts_pid2proc_opt(c_p, ERTS_PROC_LOCK_MAIN, receiver, rp_locks, ERTS_P2P_FLG_SMP_INC_REFC)); if (rp == NULL) { - ASSERT(env == NULL || receiver != c_p->id); + ASSERT(env == NULL || receiver != c_p->common.id); return 0; } flush_env(msg_env); @@ -397,7 +397,7 @@ static int is_offheap(const ErlOffHeap* oh) ErlNifPid* enif_self(ErlNifEnv* caller_env, ErlNifPid* pid) { - pid->pid = caller_env->proc->id; + pid->pid = caller_env->proc->common.id; return pid; } int enif_get_local_pid(ErlNifEnv* env, ERL_NIF_TERM term, ErlNifPid* pid) @@ -505,7 +505,7 @@ int enif_inspect_iolist_as_binary(ErlNifEnv* env, Eterm term, ErlNifBinary* bin) { struct enif_tmp_obj_t* tobj; ErtsAlcType_t allocator; - Uint sz; + ErlDrvSizeT sz; if (is_binary(term)) { return enif_inspect_binary(env,term,bin); } @@ -531,7 +531,7 @@ int enif_inspect_iolist_as_binary(ErlNifEnv* env, Eterm term, ErlNifBinary* bin) bin->size = sz; bin->bin_term = THE_NON_VALUE; bin->ref_bin = NULL; - io_list_to_buf(term, (char*) bin->data, sz); + erts_iolist_to_buf(term, (char*) bin->data, sz); ADD_READONLY_CHECK(env, bin->data, bin->size); return 1; } @@ -1396,6 +1396,44 @@ size_t enif_sizeof_resource(void* obj) return ERTS_MAGIC_BIN_DATA_SIZE(bin) - offsetof(ErlNifResource,data); } + +void* enif_dlopen(const char* lib, + void (*err_handler)(void*,const char*), void* err_arg) +{ + ErtsSysDdllError errdesc = ERTS_SYS_DDLL_ERROR_INIT; + void* handle; + void* init_func; + if (erts_sys_ddll_open2(lib, &handle, &errdesc) == ERL_DE_NO_ERROR) { + if (erts_sys_ddll_load_nif_init(handle, &init_func, &errdesc) == ERL_DE_NO_ERROR) { + erts_sys_ddll_call_nif_init(init_func); + } + } + else { + if (err_handler != NULL) { + (*err_handler)(err_arg, errdesc.str); + } + handle = NULL; + } + erts_sys_ddll_free_error(&errdesc); + return handle; +} + +void* enif_dlsym(void* handle, const char* symbol, + void (*err_handler)(void*,const char*), void* err_arg) +{ + ErtsSysDdllError errdesc = ERTS_SYS_DDLL_ERROR_INIT; + void* ret; + if (erts_sys_ddll_sym2(handle, symbol, &ret, &errdesc) != ERL_DE_NO_ERROR) { + if (err_handler != NULL) { + (*err_handler)(err_arg, errdesc.str); + } + erts_sys_ddll_free_error(&errdesc); + return NULL; + } + return ret; +} + + /*************************************************************************** ** load_nif/2 ** ***************************************************************************/ @@ -1806,7 +1844,7 @@ void erl_nif_init() #ifdef USE_VM_PROBES void dtrace_nifenv_str(ErlNifEnv *env, char *process_buf) { - dtrace_pid_str(env->proc->id, process_buf); + dtrace_pid_str(env->proc->common.id, process_buf); } #endif diff --git a/erts/emulator/beam/erl_nif_api_funcs.h b/erts/emulator/beam/erl_nif_api_funcs.h index 6396af09d0..51ff1eaa48 100644 --- a/erts/emulator/beam/erl_nif_api_funcs.h +++ b/erts/emulator/beam/erl_nif_api_funcs.h @@ -138,6 +138,8 @@ ERL_NIF_API_FUNC_DECL(ERL_NIF_TERM,enif_make_uint64,(ErlNifEnv*, ErlNifUInt64)); ERL_NIF_API_FUNC_DECL(int,enif_is_exception,(ErlNifEnv*, ERL_NIF_TERM term)); ERL_NIF_API_FUNC_DECL(int,enif_make_reverse_list,(ErlNifEnv*, ERL_NIF_TERM term, ERL_NIF_TERM *list)); ERL_NIF_API_FUNC_DECL(int,enif_is_number,(ErlNifEnv*, ERL_NIF_TERM term)); +ERL_NIF_API_FUNC_DECL(void*,enif_dlopen,(const char* lib, void (*err_handler)(void*,const char*), void* err_arg)); +ERL_NIF_API_FUNC_DECL(void*,enif_dlsym,(void* handle, const char* symbol, void (*err_handler)(void*,const char*), void* err_arg)); /* ** Add new entries here to keep compatibility on Windows!!! @@ -260,6 +262,8 @@ ERL_NIF_API_FUNC_DECL(int,enif_is_number,(ErlNifEnv*, ERL_NIF_TERM term)); # define enif_is_exception ERL_NIF_API_FUNC_MACRO(enif_is_exception) # define enif_make_reverse_list ERL_NIF_API_FUNC_MACRO(enif_make_reverse_list) # define enif_is_number ERL_NIF_API_FUNC_MACRO(enif_is_number) +# define enif_dlopen ERL_NIF_API_FUNC_MACRO(enif_dlopen) +# define enif_dlsym ERL_NIF_API_FUNC_MACRO(enif_dlsym) /* ** Add new entries here diff --git a/erts/emulator/beam/erl_node_container_utils.h b/erts/emulator/beam/erl_node_container_utils.h index 7b4cb7b042..667bda255b 100644 --- a/erts/emulator/beam/erl_node_container_utils.h +++ b/erts/emulator/beam/erl_node_container_utils.h @@ -20,7 +20,7 @@ #ifndef ERL_NODE_CONTAINER_UTILS_H__ #define ERL_NODE_CONTAINER_UTILS_H__ -#include "erl_term.h" +#include "erl_ptab.h" /* * Note regarding node containers: @@ -29,9 +29,6 @@ * the emulator) for the Erlang data types that contain a reference * to a node, i.e. pids, ports, and references. * - * Observe! The layouts of the node container data types have been - * changed in R9. - * * Node containers are divided into internal and external node containers. * An internal node container refer to the current incarnation of the * node which it reside on. An external node container refer to @@ -52,13 +49,6 @@ * reference is a boxed data type. An internal node container have an * implicit reference to the 'erts_this_node' element in the node table. * - * Due to the R9 changes in layouts of node containers there are room to - * store more data than previously. Today (R9) this extra space is unused, - * but it is planned to be used in the future. For example only 18 bits - * are used for data in a pid but there is room for 28 bits of data (on a - * 32-bit machine). Some preparations have been made in the emulator for - * usage of this extra space. - * * OBSERVE! Pids doesn't use fixed size 'serial' and 'number' fields any * more. Previously the 15 bit 'number' field of a pid was used as index * into the process table, and the 3 bit 'serial' field was used as a @@ -104,8 +94,6 @@ #define internal_dist_entry(x) (erts_this_node->dist_entry) #define external_dist_entry(x) (external_node((x))->dist_entry) -extern int erts_use_r9_pids_ports; - /* * For this node (and previous incarnations of this node), 0 is used as * channel no. For other nodes, the atom index of the atom corresponding @@ -128,47 +116,20 @@ extern int erts_use_r9_pids_ports; * Pids * \* */ -#define erts_max_processes erts_proc.max - -typedef struct { - erts_smp_atomic_t *tab; - int max; - int tab_cache_lines; - int pix_per_cache_line; - int pix_cl_mask; - int pix_cl_shift; - int pix_cli_mask; - int pix_cli_shift; -} ErtsProcTab; - -extern ErtsProcTab erts_proc; - -ERTS_GLB_INLINE int erts_pid_data2ix(Eterm pid_data); - -#if ERTS_GLB_INLINE_INCL_FUNC_DEF - -ERTS_GLB_INLINE int erts_pid_data2ix(Eterm pid_data) -{ - int n, pix; - - n = (int) pid_data; - if (erts_proc.pix_cl_mask) { - pix = ((n & erts_proc.pix_cl_mask) << erts_proc.pix_cl_shift); - pix += ((n >> erts_proc.pix_cli_shift) & erts_proc.pix_cli_mask); - } - else { - n %= erts_proc.max; - pix = n % erts_proc.tab_cache_lines; - pix *= erts_proc.pix_per_cache_line; - pix += n / erts_proc.tab_cache_lines; - } - ASSERT(0 <= pix && pix < erts_proc.max); - return pix; -} +extern ErtsPTab erts_proc; -#endif +#define make_internal_pid(D) erts_ptab_make_id(&erts_proc, \ + (D), \ + _TAG_IMMED1_PID) -#define internal_pid_index(x) erts_pid_data2ix(internal_pid_data((x))) +#define internal_pid_index(PID) (ASSERT_EXPR(is_internal_pid((PID))), \ + erts_ptab_id2pix(&erts_proc, (PID))) + +#define internal_pid_data(PID) (ASSERT_EXPR(is_internal_pid((PID))), \ + erts_ptab_id2data(&erts_proc, (PID))) + +#define internal_pid_number(x) _GET_PID_NUM(internal_pid_data((x))) +#define internal_pid_serial(x) _GET_PID_SER(internal_pid_data((x))) #define internal_pid_node_name(x) (internal_pid_node((x))->sysname) #define external_pid_node_name(x) (external_pid_node((x))->sysname) @@ -208,34 +169,37 @@ ERTS_GLB_INLINE int erts_pid_data2ix(Eterm pid_data) || is_external_pid((x))) #define is_not_pid(x) (!is_pid(x)) -#define ERTS_MAX_R9_PROCESSES (1 << ERTS_R9_PROC_BITS) - /* * Maximum number of processes. We want the number to fit in a SMALL on * 32-bit CPU. */ -#define ERTS_MAX_PROCESSES ((SWORD_CONSTANT(1) << 27)-1) -#if (ERTS_MAX_PROCESSES > MAX_SMALL) -# error "The maximum number of processes must fit in a SMALL." -#endif - +#define ERTS_MAX_PROCESSES (ERTS_PTAB_MAX_SIZE-1) #define ERTS_MAX_PID_DATA ((1 << _PID_DATA_SIZE) - 1) #define ERTS_MAX_PID_NUMBER ((1 << _PID_NUM_SIZE) - 1) #define ERTS_MAX_PID_SERIAL ((1 << _PID_SER_SIZE) - 1) -#define ERTS_MAX_PID_R9_SERIAL ((1 << _PID_R9_SER_SIZE) - 1) -#define ERTS_R9_PROC_BITS (_PID_R9_SER_SIZE + _PID_NUM_SIZE) #define ERTS_PROC_BITS (_PID_SER_SIZE + _PID_NUM_SIZE) -#define ERTS_INVALID_PID make_internal_pid(ERTS_MAX_PID_DATA) +#define ERTS_INVALID_PID ERTS_PTAB_INVALID_ID(_TAG_IMMED1_PID) /* * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * *\ * Ports * \* */ -#define internal_port_index(x) (internal_port_data((x)) \ - & erts_port_tab_index_mask) +extern ErtsPTab erts_port; + +#define make_internal_port(D) erts_ptab_make_id(&erts_port, \ + (D), \ + _TAG_IMMED1_PORT) + +#define internal_port_index(PRT) (ASSERT_EXPR(is_internal_port((PRT))), \ + erts_ptab_id2pix(&erts_port, (PRT))) + +#define internal_port_data(PRT) (ASSERT_EXPR(is_internal_port((PRT))), \ + erts_ptab_id2data(&erts_port, (PRT))) + +#define internal_port_number(x) _GET_PORT_NUM(internal_port_data((x))) #define internal_port_node_name(x) (internal_port_node((x))->sysname) #define external_port_node_name(x) (external_port_node((x))->sysname) @@ -274,18 +238,18 @@ ERTS_GLB_INLINE int erts_pid_data2ix(Eterm pid_data) #define is_not_port(x) (!is_port(x)) /* Highest port-ID part in a term of type Port - Not necessarily the same as the variable erts_max_ports + Not necessarily the same as current maximum port table size which defines the maximum number of simultaneous Ports in the Erlang node. ERTS_MAX_PORTS is a hard upper limit. */ -#define ERTS_MAX_R9_PORTS (1 << ERTS_R9_PORTS_BITS) -#define ERTS_MAX_PORTS (1 << ERTS_PORTS_BITS) - +#define ERTS_MAX_PORTS (ERTS_PTAB_MAX_SIZE-1) #define ERTS_MAX_PORT_DATA ((1 << _PORT_DATA_SIZE) - 1) #define ERTS_MAX_PORT_NUMBER ((1 << _PORT_NUM_SIZE) - 1) -#define ERTS_R9_PORTS_BITS (_PORT_R9_NUM_SIZE) #define ERTS_PORTS_BITS (_PORT_NUM_SIZE) + +#define ERTS_INVALID_PORT ERTS_PTAB_INVALID_ID(_TAG_IMMED1_PORT) + /* * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * *\ * Refs * \* */ diff --git a/erts/emulator/beam/erl_node_tables.c b/erts/emulator/beam/erl_node_tables.c index 40837d3817..ebfba065d1 100644 --- a/erts/emulator/beam/erl_node_tables.c +++ b/erts/emulator/beam/erl_node_tables.c @@ -116,8 +116,7 @@ dist_table_alloc(void *dep_tmpl) dep->qsize = 0; dep->out_queue.first = NULL; dep->out_queue.last = NULL; - dep->suspended.first = NULL; - dep->suspended.last = NULL; + dep->suspended = NULL; dep->finalized_out_queue.first = NULL; dep->finalized_out_queue.last = NULL; @@ -769,8 +768,7 @@ void erts_init_node_tables(void) erts_this_dist_entry->qsize = 0; erts_this_dist_entry->out_queue.first = NULL; erts_this_dist_entry->out_queue.last = NULL; - erts_this_dist_entry->suspended.first = NULL; - erts_this_dist_entry->suspended.last = NULL; + erts_this_dist_entry->suspended = NULL; erts_this_dist_entry->finalized_out_queue.first = NULL; erts_this_dist_entry->finalized_out_queue.last = NULL; @@ -1268,7 +1266,7 @@ setup_reference_table(void) ErlHeapFragment *hfp; DistEntry *dep; HashInfo hi; - int i; + int i, max; DeclareTmpHeapNoproc(heap,3); inserted_bins = NULL; @@ -1297,8 +1295,9 @@ setup_reference_table(void) UnUseTmpHeapNoproc(3); + max = erts_ptab_max(&erts_proc); /* Insert all processes */ - for (i = 0; i < erts_max_processes; i++) { + for (i = 0; i < max; i++) { Process *proc = erts_pix2proc(i); if (proc) { ErlMessage *msg; @@ -1306,12 +1305,12 @@ setup_reference_table(void) /* Insert Heap */ insert_offheap(&(proc->off_heap), HEAP_REF, - proc->id); + proc->common.id); /* Insert message buffers */ for(hfp = proc->mbuf; hfp; hfp = hfp->next) insert_offheap(&(hfp->off_heap), HEAP_REF, - proc->id); + proc->common.id); /* Insert msg msg buffers */ for (msg = proc->msg.first; msg; msg = msg->next) { ErlHeapFragment *heap_frag = NULL; @@ -1321,7 +1320,7 @@ setup_reference_table(void) else { if (msg->data.dist_ext->dep) insert_dist_entry(msg->data.dist_ext->dep, - HEAP_REF, proc->id, 0); + HEAP_REF, proc->common.id, 0); if (is_not_nil(ERL_MESSAGE_TOKEN(msg))) heap_frag = erts_dist_ext_trailer(msg->data.dist_ext); } @@ -1329,10 +1328,10 @@ setup_reference_table(void) if (heap_frag) insert_offheap(&(heap_frag->off_heap), HEAP_REF, - proc->id); + proc->common.id); } #ifdef ERTS_SMP - for (msg = proc->u.alive.msg_inq.first; msg; msg = msg->next) { + for (msg = proc->msg_inq.first; msg; msg = msg->next) { ErlHeapFragment *heap_frag = NULL; if (msg->data.attached) { if (is_value(ERL_MESSAGE_TERM(msg))) @@ -1340,7 +1339,7 @@ setup_reference_table(void) else { if (msg->data.dist_ext->dep) insert_dist_entry(msg->data.dist_ext->dep, - HEAP_REF, proc->id, 0); + HEAP_REF, proc->common.id, 0); if (is_not_nil(ERL_MESSAGE_TOKEN(msg))) heap_frag = erts_dist_ext_trailer(msg->data.dist_ext); } @@ -1348,19 +1347,19 @@ setup_reference_table(void) if (heap_frag) insert_offheap(&(heap_frag->off_heap), HEAP_REF, - proc->id); + proc->common.id); } #endif /* Insert links */ - if(proc->nlinks) - insert_links(proc->nlinks, proc->id); - if(proc->monitors) - insert_monitors(proc->monitors, proc->id); + if (ERTS_P_LINKS(proc)) + insert_links(ERTS_P_LINKS(proc), proc->common.id); + if (ERTS_P_MONITORS(proc)) + insert_monitors(ERTS_P_MONITORS(proc), proc->common.id); /* Insert controller */ { DistEntry *dep = ERTS_PROC_GET_DIST_ENTRY(proc); if (dep) - insert_dist_entry(dep, CTRL_REF, proc->id, 0); + insert_dist_entry(dep, CTRL_REF, proc->common.id, 0); } } } @@ -1370,21 +1369,33 @@ setup_reference_table(void) #endif /* Insert all ports */ - for (i = 0; i < erts_max_ports; i++) { - if (erts_port[i].status & ERTS_PORT_SFLGS_DEAD) + max = erts_ptab_max(&erts_port); + for (i = 0; i < max; i++) { + erts_aint32_t state; + Port *prt; + + prt = erts_pix2port(i); + if (!prt) + continue; + + state = erts_atomic32_read_nob(&prt->state); + if (state & ERTS_PORT_SFLGS_DEAD) continue; /* Insert links */ - if(erts_port[i].nlinks) - insert_links(erts_port[i].nlinks, erts_port[i].id); + if (ERTS_P_LINKS(prt)) + insert_links(ERTS_P_LINKS(prt), prt->common.id); + /* Insert monitors */ + if (ERTS_P_MONITORS(prt)) + insert_monitors(ERTS_P_MONITORS(prt), prt->common.id); /* Insert port data */ - for(hfp = erts_port[i].bp; hfp; hfp = hfp->next) - insert_offheap(&(hfp->off_heap), HEAP_REF, erts_port[i].id); + for(hfp = prt->bp; hfp; hfp = hfp->next) + insert_offheap(&(hfp->off_heap), HEAP_REF, prt->common.id); /* Insert controller */ - if (erts_port[i].dist_entry) - insert_dist_entry(erts_port[i].dist_entry, + if (prt->dist_entry) + insert_dist_entry(prt->dist_entry, CTRL_REF, - erts_port[i].id, + prt->common.id, 0); } diff --git a/erts/emulator/beam/erl_node_tables.h b/erts/emulator/beam/erl_node_tables.h index 4a015bdef9..af60071ea5 100644 --- a/erts/emulator/beam/erl_node_tables.h +++ b/erts/emulator/beam/erl_node_tables.h @@ -84,10 +84,6 @@ typedef struct { } ErtsDistOutputQueue; struct ErtsProcList_; -typedef struct { - struct ErtsProcList_ *first; - struct ErtsProcList_ *last; -} ErtsDistSuspended; /* * Lock order: @@ -100,7 +96,6 @@ typedef struct { */ struct erl_link; -struct port; typedef struct dist_entry_ { HashBucket hash_bucket; /* Hash bucket */ @@ -135,13 +130,13 @@ typedef struct dist_entry_ { Uint32 qflgs; Sint qsize; ErtsDistOutputQueue out_queue; - ErtsDistSuspended suspended; + struct ErtsProcList_ *suspended; ErtsDistOutputQueue finalized_out_queue; erts_smp_atomic_t dist_cmd_scheduled; ErtsPortTaskHandle dist_cmd; - Uint (*send)(struct port *prt, ErtsDistOutputBuf *obuf); + Uint (*send)(Port *prt, ErtsDistOutputBuf *obuf); struct cache* cache; /* The atom cache */ } DistEntry; diff --git a/erts/emulator/beam/erl_port.h b/erts/emulator/beam/erl_port.h new file mode 100644 index 0000000000..fb07f3d5bc --- /dev/null +++ b/erts/emulator/beam/erl_port.h @@ -0,0 +1,944 @@ +/* + * %CopyrightBegin% + * + * Copyright Ericsson AB 2012. All Rights Reserved. + * + * The contents of this file are subject to the Erlang Public License, + * Version 1.1, (the "License"); you may not use this file except in + * compliance with the License. You should have received a copy of the + * Erlang Public License along with this software. If not, it can be + * retrieved online at http://www.erlang.org/. + * + * Software distributed under the License is distributed on an "AS IS" + * basis, WITHOUT WARRANTY OF ANY KIND, either express or implied. See + * the License for the specific language governing rights and limitations + * under the License. + * + * %CopyrightEnd% + */ + +#ifndef ERL_PORT_TYPE__ +#define ERL_PORT_TYPE__ +typedef struct _erl_drv_port Port; +typedef struct ErtsProc2PortSigData_ ErtsProc2PortSigData; +#endif + +#if !defined(ERL_PORT_H__) && !defined(ERL_PORT_GET_PORT_TYPE_ONLY__) +#define ERL_PORT_H__ + +#include "erl_port_task.h" +#include "erl_ptab.h" +#include "erl_thr_progress.h" +#include "erl_trace.h" + +#define ERTS_DEFAULT_MAX_PORTS (1 << 16) +#define ERTS_MIN_PORTS 1024 + +extern int erts_port_synchronous_ops; +extern int erts_port_schedule_all_ops; +extern int erts_port_parallelism; + +typedef struct erts_driver_t_ erts_driver_t; + +#define ERTS_INVALID_ERL_DRV_PORT ((ErlDrvPort) (SWord) -1) +#define SMALL_IO_QUEUE 5 /* Number of fixed elements */ + +typedef struct { + ErlDrvSizeT size; /* total size in bytes */ + + SysIOVec* v_start; + SysIOVec* v_end; + SysIOVec* v_head; + SysIOVec* v_tail; + SysIOVec v_small[SMALL_IO_QUEUE]; + + ErlDrvBinary** b_start; + ErlDrvBinary** b_end; + ErlDrvBinary** b_head; + ErlDrvBinary** b_tail; + ErlDrvBinary* b_small[SMALL_IO_QUEUE]; +} ErlIOQueue; + +typedef struct line_buf { /* Buffer used in line oriented I/O */ + ErlDrvSizeT bufsiz; /* Size of character buffer */ + ErlDrvSizeT ovlen; /* Length of overflow data */ + ErlDrvSizeT ovsiz; /* Actual size of overflow buffer */ + char data[1]; /* Starting point of buffer data, + data[0] is a flag indicating an unprocess CR, + The rest is the overflow buffer. */ +} LineBuf; + +/* + * Items part of erlang:port_info/1 result. Note am_registered_name + * *need* to be first. + */ + +#define ERTS_PORT_INFO_1_ITEMS \ + { am_registered_name, /* Needs to be first */ \ + am_name, \ + am_links, \ + am_id, \ + am_connected, \ + am_input, \ + am_output, \ + am_os_pid } + +/* + * Port Specific Data. + * + * Only use PrtSD for very rarely used data. + */ + +#define ERTS_PRTSD_SCHED_ID 0 + +#define ERTS_PRTSD_SIZE 1 + +typedef struct { + void *data[ERTS_PRTSD_SIZE]; +} ErtsPrtSD; + +#ifdef ERTS_SMP +typedef struct ErtsXPortsList_ ErtsXPortsList; +#endif + +/* + * Port locking: + * + * Locking is done either driver specific or port specific. When + * driver specific locking is used, all instances of the driver, + * i.e. ports running the driver, share the same lock. When port + * specific locking is used each instance have its own lock. + * + * Most fields in the Port structure are protected by the lock + * referred to by the 'lock' field. This lock is shared between + * all ports running the same driver when driver specific locking + * is used. + * + * The 'sched' field is protected by the run queue lock that the + * port currently is assigned to. + * + */ + +struct _erl_drv_port { + ErtsPTabElementCommon common; /* *Need* to be first in struct */ + + ErtsPortTaskSched sched; + ErtsPortTaskHandle timeout_task; +#ifdef ERTS_SMP + erts_mtx_t *lock; + ErtsXPortsList *xports; + erts_smp_atomic_t run_queue; +#else + erts_atomic32_t refc; + int cleanup; +#endif + erts_atomic_t connected; /* A connected process */ + Eterm caller; /* Current caller. */ + Eterm data; /* Data associated with port. */ + ErlHeapFragment* bp; /* Heap fragment holding data (NULL if imm data). */ + Uint bytes_in; /* Number of bytes read */ + Uint bytes_out; /* Number of bytes written */ + + ErlIOQueue ioq; /* driver accessible i/o queue */ + DistEntry *dist_entry; /* Dist entry used in DISTRIBUTION */ + char *name; /* String used in the open */ + erts_driver_t* drv_ptr; + UWord drv_data; + SWord os_pid; /* Child process ID */ + ErtsProcList *suspended; /* List of suspended processes. */ + LineBuf *linebuf; /* Buffer to hold data not ready for + process to get (line oriented I/O)*/ + erts_atomic32_t state; /* Status and type flags */ + int control_flags; /* Flags for port_control() */ + ErlDrvPDL port_data_lock; + + ErtsPrtSD *psd; /* Port specific data */ +}; + +#define ERTS_PORT_GET_CONNECTED(PRT) \ + ((Eterm) erts_atomic_read_nob(&(PRT)->connected)) +#define ERTS_PORT_SET_CONNECTED(PRT, PID) \ + erts_atomic_set_relb(&(PRT)->connected, (erts_aint_t) (PID)) +#define ERTS_PORT_INIT_CONNECTED(PRT, PID) \ + erts_atomic_init_nob(&(PRT)->connected, (erts_aint_t) (PID)) + + +struct erl_drv_port_data_lock { + erts_mtx_t mtx; + erts_atomic_t refc; + Port *prt; +}; + +ERTS_GLB_INLINE ErtsRunQueue *erts_port_runq(Port *prt); + +#if ERTS_GLB_INLINE_INCL_FUNC_DEF + +ERTS_GLB_INLINE ErtsRunQueue * +erts_port_runq(Port *prt) +{ +#ifdef ERTS_SMP + ErtsRunQueue *rq1, *rq2; + rq1 = (ErtsRunQueue *) erts_smp_atomic_read_nob(&prt->run_queue); + if (!rq1) + return NULL; + while (1) { + erts_smp_runq_lock(rq1); + rq2 = (ErtsRunQueue *) erts_smp_atomic_read_nob(&prt->run_queue); + if (rq1 == rq2) + return rq1; + erts_smp_runq_unlock(rq1); + rq1 = rq2; + if (!rq1) + return NULL; + } +#else + return ERTS_RUNQ_IX(0); +#endif +} + +#endif + + +ERTS_GLB_INLINE void *erts_prtsd_get(Port *p, int ix); +ERTS_GLB_INLINE void *erts_prtsd_set(Port *p, int ix, void *new); + +#if ERTS_GLB_INLINE_INCL_FUNC_DEF + +ERTS_GLB_INLINE void * +erts_prtsd_get(Port *prt, int ix) +{ + return prt->psd ? prt->psd->data[ix] : NULL; +} + +ERTS_GLB_INLINE void * +erts_prtsd_set(Port *prt, int ix, void *data) +{ + if (prt->psd) { + void *old = prt->psd->data[ix]; + prt->psd->data[ix] = data; + return old; + } + else { + prt->psd = erts_alloc(ERTS_ALC_T_PRTSD, sizeof(ErtsPrtSD)); + prt->psd->data[ix] = data; + return NULL; + } +} + +#endif + +extern erts_smp_atomic_t erts_bytes_out; /* no bytes written out */ +extern erts_smp_atomic_t erts_bytes_in; /* no bytes sent into the system */ + + +/* port status flags */ + +#define ERTS_PORT_SFLG_CONNECTED ((Uint32) (1 << 0)) +/* Port have begun exiting */ +#define ERTS_PORT_SFLG_EXITING ((Uint32) (1 << 1)) +/* Distribution port */ +#define ERTS_PORT_SFLG_DISTRIBUTION ((Uint32) (1 << 2)) +#define ERTS_PORT_SFLG_BINARY_IO ((Uint32) (1 << 3)) +#define ERTS_PORT_SFLG_SOFT_EOF ((Uint32) (1 << 4)) +/* Flow control */ +/* Port is closing (no i/o accepted) */ +#define ERTS_PORT_SFLG_CLOSING ((Uint32) (1 << 5)) +/* Send a closed message when terminating */ +#define ERTS_PORT_SFLG_SEND_CLOSED ((Uint32) (1 << 6)) +/* Line orinted io on port */ +#define ERTS_PORT_SFLG_LINEBUF_IO ((Uint32) (1 << 7)) +/* Immortal port (only certain system ports) */ +#define ERTS_PORT_SFLG_FREE ((Uint32) (1 << 8)) +#define ERTS_PORT_SFLG_INITIALIZING ((Uint32) (1 << 9)) +/* Port uses port specific locking (opposed to driver specific locking) */ +#define ERTS_PORT_SFLG_PORT_SPECIFIC_LOCK ((Uint32) (1 << 10)) +#define ERTS_PORT_SFLG_INVALID ((Uint32) (1 << 11)) +/* Last port to terminate halts the emulator */ +#define ERTS_PORT_SFLG_HALT ((Uint32) (1 << 12)) +#ifdef DEBUG +/* Only debug: make sure all flags aren't cleared unintentionally */ +#define ERTS_PORT_SFLG_PORT_DEBUG ((Uint32) (1 << 31)) +#endif + +/* Combinations of port status flags */ +#define ERTS_PORT_SFLGS_DEAD \ + (ERTS_PORT_SFLG_FREE | ERTS_PORT_SFLG_INITIALIZING) +#define ERTS_PORT_SFLGS_INVALID_DRIVER_LOOKUP \ + (ERTS_PORT_SFLGS_DEAD | ERTS_PORT_SFLG_INVALID) +#define ERTS_PORT_SFLGS_INVALID_LOOKUP \ + (ERTS_PORT_SFLGS_INVALID_DRIVER_LOOKUP \ + | ERTS_PORT_SFLG_EXITING \ + | ERTS_PORT_SFLG_CLOSING) +#define ERTS_PORT_SFLGS_INVALID_TRACER_LOOKUP \ + (ERTS_PORT_SFLGS_INVALID_LOOKUP \ + | ERTS_PORT_SFLG_DISTRIBUTION) + + +/* + * Costs in reductions for some port operations. + */ +#define ERTS_PORT_REDS_EXECUTE 10 +#define ERTS_PORT_REDS_FREE 100 +#define ERTS_PORT_REDS_TIMEOUT 400 +#define ERTS_PORT_REDS_INPUT 400 +#define ERTS_PORT_REDS_OUTPUT 400 +#define ERTS_PORT_REDS_EVENT 400 +#define ERTS_PORT_REDS_CMD_OUTPUTV 400 +#define ERTS_PORT_REDS_CMD_OUTPUT 400 +#define ERTS_PORT_REDS_EXIT 300 +#define ERTS_PORT_REDS_CONNECT 40 +#define ERTS_PORT_REDS_UNLINK 40 +#define ERTS_PORT_REDS_LINK 40 +#define ERTS_PORT_REDS_BADSIG 40 +#define ERTS_PORT_REDS_CONTROL 400 +#define ERTS_PORT_REDS_CALL 400 +#define ERTS_PORT_REDS_INFO 100 +#define ERTS_PORT_REDS_SET_DATA 40 +#define ERTS_PORT_REDS_GET_DATA 40 +#define ERTS_PORT_REDS_TERMINATE 200 + +void print_port_info(Port *, int, void *); +void erts_port_free(Port *); +#ifndef ERTS_SMP +void erts_port_cleanup(Port *); +#endif +void erts_fire_port_monitor(Port *prt, Eterm ref); +#ifdef ERTS_SMP +int erts_port_handle_xports(Port *); +#endif + +#if defined(ERTS_SMP) && defined(ERTS_ENABLE_LOCK_CHECK) +int erts_lc_is_port_locked(Port *); +#endif + +ERTS_GLB_INLINE void erts_port_inc_refc(Port *prt); +ERTS_GLB_INLINE void erts_port_dec_refc(Port *prt); +ERTS_GLB_INLINE void erts_port_add_refc(Port *prt, Sint32 add_refc); + +ERTS_GLB_INLINE int erts_smp_port_trylock(Port *prt); +ERTS_GLB_INLINE void erts_smp_port_lock(Port *prt); +ERTS_GLB_INLINE void erts_smp_port_unlock(Port *prt); + +#if ERTS_GLB_INLINE_INCL_FUNC_DEF + +ERTS_GLB_INLINE void erts_port_inc_refc(Port *prt) +{ +#ifdef ERTS_SMP + erts_ptab_inc_refc(&prt->common); +#else + erts_atomic32_inc_nob(&prt->refc); +#endif +} + +ERTS_GLB_INLINE void erts_port_dec_refc(Port *prt) +{ +#ifdef ERTS_SMP + int referred = erts_ptab_dec_test_refc(&prt->common); + if (!referred) + erts_port_free(prt); +#else + int refc = erts_atomic32_dec_read_nob(&prt->refc); + if (refc == 0) + erts_port_free(prt); +#endif +} + +ERTS_GLB_INLINE void erts_port_add_refc(Port *prt, Sint32 add_refc) +{ +#ifdef ERTS_SMP + int referred = erts_ptab_add_test_refc(&prt->common, add_refc); + if (!referred) + erts_port_free(prt); +#else + int refc = erts_atomic32_add_read_nob(&prt->refc, add_refc); + if (refc == 0) + erts_port_free(prt); +#endif +} + +ERTS_GLB_INLINE int +erts_smp_port_trylock(Port *prt) +{ +#ifdef ERTS_SMP + /* *Need* to be a managed thread */ + ERTS_SMP_LC_ASSERT(erts_thr_progress_is_managed_thread()); + return erts_mtx_trylock(prt->lock); +#else + return 0; +#endif +} + +ERTS_GLB_INLINE void +erts_smp_port_lock(Port *prt) +{ +#ifdef ERTS_SMP + /* *Need* to be a managed thread */ + ERTS_SMP_LC_ASSERT(erts_thr_progress_is_managed_thread()); + erts_mtx_lock(prt->lock); +#endif +} + +ERTS_GLB_INLINE void +erts_smp_port_unlock(Port *prt) +{ +#ifdef ERTS_SMP + /* *Need* to be a managed thread */ + ERTS_SMP_LC_ASSERT(erts_thr_progress_is_managed_thread()); + erts_mtx_unlock(prt->lock); +#endif +} + +#endif /* #if ERTS_GLB_INLINE_INCL_FUNC_DEF */ + + +#define ERTS_INVALID_PORT_OPT(PP, ID, FLGS) \ + (!(PP) \ + || (erts_atomic32_read_nob(&(PP)->state) & (FLGS)) \ + || (PP)->common.id != (ID)) + +/* port lookup */ + +#define INVALID_PORT(PP, ID) \ + ERTS_INVALID_PORT_OPT((PP), (ID), ERTS_PORT_SFLGS_INVALID_LOOKUP) + +/* Invalidate trace port if anything suspicious, for instance + * that the port is a distribution port or it is busy. + */ +#define INVALID_TRACER_PORT(PP, ID) \ + ERTS_INVALID_PORT_OPT((PP), (ID), ERTS_PORT_SFLGS_INVALID_TRACER_LOOKUP) + +#define ERTS_PORT_SCHED_ID(P, ID) \ + ((Uint) (UWord) erts_prtsd_set((P), ERTS_PSD_SCHED_ID, (void *) (UWord) (ID))) + +extern const Port erts_invalid_port; +#define ERTS_PORT_LOCK_BUSY ((Port *) &erts_invalid_port) + +int erts_is_port_ioq_empty(Port *); +void erts_terminate_port(Port *); + +#ifdef ERTS_SMP +Port *erts_de2port(DistEntry *, Process *, ErtsProcLocks); +#endif + +ERTS_GLB_INLINE Port *erts_pix2port(int); +ERTS_GLB_INLINE Port *erts_port_lookup_raw(Eterm); +ERTS_GLB_INLINE Port *erts_port_lookup(Eterm, Uint32); +ERTS_GLB_INLINE Port*erts_id2port(Eterm id); +ERTS_GLB_INLINE Port *erts_id2port_sflgs(Eterm, Process *, ErtsProcLocks, Uint32); +ERTS_GLB_INLINE void erts_port_release(Port *); +#ifdef ERTS_SMP +ERTS_GLB_INLINE Port *erts_thr_id2port_sflgs(Eterm id, Uint32 invalid_sflgs); +ERTS_GLB_INLINE void erts_thr_port_release(Port *prt); +#endif +ERTS_GLB_INLINE Port *erts_thr_drvport2port_raw(ErlDrvPort, int); +ERTS_GLB_INLINE Port *erts_drvport2port_raw(ErlDrvPort drvport); +ERTS_GLB_INLINE Port *erts_drvport2port(ErlDrvPort, erts_aint32_t *); +ERTS_GLB_INLINE Port *erts_drvportid2port(Eterm); +ERTS_GLB_INLINE Eterm erts_drvport2id(ErlDrvPort); +ERTS_GLB_INLINE Uint32 erts_portid2status(Eterm); +ERTS_GLB_INLINE int erts_is_port_alive(Eterm); +ERTS_GLB_INLINE int erts_is_valid_tracer_port(Eterm); +ERTS_GLB_INLINE int erts_port_driver_callback_epilogue(Port *, erts_aint32_t *); + +#if ERTS_GLB_INLINE_INCL_FUNC_DEF + +ERTS_GLB_INLINE Port *erts_pix2port(int ix) +{ + Port *prt; + ASSERT(0 <= ix && ix < erts_ptab_max(&erts_port)); + prt = (Port *) erts_ptab_pix2intptr_nob(&erts_port, ix); + return prt == ERTS_PORT_LOCK_BUSY ? NULL : prt; +} + +ERTS_GLB_INLINE Port * +erts_port_lookup_raw(Eterm id) +{ + Port *prt; + + ERTS_SMP_LC_ASSERT(erts_thr_progress_lc_is_delaying()); + + if (is_not_internal_port(id)) + return NULL; + + prt = (Port *) erts_ptab_pix2intptr_ddrb(&erts_port, + internal_port_index(id)); + return prt && prt->common.id == id ? prt : NULL; +} + +ERTS_GLB_INLINE Port * +erts_port_lookup(Eterm id, Uint32 invalid_sflgs) +{ + Port *prt = erts_port_lookup_raw(id); + return (!prt + ? NULL + : ((invalid_sflgs & erts_atomic32_read_nob(&prt->state)) + ? NULL + : prt)); +} + + +ERTS_GLB_INLINE Port* +erts_id2port(Eterm id) +{ + erts_aint32_t state; + Port *prt; + + /* Only allowed to be called from managed threads */ + ERTS_SMP_LC_ASSERT(erts_thr_progress_is_managed_thread()); + + if (is_not_internal_port(id)) + return NULL; + + prt = (Port *) erts_ptab_pix2intptr_ddrb(&erts_port, + internal_port_index(id)); + + if (!prt || prt->common.id != id) + return NULL; + + erts_smp_port_lock(prt); + state = erts_atomic32_read_nob(&prt->state); + if (state & ERTS_PORT_SFLGS_INVALID_LOOKUP) { + erts_smp_port_unlock(prt); + return NULL; + } + + return prt; +} + + +ERTS_GLB_INLINE Port* +erts_id2port_sflgs(Eterm id, + Process *c_p, ErtsProcLocks c_p_locks, + Uint32 invalid_sflgs) +{ +#ifdef ERTS_SMP + int no_proc_locks = !c_p || !c_p_locks; +#endif + erts_aint32_t state; + Port *prt; + + /* Only allowed to be called from managed threads */ + ERTS_SMP_LC_ASSERT(erts_thr_progress_is_managed_thread()); + + if (is_not_internal_port(id)) + return NULL; + + prt = (Port *) erts_ptab_pix2intptr_ddrb(&erts_port, + internal_port_index(id)); + + if (!prt || prt->common.id != id) + return NULL; + +#ifdef ERTS_SMP + if (no_proc_locks) + erts_smp_port_lock(prt); + else if (erts_smp_port_trylock(prt) == EBUSY) { + /* Unlock process locks, and acquire locks in lock order... */ + erts_smp_proc_unlock(c_p, c_p_locks); + erts_smp_port_lock(prt); + erts_smp_proc_lock(c_p, c_p_locks); + } +#endif + state = erts_atomic32_read_nob(&prt->state); + if (state & invalid_sflgs) { +#ifdef ERTS_SMP + erts_smp_port_unlock(prt); +#endif + return NULL; + } + + return prt; +} + +ERTS_GLB_INLINE void +erts_port_release(Port *prt) +{ + /* Only allowed to be called from managed threads */ + ERTS_SMP_LC_ASSERT(erts_thr_progress_is_managed_thread()); +#ifdef ERTS_SMP + erts_smp_port_unlock(prt); +#else + if (prt->cleanup) { + prt->cleanup = 0; + erts_port_cleanup(prt); + } +#endif +} + +#ifdef ERTS_SMP + +/* + * erts_thr_id2port_sflgs() and erts_thr_port_release() can + * be used by unmanaged threads in the SMP case. + */ +ERTS_GLB_INLINE Port * +erts_thr_id2port_sflgs(Eterm id, Uint32 invalid_sflgs) +{ + Port *prt; + ErtsThrPrgrDelayHandle dhndl; + + if (is_not_internal_port(id)) + return NULL; + + dhndl = erts_thr_progress_unmanaged_delay(); + + prt = (Port *) erts_ptab_pix2intptr_ddrb(&erts_port, + internal_port_index(id)); + + if (!prt || prt->common.id != id) { + erts_thr_progress_unmanaged_continue(dhndl); + prt = NULL; + } + else { + erts_aint32_t state; + if (dhndl != ERTS_THR_PRGR_DHANDLE_MANAGED) { + erts_port_inc_refc(prt); + erts_thr_progress_unmanaged_continue(dhndl); + } + + erts_mtx_lock(prt->lock); + state = erts_atomic32_read_nob(&prt->state); + if (state & invalid_sflgs) { + erts_mtx_unlock(prt->lock); + if (dhndl != ERTS_THR_PRGR_DHANDLE_MANAGED) + erts_port_dec_refc(prt); + prt = NULL; + } + } + + return prt; +} + +ERTS_GLB_INLINE void +erts_thr_port_release(Port *prt) +{ + erts_mtx_unlock(prt->lock); +#ifdef ERTS_SMP + if (!erts_thr_progress_is_managed_thread()) + erts_port_dec_refc(prt); +#endif +} + +#endif + +ERTS_GLB_INLINE Port* +erts_thr_drvport2port_raw(ErlDrvPort drvport, int lock_pdl) +{ +#if ERTS_ENABLE_LOCK_CHECK + int emu_thread = erts_lc_is_emu_thr(); +#endif + if (drvport == ERTS_INVALID_ERL_DRV_PORT) + return NULL; + else { + Port *prt = (Port *) drvport; + if (lock_pdl && prt->port_data_lock) + driver_pdl_lock(prt->port_data_lock); +#if ERTS_ENABLE_LOCK_CHECK + if (!ERTS_IS_CRASH_DUMPING) { + if (emu_thread) { + ERTS_SMP_LC_ASSERT(erts_lc_is_port_locked(prt)); + ERTS_LC_ASSERT(!prt->port_data_lock + || erts_lc_mtx_is_locked(&prt->port_data_lock->mtx)); + } + else { + ERTS_LC_ASSERT(prt->port_data_lock); + ERTS_LC_ASSERT(erts_lc_mtx_is_locked(&prt->port_data_lock->mtx)); + } + } +#endif + return prt; + } +} + +ERTS_GLB_INLINE Port* +erts_drvport2port_raw(ErlDrvPort drvport) +{ + ERTS_LC_ASSERT(erts_lc_is_emu_thr()); + if (drvport == ERTS_INVALID_ERL_DRV_PORT) + return NULL; + else { + Port *prt = (Port *) drvport; + ERTS_SMP_LC_ASSERT(erts_lc_is_port_locked(prt) + || ERTS_IS_CRASH_DUMPING); + return prt; + } +} + +ERTS_GLB_INLINE Port* +erts_drvport2port(ErlDrvPort drvport, erts_aint32_t *statep) +{ + Port *prt = erts_drvport2port_raw(drvport); + erts_aint32_t state; + if (!prt) + return NULL; + state = erts_atomic32_read_nob(&prt->state); + if (state & ERTS_PORT_SFLGS_INVALID_DRIVER_LOOKUP) + return NULL; + if (statep) + *statep = state; + return prt; +} + +ERTS_GLB_INLINE Port* +erts_drvportid2port(Eterm id) +{ + Port *prt; + erts_aint32_t state; + if (is_not_internal_port(id)) + return NULL; + prt = (Port *) erts_ptab_pix2intptr_nob(&erts_port, + internal_port_index(id)); + if (!prt) + return NULL; + ERTS_SMP_LC_ASSERT(erts_lc_is_port_locked(prt) + || ERTS_IS_CRASH_DUMPING); + if (prt->common.id != id) + return NULL; + state = erts_atomic32_read_nob(&prt->state); + if (state & ERTS_PORT_SFLGS_INVALID_DRIVER_LOOKUP) + return NULL; + return prt; +} + +ERTS_GLB_INLINE Eterm +erts_drvport2id(ErlDrvPort drvport) +{ + Port *prt = erts_drvport2port_raw(drvport); + if (!prt) + return am_undefined; + else + return prt->common.id; +} + +ERTS_GLB_INLINE Uint32 +erts_portid2status(Eterm id) +{ + Port *prt = erts_port_lookup_raw(id); + if (prt) + return (Uint32) erts_atomic32_read_acqb(&prt->state); + else + return ERTS_PORT_SFLG_INVALID; +} + +ERTS_GLB_INLINE int +erts_is_port_alive(Eterm id) +{ + return !(erts_portid2status(id) & (ERTS_PORT_SFLG_INVALID + | ERTS_PORT_SFLGS_DEAD)); +} + +ERTS_GLB_INLINE int +erts_is_valid_tracer_port(Eterm id) +{ + return !(erts_portid2status(id) & ERTS_PORT_SFLGS_INVALID_TRACER_LOOKUP); +} + +ERTS_GLB_INLINE int +erts_port_driver_callback_epilogue(Port *prt, erts_aint32_t *statep) +{ + int reds = 0; + erts_aint32_t state; + + ERTS_SMP_LC_ASSERT(erts_lc_is_port_locked(prt)); + + state = erts_atomic32_read_nob(&prt->state); + if ((state & ERTS_PORT_SFLG_CLOSING) && erts_is_port_ioq_empty(prt)) { + reds += ERTS_PORT_REDS_TERMINATE; + erts_terminate_port(prt); + state = erts_atomic32_read_nob(&prt->state); + ERTS_SMP_LC_ASSERT(erts_lc_is_port_locked(prt)); + } + +#ifdef ERTS_SMP + if (prt->xports) { + reds += erts_port_handle_xports(prt); + ERTS_SMP_LC_ASSERT(erts_lc_is_port_locked(prt)); + ASSERT(!prt->xports); + } +#endif + + if (statep) + *statep = state; + + return reds; +} + +#endif /* #if ERTS_GLB_INLINE_INCL_FUNC_DEF */ + +void erts_port_resume_procs(Port *); + +struct binary; + +#define ERTS_P2P_SIG_TYPE_BAD 0 +#define ERTS_P2P_SIG_TYPE_OUTPUT 1 +#define ERTS_P2P_SIG_TYPE_OUTPUTV 2 +#define ERTS_P2P_SIG_TYPE_CONNECT 3 +#define ERTS_P2P_SIG_TYPE_EXIT 4 +#define ERTS_P2P_SIG_TYPE_CONTROL 5 +#define ERTS_P2P_SIG_TYPE_CALL 6 +#define ERTS_P2P_SIG_TYPE_INFO 7 +#define ERTS_P2P_SIG_TYPE_LINK 8 +#define ERTS_P2P_SIG_TYPE_UNLINK 9 +#define ERTS_P2P_SIG_TYPE_SET_DATA 10 +#define ERTS_P2P_SIG_TYPE_GET_DATA 11 + +#define ERTS_P2P_SIG_TYPE_BITS 4 +#define ERTS_P2P_SIG_TYPE_MASK \ + ((1 << ERTS_P2P_SIG_TYPE_BITS) - 1) + +#define ERTS_P2P_SIG_DATA_FLG(N) \ + (1 << (ERTS_P2P_SIG_TYPE_BITS + (N))) +#define ERTS_P2P_SIG_DATA_FLG_BANG_OP ERTS_P2P_SIG_DATA_FLG(0) +#define ERTS_P2P_SIG_DATA_FLG_REPLY ERTS_P2P_SIG_DATA_FLG(1) +#define ERTS_P2P_SIG_DATA_FLG_NOSUSPEND ERTS_P2P_SIG_DATA_FLG(2) +#define ERTS_P2P_SIG_DATA_FLG_FORCE ERTS_P2P_SIG_DATA_FLG(3) +#define ERTS_P2P_SIG_DATA_FLG_BAD_OUTPUT ERTS_P2P_SIG_DATA_FLG(4) +#define ERTS_P2P_SIG_DATA_FLG_BROKEN_LINK ERTS_P2P_SIG_DATA_FLG(5) +#define ERTS_P2P_SIG_DATA_FLG_SCHED ERTS_P2P_SIG_DATA_FLG(6) + +struct ErtsProc2PortSigData_ { + int flags; + Eterm caller; + Uint32 ref[ERTS_MAX_REF_NUMBERS]; + union { + struct { + Eterm from; + ErlIOVec *evp; + ErlDrvBinary *cbinp; + } outputv; + struct { + Eterm from; + char *bufp; + ErlDrvSizeT size; + } output; + struct { + Eterm from; + Eterm connected; + } connect; + struct { + Eterm from; + Eterm reason; + ErlHeapFragment *bp; + } exit; + struct { + struct binary *binp; + unsigned int command; + char *bufp; + ErlDrvSizeT size; + } control; + struct { + unsigned int command; + char *bufp; + ErlDrvSizeT size; + } call; + struct { + Eterm item; + } info; + struct { + Eterm port; + Eterm to; + } link; + struct { + Eterm from; + } unlink; + struct { + ErlHeapFragment *bp; + Eterm data; + } set_data; + } u; +} ; + +ERTS_GLB_INLINE int +erts_proc2port_sig_is_command_op(ErtsProc2PortSigData *sigdp); +ERTS_GLB_INLINE ErlDrvSizeT +erts_proc2port_sig_command_data_size(ErtsProc2PortSigData *sigdp); + +#if ERTS_GLB_INLINE_INCL_FUNC_DEF + +ERTS_GLB_INLINE int +erts_proc2port_sig_is_command_op(ErtsProc2PortSigData *sigdp) +{ + switch (sigdp->flags & ERTS_P2P_SIG_TYPE_MASK) { + case ERTS_P2P_SIG_TYPE_OUTPUT: return !0; + case ERTS_P2P_SIG_TYPE_OUTPUTV: return !0; + default: return 0; + } +} + +ERTS_GLB_INLINE ErlDrvSizeT +erts_proc2port_sig_command_data_size(ErtsProc2PortSigData *sigdp) +{ + switch (sigdp->flags & ERTS_P2P_SIG_TYPE_MASK) { + case ERTS_P2P_SIG_TYPE_OUTPUT: return sigdp->u.output.size; + case ERTS_P2P_SIG_TYPE_OUTPUTV: return sigdp->u.outputv.evp->size; + default: return (ErlDrvSizeT) 0; + } +} + +#endif + +#define ERTS_PROC2PORT_SIG_EXEC 0 +#define ERTS_PROC2PORT_SIG_ABORT 1 +#define ERTS_PROC2PORT_SIG_ABORT_NOSUSPEND 2 +#define ERTS_PROC2PORT_SIG_ABORT_CLOSED 3 + +typedef int (*ErtsProc2PortSigCallback)(Port *, + erts_aint32_t, + int, + ErtsProc2PortSigData *); + +typedef enum { + ERTS_PORT_OP_BADARG, + ERTS_PORT_OP_CALLER_EXIT, + ERTS_PORT_OP_BUSY, + ERTS_PORT_OP_BUSY_SCHEDULED, + ERTS_PORT_OP_SCHEDULED, + ERTS_PORT_OP_DROPPED, + ERTS_PORT_OP_DONE +} ErtsPortOpResult; + +ErtsPortOpResult +erts_schedule_proc2port_signal(Process *, + Port *, + Eterm, + Eterm *, + ErtsProc2PortSigData *, + int, + ErtsProc2PortSigCallback); + +int erts_deliver_port_exit(Port *, Eterm, Eterm, int); + +/* + * Port signal flags + */ +#define ERTS_PORT_SIG_FLG_BANG_OP ERTS_P2P_SIG_DATA_FLG_BANG_OP +#define ERTS_PORT_SIG_FLG_NOSUSPEND ERTS_P2P_SIG_DATA_FLG_NOSUSPEND +#define ERTS_PORT_SIG_FLG_FORCE ERTS_P2P_SIG_DATA_FLG_FORCE +#define ERTS_PORT_SIG_FLG_BROKEN_LINK ERTS_P2P_SIG_DATA_FLG_BROKEN_LINK +#define ERTS_PORT_SIG_FLG_BAD_OUTPUT ERTS_P2P_SIG_DATA_FLG_BAD_OUTPUT +#define ERTS_PORT_SIG_FLG_FORCE_SCHED ERTS_P2P_SIG_DATA_FLG_SCHED +/* ERTS_PORT_SIG_FLG_FORCE_IMM_CALL only when crash dumping... */ +#define ERTS_PORT_SIG_FLG_FORCE_IMM_CALL ERTS_P2P_SIG_DATA_FLG_BAD_OUTPUT + +/* + * Port ! {Owner, {command, Data}} + * Port ! {Owner, {connect, NewOwner}} + * Port ! {Owner, close} + */ +ErtsPortOpResult erts_port_command(Process *, int, Port *, Eterm, Eterm *); + +/* + * Signals from processes to ports. + */ +ErtsPortOpResult erts_port_output(Process *, int, Port *, Eterm, Eterm, Eterm *); +ErtsPortOpResult erts_port_exit(Process *, int, Port *, Eterm, Eterm, Eterm *); +ErtsPortOpResult erts_port_connect(Process *, int, Port *, Eterm, Eterm, Eterm *); +ErtsPortOpResult erts_port_link(Process *, Port *, Eterm, Eterm *); +ErtsPortOpResult erts_port_unlink(Process *, Port *, Eterm, Eterm *); +ErtsPortOpResult erts_port_control(Process *, Port *, unsigned int, Eterm, Eterm *); +ErtsPortOpResult erts_port_call(Process *, Port *, unsigned int, Eterm, Eterm *); +ErtsPortOpResult erts_port_info(Process *, Port *, Eterm, Eterm *); +ErtsPortOpResult erts_port_set_data(Process *, Port *, Eterm, Eterm *); +ErtsPortOpResult erts_port_get_data(Process *, Port *, Eterm *); + +#endif diff --git a/erts/emulator/beam/erl_port_task.c b/erts/emulator/beam/erl_port_task.c index b6bc59a1c3..8ceadcdb8c 100644 --- a/erts/emulator/beam/erl_port_task.c +++ b/erts/emulator/beam/erl_port_task.c @@ -33,36 +33,29 @@ #include "erl_port_task.h" #include "dist.h" #include "dtrace-wrapper.h" +#include <stdarg.h> #if defined(DEBUG) && 0 -#define HARD_DEBUG +#define ERTS_HARD_DEBUG_TASK_QUEUES +#else +#undef ERTS_HARD_DEBUG_TASK_QUEUES #endif -/* - * Costs in reductions for some port operations. - */ -#define ERTS_PORT_REDS_EXECUTE 0 -#define ERTS_PORT_REDS_FREE 50 -#define ERTS_PORT_REDS_TIMEOUT 200 -#define ERTS_PORT_REDS_INPUT 200 -#define ERTS_PORT_REDS_OUTPUT 200 -#define ERTS_PORT_REDS_EVENT 200 -#define ERTS_PORT_REDS_TERMINATE 100 - - -#define ERTS_PORT_TASK_INVALID_PORT(P, ID) \ - ((erts_port_status_get((P)) & ERTS_PORT_SFLGS_DEAD) || (P)->id != (ID)) - -#define ERTS_PORT_IS_IN_RUNQ(RQ, P) \ - ((P)->sched.next || (P)->sched.prev || (RQ)->ports.start == (P)) +#ifdef ERTS_HARD_DEBUG_TASK_QUEUES +static void chk_task_queues(Port *pp, ErtsPortTask *execq, int processing_busy_queue); +#define ERTS_PT_DBG_CHK_TASK_QS(PP, EQ, PBQ) \ + chk_task_queues((PP), (EQ), (PBQ)) +#else +#define ERTS_PT_DBG_CHK_TASK_QS(PP, EQ, PBQ) +#endif #ifdef USE_VM_PROBES #define DTRACE_DRIVER(PROBE_NAME, PP) \ - if (DTRACE_ENABLED(driver_ready_input)) { \ + if (DTRACE_ENABLED(PROBE_NAME)) { \ DTRACE_CHARBUF(process_str, DTRACE_TERM_BUF_SIZE); \ DTRACE_CHARBUF(port_str, DTRACE_TERM_BUF_SIZE); \ \ - dtrace_pid_str(PP->connected, process_str); \ + dtrace_pid_str(ERTS_PORT_GET_CONNECTED(PP), process_str); \ dtrace_port_str(PP, port_str); \ DTRACE3(PROBE_NAME, process_str, port_str, PP->name); \ } @@ -72,83 +65,768 @@ erts_smp_atomic_t erts_port_task_outstanding_io_tasks; -struct ErtsPortTaskQueue_ { - ErtsPortTask *first; - ErtsPortTask *last; - Port *port; -}; +#define ERTS_PT_STATE_SCHEDULED 0 +#define ERTS_PT_STATE_ABORTED 1 +#define ERTS_PT_STATE_EXECUTING 2 + +typedef union { + struct { /* I/O tasks */ + ErlDrvEvent event; + ErlDrvEventData event_data; + } io; + struct { + ErtsProc2PortSigCallback callback; + ErtsProc2PortSigData data; + } psig; +} ErtsPortTaskTypeData; struct ErtsPortTask_ { - ErtsPortTask *prev; - ErtsPortTask *next; - ErtsPortTaskQueue *queue; - ErtsPortTaskHandle *handle; + erts_smp_atomic32_t state; ErtsPortTaskType type; - ErlDrvEvent event; - ErlDrvEventData event_data; + union { + struct { + ErtsPortTask *next; + ErtsPortTaskHandle *handle; + int flags; + Uint32 ref[ERTS_MAX_REF_NUMBERS]; + ErtsPortTaskTypeData td; + } alive; + ErtsThrPrgrLaterOp release; + } u; }; -#ifdef HARD_DEBUG -#define ERTS_PT_CHK_PORTQ(RQ) check_port_queue((RQ), NULL, 0) -#define ERTS_PT_CHK_PRES_PORTQ(RQ, PP) check_port_queue((RQ), (PP), -1) -#define ERTS_PT_CHK_IN_PORTQ(RQ, PP) check_port_queue((RQ), (PP), 1) -#define ERTS_PT_CHK_NOT_IN_PORTQ(RQ, PP) check_port_queue((RQ), (PP), 0) -#define ERTS_PT_CHK_TASKQ(Q) check_task_queue((Q), NULL, 0) -#define ERTS_PT_CHK_IN_TASKQ(Q, T) check_task_queue((Q), (T), 1) -#define ERTS_PT_CHK_NOT_IN_TASKQ(Q, T) check_task_queue((Q), (T), 0) -static void -check_port_queue(Port *chk_pp, int inq); -static void -check_task_queue(ErtsPortTaskQueue *ptqp, - ErtsPortTask *chk_ptp, - int inq); -#else -#define ERTS_PT_CHK_PORTQ(RQ) -#define ERTS_PT_CHK_PRES_PORTQ(RQ, PP) -#define ERTS_PT_CHK_IN_PORTQ(RQ, PP) -#define ERTS_PT_CHK_NOT_IN_PORTQ(RQ, PP) -#define ERTS_PT_CHK_TASKQ(Q) -#define ERTS_PT_CHK_IN_TASKQ(Q, T) -#define ERTS_PT_CHK_NOT_IN_TASKQ(Q, T) +struct ErtsPortTaskHandleList_ { + ErtsPortTaskHandle handle; + union { + ErtsPortTaskHandleList *next; +#ifdef ERTS_SMP + ErtsThrPrgrLaterOp release; #endif + } u; +}; + +typedef struct ErtsPortTaskBusyCaller_ ErtsPortTaskBusyCaller; +struct ErtsPortTaskBusyCaller_ { + ErtsPortTaskBusyCaller *next; + Eterm caller; + SWord count; + ErtsPortTask *last; +}; + +#define ERTS_PORT_TASK_BUSY_CALLER_TABLE_BUCKETS 17 +struct ErtsPortTaskBusyCallerTable_ { + ErtsPortTaskBusyCaller *bucket[ERTS_PORT_TASK_BUSY_CALLER_TABLE_BUCKETS]; + ErtsPortTaskBusyCaller pre_alloc_busy_caller; +}; + -static void handle_remaining_tasks(ErtsRunQueue *runq, Port *pp); +static void begin_port_cleanup(Port *pp, + ErtsPortTask **execq, + int *processing_busy_q_p); ERTS_SCHED_PREF_QUICK_ALLOC_IMPL(port_task, ErtsPortTask, - 200, + 1000, ERTS_ALC_T_PORT_TASK) -ERTS_SCHED_PREF_QUICK_ALLOC_IMPL(port_taskq, - ErtsPortTaskQueue, + +ERTS_SCHED_PREF_QUICK_ALLOC_IMPL(busy_caller_table, + ErtsPortTaskBusyCallerTable, 50, - ERTS_ALC_T_PORT_TASKQ) + ERTS_ALC_T_BUSY_CALLER_TAB) + +#ifdef ERTS_SMP +static void +call_port_task_free(void *vptp) +{ + port_task_free((ErtsPortTask *) vptp); +} +#endif + +static ERTS_INLINE void +schedule_port_task_free(ErtsPortTask *ptp) +{ +#ifdef ERTS_SMP + erts_schedule_thr_prgr_later_op(call_port_task_free, + (void *) ptp, + &ptp->u.release); +#else + port_task_free(ptp); +#endif +} + +static ERTS_INLINE ErtsPortTask * +p2p_sig_data_to_task(ErtsProc2PortSigData *sigdp) +{ + ErtsPortTask *ptp; + char *ptr = (char *) sigdp; + ptr -= offsetof(ErtsPortTask, u.alive.td.psig.data); + ptp = (ErtsPortTask *) ptr; + ASSERT(ptp->type == ERTS_PORT_TASK_PROC_SIG); + return ptp; +} + +ErtsProc2PortSigData * +erts_port_task_alloc_p2p_sig_data(void) +{ + ErtsPortTask *ptp = port_task_alloc(); + + ptp->type = ERTS_PORT_TASK_PROC_SIG; + ptp->u.alive.flags = ERTS_PT_FLG_SIG_DEP; + erts_smp_atomic32_init_nob(&ptp->state, ERTS_PT_STATE_SCHEDULED); + + ASSERT(ptp == p2p_sig_data_to_task(&ptp->u.alive.td.psig.data)); + + return &ptp->u.alive.td.psig.data; +} + +static ERTS_INLINE Eterm +task_caller(ErtsPortTask *ptp) +{ + Eterm caller; + + ASSERT(ptp->type == ERTS_PORT_TASK_PROC_SIG); + + caller = ptp->u.alive.td.psig.data.caller; + + ASSERT(is_internal_pid(caller) || is_internal_port(caller)); + + return caller; +} + +/* + * Busy queue management + */ + +static ERTS_INLINE int +caller2bix(Eterm caller) +{ + ASSERT(is_internal_pid(caller) || is_internal_port(caller)); + return (int) (_GET_PID_DATA(caller) % ERTS_PORT_TASK_BUSY_CALLER_TABLE_BUCKETS); +} + + +static void +popped_from_busy_queue(Port *pp, ErtsPortTask *ptp, int last) +{ + ErtsPortTaskBusyCaller **prev_bcpp = NULL, *bcp; + ErtsPortTaskBusyCallerTable *tabp = pp->sched.taskq.local.busy.table; + Eterm caller = task_caller(ptp); + int bix = caller2bix(caller); + + ASSERT(is_internal_pid(caller)); + + ASSERT(tabp); + bcp = tabp->bucket[bix]; + prev_bcpp = &tabp->bucket[bix]; + ASSERT(bcp); + while (bcp->caller != caller) { + prev_bcpp = &bcp->next; + bcp = bcp->next; + ASSERT(bcp); + } + ASSERT(bcp->count > 0); + if (--bcp->count != 0) { + ASSERT(!last); + } + else { + *prev_bcpp = bcp->next; + if (bcp == &tabp->pre_alloc_busy_caller) + bcp->caller = am_undefined; + else + erts_free(ERTS_ALC_T_BUSY_CALLER, bcp); + if (last) { +#ifdef DEBUG + erts_aint32_t flags = +#endif + erts_smp_atomic32_read_band_nob( + &pp->sched.flags, + ~ERTS_PTS_FLG_HAVE_BUSY_TASKS); + ASSERT(flags & ERTS_PTS_FLG_HAVE_BUSY_TASKS); +#ifdef DEBUG + for (bix = 0; bix < ERTS_PORT_TASK_BUSY_CALLER_TABLE_BUCKETS; bix++) { + ASSERT(!tabp->bucket[bix]); + } +#endif + busy_caller_table_free(tabp); + pp->sched.taskq.local.busy.first = NULL; + pp->sched.taskq.local.busy.last = NULL; + pp->sched.taskq.local.busy.table = NULL; + } + } +} + +static void +busy_wait_move_to_busy_queue(Port *pp, ErtsPortTask *ptp) +{ + ErtsPortTaskBusyCallerTable *tabp = pp->sched.taskq.local.busy.table; + Eterm caller = task_caller(ptp); + ErtsPortTaskBusyCaller *bcp; + int bix; + + ASSERT(is_internal_pid(caller)); + /* + * Port is busy and this task type needs to wait until not busy. + */ + + ASSERT(ptp->u.alive.flags & ERTS_PT_FLG_WAIT_BUSY); + + ptp->u.alive.next = NULL; + if (pp->sched.taskq.local.busy.last) { + ASSERT(pp->sched.taskq.local.busy.first); + pp->sched.taskq.local.busy.last->u.alive.next = ptp; + } + else { + int i; + erts_aint32_t flags; + + pp->sched.taskq.local.busy.first = ptp; + flags = erts_smp_atomic32_read_bor_nob(&pp->sched.flags, + ERTS_PTS_FLG_HAVE_BUSY_TASKS); + ASSERT(!(flags & ERTS_PTS_FLG_HAVE_BUSY_TASKS)); + + ASSERT(!tabp); + + tabp = busy_caller_table_alloc(); + pp->sched.taskq.local.busy.table = tabp; + for (i = 0; i < ERTS_PORT_TASK_BUSY_CALLER_TABLE_BUCKETS; i++) + tabp->bucket[i] = NULL; + tabp->pre_alloc_busy_caller.caller = am_undefined; + } + pp->sched.taskq.local.busy.last = ptp; + + bix = caller2bix(caller); + ASSERT(tabp); + bcp = tabp->bucket[bix]; + + while (bcp && bcp->caller != caller) + bcp = bcp->next; + + if (bcp) + bcp->count++; + else { + if (tabp->pre_alloc_busy_caller.caller == am_undefined) + bcp = &tabp->pre_alloc_busy_caller; + else + bcp = erts_alloc(ERTS_ALC_T_BUSY_CALLER, + sizeof(ErtsPortTaskBusyCaller)); + bcp->caller = caller; + bcp->count = 1; + bcp->next = tabp->bucket[bix]; + tabp->bucket[bix] = bcp; + } + + bcp->last = ptp; +} + +static ERTS_INLINE int +check_sig_dep_move_to_busy_queue(Port *pp, ErtsPortTask *ptp) +{ + ErtsPortTaskBusyCallerTable *tabp = pp->sched.taskq.local.busy.table; + ErtsPortTask *last_ptp; + ErtsPortTaskBusyCaller *bcp; + int bix; + Eterm caller; + + ASSERT(ptp->u.alive.flags & ERTS_PT_FLG_SIG_DEP); + ASSERT(pp->sched.taskq.local.busy.last); + ASSERT(tabp); + + + /* + * We are either not busy, or the task does not imply wait on busy port. + * However, due to the signaling order requirements the task might depend + * on other tasks in the busy queue. + */ + + caller = task_caller(ptp); + bix = caller2bix(caller); + bcp = tabp->bucket[bix]; + while (bcp && bcp->caller != caller) + bcp = bcp->next; + + if (!bcp) + return 0; + + /* + * There are other tasks that we depend on in the busy queue; + * move into busy queue. + */ + + bcp->count++; + last_ptp = bcp->last; + ptp->u.alive.next = last_ptp->u.alive.next; + if (!ptp->u.alive.next) { + ASSERT(pp->sched.taskq.local.busy.last == last_ptp); + pp->sched.taskq.local.busy.last = ptp; + } + last_ptp->u.alive.next = ptp; + bcp->last = ptp; + + return 1; +} + +static void +no_sig_dep_move_from_busyq(Port *pp) +{ + ErtsPortTaskBusyCallerTable *tabp = pp->sched.taskq.local.busy.table; + ErtsPortTask *first_ptp, *last_ptp, *ptp; + ErtsPortTaskBusyCaller **prev_bcpp = NULL, *bcp = NULL; + + /* + * Move tasks at the head of the busy queue that no longer + * have any dependencies to busy wait tasks into the ordinary + * queue. + */ + + first_ptp = ptp = pp->sched.taskq.local.busy.first; + + ASSERT(ptp && !(ptp->u.alive.flags & ERTS_PT_FLG_WAIT_BUSY)); + ASSERT(tabp); + + do { + Eterm caller = task_caller(ptp); + + if (!bcp || bcp->caller != caller) { + int bix = caller2bix(caller); + + prev_bcpp = &tabp->bucket[bix]; + bcp = tabp->bucket[bix]; + ASSERT(bcp); + while (bcp->caller != caller) { + ASSERT(bcp); + prev_bcpp = &bcp->next; + bcp = bcp->next; + } + } + + ASSERT(bcp->caller == caller); + ASSERT(bcp->count > 0); + + if (--bcp->count == 0) { + *prev_bcpp = bcp->next; + if (bcp == &tabp->pre_alloc_busy_caller) + bcp->caller = am_undefined; + else + erts_free(ERTS_ALC_T_BUSY_CALLER, bcp); + } + + last_ptp = ptp; + ptp = ptp->u.alive.next; + } while (ptp && !(ptp->u.alive.flags & ERTS_PT_FLG_WAIT_BUSY)); + + pp->sched.taskq.local.busy.first = last_ptp->u.alive.next; + if (!pp->sched.taskq.local.busy.first) { +#ifdef DEBUG + int bix; + erts_aint32_t flags = +#endif + erts_smp_atomic32_read_band_nob( + &pp->sched.flags, + ~ERTS_PTS_FLG_HAVE_BUSY_TASKS); + ASSERT(flags & ERTS_PTS_FLG_HAVE_BUSY_TASKS); +#ifdef DEBUG + for (bix = 0; bix < ERTS_PORT_TASK_BUSY_CALLER_TABLE_BUCKETS; bix++) { + ASSERT(!tabp->bucket[bix]); + } +#endif + busy_caller_table_free(tabp); + pp->sched.taskq.local.busy.last = NULL; + pp->sched.taskq.local.busy.table = NULL; + } + last_ptp->u.alive.next = pp->sched.taskq.local.first; + pp->sched.taskq.local.first = first_ptp; +} + +#ifdef ERTS_HARD_DEBUG_TASK_QUEUES + +static void +chk_task_queues(Port *pp, ErtsPortTask *execq, int processing_busy_queue) +{ + Sint tot_count, tot_table_count; + int bix; + ErtsPortTask *ptp, *last; + ErtsPortTask *first = processing_busy_queue ? execq : pp->sched.taskq.local.busy.first; + ErtsPortTask *nb_task_queue = processing_busy_queue ? pp->sched.taskq.local.first : execq; + ErtsPortTaskBusyCallerTable *tabp = pp->sched.taskq.local.busy.table; + ErtsPortTaskBusyCaller *bcp; + + if (!first) { + ASSERT(!tabp); + ASSERT(!pp->sched.taskq.local.busy.last); + ASSERT(!(erts_smp_atomic32_read_nob(&pp->sched.flags) & ERTS_PTS_FLG_HAVE_BUSY_TASKS)); + return; + } + + ASSERT(erts_smp_atomic32_read_nob(&pp->sched.flags) & ERTS_PTS_FLG_HAVE_BUSY_TASKS); + ASSERT(tabp); + + tot_count = 0; + ptp = first; + while (ptp) { + Sint count = 0; + Eterm caller = task_caller(ptp); + int bix = caller2bix(caller); + for (bcp = tabp->bucket[bix]; bcp; bcp = bcp->next) + if (bcp->caller == caller) + break; + ASSERT(bcp && bcp->caller == caller); + + ASSERT(bcp->last); + while (1) { + ErtsPortTask *ptp2; + + ASSERT(caller == task_caller(ptp)); + count++; + tot_count++; + last = ptp; + + for (ptp2 = nb_task_queue; ptp2; ptp2 = ptp2->u.alive.next) { + ASSERT(ptp != ptp2); + } + + if (ptp == bcp->last) + break; + ptp = ptp->u.alive.next; + } + + ASSERT(count == bcp->count); + ptp = ptp->u.alive.next; + } + + tot_table_count = 0; + for (bix = 0; bix < ERTS_PORT_TASK_BUSY_CALLER_TABLE_BUCKETS; bix++) { + for (bcp = tabp->bucket[bix]; bcp; bcp = bcp->next) + tot_table_count += bcp->count; + } + + ASSERT(tot_count == tot_table_count); + + ASSERT(last == pp->sched.taskq.local.busy.last); +} + +#endif /* ERTS_HARD_DEBUG_TASK_QUEUES */ /* * Task handle manipulation. */ +static ERTS_INLINE void +reset_port_task_handle(ErtsPortTaskHandle *pthp) +{ + erts_smp_atomic_set_relb(pthp, (erts_aint_t) NULL); +} + static ERTS_INLINE ErtsPortTask * handle2task(ErtsPortTaskHandle *pthp) { - return (ErtsPortTask *) erts_smp_atomic_read_nob(pthp); + return (ErtsPortTask *) erts_smp_atomic_read_acqb(pthp); } static ERTS_INLINE void reset_handle(ErtsPortTask *ptp) { - if (ptp->handle) { - ASSERT(ptp == handle2task(ptp->handle)); - erts_smp_atomic_set_nob(ptp->handle, (erts_aint_t) NULL); + if (ptp->u.alive.handle) { + ASSERT(ptp == handle2task(ptp->u.alive.handle)); + reset_port_task_handle(ptp->u.alive.handle); } } static ERTS_INLINE void set_handle(ErtsPortTask *ptp, ErtsPortTaskHandle *pthp) { - ptp->handle = pthp; + ptp->u.alive.handle = pthp; if (pthp) { - erts_smp_atomic_set_nob(pthp, (erts_aint_t) ptp); - ASSERT(ptp == handle2task(ptp->handle)); + erts_smp_atomic_set_relb(pthp, (erts_aint_t) ptp); + ASSERT(ptp == handle2task(ptp->u.alive.handle)); + } +} + + +/* + * Busy port queue management + */ + +static erts_aint32_t +check_unset_busy_port_q(Port *pp, + erts_aint32_t flags, + ErtsPortTaskBusyPortQ *bpq) +{ + ErlDrvSizeT qsize, low; + int resume_procs = 0; + + ASSERT(bpq); + ERTS_SMP_LC_ASSERT(erts_lc_is_port_locked(pp)); + + erts_port_task_sched_lock(&pp->sched); + qsize = (ErlDrvSizeT) erts_smp_atomic_read_nob(&bpq->size); + low = (ErlDrvSizeT) erts_smp_atomic_read_nob(&bpq->low); + if (qsize < low) { + erts_aint32_t mask = ~(ERTS_PTS_FLG_CHK_UNSET_BUSY_PORT_Q + | ERTS_PTS_FLG_BUSY_PORT_Q); + flags = erts_smp_atomic32_read_band_relb(&pp->sched.flags, mask); + if ((flags & ERTS_PTS_FLGS_BUSY) == ERTS_PTS_FLG_BUSY_PORT_Q) + resume_procs = 1; + } + else if (flags & ERTS_PTS_FLG_CHK_UNSET_BUSY_PORT_Q) { + flags = erts_smp_atomic32_read_band_relb(&pp->sched.flags, + ~ERTS_PTS_FLG_CHK_UNSET_BUSY_PORT_Q); + flags &= ~ERTS_PTS_FLG_CHK_UNSET_BUSY_PORT_Q; + } + erts_port_task_sched_unlock(&pp->sched); + if (resume_procs) + erts_port_resume_procs(pp); + + return flags; +} + +static ERTS_INLINE void +aborted_proc2port_data(Port *pp, ErlDrvSizeT size) +{ + ErtsPortTaskBusyPortQ *bpq; + erts_aint32_t flags; + ErlDrvSizeT qsz; + + ASSERT(pp->sched.taskq.bpq); + + if (size == 0) + return; + + bpq = pp->sched.taskq.bpq; + + qsz = (ErlDrvSizeT) erts_smp_atomic_add_read_acqb(&bpq->size, + (erts_aint_t) -size); + ASSERT(qsz + size > qsz); + flags = erts_smp_atomic32_read_nob(&pp->sched.flags); + ASSERT(pp->sched.taskq.bpq); + if ((flags & (ERTS_PTS_FLG_CHK_UNSET_BUSY_PORT_Q + | ERTS_PTS_FLG_BUSY_PORT_Q)) != ERTS_PTS_FLG_BUSY_PORT_Q) + return; + if (qsz < (ErlDrvSizeT) erts_smp_atomic_read_nob(&bpq->low)) + erts_smp_atomic32_read_bor_nob(&pp->sched.flags, + ERTS_PTS_FLG_CHK_UNSET_BUSY_PORT_Q); +} + +static ERTS_INLINE void +dequeued_proc2port_data(Port *pp, ErlDrvSizeT size) +{ + ErtsPortTaskBusyPortQ *bpq; + erts_aint32_t flags; + ErlDrvSizeT qsz; + + ASSERT(pp->sched.taskq.bpq); + + if (size == 0) + return; + + bpq = pp->sched.taskq.bpq; + + qsz = (ErlDrvSizeT) erts_smp_atomic_add_read_acqb(&bpq->size, + (erts_aint_t) -size); + ASSERT(qsz + size > qsz); + flags = erts_smp_atomic32_read_nob(&pp->sched.flags); + if (!(flags & ERTS_PTS_FLG_BUSY_PORT_Q)) + return; + if (qsz < (ErlDrvSizeT) erts_smp_atomic_read_acqb(&bpq->low)) + check_unset_busy_port_q(pp, flags, bpq); +} + +static ERTS_INLINE erts_aint32_t +enqueue_proc2port_data(Port *pp, + ErtsProc2PortSigData *sigdp, + erts_aint32_t flags) +{ + ErtsPortTaskBusyPortQ *bpq = pp->sched.taskq.bpq; + if (sigdp && bpq) { + ErlDrvSizeT size = erts_proc2port_sig_command_data_size(sigdp); + if (size) { + erts_aint_t asize = erts_smp_atomic_add_read_acqb(&bpq->size, + (erts_aint_t) size); + ErlDrvSizeT qsz = (ErlDrvSizeT) asize; + + ASSERT(qsz - size < qsz); + + if (!(flags & ERTS_PTS_FLG_BUSY_PORT_Q) && qsz > bpq->high) { + flags = erts_smp_atomic32_read_bor_acqb(&pp->sched.flags, + ERTS_PTS_FLG_BUSY_PORT_Q); + flags |= ERTS_PTS_FLG_BUSY_PORT_Q; + qsz = (ErlDrvSizeT) erts_smp_atomic_read_acqb(&bpq->size); + if (qsz < (ErlDrvSizeT) erts_smp_atomic_read_nob(&bpq->low)) { + flags = (erts_smp_atomic32_read_bor_relb( + &pp->sched.flags, + ERTS_PTS_FLG_CHK_UNSET_BUSY_PORT_Q)); + flags |= ERTS_PTS_FLG_CHK_UNSET_BUSY_PORT_Q; + } + } + ASSERT(!(flags & ERTS_PTS_FLG_EXIT)); + } + } + return flags; +} + +/* + * erl_drv_busy_msgq_limits() is called by drivers either reading or + * writing the limits. + * + * A limit of zero is interpreted as a read only request (using a + * limit of zero would not be useful). Other values are interpreted + * as a write-read request. + */ + +void +erl_drv_busy_msgq_limits(ErlDrvPort dport, ErlDrvSizeT *lowp, ErlDrvSizeT *highp) +{ + Port *pp = erts_drvport2port(dport, NULL); + ErtsPortTaskBusyPortQ *bpq = pp->sched.taskq.bpq; + int written = 0, resume_procs = 0; + ErlDrvSizeT low, high; + + if (!pp || !bpq) { + if (lowp) + *lowp = ERL_DRV_BUSY_MSGQ_DISABLED; + if (highp) + *highp = ERL_DRV_BUSY_MSGQ_DISABLED; + return; + } + + low = lowp ? *lowp : 0; + high = highp ? *highp : 0; + + erts_port_task_sched_lock(&pp->sched); + + if (low == ERL_DRV_BUSY_MSGQ_DISABLED + || high == ERL_DRV_BUSY_MSGQ_DISABLED) { + /* Disable busy msgq feature */ + erts_aint32_t flags; + pp->sched.taskq.bpq = NULL; + flags = ~(ERTS_PTS_FLG_BUSY_PORT_Q|ERTS_PTS_FLG_CHK_UNSET_BUSY_PORT_Q); + flags = erts_smp_atomic32_read_band_acqb(&pp->sched.flags, flags); + if ((flags & ERTS_PTS_FLGS_BUSY) == ERTS_PTS_FLG_BUSY_PORT_Q) + resume_procs = 1; + } + else { + + if (!low) + low = (ErlDrvSizeT) erts_smp_atomic_read_nob(&bpq->low); + else { + if (bpq->high < low) + bpq->high = low; + erts_smp_atomic_set_relb(&bpq->low, (erts_aint_t) low); + written = 1; + } + + if (!high) + high = bpq->high; + else { + if (low > high) { + low = high; + erts_smp_atomic_set_relb(&bpq->low, (erts_aint_t) low); + } + bpq->high = high; + written = 1; + } + + if (written) { + ErlDrvSizeT size = (ErlDrvSizeT) erts_smp_atomic_read_nob(&bpq->size); + if (size > high) + erts_smp_atomic32_read_bor_relb(&pp->sched.flags, + ERTS_PTS_FLG_BUSY_PORT_Q); + else if (size < low) + erts_smp_atomic32_read_bor_relb(&pp->sched.flags, + ERTS_PTS_FLG_CHK_UNSET_BUSY_PORT_Q); + } + } + + erts_port_task_sched_unlock(&pp->sched); + + if (resume_procs) + erts_port_resume_procs(pp); + if (lowp) + *lowp = low; + if (highp) + *highp = high; +} + +/* + * No-suspend handles. + */ + +#ifdef ERTS_SMP +static void +free_port_task_handle_list(void *vpthlp) +{ + erts_free(ERTS_ALC_T_PT_HNDL_LIST, vpthlp); +} +#endif + +static void +schedule_port_task_handle_list_free(ErtsPortTaskHandleList *pthlp) +{ +#ifdef ERTS_SMP + erts_schedule_thr_prgr_later_op(free_port_task_handle_list, + (void *) pthlp, + &pthlp->u.release); +#else + erts_free(ERTS_ALC_T_PT_HNDL_LIST, pthlp); +#endif +} + +static ERTS_INLINE void +abort_nosuspend_task(Port *pp, + ErtsPortTaskType type, + ErtsPortTaskTypeData *tdp) +{ + + ASSERT(type == ERTS_PORT_TASK_PROC_SIG); + + if (!pp->sched.taskq.bpq) + tdp->psig.callback(NULL, + ERTS_PORT_SFLG_INVALID, + ERTS_PROC2PORT_SIG_ABORT_NOSUSPEND, + &tdp->psig.data); + else { + ErlDrvSizeT size = erts_proc2port_sig_command_data_size(&tdp->psig.data); + tdp->psig.callback(NULL, + ERTS_PORT_SFLG_INVALID, + ERTS_PROC2PORT_SIG_ABORT_NOSUSPEND, + &tdp->psig.data); + aborted_proc2port_data(pp, size); + } +} + +static ErtsPortTaskHandleList * +get_free_nosuspend_handles(Port *pp) +{ + ErtsPortTaskHandleList *nshp, *last_nshp = NULL; + + ERTS_SMP_LC_ASSERT(erts_port_task_sched_lock_is_locked(&pp->sched)); + + nshp = pp->sched.taskq.local.busy.nosuspend; + + while (nshp && !erts_port_task_is_scheduled(&nshp->handle)) { + last_nshp = nshp; + nshp = nshp->u.next; + } + + if (!last_nshp) + nshp = NULL; + else { + nshp = pp->sched.taskq.local.busy.nosuspend; + pp->sched.taskq.local.busy.nosuspend = last_nshp->u.next; + last_nshp->u.next = NULL; + if (!pp->sched.taskq.local.busy.nosuspend) + erts_smp_atomic32_read_band_nob(&pp->sched.flags, + ~ERTS_PTS_FLG_HAVE_NS_TASKS); + } + return nshp; +} + +static void +free_nosuspend_handles(ErtsPortTaskHandleList *free_nshp) +{ + while (free_nshp) { + ErtsPortTaskHandleList *nshp = free_nshp; + free_nshp = free_nshp->u.next; + schedule_port_task_handle_list_free(nshp); } } @@ -161,7 +839,6 @@ enqueue_port(ErtsRunQueue *runq, Port *pp) { ERTS_SMP_LC_ASSERT(erts_smp_lc_runq_is_locked(runq)); pp->sched.next = NULL; - pp->sched.in_runq = 1; if (runq->ports.end) { ASSERT(runq->ports.start); runq->ports.end->sched.next = pp; @@ -199,285 +876,423 @@ pop_port(ErtsRunQueue *runq) return pp; } +/* + * Task queue operations + */ -#ifdef HARD_DEBUG +static ERTS_INLINE int +enqueue_task(Port *pp, + ErtsPortTask *ptp, + ErtsProc2PortSigData *sigdp, + ErtsPortTaskHandleList *ns_pthlp, + erts_aint32_t *flagsp) -static void -check_port_queue(ErtsRunQueue *runq, Port *chk_pp, int inq) { - Port *pp; - Port *last_pp; - Port *first_pp = runq->ports.start; - int no_forward = 0, no_backward = 0; - int found_forward = 0, found_backward = 0; - if (!first_pp) { - ASSERT(!runq->ports.end); - } + int res; + erts_aint32_t fail_flags = ERTS_PTS_FLG_EXIT; + erts_aint32_t flags; + ptp->u.alive.next = NULL; + if (ns_pthlp) + fail_flags |= ERTS_PTS_FLG_BUSY_PORT; + erts_port_task_sched_lock(&pp->sched); + flags = erts_smp_atomic32_read_nob(&pp->sched.flags); + if (flags & fail_flags) + res = 0; else { - ASSERT(!first_pp->sched.prev); - for (pp = first_pp; pp; pp = pp->sched.next) { - ASSERT(pp->sched.taskq); - if (pp->sched.taskq->first) - no_forward++; - if (chk_pp == pp) - found_forward = 1; - if (!pp->sched.prev) { - ASSERT(first_pp == pp); - } - if (!pp->sched.next) { - ASSERT(runq->ports.end == pp); - last_pp = pp; - } - } - for (pp = last_pp; pp; pp = pp->sched.prev) { - ASSERT(pp->sched.taskq); - if (pp->sched.taskq->last) - no_backward++; - if (chk_pp == pp) - found_backward = 1; - if (!pp->sched.prev) { - ASSERT(first_pp == pp); - } - if (!pp->sched.next) { - ASSERT(runq->ports.end == pp); - } - check_task_queue(pp->sched.taskq, NULL, 0); - } - ASSERT(no_forward == no_backward); - } - ASSERT(no_forward == RUNQ_READ_LEN(&runq->ports.info.len)); - if (chk_pp) { - if (chk_pp->sched.taskq || chk_pp->sched.exe_taskq) { - ASSERT(chk_pp->sched.taskq != chk_pp->sched.exe_taskq); + if (ns_pthlp) { + ns_pthlp->u.next = pp->sched.taskq.local.busy.nosuspend; + pp->sched.taskq.local.busy.nosuspend = ns_pthlp; } - ASSERT(!chk_pp->sched.taskq || chk_pp->sched.taskq->first); - if (inq < 0) - inq = chk_pp->sched.taskq && !chk_pp->sched.exe_taskq; - if (inq) { - ASSERT(found_forward && found_backward); + if (pp->sched.taskq.in.last) { + ASSERT(pp->sched.taskq.in.first); + ASSERT(!pp->sched.taskq.in.last->u.alive.next); + + pp->sched.taskq.in.last->u.alive.next = ptp; } else { - ASSERT(!found_forward && !found_backward); - } - } -} - -#endif - -/* - * Task queue operations - */ + ASSERT(!pp->sched.taskq.in.first); -static ERTS_INLINE ErtsPortTaskQueue * -port_taskq_init(ErtsPortTaskQueue *ptqp, Port *pp) -{ - if (ptqp) { - ptqp->first = NULL; - ptqp->last = NULL; - ptqp->port = pp; + pp->sched.taskq.in.first = ptp; + } + pp->sched.taskq.in.last = ptp; + flags = enqueue_proc2port_data(pp, sigdp, flags); + res = 1; } - return ptqp; + erts_port_task_sched_unlock(&pp->sched); + *flagsp = flags; + return res; } static ERTS_INLINE void -enqueue_task(ErtsPortTaskQueue *ptqp, ErtsPortTask *ptp) +prepare_exec(Port *pp, ErtsPortTask **execqp, int *processing_busy_q_p) { - ERTS_PT_CHK_NOT_IN_TASKQ(ptqp, ptp); - ptp->next = NULL; - ptp->prev = ptqp->last; - ptp->queue = ptqp; - if (ptqp->last) { - ASSERT(ptqp->first); - ptqp->last->next = ptp; + erts_aint32_t act = erts_smp_atomic32_read_nob(&pp->sched.flags); + + if (!pp->sched.taskq.local.busy.first || (act & ERTS_PTS_FLG_BUSY_PORT)) { + *execqp = pp->sched.taskq.local.first; + *processing_busy_q_p = 0; } else { - ASSERT(!ptqp->first); - ptqp->first = ptp; + *execqp = pp->sched.taskq.local.busy.first; + *processing_busy_q_p = 1; + } + + ERTS_PT_DBG_CHK_TASK_QS(pp, *execqp, *processing_busy_q_p); + + while (1) { + erts_aint32_t new, exp; + + new = exp = act; + + new &= ~ERTS_PTS_FLG_IN_RUNQ; + new |= ERTS_PTS_FLG_EXEC; + + act = erts_smp_atomic32_cmpxchg_nob(&pp->sched.flags, new, exp); + + ASSERT(act & ERTS_PTS_FLG_IN_RUNQ); + + if (exp == act) + break; } - ptqp->last = ptp; - ERTS_PT_CHK_IN_TASKQ(ptqp, ptp); } -static ERTS_INLINE void -push_task(ErtsPortTaskQueue *ptqp, ErtsPortTask *ptp) +/* finalize_exec() return value != 0 if port should remain active */ +static ERTS_INLINE int +finalize_exec(Port *pp, ErtsPortTask **execq, int processing_busy_q) { - ERTS_PT_CHK_NOT_IN_TASKQ(ptqp, ptp); - ptp->next = ptqp->first; - ptp->prev = NULL; - ptp->queue = ptqp; - if (ptqp->first) { - ASSERT(ptqp->last); - ptqp->first->prev = ptp; - } + erts_aint32_t act; + + if (!processing_busy_q) + pp->sched.taskq.local.first = *execq; else { - ASSERT(!ptqp->last); - ptqp->last = ptp; + pp->sched.taskq.local.busy.first = *execq; + ASSERT(*execq); } - ptqp->first = ptp; - ERTS_PT_CHK_IN_TASKQ(ptqp, ptp); + + ERTS_PT_DBG_CHK_TASK_QS(pp, *execq, processing_busy_q); + + *execq = NULL; + + act = erts_smp_atomic32_read_nob(&pp->sched.flags); + if (act & ERTS_PTS_FLG_CHK_UNSET_BUSY_PORT_Q) + act = check_unset_busy_port_q(pp, act, pp->sched.taskq.bpq); + + while (1) { + erts_aint32_t new, exp; + + new = exp = act; + + new &= ~ERTS_PTS_FLG_EXEC; + if (act & ERTS_PTS_FLG_HAVE_TASKS) + new |= ERTS_PTS_FLG_IN_RUNQ; + + act = erts_smp_atomic32_cmpxchg_relb(&pp->sched.flags, new, exp); + + ASSERT(!(act & ERTS_PTS_FLG_IN_RUNQ)); + + if (exp == act) + break; + } + + return (act & ERTS_PTS_FLG_HAVE_TASKS) != 0; } -static ERTS_INLINE void -dequeue_task(ErtsPortTask *ptp) +static ERTS_INLINE erts_aint32_t +select_queue_for_exec(Port *pp, ErtsPortTask **execqp, int *processing_busy_q_p) { - ASSERT(ptp); - ASSERT(ptp->queue); - ERTS_PT_CHK_IN_TASKQ(ptp->queue, ptp); - if (ptp->next) - ptp->next->prev = ptp->prev; - else { - ASSERT(ptp->queue->last == ptp); - ptp->queue->last = ptp->prev; + erts_aint32_t flags = erts_smp_atomic32_read_nob(&pp->sched.flags); + + if (flags & ERTS_PTS_FLG_CHK_UNSET_BUSY_PORT_Q) + flags = check_unset_busy_port_q(pp, flags, pp->sched.taskq.bpq); + + ERTS_PT_DBG_CHK_TASK_QS(pp, *execqp, *processing_busy_q_p); + + if (flags & ERTS_PTS_FLG_BUSY_PORT) { + if (*processing_busy_q_p) { + ErtsPortTask *ptp; + + ptp = pp->sched.taskq.local.busy.first = *execqp; + if (!ptp) + pp->sched.taskq.local.busy.last = NULL; + else if (!(ptp->u.alive.flags & ERTS_PT_FLG_WAIT_BUSY)) + no_sig_dep_move_from_busyq(pp); + + *execqp = pp->sched.taskq.local.first; + *processing_busy_q_p = 0; + + ERTS_PT_DBG_CHK_TASK_QS(pp, *execqp, *processing_busy_q_p); + } + + return flags; } - if (ptp->prev) - ptp->prev->next = ptp->next; - else { - ASSERT(ptp->queue->first == ptp); - ptp->queue->first = ptp->next; + + /* Not busy */ + + if (!*processing_busy_q_p && pp->sched.taskq.local.busy.first) { + pp->sched.taskq.local.first = *execqp; + *execqp = pp->sched.taskq.local.busy.first; + *processing_busy_q_p = 1; + + ERTS_PT_DBG_CHK_TASK_QS(pp, *execqp, *processing_busy_q_p); } - ASSERT(ptp->queue->first || !ptp->queue->last); - ASSERT(ptp->queue->last || !ptp->queue->first); - ERTS_PT_CHK_NOT_IN_TASKQ(ptp->queue, ptp); + return flags; } -static ERTS_INLINE ErtsPortTask * -pop_task(ErtsPortTaskQueue *ptqp) +/* + * check_task_for_exec() returns a value !0 if the task + * is ok to execute; otherwise 0. + */ +static ERTS_INLINE int +check_task_for_exec(Port *pp, + erts_aint32_t flags, + ErtsPortTask **execqp, + int *processing_busy_q_p, + ErtsPortTask *ptp) { - ErtsPortTask *ptp = ptqp->first; - if (!ptp) { - ASSERT(!ptqp->last); + + if (!*processing_busy_q_p) { + /* Processing normal queue */ + + ERTS_PT_DBG_CHK_TASK_QS(pp, ptp, *processing_busy_q_p); + + if ((flags & ERTS_PTS_FLG_BUSY_PORT) + && (ptp->u.alive.flags & ERTS_PT_FLG_WAIT_BUSY)) { + + busy_wait_move_to_busy_queue(pp, ptp); + ERTS_PT_DBG_CHK_TASK_QS(pp, *execqp, *processing_busy_q_p); + + return 0; + } + + if (pp->sched.taskq.local.busy.last + && (ptp->u.alive.flags & ERTS_PT_FLG_SIG_DEP)) { + + int res = !check_sig_dep_move_to_busy_queue(pp, ptp); + ERTS_PT_DBG_CHK_TASK_QS(pp, *execqp, *processing_busy_q_p); + + return res; + } + } else { - ERTS_PT_CHK_IN_TASKQ(ptqp, ptp); - ASSERT(!ptp->prev); - ptqp->first = ptp->next; - if (ptqp->first) - ptqp->first->prev = NULL; - else { - ASSERT(ptqp->last == ptp); - ptqp->last = NULL; + /* Processing busy queue */ + + ASSERT(!(flags & ERTS_PTS_FLG_BUSY_PORT)); + + ERTS_PT_DBG_CHK_TASK_QS(pp, ptp, *processing_busy_q_p); + + popped_from_busy_queue(pp, ptp, !*execqp); + + if (!*execqp) { + *execqp = pp->sched.taskq.local.first; + *processing_busy_q_p = 0; } - ASSERT(ptp->queue->first || !ptp->queue->last); - ASSERT(ptp->queue->last || !ptp->queue->first); + + ERTS_PT_DBG_CHK_TASK_QS(pp, *execqp, *processing_busy_q_p); + } - ERTS_PT_CHK_NOT_IN_TASKQ(ptqp, ptp); - return ptp; + + return 1; } -#ifdef HARD_DEBUG +static ErtsPortTask * +fetch_in_queue(Port *pp, ErtsPortTask **execqp) +{ + ErtsPortTask *ptp; + ErtsPortTaskHandleList *free_nshp = NULL; + + erts_port_task_sched_lock(&pp->sched); -static void -check_task_queue(ErtsPortTaskQueue *ptqp, - ErtsPortTask *chk_ptp, - int inq) + ptp = pp->sched.taskq.in.first; + pp->sched.taskq.in.first = NULL; + pp->sched.taskq.in.last = NULL; + if (ptp) + *execqp = ptp->u.alive.next; + else + erts_smp_atomic32_read_band_nob(&pp->sched.flags, + ~ERTS_PTS_FLG_HAVE_TASKS); + + + if (pp->sched.taskq.local.busy.nosuspend) + free_nshp = get_free_nosuspend_handles(pp); + + erts_port_task_sched_unlock(&pp->sched); + + if (free_nshp) + free_nosuspend_handles(free_nshp); + + return ptp; +} + +static ERTS_INLINE ErtsPortTask * +select_task_for_exec(Port *pp, + ErtsPortTask **execqp, + int *processing_busy_q_p) { ErtsPortTask *ptp; - ErtsPortTask *last_ptp; - ErtsPortTask *first_ptp = ptqp->first; - int found_forward = 0, found_backward = 0; - if (!first_ptp) { - ASSERT(!ptqp->last); - } - else { - ASSERT(!first_ptp->prev); - for (ptp = first_ptp; ptp; ptp = ptp->next) { - ASSERT(ptp->queue == ptqp); - if (chk_ptp == ptp) - found_forward = 1; - if (!ptp->prev) { - ASSERT(first_ptp == ptp); - } - if (!ptp->next) { - ASSERT(ptqp->last == ptp); - last_ptp = ptp; - } - } - for (ptp = last_ptp; ptp; ptp = ptp->prev) { - ASSERT(ptp->queue == ptqp); - if (chk_ptp == ptp) - found_backward = 1; - if (!ptp->prev) { - ASSERT(first_ptp == ptp); - } - if (!ptp->next) { - ASSERT(ptqp->last == ptp); - } - } - } - if (chk_ptp) { - if (inq) { - ASSERT(found_forward && found_backward); - } + erts_aint32_t flags; + + flags = select_queue_for_exec(pp, execqp, processing_busy_q_p); + + while (1) { + ptp = *execqp; + if (ptp) + *execqp = ptp->u.alive.next; else { - ASSERT(!found_forward && !found_backward); + ptp = fetch_in_queue(pp, execqp); + if (!ptp) + return NULL; } + if (check_task_for_exec(pp, flags, execqp, processing_busy_q_p, ptp)) + return ptp; } } -#endif /* * Abort a scheduled task. */ int -erts_port_task_abort(Eterm id, ErtsPortTaskHandle *pthp) +erts_port_task_abort(ErtsPortTaskHandle *pthp) { - ErtsRunQueue *runq; - ErtsPortTaskQueue *ptqp; + int res; ErtsPortTask *ptp; - Port *pp; - - pp = &erts_port[internal_port_index(id)]; - runq = erts_port_runq(pp); - if (!runq) - return 1; +#ifdef ERTS_SMP + ErtsThrPrgrDelayHandle dhndl = erts_thr_progress_unmanaged_delay(); +#endif ptp = handle2task(pthp); + if (!ptp) + res = -1; + else { + erts_aint32_t old_state; + +#ifdef DEBUG + ErtsPortTaskHandle *saved_pthp = ptp->u.alive.handle; + ERTS_SMP_READ_MEMORY_BARRIER; + old_state = erts_smp_atomic32_read_nob(&ptp->state); + if (old_state == ERTS_PT_STATE_SCHEDULED) { + ASSERT(saved_pthp == pthp); + } +#endif - if (!ptp) { - erts_smp_runq_unlock(runq); - return 1; + old_state = erts_smp_atomic32_cmpxchg_nob(&ptp->state, + ERTS_PT_STATE_ABORTED, + ERTS_PT_STATE_SCHEDULED); + if (old_state != ERTS_PT_STATE_SCHEDULED) + res = - 1; /* Task already aborted, executing, or executed */ + else { + + reset_port_task_handle(pthp); + + switch (ptp->type) { + case ERTS_PORT_TASK_INPUT: + case ERTS_PORT_TASK_OUTPUT: + case ERTS_PORT_TASK_EVENT: + ASSERT(erts_smp_atomic_read_nob( + &erts_port_task_outstanding_io_tasks) > 0); + erts_smp_atomic_dec_relb(&erts_port_task_outstanding_io_tasks); + break; + case ERTS_PORT_TASK_PROC_SIG: + ERTS_INTERNAL_ERROR("Aborted process to port signal"); + break; + default: + break; + } + + res = 0; + } } - ASSERT(ptp->handle == pthp); - ptqp = ptp->queue; - ASSERT(pp == ptqp->port); +#ifdef ERTS_SMP + erts_thr_progress_unmanaged_continue(dhndl); +#endif - ERTS_PT_CHK_PRES_PORTQ(runq, pp); - ASSERT(ptqp); - ASSERT(ptqp->first); + return res; +} - dequeue_task(ptp); - reset_handle(ptp); +void +erts_port_task_abort_nosuspend_tasks(Port *pp) +{ + erts_aint32_t flags; + ErtsPortTaskHandleList *abort_list; +#ifdef ERTS_SMP + ErtsThrPrgrDelayHandle dhndl = ERTS_THR_PRGR_DHANDLE_INVALID; +#endif - switch (ptp->type) { - case ERTS_PORT_TASK_INPUT: - case ERTS_PORT_TASK_OUTPUT: - case ERTS_PORT_TASK_EVENT: - ASSERT(erts_smp_atomic_read_nob(&erts_port_task_outstanding_io_tasks) > 0); - erts_smp_atomic_dec_relb(&erts_port_task_outstanding_io_tasks); - break; - default: - break; - } + erts_port_task_sched_lock(&pp->sched); + flags = erts_smp_atomic32_read_band_nob(&pp->sched.flags, + ~ERTS_PTS_FLG_HAVE_NS_TASKS); + abort_list = pp->sched.taskq.local.busy.nosuspend; + pp->sched.taskq.local.busy.nosuspend = NULL; + erts_port_task_sched_unlock(&pp->sched); - ASSERT(ptqp == pp->sched.taskq || ptqp == pp->sched.exe_taskq); + while (abort_list) { +#ifdef DEBUG + ErtsPortTaskHandle *saved_pthp; +#endif + ErtsPortTaskType type; + ErtsPortTaskTypeData td; + ErtsPortTaskHandle *pthp; + ErtsPortTask *ptp; + ErtsPortTaskHandleList *pthlp; + erts_aint32_t old_state; - if (ptqp->first || pp->sched.taskq != ptqp) - ptqp = NULL; - else - pp->sched.taskq = NULL; + pthlp = abort_list; + abort_list = pthlp->u.next; - ERTS_PT_CHK_PRES_PORTQ(runq, pp); +#ifdef ERTS_SMP + if (dhndl != ERTS_THR_PRGR_DHANDLE_MANAGED) + dhndl = erts_thr_progress_unmanaged_delay(); +#endif - erts_smp_runq_unlock(runq); + pthp = &pthlp->handle; + ptp = handle2task(pthp); + if (!ptp) { +#ifdef ERTS_SMP + if (dhndl != ERTS_THR_PRGR_DHANDLE_MANAGED) + erts_thr_progress_unmanaged_continue(dhndl); +#endif + schedule_port_task_handle_list_free(pthlp); + continue; + } - port_task_free(ptp); - if (ptqp) - port_taskq_free(ptqp); +#ifdef DEBUG + saved_pthp = ptp->u.alive.handle; + ERTS_SMP_READ_MEMORY_BARRIER; + old_state = erts_smp_atomic32_read_nob(&ptp->state); + if (old_state == ERTS_PT_STATE_SCHEDULED) { + ASSERT(saved_pthp == pthp); + } +#endif - return 0; + old_state = erts_smp_atomic32_cmpxchg_nob(&ptp->state, + ERTS_PT_STATE_ABORTED, + ERTS_PT_STATE_SCHEDULED); + if (old_state != ERTS_PT_STATE_SCHEDULED) { + /* Task already aborted, executing, or executed */ +#ifdef ERTS_SMP + if (dhndl != ERTS_THR_PRGR_DHANDLE_MANAGED) + erts_thr_progress_unmanaged_continue(dhndl); +#endif + schedule_port_task_handle_list_free(pthlp); + continue; + } + + reset_port_task_handle(pthp); + + type = ptp->type; + td = ptp->u.alive.td; + +#ifdef ERTS_SMP + if (dhndl != ERTS_THR_PRGR_DHANDLE_MANAGED) + erts_thr_progress_unmanaged_continue(dhndl); +#endif + schedule_port_task_handle_list_free(pthlp); + + abort_nosuspend_task(pp, type, &td); + } } /* @@ -488,243 +1303,264 @@ int erts_port_task_schedule(Eterm id, ErtsPortTaskHandle *pthp, ErtsPortTaskType type, - ErlDrvEvent event, - ErlDrvEventData event_data) + ...) { + ErtsProc2PortSigData *sigdp = NULL; + ErtsPortTaskHandleList *ns_pthlp = NULL; +#ifdef ERTS_SMP + ErtsRunQueue *xrunq; + ErtsThrPrgrDelayHandle dhndl; +#endif ErtsRunQueue *runq; Port *pp; - ErtsPortTask *ptp; - int enq_port = 0; - - /* - * NOTE: We might not have the port lock here. We are only - * allowed to access the 'sched', 'tab_status', - * and 'id' fields of the port struct while - * tasks_lock is held. - */ + ErtsPortTask *ptp = NULL; + erts_aint32_t act, add_flags; if (pthp && erts_port_task_is_scheduled(pthp)) { ASSERT(0); - erts_port_task_abort(id, pthp); + erts_port_task_abort(pthp); } - ptp = port_task_alloc(); - ASSERT(is_internal_port(id)); - pp = &erts_port[internal_port_index(id)]; - runq = erts_port_runq(pp); - - if (!runq || ERTS_PORT_TASK_INVALID_PORT(pp, id)) { - if (runq) - erts_smp_runq_unlock(runq); - return -1; - } - ASSERT(!erts_port_task_is_scheduled(pthp)); +#ifdef ERTS_SMP + dhndl = erts_thr_progress_unmanaged_delay(); +#endif - ERTS_PT_CHK_PRES_PORTQ(runq, pp); + pp = erts_port_lookup_raw(id); - if (!pp->sched.taskq && !pp->sched.in_runq && !pp->sched.exe_taskq) { #ifdef ERTS_SMP - ErtsRunQueue *xrunq = erts_check_emigration_need(runq, ERTS_PORT_PRIO_LEVEL); - if (xrunq) { - /* Port emigrated ... */ - erts_smp_atomic_set_nob(&pp->run_queue, (erts_aint_t) xrunq); - erts_smp_runq_unlock(runq); - runq = erts_port_runq(pp); - if (!runq) - return -1; - } - enq_port = !pp->sched.taskq && !pp->sched.in_runq && !pp->sched.exe_taskq; -#else - enq_port = 1; -#endif + if (dhndl != ERTS_THR_PRGR_DHANDLE_MANAGED) { + if (pp) + erts_port_inc_refc(pp); + erts_thr_progress_unmanaged_continue(dhndl); } +#endif - ASSERT(!enq_port - || !(ERTS_RUNQ_FLGS_GET_NOB(runq) & ERTS_RUNQ_FLG_SUSPENDED)); + if (!pp) + goto fail; - if (!pp->sched.taskq) - pp->sched.taskq = port_taskq_init(port_taskq_alloc(), pp); + if (type != ERTS_PORT_TASK_PROC_SIG) { + ptp = port_task_alloc(); - ASSERT(ptp); + ptp->type = type; + ptp->u.alive.flags = 0; - ptp->type = type; - ptp->event = event; - ptp->event_data = event_data; + erts_smp_atomic32_init_nob(&ptp->state, ERTS_PT_STATE_SCHEDULED); - set_handle(ptp, pthp); + set_handle(ptp, pthp); + } switch (type) { - case ERTS_PORT_TASK_FREE: - erl_exit(ERTS_ABORT_EXIT, - "erts_port_task_schedule(): Cannot schedule free task\n"); - break; case ERTS_PORT_TASK_INPUT: - case ERTS_PORT_TASK_OUTPUT: - case ERTS_PORT_TASK_EVENT: + case ERTS_PORT_TASK_OUTPUT: { + va_list argp; + va_start(argp, type); + ptp->u.alive.td.io.event = va_arg(argp, ErlDrvEvent); + va_end(argp); + erts_smp_atomic_inc_relb(&erts_port_task_outstanding_io_tasks); + break; + } + case ERTS_PORT_TASK_EVENT: { + va_list argp; + va_start(argp, type); + ptp->u.alive.td.io.event = va_arg(argp, ErlDrvEvent); + ptp->u.alive.td.io.event_data = va_arg(argp, ErlDrvEventData); + va_end(argp); erts_smp_atomic_inc_relb(&erts_port_task_outstanding_io_tasks); - /* Fall through... */ + break; + } + case ERTS_PORT_TASK_PROC_SIG: { + va_list argp; + ASSERT(!pthp); + va_start(argp, type); + sigdp = va_arg(argp, ErtsProc2PortSigData *); + ptp = p2p_sig_data_to_task(sigdp); + ptp->u.alive.td.psig.callback = va_arg(argp, ErtsProc2PortSigCallback); + ptp->u.alive.flags |= va_arg(argp, int); + va_end(argp); + if (!(ptp->u.alive.flags & ERTS_PT_FLG_NOSUSPEND)) + set_handle(ptp, pthp); + else { + ns_pthlp = erts_alloc(ERTS_ALC_T_PT_HNDL_LIST, + sizeof(ErtsPortTaskHandleList)); + set_handle(ptp, &ns_pthlp->handle); + } + break; + } default: - enqueue_task(pp->sched.taskq, ptp); break; } -#ifndef ERTS_SMP - /* - * When (!enq_port && !pp->sched.exe_taskq) is true in the smp case, - * the port might not be in the run queue. If this is the case, another - * thread is in the process of enqueueing the port. This very seldom - * occur, but do occur and is a valid scenario. Debug info showing this - * enqueue in progress must be introduced before we can enable (modified - * versions of these) assertions in the smp case again. - */ -#if defined(HARD_DEBUG) - if (pp->sched.exe_taskq || enq_port) - ERTS_PT_CHK_NOT_IN_PORTQ(runq, pp); - else - ERTS_PT_CHK_IN_PORTQ(runq, pp); -#elif defined(DEBUG) - if (!enq_port && !pp->sched.exe_taskq) { - /* We should be in port run q */ - ASSERT(pp->sched.in_runq); + if (!enqueue_task(pp, ptp, sigdp, ns_pthlp, &act)) { + reset_handle(ptp); + if (ns_pthlp && !(act & ERTS_PTS_FLG_EXIT)) + goto abort_nosuspend; + else + goto fail; } -#endif -#endif - if (!enq_port) { - ERTS_PT_CHK_PRES_PORTQ(runq, pp); - erts_smp_runq_unlock(runq); - } - else { - enqueue_port(runq, pp); - ERTS_PT_CHK_PRES_PORTQ(runq, pp); - - if (erts_system_profile_flags.runnable_ports) { - profile_runnable_port(pp, am_active); + add_flags = ERTS_PTS_FLG_HAVE_TASKS; + if (ns_pthlp) + add_flags |= ERTS_PTS_FLG_HAVE_NS_TASKS; + + while (1) { + erts_aint32_t new, exp; + + if ((act & add_flags) == add_flags + && (act & (ERTS_PTS_FLG_IN_RUNQ|ERTS_PTS_FLG_EXEC))) + goto done; /* Done */ + + new = exp = act; + new |= add_flags; + if (!(act & (ERTS_PTS_FLG_IN_RUNQ|ERTS_PTS_FLG_EXEC))) + new |= ERTS_PTS_FLG_IN_RUNQ; + + act = erts_smp_atomic32_cmpxchg_relb(&pp->sched.flags, new, exp); + + if (exp == act) { + if (!(act & (ERTS_PTS_FLG_IN_RUNQ|ERTS_PTS_FLG_EXEC))) + break; /* Need to enqueue port */ + goto done; /* Done */ } + if (act & ERTS_PTS_FLG_EXIT) + goto done; /* Died after our task insert... */ + } + + /* Enqueue port on run-queue */ + + runq = erts_port_runq(pp); + if (!runq) + ERTS_INTERNAL_ERROR("Missing run-queue"); + +#ifdef ERTS_SMP + xrunq = erts_check_emigration_need(runq, ERTS_PORT_PRIO_LEVEL); + if (xrunq) { + /* Port emigrated ... */ + erts_smp_atomic_set_nob(&pp->run_queue, (erts_aint_t) xrunq); erts_smp_runq_unlock(runq); + runq = erts_port_runq(pp); + if (!runq) + ERTS_INTERNAL_ERROR("Missing run-queue"); + } +#endif - erts_smp_notify_inc_runq(runq); + enqueue_port(runq, pp); + + if (erts_system_profile_flags.runnable_ports) { + profile_runnable_port(pp, am_active); } + + erts_smp_runq_unlock(runq); + + erts_smp_notify_inc_runq(runq); + +done: + +#ifdef ERTS_SMP + if (dhndl != ERTS_THR_PRGR_DHANDLE_MANAGED) + erts_port_dec_refc(pp); +#endif + return 0; + +abort_nosuspend: + +#ifdef ERTS_SMP + if (dhndl != ERTS_THR_PRGR_DHANDLE_MANAGED) + erts_port_dec_refc(pp); +#endif + + abort_nosuspend_task(pp, ptp->type, &ptp->u.alive.td); + + ASSERT(ns_pthlp); + erts_free(ERTS_ALC_T_PT_HNDL_LIST, ns_pthlp); + if (ptp) + port_task_free(ptp); + + return 0; + +fail: + +#ifdef ERTS_SMP + if (dhndl != ERTS_THR_PRGR_DHANDLE_MANAGED) + erts_port_dec_refc(pp); +#endif + + if (ns_pthlp) + erts_free(ERTS_ALC_T_PT_HNDL_LIST, ns_pthlp); + + if (ptp) + port_task_free(ptp); + + return -1; } void erts_port_task_free_port(Port *pp) { + ErtsProcList *suspended; + erts_aint32_t flags; ErtsRunQueue *runq; - ErtsPortTaskQueue *ptqp; ERTS_SMP_LC_ASSERT(erts_lc_is_port_locked(pp)); - ASSERT(!(pp->status & ERTS_PORT_SFLGS_DEAD)); + ASSERT(!(erts_atomic32_read_nob(&pp->state) & ERTS_PORT_SFLGS_DEAD)); + runq = erts_port_runq(pp); - ASSERT(runq); - ERTS_PT_CHK_PRES_PORTQ(runq, pp); - ptqp = pp->sched.exe_taskq; - if (ptqp) { - /* I (this thread) am currently executing this port, free it - when scheduled out... */ - ErtsPortTask *ptp; - enqueue_free: - ptp = port_task_alloc(); - erts_smp_port_state_lock(pp); - pp->status &= ~ERTS_PORT_SFLG_CLOSING; - pp->status |= ERTS_PORT_SFLG_FREE_SCHEDULED; - erts_may_save_closed_port(pp); - erts_smp_port_state_unlock(pp); - ERTS_LC_ASSERT(erts_smp_atomic_read_nob(&pp->refc) > 1); - ptp->type = ERTS_PORT_TASK_FREE; - ptp->event = (ErlDrvEvent) -1; - ptp->event_data = NULL; - set_handle(ptp, NULL); - push_task(ptqp, ptp); - ERTS_PT_CHK_PRES_PORTQ(runq, pp); - erts_smp_runq_unlock(runq); - } - else { - if (pp->sched.in_runq) { - ptqp = pp->sched.taskq; - if (!ptqp) - pp->sched.taskq = ptqp = port_taskq_init(port_taskq_alloc(), pp); - goto enqueue_free; - } - ASSERT(!pp->sched.taskq); - erts_smp_port_state_lock(pp); - pp->status &= ~ERTS_PORT_SFLG_CLOSING; - pp->status |= ERTS_PORT_SFLG_FREE_SCHEDULED; - erts_may_save_closed_port(pp); - erts_smp_port_state_unlock(pp); - erts_smp_atomic_dec_nob(&pp->refc); /* Not alive */ - ERTS_LC_ASSERT(erts_smp_atomic_read_nob(&pp->refc) > 0); /* Lock */ - handle_remaining_tasks(runq, pp); /* May release runq lock */ - ASSERT(!pp->sched.exe_taskq && (!ptqp || !ptqp->first)); - pp->sched.taskq = NULL; - ERTS_PT_CHK_PRES_PORTQ(runq, pp); - erts_smp_runq_unlock(runq); - } -} + if (!runq) + ERTS_INTERNAL_ERROR("Missing run-queue"); + erts_port_task_sched_lock(&pp->sched); + flags = erts_smp_atomic32_read_bor_relb(&pp->sched.flags, + ERTS_PTS_FLG_EXIT); + suspended = pp->suspended; + pp->suspended = NULL; + erts_port_task_sched_unlock(&pp->sched); + erts_atomic32_read_bset_relb(&pp->state, + (ERTS_PORT_SFLG_CLOSING + | ERTS_PORT_SFLG_FREE), + ERTS_PORT_SFLG_FREE); -typedef struct { - ErtsRunQueue *runq; - int *resp; -} ErtsPortTaskExeBlockData; + erts_smp_runq_unlock(runq); + + if (erts_proclist_fetch(&suspended, NULL)) + erts_resume_processes(suspended); + + if (!(flags & (ERTS_PTS_FLG_IN_RUNQ|ERTS_PTS_FLG_EXEC))) + begin_port_cleanup(pp, NULL, NULL); +} /* - * Run all scheduled tasks for the first port in run queue. If - * new tasks appear while running reschedule port (free task is - * an exception; it is always handled instantly). + * Execute scheduled tasks of a port. * * erts_port_task_execute() is called by scheduler threads between - * scheduleing of processes. Sched lock should be held by caller. + * scheduling of processes. Run-queue lock should be held by caller. */ int erts_port_task_execute(ErtsRunQueue *runq, Port **curr_port_pp) { Port *pp; - ErtsPortTaskQueue *ptqp; - ErtsPortTask *ptp; + ErtsPortTask *execq; + int processing_busy_q; int res = 0; int reds = ERTS_PORT_REDS_EXECUTE; erts_aint_t io_tasks_executed = 0; int fpe_was_unmasked; + erts_aint32_t state; + int active; ERTS_SMP_LC_ASSERT(erts_smp_lc_runq_is_locked(runq)); - ERTS_PT_CHK_PORTQ(runq); - pp = pop_port(runq); if (!pp) { res = 0; goto done; } - ASSERT(pp->sched.in_runq); - pp->sched.in_runq = 0; - if (!pp->sched.taskq) { - if (erts_system_profile_flags.runnable_ports) - profile_runnable_port(pp, am_inactive); - res = (erts_smp_atomic_read_nob(&erts_port_task_outstanding_io_tasks) - != (erts_aint_t) 0); - goto done; - } + erts_smp_runq_unlock(runq); *curr_port_pp = pp; - - ASSERT(pp->sched.taskq->first); - ptqp = pp->sched.taskq; - pp->sched.taskq = NULL; - - ASSERT(!pp->sched.exe_taskq); - pp->sched.exe_taskq = ptqp; - - if (erts_smp_port_trylock(pp) == EBUSY) { - erts_smp_runq_unlock(runq); - erts_smp_port_lock(pp); - erts_smp_runq_lock(runq); - } if (erts_sched_stat.enabled) { ErtsSchedulerData *esdp = erts_get_scheduler_data(); @@ -741,77 +1577,94 @@ erts_port_task_execute(ErtsRunQueue *runq, Port **curr_port_pp) erts_smp_spin_unlock(&erts_sched_stat.lock); } + prepare_exec(pp, &execq, &processing_busy_q); + + erts_smp_port_lock(pp); + /* trace port scheduling, in */ if (IS_TRACED_FL(pp, F_TRACE_SCHED_PORTS)) { trace_sched_ports(pp, am_in); } - ERTS_SMP_LC_ASSERT(erts_lc_is_port_locked(pp)); + fpe_was_unmasked = erts_block_fpe(); - ERTS_PT_CHK_PRES_PORTQ(runq, pp); - ptp = pop_task(ptqp); + state = erts_atomic32_read_nob(&pp->state); + goto begin_handle_tasks; - fpe_was_unmasked = erts_block_fpe(); + while (1) { + erts_aint32_t task_state; + ErtsPortTask *ptp; - while (ptp) { - ASSERT(pp->sched.taskq != pp->sched.exe_taskq); + ptp = select_task_for_exec(pp, &execq, &processing_busy_q); + if (!ptp) + break; + + task_state = erts_smp_atomic32_cmpxchg_nob(&ptp->state, + ERTS_PT_STATE_EXECUTING, + ERTS_PT_STATE_SCHEDULED); + if (task_state != ERTS_PT_STATE_SCHEDULED) { + ASSERT(task_state == ERTS_PT_STATE_ABORTED); + goto aborted_port_task; + } reset_handle(ptp); - erts_smp_runq_unlock(runq); ERTS_SMP_LC_ASSERT(erts_lc_is_port_locked(pp)); ERTS_SMP_CHK_NO_PROC_LOCKS; ASSERT(pp->drv_ptr); switch (ptp->type) { - case ERTS_PORT_TASK_FREE: /* May be pushed in q at any time */ - reds += ERTS_PORT_REDS_FREE; - erts_smp_runq_lock(runq); - - erts_unblock_fpe(fpe_was_unmasked); - ASSERT(pp->status & ERTS_PORT_SFLG_FREE_SCHEDULED); - if (ptqp->first || (pp->sched.taskq && pp->sched.taskq->first)) - handle_remaining_tasks(runq, pp); - ASSERT(!ptqp->first - && (!pp->sched.taskq || !pp->sched.taskq->first)); - erts_smp_atomic_dec_nob(&pp->refc); /* Not alive */ - ERTS_LC_ASSERT(erts_smp_atomic_read_nob(&pp->refc) > 0); /* Lock */ - - port_task_free(ptp); - if (pp->sched.taskq) - port_taskq_free(pp->sched.taskq); - pp->sched.taskq = NULL; - - goto tasks_done; case ERTS_PORT_TASK_TIMEOUT: reds += ERTS_PORT_REDS_TIMEOUT; - if (!(pp->status & ERTS_PORT_SFLGS_DEAD)) { + if (!(state & ERTS_PORT_SFLGS_DEAD)) { DTRACE_DRIVER(driver_timeout, pp); (*pp->drv_ptr->timeout)((ErlDrvData) pp->drv_data); } break; case ERTS_PORT_TASK_INPUT: reds += ERTS_PORT_REDS_INPUT; - ASSERT((pp->status & ERTS_PORT_SFLGS_DEAD) == 0); + ASSERT((state & ERTS_PORT_SFLGS_DEAD) == 0); DTRACE_DRIVER(driver_ready_input, pp); /* NOTE some windows drivers use ->ready_input for input and output */ - (*pp->drv_ptr->ready_input)((ErlDrvData) pp->drv_data, ptp->event); + (*pp->drv_ptr->ready_input)((ErlDrvData) pp->drv_data, + ptp->u.alive.td.io.event); io_tasks_executed++; break; case ERTS_PORT_TASK_OUTPUT: reds += ERTS_PORT_REDS_OUTPUT; - ASSERT((pp->status & ERTS_PORT_SFLGS_DEAD) == 0); + ASSERT((state & ERTS_PORT_SFLGS_DEAD) == 0); DTRACE_DRIVER(driver_ready_output, pp); - (*pp->drv_ptr->ready_output)((ErlDrvData) pp->drv_data, ptp->event); + (*pp->drv_ptr->ready_output)((ErlDrvData) pp->drv_data, + ptp->u.alive.td.io.event); io_tasks_executed++; break; case ERTS_PORT_TASK_EVENT: reds += ERTS_PORT_REDS_EVENT; - ASSERT((pp->status & ERTS_PORT_SFLGS_DEAD) == 0); + ASSERT((state & ERTS_PORT_SFLGS_DEAD) == 0); DTRACE_DRIVER(driver_event, pp); - (*pp->drv_ptr->event)((ErlDrvData) pp->drv_data, ptp->event, ptp->event_data); + (*pp->drv_ptr->event)((ErlDrvData) pp->drv_data, + ptp->u.alive.td.io.event, + ptp->u.alive.td.io.event_data); io_tasks_executed++; break; + case ERTS_PORT_TASK_PROC_SIG: { + ErtsProc2PortSigData *sigdp = &ptp->u.alive.td.psig.data; + ASSERT((state & ERTS_PORT_SFLGS_DEAD) == 0); + if (!pp->sched.taskq.bpq) + reds += ptp->u.alive.td.psig.callback(pp, + state, + ERTS_PROC2PORT_SIG_EXEC, + sigdp); + else { + ErlDrvSizeT size = erts_proc2port_sig_command_data_size(sigdp); + reds += ptp->u.alive.td.psig.callback(pp, + state, + ERTS_PROC2PORT_SIG_EXEC, + sigdp); + dequeued_proc2port_data(pp, size); + } + break; + } case ERTS_PORT_TASK_DIST_CMD: reds += erts_dist_command(pp, CONTEXT_REDS-reds); break; @@ -822,33 +1675,30 @@ erts_port_task_execute(ErtsRunQueue *runq, Port **curr_port_pp) break; } - if ((pp->status & ERTS_PORT_SFLG_CLOSING) - && erts_is_port_ioq_empty(pp)) { - reds += ERTS_PORT_REDS_TERMINATE; - erts_terminate_port(pp); - } - - ERTS_SMP_LC_ASSERT(erts_lc_is_port_locked(pp)); - -#ifdef ERTS_SMP - if (pp->xports) - erts_smp_xports_unlock(pp); - ASSERT(!pp->xports); -#endif + reds += erts_port_driver_callback_epilogue(pp, &state); - ERTS_SMP_LC_ASSERT(erts_lc_is_port_locked(pp)); + aborted_port_task: + schedule_port_task_free(ptp); - port_task_free(ptp); + begin_handle_tasks: + if (state & ERTS_PORT_SFLG_FREE) { + reds += ERTS_PORT_REDS_FREE; + begin_port_cleanup(pp, &execq, &processing_busy_q); - erts_smp_runq_lock(runq); + break; + } - ptp = pop_task(ptqp); + if (reds >= CONTEXT_REDS) + break; } - tasks_done: - erts_unblock_fpe(fpe_was_unmasked); + /* trace port scheduling, out */ + if (IS_TRACED_FL(pp, F_TRACE_SCHED_PORTS)) { + trace_sched_ports(pp, am_out); + } + if (io_tasks_executed) { ASSERT(erts_smp_atomic_read_nob(&erts_port_task_outstanding_io_tasks) >= io_tasks_executed); @@ -856,15 +1706,19 @@ erts_port_task_execute(ErtsRunQueue *runq, Port **curr_port_pp) -1*io_tasks_executed); } - *curr_port_pp = NULL; - #ifdef ERTS_SMP ASSERT(runq == (ErtsRunQueue *) erts_smp_atomic_read_nob(&pp->run_queue)); #endif - if (!pp->sched.taskq) { - ASSERT(pp->sched.exe_taskq); - pp->sched.exe_taskq = NULL; + active = finalize_exec(pp, &execq, processing_busy_q); + + erts_port_release(pp); + + *curr_port_pp = NULL; + + erts_smp_runq_lock(runq); + + if (!active) { if (erts_system_profile_flags.runnable_ports) profile_runnable_port(pp, am_inactive); } @@ -873,16 +1727,13 @@ erts_port_task_execute(ErtsRunQueue *runq, Port **curr_port_pp) ErtsRunQueue *xrunq; #endif - ASSERT(!(pp->status & ERTS_PORT_SFLGS_DEAD)); - ASSERT(pp->sched.taskq->first); + ASSERT(!(erts_atomic32_read_nob(&pp->state) & ERTS_PORT_SFLGS_DEAD)); #ifdef ERTS_SMP xrunq = erts_check_emigration_need(runq, ERTS_PORT_PRIO_LEVEL); if (!xrunq) { #endif enqueue_port(runq, pp); - ASSERT(pp->sched.exe_taskq); - pp->sched.exe_taskq = NULL; /* No need to notify ourselves about inc in runq. */ #ifdef ERTS_SMP } @@ -892,49 +1743,20 @@ erts_port_task_execute(ErtsRunQueue *runq, Port **curr_port_pp) erts_smp_runq_unlock(runq); xrunq = erts_port_runq(pp); - if (xrunq) { - enqueue_port(xrunq, pp); - ASSERT(pp->sched.exe_taskq); - pp->sched.exe_taskq = NULL; - erts_smp_runq_unlock(xrunq); - erts_smp_notify_inc_runq(xrunq); - } + ASSERT(xrunq); + enqueue_port(xrunq, pp); + erts_smp_runq_unlock(xrunq); + erts_smp_notify_inc_runq(xrunq); erts_smp_runq_lock(runq); } #endif } + done: res = (erts_smp_atomic_read_nob(&erts_port_task_outstanding_io_tasks) != (erts_aint_t) 0); - ERTS_PT_CHK_PRES_PORTQ(runq, pp); - - port_taskq_free(ptqp); - - /* trace port scheduling, out */ - if (IS_TRACED_FL(pp, F_TRACE_SCHED_PORTS)) { - trace_sched_ports(pp, am_out); - } -#ifndef ERTS_SMP - erts_port_release(pp); -#else - { - erts_aint_t refc; - erts_smp_mtx_unlock(pp->lock); - refc = erts_smp_atomic_dec_read_nob(&pp->refc); - ASSERT(refc >= 0); - if (refc == 0) { - erts_smp_runq_unlock(runq); - erts_port_cleanup(pp); /* Might aquire runq lock */ - erts_smp_runq_lock(runq); - res = (erts_smp_atomic_read_nob(&erts_port_task_outstanding_io_tasks) - != (erts_aint_t) 0); - } - } -#endif - - done: runq->scheduler->reductions += reds; ERTS_SMP_LC_ASSERT(erts_smp_lc_runq_is_locked(runq)); @@ -943,78 +1765,191 @@ erts_port_task_execute(ErtsRunQueue *runq, Port **curr_port_pp) return res; } -/* - * Handle remaining tasks after a free task. - */ +#ifdef ERTS_SMP +static void +release_port(void *vport) +{ + erts_port_dec_refc((Port *) vport); +} +#endif static void -handle_remaining_tasks(ErtsRunQueue *runq, Port *pp) +begin_port_cleanup(Port *pp, ErtsPortTask **execqp, int *processing_busy_q_p) { - int i; - ErtsPortTask *ptp; - ErtsPortTaskQueue *ptqps[] = {pp->sched.exe_taskq, pp->sched.taskq}; + int i, max; + ErtsPortTaskBusyCallerTable *tabp; + ErtsPortTask *qs[3]; ERTS_SMP_LC_ASSERT(erts_lc_is_port_locked(pp)); - for (i = 0; i < sizeof(ptqps)/sizeof(ErtsPortTaskQueue *); i++) { - if (!ptqps[i]) - continue; - ptp = pop_task(ptqps[i]); - while (ptp) { + /* + * Abort remaining tasks... + * + * We want to process queues in the following order in order + * to preserve signal ordering guarantees: + * 1. Local busy queue + * 2. Local queue + * 3. In queue + */ + + max = 0; + if (!execqp) { + if (pp->sched.taskq.local.busy.first) + qs[max++] = pp->sched.taskq.local.busy.first; + if (pp->sched.taskq.local.first) + qs[max++] = pp->sched.taskq.local.first; + } + else { + if (*processing_busy_q_p) { + if (*execqp) + qs[max++] = *execqp; + if (pp->sched.taskq.local.first) + qs[max++] = pp->sched.taskq.local.first; + } + else { + if (pp->sched.taskq.local.busy.first) + qs[max++] = pp->sched.taskq.local.busy.first; + if (*execqp) + qs[max++] = *execqp; + } + *execqp = NULL; + *processing_busy_q_p = 0; + } + pp->sched.taskq.local.busy.first = NULL; + pp->sched.taskq.local.busy.last = NULL; + pp->sched.taskq.local.first = NULL; + tabp = pp->sched.taskq.local.busy.table; + if (tabp) { + int bix; + for (bix = 0; bix < ERTS_PORT_TASK_BUSY_CALLER_TABLE_BUCKETS; bix++) { + ErtsPortTaskBusyCaller *bcp = tabp->bucket[bix]; + while (bcp) { + ErtsPortTaskBusyCaller *free_bcp = bcp; + bcp = bcp->next; + if (free_bcp != &tabp->pre_alloc_busy_caller) + erts_free(ERTS_ALC_T_BUSY_CALLER, free_bcp); + } + } + + busy_caller_table_free(tabp); + pp->sched.taskq.local.busy.table = NULL; + } + + erts_port_task_sched_lock(&pp->sched); + qs[max] = pp->sched.taskq.in.first; + pp->sched.taskq.in.first = NULL; + pp->sched.taskq.in.last = NULL; + erts_port_task_sched_unlock(&pp->sched); + if (qs[max]) + max++; + + for (i = 0; i < max; i++) { + while (1) { + erts_aint32_t state; + ErtsPortTask *ptp = qs[i]; + if (!ptp) + break; + + qs[i] = ptp->u.alive.next; + + /* Normal case here is aborted tasks... */ + state = erts_smp_atomic32_read_nob(&ptp->state); + if (state == ERTS_PT_STATE_ABORTED) + goto aborted_port_task; + + state = erts_smp_atomic32_cmpxchg_nob(&ptp->state, + ERTS_PT_STATE_EXECUTING, + ERTS_PT_STATE_SCHEDULED); + if (state != ERTS_PT_STATE_SCHEDULED) { + ASSERT(state == ERTS_PT_STATE_ABORTED); + goto aborted_port_task; + } + reset_handle(ptp); - erts_smp_runq_unlock(runq); switch (ptp->type) { - case ERTS_PORT_TASK_FREE: case ERTS_PORT_TASK_TIMEOUT: break; case ERTS_PORT_TASK_INPUT: - erts_stale_drv_select(pp->id, ptp->event, DO_READ, 1); + erts_stale_drv_select(pp->common.id, + ptp->u.alive.td.io.event, + DO_READ, + 1); break; case ERTS_PORT_TASK_OUTPUT: - erts_stale_drv_select(pp->id, ptp->event, DO_WRITE, 1); + erts_stale_drv_select(pp->common.id, + ptp->u.alive.td.io.event, + DO_WRITE, + 1); break; case ERTS_PORT_TASK_EVENT: - erts_stale_drv_select(pp->id, ptp->event, 0, 1); + erts_stale_drv_select(pp->common.id, + ptp->u.alive.td.io.event, + 0, + 1); break; case ERTS_PORT_TASK_DIST_CMD: break; + case ERTS_PORT_TASK_PROC_SIG: { + ErtsProc2PortSigData *sigdp = &ptp->u.alive.td.psig.data; + if (!pp->sched.taskq.bpq) + ptp->u.alive.td.psig.callback(NULL, + ERTS_PORT_SFLG_INVALID, + ERTS_PROC2PORT_SIG_ABORT_CLOSED, + sigdp); + else { + ErlDrvSizeT size = erts_proc2port_sig_command_data_size(sigdp); + ptp->u.alive.td.psig.callback(NULL, + ERTS_PORT_SFLG_INVALID, + ERTS_PROC2PORT_SIG_ABORT_CLOSED, + sigdp); + aborted_proc2port_data(pp, size); + } + break; + } default: erl_exit(ERTS_ABORT_EXIT, "Invalid port task type: %d\n", (int) ptp->type); } - port_task_free(ptp); - - erts_smp_runq_lock(runq); - ptp = pop_task(ptqps[i]); + aborted_port_task: + schedule_port_task_free(ptp); } } - ASSERT(!pp->sched.taskq || !pp->sched.taskq->first); + erts_smp_atomic32_read_band_nob(&pp->sched.flags, + ~(ERTS_PTS_FLG_HAVE_BUSY_TASKS + |ERTS_PTS_FLG_HAVE_TASKS)); + + /* + * Schedule cleanup of port structure... + */ +#ifdef ERTS_SMP + erts_schedule_thr_prgr_later_op(release_port, + (void *) pp, + &pp->common.u.release); +#else + pp->cleanup = 1; +#endif } int erts_port_is_scheduled(Port *pp) { - int res; - ErtsRunQueue *runq = erts_port_runq(pp); - if (!runq) - return 0; - res = pp->sched.taskq || pp->sched.exe_taskq; - erts_smp_runq_unlock(runq); - return res; + erts_aint32_t flags = erts_smp_atomic32_read_acqb(&pp->sched.flags); + return (flags & (ERTS_PTS_FLG_IN_RUNQ|ERTS_PTS_FLG_EXEC)) != 0; } #ifdef ERTS_SMP + void erts_enqueue_port(ErtsRunQueue *rq, Port *pp) { ERTS_SMP_LC_ASSERT(erts_smp_lc_runq_is_locked(rq)); ASSERT(rq == (ErtsRunQueue *) erts_smp_atomic_read_nob(&pp->run_queue)); - ASSERT(pp->sched.in_runq); + ASSERT(erts_smp_atomic32_read_nob(&pp->sched.flags) & ERTS_PTS_FLG_IN_RUNQ); enqueue_port(rq, pp); } @@ -1026,7 +1961,8 @@ erts_dequeue_port(ErtsRunQueue *rq) pp = pop_port(rq); ASSERT(!pp || rq == (ErtsRunQueue *) erts_smp_atomic_read_nob(&pp->run_queue)); - ASSERT(!pp || pp->sched.in_runq); + ASSERT(!pp || (erts_smp_atomic32_read_nob(&pp->sched.flags) + & ERTS_PTS_FLG_IN_RUNQ)); return pp; } @@ -1041,5 +1977,5 @@ erts_port_task_init(void) erts_smp_atomic_init_nob(&erts_port_task_outstanding_io_tasks, (erts_aint_t) 0); init_port_task_alloc(); - init_port_taskq_alloc(); + init_busy_caller_table_alloc(); } diff --git a/erts/emulator/beam/erl_port_task.h b/erts/emulator/beam/erl_port_task.h index fd88b1c1ff..ae6cd69ae2 100644 --- a/erts/emulator/beam/erl_port_task.h +++ b/erts/emulator/beam/erl_port_task.h @@ -1,7 +1,7 @@ /* * %CopyrightBegin% * - * Copyright Ericsson AB 2006-2011. All Rights Reserved. + * Copyright Ericsson AB 2006-2012. All Rights Reserved. * * The contents of this file are subject to the Erlang Public License, * Version 1.1, (the "License"); you may not use this file except in @@ -27,6 +27,9 @@ #define ERTS_PORT_TASK_H_BASIC_TYPES__ #include "erl_sys_driver.h" #include "erl_smp.h" +#define ERL_PORT_GET_PORT_TYPE_ONLY__ +#include "erl_port.h" +#undef ERL_PORT_GET_PORT_TYPE_ONLY__ typedef erts_smp_atomic_t ErtsPortTaskHandle; #endif @@ -43,13 +46,19 @@ typedef erts_smp_atomic_t ErtsPortTaskHandle; #define ERTS_INCLUDE_SCHEDULER_INTERNALS #endif +#define ERTS_PT_FLG_WAIT_BUSY (1 << 0) +#define ERTS_PT_FLG_SIG_DEP (1 << 1) +#define ERTS_PT_FLG_NOSUSPEND (1 << 2) +#define ERTS_PT_FLG_REF (1 << 3) +#define ERTS_PT_FLG_BAD_OUTPUT (1 << 4) + typedef enum { - ERTS_PORT_TASK_FREE, ERTS_PORT_TASK_INPUT, ERTS_PORT_TASK_OUTPUT, ERTS_PORT_TASK_EVENT, ERTS_PORT_TASK_TIMEOUT, - ERTS_PORT_TASK_DIST_CMD + ERTS_PORT_TASK_DIST_CMD, + ERTS_PORT_TASK_PROC_SIG } ErtsPortTaskType; #ifdef ERTS_INCLUDE_SCHEDULER_INTERNALS @@ -57,19 +66,76 @@ typedef enum { extern erts_smp_atomic_t erts_port_task_outstanding_io_tasks; #endif +#define ERTS_PTS_FLG_IN_RUNQ (((erts_aint32_t) 1) << 0) +#define ERTS_PTS_FLG_EXEC (((erts_aint32_t) 1) << 1) +#define ERTS_PTS_FLG_HAVE_TASKS (((erts_aint32_t) 1) << 2) +#define ERTS_PTS_FLG_EXIT (((erts_aint32_t) 1) << 3) +#define ERTS_PTS_FLG_BUSY_PORT (((erts_aint32_t) 1) << 4) +#define ERTS_PTS_FLG_BUSY_PORT_Q (((erts_aint32_t) 1) << 5) +#define ERTS_PTS_FLG_CHK_UNSET_BUSY_PORT_Q (((erts_aint32_t) 1) << 6) +#define ERTS_PTS_FLG_HAVE_BUSY_TASKS (((erts_aint32_t) 1) << 7) +#define ERTS_PTS_FLG_HAVE_NS_TASKS (((erts_aint32_t) 1) << 8) +#define ERTS_PTS_FLG_PARALLELISM (((erts_aint32_t) 1) << 9) +#define ERTS_PTS_FLG_FORCE_SCHED (((erts_aint32_t) 1) << 10) + +#define ERTS_PTS_FLGS_BUSY \ + (ERTS_PTS_FLG_BUSY_PORT | ERTS_PTS_FLG_BUSY_PORT_Q) + +#define ERTS_PTS_FLGS_FORCE_SCHEDULE_OP \ + (ERTS_PTS_FLG_EXIT \ + | ERTS_PTS_FLG_HAVE_BUSY_TASKS \ + | ERTS_PTS_FLG_HAVE_TASKS \ + | ERTS_PTS_FLG_EXEC \ + | ERTS_PTS_FLG_FORCE_SCHED) + +#define ERTS_PORT_TASK_DEFAULT_BUSY_PORT_Q_HIGH 8192 +#define ERTS_PORT_TASK_DEFAULT_BUSY_PORT_Q_LOW 4096 + +typedef struct { + ErlDrvSizeT high; + erts_smp_atomic_t low; + erts_smp_atomic_t size; +} ErtsPortTaskBusyPortQ; + typedef struct ErtsPortTask_ ErtsPortTask; -typedef struct ErtsPortTaskQueue_ ErtsPortTaskQueue; +typedef struct ErtsPortTaskBusyCallerTable_ ErtsPortTaskBusyCallerTable; +typedef struct ErtsPortTaskHandleList_ ErtsPortTaskHandleList; typedef struct { Port *next; - int in_runq; - ErtsPortTaskQueue *taskq; - ErtsPortTaskQueue *exe_taskq; + struct { + struct { + struct { + ErtsPortTask *first; + ErtsPortTask *last; + ErtsPortTaskBusyCallerTable *table; + ErtsPortTaskHandleList *nosuspend; + } busy; + ErtsPortTask *first; + } local; + struct { + ErtsPortTask *first; + ErtsPortTask *last; + } in; + ErtsPortTaskBusyPortQ *bpq; + } taskq; + erts_smp_atomic32_t flags; +#ifdef ERTS_SMP + erts_mtx_t mtx; +#endif } ErtsPortTaskSched; ERTS_GLB_INLINE void erts_port_task_handle_init(ErtsPortTaskHandle *pthp); ERTS_GLB_INLINE int erts_port_task_is_scheduled(ErtsPortTaskHandle *pthp); -ERTS_GLB_INLINE void erts_port_task_init_sched(ErtsPortTaskSched *ptsp); +ERTS_GLB_INLINE void erts_port_task_pre_init_sched(ErtsPortTaskSched *ptsp, + ErtsPortTaskBusyPortQ *bpq); +ERTS_GLB_INLINE void erts_port_task_init_sched(ErtsPortTaskSched *ptsp, + Eterm id); +ERTS_GLB_INLINE void erts_port_task_fini_sched(ErtsPortTaskSched *ptsp); +ERTS_GLB_INLINE void erts_port_task_sched_lock(ErtsPortTaskSched *ptsp); +ERTS_GLB_INLINE void erts_port_task_sched_unlock(ErtsPortTaskSched *ptsp); +ERTS_GLB_INLINE int erts_port_task_sched_lock_is_locked(ErtsPortTaskSched *ptsp); + #ifdef ERTS_INCLUDE_SCHEDULER_INTERNALS ERTS_GLB_INLINE int erts_port_task_have_outstanding_io_tasks(void); #endif @@ -88,13 +154,75 @@ erts_port_task_is_scheduled(ErtsPortTaskHandle *pthp) return ((void *) erts_smp_atomic_read_nob(pthp)) != NULL; } +ERTS_GLB_INLINE void erts_port_task_pre_init_sched(ErtsPortTaskSched *ptsp, + ErtsPortTaskBusyPortQ *bpq) +{ + if (bpq) { + erts_aint_t low = (erts_aint_t) ERTS_PORT_TASK_DEFAULT_BUSY_PORT_Q_LOW; + erts_smp_atomic_init_nob(&bpq->low, low); + bpq->high = (ErlDrvSizeT) ERTS_PORT_TASK_DEFAULT_BUSY_PORT_Q_HIGH; + erts_smp_atomic_init_nob(&bpq->size, (erts_aint_t) 0); + } + ptsp->taskq.bpq = bpq; +} + ERTS_GLB_INLINE void -erts_port_task_init_sched(ErtsPortTaskSched *ptsp) +erts_port_task_init_sched(ErtsPortTaskSched *ptsp, Eterm instr_id) { +#ifdef ERTS_SMP + char *lock_str = "port_sched_lock"; +#endif ptsp->next = NULL; - ptsp->in_runq = 0; - ptsp->taskq = NULL; - ptsp->exe_taskq = NULL; + ptsp->taskq.local.busy.first = NULL; + ptsp->taskq.local.busy.last = NULL; + ptsp->taskq.local.busy.table = NULL; + ptsp->taskq.local.busy.nosuspend = NULL; + ptsp->taskq.local.first = NULL; + ptsp->taskq.in.first = NULL; + ptsp->taskq.in.last = NULL; + erts_smp_atomic32_init_nob(&ptsp->flags, 0); +#ifdef ERTS_SMP +#ifdef ERTS_ENABLE_LOCK_COUNT + if (!(erts_lcnt_rt_options & ERTS_LCNT_OPT_PORTLOCK)) + lock_str = NULL; +#endif + erts_mtx_init_x(&ptsp->mtx, lock_str, instr_id); +#endif +} + +ERTS_GLB_INLINE void +erts_port_task_sched_lock(ErtsPortTaskSched *ptsp) +{ +#ifdef ERTS_SMP + erts_mtx_lock(&ptsp->mtx); +#endif +} + +ERTS_GLB_INLINE void +erts_port_task_sched_unlock(ErtsPortTaskSched *ptsp) +{ +#ifdef ERTS_SMP + erts_mtx_unlock(&ptsp->mtx); +#endif +} + +ERTS_GLB_INLINE int +erts_port_task_sched_lock_is_locked(ErtsPortTaskSched *ptsp) +{ +#if defined(ERTS_SMP) && defined(ERTS_ENABLE_LOCK_CHECK) + return erts_lc_mtx_is_locked(&ptsp->mtx); +#else + return 0; +#endif +} + + +ERTS_GLB_INLINE void +erts_port_task_fini_sched(ErtsPortTaskSched *ptsp) +{ +#ifdef ERTS_SMP + erts_mtx_destroy(&ptsp->mtx); +#endif } #ifdef ERTS_INCLUDE_SCHEDULER_INTERNALS @@ -115,14 +243,16 @@ int erts_port_task_execute(ErtsRunQueue *, Port **); void erts_port_task_init(void); #endif -int erts_port_task_abort(Eterm id, ErtsPortTaskHandle *); +int erts_port_task_abort(ErtsPortTaskHandle *); +void erts_port_task_abort_nosuspend_tasks(Port *); + int erts_port_task_schedule(Eterm, ErtsPortTaskHandle *, ErtsPortTaskType, - ErlDrvEvent, - ErlDrvEventData); + ...); void erts_port_task_free_port(Port *); int erts_port_is_scheduled(Port *); +ErtsProc2PortSigData *erts_port_task_alloc_p2p_sig_data(void); #ifdef ERTS_SMP void erts_enqueue_port(ErtsRunQueue *rq, Port *pp); diff --git a/erts/emulator/beam/erl_process.c b/erts/emulator/beam/erl_process.c index 61554780c4..aaca4b5f59 100644 --- a/erts/emulator/beam/erl_process.c +++ b/erts/emulator/beam/erl_process.c @@ -42,6 +42,7 @@ #include "erl_thr_queue.h" #include "erl_async.h" #include "dtrace-wrapper.h" +#include "erl_ptab.h" #define ERTS_DELAYED_WAKEUP_INFINITY (~(Uint64) 0) #define ERTS_DELAYED_WAKEUP_REDUCTIONS ((Uint64) CONTEXT_REDS/2) @@ -136,155 +137,16 @@ do { \ #define ERTS_EMPTY_RUNQ_PORTS(RQ) \ (RUNQ_READ_LEN(&(RQ)->ports.info.len) == 0 && (RQ)->misc.start == NULL) +const Process erts_invalid_process = {{ERTS_INVALID_PID}}; + extern BeamInstr beam_apply[]; extern BeamInstr beam_exit[]; extern BeamInstr beam_continue_exit[]; -#ifdef ARCH_32 - -union { - erts_smp_dw_atomic_t pid_data; - char align[ERTS_CACHE_LINE_SIZE]; -} last erts_align_attribute(ERTS_CACHE_LINE_SIZE); - - -static ERTS_INLINE Uint64 -dw_aint_to_uint64(erts_dw_aint_t *dw) -{ -#ifdef ETHR_SU_DW_NAINT_T__ - return (Uint64) dw->dw_sint; -#else - Uint64 res; - res = (Uint64) ((Uint32) dw->sint[ERTS_DW_AINT_HIGH_WORD]); - res <<= 32; - res |= (Uint64) ((Uint32) dw->sint[ERTS_DW_AINT_LOW_WORD]); - return res; -#endif -} - -static void -unint64_to_dw_aint(erts_dw_aint_t *dw, Uint64 val) -{ -#ifdef ETHR_SU_DW_NAINT_T__ - dw->dw_sint = (ETHR_SU_DW_NAINT_T__) val; -#else - dw->sint[ERTS_DW_AINT_LOW_WORD] = (erts_aint_t) (val & 0xffffffff); - dw->sint[ERTS_DW_AINT_HIGH_WORD] = (erts_aint_t) ((val >> 32) & 0xffffffff); -#endif -} - -static ERTS_INLINE void -last_pid_data_init_nob(Uint64 val) -{ - erts_dw_aint_t dw; - unint64_to_dw_aint(&dw, val); - erts_smp_dw_atomic_init_nob(&last.pid_data, &dw); -} - -static ERTS_INLINE void -last_pid_data_set_relb(Uint64 val) -{ - erts_dw_aint_t dw; - unint64_to_dw_aint(&dw, val); - erts_smp_dw_atomic_set_relb(&last.pid_data, &dw); -} - -static ERTS_INLINE Uint64 -last_pid_data_read_nob(void) -{ - erts_dw_aint_t dw; - erts_smp_dw_atomic_read_nob(&last.pid_data, &dw); - return dw_aint_to_uint64(&dw); -} - -static ERTS_INLINE Uint64 -last_pid_data_read_acqb(void) -{ - erts_dw_aint_t dw; - erts_smp_dw_atomic_read_acqb(&last.pid_data, &dw); - return dw_aint_to_uint64(&dw); -} - -static ERTS_INLINE Uint64 -last_pid_data_cmpxchg_relb(Uint64 new, Uint64 exp) -{ - erts_dw_aint_t dw_new, dw_xchg; - - unint64_to_dw_aint(&dw_new, new); - unint64_to_dw_aint(&dw_xchg, exp); - - if (erts_smp_dw_atomic_cmpxchg_relb(&last.pid_data, &dw_new, &dw_xchg)) - return exp; - else - return dw_aint_to_uint64(&dw_xchg); -} - -#elif defined(ARCH_64) - -union { - erts_smp_atomic_t pid_data; - char align[ERTS_CACHE_LINE_SIZE]; -} last erts_align_attribute(ERTS_CACHE_LINE_SIZE); - -static ERTS_INLINE void -last_pid_data_init_nob(Uint64 val) -{ - erts_smp_atomic_init_nob(&last.pid_data, (erts_aint_t) val); -} - -static ERTS_INLINE void -last_pid_data_set_relb(Uint64 val) -{ - erts_smp_atomic_set_relb(&last.pid_data, (erts_aint_t) val); -} - -static ERTS_INLINE Uint64 -last_pid_data_read_nob(void) -{ - return (Uint64) erts_smp_atomic_read_nob(&last.pid_data); -} - -static ERTS_INLINE Uint64 -last_pid_data_read_acqb(void) -{ - return (Uint64) erts_smp_atomic_read_acqb(&last.pid_data); -} - -static ERTS_INLINE Uint64 -last_pid_data_cmpxchg_relb(Uint64 new, Uint64 exp) -{ - return (Uint64) erts_smp_atomic_cmpxchg_relb(&last.pid_data, - (erts_aint_t) new, - (erts_aint_t) exp); -} - -#else -# error "Not 64-bit, nor 32-bit architecture..." -#endif - -static ERTS_INLINE int -last_pid_data_cmp(Uint64 lpd1, Uint64 lpd2) -{ - Uint64 lpd1_wrap; - - if (lpd1 == lpd2) - return 0; - - lpd1_wrap = lpd1 + (((Uint64) 1) << 63); - - if (lpd1 < lpd1_wrap) - return (lpd1 < lpd2 && lpd2 < lpd1_wrap) ? -1 : 1; - else - return (lpd1_wrap <= lpd2 && lpd2 < lpd1) ? 1 : -1; -} - - -#define ERTS_PID_DATA_MASK__ ((1 << _PID_DATA_SIZE) - 1) - int erts_sched_compact_load; Uint erts_no_schedulers; -ErtsProcTab erts_proc erts_align_attribute(ERTS_CACHE_LINE_SIZE); +ErtsPTab erts_proc erts_align_attribute(ERTS_CACHE_LINE_SIZE); int erts_sched_thread_suggested_stack_size = -1; @@ -371,8 +233,6 @@ erts_sched_stat_t erts_sched_stat; static erts_tsd_key_t sched_data_key; #endif -erts_smp_rwmtx_t erts_proc_tab_rwmtx; - static erts_smp_atomic32_t function_calls; #ifdef ERTS_SMP @@ -409,29 +269,6 @@ struct erts_system_profile_flags_t erts_system_profile_flags; #if ERTS_MAX_PROCESSES > 0x7fffffff #error "Need to store process_count in another type" #endif -static erts_smp_atomic32_t process_count; - -typedef struct ErtsTermProcElement_ ErtsTermProcElement; -struct ErtsTermProcElement_ { - ErtsTermProcElement *next; - ErtsTermProcElement *prev; - int ix; - union { - struct { - Eterm pid; - Uint64 spawned; - Uint64 exited; - } process; - struct { - Uint64 interval; - } bif_invocation; - } u; -}; - -static struct { - ErtsTermProcElement *start; - ErtsTermProcElement *end; -} saved_term_procs; ERTS_SCHED_PREF_QUICK_ALLOC_IMPL(misc_op_list, ErtsMiscOpList, @@ -493,8 +330,6 @@ do { \ * Local functions. */ -static void init_processes_bif(void); -static void save_terminating_process(Process *p); static void exec_misc_ops(ErtsRunQueue *); static void print_function_from_pc(int to, void *to_arg, BeamInstr* x); static int stack_element_dump(int to, void *to_arg, Process* p, Eterm* sp, @@ -562,39 +397,17 @@ erts_smp_lc_runq_is_locked(ErtsRunQueue *runq) } #endif -static erts_interval_t *proc_interval; - -static void -proc_interval_init(void) -{ - proc_interval = erts_alloc_permanent_cache_aligned( - ERTS_ALC_T_PROC_INTERVAL, - sizeof(erts_interval_t)); - erts_smp_interval_init(proc_interval); -} - -static ERTS_INLINE Uint64 -get_proc_interval(void) -{ - return erts_smp_current_interval_nob(proc_interval); -} static ERTS_INLINE Uint64 ensure_later_proc_interval(Uint64 interval) { - return erts_smp_ensure_later_interval_nob(proc_interval, interval); -} - -static ERTS_INLINE Uint64 -step_proc_interval(void) -{ - return erts_smp_step_interval_nob(proc_interval); + return erts_smp_ensure_later_interval_nob(erts_ptab_interval(&erts_proc), interval); } Uint64 erts_get_proc_interval(void) { - return get_proc_interval(); + return erts_smp_current_interval_nob(erts_ptab_interval(&erts_proc)); } Uint64 @@ -606,7 +419,7 @@ erts_ensure_later_proc_interval(Uint64 interval) Uint64 erts_step_proc_interval(void) { - return step_proc_interval(); + return erts_smp_step_interval_nob(erts_ptab_interval(&erts_proc)); } void @@ -654,20 +467,18 @@ erts_pre_init_process(void) #endif } +#ifdef ERTS_SMP +static void +release_process(void *vproc) +{ + erts_smp_proc_dec_refc((Process *) vproc); +} +#endif + /* initialize the scheduler */ void -erts_init_process(int ncpu) +erts_init_process(int ncpu, int proc_tab_size) { - int proc_tab_sz; - int max_proc_bits; - int proc_bits = ERTS_PROC_BITS; - erts_smp_atomic_t *proc_entry; - char *proc_tab_end; - erts_smp_rwmtx_opt_t proc_tab_rwmtx_opts = ERTS_SMP_RWMTX_OPT_DEFAULT_INITER; - proc_tab_rwmtx_opts.type = ERTS_SMP_RWMTX_TYPE_EXTREMELY_FREQUENT_READ; - proc_tab_rwmtx_opts.lived = ERTS_SMP_RWMTX_LONG_LIVED; - - proc_interval_init(); #ifdef ERTS_SMP erts_disable_proc_not_running_opt = 0; @@ -676,52 +487,16 @@ erts_init_process(int ncpu) init_proclist_alloc(); - erts_smp_atomic32_init_nob(&process_count, 0); - - if (erts_use_r9_pids_ports) - proc_bits = ERTS_R9_PROC_BITS; - - if (erts_proc.max > (1 << proc_bits)) - erts_proc.max = 1 << proc_bits; - - proc_tab_sz = ERTS_ALC_CACHE_LINE_ALIGN_SIZE(erts_proc.max - * sizeof(erts_smp_atomic_t)); - erts_proc.tab = erts_alloc(ERTS_ALC_T_PROC_TABLE, proc_tab_sz); - proc_tab_end = ((char *) erts_proc.tab) + proc_tab_sz; - proc_entry = erts_proc.tab; - while (proc_tab_end > ((char *) proc_entry)) { - erts_smp_atomic_init_nob(proc_entry, ERTS_AINT_NULL); - proc_entry++; - } - - erts_smp_rwmtx_init_opt(&erts_proc_tab_rwmtx, - &proc_tab_rwmtx_opts, - "proc_tab"); - last_pid_data_init_nob(~((Uint64) 0)); - - max_proc_bits = erts_fit_in_bits_int32((Sint32) erts_proc.max - 1); - - erts_proc.tab_cache_lines = proc_tab_sz/ERTS_CACHE_LINE_SIZE; - erts_proc.pix_per_cache_line = ERTS_CACHE_LINE_SIZE/sizeof(erts_smp_atomic_t); - if ((erts_proc.max & (erts_proc.max - 1)) - | (erts_proc.pix_per_cache_line & (erts_proc.pix_per_cache_line - 1))) { - /* - * erts_proc.max or erts_proc.pix_per_cache_line - * not a power of 2 :( - */ - erts_proc.pix_cl_mask = 0; - erts_proc.pix_cl_shift = 0; - erts_proc.pix_cli_mask = 0; - erts_proc.pix_cli_shift = 0; - } - else { - ASSERT((erts_proc.tab_cache_lines - & (erts_proc.tab_cache_lines - 1)) == 0); - erts_proc.pix_cl_mask = erts_proc.tab_cache_lines-1; - erts_proc.pix_cl_shift = erts_fit_in_bits_int32(erts_proc.pix_per_cache_line-1); - erts_proc.pix_cli_shift = erts_fit_in_bits_int32(erts_proc.pix_cl_mask); - erts_proc.pix_cli_mask = (1 << (max_proc_bits - erts_proc.pix_cli_shift)) - 1; - } + erts_ptab_init_table(&erts_proc, + ERTS_ALC_T_PROC_TABLE, +#ifdef ERTS_SMP + release_process, +#else + NULL, +#endif + (ErtsPTabElementCommon *) &erts_invalid_process.common, + proc_tab_size, + "process_table"); last_reductions = 0; last_exact_reductions = 0; @@ -732,7 +507,6 @@ void erts_late_init_process(void) { int ix; - init_processes_bif(); erts_smp_spinlock_init(&erts_sched_stat.lock, "sched_stat"); for (ix = 0; ix < ERTS_NO_PRIO_LEVELS; ix++) { @@ -980,9 +754,9 @@ static ERTS_INLINE ErtsProcList * proclist_create(Process *p) { ErtsProcList *plp = proclist_alloc(); - ensure_later_proc_interval(p->started_interval); - plp->pid = p->id; - plp->started_interval = p->started_interval; + ensure_later_proc_interval(p->common.u.alive.started_interval); + plp->pid = p->common.id; + plp->started_interval = p->common.u.alive.started_interval; return plp; } @@ -992,12 +766,6 @@ proclist_destroy(ErtsProcList *plp) proclist_free(plp); } -static ERTS_INLINE int -proclist_same(ErtsProcList *plp, Process *p) -{ - return plp->pid == p->id && plp->started_interval == p->started_interval; -} - ErtsProcList * erts_proclist_create(Process *p) { @@ -1010,12 +778,6 @@ erts_proclist_destroy(ErtsProcList *plp) proclist_destroy(plp); } -int -erts_proclist_same(ErtsProcList *plp, Process *p) -{ - return proclist_same(plp, p); -} - void * erts_psd_set_init(Process *p, ErtsProcLocks plocks, int ix, void *data) { @@ -1776,37 +1538,32 @@ handle_reap_ports(ErtsAuxWorkData *awdp, erts_aint32_t aux_work, int waiting) unset_aux_work_flags(awdp->ssi, ERTS_SSI_AUX_WORK_REAP_PORTS); awdp->esdp->run_queue->halt_in_progress = 1; if (erts_smp_atomic32_dec_read_acqb(&erts_halt_progress) == 0) { - int i; + int i, max = erts_ptab_max(&erts_port); erts_smp_atomic32_set_nob(&erts_halt_progress, 1); - for (i = 0; i < erts_max_ports; i++) { - Port *prt = &erts_port[i]; - erts_smp_port_state_lock(prt); - if ((prt->status & (ERTS_PORT_SFLGS_INVALID_DRIVER_LOOKUP - | ERTS_PORT_SFLG_HALT))) { - erts_smp_port_state_unlock(prt); - continue; - } - /* We need to set the halt flag - get the port lock */ -#ifdef ERTS_SMP - erts_smp_atomic_inc_nob(&prt->refc); -#endif - erts_smp_port_state_unlock(prt); -#ifdef ERTS_SMP - erts_smp_mtx_lock(prt->lock); -#endif - if ((prt->status & (ERTS_PORT_SFLGS_INVALID_DRIVER_LOOKUP - | ERTS_PORT_SFLG_HALT))) { - erts_port_release(prt); + for (i = 0; i < max; i++) { + erts_aint32_t state; + Port *prt = erts_pix2port(i); + if (!prt) continue; - } - erts_port_status_bor_set(prt, ERTS_PORT_SFLG_HALT); - erts_smp_atomic32_inc_nob(&erts_halt_progress); - if (prt->status & (ERTS_PORT_SFLG_EXITING - | ERTS_PORT_SFLG_CLOSING)) { - erts_port_release(prt); + state = erts_atomic32_read_acqb(&prt->state); + if (state & (ERTS_PORT_SFLGS_INVALID_DRIVER_LOOKUP + | ERTS_PORT_SFLG_HALT)) continue; + + /* We need to set the halt flag - get the port lock */ + + erts_smp_port_lock(prt); + + state = erts_atomic32_read_nob(&prt->state); + if (!(state & (ERTS_PORT_SFLGS_INVALID_DRIVER_LOOKUP + | ERTS_PORT_SFLG_HALT))) { + state = erts_atomic32_read_bor_relb(&prt->state, + ERTS_PORT_SFLG_HALT); + erts_smp_atomic32_inc_nob(&erts_halt_progress); + if (!(state & (ERTS_PORT_SFLG_EXITING|ERTS_PORT_SFLG_CLOSING))) + erts_deliver_port_exit(prt, prt->common.id, am_killed, 0); } - erts_do_exit_port(prt, prt->id, am_killed); + erts_port_release(prt); } if (erts_smp_atomic32_dec_read_nob(&erts_halt_progress) == 0) { @@ -3183,12 +2940,12 @@ resume_run_queue(ErtsRunQueue *rq) erts_smp_runq_lock(rq); - (void) ERTS_RUNQ_FLGS_MASK_SET(rq, - (ERTS_RUNQ_FLG_OUT_OF_WORK - | ERTS_RUNQ_FLG_HALFTIME_OUT_OF_WORK - | ERTS_RUNQ_FLG_SUSPENDED), - (ERTS_RUNQ_FLG_OUT_OF_WORK - | ERTS_RUNQ_FLG_HALFTIME_OUT_OF_WORK)); + (void) ERTS_RUNQ_FLGS_READ_BSET(rq, + (ERTS_RUNQ_FLG_OUT_OF_WORK + | ERTS_RUNQ_FLG_HALFTIME_OUT_OF_WORK + | ERTS_RUNQ_FLG_SUSPENDED), + (ERTS_RUNQ_FLG_OUT_OF_WORK + | ERTS_RUNQ_FLG_HALFTIME_OUT_OF_WORK)); rq->check_balance_reds = ERTS_RUNQ_CALL_CHECK_BALANCE_REDS; for (pix = 0; pix < ERTS_NO_PROC_PRIO_LEVELS; pix++) { @@ -4210,7 +3967,7 @@ erts_fprintf(stderr, "--------------------------------\n"); ERTS_DBG_CHK_FULL_REDS_HISTORY(rq); rq->out_of_work_count = 0; - (void) ERTS_RUNQ_FLGS_MASK_SET(rq, ERTS_RUNQ_FLGS_MIGRATION_INFO, flags); + (void) ERTS_RUNQ_FLGS_READ_BSET(rq, ERTS_RUNQ_FLGS_MIGRATION_INFO, flags); rq->max_len = rq->len; for (pix = 0; pix < ERTS_NO_PRIO_LEVELS; pix++) { @@ -4896,6 +4653,11 @@ erts_init_scheduling(int no_schedulers, int no_schedulers_online) erts_smp_atomic32_init_relb(&erts_halt_progress, -1); erts_halt_code = 0; + +#if !defined(ERTS_SMP) && defined(ERTS_ENABLE_LOCK_CHECK) + erts_lc_set_thread_name("scheduler 1"); +#endif + } ErtsRunQueue * @@ -5573,8 +5335,7 @@ erts_block_multi_scheduling(Process *p, ErtsProcLocks plocks, int on, int all) else if (on) { /* ------ BLOCK ------ */ if (schdlr_sspnd.msb.procs) { plp = proclist_create(p); - plp->next = schdlr_sspnd.msb.procs; - schdlr_sspnd.msb.procs = plp; + erts_proclist_store_last(&schdlr_sspnd.msb.procs, plp); p->flags |= F_HAVE_BLCKD_MSCHED; ASSERT(erts_smp_atomic32_read_nob(&schdlr_sspnd.active) == 1); ASSERT(p->scheduler_data->no == 1); @@ -5659,8 +5420,7 @@ erts_block_multi_scheduling(Process *p, ErtsProcLocks plocks, int on, int all) ~ERTS_SCHDLR_SSPND_CHNG_WAITER); } plp = proclist_create(p); - plp->next = schdlr_sspnd.msb.procs; - schdlr_sspnd.msb.procs = plp; + erts_proclist_store_last(&schdlr_sspnd.msb.procs, plp); ASSERT(p->scheduler_data); } } @@ -5671,20 +5431,16 @@ erts_block_multi_scheduling(Process *p, ErtsProcLocks plocks, int on, int all) } else { /* ------ UNBLOCK ------ */ if (p->flags & F_HAVE_BLCKD_MSCHED) { - ErtsProcList **plpp = &schdlr_sspnd.msb.procs; - plp = schdlr_sspnd.msb.procs; + ErtsProcList *plp = erts_proclist_peek_first(schdlr_sspnd.msb.procs); while (plp) { - if (!proclist_same(plp, p)){ - plpp = &plp->next; - plp = plp->next; - } - else { - *plpp = plp->next; - proclist_destroy(plp); + ErtsProcList *tmp_plp = plp; + plp = erts_proclist_peek_next(schdlr_sspnd.msb.procs, plp); + if (erts_proclist_same(tmp_plp, p)) { + erts_proclist_remove(&schdlr_sspnd.msb.procs, tmp_plp); + proclist_destroy(tmp_plp); if (!all) break; - plp = *plpp; } } } @@ -5753,23 +5509,25 @@ erts_multi_scheduling_blockers(Process *p) Eterm res = NIL; erts_smp_mtx_lock(&schdlr_sspnd.mtx); - if (schdlr_sspnd.msb.procs) { + if (!erts_proclist_is_empty(schdlr_sspnd.msb.procs)) { Eterm *hp, *hp_end; ErtsProcList *plp1, *plp2; - Uint max_size; - ASSERT(schdlr_sspnd.msb.procs); - for (max_size = 0, plp1 = schdlr_sspnd.msb.procs; + Uint max_size = 0; + + for (plp1 = erts_proclist_peek_first(schdlr_sspnd.msb.procs); plp1; - plp1 = plp1->next) { + plp1 = erts_proclist_peek_next(schdlr_sspnd.msb.procs, plp1)) { max_size += 2; } ASSERT(max_size); hp = HAlloc(p, max_size); hp_end = hp + max_size; - for (plp1 = schdlr_sspnd.msb.procs; plp1; plp1 = plp1->next) { - for (plp2 = schdlr_sspnd.msb.procs; + for (plp1 = erts_proclist_peek_first(schdlr_sspnd.msb.procs); + plp1; + plp1 = erts_proclist_peek_next(schdlr_sspnd.msb.procs, plp1)) { + for (plp2 = erts_proclist_peek_first(schdlr_sspnd.msb.procs); plp2->pid != plp1->pid; - plp2 = plp2->next); + plp2 = erts_proclist_peek_next(schdlr_sspnd.msb.procs, plp2)); if (plp2 == plp1) { res = CONS(hp, plp1->pid, res); hp += 2; @@ -6021,7 +5779,7 @@ handle_pend_sync_suspend(Process *suspendee, ASSERT(is_nil(suspender->suspendee)); if (suspendee_alive) { erts_suspend(suspendee, suspendee_locks, NULL); - suspender->suspendee = suspendee->id; + suspender->suspendee = suspendee->common.id; } /* suspender is suspended waiting for suspendee to suspend; resume suspender */ @@ -6042,7 +5800,7 @@ pid2proc_not_running(Process *c_p, ErtsProcLocks c_p_locks, ERTS_SMP_LC_ASSERT(c_p_locks & ERTS_PROC_LOCK_MAIN); ERTS_SMP_LC_ASSERT(pid_locks & (ERTS_PROC_LOCK_MAIN|ERTS_PROC_LOCK_STATUS)); - if (c_p->id == pid) + if (c_p->common.id == pid) return erts_pid2proc(c_p, c_p_locks, pid, pid_locks); if (c_p_locks & ERTS_PROC_LOCK_STATUS) @@ -6094,7 +5852,7 @@ pid2proc_not_running(Process *c_p, ErtsProcLocks c_p_locks, */ if (!c_p->pending_suspenders) { /* Mark rp pending for suspend by c_p */ - add_pend_suspend(rp, c_p->id, handle_pend_sync_suspend); + add_pend_suspend(rp, c_p->common.id, handle_pend_sync_suspend); ASSERT(is_nil(c_p->suspendee)); /* Suspend c_p; when rp is suspended c_p will be resumed. */ @@ -6200,20 +5958,20 @@ handle_pend_bif_sync_suspend(Process *suspendee, ASSERT(is_nil(suspender->suspendee)); if (!suspendee_alive) erts_delete_suspend_monitor(&suspender->suspend_monitors, - suspendee->id); + suspendee->common.id); else { #ifdef DEBUG int res; #endif ErtsSuspendMonitor *smon; smon = erts_lookup_suspend_monitor(suspender->suspend_monitors, - suspendee->id); + suspendee->common.id); #ifdef DEBUG res = #endif do_bif_suspend_process(suspendee, smon, suspendee); ASSERT(!smon || res != 0); - suspender->suspendee = suspendee->id; + suspender->suspendee = suspendee->common.id; } /* suspender is suspended waiting for suspendee to suspend; resume suspender */ @@ -6242,14 +6000,14 @@ handle_pend_bif_async_suspend(Process *suspendee, ASSERT(is_nil(suspender->suspendee)); if (!suspendee_alive) erts_delete_suspend_monitor(&suspender->suspend_monitors, - suspendee->id); + suspendee->common.id); else { #ifdef DEBUG int res; #endif ErtsSuspendMonitor *smon; smon = erts_lookup_suspend_monitor(suspender->suspend_monitors, - suspendee->id); + suspendee->common.id); #ifdef DEBUG res = #endif @@ -6294,7 +6052,7 @@ suspend_process_2(BIF_ALIST_2) int unless_suspending = 0; - if (BIF_P->id == BIF_ARG_1) + if (BIF_P->common.id == BIF_ARG_1) goto badarg; /* We are not allowed to suspend ourselves */ if (is_not_nil(BIF_ARG_2)) { @@ -6390,7 +6148,7 @@ suspend_process_2(BIF_ALIST_2) if (!do_bif_suspend_process(BIF_P, smon, suspendee)) add_pend_suspend(suspendee, - BIF_P->id, + BIF_P->common.id, handle_pend_bif_async_suspend); res = am_true; @@ -6453,7 +6211,7 @@ suspend_process_2(BIF_ALIST_2) else { /* Mark suspendee pending for suspend by BIF_P */ add_pend_suspend(suspendee, - BIF_P->id, + BIF_P->common.id, handle_pend_bif_sync_suspend); ASSERT(is_nil(BIF_P->suspendee)); @@ -6527,7 +6285,7 @@ resume_process_1(BIF_ALIST_1) Process *suspendee; int is_active; - if (BIF_P->id == BIF_ARG_1) + if (BIF_P->common.id == BIF_ARG_1) BIF_ERROR(BIF_P, BADARG); erts_smp_proc_lock(BIF_P, ERTS_PROC_LOCK_LINK); @@ -6652,7 +6410,8 @@ erts_process_status(Process *c_p, ErtsProcLocks c_p_locks, for (i = 0; i < erts_no_schedulers; i++) { esdp = ERTS_SCHEDULER_IX(i); erts_smp_runq_lock(esdp->run_queue); - if (esdp->free_process && esdp->free_process->id == rpid) { + if (esdp->free_process + && esdp->free_process->common.id == rpid) { res = am_free; erts_smp_runq_unlock(esdp->run_queue); break; @@ -6673,27 +6432,31 @@ erts_process_status(Process *c_p, ErtsProcLocks c_p_locks, void erts_suspend(Process* c_p, ErtsProcLocks c_p_locks, Port *busy_port) { -#ifdef DEBUG - int res; -#endif + int suspend; + ASSERT(c_p == erts_get_current_process()); ERTS_SMP_LC_ASSERT(c_p_locks == erts_proc_lc_my_proc_locks(c_p)); if (!(c_p_locks & ERTS_PROC_LOCK_STATUS)) erts_smp_proc_lock(c_p, ERTS_PROC_LOCK_STATUS); + if (busy_port) + suspend = erts_save_suspend_process_on_port(busy_port, c_p); + else + suspend = 1; + + if (suspend) { #ifdef DEBUG - res = + int res = #endif - suspend_process(c_p, c_p); - - ASSERT(res); - - if (busy_port) - erts_wake_process_later(busy_port, c_p); + suspend_process(c_p, c_p); + ASSERT(res); + } if (!(c_p_locks & ERTS_PROC_LOCK_STATUS)) erts_smp_proc_unlock(c_p, ERTS_PROC_LOCK_STATUS); + if (suspend && busy_port && erts_system_monitor_flags.busy_port) + monitor_generic(c_p, am_busy_port, busy_port->common.id); } void @@ -6708,16 +6471,19 @@ erts_resume(Process* process, ErtsProcLocks process_locks) } int -erts_resume_processes(ErtsProcList *plp) +erts_resume_processes(ErtsProcList *list) { + /* 'list' is expected to have been fetched (i.e. not a ring anymore) */ int nresumed = 0; + ErtsProcList *plp = list; + while (plp) { Process *proc; ErtsProcList *fplp; ASSERT(is_internal_pid(plp->pid)); proc = erts_pid2proc(NULL, 0, plp->pid, ERTS_PROC_LOCK_STATUS); if (proc) { - if (proclist_same(plp, proc)) { + if (erts_proclist_same(plp, proc)) { resume_process(proc); nresumed++; } @@ -6870,9 +6636,8 @@ Process *schedule(Process *p, int calls) state = erts_smp_atomic32_read_acqb(&p->state); if (IS_TRACED(p)) { - if (IS_TRACED_FL(p, F_TRACE_CALLS) && !(state & ERTS_PSFLG_FREE)) { + if (IS_TRACED_FL(p, F_TRACE_CALLS) && !(state & ERTS_PSFLG_FREE)) erts_schedule_time_break(p, ERTS_BP_CALL_TIME_SCHEDULE_OUT); - } if (state & (ERTS_PSFLG_FREE|ERTS_PSFLG_EXITING)) { if (ARE_TRACE_FLAGS_ON(p, F_TRACE_SCHED_EXIT)) trace_sched(p, ((state & ERTS_PSFLG_FREE) @@ -6885,7 +6650,7 @@ Process *schedule(Process *p, int calls) else if (ARE_TRACE_FLAGS_ON(p, F_TRACE_SCHED_PROCS)) trace_virtual_sched(p, am_out); } - } + } #ifdef ERTS_SMP if (state & ERTS_PSFLG_PENDING_EXIT) @@ -6949,9 +6714,8 @@ Process *schedule(Process *p, int calls) #ifdef ERTS_SMP { ErtsProcList *pnd_xtrs = rq->procs.pending_exiters; - rq->procs.pending_exiters = NULL; - - if (pnd_xtrs) { + if (erts_proclist_fetch(&pnd_xtrs, NULL)) { + rq->procs.pending_exiters = NULL; erts_smp_runq_unlock(rq); handle_pending_exiters(pnd_xtrs); erts_smp_runq_lock(rq); @@ -7247,7 +7011,7 @@ Process *schedule(Process *p, int calls) erts_smp_proc_unlock(p, ERTS_PROC_LOCK_STATUS); #ifdef ERTS_SMP - if (is_not_nil(p->tracer_proc)) + if (is_not_nil(ERTS_TRACER_PROC(p))) erts_check_my_tracer_proc(p); #endif @@ -7450,70 +7214,6 @@ erts_get_exact_total_reductions(Process *c_p, Uint *redsp, Uint *diffp) erts_smp_proc_lock(c_p, ERTS_PROC_LOCK_MAIN); } -/* - * erts_test_next_pid() is only used for testing. - */ -Sint -erts_test_next_pid(int set, Uint next) -{ - Uint64 lpd; - Sint res; - Eterm pid_data; - int first_pix = -1; - - erts_smp_rwmtx_rwlock(&erts_proc_tab_rwmtx); - - if (!set) - lpd = last_pid_data_read_nob(); - else { - - lpd = (Uint64) next; - pid_data = (Eterm) (lpd & ERTS_PID_DATA_MASK__); - if (ERTS_INVALID_PID == make_internal_pid(pid_data)) { - lpd += erts_proc.max; - ASSERT(erts_pid_data2ix(pid_data) - == erts_pid_data2ix(lpd & ERTS_PID_DATA_MASK__)); - } - last_pid_data_set_relb(lpd); - } - - while (1) { - int pix; - lpd++; - pix = (int) (lpd % erts_proc.max); - if (first_pix < 0) - first_pix = pix; - else if (pix == first_pix) { - res = -1; - break; - } - if (ERTS_AINT_NULL == erts_smp_atomic_read_nob(&erts_proc.tab[pix])) { - pid_data = (Eterm) (lpd & ERTS_PID_DATA_MASK__); - if (ERTS_INVALID_PID == make_internal_pid(pid_data)) { - lpd += erts_proc.max; - ASSERT(erts_pid_data2ix(pid_data) - == erts_pid_data2ix(lpd & ERTS_PID_DATA_MASK__)); - } - res = lpd & ERTS_PID_DATA_MASK__; - break; - } - } - - erts_smp_rwmtx_rwunlock(&erts_proc_tab_rwmtx); - - return res; - -} - -Uint erts_process_count(void) -{ - erts_aint32_t res = erts_smp_atomic32_read_nob(&process_count); - if (res > erts_proc.max) - return erts_proc.max; - ASSERT(res >= 0); - return (Uint) res; -} - void erts_free_proc(Process *p) { @@ -7523,132 +7223,66 @@ erts_free_proc(Process *p) erts_free(ERTS_ALC_T_PROC, (void *) p); } +typedef struct { + Process *proc; + erts_aint32_t state; + ErtsRunQueue *run_queue; +} ErtsEarlyProcInit; + +static void early_init_process_struct(void *varg, Eterm data) +{ + ErtsEarlyProcInit *arg = (ErtsEarlyProcInit *) varg; + Process *proc = arg->proc; + + proc->common.id = make_internal_pid(data); + erts_smp_atomic32_init_relb(&proc->state, arg->state); + +#ifdef ERTS_SMP + RUNQ_SET_RQ(&proc->run_queue, arg->run_queue); + + erts_proc_lock_init(proc); /* All locks locked */ +#endif + +} + /* ** Allocate process and find out where to place next process. */ static Process* alloc_process(ErtsRunQueue *rq, erts_aint32_t state) { - int pix; - Process* p; - Uint64 lpd, exp_lpd; - Eterm pid_data; - erts_aint32_t proc_count; -#ifdef DEBUG - Eterm pid; -#endif - - erts_smp_rwmtx_rlock(&erts_proc_tab_rwmtx); - - proc_count = erts_smp_atomic32_inc_read_acqb(&process_count); - if (proc_count > erts_proc.max) { - while (1) { - erts_aint32_t act_proc_count; - - act_proc_count = erts_smp_atomic32_cmpxchg_relb(&process_count, - proc_count-1, - proc_count); - if (act_proc_count == proc_count) - goto system_limit; - proc_count = act_proc_count; - if (proc_count <= erts_proc.max) - break; - } - } + ErtsEarlyProcInit init_arg; + Process *p; - p = (Process*) erts_alloc_fnf(ERTS_ALC_T_PROC, sizeof(Process)); + p = erts_alloc_fnf(ERTS_ALC_T_PROC, sizeof(Process)); if (!p) - goto enomem; - - p->approx_started = erts_get_approx_time(); - p->started_interval = get_proc_interval(); - - lpd = last_pid_data_read_acqb(); - - /* Reserve slot */ - while (1) { - lpd++; - pix = erts_pid_data2ix((Eterm) (lpd & ERTS_PID_DATA_MASK__)); - if (erts_smp_atomic_read_nob(&erts_proc.tab[pix]) == ERTS_AINT_NULL) { - erts_aint_t val; - val = erts_smp_atomic_cmpxchg_relb(&erts_proc.tab[pix], - ((erts_aint_t) - ERTS_PROC_LOCK_BUSY), - ERTS_AINT_NULL); - - if (ERTS_AINT_NULL == val) - break; - } - } - - pid_data = (Eterm) lpd & ERTS_PID_DATA_MASK__; + return NULL; - p->id = make_internal_pid(pid_data); - if (p->id == ERTS_INVALID_PID) { - /* Do not use the invalid pid; change serial */ - lpd += erts_proc.max; - ASSERT(pix == erts_pid_data2ix((Eterm) (lpd & ERTS_PID_DATA_MASK__))); - pid_data = (Eterm) lpd & ERTS_PID_DATA_MASK__; - p->id = make_internal_pid(pid_data); - ASSERT(p->id != ERTS_INVALID_PID); - } + init_arg.proc = (Process *) p; + init_arg.run_queue = rq; + init_arg.state = state; - exp_lpd = last_pid_data_read_nob(); + ASSERT(((char *) p) == ((char *) &p->common)); - /* Move last pid data forward */ - while (1) { - Uint64 act_lpd; - if (last_pid_data_cmp(lpd, exp_lpd) < 0) - break; - act_lpd = last_pid_data_cmpxchg_relb(lpd, exp_lpd); - if (act_lpd == exp_lpd) - break; - exp_lpd = act_lpd; + if (!erts_ptab_new_element(&erts_proc, + &p->common, + (void *) &init_arg, + early_init_process_struct)) { + erts_free(ERTS_ALC_T_PROC, p); + return NULL; } -#ifdef ERTS_SMP - RUNQ_SET_RQ(&p->run_queue, rq); -#endif - - erts_smp_atomic32_init_relb(&p->state, state); - -#ifdef DEBUG - pid = p->id; -#endif - -#ifdef ERTS_SMP - erts_proc_lock_init(p); /* All locks locked */ -#endif - - /* Move into slot reserved */ -#ifdef DEBUG - ASSERT(ERTS_PROC_LOCK_BUSY - == (Process *) erts_smp_atomic_xchg_relb(&erts_proc.tab[pix], - (erts_aint_t) p)); -#else - erts_smp_atomic_set_relb(&erts_proc.tab[pix], (erts_aint_t) p); -#endif - - ASSERT(internal_pid_serial(p->id) <= (erts_use_r9_pids_ports - ? ERTS_MAX_PID_R9_SERIAL - : ERTS_MAX_PID_SERIAL)); - - erts_smp_rwmtx_runlock(&erts_proc_tab_rwmtx); - + ASSERT(internal_pid_serial(p->common.id) <= ERTS_MAX_PID_SERIAL); + + p->approx_started = erts_get_approx_time(); p->rcount = 0; - ASSERT(p == (Process *) - erts_smp_atomic_read_nob( - &erts_proc.tab[internal_pid_index(pid)])); - - return p; -enomem: -system_limit: - - erts_smp_rwmtx_runlock(&erts_proc_tab_rwmtx); - return NULL; + ASSERT(p == (Process *) (erts_ptab_pix2intptr_nob( + &erts_proc, + internal_pid_index(p->common.id)))); + return p; } Eterm @@ -7795,21 +7429,21 @@ erl_create_process(Process* parent, /* Parent of process (default group leader). p->reds = 0; #ifdef ERTS_SMP - p->u.alive.ptimer = NULL; + p->common.u.alive.ptimer = NULL; #else - sys_memset(&p->u.alive.tm, 0, sizeof(ErlTimer)); + sys_memset(&p->common.u.alive.tm, 0, sizeof(ErlTimer)); #endif - p->reg = NULL; - p->nlinks = NULL; - p->monitors = NULL; + p->common.u.alive.reg = NULL; + ERTS_P_LINKS(p) = NULL; + ERTS_P_MONITORS(p) = NULL; p->nodes_monitors = NULL; p->suspend_monitors = NULL; ASSERT(is_pid(parent->group_leader)); if (parent->group_leader == ERTS_INVALID_PID) - p->group_leader = p->id; + p->group_leader = p->common.id; else { /* Needs to be done after the heap has been set up */ p->group_leader = @@ -7818,18 +7452,18 @@ erl_create_process(Process* parent, /* Parent of process (default group leader). : STORE_NC(&p->htop, &p->off_heap, parent->group_leader); } - erts_get_default_tracing(&p->trace_flags, &p->tracer_proc); + erts_get_default_tracing(&ERTS_TRACE_FLAGS(p), &ERTS_TRACER_PROC(p)); p->msg.first = NULL; p->msg.last = &p->msg.first; p->msg.save = &p->msg.first; p->msg.len = 0; #ifdef ERTS_SMP - p->u.alive.msg_inq.first = NULL; - p->u.alive.msg_inq.last = &p->u.alive.msg_inq.first; - p->u.alive.msg_inq.len = 0; + p->msg_inq.first = NULL; + p->msg_inq.last = &p->msg_inq.first; + p->msg_inq.len = 0; #endif - p->bif_timers = NULL; + p->u.bif_timers = NULL; p->mbuf = NULL; p->mbuf_sz = 0; p->psd = NULL; @@ -7841,7 +7475,9 @@ erl_create_process(Process* parent, /* Parent of process (default group leader). DT_UTAG(p) = NIL; DT_UTAG_FLAGS(p) = 0; #endif - p->parent = parent->id == ERTS_INVALID_PID ? NIL : parent->id; + p->parent = (parent->common.id == ERTS_INVALID_PID + ? NIL + : parent->common.id); INIT_HOLE_CHECK(p); #ifdef DEBUG @@ -7849,18 +7485,19 @@ erl_create_process(Process* parent, /* Parent of process (default group leader). #endif if (IS_TRACED(parent)) { - if (parent->trace_flags & F_TRACE_SOS) { - p->trace_flags |= (parent->trace_flags & TRACEE_FLAGS); - p->tracer_proc = parent->tracer_proc; + if (ERTS_TRACE_FLAGS(parent) & F_TRACE_SOS) { + ERTS_TRACE_FLAGS(p) |= (ERTS_TRACE_FLAGS(parent) & TRACEE_FLAGS); + ERTS_TRACER_PROC(p) = ERTS_TRACER_PROC(parent); } if (ARE_TRACE_FLAGS_ON(parent, F_TRACE_PROCS)) { - trace_proc_spawn(parent, p->id, mod, func, args); + trace_proc_spawn(parent, p->common.id, mod, func, args); } - if (parent->trace_flags & F_TRACE_SOS1) { /* Overrides TRACE_CHILDREN */ - p->trace_flags |= (parent->trace_flags & TRACEE_FLAGS); - p->tracer_proc = parent->tracer_proc; - p->trace_flags &= ~(F_TRACE_SOS1 | F_TRACE_SOS); - parent->trace_flags &= ~(F_TRACE_SOS1 | F_TRACE_SOS); + if (ERTS_TRACE_FLAGS(parent) & F_TRACE_SOS1) { + /* Overrides TRACE_CHILDREN */ + ERTS_TRACE_FLAGS(p) |= (ERTS_TRACE_FLAGS(parent) & TRACEE_FLAGS); + ERTS_TRACER_PROC(p) = ERTS_TRACER_PROC(parent); + ERTS_TRACE_FLAGS(p) &= ~(F_TRACE_SOS1 | F_TRACE_SOS); + ERTS_TRACE_FLAGS(parent) &= ~(F_TRACE_SOS1 | F_TRACE_SOS); } } @@ -7873,27 +7510,27 @@ erl_create_process(Process* parent, /* Parent of process (default group leader). int ret; #endif if (IS_TRACED_FL(parent, F_TRACE_PROCS)) { - trace_proc(parent, parent, am_link, p->id); + trace_proc(parent, parent, am_link, p->common.id); } #ifdef DEBUG - ret = erts_add_link(&(parent->nlinks), LINK_PID, p->id); + ret = erts_add_link(&ERTS_P_LINKS(parent), LINK_PID, p->common.id); ASSERT(ret == 0); - ret = erts_add_link(&(p->nlinks), LINK_PID, parent->id); + ret = erts_add_link(&ERTS_P_LINKS(p), LINK_PID, parent->common.id); ASSERT(ret == 0); #else - erts_add_link(&(parent->nlinks), LINK_PID, p->id); - erts_add_link(&(p->nlinks), LINK_PID, parent->id); + erts_add_link(&ERTS_P_LINKS(parent), LINK_PID, p->common.id); + erts_add_link(&ERTS_P_LINKS(p), LINK_PID, parent->common.id); #endif if (IS_TRACED(parent)) { - if (parent->trace_flags & (F_TRACE_SOL|F_TRACE_SOL1)) { - p->trace_flags |= (parent->trace_flags & TRACEE_FLAGS); - p->tracer_proc = parent->tracer_proc; /* maybe steal */ + if (ERTS_TRACE_FLAGS(parent) & (F_TRACE_SOL|F_TRACE_SOL1)) { + ERTS_TRACE_FLAGS(p) |= (ERTS_TRACE_FLAGS(parent)&TRACEE_FLAGS); + ERTS_TRACER_PROC(p) = ERTS_TRACER_PROC(parent); /*maybe steal*/ - if (parent->trace_flags & F_TRACE_SOL1) { /* maybe override */ - p ->trace_flags &= ~(F_TRACE_SOL1 | F_TRACE_SOL); - parent->trace_flags &= ~(F_TRACE_SOL1 | F_TRACE_SOL); + if (ERTS_TRACE_FLAGS(parent) & F_TRACE_SOL1) {/*maybe override*/ + ERTS_TRACE_FLAGS(p) &= ~(F_TRACE_SOL1 | F_TRACE_SOL); + ERTS_TRACE_FLAGS(parent) &= ~(F_TRACE_SOL1 | F_TRACE_SOL); } } } @@ -7906,8 +7543,8 @@ erl_create_process(Process* parent, /* Parent of process (default group leader). Eterm mref; mref = erts_make_ref(parent); - erts_add_monitor(&(parent->monitors), MON_ORIGIN, mref, p->id, NIL); - erts_add_monitor(&(p->monitors), MON_TARGET, mref, parent->id, NIL); + erts_add_monitor(&ERTS_P_MONITORS(parent), MON_ORIGIN, mref, p->common.id, NIL); + erts_add_monitor(&ERTS_P_MONITORS(p), MON_TARGET, mref, parent->common.id, NIL); so->mref = mref; } @@ -7915,8 +7552,8 @@ erl_create_process(Process* parent, /* Parent of process (default group leader). p->scheduler_data = NULL; p->suspendee = NIL; p->pending_suspenders = NULL; - p->u.alive.pending_exit.reason = THE_NON_VALUE; - p->u.alive.pending_exit.bp = NULL; + p->pending_exit.reason = THE_NON_VALUE; + p->pending_exit.bp = NULL; #endif #if !defined(NO_FPE_SIGNALS) || defined(HIPE) @@ -7925,7 +7562,7 @@ erl_create_process(Process* parent, /* Parent of process (default group leader). erts_smp_proc_unlock(p, ERTS_PROC_LOCKS_ALL); - res = p->id; + res = p->common.id; /* * Schedule process for execution. @@ -7933,7 +7570,7 @@ erl_create_process(Process* parent, /* Parent of process (default group leader). schedule_process(p, state, 0); - VERBOSE(DEBUG_PROCESSES, ("Created a new process: %T\n",p->id)); + VERBOSE(DEBUG_PROCESSES, ("Created a new process: %T\n",p->common.id)); #ifdef USE_VM_PROBES if (DTRACE_ENABLED(process_spawn)) { @@ -7968,10 +7605,10 @@ void erts_init_empty_process(Process *p) p->min_heap_size = 0; p->min_vheap_size = 0; p->rcount = 0; - p->id = ERTS_INVALID_PID; + p->common.id = ERTS_INVALID_PID; p->reds = 0; - p->tracer_proc = NIL; - p->trace_flags = F_INITIAL_TRACE_FLAGS; + ERTS_TRACER_PROC(p) = NIL; + ERTS_TRACE_FLAGS(p) = F_INITIAL_TRACE_FLAGS; p->group_leader = ERTS_INVALID_PID; p->flags = 0; p->fvalue = NIL; @@ -7984,14 +7621,14 @@ void erts_init_empty_process(Process *p) p->bin_old_vheap = 0; p->bin_vheap_mature = 0; #ifdef ERTS_SMP - p->u.alive.ptimer = NULL; + p->common.u.alive.ptimer = NULL; #else - memset(&(p->u.alive.tm), 0, sizeof(ErlTimer)); + memset(&(p->common.u.alive.tm), 0, sizeof(ErlTimer)); #endif p->next = NULL; p->off_heap.first = NULL; p->off_heap.overhead = 0; - p->reg = NULL; + p->common.u.alive.reg = NULL; p->heap_sz = 0; p->high_water = NULL; p->old_hend = NULL; @@ -8000,15 +7637,15 @@ void erts_init_empty_process(Process *p) p->mbuf = NULL; p->mbuf_sz = 0; p->psd = NULL; - p->monitors = NULL; - p->nlinks = NULL; /* List of links */ + ERTS_P_MONITORS(p) = NULL; + ERTS_P_LINKS(p) = NULL; /* List of links */ p->nodes_monitors = NULL; p->suspend_monitors = NULL; p->msg.first = NULL; p->msg.last = &p->msg.first; p->msg.save = &p->msg.first; p->msg.len = 0; - p->bif_timers = NULL; + p->u.bif_timers = NULL; p->dictionary = NULL; p->seq_trace_clock = 0; p->seq_trace_lastcnt = 0; @@ -8036,7 +7673,7 @@ void erts_init_empty_process(Process *p) p->parent = NIL; p->approx_started = 0; - p->started_interval = 0; + p->common.u.alive.started_interval = 0; #ifdef HIPE hipe_init_process(&p->hipe); @@ -8054,13 +7691,13 @@ void erts_init_empty_process(Process *p) #ifdef ERTS_SMP p->scheduler_data = NULL; - p->u.alive.msg_inq.first = NULL; - p->u.alive.msg_inq.last = &p->u.alive.msg_inq.first; - p->u.alive.msg_inq.len = 0; + p->msg_inq.first = NULL; + p->msg_inq.last = &p->msg_inq.first; + p->msg_inq.len = 0; p->suspendee = NIL; p->pending_suspenders = NULL; - p->u.alive.pending_exit.reason = THE_NON_VALUE; - p->u.alive.pending_exit.bp = NULL; + p->pending_exit.reason = THE_NON_VALUE; + p->pending_exit.bp = NULL; erts_proc_lock_init(p); erts_smp_proc_unlock(p, ERTS_PROC_LOCKS_ALL); RUNQ_SET_RQ(&p->run_queue, ERTS_RUNQ_IX(0)); @@ -8082,25 +7719,25 @@ erts_debug_verify_clean_empty_process(Process* p) ASSERT(p->stop == NULL); ASSERT(p->hend == NULL); ASSERT(p->heap == NULL); - ASSERT(p->id == ERTS_INVALID_PID); - ASSERT(p->tracer_proc == NIL); - ASSERT(p->trace_flags == F_INITIAL_TRACE_FLAGS); + ASSERT(p->common.id == ERTS_INVALID_PID); + ASSERT(ERTS_TRACER_PROC(p) == NIL); + ASSERT(ERTS_TRACE_FLAGS(p) == F_INITIAL_TRACE_FLAGS); ASSERT(p->group_leader == ERTS_INVALID_PID); ASSERT(p->next == NULL); - ASSERT(p->reg == NULL); + ASSERT(p->common.u.alive.reg == NULL); ASSERT(p->heap_sz == 0); ASSERT(p->high_water == NULL); ASSERT(p->old_hend == NULL); ASSERT(p->old_htop == NULL); ASSERT(p->old_heap == NULL); - ASSERT(p->monitors == NULL); - ASSERT(p->nlinks == NULL); + ASSERT(ERTS_P_MONITORS(p) == NULL); + ASSERT(ERTS_P_LINKS(p) == NULL); ASSERT(p->nodes_monitors == NULL); ASSERT(p->suspend_monitors == NULL); ASSERT(p->msg.first == NULL); ASSERT(p->msg.len == 0); - ASSERT(p->bif_timers == NULL); + ASSERT(p->u.bif_timers == NULL); ASSERT(p->dictionary == NULL); ASSERT(p->catches == 0); ASSERT(p->cp == NULL); @@ -8110,12 +7747,12 @@ erts_debug_verify_clean_empty_process(Process* p) ASSERT(p->parent == NIL); #ifdef ERTS_SMP - ASSERT(p->u.alive.msg_inq.first == NULL); - ASSERT(p->u.alive.msg_inq.len == 0); + ASSERT(p->msg_inq.first == NULL); + ASSERT(p->msg_inq.len == 0); ASSERT(p->suspendee == NIL); ASSERT(p->pending_suspenders == NULL); - ASSERT(p->u.alive.pending_exit.reason == THE_NON_VALUE); - ASSERT(p->u.alive.pending_exit.bp == NULL); + ASSERT(p->pending_exit.reason == THE_NON_VALUE); + ASSERT(p->pending_exit.bp == NULL); #endif /* Thing that erts_cleanup_empty_process() cleans up */ @@ -8157,7 +7794,7 @@ delete_process(Process* p) { ErlMessage* mp; - VERBOSE(DEBUG_PROCESSES, ("Removing process: %T\n",p->id)); + VERBOSE(DEBUG_PROCESSES, ("Removing process: %T\n",p->common.id)); /* Cleanup psd */ @@ -8231,8 +7868,6 @@ delete_process(Process* p) mp = next_mp; } - ASSERT(!p->monitors); - ASSERT(!p->nlinks); ASSERT(!p->nodes_monitors); ASSERT(!p->suspend_monitors); @@ -8296,7 +7931,7 @@ void erts_handle_pending_exit(Process *c_p, ErtsProcLocks locks) { ErtsProcLocks xlocks; - ASSERT(is_value(c_p->u.alive.pending_exit.reason)); + ASSERT(is_value(c_p->pending_exit.reason)); ERTS_SMP_LC_ASSERT(erts_proc_lc_my_proc_locks(c_p) == locks); ERTS_SMP_LC_ASSERT(locks & ERTS_PROC_LOCK_MAIN); ERTS_SMP_LC_ASSERT(!((ERTS_PSFLG_EXITING|ERTS_PSFLG_FREE) @@ -8315,10 +7950,10 @@ erts_handle_pending_exit(Process *c_p, ErtsProcLocks locks) set_proc_exiting(c_p, erts_smp_atomic32_read_acqb(&c_p->state), - c_p->u.alive.pending_exit.reason, - c_p->u.alive.pending_exit.bp); - c_p->u.alive.pending_exit.reason = THE_NON_VALUE; - c_p->u.alive.pending_exit.bp = NULL; + c_p->pending_exit.reason, + c_p->pending_exit.bp); + c_p->pending_exit.reason = THE_NON_VALUE; + c_p->pending_exit.bp = NULL; if (xlocks) erts_smp_proc_unlock(c_p, xlocks); @@ -8327,12 +7962,14 @@ erts_handle_pending_exit(Process *c_p, ErtsProcLocks locks) static void handle_pending_exiters(ErtsProcList *pnd_xtrs) { + /* 'list' is expected to have been fetched (i.e. not a ring anymore) */ ErtsProcList *plp = pnd_xtrs; - ErtsProcList *free_plp; + while (plp) { + ErtsProcList *free_plp; Process *p = erts_pid2proc(NULL, 0, plp->pid, ERTS_PROC_LOCKS_ALL); if (p) { - if (proclist_same(plp, p)) { + if (erts_proclist_same(plp, p)) { erts_aint32_t state = erts_smp_atomic32_read_acqb(&p->state); if (!(state & ERTS_PSFLG_RUNNING)) { ASSERT(state & ERTS_PSFLG_PENDING_EXIT); @@ -8361,8 +7998,7 @@ save_pending_exiter(Process *p) erts_smp_runq_lock(rq); - plp->next = rq->procs.pending_exiters; - rq->procs.pending_exiters = plp; + erts_proclist_store_last(&rq->procs.pending_exiters, plp); erts_smp_runq_unlock(rq); wake_scheduler(rq, 1); @@ -8409,7 +8045,7 @@ send_exit_message(Process *to, ErtsProcLocks *to_locksp, hp = bp->mem; mess = copy_struct(exit_term, term_size, &hp, &bp->off_heap); /* the trace token must in this case be updated by the caller */ - seq_trace_output(token, mess, SEQ_TRACE_SEND, to->id, NULL); + seq_trace_output(token, mess, SEQ_TRACE_SEND, to->common.id, NULL); temp_token = copy_struct(token, sz_token, &hp, &bp->off_heap); erts_queue_message(to, to_locksp, bp, mess, temp_token #ifdef USE_VM_PROBES @@ -8539,7 +8175,7 @@ send_exit_signal(Process *c_p, /* current process if and only else if (reason != am_normal || (flags & ERTS_XSIG_FLG_NO_IGN_NORMAL)) { #ifdef ERTS_SMP if (!(state & (ERTS_PSFLG_EXITING|ERTS_PSFLG_PENDING_EXIT))) { - ASSERT(!rp->u.alive.pending_exit.bp); + ASSERT(!rp->pending_exit.bp); if (rp == c_p && (*rp_locks & ERTS_PROC_LOCK_MAIN)) { /* Ensure that all locks on c_p are locked before @@ -8589,7 +8225,7 @@ send_exit_signal(Process *c_p, /* current process if and only set_pending_exit: if (is_immed(rsn)) { - rp->u.alive.pending_exit.reason = rsn; + rp->pending_exit.reason = rsn; } else { Eterm *hp; @@ -8597,11 +8233,11 @@ send_exit_signal(Process *c_p, /* current process if and only ErlHeapFragment *bp = new_message_buffer(sz); hp = &bp->mem[0]; - rp->u.alive.pending_exit.reason = copy_struct(rsn, - sz, - &hp, - &bp->off_heap); - rp->u.alive.pending_exit.bp = bp; + rp->pending_exit.reason = copy_struct(rsn, + sz, + &hp, + &bp->off_heap); + rp->pending_exit.bp = bp; } erts_smp_atomic32_read_bor_relb(&rp->state, ERTS_PSFLG_PENDING_EXIT); @@ -8700,7 +8336,7 @@ static void doit_exit_monitor(ErtsMonitor *mon, void *vpcontext) if (!rp) { goto done; } - rmon = erts_remove_monitor(&(rp->monitors),mon->ref); + rmon = erts_remove_monitor(&ERTS_P_MONITORS(rp), mon->ref); erts_smp_proc_unlock(rp, ERTS_PROC_LOCK_LINK); if (rmon == NULL) { goto done; @@ -8735,7 +8371,7 @@ static void doit_exit_monitor(ErtsMonitor *mon, void *vpcontext) ASSERT(mon->type == MON_TARGET); ASSERT(is_pid(mon->pid) || is_internal_port(mon->pid)); if (is_internal_port(mon->pid)) { - Port *prt = erts_id2port(mon->pid, NULL, 0); + Port *prt = erts_id2port(mon->pid); if (prt == NULL) { goto done; } @@ -8751,13 +8387,13 @@ static void doit_exit_monitor(ErtsMonitor *mon, void *vpcontext) goto done; } UseTmpHeapNoproc(3); - rmon = erts_remove_monitor(&(rp->monitors),mon->ref); + rmon = erts_remove_monitor(&ERTS_P_MONITORS(rp), mon->ref); if (rmon) { erts_destroy_monitor(rmon); watched = (is_atom(mon->name) ? TUPLE2(lhp, mon->name, erts_this_dist_entry->sysname) - : pcontext->p->id); + : pcontext->p->common.id); erts_queue_monitor_message(rp, &rp_locks, mon->ref, am_process, watched, pcontext->reason); } @@ -8822,21 +8458,22 @@ static void doit_exit_link(ErtsLink *lnk, void *vpcontext) switch(lnk->type) { case LINK_PID: if(is_internal_port(item)) { - Port *prt = erts_id2port(item, NULL, 0); - if (prt) { - rlnk = erts_remove_link(&prt->nlinks, p->id); - if (rlnk) - erts_destroy_link(rlnk); - erts_do_exit_port(prt, p->id, reason); - erts_port_release(prt); - } + Port *prt = erts_port_lookup(item, ERTS_PORT_SFLGS_INVALID_LOOKUP); + if (prt) + erts_port_exit(NULL, + (ERTS_PORT_SIG_FLG_FORCE_SCHED + | ERTS_PORT_SIG_FLG_BROKEN_LINK), + prt, + p->common.id, + reason, + NULL); } else if(is_external_port(item)) { erts_dsprintf_buf_t *dsbufp = erts_create_logger_dsbuf(); erts_dsprintf(dsbufp, "Erroneous link between %T and external port %T " "found\n", - p->id, + p->common.id, item); erts_send_error_to_logger_nogl(dsbufp); ASSERT(0); /* It isn't possible to setup such a link... */ @@ -8846,14 +8483,14 @@ static void doit_exit_link(ErtsLink *lnk, void *vpcontext) | ERTS_PROC_LOCKS_XSIG_SEND); rp = erts_pid2proc(NULL, 0, item, rp_locks); if (rp) { - rlnk = erts_remove_link(&(rp->nlinks), p->id); + rlnk = erts_remove_link(&ERTS_P_LINKS(rp), p->common.id); /* If rlnk == NULL, we got unlinked while exiting, i.e., do nothing... */ if (rlnk) { int xres; erts_destroy_link(rlnk); xres = send_exit_signal(NULL, - p->id, + p->common.id, rp, &rp_locks, reason, @@ -8865,7 +8502,7 @@ static void doit_exit_link(ErtsLink *lnk, void *vpcontext) if (xres >= 0 && IS_TRACED_FL(rp, F_TRACE_PROCS)) { /* We didn't exit the process and it is traced */ if (IS_TRACED_FL(rp, F_TRACE_PROCS)) { - trace_proc(p, rp, am_getting_unlinked, p->id); + trace_proc(p, rp, am_getting_unlinked, p->common.id); } } } @@ -8879,12 +8516,12 @@ static void doit_exit_link(ErtsLink *lnk, void *vpcontext) ErtsDSigData dsd; int code; ErtsDistLinkData dld; - erts_remove_dist_link(&dld, p->id, item, dep); + erts_remove_dist_link(&dld, p->common.id, item, dep); erts_smp_proc_lock(p, ERTS_PROC_LOCK_MAIN); code = erts_dsig_prepare(&dsd, dep, p, ERTS_DSP_NO_LOCK, 0); if (code == ERTS_DSIG_PREP_CONNECTED) { - code = erts_dsig_send_exit_tt(&dsd, p->id, item, reason, - SEQ_TRACE_TOKEN(p)); + code = erts_dsig_send_exit_tt(&dsd, p->common.id, item, + reason, SEQ_TRACE_TOKEN(p)); ASSERT(code == ERTS_DSIG_SEND_OK); } erts_smp_proc_unlock(p, ERTS_PROC_LOCK_MAIN); @@ -8899,7 +8536,7 @@ static void doit_exit_link(ErtsLink *lnk, void *vpcontext) /* dist entries have node links in a separate structure to avoid confusion */ erts_smp_de_links_lock(dep); - rlnk = erts_remove_link(&(dep->node_links), p->id); + rlnk = erts_remove_link(&(dep->node_links), p->common.id); erts_smp_de_links_unlock(dep); if (rlnk) erts_destroy_link(rlnk); @@ -8927,15 +8564,6 @@ resume_suspend_monitor(ErtsSuspendMonitor *smon, void *vc_p) erts_destroy_suspend_monitor(smon); } -#ifdef ERTS_SMP -static void -proc_dec_refc(void *vproc) -{ - erts_smp_proc_dec_refc((Process *) vproc); -} -#endif - - /* this function fishishes a process and propagates exit messages - called by process_main when a process dies */ void @@ -8973,10 +8601,10 @@ erts_do_exit_process(Process* p, Eterm reason) state = set_proc_exiting_state(p, erts_smp_atomic32_read_nob(&p->state)); if (state & ERTS_PSFLG_PENDING_EXIT) { /* Process exited before pending exit was received... */ - p->u.alive.pending_exit.reason = THE_NON_VALUE; - if (p->u.alive.pending_exit.bp) { - free_message_buffer(p->u.alive.pending_exit.bp); - p->u.alive.pending_exit.bp = NULL; + p->pending_exit.reason = THE_NON_VALUE; + if (p->pending_exit.bp) { + free_message_buffer(p->pending_exit.bp); + p->pending_exit.bp = NULL; } } @@ -8993,23 +8621,25 @@ erts_do_exit_process(Process* p, Eterm reason) trace_proc(p, p, am_exit, reason); } - erts_trace_check_exiting(p->id); + erts_trace_check_exiting(p->common.id); - ASSERT((p->trace_flags & F_INITIAL_TRACE_FLAGS) == F_INITIAL_TRACE_FLAGS); + ASSERT((ERTS_TRACE_FLAGS(p) & F_INITIAL_TRACE_FLAGS) + == F_INITIAL_TRACE_FLAGS); cancel_timer(p); /* Always cancel timer just in case */ - /* - * The timer of this process can *not* be used anymore. The field used - * for the timer is now used for misc exiting data. - */ - p->u.exit_data = NULL; - - if (p->bif_timers) + if (p->u.bif_timers) erts_cancel_bif_timers(p, ERTS_PROC_LOCKS_ALL); erts_smp_proc_unlock(p, ERTS_PROC_LOCKS_ALL_MINOR); + /* + * The p->u.bif_timers of this process can *not* be used anymore; + * will be overwritten by misc termination data. + */ + p->u.terminate = NULL; + + erts_continue_exit_process(p); } @@ -9081,9 +8711,9 @@ erts_continue_exit_process(Process *p) * The registered name *should* be the last "erlang resource" to * cleanup. */ - if (p->reg) { + if (p->common.u.alive.reg) { (void) erts_unregister_name(p, ERTS_PROC_LOCK_MAIN, NULL, THE_NON_VALUE); - ASSERT(!p->reg); + ASSERT(!p->common.u.alive.reg); } erts_smp_proc_lock(p, ERTS_PROC_LOCKS_ALL_MINOR); @@ -9097,22 +8727,18 @@ erts_continue_exit_process(Process *p) yield_allowed = 0; #endif + /* + * Note! The monitor and link fields will be overwritten + * by erts_ptab_delete_element() below. + */ + mon = ERTS_P_MONITORS(p); + lnk = ERTS_P_LINKS(p); + { - int maybe_save; - int pix; /* Do *not* use erts_get_runq_proc() */ ErtsRunQueue *rq; rq = erts_get_runq_current(ERTS_GET_SCHEDULER_DATA_FROM_PROC(p)); - pix = internal_pid_index(p->id); - - erts_smp_rwmtx_rlock(&erts_proc_tab_rwmtx); - maybe_save = saved_term_procs.end != NULL; - if (maybe_save) { - erts_smp_rwmtx_runlock(&erts_proc_tab_rwmtx); - erts_smp_rwmtx_rwlock(&erts_proc_tab_rwmtx); - } - erts_smp_runq_lock(rq); #ifdef ERTS_SMP @@ -9123,22 +8749,11 @@ erts_continue_exit_process(Process *p) p->scheduler_data->current_process = NULL; p->scheduler_data->free_process = p; #endif - /* Time of death! */ - erts_smp_atomic_set_relb(&erts_proc.tab[pix], ERTS_AINT_NULL); - ASSERT(erts_smp_atomic32_read_nob(&process_count) > 0); - erts_smp_atomic32_dec_relb(&process_count); + /* Time of death! */ + erts_ptab_delete_element(&erts_proc, &p->common); erts_smp_runq_unlock(rq); - - if (!maybe_save) - erts_smp_rwmtx_runlock(&erts_proc_tab_rwmtx); - else { - if (saved_term_procs.end) - save_terminating_process(p); - erts_smp_rwmtx_rwunlock(&erts_proc_tab_rwmtx); - } - } /* @@ -9148,12 +8763,6 @@ erts_continue_exit_process(Process *p) * when the monitors and/or links hit. */ - mon = p->monitors; - p->monitors = NULL; /* to avoid recursive deletion during traversal */ - - lnk = p->nlinks; - p->nlinks = NULL; - { /* Inactivate and notify free */ erts_aint32_t n, e, a = erts_smp_atomic32_read_nob(&p->state); @@ -9197,7 +8806,7 @@ erts_continue_exit_process(Process *p) UseTmpHeap(4,p); hp = &tmp_heap[0]; - exit_tuple = TUPLE3(hp, am_EXIT, p->id, reason); + exit_tuple = TUPLE3(hp, am_EXIT, p->common.id, reason); exit_tuple_sz = size_object(exit_tuple); @@ -9223,9 +8832,6 @@ erts_continue_exit_process(Process *p) delete_process(p); #ifdef ERTS_SMP - erts_schedule_thr_prgr_later_op(proc_dec_refc, - (void *) p, - &p->u.release_data); erts_smp_proc_lock(p, ERTS_PROC_LOCK_MAIN); ERTS_SMP_CHK_HAVE_ONLY_MAIN_PROC_LOCK(p); #endif @@ -9277,9 +8883,9 @@ cancel_timer(Process* p) ERTS_SMP_LC_ASSERT(ERTS_PROC_LOCK_MAIN & erts_proc_lc_my_proc_locks(p)); p->flags &= ~(F_INSLPQUEUE|F_TIMO); #ifdef ERTS_SMP - erts_cancel_smp_ptimer(p->u.alive.ptimer); + erts_cancel_smp_ptimer(p->common.u.alive.ptimer); #else - erts_cancel_timer(&p->u.alive.tm); + erts_cancel_timer(&p->common.u.alive.tm); #endif } @@ -9300,12 +8906,12 @@ set_timer(Process* p, Uint timeout) p->flags &= ~F_TIMO; #ifdef ERTS_SMP - erts_create_smp_ptimer(&p->u.alive.ptimer, - p->id, + erts_create_smp_ptimer(&p->common.u.alive.ptimer, + p->common.id, (ErlTimeoutProc) timeout_proc, timeout); #else - erts_set_timer(&p->u.alive.tm, + erts_set_timer(&p->common.u.alive.tm, (ErlTimeoutProc) timeout_proc, NULL, (void*) p, @@ -9323,7 +8929,7 @@ erts_stack_dump(int to, void *to_arg, Process *p) Eterm* sp; int yreg = -1; - if (p->trace_flags & F_SENSITIVE) { + if (ERTS_TRACE_FLAGS(p) & F_SENSITIVE) { return; } erts_program_counter_info(to, to_arg, p); @@ -9411,1068 +9017,6 @@ stack_element_dump(int to, void *to_arg, Process* p, Eterm* sp, int yreg) return yreg; } -/* * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * *\ - * The processes/0 BIF implementation. * -\* */ - - -#define ERTS_PROCESSES_BIF_TAB_INSPECT_INDICES_PER_RED 25 -#define ERTS_PROCESSES_BIF_TAB_CHUNK_SIZE 1000 -#define ERTS_PROCESSES_BIF_MIN_START_REDS \ - (ERTS_PROCESSES_BIF_TAB_CHUNK_SIZE \ - / ERTS_PROCESSES_BIF_TAB_INSPECT_INDICES_PER_RED) - -#define ERTS_PROCESSES_BIF_TAB_FREE_TERM_PROC_REDS 1 - -#define ERTS_PROCESSES_BIF_INSPECT_TERM_PROC_PER_RED 10 - -#define ERTS_PROCESSES_INSPECT_TERM_PROC_MAX_REDS \ - (ERTS_PROCESSES_BIF_TAB_CHUNK_SIZE \ - / ERTS_PROCESSES_BIF_TAB_INSPECT_INDICES_PER_RED) - - -#define ERTS_PROCESSES_BIF_BUILD_RESULT_CONSES_PER_RED 75 - -#define ERTS_PROCS_DBG_DO_TRACE 0 - -#ifdef DEBUG -# define ERTS_PROCESSES_BIF_DEBUGLEVEL 100 -#else -# define ERTS_PROCESSES_BIF_DEBUGLEVEL 0 -#endif - -#define ERTS_PROCS_DBGLVL_CHK_HALLOC 1 -#define ERTS_PROCS_DBGLVL_CHK_FOUND_PIDS 5 -#define ERTS_PROCS_DBGLVL_CHK_PIDS 10 -#define ERTS_PROCS_DBGLVL_CHK_TERM_PROC_LIST 20 -#define ERTS_PROCS_DBGLVL_CHK_RESLIST 20 - -#if ERTS_PROCESSES_BIF_DEBUGLEVEL == 0 -# define ERTS_PROCS_ASSERT(EXP) -#else -# define ERTS_PROCS_ASSERT(EXP) \ - ((void) ((EXP) \ - ? 1 \ - : (debug_processes_assert_error(#EXP, __FILE__, __LINE__), 0))) -#endif - - -#if ERTS_PROCESSES_BIF_DEBUGLEVEL >= ERTS_PROCS_DBGLVL_CHK_HALLOC -# define ERTS_PROCS_DBG_SAVE_HEAP_ALLOC(PBDP, HP, SZ) \ -do { \ - ERTS_PROCS_ASSERT(!(PBDP)->debug.heap); \ - ERTS_PROCS_ASSERT(!(PBDP)->debug.heap_size); \ - (PBDP)->debug.heap = (HP); \ - (PBDP)->debug.heap_size = (SZ); \ -} while (0) -# define ERTS_PROCS_DBG_VERIFY_HEAP_ALLOC_USED(PBDP, HP) \ -do { \ - ERTS_PROCS_ASSERT((PBDP)->debug.heap); \ - ERTS_PROCS_ASSERT((PBDP)->debug.heap_size); \ - ERTS_PROCS_ASSERT((PBDP)->debug.heap + (PBDP)->debug.heap_size == (HP));\ - (PBDP)->debug.heap = NULL; \ - (PBDP)->debug.heap_size = 0; \ -} while (0) -# define ERTS_PROCS_DBG_HEAP_ALLOC_INIT(PBDP) \ -do { \ - (PBDP)->debug.heap = NULL; \ - (PBDP)->debug.heap_size = 0; \ -} while (0) -#else -# define ERTS_PROCS_DBG_SAVE_HEAP_ALLOC(PBDP, HP, SZ) -# define ERTS_PROCS_DBG_VERIFY_HEAP_ALLOC_USED(PBDP, HP) -# define ERTS_PROCS_DBG_HEAP_ALLOC_INIT(PBDP) -#endif - -#if ERTS_PROCESSES_BIF_DEBUGLEVEL >= ERTS_PROCS_DBGLVL_CHK_RESLIST -# define ERTS_PROCS_DBG_CHK_RESLIST(R) debug_processes_check_res_list((R)) -#else -# define ERTS_PROCS_DBG_CHK_RESLIST(R) -#endif - -#if ERTS_PROCESSES_BIF_DEBUGLEVEL >= ERTS_PROCS_DBGLVL_CHK_PIDS -# define ERTS_PROCS_DBG_SAVE_PIDS(PBDP) debug_processes_save_all_pids((PBDP)) -# define ERTS_PROCS_DBG_VERIFY_PIDS(PBDP) \ -do { \ - if (!(PBDP)->debug.correct_pids_verified) \ - debug_processes_verify_all_pids((PBDP)); \ -} while (0) -# define ERTS_PROCS_DBG_CLEANUP_CHK_PIDS(PBDP) \ -do { \ - if ((PBDP)->debug.correct_pids) { \ - erts_free(ERTS_ALC_T_PROCS_PIDS, \ - (PBDP)->debug.correct_pids); \ - (PBDP)->debug.correct_pids = NULL; \ - } \ -} while(0) -# define ERTS_PROCS_DBG_CHK_PIDS_INIT(PBDP) \ -do { \ - (PBDP)->debug.correct_pids_verified = 0; \ - (PBDP)->debug.correct_pids = NULL; \ -} while (0) -#else -# define ERTS_PROCS_DBG_SAVE_PIDS(PBDP) -# define ERTS_PROCS_DBG_VERIFY_PIDS(PBDP) -# define ERTS_PROCS_DBG_CLEANUP_CHK_PIDS(PBDP) -# define ERTS_PROCS_DBG_CHK_PIDS_INIT(PBDP) -#endif - -#if ERTS_PROCESSES_BIF_DEBUGLEVEL >= ERTS_PROCS_DBGLVL_CHK_FOUND_PIDS -# define ERTS_PROCS_DBG_CHK_PID_FOUND(PBDP, PID, IC) \ - debug_processes_check_found_pid((PBDP), (PID), (IC), 1) -# define ERTS_PROCS_DBG_CHK_PID_NOT_FOUND(PBDP, PID, IC) \ - debug_processes_check_found_pid((PBDP), (PID), (IC), 0) -#else -# define ERTS_PROCS_DBG_CHK_PID_FOUND(PBDP, PID, IC) -# define ERTS_PROCS_DBG_CHK_PID_NOT_FOUND(PBDP, PID, IC) -#endif - -#if ERTS_PROCESSES_BIF_DEBUGLEVEL >= ERTS_PROCS_DBGLVL_CHK_TERM_PROC_LIST -# define ERTS_PROCS_DBG_CHK_TPLIST() \ - debug_processes_check_term_proc_list() -# define ERTS_PROCS_DBG_CHK_FREELIST(FL) \ - debug_processes_check_term_proc_free_list(FL) -#else -# define ERTS_PROCS_DBG_CHK_TPLIST() -# define ERTS_PROCS_DBG_CHK_FREELIST(FL) -#endif - -#if ERTS_PROCESSES_BIF_DEBUGLEVEL == 0 -#if ERTS_PROCS_DBG_DO_TRACE -# define ERTS_PROCS_DBG_INIT(P, PBDP) (PBDP)->debug.caller = (P)->id -# else -# define ERTS_PROCS_DBG_INIT(P, PBDP) -# endif -# define ERTS_PROCS_DBG_CLEANUP(PBDP) -#else -# define ERTS_PROCS_DBG_INIT(P, PBDP) \ -do { \ - (PBDP)->debug.caller = (P)->id; \ - ERTS_PROCS_DBG_HEAP_ALLOC_INIT((PBDP)); \ - ERTS_PROCS_DBG_CHK_PIDS_INIT((PBDP)); \ -} while (0) -# define ERTS_PROCS_DBG_CLEANUP(PBDP) \ -do { \ - ERTS_PROCS_DBG_CLEANUP_CHK_PIDS((PBDP)); \ -} while (0) -#endif - -#if ERTS_PROCS_DBG_DO_TRACE -# define ERTS_PROCS_DBG_TRACE(PID, FUNC, WHAT) \ - erts_fprintf(stderr, "%T %s:%d:%s(): %s\n", \ - (PID), __FILE__, __LINE__, #FUNC, #WHAT) -#else -# define ERTS_PROCS_DBG_TRACE(PID, FUNC, WHAT) -#endif - -static Uint processes_bif_tab_chunks; -static Export processes_trap_export; - -typedef struct { - Uint64 interval; -} ErtsProcessesBifChunkInfo; - -typedef enum { - INITIALIZING, - INSPECTING_TABLE, - INSPECTING_TERMINATED_PROCESSES, - BUILDING_RESULT, - RETURN_RESULT -} ErtsProcessesBifState; - -typedef struct { - ErtsProcessesBifState state; - Eterm caller; - ErtsProcessesBifChunkInfo *chunk; - int tix; - int pid_ix; - int pid_sz; - Eterm *pid; - ErtsTermProcElement *bif_invocation; /* Only used when > 1 chunk */ - -#if ERTS_PROCESSES_BIF_DEBUGLEVEL != 0 || ERTS_PROCS_DBG_DO_TRACE - struct { - Eterm caller; -#if ERTS_PROCESSES_BIF_DEBUGLEVEL >= ERTS_PROCS_DBGLVL_CHK_FOUND_PIDS - Uint64 *pid_started; -#endif -#if ERTS_PROCESSES_BIF_DEBUGLEVEL >= ERTS_PROCS_DBGLVL_CHK_HALLOC - Eterm *heap; - Uint heap_size; -#endif -#if ERTS_PROCESSES_BIF_DEBUGLEVEL >= ERTS_PROCS_DBGLVL_CHK_PIDS - int correct_pids_verified; - Eterm *correct_pids; -#endif - } debug; -#endif - -} ErtsProcessesBifData; - - -#if ERTS_PROCESSES_BIF_DEBUGLEVEL != 0 -static void debug_processes_assert_error(char* expr, char* file, int line); -#endif -#if ERTS_PROCESSES_BIF_DEBUGLEVEL >= ERTS_PROCS_DBGLVL_CHK_RESLIST -static void debug_processes_check_res_list(Eterm list); -#endif -#if ERTS_PROCESSES_BIF_DEBUGLEVEL >= ERTS_PROCS_DBGLVL_CHK_PIDS -static void debug_processes_save_all_pids(ErtsProcessesBifData *pbdp); -static void debug_processes_verify_all_pids(ErtsProcessesBifData *pbdp); -#endif -#if ERTS_PROCESSES_BIF_DEBUGLEVEL >= ERTS_PROCS_DBGLVL_CHK_FOUND_PIDS -static void debug_processes_check_found_pid(ErtsProcessesBifData *pbdp, - Eterm pid, - Uint64 ic, - int pid_should_be_found); -#endif -#if ERTS_PROCESSES_BIF_DEBUGLEVEL >= ERTS_PROCS_DBGLVL_CHK_TERM_PROC_LIST -static void debug_processes_check_term_proc_list(void); -static void debug_processes_check_term_proc_free_list(ErtsTermProcElement *tpep); -#endif - -static void -save_terminating_process(Process *p) -{ - ErtsTermProcElement *tpep = erts_alloc(ERTS_ALC_T_PROCS_TPROC_EL, - sizeof(ErtsTermProcElement)); - ERTS_PROCS_ASSERT(saved_term_procs.start && saved_term_procs.end); - ERTS_SMP_LC_ASSERT(erts_lc_rwmtx_is_rwlocked(&erts_proc_tab_rwmtx)); - - ERTS_PROCS_DBG_CHK_TPLIST(); - - tpep->prev = saved_term_procs.end; - tpep->next = NULL; - tpep->ix = internal_pid_index(p->id); - tpep->u.process.pid = p->id; - tpep->u.process.spawned = p->started_interval; - tpep->u.process.exited = get_proc_interval(); - - saved_term_procs.end->next = tpep; - saved_term_procs.end = tpep; - - ERTS_PROCS_DBG_CHK_TPLIST(); - - ERTS_PROCS_ASSERT(tpep->prev->ix >= 0 - ? (tpep->u.process.exited - >= tpep->prev->u.process.exited) - : (tpep->u.process.exited - >= tpep->prev->u.bif_invocation.interval)); -} - -static void -cleanup_processes_bif_data(Binary *bp) -{ - ErtsProcessesBifData *pbdp = ERTS_MAGIC_BIN_DATA(bp); - - ERTS_PROCS_DBG_TRACE(pbdp->debug.caller, cleanup_processes_bif_data, call); - - if (pbdp->state != INITIALIZING) { - - if (pbdp->chunk) { - erts_free(ERTS_ALC_T_PROCS_CNKINF, pbdp->chunk); - pbdp->chunk = NULL; - } - if (pbdp->pid) { - erts_free(ERTS_ALC_T_PROCS_PIDS, pbdp->pid); - pbdp->pid = NULL; - } - -#if ERTS_PROCESSES_BIF_DEBUGLEVEL >= ERTS_PROCS_DBGLVL_CHK_FOUND_PIDS - if (pbdp->debug.pid_started) { - erts_free(ERTS_ALC_T_PROCS_PIDS, pbdp->debug.pid_started); - pbdp->debug.pid_started = NULL; - } -#endif - - if (pbdp->bif_invocation) { - ErtsTermProcElement *tpep; - - erts_smp_rwmtx_rwlock(&erts_proc_tab_rwmtx); - - ERTS_PROCS_DBG_TRACE(pbdp->debug.caller, - cleanup_processes_bif_data, - term_proc_cleanup); - - tpep = pbdp->bif_invocation; - pbdp->bif_invocation = NULL; - - ERTS_PROCS_DBG_CHK_TPLIST(); - - if (tpep->prev) { - /* - * Only remove this bif invokation when we - * have preceding invokations. - */ - tpep->prev->next = tpep->next; - if (tpep->next) - tpep->next->prev = tpep->prev; - else { - /* - * At the time of writing this branch cannot be - * reached. I don't want to remove this code though - * since it may be possible to reach this line - * in the future if the cleanup order in - * erts_do_exit_process() is changed. The ASSERT(0) - * is only here to make us aware that the reorder - * has happened. /rickard - */ - ASSERT(0); - saved_term_procs.end = tpep->prev; - } - erts_free(ERTS_ALC_T_PROCS_TPROC_EL, tpep); - } - else { - /* - * Free all elements until next bif invokation - * is found. - */ - ERTS_PROCS_ASSERT(saved_term_procs.start == tpep); - do { - ErtsTermProcElement *ftpep = tpep; - tpep = tpep->next; - erts_free(ERTS_ALC_T_PROCS_TPROC_EL, ftpep); - } while (tpep && tpep->ix >= 0); - saved_term_procs.start = tpep; - if (tpep) - tpep->prev = NULL; - else - saved_term_procs.end = NULL; - } - - ERTS_PROCS_DBG_CHK_TPLIST(); - - erts_smp_rwmtx_rwunlock(&erts_proc_tab_rwmtx); - - } - } - - ERTS_PROCS_DBG_TRACE(pbdp->debug.caller, - cleanup_processes_bif_data, - return); - ERTS_PROCS_DBG_CLEANUP(pbdp); -} - -static int -processes_bif_engine(Process *p, Eterm *res_accp, Binary *mbp) -{ - ErtsProcessesBifData *pbdp = ERTS_MAGIC_BIN_DATA(mbp); - int have_reds; - int reds; - int locked = 0; - - do { - switch (pbdp->state) { - case INITIALIZING: - pbdp->chunk = erts_alloc(ERTS_ALC_T_PROCS_CNKINF, - (sizeof(ErtsProcessesBifChunkInfo) - * processes_bif_tab_chunks)); - pbdp->tix = 0; - pbdp->pid_ix = 0; - - erts_smp_rwmtx_rwlock(&erts_proc_tab_rwmtx); - locked = 1; - - ERTS_PROCS_DBG_TRACE(p->id, processes_bif_engine, init); - - pbdp->pid_sz = erts_process_count(); - pbdp->pid = erts_alloc(ERTS_ALC_T_PROCS_PIDS, - sizeof(Eterm)*pbdp->pid_sz); - -#if ERTS_PROCESSES_BIF_DEBUGLEVEL >= ERTS_PROCS_DBGLVL_CHK_FOUND_PIDS - pbdp->debug.pid_started = erts_alloc(ERTS_ALC_T_PROCS_PIDS, - sizeof(Uint64)*pbdp->pid_sz); -#endif - - ERTS_PROCS_DBG_SAVE_PIDS(pbdp); - - if (processes_bif_tab_chunks == 1) - pbdp->bif_invocation = NULL; - else { - /* - * We will have to access the table multiple times - * releasing the table lock in between chunks. - */ - pbdp->bif_invocation = erts_alloc(ERTS_ALC_T_PROCS_TPROC_EL, - sizeof(ErtsTermProcElement)); - pbdp->bif_invocation->ix = -1; - pbdp->bif_invocation->u.bif_invocation.interval - = step_proc_interval(); - ERTS_PROCS_DBG_CHK_TPLIST(); - - pbdp->bif_invocation->next = NULL; - if (saved_term_procs.end) { - pbdp->bif_invocation->prev = saved_term_procs.end; - saved_term_procs.end->next = pbdp->bif_invocation; - ERTS_PROCS_ASSERT(saved_term_procs.start); - } - else { - pbdp->bif_invocation->prev = NULL; - saved_term_procs.start = pbdp->bif_invocation; - } - saved_term_procs.end = pbdp->bif_invocation; - - ERTS_PROCS_DBG_CHK_TPLIST(); - - } - - pbdp->state = INSPECTING_TABLE; - /* Fall through */ - - case INSPECTING_TABLE: { - int ix = pbdp->tix; - int indices = ERTS_PROCESSES_BIF_TAB_CHUNK_SIZE; - int cix = ix / ERTS_PROCESSES_BIF_TAB_CHUNK_SIZE; - int end_ix = ix + indices; - Uint64 *invocation_interval_p; - - invocation_interval_p - = (pbdp->bif_invocation - ? &pbdp->bif_invocation->u.bif_invocation.interval - : NULL); - - ERTS_PROCS_ASSERT(is_nil(*res_accp)); - if (!locked) { - erts_smp_rwmtx_rwlock(&erts_proc_tab_rwmtx); - locked = 1; - } - - ERTS_SMP_LC_ASSERT(erts_lc_rwmtx_is_rwlocked(&erts_proc_tab_rwmtx)); - ERTS_PROCS_DBG_TRACE(p->id, processes_bif_engine, insp_table); - - if (cix != 0) - pbdp->chunk[cix].interval = step_proc_interval(); - else if (pbdp->bif_invocation) - pbdp->chunk[0].interval = *invocation_interval_p; - /* else: interval is irrelevant */ - - if (end_ix >= erts_proc.max) { - ERTS_PROCS_ASSERT(cix+1 == processes_bif_tab_chunks); - end_ix = erts_proc.max; - indices = end_ix - ix; - /* What to do when done with this chunk */ - pbdp->state = (processes_bif_tab_chunks == 1 - ? BUILDING_RESULT - : INSPECTING_TERMINATED_PROCESSES); - } - - for (; ix < end_ix; ix++) { - Process *rp = erts_pix2proc(ix); - if (rp - && (!invocation_interval_p - || rp->started_interval < *invocation_interval_p)) { - ERTS_PROCS_ASSERT(is_internal_pid(rp->id)); - pbdp->pid[pbdp->pid_ix] = rp->id; - -#if ERTS_PROCESSES_BIF_DEBUGLEVEL >= ERTS_PROCS_DBGLVL_CHK_FOUND_PIDS - pbdp->debug.pid_started[pbdp->pid_ix] = rp->started_interval; -#endif - - pbdp->pid_ix++; - ERTS_PROCS_ASSERT(pbdp->pid_ix <= pbdp->pid_sz); - } - } - - pbdp->tix = end_ix; - - erts_smp_rwmtx_rwunlock(&erts_proc_tab_rwmtx); - locked = 0; - - reds = indices/ERTS_PROCESSES_BIF_TAB_INSPECT_INDICES_PER_RED; - BUMP_REDS(p, reds); - - have_reds = ERTS_BIF_REDS_LEFT(p); - - if (have_reds && pbdp->state == INSPECTING_TABLE) { - ix = pbdp->tix; - indices = ERTS_PROCESSES_BIF_TAB_CHUNK_SIZE; - end_ix = ix + indices; - if (end_ix > erts_proc.max) { - end_ix = erts_proc.max; - indices = end_ix - ix; - } - - reds = indices/ERTS_PROCESSES_BIF_TAB_INSPECT_INDICES_PER_RED; - - /* Pretend we have no reds left if we haven't got enough - reductions to complete next chunk */ - if (reds > have_reds) - have_reds = 0; - } - - break; - } - - case INSPECTING_TERMINATED_PROCESSES: { - int i; - int max_reds; - int free_term_procs = 0; - Uint64 invocation_interval; - ErtsTermProcElement *tpep; - ErtsTermProcElement *free_list = NULL; - - tpep = pbdp->bif_invocation; - ERTS_PROCS_ASSERT(tpep); - invocation_interval = tpep->u.bif_invocation.interval; - - max_reds = have_reds = ERTS_BIF_REDS_LEFT(p); - if (max_reds > ERTS_PROCESSES_INSPECT_TERM_PROC_MAX_REDS) - max_reds = ERTS_PROCESSES_INSPECT_TERM_PROC_MAX_REDS; - - reds = 0; - erts_smp_rwmtx_rwlock(&erts_proc_tab_rwmtx); - ERTS_PROCS_DBG_TRACE(p->id, processes_bif_engine, insp_term_procs); - - ERTS_PROCS_DBG_CHK_TPLIST(); - - if (tpep->prev) - tpep->prev->next = tpep->next; - else { - ERTS_PROCS_ASSERT(saved_term_procs.start == tpep); - saved_term_procs.start = tpep->next; - - if (saved_term_procs.start && saved_term_procs.start->ix >= 0) { - free_list = saved_term_procs.start; - free_term_procs = 1; - } - } - - if (tpep->next) - tpep->next->prev = tpep->prev; - else - saved_term_procs.end = tpep->prev; - - tpep = tpep->next; - - i = 0; - while (reds < max_reds && tpep) { - if (tpep->ix < 0) { - if (free_term_procs) { - ERTS_PROCS_ASSERT(free_list); - ERTS_PROCS_ASSERT(tpep->prev); - - tpep->prev->next = NULL; /* end of free_list */ - saved_term_procs.start = tpep; - tpep->prev = NULL; - free_term_procs = 0; - } - } - else { - int cix = tpep->ix/ERTS_PROCESSES_BIF_TAB_CHUNK_SIZE; - Uint64 chunk_interval = pbdp->chunk[cix].interval; - Eterm pid = tpep->u.process.pid; - ERTS_PROCS_ASSERT(is_internal_pid(pid)); - - if (tpep->u.process.spawned < invocation_interval) { - if (tpep->u.process.exited < chunk_interval) { - ERTS_PROCS_DBG_CHK_PID_NOT_FOUND(pbdp, - pid, - tpep->u.process.spawned); - pbdp->pid[pbdp->pid_ix] = pid; -#if ERTS_PROCESSES_BIF_DEBUGLEVEL >= ERTS_PROCS_DBGLVL_CHK_FOUND_PIDS - pbdp->debug.pid_started[pbdp->pid_ix] - = tpep->u.process.spawned; -#endif - pbdp->pid_ix++; - ERTS_PROCS_ASSERT(pbdp->pid_ix <= pbdp->pid_sz); - } - else { - ERTS_PROCS_DBG_CHK_PID_FOUND(pbdp, - pid, - tpep->u.process.spawned); - } - } - else { - ERTS_PROCS_DBG_CHK_PID_NOT_FOUND(pbdp, - pid, - tpep->u.process.spawned); - } - - i++; - if (i == ERTS_PROCESSES_BIF_INSPECT_TERM_PROC_PER_RED) { - reds++; - i = 0; - } - if (free_term_procs) - reds += ERTS_PROCESSES_BIF_TAB_FREE_TERM_PROC_REDS; - } - tpep = tpep->next; - } - - if (free_term_procs) { - ERTS_PROCS_ASSERT(free_list); - saved_term_procs.start = tpep; - if (!tpep) - saved_term_procs.end = NULL; - else { - ERTS_PROCS_ASSERT(tpep->prev); - tpep->prev->next = NULL; /* end of free_list */ - tpep->prev = NULL; - } - } - - if (!tpep) { - /* Done */ - ERTS_PROCS_ASSERT(pbdp->pid_ix == pbdp->pid_sz); - pbdp->state = BUILDING_RESULT; - pbdp->bif_invocation->next = free_list; - free_list = pbdp->bif_invocation; - pbdp->bif_invocation = NULL; - } - else { - /* Link in bif_invocation again where we left off */ - pbdp->bif_invocation->prev = tpep->prev; - pbdp->bif_invocation->next = tpep; - tpep->prev = pbdp->bif_invocation; - if (pbdp->bif_invocation->prev) - pbdp->bif_invocation->prev->next = pbdp->bif_invocation; - else { - ERTS_PROCS_ASSERT(saved_term_procs.start == tpep); - saved_term_procs.start = pbdp->bif_invocation; - } - } - - ERTS_PROCS_DBG_CHK_TPLIST(); - ERTS_PROCS_DBG_CHK_FREELIST(free_list); - erts_smp_rwmtx_rwunlock(&erts_proc_tab_rwmtx); - - /* - * We do the actual free of term proc structures now when we - * have released the table lock instead of when we encountered - * them. This since free() isn't for free and we don't want to - * unnecessarily block other schedulers. - */ - while (free_list) { - tpep = free_list; - free_list = tpep->next; - erts_free(ERTS_ALC_T_PROCS_TPROC_EL, tpep); - } - - have_reds -= reds; - if (have_reds < 0) - have_reds = 0; - BUMP_REDS(p, reds); - break; - } - - case BUILDING_RESULT: { - int conses, ix, min_ix; - Eterm *hp; - Eterm res = *res_accp; - - ERTS_PROCS_DBG_VERIFY_PIDS(pbdp); - ERTS_PROCS_DBG_CHK_RESLIST(res); - - ERTS_PROCS_DBG_TRACE(p->id, processes_bif_engine, begin_build_res); - - have_reds = ERTS_BIF_REDS_LEFT(p); - conses = ERTS_PROCESSES_BIF_BUILD_RESULT_CONSES_PER_RED*have_reds; - min_ix = pbdp->pid_ix - conses; - if (min_ix < 0) { - min_ix = 0; - conses = pbdp->pid_ix; - } - - hp = HAlloc(p, conses*2); - ERTS_PROCS_DBG_SAVE_HEAP_ALLOC(pbdp, hp, conses*2); - - for (ix = pbdp->pid_ix - 1; ix >= min_ix; ix--) { - ERTS_PROCS_ASSERT(is_internal_pid(pbdp->pid[ix])); - res = CONS(hp, pbdp->pid[ix], res); - hp += 2; - } - - ERTS_PROCS_DBG_VERIFY_HEAP_ALLOC_USED(pbdp, hp); - - pbdp->pid_ix = min_ix; - if (min_ix == 0) - pbdp->state = RETURN_RESULT; - else { - pbdp->pid_sz = min_ix; - pbdp->pid = erts_realloc(ERTS_ALC_T_PROCS_PIDS, - pbdp->pid, - sizeof(Eterm)*pbdp->pid_sz); -#if ERTS_PROCESSES_BIF_DEBUGLEVEL >= ERTS_PROCS_DBGLVL_CHK_FOUND_PIDS - pbdp->debug.pid_started = erts_realloc(ERTS_ALC_T_PROCS_PIDS, - pbdp->debug.pid_started, - (sizeof(Uint64) - * pbdp->pid_sz)); -#endif - } - reds = conses/ERTS_PROCESSES_BIF_BUILD_RESULT_CONSES_PER_RED; - BUMP_REDS(p, reds); - have_reds -= reds; - - ERTS_PROCS_DBG_CHK_RESLIST(res); - ERTS_PROCS_DBG_TRACE(p->id, processes_bif_engine, end_build_res); - *res_accp = res; - break; - } - case RETURN_RESULT: - cleanup_processes_bif_data(mbp); - return 1; - - default: - erl_exit(ERTS_ABORT_EXIT, - "erlang:processes/0: Invalid state: %d\n", - (int) pbdp->state); - } - - - } while (have_reds || pbdp->state == RETURN_RESULT); - - return 0; -} - -/* - * processes_trap/2 is a hidden BIF that processes/0 traps to. - */ - -static BIF_RETTYPE processes_trap(BIF_ALIST_2) -{ - Eterm res_acc; - Binary *mbp; - - /* - * This bif cannot be called from erlang code. It can only be - * trapped to from processes/0; therefore, a bad argument - * is a processes/0 internal error. - */ - - ERTS_PROCS_DBG_TRACE(BIF_P->id, processes_trap, call); - ERTS_PROCS_ASSERT(is_nil(BIF_ARG_1) || is_list(BIF_ARG_1)); - - res_acc = BIF_ARG_1; - - ERTS_PROCS_ASSERT(ERTS_TERM_IS_MAGIC_BINARY(BIF_ARG_2)); - - mbp = ((ProcBin *) binary_val(BIF_ARG_2))->val; - - ERTS_PROCS_ASSERT(ERTS_MAGIC_BIN_DESTRUCTOR(mbp) - == cleanup_processes_bif_data); - ERTS_PROCS_ASSERT( - ((ErtsProcessesBifData *) ERTS_MAGIC_BIN_DATA(mbp))->debug.caller - == BIF_P->id); - - if (processes_bif_engine(BIF_P, &res_acc, mbp)) { - ERTS_PROCS_DBG_TRACE(BIF_P->id, processes_trap, return); - BIF_RET(res_acc); - } - else { - ERTS_PROCS_DBG_TRACE(BIF_P->id, processes_trap, trap); - ERTS_BIF_YIELD2(&processes_trap_export, BIF_P, res_acc, BIF_ARG_2); - } -} - - - -/* - * The actual processes/0 BIF. - */ - -BIF_RETTYPE processes_0(BIF_ALIST_0) -{ - /* - * A requirement: The list of pids returned should be a consistent - * snapshot of all processes existing at some point - * in time during the execution of processes/0. Since - * processes might terminate while processes/0 is - * executing, we have to keep track of terminated - * processes and add them to the result. We also - * ignore processes created after processes/0 has - * begun executing. - */ - Eterm res_acc = NIL; - Binary *mbp = erts_create_magic_binary(sizeof(ErtsProcessesBifData), - cleanup_processes_bif_data); - ErtsProcessesBifData *pbdp = ERTS_MAGIC_BIN_DATA(mbp); - - ERTS_PROCS_DBG_TRACE(BIF_P->id, processes_0, call); - pbdp->state = INITIALIZING; - ERTS_PROCS_DBG_INIT(BIF_P, pbdp); - - if (ERTS_BIF_REDS_LEFT(BIF_P) >= ERTS_PROCESSES_BIF_MIN_START_REDS - && processes_bif_engine(BIF_P, &res_acc, mbp)) { - erts_bin_free(mbp); - ERTS_PROCS_DBG_CHK_RESLIST(res_acc); - ERTS_PROCS_DBG_TRACE(BIF_P->id, processes_0, return); - BIF_RET(res_acc); - } - else { - Eterm *hp; - Eterm magic_bin; - ERTS_PROCS_DBG_CHK_RESLIST(res_acc); - hp = HAlloc(BIF_P, PROC_BIN_SIZE); - ERTS_PROCS_DBG_SAVE_HEAP_ALLOC(pbdp, hp, PROC_BIN_SIZE); - magic_bin = erts_mk_magic_binary_term(&hp, &MSO(BIF_P), mbp); - ERTS_PROCS_DBG_VERIFY_HEAP_ALLOC_USED(pbdp, hp); - ERTS_PROCS_DBG_TRACE(BIF_P->id, processes_0, trap); - ERTS_BIF_YIELD2(&processes_trap_export, BIF_P, res_acc, magic_bin); - } -} - -static void -init_processes_bif(void) -{ - saved_term_procs.start = NULL; - saved_term_procs.end = NULL; - processes_bif_tab_chunks = (((erts_proc.max - 1) - / ERTS_PROCESSES_BIF_TAB_CHUNK_SIZE) - + 1); - - /* processes_trap/2 is a hidden BIF that the processes/0 BIF traps to. */ - erts_init_trap_export(&processes_trap_export, am_erlang, am_processes_trap, 2, - &processes_trap); - -} - -/* - * Debug stuff - */ - -#if defined(ERTS_SMP) && defined(ERTS_ENABLE_LOCK_CHECK) -int -erts_dbg_check_halloc_lock(Process *p) -{ - if (ERTS_PROC_LOCK_MAIN & erts_proc_lc_my_proc_locks(p)) - return 1; - if (p->id == ERTS_INVALID_PID) - return 1; - if (p->scheduler_data && p == p->scheduler_data->match_pseudo_process) - return 1; - if (erts_thr_progress_is_blocking()) - return 1; - return 0; -} -#endif - -Eterm -erts_debug_processes(Process *c_p) -{ - /* This is the old processes/0 BIF. */ - int i; - Uint need; - Eterm res; - Eterm* hp; - Process *p; - Eterm *hp_end; - - erts_smp_rwmtx_rwlock(&erts_proc_tab_rwmtx); - - res = NIL; - need = erts_process_count() * 2; - hp = HAlloc(c_p, need); /* we need two heap words for each pid */ - hp_end = hp + need; - - /* make the list by scanning bakward */ - - - for (i = erts_proc.max-1; i >= 0; i--) { - p = erts_pix2proc(i); - if (p) { - res = CONS(hp, p->id, res); - hp += 2; - } - } - - erts_smp_rwmtx_rwunlock(&erts_proc_tab_rwmtx); - - HRelease(c_p, hp_end, hp); - - return res; -} - -Eterm -erts_debug_processes_bif_info(Process *c_p) -{ - ERTS_DECL_AM(processes_bif_info); - Eterm elements[] = { - AM_processes_bif_info, - make_small((Uint) ERTS_PROCESSES_BIF_MIN_START_REDS), - make_small((Uint) processes_bif_tab_chunks), - make_small((Uint) ERTS_PROCESSES_BIF_TAB_CHUNK_SIZE), - make_small((Uint) ERTS_PROCESSES_BIF_TAB_INSPECT_INDICES_PER_RED), - make_small((Uint) ERTS_PROCESSES_BIF_TAB_FREE_TERM_PROC_REDS), - make_small((Uint) ERTS_PROCESSES_BIF_INSPECT_TERM_PROC_PER_RED), - make_small((Uint) ERTS_PROCESSES_INSPECT_TERM_PROC_MAX_REDS), - make_small((Uint) ERTS_PROCESSES_BIF_BUILD_RESULT_CONSES_PER_RED), - make_small((Uint) ERTS_PROCESSES_BIF_DEBUGLEVEL) - }; - Uint sz = 0; - Eterm *hp; - (void) erts_bld_tuplev(NULL, &sz, sizeof(elements)/sizeof(Eterm), elements); - hp = HAlloc(c_p, sz); - return erts_bld_tuplev(&hp, NULL, sizeof(elements)/sizeof(Eterm), elements); -} - -#if ERTS_PROCESSES_BIF_DEBUGLEVEL >= ERTS_PROCS_DBGLVL_CHK_FOUND_PIDS -static void -debug_processes_check_found_pid(ErtsProcessesBifData *pbdp, - Eterm pid, - Uint64 ic, - int pid_should_be_found) -{ - int i; - for (i = 0; i < pbdp->pid_ix; i++) { - if (pbdp->pid[i] == pid && pbdp->debug.pid_started[i] == ic) { - ERTS_PROCS_ASSERT(pid_should_be_found); - return; - } - } - ERTS_PROCS_ASSERT(!pid_should_be_found); -} -#endif - -#if ERTS_PROCESSES_BIF_DEBUGLEVEL >= ERTS_PROCS_DBGLVL_CHK_RESLIST -static void -debug_processes_check_res_list(Eterm list) -{ - while (is_list(list)) { - Eterm* consp = list_val(list); - Eterm hd = CAR(consp); - ERTS_PROCS_ASSERT(is_internal_pid(hd)); - list = CDR(consp); - } - - ERTS_PROCS_ASSERT(is_nil(list)); -} -#endif - -#if ERTS_PROCESSES_BIF_DEBUGLEVEL >= ERTS_PROCS_DBGLVL_CHK_PIDS - -static void -debug_processes_save_all_pids(ErtsProcessesBifData *pbdp) -{ - int ix, tix, cpix; - pbdp->debug.correct_pids_verified = 0; - pbdp->debug.correct_pids = erts_alloc(ERTS_ALC_T_PROCS_PIDS, - sizeof(Eterm)*pbdp->pid_sz); - - for (tix = 0, cpix = 0; tix < erts_proc.max; tix++) { - Process *rp = erts_pix2proc(tix); - if (rp) { - ERTS_PROCS_ASSERT(is_internal_pid(rp->id)); - pbdp->debug.correct_pids[cpix++] = rp->id; - ERTS_PROCS_ASSERT(cpix <= pbdp->pid_sz); - } - } - ERTS_PROCS_ASSERT(cpix == pbdp->pid_sz); - - for (ix = 0; ix < pbdp->pid_sz; ix++) - pbdp->pid[ix] = make_small(ix); -} - -static void -debug_processes_verify_all_pids(ErtsProcessesBifData *pbdp) -{ - int ix, cpix; - - ERTS_PROCS_ASSERT(pbdp->pid_ix == pbdp->pid_sz); - - for (ix = 0; ix < pbdp->pid_sz; ix++) { - int found = 0; - Eterm pid = pbdp->pid[ix]; - ERTS_PROCS_ASSERT(is_internal_pid(pid)); - for (cpix = ix; cpix < pbdp->pid_sz; cpix++) { - if (pbdp->debug.correct_pids[cpix] == pid) { - pbdp->debug.correct_pids[cpix] = NIL; - found = 1; - break; - } - } - if (!found) { - for (cpix = 0; cpix < ix; cpix++) { - if (pbdp->debug.correct_pids[cpix] == pid) { - pbdp->debug.correct_pids[cpix] = NIL; - found = 1; - break; - } - } - } - ERTS_PROCS_ASSERT(found); - } - pbdp->debug.correct_pids_verified = 1; - - erts_free(ERTS_ALC_T_PROCS_PIDS, pbdp->debug.correct_pids); - pbdp->debug.correct_pids = NULL; -} -#endif /* ERTS_PROCESSES_BIF_DEBUGLEVEL >= ERTS_PROCS_DBGLVL_CHK_PIDS */ - -#if ERTS_PROCESSES_BIF_DEBUGLEVEL >= ERTS_PROCS_DBGLVL_CHK_TERM_PROC_LIST -static void -debug_processes_check_term_proc_list(void) -{ - ERTS_SMP_LC_ASSERT(erts_lc_rwmtx_is_rwlocked(&erts_proc_tab_rwmtx)); - if (!saved_term_procs.start) - ERTS_PROCS_ASSERT(!saved_term_procs.end); - else { - Uint64 curr_interval = get_proc_interval(); - Uint64 *prev_x_interval_p = NULL; - ErtsTermProcElement *tpep; - - for (tpep = saved_term_procs.start; tpep; tpep = tpep->next) { - if (!tpep->prev) - ERTS_PROCS_ASSERT(saved_term_procs.start == tpep); - else - ERTS_PROCS_ASSERT(tpep->prev->next == tpep); - if (!tpep->next) - ERTS_PROCS_ASSERT(saved_term_procs.end == tpep); - else - ERTS_PROCS_ASSERT(tpep->next->prev == tpep); - if (tpep->ix < 0) { - Uint64 interval = tpep->u.bif_invocation.interval; - ERTS_PROCS_ASSERT(interval <= curr_interval); - } - else { - Uint64 s_interval = tpep->u.process.spawned; - Uint64 x_interval = tpep->u.process.exited; - - ERTS_PROCS_ASSERT(s_interval <= x_interval); - if (prev_x_interval_p) - ERTS_PROCS_ASSERT(*prev_x_interval_p <= x_interval); - prev_x_interval_p = &tpep->u.process.exited; - ERTS_PROCS_ASSERT(is_internal_pid(tpep->u.process.pid)); - ERTS_PROCS_ASSERT(tpep->ix - == internal_pid_index(tpep->u.process.pid)); - } - } - - } -} - -static void -debug_processes_check_term_proc_free_list(ErtsTermProcElement *free_list) -{ - if (saved_term_procs.start) { - ErtsTermProcElement *ftpep; - ErtsTermProcElement *tpep; - - for (ftpep = free_list; ftpep; ftpep = ftpep->next) { - for (tpep = saved_term_procs.start; tpep; tpep = tpep->next) - ERTS_PROCS_ASSERT(ftpep != tpep); - } - } -} - -#endif - -#if ERTS_PROCESSES_BIF_DEBUGLEVEL != 0 - -static void -debug_processes_assert_error(char* expr, char* file, int line) -{ - fflush(stdout); - erts_fprintf(stderr, "%s:%d: Assertion failed: %s\n", file, line, expr); - fflush(stderr); - abort(); -} - -#endif - -/* *\ - * End of the processes/0 BIF implementation. * -\* * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * */ - /* * A nice system halt closing all open port goes as follows: * 1) This function schedules the aux work ERTS_SSI_AUX_WORK_REAP_PORTS @@ -10499,3 +9043,19 @@ void erl_halt(int code) notify_reap_ports_relb(); } } + +#if defined(ERTS_SMP) && defined(ERTS_ENABLE_LOCK_CHECK) +int +erts_dbg_check_halloc_lock(Process *p) +{ + if (ERTS_PROC_LOCK_MAIN & erts_proc_lc_my_proc_locks(p)) + return 1; + if (p->common.id == ERTS_INVALID_PID) + return 1; + if (p->scheduler_data && p == p->scheduler_data->match_pseudo_process) + return 1; + if (erts_thr_progress_is_blocking()) + return 1; + return 0; +} +#endif diff --git a/erts/emulator/beam/erl_process.h b/erts/emulator/beam/erl_process.h index e789c873fb..6d1032c292 100644 --- a/erts/emulator/beam/erl_process.h +++ b/erts/emulator/beam/erl_process.h @@ -42,6 +42,9 @@ typedef struct process Process; #include "erl_process_lock.h" /* Only pull out important types... */ #undef ERTS_PROCESS_LOCK_ONLY_PROC_LOCK_TYPE__ +#define ERL_PORT_GET_PORT_TYPE_ONLY__ +#include "erl_port.h" +#undef ERL_PORT_GET_PORT_TYPE_ONLY__ #include "erl_vm.h" #include "erl_smp.h" #include "erl_message.h" @@ -66,11 +69,10 @@ typedef struct process Process; #undef ERL_THR_PROGRESS_TSD_TYPE_ONLY struct ErtsNodesMonitor_; -struct port; #define ERTS_MAX_NO_OF_SCHEDULERS 1024 -#define ERTS_DEFAULT_MAX_PROCESSES (1 << 15) +#define ERTS_DEFAULT_MAX_PROCESSES (1 << 18) #define ERTS_HEAP_ALLOC(Type, Size) \ erts_alloc((Type), (Size)) @@ -205,32 +207,10 @@ extern int erts_sched_thread_suggested_stack_size; ((Uint32) erts_smp_atomic32_read_nob(&(RQ)->flags)) #define ERTS_RUNQ_FLGS_GET_MB(RQ) \ ((Uint32) erts_smp_atomic32_read_mb(&(RQ)->flags)) -#define ERTS_RUNQ_FLGS_MASK_SET(RQ, MSK, FLGS) \ - ((Uint32) erts_smp_atomic32_mask_set_relb(&(RQ)->flags, \ - (erts_aint32_t) (MSK), \ - (erts_aint32_t) (FLGS))) - -ERTS_GLB_INLINE erts_aint32_t -erts_smp_atomic32_mask_set_relb(erts_smp_atomic32_t *a32p, - erts_aint32_t mask, - erts_aint32_t set); -#if ERTS_GLB_INLINE_INCL_FUNC_DEF -ERTS_GLB_INLINE erts_aint32_t -erts_smp_atomic32_mask_set_relb(erts_smp_atomic32_t *a32p, - erts_aint32_t mask, - erts_aint32_t set) -{ - erts_aint32_t act = erts_smp_atomic32_read_nob(a32p); - while (1) { - erts_aint32_t exp = act; - erts_aint32_t new = exp & ~mask; - new |= (mask & set); - act = erts_smp_atomic32_cmpxchg_relb(a32p, new, exp); - if (act == exp) - return act; - } -} -#endif +#define ERTS_RUNQ_FLGS_READ_BSET(RQ, MSK, FLGS) \ + ((Uint32) erts_smp_atomic32_read_bset_relb(&(RQ)->flags, \ + (erts_aint32_t) (MSK), \ + (erts_aint32_t) (FLGS))) typedef enum { ERTS_SCHDLR_SSPND_DONE_MSCHED_BLOCKED, @@ -311,6 +291,7 @@ struct ErtsProcList_ { Eterm pid; Uint64 started_interval; ErtsProcList* next; + ErtsProcList* prev; }; typedef struct ErtsMiscOpList_ ErtsMiscOpList; @@ -401,8 +382,8 @@ struct ErtsRunQueue_ { struct { ErtsRunQueueInfo info; - struct port *start; - struct port *end; + Port *start; + Port *end; } ports; }; @@ -512,7 +493,7 @@ struct ErtsSchedulerData_ { ErtsSchedulerSleepInfo *ssi; Process *current_process; Uint no; /* Scheduler number */ - struct port *current_port; + Port *current_port; ErtsRunQueue *run_queue; int virtual_reds; int cpu_id; /* >= 0 when bound */ @@ -737,8 +718,8 @@ struct ErtsPendingSuspend_ { # define BIN_OLD_VHEAP(p) (p)->bin_old_vheap struct process { - Eterm id; /* The pid of this process - (need to be first in struct) */ + ErtsPTabElementCommon common; /* *Need* to be first in struct */ + /* All fields in the PCB that differs between different heap * architectures, have been moved to the end of this struct to * make sure that as few offsets as possible differ. Different @@ -782,12 +763,8 @@ struct process { * Only valid for the current process. */ Uint32 rcount; /* suspend count */ - int prio; /* Priority of process */ int schedule_count; /* Times left to reschedule a low prio process */ Uint reds; /* No of reductions for this process */ - Eterm tracer_proc; /* If proc is traced, this is the tracer - (can NOT be boxed) */ - Uint trace_flags; /* Trace flags (used to be in flags) */ Eterm group_leader; /* Pid in charge (can be boxed) */ Uint flags; /* Trap exit, etc (no trace flags anymore) */ @@ -797,10 +774,6 @@ struct process { Process *next; /* Pointer to next process in run queue */ - struct reg_proc *reg; /* NULL iff not registered */ - ErtsLink *nlinks; - ErtsMonitor *monitors; /* The process monitors, both ends */ - struct ErtsNodesMonitor_ *nodes_monitors; ErtsSuspendMonitor *suspend_monitors; /* Processes suspended by @@ -809,7 +782,10 @@ struct process { ErlMessageQueue msg; /* Message queue */ - ErtsBifTimer *bif_timers; /* Bif timers aiming at this process */ + union { + ErtsBifTimer *bif_timers; /* Bif timers aiming at this process */ + void *terminate; + } u; ProcDict *dictionary; /* Process dictionary, may be NULL */ @@ -834,7 +810,6 @@ struct process { */ Eterm parent; /* Pid of process that created this process. */ erts_approx_time_t approx_started; /* Time when started. */ - Uint64 started_interval; /* This is the place, where all fields that differs between memory * architectures, have gone to. @@ -856,25 +831,11 @@ struct process { Uint64 bin_old_vheap_sz; /* Virtual old heap block size for binaries */ Uint64 bin_old_vheap; /* Virtual old heap size for binaries */ - union { - struct { -#ifdef ERTS_SMP - ErtsSmpPTimer *ptimer; - ErlMessageInQueue msg_inq; - ErtsPendExit pending_exit; -#else - ErlTimer tm; /* Timer entry */ -#endif - } alive; /* when process is alive */ -#ifdef ERTS_SMP - ErtsThrPrgrLaterOp release_data; /* when releasing process struct */ -#endif - void *exit_data; /* Misc data referred during termination */ - } u; - erts_smp_atomic32_t state; /* Process state flags (see ERTS_PSFLG_*) */ #ifdef ERTS_SMP + ErlMessageInQueue msg_inq; + ErtsPendExit pending_exit; erts_proc_lock_t lock; ErtsSchedulerData *scheduler_data; Eterm suspendee; @@ -904,6 +865,8 @@ struct process { #endif }; +extern const Process erts_invalid_process; + #ifdef CHECK_FOR_HOLES # define INIT_HOLE_CHECK(p) \ do { \ @@ -1028,8 +991,6 @@ Eterm* erts_heap_alloc(Process* p, Uint need, Uint xtra); Eterm* erts_set_hole_marker(Eterm* ptr, Uint sz); #endif -extern erts_smp_rwmtx_t erts_proc_tab_rwmtx; -extern erts_smp_atomic_t *erts_proc_tab; extern Uint erts_default_process_flags; extern erts_smp_rwmtx_t erts_cpu_bind_rwmtx; /* If any of the erts_system_monitor_* variables are set (enabled), @@ -1057,10 +1018,6 @@ struct erts_system_profile_flags_t { unsigned int exclusive : 1; }; extern struct erts_system_profile_flags_t erts_system_profile_flags; - -#define IS_TRACED(p) ( (p)->tracer_proc != NIL ) -#define ARE_TRACE_FLAGS_ON(p,tf) ( ((p)->trace_flags & (tf|F_SENSITIVE)) == (tf) ) -#define IS_TRACED_FL(p,tf) ( IS_TRACED(p) && ARE_TRACE_FLAGS_ON(p,tf) ) /* process flags */ #define F_HIBERNATE_SCHED (1 << 0) /* Schedule out after hibernate op */ @@ -1171,7 +1128,172 @@ Uint64 erts_step_proc_interval(void); ErtsProcList *erts_proclist_create(Process *); void erts_proclist_destroy(ErtsProcList *); -int erts_proclist_same(ErtsProcList *, Process *); + +ERTS_GLB_INLINE int erts_proclist_same(ErtsProcList *, Process *); +ERTS_GLB_INLINE void erts_proclist_store_first(ErtsProcList **, ErtsProcList *); +ERTS_GLB_INLINE void erts_proclist_store_last(ErtsProcList **, ErtsProcList *); +ERTS_GLB_INLINE ErtsProcList *erts_proclist_peek_first(ErtsProcList *); +ERTS_GLB_INLINE ErtsProcList *erts_proclist_peek_last(ErtsProcList *); +ERTS_GLB_INLINE ErtsProcList *erts_proclist_peek_next(ErtsProcList *, ErtsProcList *); +ERTS_GLB_INLINE ErtsProcList *erts_proclist_peek_prev(ErtsProcList *, ErtsProcList *); +ERTS_GLB_INLINE ErtsProcList *erts_proclist_fetch_first(ErtsProcList **); +ERTS_GLB_INLINE ErtsProcList *erts_proclist_fetch_last(ErtsProcList **); +ERTS_GLB_INLINE int erts_proclist_fetch(ErtsProcList **, ErtsProcList **); +ERTS_GLB_INLINE void erts_proclist_remove(ErtsProcList **, ErtsProcList *); +ERTS_GLB_INLINE int erts_proclist_is_empty(ErtsProcList *); +ERTS_GLB_INLINE int erts_proclist_is_first(ErtsProcList *, ErtsProcList *); +ERTS_GLB_INLINE int erts_proclist_is_last(ErtsProcList *, ErtsProcList *); + +#if ERTS_GLB_INLINE_INCL_FUNC_DEF + +ERTS_GLB_INLINE int +erts_proclist_same(ErtsProcList *plp, Process *p) +{ + return (plp->pid == p->common.id + && (plp->started_interval + == p->common.u.alive.started_interval)); +} + +ERTS_GLB_INLINE void erts_proclist_store_first(ErtsProcList **list, + ErtsProcList *element) +{ + if (!*list) + element->next = element->prev = element; + else { + element->prev = (*list)->prev; + element->next = *list; + element->prev->next = element; + element->next->prev = element; + } + *list = element; +} + +ERTS_GLB_INLINE void erts_proclist_store_last(ErtsProcList **list, + ErtsProcList *element) +{ + if (!*list) { + element->next = element->prev = element; + *list = element; + } + else { + element->prev = (*list)->prev; + element->next = *list; + element->prev->next = element; + element->next->prev = element; + } +} + +ERTS_GLB_INLINE ErtsProcList *erts_proclist_peek_first(ErtsProcList *list) +{ + return list; +} + +ERTS_GLB_INLINE ErtsProcList *erts_proclist_peek_last(ErtsProcList *list) +{ + if (!list) + return NULL; + else + return list->prev; +} + +ERTS_GLB_INLINE ErtsProcList *erts_proclist_peek_next(ErtsProcList *list, + ErtsProcList *element) +{ + ErtsProcList *next; + ASSERT(list && element); + next = element->next; + return list == next ? NULL : next; +} + +ERTS_GLB_INLINE ErtsProcList *erts_proclist_peek_prev(ErtsProcList *list, + ErtsProcList *element) +{ + ErtsProcList *prev; + ASSERT(list && element); + prev = element->prev; + return list == element ? NULL : prev; +} + +ERTS_GLB_INLINE ErtsProcList *erts_proclist_fetch_first(ErtsProcList **list) +{ + if (!*list) + return NULL; + else { + ErtsProcList *res = *list; + if (res == *list) + *list = NULL; + else + *list = res->next; + res->next->prev = res->prev; + res->prev->next = res->next; + return res; + } +} + +ERTS_GLB_INLINE ErtsProcList *erts_proclist_fetch_last(ErtsProcList **list) +{ + if (!*list) + return NULL; + else { + ErtsProcList *res = (*list)->prev; + if (res == *list) + *list = NULL; + res->next->prev = res->prev; + res->prev->next = res->next; + return res; + } +} + +ERTS_GLB_INLINE int erts_proclist_fetch(ErtsProcList **list_first, + ErtsProcList **list_last) +{ + if (!*list_first) { + if (list_last) + *list_last = NULL; + return 0; + } + else { + if (list_last) + *list_last = (*list_first)->prev; + (*list_first)->prev->next = NULL; + (*list_first)->prev = NULL; + return !0; + } +} + +ERTS_GLB_INLINE void erts_proclist_remove(ErtsProcList **list, + ErtsProcList *element) +{ + ASSERT(list && *list); + if (*list == element) { + *list = element->next; + if (*list == element) + *list = NULL; + } + element->next->prev = element->prev; + element->prev->next = element->next; +} + +ERTS_GLB_INLINE int erts_proclist_is_empty(ErtsProcList *list) +{ + return list == NULL; +} + +ERTS_GLB_INLINE int erts_proclist_is_first(ErtsProcList *list, + ErtsProcList *element) +{ + ASSERT(list && element); + return list == element; +} + +ERTS_GLB_INLINE int erts_proclist_is_last(ErtsProcList *list, + ErtsProcList *element) +{ + ASSERT(list && element); + return list->prev == element; +} + +#endif int erts_sched_set_wakeup_other_thresold(char *str); int erts_sched_set_wakeup_other_type(char *str); @@ -1221,7 +1343,7 @@ void erts_schedule_multi_misc_aux_work(int ignore_self, erts_aint32_t erts_set_aux_work_timeout(int, erts_aint32_t, int); void erts_sched_notify_check_cpu_bind(void); Uint erts_active_schedulers(void); -void erts_init_process(int); +void erts_init_process(int, int); Eterm erts_process_status(Process *, ErtsProcLocks, Process *, Eterm); Uint erts_run_queues_len(Uint *); void erts_add_to_runq(Process *); @@ -1240,7 +1362,6 @@ void set_timer(Process*, Uint); void cancel_timer(Process*); /* Begin System profile */ Uint erts_runnable_process_count(void); -Uint erts_process_count(void); /* End System profile */ void erts_init_empty_process(Process *p); void erts_cleanup_empty_process(Process* p); @@ -1264,7 +1385,7 @@ Eterm erts_sched_stat_term(Process *p, int total); void erts_free_proc(Process *); -void erts_suspend(Process*, ErtsProcLocks, struct port*); +void erts_suspend(Process*, ErtsProcLocks, Port*); void erts_resume(Process*, ErtsProcLocks); int erts_resume_processes(ErtsProcList *); @@ -1289,9 +1410,6 @@ void erts_deep_process_dump(int, void *); Eterm erts_get_reader_groups_map(Process *c_p); Eterm erts_debug_reader_groups_map(Process *c_p, int groups); -Sint erts_test_next_pid(int, Uint); -Eterm erts_debug_processes(Process *c_p); -Eterm erts_debug_processes_bif_info(Process *c_p); Uint erts_debug_nbalance(void); int erts_debug_wait_deallocations(Process *c_p); @@ -1603,7 +1721,7 @@ ERTS_GLB_INLINE Eterm erts_get_current_pid(void) { Process *proc = erts_get_current_process(); - return proc ? proc->id : THE_NON_VALUE; + return proc ? proc->common.id : THE_NON_VALUE; } ERTS_GLB_INLINE @@ -1812,10 +1930,10 @@ extern int erts_disable_proc_not_running_opt; /* Minimum NUMBER of processes for a small system to start */ -#ifdef ERTS_SMP +#define ERTS_MIN_PROCESSES 1024 +#if defined(ERTS_SMP) && ERTS_MIN_PROCESSES < ERTS_NO_OF_PIX_LOCKS +#undef ERTS_MIN_PROCESSES #define ERTS_MIN_PROCESSES ERTS_NO_OF_PIX_LOCKS -#else -#define ERTS_MIN_PROCESSES 16 #endif void erts_smp_notify_inc_runq(ErtsRunQueue *runq); diff --git a/erts/emulator/beam/erl_process_dict.c b/erts/emulator/beam/erl_process_dict.c index 93466da3aa..bf384c66e1 100644 --- a/erts/emulator/beam/erl_process_dict.c +++ b/erts/emulator/beam/erl_process_dict.c @@ -360,7 +360,7 @@ static void pd_hash_erase(Process *p, Eterm id, Eterm *ret) erts_fprintf(stderr, "Process dictionary for process %T is broken, trying to " "display term found in line %d:\n" - "%T\n", p->id, __LINE__, old); + "%T\n", p->common.id, __LINE__, old); #endif erl_exit(1, "Damaged process dictionary found during erase/1."); } @@ -405,7 +405,7 @@ Eterm erts_pd_hash_get(Process *p, Eterm id) erts_fprintf(stderr, "Process dictionary for process %T is broken, trying to " "display term found in line %d:\n" - "%T\n", p->id, __LINE__, tmp); + "%T\n", p->common.id, __LINE__, tmp); #endif erl_exit(1, "Damaged process dictionary found during get/1."); } @@ -614,7 +614,7 @@ static Eterm pd_hash_put(Process *p, Eterm id, Eterm value) erts_fprintf(stderr, "Process dictionary for process %T is broken, trying to " "display term found in line %d:\n" - "%T\n", p->id, __LINE__, old); + "%T\n", p->common.id, __LINE__, old); #endif erl_exit(1, "Damaged process dictionary found during put/2."); diff --git a/erts/emulator/beam/erl_process_dump.c b/erts/emulator/beam/erl_process_dump.c index 542c5ed0d9..ba74dfd6a1 100644 --- a/erts/emulator/beam/erl_process_dump.c +++ b/erts/emulator/beam/erl_process_dump.c @@ -60,11 +60,11 @@ extern BeamInstr beam_continue_exit[]; void erts_deep_process_dump(int to, void *to_arg) { - int i; + int i, max = erts_ptab_max(&erts_proc); all_binaries = NULL; - for (i = 0; i < erts_max_processes; i++) { + for (i = 0; i < max; i++) { Process *p = erts_pix2proc(i); if (p && p->i != ENULL) { erts_aint32_t state = erts_smp_atomic32_read_acqb(&p->state); @@ -85,8 +85,8 @@ dump_process_info(int to, void *to_arg, Process *p) ERTS_SMP_MSGQ_MV_INQ2PRIVQ(p); - if ((p->trace_flags & F_SENSITIVE) == 0 && p->msg.first) { - erts_print(to, to_arg, "=proc_messages:%T\n", p->id); + if ((ERTS_TRACE_FLAGS(p) & F_SENSITIVE) == 0 && p->msg.first) { + erts_print(to, to_arg, "=proc_messages:%T\n", p->common.id); for (mp = p->msg.first; mp != NULL; mp = mp->next) { Eterm mesg = ERL_MESSAGE_TERM(mp); if (is_value(mesg)) @@ -100,21 +100,21 @@ dump_process_info(int to, void *to_arg, Process *p) } } - if ((p->trace_flags & F_SENSITIVE) == 0) { + if ((ERTS_TRACE_FLAGS(p) & F_SENSITIVE) == 0) { if (p->dictionary) { - erts_print(to, to_arg, "=proc_dictionary:%T\n", p->id); + erts_print(to, to_arg, "=proc_dictionary:%T\n", p->common.id); erts_deep_dictionary_dump(to, to_arg, p->dictionary, dump_element_nl); } } - if ((p->trace_flags & F_SENSITIVE) == 0) { - erts_print(to, to_arg, "=proc_stack:%T\n", p->id); + if ((ERTS_TRACE_FLAGS(p) & F_SENSITIVE) == 0) { + erts_print(to, to_arg, "=proc_stack:%T\n", p->common.id); for (sp = p->stop; sp < STACK_START(p); sp++) { yreg = stack_element_dump(to, to_arg, p, sp, yreg); } - erts_print(to, to_arg, "=proc_heap:%T\n", p->id); + erts_print(to, to_arg, "=proc_heap:%T\n", p->common.id); for (sp = p->stop; sp < STACK_START(p); sp++) { Eterm term = *sp; diff --git a/erts/emulator/beam/erl_process_lock.c b/erts/emulator/beam/erl_process_lock.c index 84a8270d06..2db5df06b4 100644 --- a/erts/emulator/beam/erl_process_lock.c +++ b/erts/emulator/beam/erl_process_lock.c @@ -66,8 +66,7 @@ #endif #include "erl_process.h" - -const Process erts_proc_lock_busy = {ERTS_INVALID_PID}; +#include "erl_thr_progress.h" #ifdef ERTS_SMP @@ -399,7 +398,7 @@ wait_for_locks(Process *p, ErtsProcLocks need_locks, ErtsProcLocks olflgs) { - erts_pix_lock_t *pix_lock = pixlck ? pixlck : ERTS_PID2PIXLOCK(p->id); + erts_pix_lock_t *pix_lock = pixlck ? pixlck : ERTS_PID2PIXLOCK(p->common.id); erts_tse_t *wtr; /* Acquire a waiter object on which this thread can wait. */ @@ -553,7 +552,7 @@ erts_proc_unlock_failed(Process *p, erts_pix_lock_t *pixlck, ErtsProcLocks wait_locks) { - erts_pix_lock_t *pix_lock = pixlck ? pixlck : ERTS_PID2PIXLOCK(p->id); + erts_pix_lock_t *pix_lock = pixlck ? pixlck : ERTS_PID2PIXLOCK(p->common.id); #if ERTS_PROC_LOCK_ATOMIC_IMPL erts_pix_lock(pix_lock); @@ -580,7 +579,7 @@ erts_proc_lock_prepare_proc_lock_waiter(void) */ static void -proc_safelock(int is_sched, +proc_safelock(int is_managed, Process *a_proc, ErtsProcLocks a_have_locks, ErtsProcLocks a_need_locks, @@ -603,40 +602,40 @@ proc_safelock(int is_sched, * Locks with the same lock order should be locked on p1 before p2. */ if (a_proc) { - if (a_proc->id < b_proc->id) { + if (a_proc->common.id < b_proc->common.id) { p1 = a_proc; #ifdef ERTS_ENABLE_LOCK_CHECK - pid1 = a_proc->id; + pid1 = a_proc->common.id; #endif need_locks1 = a_need_locks; have_locks1 = a_have_locks; p2 = b_proc; #ifdef ERTS_ENABLE_LOCK_CHECK - pid2 = b_proc->id; + pid2 = b_proc->common.id; #endif need_locks2 = b_need_locks; have_locks2 = b_have_locks; } - else if (a_proc->id > b_proc->id) { + else if (a_proc->common.id > b_proc->common.id) { p1 = b_proc; #ifdef ERTS_ENABLE_LOCK_CHECK - pid1 = b_proc->id; + pid1 = b_proc->common.id; #endif need_locks1 = b_need_locks; have_locks1 = b_have_locks; p2 = a_proc; #ifdef ERTS_ENABLE_LOCK_CHECK - pid2 = a_proc->id; + pid2 = a_proc->common.id; #endif need_locks2 = a_need_locks; have_locks2 = a_have_locks; } else { ERTS_LC_ASSERT(a_proc == b_proc); - ERTS_LC_ASSERT(a_proc->id == b_proc->id); + ERTS_LC_ASSERT(a_proc->common.id == b_proc->common.id); p1 = a_proc; #ifdef ERTS_ENABLE_LOCK_CHECK - pid1 = a_proc->id; + pid1 = a_proc->common.id; #endif need_locks1 = a_need_locks | b_need_locks; have_locks1 = a_have_locks | b_have_locks; @@ -651,7 +650,7 @@ proc_safelock(int is_sched, else { p1 = b_proc; #ifdef ERTS_ENABLE_LOCK_CHECK - pid1 = b_proc->id; + pid1 = b_proc->common.id; #endif need_locks1 = b_need_locks; have_locks1 = b_have_locks; @@ -706,7 +705,7 @@ proc_safelock(int is_sched, if (unlock_locks) { have_locks1 &= ~unlock_locks; need_locks1 |= unlock_locks; - if (!is_sched && !have_locks1) { + if (!is_managed && !have_locks1) { refc1 = 1; erts_smp_proc_inc_refc(p1); } @@ -716,7 +715,7 @@ proc_safelock(int is_sched, if (unlock_locks) { have_locks2 &= ~unlock_locks; need_locks2 |= unlock_locks; - if (!is_sched && !have_locks2) { + if (!is_managed && !have_locks2) { refc2 = 1; erts_smp_proc_inc_refc(p2); } @@ -797,7 +796,7 @@ proc_safelock(int is_sched, } #endif - if (!is_sched) { + if (!is_managed) { if (refc1) erts_smp_proc_dec_refc(p1); if (refc2) @@ -830,7 +829,7 @@ erts_pid2proc_opt(Process *c_p, int flags) { Process *dec_refc_proc = NULL; - int need_ptl; + ErtsThrPrgrDelayHandle dhndl; ErtsProcLocks need_locks; Uint pix; Process *proc; @@ -853,8 +852,8 @@ erts_pid2proc_opt(Process *c_p, ERTS_LC_ASSERT((pid_need_locks & ERTS_PROC_LOCKS_ALL) == pid_need_locks); need_locks = pid_need_locks; - if (c_p && c_p->id == pid) { - ASSERT(c_p->id != ERTS_INVALID_PID); + if (c_p && c_p->common.id == pid) { + ASSERT(c_p->common.id != ERTS_INVALID_PID); ASSERT(c_p == erts_pix2proc(pix)); if (!(flags & ERTS_P2P_FLG_ALLOW_OTHER_X) @@ -868,15 +867,12 @@ erts_pid2proc_opt(Process *c_p, } } - need_ptl = !erts_get_scheduler_id(); - - if (need_ptl) - erts_smp_rwmtx_rwlock(&erts_proc_tab_rwmtx); + dhndl = erts_thr_progress_unmanaged_delay(); - proc = (Process *) erts_smp_atomic_read_ddrb(&erts_proc.tab[pix]); + proc = (Process *) erts_ptab_pix2intptr_ddrb(&erts_proc, pix); if (proc) { - if (proc->id != pid) + if (proc->common.id != pid) proc = NULL; else if (!need_locks) { if (flags & ERTS_P2P_FLG_SMP_INC_REFC) @@ -935,6 +931,7 @@ erts_pid2proc_opt(Process *c_p, if (flags & ERTS_P2P_FLG_TRY_LOCK) proc = ERTS_PROC_LOCK_BUSY; else { + int managed; if (flags & ERTS_P2P_FLG_SMP_INC_REFC) erts_smp_proc_inc_refc(proc); @@ -942,14 +939,21 @@ erts_pid2proc_opt(Process *c_p, erts_lcnt_proc_lock_unaquire(&proc->lock, lcnt_locks); #endif - if (need_ptl) { + managed = dhndl == ERTS_THR_PRGR_DHANDLE_MANAGED; + if (!managed) { erts_smp_proc_inc_refc(proc); + erts_thr_progress_unmanaged_continue(dhndl); dec_refc_proc = proc; - erts_smp_rwmtx_rwunlock(&erts_proc_tab_rwmtx); - need_ptl = 0; + + /* + * We don't want to call + * erts_thr_progress_unmanaged_continue() + * again. + */ + dhndl = ERTS_THR_PRGR_DHANDLE_MANAGED; } - proc_safelock(!need_ptl, + proc_safelock(managed, c_p, c_p_have_locks, c_p_have_locks, @@ -961,8 +965,8 @@ erts_pid2proc_opt(Process *c_p, } } - if (need_ptl) - erts_smp_rwmtx_rwunlock(&erts_proc_tab_rwmtx); + if (dhndl != ERTS_THR_PRGR_DHANDLE_MANAGED) + erts_thr_progress_unmanaged_continue(dhndl); if (need_locks && proc @@ -970,7 +974,7 @@ erts_pid2proc_opt(Process *c_p, && (!(flags & ERTS_P2P_FLG_ALLOW_OTHER_X) ? ERTS_PROC_IS_EXITING(proc) : (proc - != (Process *) erts_smp_atomic_read_nob(&erts_proc.tab[pix])))) { + != (Process *) erts_ptab_pix2intptr_nob(&erts_proc, pix)))) { erts_smp_proc_unlock(proc, need_locks); @@ -1012,22 +1016,22 @@ erts_proc_lock_init(Process *p) erts_proc_lc_trylock(p, ERTS_PROC_LOCKS_ALL, 1); #endif #elif ERTS_PROC_LOCK_RAW_MUTEX_IMPL - erts_mtx_init_x(&p->lock.main, "proc_main", p->id); + erts_mtx_init_x(&p->lock.main, "proc_main", p->common.id); ethr_mutex_lock(&p->lock.main.mtx); #ifdef ERTS_ENABLE_LOCK_CHECK erts_lc_trylock(1, &p->lock.main.lc); #endif - erts_mtx_init_x(&p->lock.link, "proc_link", p->id); + erts_mtx_init_x(&p->lock.link, "proc_link", p->common.id); ethr_mutex_lock(&p->lock.link.mtx); #ifdef ERTS_ENABLE_LOCK_CHECK erts_lc_trylock(1, &p->lock.link.lc); #endif - erts_mtx_init_x(&p->lock.msgq, "proc_msgq", p->id); + erts_mtx_init_x(&p->lock.msgq, "proc_msgq", p->common.id); ethr_mutex_lock(&p->lock.msgq.mtx); #ifdef ERTS_ENABLE_LOCK_CHECK erts_lc_trylock(1, &p->lock.msgq.lc); #endif - erts_mtx_init_x(&p->lock.status, "proc_status", p->id); + erts_mtx_init_x(&p->lock.status, "proc_status", p->common.id); ethr_mutex_lock(&p->lock.status.mtx); #ifdef ERTS_ENABLE_LOCK_CHECK erts_lc_trylock(1, &p->lock.status.lc); @@ -1064,11 +1068,11 @@ erts_proc_lock_fin(Process *p) #if ERTS_PROC_LOCK_OWN_IMPL && defined(ERTS_ENABLE_LOCK_COUNT) void erts_lcnt_proc_lock_init(Process *p) { if (erts_lcnt_rt_options & ERTS_LCNT_OPT_PROCLOCK) { - if (p->id != ERTS_INVALID_PID) { - erts_lcnt_init_lock_x(&(p->lock.lcnt_main), "proc_main", ERTS_LCNT_LT_PROCLOCK, p->id); - erts_lcnt_init_lock_x(&(p->lock.lcnt_msgq), "proc_msgq", ERTS_LCNT_LT_PROCLOCK, p->id); - erts_lcnt_init_lock_x(&(p->lock.lcnt_link), "proc_link", ERTS_LCNT_LT_PROCLOCK, p->id); - erts_lcnt_init_lock_x(&(p->lock.lcnt_status), "proc_status", ERTS_LCNT_LT_PROCLOCK, p->id); + if (p->common.id != ERTS_INVALID_PID) { + erts_lcnt_init_lock_x(&(p->lock.lcnt_main), "proc_main", ERTS_LCNT_LT_PROCLOCK, p->common.id); + erts_lcnt_init_lock_x(&(p->lock.lcnt_msgq), "proc_msgq", ERTS_LCNT_LT_PROCLOCK, p->common.id); + erts_lcnt_init_lock_x(&(p->lock.lcnt_link), "proc_link", ERTS_LCNT_LT_PROCLOCK, p->common.id); + erts_lcnt_init_lock_x(&(p->lock.lcnt_status), "proc_status", ERTS_LCNT_LT_PROCLOCK, p->common.id); } else { erts_lcnt_init_lock(&(p->lock.lcnt_main), "proc_main", ERTS_LCNT_LT_PROCLOCK); erts_lcnt_init_lock(&(p->lock.lcnt_msgq), "proc_msgq", ERTS_LCNT_LT_PROCLOCK); @@ -1176,10 +1180,11 @@ void erts_lcnt_proc_trylock(erts_proc_lock_t *lock, ErtsProcLocks locks, int res } -void erts_lcnt_enable_proc_lock_count(int enable) { - int i; +void erts_lcnt_enable_proc_lock_count(int enable) +{ + int i, max = erts_ptab_max(&erts_proc); - for (i = 0; i < erts_max_processes; ++i) { + for (i = 0; i < max; ++i) { Process* p = erts_pix2proc(i); if (p) { if (enable) { @@ -1208,7 +1213,7 @@ void erts_proc_lc_lock(Process *p, ErtsProcLocks locks) { erts_lc_lock_t lck = ERTS_LC_LOCK_INIT(-1, - p->id, + p->common.id, ERTS_LC_FLG_LT_PROCLOCK); if (locks & ERTS_PROC_LOCK_MAIN) { lck.id = lc_id.proc_lock_main; @@ -1232,7 +1237,7 @@ void erts_proc_lc_trylock(Process *p, ErtsProcLocks locks, int locked) { erts_lc_lock_t lck = ERTS_LC_LOCK_INIT(-1, - p->id, + p->common.id, ERTS_LC_FLG_LT_PROCLOCK); if (locks & ERTS_PROC_LOCK_MAIN) { lck.id = lc_id.proc_lock_main; @@ -1256,7 +1261,7 @@ void erts_proc_lc_unlock(Process *p, ErtsProcLocks locks) { erts_lc_lock_t lck = ERTS_LC_LOCK_INIT(-1, - p->id, + p->common.id, ERTS_LC_FLG_LT_PROCLOCK); if (locks & ERTS_PROC_LOCK_STATUS) { lck.id = lc_id.proc_lock_status; @@ -1283,7 +1288,7 @@ erts_proc_lc_might_unlock(Process *p, ErtsProcLocks locks) { #if ERTS_PROC_LOCK_OWN_IMPL erts_lc_lock_t lck = ERTS_LC_LOCK_INIT(-1, - p->id, + p->common.id, ERTS_LC_FLG_LT_PROCLOCK); if (locks & ERTS_PROC_LOCK_STATUS) { lck.id = lc_id.proc_lock_status; @@ -1318,7 +1323,7 @@ erts_proc_lc_require_lock(Process *p, ErtsProcLocks locks) { #if ERTS_PROC_LOCK_OWN_IMPL erts_lc_lock_t lck = ERTS_LC_LOCK_INIT(-1, - p->id, + p->common.id, ERTS_LC_FLG_LT_PROCLOCK); if (locks & ERTS_PROC_LOCK_MAIN) { lck.id = lc_id.proc_lock_main; @@ -1353,7 +1358,7 @@ erts_proc_lc_unrequire_lock(Process *p, ErtsProcLocks locks) { #if ERTS_PROC_LOCK_OWN_IMPL erts_lc_lock_t lck = ERTS_LC_LOCK_INIT(-1, - p->id, + p->common.id, ERTS_LC_FLG_LT_PROCLOCK); if (locks & ERTS_PROC_LOCK_STATUS) { lck.id = lc_id.proc_lock_status; @@ -1390,7 +1395,7 @@ erts_proc_lc_trylock_force_busy(Process *p, ErtsProcLocks locks) { if (locks & ERTS_PROC_LOCKS_ALL) { erts_lc_lock_t lck = ERTS_LC_LOCK_INIT(-1, - p->id, + p->common.id, ERTS_LC_FLG_LT_PROCLOCK); if (locks & ERTS_PROC_LOCK_MAIN) @@ -1415,7 +1420,7 @@ void erts_proc_lc_chk_only_proc_main(Process *p) { #if ERTS_PROC_LOCK_OWN_IMPL erts_lc_lock_t proc_main = ERTS_LC_LOCK_INIT(lc_id.proc_lock_main, - p->id, + p->common.id, ERTS_LC_FLG_LT_PROCLOCK); erts_lc_check_exact(&proc_main, 1); #elif ERTS_PROC_LOCK_RAW_MUTEX_IMPL @@ -1439,19 +1444,19 @@ erts_proc_lc_chk_have_proc_locks(Process *p, ErtsProcLocks locks) ERTS_PROC_LC_EMPTY_LOCK_INIT}; if (locks & ERTS_PROC_LOCK_MAIN) { have_locks[have_locks_len].id = lc_id.proc_lock_main; - have_locks[have_locks_len++].extra = p->id; + have_locks[have_locks_len++].extra = p->common.id; } if (locks & ERTS_PROC_LOCK_LINK) { have_locks[have_locks_len].id = lc_id.proc_lock_link; - have_locks[have_locks_len++].extra = p->id; + have_locks[have_locks_len++].extra = p->common.id; } if (locks & ERTS_PROC_LOCK_MSGQ) { have_locks[have_locks_len].id = lc_id.proc_lock_msgq; - have_locks[have_locks_len++].extra = p->id; + have_locks[have_locks_len++].extra = p->common.id; } if (locks & ERTS_PROC_LOCK_STATUS) { have_locks[have_locks_len].id = lc_id.proc_lock_status; - have_locks[have_locks_len++].extra = p->id; + have_locks[have_locks_len++].extra = p->common.id; } #elif ERTS_PROC_LOCK_RAW_MUTEX_IMPL erts_lc_lock_t have_locks[4]; @@ -1484,35 +1489,35 @@ erts_proc_lc_chk_proc_locks(Process *p, ErtsProcLocks locks) if (locks & ERTS_PROC_LOCK_MAIN) { have_locks[have_locks_len].id = lc_id.proc_lock_main; - have_locks[have_locks_len++].extra = p->id; + have_locks[have_locks_len++].extra = p->common.id; } else { have_not_locks[have_not_locks_len].id = lc_id.proc_lock_main; - have_not_locks[have_not_locks_len++].extra = p->id; + have_not_locks[have_not_locks_len++].extra = p->common.id; } if (locks & ERTS_PROC_LOCK_LINK) { have_locks[have_locks_len].id = lc_id.proc_lock_link; - have_locks[have_locks_len++].extra = p->id; + have_locks[have_locks_len++].extra = p->common.id; } else { have_not_locks[have_not_locks_len].id = lc_id.proc_lock_link; - have_not_locks[have_not_locks_len++].extra = p->id; + have_not_locks[have_not_locks_len++].extra = p->common.id; } if (locks & ERTS_PROC_LOCK_MSGQ) { have_locks[have_locks_len].id = lc_id.proc_lock_msgq; - have_locks[have_locks_len++].extra = p->id; + have_locks[have_locks_len++].extra = p->common.id; } else { have_not_locks[have_not_locks_len].id = lc_id.proc_lock_msgq; - have_not_locks[have_not_locks_len++].extra = p->id; + have_not_locks[have_not_locks_len++].extra = p->common.id; } if (locks & ERTS_PROC_LOCK_STATUS) { have_locks[have_locks_len].id = lc_id.proc_lock_status; - have_locks[have_locks_len++].extra = p->id; + have_locks[have_locks_len++].extra = p->common.id; } else { have_not_locks[have_not_locks_len].id = lc_id.proc_lock_status; - have_not_locks[have_not_locks_len++].extra = p->id; + have_not_locks[have_not_locks_len++].extra = p->common.id; } #elif ERTS_PROC_LOCK_RAW_MUTEX_IMPL erts_lc_lock_t have_locks[4]; @@ -1547,16 +1552,16 @@ erts_proc_lc_my_proc_locks(Process *p) ErtsProcLocks res = 0; #if ERTS_PROC_LOCK_OWN_IMPL erts_lc_lock_t locks[4] = {ERTS_LC_LOCK_INIT(lc_id.proc_lock_main, - p->id, + p->common.id, ERTS_LC_FLG_LT_PROCLOCK), ERTS_LC_LOCK_INIT(lc_id.proc_lock_link, - p->id, + p->common.id, ERTS_LC_FLG_LT_PROCLOCK), ERTS_LC_LOCK_INIT(lc_id.proc_lock_msgq, - p->id, + p->common.id, ERTS_LC_FLG_LT_PROCLOCK), ERTS_LC_LOCK_INIT(lc_id.proc_lock_status, - p->id, + p->common.id, ERTS_LC_FLG_LT_PROCLOCK)}; #elif ERTS_PROC_LOCK_RAW_MUTEX_IMPL erts_lc_lock_t locks[4] = {p->lock.main.lc, diff --git a/erts/emulator/beam/erl_process_lock.h b/erts/emulator/beam/erl_process_lock.h index 4aec19c8c3..9dd503f3cb 100644 --- a/erts/emulator/beam/erl_process_lock.h +++ b/erts/emulator/beam/erl_process_lock.h @@ -176,8 +176,8 @@ typedef struct erts_proc_lock_t_ { * on multiple processes, locks on processes with low process ids * have to be locked before locks on processes with high process * ids. E.g., if the main and the message queue locks are to be - * locked on processes p1 and p2 and p1->id < p2->id, then locks - * should be locked in the following order: + * locked on processes p1 and p2 and p1->common.id < p2->common.id, + * then locks should be locked in the following order: * 1. main lock on p1 * 2. main lock on p2 * 3. message queue lock on p1 @@ -203,7 +203,7 @@ typedef struct erts_proc_lock_t_ { & ~ERTS_PROC_LOCK_MAIN) -#define ERTS_PIX_LOCKS_BITS 8 +#define ERTS_PIX_LOCKS_BITS 10 #define ERTS_NO_OF_PIX_LOCKS (1 << ERTS_PIX_LOCKS_BITS) @@ -767,7 +767,7 @@ erts_smp_proc_lock(Process *p, ErtsProcLocks locks) #if ERTS_PROC_LOCK_ATOMIC_IMPL NULL, #else - ERTS_PID2PIXLOCK(p->id), + ERTS_PID2PIXLOCK(p->common.id), #endif /*ERTS_PROC_LOCK_ATOMIC_IMPL*/ locks, file, line); #elif defined(ERTS_SMP) @@ -775,7 +775,7 @@ erts_smp_proc_lock(Process *p, ErtsProcLocks locks) #if ERTS_PROC_LOCK_ATOMIC_IMPL NULL, #else - ERTS_PID2PIXLOCK(p->id), + ERTS_PID2PIXLOCK(p->common.id), #endif /*ERTS_PROC_LOCK_ATOMIC_IMPL*/ locks); #endif /*ERTS_SMP*/ @@ -789,7 +789,7 @@ erts_smp_proc_unlock(Process *p, ErtsProcLocks locks) #if ERTS_PROC_LOCK_ATOMIC_IMPL NULL, #else - ERTS_PID2PIXLOCK(p->id), + ERTS_PID2PIXLOCK(p->common.id), #endif locks); #endif @@ -805,7 +805,7 @@ erts_smp_proc_trylock(Process *p, ErtsProcLocks locks) #if ERTS_PROC_LOCK_ATOMIC_IMPL NULL, #else - ERTS_PID2PIXLOCK(p->id), + ERTS_PID2PIXLOCK(p->common.id), #endif locks); #endif @@ -814,21 +814,15 @@ erts_smp_proc_trylock(Process *p, ErtsProcLocks locks) ERTS_GLB_INLINE void erts_smp_proc_inc_refc(Process *p) { #ifdef ERTS_SMP -#ifdef ERTS_ENABLE_LOCK_CHECK - erts_aint32_t refc = erts_atomic32_inc_read_nob(&p->lock.refc); - ERTS_SMP_LC_ASSERT(refc > 1); -#else - erts_atomic32_inc_nob(&p->lock.refc); -#endif + erts_ptab_inc_refc(&p->common); #endif } ERTS_GLB_INLINE void erts_smp_proc_dec_refc(Process *p) { #ifdef ERTS_SMP - erts_aint32_t refc = erts_atomic32_dec_read_nob(&p->lock.refc); - ERTS_SMP_LC_ASSERT(refc >= 0); - if (refc == 0) + int referred = erts_ptab_dec_test_refc(&p->common); + if (!referred) erts_free_proc(p); #endif } @@ -836,10 +830,8 @@ ERTS_GLB_INLINE void erts_smp_proc_dec_refc(Process *p) ERTS_GLB_INLINE void erts_smp_proc_add_refc(Process *p, Sint32 add_refc) { #ifdef ERTS_SMP - erts_aint32_t refc = erts_atomic32_add_read_nob(&p->lock.refc, - (erts_aint32_t) add_refc); - ERTS_SMP_LC_ASSERT(refc >= 0); - if (refc == 0) + int referred = erts_ptab_add_test_refc(&p->common, add_refc); + if (!referred) erts_free_proc(p); #endif } @@ -875,8 +867,7 @@ void erts_proc_safelock(Process *a_proc, #define ERTS_P2P_FLG_TRY_LOCK (1 << 1) #define ERTS_P2P_FLG_SMP_INC_REFC (1 << 2) -#define ERTS_PROC_LOCK_BUSY ((Process *) &erts_proc_lock_busy) -extern const Process erts_proc_lock_busy; +#define ERTS_PROC_LOCK_BUSY ((Process *) &erts_invalid_process) #define erts_pid2proc(PROC, HL, PID, NL) \ erts_pid2proc_opt((PROC), (HL), (PID), (NL), 0) @@ -896,33 +887,24 @@ Process *erts_pid2proc_opt(Process *, ErtsProcLocks, Eterm, ErtsProcLocks, int); ERTS_GLB_INLINE Process *erts_pix2proc(int ix) { Process *proc; - ASSERT(0 <= ix && ix < erts_proc.max); - proc = (Process *) erts_smp_atomic_read_nob(&erts_proc.tab[ix]); + ASSERT(0 <= ix && ix < erts_ptab_max(&erts_proc)); + proc = (Process *) erts_ptab_pix2intptr_nob(&erts_proc, ix); return proc == ERTS_PROC_LOCK_BUSY ? NULL : proc; } ERTS_GLB_INLINE Process *erts_proc_lookup_raw(Eterm pid) { Process *proc; - int pix; - /* - * In SMP case: Only scheduler threads are allowed - * to use this function. Other threads need to - * atomicaly increment refc at lookup, i.e., use - * erts_pid2proc_opt() with ERTS_P2P_FLG_SMP_INC_REFC. - */ - ERTS_SMP_LC_ASSERT(erts_get_scheduler_id()); + ERTS_SMP_LC_ASSERT(erts_thr_progress_lc_is_delaying()); if (is_not_internal_pid(pid)) return NULL; - pix = internal_pid_index(pid); - proc = (Process *) erts_smp_atomic_read_ddrb(&erts_proc.tab[pix]); - - if (proc && proc->id != pid) + proc = (Process *) erts_ptab_pix2intptr_ddrb(&erts_proc, + internal_pid_index(pid)); + if (proc && proc->common.id != pid) return NULL; - return proc; } diff --git a/erts/emulator/beam/erl_ptab.c b/erts/emulator/beam/erl_ptab.c new file mode 100644 index 0000000000..87beeafa1a --- /dev/null +++ b/erts/emulator/beam/erl_ptab.c @@ -0,0 +1,1566 @@ +/* + * %CopyrightBegin% + * + * Copyright Ericsson AB 2012. All Rights Reserved. + * + * The contents of this file are subject to the Erlang Public License, + * Version 1.1, (the "License"); you may not use this file except in + * compliance with the License. You should have received a copy of the + * Erlang Public License along with this software. If not, it can be + * retrieved online at http://www.erlang.org/. + * + * Software distributed under the License is distributed on an "AS IS" + * basis, WITHOUT WARRANTY OF ANY KIND, either express or implied. See + * the License for the specific language governing rights and limitations + * under the License. + * + * %CopyrightEnd% + */ + +/* + * Description: Process/Port table implementation. + * + * Author: Rickard Green + */ + +#ifdef HAVE_CONFIG_H +# include "config.h" +#endif +#define ERTS_PTAB_WANT_BIF_IMPL__ +#define ERTS_PTAB_WANT_DEBUG_FUNCS__ +#include "erl_ptab.h" +#include "global.h" +#include "erl_binary.h" + +typedef struct ErtsPTabListBifData_ ErtsPTabListBifData; + +#define ERTS_PTAB_LIST_BIF_TAB_INSPECT_INDICES_PER_RED 25 +#define ERTS_PTAB_LIST_BIF_TAB_CHUNK_SIZE 1000 +#define ERTS_PTAB_LIST_BIF_MIN_START_REDS \ + (ERTS_PTAB_LIST_BIF_TAB_CHUNK_SIZE \ + / ERTS_PTAB_LIST_BIF_TAB_INSPECT_INDICES_PER_RED) + +#define ERTS_PTAB_LIST_BIF_TAB_FREE_DELETED_REDS 1 + +#define ERTS_PTAB_LIST_BIF_INSPECT_DELETED_PER_RED 10 + +#define ERTS_PTAB_LIST_INSPECT_DELETED_MAX_REDS \ + (ERTS_PTAB_LIST_BIF_TAB_CHUNK_SIZE \ + / ERTS_PTAB_LIST_BIF_TAB_INSPECT_INDICES_PER_RED) + + +#define ERTS_PTAB_LIST_BIF_BUILD_RESULT_CONSES_PER_RED 75 + +#define ERTS_PTAB_LIST_DBG_DO_TRACE 0 + +#ifdef DEBUG +# define ERTS_PTAB_LIST_BIF_DEBUGLEVEL 100 +#else +# define ERTS_PTAB_LIST_BIF_DEBUGLEVEL 0 +#endif + +#define ERTS_PTAB_LIST_DBGLVL_CHK_HALLOC 1 +#define ERTS_PTAB_LIST_DBGLVL_CHK_FOUND_PIDS 5 +#define ERTS_PTAB_LIST_DBGLVL_CHK_PIDS 10 +#define ERTS_PTAB_LIST_DBGLVL_CHK_DEL_LIST 20 +#define ERTS_PTAB_LIST_DBGLVL_CHK_RESLIST 20 + +#if ERTS_PTAB_LIST_BIF_DEBUGLEVEL == 0 +# define ERTS_PTAB_LIST_ASSERT(EXP) +#else +# define ERTS_PTAB_LIST_ASSERT(EXP) \ + ((void) ((EXP) \ + ? 1 \ + : (debug_ptab_list_assert_error(#EXP, \ + __FILE__, \ + __LINE__, \ + __func__), \ + 0))) +#endif + + +#if ERTS_PTAB_LIST_BIF_DEBUGLEVEL >= ERTS_PTAB_LIST_DBGLVL_CHK_HALLOC +# define ERTS_PTAB_LIST_DBG_SAVE_HEAP_ALLOC(PTLBDP, HP, SZ) \ +do { \ + ERTS_PTAB_LIST_ASSERT(!(PTLBDP)->debug.heap); \ + ERTS_PTAB_LIST_ASSERT(!(PTLBDP)->debug.heap_size); \ + (PTLBDP)->debug.heap = (HP); \ + (PTLBDP)->debug.heap_size = (SZ); \ +} while (0) +# define ERTS_PTAB_LIST_DBG_VERIFY_HEAP_ALLOC_USED(PTLBDP, HP) \ +do { \ + ERTS_PTAB_LIST_ASSERT((PTLBDP)->debug.heap); \ + ERTS_PTAB_LIST_ASSERT((PTLBDP)->debug.heap_size); \ + ERTS_PTAB_LIST_ASSERT(((PTLBDP)->debug.heap \ + + (PTLBDP)->debug.heap_size) \ + == (HP)); \ + (PTLBDP)->debug.heap = NULL; \ + (PTLBDP)->debug.heap_size = 0; \ +} while (0) +# define ERTS_PTAB_LIST_DBG_HEAP_ALLOC_INIT(PTLBDP) \ +do { \ + (PTLBDP)->debug.heap = NULL; \ + (PTLBDP)->debug.heap_size = 0; \ +} while (0) +#else +# define ERTS_PTAB_LIST_DBG_SAVE_HEAP_ALLOC(PTLBDP, HP, SZ) +# define ERTS_PTAB_LIST_DBG_VERIFY_HEAP_ALLOC_USED(PTLBDP, HP) +# define ERTS_PTAB_LIST_DBG_HEAP_ALLOC_INIT(PTLBDP) +#endif + +#if ERTS_PTAB_LIST_BIF_DEBUGLEVEL >= ERTS_PTAB_LIST_DBGLVL_CHK_RESLIST +# define ERTS_PTAB_LIST_DBG_CHK_RESLIST(R) \ + debug_ptab_list_check_res_list((R)) +#else +# define ERTS_PTAB_LIST_DBG_CHK_RESLIST(R) +#endif + +#if ERTS_PTAB_LIST_BIF_DEBUGLEVEL >= ERTS_PTAB_LIST_DBGLVL_CHK_PIDS +# define ERTS_PTAB_LIST_DBG_SAVE_PIDS(PTLBDP) \ + debug_ptab_list_save_all_pids((PTLBDP)) +# define ERTS_PTAB_LIST_DBG_VERIFY_PIDS(PTLBDP) \ +do { \ + if (!(PTLBDP)->debug.correct_pids_verified) \ + debug_ptab_list_verify_all_pids((PTLBDP)); \ +} while (0) +# define ERTS_PTAB_LIST_DBG_CLEANUP_CHK_PIDS(PTLBDP) \ +do { \ + if ((PTLBDP)->debug.correct_pids) { \ + erts_free(ERTS_ALC_T_PTAB_LIST_PIDS, \ + (PTLBDP)->debug.correct_pids); \ + (PTLBDP)->debug.correct_pids = NULL; \ + } \ +} while(0) +# define ERTS_PTAB_LIST_DBG_CHK_PIDS_INIT(PTLBDP) \ +do { \ + (PTLBDP)->debug.correct_pids_verified = 0; \ + (PTLBDP)->debug.correct_pids = NULL; \ +} while (0) +#else +# define ERTS_PTAB_LIST_DBG_SAVE_PIDS(PTLBDP) +# define ERTS_PTAB_LIST_DBG_VERIFY_PIDS(PTLBDP) +# define ERTS_PTAB_LIST_DBG_CLEANUP_CHK_PIDS(PTLBDP) +# define ERTS_PTAB_LIST_DBG_CHK_PIDS_INIT(PTLBDP) +#endif + +#if ERTS_PTAB_LIST_BIF_DEBUGLEVEL >= ERTS_PTAB_LIST_DBGLVL_CHK_FOUND_PIDS +# define ERTS_PTAB_LIST_DBG_CHK_PID_FOUND(PTLBDP, PID, IC) \ + debug_ptab_list_check_found_pid((PTLBDP), (PID), (IC), 1) +# define ERTS_PTAB_LIST_DBG_CHK_PID_NOT_FOUND(PTLBDP, PID, IC) \ + debug_ptab_list_check_found_pid((PTLBDP), (PID), (IC), 0) +#else +# define ERTS_PTAB_LIST_DBG_CHK_PID_FOUND(PTLBDP, PID, IC) +# define ERTS_PTAB_LIST_DBG_CHK_PID_NOT_FOUND(PTLBDP, PID, IC) +#endif + +#if ERTS_PTAB_LIST_BIF_DEBUGLEVEL >= ERTS_PTAB_LIST_DBGLVL_CHK_DEL_LIST +# define ERTS_PTAB_LIST_DBG_CHK_DEL_LIST(PTab) \ + debug_ptab_list_check_del_list((PTab)) +# define ERTS_PTAB_LIST_DBG_CHK_FREELIST(PTab, FL) \ + debug_ptab_list_check_del_free_list((PTab), (FL)) +#else +# define ERTS_PTAB_LIST_DBG_CHK_DEL_LIST(PTab) +# define ERTS_PTAB_LIST_DBG_CHK_FREELIST(PTab, FL) +#endif + +#if ERTS_PTAB_LIST_BIF_DEBUGLEVEL == 0 +#if ERTS_PTAB_LIST_DBG_DO_TRACE +# define ERTS_PTAB_LIST_DBG_INIT(P, PTLBDP) \ + (PTLBDP)->debug.caller = (P)->common.id +# else +# define ERTS_PTAB_LIST_DBG_INIT(P, PTLBDP) +# endif +# define ERTS_PTAB_LIST_DBG_CLEANUP(PTLBDP) +#else +# define ERTS_PTAB_LIST_DBG_INIT(P, PTLBDP) \ +do { \ + (PTLBDP)->debug.caller = (P)->common.id; \ + ERTS_PTAB_LIST_DBG_HEAP_ALLOC_INIT((PTLBDP)); \ + ERTS_PTAB_LIST_DBG_CHK_PIDS_INIT((PTLBDP)); \ +} while (0) +# define ERTS_PTAB_LIST_DBG_CLEANUP(PTLBDP) \ +do { \ + ERTS_PTAB_LIST_DBG_CLEANUP_CHK_PIDS((PTLBDP)); \ +} while (0) +#endif + +#if ERTS_PTAB_LIST_DBG_DO_TRACE +# define ERTS_PTAB_LIST_DBG_TRACE(PID, WHAT) \ + erts_fprintf(stderr, "%T %s:%d:%s(): %s\n", \ + (PID), __FILE__, __LINE__, __func__, #WHAT) +#else +# define ERTS_PTAB_LIST_DBG_TRACE(PID, WHAT) +#endif + + +#if ERTS_PTAB_LIST_BIF_DEBUGLEVEL != 0 +static void debug_ptab_list_assert_error(char* expr, + const char* file, + int line, + const char *func); +#endif +#if ERTS_PTAB_LIST_BIF_DEBUGLEVEL >= ERTS_PTAB_LIST_DBGLVL_CHK_RESLIST +static void debug_ptab_list_check_res_list(Eterm list); +#endif +#if ERTS_PTAB_LIST_BIF_DEBUGLEVEL >= ERTS_PTAB_LIST_DBGLVL_CHK_PIDS +static void debug_ptab_list_save_all_pids(ErtsPTabListBifData *ptlbdp); +static void debug_ptab_list_verify_all_pids(ErtsPTabListBifData *ptlbdp); +#endif +#if ERTS_PTAB_LIST_BIF_DEBUGLEVEL >= ERTS_PTAB_LIST_DBGLVL_CHK_FOUND_PIDS +static void debug_ptab_list_check_found_pid(ErtsPTabListBifData *ptlbdp, + Eterm pid, + Uint64 ic, + int pid_should_be_found); +#endif +#if ERTS_PTAB_LIST_BIF_DEBUGLEVEL >= ERTS_PTAB_LIST_DBGLVL_CHK_DEL_LIST +static void debug_ptab_list_check_del_list(ErtsPTab *ptab); +static void debug_ptab_list_check_del_free_list(ErtsPTab *ptab, + ErtsPTabDeletedElement *ptdep); +#endif + +struct ErtsPTabDeletedElement_ { + ErtsPTabDeletedElement *next; + ErtsPTabDeletedElement *prev; + int ix; + union { + struct { + Eterm id; + Uint64 inserted; + Uint64 deleted; + } element; + struct { + Uint64 interval; + } bif_invocation; + } u; +}; + +static Export ptab_list_continue_export; + +typedef struct { + Uint64 interval; +} ErtsPTabListBifChunkInfo; + +typedef enum { + INITIALIZING, + INSPECTING_TABLE, + INSPECTING_DELETED, + BUILDING_RESULT, + RETURN_RESULT +} ErtsPTabListBifState; + +struct ErtsPTabListBifData_ { + ErtsPTab *ptab; + ErtsPTabListBifState state; + Eterm caller; + ErtsPTabListBifChunkInfo *chunk; + int tix; + int pid_ix; + int pid_sz; + Eterm *pid; + ErtsPTabDeletedElement *bif_invocation; /* Only used when > 1 chunk */ + +#if ERTS_PTAB_LIST_BIF_DEBUGLEVEL != 0 || ERTS_PTAB_LIST_DBG_DO_TRACE + struct { + Eterm caller; +#if ERTS_PTAB_LIST_BIF_DEBUGLEVEL >= ERTS_PTAB_LIST_DBGLVL_CHK_FOUND_PIDS + Uint64 *pid_started; +#endif +#if ERTS_PTAB_LIST_BIF_DEBUGLEVEL >= ERTS_PTAB_LIST_DBGLVL_CHK_HALLOC + Eterm *heap; + Uint heap_size; +#endif +#if ERTS_PTAB_LIST_BIF_DEBUGLEVEL >= ERTS_PTAB_LIST_DBGLVL_CHK_PIDS + int correct_pids_verified; + Eterm *correct_pids; +#endif + } debug; +#endif + +}; + +#ifdef ARCH_32 + +static ERTS_INLINE Uint64 +dw_aint_to_uint64(erts_dw_aint_t *dw) +{ +#ifdef ETHR_SU_DW_NAINT_T__ + return (Uint64) dw->dw_sint; +#else + Uint64 res; + res = (Uint64) ((Uint32) dw->sint[ERTS_DW_AINT_HIGH_WORD]); + res <<= 32; + res |= (Uint64) ((Uint32) dw->sint[ERTS_DW_AINT_LOW_WORD]); + return res; +#endif +} + +static void +unint64_to_dw_aint(erts_dw_aint_t *dw, Uint64 val) +{ +#ifdef ETHR_SU_DW_NAINT_T__ + dw->dw_sint = (ETHR_SU_DW_NAINT_T__) val; +#else + dw->sint[ERTS_DW_AINT_LOW_WORD] = (erts_aint_t) (val & 0xffffffff); + dw->sint[ERTS_DW_AINT_HIGH_WORD] = (erts_aint_t) ((val >> 32) & 0xffffffff); +#endif +} + +static ERTS_INLINE void +last_data_init_nob(ErtsPTab *ptab, Uint64 val) +{ + erts_dw_aint_t dw; + unint64_to_dw_aint(&dw, val); + erts_smp_dw_atomic_init_nob(&ptab->vola.tile.last_data, &dw); +} + +static ERTS_INLINE void +last_data_set_relb(ErtsPTab *ptab, Uint64 val) +{ + erts_dw_aint_t dw; + unint64_to_dw_aint(&dw, val); + erts_smp_dw_atomic_set_relb(&ptab->vola.tile.last_data, &dw); +} + +static ERTS_INLINE Uint64 +last_data_read_nob(ErtsPTab *ptab) +{ + erts_dw_aint_t dw; + erts_smp_dw_atomic_read_nob(&ptab->vola.tile.last_data, &dw); + return dw_aint_to_uint64(&dw); +} + +static ERTS_INLINE Uint64 +last_data_read_acqb(ErtsPTab *ptab) +{ + erts_dw_aint_t dw; + erts_smp_dw_atomic_read_acqb(&ptab->vola.tile.last_data, &dw); + return dw_aint_to_uint64(&dw); +} + +static ERTS_INLINE Uint64 +last_data_cmpxchg_relb(ErtsPTab *ptab, Uint64 new, Uint64 exp) +{ + erts_dw_aint_t dw_new, dw_xchg; + + unint64_to_dw_aint(&dw_new, new); + unint64_to_dw_aint(&dw_xchg, exp); + + if (erts_smp_dw_atomic_cmpxchg_relb(&ptab->vola.tile.last_data, + &dw_new, + &dw_xchg)) + return exp; + else + return dw_aint_to_uint64(&dw_xchg); +} + +#elif defined(ARCH_64) + +union { + erts_smp_atomic_t pid_data; + char align[ERTS_CACHE_LINE_SIZE]; +} last erts_align_attribute(ERTS_CACHE_LINE_SIZE); + +static ERTS_INLINE void +last_data_init_nob(ErtsPTab *ptab, Uint64 val) +{ + erts_smp_atomic_init_nob(&ptab->vola.tile.last_data, (erts_aint_t) val); +} + +static ERTS_INLINE void +last_data_set_relb(ErtsPTab *ptab, Uint64 val) +{ + erts_smp_atomic_set_relb(&ptab->vola.tile.last_data, (erts_aint_t) val); +} + +static ERTS_INLINE Uint64 +last_data_read_nob(ErtsPTab *ptab) +{ + return (Uint64) erts_smp_atomic_read_nob(&ptab->vola.tile.last_data); +} + +static ERTS_INLINE Uint64 +last_data_read_acqb(ErtsPTab *ptab) +{ + return (Uint64) erts_smp_atomic_read_acqb(&ptab->vola.tile.last_data); +} + +static ERTS_INLINE Uint64 +last_data_cmpxchg_relb(ErtsPTab *ptab, Uint64 new, Uint64 exp) +{ + return (Uint64) erts_smp_atomic_cmpxchg_relb(&ptab->vola.tile.last_data, + (erts_aint_t) new, + (erts_aint_t) exp); +} + +#else +# error "Not 64-bit, nor 32-bit architecture..." +#endif + +static ERTS_INLINE int +last_data_cmp(Uint64 ld1, Uint64 ld2) +{ + Uint64 ld1_wrap; + + if (ld1 == ld2) + return 0; + + ld1_wrap = ld1 + (((Uint64) 1) << 63); + + if (ld1 < ld1_wrap) + return (ld1 < ld2 && ld2 < ld1_wrap) ? -1 : 1; + else + return (ld1_wrap <= ld2 && ld2 < ld1) ? 1 : -1; +} + +#define ERTS_PTAB_LastData2EtermData(LD) \ + ((Eterm) ((LD) & ~(~((Uint64) 0) << ERTS_PTAB_ID_DATA_SIZE))) + +void +erts_ptab_init_table(ErtsPTab *ptab, + ErtsAlcType_t atype, + void (*release_element)(void *), + ErtsPTabElementCommon *invalid_element, + int size, + char *name) +{ + size_t tab_sz; + int bits; + char *tab_end; + erts_smp_atomic_t *tab_entry; + erts_smp_rwmtx_opt_t rwmtx_opts = ERTS_SMP_RWMTX_OPT_DEFAULT_INITER; + rwmtx_opts.type = ERTS_SMP_RWMTX_TYPE_EXTREMELY_FREQUENT_READ; + rwmtx_opts.lived = ERTS_SMP_RWMTX_LONG_LIVED; + + erts_smp_rwmtx_init_opt(&ptab->list.data.rwmtx, &rwmtx_opts, name); + erts_smp_atomic32_init_nob(&ptab->vola.tile.count, 0); + last_data_init_nob(ptab, ~((Uint64) 0)); + + /* A size that is a power of 2 is to prefer performance wise */ + bits = erts_fit_in_bits_int32(size-1); + size = 1 << bits; + if (size > ERTS_PTAB_MAX_SIZE) { + size = ERTS_PTAB_MAX_SIZE; + bits = erts_fit_in_bits_int32((Sint32) size - 1); + } + + ptab->r.o.max = size; + + tab_sz = ERTS_ALC_CACHE_LINE_ALIGN_SIZE(size*sizeof(erts_smp_atomic_t)); + ptab->r.o.tab = erts_alloc_permanent_cache_aligned(atype, tab_sz); + tab_end = ((char *) ptab->r.o.tab) + tab_sz; + tab_entry = ptab->r.o.tab; + while (tab_end > ((char *) tab_entry)) { + erts_smp_atomic_init_nob(tab_entry, ERTS_AINT_NULL); + tab_entry++; + } + + ptab->r.o.tab_cache_lines = tab_sz/ERTS_CACHE_LINE_SIZE; + ptab->r.o.pix_per_cache_line = (ERTS_CACHE_LINE_SIZE + / sizeof(erts_smp_atomic_t)); + ASSERT((ptab->r.o.max & (ptab->r.o.max - 1)) == 0); /* power of 2 */ + ASSERT((ptab->r.o.pix_per_cache_line + & (ptab->r.o.pix_per_cache_line - 1)) == 0); /* power of 2 */ + ASSERT((ptab->r.o.tab_cache_lines + & (ptab->r.o.tab_cache_lines - 1)) == 0); /* power of 2 */ + + ptab->r.o.pix_mask + = (1 << bits) - 1; + ptab->r.o.pix_cl_mask + = ptab->r.o.tab_cache_lines-1; + ptab->r.o.pix_cl_shift + = erts_fit_in_bits_int32(ptab->r.o.pix_per_cache_line-1); + ptab->r.o.pix_cli_shift + = erts_fit_in_bits_int32(ptab->r.o.pix_cl_mask); + ptab->r.o.pix_cli_mask + = (1 << (bits - ptab->r.o.pix_cli_shift)) - 1; + + ASSERT(ptab->r.o.pix_cl_shift + ptab->r.o.pix_cli_shift == bits); + + ptab->r.o.invalid_element = invalid_element; + ptab->r.o.invalid_data = erts_ptab_id2data(ptab, invalid_element->id); + ptab->r.o.release_element = release_element; + + erts_smp_interval_init(&ptab->list.data.interval); + ptab->list.data.deleted.start = NULL; + ptab->list.data.deleted.end = NULL; + ptab->list.data.chunks = (((ptab->r.o.max - 1) + / ERTS_PTAB_LIST_BIF_TAB_CHUNK_SIZE) + + 1); + + if (size == ERTS_PTAB_MAX_SIZE) { + int pix; + /* + * We want a table size of a power of 2 which ERTS_PTAB_MAX_SIZE + * is. We only have ERTS_PTAB_MAX_SIZE-1 unique identifiers and + * we don't want to shrink the size to ERTS_PTAB_MAX_SIZE/2. + * + * In order to fix this, we insert a pointer from the table + * to the invalid_element, wich will be interpreted as a + * slot currently being modified. This way we will be able to + * have ERTS_PTAB_MAX_SIZE-1 valid elements in the table while + * still having a table size of the power of 2. + */ + erts_smp_atomic32_inc_nob(&ptab->vola.tile.count); + pix = erts_ptab_data2pix(ptab, ptab->r.o.invalid_data); + erts_smp_atomic_set_relb(&ptab->r.o.tab[pix], + (erts_aint_t) ptab->r.o.invalid_element); + } + +} + +int +erts_ptab_initialized(ErtsPTab *ptab) +{ + return ptab->r.o.tab != NULL; +} + +int +erts_ptab_new_element(ErtsPTab *ptab, + ErtsPTabElementCommon *ptab_el, + void *init_arg, + void (*init_ptab_el)(void *, Eterm)) +{ + int pix; + Uint64 ld, exp_ld; + Eterm data; + erts_aint32_t count; + erts_aint_t invalid = (erts_aint_t) ptab->r.o.invalid_element; + + erts_ptab_rlock(ptab); + + count = erts_smp_atomic32_inc_read_acqb(&ptab->vola.tile.count); + if (count > ptab->r.o.max) { + while (1) { + erts_aint32_t act_count; + + act_count = erts_smp_atomic32_cmpxchg_relb(&ptab->vola.tile.count, + count-1, + count); + if (act_count == count) { + erts_ptab_runlock(ptab); + return 0; + } + count = act_count; + if (count <= ptab->r.o.max) + break; + } + } + + ptab_el->u.alive.started_interval + = erts_smp_current_interval_nob(erts_ptab_interval(ptab)); + + ld = last_data_read_acqb(ptab); + + /* Reserve slot */ + while (1) { + ld++; + pix = erts_ptab_data2pix(ptab, ERTS_PTAB_LastData2EtermData(ld)); + if (erts_smp_atomic_read_nob(&ptab->r.o.tab[pix]) == ERTS_AINT_NULL) { + erts_aint_t val; + val = erts_smp_atomic_cmpxchg_relb(&ptab->r.o.tab[pix], + invalid, + ERTS_AINT_NULL); + + if (ERTS_AINT_NULL == val) + break; + } + } + + data = ERTS_PTAB_LastData2EtermData(ld); + + if (data == ptab->r.o.invalid_data) { + /* Do not use invalid data; fix it... */ + ld += ptab->r.o.max; + ASSERT(pix == erts_ptab_data2pix(ptab, + ERTS_PTAB_LastData2EtermData(ld))); + data = ERTS_PTAB_LastData2EtermData(ld); + ASSERT(data != ptab->r.o.invalid_data); + } + + exp_ld = last_data_read_nob(ptab); + + /* Move last data forward */ + while (1) { + Uint64 act_ld; + if (last_data_cmp(ld, exp_ld) < 0) + break; + act_ld = last_data_cmpxchg_relb(ptab, ld, exp_ld); + if (act_ld == exp_ld) + break; + exp_ld = act_ld; + } + + init_ptab_el(init_arg, data); + +#ifdef ERTS_SMP + erts_smp_atomic32_init_nob(&ptab_el->refc, 1); +#endif + + /* Move into slot reserved */ +#ifdef DEBUG + ASSERT(invalid == erts_smp_atomic_xchg_relb(&ptab->r.o.tab[pix], + (erts_aint_t) ptab_el)); +#else + erts_smp_atomic_set_relb(&ptab->r.o.tab[pix], (erts_aint_t) ptab_el); +#endif + + erts_ptab_runlock(ptab); + + return 1; +} + +static void +save_deleted_element(ErtsPTab *ptab, ErtsPTabElementCommon *ptab_el) +{ + ErtsPTabDeletedElement *ptdep = erts_alloc(ERTS_ALC_T_PTAB_LIST_DEL, + sizeof(ErtsPTabDeletedElement)); + ERTS_PTAB_LIST_ASSERT(ptab->list.data.deleted.start + && ptab->list.data.deleted.end); + ERTS_SMP_LC_ASSERT(erts_smp_lc_ptab_is_rwlocked(ptab)); + + ERTS_PTAB_LIST_DBG_CHK_DEL_LIST(ptab); + + ptdep->prev = ptab->list.data.deleted.end; + ptdep->next = NULL; + ptdep->ix = erts_ptab_id2pix(ptab, ptab_el->id); + ptdep->u.element.id = ptab_el->id; + ptdep->u.element.inserted = ptab_el->u.alive.started_interval; + ptdep->u.element.deleted = + erts_smp_current_interval_nob(erts_ptab_interval(ptab)); + + ptab->list.data.deleted.end->next = ptdep; + ptab->list.data.deleted.end = ptdep; + + ERTS_PTAB_LIST_DBG_CHK_DEL_LIST(ptab); + + ERTS_PTAB_LIST_ASSERT(ptdep->prev->ix >= 0 + ? (ptdep->u.element.deleted + >= ptdep->prev->u.element.deleted) + : (ptdep->u.element.deleted + >= ptdep->prev->u.bif_invocation.interval)); +} + +void +erts_ptab_delete_element(ErtsPTab *ptab, + ErtsPTabElementCommon *ptab_el) +{ + int maybe_save; + int pix = erts_ptab_id2pix(ptab, ptab_el->id); + + ASSERT(erts_get_scheduler_id()); /* *Need* to be a scheduler */ + + erts_ptab_rlock(ptab); + maybe_save = ptab->list.data.deleted.end != NULL; + if (maybe_save) { + erts_ptab_runlock(ptab); + erts_ptab_rwlock(ptab); + } + + erts_smp_atomic_set_relb(&ptab->r.o.tab[pix], ERTS_AINT_NULL); + + ASSERT(erts_smp_atomic32_read_nob(&ptab->vola.tile.count) > 0); + erts_smp_atomic32_dec_relb(&ptab->vola.tile.count); + + if (!maybe_save) + erts_ptab_runlock(ptab); + else { + if (ptab->list.data.deleted.end) + save_deleted_element(ptab, ptab_el); + erts_ptab_rwunlock(ptab); + } + + if (ptab->r.o.release_element) + erts_schedule_thr_prgr_later_op(ptab->r.o.release_element, + (void *) ptab_el, + &ptab_el->u.release); +} + +/* + * erts_ptab_list() implements BIFs listing the content of the table, + * e.g. erlang:processes/0. + */ +static void cleanup_ptab_list_bif_data(Binary *bp); +static int ptab_list_bif_engine(Process *c_p, Eterm *res_accp, Binary *mbp); + + +BIF_RETTYPE +erts_ptab_list(Process *c_p, ErtsPTab *ptab) +{ + /* + * A requirement: The list of identifiers returned should be a + * consistent snapshot of all elements existing + * in the table at some point in time during the + * execution of the BIF calling this function. + * Since elements might be deleted while the BIF + * is executing, we have to keep track of all + * deleted elements and add them to the result. + * We also ignore elements created after the BIF + * has begun executing. + */ + BIF_RETTYPE ret_val; + Eterm res_acc = NIL; + Binary *mbp = erts_create_magic_binary(sizeof(ErtsPTabListBifData), + cleanup_ptab_list_bif_data); + ErtsPTabListBifData *ptlbdp = ERTS_MAGIC_BIN_DATA(mbp); + + ERTS_PTAB_LIST_DBG_TRACE(c_p->common.id, call); + ptlbdp->ptab = ptab; + ptlbdp->state = INITIALIZING; + ERTS_PTAB_LIST_DBG_INIT(c_p, ptlbdp); + + if (ERTS_BIF_REDS_LEFT(c_p) >= ERTS_PTAB_LIST_BIF_MIN_START_REDS + && ptab_list_bif_engine(c_p, &res_acc, mbp)) { + erts_bin_free(mbp); + ERTS_PTAB_LIST_DBG_CHK_RESLIST(res_acc); + ERTS_PTAB_LIST_DBG_TRACE(c_p->common.id, return); + ERTS_BIF_PREP_RET(ret_val, res_acc); + } + else { + Eterm *hp; + Eterm magic_bin; + ERTS_PTAB_LIST_DBG_CHK_RESLIST(res_acc); + hp = HAlloc(c_p, PROC_BIN_SIZE); + ERTS_PTAB_LIST_DBG_SAVE_HEAP_ALLOC(ptlbdp, hp, PROC_BIN_SIZE); + magic_bin = erts_mk_magic_binary_term(&hp, &MSO(c_p), mbp); + ERTS_PTAB_LIST_DBG_VERIFY_HEAP_ALLOC_USED(ptlbdp, hp); + ERTS_PTAB_LIST_DBG_TRACE(c_p->common.id, trap); + ERTS_BIF_PREP_YIELD2(ret_val, + &ptab_list_continue_export, + c_p, + res_acc, + magic_bin); + } + return ret_val; +} + +static void +cleanup_ptab_list_bif_data(Binary *bp) +{ + ErtsPTabListBifData *ptlbdp = ERTS_MAGIC_BIN_DATA(bp); + ErtsPTab *ptab = ptlbdp->ptab; + + ERTS_PTAB_LIST_DBG_TRACE(ptlbdp->debug.caller, call); + + if (ptlbdp->state != INITIALIZING) { + + if (ptlbdp->chunk) { + erts_free(ERTS_ALC_T_PTAB_LIST_CNKI, ptlbdp->chunk); + ptlbdp->chunk = NULL; + } + if (ptlbdp->pid) { + erts_free(ERTS_ALC_T_PTAB_LIST_PIDS, ptlbdp->pid); + ptlbdp->pid = NULL; + } + +#if ERTS_PTAB_LIST_BIF_DEBUGLEVEL >= ERTS_PTAB_LIST_DBGLVL_CHK_FOUND_PIDS + if (ptlbdp->debug.pid_started) { + erts_free(ERTS_ALC_T_PTAB_LIST_PIDS, ptlbdp->debug.pid_started); + ptlbdp->debug.pid_started = NULL; + } +#endif + + if (ptlbdp->bif_invocation) { + ErtsPTabDeletedElement *ptdep; + + erts_ptab_rwlock(ptab); + + ERTS_PTAB_LIST_DBG_TRACE(ptlbdp->debug.caller, deleted_cleanup); + + ptdep = ptlbdp->bif_invocation; + ptlbdp->bif_invocation = NULL; + + ERTS_PTAB_LIST_DBG_CHK_DEL_LIST(ptab); + + if (ptdep->prev) { + /* + * Only remove this bif invokation when we + * have preceding invokations. + */ + ptdep->prev->next = ptdep->next; + if (ptdep->next) + ptdep->next->prev = ptdep->prev; + else { + /* + * At the time of writing this branch cannot be + * reached. I don't want to remove this code though + * since it may be possible to reach this line + * in the future if the cleanup order in + * erts_do_exit_process() is changed. The ASSERT(0) + * is only here to make us aware that the reorder + * has happened. /rickard + */ + ASSERT(0); + ptab->list.data.deleted.end = ptdep->prev; + } + erts_free(ERTS_ALC_T_PTAB_LIST_DEL, ptdep); + } + else { + /* + * Free all elements until next bif invokation + * is found. + */ + ERTS_PTAB_LIST_ASSERT(ptab->list.data.deleted.start == ptdep); + do { + ErtsPTabDeletedElement *fptdep = ptdep; + ptdep = ptdep->next; + erts_free(ERTS_ALC_T_PTAB_LIST_DEL, fptdep); + } while (ptdep && ptdep->ix >= 0); + ptab->list.data.deleted.start = ptdep; + if (ptdep) + ptdep->prev = NULL; + else + ptab->list.data.deleted.end = NULL; + } + + ERTS_PTAB_LIST_DBG_CHK_DEL_LIST(ptab); + + erts_ptab_rwunlock(ptab); + + } + } + + ERTS_PTAB_LIST_DBG_TRACE(ptlbdp->debug.caller, return); + ERTS_PTAB_LIST_DBG_CLEANUP(ptlbdp); +} + +static int +ptab_list_bif_engine(Process *c_p, Eterm *res_accp, Binary *mbp) +{ + ErtsPTabListBifData *ptlbdp = ERTS_MAGIC_BIN_DATA(mbp); + ErtsPTab *ptab = ptlbdp->ptab; + int have_reds; + int reds; + int locked = 0; + + do { + switch (ptlbdp->state) { + case INITIALIZING: + ptlbdp->chunk = erts_alloc(ERTS_ALC_T_PTAB_LIST_CNKI, + (sizeof(ErtsPTabListBifChunkInfo) + * ptab->list.data.chunks)); + ptlbdp->tix = 0; + ptlbdp->pid_ix = 0; + + erts_ptab_rwlock(ptab); + locked = 1; + + ERTS_PTAB_LIST_DBG_TRACE(c_p->common.id, init); + + ptlbdp->pid_sz = erts_ptab_count(ptab); + ptlbdp->pid = erts_alloc(ERTS_ALC_T_PTAB_LIST_PIDS, + sizeof(Eterm)*ptlbdp->pid_sz); + +#if ERTS_PTAB_LIST_BIF_DEBUGLEVEL >= ERTS_PTAB_LIST_DBGLVL_CHK_FOUND_PIDS + ptlbdp->debug.pid_started + = erts_alloc(ERTS_ALC_T_PTAB_LIST_PIDS, + sizeof(Uint64)*ptlbdp->pid_sz); +#endif + + ERTS_PTAB_LIST_DBG_SAVE_PIDS(ptlbdp); + + if (ptab->list.data.chunks == 1) + ptlbdp->bif_invocation = NULL; + else { + /* + * We will have to access the table multiple times + * releasing the table lock in between chunks. + */ + ptlbdp->bif_invocation + = erts_alloc(ERTS_ALC_T_PTAB_LIST_DEL, + sizeof(ErtsPTabDeletedElement)); + ptlbdp->bif_invocation->ix = -1; + ptlbdp->bif_invocation->u.bif_invocation.interval + = erts_smp_step_interval_nob(erts_ptab_interval(ptab)); + ERTS_PTAB_LIST_DBG_CHK_DEL_LIST(ptab); + + ptlbdp->bif_invocation->next = NULL; + if (ptab->list.data.deleted.end) { + ptlbdp->bif_invocation->prev = ptab->list.data.deleted.end; + ptab->list.data.deleted.end->next = ptlbdp->bif_invocation; + ERTS_PTAB_LIST_ASSERT(ptab->list.data.deleted.start); + } + else { + ptlbdp->bif_invocation->prev = NULL; + ptab->list.data.deleted.start = ptlbdp->bif_invocation; + } + ptab->list.data.deleted.end = ptlbdp->bif_invocation; + + ERTS_PTAB_LIST_DBG_CHK_DEL_LIST(ptab); + + } + + ptlbdp->state = INSPECTING_TABLE; + /* Fall through */ + + case INSPECTING_TABLE: { + int ix = ptlbdp->tix; + int indices = ERTS_PTAB_LIST_BIF_TAB_CHUNK_SIZE; + int cix = ix / ERTS_PTAB_LIST_BIF_TAB_CHUNK_SIZE; + int end_ix = ix + indices; + Uint64 *invocation_interval_p; + ErtsPTabElementCommon *invalid_element; + + invocation_interval_p + = (ptlbdp->bif_invocation + ? &ptlbdp->bif_invocation->u.bif_invocation.interval + : NULL); + + ERTS_PTAB_LIST_ASSERT(is_nil(*res_accp)); + if (!locked) { + erts_ptab_rwlock(ptab); + locked = 1; + } + + ERTS_SMP_LC_ASSERT(erts_smp_lc_ptab_is_rwlocked(ptab)); + ERTS_PTAB_LIST_DBG_TRACE(p->common.id, insp_table); + + if (cix != 0) + ptlbdp->chunk[cix].interval + = erts_smp_step_interval_nob(erts_ptab_interval(ptab)); + else if (ptlbdp->bif_invocation) + ptlbdp->chunk[0].interval = *invocation_interval_p; + /* else: interval is irrelevant */ + + if (end_ix >= ptab->r.o.max) { + ERTS_PTAB_LIST_ASSERT(cix+1 == ptab->list.data.chunks); + end_ix = ptab->r.o.max; + indices = end_ix - ix; + /* What to do when done with this chunk */ + ptlbdp->state = (ptab->list.data.chunks == 1 + ? BUILDING_RESULT + : INSPECTING_DELETED); + } + + invalid_element = ptab->r.o.invalid_element; + for (; ix < end_ix; ix++) { + ErtsPTabElementCommon *el; + el = (ErtsPTabElementCommon *) erts_ptab_pix2intptr_nob(ptab, + ix); + if (el + && el != invalid_element + && (!invocation_interval_p + || el->u.alive.started_interval < *invocation_interval_p)) { + ERTS_PTAB_LIST_ASSERT(erts_ptab_is_valid_id(el->id)); + ptlbdp->pid[ptlbdp->pid_ix] = el->id; + +#if ERTS_PTAB_LIST_BIF_DEBUGLEVEL >= ERTS_PTAB_LIST_DBGLVL_CHK_FOUND_PIDS + ptlbdp->debug.pid_started[ptlbdp->pid_ix] + = el->u.alive.started_interval; +#endif + + ptlbdp->pid_ix++; + ERTS_PTAB_LIST_ASSERT(ptlbdp->pid_ix <= ptlbdp->pid_sz); + } + } + + ptlbdp->tix = end_ix; + + erts_ptab_rwunlock(ptab); + locked = 0; + + reds = indices/ERTS_PTAB_LIST_BIF_TAB_INSPECT_INDICES_PER_RED; + BUMP_REDS(c_p, reds); + + have_reds = ERTS_BIF_REDS_LEFT(c_p); + + if (have_reds && ptlbdp->state == INSPECTING_TABLE) { + ix = ptlbdp->tix; + indices = ERTS_PTAB_LIST_BIF_TAB_CHUNK_SIZE; + end_ix = ix + indices; + if (end_ix > ptab->r.o.max) { + end_ix = ptab->r.o.max; + indices = end_ix - ix; + } + + reds = indices/ERTS_PTAB_LIST_BIF_TAB_INSPECT_INDICES_PER_RED; + + /* Pretend we have no reds left if we haven't got enough + reductions to complete next chunk */ + if (reds > have_reds) + have_reds = 0; + } + + break; + } + + case INSPECTING_DELETED: { + int i; + int max_reds; + int free_deleted = 0; + Uint64 invocation_interval; + ErtsPTabDeletedElement *ptdep; + ErtsPTabDeletedElement *free_list = NULL; + + ptdep = ptlbdp->bif_invocation; + ERTS_PTAB_LIST_ASSERT(ptdep); + invocation_interval = ptdep->u.bif_invocation.interval; + + max_reds = have_reds = ERTS_BIF_REDS_LEFT(c_p); + if (max_reds > ERTS_PTAB_LIST_INSPECT_DELETED_MAX_REDS) + max_reds = ERTS_PTAB_LIST_INSPECT_DELETED_MAX_REDS; + + reds = 0; + erts_ptab_rwlock(ptab); + ERTS_PTAB_LIST_DBG_TRACE(p->common.id, insp_term_procs); + + ERTS_PTAB_LIST_DBG_CHK_DEL_LIST(ptab); + + if (ptdep->prev) + ptdep->prev->next = ptdep->next; + else { + ERTS_PTAB_LIST_ASSERT(ptab->list.data.deleted.start == ptdep); + ptab->list.data.deleted.start = ptdep->next; + + if (ptab->list.data.deleted.start + && ptab->list.data.deleted.start->ix >= 0) { + free_list = ptab->list.data.deleted.start; + free_deleted = 1; + } + } + + if (ptdep->next) + ptdep->next->prev = ptdep->prev; + else + ptab->list.data.deleted.end = ptdep->prev; + + ptdep = ptdep->next; + + i = 0; + while (reds < max_reds && ptdep) { + if (ptdep->ix < 0) { + if (free_deleted) { + ERTS_PTAB_LIST_ASSERT(free_list); + ERTS_PTAB_LIST_ASSERT(ptdep->prev); + + ptdep->prev->next = NULL; /* end of free_list */ + ptab->list.data.deleted.start = ptdep; + ptdep->prev = NULL; + free_deleted = 0; + } + } + else { + int cix = ptdep->ix/ERTS_PTAB_LIST_BIF_TAB_CHUNK_SIZE; + Uint64 chunk_interval = ptlbdp->chunk[cix].interval; + Eterm pid = ptdep->u.element.id; + ERTS_PTAB_LIST_ASSERT(erts_ptab_is_valid_id(pid)); + + if (ptdep->u.element.inserted < invocation_interval) { + if (ptdep->u.element.deleted < chunk_interval) { + ERTS_PTAB_LIST_DBG_CHK_PID_NOT_FOUND( + ptlbdp, + pid, + ptdep->u.element.inserted); + ptlbdp->pid[ptlbdp->pid_ix] = pid; +#if ERTS_PTAB_LIST_BIF_DEBUGLEVEL >= ERTS_PTAB_LIST_DBGLVL_CHK_FOUND_PIDS + ptlbdp->debug.pid_started[ptlbdp->pid_ix] + = ptdep->u.element.inserted; +#endif + ptlbdp->pid_ix++; + ERTS_PTAB_LIST_ASSERT(ptlbdp->pid_ix + <= ptlbdp->pid_sz); + } + else { + ERTS_PTAB_LIST_DBG_CHK_PID_FOUND( + ptlbdp, + pid, + ptdep->u.element.inserted); + } + } + else { + ERTS_PTAB_LIST_DBG_CHK_PID_NOT_FOUND( + ptlbdp, + pid, + ptdep->u.element.inserted); + } + + i++; + if (i == ERTS_PTAB_LIST_BIF_INSPECT_DELETED_PER_RED) { + reds++; + i = 0; + } + if (free_deleted) + reds += ERTS_PTAB_LIST_BIF_TAB_FREE_DELETED_REDS; + } + ptdep = ptdep->next; + } + + if (free_deleted) { + ERTS_PTAB_LIST_ASSERT(free_list); + ptab->list.data.deleted.start = ptdep; + if (!ptdep) + ptab->list.data.deleted.end = NULL; + else { + ERTS_PTAB_LIST_ASSERT(ptdep->prev); + ptdep->prev->next = NULL; /* end of free_list */ + ptdep->prev = NULL; + } + } + + if (!ptdep) { + /* Done */ + ERTS_PTAB_LIST_ASSERT(ptlbdp->pid_ix == ptlbdp->pid_sz); + ptlbdp->state = BUILDING_RESULT; + ptlbdp->bif_invocation->next = free_list; + free_list = ptlbdp->bif_invocation; + ptlbdp->bif_invocation = NULL; + } + else { + /* Link in bif_invocation again where we left off */ + ptlbdp->bif_invocation->prev = ptdep->prev; + ptlbdp->bif_invocation->next = ptdep; + ptdep->prev = ptlbdp->bif_invocation; + if (ptlbdp->bif_invocation->prev) + ptlbdp->bif_invocation->prev->next = ptlbdp->bif_invocation; + else { + ERTS_PTAB_LIST_ASSERT(ptab->list.data.deleted.start + == ptdep); + ptab->list.data.deleted.start = ptlbdp->bif_invocation; + } + } + + ERTS_PTAB_LIST_DBG_CHK_DEL_LIST(ptab); + ERTS_PTAB_LIST_DBG_CHK_FREELIST(ptab, free_list); + erts_ptab_rwunlock(ptab); + + /* + * We do the actual free of deleted structures now when we + * have released the table lock instead of when we encountered + * them. This since free() isn't for free and we don't want to + * unnecessarily block other schedulers. + */ + while (free_list) { + ptdep = free_list; + free_list = ptdep->next; + erts_free(ERTS_ALC_T_PTAB_LIST_DEL, ptdep); + } + + have_reds -= reds; + if (have_reds < 0) + have_reds = 0; + BUMP_REDS(c_p, reds); + break; + } + + case BUILDING_RESULT: { + int conses, ix, min_ix; + Eterm *hp; + Eterm res = *res_accp; + + ERTS_PTAB_LIST_DBG_VERIFY_PIDS(ptlbdp); + ERTS_PTAB_LIST_DBG_CHK_RESLIST(res); + + ERTS_PTAB_LIST_DBG_TRACE(p->common.id, begin_build_res); + + have_reds = ERTS_BIF_REDS_LEFT(c_p); + conses = ERTS_PTAB_LIST_BIF_BUILD_RESULT_CONSES_PER_RED*have_reds; + min_ix = ptlbdp->pid_ix - conses; + if (min_ix < 0) { + min_ix = 0; + conses = ptlbdp->pid_ix; + } + + if (conses) { + hp = HAlloc(c_p, conses*2); + ERTS_PTAB_LIST_DBG_SAVE_HEAP_ALLOC(ptlbdp, hp, conses*2); + + for (ix = ptlbdp->pid_ix - 1; ix >= min_ix; ix--) { + ERTS_PTAB_LIST_ASSERT(erts_ptab_is_valid_id(ptlbdp->pid[ix])); + res = CONS(hp, ptlbdp->pid[ix], res); + hp += 2; + } + + ERTS_PTAB_LIST_DBG_VERIFY_HEAP_ALLOC_USED(ptlbdp, hp); + } + + ptlbdp->pid_ix = min_ix; + if (min_ix == 0) + ptlbdp->state = RETURN_RESULT; + else { + ptlbdp->pid_sz = min_ix; + ptlbdp->pid = erts_realloc(ERTS_ALC_T_PTAB_LIST_PIDS, + ptlbdp->pid, + sizeof(Eterm)*ptlbdp->pid_sz); +#if ERTS_PTAB_LIST_BIF_DEBUGLEVEL >= ERTS_PTAB_LIST_DBGLVL_CHK_FOUND_PIDS + ptlbdp->debug.pid_started + = erts_realloc(ERTS_ALC_T_PTAB_LIST_PIDS, + ptlbdp->debug.pid_started, + sizeof(Uint64) * ptlbdp->pid_sz); +#endif + } + reds = conses/ERTS_PTAB_LIST_BIF_BUILD_RESULT_CONSES_PER_RED; + BUMP_REDS(c_p, reds); + have_reds -= reds; + + ERTS_PTAB_LIST_DBG_CHK_RESLIST(res); + ERTS_PTAB_LIST_DBG_TRACE(c_p->common.id, end_build_res); + *res_accp = res; + break; + } + case RETURN_RESULT: + cleanup_ptab_list_bif_data(mbp); + return 1; + + default: + erl_exit(ERTS_ABORT_EXIT, + "%s:%d:ptab_list_bif_engine(): Invalid state: %d\n", + __FILE__, __LINE__, (int) ptlbdp->state); + } + + + } while (have_reds || ptlbdp->state == RETURN_RESULT); + + return 0; +} + +/* + * ptab_list_continue/2 is a hidden BIF that the original BIF traps to + * if there are too much work to do in one go. + */ + +static BIF_RETTYPE ptab_list_continue(BIF_ALIST_2) +{ + Eterm res_acc; + Binary *mbp; + + /* + * This bif cannot be called from erlang code. It can only be + * trapped to from other BIFs; therefore, a bad argument + * is an internal error and should never occur... + */ + + ERTS_PTAB_LIST_DBG_TRACE(BIF_P->common.id, call); + ERTS_PTAB_LIST_ASSERT(is_nil(BIF_ARG_1) || is_list(BIF_ARG_1)); + + res_acc = BIF_ARG_1; + + ERTS_PTAB_LIST_ASSERT(ERTS_TERM_IS_MAGIC_BINARY(BIF_ARG_2)); + + mbp = ((ProcBin *) binary_val(BIF_ARG_2))->val; + + ERTS_PTAB_LIST_ASSERT(ERTS_MAGIC_BIN_DESTRUCTOR(mbp) + == cleanup_ptab_list_bif_data); + ERTS_PTAB_LIST_ASSERT( + ((ErtsPTabListBifData *) ERTS_MAGIC_BIN_DATA(mbp))->debug.caller + == BIF_P->common.id); + + if (ptab_list_bif_engine(BIF_P, &res_acc, mbp)) { + ERTS_PTAB_LIST_DBG_TRACE(BIF_P->common.id, return); + BIF_RET(res_acc); + } + else { + ERTS_PTAB_LIST_DBG_TRACE(BIF_P->common.id, trap); + ERTS_BIF_YIELD2(&ptab_list_continue_export, BIF_P, res_acc, BIF_ARG_2); + } +} + +void +erts_ptab_init(void) +{ + /* ptab_list_continue/2 is a hidden BIF that the original BIF traps to. */ + erts_init_trap_export(&ptab_list_continue_export, + am_erlang, am_ptab_list_continue, 2, + &ptab_list_continue); + +} + +/* + * Debug stuff + */ + +Sint +erts_ptab_test_next_id(ErtsPTab *ptab, int set, Uint next) +{ + Uint64 ld; + Sint res; + Eterm data; + int first_pix = -1; + + erts_ptab_rwlock(ptab); + + if (!set) + ld = last_data_read_nob(ptab); + else { + + ld = (Uint64) next; + data = ERTS_PTAB_LastData2EtermData(ld); + if (ptab->r.o.invalid_data == data) { + ld += ptab->r.o.max; + ASSERT(erts_ptab_data2pix(ptab, data) + == erts_ptab_data2pix(ptab, + ERTS_PTAB_LastData2EtermData(ld))); + } + last_data_set_relb(ptab, ld); + } + + while (1) { + int pix; + ld++; + pix = (int) (ld % ptab->r.o.max); + if (first_pix < 0) + first_pix = pix; + else if (pix == first_pix) { + res = -1; + break; + } + if (ERTS_AINT_NULL == erts_ptab_pix2intptr_nob(ptab, pix)) { + data = ERTS_PTAB_LastData2EtermData(ld); + if (ptab->r.o.invalid_data == data) { + ld += ptab->r.o.max; + ASSERT(erts_ptab_data2pix(ptab, data) + == erts_ptab_data2pix(ptab, + ERTS_PTAB_LastData2EtermData(ld))); + data = ERTS_PTAB_LastData2EtermData(ld); + } + res = data; + break; + } + } + + erts_ptab_rwunlock(ptab); + + return res; +} + +static ERTS_INLINE ErtsPTabElementCommon * +ptab_pix2el(ErtsPTab *ptab, int ix) +{ + ErtsPTabElementCommon *ptab_el; + ASSERT(0 <= ix && ix < ptab->r.o.max); + ptab_el = (ErtsPTabElementCommon *) erts_ptab_pix2intptr_nob(ptab, ix); + if (ptab_el == ptab->r.o.invalid_element) + return NULL; + else + return ptab_el; +} + +Eterm +erts_debug_ptab_list(Process *c_p, ErtsPTab *ptab) +{ + int i; + Uint need; + Eterm res; + Eterm* hp; + Eterm *hp_end; + + erts_ptab_rwlock(ptab); + + res = NIL; + need = erts_ptab_count(ptab) * 2; + hp = HAlloc(c_p, need); /* we need two heap words for each id */ + hp_end = hp + need; + + /* make the list by scanning bakward */ + + + for (i = ptab->r.o.max-1; i >= 0; i--) { + ErtsPTabElementCommon *el = ptab_pix2el(ptab, i); + if (el) { + res = CONS(hp, el->id, res); + hp += 2; + } + } + + erts_ptab_rwunlock(ptab); + + HRelease(c_p, hp_end, hp); + + return res; +} + +Eterm +erts_debug_ptab_list_bif_info(Process *c_p, ErtsPTab *ptab) +{ + ERTS_DECL_AM(ptab_list_bif_info); + Eterm elements[] = { + AM_ptab_list_bif_info, + make_small((Uint) ERTS_PTAB_LIST_BIF_MIN_START_REDS), + make_small((Uint) ptab->list.data.chunks), + make_small((Uint) ERTS_PTAB_LIST_BIF_TAB_CHUNK_SIZE), + make_small((Uint) ERTS_PTAB_LIST_BIF_TAB_INSPECT_INDICES_PER_RED), + make_small((Uint) ERTS_PTAB_LIST_BIF_TAB_FREE_DELETED_REDS), + make_small((Uint) ERTS_PTAB_LIST_BIF_INSPECT_DELETED_PER_RED), + make_small((Uint) ERTS_PTAB_LIST_INSPECT_DELETED_MAX_REDS), + make_small((Uint) ERTS_PTAB_LIST_BIF_BUILD_RESULT_CONSES_PER_RED), + make_small((Uint) ERTS_PTAB_LIST_BIF_DEBUGLEVEL) + }; + Uint sz = 0; + Eterm *hp; + (void) erts_bld_tuplev(NULL, &sz, sizeof(elements)/sizeof(Eterm), elements); + hp = HAlloc(c_p, sz); + return erts_bld_tuplev(&hp, NULL, sizeof(elements)/sizeof(Eterm), elements); +} + +#if ERTS_PTAB_LIST_BIF_DEBUGLEVEL >= ERTS_PTAB_LIST_DBGLVL_CHK_FOUND_PIDS +static void +debug_ptab_list_check_found_pid(ErtsPTabListBifData *ptlbdp, + Eterm pid, + Uint64 ic, + int pid_should_be_found) +{ + int i; + for (i = 0; i < ptlbdp->pid_ix; i++) { + if (ptlbdp->pid[i] == pid && ptlbdp->debug.pid_started[i] == ic) { + ERTS_PTAB_LIST_ASSERT(pid_should_be_found); + return; + } + } + ERTS_PTAB_LIST_ASSERT(!pid_should_be_found); +} +#endif + +#if ERTS_PTAB_LIST_BIF_DEBUGLEVEL >= ERTS_PTAB_LIST_DBGLVL_CHK_RESLIST +static void +debug_ptab_list_check_res_list(Eterm list) +{ + while (is_list(list)) { + Eterm* consp = list_val(list); + Eterm hd = CAR(consp); + ERTS_PTAB_LIST_ASSERT(erts_ptab_is_valid_id(hd)); + list = CDR(consp); + } + + ERTS_PTAB_LIST_ASSERT(is_nil(list)); +} +#endif + +#if ERTS_PTAB_LIST_BIF_DEBUGLEVEL >= ERTS_PTAB_LIST_DBGLVL_CHK_PIDS + +static void +debug_ptab_list_save_all_pids(ErtsPTabListBifData *ptlbdp) +{ + int ix, tix, cpix; + ErtsPTab *ptab = ptlbdp->ptab; + ptlbdp->debug.correct_pids_verified = 0; + ptlbdp->debug.correct_pids = erts_alloc(ERTS_ALC_T_PTAB_LIST_PIDS, + sizeof(Eterm)*ptlbdp->pid_sz); + + for (tix = 0, cpix = 0; tix < ptab->r.o.max; tix++) { + ErtsPTabElementCommon *el = ptab_pix2el(ptab, tix); + if (el) { + ERTS_PTAB_LIST_ASSERT(erts_ptab_is_valid_id(el->id)); + ptlbdp->debug.correct_pids[cpix++] = el->id; + ERTS_PTAB_LIST_ASSERT(cpix <= ptlbdp->pid_sz); + } + } + ERTS_PTAB_LIST_ASSERT(cpix == ptlbdp->pid_sz); + + for (ix = 0; ix < ptlbdp->pid_sz; ix++) + ptlbdp->pid[ix] = make_small(ix); +} + +static void +debug_ptab_list_verify_all_pids(ErtsPTabListBifData *ptlbdp) +{ + int ix, cpix; + + ERTS_PTAB_LIST_ASSERT(ptlbdp->pid_ix == ptlbdp->pid_sz); + + for (ix = 0; ix < ptlbdp->pid_sz; ix++) { + int found = 0; + Eterm pid = ptlbdp->pid[ix]; + ERTS_PTAB_LIST_ASSERT(erts_ptab_is_valid_id(pid)); + for (cpix = ix; cpix < ptlbdp->pid_sz; cpix++) { + if (ptlbdp->debug.correct_pids[cpix] == pid) { + ptlbdp->debug.correct_pids[cpix] = NIL; + found = 1; + break; + } + } + if (!found) { + for (cpix = 0; cpix < ix; cpix++) { + if (ptlbdp->debug.correct_pids[cpix] == pid) { + ptlbdp->debug.correct_pids[cpix] = NIL; + found = 1; + break; + } + } + } + ERTS_PTAB_LIST_ASSERT(found); + } + ptlbdp->debug.correct_pids_verified = 1; + + erts_free(ERTS_ALC_T_PTAB_LIST_PIDS, ptlbdp->debug.correct_pids); + ptlbdp->debug.correct_pids = NULL; +} +#endif /* ERTS_PTAB_LIST_BIF_DEBUGLEVEL >= ERTS_PTAB_LIST_DBGLVL_CHK_PIDS */ + +#if ERTS_PTAB_LIST_BIF_DEBUGLEVEL >= ERTS_PTAB_LIST_DBGLVL_CHK_DEL_LIST +static void +debug_ptab_list_check_del_list(ErtsPTab *ptab) +{ + ERTS_SMP_LC_ASSERT(erts_smp_lc_ptab_is_rwlocked(ptab)); + if (!ptab->list.data.deleted.start) + ERTS_PTAB_LIST_ASSERT(!ptab->list.data.deleted.end); + else { + Uint64 curr_interval = erts_smp_current_interval_nob(erts_ptab_interval(ptab)); + Uint64 *prev_x_interval_p = NULL; + ErtsPTabDeletedElement *ptdep; + + for (ptdep = ptab->list.data.deleted.start; + ptdep; + ptdep = ptdep->next) { + if (!ptdep->prev) + ERTS_PTAB_LIST_ASSERT(ptab->list.data.deleted.start == ptdep); + else + ERTS_PTAB_LIST_ASSERT(ptdep->prev->next == ptdep); + if (!ptdep->next) + ERTS_PTAB_LIST_ASSERT(ptab->list.data.deleted.end == ptdep); + else + ERTS_PTAB_LIST_ASSERT(ptdep->next->prev == ptdep); + if (ptdep->ix < 0) { + Uint64 interval = ptdep->u.bif_invocation.interval; + ERTS_PTAB_LIST_ASSERT(interval <= curr_interval); + } + else { + Uint64 s_interval = ptdep->u.element.inserted; + Uint64 x_interval = ptdep->u.element.deleted; + + ERTS_PTAB_LIST_ASSERT(s_interval <= x_interval); + if (prev_x_interval_p) + ERTS_PTAB_LIST_ASSERT(*prev_x_interval_p <= x_interval); + prev_x_interval_p = &ptdep->u.element.deleted; + ERTS_PTAB_LIST_ASSERT( + erts_ptab_is_valid_id(ptdep->u.element.id)); + ERTS_PTAB_LIST_ASSERT(erts_ptab_id2pix(ptab, + ptdep->u.element.id) + == ptdep->ix); + + } + } + + } +} + +static void +debug_ptab_list_check_del_free_list(ErtsPTab *ptab, + ErtsPTabDeletedElement *free_list) +{ + if (ptab->list.data.deleted.start) { + ErtsPTabDeletedElement *fptdep; + ErtsPTabDeletedElement *ptdep; + + for (fptdep = free_list; fptdep; fptdep = fptdep->next) { + for (ptdep = ptab->list.data.deleted.start; + ptdep; + ptdep = ptdep->next) { + ERTS_PTAB_LIST_ASSERT(fptdep != ptdep); + } + } + } +} + +#endif + +#if ERTS_PTAB_LIST_BIF_DEBUGLEVEL != 0 + +static void +debug_ptab_list_assert_error(char* expr, const char* file, int line, const char *func) +{ + fflush(stdout); + erts_fprintf(stderr, "%s:%d:%s(): Assertion failed: %s\n", + (char *) file, line, (char *) func, expr); + fflush(stderr); + abort(); +} + +#endif diff --git a/erts/emulator/beam/erl_ptab.h b/erts/emulator/beam/erl_ptab.h new file mode 100644 index 0000000000..8a130f42a3 --- /dev/null +++ b/erts/emulator/beam/erl_ptab.h @@ -0,0 +1,472 @@ +/* + * %CopyrightBegin% + * + * Copyright Ericsson AB 2012. All Rights Reserved. + * + * The contents of this file are subject to the Erlang Public License, + * Version 1.1, (the "License"); you may not use this file except in + * compliance with the License. You should have received a copy of the + * Erlang Public License along with this software. If not, it can be + * retrieved online at http://www.erlang.org/. + * + * Software distributed under the License is distributed on an "AS IS" + * basis, WITHOUT WARRANTY OF ANY KIND, either express or implied. See + * the License for the specific language governing rights and limitations + * under the License. + * + * %CopyrightEnd% + */ + +/* + * Description: Process/Port table implementation. + * + * Author: Rickard Green + */ + +#ifndef ERL_PTAB_H__ +#define ERL_PTAB_H__ + +#include "sys.h" +#include "erl_term.h" +#include "erl_time.h" +#include "erl_utils.h" +#define ERL_THR_PROGRESS_TSD_TYPE_ONLY +#include "erl_thr_progress.h" +#undef ERL_THR_PROGRESS_TSD_TYPE_ONLY +#include "erl_alloc.h" +#include "erl_monitors.h" + +#define ERTS_TRACER_PROC(P) ((P)->common.tracer_proc) +#define ERTS_TRACE_FLAGS(P) ((P)->common.trace_flags) + +#define ERTS_P_LINKS(P) ((P)->common.u.alive.links) +#define ERTS_P_MONITORS(P) ((P)->common.u.alive.monitors) + +#define IS_TRACED(p) \ + (ERTS_TRACER_PROC((p)) != NIL) +#define ARE_TRACE_FLAGS_ON(p,tf) \ + ((ERTS_TRACE_FLAGS((p)) & (tf|F_SENSITIVE)) == (tf)) +#define IS_TRACED_FL(p,tf) \ + ( IS_TRACED(p) && ARE_TRACE_FLAGS_ON(p,tf) ) + +typedef struct { + Eterm id; +#ifdef ERTS_SMP + erts_atomic32_t refc; +#endif + Eterm tracer_proc; + Uint trace_flags; + union { + /* --- While being alive --- */ + struct { + Uint64 started_interval; + struct reg_proc *reg; + ErtsLink *links; + ErtsMonitor *monitors; +#ifdef ERTS_SMP + ErtsSmpPTimer *ptimer; +#else + ErlTimer tm; +#endif + } alive; + + /* --- While being released --- */ + ErtsThrPrgrLaterOp release; + } u; +} ErtsPTabElementCommon; + +typedef struct ErtsPTabDeletedElement_ ErtsPTabDeletedElement; + +typedef struct { + erts_smp_rwmtx_t rwmtx; + erts_interval_t interval; + struct { + ErtsPTabDeletedElement *start; + ErtsPTabDeletedElement *end; + } deleted; + int chunks; +} ErtsPTabListData; + +typedef struct { +#ifdef ARCH_32 + erts_smp_dw_atomic_t last_data; +#else + erts_smp_atomic_t last_data; +#endif + erts_smp_atomic32_t count; +} ErtsPTabVolatileData; + +typedef struct { + erts_smp_atomic_t *tab; + Uint32 max; + Uint32 tab_cache_lines; + Uint32 pix_per_cache_line; + Uint32 pix_mask; + Uint32 pix_cl_mask; + Uint32 pix_cl_shift; + Uint32 pix_cli_mask; + Uint32 pix_cli_shift; + ErtsPTabElementCommon *invalid_element; + Eterm invalid_data; + void (*release_element)(void *); +} ErtsPTabReadOnlyData; + +typedef struct { + /* + * Data mainly modified when someone is listing + * the content of the table. + */ + union { + ErtsPTabListData data; + char algn[ERTS_ALC_CACHE_LINE_ALIGN_SIZE(sizeof(ErtsPTabListData))]; + } list; + + /* + * Frequently modified data. + */ + union { + ErtsPTabVolatileData tile; + char algn[ERTS_ALC_CACHE_LINE_ALIGN_SIZE(sizeof(ErtsPTabVolatileData))]; + } vola; + + /* + * Read only data. + */ + union { + ErtsPTabReadOnlyData o; + char algn[ERTS_ALC_CACHE_LINE_ALIGN_SIZE(sizeof(ErtsPTabReadOnlyData))]; + } r; +} ErtsPTab; + +#define ERTS_PTAB_ID_DATA_SIZE 28 +#define ERTS_PTAB_ID_DATA_SHIFT (_TAG_IMMED1_SIZE) +/* ERTS_PTAB_MAX_SIZE must be a power of 2 */ +#define ERTS_PTAB_MAX_SIZE (SWORD_CONSTANT(1) << 27) +#if (ERTS_PTAB_MAX_SIZE-1) > MAX_SMALL +# error "The maximum number of processes/ports must fit in a SMALL." +#endif + + +/* + * Currently pids and ports are allowed. + */ +#if _PID_DATA_SIZE != ERTS_PTAB_ID_DATA_SIZE +# error "Unexpected pid data size" +#endif +#if _PID_DATA_SHIFT != ERTS_PTAB_ID_DATA_SHIFT +# error "Unexpected pid tag size" +#endif +#if _PORT_DATA_SIZE != ERTS_PTAB_ID_DATA_SIZE +# error "Unexpected port data size" +#endif +#if _PORT_DATA_SHIFT != ERTS_PTAB_ID_DATA_SHIFT +# error "Unexpected port tag size" +#endif + +#define ERTS_PTAB_INVALID_ID(TAG) \ + ((Eterm) \ + ((((1 << ERTS_PTAB_ID_DATA_SIZE) - 1) << ERTS_PTAB_ID_DATA_SHIFT) \ + | (TAG))) + +#define erts_ptab_is_valid_id(ID) \ + (is_internal_pid((ID)) || is_internal_port((ID))) + +void erts_ptab_init(void); +void erts_ptab_init_table(ErtsPTab *ptab, + ErtsAlcType_t atype, + void (*release_element)(void *), + ErtsPTabElementCommon *invalid_element, + int size, + char *name); +int erts_ptab_new_element(ErtsPTab *ptab, + ErtsPTabElementCommon *ptab_el, + void *init_arg, + void (*init_ptab_el)(void *, Eterm)); +void erts_ptab_delete_element(ErtsPTab *ptab, + ErtsPTabElementCommon *ptab_el); +int erts_ptab_initialized(ErtsPTab *ptab); + +ERTS_GLB_INLINE erts_interval_t *erts_ptab_interval(ErtsPTab *ptab); +ERTS_GLB_INLINE int erts_ptab_max(ErtsPTab *ptab); +ERTS_GLB_INLINE int erts_ptab_count(ErtsPTab *ptab); +ERTS_GLB_INLINE Uint erts_ptab_pixdata2data(ErtsPTab *ptab, Eterm pixdata); +ERTS_GLB_INLINE Uint32 erts_ptab_pixdata2pix(ErtsPTab *ptab, Eterm pixdata); +ERTS_GLB_INLINE Uint32 erts_ptab_data2pix(ErtsPTab *ptab, Eterm data); +ERTS_GLB_INLINE Uint erts_ptab_data2pixdata(ErtsPTab *ptab, Eterm data); +ERTS_GLB_INLINE Eterm erts_ptab_make_id(ErtsPTab *ptab, Eterm data, Eterm tag); +ERTS_GLB_INLINE int erts_ptab_id2pix(ErtsPTab *ptab, Eterm id); +ERTS_GLB_INLINE Uint erts_ptab_id2data(ErtsPTab *ptab, Eterm id); +ERTS_GLB_INLINE erts_aint_t erts_ptab_pix2intptr_nob(ErtsPTab *ptab, int ix); +ERTS_GLB_INLINE erts_aint_t erts_ptab_pix2intptr_ddrb(ErtsPTab *ptab, int ix); +ERTS_GLB_INLINE erts_aint_t erts_ptab_pix2intptr_rb(ErtsPTab *ptab, int ix); +ERTS_GLB_INLINE erts_aint_t erts_ptab_pix2intptr_acqb(ErtsPTab *ptab, int ix); +ERTS_GLB_INLINE void erts_ptab_inc_refc(ErtsPTabElementCommon *ptab_el); +ERTS_GLB_INLINE int erts_ptab_dec_test_refc(ErtsPTabElementCommon *ptab_el); +ERTS_GLB_INLINE int erts_ptab_add_test_refc(ErtsPTabElementCommon *ptab_el, + Sint32 add_refc); +ERTS_GLB_INLINE void erts_ptab_rlock(ErtsPTab *ptab); +ERTS_GLB_INLINE int erts_ptab_tryrlock(ErtsPTab *ptab); +ERTS_GLB_INLINE void erts_ptab_runlock(ErtsPTab *ptab); +ERTS_GLB_INLINE void erts_ptab_rwlock(ErtsPTab *ptab); +ERTS_GLB_INLINE int erts_ptab_tryrwlock(ErtsPTab *ptab); +ERTS_GLB_INLINE void erts_ptab_rwunlock(ErtsPTab *ptab); +ERTS_GLB_INLINE int erts_smp_lc_ptab_is_rlocked(ErtsPTab *ptab); +ERTS_GLB_INLINE int erts_smp_lc_ptab_is_rwlocked(ErtsPTab *ptab); + +#if ERTS_GLB_INLINE_INCL_FUNC_DEF + +ERTS_GLB_INLINE erts_interval_t * +erts_ptab_interval(ErtsPTab *ptab) +{ + return &ptab->list.data.interval; +} + +ERTS_GLB_INLINE int +erts_ptab_max(ErtsPTab *ptab) +{ + int max = ptab->r.o.max; + return max == ERTS_PTAB_MAX_SIZE ? max - 1 : max; +} + +ERTS_GLB_INLINE int +erts_ptab_count(ErtsPTab *ptab) +{ + int max = ptab->r.o.max; + erts_aint32_t res = erts_smp_atomic32_read_nob(&ptab->vola.tile.count); + if (max == ERTS_PTAB_MAX_SIZE) { + max--; + res--; + } + if (res > max) + return max; + ASSERT(res >= 0); + return (int) res; + +} + +ERTS_GLB_INLINE Uint erts_ptab_pixdata2data(ErtsPTab *ptab, Eterm pixdata) +{ + Uint32 data = ((Uint32) pixdata) & ~ptab->r.o.pix_mask; + data |= (pixdata >> ptab->r.o.pix_cl_shift) & ptab->r.o.pix_cl_mask; + data |= (pixdata & ptab->r.o.pix_cli_mask) << ptab->r.o.pix_cli_shift; + return data; +} + +ERTS_GLB_INLINE Uint32 erts_ptab_pixdata2pix(ErtsPTab *ptab, Eterm pixdata) +{ + return ((Uint32) pixdata) & ptab->r.o.pix_mask; +} + +ERTS_GLB_INLINE Uint32 erts_ptab_data2pix(ErtsPTab *ptab, Eterm data) +{ + Uint32 n, pix; + n = (Uint32) data; + pix = ((n & ptab->r.o.pix_cl_mask) << ptab->r.o.pix_cl_shift); + pix += ((n >> ptab->r.o.pix_cli_shift) & ptab->r.o.pix_cli_mask); + ASSERT(0 <= pix && pix < ptab->r.o.max); + return pix; +} + +ERTS_GLB_INLINE Uint erts_ptab_data2pixdata(ErtsPTab *ptab, Eterm data) +{ + Uint pixdata = data & ~((Uint) ptab->r.o.pix_mask); + pixdata |= (Uint) erts_ptab_data2pix(ptab, data); + ASSERT(data == erts_ptab_pixdata2data(ptab, pixdata)); + return pixdata; +} + +#if ERTS_SIZEOF_TERM == 8 + +ERTS_GLB_INLINE Eterm +erts_ptab_make_id(ErtsPTab *ptab, Eterm data, Eterm tag) +{ + HUint huint; + Uint32 low_data = (Uint32) data; + low_data &= (1 << ERTS_PTAB_ID_DATA_SIZE) - 1; + low_data <<= ERTS_PTAB_ID_DATA_SHIFT; + huint.hval[ERTS_HUINT_HVAL_HIGH] = erts_ptab_data2pix(ptab, data); + huint.hval[ERTS_HUINT_HVAL_LOW] = low_data | ((Uint32) tag); + return (Eterm) huint.val; +} + +ERTS_GLB_INLINE int +erts_ptab_id2pix(ErtsPTab *ptab, Eterm id) +{ + HUint huint; + huint.val = id; + return (int) huint.hval[ERTS_HUINT_HVAL_HIGH]; +} + +ERTS_GLB_INLINE Uint +erts_ptab_id2data(ErtsPTab *ptab, Eterm id) +{ + HUint huint; + huint.val = id; + return (Uint) (huint.hval[ERTS_HUINT_HVAL_LOW] >> ERTS_PTAB_ID_DATA_SHIFT); +} + +#elif ERTS_SIZEOF_TERM == 4 + +ERTS_GLB_INLINE Eterm +erts_ptab_make_id(ErtsPTab *ptab, Eterm data, Eterm tag) +{ + Eterm id; + data &= ((1 << ERTS_PTAB_ID_DATA_SIZE) - 1); + id = (Eterm) erts_ptab_data2pixdata(ptab, data); + return (id << ERTS_PTAB_ID_DATA_SHIFT) | tag; +} + +ERTS_GLB_INLINE int +erts_ptab_id2pix(ErtsPTab *ptab, Eterm id) +{ + Uint pixdata = (Uint) id; + pixdata >>= ERTS_PTAB_ID_DATA_SHIFT; + return (int) erts_ptab_pixdata2pix(ptab, pixdata); +} + +ERTS_GLB_INLINE Uint +erts_ptab_id2data(ErtsPTab *ptab, Eterm id) +{ + Uint pixdata = (Uint) id; + pixdata >>= ERTS_PTAB_ID_DATA_SHIFT; + return erts_ptab_pixdata2data(ptab, pixdata); +} + +#else +#error "Unsupported size of term" +#endif + +ERTS_GLB_INLINE erts_aint_t erts_ptab_pix2intptr_nob(ErtsPTab *ptab, int ix) +{ + ASSERT(0 <= ix && ix < ptab->r.o.max); + return erts_smp_atomic_read_nob(&ptab->r.o.tab[ix]); +} + +ERTS_GLB_INLINE erts_aint_t erts_ptab_pix2intptr_ddrb(ErtsPTab *ptab, int ix) +{ + ASSERT(0 <= ix && ix < ptab->r.o.max); + return erts_smp_atomic_read_ddrb(&ptab->r.o.tab[ix]); +} + +ERTS_GLB_INLINE erts_aint_t erts_ptab_pix2intptr_rb(ErtsPTab *ptab, int ix) +{ + ASSERT(0 <= ix && ix < ptab->r.o.max); + return erts_smp_atomic_read_rb(&ptab->r.o.tab[ix]); +} + +ERTS_GLB_INLINE erts_aint_t erts_ptab_pix2intptr_acqb(ErtsPTab *ptab, int ix) +{ + ASSERT(0 <= ix && ix < ptab->r.o.max); + return erts_smp_atomic_read_acqb(&ptab->r.o.tab[ix]); +} + +ERTS_GLB_INLINE void erts_ptab_inc_refc(ErtsPTabElementCommon *ptab_el) +{ +#ifdef ERTS_SMP +#ifdef ERTS_ENABLE_LOCK_CHECK + erts_aint32_t refc = erts_atomic32_inc_read_nob(&ptab_el->refc); + ERTS_SMP_LC_ASSERT(refc > 1); +#else + erts_atomic32_inc_nob(&ptab_el->refc); +#endif +#endif +} + +ERTS_GLB_INLINE int erts_ptab_dec_test_refc(ErtsPTabElementCommon *ptab_el) +{ +#ifdef ERTS_SMP + erts_aint32_t refc = erts_atomic32_dec_read_nob(&ptab_el->refc); + ERTS_SMP_LC_ASSERT(refc >= 0); + return (int) refc; +#else + return 0; +#endif +} + +ERTS_GLB_INLINE int erts_ptab_add_test_refc(ErtsPTabElementCommon *ptab_el, + Sint32 add_refc) +{ +#ifdef ERTS_SMP + erts_aint32_t refc; + +#ifndef ERTS_ENABLE_LOCK_CHECK + if (add_refc >= 0) { + erts_atomic32_add_nob(&ptab_el->refc, + (erts_aint32_t) add_refc); + return 1; + } +#endif + + refc = erts_atomic32_add_read_nob(&ptab_el->refc, + (erts_aint32_t) add_refc); + ERTS_SMP_LC_ASSERT(refc >= 0); + return (int) refc; +#else + return 0; +#endif +} + +ERTS_GLB_INLINE void erts_ptab_rlock(ErtsPTab *ptab) +{ + erts_smp_rwmtx_rlock(&ptab->list.data.rwmtx); +} + +ERTS_GLB_INLINE int erts_ptab_tryrlock(ErtsPTab *ptab) +{ + return erts_smp_rwmtx_tryrlock(&ptab->list.data.rwmtx); +} + +ERTS_GLB_INLINE void erts_ptab_runlock(ErtsPTab *ptab) +{ + erts_smp_rwmtx_runlock(&ptab->list.data.rwmtx); +} + +ERTS_GLB_INLINE void erts_ptab_rwlock(ErtsPTab *ptab) +{ + erts_smp_rwmtx_rwlock(&ptab->list.data.rwmtx); +} + +ERTS_GLB_INLINE int erts_ptab_tryrwlock(ErtsPTab *ptab) +{ + return erts_smp_rwmtx_tryrwlock(&ptab->list.data.rwmtx); +} + +ERTS_GLB_INLINE void erts_ptab_rwunlock(ErtsPTab *ptab) +{ + erts_smp_rwmtx_rwunlock(&ptab->list.data.rwmtx); +} + +ERTS_GLB_INLINE int erts_smp_lc_ptab_is_rlocked(ErtsPTab *ptab) +{ + return erts_smp_lc_rwmtx_is_rlocked(&ptab->list.data.rwmtx); +} + +ERTS_GLB_INLINE int erts_smp_lc_ptab_is_rwlocked(ErtsPTab *ptab) +{ + return erts_smp_lc_rwmtx_is_rwlocked(&ptab->list.data.rwmtx); +} + +#endif + +#endif + +#if defined(ERTS_PTAB_WANT_BIF_IMPL__) && !defined(ERTS_PTAB_LIST__) +#define ERTS_PTAB_LIST__ + +#include "erl_process.h" +#include "bif.h" + +BIF_RETTYPE erts_ptab_list(struct process *c_p, ErtsPTab *ptab); + +#endif + +#if defined(ERTS_PTAB_WANT_DEBUG_FUNCS__) && !defined(ERTS_PTAB_DEBUG_FUNCS__) +#define ERTS_PTAB_DEBUG_FUNCS__ +#include "erl_process.h" + +/* Debug functions */ +Sint erts_ptab_test_next_id(ErtsPTab *ptab, int set, Uint next); +Eterm erts_debug_ptab_list(Process *c_p, ErtsPTab *ptab); +Eterm erts_debug_ptab_list_bif_info(Process *c_p, ErtsPTab *ptab); + +#endif diff --git a/erts/emulator/beam/erl_smp.h b/erts/emulator/beam/erl_smp.h index a32e9d9d7c..34c90c0bda 100644 --- a/erts/emulator/beam/erl_smp.h +++ b/erts/emulator/beam/erl_smp.h @@ -274,6 +274,7 @@ ERTS_GLB_INLINE void erts_smp_thr_sigwait(const sigset_t *set, int *sig); #define erts_smp_atomic_read_band_nob erts_atomic_read_band_nob #define erts_smp_atomic_xchg_nob erts_atomic_xchg_nob #define erts_smp_atomic_cmpxchg_nob erts_atomic_cmpxchg_nob +#define erts_smp_atomic_read_bset_nob erts_atomic_read_bset_nob #define erts_smp_atomic_init_mb erts_atomic_init_mb #define erts_smp_atomic_set_mb erts_atomic_set_mb @@ -288,6 +289,7 @@ ERTS_GLB_INLINE void erts_smp_thr_sigwait(const sigset_t *set, int *sig); #define erts_smp_atomic_read_band_mb erts_atomic_read_band_mb #define erts_smp_atomic_xchg_mb erts_atomic_xchg_mb #define erts_smp_atomic_cmpxchg_mb erts_atomic_cmpxchg_mb +#define erts_smp_atomic_read_bset_mb erts_atomic_read_bset_mb #define erts_smp_atomic_init_acqb erts_atomic_init_acqb #define erts_smp_atomic_set_acqb erts_atomic_set_acqb @@ -302,6 +304,7 @@ ERTS_GLB_INLINE void erts_smp_thr_sigwait(const sigset_t *set, int *sig); #define erts_smp_atomic_read_band_acqb erts_atomic_read_band_acqb #define erts_smp_atomic_xchg_acqb erts_atomic_xchg_acqb #define erts_smp_atomic_cmpxchg_acqb erts_atomic_cmpxchg_acqb +#define erts_smp_atomic_read_bset_acqb erts_atomic_read_bset_acqb #define erts_smp_atomic_init_relb erts_atomic_init_relb #define erts_smp_atomic_set_relb erts_atomic_set_relb @@ -316,6 +319,7 @@ ERTS_GLB_INLINE void erts_smp_thr_sigwait(const sigset_t *set, int *sig); #define erts_smp_atomic_read_band_relb erts_atomic_read_band_relb #define erts_smp_atomic_xchg_relb erts_atomic_xchg_relb #define erts_smp_atomic_cmpxchg_relb erts_atomic_cmpxchg_relb +#define erts_smp_atomic_read_bset_relb erts_atomic_read_bset_relb #define erts_smp_atomic_init_ddrb erts_atomic_init_ddrb #define erts_smp_atomic_set_ddrb erts_atomic_set_ddrb @@ -330,6 +334,7 @@ ERTS_GLB_INLINE void erts_smp_thr_sigwait(const sigset_t *set, int *sig); #define erts_smp_atomic_read_band_ddrb erts_atomic_read_band_ddrb #define erts_smp_atomic_xchg_ddrb erts_atomic_xchg_ddrb #define erts_smp_atomic_cmpxchg_ddrb erts_atomic_cmpxchg_ddrb +#define erts_smp_atomic_read_bset_ddrb erts_atomic_read_bset_ddrb #define erts_smp_atomic_init_rb erts_atomic_init_rb #define erts_smp_atomic_set_rb erts_atomic_set_rb @@ -344,6 +349,7 @@ ERTS_GLB_INLINE void erts_smp_thr_sigwait(const sigset_t *set, int *sig); #define erts_smp_atomic_read_band_rb erts_atomic_read_band_rb #define erts_smp_atomic_xchg_rb erts_atomic_xchg_rb #define erts_smp_atomic_cmpxchg_rb erts_atomic_cmpxchg_rb +#define erts_smp_atomic_read_bset_rb erts_atomic_read_bset_rb #define erts_smp_atomic_init_wb erts_atomic_init_wb #define erts_smp_atomic_set_wb erts_atomic_set_wb @@ -358,6 +364,7 @@ ERTS_GLB_INLINE void erts_smp_thr_sigwait(const sigset_t *set, int *sig); #define erts_smp_atomic_read_band_wb erts_atomic_read_band_wb #define erts_smp_atomic_xchg_wb erts_atomic_xchg_wb #define erts_smp_atomic_cmpxchg_wb erts_atomic_cmpxchg_wb +#define erts_smp_atomic_read_bset_wb erts_atomic_read_bset_wb /* 32-bit atomics */ @@ -374,6 +381,7 @@ ERTS_GLB_INLINE void erts_smp_thr_sigwait(const sigset_t *set, int *sig); #define erts_smp_atomic32_read_band_nob erts_atomic32_read_band_nob #define erts_smp_atomic32_xchg_nob erts_atomic32_xchg_nob #define erts_smp_atomic32_cmpxchg_nob erts_atomic32_cmpxchg_nob +#define erts_smp_atomic32_read_bset_nob erts_atomic32_read_bset_nob #define erts_smp_atomic32_init_mb erts_atomic32_init_mb #define erts_smp_atomic32_set_mb erts_atomic32_set_mb @@ -388,6 +396,7 @@ ERTS_GLB_INLINE void erts_smp_thr_sigwait(const sigset_t *set, int *sig); #define erts_smp_atomic32_read_band_mb erts_atomic32_read_band_mb #define erts_smp_atomic32_xchg_mb erts_atomic32_xchg_mb #define erts_smp_atomic32_cmpxchg_mb erts_atomic32_cmpxchg_mb +#define erts_smp_atomic32_read_bset_mb erts_atomic32_read_bset_mb #define erts_smp_atomic32_init_acqb erts_atomic32_init_acqb #define erts_smp_atomic32_set_acqb erts_atomic32_set_acqb @@ -402,6 +411,7 @@ ERTS_GLB_INLINE void erts_smp_thr_sigwait(const sigset_t *set, int *sig); #define erts_smp_atomic32_read_band_acqb erts_atomic32_read_band_acqb #define erts_smp_atomic32_xchg_acqb erts_atomic32_xchg_acqb #define erts_smp_atomic32_cmpxchg_acqb erts_atomic32_cmpxchg_acqb +#define erts_smp_atomic32_read_bset_acqb erts_atomic32_read_bset_acqb #define erts_smp_atomic32_init_relb erts_atomic32_init_relb #define erts_smp_atomic32_set_relb erts_atomic32_set_relb @@ -416,6 +426,7 @@ ERTS_GLB_INLINE void erts_smp_thr_sigwait(const sigset_t *set, int *sig); #define erts_smp_atomic32_read_band_relb erts_atomic32_read_band_relb #define erts_smp_atomic32_xchg_relb erts_atomic32_xchg_relb #define erts_smp_atomic32_cmpxchg_relb erts_atomic32_cmpxchg_relb +#define erts_smp_atomic32_read_bset_relb erts_atomic32_read_bset_relb #define erts_smp_atomic32_init_ddrb erts_atomic32_init_ddrb #define erts_smp_atomic32_set_ddrb erts_atomic32_set_ddrb @@ -430,6 +441,7 @@ ERTS_GLB_INLINE void erts_smp_thr_sigwait(const sigset_t *set, int *sig); #define erts_smp_atomic32_read_band_ddrb erts_atomic32_read_band_ddrb #define erts_smp_atomic32_xchg_ddrb erts_atomic32_xchg_ddrb #define erts_smp_atomic32_cmpxchg_ddrb erts_atomic32_cmpxchg_ddrb +#define erts_smp_atomic32_read_bset_ddrb erts_atomic32_read_bset_ddrb #define erts_smp_atomic32_init_rb erts_atomic32_init_rb #define erts_smp_atomic32_set_rb erts_atomic32_set_rb @@ -444,6 +456,7 @@ ERTS_GLB_INLINE void erts_smp_thr_sigwait(const sigset_t *set, int *sig); #define erts_smp_atomic32_read_band_rb erts_atomic32_read_band_rb #define erts_smp_atomic32_xchg_rb erts_atomic32_xchg_rb #define erts_smp_atomic32_cmpxchg_rb erts_atomic32_cmpxchg_rb +#define erts_smp_atomic32_read_bset_rb erts_atomic32_read_bset_rb #define erts_smp_atomic32_init_wb erts_atomic32_init_wb #define erts_smp_atomic32_set_wb erts_atomic32_set_wb @@ -458,6 +471,7 @@ ERTS_GLB_INLINE void erts_smp_thr_sigwait(const sigset_t *set, int *sig); #define erts_smp_atomic32_read_band_wb erts_atomic32_read_band_wb #define erts_smp_atomic32_xchg_wb erts_atomic32_xchg_wb #define erts_smp_atomic32_cmpxchg_wb erts_atomic32_cmpxchg_wb +#define erts_smp_atomic32_read_bset_wb erts_atomic32_read_bset_wb #else /* !ERTS_SMP */ @@ -513,6 +527,7 @@ ERTS_GLB_INLINE void erts_smp_thr_sigwait(const sigset_t *set, int *sig); #define erts_smp_atomic_read_band_nob erts_no_atomic_read_band #define erts_smp_atomic_xchg_nob erts_no_atomic_xchg #define erts_smp_atomic_cmpxchg_nob erts_no_atomic_cmpxchg +#define erts_smp_atomic_read_bset_nob erts_no_atomic_read_bset #define erts_smp_atomic_init_mb erts_no_atomic_set #define erts_smp_atomic_set_mb erts_no_atomic_set @@ -527,6 +542,7 @@ ERTS_GLB_INLINE void erts_smp_thr_sigwait(const sigset_t *set, int *sig); #define erts_smp_atomic_read_band_mb erts_no_atomic_read_band #define erts_smp_atomic_xchg_mb erts_no_atomic_xchg #define erts_smp_atomic_cmpxchg_mb erts_no_atomic_cmpxchg +#define erts_smp_atomic_read_bset_mb erts_no_atomic_read_bset #define erts_smp_atomic_init_acqb erts_no_atomic_set #define erts_smp_atomic_set_acqb erts_no_atomic_set @@ -541,6 +557,7 @@ ERTS_GLB_INLINE void erts_smp_thr_sigwait(const sigset_t *set, int *sig); #define erts_smp_atomic_read_band_acqb erts_no_atomic_read_band #define erts_smp_atomic_xchg_acqb erts_no_atomic_xchg #define erts_smp_atomic_cmpxchg_acqb erts_no_atomic_cmpxchg +#define erts_smp_atomic_read_bset_acqb erts_no_atomic_read_bset #define erts_smp_atomic_init_relb erts_no_atomic_set #define erts_smp_atomic_set_relb erts_no_atomic_set @@ -555,6 +572,7 @@ ERTS_GLB_INLINE void erts_smp_thr_sigwait(const sigset_t *set, int *sig); #define erts_smp_atomic_read_band_relb erts_no_atomic_read_band #define erts_smp_atomic_xchg_relb erts_no_atomic_xchg #define erts_smp_atomic_cmpxchg_relb erts_no_atomic_cmpxchg +#define erts_smp_atomic_read_bset_relb erts_no_atomic_read_bset #define erts_smp_atomic_init_ddrb erts_no_atomic_set #define erts_smp_atomic_set_ddrb erts_no_atomic_set @@ -569,6 +587,7 @@ ERTS_GLB_INLINE void erts_smp_thr_sigwait(const sigset_t *set, int *sig); #define erts_smp_atomic_read_band_ddrb erts_no_atomic_read_band #define erts_smp_atomic_xchg_ddrb erts_no_atomic_xchg #define erts_smp_atomic_cmpxchg_ddrb erts_no_atomic_cmpxchg +#define erts_smp_atomic_read_bset_ddrb erts_no_atomic_read_bset #define erts_smp_atomic_init_rb erts_no_atomic_set #define erts_smp_atomic_set_rb erts_no_atomic_set @@ -583,6 +602,7 @@ ERTS_GLB_INLINE void erts_smp_thr_sigwait(const sigset_t *set, int *sig); #define erts_smp_atomic_read_band_rb erts_no_atomic_read_band #define erts_smp_atomic_xchg_rb erts_no_atomic_xchg #define erts_smp_atomic_cmpxchg_rb erts_no_atomic_cmpxchg +#define erts_smp_atomic_read_bset_rb erts_no_atomic_read_bset #define erts_smp_atomic_init_wb erts_no_atomic_set #define erts_smp_atomic_set_wb erts_no_atomic_set @@ -597,6 +617,7 @@ ERTS_GLB_INLINE void erts_smp_thr_sigwait(const sigset_t *set, int *sig); #define erts_smp_atomic_read_band_wb erts_no_atomic_read_band #define erts_smp_atomic_xchg_wb erts_no_atomic_xchg #define erts_smp_atomic_cmpxchg_wb erts_no_atomic_cmpxchg +#define erts_smp_atomic_read_bset_wb erts_no_atomic_read_bset /* 32-bit atomics */ @@ -613,6 +634,7 @@ ERTS_GLB_INLINE void erts_smp_thr_sigwait(const sigset_t *set, int *sig); #define erts_smp_atomic32_read_band_nob erts_no_atomic32_read_band #define erts_smp_atomic32_xchg_nob erts_no_atomic32_xchg #define erts_smp_atomic32_cmpxchg_nob erts_no_atomic32_cmpxchg +#define erts_smp_atomic32_read_bset_nob erts_no_atomic32_read_bset #define erts_smp_atomic32_init_mb erts_no_atomic32_set #define erts_smp_atomic32_set_mb erts_no_atomic32_set @@ -627,6 +649,7 @@ ERTS_GLB_INLINE void erts_smp_thr_sigwait(const sigset_t *set, int *sig); #define erts_smp_atomic32_read_band_mb erts_no_atomic32_read_band #define erts_smp_atomic32_xchg_mb erts_no_atomic32_xchg #define erts_smp_atomic32_cmpxchg_mb erts_no_atomic32_cmpxchg +#define erts_smp_atomic32_read_bset_mb erts_no_atomic32_read_bset #define erts_smp_atomic32_init_acqb erts_no_atomic32_set #define erts_smp_atomic32_set_acqb erts_no_atomic32_set @@ -641,6 +664,7 @@ ERTS_GLB_INLINE void erts_smp_thr_sigwait(const sigset_t *set, int *sig); #define erts_smp_atomic32_read_band_acqb erts_no_atomic32_read_band #define erts_smp_atomic32_xchg_acqb erts_no_atomic32_xchg #define erts_smp_atomic32_cmpxchg_acqb erts_no_atomic32_cmpxchg +#define erts_smp_atomic32_read_bset_acqb erts_no_atomic32_read_bset #define erts_smp_atomic32_init_relb erts_no_atomic32_set #define erts_smp_atomic32_set_relb erts_no_atomic32_set @@ -655,6 +679,7 @@ ERTS_GLB_INLINE void erts_smp_thr_sigwait(const sigset_t *set, int *sig); #define erts_smp_atomic32_read_band_relb erts_no_atomic32_read_band #define erts_smp_atomic32_xchg_relb erts_no_atomic32_xchg #define erts_smp_atomic32_cmpxchg_relb erts_no_atomic32_cmpxchg +#define erts_smp_atomic32_read_bset_relb erts_no_atomic32_read_bset #define erts_smp_atomic32_init_ddrb erts_no_atomic32_set #define erts_smp_atomic32_set_ddrb erts_no_atomic32_set @@ -669,6 +694,7 @@ ERTS_GLB_INLINE void erts_smp_thr_sigwait(const sigset_t *set, int *sig); #define erts_smp_atomic32_read_band_ddrb erts_no_atomic32_read_band #define erts_smp_atomic32_xchg_ddrb erts_no_atomic32_xchg #define erts_smp_atomic32_cmpxchg_ddrb erts_no_atomic32_cmpxchg +#define erts_smp_atomic32_read_bset_ddrb erts_no_atomic32_read_bset #define erts_smp_atomic32_init_rb erts_no_atomic32_set #define erts_smp_atomic32_set_rb erts_no_atomic32_set @@ -683,6 +709,7 @@ ERTS_GLB_INLINE void erts_smp_thr_sigwait(const sigset_t *set, int *sig); #define erts_smp_atomic32_read_band_rb erts_no_atomic32_read_band #define erts_smp_atomic32_xchg_rb erts_no_atomic32_xchg #define erts_smp_atomic32_cmpxchg_rb erts_no_atomic32_cmpxchg +#define erts_smp_atomic32_read_bset_rb erts_no_atomic32_read_bset #define erts_smp_atomic32_init_wb erts_no_atomic32_set #define erts_smp_atomic32_set_wb erts_no_atomic32_set @@ -697,6 +724,7 @@ ERTS_GLB_INLINE void erts_smp_thr_sigwait(const sigset_t *set, int *sig); #define erts_smp_atomic32_read_band_wb erts_no_atomic32_read_band #define erts_smp_atomic32_xchg_wb erts_no_atomic32_xchg #define erts_smp_atomic32_cmpxchg_wb erts_no_atomic32_cmpxchg +#define erts_smp_atomic32_read_bset_wb erts_no_atomic32_read_bset #endif /* !ERTS_SMP */ diff --git a/erts/emulator/beam/erl_sys_driver.h b/erts/emulator/beam/erl_sys_driver.h index d429d0ce96..b991a2840c 100644 --- a/erts/emulator/beam/erl_sys_driver.h +++ b/erts/emulator/beam/erl_sys_driver.h @@ -31,7 +31,6 @@ #define ERL_SYS_DRV typedef long ErlDrvEvent; /* An event to be selected on. */ -typedef long ErlDrvPort; /* A port descriptor. */ /* typedef struct _SysDriverOpts SysDriverOpts; defined in sys.h */ diff --git a/erts/emulator/beam/erl_term.c b/erts/emulator/beam/erl_term.c index bf7774f882..4587cd84d1 100644 --- a/erts/emulator/beam/erl_term.c +++ b/erts/emulator/beam/erl_term.c @@ -133,7 +133,7 @@ ET_DEFINE_CHECKED(Uint,unsigned_val,Eterm,is_small); ET_DEFINE_CHECKED(Sint,signed_val,Eterm,is_small); ET_DEFINE_CHECKED(Uint,atom_val,Eterm,is_atom); ET_DEFINE_CHECKED(Uint,header_arity,Eterm,is_header); -ET_DEFINE_CHECKED(Uint,arityval,Eterm,is_arity_value); +ET_DEFINE_CHECKED(Uint,arityval,Eterm,is_sane_arity_value); ET_DEFINE_CHECKED(Uint,thing_arityval,Eterm,is_thing); ET_DEFINE_CHECKED(Uint,thing_subtag,Eterm,is_thing); ET_DEFINE_CHECKED(Eterm*,binary_val,Wterm,is_binary); @@ -144,9 +144,7 @@ ET_DEFINE_CHECKED(Uint,bignum_header_arity,Eterm,_is_bignum_header); ET_DEFINE_CHECKED(Eterm*,big_val,Wterm,is_big); ET_DEFINE_CHECKED(Eterm*,float_val,Wterm,is_float); ET_DEFINE_CHECKED(Eterm*,tuple_val,Wterm,is_tuple); -ET_DEFINE_CHECKED(Uint,internal_pid_data,Eterm,is_internal_pid); ET_DEFINE_CHECKED(struct erl_node_*,internal_pid_node,Eterm,is_internal_pid); -ET_DEFINE_CHECKED(Uint,internal_port_data,Eterm,is_internal_port); ET_DEFINE_CHECKED(struct erl_node_*,internal_port_node,Eterm,is_internal_port); ET_DEFINE_CHECKED(Eterm*,internal_ref_val,Wterm,is_internal_ref); ET_DEFINE_CHECKED(Uint,internal_ref_data_words,Wterm,is_internal_ref); diff --git a/erts/emulator/beam/erl_term.h b/erts/emulator/beam/erl_term.h index c270d13365..fb3ef9cd6c 100644 --- a/erts/emulator/beam/erl_term.h +++ b/erts/emulator/beam/erl_term.h @@ -300,8 +300,17 @@ _ET_DECLARE_CHECKED(Uint,header_arity,Eterm) #define header_arity(x) _ET_APPLY(header_arity,(x)) /* arityval access methods */ +/* Erlang Spec. 4.7.3 defines max arity to 65535 + * we will however enforce max arity of 16777215 (24 bits) + * (checked in bifs and asserted in debug) + */ +#define MAX_ARITYVAL ((((Uint)1) << 24) - 1) +#define ERTS_MAX_TUPLE_SIZE MAX_ARITYVAL + #define make_arityval(sz) _make_header((sz),_TAG_HEADER_ARITYVAL) #define is_arity_value(x) (((x) & _TAG_HEADER_MASK) == _TAG_HEADER_ARITYVAL) +#define is_sane_arity_value(x) ((((x) & _TAG_HEADER_MASK) == _TAG_HEADER_ARITYVAL) && \ + (((x) >> _HEADER_ARITY_OFFS) <= MAX_ARITYVAL)) #define is_not_arity_value(x) (!is_arity_value((x))) #define _unchecked_arityval(x) _unchecked_header_arity((x)) _ET_DECLARE_CHECKED(Uint,arityval,Eterm) @@ -542,12 +551,6 @@ _ET_DECLARE_CHECKED(Eterm*,tuple_val,Wterm) #define _GETBITS(X,Pos,Size) (((X) >> (Pos)) & ~(~((Uint) 0) << (Size))) /* - * Observe! New layout for pids, ports and references in R9 (see also note - * in erl_node_container_utils.h). - */ - - -/* * Creation in node specific data (pids, ports, refs) */ @@ -584,7 +587,6 @@ _ET_DECLARE_CHECKED(Eterm*,tuple_val,Wterm) * */ -#define _PID_R9_SER_SIZE 3 #define _PID_SER_SIZE (_PID_DATA_SIZE - _PID_NUM_SIZE) #define _PID_NUM_SIZE 15 @@ -598,23 +600,13 @@ _ET_DECLARE_CHECKED(Eterm*,tuple_val,Wterm) #define make_pid_data(Ser, Num) \ ((Uint) ((Ser) << _PID_NUM_SIZE | (Num))) -#define make_internal_pid(X) \ - ((Eterm) (((X) << _PID_DATA_SHIFT) | _TAG_IMMED1_PID)) - #define is_internal_pid(x) (((x) & _TAG_IMMED1_MASK) == _TAG_IMMED1_PID) #define is_not_internal_pid(x) (!is_internal_pid((x))) -#define _unchecked_internal_pid_data(x) _GET_PID_DATA((x)) -_ET_DECLARE_CHECKED(Uint,internal_pid_data,Eterm) -#define internal_pid_data(x) _ET_APPLY(internal_pid_data,(x)) - #define _unchecked_internal_pid_node(x) erts_this_node _ET_DECLARE_CHECKED(struct erl_node_*,internal_pid_node,Eterm) #define internal_pid_node(x) _ET_APPLY(internal_pid_node,(x)) -#define internal_pid_number(x) _GET_PID_NUM(internal_pid_data((x))) -#define internal_pid_serial(x) _GET_PID_SER(internal_pid_data((x))) - #define internal_pid_data_words(x) (1) /* @@ -644,7 +636,6 @@ _ET_DECLARE_CHECKED(struct erl_node_*,internal_pid_node,Eterm) * N : node number * */ -#define _PORT_R9_NUM_SIZE 18 #define _PORT_NUM_SIZE _PORT_DATA_SIZE #define _PORT_DATA_SIZE 28 @@ -654,18 +645,9 @@ _ET_DECLARE_CHECKED(struct erl_node_*,internal_pid_node,Eterm) #define _GET_PORT_NUM(X) _GETBITS((X), 0, _PORT_NUM_SIZE) -#define make_internal_port(X) \ - ((Eterm) (((X) << _PORT_DATA_SHIFT) | _TAG_IMMED1_PORT)) - #define is_internal_port(x) (((x) & _TAG_IMMED1_MASK) == _TAG_IMMED1_PORT) #define is_not_internal_port(x) (!is_internal_port(x)) -#define _unchecked_internal_port_data(x) _GET_PORT_DATA((x)) -_ET_DECLARE_CHECKED(Uint,internal_port_data,Eterm) -#define internal_port_data(x) _ET_APPLY(internal_port_data,(x)) - -#define internal_port_number(x) _GET_PORT_NUM(internal_port_data((x))) - #define _unchecked_internal_port_node(x) erts_this_node _ET_DECLARE_CHECKED(struct erl_node_*,internal_port_node,Eterm) #define internal_port_node(x) _ET_APPLY(internal_port_node,(x)) diff --git a/erts/emulator/beam/erl_thr_progress.c b/erts/emulator/beam/erl_thr_progress.c index 88524bdd4c..9678d7e08b 100644 --- a/erts/emulator/beam/erl_thr_progress.c +++ b/erts/emulator/beam/erl_thr_progress.c @@ -96,17 +96,14 @@ #define ERTS_THR_PRGR_LFLG_BLOCK (((erts_aint32_t) 1) << 31) #define ERTS_THR_PRGR_LFLG_NO_LEADER (((erts_aint32_t) 1) << 30) -#define ERTS_THR_PRGR_LFLG_ACTIVE_MASK (~(ERTS_THR_PRGR_LFLG_NO_LEADER \ - | ERTS_THR_PRGR_LFLG_BLOCK)) +#define ERTS_THR_PRGR_LFLG_WAITING_UM (((erts_aint32_t) 1) << 29) +#define ERTS_THR_PRGR_LFLG_ACTIVE_MASK (~(ERTS_THR_PRGR_LFLG_NO_LEADER \ + | ERTS_THR_PRGR_LFLG_BLOCK \ + | ERTS_THR_PRGR_LFLG_WAITING_UM)) -#define ERTS_THR_PRGR_LFLGS_ACTIVE(LFLGS) \ +#define ERTS_THR_PRGR_LFLGS_ACTIVE(LFLGS) \ ((LFLGS) & ERTS_THR_PRGR_LFLG_ACTIVE_MASK) -#define ERTS_THR_PRGR_LFLGS_ALL_WAITING(LFLGS) \ - (((LFLGS) & (ERTS_THR_PRGR_LFLG_NO_LEADER \ - |ERTS_THR_PRGR_LFLG_ACTIVE_MASK)) \ - == ERTS_THR_PRGR_LFLG_NO_LEADER) - /* * We use a 64-bit value for thread progress. By this wrapping of * the thread progress will more or less never occur. @@ -262,6 +259,11 @@ typedef struct { erts_atomic32_t managed_count; erts_atomic32_t managed_id; erts_atomic32_t unmanaged_id; + int chk_next_ix; + struct { + int waiting; + erts_atomic32_t current; + } umrefc_ix; } ErtsThrPrgrMiscData; typedef struct { @@ -276,12 +278,18 @@ typedef union { char align__[ERTS_ALC_CACHE_LINE_ALIGN_SIZE(sizeof(ErtsThrPrgrElement))]; } ErtsThrPrgrArray; +typedef union { + erts_atomic_t refc; + char align__[ERTS_ALC_CACHE_LINE_ALIGN_SIZE(sizeof(erts_atomic_t))]; +} ErtsThrPrgrUnmanagedRefc; + typedef struct { union { ErtsThrPrgrMiscData data; char align__[ERTS_ALC_CACHE_LINE_ALIGN_SIZE( sizeof(ErtsThrPrgrMiscData))]; } misc; + ErtsThrPrgrUnmanagedRefc umrefc[2]; ErtsThrPrgrArray *thr; struct { int no; @@ -346,7 +354,9 @@ init_tmp_thr_prgr_data(ErtsThrPrgrData *tpd) tpd->is_managed = 0; tpd->is_blocking = 0; tpd->is_temporary = 1; - +#ifdef ERTS_ENABLE_LOCK_CHECK + tpd->is_delaying = 0; +#endif erts_tsd_set(erts_thr_prgr_data_key__, (void *) tpd); } @@ -461,6 +471,12 @@ erts_thr_progress_init(int no_schedulers, int managed, int unmanaged) erts_atomic32_init_nob(&intrnl->misc.data.managed_count, 0); erts_atomic32_init_nob(&intrnl->misc.data.managed_id, no_schedulers); erts_atomic32_init_nob(&intrnl->misc.data.unmanaged_id, -1); + intrnl->misc.data.chk_next_ix = 0; + intrnl->misc.data.umrefc_ix.waiting = -1; + erts_atomic32_init_nob(&intrnl->misc.data.umrefc_ix.current, 0); + + erts_atomic_init_nob(&intrnl->umrefc[0].refc, (erts_aint_t) 0); + erts_atomic_init_nob(&intrnl->umrefc[1].refc, (erts_aint_t) 0); intrnl->thr = (ErtsThrPrgrArray *) ptr; ptr += thr_arr_sz; @@ -547,6 +563,9 @@ erts_thr_progress_register_unmanaged_thread(ErtsThrPrgrCallbacks *callbacks) tpd->is_managed = 0; tpd->is_blocking = is_blocking; tpd->is_temporary = 0; +#ifdef ERTS_ENABLE_LOCK_CHECK + tpd->is_delaying = 0; +#endif ASSERT(tpd->id >= 0); if (tpd->id >= intrnl->unmanaged.no) erl_exit(ERTS_ABORT_EXIT, @@ -600,6 +619,9 @@ erts_thr_progress_register_managed_thread(ErtsSchedulerData *esdp, tpd->is_managed = 1; tpd->is_blocking = is_blocking; tpd->is_temporary = 0; +#ifdef ERTS_ENABLE_LOCK_CHECK + tpd->is_delaying = 1; +#endif init_wakeup_request_array(&tpd->wakeup_request[0]); @@ -607,8 +629,8 @@ erts_thr_progress_register_managed_thread(ErtsSchedulerData *esdp, tpd->leader = 0; tpd->active = 1; - tpd->previous.local = 0; - tpd->previous.current = ERTS_THR_PRGR_VAL_WAITING; + tpd->confirmed = 0; + tpd->leader_state.current = ERTS_THR_PRGR_VAL_WAITING; erts_tsd_set(erts_thr_prgr_data_key__, (void *) tpd); erts_atomic32_inc_nob(&intrnl->misc.data.lflgs); @@ -651,60 +673,113 @@ leader_update(ErtsThrPrgrData *tpd) block_thread(tpd); } else { + ErtsThrPrgrVal current; + int ix, chk_next_ix, umrefc_ix, my_ix, no_managed, waiting_unmanaged; erts_aint32_t lflgs; ErtsThrPrgrVal next; - int ix, sz, make_progress; + erts_aint_t refc; - if (tpd->previous.current == ERTS_THR_PRGR_VAL_WAITING) { - /* Took over as leader from another thread */ - tpd->previous.current = read_acqb(&erts_thr_prgr__.current); - tpd->previous.next = tpd->previous.current; - tpd->previous.next++; - if (tpd->previous.next == ERTS_THR_PRGR_VAL_WAITING) - tpd->previous.next = 0; - } + my_ix = tpd->id; - if (tpd->previous.local == tpd->previous.current) { - ErtsThrPrgrVal val = tpd->previous.current + 1; - if (val == ERTS_THR_PRGR_VAL_WAITING) - val = 0; - tpd->previous.local = val; - set_mb(&intrnl->thr[tpd->id].data.current, val); + if (tpd->leader_state.current == ERTS_THR_PRGR_VAL_WAITING) { + /* Took over as leader from another thread */ + tpd->leader_state.current = read_nob(&erts_thr_prgr__.current); + tpd->leader_state.next = tpd->leader_state.current; + tpd->leader_state.next++; + if (tpd->leader_state.next == ERTS_THR_PRGR_VAL_WAITING) + tpd->leader_state.next = 0; + tpd->leader_state.chk_next_ix = intrnl->misc.data.chk_next_ix; + tpd->leader_state.umrefc_ix.waiting = intrnl->misc.data.umrefc_ix.waiting; + tpd->leader_state.umrefc_ix.current = + (int) erts_atomic32_read_nob(&intrnl->misc.data.umrefc_ix.current); + + if (tpd->confirmed == tpd->leader_state.current) { + ErtsThrPrgrVal val = tpd->leader_state.current + 1; + if (val == ERTS_THR_PRGR_VAL_WAITING) + val = 0; + tpd->confirmed = val; + set_mb(&intrnl->thr[my_ix].data.current, val); + } } - next = tpd->previous.next; - make_progress = 1; - sz = intrnl->managed.no; - for (ix = 0; ix < sz; ix++) { - ErtsThrPrgrVal tmp; - tmp = read_nob(&intrnl->thr[ix].data.current); - if (tmp != next && tmp != ERTS_THR_PRGR_VAL_WAITING) { - make_progress = 0; - ASSERT(erts_thr_progress_has_passed__(next, tmp)); - break; + next = tpd->leader_state.next; + + waiting_unmanaged = 0; + umrefc_ix = -1; /* Shut up annoying warning */ + + chk_next_ix = tpd->leader_state.chk_next_ix; + no_managed = intrnl->managed.no; + ASSERT(0 <= chk_next_ix && chk_next_ix <= no_managed); + /* Check manged threads */ + if (chk_next_ix < no_managed) { + for (ix = chk_next_ix; ix < no_managed; ix++) { + ErtsThrPrgrVal tmp; + if (ix == my_ix) + continue; + tmp = read_nob(&intrnl->thr[ix].data.current); + if (tmp != next && tmp != ERTS_THR_PRGR_VAL_WAITING) { + tpd->leader_state.chk_next_ix = ix; + ASSERT(erts_thr_progress_has_passed__(next, tmp)); + goto done; + } } } - if (make_progress) { - ErtsThrPrgrVal current = next; + /* Check unmanged threads */ + waiting_unmanaged = tpd->leader_state.umrefc_ix.waiting != -1; + umrefc_ix = (waiting_unmanaged + ? tpd->leader_state.umrefc_ix.waiting + : tpd->leader_state.umrefc_ix.current); + refc = erts_atomic_read_nob(&intrnl->umrefc[umrefc_ix].refc); + ASSERT(refc >= 0); + if (refc != 0) { + int new_umrefc_ix; + + if (waiting_unmanaged) + goto done; + + new_umrefc_ix = (umrefc_ix + 1) & 0x1; + tpd->leader_state.umrefc_ix.waiting = umrefc_ix; + tpd->leader_state.chk_next_ix = no_managed; + erts_atomic32_set_nob(&intrnl->misc.data.umrefc_ix.current, + (erts_aint32_t) new_umrefc_ix); + ETHR_MEMBAR(ETHR_StoreLoad); + refc = erts_atomic_read_nob(&intrnl->umrefc[umrefc_ix].refc); + ASSERT(refc >= 0); + waiting_unmanaged = 1; + if (refc != 0) + goto done; + } - next++; - if (next == ERTS_THR_PRGR_VAL_WAITING) - next = 0; + /* Make progress */ + current = next; - set_nob(&intrnl->thr[tpd->id].data.current, next); - set_mb(&erts_thr_prgr__.current, current); - tpd->previous.local = next; - tpd->previous.next = next; - tpd->previous.current = current; + next++; + if (next == ERTS_THR_PRGR_VAL_WAITING) + next = 0; + + set_nob(&intrnl->thr[my_ix].data.current, next); + set_mb(&erts_thr_prgr__.current, current); + tpd->confirmed = next; + tpd->leader_state.next = next; + tpd->leader_state.current = current; #if ERTS_THR_PRGR_PRINT_VAL - if (current % 1000 == 0) - erts_fprintf(stderr, "%b64u\n", current); + if (current % 1000 == 0) + erts_fprintf(stderr, "%b64u\n", current); #endif - handle_wakeup_requests(current); + handle_wakeup_requests(current); + + if (waiting_unmanaged) { + waiting_unmanaged = 0; + tpd->leader_state.umrefc_ix.waiting = -1; + erts_atomic32_read_band_nob(&intrnl->misc.data.lflgs, + ~ERTS_THR_PRGR_LFLG_WAITING_UM); } + tpd->leader_state.chk_next_ix = 0; + + done: if (tpd->active) { lflgs = erts_atomic32_read_nob(&intrnl->misc.data.lflgs); @@ -712,20 +787,44 @@ leader_update(ErtsThrPrgrData *tpd) (void) block_thread(tpd); } else { + int force_wakeup_check = 0; + erts_aint32_t set_flags = ERTS_THR_PRGR_LFLG_NO_LEADER; tpd->leader = 0; - tpd->previous.current = ERTS_THR_PRGR_VAL_WAITING; + tpd->leader_state.current = ERTS_THR_PRGR_VAL_WAITING; #if ERTS_THR_PRGR_PRINT_LEADER erts_fprintf(stderr, "L <- %d\n", tpd->id); #endif ERTS_THR_PROGRESS_STATE_DEBUG_SET_LEADER(tpd->id, 0); + if (waiting_unmanaged) + set_flags |= ERTS_THR_PRGR_LFLG_WAITING_UM; + lflgs = erts_atomic32_read_bor_relb(&intrnl->misc.data.lflgs, - ERTS_THR_PRGR_LFLG_NO_LEADER); + set_flags); + lflgs |= set_flags; if (lflgs & ERTS_THR_PRGR_LFLG_BLOCK) lflgs = block_thread(tpd); - if (ERTS_THR_PRGR_LFLGS_ACTIVE(lflgs) == 0 && got_sched_wakeups()) + + if (waiting_unmanaged) { + /* Need to check umrefc again */ + ETHR_MEMBAR(ETHR_StoreLoad); + refc = erts_atomic_read_nob(&intrnl->umrefc[umrefc_ix].refc); + if (refc == 0) { + /* Need to force wakeup check */ + force_wakeup_check = 1; + } + } + + if ((force_wakeup_check + || ((lflgs & (ERTS_THR_PRGR_LFLG_NO_LEADER + | ERTS_THR_PRGR_LFLG_WAITING_UM + | ERTS_THR_PRGR_LFLG_ACTIVE_MASK)) + == ERTS_THR_PRGR_LFLG_NO_LEADER)) + && got_sched_wakeups()) { + /* Someone need to make progress */ wakeup_managed(0); + } } } @@ -744,11 +843,11 @@ update(ErtsThrPrgrData *tpd) erts_aint32_t lflgs; res = 0; val = read_acqb(&erts_thr_prgr__.current); - if (tpd->previous.local == val) { + if (tpd->confirmed == val) { val++; if (val == ERTS_THR_PRGR_VAL_WAITING) val = 0; - tpd->previous.local = val; + tpd->confirmed = val; set_mb(&intrnl->thr[tpd->id].data.current, val); } @@ -801,12 +900,19 @@ erts_thr_progress_prepare_wait(ErtsSchedulerData *esdp) block_count_dec(); - tpd->previous.local = ERTS_THR_PRGR_VAL_WAITING; + tpd->confirmed = ERTS_THR_PRGR_VAL_WAITING; set_mb(&intrnl->thr[tpd->id].data.current, ERTS_THR_PRGR_VAL_WAITING); lflgs = erts_atomic32_read_nob(&intrnl->misc.data.lflgs); - if (ERTS_THR_PRGR_LFLGS_ALL_WAITING(lflgs) && got_sched_wakeups()) - wakeup_managed(0); /* Someone need to make progress */ + + if ((lflgs & (ERTS_THR_PRGR_LFLG_NO_LEADER + | ERTS_THR_PRGR_LFLG_WAITING_UM + | ERTS_THR_PRGR_LFLG_ACTIVE_MASK)) + == ERTS_THR_PRGR_LFLG_NO_LEADER + && got_sched_wakeups()) { + /* Someone need to make progress */ + wakeup_managed(0); + } } void @@ -828,7 +934,7 @@ erts_thr_progress_finalize_wait(ErtsSchedulerData *esdp) val++; if (val == ERTS_THR_PRGR_VAL_WAITING) val = 0; - tpd->previous.local = val; + tpd->confirmed = val; set_mb(&intrnl->thr[tpd->id].data.current, val); val = read_acqb(&erts_thr_prgr__.current); if (current == val) @@ -875,6 +981,68 @@ erts_thr_progress_active(ErtsSchedulerData *esdp, int on) } +static ERTS_INLINE void +unmanaged_continue(ErtsThrPrgrDelayHandle handle) +{ + int umrefc_ix = (int) handle; + erts_aint_t refc; + + ASSERT(umrefc_ix == 0 || umrefc_ix == 1); + refc = erts_atomic_dec_read_relb(&intrnl->umrefc[umrefc_ix].refc); + ASSERT(refc >= 0); + if (refc == 0) { + erts_aint_t lflgs; + ERTS_THR_READ_MEMORY_BARRIER; + lflgs = erts_atomic32_read_nob(&intrnl->misc.data.lflgs); + if ((lflgs & (ERTS_THR_PRGR_LFLG_NO_LEADER + | ERTS_THR_PRGR_LFLG_WAITING_UM + | ERTS_THR_PRGR_LFLG_ACTIVE_MASK)) + == (ERTS_THR_PRGR_LFLG_NO_LEADER|ERTS_THR_PRGR_LFLG_WAITING_UM) + && got_sched_wakeups()) { + /* Others waiting for us... */ + wakeup_managed(0); + } + } +} + +void +erts_thr_progress_unmanaged_continue__(ErtsThrPrgrDelayHandle handle) +{ +#ifdef ERTS_ENABLE_LOCK_CHECK + ErtsThrPrgrData *tpd = perhaps_thr_prgr_data(NULL); + ERTS_LC_ASSERT(tpd && tpd->is_delaying); + tpd->is_delaying = 0; + return_tmp_thr_prgr_data(tpd); +#endif + ASSERT(!erts_thr_progress_is_managed_thread()); + + unmanaged_continue(handle); +} + +ErtsThrPrgrDelayHandle +erts_thr_progress_unmanaged_delay__(void) +{ + int umrefc_ix; + ASSERT(!erts_thr_progress_is_managed_thread()); + umrefc_ix = (int) erts_atomic32_read_acqb(&intrnl->misc.data.umrefc_ix.current); + while (1) { + int tmp_ix; + erts_atomic_inc_acqb(&intrnl->umrefc[umrefc_ix].refc); + tmp_ix = (int) erts_atomic32_read_acqb(&intrnl->misc.data.umrefc_ix.current); + if (tmp_ix == umrefc_ix) + break; + unmanaged_continue(umrefc_ix); + umrefc_ix = tmp_ix; + } +#ifdef ERTS_ENABLE_LOCK_CHECK + { + ErtsThrPrgrData *tpd = tmp_thr_prgr_data(NULL); + tpd->is_delaying = 1; + } +#endif + return (ErtsThrPrgrDelayHandle) umrefc_ix; +} + static ERTS_INLINE int has_reached_wakeup(ErtsThrPrgrVal wakeup) { @@ -931,7 +1099,7 @@ request_wakeup_managed(ErtsThrPrgrData *tpd, ErtsThrPrgrVal value) */ ASSERT(tpd->is_managed); - ASSERT(tpd->previous.local != ERTS_THR_PRGR_VAL_WAITING); + ASSERT(tpd->confirmed != ERTS_THR_PRGR_VAL_WAITING); if (has_reached_wakeup(value)) { wakeup_managed(tpd->id); @@ -946,7 +1114,7 @@ request_wakeup_managed(ErtsThrPrgrData *tpd, ErtsThrPrgrVal value) tpd->wakeup_request[wix])); - if (tpd->previous.local == value) { + if (tpd->confirmed == value) { /* * We have already confirmed this value. We need to request * wakeup for a value later than our latest confirmed value in diff --git a/erts/emulator/beam/erl_thr_progress.h b/erts/emulator/beam/erl_thr_progress.h index e72321cf48..1416aa6166 100644 --- a/erts/emulator/beam/erl_thr_progress.h +++ b/erts/emulator/beam/erl_thr_progress.h @@ -53,9 +53,22 @@ typedef Uint64 ErtsThrPrgrVal; #define ERTS_THR_PRGR_WAKEUP_DATA_SIZE 4 /* Need to be an even power of 2. */ typedef struct { + ErtsThrPrgrVal next; + ErtsThrPrgrVal current; + int chk_next_ix; + struct { + int current; + int waiting; + } umrefc_ix; +} ErtsThrPrgrLeaderState; + +typedef struct { int id; int is_managed; int is_blocking; +#ifdef ERTS_ENABLE_LOCK_CHECK + int is_delaying; /* managed is always delaying */ +#endif int is_temporary; /* --- Part below only for registered threads --- */ @@ -66,11 +79,8 @@ typedef struct { int leader; /* Needs to be first in the managed threads part */ int active; - struct { - ErtsThrPrgrVal local; - ErtsThrPrgrVal next; - ErtsThrPrgrVal current; - } previous; + ErtsThrPrgrVal confirmed; + ErtsThrPrgrLeaderState leader_state; } ErtsThrPrgrData; void erts_thr_progress_fatal_error_block(SWord timeout, @@ -121,6 +131,11 @@ typedef struct { ERTS_THR_PRGR_ATOMIC current; } ErtsThrPrgr; +typedef int ErtsThrPrgrDelayHandle; +#define ERTS_THR_PRGR_DHANDLE_MANAGED ((ErtsThrPrgrDelayHandle) -1) +/* ERTS_THR_PRGR_DHANDLE_MANAGED implies managed thread */ +#define ERTS_THR_PRGR_DHANDLE_INVALID ((ErtsThrPrgrDelayHandle) -2) + extern ErtsThrPrgr erts_thr_prgr__; void erts_thr_progress_pre_init(void); @@ -136,6 +151,8 @@ int erts_thr_progress_update(ErtsSchedulerData *esdp); int erts_thr_progress_leader_update(ErtsSchedulerData *esdp); void erts_thr_progress_prepare_wait(ErtsSchedulerData *esdp); void erts_thr_progress_finalize_wait(ErtsSchedulerData *esdp); +ErtsThrPrgrDelayHandle erts_thr_progress_unmanaged_delay__(void); +void erts_thr_progress_unmanaged_continue__(int umrefc_ix); void erts_thr_progress_dbg_print_state(void); @@ -148,6 +165,11 @@ ERTS_GLB_INLINE ErtsThrPrgrVal erts_thr_prgr_read_acqb__(ERTS_THR_PRGR_ATOMIC *a ERTS_GLB_INLINE ErtsThrPrgrVal erts_thr_prgr_read_mb__(ERTS_THR_PRGR_ATOMIC *atmc); ERTS_GLB_INLINE int erts_thr_progress_is_managed_thread(void); +ERTS_GLB_INLINE ErtsThrPrgrDelayHandle erts_thr_progress_unmanaged_delay(void); +ERTS_GLB_INLINE void erts_thr_progress_unmanaged_continue(ErtsThrPrgrDelayHandle handle); +#ifdef ERTS_ENABLE_LOCK_CHECK +ERTS_GLB_INLINE int erts_thr_progress_lc_is_delaying(void); +#endif ERTS_GLB_INLINE ErtsThrPrgrVal erts_thr_progress_current_to_later__(ErtsThrPrgrVal val); ERTS_GLB_INLINE ErtsThrPrgrVal erts_thr_progress_later(ErtsSchedulerData *); ERTS_GLB_INLINE ErtsThrPrgrVal erts_thr_progress_current(void); @@ -229,6 +251,35 @@ erts_thr_progress_is_managed_thread(void) return tpd && tpd->is_managed; } +ERTS_GLB_INLINE ErtsThrPrgrDelayHandle +erts_thr_progress_unmanaged_delay(void) +{ + if (erts_thr_progress_is_managed_thread()) + return ERTS_THR_PRGR_DHANDLE_MANAGED; /* Nothing to do */ + else + return erts_thr_progress_unmanaged_delay__(); +} + +ERTS_GLB_INLINE void +erts_thr_progress_unmanaged_continue(ErtsThrPrgrDelayHandle handle) +{ + ASSERT(handle != ERTS_THR_PRGR_DHANDLE_MANAGED + || erts_thr_progress_is_managed_thread()); + if (handle != ERTS_THR_PRGR_DHANDLE_MANAGED) + erts_thr_progress_unmanaged_continue__(handle); +} + +#ifdef ERTS_ENABLE_LOCK_CHECK + +ERTS_GLB_INLINE int +erts_thr_progress_lc_is_delaying(void) +{ + ErtsThrPrgrData *tpd = erts_tsd_get(erts_thr_prgr_data_key__); + return tpd && tpd->is_delaying; +} + +#endif + ERTS_GLB_INLINE ErtsThrPrgrVal erts_thr_progress_current_to_later__(ErtsThrPrgrVal val) { @@ -248,7 +299,7 @@ erts_thr_progress_later(ErtsSchedulerData *esdp) if (esdp) { tpd = &esdp->thr_progress_data; managed_thread: - val = tpd->previous.local; + val = tpd->confirmed; ERTS_THR_MEMORY_BARRIER; } else { diff --git a/erts/emulator/beam/erl_threads.h b/erts/emulator/beam/erl_threads.h index 17628286bc..1dc3ffeb3c 100644 --- a/erts/emulator/beam/erl_threads.h +++ b/erts/emulator/beam/erl_threads.h @@ -533,6 +533,9 @@ ERTS_GLB_INLINE erts_aint_t erts_no_atomic_xchg(erts_no_atomic_t *xchgp, ERTS_GLB_INLINE erts_aint_t erts_no_atomic_cmpxchg(erts_no_atomic_t *xchgp, erts_aint_t new, erts_aint_t expected); +ERTS_GLB_INLINE erts_aint_t erts_no_atomic_read_bset(erts_no_atomic_t *var, + erts_aint_t mask, + erts_aint_t set); ERTS_GLB_INLINE void erts_no_atomic32_set(erts_no_atomic32_t *var, erts_aint32_t i); ERTS_GLB_INLINE erts_aint32_t erts_no_atomic32_read(erts_no_atomic32_t *var); @@ -553,6 +556,9 @@ ERTS_GLB_INLINE erts_aint32_t erts_no_atomic32_xchg(erts_no_atomic32_t *xchgp, ERTS_GLB_INLINE erts_aint32_t erts_no_atomic32_cmpxchg(erts_no_atomic32_t *xchgp, erts_aint32_t new, erts_aint32_t expected); +ERTS_GLB_INLINE erts_aint32_t erts_no_atomic32_read_bset(erts_no_atomic32_t *var, + erts_aint32_t mask, + erts_aint32_t set); ERTS_GLB_INLINE void erts_spinlock_init_x_opt(erts_spinlock_t *lock, char *name, @@ -612,6 +618,78 @@ ERTS_GLB_INLINE void erts_thr_sigwait(const sigset_t *set, int *sig); #ifdef USE_THREADS +ERTS_GLB_INLINE erts_aint_t +erts_atomic_read_bset_nob(erts_atomic_t *var, + erts_aint_t mask, + erts_aint_t set); +ERTS_GLB_INLINE erts_aint_t +erts_atomic_read_bset_ddrb(erts_atomic_t *var, + erts_aint_t mask, + erts_aint_t set); +ERTS_GLB_INLINE erts_aint_t +erts_atomic_read_bset_rb(erts_atomic_t *var, + erts_aint_t mask, + erts_aint_t set); +ERTS_GLB_INLINE erts_aint_t +erts_atomic_read_bset_wb(erts_atomic_t *var, + erts_aint_t mask, + erts_aint_t set); +ERTS_GLB_INLINE erts_aint_t +erts_atomic_read_bset_acqb(erts_atomic_t *var, + erts_aint_t mask, + erts_aint_t set); +ERTS_GLB_INLINE erts_aint_t +erts_atomic_read_bset_relb(erts_atomic_t *var, + erts_aint_t mask, + erts_aint_t set); +ERTS_GLB_INLINE erts_aint_t +erts_atomic_read_bset_mb(erts_atomic_t *var, + erts_aint_t mask, + erts_aint_t set); +ERTS_GLB_INLINE erts_aint32_t +erts_atomic32_read_bset_nob(erts_atomic32_t *var, + erts_aint32_t mask, + erts_aint32_t set); +ERTS_GLB_INLINE erts_aint32_t +erts_atomic32_read_bset_ddrb(erts_atomic32_t *var, + erts_aint32_t mask, + erts_aint32_t set); +ERTS_GLB_INLINE erts_aint32_t +erts_atomic32_read_bset_rb(erts_atomic32_t *var, + erts_aint32_t mask, + erts_aint32_t set); +ERTS_GLB_INLINE erts_aint32_t +erts_atomic32_read_bset_wb(erts_atomic32_t *var, + erts_aint32_t mask, + erts_aint32_t set); +ERTS_GLB_INLINE erts_aint32_t +erts_atomic32_read_bset_acqb(erts_atomic32_t *var, + erts_aint32_t mask, + erts_aint32_t set); +ERTS_GLB_INLINE erts_aint32_t +erts_atomic32_read_bset_relb(erts_atomic32_t *var, + erts_aint32_t mask, + erts_aint32_t set); +ERTS_GLB_INLINE erts_aint32_t +erts_atomic32_read_bset_mb(erts_atomic32_t *var, + erts_aint32_t mask, + erts_aint32_t set); + +#if ERTS_GLB_INLINE_INCL_FUNC_DEF +#define ERTS_ATOMIC_BSET_IMPL__(Type, ReadOp, CmpxchgOp, VarP, Mask, Set) \ +do { \ + Type act = ReadOp((VarP)); \ + while (1) { \ + Type exp = act; \ + Type new = exp & ~(Mask); \ + new |= ((Mask) & (Set)); \ + act = CmpxchgOp((VarP), new, exp); \ + if (act == exp) \ + return act; \ + } \ +} while (0) +#endif + /* * See "Documentation of atomics and memory barriers" at the top * of this file for info on atomics. @@ -670,6 +748,19 @@ ERTS_GLB_INLINE void erts_thr_sigwait(const sigset_t *set, int *sig); #define erts_atomic_xchg_nob ethr_atomic_xchg #define erts_atomic_cmpxchg_nob ethr_atomic_cmpxchg +#if ERTS_GLB_INLINE_INCL_FUNC_DEF +ERTS_GLB_INLINE erts_aint_t +erts_atomic_read_bset_nob(erts_atomic_t *var, + erts_aint_t mask, + erts_aint_t set) +{ + ERTS_ATOMIC_BSET_IMPL__(erts_aint_t, + ethr_atomic_read, + ethr_atomic_cmpxchg, + var, mask, set); +} +#endif + #define erts_atomic_init_mb ethr_atomic_init_mb #define erts_atomic_set_mb ethr_atomic_set_mb #define erts_atomic_read_mb ethr_atomic_read_mb @@ -684,6 +775,19 @@ ERTS_GLB_INLINE void erts_thr_sigwait(const sigset_t *set, int *sig); #define erts_atomic_xchg_mb ethr_atomic_xchg_mb #define erts_atomic_cmpxchg_mb ethr_atomic_cmpxchg_mb +#if ERTS_GLB_INLINE_INCL_FUNC_DEF +ERTS_GLB_INLINE erts_aint_t +erts_atomic_read_bset_mb(erts_atomic_t *var, + erts_aint_t mask, + erts_aint_t set) +{ + ERTS_ATOMIC_BSET_IMPL__(erts_aint_t, + ethr_atomic_read, + ethr_atomic_cmpxchg_mb, + var, mask, set); +} +#endif + #define erts_atomic_init_acqb ethr_atomic_init_acqb #define erts_atomic_set_acqb ethr_atomic_set_acqb #define erts_atomic_read_acqb ethr_atomic_read_acqb @@ -698,6 +802,19 @@ ERTS_GLB_INLINE void erts_thr_sigwait(const sigset_t *set, int *sig); #define erts_atomic_xchg_acqb ethr_atomic_xchg_acqb #define erts_atomic_cmpxchg_acqb ethr_atomic_cmpxchg_acqb +#if ERTS_GLB_INLINE_INCL_FUNC_DEF +ERTS_GLB_INLINE erts_aint_t +erts_atomic_read_bset_acqb(erts_atomic_t *var, + erts_aint_t mask, + erts_aint_t set) +{ + ERTS_ATOMIC_BSET_IMPL__(erts_aint_t, + ethr_atomic_read, + ethr_atomic_cmpxchg_acqb, + var, mask, set); +} +#endif + #define erts_atomic_init_relb ethr_atomic_init_relb #define erts_atomic_set_relb ethr_atomic_set_relb #define erts_atomic_read_relb ethr_atomic_read_relb @@ -712,6 +829,19 @@ ERTS_GLB_INLINE void erts_thr_sigwait(const sigset_t *set, int *sig); #define erts_atomic_xchg_relb ethr_atomic_xchg_relb #define erts_atomic_cmpxchg_relb ethr_atomic_cmpxchg_relb +#if ERTS_GLB_INLINE_INCL_FUNC_DEF +ERTS_GLB_INLINE erts_aint_t +erts_atomic_read_bset_relb(erts_atomic_t *var, + erts_aint_t mask, + erts_aint_t set) +{ + ERTS_ATOMIC_BSET_IMPL__(erts_aint_t, + ethr_atomic_read, + ethr_atomic_cmpxchg_relb, + var, mask, set); +} +#endif + #define erts_atomic_init_ddrb ethr_atomic_init_ddrb #define erts_atomic_set_ddrb ethr_atomic_set_ddrb #define erts_atomic_read_ddrb ethr_atomic_read_ddrb @@ -726,6 +856,19 @@ ERTS_GLB_INLINE void erts_thr_sigwait(const sigset_t *set, int *sig); #define erts_atomic_xchg_ddrb ethr_atomic_xchg_ddrb #define erts_atomic_cmpxchg_ddrb ethr_atomic_cmpxchg_ddrb +#if ERTS_GLB_INLINE_INCL_FUNC_DEF +ERTS_GLB_INLINE erts_aint_t +erts_atomic_read_bset_ddrb(erts_atomic_t *var, + erts_aint_t mask, + erts_aint_t set) +{ + ERTS_ATOMIC_BSET_IMPL__(erts_aint_t, + ethr_atomic_read, + ethr_atomic_cmpxchg_ddrb, + var, mask, set); +} +#endif + #define erts_atomic_init_rb ethr_atomic_init_rb #define erts_atomic_set_rb ethr_atomic_set_rb #define erts_atomic_read_rb ethr_atomic_read_rb @@ -740,6 +883,19 @@ ERTS_GLB_INLINE void erts_thr_sigwait(const sigset_t *set, int *sig); #define erts_atomic_xchg_rb ethr_atomic_xchg_rb #define erts_atomic_cmpxchg_rb ethr_atomic_cmpxchg_rb +#if ERTS_GLB_INLINE_INCL_FUNC_DEF +ERTS_GLB_INLINE erts_aint_t +erts_atomic_read_bset_rb(erts_atomic_t *var, + erts_aint_t mask, + erts_aint_t set) +{ + ERTS_ATOMIC_BSET_IMPL__(erts_aint_t, + ethr_atomic_read, + ethr_atomic_cmpxchg_rb, + var, mask, set); +} +#endif + #define erts_atomic_init_wb ethr_atomic_init_wb #define erts_atomic_set_wb ethr_atomic_set_wb #define erts_atomic_read_wb ethr_atomic_read_wb @@ -754,6 +910,19 @@ ERTS_GLB_INLINE void erts_thr_sigwait(const sigset_t *set, int *sig); #define erts_atomic_xchg_wb ethr_atomic_xchg_wb #define erts_atomic_cmpxchg_wb ethr_atomic_cmpxchg_wb +#if ERTS_GLB_INLINE_INCL_FUNC_DEF +ERTS_GLB_INLINE erts_aint_t +erts_atomic_read_bset_wb(erts_atomic_t *var, + erts_aint_t mask, + erts_aint_t set) +{ + ERTS_ATOMIC_BSET_IMPL__(erts_aint_t, + ethr_atomic_read, + ethr_atomic_cmpxchg_wb, + var, mask, set); +} +#endif + /* 32-bit atomics */ #define erts_atomic32_init_nob ethr_atomic32_init @@ -770,6 +939,19 @@ ERTS_GLB_INLINE void erts_thr_sigwait(const sigset_t *set, int *sig); #define erts_atomic32_xchg_nob ethr_atomic32_xchg #define erts_atomic32_cmpxchg_nob ethr_atomic32_cmpxchg +#if ERTS_GLB_INLINE_INCL_FUNC_DEF +ERTS_GLB_INLINE erts_aint32_t +erts_atomic32_read_bset_nob(erts_atomic32_t *var, + erts_aint32_t mask, + erts_aint32_t set) +{ + ERTS_ATOMIC_BSET_IMPL__(erts_aint32_t, + ethr_atomic32_read, + ethr_atomic32_cmpxchg, + var, mask, set); +} +#endif + #define erts_atomic32_init_mb ethr_atomic32_init_mb #define erts_atomic32_set_mb ethr_atomic32_set_mb #define erts_atomic32_read_mb ethr_atomic32_read_mb @@ -784,6 +966,19 @@ ERTS_GLB_INLINE void erts_thr_sigwait(const sigset_t *set, int *sig); #define erts_atomic32_xchg_mb ethr_atomic32_xchg_mb #define erts_atomic32_cmpxchg_mb ethr_atomic32_cmpxchg_mb +#if ERTS_GLB_INLINE_INCL_FUNC_DEF +ERTS_GLB_INLINE erts_aint32_t +erts_atomic32_read_bset_mb(erts_atomic32_t *var, + erts_aint32_t mask, + erts_aint32_t set) +{ + ERTS_ATOMIC_BSET_IMPL__(erts_aint32_t, + ethr_atomic32_read, + ethr_atomic32_cmpxchg_mb, + var, mask, set); +} +#endif + #define erts_atomic32_init_acqb ethr_atomic32_init_acqb #define erts_atomic32_set_acqb ethr_atomic32_set_acqb #define erts_atomic32_read_acqb ethr_atomic32_read_acqb @@ -798,6 +993,19 @@ ERTS_GLB_INLINE void erts_thr_sigwait(const sigset_t *set, int *sig); #define erts_atomic32_xchg_acqb ethr_atomic32_xchg_acqb #define erts_atomic32_cmpxchg_acqb ethr_atomic32_cmpxchg_acqb +#if ERTS_GLB_INLINE_INCL_FUNC_DEF +ERTS_GLB_INLINE erts_aint32_t +erts_atomic32_read_bset_acqb(erts_atomic32_t *var, + erts_aint32_t mask, + erts_aint32_t set) +{ + ERTS_ATOMIC_BSET_IMPL__(erts_aint32_t, + ethr_atomic32_read, + ethr_atomic32_cmpxchg_acqb, + var, mask, set); +} +#endif + #define erts_atomic32_init_relb ethr_atomic32_init_relb #define erts_atomic32_set_relb ethr_atomic32_set_relb #define erts_atomic32_read_relb ethr_atomic32_read_relb @@ -812,6 +1020,19 @@ ERTS_GLB_INLINE void erts_thr_sigwait(const sigset_t *set, int *sig); #define erts_atomic32_xchg_relb ethr_atomic32_xchg_relb #define erts_atomic32_cmpxchg_relb ethr_atomic32_cmpxchg_relb +#if ERTS_GLB_INLINE_INCL_FUNC_DEF +ERTS_GLB_INLINE erts_aint32_t +erts_atomic32_read_bset_relb(erts_atomic32_t *var, + erts_aint32_t mask, + erts_aint32_t set) +{ + ERTS_ATOMIC_BSET_IMPL__(erts_aint32_t, + ethr_atomic32_read, + ethr_atomic32_cmpxchg_relb, + var, mask, set); +} +#endif + #define erts_atomic32_init_ddrb ethr_atomic32_init_ddrb #define erts_atomic32_set_ddrb ethr_atomic32_set_ddrb #define erts_atomic32_read_ddrb ethr_atomic32_read_ddrb @@ -826,6 +1047,19 @@ ERTS_GLB_INLINE void erts_thr_sigwait(const sigset_t *set, int *sig); #define erts_atomic32_xchg_ddrb ethr_atomic32_xchg_ddrb #define erts_atomic32_cmpxchg_ddrb ethr_atomic32_cmpxchg_ddrb +#if ERTS_GLB_INLINE_INCL_FUNC_DEF +ERTS_GLB_INLINE erts_aint32_t +erts_atomic32_read_bset_ddrb(erts_atomic32_t *var, + erts_aint32_t mask, + erts_aint32_t set) +{ + ERTS_ATOMIC_BSET_IMPL__(erts_aint32_t, + ethr_atomic32_read, + ethr_atomic32_cmpxchg_ddrb, + var, mask, set); +} +#endif + #define erts_atomic32_init_rb ethr_atomic32_init_rb #define erts_atomic32_set_rb ethr_atomic32_set_rb #define erts_atomic32_read_rb ethr_atomic32_read_rb @@ -840,6 +1074,19 @@ ERTS_GLB_INLINE void erts_thr_sigwait(const sigset_t *set, int *sig); #define erts_atomic32_xchg_rb ethr_atomic32_xchg_rb #define erts_atomic32_cmpxchg_rb ethr_atomic32_cmpxchg_rb +#if ERTS_GLB_INLINE_INCL_FUNC_DEF +ERTS_GLB_INLINE erts_aint32_t +erts_atomic32_read_bset_rb(erts_atomic32_t *var, + erts_aint32_t mask, + erts_aint32_t set) +{ + ERTS_ATOMIC_BSET_IMPL__(erts_aint32_t, + ethr_atomic32_read, + ethr_atomic32_cmpxchg_rb, + var, mask, set); +} +#endif + #define erts_atomic32_init_wb ethr_atomic32_init_wb #define erts_atomic32_set_wb ethr_atomic32_set_wb #define erts_atomic32_read_wb ethr_atomic32_read_wb @@ -854,6 +1101,21 @@ ERTS_GLB_INLINE void erts_thr_sigwait(const sigset_t *set, int *sig); #define erts_atomic32_xchg_wb ethr_atomic32_xchg_wb #define erts_atomic32_cmpxchg_wb ethr_atomic32_cmpxchg_wb +#if ERTS_GLB_INLINE_INCL_FUNC_DEF +ERTS_GLB_INLINE erts_aint32_t +erts_atomic32_read_bset_wb(erts_atomic32_t *var, + erts_aint32_t mask, + erts_aint32_t set) +{ + ERTS_ATOMIC_BSET_IMPL__(erts_aint32_t, + ethr_atomic32_read, + ethr_atomic32_cmpxchg_wb, + var, mask, set); +} +#endif + +#undef ERTS_ATOMIC_BSET_IMPL__ + #else /* !USE_THREADS */ /* Double word size atomics */ @@ -908,6 +1170,7 @@ ERTS_GLB_INLINE void erts_thr_sigwait(const sigset_t *set, int *sig); #define erts_atomic_read_band_nob erts_no_atomic_read_band #define erts_atomic_xchg_nob erts_no_atomic_xchg #define erts_atomic_cmpxchg_nob erts_no_atomic_cmpxchg +#define erts_atomic_read_bset_nob erts_no_atomic_read_bset #define erts_atomic_init_mb erts_no_atomic_set #define erts_atomic_set_mb erts_no_atomic_set @@ -922,6 +1185,7 @@ ERTS_GLB_INLINE void erts_thr_sigwait(const sigset_t *set, int *sig); #define erts_atomic_read_band_mb erts_no_atomic_read_band #define erts_atomic_xchg_mb erts_no_atomic_xchg #define erts_atomic_cmpxchg_mb erts_no_atomic_cmpxchg +#define erts_atomic_read_bset_mb erts_no_atomic_read_bset #define erts_atomic_init_acqb erts_no_atomic_set #define erts_atomic_set_acqb erts_no_atomic_set @@ -936,6 +1200,7 @@ ERTS_GLB_INLINE void erts_thr_sigwait(const sigset_t *set, int *sig); #define erts_atomic_read_band_acqb erts_no_atomic_read_band #define erts_atomic_xchg_acqb erts_no_atomic_xchg #define erts_atomic_cmpxchg_acqb erts_no_atomic_cmpxchg +#define erts_atomic_read_bset_acqb erts_no_atomic_read_bset #define erts_atomic_init_relb erts_no_atomic_set #define erts_atomic_set_relb erts_no_atomic_set @@ -950,6 +1215,7 @@ ERTS_GLB_INLINE void erts_thr_sigwait(const sigset_t *set, int *sig); #define erts_atomic_read_band_relb erts_no_atomic_read_band #define erts_atomic_xchg_relb erts_no_atomic_xchg #define erts_atomic_cmpxchg_relb erts_no_atomic_cmpxchg +#define erts_atomic_read_bset_relb erts_no_atomic_read_bset #define erts_atomic_init_ddrb erts_no_atomic_set #define erts_atomic_set_ddrb erts_no_atomic_set @@ -964,6 +1230,7 @@ ERTS_GLB_INLINE void erts_thr_sigwait(const sigset_t *set, int *sig); #define erts_atomic_read_band_ddrb erts_no_atomic_read_band #define erts_atomic_xchg_ddrb erts_no_atomic_xchg #define erts_atomic_cmpxchg_ddrb erts_no_atomic_cmpxchg +#define erts_atomic_read_bset_ddrb erts_no_atomic_read_bset #define erts_atomic_init_rb erts_no_atomic_set #define erts_atomic_set_rb erts_no_atomic_set @@ -978,6 +1245,7 @@ ERTS_GLB_INLINE void erts_thr_sigwait(const sigset_t *set, int *sig); #define erts_atomic_read_band_rb erts_no_atomic_read_band #define erts_atomic_xchg_rb erts_no_atomic_xchg #define erts_atomic_cmpxchg_rb erts_no_atomic_cmpxchg +#define erts_atomic_read_bset_rb erts_no_atomic_read_bset #define erts_atomic_init_wb erts_no_atomic_set #define erts_atomic_set_wb erts_no_atomic_set @@ -992,6 +1260,7 @@ ERTS_GLB_INLINE void erts_thr_sigwait(const sigset_t *set, int *sig); #define erts_atomic_read_band_wb erts_no_atomic_read_band #define erts_atomic_xchg_wb erts_no_atomic_xchg #define erts_atomic_cmpxchg_wb erts_no_atomic_cmpxchg +#define erts_atomic_read_bset_wb erts_no_atomic_read_bset /* 32-bit atomics */ @@ -1008,6 +1277,7 @@ ERTS_GLB_INLINE void erts_thr_sigwait(const sigset_t *set, int *sig); #define erts_atomic32_read_band_nob erts_no_atomic32_read_band #define erts_atomic32_xchg_nob erts_no_atomic32_xchg #define erts_atomic32_cmpxchg_nob erts_no_atomic32_cmpxchg +#define erts_atomic32_read_bset_nob erts_no_atomic32_read_bset #define erts_atomic32_init_mb erts_no_atomic32_set #define erts_atomic32_set_mb erts_no_atomic32_set @@ -1022,6 +1292,7 @@ ERTS_GLB_INLINE void erts_thr_sigwait(const sigset_t *set, int *sig); #define erts_atomic32_read_band_mb erts_no_atomic32_read_band #define erts_atomic32_xchg_mb erts_no_atomic32_xchg #define erts_atomic32_cmpxchg_mb erts_no_atomic32_cmpxchg +#define erts_atomic32_read_bset_mb erts_no_atomic32_read_bset #define erts_atomic32_init_acqb erts_no_atomic32_set #define erts_atomic32_set_acqb erts_no_atomic32_set @@ -1036,6 +1307,7 @@ ERTS_GLB_INLINE void erts_thr_sigwait(const sigset_t *set, int *sig); #define erts_atomic32_read_band_acqb erts_no_atomic32_read_band #define erts_atomic32_xchg_acqb erts_no_atomic32_xchg #define erts_atomic32_cmpxchg_acqb erts_no_atomic32_cmpxchg +#define erts_atomic32_read_bset_acqb erts_no_atomic32_read_bset #define erts_atomic32_init_relb erts_no_atomic32_set #define erts_atomic32_set_relb erts_no_atomic32_set @@ -1050,6 +1322,7 @@ ERTS_GLB_INLINE void erts_thr_sigwait(const sigset_t *set, int *sig); #define erts_atomic32_read_band_relb erts_no_atomic32_read_band #define erts_atomic32_xchg_relb erts_no_atomic32_xchg #define erts_atomic32_cmpxchg_relb erts_no_atomic32_cmpxchg +#define erts_atomic32_read_bset_relb erts_no_atomic32_read_bset #define erts_atomic32_init_ddrb erts_no_atomic32_set #define erts_atomic32_set_ddrb erts_no_atomic32_set @@ -1064,6 +1337,7 @@ ERTS_GLB_INLINE void erts_thr_sigwait(const sigset_t *set, int *sig); #define erts_atomic32_read_band_ddrb erts_no_atomic32_read_band #define erts_atomic32_xchg_ddrb erts_no_atomic32_xchg #define erts_atomic32_cmpxchg_ddrb erts_no_atomic32_cmpxchg +#define erts_atomic32_read_bset_ddrb erts_no_atomic32_read_bset #define erts_atomic32_init_rb erts_no_atomic32_set #define erts_atomic32_set_rb erts_no_atomic32_set @@ -1078,6 +1352,7 @@ ERTS_GLB_INLINE void erts_thr_sigwait(const sigset_t *set, int *sig); #define erts_atomic32_read_band_rb erts_no_atomic32_read_band #define erts_atomic32_xchg_rb erts_no_atomic32_xchg #define erts_atomic32_cmpxchg_rb erts_no_atomic32_cmpxchg +#define erts_atomic32_read_bset_rb erts_no_atomic32_read_bset #define erts_atomic32_init_wb erts_no_atomic32_set #define erts_atomic32_set_wb erts_no_atomic32_set @@ -1092,6 +1367,7 @@ ERTS_GLB_INLINE void erts_thr_sigwait(const sigset_t *set, int *sig); #define erts_atomic32_read_band_wb erts_no_atomic32_read_band #define erts_atomic32_xchg_wb erts_no_atomic32_xchg #define erts_atomic32_cmpxchg_wb erts_no_atomic32_cmpxchg +#define erts_atomic32_read_bset_wb erts_no_atomic32_read_bset #endif /* !USE_THREADS */ @@ -1856,6 +2132,17 @@ erts_no_atomic_cmpxchg(erts_no_atomic_t *xchgp, return old; } +ERTS_GLB_INLINE erts_aint_t +erts_no_atomic_read_bset(erts_no_atomic_t *var, + erts_aint_t mask, + erts_aint_t set) +{ + erts_aint_t old = *var; + *var &= ~mask; + *var |= (mask & set); + return old; +} + /* atomic32 */ ERTS_GLB_INLINE void @@ -1943,6 +2230,17 @@ erts_no_atomic32_cmpxchg(erts_no_atomic32_t *xchgp, return old; } +ERTS_GLB_INLINE erts_aint32_t +erts_no_atomic32_read_bset(erts_no_atomic32_t *var, + erts_aint32_t mask, + erts_aint32_t set) +{ + erts_aint32_t old = *var; + *var &= ~mask; + *var |= (mask & set); + return old; +} + /* spinlock */ ERTS_GLB_INLINE void diff --git a/erts/emulator/beam/erl_trace.c b/erts/emulator/beam/erl_trace.c index d04a91f18c..848877d43e 100644 --- a/erts/emulator/beam/erl_trace.c +++ b/erts/emulator/beam/erl_trace.c @@ -44,9 +44,9 @@ #undef DEBUG_PRINTOUTS #endif -extern Eterm beam_return_to_trace[1]; /* OpCode(i_return_to_trace) */ -extern Eterm beam_return_trace[1]; /* OpCode(i_return_trace) */ -extern Eterm beam_return_time_trace[1]; /* OpCode(i_return_time_trace) */ +extern BeamInstr beam_return_to_trace[1]; /* OpCode(i_return_to_trace) */ +extern BeamInstr beam_return_trace[1]; /* OpCode(i_return_trace) */ +extern BeamInstr beam_return_time_trace[1]; /* OpCode(i_return_time_trace) */ /* Pseudo export entries. Never filled in with data, only used to yield unique pointers of the correct type. */ @@ -156,8 +156,8 @@ do { (RES) = (TPID); } while(0) #define ERTS_TRACER_REF_TYPE Process * #define ERTS_GET_TRACER_REF(RES, TPID, TRACEE_FLGS) \ do { \ - (RES) = erts_proc_lookup((TPID)); \ - if (!(RES) || !((RES)->trace_flags & F_TRACER)) { \ + (RES) = erts_proc_lookup((TPID)); \ + if (!(RES) || !(ERTS_TRACE_FLAGS((RES)) & F_TRACER)) { \ (TPID) = NIL; \ (TRACEE_FLGS) &= ~TRACEE_FLAGS; \ return; \ @@ -409,7 +409,7 @@ WRITE_SYS_MSG_TO_PORT(Eterm unused_to, } #ifndef ERTS_SMP - if (!INVALID_TRACER_PORT(trace_port, trace_port->id)) + if (!INVALID_TRACER_PORT(trace_port, trace_port->common.id)) #endif erts_raw_port_command(trace_port, buffer, ptr-buffer); @@ -441,7 +441,7 @@ do_send_schedfix_to_port(Port *trace_port, Eterm pid, Eterm timestamp) { message = TUPLE5(hp, am_trace_ts, pid, am_out, mfarity, timestamp); /* Note, hp is deliberately NOT incremented since it will be reused */ - do_send_to_port(trace_port->id, + do_send_to_port(trace_port->common.id, trace_port, pid, SYS_MSG_TYPE_UNDEFINED, @@ -451,7 +451,7 @@ do_send_schedfix_to_port(Port *trace_port, Eterm pid, Eterm timestamp) { hp += 5; hp = patch_ts(message, hp); - do_send_to_port(trace_port->id, + do_send_to_port(trace_port->common.id, trace_port, pid, SYS_MSG_TYPE_UNDEFINED, @@ -508,7 +508,7 @@ send_to_port(Process *c_p, Eterm message, #endif do_send_to_port(*tracer_pid, trace_port, - c_p ? c_p->id : NIL, + c_p ? c_p->common.id : NIL, SYS_MSG_TYPE_TRACE, message); #ifndef ERTS_SMP @@ -543,7 +543,7 @@ send_to_port(Process *c_p, Eterm message, trace_port->control_flags &= ~PORT_CONTROL_FLAG_HEAVY; do_send_to_port(*tracer_pid, trace_port, - c_p ? c_p->id : NIL, + c_p ? c_p->common.id : NIL, SYS_MSG_TYPE_TRACE, message); @@ -557,7 +557,7 @@ send_to_port(Process *c_p, Eterm message, * just after writning the real trace message, and now gets scheduled * in again. */ - do_send_schedfix_to_port(trace_port, c_p->id, ts); + do_send_schedfix_to_port(trace_port, c_p->common.id, ts); } erts_port_release(trace_port); @@ -599,15 +599,14 @@ profile_send(Eterm from, Eterm message) { if (profiler_port) { do_send_to_port(profiler, profiler_port, - NIL, /* or current process->id */ + NIL, /* or current process->common.id */ SYS_MSG_TYPE_SYSPROF, message); erts_port_release(profiler_port); } } else { - ASSERT(is_internal_pid(profiler) - && internal_pid_index(profiler) < erts_max_processes); + ASSERT(is_internal_pid(profiler)); profile_p = erts_proc_lookup(profiler); @@ -673,7 +672,7 @@ seq_trace_send_to_port(Process *c_p, #endif do_send_to_port(seq_tracer, trace_port, - c_p ? c_p->id : NIL, + c_p ? c_p->common.id : NIL, SYS_MSG_TYPE_SEQTRACE, message); @@ -704,7 +703,7 @@ seq_trace_send_to_port(Process *c_p, trace_port->control_flags &= ~PORT_CONTROL_FLAG_HEAVY; do_send_to_port(seq_tracer, trace_port, - c_p ? c_p->id : NIL, + c_p ? c_p->common.id : NIL, SYS_MSG_TYPE_SEQTRACE, message); @@ -718,7 +717,7 @@ seq_trace_send_to_port(Process *c_p, * just after writing the real trace message, and now gets scheduled * in again. */ - do_send_schedfix_to_port(trace_port, c_p->id, ts); + do_send_schedfix_to_port(trace_port, c_p->common.id, ts); } erts_port_release(trace_port); @@ -729,7 +728,9 @@ seq_trace_send_to_port(Process *c_p, } #define TS_HEAP_WORDS 5 -#define TS_SIZE(p) (((p)->trace_flags & F_TIMESTAMP) ? TS_HEAP_WORDS : 0) +#define TS_SIZE(p) ((ERTS_TRACE_FLAGS((p)) & F_TIMESTAMP) \ + ? TS_HEAP_WORDS \ + : 0) /* * Patch a timestamp into a tuple. The tuple must be the last thing @@ -764,17 +765,17 @@ send_to_tracer(Process *tracee, erts_smp_mtx_lock(&smq_mtx); - if (tracee->trace_flags & F_TIMESTAMP) + if (ERTS_TRACE_FLAGS(tracee) & F_TIMESTAMP) *hpp = patch_ts(msg, *hpp); - if (is_internal_pid(tracee->tracer_proc)) - ERTS_ENQ_TRACE_MSG(tracee->id, tracer_ref, msg, bp); + if (is_internal_pid(ERTS_TRACER_PROC(tracee))) + ERTS_ENQ_TRACE_MSG(tracee->common.id, tracer_ref, msg, bp); else { - ASSERT(is_internal_port(tracee->tracer_proc)); + ASSERT(is_internal_port(ERTS_TRACER_PROC(tracee))); send_to_port(no_fake_sched ? NULL : tracee, msg, - &tracee->tracer_proc, - &tracee->trace_flags); + &ERTS_TRACER_PROC(tracee), + &ERTS_TRACE_FLAGS(tracee)); } erts_smp_mtx_unlock(&smq_mtx); @@ -792,7 +793,7 @@ trace_sched_aux(Process *p, Eterm what, int never_fake_sched) ERTS_TRACER_REF_TYPE tracer_ref = ERTS_NULL_TRACER_REF; int sched_no, curr_func, to_port, no_fake_sched; - if (is_nil(p->tracer_proc)) + if (is_nil(ERTS_TRACER_PROC(p))) return; no_fake_sched = never_fake_sched; @@ -812,13 +813,14 @@ trace_sched_aux(Process *p, Eterm what, int never_fake_sched) } sched_no = IS_TRACED_FL(p, F_TRACE_SCHED_NO); - to_port = is_internal_port(p->tracer_proc); + to_port = is_internal_port(ERTS_TRACER_PROC(p)); if (!to_port) { - ASSERT(is_internal_pid(p->tracer_proc) - && internal_pid_index(p->tracer_proc) < erts_max_processes); + ASSERT(is_internal_pid(ERTS_TRACER_PROC(p))); - ERTS_GET_TRACER_REF(tracer_ref, p->tracer_proc, p->trace_flags); + ERTS_GET_TRACER_REF(tracer_ref, + ERTS_TRACER_PROC(p), + ERTS_TRACE_FLAGS(p)); } if (ERTS_PROC_IS_EXITING(p)) @@ -851,7 +853,7 @@ trace_sched_aux(Process *p, Eterm what, int never_fake_sched) } if (!sched_no) { - mess = TUPLE4(hp, am_trace, p->id, what, tmp); + mess = TUPLE4(hp, am_trace, p->common.id, what, tmp); hp += 5; } else { @@ -860,7 +862,7 @@ trace_sched_aux(Process *p, Eterm what, int never_fake_sched) #else Eterm sched_id = make_small(1); #endif - mess = TUPLE5(hp, am_trace, p->id, what, sched_id, tmp); + mess = TUPLE5(hp, am_trace, p->common.id, what, sched_id, tmp); hp += 6; } @@ -912,19 +914,19 @@ trace_send(Process *p, Eterm to, Eterm msg) operation = am_atom_put(s, sys_strlen(s)); } - if (is_internal_port(p->tracer_proc)) { + if (is_internal_port(ERTS_TRACER_PROC(p))) { #define LOCAL_HEAP_SIZE (11) DeclareTmpHeapNoproc(local_heap,LOCAL_HEAP_SIZE); UseTmpHeapNoproc(LOCAL_HEAP_SIZE); hp = local_heap; - mess = TUPLE5(hp, am_trace, p->id, operation, msg, to); + mess = TUPLE5(hp, am_trace, p->common.id, operation, msg, to); hp += 6; erts_smp_mtx_lock(&smq_mtx); - if (p->trace_flags & F_TIMESTAMP) { + if (ERTS_TRACE_FLAGS(p) & F_TIMESTAMP) { hp = patch_ts(mess, hp); } - send_to_port(p, mess, &p->tracer_proc, &p->trace_flags); + send_to_port(p, mess, &ERTS_TRACER_PROC(p), &ERTS_TRACE_FLAGS(p)); UnUseTmpHeapNoproc(LOCAL_HEAP_SIZE); #undef LOCAL_HEAP_SIZE erts_smp_mtx_unlock(&smq_mtx); @@ -934,10 +936,11 @@ trace_send(Process *p, Eterm to, Eterm msg) ErlOffHeap *off_heap; ERTS_TRACER_REF_TYPE tracer_ref; - ASSERT(is_internal_pid(p->tracer_proc) - && internal_pid_index(p->tracer_proc) < erts_max_processes); + ASSERT(is_internal_pid(ERTS_TRACER_PROC(p))); - ERTS_GET_TRACER_REF(tracer_ref, p->tracer_proc, p->trace_flags); + ERTS_GET_TRACER_REF(tracer_ref, + ERTS_TRACER_PROC(p), + ERTS_TRACE_FLAGS(p)); sz_msg = size_object(msg); sz_to = size_object(to); @@ -953,16 +956,16 @@ trace_send(Process *p, Eterm to, Eterm msg) sz_msg, &hp, off_heap); - mess = TUPLE5(hp, am_trace, p->id/* Local pid */, operation, msg, to); + mess = TUPLE5(hp, am_trace, p->common.id, operation, msg, to); hp += 6; erts_smp_mtx_lock(&smq_mtx); - if (p->trace_flags & F_TIMESTAMP) { + if (ERTS_TRACE_FLAGS(p) & F_TIMESTAMP) { patch_ts(mess, hp); } - ERTS_ENQ_TRACE_MSG(p->id, tracer_ref, mess, bp); + ERTS_ENQ_TRACE_MSG(p->common.id, tracer_ref, mess, bp); erts_smp_mtx_unlock(&smq_mtx); } } @@ -977,19 +980,19 @@ trace_receive(Process *rp, Eterm msg) size_t sz_msg; Eterm* hp; - if (is_internal_port(rp->tracer_proc)) { + if (is_internal_port(ERTS_TRACER_PROC(rp))) { #define LOCAL_HEAP_SIZE (10) DeclareTmpHeapNoproc(local_heap,LOCAL_HEAP_SIZE); UseTmpHeapNoproc(LOCAL_HEAP_SIZE); hp = local_heap; - mess = TUPLE4(hp, am_trace, rp->id, am_receive, msg); + mess = TUPLE4(hp, am_trace, rp->common.id, am_receive, msg); hp += 5; erts_smp_mtx_lock(&smq_mtx); - if (rp->trace_flags & F_TIMESTAMP) { + if (ERTS_TRACE_FLAGS(rp) & F_TIMESTAMP) { hp = patch_ts(mess, hp); } - send_to_port(rp, mess, &rp->tracer_proc, &rp->trace_flags); + send_to_port(rp, mess, &ERTS_TRACER_PROC(rp), &ERTS_TRACE_FLAGS(rp)); UnUseTmpHeapNoproc(LOCAL_HEAP_SIZE); #undef LOCAL_HEAP_SIZE erts_smp_mtx_unlock(&smq_mtx); @@ -999,10 +1002,11 @@ trace_receive(Process *rp, Eterm msg) ErlOffHeap *off_heap; ERTS_TRACER_REF_TYPE tracer_ref; - ASSERT(is_internal_pid(rp->tracer_proc) - && internal_pid_index(rp->tracer_proc) < erts_max_processes); + ASSERT(is_internal_pid(ERTS_TRACER_PROC(rp))); - ERTS_GET_TRACER_REF(tracer_ref, rp->tracer_proc, rp->trace_flags); + ERTS_GET_TRACER_REF(tracer_ref, + ERTS_TRACER_PROC(rp), + ERTS_TRACE_FLAGS(rp)); sz_msg = size_object(msg); @@ -1011,16 +1015,16 @@ trace_receive(Process *rp, Eterm msg) hp = ERTS_ALLOC_SYSMSG_HEAP(hsz, &bp, &off_heap, tracer_ref); msg = copy_struct(msg, sz_msg, &hp, off_heap); - mess = TUPLE4(hp, am_trace, rp->id/* Local pid */, am_receive, msg); + mess = TUPLE4(hp, am_trace, rp->common.id, am_receive, msg); hp += 5; erts_smp_mtx_lock(&smq_mtx); - if (rp->trace_flags & F_TIMESTAMP) { + if (ERTS_TRACE_FLAGS(rp) & F_TIMESTAMP) { patch_ts(mess, hp); } - ERTS_ENQ_TRACE_MSG(rp->id, tracer_ref, mess, bp); + ERTS_ENQ_TRACE_MSG(rp->common.id, tracer_ref, mess, bp); erts_smp_mtx_unlock(&smq_mtx); } } @@ -1030,14 +1034,14 @@ seq_trace_update_send(Process *p) { Eterm seq_tracer = erts_get_system_seq_tracer(); ASSERT((is_tuple(SEQ_TRACE_TOKEN(p)) || is_nil(SEQ_TRACE_TOKEN(p)))); - if ( (p->id == seq_tracer) || (SEQ_TRACE_TOKEN(p) == NIL) + if ( (p->common.id == seq_tracer) || (SEQ_TRACE_TOKEN(p) == NIL) #ifdef USE_VM_PROBES || (SEQ_TRACE_TOKEN(p) == am_have_dt_utag) #endif ) { return 0; } - SEQ_TRACE_TOKEN_SENDER(p) = p->id; /* Internal pid */ + SEQ_TRACE_TOKEN_SENDER(p) = p->common.id; SEQ_TRACE_TOKEN_SERIAL(p) = make_small(++(p -> seq_trace_clock)); SEQ_TRACE_TOKEN_LASTCNT(p) = @@ -1074,7 +1078,7 @@ seq_trace_output_generic(Eterm token, Eterm msg, Uint type, ASSERT(is_tuple(token) || is_nil(token)); if (SEQ_TRACE_T_SENDER(token) == seq_tracer || token == NIL || - (process && process->trace_flags & F_SENSITIVE)) { + (process && ERTS_TRACE_FLAGS(process) & F_SENSITIVE)) { return; } @@ -1138,8 +1142,7 @@ seq_trace_output_generic(Eterm token, Eterm msg, Uint type, Uint sz_label, sz_lastcnt_serial, sz_msg, sz_ts, sz_sender, sz_exitfrom, sz_receiver; - ASSERT(is_internal_pid(seq_tracer) - && internal_pid_index(seq_tracer) < erts_max_processes); + ASSERT(is_internal_pid(seq_tracer)); #ifndef ERTS_SMP @@ -1253,17 +1256,17 @@ erts_trace_return_to(Process *p, BeamInstr *pc) hp += 4; } - mess = TUPLE4(hp, am_trace, p->id, am_return_to, mfa); + mess = TUPLE4(hp, am_trace, p->common.id, am_return_to, mfa); hp += 5; erts_smp_mtx_lock(&smq_mtx); - if (p->trace_flags & F_TIMESTAMP) { + if (ERTS_TRACE_FLAGS(p) & F_TIMESTAMP) { hp = patch_ts(mess, hp); } - if (is_internal_port(p->tracer_proc)) { - send_to_port(p, mess, &p->tracer_proc, &p->trace_flags); + if (is_internal_port(ERTS_TRACER_PROC(p))) { + send_to_port(p, mess, &ERTS_TRACER_PROC(p), &ERTS_TRACE_FLAGS(p)); } else { ErlHeapFragment *bp; ErlOffHeap *off_heap; @@ -1273,10 +1276,11 @@ erts_trace_return_to(Process *p, BeamInstr *pc) /* * Find the tracer. */ - ASSERT(is_internal_pid(p->tracer_proc) - && internal_pid_index(p->tracer_proc) < erts_max_processes); + ASSERT(is_internal_pid(ERTS_TRACER_PROC(p))); - ERTS_GET_TRACER_REF(tracer_ref, p->tracer_proc, p->trace_flags); + ERTS_GET_TRACER_REF(tracer_ref, + ERTS_TRACER_PROC(p), + ERTS_TRACE_FLAGS(p)); size = size_object(mess); @@ -1286,7 +1290,7 @@ erts_trace_return_to(Process *p, BeamInstr *pc) * Copy the trace message into the buffer and enqueue it. */ mess = copy_struct(mess, size, &hp, off_heap); - ERTS_ENQ_TRACE_MSG(p->id, tracer_ref, mess, bp); + ERTS_ENQ_TRACE_MSG(p->common.id, tracer_ref, mess, bp); } UnUseTmpHeapNoproc(LOCAL_HEAP_SIZE); #undef LOCAL_HEAP_SIZE @@ -1315,25 +1319,25 @@ erts_trace_return(Process* p, BeamInstr* fi, Eterm retval, Eterm *tracer_pid) /* Breakpoint trace enabled without specifying tracer => * use process tracer and flags */ - tracer_pid = &p->tracer_proc; + tracer_pid = &ERTS_TRACER_PROC(p); } if (is_nil(*tracer_pid)) { /* Trace disabled */ return; } ASSERT(is_internal_pid(*tracer_pid) || is_internal_port(*tracer_pid)); - if (*tracer_pid == p->id) { + if (*tracer_pid == p->common.id) { /* Do not generate trace messages to oneself */ return; } - if (tracer_pid == &p->tracer_proc) { + if (tracer_pid == &ERTS_TRACER_PROC(p)) { /* Tracer specified in process structure => * non-breakpoint trace => * use process flags */ - tracee_flags = &p->trace_flags; + tracee_flags = &ERTS_TRACE_FLAGS(p); #ifdef ERTS_SMP - tracee = p->id; + tracee = p->common.id; #endif } else { /* Tracer not specified in process structure => @@ -1362,7 +1366,7 @@ erts_trace_return(Process* p, BeamInstr* fi, Eterm retval, Eterm *tracer_pid) hp = local_heap; mfa = TUPLE3(hp, mod, name, make_small(arity)); hp += 4; - mess = TUPLE5(hp, am_trace, p->id, am_return_from, mfa, retval); + mess = TUPLE5(hp, am_trace, p->common.id, am_return_from, mfa, retval); hp += 6; erts_smp_mtx_lock(&smq_mtx); if (*tracee_flags & F_TIMESTAMP) { @@ -1382,8 +1386,7 @@ erts_trace_return(Process* p, BeamInstr* fi, Eterm retval, Eterm *tracer_pid) Eterm* limit; #endif - ASSERT(is_internal_pid(*tracer_pid) - && internal_pid_index(*tracer_pid) < erts_max_processes); + ASSERT(is_internal_pid(*tracer_pid)); ERTS_GET_TRACER_REF(tracer_ref, *tracer_pid, *tracee_flags); @@ -1405,7 +1408,7 @@ erts_trace_return(Process* p, BeamInstr* fi, Eterm retval, Eterm *tracer_pid) mfa = TUPLE3(hp, mod, name, make_small(arity)); hp += 4; retval = copy_struct(retval, retval_size, &hp, off_heap); - mess = TUPLE5(hp, am_trace, p->id/* Local pid */, am_return_from, mfa, retval); + mess = TUPLE5(hp, am_trace, p->common.id, am_return_from, mfa, retval); hp += 6; erts_smp_mtx_lock(&smq_mtx); @@ -1446,25 +1449,25 @@ erts_trace_exception(Process* p, BeamInstr mfa[3], Eterm class, Eterm value, /* Breakpoint trace enabled without specifying tracer => * use process tracer and flags */ - tracer_pid = &p->tracer_proc; + tracer_pid = &ERTS_TRACER_PROC(p); } if (is_nil(*tracer_pid)) { /* Trace disabled */ return; } ASSERT(is_internal_pid(*tracer_pid) || is_internal_port(*tracer_pid)); - if (*tracer_pid == p->id) { + if (*tracer_pid == p->common.id) { /* Do not generate trace messages to oneself */ return; } - if (tracer_pid == &p->tracer_proc) { + if (tracer_pid == &ERTS_TRACER_PROC(p)) { /* Tracer specified in process structure => * non-breakpoint trace => * use process flags */ - tracee_flags = &p->trace_flags; + tracee_flags = &ERTS_TRACE_FLAGS(p); #ifdef ERTS_SMP - tracee = p->id; + tracee = p->common.id; #endif if (! (*tracee_flags & F_TRACE_CALLS)) { return; @@ -1492,7 +1495,7 @@ erts_trace_exception(Process* p, BeamInstr mfa[3], Eterm class, Eterm value, hp += 4; cv = TUPLE2(hp, class, value); hp += 3; - mess = TUPLE5(hp, am_trace, p->id, am_exception_from, mfa_tuple, cv); + mess = TUPLE5(hp, am_trace, p->common.id, am_exception_from, mfa_tuple, cv); hp += 6; ASSERT((hp - local_heap) <= LOCAL_HEAP_SIZE); erts_smp_mtx_lock(&smq_mtx); @@ -1514,8 +1517,7 @@ erts_trace_exception(Process* p, BeamInstr mfa[3], Eterm class, Eterm value, Eterm* limit; #endif - ASSERT(is_internal_pid(*tracer_pid) - && internal_pid_index(*tracer_pid) < erts_max_processes); + ASSERT(is_internal_pid(*tracer_pid)); ERTS_GET_TRACER_REF(tracer_ref, *tracer_pid, *tracee_flags); @@ -1539,7 +1541,7 @@ erts_trace_exception(Process* p, BeamInstr mfa[3], Eterm class, Eterm value, value = copy_struct(value, value_size, &hp, off_heap); cv = TUPLE2(hp, class, value); hp += 3; - mess = TUPLE5(hp, am_trace, p->id/* Local pid */, + mess = TUPLE5(hp, am_trace, p->common.id, am_exception_from, mfa_tuple, cv); hp += 6; @@ -1593,25 +1595,25 @@ erts_call_trace(Process* p, BeamInstr mfa[3], Binary *match_spec, /* Breakpoint trace enabled without specifying tracer => * use process tracer and flags */ - tracer_pid = &p->tracer_proc; + tracer_pid = &ERTS_TRACER_PROC(p); } if (is_nil(*tracer_pid)) { /* Trace disabled */ return 0; } ASSERT(is_internal_pid(*tracer_pid) || is_internal_port(*tracer_pid)); - if (*tracer_pid == p->id) { + if (*tracer_pid == p->common.id) { /* Do not generate trace messages to oneself */ return 0; } - if (tracer_pid == &p->tracer_proc) { + if (tracer_pid == &ERTS_TRACER_PROC(p)) { /* Tracer specified in process structure => * non-breakpoint trace => * use process flags */ - tracee_flags = &p->trace_flags; + tracee_flags = &ERTS_TRACE_FLAGS(p); #ifdef ERTS_SMP - tracee = p->id; + tracee = p->common.id; #endif } else { /* Tracer not specified in process structure => @@ -1619,7 +1621,7 @@ erts_call_trace(Process* p, BeamInstr mfa[3], Binary *match_spec, * meta trace => * use fixed flag set instead of process flags */ - if (p->trace_flags & F_SENSITIVE) { + if (ERTS_TRACE_FLAGS(p) & F_SENSITIVE) { /* No trace messages for sensitive processes. */ return 0; } @@ -1677,7 +1679,7 @@ erts_call_trace(Process* p, BeamInstr mfa[3], Binary *match_spec, if (!erts_is_valid_tracer_port(*tracer_pid)) { #ifdef ERTS_SMP - ASSERT(is_nil(tracee) || tracer_pid == &p->tracer_proc); + ASSERT(is_nil(tracee) || tracer_pid == &ERTS_TRACER_PROC(p)); if (is_not_nil(tracee)) erts_smp_proc_lock(p, ERTS_PROC_LOCKS_ALL_MINOR); #endif @@ -1779,7 +1781,7 @@ erts_call_trace(Process* p, BeamInstr mfa[3], Binary *match_spec, * Build the trace tuple and send it to the port. */ - mess = TUPLE4(hp, am_trace, p->id, am_call, mfa_tuple); + mess = TUPLE4(hp, am_trace, p->common.id, am_call, mfa_tuple); hp += 5; if (pam_result != am_true) { hp[-5] = make_arityval(5); @@ -1814,21 +1816,21 @@ erts_call_trace(Process* p, BeamInstr mfa[3], Binary *match_spec, Eterm* limit; #endif - ASSERT(is_internal_pid(*tracer_pid) - && internal_pid_index(*tracer_pid) < erts_max_processes); + ASSERT(is_internal_pid(*tracer_pid)); tracer = erts_pid2proc(p, ERTS_PROC_LOCK_MAIN, *tracer_pid, ERTS_PROC_LOCK_STATUS); if (!tracer) invalid_tracer = 1; else { - invalid_tracer = (tracer->trace_flags & F_TRACER) == 0; + invalid_tracer = !(ERTS_TRACE_FLAGS(tracer) & F_TRACER); erts_smp_proc_unlock(tracer, ERTS_PROC_LOCK_STATUS); } if (invalid_tracer) { #ifdef ERTS_SMP - ASSERT(is_nil(tracee) || tracer_pid == &p->tracer_proc); + ASSERT(is_nil(tracee) + || tracer_pid == &ERTS_TRACER_PROC(p)); if (is_not_nil(tracee)) erts_smp_proc_lock(p, ERTS_PROC_LOCKS_ALL_MINOR); #endif @@ -1952,7 +1954,7 @@ erts_call_trace(Process* p, BeamInstr mfa[3], Binary *match_spec, * Build the trace tuple and enqueue it. */ - mess = TUPLE4(hp, am_trace, p->id/* Local pid */, am_call, mfa_tuple); + mess = TUPLE4(hp, am_trace, p->common.id, am_call, mfa_tuple); hp += 5; if (pam_result != am_true) { hp[-5] = make_arityval(5); @@ -1990,17 +1992,17 @@ trace_proc(Process *c_p, Process *t_p, Eterm what, Eterm data) ERTS_SMP_LC_ASSERT((erts_proc_lc_my_proc_locks(t_p) != 0) || erts_thr_progress_is_blocking()); - if (is_internal_port(t_p->tracer_proc)) { + if (is_internal_port(ERTS_TRACER_PROC(t_p))) { #define LOCAL_HEAP_SIZE (5+5) DeclareTmpHeapNoproc(local_heap,LOCAL_HEAP_SIZE); UseTmpHeapNoproc(LOCAL_HEAP_SIZE); hp = local_heap; - mess = TUPLE4(hp, am_trace, t_p->id, what, data); + mess = TUPLE4(hp, am_trace, t_p->common.id, what, data); hp += 5; erts_smp_mtx_lock(&smq_mtx); - if (t_p->trace_flags & F_TIMESTAMP) { + if (ERTS_TRACE_FLAGS(t_p) & F_TIMESTAMP) { hp = patch_ts(mess, hp); } send_to_port( @@ -2011,7 +2013,9 @@ trace_proc(Process *c_p, Process *t_p, Eterm what, Eterm data) /* Fake schedule out and in are never sent when smp enabled */ c_p, #endif - mess, &t_p->tracer_proc, &t_p->trace_flags); + mess, + &ERTS_TRACER_PROC(t_p), + &ERTS_TRACE_FLAGS(t_p)); UnUseTmpHeapNoproc(LOCAL_HEAP_SIZE); #undef LOCAL_HEAP_SIZE erts_smp_mtx_unlock(&smq_mtx); @@ -2022,10 +2026,11 @@ trace_proc(Process *c_p, Process *t_p, Eterm what, Eterm data) ERTS_TRACER_REF_TYPE tracer_ref; size_t sz_data; - ASSERT(is_internal_pid(t_p->tracer_proc) - && internal_pid_index(t_p->tracer_proc) < erts_max_processes); + ASSERT(is_internal_pid(ERTS_TRACER_PROC(t_p))); - ERTS_GET_TRACER_REF(tracer_ref, t_p->tracer_proc, t_p->trace_flags); + ERTS_GET_TRACER_REF(tracer_ref, + ERTS_TRACER_PROC(t_p), + ERTS_TRACE_FLAGS(t_p)); sz_data = size_object(data); @@ -2034,16 +2039,16 @@ trace_proc(Process *c_p, Process *t_p, Eterm what, Eterm data) hp = ERTS_ALLOC_SYSMSG_HEAP(need, &bp, &off_heap, tracer_ref); tmp = copy_struct(data, sz_data, &hp, off_heap); - mess = TUPLE4(hp, am_trace, t_p->id/* Local pid */, what, tmp); + mess = TUPLE4(hp, am_trace, t_p->common.id, what, tmp); hp += 5; erts_smp_mtx_lock(&smq_mtx); - if (t_p->trace_flags & F_TIMESTAMP) { + if (ERTS_TRACE_FLAGS(t_p) & F_TIMESTAMP) { hp = patch_ts(mess, hp); } - ERTS_ENQ_TRACE_MSG(t_p->id, tracer_ref, mess, bp); + ERTS_ENQ_TRACE_MSG(t_p->common.id, tracer_ref, mess, bp); erts_smp_mtx_unlock(&smq_mtx); } } @@ -2064,7 +2069,7 @@ trace_proc_spawn(Process *p, Eterm pid, Eterm mess; Eterm* hp; - if (is_internal_port(p->tracer_proc)) { + if (is_internal_port(ERTS_TRACER_PROC(p))) { #define LOCAL_HEAP_SIZE (4+6+5) DeclareTmpHeapNoproc(local_heap,LOCAL_HEAP_SIZE); UseTmpHeapNoproc(LOCAL_HEAP_SIZE); @@ -2072,13 +2077,13 @@ trace_proc_spawn(Process *p, Eterm pid, hp = local_heap; mfa = TUPLE3(hp, mod, func, args); hp += 4; - mess = TUPLE5(hp, am_trace, p->id, am_spawn, pid, mfa); + mess = TUPLE5(hp, am_trace, p->common.id, am_spawn, pid, mfa); hp += 6; erts_smp_mtx_lock(&smq_mtx); - if (p->trace_flags & F_TIMESTAMP) { + if (ERTS_TRACE_FLAGS(p) & F_TIMESTAMP) { hp = patch_ts(mess, hp); } - send_to_port(p, mess, &p->tracer_proc, &p->trace_flags); + send_to_port(p, mess, &ERTS_TRACER_PROC(p), &ERTS_TRACE_FLAGS(p)); UnUseTmpHeapNoproc(LOCAL_HEAP_SIZE); #undef LOCAL_HEAP_SIZE erts_smp_mtx_unlock(&smq_mtx); @@ -2090,10 +2095,11 @@ trace_proc_spawn(Process *p, Eterm pid, size_t sz_args, sz_pid; Uint need; - ASSERT(is_internal_pid(p->tracer_proc) - && internal_pid_index(p->tracer_proc) < erts_max_processes); + ASSERT(is_internal_pid(ERTS_TRACER_PROC(p))); - ERTS_GET_TRACER_REF(tracer_ref, p->tracer_proc, p->trace_flags); + ERTS_GET_TRACER_REF(tracer_ref, + ERTS_TRACER_PROC(p), + ERTS_TRACE_FLAGS(p)); sz_args = size_object(args); sz_pid = size_object(pid); @@ -2105,16 +2111,16 @@ trace_proc_spawn(Process *p, Eterm pid, mfa = TUPLE3(hp, mod, func, tmp); hp += 4; tmp = copy_struct(pid, sz_pid, &hp, off_heap); - mess = TUPLE5(hp, am_trace, p->id, am_spawn, tmp, mfa); + mess = TUPLE5(hp, am_trace, p->common.id, am_spawn, tmp, mfa); hp += 6; erts_smp_mtx_lock(&smq_mtx); - if (p->trace_flags & F_TIMESTAMP) { + if (ERTS_TRACE_FLAGS(p) & F_TIMESTAMP) { hp = patch_ts(mess, hp); } - ERTS_ENQ_TRACE_MSG(p->id, tracer_ref, mess, bp); + ERTS_ENQ_TRACE_MSG(p->common.id, tracer_ref, mess, bp); erts_smp_mtx_unlock(&smq_mtx); } } @@ -2204,7 +2210,7 @@ trace_gc(Process *p, Eterm what) UseTmpHeap(LOCAL_HEAP_SIZE,p); - if (is_internal_port(p->tracer_proc)) { + if (is_internal_port(ERTS_TRACER_PROC(p))) { hp = local_heap; #ifdef DEBUG size = 0; @@ -2216,10 +2222,11 @@ trace_gc(Process *p, Eterm what) size += 5/*4-tuple*/ + TS_SIZE(p); #endif } else { - ASSERT(is_internal_pid(p->tracer_proc) - && internal_pid_index(p->tracer_proc) < erts_max_processes); + ASSERT(is_internal_pid(ERTS_TRACER_PROC(p))); - ERTS_GET_TRACER_REF(tracer_ref, p->tracer_proc, p->trace_flags); + ERTS_GET_TRACER_REF(tracer_ref, + ERTS_TRACER_PROC(p), + ERTS_TRACE_FLAGS(p)); size = 0; (void) erts_bld_atom_uint_2tup_list(NULL, @@ -2243,19 +2250,19 @@ trace_gc(Process *p, Eterm what) tags, values); - msg = TUPLE4(hp, am_trace, p->id/* Local pid */, what, msg); + msg = TUPLE4(hp, am_trace, p->common.id, what, msg); hp += 5; erts_smp_mtx_lock(&smq_mtx); - if (p->trace_flags & F_TIMESTAMP) { + if (ERTS_TRACE_FLAGS(p) & F_TIMESTAMP) { hp = patch_ts(msg, hp); } ASSERT(hp == limit); - if (is_internal_port(p->tracer_proc)) - send_to_port(p, msg, &p->tracer_proc, &p->trace_flags); + if (is_internal_port(ERTS_TRACER_PROC(p))) + send_to_port(p, msg, &ERTS_TRACER_PROC(p), &ERTS_TRACE_FLAGS(p)); else - ERTS_ENQ_TRACE_MSG(p->id, tracer_ref, msg, bp); + ERTS_ENQ_TRACE_MSG(p->common.id, tracer_ref, msg, bp); erts_smp_mtx_unlock(&smq_mtx); UnUseTmpHeap(LOCAL_HEAP_SIZE,p); #undef LOCAL_HEAP_SIZE @@ -2295,8 +2302,7 @@ monitor_long_gc(Process *p, Uint time) { #endif #ifndef ERTS_SMP - ASSERT(is_internal_pid(system_monitor) - && internal_pid_index(system_monitor) < erts_max_processes); + ASSERT(is_internal_pid(system_monitor)); monitor_p = erts_proc_lookup(system_monitor); if (!monitor_p || p == monitor_p) return; @@ -2321,7 +2327,7 @@ monitor_long_gc(Process *p, Uint time) { sizeof(values)/sizeof(Uint), tags, values); - msg = TUPLE4(hp, am_monitor, p->id/* Local pid */, am_long_gc, list); + msg = TUPLE4(hp, am_monitor, p->common.id, am_long_gc, list); #ifdef DEBUG hp += 5 /* 4-tuple */; @@ -2329,7 +2335,7 @@ monitor_long_gc(Process *p, Uint time) { #endif #ifdef ERTS_SMP - enqueue_sys_msg(SYS_MSG_TYPE_SYSMON, p->id, NIL, msg, bp); + enqueue_sys_msg(SYS_MSG_TYPE_SYSMON, p->common.id, NIL, msg, bp); #else erts_queue_message(monitor_p, NULL, bp, msg, NIL #ifdef USE_VM_PROBES @@ -2370,8 +2376,7 @@ monitor_large_heap(Process *p) { #ifndef ERTS_SMP - ASSERT(is_internal_pid(system_monitor) - && internal_pid_index(system_monitor) < erts_max_processes); + ASSERT(is_internal_pid(system_monitor)); monitor_p = erts_proc_lookup(system_monitor); if (monitor_p || p == monitor_p) { return; @@ -2397,7 +2402,7 @@ monitor_large_heap(Process *p) { sizeof(values)/sizeof(Uint), tags, values); - msg = TUPLE4(hp, am_monitor, p->id/* Local pid */, am_large_heap, list); + msg = TUPLE4(hp, am_monitor, p->common.id, am_large_heap, list); #ifdef DEBUG hp += 5 /* 4-tuple */; @@ -2405,7 +2410,7 @@ monitor_large_heap(Process *p) { #endif #ifdef ERTS_SMP - enqueue_sys_msg(SYS_MSG_TYPE_SYSMON, p->id, NIL, msg, bp); + enqueue_sys_msg(SYS_MSG_TYPE_SYSMON, p->common.id, NIL, msg, bp); #else erts_queue_message(monitor_p, NULL, bp, msg, NIL #ifdef USE_VM_PROBES @@ -2425,8 +2430,7 @@ monitor_generic(Process *p, Eterm type, Eterm spec) { Eterm *hp, msg; #ifndef ERTS_SMP - ASSERT(is_internal_pid(system_monitor) - && internal_pid_index(system_monitor) < erts_max_processes); + ASSERT(is_internal_pid(system_monitor)); monitor_p = erts_proc_lookup(system_monitor); if (!monitor_p || p == monitor_p) return; @@ -2434,11 +2438,11 @@ monitor_generic(Process *p, Eterm type, Eterm spec) { hp = ERTS_ALLOC_SYSMSG_HEAP(5, &bp, &off_heap, monitor_p); - msg = TUPLE4(hp, am_monitor, p->id/* Local pid */, type, spec); + msg = TUPLE4(hp, am_monitor, p->common.id, type, spec); hp += 5; #ifdef ERTS_SMP - enqueue_sys_msg(SYS_MSG_TYPE_SYSMON, p->id, NIL, msg, bp); + enqueue_sys_msg(SYS_MSG_TYPE_SYSMON, p->common.id, NIL, msg, bp); #else erts_queue_message(monitor_p, NULL, bp, msg, NIL #ifdef USE_VM_PROBES @@ -2560,21 +2564,21 @@ trace_port_open(Port *p, Eterm calling_pid, Eterm drv_name) { Eterm mess; Eterm* hp; - if (is_internal_port(p->tracer_proc)) { + if (is_internal_port(ERTS_TRACER_PROC(p))) { #define LOCAL_HEAP_SIZE (5+6) DeclareTmpHeapNoproc(local_heap,LOCAL_HEAP_SIZE); UseTmpHeapNoproc(LOCAL_HEAP_SIZE); hp = local_heap; - mess = TUPLE5(hp, am_trace, calling_pid, am_open, p->id, drv_name); + mess = TUPLE5(hp, am_trace, calling_pid, am_open, p->common.id, drv_name); hp += 6; erts_smp_mtx_lock(&smq_mtx); - if (p->trace_flags & F_TIMESTAMP) { + if (ERTS_TRACE_FLAGS(p) & F_TIMESTAMP) { hp = patch_ts(mess, hp); } /* No fake schedule */ - send_to_port(NULL, mess, &p->tracer_proc, &p->trace_flags); + send_to_port(NULL, mess, &ERTS_TRACER_PROC(p), &ERTS_TRACE_FLAGS(p)); UnUseTmpHeapNoproc(LOCAL_HEAP_SIZE); #undef LOCAL_HEAP_SIZE erts_smp_mtx_unlock(&smq_mtx); @@ -2584,25 +2588,26 @@ trace_port_open(Port *p, Eterm calling_pid, Eterm drv_name) { size_t sz_data; ERTS_TRACER_REF_TYPE tracer_ref; - ASSERT(is_internal_pid(p->tracer_proc) - && internal_pid_index(p->tracer_proc) < erts_max_processes); + ASSERT(is_internal_pid(ERTS_TRACER_PROC(p))); sz_data = 6 + TS_SIZE(p); - ERTS_GET_TRACER_REF(tracer_ref, p->tracer_proc, p->trace_flags); + ERTS_GET_TRACER_REF(tracer_ref, + ERTS_TRACER_PROC(p), + ERTS_TRACE_FLAGS(p)); hp = ERTS_ALLOC_SYSMSG_HEAP(sz_data, &bp, &off_heap, tracer_ref); - mess = TUPLE5(hp, am_trace, calling_pid, am_open, p->id, drv_name); + mess = TUPLE5(hp, am_trace, calling_pid, am_open, p->common.id, drv_name); hp += 6; erts_smp_mtx_lock(&smq_mtx); - if (p->trace_flags & F_TIMESTAMP) { + if (ERTS_TRACE_FLAGS(p) & F_TIMESTAMP) { hp = patch_ts(mess, hp); } - ERTS_ENQ_TRACE_MSG(p->id, tracer_ref, mess, bp); + ERTS_ENQ_TRACE_MSG(p->common.id, tracer_ref, mess, bp); erts_smp_mtx_unlock(&smq_mtx); } @@ -2623,20 +2628,20 @@ trace_port(Port *t_p, Eterm what, Eterm data) { ERTS_SMP_LC_ASSERT(erts_lc_is_port_locked(t_p) || erts_thr_progress_is_blocking()); - if (is_internal_port(t_p->tracer_proc)) { + if (is_internal_port(ERTS_TRACER_PROC(t_p))) { #define LOCAL_HEAP_SIZE (5+5) DeclareTmpHeapNoproc(local_heap,LOCAL_HEAP_SIZE); UseTmpHeapNoproc(LOCAL_HEAP_SIZE); hp = local_heap; - mess = TUPLE4(hp, am_trace, t_p->id, what, data); + mess = TUPLE4(hp, am_trace, t_p->common.id, what, data); hp += 5; erts_smp_mtx_lock(&smq_mtx); - if (t_p->trace_flags & F_TIMESTAMP) { + if (ERTS_TRACE_FLAGS(t_p) & F_TIMESTAMP) { hp = patch_ts(mess, hp); } /* No fake schedule */ - send_to_port(NULL, mess, &t_p->tracer_proc, &t_p->trace_flags); + send_to_port(NULL,mess,&ERTS_TRACER_PROC(t_p),&ERTS_TRACE_FLAGS(t_p)); UnUseTmpHeapNoproc(LOCAL_HEAP_SIZE); #undef LOCAL_HEAP_SIZE erts_smp_mtx_unlock(&smq_mtx); @@ -2646,25 +2651,26 @@ trace_port(Port *t_p, Eterm what, Eterm data) { size_t sz_data; ERTS_TRACER_REF_TYPE tracer_ref; - ASSERT(is_internal_pid(t_p->tracer_proc) - && internal_pid_index(t_p->tracer_proc) < erts_max_processes); + ASSERT(is_internal_pid(ERTS_TRACER_PROC(t_p))); sz_data = 5 + TS_SIZE(t_p); - ERTS_GET_TRACER_REF(tracer_ref, t_p->tracer_proc, t_p->trace_flags); + ERTS_GET_TRACER_REF(tracer_ref, + ERTS_TRACER_PROC(t_p), + ERTS_TRACE_FLAGS(t_p)); hp = ERTS_ALLOC_SYSMSG_HEAP(sz_data, &bp, &off_heap, tracer_ref); - mess = TUPLE4(hp, am_trace, t_p->id, what, data); + mess = TUPLE4(hp, am_trace, t_p->common.id, what, data); hp += 5; erts_smp_mtx_lock(&smq_mtx); - if (t_p->trace_flags & F_TIMESTAMP) { + if (ERTS_TRACE_FLAGS(t_p) & F_TIMESTAMP) { hp = patch_ts(mess, hp); } - ERTS_ENQ_TRACE_MSG(t_p->id, tracer_ref, mess, bp); + ERTS_ENQ_TRACE_MSG(t_p->common.id, tracer_ref, mess, bp); erts_smp_mtx_unlock(&smq_mtx); } } @@ -2689,7 +2695,7 @@ trace_sched_ports_where(Port *p, Eterm what, Eterm where) { int ws = 5; Eterm sched_id = am_undefined; - if (is_internal_port(p->tracer_proc)) { + if (is_internal_port(ERTS_TRACER_PROC(p))) { #define LOCAL_HEAP_SIZE (5+6) DeclareTmpHeapNoproc(local_heap,LOCAL_HEAP_SIZE); UseTmpHeapNoproc(LOCAL_HEAP_SIZE); @@ -2704,21 +2710,21 @@ trace_sched_ports_where(Port *p, Eterm what, Eterm where) { #else sched_id = make_small(1); #endif - mess = TUPLE5(hp, am_trace, p->id, what, sched_id, where); + mess = TUPLE5(hp, am_trace, p->common.id, what, sched_id, where); ws = 6; } else { - mess = TUPLE4(hp, am_trace, p->id, what, where); + mess = TUPLE4(hp, am_trace, p->common.id, what, where); ws = 5; } hp += ws; erts_smp_mtx_lock(&smq_mtx); - if (p->trace_flags & F_TIMESTAMP) { + if (ERTS_TRACE_FLAGS(p) & F_TIMESTAMP) { hp = patch_ts(mess, hp); } /* No fake scheduling */ - send_to_port(NULL, mess, &p->tracer_proc, &p->trace_flags); + send_to_port(NULL, mess, &ERTS_TRACER_PROC(p), &ERTS_TRACE_FLAGS(p)); UnUseTmpHeapNoproc(LOCAL_HEAP_SIZE); #undef LOCAL_HEAP_SIZE erts_smp_mtx_unlock(&smq_mtx); @@ -2727,12 +2733,13 @@ trace_sched_ports_where(Port *p, Eterm what, Eterm where) { ErlOffHeap *off_heap; ERTS_TRACER_REF_TYPE tracer_ref; - ASSERT(is_internal_pid(p->tracer_proc) - && internal_pid_index(p->tracer_proc) < erts_max_processes); + ASSERT(is_internal_pid(ERTS_TRACER_PROC(p))); if (IS_TRACED_FL(p, F_TRACE_SCHED_NO)) ws = 6; /* Make place for scheduler id */ - ERTS_GET_TRACER_REF(tracer_ref, p->tracer_proc, p->trace_flags); + ERTS_GET_TRACER_REF(tracer_ref, + ERTS_TRACER_PROC(p), + ERTS_TRACE_FLAGS(p)); hp = ERTS_ALLOC_SYSMSG_HEAP(ws+TS_SIZE(p), &bp, &off_heap, tracer_ref); @@ -2744,19 +2751,19 @@ trace_sched_ports_where(Port *p, Eterm what, Eterm where) { #else sched_id = make_small(1); #endif - mess = TUPLE5(hp, am_trace, p->id, what, sched_id, where); + mess = TUPLE5(hp, am_trace, p->common.id, what, sched_id, where); } else { - mess = TUPLE4(hp, am_trace, p->id, what, where); + mess = TUPLE4(hp, am_trace, p->common.id, what, where); } hp += ws; erts_smp_mtx_lock(&smq_mtx); - if (p->trace_flags & F_TIMESTAMP) { + if (ERTS_TRACE_FLAGS(p) & F_TIMESTAMP) { hp = patch_ts(mess, hp); } - ERTS_ENQ_TRACE_MSG(p->id, tracer_ref, mess, bp); + ERTS_ENQ_TRACE_MSG(p->common.id, tracer_ref, mess, bp); erts_smp_mtx_unlock(&smq_mtx); } } @@ -2792,14 +2799,14 @@ profile_runnable_port(Port *p, Eterm status) { GET_NOW(&Ms, &s, &us); timestamp = TUPLE3(hp, make_small(Ms), make_small(s), make_small(us)); hp += 4; - msg = TUPLE5(hp, am_profile, p->id, status, count, timestamp); hp += 6; + msg = TUPLE5(hp, am_profile, p->common.id, status, count, timestamp); hp += 6; #ifndef ERTS_SMP - profile_send(p->id, msg); + profile_send(p->common.id, msg); UnUseTmpHeapNoproc(LOCAL_HEAP_SIZE); #undef LOCAL_HEAP_SIZE #else - enqueue_sys_msg_unlocked(SYS_MSG_TYPE_SYSPROF, p->id, NIL, msg, bp); + enqueue_sys_msg_unlocked(SYS_MSG_TYPE_SYSPROF, p->common.id, NIL, msg, bp); #endif erts_smp_mtx_unlock(&smq_mtx); } @@ -2846,13 +2853,13 @@ profile_runnable_proc(Process *p, Eterm status){ GET_NOW(&Ms, &s, &us); timestamp = TUPLE3(hp, make_small(Ms), make_small(s), make_small(us)); hp += 4; - msg = TUPLE5(hp, am_profile, p->id, status, where, timestamp); hp += 6; + msg = TUPLE5(hp, am_profile, p->common.id, status, where, timestamp); hp += 6; #ifndef ERTS_SMP - profile_send(p->id, msg); + profile_send(p->common.id, msg); UnUseTmpHeapNoproc(LOCAL_HEAP_SIZE); #undef LOCAL_HEAP_SIZE #else - enqueue_sys_msg_unlocked(SYS_MSG_TYPE_SYSPROF, p->id, NIL, msg, bp); + enqueue_sys_msg_unlocked(SYS_MSG_TYPE_SYSPROF, p->common.id, NIL, msg, bp); #endif erts_smp_mtx_unlock(&smq_mtx); } @@ -2865,16 +2872,19 @@ profile_runnable_proc(Process *p, Eterm status){ void erts_check_my_tracer_proc(Process *p) { - if (is_internal_pid(p->tracer_proc)) { - Process *tracer = erts_pid2proc(p, ERTS_PROC_LOCK_MAIN, - p->tracer_proc, ERTS_PROC_LOCK_STATUS); - int invalid_tracer = !tracer || !(tracer->trace_flags & F_TRACER); + if (is_internal_pid(ERTS_TRACER_PROC(p))) { + Process *tracer = erts_pid2proc(p, + ERTS_PROC_LOCK_MAIN, + ERTS_TRACER_PROC(p), + ERTS_PROC_LOCK_STATUS); + int invalid_tracer = (!tracer + || !(ERTS_TRACE_FLAGS(tracer) & F_TRACER)); if (tracer) erts_smp_proc_unlock(tracer, ERTS_PROC_LOCK_STATUS); if (invalid_tracer) { erts_smp_proc_lock(p, ERTS_PROC_LOCKS_ALL_MINOR); - p->trace_flags &= ~TRACEE_FLAGS; - p->tracer_proc = NIL; + ERTS_TRACE_FLAGS(p) &= ~TRACEE_FLAGS; + ERTS_TRACER_PROC(p) = NIL; erts_smp_proc_unlock(p, ERTS_PROC_LOCKS_ALL_MINOR); } } @@ -3218,7 +3228,7 @@ sys_msg_dispatcher_func(void *unused) proc = erts_pid2proc(NULL, 0, receiver, proc_locks); if (!proc || (smqp->type == SYS_MSG_TYPE_TRACE - && !(proc->trace_flags & F_TRACER))) { + && !(ERTS_TRACE_FLAGS(proc) & F_TRACER))) { /* Bad tracer */ #ifdef DEBUG_PRINTOUTS if (smqp->type == SYS_MSG_TYPE_TRACE && proc) @@ -3245,16 +3255,14 @@ sys_msg_dispatcher_func(void *unused) proc = erts_whereis_process(NULL,0,receiver,proc_locks,0); if (!proc) goto failure; - else if (smqp->from == proc->id) + else if (smqp->from == proc->common.id) goto drop_sys_msg; else goto queue_proc_msg; } else if (is_internal_port(receiver)) { - port = erts_id2port_sflgs(receiver, - NULL, - 0, - ERTS_PORT_SFLGS_INVALID_TRACER_LOOKUP); + port = erts_thr_id2port_sflgs(receiver, + ERTS_PORT_SFLGS_INVALID_TRACER_LOOKUP); if (!port) goto failure; else { @@ -3268,7 +3276,7 @@ sys_msg_dispatcher_func(void *unused) #ifdef DEBUG_PRINTOUTS erts_fprintf(stderr, "delivered\n"); #endif - erts_port_release(port); + erts_thr_port_release(port); if (smqp->bp) free_message_buffer(smqp->bp); } diff --git a/erts/emulator/beam/erl_trace.h b/erts/emulator/beam/erl_trace.h new file mode 100644 index 0000000000..50fb27aab0 --- /dev/null +++ b/erts/emulator/beam/erl_trace.h @@ -0,0 +1,141 @@ +/* + * %CopyrightBegin% + * + * Copyright Ericsson AB 2012. All Rights Reserved. + * + * The contents of this file are subject to the Erlang Public License, + * Version 1.1, (the "License"); you may not use this file except in + * compliance with the License. You should have received a copy of the + * Erlang Public License along with this software. If not, it can be + * retrieved online at http://www.erlang.org/. + * + * Software distributed under the License is distributed on an "AS IS" + * basis, WITHOUT WARRANTY OF ANY KIND, either express or implied. See + * the License for the specific language governing rights and limitations + * under the License. + * + * %CopyrightEnd% + */ + + +#ifndef ERL_TRACE_H__ +#define ERL_TRACE_H__ + +struct binary; + +/* erl_bif_trace.c */ +Eterm erl_seq_trace_info(Process *p, Eterm arg1); +void erts_system_monitor_clear(Process *c_p); +void erts_system_profile_clear(Process *c_p); + +/* erl_trace.c */ +void erts_init_trace(void); +void erts_trace_check_exiting(Eterm exiting); +Eterm erts_set_system_seq_tracer(Process *c_p, + ErtsProcLocks c_p_locks, + Eterm new); +Eterm erts_get_system_seq_tracer(void); +void erts_change_default_tracing(int setflags, Uint *flagsp, Eterm *tracerp); +void erts_get_default_tracing(Uint *flagsp, Eterm *tracerp); +void erts_set_system_monitor(Eterm monitor); +Eterm erts_get_system_monitor(void); + +#ifdef ERTS_SMP +void erts_check_my_tracer_proc(Process *); +void erts_block_sys_msg_dispatcher(void); +void erts_release_sys_msg_dispatcher(void); +void erts_foreach_sys_msg_in_q(void (*func)(Eterm, + Eterm, + Eterm, + ErlHeapFragment *)); +void erts_queue_error_logger_message(Eterm, Eterm, ErlHeapFragment *); +#endif + +void erts_send_sys_msg_proc(Eterm, Eterm, Eterm, ErlHeapFragment *); +void trace_send(Process*, Eterm, Eterm); +void trace_receive(Process*, Eterm); +Uint32 erts_call_trace(Process *p, BeamInstr mfa[], struct binary *match_spec, Eterm* args, + int local, Eterm *tracer_pid); +void erts_trace_return(Process* p, BeamInstr* fi, Eterm retval, Eterm *tracer_pid); +void erts_trace_exception(Process* p, BeamInstr mfa[], Eterm class, Eterm value, + Eterm *tracer); +void erts_trace_return_to(Process *p, BeamInstr *pc); +void trace_sched(Process*, Eterm); +void trace_proc(Process*, Process*, Eterm, Eterm); +void trace_proc_spawn(Process*, Eterm pid, Eterm mod, Eterm func, Eterm args); +void save_calls(Process *p, Export *); +void trace_gc(Process *p, Eterm what); +/* port tracing */ +void trace_virtual_sched(Process*, Eterm); +void trace_sched_ports(Port *pp, Eterm); +void trace_sched_ports_where(Port *pp, Eterm, Eterm); +void trace_port(Port *, Eterm what, Eterm data); +void trace_port_open(Port *, Eterm calling_pid, Eterm drv_name); + +/* system_profile */ +void erts_set_system_profile(Eterm profile); +Eterm erts_get_system_profile(void); +void profile_scheduler(Eterm scheduler_id, Eterm); +void profile_scheduler_q(Eterm scheduler_id, Eterm state, Eterm no_schedulers, Uint Ms, Uint s, Uint us); +void profile_runnable_proc(Process* p, Eterm status); +void profile_runnable_port(Port* p, Eterm status); +void erts_system_profile_setup_active_schedulers(void); + +/* system_monitor */ +void monitor_long_gc(Process *p, Uint time); +void monitor_large_heap(Process *p); +void monitor_generic(Process *p, Eterm type, Eterm spec); +Uint erts_trace_flag2bit(Eterm flag); +int erts_trace_flags(Eterm List, + Uint *pMask, Eterm *pTracer, int *pCpuTimestamp); +Eterm erts_bif_trace(int bif_index, Process* p, Eterm* args, BeamInstr *I); + +#ifdef ERTS_SMP +void erts_send_pending_trace_msgs(ErtsSchedulerData *esdp); +#define ERTS_SMP_CHK_PEND_TRACE_MSGS(ESDP) \ +do { \ + if ((ESDP)->pending_trace_msgs) \ + erts_send_pending_trace_msgs((ESDP)); \ +} while (0) +#else +#define ERTS_SMP_CHK_PEND_TRACE_MSGS(ESDP) +#endif + +#define seq_trace_output(token, msg, type, receiver, process) \ +seq_trace_output_generic((token), (msg), (type), (receiver), (process), NIL) +#define seq_trace_output_exit(token, msg, type, receiver, exitfrom) \ +seq_trace_output_generic((token), (msg), (type), (receiver), NULL, (exitfrom)) +void seq_trace_output_generic(Eterm token, Eterm msg, Uint type, + Eterm receiver, Process *process, Eterm exitfrom); + +int seq_trace_update_send(Process *process); + +Eterm erts_seq_trace(Process *process, + Eterm atom_type, Eterm atom_true_or_false, + int build_result); + +struct trace_pattern_flags { + unsigned int breakpoint : 1; /* Set if any other is set */ + unsigned int local : 1; /* Local call trace breakpoint */ + unsigned int meta : 1; /* Metadata trace breakpoint */ + unsigned int call_count : 1; /* Fast call count breakpoint */ + unsigned int call_time : 1; /* Fast call time breakpoint */ +}; +extern const struct trace_pattern_flags erts_trace_pattern_flags_off; +extern int erts_call_time_breakpoint_tracing; +int erts_set_trace_pattern(Process*p, Eterm* mfa, int specified, + struct binary* match_prog_set, + struct binary *meta_match_prog_set, + int on, struct trace_pattern_flags, + Eterm meta_tracer_pid, int is_blocking); +void +erts_get_default_trace_pattern(int *trace_pattern_is_on, + struct binary **match_spec, + struct binary **meta_match_spec, + struct trace_pattern_flags *trace_pattern_flags, + Eterm *meta_tracer_pid); +int erts_is_default_trace_enabled(void); +void erts_bif_trace_init(void); +int erts_finish_breakpointing(void); + +#endif /* ERL_TRACE_H__ */ diff --git a/erts/emulator/beam/erl_utils.h b/erts/emulator/beam/erl_utils.h new file mode 100644 index 0000000000..a2064bd8a3 --- /dev/null +++ b/erts/emulator/beam/erl_utils.h @@ -0,0 +1,215 @@ +/* + * %CopyrightBegin% + * + * Copyright Ericsson AB 2012. All Rights Reserved. + * + * The contents of this file are subject to the Erlang Public License, + * Version 1.1, (the "License"); you may not use this file except in + * compliance with the License. You should have received a copy of the + * Erlang Public License along with this software. If not, it can be + * retrieved online at http://www.erlang.org/. + * + * Software distributed under the License is distributed on an "AS IS" + * basis, WITHOUT WARRANTY OF ANY KIND, either express or implied. See + * the License for the specific language governing rights and limitations + * under the License. + * + * %CopyrightEnd% + */ + +#ifndef ERL_UTILS_H__ +#define ERL_UTILS_H__ + +#include "sys.h" +#include "erl_smp.h" +#include "erl_printf.h" + +typedef struct { +#ifdef DEBUG + int smp_api; +#endif + union { + Uint64 not_atomic; +#ifdef ARCH_64 + erts_atomic_t atomic; +#else + erts_dw_atomic_t atomic; +#endif + } counter; +} erts_interval_t; + +void erts_interval_init(erts_interval_t *); +void erts_smp_interval_init(erts_interval_t *); +Uint64 erts_step_interval_nob(erts_interval_t *); +Uint64 erts_step_interval_relb(erts_interval_t *); +Uint64 erts_smp_step_interval_nob(erts_interval_t *); +Uint64 erts_smp_step_interval_relb(erts_interval_t *); +Uint64 erts_ensure_later_interval_nob(erts_interval_t *, Uint64); +Uint64 erts_ensure_later_interval_acqb(erts_interval_t *, Uint64); +Uint64 erts_smp_ensure_later_interval_nob(erts_interval_t *, Uint64); +Uint64 erts_smp_ensure_later_interval_acqb(erts_interval_t *, Uint64); +#ifdef ARCH_32 +ERTS_GLB_INLINE Uint64 erts_interval_dw_aint_to_val__(erts_dw_aint_t *); +#endif +ERTS_GLB_INLINE Uint64 erts_current_interval_nob__(erts_interval_t *); +ERTS_GLB_INLINE Uint64 erts_current_interval_acqb__(erts_interval_t *); +ERTS_GLB_INLINE Uint64 erts_current_interval_nob(erts_interval_t *); +ERTS_GLB_INLINE Uint64 erts_current_interval_acqb(erts_interval_t *); +ERTS_GLB_INLINE Uint64 erts_smp_current_interval_nob(erts_interval_t *); +ERTS_GLB_INLINE Uint64 erts_smp_current_interval_acqb(erts_interval_t *); + +#if ERTS_GLB_INLINE_INCL_FUNC_DEF + +#ifdef ARCH_32 + +ERTS_GLB_INLINE Uint64 +erts_interval_dw_aint_to_val__(erts_dw_aint_t *dw) +{ +#ifdef ETHR_SU_DW_NAINT_T__ + return (Uint64) dw->dw_sint; +#else + Uint64 res; + res = (Uint64) ((Uint32) dw->sint[ERTS_DW_AINT_HIGH_WORD]); + res <<= 32; + res |= (Uint64) ((Uint32) dw->sint[ERTS_DW_AINT_LOW_WORD]); + return res; +#endif +} + +#endif + +ERTS_GLB_INLINE Uint64 +erts_current_interval_nob__(erts_interval_t *icp) +{ +#ifdef ARCH_64 + return (Uint64) erts_atomic_read_nob(&icp->counter.atomic); +#else + erts_dw_aint_t dw; + erts_dw_atomic_read_nob(&icp->counter.atomic, &dw); + return erts_interval_dw_aint_to_val__(&dw); +#endif +} + +ERTS_GLB_INLINE Uint64 +erts_current_interval_acqb__(erts_interval_t *icp) +{ +#ifdef ARCH_64 + return (Uint64) erts_atomic_read_acqb(&icp->counter.atomic); +#else + erts_dw_aint_t dw; + erts_dw_atomic_read_acqb(&icp->counter.atomic, &dw); + return erts_interval_dw_aint_to_val__(&dw); +#endif +} + +ERTS_GLB_INLINE Uint64 +erts_current_interval_nob(erts_interval_t *icp) +{ + ASSERT(!icp->smp_api); + return erts_current_interval_nob__(icp); +} + +ERTS_GLB_INLINE Uint64 +erts_current_interval_acqb(erts_interval_t *icp) +{ + ASSERT(!icp->smp_api); + return erts_current_interval_acqb__(icp); +} + +ERTS_GLB_INLINE Uint64 +erts_smp_current_interval_nob(erts_interval_t *icp) +{ + ASSERT(icp->smp_api); +#ifdef ERTS_SMP + return erts_current_interval_nob__(icp); +#else + return icp->counter.not_atomic; +#endif +} + +ERTS_GLB_INLINE Uint64 +erts_smp_current_interval_acqb(erts_interval_t *icp) +{ + ASSERT(icp->smp_api); +#ifdef ERTS_SMP + return erts_current_interval_acqb__(icp); +#else + return icp->counter.not_atomic; +#endif +} + +#endif /* ERTS_GLB_INLINE_INCL_FUNC_DEF */ + +/* + * To be used to silence unused result warnings, but do not abuse it. + */ +void erts_silence_warn_unused_result(long unused); + + +int erts_fit_in_bits_int64(Sint64); +int erts_fit_in_bits_int32(Sint32); +int list_length(Eterm); +int erts_is_builtin(Eterm, Eterm, int); +Uint32 make_broken_hash(Eterm); +Uint32 block_hash(byte *, unsigned, Uint32); +Uint32 make_hash2(Eterm); +Uint32 make_hash(Eterm); + + +Eterm erts_bld_atom(Uint **hpp, Uint *szp, char *str); +Eterm erts_bld_uint(Uint **hpp, Uint *szp, Uint ui); +Eterm erts_bld_uword(Uint **hpp, Uint *szp, UWord uw); +Eterm erts_bld_uint64(Uint **hpp, Uint *szp, Uint64 ui64); +Eterm erts_bld_sint64(Uint **hpp, Uint *szp, Sint64 si64); +Eterm erts_bld_cons(Uint **hpp, Uint *szp, Eterm car, Eterm cdr); +Eterm erts_bld_tuple(Uint **hpp, Uint *szp, Uint arity, ...); +Eterm erts_bld_tuplev(Uint **hpp, Uint *szp, Uint arity, Eterm terms[]); +Eterm erts_bld_string_n(Uint **hpp, Uint *szp, const char *str, Sint len); +#define erts_bld_string(hpp,szp,str) erts_bld_string_n(hpp,szp,str,strlen(str)) +Eterm erts_bld_list(Uint **hpp, Uint *szp, Sint length, Eterm terms[]); +Eterm erts_bld_2tup_list(Uint **hpp, Uint *szp, + Sint length, Eterm terms1[], Uint terms2[]); +Eterm +erts_bld_atom_uint_2tup_list(Uint **hpp, Uint *szp, + Sint length, Eterm atoms[], Uint uints[]); +Eterm +erts_bld_atom_2uint_3tup_list(Uint **hpp, Uint *szp, Sint length, + Eterm atoms[], Uint uints1[], Uint uints2[]); + +void erts_init_utils(void); +void erts_init_utils_mem(void); + +erts_dsprintf_buf_t *erts_create_tmp_dsbuf(Uint); +void erts_destroy_tmp_dsbuf(erts_dsprintf_buf_t *); + +#if HALFWORD_HEAP +int eq_rel(Eterm a, Eterm* a_base, Eterm b, Eterm* b_base); +# define eq(A,B) eq_rel(A,NULL,B,NULL) +#else +int eq(Eterm, Eterm); +# define eq_rel(A,A_BASE,B,B_BASE) eq(A,B) +#endif + +#define EQ(x,y) (((x) == (y)) || (is_not_both_immed((x),(y)) && eq((x),(y)))) + +#if HALFWORD_HEAP +Sint cmp_rel(Eterm, Eterm*, Eterm, Eterm*); +#define CMP(A,B) cmp_rel(A,NULL,B,NULL) +#else +Sint cmp(Eterm, Eterm); +#define cmp_rel(A,A_BASE,B,B_BASE) cmp(A,B) +#define CMP(A,B) cmp(A,B) +#endif +#define cmp_lt(a,b) (CMP((a),(b)) < 0) +#define cmp_le(a,b) (CMP((a),(b)) <= 0) +#define cmp_eq(a,b) (CMP((a),(b)) == 0) +#define cmp_ne(a,b) (CMP((a),(b)) != 0) +#define cmp_ge(a,b) (CMP((a),(b)) >= 0) +#define cmp_gt(a,b) (CMP((a),(b)) > 0) + +#define CMP_LT(a,b) ((a) != (b) && cmp_lt((a),(b))) +#define CMP_GE(a,b) ((a) == (b) || cmp_ge((a),(b))) +#define CMP_EQ(a,b) ((a) == (b) || cmp_eq((a),(b))) +#define CMP_NE(a,b) ((a) != (b) && cmp_ne((a),(b))) + +#endif diff --git a/erts/emulator/beam/global.h b/erts/emulator/beam/global.h index 4c0d3421c8..298241618f 100755 --- a/erts/emulator/beam/global.h +++ b/erts/emulator/beam/global.h @@ -39,38 +39,8 @@ #include "erl_sys_driver.h" #include "erl_debug.h" #include "error.h" - -typedef struct port Port; -#include "erl_port_task.h" - -typedef struct erts_driver_t_ erts_driver_t; - -#define SMALL_IO_QUEUE 5 /* Number of fixed elements */ - -typedef struct { - ErlDrvSizeT size; /* total size in bytes */ - - SysIOVec* v_start; - SysIOVec* v_end; - SysIOVec* v_head; - SysIOVec* v_tail; - SysIOVec v_small[SMALL_IO_QUEUE]; - - ErlDrvBinary** b_start; - ErlDrvBinary** b_end; - ErlDrvBinary** b_head; - ErlDrvBinary** b_tail; - ErlDrvBinary* b_small[SMALL_IO_QUEUE]; -} ErlIOQueue; - -typedef struct line_buf { /* Buffer used in line oriented I/O */ - ErlDrvSizeT bufsiz; /* Size of character buffer */ - ErlDrvSizeT ovlen; /* Length of overflow data */ - ErlDrvSizeT ovsiz; /* Actual size of overflow buffer */ - char data[1]; /* Starting point of buffer data, - data[0] is a flag indicating an unprocess CR, - The rest is the overflow buffer. */ -} LineBuf; +#include "erl_utils.h" +#include "erl_port.h" struct enif_environment_t /* ErlNifEnv */ { @@ -90,162 +60,6 @@ extern void erts_print_nif_taints(int to, void* to_arg); void erts_unload_nif(struct erl_module_nif* nif); extern void erl_nif_init(void); -/* - * Port Specific Data. - * - * Only use PrtSD for very rarely used data. - */ - -#define ERTS_PRTSD_SCHED_ID 0 - -#define ERTS_PRTSD_SIZE 1 - -typedef struct { - void *data[ERTS_PRTSD_SIZE]; -} ErtsPrtSD; - -#ifdef ERTS_SMP -typedef struct ErtsXPortsList_ ErtsXPortsList; -#endif - -/* - * Port locking: - * - * Locking is done either driver specific or port specific. When - * driver specific locking is used, all instances of the driver, - * i.e. ports running the driver, share the same lock. When port - * specific locking is used each instance have its own lock. - * - * Most fields in the Port structure are protected by the lock - * referred to by the lock field. I'v called it the port lock. - * This lock is shared between all ports running the same driver - * when driver specific locking is used. - * - * The 'sched' field is protected by the port tasks lock - * (see erl_port_tasks.c) - * - * The 'status' field is protected by a combination of the port lock, - * the port tasks lock, and the state_lck. It may be read if - * the state_lck, or the port lock is held. It may only be - * modified if both the port lock and the state_lck is held - * (with one exception; see below). When changeing status from alive - * to dead or vice versa, also the port task lock has to be held. - * This in order to guarantee that tasks are scheduled only for - * ports that are alive. - * - * The status field may be modified with only the state_lck - * held when status is changed from dead to alive. This since no - * threads can have any references to the port other than via the - * port table. - * - * /rickard - */ - -struct port { - ErtsPortTaskSched sched; - ErtsPortTaskHandle timeout_task; - erts_smp_atomic_t refc; -#ifdef ERTS_SMP - erts_smp_mtx_t *lock; - ErtsXPortsList *xports; - erts_smp_atomic_t run_queue; - erts_smp_spinlock_t state_lck; /* protects: id, status, snapshot */ -#endif - Eterm id; /* The Port id of this port */ - Eterm connected; /* A connected process */ - Eterm caller; /* Current caller. */ - Eterm data; /* Data associated with port. */ - ErlHeapFragment* bp; /* Heap fragment holding data (NULL if imm data). */ - ErtsLink *nlinks; - ErtsMonitor *monitors; /* Only MON_ORIGIN monitors of pid's */ - Uint bytes_in; /* Number of bytes read */ - Uint bytes_out; /* Number of bytes written */ -#ifdef ERTS_SMP - ErtsSmpPTimer *ptimer; -#else - ErlTimer tm; /* Timer entry */ -#endif - - Eterm tracer_proc; /* If the port is traced, this is the tracer */ - Uint trace_flags; /* Trace flags */ - - ErlIOQueue ioq; /* driver accessible i/o queue */ - DistEntry *dist_entry; /* Dist entry used in DISTRIBUTION */ - char *name; /* String used in the open */ - erts_driver_t* drv_ptr; - UWord drv_data; - SWord os_pid; /* Child process ID */ - ErtsProcList *suspended; /* List of suspended processes. */ - LineBuf *linebuf; /* Buffer to hold data not ready for - process to get (line oriented I/O)*/ - Uint32 status; /* Status and type flags */ - int control_flags; /* Flags for port_control() */ - erts_aint32_t snapshot; /* Next snapshot that port should be part of */ - struct reg_proc *reg; - ErlDrvPDL port_data_lock; - - ErtsPrtSD *psd; /* Port specific data */ -}; - - -ERTS_GLB_INLINE ErtsRunQueue *erts_port_runq(Port *prt); - -#if ERTS_GLB_INLINE_INCL_FUNC_DEF - -ERTS_GLB_INLINE ErtsRunQueue * -erts_port_runq(Port *prt) -{ -#ifdef ERTS_SMP - ErtsRunQueue *rq1, *rq2; - rq1 = (ErtsRunQueue *) erts_smp_atomic_read_nob(&prt->run_queue); - if (!rq1) - return NULL; - while (1) { - erts_smp_runq_lock(rq1); - rq2 = (ErtsRunQueue *) erts_smp_atomic_read_nob(&prt->run_queue); - if (rq1 == rq2) - return rq1; - erts_smp_runq_unlock(rq1); - rq1 = rq2; - if (!rq1) - return NULL; - } -#else - return ERTS_RUNQ_IX(0); -#endif -} - -#endif - - -ERTS_GLB_INLINE void *erts_prtsd_get(Port *p, int ix); -ERTS_GLB_INLINE void *erts_prtsd_set(Port *p, int ix, void *new); - -#if ERTS_GLB_INLINE_INCL_FUNC_DEF - -ERTS_GLB_INLINE void * -erts_prtsd_get(Port *prt, int ix) -{ - return prt->psd ? prt->psd->data[ix] : NULL; -} - -ERTS_GLB_INLINE void * -erts_prtsd_set(Port *prt, int ix, void *data) -{ - if (prt->psd) { - void *old = prt->psd->data[ix]; - prt->psd->data[ix] = data; - return old; - } - else { - prt->psd = erts_alloc(ERTS_ALC_T_PRTSD, sizeof(ErtsPrtSD)); - prt->psd->data[ix] = data; - return NULL; - } -} - -#endif - /* Driver handle (wrapper for old plain handle) */ #define ERL_DE_OK 0 #define ERL_DE_UNLOAD 1 @@ -297,7 +111,7 @@ typedef struct { or that wait for it to change state */ erts_refc_t refc; /* Number of ports/processes having references to the driver */ - Uint port_count; /* Number of ports using the driver */ + erts_smp_atomic32_t port_count; /* Number of ports using the driver */ Uint flags; /* ERL_DE_FL_KILL_PORTS */ int status; /* ERL_DE_xxx */ char *full_path; /* Full path of the driver */ @@ -349,7 +163,7 @@ struct erts_driver_t_ { }; extern erts_driver_t *driver_list; -extern erts_smp_mtx_t erts_driver_list_lock; +extern erts_smp_rwmtx_t erts_driver_list_lock; extern void erts_ddll_init(void); extern void erts_ddll_lock_driver(DE_Handle *dh, char *name); @@ -529,40 +343,9 @@ union erl_off_heap_ptr { void* voidp; }; -/* arrays that get malloced at startup */ -extern Port* erts_port; - -extern Uint erts_max_ports; -extern Uint erts_port_tab_index_mask; -extern erts_smp_atomic32_t erts_ports_snapshot; -extern erts_smp_atomic_t erts_dead_ports_ptr; - -ERTS_GLB_INLINE void erts_may_save_closed_port(Port *prt); - -#if ERTS_GLB_INLINE_INCL_FUNC_DEF - -ERTS_GLB_INLINE void erts_may_save_closed_port(Port *prt) -{ - ERTS_SMP_LC_ASSERT(erts_smp_lc_spinlock_is_locked(&prt->state_lck)); - if (prt->snapshot != erts_smp_atomic32_read_acqb(&erts_ports_snapshot)) { - /* Dead ports are added from the end of the snapshot buffer */ - Eterm* tombstone; - tombstone = (Eterm*) erts_smp_atomic_add_read_nob(&erts_dead_ports_ptr, - -(erts_aint_t)sizeof(Eterm)); - ASSERT(tombstone+1 != NULL); - ASSERT(prt->snapshot == erts_smp_atomic32_read_nob(&erts_ports_snapshot) - 1); - *tombstone = prt->id; - } - /*else no ongoing snapshot or port was already included or created after snapshot */ -} - -#endif - /* controls warning mapping in error_logger */ extern Eterm node_cookie; -extern erts_smp_atomic_t erts_bytes_out; /* no bytes written out */ -extern erts_smp_atomic_t erts_bytes_in; /* no bytes sent into the system */ extern Uint display_items; /* no of items to display in traces etc */ extern int erts_backtrace_depth; @@ -700,54 +483,6 @@ do { \ #define WSTACK_ISEMPTY(s) (WSTK_CONCAT(s,_sp) == WSTK_CONCAT(s,_start)) #define WSTACK_POP(s) (*(--WSTK_CONCAT(s,_sp))) - -/* port status flags */ - -#define ERTS_PORT_SFLG_CONNECTED ((Uint32) (1 << 0)) -/* Port have begun exiting */ -#define ERTS_PORT_SFLG_EXITING ((Uint32) (1 << 1)) -/* Distribution port */ -#define ERTS_PORT_SFLG_DISTRIBUTION ((Uint32) (1 << 2)) -#define ERTS_PORT_SFLG_BINARY_IO ((Uint32) (1 << 3)) -#define ERTS_PORT_SFLG_SOFT_EOF ((Uint32) (1 << 4)) -/* Flow control */ -#define ERTS_PORT_SFLG_PORT_BUSY ((Uint32) (1 << 5)) -/* Port is closing (no i/o accepted) */ -#define ERTS_PORT_SFLG_CLOSING ((Uint32) (1 << 6)) -/* Send a closed message when terminating */ -#define ERTS_PORT_SFLG_SEND_CLOSED ((Uint32) (1 << 7)) -/* Line orinted io on port */ -#define ERTS_PORT_SFLG_LINEBUF_IO ((Uint32) (1 << 8)) -/* Immortal port (only certain system ports) */ -#define ERTS_PORT_SFLG_IMMORTAL ((Uint32) (1 << 9)) -#define ERTS_PORT_SFLG_FREE ((Uint32) (1 << 10)) -#define ERTS_PORT_SFLG_FREE_SCHEDULED ((Uint32) (1 << 11)) -#define ERTS_PORT_SFLG_INITIALIZING ((Uint32) (1 << 12)) -/* Port uses port specific locking (opposed to driver specific locking) */ -#define ERTS_PORT_SFLG_PORT_SPECIFIC_LOCK ((Uint32) (1 << 13)) -#define ERTS_PORT_SFLG_INVALID ((Uint32) (1 << 14)) -/* Last port to terminate halts the emulator */ -#define ERTS_PORT_SFLG_HALT ((Uint32) (1 << 15)) -#ifdef DEBUG -/* Only debug: make sure all flags aren't cleared unintentionally */ -#define ERTS_PORT_SFLG_PORT_DEBUG ((Uint32) (1 << 31)) -#endif - -/* Combinations of port status flags */ -#define ERTS_PORT_SFLGS_DEAD \ - (ERTS_PORT_SFLG_FREE \ - | ERTS_PORT_SFLG_FREE_SCHEDULED \ - | ERTS_PORT_SFLG_INITIALIZING) -#define ERTS_PORT_SFLGS_INVALID_DRIVER_LOOKUP \ - (ERTS_PORT_SFLGS_DEAD | ERTS_PORT_SFLG_INVALID) -#define ERTS_PORT_SFLGS_INVALID_LOOKUP \ - (ERTS_PORT_SFLGS_INVALID_DRIVER_LOOKUP \ - | ERTS_PORT_SFLG_CLOSING) -#define ERTS_PORT_SFLGS_INVALID_TRACER_LOOKUP \ - (ERTS_PORT_SFLGS_INVALID_LOOKUP \ - | ERTS_PORT_SFLG_PORT_BUSY \ - | ERTS_PORT_SFLG_DISTRIBUTION) - /* binary.c */ void erts_emasculate_writable_binary(ProcBin* pb); @@ -758,11 +493,35 @@ Eterm erts_realloc_binary(Eterm bin, size_t size); /* erl_bif_info.c */ +Eterm +erts_bld_port_info(Eterm **hpp, + ErlOffHeap *ohp, + Uint *szp, + Port *prt, + Eterm item); + void erts_bif_info_init(void); /* bif.c */ Eterm erts_make_ref(Process *); Eterm erts_make_ref_in_buffer(Eterm buffer[REF_THING_SIZE]); +void erts_make_ref_in_array(Uint32 ref[ERTS_MAX_REF_NUMBERS]); + +ERTS_GLB_INLINE Eterm +erts_proc_store_ref(Process *c_p, Uint32 ref[ERTS_MAX_REF_NUMBERS]); + +#if ERTS_GLB_INLINE_INCL_FUNC_DEF + +ERTS_GLB_INLINE Eterm +erts_proc_store_ref(Process *c_p, Uint32 ref[ERTS_MAX_REF_NUMBERS]) +{ + Eterm *hp = HAlloc(c_p, REF_THING_SIZE); + write_ref_thing(hp, ref[0], ref[1], ref[2]); + return make_internal_ref(hp); +} + +#endif + void erts_queue_monitor_message(Process *, ErtsProcLocks*, Eterm, @@ -778,13 +537,6 @@ Eterm erl_send(Process *p, Eterm to, Eterm msg); Eterm erl_is_function(Process* p, Eterm arg1, Eterm arg2); -/* erl_bif_port.c */ - -/* erl_bif_trace.c */ -Eterm erl_seq_trace_info(Process *p, Eterm arg1); -void erts_system_monitor_clear(Process *c_p); -void erts_system_profile_clear(Process *c_p); - /* beam_load.c */ typedef struct { BeamInstr* current; /* Pointer to: Mod, Name, Arity */ @@ -960,11 +712,6 @@ void erts_free_heap_frags(Process* p); /* io.c */ -struct erl_drv_port_data_lock { - erts_mtx_t mtx; - erts_atomic_t refc; -}; - typedef struct { char *name; char *driver_name; @@ -973,477 +720,33 @@ typedef struct { #define ERTS_SPAWN_DRIVER 1 #define ERTS_SPAWN_EXECUTABLE 2 #define ERTS_SPAWN_ANY (ERTS_SPAWN_DRIVER | ERTS_SPAWN_EXECUTABLE) - int erts_add_driver_entry(ErlDrvEntry *drv, DE_Handle *handle, int driver_list_locked); void erts_destroy_driver(erts_driver_t *drv); -void erts_wake_process_later(Port*, Process*); -int erts_open_driver(erts_driver_t*, Eterm, char*, SysDriverOpts*, int *); -int erts_is_port_ioq_empty(Port *); -void erts_terminate_port(Port *); -void close_port(Eterm); -void init_io(void); -void cleanup_io(void); -void erts_do_exit_port(Port *, Eterm, Eterm); -void erts_port_command(Process *, Eterm, Port *, Eterm); -Eterm erts_port_control(Process*, Port*, Uint, Eterm); -int erts_write_to_port(Eterm caller_id, Port *p, Eterm list); -void print_port_info(int, void *, int); +int erts_save_suspend_process_on_port(Port*, Process*); +Port *erts_open_driver(erts_driver_t*, Eterm, char*, SysDriverOpts*, int *, int *); +void erts_init_io(int, int); void erts_raw_port_command(Port*, byte*, Uint); -void driver_report_exit(int, int); +void driver_report_exit(ErlDrvPort, int); LineBuf* allocate_linebuf(int); int async_ready(Port *, void*); -Sint erts_test_next_port(int, Uint); ErtsPortNames *erts_get_port_names(Eterm); void erts_free_port_names(ErtsPortNames *); Uint erts_port_ioq_size(Port *pp); void erts_stale_drv_select(Eterm, ErlDrvEvent, int, int); -void erts_port_cleanup(Port *); -void erts_fire_port_monitor(Port *prt, Eterm ref); Port *erts_get_heart_port(void); -#ifdef ERTS_SMP -void erts_smp_xports_unlock(Port *); -#endif - #if defined(ERTS_SMP) && defined(ERTS_ENABLE_LOCK_COUNT) void erts_lcnt_enable_io_lock_count(int enable); #endif -#if defined(ERTS_SMP) && defined(ERTS_ENABLE_LOCK_CHECK) -int erts_lc_is_port_locked(Port *); -#endif - -ERTS_GLB_INLINE void erts_smp_port_state_lock(Port*); -ERTS_GLB_INLINE void erts_smp_port_state_unlock(Port*); - -ERTS_GLB_INLINE int erts_smp_port_trylock(Port *prt); -ERTS_GLB_INLINE void erts_smp_port_lock(Port *prt); -ERTS_GLB_INLINE void erts_smp_port_unlock(Port *prt); - -#if ERTS_GLB_INLINE_INCL_FUNC_DEF - -ERTS_GLB_INLINE void -erts_smp_port_state_lock(Port* prt) -{ -#ifdef ERTS_SMP - erts_smp_spin_lock(&prt->state_lck); -#endif -} - -ERTS_GLB_INLINE void -erts_smp_port_state_unlock(Port *prt) -{ -#ifdef ERTS_SMP - erts_smp_spin_unlock(&prt->state_lck); -#endif -} - - -ERTS_GLB_INLINE int -erts_smp_port_trylock(Port *prt) -{ - int res; - - ASSERT(erts_smp_atomic_read_nob(&prt->refc) > 0); - erts_smp_atomic_inc_nob(&prt->refc); - -#ifdef ERTS_SMP - res = erts_smp_mtx_trylock(prt->lock); - if (res == EBUSY) { - erts_smp_atomic_dec_nob(&prt->refc); - } -#else - res = 0; -#endif - - return res; -} - -ERTS_GLB_INLINE void -erts_smp_port_lock(Port *prt) -{ - ASSERT(erts_smp_atomic_read_nob(&prt->refc) > 0); - erts_smp_atomic_inc_nob(&prt->refc); -#ifdef ERTS_SMP - erts_smp_mtx_lock(prt->lock); -#endif -} - -ERTS_GLB_INLINE void -erts_smp_port_unlock(Port *prt) -{ - erts_aint_t refc; -#ifdef ERTS_SMP - erts_smp_mtx_unlock(prt->lock); -#endif - refc = erts_smp_atomic_dec_read_nob(&prt->refc); - ASSERT(refc >= 0); - if (refc == 0) - erts_port_cleanup(prt); -} - -#endif /* #if ERTS_GLB_INLINE_INCL_FUNC_DEF */ - - -#define ERTS_INVALID_PORT_OPT(PP, ID, FLGS) \ - (!(PP) || ((PP)->status & (FLGS)) || (PP)->id != (ID)) - -/* port lookup */ - -#define INVALID_PORT(PP, ID) \ - ERTS_INVALID_PORT_OPT((PP), (ID), ERTS_PORT_SFLGS_INVALID_LOOKUP) - -/* Invalidate trace port if anything suspicious, for instance - * that the port is a distribution port or it is busy. - */ -#define INVALID_TRACER_PORT(PP, ID) \ - ERTS_INVALID_PORT_OPT((PP), (ID), ERTS_PORT_SFLGS_INVALID_TRACER_LOOKUP) - -#define ERTS_PORT_SCHED_ID(P, ID) \ - ((Uint) (UWord) erts_prtsd_set((P), ERTS_PSD_SCHED_ID, (void *) (UWord) (ID))) - -#ifdef ERTS_SMP -Port *erts_de2port(DistEntry *, Process *, ErtsProcLocks); -#endif - -#define erts_id2port(ID, P, PL) \ - erts_id2port_sflgs((ID), (P), (PL), ERTS_PORT_SFLGS_INVALID_LOOKUP) - -ERTS_GLB_INLINE Port*erts_id2port_sflgs(Eterm, Process *, ErtsProcLocks, Uint32); -ERTS_GLB_INLINE void erts_port_release(Port *); -ERTS_GLB_INLINE Port*erts_drvport2port(ErlDrvPort); -ERTS_GLB_INLINE Port*erts_drvportid2port(Eterm); -ERTS_GLB_INLINE Uint32 erts_portid2status(Eterm id); -ERTS_GLB_INLINE int erts_is_port_alive(Eterm id); -ERTS_GLB_INLINE int erts_is_valid_tracer_port(Eterm id); -ERTS_GLB_INLINE void erts_port_status_bandor_set(Port *, Uint32, Uint32); -ERTS_GLB_INLINE void erts_port_status_band_set(Port *, Uint32); -ERTS_GLB_INLINE void erts_port_status_bor_set(Port *, Uint32); -ERTS_GLB_INLINE void erts_port_status_set(Port *, Uint32); -ERTS_GLB_INLINE Uint32 erts_port_status_get(Port *); - -#if ERTS_GLB_INLINE_INCL_FUNC_DEF - -ERTS_GLB_INLINE Port* -erts_id2port_sflgs(Eterm id, Process *c_p, ErtsProcLocks c_p_locks, Uint32 sflgs) -{ -#ifdef ERTS_SMP - int no_proc_locks = !c_p || !c_p_locks; -#endif - Port *prt; - - if (is_not_internal_port(id)) - return NULL; - - prt = &erts_port[internal_port_index(id)]; - - erts_smp_port_state_lock(prt); - if (ERTS_INVALID_PORT_OPT(prt, id, sflgs)) { - erts_smp_port_state_unlock(prt); - prt = NULL; - } - else { - erts_smp_atomic_inc_nob(&prt->refc); - erts_smp_port_state_unlock(prt); - -#ifdef ERTS_SMP - if (no_proc_locks) - erts_smp_mtx_lock(prt->lock); - else if (erts_smp_mtx_trylock(prt->lock) == EBUSY) { - /* Unlock process locks, and acquire locks in lock order... */ - erts_smp_proc_unlock(c_p, c_p_locks); - erts_smp_mtx_lock(prt->lock); - erts_smp_proc_lock(c_p, c_p_locks); - } - - /* The id may not have changed... */ - ERTS_SMP_LC_ASSERT(prt->id == id); - /* ... but status may have... */ - if (prt->status & sflgs) { - erts_smp_port_unlock(prt); /* Also decrements refc... */ - prt = NULL; - } -#endif - - } - - return prt; -} - -ERTS_GLB_INLINE void -erts_port_release(Port *prt) -{ - erts_smp_port_unlock(prt); -} - -ERTS_GLB_INLINE Port* -erts_drvport2port(ErlDrvPort drvport) -{ - int ix = (int) drvport; - if (ix < 0 || erts_max_ports <= ix) - return NULL; - if (erts_port[ix].status & ERTS_PORT_SFLGS_INVALID_DRIVER_LOOKUP) - return NULL; - ERTS_SMP_LC_ASSERT(erts_lc_is_port_locked(&erts_port[ix])); - return &erts_port[ix]; -} - -ERTS_GLB_INLINE Port* -erts_drvportid2port(Eterm id) -{ - int ix; - if (is_not_internal_port(id)) - return NULL; - ix = (int) internal_port_index(id); - if (erts_max_ports <= ix) - return NULL; - if (erts_port[ix].status & ERTS_PORT_SFLGS_INVALID_DRIVER_LOOKUP) - return NULL; - if (erts_port[ix].id != id) - return NULL; - ERTS_SMP_LC_ASSERT(erts_lc_is_port_locked(&erts_port[ix])); - return &erts_port[ix]; -} - -ERTS_GLB_INLINE Uint32 -erts_portid2status(Eterm id) -{ - if (is_not_internal_port(id)) - return ERTS_PORT_SFLG_INVALID; - else { - Uint32 status; - int ix = internal_port_index(id); - if (erts_max_ports <= ix) - return ERTS_PORT_SFLG_INVALID; - erts_smp_port_state_lock(&erts_port[ix]); - if (erts_port[ix].id == id) - status = erts_port[ix].status; - else - status = ERTS_PORT_SFLG_INVALID; - erts_smp_port_state_unlock(&erts_port[ix]); - return status; - } -} - -ERTS_GLB_INLINE int -erts_is_port_alive(Eterm id) -{ - return !(erts_portid2status(id) & (ERTS_PORT_SFLG_INVALID - | ERTS_PORT_SFLGS_DEAD)); -} - -ERTS_GLB_INLINE int -erts_is_valid_tracer_port(Eterm id) -{ - return !(erts_portid2status(id) & ERTS_PORT_SFLGS_INVALID_TRACER_LOOKUP); -} - -ERTS_GLB_INLINE void erts_port_status_bandor_set(Port *prt, - Uint32 band_status, - Uint32 bor_status) -{ - ERTS_SMP_LC_ASSERT(erts_lc_is_port_locked(prt)); - erts_smp_port_state_lock(prt); - prt->status &= band_status; - prt->status |= bor_status; - erts_smp_port_state_unlock(prt); -} - -ERTS_GLB_INLINE void erts_port_status_band_set(Port *prt, Uint32 status) -{ - ERTS_SMP_LC_ASSERT(erts_lc_is_port_locked(prt)); - erts_smp_port_state_lock(prt); - prt->status &= status; - erts_smp_port_state_unlock(prt); -} - -ERTS_GLB_INLINE void erts_port_status_bor_set(Port *prt, Uint32 status) -{ - ERTS_SMP_LC_ASSERT(erts_lc_is_port_locked(prt)); - erts_smp_port_state_lock(prt); - prt->status |= status; - erts_smp_port_state_unlock(prt); -} - -ERTS_GLB_INLINE void erts_port_status_set(Port *prt, Uint32 status) -{ - ERTS_SMP_LC_ASSERT(erts_lc_is_port_locked(prt)); - erts_smp_port_state_lock(prt); - prt->status = status; - erts_smp_port_state_unlock(prt); -} - -ERTS_GLB_INLINE Uint32 erts_port_status_get(Port *prt) -{ - Uint32 res; - erts_smp_port_state_lock(prt); - res = prt->status; - erts_smp_port_state_unlock(prt); - return res; -} -#endif /* #if ERTS_GLB_INLINE_INCL_FUNC_DEF */ - /* erl_drv_thread.c */ void erl_drv_thr_init(void); -/* time.c */ - /* utils.c */ - -typedef struct { -#ifdef DEBUG - int smp_api; -#endif - union { - Uint64 not_atomic; -#ifdef ARCH_64 - erts_atomic_t atomic; -#else - erts_dw_atomic_t atomic; -#endif - } counter; -} erts_interval_t; - -void erts_interval_init(erts_interval_t *); -void erts_smp_interval_init(erts_interval_t *); -Uint64 erts_step_interval_nob(erts_interval_t *); -Uint64 erts_step_interval_relb(erts_interval_t *); -Uint64 erts_smp_step_interval_nob(erts_interval_t *); -Uint64 erts_smp_step_interval_relb(erts_interval_t *); -Uint64 erts_ensure_later_interval_nob(erts_interval_t *, Uint64); -Uint64 erts_ensure_later_interval_acqb(erts_interval_t *, Uint64); -Uint64 erts_smp_ensure_later_interval_nob(erts_interval_t *, Uint64); -Uint64 erts_smp_ensure_later_interval_acqb(erts_interval_t *, Uint64); -#ifdef ARCH_32 -ERTS_GLB_INLINE Uint64 erts_interval_dw_aint_to_val__(erts_dw_aint_t *); -#endif -ERTS_GLB_INLINE Uint64 erts_current_interval_nob__(erts_interval_t *); -ERTS_GLB_INLINE Uint64 erts_current_interval_acqb__(erts_interval_t *); -ERTS_GLB_INLINE Uint64 erts_current_interval_nob(erts_interval_t *); -ERTS_GLB_INLINE Uint64 erts_current_interval_acqb(erts_interval_t *); -ERTS_GLB_INLINE Uint64 erts_smp_current_interval_nob(erts_interval_t *); -ERTS_GLB_INLINE Uint64 erts_smp_current_interval_acqb(erts_interval_t *); - -#if ERTS_GLB_INLINE_INCL_FUNC_DEF - -#ifdef ARCH_32 - -ERTS_GLB_INLINE Uint64 -erts_interval_dw_aint_to_val__(erts_dw_aint_t *dw) -{ -#ifdef ETHR_SU_DW_NAINT_T__ - return (Uint64) dw->dw_sint; -#else - Uint64 res; - res = (Uint64) ((Uint32) dw->sint[ERTS_DW_AINT_HIGH_WORD]); - res <<= 32; - res |= (Uint64) ((Uint32) dw->sint[ERTS_DW_AINT_LOW_WORD]); - return res; -#endif -} - -#endif - -ERTS_GLB_INLINE Uint64 -erts_current_interval_nob__(erts_interval_t *icp) -{ -#ifdef ARCH_64 - return (Uint64) erts_atomic_read_nob(&icp->counter.atomic); -#else - erts_dw_aint_t dw; - erts_dw_atomic_read_nob(&icp->counter.atomic, &dw); - return erts_interval_dw_aint_to_val__(&dw); -#endif -} - -ERTS_GLB_INLINE Uint64 -erts_current_interval_acqb__(erts_interval_t *icp) -{ -#ifdef ARCH_64 - return (Uint64) erts_atomic_read_acqb(&icp->counter.atomic); -#else - erts_dw_aint_t dw; - erts_dw_atomic_read_acqb(&icp->counter.atomic, &dw); - return erts_interval_dw_aint_to_val__(&dw); -#endif -} - -ERTS_GLB_INLINE Uint64 -erts_current_interval_nob(erts_interval_t *icp) -{ - ASSERT(!icp->smp_api); - return erts_current_interval_nob__(icp); -} - -ERTS_GLB_INLINE Uint64 -erts_current_interval_acqb(erts_interval_t *icp) -{ - ASSERT(!icp->smp_api); - return erts_current_interval_acqb__(icp); -} - -ERTS_GLB_INLINE Uint64 -erts_smp_current_interval_nob(erts_interval_t *icp) -{ - ASSERT(icp->smp_api); -#ifdef ERTS_SMP - return erts_current_interval_nob__(icp); -#else - return icp->counter.not_atomic; -#endif -} - -ERTS_GLB_INLINE Uint64 -erts_smp_current_interval_acqb(erts_interval_t *icp) -{ - ASSERT(icp->smp_api); -#ifdef ERTS_SMP - return erts_current_interval_acqb__(icp); -#else - return icp->counter.not_atomic; -#endif -} - -#endif /* ERTS_GLB_INLINE_INCL_FUNC_DEF */ - -/* - * To be used to silence unused result warnings, but do not abuse it. - */ -void erts_silence_warn_unused_result(long unused); - void erts_cleanup_offheap(ErlOffHeap *offheap); -int erts_fit_in_bits_int64(Sint64); -int erts_fit_in_bits_int32(Sint32); -int list_length(Eterm); Export* erts_find_function(Eterm, Eterm, unsigned int, ErtsCodeIndex); -int erts_is_builtin(Eterm, Eterm, int); -Uint32 make_broken_hash(Eterm); -Uint32 block_hash(byte *, unsigned, Uint32); -Uint32 make_hash2(Eterm); -Uint32 make_hash(Eterm); - - -Eterm erts_bld_atom(Uint **hpp, Uint *szp, char *str); -Eterm erts_bld_uint(Uint **hpp, Uint *szp, Uint ui); -Eterm erts_bld_uword(Uint **hpp, Uint *szp, UWord uw); -Eterm erts_bld_uint64(Uint **hpp, Uint *szp, Uint64 ui64); -Eterm erts_bld_sint64(Uint **hpp, Uint *szp, Sint64 si64); -Eterm erts_bld_cons(Uint **hpp, Uint *szp, Eterm car, Eterm cdr); -Eterm erts_bld_tuple(Uint **hpp, Uint *szp, Uint arity, ...); -Eterm erts_bld_tuplev(Uint **hpp, Uint *szp, Uint arity, Eterm terms[]); -Eterm erts_bld_string_n(Uint **hpp, Uint *szp, const char *str, Sint len); -#define erts_bld_string(hpp,szp,str) erts_bld_string_n(hpp,szp,str,strlen(str)) -Eterm erts_bld_list(Uint **hpp, Uint *szp, Sint length, Eterm terms[]); -Eterm erts_bld_2tup_list(Uint **hpp, Uint *szp, - Sint length, Eterm terms1[], Uint terms2[]); -Eterm -erts_bld_atom_uint_2tup_list(Uint **hpp, Uint *szp, - Sint length, Eterm atoms[], Uint uints[]); -Eterm -erts_bld_atom_2uint_3tup_list(Uint **hpp, Uint *szp, Sint length, - Eterm atoms[], Uint uints1[], Uint uints2[]); Eterm store_external_or_ref_in_proc_(Process *, Eterm); Eterm store_external_or_ref_(Uint **, ErlOffHeap*, Eterm); @@ -1458,42 +761,6 @@ Eterm store_external_or_ref_(Uint **, ErlOffHeap*, Eterm); (ASSERT_EXPR(is_node_container((NC))), \ IS_CONST((NC)) ? (NC) : store_external_or_ref_in_proc_((Pp), (NC))) -void erts_init_utils(void); -void erts_init_utils_mem(void); - -erts_dsprintf_buf_t *erts_create_tmp_dsbuf(Uint); -void erts_destroy_tmp_dsbuf(erts_dsprintf_buf_t *); - -#if HALFWORD_HEAP -int eq_rel(Eterm a, Eterm* a_base, Eterm b, Eterm* b_base); -# define eq(A,B) eq_rel(A,NULL,B,NULL) -#else -int eq(Eterm, Eterm); -# define eq_rel(A,A_BASE,B,B_BASE) eq(A,B) -#endif - -#define EQ(x,y) (((x) == (y)) || (is_not_both_immed((x),(y)) && eq((x),(y)))) - -#if HALFWORD_HEAP -Sint cmp_rel(Eterm, Eterm*, Eterm, Eterm*); -#define CMP(A,B) cmp_rel(A,NULL,B,NULL) -#else -Sint cmp(Eterm, Eterm); -#define cmp_rel(A,A_BASE,B,B_BASE) cmp(A,B) -#define CMP(A,B) cmp(A,B) -#endif -#define cmp_lt(a,b) (CMP((a),(b)) < 0) -#define cmp_le(a,b) (CMP((a),(b)) <= 0) -#define cmp_eq(a,b) (CMP((a),(b)) == 0) -#define cmp_ne(a,b) (CMP((a),(b)) != 0) -#define cmp_ge(a,b) (CMP((a),(b)) >= 0) -#define cmp_gt(a,b) (CMP((a),(b)) > 0) - -#define CMP_LT(a,b) ((a) != (b) && cmp_lt((a),(b))) -#define CMP_GE(a,b) ((a) == (b) || cmp_ge((a),(b))) -#define CMP_EQ(a,b) ((a) == (b) || cmp_eq((a),(b))) -#define CMP_NE(a,b) ((a) != (b) && cmp_ne((a),(b))) - /* duplicates from big.h */ int term_to_Uint(Eterm term, Uint *up); int term_to_UWord(Eterm, UWord*); @@ -1530,79 +797,6 @@ Eterm erts_convert_native_to_filename(Process *p, byte *bytes); #define ERTS_UTF8_ERROR 2 #define ERTS_UTF8_ANALYZE_MORE 3 -/* erl_trace.c */ -void erts_init_trace(void); -void erts_trace_check_exiting(Eterm exiting); -Eterm erts_set_system_seq_tracer(Process *c_p, - ErtsProcLocks c_p_locks, - Eterm new); -Eterm erts_get_system_seq_tracer(void); -void erts_change_default_tracing(int setflags, Uint *flagsp, Eterm *tracerp); -void erts_get_default_tracing(Uint *flagsp, Eterm *tracerp); -void erts_set_system_monitor(Eterm monitor); -Eterm erts_get_system_monitor(void); - -#ifdef ERTS_SMP -void erts_check_my_tracer_proc(Process *); -void erts_block_sys_msg_dispatcher(void); -void erts_release_sys_msg_dispatcher(void); -void erts_foreach_sys_msg_in_q(void (*func)(Eterm, - Eterm, - Eterm, - ErlHeapFragment *)); -void erts_queue_error_logger_message(Eterm, Eterm, ErlHeapFragment *); -#endif - -void erts_send_sys_msg_proc(Eterm, Eterm, Eterm, ErlHeapFragment *); -void trace_send(Process*, Eterm, Eterm); -void trace_receive(Process*, Eterm); -Uint32 erts_call_trace(Process *p, BeamInstr mfa[], Binary *match_spec, Eterm* args, - int local, Eterm *tracer_pid); -void erts_trace_return(Process* p, BeamInstr* fi, Eterm retval, Eterm *tracer_pid); -void erts_trace_exception(Process* p, BeamInstr mfa[], Eterm class, Eterm value, - Eterm *tracer); -void erts_trace_return_to(Process *p, BeamInstr *pc); -void trace_sched(Process*, Eterm); -void trace_proc(Process*, Process*, Eterm, Eterm); -void trace_proc_spawn(Process*, Eterm pid, Eterm mod, Eterm func, Eterm args); -void save_calls(Process *p, Export *); -void trace_gc(Process *p, Eterm what); -/* port tracing */ -void trace_virtual_sched(Process*, Eterm); -void trace_sched_ports(Port *pp, Eterm); -void trace_sched_ports_where(Port *pp, Eterm, Eterm); -void trace_port(Port *, Eterm what, Eterm data); -void trace_port_open(Port *, Eterm calling_pid, Eterm drv_name); - -/* system_profile */ -void erts_set_system_profile(Eterm profile); -Eterm erts_get_system_profile(void); -void profile_scheduler(Eterm scheduler_id, Eterm); -void profile_scheduler_q(Eterm scheduler_id, Eterm state, Eterm no_schedulers, Uint Ms, Uint s, Uint us); -void profile_runnable_proc(Process* p, Eterm status); -void profile_runnable_port(Port* p, Eterm status); -void erts_system_profile_setup_active_schedulers(void); - -/* system_monitor */ -void monitor_long_gc(Process *p, Uint time); -void monitor_large_heap(Process *p); -void monitor_generic(Process *p, Eterm type, Eterm spec); -Uint erts_trace_flag2bit(Eterm flag); -int erts_trace_flags(Eterm List, - Uint *pMask, Eterm *pTracer, int *pCpuTimestamp); -Eterm erts_bif_trace(int bif_index, Process* p, Eterm* args, BeamInstr *I); - -#ifdef ERTS_SMP -void erts_send_pending_trace_msgs(ErtsSchedulerData *esdp); -#define ERTS_SMP_CHK_PEND_TRACE_MSGS(ESDP) \ -do { \ - if ((ESDP)->pending_trace_msgs) \ - erts_send_pending_trace_msgs((ESDP)); \ -} while (0) -#else -#define ERTS_SMP_CHK_PEND_TRACE_MSGS(ESDP) -#endif - void bin_write(int, void*, byte*, size_t); int intlist_to_buf(Eterm, char*, int); /* most callers pass plain char*'s */ @@ -1620,9 +814,16 @@ char* Sint_to_buf(Sint, struct Sint_buf*); #define ERTS_IOLIST_TYPE 2 Eterm buf_to_intlist(Eterm**, char*, size_t, Eterm); /* most callers pass plain char*'s */ -int io_list_to_buf(Eterm, char*, int); -int io_list_to_buf2(Eterm, char*, int); -int erts_iolist_size(Eterm, Uint *); + +#define ERTS_IOLIST_TO_BUF_OVERFLOW (~((ErlDrvSizeT) 0)) +#define ERTS_IOLIST_TO_BUF_TYPE_ERROR (~((ErlDrvSizeT) 1)) +#define ERTS_IOLIST_TO_BUF_FAILED(R) \ + (((R) & (~((ErlDrvSizeT) 1))) == (~((ErlDrvSizeT) 1))) +#define ERTS_IOLIST_TO_BUF_SUCCEEDED(R) \ + (!ERTS_IOLIST_TO_BUF_FAILED((R))) + +ErlDrvSizeT erts_iolist_to_buf(Eterm, char*, ErlDrvSizeT); +int erts_iolist_size(Eterm, ErlDrvSizeT *); int is_string(Eterm); void erl_at_exit(void (*) (void*), void*); Eterm collect_memory(Process *); @@ -1667,41 +868,6 @@ Uint erts_current_reductions(Process* current, Process *p); int erts_print_system_version(int to, void *arg, Process *c_p); int erts_hibernate(Process* c_p, Eterm module, Eterm function, Eterm args, Eterm* reg); -#define seq_trace_output(token, msg, type, receiver, process) \ -seq_trace_output_generic((token), (msg), (type), (receiver), (process), NIL) -#define seq_trace_output_exit(token, msg, type, receiver, exitfrom) \ -seq_trace_output_generic((token), (msg), (type), (receiver), NULL, (exitfrom)) -void seq_trace_output_generic(Eterm token, Eterm msg, Uint type, - Eterm receiver, Process *process, Eterm exitfrom); - -int seq_trace_update_send(Process *process); - -Eterm erts_seq_trace(Process *process, - Eterm atom_type, Eterm atom_true_or_false, - int build_result); - -struct trace_pattern_flags { - unsigned int breakpoint : 1; /* Set if any other is set */ - unsigned int local : 1; /* Local call trace breakpoint */ - unsigned int meta : 1; /* Metadata trace breakpoint */ - unsigned int call_count : 1; /* Fast call count breakpoint */ - unsigned int call_time : 1; /* Fast call time breakpoint */ -}; -extern const struct trace_pattern_flags erts_trace_pattern_flags_off; -extern int erts_call_time_breakpoint_tracing; -int erts_set_trace_pattern(Process*p, Eterm* mfa, int specified, - Binary* match_prog_set, Binary *meta_match_prog_set, - int on, struct trace_pattern_flags, - Eterm meta_tracer_pid, int is_blocking); -void -erts_get_default_trace_pattern(int *trace_pattern_is_on, - Binary **match_spec, - Binary **meta_match_spec, - struct trace_pattern_flags *trace_pattern_flags, - Eterm *meta_tracer_pid); -int erts_is_default_trace_enabled(void); -void erts_bif_trace_init(void); -int erts_finish_breakpointing(void); /* ** Call_trace uses this API for the parameter matching functions @@ -1943,15 +1109,15 @@ dtrace_pid_str(Eterm pid, char *process_buf) ERTS_GLB_INLINE void dtrace_proc_str(Process *process, char *process_buf) { - dtrace_pid_str(process->id, process_buf); + dtrace_pid_str(process->common.id, process_buf); } ERTS_GLB_INLINE void dtrace_port_str(Port *port, char *port_buf) { erts_snprintf(port_buf, DTRACE_TERM_BUF_SIZE, "#Port<%lu.%lu>", - port_channel_no(port->id), - port_number(port->id)); + port_channel_no(port->common.id), + port_number(port->common.id)); } ERTS_GLB_INLINE void diff --git a/erts/emulator/beam/index.c b/erts/emulator/beam/index.c index c981a0a55e..79c3ecf1b3 100644 --- a/erts/emulator/beam/index.c +++ b/erts/emulator/beam/index.c @@ -1,7 +1,7 @@ /* * %CopyrightBegin% * - * Copyright Ericsson AB 1996-2009. All Rights Reserved. + * Copyright Ericsson AB 1996-2012. All Rights Reserved. * * The contents of this file are subject to the Erlang Public License, * Version 1.1, (the "License"); you may not use this file except in diff --git a/erts/emulator/beam/io.c b/erts/emulator/beam/io.c index 60b9238d38..e466f0e299 100644 --- a/erts/emulator/beam/io.c +++ b/erts/emulator/beam/io.c @@ -43,6 +43,8 @@ #include "erl_version.h" #include "error.h" #include "erl_async.h" +#define ERTS_WANT_EXTERNAL_TAGS +#include "external.h" #include "dtrace-wrapper.h" extern ErlDrvEntry fd_driver_entry; @@ -51,34 +53,40 @@ extern ErlDrvEntry spawn_driver_entry; extern ErlDrvEntry *driver_tab[]; /* table of static drivers, only used during initialization */ erts_driver_t *driver_list; /* List of all drivers, static and dynamic. */ -erts_smp_mtx_t erts_driver_list_lock; /* Mutex for driver list */ +erts_smp_rwmtx_t erts_driver_list_lock; /* Mutex for driver list */ static erts_smp_tsd_key_t driver_list_lock_status_key; /*stop recursive locks when calling driver init */ static erts_smp_tsd_key_t driver_list_last_error_key; /* Save last DDLL error on a per thread basis (for BC interfaces) */ -Port* erts_port; /* The port table */ +ErtsPTab erts_port erts_align_attribute(ERTS_CACHE_LINE_SIZE); /* The port table */ erts_smp_atomic_t erts_bytes_out; /* No bytes sent out of the system */ erts_smp_atomic_t erts_bytes_in; /* No bytes gotten into the system */ -Uint erts_max_ports; -Uint erts_port_tab_index_mask; - const ErlDrvTermData driver_term_nil = (ErlDrvTermData)NIL; +const Port erts_invalid_port = {{ERTS_INVALID_PORT}}; + erts_driver_t vanilla_driver; erts_driver_t spawn_driver; erts_driver_t fd_driver; +int erts_port_synchronous_ops = 0; +int erts_port_schedule_all_ops = 0; +int erts_port_parallelism = 0; + +static void deliver_result(Eterm sender, Eterm pid, Eterm res); static int init_driver(erts_driver_t *, ErlDrvEntry *, DE_Handle *); static void terminate_port(Port *p); static void pdl_init(void); #ifdef ERTS_SMP static void driver_monitor_lock_pdl(Port *p); static void driver_monitor_unlock_pdl(Port *p); +#define DRV_MONITOR_LOOKUP_PORT_LOCK_PDL(Port) erts_thr_drvport2port_raw((Port), 1) #define DRV_MONITOR_LOCK_PDL(Port) driver_monitor_lock_pdl(Port) #define DRV_MONITOR_UNLOCK_PDL(Port) driver_monitor_unlock_pdl(Port) #else +#define DRV_MONITOR_LOOKUP_PORT_LOCK_PDL(Port) erts_thr_drvport2port_raw((Port), 0) #define DRV_MONITOR_LOCK_PDL(Port) /* nothing */ #define DRV_MONITOR_UNLOCK_PDL(Port) /* nothing */ #endif @@ -89,36 +97,12 @@ static void driver_monitor_unlock_pdl(Port *p); static ERTS_INLINE ErlIOQueue* drvport2ioq(ErlDrvPort drvport) { - int ix = (int) drvport; - Uint32 status; - - if (ix < 0 || erts_max_ports <= ix) + Port *prt = erts_thr_drvport2port_raw(drvport, 0); + erts_aint32_t state = erts_atomic32_read_nob(&prt->state); + if (state & ERTS_PORT_SFLGS_INVALID_DRIVER_LOOKUP) return NULL; - - if (erts_get_scheduler_data()) { - ERTS_SMP_LC_ASSERT(erts_lc_is_port_locked(&erts_port[ix])); - ERTS_LC_ASSERT(!erts_port[ix].port_data_lock - || erts_lc_mtx_is_locked( - &erts_port[ix].port_data_lock->mtx)); - - status = erts_port[ix].status; - } - else { - erts_smp_port_state_lock(&erts_port[ix]); - status = erts_port[ix].status; - erts_smp_port_state_unlock(&erts_port[ix]); - - ERTS_LC_ASSERT((status & ERTS_PORT_SFLGS_INVALID_DRIVER_LOOKUP) - || erts_port[ix].port_data_lock); - ERTS_LC_ASSERT(!erts_port[ix].port_data_lock - || erts_lc_mtx_is_locked( - &erts_port[ix].port_data_lock->mtx)); - - } - - return ((status & ERTS_PORT_SFLGS_INVALID_DRIVER_LOOKUP) - ? NULL - : &erts_port[ix].ioq); + else + return &prt->ioq; } static ERTS_INLINE int @@ -196,27 +180,13 @@ typedef struct line_buf_context { dtrace_port_str((PORT), port_str); #endif -/* The 'number' field in a port now has two parts: the lowest bits - contain the index in the port table, and the higher bits are a counter - which is incremented each time we look for a free port and start from - the beginning of the table. erts_max_ports is the number of file descriptors, - rounded up to a power of 2. - To get the index from a port, use the macro 'internal_port_index'; - 'port_number' returns the whole number field. -*/ - -static erts_smp_spinlock_t get_free_port_lck; -static Uint last_port_num; -static Uint port_num_mask; -erts_smp_atomic32_t erts_ports_snapshot; /* Identifies the _next_ snapshot (not the ongoing) */ - - static ERTS_INLINE void kill_port(Port *pp) { ERTS_SMP_LC_ASSERT(erts_lc_is_port_locked(pp)); + erts_ptab_delete_element(&erts_port, &pp->common); /* Time of death */ erts_port_task_free_port(pp); - ASSERT(pp->status & ERTS_PORT_SFLGS_DEAD); + /* In non-smp case the port structure may have been deallocated now */ } #ifdef ERTS_SMP @@ -227,146 +197,280 @@ erts_lc_is_port_locked(Port *prt) { if (!prt) return 0; + ERTS_SMP_LC_ASSERT(prt->lock); return erts_smp_lc_mtx_is_locked(prt->lock); } #endif #endif /* #ifdef ERTS_SMP */ -static int -get_free_port(void) -{ - Uint num; - Uint tries = erts_max_ports; - Port* port; +static void initq(Port* prt); - erts_smp_spin_lock(&get_free_port_lck); - num = last_port_num + 1; - for (;; ++num) { - port = &erts_port[num & erts_port_tab_index_mask]; +#if defined(ERTS_ENABLE_LOCK_CHECK) || defined(ERTS_ENABLE_LOCK_COUNT) +#define ERTS_PORT_INIT_INSTR_NEED_ID 1 +#else +#define ERTS_PORT_INIT_INSTR_NEED_ID 0 +#endif - erts_smp_port_state_lock(port); - if (port->status & ERTS_PORT_SFLG_FREE) { - last_port_num = num; - erts_smp_spin_unlock(&get_free_port_lck); - break; - } - erts_smp_port_state_unlock(port); +static ERTS_INLINE void port_init_instr(Port *prt +#if ERTS_PORT_INIT_INSTR_NEED_ID + , Eterm id +#endif + ) +{ +#if !ERTS_PORT_INIT_INSTR_NEED_ID + Eterm id = NIL; /* Not used */ +#endif - if (--tries == 0) { - erts_smp_spin_unlock(&get_free_port_lck); - return -1; - } + /* + * Stuff that need to be initialized with the port id + * in the instrumented case, but not in the normal case. + */ +#ifdef ERTS_SMP + ASSERT(prt->drv_ptr && prt->lock); + if (!prt->drv_ptr->lock) { + char *lock_str = "port_lock"; +#ifdef ERTS_ENABLE_LOCK_COUNT + if (!(erts_lcnt_rt_options & ERTS_LCNT_OPT_PORTLOCK)) + lock_str = NULL; +#endif + erts_mtx_init_locked_x(prt->lock, lock_str, id); } - port->status = ERTS_PORT_SFLG_INITIALIZING; - ERTS_LC_ASSERT(erts_smp_atomic_read_nob(&port->refc) == 0); - erts_smp_atomic_set_nob(&port->refc, 2); /* Port alive + lock */ - erts_smp_port_state_unlock(port); - return num & port_num_mask; +#endif + erts_port_task_init_sched(&prt->sched, id); } -/* - * erts_test_next_port() is only used for testing. - */ -Sint -erts_test_next_port(int set, Uint next) +#if !ERTS_PORT_INIT_INSTR_NEED_ID +static ERTS_INLINE void port_init_instr_abort(Port *prt) { - Uint i, num; - Sint res = -1; - - erts_smp_spin_lock(&get_free_port_lck); - if (set) { - last_port_num = (next - 1) & port_num_mask; +#ifdef ERTS_SMP + ASSERT(prt->drv_ptr && prt->lock); + if (!prt->drv_ptr->lock) { + erts_mtx_unlock(prt->lock); + erts_mtx_destroy(prt->lock); } - num = last_port_num + 1; +#endif + erts_port_task_fini_sched(&prt->sched); +} +#endif - for (i=0; i < erts_max_ports && res<0; ++i, ++num) { - - Port* port = &erts_port[num & erts_port_tab_index_mask]; +static void insert_port_struct(void *vprt, Eterm data) +{ + Port *prt = (Port *) vprt; + Eterm id = make_internal_port(data); +#if ERTS_PORT_INIT_INSTR_NEED_ID + /* + * This cannot be done earlier in the instrumented + * case since we don't now 'id' until now. + */ + port_init_instr(prt, id); +#endif + prt->common.id = id; + erts_atomic32_init_relb(&prt->state, ERTS_PORT_SFLG_INITIALIZING); +} - erts_smp_port_state_lock(port); +#define ERTS_CREATE_PORT_FLAG_PARALLELISM (1 << 0) - if (port->status & ERTS_PORT_SFLG_FREE) { - last_port_num = num - 1; - res = num & port_num_mask; - } - erts_smp_port_state_unlock(port); +static Port *create_port(char *name, + erts_driver_t *driver, + erts_mtx_t *driver_lock, + int create_flags, + Eterm pid, + int *enop) +{ + ErtsPortTaskBusyPortQ *busy_port_queue; + Port *prt; + char *p; + size_t port_size, busy_port_queue_size, size; + erts_aint32_t state = ERTS_PORT_SFLG_CONNECTED; + erts_aint32_t x_pts_flgs = 0; +#ifdef DEBUG + /* Make sure the debug flags survives until port is freed */ + state |= ERTS_PORT_SFLG_PORT_DEBUG; +#endif + +#ifdef ERTS_SMP + if (!driver_lock) { + /* Align size for mutex following port struct */ + port_size = size = ERTS_ALC_DATA_ALIGN_SIZE(sizeof(Port)); + size += sizeof(erts_mtx_t); } - erts_smp_spin_unlock(&get_free_port_lck); - return res; -} + else +#endif + port_size = size = ERTS_ALC_DATA_ALIGN_SIZE(sizeof(Port)); + busy_port_queue_size + = ((driver->flags & ERL_DRV_FLAG_NO_BUSY_MSGQ) + ? 0 + : ERTS_ALC_DATA_ALIGN_SIZE(sizeof(ErtsPortTaskBusyPortQ))); + size += busy_port_queue_size; -static void port_cleanup(Port *prt); + size += sys_strlen(name) + 1; -#ifdef ERTS_SMP + p = erts_alloc_fnf(ERTS_ALC_T_PORT, size); + if (!p) { + if (enop) + *enop = ENOMEM; + return NULL; + } -static void -sched_port_cleanup(void *vprt) -{ - Port *prt = (Port *) vprt; - erts_smp_mtx_lock(prt->lock); - port_cleanup(prt); -} + prt = (Port *) p; + p += port_size; -#endif + if (!busy_port_queue_size) + busy_port_queue = NULL; + else { + busy_port_queue = (ErtsPortTaskBusyPortQ *) p; + p += busy_port_queue_size; + } -void -erts_port_cleanup(Port *prt) -{ #ifdef ERTS_SMP - if (erts_smp_mtx_trylock(prt->lock) == EBUSY) - erts_schedule_misc_op(sched_port_cleanup, (void *) prt); - else + if (driver_lock) { + prt->lock = driver_lock; + erts_mtx_lock(driver_lock); + } + else { + prt->lock = (erts_mtx_t *) p; + p += sizeof(erts_mtx_t); + state |= ERTS_PORT_SFLG_PORT_SPECIFIC_LOCK; + } + erts_smp_atomic_set_nob(&prt->run_queue, + (erts_aint_t) erts_get_runq_current(NULL)); + prt->xports = NULL; +#else + erts_atomic32_init_nob(&prt->refc, 1); + prt->cleanup = 0; #endif - port_cleanup(prt); -} + + erts_port_task_pre_init_sched(&prt->sched, busy_port_queue); -void -port_cleanup(Port *prt) -{ + prt->name = p; + sys_strcpy(p, name); + prt->drv_ptr = driver; + ERTS_P_LINKS(prt) = NULL; + ERTS_P_MONITORS(prt) = NULL; + prt->linebuf = NULL; + prt->bp = NULL; + prt->suspended = NULL; + prt->data = am_undefined; + prt->port_data_lock = NULL; + prt->control_flags = 0; + prt->bytes_in = 0; + prt->bytes_out = 0; + prt->dist_entry = NULL; + ERTS_PORT_INIT_CONNECTED(prt, pid); + prt->common.u.alive.reg = NULL; #ifdef ERTS_SMP - Uint32 port_specific; - erts_smp_mtx_t *mtx; + prt->common.u.alive.ptimer = NULL; +#else + sys_memset(&prt->common.u.alive.tm, 0, sizeof(ErlTimer)); #endif - erts_driver_t *driver; + erts_port_task_handle_init(&prt->timeout_task); + prt->psd = NULL; + prt->drv_data = (SWord) 0; + prt->os_pid = -1; - erts_smp_port_state_lock(prt); + /* Set default tracing */ + erts_get_default_tracing(&ERTS_TRACE_FLAGS(prt), &ERTS_TRACER_PROC(prt)); - ERTS_SMP_LC_ASSERT(erts_lc_is_port_locked(prt)); - driver = prt->drv_ptr; - prt->drv_ptr = NULL; - ASSERT(driver); + ASSERT(((char *) prt) == ((char *) &prt->common)); - ASSERT(prt->status & ERTS_PORT_SFLG_FREE_SCHEDULED); - ERTS_LC_ASSERT(erts_smp_atomic_read_nob(&prt->refc) == 0); +#if !ERTS_PORT_INIT_INSTR_NEED_ID + /* + * When 'id' isn't needed (the normal case), it is better to + * do the initialization here avoiding unnecessary contention + * on table... + */ + port_init_instr(prt); +#endif - ASSERT(prt->status & ERTS_PORT_SFLG_PORT_DEBUG); - ASSERT(!(prt->status & ERTS_PORT_SFLG_FREE)); - prt->status = ERTS_PORT_SFLG_FREE; + if (!erts_ptab_new_element(&erts_port, + &prt->common, + (void *) prt, + insert_port_struct)) { +#if !ERTS_PORT_INIT_INSTR_NEED_ID + port_init_instr_abort(prt); +#endif #ifdef ERTS_SMP + if (driver_lock) + erts_mtx_unlock(driver_lock); +#endif + if (enop) + *enop = 0; + return NULL; + } - port_specific = (prt->status & ERTS_PORT_SFLG_PORT_SPECIFIC_LOCK); + ASSERT(prt == (Port *) (erts_ptab_pix2intptr_nob( + &erts_port, + internal_port_index(prt->common.id)))); - mtx = prt->lock; - ASSERT(mtx); + initq(prt); - prt->lock = NULL; + ERTS_SMP_LC_ASSERT(erts_lc_is_port_locked(prt)); - erts_smp_port_state_unlock(prt); - erts_smp_mtx_unlock(mtx); + if (erts_port_schedule_all_ops) + x_pts_flgs |= ERTS_PTS_FLG_FORCE_SCHED; - if (port_specific) { - erts_smp_mtx_destroy(mtx); - erts_free(ERTS_ALC_T_PORT_LOCK, mtx); - } -#endif + if (create_flags & ERTS_CREATE_PORT_FLAG_PARALLELISM) + x_pts_flgs |= ERTS_PTS_FLG_PARALLELISM; - if (driver->handle) - erts_ddll_dereference_driver(driver->handle); + if (x_pts_flgs) + erts_smp_atomic32_read_bor_nob(&prt->sched.flags, x_pts_flgs); + + erts_atomic32_set_relb(&prt->state, state); + return prt; +} + +#ifndef ERTS_SMP +void +erts_port_cleanup(Port *prt) +{ + if (prt->drv_ptr && prt->drv_ptr->handle) + erts_ddll_dereference_driver(prt->drv_ptr->handle); + prt->drv_ptr = NULL; + erts_port_dec_refc(prt); } +#endif +void +erts_port_free(Port *prt) +{ +#if defined(ERTS_SMP) || defined(DEBUG) || defined(ERTS_ENABLE_LOCK_CHECK) + erts_aint32_t state = erts_atomic32_read_nob(&prt->state); +#endif + ERTS_LC_ASSERT(state & (ERTS_PORT_SFLG_INITIALIZING + | ERTS_PORT_SFLG_FREE)); + ASSERT(state & ERTS_PORT_SFLG_PORT_DEBUG); + +#ifdef ERTS_SMP + ERTS_LC_ASSERT(erts_atomic32_read_nob(&prt->common.refc) == 0); +#else + ERTS_LC_ASSERT(erts_atomic32_read_nob(&prt->refc) == 0); +#endif + + erts_port_task_fini_sched(&prt->sched); + +#ifdef ERTS_SMP + ASSERT(prt->lock); + if (state & ERTS_PORT_SFLG_PORT_SPECIFIC_LOCK) + erts_mtx_destroy(prt->lock); + + /* + * We cannot dereference a driver using driver + * locking until here in smp case. Otherwise, + * the driver lock may still be in use by others. + * + * In the non-smp case we cannot do it here since + * this function may be called by non-scheduler + * threads. This is done in erts_port_cleanup() + * in the non-smp case. + */ + if (prt->drv_ptr->handle) + erts_ddll_dereference_driver(prt->drv_ptr->handle); +#endif + erts_free(ERTS_ALC_T_PORT, prt); +} /* ** Initialize v_start to point to the small fixed vector. @@ -414,94 +518,21 @@ static void stopq(Port* prt) if (prt->port_data_lock) { driver_pdl_unlock(prt->port_data_lock); driver_pdl_dec_refc(prt->port_data_lock); - prt->port_data_lock = NULL; - } -} - - - -static void -setup_port(Port* prt, Eterm pid, erts_driver_t *driver, - ErlDrvData drv_data, char *name, Uint32 xstatus) -{ - ErtsRunQueue *runq = erts_get_runq_current(NULL); - char *new_name, *old_name; -#ifdef DEBUG - /* Make sure the debug flags survives until port is freed */ - xstatus |= ERTS_PORT_SFLG_PORT_DEBUG; -#endif - ASSERT(runq); - ERTS_SMP_LC_ASSERT(erts_lc_is_port_locked(prt)); - - - new_name = (char*) erts_alloc(ERTS_ALC_T_PORT_NAME, sys_strlen(name)+1); - sys_strcpy(new_name, name); - erts_smp_runq_lock(runq); - erts_smp_port_state_lock(prt); - prt->os_pid = -1; - prt->status = ERTS_PORT_SFLG_CONNECTED | xstatus; - prt->snapshot = erts_smp_atomic32_read_nob(&erts_ports_snapshot); - old_name = prt->name; - prt->name = new_name; -#ifdef ERTS_SMP - erts_smp_atomic_set_nob(&prt->run_queue, (erts_aint_t) runq); -#endif - ASSERT(!prt->drv_ptr); - prt->drv_ptr = driver; - erts_smp_port_state_unlock(prt); - erts_smp_runq_unlock(runq); -#ifdef ERTS_SMP - ASSERT(!prt->xports); -#endif - if (old_name) { - erts_free(ERTS_ALC_T_PORT_NAME, (void *) old_name); } - - prt->control_flags = 0; - prt->connected = pid; - prt->drv_data = (SWord) drv_data; - prt->bytes_in = 0; - prt->bytes_out = 0; - prt->dist_entry = NULL; - prt->reg = NULL; -#ifdef ERTS_SMP - prt->ptimer = NULL; -#else - sys_memset(&prt->tm, 0, sizeof(ErlTimer)); -#endif - erts_port_task_handle_init(&prt->timeout_task); - prt->suspended = NULL; - sys_strcpy(prt->name, name); - prt->nlinks = NULL; - prt->monitors = NULL; - prt->linebuf = NULL; - prt->bp = NULL; - prt->data = am_undefined; - /* Set default tracing */ - erts_get_default_tracing(&(prt->trace_flags), &(prt->tracer_proc)); - - prt->psd = NULL; - - initq(prt); } -void -erts_wake_process_later(Port *prt, Process *process) +int +erts_save_suspend_process_on_port(Port *prt, Process *process) { - ErtsProcList** p; - ErtsProcList* new_p; - - ERTS_SMP_LC_ASSERT(erts_lc_is_port_locked(prt)); - - if (prt->status & ERTS_PORT_SFLGS_DEAD) - return; - - for (p = &(prt->suspended); *p != NULL; p = &((*p)->next)) - /* Empty loop body */; - - new_p = erts_proclist_create(process); - new_p->next = NULL; - *p = new_p; + int saved; + erts_aint32_t flags; + erts_port_task_sched_lock(&prt->sched); + flags = erts_smp_atomic32_read_nob(&prt->sched.flags); + saved = (flags & ERTS_PTS_FLGS_BUSY) && !(flags & ERTS_PTS_FLG_EXIT); + if (saved) + erts_proclist_store_last(&prt->suspended, erts_proclist_create(process)); + erts_port_task_sched_unlock(&prt->sched); + return saved; } /* @@ -513,47 +544,44 @@ erts_wake_process_later(Port *prt, Process *process) (*error_number_ptr must contain either BADARG or SYSTEM_LIMIT). The driver start function must obey the same conventions. */ -int +Port * erts_open_driver(erts_driver_t* driver, /* Pointer to driver. */ Eterm pid, /* Current process. */ char* name, /* Driver name. */ SysDriverOpts* opts, /* Options. */ - int *error_number_ptr) /* errno in case -2 is returned */ + int *error_type_ptr, /* error type */ + int *error_number_ptr) /* errno in case of error type -2 */ { - int port_num; - int port_ix; + +#undef ERTS_OPEN_DRIVER_RET +#define ERTS_OPEN_DRIVER_RET(Prt, EType, ENo) \ + do { \ + if (error_type_ptr) \ + *error_type_ptr = (EType); \ + if (error_number_ptr) \ + *error_number_ptr = (ENo); \ + return (Prt); \ + } while (0) + ErlDrvData drv_data = 0; - Uint32 xstatus = 0; Port *port; int fpe_was_unmasked; - - if (error_number_ptr) - *error_number_ptr = 0; + int error_type, error_number; + int port_errno = 0; + erts_mtx_t *driver_lock = NULL; + int cprt_flgs = 0; ERTS_SMP_CHK_NO_PROC_LOCKS; - if ((port_num = get_free_port()) < 0) { - if (error_number_ptr) { - *error_number_ptr = SYSTEM_LIMIT; - } - return -3; - } - - port_ix = port_num & erts_port_tab_index_mask; - port = &erts_port[port_ix]; - port->id = make_internal_port(port_num); - - erts_smp_mtx_lock(&erts_driver_list_lock); + erts_smp_rwmtx_rlock(&erts_driver_list_lock); if (!driver) { for (driver = driver_list; driver; driver = driver->next) { if (sys_strcmp(driver->name, name) == 0) break; } if (!driver) { - erts_smp_mtx_unlock(&erts_driver_list_lock); - if (error_number_ptr) - *error_number_ptr = BADARG; - return -3; + erts_smp_rwmtx_runlock(&erts_driver_list_lock); + ERTS_OPEN_DRIVER_RET(NULL, -3, BADARG); } } if (driver == &spawn_driver) { @@ -597,59 +625,49 @@ erts_open_driver(erts_driver_t* driver, /* Pointer to driver. */ } if (driver == NULL || (driver != &spawn_driver && opts->exit_status)) { - erts_smp_mtx_unlock(&erts_driver_list_lock); - if (error_number_ptr) { - *error_number_ptr = BADARG; - } - /* Need to mark the port as free again */ - erts_smp_port_state_lock(port); - port->status = ERTS_PORT_SFLG_FREE; - ERTS_LC_ASSERT(erts_smp_atomic_read_nob(&port->refc) == 2); - erts_smp_atomic_set_nob(&port->refc, 0); - erts_smp_port_state_unlock(port); - return -3; + erts_smp_rwmtx_runlock(&erts_driver_list_lock); + ERTS_OPEN_DRIVER_RET(NULL, -3, BADARG); } - /* - * We'll set up the port before calling the start function, - * to allow message sending and setting timers in the start function. - */ - #ifdef ERTS_SMP - ASSERT(!port->lock); - port->lock = driver->lock; - if (!port->lock) { - port->lock = erts_alloc(ERTS_ALC_T_PORT_LOCK, - sizeof(erts_smp_mtx_t)); - erts_smp_mtx_init_x(port->lock, -#ifdef ERTS_ENABLE_LOCK_COUNT - (erts_lcnt_rt_options & ERTS_LCNT_OPT_PORTLOCK) ? "port_lock" : NULL, -#else - "port_lock", -#endif - port->id); - xstatus |= ERTS_PORT_SFLG_PORT_SPECIFIC_LOCK; - } + driver_lock = driver->lock; #endif if (driver->handle != NULL) { erts_ddll_increment_port_count(driver->handle); erts_ddll_reference_driver(driver->handle); } - erts_smp_mtx_unlock(&erts_driver_list_lock); + erts_smp_rwmtx_runlock(&erts_driver_list_lock); -#ifdef ERTS_SMP - erts_smp_mtx_lock(port->lock); -#endif + /* + * We'll set up the port before calling the start function, + * to allow message sending and setting timers in the start function. + */ + + if (opts->parallelism) + cprt_flgs |= ERTS_CREATE_PORT_FLAG_PARALLELISM; - setup_port(port, pid, driver, drv_data, name, xstatus); + port = create_port(name, driver, driver_lock, cprt_flgs, pid, &port_errno); + if (!port) { + if (driver->handle) { + erts_smp_rwmtx_rlock(&erts_driver_list_lock); + erts_ddll_decrement_port_count(driver->handle); + erts_smp_rwmtx_runlock(&erts_driver_list_lock); + erts_ddll_dereference_driver(driver->handle); + } + if (port_errno) + ERTS_OPEN_DRIVER_RET(NULL, -2, port_errno); + else + ERTS_OPEN_DRIVER_RET(NULL, -3, SYSTEM_LIMIT); + } if (IS_TRACED_FL(port, F_TRACE_PORTS)) { trace_port_open(port, pid, am_atom_put(port->name, strlen(port->name))); } - + + error_number = error_type = 0; if (driver->start) { if (IS_TRACED_FL(port, F_TRACE_SCHED_PORTS)) { trace_sched_ports_where(port, am_in, am_start); @@ -662,56 +680,63 @@ erts_open_driver(erts_driver_t* driver, /* Pointer to driver. */ } #endif fpe_was_unmasked = erts_block_fpe(); - drv_data = (*driver->start)((ErlDrvPort)(port_ix), - name, opts); + drv_data = (*driver->start)((ErlDrvPort) port, name, opts); + if (((SWord) drv_data) == -1) + error_type = -1; + else if (((SWord) drv_data) == -2) { + /* + * We need to save errno quickly after the + * call to the 'start' callback before + * something else modify it. + */ + error_type = -2; + error_number = errno; + } + else if (((SWord) drv_data) == -3) { + error_type = -3; + error_number = BADARG; + } + erts_unblock_fpe(fpe_was_unmasked); port->caller = NIL; if (IS_TRACED_FL(port, F_TRACE_SCHED_PORTS)) { trace_sched_ports_where(port, am_out, am_start); } - if (error_number_ptr && ((SWord) drv_data) == (SWord) -2) - *error_number_ptr = errno; #ifdef ERTS_SMP if (port->xports) - erts_smp_xports_unlock(port); + erts_port_handle_xports(port); ASSERT(!port->xports); #endif } - if (((SWord)drv_data) == -1 || - ((SWord)drv_data) == -2 || - ((SWord)drv_data) == -3) { - int res = (int) ((SWord) drv_data); - - if (res == -3 && error_number_ptr) { - *error_number_ptr = BADARG; - } - + if (error_type) { /* * Must clean up the port. */ #ifdef ERTS_SMP - erts_cancel_smp_ptimer(port->ptimer); + erts_cancel_smp_ptimer(port->common.u.alive.ptimer); #else - erts_cancel_timer(&(port->tm)); + erts_cancel_timer(&(port->common.u.alive.tm)); #endif stopq(port); - kill_port(port); if (port->linebuf != NULL) { erts_free(ERTS_ALC_T_LINEBUF, (void *) port->linebuf); port->linebuf = NULL; } if (driver->handle != NULL) { - erts_smp_mtx_lock(&erts_driver_list_lock); + erts_smp_rwmtx_rlock(&erts_driver_list_lock); erts_ddll_decrement_port_count(driver->handle); - erts_smp_mtx_unlock(&erts_driver_list_lock); + erts_smp_rwmtx_runlock(&erts_driver_list_lock); } + kill_port(port); erts_port_release(port); - return res; + ERTS_OPEN_DRIVER_RET(NULL, error_type, error_number); } - port->drv_data = (SWord) drv_data; - return port_ix; + port->drv_data = (UWord) drv_data; + ERTS_OPEN_DRIVER_RET(port, 0, 0); + +#undef ERTS_OPEN_DRIVER_RET } #ifdef ERTS_SMP @@ -736,102 +761,122 @@ driver_create_port(ErlDrvPort creator_port_ix, /* Creating port */ char* name, /* Driver name */ ErlDrvData drv_data) /* Driver data */ { + int cprt_flgs = 0; Port *creator_port; Port* port; erts_driver_t *driver; Process *rp; - int port_num; - Eterm port_id; - Uint32 xstatus = 0; + erts_mtx_t *driver_lock = NULL; ERTS_SMP_CHK_NO_PROC_LOCKS; - creator_port = erts_drvport2port(creator_port_ix); + /* Need to be called from a scheduler thread */ + if (!erts_get_scheduler_id()) + return ERTS_INVALID_ERL_DRV_PORT; + + creator_port = erts_drvport2port(creator_port_ix, NULL); if (!creator_port) - return (ErlDrvTermData) -1; + return ERTS_INVALID_ERL_DRV_PORT; + + rp = erts_proc_lookup(pid); + if (!rp) + return ERTS_INVALID_ERL_DRV_PORT; ERTS_SMP_LC_ASSERT(erts_lc_is_port_locked(creator_port)); driver = creator_port->drv_ptr; - erts_smp_mtx_lock(&erts_driver_list_lock); + erts_smp_rwmtx_rlock(&erts_driver_list_lock); if (!erts_ddll_driver_ok(driver->handle)) { - erts_smp_mtx_unlock(&erts_driver_list_lock); - return (ErlDrvTermData) -1; + erts_smp_rwmtx_runlock(&erts_driver_list_lock); + return ERTS_INVALID_ERL_DRV_PORT; } - rp = erts_pid2proc(NULL, 0, pid, ERTS_PROC_LOCK_LINK); - if (!rp) { - erts_smp_mtx_unlock(&erts_driver_list_lock); - return (ErlDrvTermData) -1; /* pid does not exist */ + if (driver->handle != NULL) { + erts_ddll_increment_port_count(driver->handle); + erts_ddll_reference_referenced_driver(driver->handle); + } + +#ifdef ERTS_SMP + driver_lock = driver->lock; +#endif + + erts_smp_rwmtx_runlock(&erts_driver_list_lock); + + /* Inherit parallelism flag from parent */ + if (ERTS_PTS_FLG_PARALLELISM & + erts_smp_atomic32_read_nob(&creator_port->sched.flags)) + cprt_flgs |= ERTS_CREATE_PORT_FLAG_PARALLELISM; + port = create_port(name, driver, driver_lock, cprt_flgs, pid, NULL); + if (!port) { + if (driver->handle) { + erts_smp_rwmtx_rlock(&erts_driver_list_lock); + erts_ddll_decrement_port_count(driver->handle); + erts_smp_rwmtx_runlock(&erts_driver_list_lock); + erts_ddll_dereference_driver(driver->handle); + } + return ERTS_INVALID_ERL_DRV_PORT; } - if ((port_num = get_free_port()) < 0) { - errno = SYSTEM_LIMIT; + ERTS_SMP_LC_ASSERT(erts_lc_is_port_locked(port)); + + erts_smp_proc_lock(rp, ERTS_PROC_LOCK_LINK); + if (ERTS_PROC_IS_EXITING(rp)) { erts_smp_proc_unlock(rp, ERTS_PROC_LOCK_LINK); - erts_smp_mtx_unlock(&erts_driver_list_lock); - return (ErlDrvTermData) -1; + if (driver->handle) { + erts_smp_rwmtx_rlock(&erts_driver_list_lock); + erts_ddll_decrement_port_count(driver->handle); + erts_smp_rwmtx_runlock(&erts_driver_list_lock); + } + kill_port(port); + erts_port_release(port); + return ERTS_INVALID_ERL_DRV_PORT; } - port_id = make_internal_port(port_num); - port = &erts_port[port_num & erts_port_tab_index_mask]; + erts_add_link(&ERTS_P_LINKS(port), LINK_PID, pid); + erts_add_link(&ERTS_P_LINKS(rp), LINK_PID, port->common.id); + erts_smp_proc_unlock(rp, ERTS_PROC_LOCK_LINK); #ifdef ERTS_SMP - ASSERT(!port->lock); - port->lock = driver->lock; - if (!port->lock) { + if (!driver_lock) { ErtsXPortsList *xplp = xports_list_alloc(); xplp->port = port; xplp->next = creator_port->xports; creator_port->xports = xplp; - port->lock = erts_alloc(ERTS_ALC_T_PORT_LOCK, - sizeof(erts_smp_mtx_t)); - erts_smp_mtx_init_locked_x(port->lock, -#ifdef ERTS_ENABLE_LOCK_COUNT - (erts_lcnt_rt_options & ERTS_LCNT_OPT_PORTLOCK) ? "port_lock" : NULL, -#else - "port_lock", -#endif - port_id); - xstatus |= ERTS_PORT_SFLG_PORT_SPECIFIC_LOCK; } - #endif - if (driver->handle != NULL) { - erts_ddll_increment_port_count(driver->handle); - erts_ddll_reference_referenced_driver(driver->handle); - } - erts_smp_mtx_unlock(&erts_driver_list_lock); - - ERTS_SMP_LC_ASSERT(erts_lc_is_port_locked(port)); + port->drv_data = (UWord) drv_data; - setup_port(port, pid, driver, drv_data, name, xstatus); - port->id = port_id; - - erts_add_link(&(port->nlinks), LINK_PID, pid); - erts_add_link(&(rp->nlinks), LINK_PID, port_id); - erts_smp_proc_unlock(rp, ERTS_PROC_LOCK_LINK); - return port_num & erts_port_tab_index_mask; + return (ErlDrvPort) port; } #ifdef ERTS_SMP -void -erts_smp_xports_unlock(Port *prt) +int erts_port_handle_xports(Port *prt) { + int reds = 0; ErtsXPortsList *xplp; ASSERT(prt); xplp = prt->xports; ASSERT(xplp); while (xplp) { + Port *rprt = xplp->port; ErtsXPortsList *free_xplp; - if (xplp->port->xports) - erts_smp_xports_unlock(xplp->port); - erts_port_release(xplp->port); + erts_aint32_t state; + if (rprt->xports) + reds += erts_port_handle_xports(rprt); + state = erts_atomic32_read_nob(&rprt->state); + if ((state & ERTS_PORT_SFLG_CLOSING) && erts_is_port_ioq_empty(rprt)) { + terminate_port(rprt); + reds += ERTS_PORT_REDS_TERMINATE; + } + erts_port_release(rprt); free_xplp = xplp; xplp = xplp->next; xports_list_free(free_xplp); + reds++; } prt->xports = NULL; + return reds; } #endif @@ -866,8 +911,8 @@ io_list_to_vec(Eterm obj, /* io-list */ DECLARE_ESTACK(s); Eterm* objp; char *buf = cbin->orig_bytes; - ErlDrvSizeT len = cbin->orig_size; - ErlDrvSizeT csize = 0; + Uint len = cbin->orig_size; + Uint csize = 0; int vlen = 0; char* cptr = buf; @@ -982,7 +1027,7 @@ io_list_to_vec(Eterm obj, /* io-list */ #define IO_LIST_VEC_COUNT(obj) \ do { \ - ErlDrvSizeT _size = binary_size(obj); \ + Uint _size = binary_size(obj); \ Eterm _real; \ ERTS_DECLARE_DUMMY(Uint _offset); \ int _bitoffs; \ @@ -1033,8 +1078,9 @@ do { \ */ static int -io_list_vec_len(Eterm obj, Uint* vsize, Uint* csize, - Uint* pvsize, Uint* pcsize, Uint* total_size) +io_list_vec_len(Eterm obj, int* vsize, Uint* csize, + Uint* pvsize, Uint* pcsize, + ErlDrvSizeT* total_size) { DECLARE_ESTACK(s); Eterm* objp; @@ -1045,7 +1091,7 @@ io_list_vec_len(Eterm obj, Uint* vsize, Uint* csize, Uint p_v_size = 0; Uint p_c_size = 0; Uint p_in_clist = 0; - Uint total; + Uint total; /* Uint due to halfword emulator */ goto L_jump_start; /* avoid a push */ @@ -1105,7 +1151,7 @@ io_list_vec_len(Eterm obj, Uint* vsize, Uint* csize, if (total < c_size) { goto L_overflow_error; } - *total_size = total; + *total_size = (ErlDrvSizeT) total; DESTROY_ESTACK(s); *vsize = v_size; @@ -1120,56 +1166,724 @@ io_list_vec_len(Eterm obj, Uint* vsize, Uint* csize, return 1; } -/* write data to a port */ -int erts_write_to_port(Eterm caller_id, Port *p, Eterm list) -{ - char *buf; - erts_driver_t *drv = p->drv_ptr; - Uint size; +typedef enum { + ERTS_TRY_IMM_DRV_CALL_OK, + ERTS_TRY_IMM_DRV_CALL_BUSY_LOCK, + ERTS_TRY_IMM_DRV_CALL_INVALID_PORT, + ERTS_TRY_IMM_DRV_CALL_INVALID_SCHED_FLAGS +} ErtsTryImmDrvCallResult; + +typedef struct { + Process *c_p; /* Currently executing process (unlocked) */ + Port *port; /* Port to operate on */ + Eterm port_op; /* port operation as an atom */ + erts_aint32_t state; /* in: invalid state; out: read state (if read) */ + erts_aint32_t sched_flags; /* in: invalid flags; out: read flags (if read) */ + int async; /* Asynchronous operation */ + int pre_chk_sched_flags; /* Check sched flags before lock? */ int fpe_was_unmasked; - - ERTS_SMP_LC_ASSERT(erts_lc_is_port_locked(p) || ERTS_IS_CRASH_DUMPING); +} ErtsTryImmDrvCallState; + +#define ERTS_INIT_TRY_IMM_DRV_CALL_STATE(C_P, PRT, SFLGS, PTS_FLGS, A, PRT_OP) \ + {(C_P), (PRT), (PRT_OP), (SFLGS), (PTS_FLGS), (A), 1, 0} + +/* + * Try doing an immediate driver callback call from a process. If + * this fail, the operation should be scheduled in the normal case... + * + */ +static ERTS_INLINE ErtsTryImmDrvCallResult +try_imm_drv_call(ErtsTryImmDrvCallState *sp) +{ + ErtsTryImmDrvCallResult res; + erts_aint32_t invalid_state, invalid_sched_flags; + Port *prt = sp->port; + Process *c_p = sp->c_p; + + ASSERT(is_atom(sp->port_op)); + + invalid_sched_flags = ERTS_PTS_FLGS_FORCE_SCHEDULE_OP; + invalid_sched_flags |= sp->sched_flags; + if (sp->async) + invalid_sched_flags |= ERTS_PTS_FLG_PARALLELISM; + + if (sp->pre_chk_sched_flags) { + sp->sched_flags = erts_smp_atomic32_read_nob(&prt->sched.flags); + if (sp->sched_flags & invalid_sched_flags) + return ERTS_TRY_IMM_DRV_CALL_INVALID_SCHED_FLAGS; + } + + if (erts_smp_port_trylock(prt) == EBUSY) + return ERTS_TRY_IMM_DRV_CALL_BUSY_LOCK; + + invalid_state = sp->state; + sp->state = erts_atomic32_read_nob(&prt->state); + if (sp->state & invalid_state) { + res = ERTS_TRY_IMM_DRV_CALL_INVALID_PORT; + goto locked_fail; + } + + sp->sched_flags = erts_smp_atomic32_read_nob(&prt->sched.flags); + if (sp->sched_flags & invalid_sched_flags) { + res = ERTS_TRY_IMM_DRV_CALL_INVALID_SCHED_FLAGS; + goto locked_fail; + } + + if (c_p) { + if (IS_TRACED_FL(c_p, F_TRACE_SCHED_PROCS)) + trace_virtual_sched(c_p, am_out); + if (erts_system_profile_flags.runnable_procs + && erts_system_profile_flags.exclusive) + profile_runnable_proc(c_p, am_inactive); + + erts_smp_proc_unlock(c_p, ERTS_PROC_LOCK_MAIN); + } + ERTS_SMP_CHK_NO_PROC_LOCKS; - p->caller = caller_id; - if (drv->outputv != NULL) { - Uint vsize; - Uint csize; - Uint pvsize; - Uint pcsize; - ErlDrvSizeT blimit; + if (IS_TRACED_FL(prt, F_TRACE_SCHED_PORTS)) + trace_sched_ports_where(prt, am_in, sp->port_op); + if (erts_system_profile_flags.runnable_ports + && !erts_port_is_scheduled(prt)) + profile_runnable_port(prt, am_active); + + sp->fpe_was_unmasked = erts_block_fpe(); + + return ERTS_TRY_IMM_DRV_CALL_OK; + +locked_fail: + erts_port_release(prt); + return res; +} + +static ERTS_INLINE void +finalize_imm_drv_call(ErtsTryImmDrvCallState *sp) +{ + Port *prt = sp->port; + Process *c_p = sp->c_p; + + erts_port_driver_callback_epilogue(prt, NULL); + + erts_unblock_fpe(sp->fpe_was_unmasked); + + if (IS_TRACED_FL(prt, F_TRACE_SCHED_PORTS)) + trace_sched_ports_where(prt, am_out, sp->port_op); + if (erts_system_profile_flags.runnable_ports + && !erts_port_is_scheduled(prt)) + profile_runnable_port(prt, am_inactive); + + erts_port_release(prt); + + if (c_p) { + erts_smp_proc_lock(c_p, ERTS_PROC_LOCK_MAIN); + + if (IS_TRACED_FL(c_p, F_TRACE_SCHED_PROCS)) + trace_virtual_sched(c_p, am_in); + if (erts_system_profile_flags.runnable_procs + && erts_system_profile_flags.exclusive) + profile_runnable_proc(c_p, am_active); + } +} + +/* + * force_imm_drv_call()/finalize_force_imm_drv_call() should *only* + * be used while crash dumping... + */ +static ErtsTryImmDrvCallResult +force_imm_drv_call(ErtsTryImmDrvCallState *sp) +{ + erts_aint32_t invalid_state; + Port *prt = sp->port; + + ASSERT(ERTS_IS_CRASH_DUMPING) + ASSERT(is_atom(sp->port_op)); + + invalid_state = sp->state; + sp->state = erts_atomic32_read_nob(&prt->state); + if (sp->state & invalid_state) + return ERTS_TRY_IMM_DRV_CALL_INVALID_PORT; + + sp->fpe_was_unmasked = erts_block_fpe(); + + return ERTS_TRY_IMM_DRV_CALL_OK; +} + +static void +finalize_force_imm_drv_call(ErtsTryImmDrvCallState *sp) +{ + erts_unblock_fpe(sp->fpe_was_unmasked); +} + +#define ERTS_QUEUE_PORT_SCHED_OP_REPLY_SIZE (REF_THING_SIZE + 3) + +static ERTS_INLINE void +queue_port_sched_op_reply(Process *rp, + ErtsProcLocks *rp_locksp, + Eterm *hp_start, + Eterm *hp, + Uint h_size, + ErlHeapFragment* bp, + Uint32 *ref_num, + Eterm msg) +{ + Eterm ref = make_internal_ref(hp); + write_ref_thing(hp, ref_num[0], ref_num[1], ref_num[2]); + hp += REF_THING_SIZE; + + msg = TUPLE2(hp, ref, msg); + hp += 3; + + if (!bp) { + HRelease(rp, hp_start + h_size, hp); + } + else { + Uint used_h_size = hp - hp_start; + ASSERT(h_size >= used_h_size); + if (h_size > used_h_size) + bp = erts_resize_message_buffer(bp, used_h_size, &msg, 1); + } + + erts_queue_message(rp, + rp_locksp, + bp, + msg, + NIL +#ifdef USE_VM_PROBES + , NIL +#endif + ); +} + +static void +port_sched_op_reply(Eterm to, Uint32 *ref_num, Eterm msg) +{ + Process *rp = erts_proc_lookup_raw(to); + if (rp) { + ErlOffHeap *ohp; + ErlHeapFragment* bp; + Eterm msg_copy; + Uint hsz, msg_sz; + Eterm *hp, *hp_start; + ErtsProcLocks rp_locks = 0; + + hsz = ERTS_QUEUE_PORT_SCHED_OP_REPLY_SIZE; + if (is_immed(msg)) + msg_sz = 0; + else { + msg_sz = size_object(msg); + hsz += msg_sz; + } + + hp_start = hp = erts_alloc_message_heap(hsz, + &bp, + &ohp, + rp, + &rp_locks); + if (is_immed(msg)) + msg_copy = msg; + else + msg_copy = copy_struct(msg, msg_sz, &hp, ohp); + + queue_port_sched_op_reply(rp, + &rp_locks, + hp_start, + hp, + hsz, + bp, + ref_num, + msg_copy); + + if (rp_locks) + erts_smp_proc_unlock(rp, rp_locks); + } +} + + +ErtsPortOpResult +erts_schedule_proc2port_signal(Process *c_p, + Port *prt, + Eterm caller, + Eterm *refp, + ErtsProc2PortSigData *sigdp, + int task_flags, + ErtsProc2PortSigCallback callback) +{ + int sched_res; + if (!refp) { + if (c_p) + erts_smp_proc_unlock(c_p, ERTS_PROC_LOCK_MAIN); + } + else { + ASSERT(c_p); + sigdp->flags |= ERTS_P2P_SIG_DATA_FLG_REPLY; + erts_make_ref_in_array(sigdp->ref); + *refp = erts_proc_store_ref(c_p, sigdp->ref); + + /* + * Caller needs to wait for a message containing + * the ref that we just created. No such message + * can exist in callers message queue at this time. + * We therefore move the save pointer of the + * callers message queue to the end of the queue. + * + * NOTE: It is of vital importance that the caller + * immediately do a receive unconditionaly + * waiting for the message with the reference; + * otherwise, next receive will *not* work + * as expected! + */ + erts_smp_proc_lock(c_p, ERTS_PROC_LOCKS_MSG_RECEIVE); + + if (ERTS_PROC_PENDING_EXIT(c_p)) { + /* need to exit caller instead */ + erts_smp_proc_unlock(c_p, ERTS_PROC_LOCKS_MSG_RECEIVE); + KILL_CATCHES(c_p); + c_p->freason = EXC_EXIT; + return ERTS_PORT_OP_CALLER_EXIT; + } + + ERTS_SMP_MSGQ_MV_INQ2PRIVQ(c_p); + c_p->msg.save = c_p->msg.last; + + erts_smp_proc_unlock(c_p, + (ERTS_PROC_LOCK_MAIN + | ERTS_PROC_LOCKS_MSG_RECEIVE)); + } + + + sigdp->caller = caller; + + /* Schedule port close call for later execution... */ + sched_res = erts_port_task_schedule(prt->common.id, + NULL, + ERTS_PORT_TASK_PROC_SIG, + sigdp, + callback, + task_flags); + + if (c_p) + erts_smp_proc_lock(c_p, ERTS_PROC_LOCK_MAIN); + + if (sched_res != 0) { + if (refp) + *refp = NIL; + return ERTS_PORT_OP_DROPPED; + } + return ERTS_PORT_OP_SCHEDULED; +} + +static ERTS_INLINE void +send_badsig(Port *prt) +{ + ErtsProcLocks rp_locks = ERTS_PROC_LOCKS_XSIG_SEND; + Process* rp; + Eterm connected = ERTS_PORT_GET_CONNECTED(prt); + + ERTS_SMP_CHK_NO_PROC_LOCKS; + ERTS_LC_ASSERT(erts_get_scheduler_id()); + + ASSERT(is_internal_pid(connected)); + + rp = erts_proc_lookup_raw(connected); + if (rp) { + erts_smp_proc_lock(rp, rp_locks); + if (!ERTS_PROC_IS_EXITING(rp)) + (void) erts_send_exit_signal(NULL, + prt->common.id, + rp, + &rp_locks, + am_badsig, + NIL, + NULL, + 0); + if (rp_locks) + erts_smp_proc_unlock(rp, rp_locks); + } +} + +static void +badsig_received(int bang_op, + Port *prt, + erts_aint32_t state, + int bad_output_value) +{ + /* + * if (bang_op) + * we are part of a "Prt ! Something" operation + * else + * we are part of a call to a port BIF + * behave accordingly... + */ + if (!(state & ERTS_PORT_SFLGS_INVALID_LOOKUP)) { + if (bad_output_value) { + erts_dsprintf_buf_t *dsbufp = erts_create_logger_dsbuf(); + erts_dsprintf(dsbufp, "Bad value on output port '%s'\n", prt->name); + erts_send_error_to_logger_nogl(dsbufp); + } + if (bang_op) + send_badsig(prt); + } +} + +static int +port_badsig(Port *prt, erts_aint32_t state, int op, ErtsProc2PortSigData *sigdp) +{ + if (op == ERTS_PROC2PORT_SIG_EXEC) + badsig_received(sigdp->flags & ERTS_P2P_SIG_DATA_FLG_BANG_OP, + prt, + state, + sigdp->flags & ERTS_P2P_SIG_DATA_FLG_BAD_OUTPUT); + if (sigdp->flags & ERTS_P2P_SIG_DATA_FLG_REPLY) + port_sched_op_reply(sigdp->caller, sigdp->ref, am_badarg); + return ERTS_PORT_REDS_BADSIG; +} + + +/* + * bad_port_signal() will + * - preserve signal order of signals. + * - send a 'badsig' exit signal to connected process if 'from' is an + * internal pid and the port is alive when the bad signal reaches + * it. + */ +static ErtsPortOpResult +bad_port_signal(Process *c_p, + int flags, + Port *prt, + Eterm from, + Eterm *refp, + Eterm port_op) +{ + ErtsProc2PortSigData *sigdp; + ErtsTryImmDrvCallResult try_call_res; + ErtsTryImmDrvCallState try_call_state + = ERTS_INIT_TRY_IMM_DRV_CALL_STATE( + c_p, + prt, + ERTS_PORT_SFLGS_INVALID_LOOKUP, + 0, + !refp, + port_op); + + try_call_res = try_imm_drv_call(&try_call_state); + switch (try_call_res) { + case ERTS_TRY_IMM_DRV_CALL_OK: + badsig_received(flags & ERTS_PORT_SIG_FLG_BANG_OP, + prt, + try_call_state.state, + flags & ERTS_PORT_SIG_FLG_BAD_OUTPUT); + finalize_imm_drv_call(&try_call_state); + return ERTS_PORT_OP_BADARG; + case ERTS_TRY_IMM_DRV_CALL_INVALID_PORT: + return ERTS_PORT_OP_DROPPED; + case ERTS_TRY_IMM_DRV_CALL_INVALID_SCHED_FLAGS: + case ERTS_TRY_IMM_DRV_CALL_BUSY_LOCK: + /* Schedule badsig() call instead... */ + break; + } + + sigdp = erts_port_task_alloc_p2p_sig_data(); + sigdp->flags = (flags & ~ERTS_P2P_SIG_TYPE_MASK) | ERTS_P2P_SIG_TYPE_BAD; + + return erts_schedule_proc2port_signal(c_p, + prt, + c_p->common.id, + refp, + sigdp, + 0, + port_badsig); +} + + +/* + * Driver outputv() callback + */ + +static ERTS_INLINE void +call_driver_outputv(int bang_op, + Eterm caller, + Eterm from, + Port *prt, + erts_driver_t *drv, + ErlIOVec *evp) +{ + /* + * if (bang_op) + * we are part of a "Prt ! {From, {command, Data}}" operation + * else + * we are part of a call to port_command/[2,3] + * behave accordingly... + */ + if (bang_op && from != ERTS_PORT_GET_CONNECTED(prt)) + send_badsig(prt); + else { + ErlDrvSizeT size = evp->size; + + ERTS_SMP_LC_ASSERT(erts_lc_is_port_locked(prt) + || ERTS_IS_CRASH_DUMPING); + +#ifdef USE_VM_PROBES + if (DTRACE_ENABLED(driver_outputv)) { + DTRACE_FORMAT_COMMON_PID_AND_PORT(caller, prt); + DTRACE4(driver_outputv, process_str, port_str, prt->name, size); + } +#endif + + prt->caller = caller; + (*drv->outputv)((ErlDrvData) prt->drv_data, evp); + prt->caller = NIL; + + prt->bytes_out += size; + erts_smp_atomic_add_nob(&erts_bytes_out, size); + } +} + +static ERTS_INLINE void +cleanup_scheduled_outputv(ErlIOVec *ev, ErlDrvBinary *cbinp) +{ + int i; + /* Need to free all binaries */ + for (i = 1; i < ev->vsize; i++) + if (ev->binv[i]) + driver_free_binary(ev->binv[i]); + if (cbinp) + driver_free_binary(cbinp); + erts_free(ERTS_ALC_T_DRV_CMD_DATA, ev); +} + +static int +port_sig_outputv(Port *prt, erts_aint32_t state, int op, ErtsProc2PortSigData *sigdp) +{ + Eterm reply; + + switch (op) { + case ERTS_PROC2PORT_SIG_EXEC: + /* Execution of a scheduled outputv() call */ + + ERTS_SMP_LC_ASSERT(erts_lc_is_port_locked(prt)); + + if (state & ERTS_PORT_SFLGS_INVALID_LOOKUP) + reply = am_badarg; + else { + call_driver_outputv(sigdp->flags & ERTS_P2P_SIG_DATA_FLG_BANG_OP, + sigdp->caller, + sigdp->u.outputv.from, + prt, + prt->drv_ptr, + sigdp->u.outputv.evp); + reply = am_true; + } + break; + case ERTS_PROC2PORT_SIG_ABORT_NOSUSPEND: + reply = am_false; + break; + default: + reply = am_badarg; + break; + } + + if (sigdp->flags & ERTS_P2P_SIG_DATA_FLG_REPLY) + port_sched_op_reply(sigdp->caller, sigdp->ref, reply); + + cleanup_scheduled_outputv(sigdp->u.outputv.evp, + sigdp->u.outputv.cbinp); + + return ERTS_PORT_REDS_CMD_OUTPUTV; +} + +/* + * Driver output() callback + */ + +static ERTS_INLINE void +call_driver_output(int bang_op, + Eterm caller, + Eterm from, + Port *prt, + erts_driver_t *drv, + char *bufp, + ErlDrvSizeT size) +{ + /* + * if (bang_op) + * we are part of a "Prt ! {From, {command, Data}}" operation + * else + * we are part of a call to port_command/[2,3] + * behave accordingly... + */ + if (bang_op && from != ERTS_PORT_GET_CONNECTED(prt)) + send_badsig(prt); + else { + + ERTS_SMP_LC_ASSERT(erts_lc_is_port_locked(prt) + || ERTS_IS_CRASH_DUMPING); + +#ifdef USE_VM_PROBES + if (DTRACE_ENABLED(driver_output)) { + DTRACE_FORMAT_COMMON_PID_AND_PORT(caller, prt); + DTRACE4(driver_output, process_str, port_str, prt->name, size); + } +#endif + + prt->caller = caller; + (*drv->output)((ErlDrvData) prt->drv_data, bufp, size); + prt->caller = NIL; + + prt->bytes_out += size; + erts_smp_atomic_add_nob(&erts_bytes_out, size); + } +} + +static ERTS_INLINE void +cleanup_scheduled_output(char *bufp) +{ + erts_free(ERTS_ALC_T_DRV_CMD_DATA, bufp); +} + +static int +port_sig_output(Port *prt, erts_aint32_t state, int op, ErtsProc2PortSigData *sigdp) +{ + Eterm reply; + + switch (op) { + case ERTS_PROC2PORT_SIG_EXEC: + /* Execution of a scheduled output() call */ + + ERTS_SMP_LC_ASSERT(erts_lc_is_port_locked(prt)); + + if (state & ERTS_PORT_SFLGS_INVALID_LOOKUP) + reply = am_badarg; + else { + call_driver_output(sigdp->flags & ERTS_P2P_SIG_DATA_FLG_BANG_OP, + sigdp->caller, + sigdp->u.output.from, + prt, + prt->drv_ptr, + sigdp->u.output.bufp, + sigdp->u.output.size); + reply = am_true; + } + break; + case ERTS_PROC2PORT_SIG_ABORT_NOSUSPEND: + reply = am_false; + break; + default: + reply = am_badarg; + break; + } + + if (sigdp->flags & ERTS_P2P_SIG_DATA_FLG_REPLY) + port_sched_op_reply(sigdp->caller, sigdp->ref, reply); + + cleanup_scheduled_output(sigdp->u.output.bufp); + + return ERTS_PORT_REDS_CMD_OUTPUT; +} + +ErtsPortOpResult +erts_port_output(Process *c_p, + int flags, + Port *prt, + Eterm from, + Eterm list, + Eterm *refp) +{ + ErtsPortOpResult res; + ErtsProc2PortSigData *sigdp; + erts_driver_t *drv = prt->drv_ptr; + size_t size; + int try_call; + erts_aint32_t sched_flags, busy_flgs, invalid_flags; + int task_flags; + ErtsProc2PortSigCallback port_sig_callback; + ErlDrvBinary *cbin = NULL; + ErlIOVec *evp = NULL; + char *buf = NULL; + int force_immediate_call = (flags & ERTS_PORT_SIG_FLG_FORCE_IMM_CALL); + + ASSERT((flags & ~(ERTS_PORT_SIG_FLG_BANG_OP + | ERTS_PORT_SIG_FLG_NOSUSPEND + | ERTS_PORT_SIG_FLG_FORCE + | ERTS_PORT_SIG_FLG_FORCE_IMM_CALL)) == 0); + + busy_flgs = ((flags & ERTS_PORT_SIG_FLG_FORCE) + ? ((erts_aint32_t) 0) + : ERTS_PTS_FLGS_BUSY); + invalid_flags = busy_flgs; + if (!refp) + invalid_flags |= ERTS_PTS_FLG_PARALLELISM; + + /* + * Assumes caller have checked that port is valid... + */ + + sched_flags = erts_smp_atomic32_read_nob(&prt->sched.flags); + if (sched_flags & (busy_flgs|ERTS_PTS_FLG_EXIT)) + return ((sched_flags & ERTS_PTS_FLG_EXIT) + ? ERTS_PORT_OP_DROPPED + : ERTS_PORT_OP_BUSY); + + try_call = (force_immediate_call /* crash dumping */ + || !(sched_flags & (invalid_flags + | ERTS_PTS_FLGS_FORCE_SCHEDULE_OP))); + +#ifdef USE_VM_PROBES + if(DTRACE_ENABLED(port_command)) { + DTRACE_FORMAT_COMMON_PID_AND_PORT(c_p ? c_p->common.id : ERTS_INVALID_PID, prt); + DTRACE4(port_command, process_str, port_str, prt->name, "command"); + } +#endif + + if (drv->outputv) { + ErlIOVec ev; SysIOVec iv[SMALL_WRITE_VEC]; ErlDrvBinary* bv[SMALL_WRITE_VEC]; SysIOVec* ivp; ErlDrvBinary** bvp; - ErlDrvBinary* cbin; - ErlIOVec ev; + int vsize; + Uint csize; + Uint pvsize; + Uint pcsize; + Uint blimit; + size_t iov_offset, binv_offset, alloc_size; - if (io_list_vec_len(list, &vsize, &csize, - &pvsize, &pcsize, &size)) { + if (io_list_vec_len(list, &vsize, &csize, &pvsize, &pcsize, &size)) goto bad_value; + + iov_offset = ERTS_ALC_DATA_ALIGN_SIZE(sizeof(ErlIOVec)); + binv_offset = iov_offset; + binv_offset += ERTS_ALC_DATA_ALIGN_SIZE((vsize+1)*sizeof(SysIOVec)); + alloc_size = binv_offset; + alloc_size += (vsize+1)*sizeof(ErlDrvBinary *); + + if (try_call && vsize < SMALL_WRITE_VEC) { + ivp = ev.iov = iv; + bvp = ev.binv = bv; + evp = &ev; + } + else { + char *ptr = erts_alloc((try_call + ? ERTS_ALC_T_TMP + : ERTS_ALC_T_DRV_CMD_DATA), alloc_size); + + evp = (ErlIOVec *) ptr; + ivp = evp->iov = (SysIOVec *) (ptr + iov_offset); + bvp = evp->binv = (ErlDrvBinary **) (ptr + binv_offset); } + /* To pack or not to pack (small binaries) ...? */ - vsize++; - if (vsize <= SMALL_WRITE_VEC) { + if (vsize < SMALL_WRITE_VEC) { /* Do NOT pack */ blimit = 0; - } else { + } + else { /* Do pack */ vsize = pvsize + 1; csize = pcsize; blimit = ERL_SMALL_IO_BIN_LIMIT; } /* Use vsize and csize from now on */ - if (vsize <= SMALL_WRITE_VEC) { - ivp = iv; - bvp = bv; - } else { - ivp = (SysIOVec *) erts_alloc(ERTS_ALC_T_TMP, - vsize * sizeof(SysIOVec)); - bvp = (ErlDrvBinary**) erts_alloc(ERTS_ALC_T_TMP, - vsize * sizeof(ErlDrvBinary*)); - } + cbin = driver_alloc_binary(csize); if (!cbin) erts_alloc_enomem(ERTS_ALC_T_DRV_BINARY, ERTS_SIZEOF_Binary(csize)); @@ -1178,210 +1892,761 @@ int erts_write_to_port(Eterm caller_id, Port *p, Eterm list) ivp[0].iov_base = NULL; ivp[0].iov_len = 0; bvp[0] = NULL; - ev.vsize = io_list_to_vec(list, ivp+1, bvp+1, cbin, blimit); - if (ev.vsize < 0) { - if (ivp != iv) { - erts_free(ERTS_ALC_T_TMP, (void *) ivp); - } - if (bvp != bv) { - erts_free(ERTS_ALC_T_TMP, (void *) bvp); - } + evp->vsize = io_list_to_vec(list, ivp+1, bvp+1, cbin, blimit); + if (evp->vsize < 0) { + if (evp != &ev) + erts_free(try_call ? ERTS_ALC_T_TMP : ERTS_ALC_T_DRV_CMD_DATA, + evp); driver_free_binary(cbin); goto bad_value; } - ev.vsize++; #if 0 /* This assertion may say something useful, but it can be falsified during the emulator test suites. */ - ASSERT(ev.vsize == vsize); + ASSERT(evp->vsize == vsize); #endif - ev.size = size; /* total size */ - ev.iov = ivp; - ev.binv = bvp; -#ifdef USE_VM_PROBES - if (DTRACE_ENABLED(driver_outputv)) { - DTRACE_FORMAT_COMMON_PID_AND_PORT(caller_id, p) - DTRACE4(driver_outputv, process_str, port_str, p->name, size); - } -#endif - fpe_was_unmasked = erts_block_fpe(); - (*drv->outputv)((ErlDrvData)p->drv_data, &ev); - erts_unblock_fpe(fpe_was_unmasked); - if (ivp != iv) { - erts_free(ERTS_ALC_T_TMP, (void *) ivp); - } - if (bvp != bv) { - erts_free(ERTS_ALC_T_TMP, (void *) bvp); + evp->vsize++; + evp->size = size; /* total size */ + + if (!try_call) { + int i; + /* Need to increase refc on all binaries */ + for (i = 1; i < evp->vsize; i++) + if (bvp[i]) + driver_binary_inc_refc(bvp[i]); } - driver_free_binary(cbin); - } else { - int r; - - /* Try with an 8KB buffer first (will often be enough I guess). */ - size = 8*1024; - /* See below why the extra byte is added. */ - buf = erts_alloc(ERTS_ALC_T_TMP, size+1); - r = io_list_to_buf(list, buf, size); + else { + int i; + ErlIOVec *new_evp; + ErtsTryImmDrvCallResult try_call_res; + ErtsTryImmDrvCallState try_call_state + = ERTS_INIT_TRY_IMM_DRV_CALL_STATE( + c_p, + prt, + ERTS_PORT_SFLGS_INVALID_LOOKUP, + invalid_flags, + !refp, + am_command); + + try_call_state.pre_chk_sched_flags = 0; /* already checked */ + if (force_immediate_call) + try_call_res = force_imm_drv_call(&try_call_state); + else + try_call_res = try_imm_drv_call(&try_call_state); + switch (try_call_res) { + case ERTS_TRY_IMM_DRV_CALL_OK: + call_driver_outputv(flags & ERTS_PORT_SIG_FLG_BANG_OP, + c_p ? c_p->common.id : ERTS_INVALID_PID, + from, + prt, + drv, + evp); + if (force_immediate_call) + finalize_force_imm_drv_call(&try_call_state); + else + finalize_imm_drv_call(&try_call_state); + /* Fall through... */ + case ERTS_TRY_IMM_DRV_CALL_INVALID_PORT: + driver_free_binary(cbin); + if (evp != &ev) + erts_free(ERTS_ALC_T_TMP, evp); + if (try_call_res == ERTS_TRY_IMM_DRV_CALL_OK) + return ERTS_PORT_OP_DONE; + else + return ERTS_PORT_OP_DROPPED; + case ERTS_TRY_IMM_DRV_CALL_INVALID_SCHED_FLAGS: + sched_flags = try_call_state.sched_flags; + case ERTS_TRY_IMM_DRV_CALL_BUSY_LOCK: + /* Schedule outputv() call instead... */ + break; + } -#ifdef USE_VM_PROBES - if(DTRACE_ENABLED(port_command)) { - DTRACE_FORMAT_COMMON_PID_AND_PORT(caller_id, p) - DTRACE4(port_command, process_str, port_str, p->name, "command"); - } + /* Need to increase refc on all binaries */ + for (i = 1; i < evp->vsize; i++) + if (bvp[i]) + driver_binary_inc_refc(bvp[i]); + + new_evp = erts_alloc(ERTS_ALC_T_DRV_CMD_DATA, alloc_size); + + if (evp != &ev) { + sys_memcpy((void *) new_evp, (void *) evp, alloc_size); + new_evp->iov = (SysIOVec *) (((char *) new_evp) + + iov_offset); + bvp = new_evp->binv = (ErlDrvBinary **) (((char *) new_evp) + + binv_offset); + +#ifdef DEBUG + ASSERT(new_evp->vsize == evp->vsize); + ASSERT(new_evp->size == evp->size); + for (i = 0; i < evp->vsize; i++) { + ASSERT(new_evp->iov[i].iov_len == evp->iov[i].iov_len); + ASSERT(new_evp->iov[i].iov_base == evp->iov[i].iov_base); + ASSERT(new_evp->binv[i] == evp->binv[i]); + } #endif - if (r >= 0) { - size -= r; -#ifdef USE_VM_PROBES - if (DTRACE_ENABLED(driver_output)) { - DTRACE_FORMAT_COMMON_PID_AND_PORT(caller_id, p) - DTRACE4(driver_output, process_str, port_str, p->name, size); - } + erts_free(ERTS_ALC_T_TMP, evp); + } + else { /* from stack allocated structure; offsets may differ */ + + sys_memcpy((void *) new_evp, (void *) evp, sizeof(ErlIOVec)); + new_evp->iov = (SysIOVec *) (((char *) new_evp) + + iov_offset); + sys_memcpy((void *) new_evp->iov, + (void *) evp->iov, + evp->vsize * sizeof(SysIOVec)); + new_evp->binv = (ErlDrvBinary **) (((char *) new_evp) + + binv_offset); + sys_memcpy((void *) new_evp->binv, + (void *) evp->binv, + evp->vsize * sizeof(ErlDrvBinary *)); + +#ifdef DEBUG + ASSERT(new_evp->vsize == evp->vsize); + ASSERT(new_evp->size == evp->size); + for (i = 0; i < evp->vsize; i++) { + ASSERT(new_evp->iov[i].iov_len == evp->iov[i].iov_len); + ASSERT(new_evp->iov[i].iov_base == evp->iov[i].iov_base); + ASSERT(new_evp->binv[i] == evp->binv[i]); + } #endif - fpe_was_unmasked = erts_block_fpe(); - (*drv->output)((ErlDrvData)p->drv_data, buf, size); - erts_unblock_fpe(fpe_was_unmasked); - erts_free(ERTS_ALC_T_TMP, buf); + + } + + evp = new_evp; } - else if (r == -2) { - erts_free(ERTS_ALC_T_TMP, buf); - goto bad_value; + + sigdp = erts_port_task_alloc_p2p_sig_data(); + sigdp->flags = ERTS_P2P_SIG_TYPE_OUTPUTV; + sigdp->u.outputv.from = from; + sigdp->u.outputv.evp = evp; + sigdp->u.outputv.cbinp = cbin; + port_sig_callback = port_sig_outputv; + } + else { + ErlDrvSizeT r; + + /* + * Apperently there exist code that write 1 byte to + * much in buffer. Where it resides I don't know, but + * we can live with one byte extra allocated... + */ + + if (!try_call) { + if (erts_iolist_size(list, &size)) + goto bad_value; + + buf = erts_alloc(ERTS_ALC_T_DRV_CMD_DATA, size + 1); + + r = erts_iolist_to_buf(list, buf, size); + ASSERT(ERTS_IOLIST_TO_BUF_SUCCEEDED(r)); } else { - ASSERT(r == -1); /* Overflow */ - erts_free(ERTS_ALC_T_TMP, buf); - if (erts_iolist_size(list, &size)) { - goto bad_value; + char *new_buf; + ErtsTryImmDrvCallResult try_call_res; + ErtsTryImmDrvCallState try_call_state + = ERTS_INIT_TRY_IMM_DRV_CALL_STATE( + c_p, + prt, + ERTS_PORT_SFLGS_INVALID_LOOKUP, + invalid_flags, + !refp, + am_command); + + /* Try with an 8KB buffer first (will often be enough I guess). */ + size = 8*1024; + + buf = erts_alloc(ERTS_ALC_T_TMP, size + 1); + r = erts_iolist_to_buf(list, buf, size); + + if (ERTS_IOLIST_TO_BUF_SUCCEEDED(r)) { + ASSERT(r <= size); + size -= r; + } + else { + erts_free(ERTS_ALC_T_TMP, buf); + if (r == ERTS_IOLIST_TO_BUF_TYPE_ERROR) + goto bad_value; + ASSERT(r == ERTS_IOLIST_TO_BUF_OVERFLOW); + if (erts_iolist_size(list, &size)) + goto bad_value; + buf = erts_alloc(ERTS_ALC_T_TMP, size + 1); + r = erts_iolist_to_buf(list, buf, size); + ASSERT(ERTS_IOLIST_TO_BUF_SUCCEEDED(r)); } - /* - * I know drivers that pad space with '\0' this is clearly - * incorrect but I don't feel like fixing them now, insted - * add ONE extra byte. - */ - buf = erts_alloc(ERTS_ALC_T_TMP, size+1); - r = io_list_to_buf(list, buf, size); -#ifdef USE_VM_PROBES - if (DTRACE_ENABLED(driver_output)) { - DTRACE_FORMAT_COMMON_PID_AND_PORT(caller_id, p) - DTRACE4(driver_output, process_str, port_str, p->name, size); - } -#endif - fpe_was_unmasked = erts_block_fpe(); - (*drv->output)((ErlDrvData)p->drv_data, buf, size); - erts_unblock_fpe(fpe_was_unmasked); + try_call_state.pre_chk_sched_flags = 0; /* already checked */ + if (force_immediate_call) + try_call_res = force_imm_drv_call(&try_call_state); + else + try_call_res = try_imm_drv_call(&try_call_state); + switch (try_call_res) { + case ERTS_TRY_IMM_DRV_CALL_OK: + call_driver_output(flags & ERTS_PORT_SIG_FLG_BANG_OP, + c_p ? c_p->common.id : ERTS_INVALID_PID, + from, + prt, + drv, + buf, + size); + if (force_immediate_call) + finalize_force_imm_drv_call(&try_call_state); + else + finalize_imm_drv_call(&try_call_state); + /* Fall through... */ + case ERTS_TRY_IMM_DRV_CALL_INVALID_PORT: + erts_free(ERTS_ALC_T_TMP, buf); + if (try_call_res == ERTS_TRY_IMM_DRV_CALL_OK) + return ERTS_PORT_OP_DONE; + else + return ERTS_PORT_OP_DROPPED; + case ERTS_TRY_IMM_DRV_CALL_INVALID_SCHED_FLAGS: + sched_flags = try_call_state.sched_flags; + case ERTS_TRY_IMM_DRV_CALL_BUSY_LOCK: + /* Schedule outputv() call instead... */ + break; + } + + new_buf = erts_alloc(ERTS_ALC_T_DRV_CMD_DATA, size + 1); + sys_memcpy(new_buf, buf, size); erts_free(ERTS_ALC_T_TMP, buf); + buf = new_buf; } + + sigdp = erts_port_task_alloc_p2p_sig_data(); + sigdp->flags = ERTS_P2P_SIG_TYPE_OUTPUT; + sigdp->u.output.from = from; + sigdp->u.output.bufp = buf; + sigdp->u.output.size = size; + port_sig_callback = port_sig_output; + } + + task_flags = ERTS_PT_FLG_WAIT_BUSY; + sigdp->flags |= flags; + if (flags & (ERTS_P2P_SIG_DATA_FLG_FORCE|ERTS_P2P_SIG_DATA_FLG_NOSUSPEND)) { + task_flags = 0; + if (flags & ERTS_P2P_SIG_DATA_FLG_FORCE) + sigdp->flags &= ~ERTS_P2P_SIG_DATA_FLG_NOSUSPEND; + else if (flags & ERTS_P2P_SIG_DATA_FLG_NOSUSPEND) + task_flags = ERTS_PT_FLG_NOSUSPEND; + } + + res = erts_schedule_proc2port_signal(c_p, + prt, + c_p ? c_p->common.id : ERTS_INVALID_PID, + refp, + sigdp, + task_flags, + port_sig_callback); + + if (res != ERTS_PORT_OP_SCHEDULED) { + if (drv->outputv) + cleanup_scheduled_outputv(evp, cbin); + else + cleanup_scheduled_output(buf); + return res; } - p->bytes_out += size; - erts_smp_atomic_add_nob(&erts_bytes_out, size); -#ifdef ERTS_SMP - if (p->xports) - erts_smp_xports_unlock(p); - ASSERT(!p->xports); + if (!(sched_flags & ERTS_PTS_FLG_EXIT) && (sched_flags & busy_flgs)) + return ERTS_PORT_OP_BUSY_SCHEDULED; + + return res; + +bad_value: + + flags |= ERTS_PORT_SIG_FLG_BAD_OUTPUT; + return bad_port_signal(c_p, flags, prt, from, refp, am_command); +} + +static ERTS_INLINE ErtsPortOpResult +call_deliver_port_exit(int bang_op, + Eterm from, + Port *prt, + erts_aint32_t state, + Eterm reason, + int broken_link) +{ + /* + * if (bang_op) + * we are part of a "Prt ! {From, close}" operation + * else + * we are part of a call to port_close(Port) + * behave accordingly... + */ + + if (state & ERTS_PORT_SFLGS_INVALID_LOOKUP) + return ERTS_PORT_OP_DROPPED; + + if (bang_op && from != ERTS_PORT_GET_CONNECTED(prt)) { + send_badsig(prt); + return ERTS_PORT_OP_DROPPED; + } + + if (broken_link) { + ErtsLink *lnk = erts_remove_link(&ERTS_P_LINKS(prt), from); + if (lnk) + erts_destroy_link(lnk); + else + return ERTS_PORT_OP_DROPPED; + } + + if (!erts_deliver_port_exit(prt, from, reason, bang_op)) + return ERTS_PORT_OP_DROPPED; + +#ifdef USE_VM_PROBES + if(DTRACE_ENABLED(port_command) && bang_op) { + DTRACE_FORMAT_COMMON_PID_AND_PORT(from, prt); + DTRACE4(port_command, process_str, port_str, prt->name, "close"); + } #endif - p->caller = NIL; - return 0; - bad_value: - p->caller = NIL; - { - erts_dsprintf_buf_t *dsbufp = erts_create_logger_dsbuf(); - erts_dsprintf(dsbufp, "Bad value on output port '%s'\n", p->name); - erts_send_error_to_logger_nogl(dsbufp); - return 1; + return ERTS_PORT_OP_DONE; +} + +static int +port_sig_exit(Port *prt, + erts_aint32_t state, + int op, + ErtsProc2PortSigData *sigdp) +{ + Eterm msg = am_badarg; + if (op == ERTS_PROC2PORT_SIG_EXEC) { + ErtsPortOpResult res; + int bang_op = sigdp->flags & ERTS_P2P_SIG_DATA_FLG_BANG_OP; + int broken_link = sigdp->flags & ERTS_P2P_SIG_DATA_FLG_BROKEN_LINK; + res = call_deliver_port_exit(bang_op, + sigdp->u.exit.from, + prt, + state, + sigdp->u.exit.reason, + broken_link); + + if (res == ERTS_PORT_OP_DONE) + msg = am_true; + } + if (sigdp->u.exit.bp) + free_message_buffer(sigdp->u.exit.bp); + if (sigdp->flags & ERTS_P2P_SIG_DATA_FLG_REPLY) + port_sched_op_reply(sigdp->caller, sigdp->ref, msg); + + return ERTS_PORT_REDS_EXIT; +} + +ErtsPortOpResult +erts_port_exit(Process *c_p, + int flags, + Port *prt, + Eterm from, + Eterm reason, + Eterm *refp) +{ + ErtsPortOpResult res; + ErtsProc2PortSigData *sigdp; + ErlHeapFragment *bp = NULL; + + ASSERT((flags & ~(ERTS_PORT_SIG_FLG_BANG_OP + | ERTS_PORT_SIG_FLG_BROKEN_LINK + | ERTS_PORT_SIG_FLG_FORCE_SCHED)) == 0); + + if (!(flags & ERTS_PORT_SIG_FLG_FORCE_SCHED)) { + ErtsTryImmDrvCallState try_call_state + = ERTS_INIT_TRY_IMM_DRV_CALL_STATE(c_p, + prt, + ERTS_PORT_SFLGS_INVALID_LOOKUP, + 0, + !refp, + am_exit); + + + switch (try_imm_drv_call(&try_call_state)) { + case ERTS_TRY_IMM_DRV_CALL_OK: { + res = call_deliver_port_exit(flags & ERTS_PORT_SIG_FLG_BANG_OP, + from, + prt, + try_call_state.state, + reason, + flags & ERTS_PORT_SIG_FLG_BROKEN_LINK); + finalize_imm_drv_call(&try_call_state); + return res; + } + case ERTS_TRY_IMM_DRV_CALL_INVALID_PORT: + return ERTS_PORT_OP_DROPPED; + default: + /* Schedule call instead... */ + break; + } } + + sigdp = erts_port_task_alloc_p2p_sig_data(); + sigdp->flags = ERTS_P2P_SIG_TYPE_EXIT | flags; + sigdp->u.exit.from = from; + + if (is_immed(reason)) { + sigdp->u.exit.reason = reason; + sigdp->u.exit.bp = NULL; + } + else { + Eterm *hp; + Uint hsz = size_object(reason); + bp = new_message_buffer(hsz); + sigdp->u.exit.bp = bp; + hp = bp->mem; + sigdp->u.exit.reason = copy_struct(reason, + hsz, + &hp, + &bp->off_heap); + } + + res = erts_schedule_proc2port_signal(c_p, + prt, + c_p ? c_p->common.id : from, + refp, + sigdp, + 0, + port_sig_exit); + + if (res == ERTS_PORT_OP_DROPPED) { + if (bp) + free_message_buffer(bp); + } + + return res; } -/* initialize the port array */ -void init_io(void) +static ErtsPortOpResult +set_port_connected(int bang_op, + Eterm from, + Port *prt, + erts_aint32_t state, + Eterm connect) { - int i; - ErlDrvEntry** dp; - char maxports[21]; /* enough for any 64-bit integer */ - size_t maxportssize = sizeof(maxports); - Uint ports_bits = ERTS_PORTS_BITS; - Sint port_extra_shift; + /* + * if (bang_op) + * we are part of a "Prt ! {From, {connect, Connect}}" operation + * else + * we are part of a call to port_connect(Port, Connect) + * behave accordingly... + */ -#ifdef ERTS_SMP - init_xports_list_alloc(); + if (state & ERTS_PORT_SFLGS_INVALID_LOOKUP) + return ERTS_PORT_OP_DROPPED; + + if (bang_op) { /* Bang operation */ + if (is_not_internal_pid(connect) || ERTS_PORT_GET_CONNECTED(prt) != from) { + send_badsig(prt); + return ERTS_PORT_OP_DROPPED; + } + + ERTS_PORT_SET_CONNECTED(prt, connect); + deliver_result(prt->common.id, from, am_connected); + +#ifdef USE_VM_PROBES + if(DTRACE_ENABLED(port_command)) { + DTRACE_FORMAT_COMMON_PID_AND_PORT(from, prt); + DTRACE4(port_command, process_str, port_str, prt->name, "connect"); + } #endif + } + else { /* Port BIF operation */ + Process *rp = erts_proc_lookup_raw(connect); + if (!rp) + return ERTS_PORT_OP_DROPPED; + erts_smp_proc_lock(rp, ERTS_PROC_LOCK_LINK); + if (ERTS_PROC_IS_EXITING(rp)) { + erts_smp_proc_unlock(rp, ERTS_PROC_LOCK_LINK); + return ERTS_PORT_OP_DROPPED; + } - pdl_init(); + erts_add_link(&ERTS_P_LINKS(rp), LINK_PID, prt->common.id); + erts_add_link(&ERTS_P_LINKS(prt), LINK_PID, connect); + + ERTS_PORT_SET_CONNECTED(prt, connect); + + erts_smp_proc_unlock(rp, ERTS_PROC_LOCK_LINK); + +#ifdef USE_VM_PROBES + if (DTRACE_ENABLED(port_connect)) { + DTRACE_CHARBUF(process_str, DTRACE_TERM_BUF_SIZE); + DTRACE_CHARBUF(port_str, DTRACE_TERM_BUF_SIZE); + DTRACE_CHARBUF(newprocess_str, DTRACE_TERM_BUF_SIZE); + + dtrace_pid_str(connect, process_str); + erts_snprintf(port_str, sizeof(port_str), "%T", prt->common.id); + dtrace_proc_str(rp, newprocess_str); + DTRACE4(port_connect, process_str, port_str, prt->name, newprocess_str); + } +#endif + } + + return ERTS_PORT_OP_DONE; +} - if (erts_sys_getenv_raw("ERL_MAX_PORTS", maxports, &maxportssize) == 0) - erts_max_ports = atoi(maxports); +static int +port_sig_connect(Port *prt, erts_aint32_t state, int op, ErtsProc2PortSigData *sigdp) +{ + Eterm msg = am_badarg; + if (op == ERTS_PROC2PORT_SIG_EXEC) { + ErtsPortOpResult res; + res = set_port_connected(sigdp->flags & ERTS_P2P_SIG_DATA_FLG_BANG_OP, + sigdp->u.connect.from, + prt, + state, + sigdp->u.connect.connected); + if (res == ERTS_PORT_OP_DONE) + msg = am_true; + } + if (sigdp->flags & ERTS_P2P_SIG_DATA_FLG_REPLY) + port_sched_op_reply(sigdp->caller, sigdp->ref, msg); + return ERTS_PORT_REDS_CONNECT; +} + +ErtsPortOpResult +erts_port_connect(Process *c_p, + int flags, + Port *prt, + Eterm from, + Eterm connect, + Eterm *refp) +{ + ErtsProc2PortSigData *sigdp; + Eterm connect_id; + ErtsTryImmDrvCallState try_call_state + = ERTS_INIT_TRY_IMM_DRV_CALL_STATE(c_p, + prt, + ERTS_PORT_SFLGS_INVALID_LOOKUP, + 0, + !refp, + am_connect); + + ASSERT((flags & ~ERTS_PORT_SIG_FLG_BANG_OP) == 0); + + if (is_not_internal_pid(connect)) + connect_id = NIL; /* Fail in op (for signal order) */ else - erts_max_ports = sys_max_files(); + connect_id = connect; + + switch (try_imm_drv_call(&try_call_state)) { + case ERTS_TRY_IMM_DRV_CALL_OK: { + ErtsPortOpResult res; + res = set_port_connected(flags & ERTS_PORT_SIG_FLG_BANG_OP, + from, + prt, + try_call_state.state, + connect_id); + finalize_imm_drv_call(&try_call_state); + return res; + } + case ERTS_TRY_IMM_DRV_CALL_INVALID_PORT: + return ERTS_PORT_OP_DROPPED; + default: + /* Schedule call instead... */ + break; + } + + sigdp = erts_port_task_alloc_p2p_sig_data(); + sigdp->flags = ERTS_P2P_SIG_TYPE_CONNECT | flags; + + sigdp->u.connect.from = from; + sigdp->u.connect.connected = connect_id; + + return erts_schedule_proc2port_signal(c_p, + prt, + c_p->common.id, + refp, + sigdp, + 0, + port_sig_connect); +} - if (erts_max_ports > ERTS_MAX_PORTS) - erts_max_ports = ERTS_MAX_PORTS; - if (erts_max_ports < 1024) - erts_max_ports = 1024; +static void +port_unlink(Port *prt, Eterm from) +{ + ErtsLink *lnk = erts_remove_link(&ERTS_P_LINKS(prt), from); + if (lnk) + erts_destroy_link(lnk); +} - if (erts_use_r9_pids_ports) { - ports_bits = ERTS_R9_PORTS_BITS; - if (erts_max_ports > ERTS_MAX_R9_PORTS) - erts_max_ports = ERTS_MAX_R9_PORTS; +static int +port_sig_unlink(Port *prt, erts_aint32_t state, int op, ErtsProc2PortSigData *sigdp) +{ + if (op == ERTS_PROC2PORT_SIG_EXEC) + port_unlink(prt, sigdp->u.unlink.from); + if (sigdp->flags & ERTS_P2P_SIG_DATA_FLG_REPLY) + port_sched_op_reply(sigdp->caller, sigdp->ref, am_true); + return ERTS_PORT_REDS_UNLINK; +} + +ErtsPortOpResult +erts_port_unlink(Process *c_p, Port *prt, Eterm from, Eterm *refp) +{ + ErtsProc2PortSigData *sigdp; + ErtsTryImmDrvCallState try_call_state + = ERTS_INIT_TRY_IMM_DRV_CALL_STATE(c_p, + prt, + ERTS_PORT_SFLGS_DEAD, + 0, + !refp, + am_unlink); + + switch (try_imm_drv_call(&try_call_state)) { + case ERTS_TRY_IMM_DRV_CALL_OK: + port_unlink(prt, from); + finalize_imm_drv_call(&try_call_state); + return ERTS_PORT_OP_DONE; + case ERTS_TRY_IMM_DRV_CALL_INVALID_PORT: + return ERTS_PORT_OP_DROPPED; + default: + /* Schedule call instead... */ + break; } - port_extra_shift = erts_fit_in_bits_int32(erts_max_ports - 1); - port_num_mask = (1 << ports_bits) - 1; + sigdp = erts_port_task_alloc_p2p_sig_data(); + sigdp->flags = ERTS_P2P_SIG_TYPE_UNLINK; + sigdp->u.unlink.from = from; + + return erts_schedule_proc2port_signal(c_p, + prt, + c_p ? c_p->common.id : from, + refp, + sigdp, + 0, + port_sig_unlink); +} - erts_port_tab_index_mask = ~(~((Uint) 0) << port_extra_shift); - erts_max_ports = 1 << port_extra_shift; +static void +port_link_failure(Eterm port_id, Eterm linker) +{ + Process *rp; + ErtsProcLocks rp_locks = ERTS_PROC_LOCK_LINK|ERTS_PROC_LOCKS_XSIG_SEND; + ASSERT(is_internal_pid(linker)); + rp = erts_pid2proc(NULL, 0, linker, rp_locks); + if (rp) { + ErtsLink *rlnk = erts_remove_link(&ERTS_P_LINKS(rp), port_id); + if (rlnk) { + int xres = erts_send_exit_signal(NULL, + port_id, + rp, + &rp_locks, + am_noproc, + NIL, + NULL, + 0); + if (xres >= 0 && IS_TRACED_FL(rp, F_TRACE_PROCS)) { + /* We didn't exit the process and it is traced */ + if (IS_TRACED_FL(rp, F_TRACE_PROCS)) + trace_proc(NULL, rp, am_getting_unlinked, port_id); + } + if (rp_locks) + erts_smp_proc_unlock(rp, rp_locks); + } + } +} - erts_smp_mtx_init(&erts_driver_list_lock,"driver_list"); - driver_list = NULL; - erts_smp_tsd_key_create(&driver_list_lock_status_key); - erts_smp_tsd_key_create(&driver_list_last_error_key); +static void +port_link(Port *prt, erts_aint32_t state, Eterm to) +{ + if (!(state & ERTS_PORT_SFLGS_INVALID_LOOKUP)) + erts_add_link(&ERTS_P_LINKS(prt), LINK_PID, to); + else + port_link_failure(prt->common.id, to); +} - if (erts_max_ports * sizeof(Port) <= erts_max_ports) { - /* More memory needed than the whole address space. */ - erts_alloc_enomem(ERTS_ALC_T_PORT_TABLE, ~((Uint) 0)); +static int +port_sig_link(Port *prt, erts_aint32_t state, int op, ErtsProc2PortSigData *sigdp) +{ + if (op == ERTS_PROC2PORT_SIG_EXEC) + port_link(prt, state, sigdp->u.link.to); + else + port_link_failure(sigdp->u.link.port, sigdp->u.link.to); + if (sigdp->flags & ERTS_P2P_SIG_DATA_FLG_REPLY) + port_sched_op_reply(sigdp->caller, sigdp->ref, am_true); + return ERTS_PORT_REDS_LINK; +} + +ErtsPortOpResult +erts_port_link(Process *c_p, Port *prt, Eterm to, Eterm *refp) +{ + ErtsProc2PortSigData *sigdp; + ErtsTryImmDrvCallState try_call_state + = ERTS_INIT_TRY_IMM_DRV_CALL_STATE(c_p, + prt, + ERTS_PORT_SFLGS_INVALID_LOOKUP, + 0, + !refp, + am_link); + + switch (try_imm_drv_call(&try_call_state)) { + case ERTS_TRY_IMM_DRV_CALL_OK: + port_link(prt, try_call_state.state, to); + finalize_imm_drv_call(&try_call_state); + return ERTS_PORT_OP_DONE; + case ERTS_TRY_IMM_DRV_CALL_INVALID_PORT: + return ERTS_PORT_OP_BADARG; + default: + /* Schedule call instead... */ + break; } - erts_port = (Port *) erts_alloc(ERTS_ALC_T_PORT_TABLE, - erts_max_ports * sizeof(Port)); + sigdp = erts_port_task_alloc_p2p_sig_data(); + sigdp->flags = ERTS_P2P_SIG_TYPE_LINK; + sigdp->u.link.port = prt->common.id; + sigdp->u.link.to = to; + + return erts_schedule_proc2port_signal(c_p, + prt, + c_p ? c_p->common.id : to, + refp, + sigdp, + 0, + port_sig_link); +} - erts_smp_atomic_init_nob(&erts_bytes_out, 0); - erts_smp_atomic_init_nob(&erts_bytes_in, 0); +void erts_init_io(int port_tab_size, + int port_tab_size_ignore_files) +{ + ErlDrvEntry** dp; + erts_smp_rwmtx_opt_t drv_list_rwmtx_opts = ERTS_SMP_RWMTX_OPT_DEFAULT_INITER; + drv_list_rwmtx_opts.type = ERTS_SMP_RWMTX_TYPE_EXTREMELY_FREQUENT_READ; + drv_list_rwmtx_opts.lived = ERTS_SMP_RWMTX_LONG_LIVED; - for (i = 0; i < erts_max_ports; i++) { - erts_port_task_init_sched(&erts_port[i].sched); - erts_smp_atomic_init_nob(&erts_port[i].refc, 0); #ifdef ERTS_SMP - erts_port[i].lock = NULL; - erts_port[i].xports = NULL; - erts_smp_spinlock_init_x(&erts_port[i].state_lck, -#ifdef ERTS_ENABLE_LOCK_COUNT - (erts_lcnt_rt_options & ERTS_LCNT_OPT_PORTLOCK) ? "port_state" : NULL, -#else - "port_state", -#endif - make_small(0)); + init_xports_list_alloc(); #endif - erts_port[i].tracer_proc = NIL; - erts_port[i].trace_flags = 0; - erts_port[i].drv_ptr = NULL; - erts_port[i].status = ERTS_PORT_SFLG_FREE; - erts_port[i].name = NULL; - erts_port[i].nlinks = NULL; - erts_port[i].monitors = NULL; - erts_port[i].linebuf = NULL; - erts_port[i].port_data_lock = NULL; + pdl_init(); + + if (!port_tab_size_ignore_files) { + int max_files = sys_max_files(); + if (port_tab_size < max_files) + port_tab_size = max_files; } - erts_smp_atomic32_init_nob(&erts_ports_snapshot, (erts_aint32_t) 0); - last_port_num = 0; - erts_smp_spinlock_init(&get_free_port_lck, "get_free_port"); + if (port_tab_size > ERTS_MAX_PORTS) + port_tab_size = ERTS_MAX_PORTS; + else if (port_tab_size < ERTS_MIN_PORTS) + port_tab_size = ERTS_MIN_PORTS; + + erts_smp_rwmtx_init_opt(&erts_driver_list_lock, + &drv_list_rwmtx_opts, + "driver_list"); + driver_list = NULL; + erts_smp_tsd_key_create(&driver_list_lock_status_key); + erts_smp_tsd_key_create(&driver_list_last_error_key); + + erts_ptab_init_table(&erts_port, + ERTS_ALC_T_PORT_TABLE, + NULL, + (ErtsPTabElementCommon *) &erts_invalid_port.common, + port_tab_size, + "port_table"); + + erts_smp_atomic_init_nob(&erts_bytes_out, 0); + erts_smp_atomic_init_nob(&erts_bytes_in, 0); sys_init_io(); erts_smp_tsd_set(driver_list_lock_status_key, (void *) 1); - erts_smp_mtx_lock(&erts_driver_list_lock); + erts_smp_rwmtx_rwlock(&erts_driver_list_lock); init_driver(&fd_driver, &fd_driver_entry, NULL); init_driver(&vanilla_driver, &vanilla_driver_entry, NULL); @@ -1390,27 +2655,63 @@ void init_io(void) erts_add_driver_entry(*dp, NULL, 1); erts_smp_tsd_set(driver_list_lock_status_key, NULL); - erts_smp_mtx_unlock(&erts_driver_list_lock); + erts_smp_rwmtx_rwunlock(&erts_driver_list_lock); } #if defined(ERTS_ENABLE_LOCK_COUNT) && defined(ERTS_SMP) -void erts_lcnt_enable_io_lock_count(int enable) { - int i; - for (i = 0; i < erts_max_ports; i++) { - Port* p = &erts_port[i]; - if (enable) { - erts_lcnt_init_lock_x(&p->state_lck.lcnt, "port_state", ERTS_LCNT_LT_SPINLOCK, make_small(i)); - if (p->lock) { - erts_lcnt_init_lock_x(&p->lock->lcnt, "port_lock", ERTS_LCNT_LT_MUTEX, make_small(i)); - } - } else { - erts_lcnt_destroy_lock(&p->state_lck.lcnt); - if (p->lock) { - erts_lcnt_destroy_lock(&p->lock->lcnt); - } - } +static ERTS_INLINE void lcnt_enable_drv_lock_count(erts_driver_t *dp, int enable) +{ + if (dp->lock) { + if (enable) + erts_lcnt_init_lock_x(&dp->lock->lcnt, + "driver_lock", + ERTS_LCNT_LT_MUTEX, + am_atom_put(dp->name, + sys_strlen(dp->name))); + else + erts_lcnt_destroy_lock(&dp->lock->lcnt); + + } +} + +static ERTS_INLINE void lcnt_enable_port_lock_count(Port *prt, int enable) +{ + erts_aint32_t state = erts_atomic32_read_nob(&prt->state); + if (!enable) { + erts_lcnt_destroy_lock(&prt->sched.mtx.lcnt); + if (state & ERTS_PORT_SFLG_PORT_SPECIFIC_LOCK) + erts_lcnt_destroy_lock(&prt->lock->lcnt); } + else { + erts_lcnt_init_lock_x(&prt->sched.mtx.lcnt, + "port_sched_lock", + ERTS_LCNT_LT_MUTEX, + prt->common.id); + if (state & ERTS_PORT_SFLG_PORT_SPECIFIC_LOCK) + erts_lcnt_init_lock_x(&prt->lock->lcnt, + "port_lock", + ERTS_LCNT_LT_MUTEX, + prt->common.id); + } +} + +void erts_lcnt_enable_io_lock_count(int enable) +{ + erts_driver_t *dp; + int i, max = erts_ptab_max(&erts_port); + + for (i = 0; i < max; i++) { + Port *prt = erts_pix2port(i); + if (prt) + lcnt_enable_port_lock_count(prt, enable); + } + + lcnt_enable_drv_lock_count(&vanilla_driver, enable); + lcnt_enable_drv_lock_count(&spawn_driver, enable); + lcnt_enable_drv_lock_count(&fd_driver, enable); + for (dp = driver_list; dp; dp = dp->next) + lcnt_enable_drv_lock_count(dp, enable); } #endif @@ -1594,9 +2895,7 @@ deliver_result(Eterm sender, Eterm pid, Eterm res) ERTS_SMP_CHK_NO_PROC_LOCKS; - ASSERT(is_internal_port(sender) - && is_internal_pid(pid) - && internal_pid_index(pid) < erts_max_processes); + ASSERT(is_internal_port(sender) && is_internal_pid(pid)); rp = (scheduler ? erts_proc_lookup(pid) @@ -1608,16 +2907,19 @@ deliver_result(Eterm sender, Eterm pid, Eterm res) ErlOffHeap *ohp; Eterm* hp; Uint sz_res; - sz_res = size_object(res); - hp = erts_alloc_message_heap(sz_res + 3, &bp, &ohp, rp, &rp_locks); - res = copy_struct(res, sz_res, &hp, ohp); - tuple = TUPLE2(hp, sender, res); + + sz_res = size_object(res); + hp = erts_alloc_message_heap(sz_res + 3, &bp, &ohp, rp, &rp_locks); + res = copy_struct(res, sz_res, &hp, ohp); + tuple = TUPLE2(hp, sender, res); erts_queue_message(rp, &rp_locks, bp, tuple, NIL #ifdef USE_VM_PROBES , NIL #endif ); - erts_smp_proc_unlock(rp, rp_locks); + + if (rp_locks) + erts_smp_proc_unlock(rp, rp_locks); if (!scheduler) erts_smp_proc_dec_refc(rp); @@ -1633,7 +2935,7 @@ deliver_result(Eterm sender, Eterm pid, Eterm res) * len -- length of data */ -static void deliver_read_message(Port* prt, Eterm to, +static void deliver_read_message(Port* prt, erts_aint32_t state, Eterm to, char *hbuf, ErlDrvSizeT hlen, char *buf, ErlDrvSizeT len, int eol) { @@ -1651,10 +2953,11 @@ static void deliver_read_message(Port* prt, Eterm to, ERTS_SMP_CHK_NO_PROC_LOCKS; need = 3 + 3 + 2*hlen; - if (prt->status & ERTS_PORT_SFLG_LINEBUF_IO) { + + if (state & ERTS_PORT_SFLG_LINEBUF_IO) { need += 3; } - if (prt->status & ERTS_PORT_SFLG_BINARY_IO && buf != NULL) { + if ((state & ERTS_PORT_SFLG_BINARY_IO) && buf != NULL) { need += PROC_BIN_SIZE; } else { need += 2*len; @@ -1670,7 +2973,7 @@ static void deliver_read_message(Port* prt, Eterm to, hp = erts_alloc_message_heap(need, &bp, &ohp, rp, &rp_locks); listp = NIL; - if ((prt->status & ERTS_PORT_SFLG_BINARY_IO) == 0) { + if ((state & ERTS_PORT_SFLG_BINARY_IO) == 0) { listp = buf_to_intlist(&hp, buf, len, listp); } else if (buf != NULL) { ProcBin* pb; @@ -1701,14 +3004,14 @@ static void deliver_read_message(Port* prt, Eterm to, listp = buf_to_intlist(&hp, hbuf, hlen, listp); } - if (prt->status & ERTS_PORT_SFLG_LINEBUF_IO){ + if (state & ERTS_PORT_SFLG_LINEBUF_IO){ listp = TUPLE2(hp, (eol) ? am_eol : am_noeol, listp); hp += 3; } tuple = TUPLE2(hp, am_data, listp); hp += 3; - tuple = TUPLE2(hp, prt->id, tuple); + tuple = TUPLE2(hp, prt->common.id, tuple); hp += 3; erts_queue_message(rp, &rp_locks, bp, tuple, am_undefined @@ -1726,7 +3029,8 @@ static void deliver_read_message(Port* prt, Eterm to, * Deliver all lines in a line buffer, repeats calls to * deliver_read_message, and takes the same parameters. */ -static void deliver_linebuf_message(Port* prt, Eterm to, +static void deliver_linebuf_message(Port* prt, erts_aint_t state, + Eterm to, char* hbuf, ErlDrvSizeT hlen, char *buf, ErlDrvSizeT len) { @@ -1735,7 +3039,7 @@ static void deliver_linebuf_message(Port* prt, Eterm to, if(init_linebuf_context(&lc,&(prt->linebuf), buf, len) < 0) return; while((ret = read_linebuf(&lc)) > LINEBUF_EMPTY) - deliver_read_message(prt, to, hbuf, hlen, LINEBUF_DATA(lc), + deliver_read_message(prt, state, to, hbuf, hlen, LINEBUF_DATA(lc), LINEBUF_DATALEN(lc), (ret == LINEBUF_EOL)); } @@ -1746,20 +3050,25 @@ static void deliver_linebuf_message(Port* prt, Eterm to, * Parameters: * prt - Pointer to a Port structure for this port. */ -static void flush_linebuf_messages(Port *prt) +static void flush_linebuf_messages(Port *prt, erts_aint32_t state) { LineBufContext lc; int ret; ERTS_SMP_LC_ASSERT(!prt || erts_lc_is_port_locked(prt)); - if(prt == NULL || !(prt->status & ERTS_PORT_SFLG_LINEBUF_IO)) + + if (!prt) + return; + + if (!(state & ERTS_PORT_SFLG_LINEBUF_IO)) return; if(init_linebuf_context(&lc,&(prt->linebuf), NULL, 0) < 0) return; while((ret = flush_linebuf(&lc)) > LINEBUF_EMPTY) deliver_read_message(prt, - prt->connected, + state, + ERTS_PORT_GET_CONNECTED(prt), NULL, 0, LINEBUF_DATA(lc), @@ -1787,6 +3096,7 @@ deliver_vec_message(Port* prt, /* Port */ ErlOffHeap *ohp; ErtsProcLocks rp_locks = 0; int scheduler = erts_get_scheduler_id() != 0; + erts_aint32_t state; ERTS_SMP_LC_ASSERT(erts_lc_is_port_locked(prt)); ERTS_SMP_CHK_NO_PROC_LOCKS; @@ -1802,12 +3112,13 @@ deliver_vec_message(Port* prt, /* Port */ if (!rp) return; + state = erts_atomic32_read_nob(&prt->state); /* * Calculate the exact number of heap words needed. */ need = 3 + 3; /* Heap space for two tuples */ - if (prt->status & ERTS_PORT_SFLG_BINARY_IO) { + if (state & ERTS_PORT_SFLG_BINARY_IO) { need += (2+PROC_BIN_SIZE)*vsize - 2 + hlen*2; } else { need += (hlen+csize)*2; @@ -1818,7 +3129,7 @@ deliver_vec_message(Port* prt, /* Port */ listp = NIL; iov += vsize; - if ((prt->status & ERTS_PORT_SFLG_BINARY_IO) == 0) { + if ((state & ERTS_PORT_SFLG_BINARY_IO) == 0) { Eterm* thp = hp; while (vsize--) { iov--; @@ -1871,7 +3182,7 @@ deliver_vec_message(Port* prt, /* Port */ tuple = TUPLE2(hp, am_data, listp); hp += 3; - tuple = TUPLE2(hp, prt->id, tuple); + tuple = TUPLE2(hp, prt->common.id, tuple); hp += 3; erts_queue_message(rp, &rp_locks, bp, tuple, am_undefined @@ -1904,7 +3215,7 @@ static void deliver_bin_message(Port* prt, /* port */ /* * Note. * - * The test for (p->status & ERTS_PORT_SFLGS_DEAD) == 0 is important since the + * The test for ERTS_PORT_SFLGS_DEAD is important since the * driver's flush function might call driver_async, which when using no * threads and being short circuited will notice that the io queue is empty * (after calling the driver's async_ready) and recursively call @@ -1920,7 +3231,7 @@ static void flush_port(Port *p) if (p->drv_ptr->flush != NULL) { #ifdef USE_VM_PROBES if (DTRACE_ENABLED(driver_flush)) { - DTRACE_FORMAT_COMMON_PID_AND_PORT(p->connected, p) + DTRACE_FORMAT_COMMON_PID_AND_PORT(ERTS_PORT_GET_CONNECTED(p), p) DTRACE3(driver_flush, process_str, port_str, p->name); } #endif @@ -1935,11 +3246,12 @@ static void flush_port(Port *p) } #ifdef ERTS_SMP if (p->xports) - erts_smp_xports_unlock(p); + erts_port_handle_xports(p); ASSERT(!p->xports); #endif } - if ((p->status & ERTS_PORT_SFLGS_DEAD) == 0 && is_port_ioq_empty(p)) { + if ((erts_atomic32_read_nob(&p->state) & ERTS_PORT_SFLGS_DEAD) == 0 + && is_port_ioq_empty(p)) { terminate_port(p); } } @@ -1951,29 +3263,29 @@ terminate_port(Port *prt) Eterm send_closed_port_id; Eterm connected_id = NIL /* Initialize to silence compiler */; erts_driver_t *drv; - int halt; + erts_aint32_t state; ERTS_SMP_CHK_NO_PROC_LOCKS; ERTS_SMP_LC_ASSERT(erts_lc_is_port_locked(prt)); - ASSERT(!prt->nlinks); - ASSERT(!prt->monitors); + ASSERT(!ERTS_P_LINKS(prt)); + ASSERT(!ERTS_P_MONITORS(prt)); - /* prt->status may be altered by kill_port()below */ - halt = (prt->status & ERTS_PORT_SFLG_HALT) != 0; - if (prt->status & ERTS_PORT_SFLG_SEND_CLOSED) { - erts_port_status_band_set(prt, ~ERTS_PORT_SFLG_SEND_CLOSED); - send_closed_port_id = prt->id; - connected_id = prt->connected; + /* state may be altered by kill_port() below */ + state = erts_atomic32_read_band_nob(&prt->state, + ~ERTS_PORT_SFLG_SEND_CLOSED); + if (state & ERTS_PORT_SFLG_SEND_CLOSED) { + send_closed_port_id = prt->common.id; + connected_id = ERTS_PORT_GET_CONNECTED(prt); } else { send_closed_port_id = NIL; } #ifdef ERTS_SMP - erts_cancel_smp_ptimer(prt->ptimer); + erts_cancel_smp_ptimer(prt->common.u.alive.ptimer); #else - erts_cancel_timer(&prt->tm); + erts_cancel_timer(&prt->common.u.alive.tm); #endif drv = prt->drv_ptr; @@ -1981,7 +3293,7 @@ terminate_port(Port *prt) int fpe_was_unmasked = erts_block_fpe(); #ifdef USE_VM_PROBES if (DTRACE_ENABLED(driver_stop)) { - DTRACE_FORMAT_COMMON_PID_AND_PORT(prt->connected, prt) + DTRACE_FORMAT_COMMON_PID_AND_PORT(connected_id, prt) DTRACE3(driver_stop, process_str, drv->name, port_str); } #endif @@ -1989,14 +3301,14 @@ terminate_port(Port *prt) erts_unblock_fpe(fpe_was_unmasked); #ifdef ERTS_SMP if (prt->xports) - erts_smp_xports_unlock(prt); + erts_port_handle_xports(prt); ASSERT(!prt->xports); #endif } if(drv->handle != NULL) { - erts_smp_mtx_lock(&erts_driver_list_lock); + erts_smp_rwmtx_rlock(&erts_driver_list_lock); erts_ddll_decrement_port_count(drv->handle); - erts_smp_mtx_unlock(&erts_driver_list_lock); + erts_smp_rwmtx_runlock(&erts_driver_list_lock); } stopq(prt); /* clear queue memory */ if(prt->linebuf != NULL){ @@ -2012,20 +3324,21 @@ terminate_port(Port *prt) if (prt->psd) erts_free(ERTS_ALC_T_PRTSD, prt->psd); + ASSERT(prt->dist_entry == NULL); + kill_port(prt); /* * We don't want to send the closed message until after the * port has been removed from the port table (in kill_port()). */ - if (halt && (erts_smp_atomic32_dec_read_nob(&erts_halt_progress) == 0)) { - erts_smp_port_unlock(prt); /* We will exit and never return */ + if ((state & ERTS_PORT_SFLG_HALT) + && (erts_smp_atomic32_dec_read_nob(&erts_halt_progress) == 0)) { + erts_port_release(prt); /* We will exit and never return */ erl_exit_flush_async(erts_halt_code, ""); } if (is_internal_port(send_closed_port_id)) deliver_result(send_closed_port_id, connected_id, am_closed); - - ASSERT(prt->dist_entry == NULL); } void @@ -2045,7 +3358,7 @@ static void sweep_one_monitor(ErtsMonitor *mon, void *vpsc) if (!rp) { goto done; } - rmon = erts_remove_monitor(&(rp->monitors),mon->ref); + rmon = erts_remove_monitor(&ERTS_P_MONITORS(rp), mon->ref); erts_smp_proc_unlock(rp, ERTS_PROC_LOCK_LINK); if (rmon == NULL) { goto done; @@ -2099,7 +3412,7 @@ static void sweep_one_link(ErtsLink *lnk, void *vpsc) ASSERT(is_internal_pid(lnk->pid)); rp = erts_pid2proc(NULL, 0, lnk->pid, rp_locks); if (rp) { - ErtsLink *rlnk = erts_remove_link(&(rp->nlinks), psc->port); + ErtsLink *rlnk = erts_remove_link(&ERTS_P_LINKS(rp), psc->port); if (rlnk) { int xres = erts_send_exit_signal(NULL, @@ -2135,11 +3448,13 @@ static void sweep_one_link(ErtsLink *lnk, void *vpsc) * that is to kill a port till reason kill. Then the port is stopped. * */ -void -erts_do_exit_port(Port *p, Eterm from, Eterm reason) + +int +erts_deliver_port_exit(Port *p, Eterm from, Eterm reason, int send_closed) { ErtsLink *lnk; Eterm rreason; + erts_aint32_t state; ERTS_SMP_CHK_NO_PROC_LOCKS; ERTS_SMP_LC_ASSERT(erts_lc_is_port_locked(p)); @@ -2159,66 +3474,76 @@ erts_do_exit_port(Port *p, Eterm from, Eterm reason) } #endif - if ((p->status & (ERTS_PORT_SFLGS_DEAD - | ERTS_PORT_SFLG_EXITING - | ERTS_PORT_SFLG_IMMORTAL)) - || ((reason == am_normal) && - ((from != p->connected) && (from != p->id)))) { - return; - } + state = erts_atomic32_read_nob(&p->state); + if (state & (ERTS_PORT_SFLGS_DEAD + | ERTS_PORT_SFLG_EXITING + | ERTS_PORT_SFLG_CLOSING)) + return 0; + + if (reason == am_normal && from != ERTS_PORT_GET_CONNECTED(p) && from != p->common.id) + return 0; + + if (send_closed) + erts_atomic32_read_bor_relb(&p->state, + ERTS_PORT_SFLG_SEND_CLOSED); if (IS_TRACED_FL(p, F_TRACE_PORTS)) { trace_port(p, am_closed, reason); } - erts_trace_check_exiting(p->id); + erts_trace_check_exiting(p->common.id); /* * Setting the port to not busy here, frees the list of pending * processes and makes them runnable. */ - set_busy_port((ErlDrvPort)internal_port_index(p->id), 0); + set_busy_port((ErlDrvPort) p, 0); - if (p->reg != NULL) - (void) erts_unregister_name(NULL, 0, p, p->reg->name); + if (p->common.u.alive.reg != NULL) + (void) erts_unregister_name(NULL, 0, p, p->common.u.alive.reg->name); - erts_port_status_bor_set(p, ERTS_PORT_SFLG_EXITING); + state = erts_atomic32_read_bor_relb(&p->state, ERTS_PORT_SFLG_EXITING); { - SweepContext sc = {p->id, rreason}; - lnk = p->nlinks; - p->nlinks = NULL; + SweepContext sc = {p->common.id, rreason}; + lnk = ERTS_P_LINKS(p); + ERTS_P_LINKS(p) = NULL; erts_sweep_links(lnk, &sweep_one_link, &sc); } DRV_MONITOR_LOCK_PDL(p); { - ErtsMonitor *moni = p->monitors; - p->monitors = NULL; + ErtsMonitor *moni = ERTS_P_MONITORS(p); + ERTS_P_MONITORS(p) = NULL; erts_sweep_monitors(moni, &sweep_one_monitor, NULL); } DRV_MONITOR_UNLOCK_PDL(p); - if ((p->status & ERTS_PORT_SFLG_DISTRIBUTION) && p->dist_entry) { + if ((state & ERTS_PORT_SFLG_DISTRIBUTION) && p->dist_entry) { erts_do_net_exits(p->dist_entry, rreason); erts_deref_dist_entry(p->dist_entry); - p->dist_entry = NULL; - erts_port_status_band_set(p, ~ERTS_PORT_SFLG_DISTRIBUTION); + p->dist_entry = NULL; + erts_atomic32_read_band_relb(&p->state, + ~ERTS_PORT_SFLG_DISTRIBUTION); } if ((reason != am_kill) && !is_port_ioq_empty(p)) { - erts_port_status_bandor_set(p, - ~ERTS_PORT_SFLG_EXITING, /* must turn it off */ - ERTS_PORT_SFLG_CLOSING); + /* must turn exiting flag off */ + erts_atomic32_read_bset_relb(&p->state, + (ERTS_PORT_SFLG_EXITING + | ERTS_PORT_SFLG_CLOSING), + ERTS_PORT_SFLG_CLOSING); flush_port(p); } else { terminate_port(p); } + + return 1; } /* About the states ERTS_PORT_SFLG_EXITING and ERTS_PORT_SFLG_CLOSING used above. ** -** ERTS_PORT_SFLG_EXITING is a recursion protection for erts_do_exit_port(). +** ERTS_PORT_SFLG_EXITING is a recursion protection for erts_deliver_port_exit(). ** It is unclear whether this state is necessary or not, it might be possible ** to merge it with ERTS_PORT_SFLG_CLOSING. ERTS_PORT_SFLG_EXITING only persists ** over a section of sequential (but highly recursive) code. @@ -2234,232 +3559,1108 @@ erts_do_exit_port(Port *p, Eterm from, Eterm reason) ** {PID, close} ** {PID, {command, io-list}} ** {PID, {connect, New_PID}} -** -** */ -void erts_port_command(Process *proc, - Eterm caller_id, - Port *port, - Eterm command) +ErtsPortOpResult +erts_port_command(Process *c_p, + int flags, + Port *port, + Eterm command, + Eterm *refp) { Eterm *tp; - Eterm pid; - if (!port) - return; + ASSERT(port); - erts_smp_proc_unlock(proc, ERTS_PROC_LOCK_MAIN); - ERTS_SMP_CHK_NO_PROC_LOCKS; - ASSERT(!INVALID_PORT(port, port->id)); + flags |= ERTS_PORT_SIG_FLG_BANG_OP; if (is_tuple_arity(command, 2)) { + Eterm cntd; tp = tuple_val(command); - if ((pid = port->connected) == tp[1]) { - /* PID must be connected */ + cntd = tp[1]; + if (is_internal_pid(cntd)) { if (tp[2] == am_close) { - erts_port_status_bor_set(port, ERTS_PORT_SFLG_SEND_CLOSED); - erts_do_exit_port(port, pid, am_normal); - -#ifdef USE_VM_PROBES - if(DTRACE_ENABLED(port_command)) { - DTRACE_FORMAT_COMMON_PROC_AND_PORT(proc, port) - DTRACE4(port_command, process_str, port_str, port->name, "close"); - } -#endif - goto done; + if (!erts_port_synchronous_ops) + refp = NULL; + flags &= ~ERTS_PORT_SIG_FLG_NOSUSPEND; + return erts_port_exit(c_p, flags, port, cntd, am_normal, refp); } else if (is_tuple_arity(tp[2], 2)) { tp = tuple_val(tp[2]); if (tp[1] == am_command) { - if (erts_write_to_port(caller_id, port, tp[2]) == 0) - goto done; - } else if ((tp[1] == am_connect) && is_internal_pid(tp[2])) { -#ifdef USE_VM_PROBES - if(DTRACE_ENABLED(port_command)) { - DTRACE_FORMAT_COMMON_PROC_AND_PORT(proc, port) - DTRACE4(port_command, process_str, port_str, port->name, "connect"); - } -#endif - port->connected = tp[2]; - deliver_result(port->id, pid, am_connected); - goto done; + if (!(flags & ERTS_PORT_SIG_FLG_NOSUSPEND) + && !erts_port_synchronous_ops) + refp = NULL; + return erts_port_output(c_p, flags, port, cntd, tp[2], refp); + } + else if (tp[1] == am_connect) { + if (!erts_port_synchronous_ops) + refp = NULL; + flags &= ~ERTS_PORT_SIG_FLG_NOSUSPEND; + return erts_port_connect(c_p, flags, port, cntd, tp[2], refp); } } } } - { - ErtsProcLocks rp_locks = ERTS_PROC_LOCKS_XSIG_SEND; - Process* rp = erts_pid2proc(NULL, 0, - port->connected, rp_locks); - if (rp) { - (void) erts_send_exit_signal(NULL, - port->id, - rp, - &rp_locks, - am_badsig, - NIL, - NULL, - 0); - erts_smp_proc_unlock(rp, rp_locks); - } + /* badsig */ + if (!erts_port_synchronous_ops) + refp = NULL; + flags &= ~ERTS_PORT_SIG_FLG_NOSUSPEND; + return bad_port_signal(c_p, flags, port, c_p->common.id, refp, am_command); +} + +static ERTS_INLINE ErtsPortOpResult +call_driver_control(Eterm caller, + Port *prt, + unsigned int command, + char *bufp, + ErlDrvSizeT size, + char **resp_bufp, + ErlDrvSizeT *from_size) +{ + ErlDrvSSizeT cres; + + if (!prt->drv_ptr->control) + return ERTS_PORT_OP_BADARG; + +#ifdef USE_VM_PROBES + if (DTRACE_ENABLED(port_control) || DTRACE_ENABLED(driver_control)) { + DTRACE_FORMAT_COMMON_PID_AND_PORT(caller, prt); + DTRACE4(port_control, process_str, port_str, prt->name, command); + DTRACE5(driver_control, process_str, port_str, prt->name, + command, size); } - done: - erts_smp_proc_lock(proc, ERTS_PROC_LOCK_MAIN); +#endif + + prt->caller = caller; + cres = prt->drv_ptr->control((ErlDrvData) prt->drv_data, + command, + bufp, + size, + resp_bufp, + *from_size); + prt->caller = NIL; + + if (cres < 0) + return ERTS_PORT_OP_BADARG; + + *from_size = (ErlDrvSizeT) cres; + + return ERTS_PORT_OP_DONE; } -/* - * Control a port synchronously. - * Returns either a list or a binary. - */ -Eterm -erts_port_control(Process* p, Port* prt, Uint command, Eterm iolist) -{ - byte* to_port = NULL; /* Buffer to write to port. */ - /* Initialization is for shutting up - warning about use before set. */ - Uint to_len = 0; /* Length of buffer. */ - int must_free = 0; /* True if the buffer should be freed. */ - char port_result[ERL_ONHEAP_BIN_LIMIT]; /* Default buffer for result from port. */ - char* port_resp; /* Pointer to result buffer. */ - ErlDrvSSizeT n; - ErlDrvSSizeT (*control) - (ErlDrvData, unsigned, char*, ErlDrvSizeT, char**, ErlDrvSizeT); - int fpe_was_unmasked; +static void +cleanup_scheduled_control(Binary *binp, char *bufp) +{ + if (binp) { + if (erts_refc_dectest(&binp->refc, 0) == 0) + erts_bin_free(binp); + } + else { + if (bufp) + erts_free(ERTS_ALC_T_DRV_CTRL_DATA, bufp); + } +} - ERTS_SMP_LC_ASSERT(erts_lc_is_port_locked(prt)); - if ((control = prt->drv_ptr->control) == NULL) { - return THE_NON_VALUE; +static ERTS_INLINE Uint +port_control_result_size(int control_flags, + char *resp_bufp, + ErlDrvSizeT *resp_size, + char *pre_alloc_buf) +{ + if (!resp_bufp) + return (Uint) 0; + + if (control_flags & PORT_CONTROL_FLAG_BINARY) { + if (resp_bufp != pre_alloc_buf) { + ErlDrvBinary *dbin = (ErlDrvBinary *) resp_bufp; + *resp_size = dbin->orig_size; + if (*resp_size > ERL_ONHEAP_BIN_LIMIT) + return PROC_BIN_SIZE; + } + ASSERT(*resp_size <= ERL_ONHEAP_BIN_LIMIT); + return (Uint) heap_bin_size((*resp_size)); } - /* - * Convert the iolist to a buffer, pointed to by to_port, - * and with its length in to_len. - */ - if (is_binary(iolist) && binary_bitoffset(iolist) == 0) { + return (Uint) 2*(*resp_size); +} + +static ERTS_INLINE Eterm +write_port_control_result(int control_flags, + char *resp_bufp, + ErlDrvSizeT resp_size, + char *pre_alloc_buf, + Eterm **hpp, + ErlHeapFragment *bp, + ErlOffHeap *ohp) +{ + Eterm res; + if (!resp_bufp) + return NIL; + if (control_flags & PORT_CONTROL_FLAG_BINARY) { + /* Binary result */ + ErlDrvBinary *dbin; + ErlHeapBin *hbin; + + if (resp_bufp == pre_alloc_buf) + dbin = NULL; + else { + dbin = (ErlDrvBinary *) resp_bufp; + if (dbin->orig_size > ERL_ONHEAP_BIN_LIMIT) { + ProcBin* pb = (ProcBin *) *hpp; + *hpp += PROC_BIN_SIZE; + pb->thing_word = HEADER_PROC_BIN; + pb->size = dbin->orig_size; + pb->next = ohp->first; + ohp->first = (struct erl_off_heap_header *) pb; + pb->val = ErlDrvBinary2Binary(dbin); + pb->bytes = (byte*) dbin->orig_bytes; + pb->flags = 0; + OH_OVERHEAD(ohp, dbin->orig_size / sizeof(Eterm)); + return make_binary(pb); + } + resp_bufp = dbin->orig_bytes; + resp_size = dbin->orig_size; + } + + hbin = (ErlHeapBin *) *hpp; + *hpp += heap_bin_size(resp_size); + ASSERT(resp_size <= ERL_ONHEAP_BIN_LIMIT); + hbin->thing_word = header_heap_bin(resp_size); + hbin->size = resp_size; + sys_memcpy(hbin->data, resp_bufp, resp_size); + if (dbin) + driver_free_binary(dbin); + return make_binary(hbin); + } + + /* List result */ + res = buf_to_intlist(hpp, resp_bufp, resp_size, NIL); + if (resp_bufp != pre_alloc_buf) + driver_free(resp_bufp); + return res; +} + +static int +port_sig_control(Port *prt, + erts_aint32_t state, + int op, + ErtsProc2PortSigData *sigdp) +{ + ASSERT(sigdp->flags & ERTS_P2P_SIG_DATA_FLG_REPLY); + + if (op == ERTS_PROC2PORT_SIG_EXEC) { + char resp_buf[ERL_ONHEAP_BIN_LIMIT]; + ErlDrvSizeT resp_size = sizeof(resp_buf); + char *resp_bufp = &resp_buf[0]; + ErtsPortOpResult res; + + res = call_driver_control(sigdp->caller, + prt, + sigdp->u.control.command, + sigdp->u.control.bufp, + sigdp->u.control.size, + &resp_bufp, + &resp_size); + + if (res == ERTS_PORT_OP_DONE) { + Eterm msg; + Eterm *hp, *hp_start; + ErlHeapFragment *bp; + ErlOffHeap *ohp; + Process *rp; + ErtsProcLocks rp_locks = 0; + Uint hsz; + int control_flags; + + rp = erts_proc_lookup_raw(sigdp->caller); + if (!rp) + goto done; + + control_flags = prt->control_flags; + + hsz = ERTS_QUEUE_PORT_SCHED_OP_REPLY_SIZE; + hsz += port_control_result_size(control_flags, + resp_bufp, + &resp_size, + &resp_buf[0]); + + hp_start = hp = erts_alloc_message_heap(hsz, + &bp, + &ohp, + rp, + &rp_locks); + + msg = write_port_control_result(control_flags, + resp_bufp, + resp_size, + &resp_buf[0], + &hp, + bp, + ohp); + + queue_port_sched_op_reply(rp, + &rp_locks, + hp_start, + hp, + hsz, + bp, + sigdp->ref, + msg); + + if (rp_locks) + erts_smp_proc_unlock(rp, rp_locks); + goto done; + } + } + + /* failure */ + + port_sched_op_reply(sigdp->caller, sigdp->ref, am_badarg); + +done: + + cleanup_scheduled_control(sigdp->u.control.binp, + sigdp->u.control.bufp); + + return ERTS_PORT_REDS_CONTROL; +} + + +ErtsPortOpResult +erts_port_control(Process* c_p, + Port *prt, + unsigned int command, + Eterm data, + Eterm *retvalp) +{ + ErtsPortOpResult res; + char *bufp = NULL; + ErlDrvSizeT size = 0; + int try_call; + int tmp_alloced = 0; + erts_aint32_t sched_flags; + Binary *binp; + int copy; + ErtsProc2PortSigData *sigdp; + + sched_flags = erts_smp_atomic32_read_nob(&prt->sched.flags); + if (sched_flags & ERTS_PTS_FLG_EXIT) + return ERTS_PORT_OP_BADARG; + + try_call = !(sched_flags & ERTS_PTS_FLGS_FORCE_SCHEDULE_OP); + + if (is_binary(data) && binary_bitoffset(data) == 0) { + byte *bytep; ERTS_DECLARE_DUMMY(Uint bitoffs); ERTS_DECLARE_DUMMY(Uint bitsize); - ERTS_GET_BINARY_BYTES(iolist, to_port, bitoffs, bitsize); - to_len = binary_size(iolist); + ERTS_GET_BINARY_BYTES(data, bytep, bitoffs, bitsize); + bufp = (char *) bytep; + size = binary_size(data); } else { int r; - /* Try with an 8KB buffer first (will often be enough I guess). */ - to_len = 8*1024; - to_port = erts_alloc(ERTS_ALC_T_TMP, to_len); - must_free = 1; + if (!try_call) { + if (erts_iolist_size(data, &size)) + return ERTS_PORT_OP_BADARG; + bufp = erts_alloc(ERTS_ALC_T_DRV_CTRL_DATA, size); + r = erts_iolist_to_buf(data, bufp, size); + ASSERT(r == 0); + } + else { + /* Try with an 8KB buffer first (will often be enough I guess). */ + size = 8*1024; + bufp = erts_alloc(ERTS_ALC_T_TMP, size); + tmp_alloced = 1; + + r = erts_iolist_to_buf(data, bufp, size); + if (ERTS_IOLIST_TO_BUF_SUCCEEDED(r)) { + size -= r; + } else { + if (r == ERTS_IOLIST_TO_BUF_TYPE_ERROR) { /* Type error */ + erts_free(ERTS_ALC_T_TMP, bufp); + return ERTS_PORT_OP_BADARG; + } + else { + ASSERT(r == ERTS_IOLIST_TO_BUF_OVERFLOW); /* Overflow */ + erts_free(ERTS_ALC_T_TMP, bufp); + if (erts_iolist_size(data, &size)) + return ERTS_PORT_OP_BADARG; /* Type error */ + } + bufp = erts_alloc(ERTS_ALC_T_TMP, size); + r = erts_iolist_to_buf(data, bufp, size); + ASSERT(r == 0); + } + } + } - /* - * In versions before R10B, we used to reserve random - * amounts of extra memory. From R10B, we allocate the - * exact amount. - */ - r = io_list_to_buf(iolist, (char*) to_port, to_len); - if (r >= 0) { - to_len -= r; - } else if (r == -2) { /* Type error */ - erts_free(ERTS_ALC_T_TMP, (void *) to_port); - return THE_NON_VALUE; - } else { - ASSERT(r == -1); /* Overflow */ - erts_free(ERTS_ALC_T_TMP, (void *) to_port); - if (erts_iolist_size(iolist, &to_len)) { /* Type error */ - return THE_NON_VALUE; + if (try_call) { + char resp_buf[ERL_ONHEAP_BIN_LIMIT]; + char* resp_bufp = &resp_buf[0]; + ErlDrvSizeT resp_size = sizeof(resp_buf); + ErtsTryImmDrvCallResult try_call_res; + ErtsTryImmDrvCallState try_call_state + = ERTS_INIT_TRY_IMM_DRV_CALL_STATE( + c_p, + prt, + ERTS_PORT_SFLGS_INVALID_LOOKUP, + 0, + 0, + am_control); + + try_call_res = try_imm_drv_call(&try_call_state); + switch (try_call_res) { + case ERTS_TRY_IMM_DRV_CALL_OK: { + Eterm *hp; + Uint hsz; + int control_flags; + + res = call_driver_control(c_p->common.id, + prt, + command, + bufp, + size, + &resp_bufp, + &resp_size); + finalize_imm_drv_call(&try_call_state); + if (tmp_alloced) + erts_free(ERTS_ALC_T_TMP, bufp); + if (res == ERTS_PORT_OP_BADARG) { + return ERTS_PORT_OP_BADARG; } - must_free = 1; - to_port = erts_alloc(ERTS_ALC_T_TMP, to_len); - r = io_list_to_buf(iolist, (char*) to_port, to_len); - ASSERT(r == 0); + + control_flags = prt->control_flags; + + hsz = port_control_result_size(control_flags, + resp_bufp, + &resp_size, + &resp_buf[0]); + hp = HAlloc(c_p, hsz); + *retvalp = write_port_control_result(control_flags, + resp_bufp, + resp_size, + &resp_buf[0], + &hp, + NULL, + &c_p->off_heap); + return ERTS_PORT_OP_DONE; + } + case ERTS_TRY_IMM_DRV_CALL_INVALID_PORT: + if (tmp_alloced) + erts_free(ERTS_ALC_T_TMP, bufp); + return ERTS_PORT_OP_BADARG; + default: + /* Schedule control() call instead... */ + break; } } - prt->caller = p->id; /* Internal pid */ + /* Convert data into something that can be scheduled */ - erts_smp_proc_unlock(p, ERTS_PROC_LOCK_MAIN); - ERTS_SMP_CHK_NO_PROC_LOCKS; + copy = tmp_alloced; + + binp = NULL; + + if (is_binary(data) && binary_bitoffset(data) == 0) { + Eterm *ebinp = binary_val_rel(data, NULL); + ASSERT(!tmp_alloced); + if (*ebinp == HEADER_SUB_BIN) + ebinp = binary_val_rel(((ErlSubBin *) ebinp)->orig, NULL); + if (*ebinp != HEADER_PROC_BIN) + copy = 1; + else { + binp = ((ProcBin *) ebinp)->val; + ASSERT(bufp < bufp + size); + ASSERT(binp->orig_bytes <= bufp + && bufp + size <= binp->orig_bytes + binp->orig_size); + erts_refc_inc(&binp->refc, 1); + } + } + + if (copy) { + char *old_bufp = bufp; + bufp = erts_alloc(ERTS_ALC_T_DRV_CTRL_DATA, size); + sys_memcpy(bufp, old_bufp, size); + if (tmp_alloced) + erts_free(ERTS_ALC_T_TMP, old_bufp); + } + + sigdp = erts_port_task_alloc_p2p_sig_data(); + sigdp->flags = ERTS_P2P_SIG_TYPE_CONTROL; + sigdp->u.control.binp = binp; + sigdp->u.control.command = command; + sigdp->u.control.bufp = bufp; + sigdp->u.control.size = size; + + res = erts_schedule_proc2port_signal(c_p, + prt, + c_p->common.id, + retvalp, + sigdp, + 0, + port_sig_control); + if (res != ERTS_PORT_OP_SCHEDULED) { + cleanup_scheduled_control(binp, bufp); + return ERTS_PORT_OP_BADARG; + } + return res; +} + +static ERTS_INLINE ErtsPortOpResult +call_driver_call(Eterm caller, + Port *prt, + unsigned int command, + char *bufp, + ErlDrvSizeT size, + char **resp_bufp, + ErlDrvSizeT *from_size, + unsigned *ret_flagsp) +{ + ErlDrvSSizeT cres; + + if (!prt->drv_ptr->call) + return ERTS_PORT_OP_BADARG; #ifdef USE_VM_PROBES - if (DTRACE_ENABLED(port_control) || DTRACE_ENABLED(driver_control)) { - DTRACE_FORMAT_COMMON_PROC_AND_PORT(p, prt); - DTRACE4(port_control, process_str, port_str, prt->name, command); - DTRACE5(driver_control, process_str, port_str, prt->name, - command, to_len); + if (DTRACE_ENABLED(driver_call)) { + DTRACE_CHARBUF(process_str, DTRACE_TERM_BUF_SIZE); + DTRACE_CHARBUF(port_str, DTRACE_TERM_BUF_SIZE); + + dtrace_pid_str(caller, process_str); + dtrace_port_str(prt, port_str); + DTRACE5(driver_call, process_str, port_str, prt->name, command, size); } #endif - /* - * Call the port's control routine. - */ + prt->caller = caller; + cres = prt->drv_ptr->call((ErlDrvData) prt->drv_data, + command, + bufp, + size, + resp_bufp, + *from_size, + ret_flagsp); + prt->caller = NIL; - port_resp = port_result; - fpe_was_unmasked = erts_block_fpe(); - n = control((ErlDrvData)prt->drv_data, command, (char*)to_port, to_len, - &port_resp, sizeof(port_result)); - erts_unblock_fpe(fpe_was_unmasked); - if (must_free) { - erts_free(ERTS_ALC_T_TMP, (void *) to_port); + if (cres <= 0 + || ((byte) (*resp_bufp)[0]) != VERSION_MAGIC) + return ERTS_PORT_OP_BADARG; + + *from_size = (ErlDrvSizeT) cres; + + return ERTS_PORT_OP_DONE; +} + + +static +void cleanup_scheduled_call(char *bufp) +{ + if (bufp) + erts_free(ERTS_ALC_T_DRV_CALL_DATA, bufp); +} + +static int +port_sig_call(Port *prt, + erts_aint32_t state, + int op, + ErtsProc2PortSigData *sigdp) +{ + char resp_buf[256]; + ErlDrvSizeT resp_size = sizeof(resp_buf); + char *resp_bufp = &resp_buf[0]; + unsigned ret_flags = 0U; + + + ASSERT(sigdp->flags & ERTS_P2P_SIG_DATA_FLG_REPLY); + + if (op == ERTS_PROC2PORT_SIG_EXEC) { + ErtsPortOpResult res; + + res = call_driver_call(sigdp->caller, + prt, + sigdp->u.call.command, + sigdp->u.call.bufp, + sigdp->u.call.size, + &resp_bufp, + &resp_size, + &ret_flags); + + if (res == ERTS_PORT_OP_DONE) { + Eterm msg; + Eterm *hp; + ErlHeapFragment *bp; + ErlOffHeap *ohp; + Process *rp; + ErtsProcLocks rp_locks = 0; + Uint hsz; + + rp = erts_proc_lookup_raw(sigdp->caller); + if (!rp) + goto done; + + hsz = erts_decode_ext_size((byte *) resp_bufp, resp_size); + if (hsz >= 0) { + Eterm *hp_start; + byte *endp; + + hsz += 3; /* ok tuple */ + hsz += ERTS_QUEUE_PORT_SCHED_OP_REPLY_SIZE; + + hp_start = hp = erts_alloc_message_heap(hsz, + &bp, + &ohp, + rp, + &rp_locks); + endp = (byte *) resp_bufp; + msg = erts_decode_ext(&hp, ohp, &endp); + if (is_value(msg)) { + msg = TUPLE2(hp, am_ok, msg); + hp += 3; + + queue_port_sched_op_reply(rp, + &rp_locks, + hp_start, + hp, + hsz, + bp, + sigdp->ref, + msg); + + if (rp_locks) + erts_smp_proc_unlock(rp, rp_locks); + goto done; + } + if (bp) + free_message_buffer(bp); + if (rp_locks) + erts_smp_proc_unlock(rp, rp_locks); + } + } } - prt->caller = NIL; -#ifdef ERTS_SMP - if (prt->xports) - erts_smp_xports_unlock(prt); - ASSERT(!prt->xports); -#endif - erts_smp_proc_lock(p, ERTS_PROC_LOCK_MAIN); - /* - * Handle the result. - */ + port_sched_op_reply(sigdp->caller, sigdp->ref, am_badarg); + +done: - if (n < 0) { - return THE_NON_VALUE; + if (resp_bufp != &resp_buf[0] && !(ret_flags & DRIVER_CALL_KEEP_BUFFER)) + driver_free(resp_bufp); + + cleanup_scheduled_call(sigdp->u.call.bufp); + + return ERTS_PORT_REDS_CALL; +} + + +ErtsPortOpResult +erts_port_call(Process* c_p, + Port *prt, + unsigned int command, + Eterm data, + Eterm *retvalp) +{ + ErtsPortOpResult res; + char input_buf[256]; + char *bufp; + byte *endp; + ErlDrvSizeT size; + int try_call; + erts_aint32_t sched_flags; + ErtsProc2PortSigData *sigdp; + + sched_flags = erts_smp_atomic32_read_nob(&prt->sched.flags); + if (sched_flags & ERTS_PTS_FLG_EXIT) { + return ERTS_PORT_OP_BADARG; } - if ((prt->control_flags & PORT_CONTROL_FLAG_BINARY) == 0) { /* List result */ - Eterm ret; - Eterm* hp = HAlloc(p, 2*n); - ret = buf_to_intlist(&hp, port_resp, n, NIL); - if (port_resp != port_result) { - driver_free(port_resp); + try_call = !(sched_flags & ERTS_PTS_FLGS_FORCE_SCHEDULE_OP); + + size = erts_encode_ext_size(data); + + if (!try_call) + bufp = erts_alloc(ERTS_ALC_T_DRV_CALL_DATA, size); + else if (size <= sizeof(input_buf)) + bufp = &input_buf[0]; + else + bufp = erts_alloc(ERTS_ALC_T_TMP, size); + + endp = (byte *) bufp; + erts_encode_ext(data, &endp); + + if (endp - (byte *) bufp > size) + ERTS_INTERNAL_ERROR("erts_internal:port_call() - Buffer overflow"); + + size = endp - (byte *) bufp; + + if (try_call) { + char resp_buf[255]; + char* resp_bufp = &resp_buf[0]; + ErlDrvSizeT resp_size = sizeof(resp_buf); + ErtsTryImmDrvCallResult try_call_res; + ErtsTryImmDrvCallState try_call_state + = ERTS_INIT_TRY_IMM_DRV_CALL_STATE( + c_p, + prt, + ERTS_PORT_SFLGS_INVALID_LOOKUP, + 0, + 0, + am_call); + + try_call_res = try_imm_drv_call(&try_call_state); + switch (try_call_res) { + case ERTS_TRY_IMM_DRV_CALL_OK: { + Eterm *hp, *hp_end; + Uint hsz; + unsigned ret_flags = 0U; + Eterm term; + + res = call_driver_call(c_p->common.id, + prt, + command, + bufp, + size, + &resp_bufp, + &resp_size, + &ret_flags); + + finalize_imm_drv_call(&try_call_state); + if (bufp != &input_buf[0]) + erts_free(ERTS_ALC_T_TMP, bufp); + if (res == ERTS_PORT_OP_BADARG) + return ERTS_PORT_OP_BADARG; + hsz = erts_decode_ext_size((byte *) resp_bufp, resp_size); + if (hsz < 0) + return ERTS_PORT_OP_BADARG; + hsz += 3; + hp = HAlloc(c_p, hsz); + hp_end = hp + hsz; + endp = (byte *) resp_bufp; + term = erts_decode_ext(&hp, &MSO(c_p), &endp); + if (term == THE_NON_VALUE) + return ERTS_PORT_OP_BADARG; + *retvalp = TUPLE2(hp, am_ok, term); + hp += 3; + HRelease(c_p, hp_end, hp); + if (resp_buf != &resp_buf[0] + && !(ret_flags & DRIVER_CALL_KEEP_BUFFER)) + driver_free(resp_buf); + return ERTS_PORT_OP_DONE; } - return ret; - } - else if (port_resp == NULL) { - return NIL; - } - else { /* Binary result */ - ErlDrvBinary *dbin; - ErlHeapBin *hbin; - if (port_resp != port_result) { - dbin = (ErlDrvBinary *) port_resp; - if (dbin->orig_size > ERL_ONHEAP_BIN_LIMIT) { - ProcBin* pb = (ProcBin *) HAlloc(p, PROC_BIN_SIZE); - pb->thing_word = HEADER_PROC_BIN; - pb->size = dbin->orig_size; - pb->next = MSO(p).first; - MSO(p).first = (struct erl_off_heap_header*)pb; - pb->val = ErlDrvBinary2Binary(dbin); - pb->bytes = (byte*) dbin->orig_bytes; - pb->flags = 0; - OH_OVERHEAD(&(MSO(p)), dbin->orig_size / sizeof(Eterm)); - return make_binary(pb); - } - port_resp = dbin->orig_bytes; - n = dbin->orig_size; - } else { - dbin = NULL; + case ERTS_TRY_IMM_DRV_CALL_INVALID_PORT: + if (bufp != &input_buf[0]) + erts_free(ERTS_ALC_T_TMP, bufp); + return ERTS_PORT_OP_BADARG; + default: + /* Schedule call() call instead... */ + break; } - hbin = (ErlHeapBin*) HAlloc(p, heap_bin_size(n)); - ASSERT(n <= ERL_ONHEAP_BIN_LIMIT); - hbin->thing_word = header_heap_bin(n); - hbin->size = n; - sys_memcpy(hbin->data, port_resp, n); - if (dbin != NULL) { - driver_free_binary(dbin); + } + + /* Convert data into something that can be scheduled */ + + if (bufp == &input_buf[0] || try_call) { + char *new_bufp = erts_alloc(ERTS_ALC_T_DRV_CALL_DATA, size); + sys_memcpy(new_bufp, bufp, size); + if (bufp != &input_buf[0]) + erts_free(ERTS_ALC_T_TMP, bufp); + bufp = new_bufp; + } + + sigdp = erts_port_task_alloc_p2p_sig_data(); + sigdp->flags = ERTS_P2P_SIG_TYPE_CALL; + sigdp->u.call.command = command; + sigdp->u.call.bufp = bufp; + sigdp->u.call.size = size; + + res = erts_schedule_proc2port_signal(c_p, + prt, + c_p->common.id, + retvalp, + sigdp, + 0, + port_sig_call); + if (res != ERTS_PORT_OP_SCHEDULED) { + cleanup_scheduled_call(bufp); + return ERTS_PORT_OP_BADARG; + } + return res; +} + +static Eterm +make_port_info_term(Eterm **hpp_start, + Eterm **hpp, + Uint *hszp, + ErlHeapFragment **bpp, + Port *prt, + Eterm item) +{ + ErlOffHeap *ohp; + + if (is_value(item)) { + if (erts_bld_port_info(NULL, NULL, hszp, prt, item) == am_false) + return THE_NON_VALUE; + if (*hszp) { + *bpp = new_message_buffer(*hszp); + *hpp_start = *hpp = (*bpp)->mem; + ohp = &(*bpp)->off_heap; } - return make_binary(hbin); + else { + *bpp = NULL; + *hpp_start = *hpp = NULL; + ohp = NULL; + } + return erts_bld_port_info(hpp, ohp, NULL, prt, item); + } + else { + int i; + int len; + int start; + static Eterm item[] = ERTS_PORT_INFO_1_ITEMS; + static Eterm value[sizeof(item)/sizeof(item[0])]; + + start = 0; + len = sizeof(item)/sizeof(item[0]); + + for (i = start; i < sizeof(item)/sizeof(item[0]); i++) { + ASSERT(is_atom(item[i])); + value[i] = erts_bld_port_info(NULL, NULL, hszp, prt, item[i]); + } + + if (value[0] == am_undefined) { + start++; + len--; + } + + erts_bld_list(NULL, hszp, len, &value[start]); + + *bpp = new_message_buffer(*hszp); + *hpp_start = *hpp = (*bpp)->mem; + ohp = &(*bpp)->off_heap; + + for (i = start; i < sizeof(item)/sizeof(item[0]); i++) + value[i] = erts_bld_port_info(hpp, ohp, NULL, prt, item[i]); + + return erts_bld_list(hpp, NULL, len, &value[start]); + } +} + +static int +port_sig_info(Port *prt, + erts_aint32_t state, + int op, + ErtsProc2PortSigData *sigdp) +{ + ASSERT(sigdp->flags & ERTS_P2P_SIG_DATA_FLG_REPLY); + if (op != ERTS_PROC2PORT_SIG_EXEC) + port_sched_op_reply(sigdp->caller, sigdp->ref, am_undefined); + else { + Eterm *hp, *hp_start; + Uint hsz; + ErlHeapFragment *bp; + Eterm value; + Process *rp; + ErtsProcLocks rp_locks = 0; + + rp = erts_proc_lookup_raw(sigdp->caller); + if (!rp) + return ERTS_PORT_REDS_INFO; + + hsz = ERTS_QUEUE_PORT_SCHED_OP_REPLY_SIZE; + value = make_port_info_term(&hp_start, + &hp, + &hsz, + &bp, + prt, + sigdp->u.info.item); + if (is_value(value)) { + queue_port_sched_op_reply(rp, + &rp_locks, + hp_start, + hp, + hsz, + bp, + sigdp->ref, + value); + } + if (rp_locks) + erts_smp_proc_unlock(rp, rp_locks); + } + return ERTS_PORT_REDS_INFO; +} + +ErtsPortOpResult +erts_port_info(Process* c_p, + Port *prt, + Eterm item, + Eterm *retvalp) +{ + ErtsProc2PortSigData *sigdp; + ErtsTryImmDrvCallResult try_call_res; + ErtsTryImmDrvCallState try_call_state + = ERTS_INIT_TRY_IMM_DRV_CALL_STATE( + c_p, + prt, + ERTS_PORT_SFLGS_INVALID_LOOKUP, + 0, + 0, + am_info); + + try_call_res = try_imm_drv_call(&try_call_state); + switch (try_call_res) { + case ERTS_TRY_IMM_DRV_CALL_OK: { + Eterm *hp, *hp_start; + ErlHeapFragment *bp; + Uint hsz = 0; + Eterm value = make_port_info_term(&hp_start, &hp, &hsz, &bp, prt, item); + finalize_imm_drv_call(&try_call_state); + if (is_non_value(value)) + return ERTS_PORT_OP_BADARG; + else if (is_immed(value)) + *retvalp = value; + else { + Uint used_h_size = hp - hp_start; + hp = HAlloc(c_p, used_h_size); + *retvalp = copy_struct(value, used_h_size, &hp, &MSO(c_p)); + free_message_buffer(bp); + } + return ERTS_PORT_OP_DONE; + } + case ERTS_TRY_IMM_DRV_CALL_INVALID_PORT: + return ERTS_PORT_OP_DROPPED; + case ERTS_TRY_IMM_DRV_CALL_INVALID_SCHED_FLAGS: + case ERTS_TRY_IMM_DRV_CALL_BUSY_LOCK: + /* Schedule call instead... */ + break; + } + + sigdp = erts_port_task_alloc_p2p_sig_data(); + sigdp->flags = ERTS_P2P_SIG_TYPE_INFO; + sigdp->u.info.item = item; + + return erts_schedule_proc2port_signal(c_p, + prt, + c_p->common.id, + retvalp, + sigdp, + 0, + port_sig_info); +} + +static int +port_sig_set_data(Port *prt, + erts_aint32_t state, + int op, + ErtsProc2PortSigData *sigdp) +{ + ASSERT(sigdp->flags & ERTS_P2P_SIG_DATA_FLG_REPLY); + + if (op == ERTS_PROC2PORT_SIG_EXEC) { + if (prt->bp) + free_message_buffer(prt->bp); + prt->bp = sigdp->u.set_data.bp; + prt->data = sigdp->u.set_data.data; + port_sched_op_reply(sigdp->caller, sigdp->ref, am_true); + } + else { + if (sigdp->u.set_data.bp) + free_message_buffer(sigdp->u.set_data.bp); + port_sched_op_reply(sigdp->caller, sigdp->ref, am_badarg); + } + return ERTS_PORT_REDS_SET_DATA; +} + +ErtsPortOpResult +erts_port_set_data(Process* c_p, + Port *prt, + Eterm data, + Eterm *refp) +{ + ErtsPortOpResult res; + Eterm set_data; + ErlHeapFragment *bp; + ErtsProc2PortSigData *sigdp; + ErtsTryImmDrvCallResult try_call_res; + ErtsTryImmDrvCallState try_call_state + = ERTS_INIT_TRY_IMM_DRV_CALL_STATE( + c_p, + prt, + ERTS_PORT_SFLGS_INVALID_LOOKUP, + 0, + !refp, + am_set_data); + + if (is_immed(data)) { + set_data = data; + bp = NULL; + } + else { + Eterm *hp; + Uint sz = size_object(data); + bp = new_message_buffer(sz); + hp = bp->mem; + set_data = copy_struct(data, sz, &hp, &bp->off_heap); + } + + try_call_res = try_imm_drv_call(&try_call_state); + switch (try_call_res) { + case ERTS_TRY_IMM_DRV_CALL_OK: + if (prt->bp) + free_message_buffer(prt->bp); + prt->bp = bp; + prt->data = set_data; + finalize_imm_drv_call(&try_call_state); + return ERTS_PORT_OP_DONE; + case ERTS_TRY_IMM_DRV_CALL_INVALID_PORT: + return ERTS_PORT_OP_DROPPED; + case ERTS_TRY_IMM_DRV_CALL_INVALID_SCHED_FLAGS: + case ERTS_TRY_IMM_DRV_CALL_BUSY_LOCK: + /* Schedule call instead... */ + break; + } + + sigdp = erts_port_task_alloc_p2p_sig_data(); + sigdp->flags = ERTS_P2P_SIG_TYPE_SET_DATA; + sigdp->u.set_data.data = set_data; + sigdp->u.set_data.bp = bp; + + res = erts_schedule_proc2port_signal(c_p, + prt, + c_p->common.id, + refp, + sigdp, + 0, + port_sig_set_data); + if (res != ERTS_PORT_OP_SCHEDULED && bp) + free_message_buffer(bp); + return res; +} + +static int +port_sig_get_data(Port *prt, + erts_aint32_t state, + int op, + ErtsProc2PortSigData *sigdp) +{ + ASSERT(sigdp->flags & ERTS_P2P_SIG_DATA_FLG_REPLY); + if (op != ERTS_PROC2PORT_SIG_EXEC) + port_sched_op_reply(sigdp->caller, sigdp->ref, am_badarg); + else { + Process *rp; + ErtsProcLocks rp_locks = 0; + + rp = erts_proc_lookup_raw(sigdp->caller); + if (rp) { + Uint hsz; + Eterm *hp, *hp_start; + Eterm data, msg; + ErlHeapFragment *bp; + ErlOffHeap *ohp; + + hsz = ERTS_QUEUE_PORT_SCHED_OP_REPLY_SIZE; + hsz += 3; + if (prt->bp) + hsz += prt->bp->used_size; + + hp_start = hp = erts_alloc_message_heap(hsz, + &bp, + &ohp, + rp, + &rp_locks); + + if (is_immed(prt->data)) + data = prt->data; + else + data = copy_struct(prt->data, + prt->bp->used_size, + &hp, + &bp->off_heap); + + + + msg = TUPLE2(hp, am_ok, data); + hp += 3; + + queue_port_sched_op_reply(rp, + &rp_locks, + hp_start, + hp, + hsz, + bp, + sigdp->ref, + msg); + if (rp_locks) + erts_smp_proc_unlock(rp, rp_locks); + } + } + return ERTS_PORT_REDS_GET_DATA; +} + +ErtsPortOpResult +erts_port_get_data(Process* c_p, + Port *prt, + Eterm *retvalp) +{ + ErtsProc2PortSigData *sigdp; + ErtsTryImmDrvCallResult try_call_res; + ErtsTryImmDrvCallState try_call_state + = ERTS_INIT_TRY_IMM_DRV_CALL_STATE( + c_p, + prt, + ERTS_PORT_SFLGS_INVALID_LOOKUP, + 0, + 0, + am_get_data); + + try_call_res = try_imm_drv_call(&try_call_state); + switch (try_call_res) { + case ERTS_TRY_IMM_DRV_CALL_OK: { + Eterm *hp; + Eterm data; + ErlHeapFragment *bp; + Uint sz; + if (is_immed(prt->data)) { + bp = NULL; + data = prt->data; + } + else { + bp = new_message_buffer(prt->bp->used_size); + data = copy_struct(prt->data, + prt->bp->used_size, + &hp, + &bp->off_heap); + } + finalize_imm_drv_call(&try_call_state); + if (is_immed(data)) + sz = 0; + else + sz = bp->used_size; + + hp = HAlloc(c_p, sz + 3); + if (is_not_immed(data)) { + data = copy_struct(data, bp->used_size, &hp, &MSO(c_p)); + free_message_buffer(bp); + } + *retvalp = TUPLE2(hp, am_ok, data); + return ERTS_PORT_OP_DONE; + } + case ERTS_TRY_IMM_DRV_CALL_INVALID_PORT: + return ERTS_PORT_OP_DROPPED; + case ERTS_TRY_IMM_DRV_CALL_INVALID_SCHED_FLAGS: + case ERTS_TRY_IMM_DRV_CALL_BUSY_LOCK: + /* Schedule call instead... */ + break; } + + sigdp = erts_port_task_alloc_p2p_sig_data(); + sigdp->flags = ERTS_P2P_SIG_TYPE_GET_DATA; + + return erts_schedule_proc2port_signal(c_p, + prt, + c_p->common.id, + retvalp, + sigdp, + 0, + port_sig_get_data); } typedef struct { @@ -2480,39 +4681,39 @@ static void prt_one_lnk(ErtsLink *lnk, void *vprtd) } void -print_port_info(int to, void *arg, int i) +print_port_info(Port *p, int to, void *arg) { - Port* p = &erts_port[i]; + erts_aint32_t state = erts_atomic32_read_nob(&p->state); - if (p->status & ERTS_PORT_SFLGS_DEAD) + if (state & ERTS_PORT_SFLGS_DEAD) return; - erts_print(to, arg, "=port:%T\n", p->id); - erts_print(to, arg, "Slot: %d\n", i); - if (p->status & ERTS_PORT_SFLG_CONNECTED) { - erts_print(to, arg, "Connected: %T", p->connected); + erts_print(to, arg, "=port:%T\n", p->common.id); + erts_print(to, arg, "Slot: %d\n", internal_port_index(p->common.id)); + if (state & ERTS_PORT_SFLG_CONNECTED) { + erts_print(to, arg, "Connected: %T", ERTS_PORT_GET_CONNECTED(p)); erts_print(to, arg, "\n"); } - if (p->nlinks != NULL) { + if (ERTS_P_LINKS(p)) { prt_one_lnk_data prtd; prtd.to = to; prtd.arg = arg; erts_print(to, arg, "Links: "); - erts_doforall_links(p->nlinks, &prt_one_lnk, &prtd); + erts_doforall_links(ERTS_P_LINKS(p), &prt_one_lnk, &prtd); erts_print(to, arg, "\n"); } - if (p->monitors != NULL) { + if (ERTS_P_MONITORS(p)) { prt_one_lnk_data prtd; prtd.to = to; prtd.arg = arg; erts_print(to, arg, "Monitors: "); - erts_doforall_monitors(p->monitors, &prt_one_monitor, &prtd); + erts_doforall_monitors(ERTS_P_MONITORS(p), &prt_one_monitor, &prtd); erts_print(to, arg, "\n"); } - if (p->reg != NULL) - erts_print(to, arg, "Registered as: %T\n", p->reg->name); + if (p->common.u.alive.reg != NULL) + erts_print(to, arg, "Registered as: %T\n", p->common.u.alive.reg->name); if (p->drv_ptr == &fd_driver) { erts_print(to, arg, "Port is UNIX fd not opened by emulator: %s\n", p->name); @@ -2526,109 +4727,143 @@ print_port_info(int to, void *arg, int i) } void -set_busy_port(ErlDrvPort port_num, int on) +set_busy_port(ErlDrvPort dprt, int on) { + Port *prt; + erts_aint32_t flags; + #ifdef USE_VM_PROBES DTRACE_CHARBUF(port_str, 16); #endif ERTS_SMP_CHK_NO_PROC_LOCKS; - ERTS_SMP_LC_ASSERT(erts_lc_is_port_locked(&erts_port[port_num])); + prt = erts_drvport2port_raw(dprt); + if (!prt) + return; if (on) { - erts_port_status_bor_set(&erts_port[port_num], - ERTS_PORT_SFLG_PORT_BUSY); + flags = erts_smp_atomic32_read_bor_acqb(&prt->sched.flags, + ERTS_PTS_FLG_BUSY_PORT); + if (flags & ERTS_PTS_FLG_BUSY_PORT) + return; /* Already busy */ + + if (flags & ERTS_PTS_FLG_HAVE_NS_TASKS) + erts_port_task_abort_nosuspend_tasks(prt); + #ifdef USE_VM_PROBES if (DTRACE_ENABLED(port_busy)) { erts_snprintf(port_str, sizeof(port_str), - "%T", erts_port[port_num].id); + "%T", prt->common.id); DTRACE1(port_busy, port_str); } #endif } else { - ErtsProcList* plp = erts_port[port_num].suspended; - erts_port_status_band_set(&erts_port[port_num], - ~ERTS_PORT_SFLG_PORT_BUSY); - erts_port[port_num].suspended = NULL; + flags = erts_smp_atomic32_read_band_acqb(&prt->sched.flags, + ~ERTS_PTS_FLG_BUSY_PORT); + if (!(flags & ERTS_PTS_FLG_BUSY_PORT)) + return; /* Already non-busy */ #ifdef USE_VM_PROBES if (DTRACE_ENABLED(port_not_busy)) { erts_snprintf(port_str, sizeof(port_str), - "%T", erts_port[port_num].id); + "%T", prt->common.id); DTRACE1(port_not_busy, port_str); } #endif - if (erts_port[port_num].dist_entry) { + if (prt->dist_entry) { /* * Processes suspended on distribution ports are * normally queued on the dist entry. */ - erts_dist_port_not_busy(&erts_port[port_num]); + erts_dist_port_not_busy(prt); } - /* - * Resume, in a round-robin fashion, all processes waiting on the port. - * - * This version submitted by Tony Rogvall. The earlier version used - * to resume the processes in order, which caused starvation of all but - * the first process. - */ + if (!(flags & ERTS_PTS_FLG_BUSY_PORT_Q)) + erts_port_resume_procs(prt); + } +} + +void +erts_port_resume_procs(Port *prt) +{ + /* + * Resume, in a round-robin fashion, all processes waiting on the port. + * + * This version submitted by Tony Rogvall. The earlier version used + * to resume the processes in order, which caused starvation of all but + * the first process. + */ + ErtsProcList *plp; + + erts_port_task_sched_lock(&prt->sched); + plp = prt->suspended; + prt->suspended = NULL; + erts_port_task_sched_unlock(&prt->sched); + + if (erts_proclist_fetch(&plp, NULL)) { - if (plp) { #ifdef USE_VM_PROBES - /* - * Hrm, for blocked dist ports, plp always seems to be NULL. - * That's not so fun. - * Well, another way to get the same info is using a D - * script to correlate an earlier process-port_blocked+pid - * event with a later process-scheduled event. That's - * subject to the multi-CPU races with how events are - * handled, but hey, that way works most of the time. - */ - if (DTRACE_ENABLED(process_port_unblocked)) { - DTRACE_CHARBUF(pid_str, 16); - ErtsProcList* plp2 = plp; - - erts_snprintf(port_str, sizeof(port_str), - "%T", erts_port[port_num]); - while (plp2 != NULL) { - erts_snprintf(pid_str, sizeof(pid_str), "%T", plp2->pid); - DTRACE2(process_port_unblocked, pid_str, port_str); - } - } -#endif - /* First proc should be resumed last */ - if (plp->next) { - erts_resume_processes(plp->next); - plp->next = NULL; + /* + * Hrm, for blocked dist ports, plp always seems to be NULL. + * That's not so fun. + * Well, another way to get the same info is using a D + * script to correlate an earlier process-port_blocked+pid + * event with a later process-scheduled event. That's + * subject to the multi-CPU races with how events are + * handled, but hey, that way works most of the time. + */ + if (DTRACE_ENABLED(process_port_unblocked)) { + DTRACE_CHARBUF(port_str, 16); + DTRACE_CHARBUF(pid_str, 16); + ErtsProcList* plp2 = plp; + + erts_snprintf(port_str, sizeof(port_str), "%T", prt->common.id); + while (plp2 != NULL) { + erts_snprintf(pid_str, sizeof(pid_str), "%T", plp2->pid); + DTRACE2(process_port_unblocked, pid_str, port_str); } - erts_resume_processes(plp); - } + } +#endif + + /* First proc should be resumed last */ + if (plp->next) { + plp->next->prev = NULL; + erts_resume_processes(plp->next); + plp->next = NULL; + } + erts_resume_processes(plp); } } void set_port_control_flags(ErlDrvPort port_num, int flags) { - - ERTS_SMP_LC_ASSERT(erts_lc_is_port_locked(&erts_port[port_num])); - - erts_port[port_num].control_flags = flags; + Port *prt = erts_drvport2port_raw(port_num); + if (prt) + prt->control_flags = flags; } -int get_port_flags(ErlDrvPort ix) { - Port* prt = erts_drvport2port(ix); +int get_port_flags(ErlDrvPort ix) +{ + int flags; + Port *prt; + erts_aint32_t state; + + prt = erts_drvport2port(ix, &state); + if (!prt) + return 0; ERTS_SMP_LC_ASSERT(erts_lc_is_port_locked(prt)); - if (prt == NULL) - return 0; + flags = 0; + if (state & ERTS_PORT_SFLG_BINARY_IO) + flags |= PORT_FLAG_BINARY; + if (state & ERTS_PORT_SFLG_LINEBUF_IO) + flags |= PORT_FLAG_LINE; - return (prt->status & ERTS_PORT_SFLG_BINARY_IO ? PORT_FLAG_BINARY : 0) - | (prt->status & ERTS_PORT_SFLG_LINEBUF_IO ? PORT_FLAG_LINE : 0); + return flags; } - void erts_raw_port_command(Port* p, byte* buf, Uint len) { int fpe_was_unmasked; @@ -2665,25 +4900,18 @@ int async_ready(Port *p, void* data) if (p) { ERTS_SMP_LC_ASSERT(erts_lc_is_port_locked(p)); - ASSERT(!(p->status & ERTS_PORT_SFLGS_DEAD)); if (p->drv_ptr->ready_async != NULL) { #ifdef USE_VM_PROBES if (DTRACE_ENABLED(driver_ready_async)) { - DTRACE_FORMAT_COMMON_PID_AND_PORT(p->connected, p) + DTRACE_FORMAT_COMMON_PID_AND_PORT(ERTS_PORT_GET_CONNECTED(p), p) DTRACE3(driver_ready_async, process_str, port_str, p->name); } #endif (*p->drv_ptr->ready_async)((ErlDrvData)p->drv_data, data); need_free = 0; -#ifdef ERTS_SMP - if (p->xports) - erts_smp_xports_unlock(p); - ASSERT(!p->xports); -#endif - } - if ((p->status & ERTS_PORT_SFLG_CLOSING) && is_port_ioq_empty(p)) { - terminate_port(p); + } + erts_port_driver_callback_epilogue(p, NULL); } return need_free; } @@ -2691,12 +4919,12 @@ int async_ready(Port *p, void* data) static void report_missing_drv_callback(Port *p, char *drv_type, char *callback) { - ErtsPortNames *pnp = erts_get_port_names(p->id); + ErtsPortNames *pnp = erts_get_port_names(p->common.id); char *unknown = "<unknown>"; char *drv_name = pnp->driver_name ? pnp->driver_name : unknown; char *prt_name = pnp->name ? pnp->name : unknown; erts_dsprintf_buf_t *dsbufp = erts_create_logger_dsbuf(); - erts_dsprintf(dsbufp, "%T: %s driver '%s' ", p->id, drv_type, drv_name); + erts_dsprintf(dsbufp, "%T: %s driver '%s' ", p->common.id, drv_type, drv_name); if (sys_strcmp(drv_name, prt_name) != 0) erts_dsprintf(dsbufp, "(%s) ", prt_name); erts_dsprintf(dsbufp, "does not implement the %s callback!\n", callback); @@ -2711,7 +4939,7 @@ erts_stale_drv_select(Eterm port, int deselect) { char *type; - ErlDrvPort drv_port = internal_port_index(port); + ErlDrvPort drv_port = (ErlDrvPort) erts_port_lookup_raw(port); ErtsPortNames *pnp = erts_get_port_names(port); erts_dsprintf_buf_t *dsbufp; @@ -2751,16 +4979,16 @@ erts_stale_drv_select(Eterm port, ErtsPortNames * erts_get_port_names(Eterm id) { + Port *prt = erts_port_lookup_raw(id); ErtsPortNames *pnp; ASSERT(is_nil(id) || is_internal_port(id)); - - if (is_not_internal_port(id)) { + + if (!prt) { pnp = erts_alloc(ERTS_ALC_T_PORT_NAMES, sizeof(ErtsPortNames)); pnp->name = NULL; pnp->driver_name = NULL; } else { - Port* prt = &erts_port[internal_port_index(id)]; int do_realloc = 1; int len = -1; size_t pnp_len = sizeof(ErtsPortNames); @@ -2776,17 +5004,10 @@ erts_get_port_names(Eterm id) pnp_len = sizeof(ErtsPortNames) + len; pnp = erts_alloc(ERTS_ALC_T_PORT_NAMES, pnp_len); } - erts_smp_port_state_lock(prt); - if (id != prt->id) { - len = nlen = 0; - name = driver_name = NULL; - } - else { - name = prt->name; - len = nlen = name ? sys_strlen(name) + 1 : 0; - driver_name = (prt->drv_ptr ? prt->drv_ptr->name : NULL); - len += driver_name ? sys_strlen(driver_name) + 1 : 0; - } + name = prt->name; + len = nlen = name ? sys_strlen(name) + 1 : 0; + driver_name = (prt->drv_ptr ? prt->drv_ptr->name : NULL); + len += driver_name ? sys_strlen(driver_name) + 1 : 0; if (len <= pnp_len - sizeof(ErtsPortNames)) { if (!name) pnp->name = NULL; @@ -2804,7 +5025,6 @@ erts_get_port_names(Eterm id) } do_realloc = 0; } - erts_smp_port_state_unlock(prt); } while (do_realloc); } return pnp; @@ -2829,11 +5049,9 @@ static void schedule_port_timeout(Port *p) * /Rickard */ ERTS_SMP_LC_ASSERT(erts_lc_is_port_locked(p)); - (void) erts_port_task_schedule(p->id, - &p->timeout_task, - ERTS_PORT_TASK_TIMEOUT, - (ErlDrvEvent) -1, - NULL); + erts_port_task_schedule(p->common.id, + &p->timeout_task, + ERTS_PORT_TASK_TIMEOUT); } ErlDrvTermData driver_mk_term_nil(void) @@ -2841,9 +5059,9 @@ ErlDrvTermData driver_mk_term_nil(void) return driver_term_nil; } -void driver_report_exit(int ix, int status) +void driver_report_exit(ErlDrvPort ix, int status) { - Port* prt = erts_drvport2port(ix); + Port* prt = erts_drvport2port(ix, NULL); Eterm* hp; Eterm tuple; Process *rp; @@ -2856,7 +5074,7 @@ void driver_report_exit(int ix, int status) ERTS_SMP_CHK_NO_PROC_LOCKS; ERTS_SMP_LC_ASSERT(erts_lc_is_port_locked(prt)); - pid = prt->connected; + pid = ERTS_PORT_GET_CONNECTED(prt); ASSERT(is_internal_pid(pid)); rp = (scheduler @@ -2869,7 +5087,7 @@ void driver_report_exit(int ix, int status) tuple = TUPLE2(hp, am_exit_status, make_small(status)); hp += 3; - tuple = TUPLE2(hp, prt->id, tuple); + tuple = TUPLE2(hp, prt->common.id, tuple); erts_queue_message(rp, &rp_locks, bp, tuple, am_undefined #ifdef USE_VM_PROBES @@ -2882,28 +5100,6 @@ void driver_report_exit(int ix, int status) erts_smp_proc_dec_refc(rp); } - -static ERTS_INLINE int -deliver_term_check_port(ErlDrvPort drvport) -{ - int res; - int ix = (int) drvport; - if (ix < 0 || erts_max_ports <= ix) - res = -1; /* invalid */ - else { - Port* prt = &erts_port[ix]; - erts_smp_port_state_lock(prt); - if (!(prt->status & ERTS_PORT_SFLGS_INVALID_LOOKUP)) - res = 1; /* ok */ - else if (prt->status & ERTS_PORT_SFLG_CLOSING) - res = 0; /* closing */ - else - res = -1; /* invalid (dead) */ - erts_smp_port_state_unlock(prt); - } - return res; -} - #define ERTS_B2T_STATES_DEF_STATES_SZ 5 #define ERTS_B2T_STATES_DEF_STATES_INC 100 @@ -2991,10 +5187,7 @@ cleanup_b2t_states(struct b2t_states__ *b2tsp) */ static int -driver_deliver_term(ErlDrvPort port, - Eterm to, - ErlDrvTermData* data, - int len) +driver_deliver_term(Eterm to, ErlDrvTermData* data, int len) { #define ERTS_DDT_FAIL do { res = -1; goto done; } while (0) Uint need = 0; @@ -3196,11 +5389,8 @@ driver_deliver_term(ErlDrvPort port, b2t.ix = 0; /* - * The term is OK. Go ahead and validate the port and process. + * The term is OK. Go ahead and validate the process. */ - res = deliver_term_check_port(port); - if (res <= 0) - goto done; /* * Increase refc on proc if done from a non-scheduler thread. @@ -3470,25 +5660,115 @@ driver_deliver_term(ErlDrvPort port, #undef ERTS_DDT_FAIL } +static ERTS_INLINE int +deliver_term_check_port(ErlDrvTermData port_id, Eterm *connected_p) +{ +#ifdef ERTS_SMP + ErtsThrPrgrDelayHandle dhndl = erts_thr_progress_unmanaged_delay(); +#endif + Port *prt = erts_port_lookup_raw((Eterm) port_id); + erts_aint32_t state = erts_atomic32_read_nob(&prt->state); + if (connected_p) { +#ifdef ERTS_SMP + if (dhndl != ERTS_THR_PRGR_DHANDLE_MANAGED) + ETHR_MEMBAR(ETHR_LoadLoad); +#endif + *connected_p = ERTS_PORT_GET_CONNECTED(prt); + } +#ifdef ERTS_SMP + if (dhndl != ERTS_THR_PRGR_DHANDLE_MANAGED) { + erts_thr_progress_unmanaged_continue(dhndl); + ETHR_MEMBAR(ETHR_LoadLoad|ETHR_LoadStore); + } +#endif + ERTS_SMP_LC_ASSERT(dhndl == ERTS_THR_PRGR_DHANDLE_MANAGED + ? erts_lc_is_port_locked(prt) + : !erts_lc_is_port_locked(prt)); + return ((state & ERTS_PORT_SFLGS_INVALID_DRIVER_LOOKUP) + ? -1 + : ((state & ERTS_PORT_SFLG_CLOSING) ? 0 : 1)); +} + +int erl_drv_output_term(ErlDrvTermData port_id, ErlDrvTermData* data, int len) +{ + /* May be called from arbitrary thread */ + Eterm connected; + int res = deliver_term_check_port(port_id, &connected); + if (res <= 0) + return res; + return driver_deliver_term(connected, data, len); +} +/* + * driver_output_term() is deprecated, and has been scheduled for + * removal in OTP-R17. It is replaced by erl_drv_output_term() + * above. + */ int -driver_output_term(ErlDrvPort ix, ErlDrvTermData* data, int len) +driver_output_term(ErlDrvPort drvport, ErlDrvTermData* data, int len) { - Port* prt = erts_drvport2port(ix); + erts_aint32_t state; + Port* prt; ERTS_SMP_CHK_NO_PROC_LOCKS; + /* NOTE! It *not* safe to access 'drvport' from unmanaged threads. */ + prt = erts_drvport2port(drvport, &state); + if (!prt) + return -1; /* invalid (dead) */ + ERTS_SMP_CHK_NO_PROC_LOCKS; ERTS_SMP_LC_ASSERT(erts_lc_is_port_locked(prt)); - - if (prt == NULL) + if (state & ERTS_PORT_SFLGS_INVALID_DRIVER_LOOKUP) return -1; - return driver_deliver_term(ix, prt->connected, data, len); + else if (state & ERTS_PORT_SFLG_CLOSING) + return 0; + + return driver_deliver_term(ERTS_PORT_GET_CONNECTED(prt), data, len); } +int erl_drv_send_term(ErlDrvTermData port_id, + ErlDrvTermData to, + ErlDrvTermData* data, + int len) +{ + /* May be called from arbitrary thread */ + int res = deliver_term_check_port(port_id, NULL); + if (res <= 0) + return res; + return driver_deliver_term(to, data, len); +} +/* + * driver_send_term() is deprecated, and has been scheduled for + * removal in OTP-R17. It is replaced by erl_drv_send_term() above. + */ int -driver_send_term(ErlDrvPort ix, ErlDrvTermData to, ErlDrvTermData* data, int len) +driver_send_term(ErlDrvPort drvport, + ErlDrvTermData to, + ErlDrvTermData* data, + int len) { - return driver_deliver_term(ix, to, data, len); + /* + * NOTE! It is *not* safe to access the 'drvport' parameter + * from unmanaged threads. Also note that it is impossible + * to make this access safe without using a less efficient + * internal data representation for ErlDrvPort. + */ + ERTS_SMP_CHK_NO_PROC_LOCKS; +#ifdef ERTS_SMP + if (erts_thr_progress_is_managed_thread()) +#endif + { + erts_aint32_t state; + Port* prt = erts_drvport2port(drvport, &state); + if (!prt) + return -1; /* invalid (dead) */ + ERTS_SMP_LC_ASSERT(erts_lc_is_port_locked(prt)); + if (state & ERTS_PORT_SFLGS_INVALID_DRIVER_LOOKUP) + return -1; + else if (state & ERTS_PORT_SFLG_CLOSING) + return 0; + } + return driver_deliver_term(to, data, len); } @@ -3500,26 +5780,27 @@ driver_send_term(ErlDrvPort ix, ErlDrvTermData to, ErlDrvTermData* data, int len int driver_output_binary(ErlDrvPort ix, char* hbuf, ErlDrvSizeT hlen, ErlDrvBinary* bin, ErlDrvSizeT offs, ErlDrvSizeT len) { - Port* prt = erts_drvport2port(ix); + erts_aint32_t state; + Port* prt = erts_drvport2port(ix, &state); ERTS_SMP_CHK_NO_PROC_LOCKS; if (prt == NULL) return -1; ERTS_SMP_LC_ASSERT(erts_lc_is_port_locked(prt)); - if (prt->status & ERTS_PORT_SFLG_CLOSING) + if (state & ERTS_PORT_SFLG_CLOSING) return 0; prt->bytes_in += (hlen + len); erts_smp_atomic_add_nob(&erts_bytes_in, (erts_aint_t) (hlen + len)); - if (prt->status & ERTS_PORT_SFLG_DISTRIBUTION) { + if (state & ERTS_PORT_SFLG_DISTRIBUTION) { return erts_net_message(prt, prt->dist_entry, (byte*) hbuf, hlen, (byte*) (bin->orig_bytes+offs), len); } else - deliver_bin_message(prt, prt->connected, + deliver_bin_message(prt, ERTS_PORT_GET_CONNECTED(prt), hbuf, hlen, bin, offs, len); return 0; } @@ -3534,7 +5815,8 @@ int driver_output_binary(ErlDrvPort ix, char* hbuf, ErlDrvSizeT hlen, int driver_output2(ErlDrvPort ix, char* hbuf, ErlDrvSizeT hlen, char* buf, ErlDrvSizeT len) { - Port* prt = erts_drvport2port(ix); + erts_aint32_t state; + Port* prt = erts_drvport2port(ix, &state); ERTS_SMP_CHK_NO_PROC_LOCKS; @@ -3543,12 +5825,12 @@ int driver_output2(ErlDrvPort ix, char* hbuf, ErlDrvSizeT hlen, ERTS_SMP_LC_ASSERT(erts_lc_is_port_locked(prt)); - if (prt->status & ERTS_PORT_SFLG_CLOSING) + if (state & ERTS_PORT_SFLG_CLOSING) return 0; prt->bytes_in += (hlen + len); erts_smp_atomic_add_nob(&erts_bytes_in, (erts_aint_t) (hlen + len)); - if (prt->status & ERTS_PORT_SFLG_DISTRIBUTION) { + if (state & ERTS_PORT_SFLG_DISTRIBUTION) { if (len == 0) return erts_net_message(prt, prt->dist_entry, @@ -3560,10 +5842,12 @@ int driver_output2(ErlDrvPort ix, char* hbuf, ErlDrvSizeT hlen, (byte*) hbuf, hlen, (byte*) buf, len); } - else if(prt->status & ERTS_PORT_SFLG_LINEBUF_IO) - deliver_linebuf_message(prt, prt->connected, hbuf, hlen, buf, len); + else if (state & ERTS_PORT_SFLG_LINEBUF_IO) + deliver_linebuf_message(prt, state, ERTS_PORT_GET_CONNECTED(prt), + hbuf, hlen, buf, len); else - deliver_read_message(prt, prt->connected, hbuf, hlen, buf, len, 0); + deliver_read_message(prt, state, ERTS_PORT_GET_CONNECTED(prt), + hbuf, hlen, buf, len, 0); return 0; } @@ -3584,6 +5868,7 @@ int driver_outputv(ErlDrvPort ix, char* hbuf, ErlDrvSizeT hlen, SysIOVec* iov; ErlDrvBinary** binv; Port* prt; + erts_aint32_t state; ERTS_SMP_CHK_NO_PROC_LOCKS; @@ -3596,13 +5881,13 @@ int driver_outputv(ErlDrvPort ix, char* hbuf, ErlDrvSizeT hlen, if (hlen < 0) hlen = 0; - prt = erts_drvport2port(ix); + prt = erts_drvport2port(ix, &state); if (prt == NULL) return -1; ERTS_SMP_LC_ASSERT(erts_lc_is_port_locked(prt)); - if (prt->status & ERTS_PORT_SFLG_CLOSING) + if (state & ERTS_PORT_SFLG_CLOSING) return 0; /* size > 0 ! */ @@ -3627,7 +5912,8 @@ int driver_outputv(ErlDrvPort ix, char* hbuf, ErlDrvSizeT hlen, /* XXX handle distribution !!! */ prt->bytes_in += (hlen + size); erts_smp_atomic_add_nob(&erts_bytes_in, (erts_aint_t) (hlen + size)); - deliver_vec_message(prt, prt->connected, hbuf, hlen, binv, iov, n, size); + deliver_vec_message(prt, ERTS_PORT_GET_CONNECTED(prt), hbuf, hlen, + binv, iov, n, size); return 0; } @@ -3738,8 +6024,7 @@ ErlDrvBinary* driver_realloc_binary(ErlDrvBinary* bin, ErlDrvSizeT size) } -void driver_free_binary(dbin) -ErlDrvBinary* dbin; +void driver_free_binary(ErlDrvBinary* dbin) { Binary *bin; if (!dbin) { @@ -3835,6 +6120,7 @@ static ERTS_INLINE void pdl_destroy(ErlDrvPDL pdl) { ERTS_LC_ASSERT(driver_pdl_get_refc(pdl) == 0); erts_mtx_destroy(&pdl->mtx); + erts_port_dec_refc(pdl->prt); erts_free(ERTS_ALC_T_PORT_DATA_LOCK, pdl); } @@ -3872,16 +6158,18 @@ ErlDrvPDL driver_pdl_create(ErlDrvPort dp) { ErlDrvPDL pdl; - Port *pp = erts_drvport2port(dp); + Port *pp = erts_drvport2port(dp, NULL); if (!pp || pp->port_data_lock) return NULL; pdl = erts_alloc(ERTS_ALC_T_PORT_DATA_LOCK, sizeof(struct erl_drv_port_data_lock)); erts_mtx_init(&pdl->mtx, "port_data_lock"); pdl_init_refc(pdl); + erts_port_inc_refc(pp); + pdl->prt = pp; pp->port_data_lock = pdl; #ifdef HARDDEBUG - erts_fprintf(stderr, "driver_pdl_create(%T) -> 0x%08X\r\n",pp->id,(unsigned) pdl); + erts_fprintf(stderr, "driver_pdl_create(%T) -> 0x%08X\r\n",pp->common.id,(unsigned) pdl); #endif return pdl; } @@ -4319,33 +6607,33 @@ static ERTS_INLINE void drv_cancel_timer(Port *prt) { #ifdef ERTS_SMP - erts_cancel_smp_ptimer(prt->ptimer); + erts_cancel_smp_ptimer(prt->common.u.alive.ptimer); #else - erts_cancel_timer(&prt->tm); + erts_cancel_timer(&prt->common.u.alive.tm); #endif if (erts_port_task_is_scheduled(&prt->timeout_task)) - erts_port_task_abort(prt->id, &prt->timeout_task); + erts_port_task_abort(&prt->timeout_task); } int driver_set_timer(ErlDrvPort ix, unsigned long t) { - Port* prt = erts_drvport2port(ix); + Port* prt = erts_drvport2port(ix, NULL); ERTS_SMP_CHK_NO_PROC_LOCKS; if (prt == NULL) return -1; - ERTS_SMP_LC_ASSERT(erts_lc_is_port_locked(prt)); + if (prt->drv_ptr->timeout == NULL) return -1; drv_cancel_timer(prt); #ifdef ERTS_SMP - erts_create_smp_ptimer(&prt->ptimer, - prt->id, + erts_create_smp_ptimer(&prt->common.u.alive.ptimer, + prt->common.id, (ErlTimeoutProc) schedule_port_timeout, t); #else - erts_set_timer(&prt->tm, + erts_set_timer(&prt->common.u.alive.tm, (ErlTimeoutProc) schedule_port_timeout, NULL, prt, @@ -4356,7 +6644,7 @@ int driver_set_timer(ErlDrvPort ix, unsigned long t) int driver_cancel_timer(ErlDrvPort ix) { - Port* prt = erts_drvport2port(ix); + Port* prt = erts_drvport2port(ix, NULL); if (prt == NULL) return -1; ERTS_SMP_LC_ASSERT(erts_lc_is_port_locked(prt)); @@ -4368,7 +6656,7 @@ int driver_cancel_timer(ErlDrvPort ix) int driver_read_timer(ErlDrvPort ix, unsigned long* t) { - Port* prt = erts_drvport2port(ix); + Port* prt = erts_drvport2port(ix, NULL); ERTS_SMP_CHK_NO_PROC_LOCKS; @@ -4376,9 +6664,11 @@ driver_read_timer(ErlDrvPort ix, unsigned long* t) return -1; ERTS_SMP_LC_ASSERT(erts_lc_is_port_locked(prt)); #ifdef ERTS_SMP - *t = prt->ptimer ? erts_time_left(&prt->ptimer->timer.tm) : 0; + *t = (prt->common.u.alive.ptimer + ? erts_time_left(&prt->common.u.alive.ptimer->timer.tm) + : 0); #else - *t = erts_time_left(&prt->tm); + *t = erts_time_left(&prt->common.u.alive.tm); #endif return 0; } @@ -4429,8 +6719,8 @@ static int do_driver_monitor_process(Port *prt, } ref = erts_make_ref_in_buffer(buf); - erts_add_monitor(&(prt->monitors), MON_ORIGIN, ref, rp->id, NIL); - erts_add_monitor(&(rp->monitors), MON_TARGET, ref, prt->id, NIL); + erts_add_monitor(&ERTS_P_MONITORS(prt), MON_ORIGIN, ref, rp->common.id, NIL); + erts_add_monitor(&ERTS_P_MONITORS(rp), MON_TARGET, ref, prt->common.id, NIL); erts_smp_proc_unlock(rp, ERTS_PROC_LOCK_LINK); ref_to_driver_monitor(ref,monitor); @@ -4440,31 +6730,22 @@ static int do_driver_monitor_process(Port *prt, /* * This can be called from a non scheduler thread iff a port_data_lock exists */ -int driver_monitor_process(ErlDrvPort port, +int driver_monitor_process(ErlDrvPort drvport, ErlDrvTermData process, ErlDrvMonitor *monitor) { Port *prt; int ret; - Uint32 status; + erts_aint32_t state; +#if !HEAP_ON_C_STACK || (defined(ERTS_SMP) && defined(ERTS_ENABLE_LOCK_CHECK)) ErtsSchedulerData *sched = erts_get_scheduler_data(); - int ix = (int) port; - if (ix < 0 || erts_max_ports <= ix) { - return -1; - } - prt = &erts_port[ix]; +#endif - DRV_MONITOR_LOCK_PDL(prt); + prt = DRV_MONITOR_LOOKUP_PORT_LOCK_PDL(drvport); - if (sched) { - status = erts_port[ix].status; - } else { - erts_smp_port_state_lock(prt); - status = erts_port[ix].status; - erts_smp_port_state_unlock(prt); - } + state = erts_atomic32_read_nob(&prt->state); - if (status & ERTS_PORT_SFLGS_INVALID_DRIVER_LOOKUP) { + if (state & ERTS_PORT_SFLGS_INVALID_DRIVER_LOOKUP) { DRV_MONITOR_UNLOCK_PDL(prt); return -1; } @@ -4502,7 +6783,7 @@ static int do_driver_demonitor_process(Port *prt, Eterm *buf, memcpy(buf,monitor,sizeof(Eterm)*REF_THING_SIZE); ref = make_internal_ref(buf); - mon = erts_lookup_monitor(prt->monitors, ref); + mon = erts_lookup_monitor(ERTS_P_MONITORS(prt), ref); if (mon == NULL) { return 1; } @@ -4514,13 +6795,13 @@ static int do_driver_demonitor_process(Port *prt, Eterm *buf, to, ERTS_PROC_LOCK_LINK, ERTS_P2P_FLG_ALLOW_OTHER_X); - mon = erts_remove_monitor(&(prt->monitors), ref); + mon = erts_remove_monitor(&ERTS_P_MONITORS(prt), ref); if (mon) { erts_destroy_monitor(mon); } if (rp) { ErtsMonitor *rmon; - rmon = erts_remove_monitor(&(rp->monitors), ref); + rmon = erts_remove_monitor(&ERTS_P_MONITORS(rp), ref); erts_smp_proc_unlock(rp, ERTS_PROC_LOCK_LINK); if (rmon != NULL) { erts_destroy_monitor(rmon); @@ -4529,30 +6810,21 @@ static int do_driver_demonitor_process(Port *prt, Eterm *buf, return 0; } -int driver_demonitor_process(ErlDrvPort port, +int driver_demonitor_process(ErlDrvPort drvport, const ErlDrvMonitor *monitor) { Port *prt; int ret; - Uint32 status; + erts_aint32_t state; +#if !HEAP_ON_C_STACK || (defined(ERTS_SMP) && defined(ERTS_ENABLE_LOCK_CHECK)) ErtsSchedulerData *sched = erts_get_scheduler_data(); - int ix = (int) port; - if (ix < 0 || erts_max_ports <= ix) { - return -1; - } - prt = &erts_port[ix]; +#endif - DRV_MONITOR_LOCK_PDL(prt); + prt = DRV_MONITOR_LOOKUP_PORT_LOCK_PDL(drvport); - if (sched) { - status = erts_port[ix].status; - } else { - erts_smp_port_state_lock(prt); - status = erts_port[ix].status; - erts_smp_port_state_unlock(prt); - } + state = erts_atomic32_read_nob(&prt->state); - if (status & ERTS_PORT_SFLGS_INVALID_DRIVER_LOOKUP) { + if (state & ERTS_PORT_SFLGS_INVALID_DRIVER_LOOKUP) { DRV_MONITOR_UNLOCK_PDL(prt); return -1; } @@ -4588,7 +6860,7 @@ static ErlDrvTermData do_driver_get_monitored_process(Port *prt, Eterm *buf, memcpy(buf,monitor,sizeof(Eterm)*REF_THING_SIZE); ref = make_internal_ref(buf); - mon = erts_lookup_monitor(prt->monitors, ref); + mon = erts_lookup_monitor(ERTS_P_MONITORS(prt), ref); if (mon == NULL) { return driver_term_nil; } @@ -4599,30 +6871,20 @@ static ErlDrvTermData do_driver_get_monitored_process(Port *prt, Eterm *buf, } -ErlDrvTermData driver_get_monitored_process(ErlDrvPort port, +ErlDrvTermData driver_get_monitored_process(ErlDrvPort drvport, const ErlDrvMonitor *monitor) { Port *prt; ErlDrvTermData ret; - Uint32 status; + erts_aint32_t state; +#if !HEAP_ON_C_STACK || (defined(ERTS_SMP) && defined(ERTS_ENABLE_LOCK_CHECK)) ErtsSchedulerData *sched = erts_get_scheduler_data(); - int ix = (int) port; - if (ix < 0 || erts_max_ports <= ix) { - return driver_term_nil; - } - prt = &erts_port[ix]; - - DRV_MONITOR_LOCK_PDL(prt); +#endif - if (sched) { - status = erts_port[ix].status; - } else { - erts_smp_port_state_lock(prt); - status = erts_port[ix].status; - erts_smp_port_state_unlock(prt); - } + prt = DRV_MONITOR_LOOKUP_PORT_LOCK_PDL(drvport); - if (status & ERTS_PORT_SFLGS_INVALID_DRIVER_LOOKUP) { + state = erts_atomic32_read_nob(&prt->state); + if (state & ERTS_PORT_SFLGS_INVALID_DRIVER_LOOKUP) { DRV_MONITOR_UNLOCK_PDL(prt); return driver_term_nil; } @@ -4667,7 +6929,7 @@ void erts_fire_port_monitor(Port *prt, Eterm ref) ERTS_SMP_LC_ASSERT(erts_lc_is_port_locked(prt)); ASSERT(prt->drv_ptr != NULL); DRV_MONITOR_LOCK_PDL(prt); - if (erts_lookup_monitor(prt->monitors,ref) == NULL) { + if (erts_lookup_monitor(ERTS_P_MONITORS(prt), ref) == NULL) { DRV_MONITOR_UNLOCK_PDL(prt); return; } @@ -4677,7 +6939,7 @@ void erts_fire_port_monitor(Port *prt, Eterm ref) DRV_MONITOR_UNLOCK_PDL(prt); #ifdef USE_VM_PROBES if (DTRACE_ENABLED(driver_process_exit)) { - DTRACE_FORMAT_COMMON_PID_AND_PORT(prt->connected, prt) + DTRACE_FORMAT_COMMON_PID_AND_PORT(ERTS_PORT_GET_CONNECTED(prt), prt) DTRACE3(driver_process_exit, process_str, port_str, prt->name); } #endif @@ -4686,7 +6948,7 @@ void erts_fire_port_monitor(Port *prt, Eterm ref) erts_unblock_fpe(fpe_was_unmasked); DRV_MONITOR_LOCK_PDL(prt); /* remove monitor *after* callback */ - rmon = erts_remove_monitor(&(prt->monitors),ref); + rmon = erts_remove_monitor(&ERTS_P_MONITORS(prt), ref); DRV_MONITOR_UNLOCK_PDL(prt); if (rmon) { erts_destroy_monitor(rmon); @@ -4697,7 +6959,8 @@ void erts_fire_port_monitor(Port *prt, Eterm ref) static int driver_failure_term(ErlDrvPort ix, Eterm term, int eof) { - Port* prt = erts_drvport2port(ix); + erts_aint32_t state; + Port* prt = erts_drvport2port(ix, &state); ERTS_SMP_CHK_NO_PROC_LOCKS; @@ -4705,19 +6968,19 @@ driver_failure_term(ErlDrvPort ix, Eterm term, int eof) return -1; ERTS_SMP_LC_ASSERT(erts_lc_is_port_locked(prt)); if (eof) - flush_linebuf_messages(prt); - if (prt->status & ERTS_PORT_SFLG_CLOSING) { + flush_linebuf_messages(prt, state); + if (state & ERTS_PORT_SFLG_CLOSING) { terminate_port(prt); - } else if (eof && (prt->status & ERTS_PORT_SFLG_SOFT_EOF)) { - deliver_result(prt->id, prt->connected, am_eof); + } else if (eof && (state & ERTS_PORT_SFLG_SOFT_EOF)) { + deliver_result(prt->common.id, ERTS_PORT_GET_CONNECTED(prt), am_eof); } else { - /* XXX UGLY WORK AROUND, Let do_exit_port terminate the port */ + /* XXX UGLY WORK AROUND, Let erts_deliver_port_exit() terminate the port */ if (prt->port_data_lock) driver_pdl_lock(prt->port_data_lock); prt->ioq.size = 0; if (prt->port_data_lock) driver_pdl_unlock(prt->port_data_lock); - erts_do_exit_port(prt, prt->id, eof ? am_normal : term); + erts_deliver_port_exit(prt, prt->common.id, eof ? am_normal : term, 0); } return 0; } @@ -4730,23 +6993,23 @@ driver_failure_term(ErlDrvPort ix, Eterm term, int eof) */ int driver_exit(ErlDrvPort ix, int err) { - Port* prt = erts_drvport2port(ix); + Port* prt = erts_drvport2port(ix, NULL); Process* rp; ErtsLink *lnk, *rlnk = NULL; + Eterm connected; ERTS_SMP_CHK_NO_PROC_LOCKS; if (prt == NULL) return -1; - ERTS_SMP_LC_ASSERT(erts_lc_is_port_locked(prt)); - - rp = erts_pid2proc(NULL, 0, prt->connected, ERTS_PROC_LOCK_LINK); + connected = ERTS_PORT_GET_CONNECTED(prt); + rp = erts_pid2proc(NULL, 0, connected, ERTS_PROC_LOCK_LINK); if (rp) { - rlnk = erts_remove_link(&(rp->nlinks),prt->id); + rlnk = erts_remove_link(&ERTS_P_LINKS(rp),prt->common.id); } - lnk = erts_remove_link(&(prt->nlinks),prt->connected); + lnk = erts_remove_link(&ERTS_P_LINKS(prt), connected); #ifdef ERTS_SMP if (rp) @@ -4803,24 +7066,24 @@ ErlDrvTermData driver_mk_atom(char* string) ErlDrvTermData driver_mk_port(ErlDrvPort ix) { - Port* prt = erts_drvport2port(ix); + Port* prt = erts_drvport2port(ix, NULL); ERTS_SMP_LC_ASSERT(erts_lc_is_port_locked(prt)); - return (ErlDrvTermData) prt->id; + return (ErlDrvTermData) prt->common.id; } ErlDrvTermData driver_connected(ErlDrvPort ix) { - Port* prt = erts_drvport2port(ix); + Port* prt = erts_drvport2port(ix, NULL); ERTS_SMP_CHK_NO_PROC_LOCKS; if (prt == NULL) return NIL; ERTS_SMP_LC_ASSERT(erts_lc_is_port_locked(prt)); - return prt->connected; + return ERTS_PORT_GET_CONNECTED(prt); } ErlDrvTermData driver_caller(ErlDrvPort ix) { - Port* prt = erts_drvport2port(ix); + Port* prt = erts_drvport2port(ix, NULL); ERTS_SMP_CHK_NO_PROC_LOCKS; if (prt == NULL) return NIL; @@ -4830,25 +7093,25 @@ ErlDrvTermData driver_caller(ErlDrvPort ix) int driver_lock_driver(ErlDrvPort ix) { - Port* prt = erts_drvport2port(ix); + Port* prt = erts_drvport2port(ix, NULL); DE_Handle* dh; ERTS_SMP_CHK_NO_PROC_LOCKS; - erts_smp_mtx_lock(&erts_driver_list_lock); + erts_smp_rwmtx_rwlock(&erts_driver_list_lock); if (prt == NULL) { - erts_smp_mtx_unlock(&erts_driver_list_lock); + erts_smp_rwmtx_rwunlock(&erts_driver_list_lock); return -1; } ERTS_SMP_LC_ASSERT(erts_lc_is_port_locked(prt)); if ((dh = (DE_Handle*)prt->drv_ptr->handle ) == NULL) { - erts_smp_mtx_unlock(&erts_driver_list_lock); + erts_smp_rwmtx_rwunlock(&erts_driver_list_lock); return -1; } erts_ddll_lock_driver(dh, prt->drv_ptr->name); - erts_smp_mtx_unlock(&erts_driver_list_lock); + erts_smp_rwmtx_rwunlock(&erts_driver_list_lock); return 0; } @@ -4858,7 +7121,7 @@ static int maybe_lock_driver_list(void) void *rec_lock; rec_lock = erts_smp_tsd_get(driver_list_lock_status_key); if (rec_lock == 0) { - erts_smp_mtx_lock(&erts_driver_list_lock); + erts_smp_rwmtx_rwlock(&erts_driver_list_lock); return 1; } return 0; @@ -4866,7 +7129,7 @@ static int maybe_lock_driver_list(void) static void maybe_unlock_driver_list(int doit) { if (doit) { - erts_smp_mtx_unlock(&erts_driver_list_lock); + erts_smp_rwmtx_rwunlock(&erts_driver_list_lock); } } /* @@ -5035,7 +7298,7 @@ no_event_callback(ErlDrvData drv_data, ErlDrvEvent event, ErlDrvEventData event_ { Port *prt = get_current_port(); report_missing_drv_callback(prt, "Event", "event()"); - driver_event((ErlDrvPort) internal_port_index(prt->id), event, NULL); + driver_event((ErlDrvPort) prt, event, NULL); } static void @@ -5043,7 +7306,7 @@ no_ready_input_callback(ErlDrvData drv_data, ErlDrvEvent event) { Port *prt = get_current_port(); report_missing_drv_callback(prt, "Input", "ready_input()"); - driver_select((ErlDrvPort) internal_port_index(prt->id), event, + driver_select((ErlDrvPort) prt, event, (ERL_DRV_READ | ERL_DRV_USE_NO_CALLBACK), 0); } @@ -5052,7 +7315,7 @@ no_ready_output_callback(ErlDrvData drv_data, ErlDrvEvent event) { Port *prt = get_current_port(); report_missing_drv_callback(prt, "Output", "ready_output()"); - driver_select((ErlDrvPort) internal_port_index(prt->id), event, + driver_select((ErlDrvPort) prt, event, (ERL_DRV_WRITE | ERL_DRV_USE_NO_CALLBACK), 0); } @@ -5087,13 +7350,13 @@ init_driver(erts_driver_t *drv, ErlDrvEntry *de, DE_Handle *handle) drv->lock = NULL; else { drv->lock = erts_alloc(ERTS_ALC_T_DRIVER_LOCK, - sizeof(erts_smp_mtx_t)); - erts_smp_mtx_init_x(drv->lock, - "driver_lock", + sizeof(erts_mtx_t)); + erts_mtx_init_x(drv->lock, + "driver_lock", #if defined(ERTS_ENABLE_LOCK_CHECK) || defined(ERTS_ENABLE_LOCK_COUNT) - am_atom_put(drv->name, sys_strlen(drv->name)) + am_atom_put(drv->name, sys_strlen(drv->name)) #else - NIL + NIL #endif ); } @@ -5165,7 +7428,7 @@ int erts_add_driver_entry(ErlDrvEntry *de, DE_Handle *handle, int driver_list_lo int res; if (!driver_list_locked) { - erts_smp_mtx_lock(&erts_driver_list_lock); + erts_smp_rwmtx_rwlock(&erts_driver_list_lock); } dp->next = driver_list; @@ -5194,7 +7457,7 @@ int erts_add_driver_entry(ErlDrvEntry *de, DE_Handle *handle, int driver_list_lo if (!driver_list_locked) { erts_smp_tsd_set(driver_list_lock_status_key, NULL); - erts_smp_mtx_unlock(&erts_driver_list_lock); + erts_smp_rwmtx_rwunlock(&erts_driver_list_lock); } return res; } @@ -5207,7 +7470,7 @@ int remove_driver_entry(ErlDrvEntry *drv) rec_lock = erts_smp_tsd_get(driver_list_lock_status_key); if (rec_lock == NULL) { - erts_smp_mtx_lock(&erts_driver_list_lock); + erts_smp_rwmtx_rwlock(&erts_driver_list_lock); } dp = driver_list; while (dp && dp->entry != drv) @@ -5215,7 +7478,7 @@ int remove_driver_entry(ErlDrvEntry *drv) if (dp) { if (dp->handle) { if (rec_lock == NULL) { - erts_smp_mtx_unlock(&erts_driver_list_lock); + erts_smp_rwmtx_rwunlock(&erts_driver_list_lock); } return -1; } @@ -5229,12 +7492,12 @@ int remove_driver_entry(ErlDrvEntry *drv) } erts_destroy_driver(dp); if (rec_lock == NULL) { - erts_smp_mtx_unlock(&erts_driver_list_lock); + erts_smp_rwmtx_rwunlock(&erts_driver_list_lock); } return 1; } if (rec_lock == NULL) { - erts_smp_mtx_unlock(&erts_driver_list_lock); + erts_smp_rwmtx_rwunlock(&erts_driver_list_lock); } return 0; } @@ -5264,18 +7527,22 @@ erl_drv_getenv(char *key, char *value, size_t *value_size) * - uses the fact that heart_port is registered when starting heart */ -Port *erts_get_heart_port() { +Port *erts_get_heart_port(void) +{ + int ix, max = erts_ptab_max(&erts_port); - Port* port; - Uint ix; + for (ix = 0; ix < max; ix++) { + struct reg_proc *reg; + Port *port = erts_pix2port(ix); - for(ix = 0; ix < erts_max_ports; ix++) { - port = &erts_port[ix]; + if (!port) + continue; /* only examine undead or alive ports */ - if (port->status & ERTS_PORT_SFLGS_DEAD) + if (erts_atomic32_read_nob(&port->state) & ERTS_PORT_SFLGS_DEAD) continue; /* immediate atom compare */ - if (port->reg && port->reg->name == am_heart_port) { + reg = port->common.u.alive.reg; + if (reg && reg->name == am_heart_port) { return port; } } diff --git a/erts/emulator/beam/register.c b/erts/emulator/beam/register.c index c02872ef80..757e2800e6 100644 --- a/erts/emulator/beam/register.c +++ b/erts/emulator/beam/register.c @@ -175,14 +175,14 @@ int erts_register_name(Process *c_p, Eterm name, Eterm id) if (is_not_atom(name) || name == am_undefined) return res; - if (c_p->id == id) /* A very common case I think... */ + if (c_p->common.id == id) /* A very common case I think... */ proc = c_p; else { if (is_not_internal_pid(id) && is_not_internal_port(id)) return res; erts_smp_proc_unlock(c_p, ERTS_PROC_LOCK_MAIN); if (is_internal_port(id)) { - port = erts_id2port(id, NULL, 0); + port = erts_id2port(id); if (!port) goto done; } @@ -204,7 +204,7 @@ int erts_register_name(Process *c_p, Eterm name, Eterm id) r.p = proc; if (!proc) goto done; - if (proc->reg) + if (proc->common.u.alive.reg) goto done; r.pt = NULL; } @@ -212,7 +212,7 @@ int erts_register_name(Process *c_p, Eterm name, Eterm id) ASSERT(!INVALID_PORT(port, id)); ERTS_SMP_LC_ASSERT(erts_lc_is_port_locked(port)); r.pt = port; - if (r.pt->reg) + if (r.pt->common.u.alive.reg) goto done; r.p = NULL; } @@ -224,23 +224,24 @@ int erts_register_name(Process *c_p, Eterm name, Eterm id) if (IS_TRACED_FL(proc, F_TRACE_PROCS)) { trace_proc(c_p, proc, am_register, name); } - proc->reg = rp; + proc->common.u.alive.reg = rp; } else if (port && rp->pt == port) { if (IS_TRACED_FL(port, F_TRACE_PORTS)) { trace_port(port, am_register, name); } - port->reg = rp; + port->common.u.alive.reg = rp; } - if ((rp->p && rp->p->id == id) || (rp->pt && rp->pt->id == id)) { + if ((rp->p && rp->p->common.id == id) + || (rp->pt && rp->pt->common.id == id)) { res = 1; } done: reg_write_unlock(); if (port) - erts_smp_port_unlock(port); + erts_port_release(port); if (c_p != proc) { if (proc) erts_smp_proc_unlock(proc, ERTS_PROC_LOCK_MAIN); @@ -291,9 +292,9 @@ erts_whereis_name_to_id(Process *c_p, Eterm name) * is read only. */ if (rp->p) - res = rp->p->id; + res = rp->p->common.id; else if (rp->pt) - res = rp->pt->id; + res = rp->pt->common.id; break; } b = b->next; @@ -396,28 +397,26 @@ erts_whereis_name(Process *c_p, if (!rp || !rp->pt) *port = NULL; else { -#ifndef ERTS_SMP - erts_smp_atomic_inc_nob(&rp->pt->refc); -#else +#ifdef ERTS_SMP if (pending_port == rp->pt) pending_port = NULL; else { if (pending_port) { /* Ahh! Registered port changed while reg lock was unlocked... */ - erts_smp_port_unlock(pending_port); + erts_port_release(pending_port); pending_port = NULL; } if (erts_smp_port_trylock(rp->pt) == EBUSY) { - Eterm id = rp->pt->id; /* id read only... */ + Eterm id = rp->pt->common.id; /* id read only... */ /* Unlock all locks, acquire port lock, and restart... */ if (current_c_p_locks) { erts_smp_proc_unlock(c_p, current_c_p_locks); current_c_p_locks = 0; } reg_read_unlock(); - pending_port = erts_id2port(id, NULL, 0); + pending_port = erts_id2port(id); goto restart; } } @@ -431,7 +430,7 @@ erts_whereis_name(Process *c_p, if (c_p && !current_c_p_locks) erts_smp_proc_lock(c_p, c_p_locks); if (pending_port) - erts_smp_port_unlock(pending_port); + erts_port_release(pending_port); #endif reg_read_unlock(); @@ -493,8 +492,8 @@ int erts_unregister_name(Process *c_p, current_c_p_locks = c_p_locks; } #endif - if (c_p->reg) { - r.name = c_p->reg->name; + if (c_p->common.u.alive.reg) { + r.name = c_p->common.u.alive.reg->name; } else { /* Name got unregistered while main lock was released */ res = 0; @@ -505,24 +504,22 @@ int erts_unregister_name(Process *c_p, if ((rp = (RegProc*) hash_get(&process_reg, (void*) &r)) != NULL) { if (rp->pt) { if (port != rp->pt) { -#ifndef ERTS_SMP - erts_smp_atomic_inc_nob(&rp->pt->refc); -#else +#ifdef ERTS_SMP if (port) { ASSERT(port != c_prt); - erts_smp_port_unlock(port); + erts_port_release(port); port = NULL; } if (erts_smp_port_trylock(rp->pt) == EBUSY) { - Eterm id = rp->pt->id; /* id read only... */ + Eterm id = rp->pt->common.id; /* id read only... */ /* Unlock all locks, acquire port lock, and restart... */ if (current_c_p_locks) { erts_smp_proc_unlock(c_p, current_c_p_locks); current_c_p_locks = 0; } reg_write_unlock(); - port = erts_id2port(id, NULL, 0); + port = erts_id2port(id); goto restart; } #endif @@ -532,7 +529,7 @@ int erts_unregister_name(Process *c_p, ASSERT(rp->pt == port); ERTS_SMP_LC_ASSERT(erts_lc_is_port_locked(port)); - rp->pt->reg = NULL; + rp->pt->common.u.alive.reg = NULL; if (IS_TRACED_FL(port, F_TRACE_PORTS)) { trace_port(port, am_unregister, r.name); @@ -549,7 +546,7 @@ int erts_unregister_name(Process *c_p, ERTS_PROC_LOCK_MAIN); current_c_p_locks = c_p_locks; #endif - rp->p->reg = NULL; + rp->p->common.u.alive.reg = NULL; if (IS_TRACED_FL(rp->p, F_TRACE_PROCS)) { trace_proc(c_p, rp->p, am_unregister, r.name); } @@ -568,7 +565,7 @@ int erts_unregister_name(Process *c_p, reg_write_unlock(); if (c_prt != port) { if (port) { - erts_smp_port_unlock(port); + erts_port_release(port); } if (c_prt) { erts_smp_port_lock(c_prt); diff --git a/erts/emulator/beam/register.h b/erts/emulator/beam/register.h index 38e8cfbf28..7170463375 100644 --- a/erts/emulator/beam/register.h +++ b/erts/emulator/beam/register.h @@ -24,26 +24,19 @@ #ifndef __REGPROC_H__ #define __REGPROC_H__ -#ifndef __SYS_H__ #include "sys.h" -#endif - -#ifndef __HASH_H__ #include "hash.h" -#endif - -#ifndef __PROCESS_H__ #include "erl_process.h" -#endif - -struct port; +#define ERL_PORT_GET_PORT_TYPE_ONLY__ +#include "erl_port.h" +#undef ERL_PORT_GET_PORT_TYPE_ONLY__ typedef struct reg_proc { HashBucket bucket; /* MUST BE LOCATED AT TOP OF STRUCT!!! */ Process *p; /* The process registered (only one of this and 'pt' is non-NULL */ - struct port *pt; /* The port registered */ + Port *pt; /* The port registered */ Eterm name; /* Atom name */ } RegProc; @@ -55,12 +48,12 @@ int erts_register_name(Process *, Eterm, Eterm); Eterm erts_whereis_name_to_id(Process *, Eterm); void erts_whereis_name(Process *, ErtsProcLocks, Eterm, Process**, ErtsProcLocks, int, - struct port**); + Port**); Process *erts_whereis_process(Process *, ErtsProcLocks, Eterm, ErtsProcLocks, int); -int erts_unregister_name(Process *, ErtsProcLocks, struct port *, Eterm); +int erts_unregister_name(Process *, ErtsProcLocks, Port *, Eterm); #endif diff --git a/erts/emulator/beam/sys.h b/erts/emulator/beam/sys.h index 8957bb5bde..898a30b010 100644 --- a/erts/emulator/beam/sys.h +++ b/erts/emulator/beam/sys.h @@ -116,6 +116,16 @@ typedef ERTS_SYS_FD_TYPE ErtsSysFdType; # define ERTS_DECLARE_DUMMY(X) X #endif +#if !defined(__func__) +# if !defined(__STDC_VERSION__) || __STDC_VERSION__ < 199901L +# if !defined(__GNUC__) || __GNUC__ < 2 +# define __func__ "[unknown_function]" +# else +# define __func__ __FUNCTION__ +# endif +# endif +#endif + #if defined(DEBUG) || defined(ERTS_ENABLE_LOCK_CHECK) # undef ERTS_CAN_INLINE # define ERTS_CAN_INLINE 0 @@ -231,9 +241,11 @@ void erl_assert_error(char* expr, char* file, int line); #if SIZEOF_VOID_P == 8 #undef ARCH_32 #define ARCH_64 +#define ERTS_SIZEOF_TERM 8 #elif SIZEOF_VOID_P == 4 #define ARCH_32 #undef ARCH_64 +#define ERTS_SIZEOF_TERM 4 #else #error Neither 32 nor 64 bit architecture #endif @@ -241,6 +253,8 @@ void erl_assert_error(char* expr, char* file, int line); # define HALFWORD_HEAP 1 # define HALFWORD_ASSERT 0 # define ASSERT_HALFWORD(COND) ASSERT(COND) +# undef ERTS_SIZEOF_TERM +# define ERTS_SIZEOF_TERM 4 #else # define HALFWORD_HEAP 0 # define HALFWORD_ASSERT 0 @@ -367,6 +381,27 @@ typedef unsigned char byte; #error 64-bit architecture, but no appropriate type to use for Uint64 and Sint64 found #endif +#ifdef WORDS_BIGENDIAN +# define ERTS_HUINT_HVAL_HIGH 0 +# define ERTS_HUINT_HVAL_LOW 1 +#else +# define ERTS_HUINT_HVAL_HIGH 1 +# define ERTS_HUINT_HVAL_LOW 0 +#endif +#if ERTS_SIZEOF_TERM == 8 +typedef union { + Uint val; + Uint32 hval[2]; +} HUint; +#elif ERTS_SIZEOF_TERM == 4 +typedef union { + Uint val; + Uint16 hval[2]; +} HUint; +#else +#error "Unsupported size of term" +#endif + # define ERTS_EXTRA_DATA_ALIGN_SZ(X) \ (((size_t) 8) - (((size_t) (X)) & ((size_t) 7))) @@ -505,6 +540,10 @@ __decl_noreturn void __noreturn erl_exit(int n, char*, ...); #define ERTS_ABORT_EXIT (INT_MIN + 1) /* no crash dump; only abort() */ #define ERTS_DUMP_EXIT (INT_MIN + 2) /* crash dump; then exit() */ +#define ERTS_INTERNAL_ERROR(What) \ + erl_exit(ERTS_ABORT_EXIT, "%s:%d:%s(): Internal error: %s\n", \ + __FILE__, __LINE__, __func__, What) + Eterm erts_check_io_info(void *p); /* Size of misc memory allocated from system dependent code */ @@ -579,6 +618,7 @@ typedef struct _SysDriverOpts { char *wd; /* Working directory. */ unsigned spawn_type; /* Bitfield of ERTS_SPAWN_DRIVER | ERTS_SPAWN_EXTERNAL | both*/ + int parallelism; /* Optimize for parallelism */ } SysDriverOpts; extern char *erts_default_arg0; @@ -618,7 +658,7 @@ typedef struct { #define ERTS_SYS_DDLL_ERROR_INIT {NULL} extern void erts_sys_ddll_free_error(ErtsSysDdllError*); extern void erl_sys_ddll_init(void); /* to initialize mutexes etc */ -extern int erts_sys_ddll_open2(char *path, void **handle, ErtsSysDdllError*); +extern int erts_sys_ddll_open2(const char *path, void **handle, ErtsSysDdllError*); #define erts_sys_ddll_open(P,H) erts_sys_ddll_open2(P,H,NULL) extern int erts_sys_ddll_open_noext(char *path, void **handle, ErtsSysDdllError*); extern int erts_sys_ddll_load_driver_init(void *handle, void **function); @@ -627,7 +667,7 @@ extern int erts_sys_ddll_close2(void *handle, ErtsSysDdllError*); #define erts_sys_ddll_close(H) erts_sys_ddll_close2(H,NULL) extern void *erts_sys_ddll_call_init(void *function); extern void *erts_sys_ddll_call_nif_init(void *function); -extern int erts_sys_ddll_sym2(void *handle, char *name, void **function, ErtsSysDdllError*); +extern int erts_sys_ddll_sym2(void *handle, const char *name, void **function, ErtsSysDdllError*); #define erts_sys_ddll_sym(H,N,F) erts_sys_ddll_sym2(H,N,F,NULL) extern char *erts_sys_ddll_error(int code); @@ -644,7 +684,7 @@ void erts_sys_schedule_interrupt_timed(int set, erts_short_time_t msec); void erts_sys_main_thread(void); #endif -extern void erts_sys_prepare_crash_dump(int secs); +extern int erts_sys_prepare_crash_dump(int secs); extern void erts_sys_pre_init(void); extern void erl_sys_init(void); extern void erl_sys_args(int *argc, char **argv); diff --git a/erts/emulator/beam/utils.c b/erts/emulator/beam/utils.c index 1969fc762c..5261effef9 100644 --- a/erts/emulator/beam/utils.c +++ b/erts/emulator/beam/utils.c @@ -46,6 +46,7 @@ #include "erl_thr_queue.h" #include "erl_sched_spec_pre_alloc.h" #include "beam_bp.h" +#include "erl_ptab.h" #undef M_TRIM_THRESHOLD #undef M_TOP_PAD @@ -3016,12 +3017,13 @@ buf_to_intlist(Eterm** hpp, char *buf, size_t len, Eterm tail) ** ; ** ** Return remaining bytes in buffer on success -** -1 on overflow -** -2 on type error (including that result would not be a whole number of bytes) +** ERTS_IOLIST_TO_BUF_OVERFLOW on overflow +** ERTS_IOLIST_TO_BUF_TYPE_ERROR on type error (including that result would not be a whole number of bytes) */ -int io_list_to_buf(Eterm obj, char* buf, int len) +ErlDrvSizeT erts_iolist_to_buf(Eterm obj, char* buf, ErlDrvSizeT alloced_len) { + ErlDrvSizeT len = (ErlDrvSizeT) alloced_len; Eterm* objp; DECLARE_ESTACK(s); goto L_again; @@ -3114,20 +3116,20 @@ int io_list_to_buf(Eterm obj, char* buf, int len) L_type_error: DESTROY_ESTACK(s); - return -2; + return ERTS_IOLIST_TO_BUF_TYPE_ERROR; L_overflow: DESTROY_ESTACK(s); - return -1; + return ERTS_IOLIST_TO_BUF_OVERFLOW; } /* * Return 0 if successful, and non-zero if unsuccessful. */ -int erts_iolist_size(Eterm obj, Uint* sizep) +int erts_iolist_size(Eterm obj, ErlDrvSizeT* sizep) { Eterm* objp; - Uint size = 0; + Uint size = 0; /* Intentionally Uint due to halfword heap */ DECLARE_ESTACK(s); goto L_again; @@ -3179,7 +3181,7 @@ int erts_iolist_size(Eterm obj, Uint* sizep) #undef SAFE_ADD DESTROY_ESTACK(s); - *sizep = size; + *sizep = (ErlDrvSizeT) size; return ERTS_IOLIST_OK; L_overflow_error: diff --git a/erts/emulator/drivers/common/inet_drv.c b/erts/emulator/drivers/common/inet_drv.c index 3210ffa92a..236b8710fb 100644 --- a/erts/emulator/drivers/common/inet_drv.c +++ b/erts/emulator/drivers/common/inet_drv.c @@ -678,6 +678,8 @@ static int my_strncasecmp(const char *s1, const char *s2, size_t n) #define INET_LOPT_UDP_READ_PACKETS 33 /* Number of packets to read */ #define INET_OPT_RAW 34 /* Raw socket options */ #define INET_LOPT_TCP_SEND_TIMEOUT_CLOSE 35 /* auto-close on send timeout or not */ +#define INET_LOPT_TCP_MSGQ_HIWTRMRK 36 /* set local high watermark */ +#define INET_LOPT_TCP_MSGQ_LOWTRMRK 37 /* set local low watermark */ /* SCTP options: a separate range, from 100: */ #define SCTP_OPT_RTOINFO 100 #define SCTP_OPT_ASSOCINFO 101 @@ -788,6 +790,8 @@ static int my_strncasecmp(const char *s1, const char *s2, size_t n) #define INET_HIGH_WATERMARK (1024*8) /* 8k pending high => busy */ #define INET_LOW_WATERMARK (1024*4) /* 4k pending => allow more */ +#define INET_HIGH_MSGQ_WATERMARK (1024*8) /* 8k pending high => busy */ +#define INET_LOW_MSGQ_WATERMARK (1024*4) /* 4k pending => allow more */ #define INET_INFINITY 0xffffffff /* infinity value */ @@ -879,7 +883,7 @@ typedef struct subs_list_ { #define NO_PROCESS 0 #define NO_SUBSCRIBERS(SLP) ((SLP)->subscriber == NO_PROCESS) -static void send_to_subscribers(ErlDrvPort, subs_list *, int, +static void send_to_subscribers(ErlDrvTermData, subs_list *, int, ErlDrvTermData [], int); static void free_subscribers(subs_list*); static int save_subscriber(subs_list *, ErlDrvTermData); @@ -1873,8 +1877,7 @@ static int deq_async(inet_descriptor* desc, int* ap, ErlDrvTermData* cp, int* rp ** {inet_async, Port, Ref, ok} */ static int -send_async_ok(ErlDrvPort port, ErlDrvTermData Port, int Ref, - ErlDrvTermData recipient) +send_async_ok(ErlDrvTermData Port, int Ref,ErlDrvTermData recipient) { ErlDrvTermData spec[2*LOAD_ATOM_CNT + LOAD_PORT_CNT + LOAD_INT_CNT + LOAD_TUPLE_CNT]; @@ -1888,14 +1891,14 @@ send_async_ok(ErlDrvPort port, ErlDrvTermData Port, int Ref, ASSERT(i == sizeof(spec)/sizeof(*spec)); - return driver_send_term(port, recipient, spec, i); + return erl_drv_send_term(Port, recipient, spec, i); } /* send message: ** {inet_async, Port, Ref, {ok,Port2}} */ static int -send_async_ok_port(ErlDrvPort port, ErlDrvTermData Port, int Ref, +send_async_ok_port(ErlDrvTermData Port, int Ref, ErlDrvTermData recipient, ErlDrvTermData Port2) { ErlDrvTermData spec[2*LOAD_ATOM_CNT + 2*LOAD_PORT_CNT + @@ -1914,14 +1917,14 @@ send_async_ok_port(ErlDrvPort port, ErlDrvTermData Port, int Ref, ASSERT(i == sizeof(spec)/sizeof(*spec)); - return driver_send_term(port, recipient, spec, i); + return erl_drv_send_term(Port, recipient, spec, i); } /* send message: ** {inet_async, Port, Ref, {error,Reason}} */ static int -send_async_error(ErlDrvPort port, ErlDrvTermData Port, int Ref, +send_async_error(ErlDrvTermData Port, int Ref, ErlDrvTermData recipient, ErlDrvTermData Reason) { ErlDrvTermData spec[3*LOAD_ATOM_CNT + LOAD_PORT_CNT + @@ -1939,7 +1942,7 @@ send_async_error(ErlDrvPort port, ErlDrvTermData Port, int Ref, i = LOAD_TUPLE(spec, i, 4); ASSERT(i == sizeof(spec)/sizeof(*spec)); DEBUGF(("send_async_error %ld %ld\r\n", recipient, Reason)); - return driver_send_term(port, recipient, spec, i); + return erl_drv_send_term(Port, recipient, spec, i); } @@ -1951,7 +1954,7 @@ static int async_ok(inet_descriptor* desc) if (deq_async(desc, &aid, &caller, &req) < 0) return -1; - return send_async_ok(desc->port, desc->dport, aid, caller); + return send_async_ok(desc->dport, aid, caller); } static int async_ok_port(inet_descriptor* desc, ErlDrvTermData Port2) @@ -1962,7 +1965,7 @@ static int async_ok_port(inet_descriptor* desc, ErlDrvTermData Port2) if (deq_async(desc, &aid, &caller, &req) < 0) return -1; - return send_async_ok_port(desc->port, desc->dport, aid, caller, Port2); + return send_async_ok_port(desc->dport, aid, caller, Port2); } static int async_error_am(inet_descriptor* desc, ErlDrvTermData reason) @@ -1973,8 +1976,7 @@ static int async_error_am(inet_descriptor* desc, ErlDrvTermData reason) if (deq_async(desc, &aid, &caller, &req) < 0) return -1; - return send_async_error(desc->port, desc->dport, aid, caller, - reason); + return send_async_error(desc->dport, aid, caller, reason); } /* dequeue all operations */ @@ -1985,8 +1987,7 @@ static int async_error_am_all(inet_descriptor* desc, ErlDrvTermData reason) ErlDrvTermData caller; while (deq_async(desc, &aid, &caller, &req) == 0) { - send_async_error(desc->port, desc->dport, aid, caller, - reason); + send_async_error(desc->dport, aid, caller, reason); } return 0; } @@ -2014,7 +2015,7 @@ static int inet_reply_ok(inet_descriptor* desc) ASSERT(i == sizeof(spec)/sizeof(*spec)); desc->caller = 0; - return driver_send_term(desc->port, caller, spec, i); + return erl_drv_send_term(desc->dport, caller, spec, i); } #ifdef HAVE_SCTP @@ -2033,7 +2034,7 @@ static int inet_reply_ok_port(inet_descriptor* desc, ErlDrvTermData dport) ASSERT(i == sizeof(spec)/sizeof(*spec)); desc->caller = 0; - return driver_send_term(desc->port, caller, spec, i); + return erl_drv_send_term(desc->dport, caller, spec, i); } #endif @@ -2056,7 +2057,7 @@ static int inet_reply_error_am(inet_descriptor* desc, ErlDrvTermData reason) desc->caller = 0; DEBUGF(("inet_reply_error_am %ld %ld\r\n", caller, reason)); - return driver_send_term(desc->port, caller, spec, i); + return erl_drv_send_term(desc->dport, caller, spec, i); } /* send: @@ -2165,12 +2166,12 @@ static int http_response_inetdrv(void *arg, int major, int minor, i = LOAD_TUPLE(spec, i, 2); i = LOAD_TUPLE(spec, i, 4); ASSERT(i<=27); - return driver_send_term(desc->inet.port, caller, spec, i); + return erl_drv_send_term(desc->inet.dport, caller, spec, i); } else { i = LOAD_TUPLE(spec, i, 3); ASSERT(i<=27); - return driver_output_term(desc->inet.port, spec, i); + return erl_drv_output_term(desc->inet.dport, spec, i); } } @@ -2262,12 +2263,12 @@ http_request_inetdrv(void* arg, const http_atom_t* meth, const char* meth_ptr, i = LOAD_TUPLE(spec, i, 2); i = LOAD_TUPLE(spec, i, 4); ASSERT(i <= 43); - return driver_send_term(desc->inet.port, caller, spec, i); + return erl_drv_send_term(desc->inet.dport, caller, spec, i); } else { i = LOAD_TUPLE(spec, i, 3); ASSERT(i <= 43); - return driver_output_term(desc->inet.port, spec, i); + return erl_drv_output_term(desc->inet.dport, spec, i); } } @@ -2316,12 +2317,12 @@ http_header_inetdrv(void* arg, const http_atom_t* name, const char* name_ptr, i = LOAD_TUPLE(spec, i, 2); i = LOAD_TUPLE(spec, i, 4); ASSERT(i <= 26); - return driver_send_term(desc->inet.port, caller, spec, i); + return erl_drv_send_term(desc->inet.dport, caller, spec, i); } else { i = LOAD_TUPLE(spec, i, 3); ASSERT(i <= 26); - return driver_output_term(desc->inet.port, spec, i); + return erl_drv_output_term(desc->inet.dport, spec, i); } } @@ -2347,7 +2348,7 @@ static int http_eoh_inetdrv(void* arg) i = LOAD_TUPLE(spec, i, 2); i = LOAD_TUPLE(spec, i, 4); ASSERT(i <= 14); - return driver_send_term(desc->inet.port, caller, spec, i); + return erl_drv_send_term(desc->inet.dport, caller, spec, i); } else { /* {http, S, http_eoh} */ @@ -2356,7 +2357,7 @@ static int http_eoh_inetdrv(void* arg) i = LOAD_ATOM(spec, i, am_http_eoh); i = LOAD_TUPLE(spec, i, 3); ASSERT(i <= 14); - return driver_output_term(desc->inet.port, spec, i); + return erl_drv_output_term(desc->inet.dport, spec, i); } } @@ -2384,7 +2385,7 @@ static int http_error_inetdrv(void* arg, const char* buf, int len) i = LOAD_TUPLE(spec, i, 2); i = LOAD_TUPLE(spec, i, 4); ASSERT(i <= 19); - return driver_send_term(desc->inet.port, caller, spec, i); + return erl_drv_send_term(desc->inet.dport, caller, spec, i); } else { /* {http, S, {http_error,Line} */ @@ -2395,7 +2396,7 @@ static int http_error_inetdrv(void* arg, const char* buf, int len) i = LOAD_TUPLE(spec, i, 2); i = LOAD_TUPLE(spec, i, 3); ASSERT(i <= 19); - return driver_output_term(desc->inet.port, spec, i); + return erl_drv_output_term(desc->inet.dport, spec, i); } } @@ -2448,11 +2449,11 @@ int ssl_tls_inetdrv(void* arg, unsigned type, unsigned major, unsigned minor, i = LOAD_TUPLE(spec, i, 2); i = LOAD_TUPLE(spec, i, 4); ASSERT(i <= 28); - ret = driver_send_term(desc->inet.port, caller, spec, i); + ret = erl_drv_send_term(desc->inet.dport, caller, spec, i); } else { ASSERT(i <= 28); - ret = driver_output_term(desc->inet.port, spec, i); + ret = erl_drv_output_term(desc->inet.dport, spec, i); } done: driver_free_binary(bin); @@ -2502,7 +2503,7 @@ static int inet_async_data(inet_descriptor* desc, const char* buf, int len) i = LOAD_TUPLE(spec, i, 4); ASSERT(i == 15); desc->caller = 0; - return driver_send_term(desc->port, caller, spec, i); + return erl_drv_send_term(desc->dport, caller, spec, i); } else { /* INET_MODE_BINARY => [H1,H2,...HSz | Binary] */ @@ -2516,7 +2517,7 @@ static int inet_async_data(inet_descriptor* desc, const char* buf, int len) i = LOAD_TUPLE(spec, i, 4); ASSERT(i <= 20); desc->caller = 0; - code = driver_send_term(desc->port, caller, spec, i); + code = erl_drv_send_term(desc->dport, caller, spec, i); return code; } } @@ -3109,7 +3110,7 @@ inet_async_binary_data ASSERT(i <= PACKET_ERL_DRV_TERM_DATA_LEN); desc->caller = 0; - return driver_send_term(desc->port, caller, spec, i); + return erl_drv_send_term(desc->dport, caller, spec, i); } /* @@ -3132,7 +3133,7 @@ static int tcp_message(inet_descriptor* desc, const char* buf, int len) i = LOAD_STRING(spec, i, buf, len); /* => [H1,H2,...Hn] */ i = LOAD_TUPLE(spec, i, 3); ASSERT(i <= 20); - return driver_output_term(desc->port, spec, i); + return erl_drv_output_term(desc->dport, spec, i); } else { /* INET_MODE_BINARY => [H1,H2,...HSz | Binary] */ @@ -3144,7 +3145,7 @@ static int tcp_message(inet_descriptor* desc, const char* buf, int len) i = LOAD_STRING_CONS(spec, i, buf, hsz); i = LOAD_TUPLE(spec, i, 3); ASSERT(i <= 20); - code = driver_output_term(desc->port, spec, i); + code = erl_drv_output_term(desc->dport, spec, i); return code; } } @@ -3179,7 +3180,7 @@ tcp_binary_message(inet_descriptor* desc, ErlDrvBinary* bin, int offs, int len) } i = LOAD_TUPLE(spec, i, 3); ASSERT(i <= 20); - return driver_output_term(desc->port, spec, i); + return erl_drv_output_term(desc->dport, spec, i); } /* @@ -3198,7 +3199,7 @@ static int tcp_closed_message(tcp_descriptor* desc) i = LOAD_PORT(spec, i, desc->inet.dport); i = LOAD_TUPLE(spec, i, 2); ASSERT(i <= 6); - return driver_output_term(desc->inet.port, spec, i); + return erl_drv_output_term(desc->inet.dport, spec, i); } return 0; } @@ -3219,7 +3220,7 @@ static int tcp_error_message(tcp_descriptor* desc, int err) i = LOAD_ATOM(spec, i, am_err); i = LOAD_TUPLE(spec, i, 3); ASSERT(i <= 8); - return driver_output_term(desc->inet.port, spec, i); + return erl_drv_output_term(desc->inet.dport, spec, i); } /* @@ -3310,7 +3311,7 @@ static int packet_binary_message /* Close up the outer 5-tuple: */ i = LOAD_TUPLE(spec, i, 5); ASSERT(i <= PACKET_ERL_DRV_TERM_DATA_LEN); - return driver_output_term(desc->port, spec, i); + return erl_drv_output_term(desc->dport, spec, i); } /* @@ -3337,7 +3338,7 @@ static int packet_error_message(udp_descriptor* udesc, int err) i = LOAD_ATOM(spec, i, am_err); i = LOAD_TUPLE(spec, i, 3); ASSERT(i == sizeof(spec)/sizeof(*spec)); - return driver_output_term(desc->port, spec, i); + return erl_drv_output_term(desc->dport, spec, i); } @@ -5465,6 +5466,28 @@ static int inet_set_opts(inet_descriptor* desc, char* ptr, int len) } continue; + case INET_LOPT_TCP_MSGQ_HIWTRMRK: + if (desc->stype == SOCK_STREAM) { + ErlDrvSizeT high; + if (ival < ERL_DRV_BUSY_MSGQ_LIM_MIN + || ERL_DRV_BUSY_MSGQ_LIM_MAX < ival) + return -1; + high = (ErlDrvSizeT) ival; + erl_drv_busy_msgq_limits(desc->port, NULL, &high); + } + continue; + + case INET_LOPT_TCP_MSGQ_LOWTRMRK: + if (desc->stype == SOCK_STREAM) { + ErlDrvSizeT low; + if (ival < ERL_DRV_BUSY_MSGQ_LIM_MIN + || ERL_DRV_BUSY_MSGQ_LIM_MAX < ival) + return -1; + low = (ErlDrvSizeT) ival; + erl_drv_busy_msgq_limits(desc->port, &low, NULL); + } + continue; + case INET_LOPT_TCP_SEND_TIMEOUT: if (desc->stype == SOCK_STREAM) { tcp_descriptor* tdesc = (tcp_descriptor*) desc; @@ -6365,6 +6388,32 @@ static ErlDrvSSizeT inet_fill_opts(inet_descriptor* desc, } continue; + case INET_LOPT_TCP_MSGQ_HIWTRMRK: + if (desc->stype == SOCK_STREAM) { + ErlDrvSizeT high = ERL_DRV_BUSY_MSGQ_READ_ONLY; + *ptr++ = opt; + erl_drv_busy_msgq_limits(desc->port, NULL, &high); + ival = high > INT_MAX ? INT_MAX : (int) high; + put_int32(ival, ptr); + } + else { + TRUNCATE_TO(0,ptr); + } + continue; + + case INET_LOPT_TCP_MSGQ_LOWTRMRK: + if (desc->stype == SOCK_STREAM) { + ErlDrvSizeT low = ERL_DRV_BUSY_MSGQ_READ_ONLY; + *ptr++ = opt; + erl_drv_busy_msgq_limits(desc->port, &low, NULL); + ival = low > INT_MAX ? INT_MAX : (int) low; + put_int32(ival, ptr); + } + else { + TRUNCATE_TO(0,ptr); + } + continue; + case INET_LOPT_TCP_SEND_TIMEOUT: if (desc->stype == SOCK_STREAM) { *ptr++ = opt; @@ -7278,7 +7327,7 @@ static ErlDrvSSizeT sctp_fill_opts(inet_descriptor* desc, i = LOAD_TUPLE(spec, i, 3); /* Now, convert "spec" into the returnable term: */ - driver_send_term(desc->port, driver_caller(desc->port), spec, i); + erl_drv_send_term(desc->dport, driver_caller(desc->port), spec, i); FREE(spec); (*dest)[0] = INET_REP; @@ -7360,7 +7409,7 @@ send_empty_out_q_msgs(inet_descriptor* desc) ASSERT(msg_len == sizeof(msg)/sizeof(*msg)); - send_to_subscribers(desc->port, + send_to_subscribers(desc->dport, &desc->empty_out_q_subs, 1, msg, @@ -8007,6 +8056,7 @@ static int tcp_inet_init(void) static ErlDrvData tcp_inet_start(ErlDrvPort port, char* args) { + ErlDrvSizeT q_low, q_high; tcp_descriptor* desc; DEBUGF(("tcp_inet_start(%ld) {\r\n", (long)port)); @@ -8016,6 +8066,17 @@ static ErlDrvData tcp_inet_start(ErlDrvPort port, char* args) return ERL_DRV_ERROR_ERRNO; desc->high = INET_HIGH_WATERMARK; desc->low = INET_LOW_WATERMARK; + q_high = INET_HIGH_MSGQ_WATERMARK; + q_low = INET_LOW_MSGQ_WATERMARK; + if (q_low < ERL_DRV_BUSY_MSGQ_LIM_MIN) + q_low = ERL_DRV_BUSY_MSGQ_LIM_MIN; + else if (q_low > ERL_DRV_BUSY_MSGQ_LIM_MAX) + q_low = ERL_DRV_BUSY_MSGQ_LIM_MAX; + if (q_high < ERL_DRV_BUSY_MSGQ_LIM_MIN) + q_high = ERL_DRV_BUSY_MSGQ_LIM_MIN; + else if (q_high > ERL_DRV_BUSY_MSGQ_LIM_MAX) + q_high = ERL_DRV_BUSY_MSGQ_LIM_MAX; + erl_drv_busy_msgq_limits(port, &q_low, &q_high); desc->send_timeout = INET_INFINITY; desc->send_timeout_close = 0; desc->busy_on_send = 0; @@ -8039,6 +8100,7 @@ static ErlDrvData tcp_inet_start(ErlDrvPort port, char* args) static tcp_descriptor* tcp_inet_copy(tcp_descriptor* desc,SOCKET s, ErlDrvTermData owner, int* err) { + ErlDrvSizeT q_low, q_high; ErlDrvPort port = desc->inet.port; tcp_descriptor* copy_desc; @@ -8076,6 +8138,13 @@ static tcp_descriptor* tcp_inet_copy(tcp_descriptor* desc,SOCKET s, FREE(copy_desc); return NULL; } + + /* Read busy msgq limits of parent */ + q_low = q_high = ERL_DRV_BUSY_MSGQ_READ_ONLY; + erl_drv_busy_msgq_limits(desc->inet.port, &q_low, &q_high); + /* Write same busy msgq limits to child */ + erl_drv_busy_msgq_limits(port, &q_low, &q_high); + copy_desc->inet.port = port; copy_desc->inet.dport = driver_mk_port(port); *err = 0; @@ -8108,7 +8177,7 @@ static void tcp_close_check(tcp_descriptor* desc) desc->inet.state = INET_STATE_LISTENING; while (deq_multi_op(desc,&id,&req,&caller,NULL,&monitor) == 0) { driver_demonitor_process(desc->inet.port, &monitor); - send_async_error(desc->inet.port, desc->inet.dport, id, caller, am_closed); + send_async_error(desc->inet.dport, id, caller, am_closed); } clean_multi_timers(&(desc->mtd), desc->inet.port); } @@ -8532,7 +8601,7 @@ static void tcp_inet_multi_timeout(ErlDrvData e, ErlDrvTermData caller) sock_select(INETP(desc),FD_ACCEPT,0); desc->inet.state = INET_STATE_LISTENING; /* restore state */ } - send_async_error(desc->inet.port, desc->inet.dport, id, caller, am_timeout); + send_async_error(desc->inet.dport, id, caller, am_timeout); } @@ -9273,7 +9342,7 @@ static int tcp_inet_input(tcp_descriptor* desc, HANDLE event) if (s == INVALID_SOCKET) { /* Not ERRNO_BLOCK, that's handled right away */ - ret = send_async_error(desc->inet.port, desc->inet.dport, + ret = send_async_error(desc->inet.dport, id, caller, error_atom(sock_errno())); goto done; } @@ -9283,7 +9352,7 @@ static int tcp_inet_input(tcp_descriptor* desc, HANDLE event) if ((accept_desc = tcp_inet_copy(desc,s,caller,&err)) == NULL) { sock_close(s); - ret = send_async_error(desc->inet.port, desc->inet.dport, + ret = send_async_error(desc->inet.dport, id, caller, error_atom(err)); goto done; } @@ -9294,7 +9363,7 @@ static int tcp_inet_input(tcp_descriptor* desc, HANDLE event) ERL_DRV_READ, 1); #endif accept_desc->inet.state = INET_STATE_CONNECTED; - ret = send_async_ok_port(desc->inet.port, desc->inet.dport, + ret = send_async_ok_port(desc->inet.dport, id, caller, accept_desc->inet.dport); } } @@ -10936,7 +11005,7 @@ subs_list *subs; static void send_to_subscribers ( - ErlDrvPort port, + ErlDrvTermData port, subs_list *subs, int free_subs, ErlDrvTermData msg[], @@ -10953,7 +11022,7 @@ static void send_to_subscribers this = subs; while(this) { - (void) driver_send_term(port, this->subscriber, msg, msg_len); + (void) erl_drv_send_term(port, this->subscriber, msg, msg_len); if(free_subs && !first) { next = this->next; diff --git a/erts/emulator/hipe/hipe_amd64_bifs.m4 b/erts/emulator/hipe/hipe_amd64_bifs.m4 index 42e8b2a9b0..0de69a617f 100644 --- a/erts/emulator/hipe/hipe_amd64_bifs.m4 +++ b/erts/emulator/hipe/hipe_amd64_bifs.m4 @@ -2,7 +2,7 @@ changecom(`/*', `*/')dnl /* * %CopyrightBegin% * - * Copyright Ericsson AB 2004-2011. All Rights Reserved. + * Copyright Ericsson AB 2004-2012. All Rights Reserved. * * The contents of this file are subject to the Erlang Public License, * Version 1.1, (the "License"); you may not use this file except in diff --git a/erts/emulator/hipe/hipe_bif_list.m4 b/erts/emulator/hipe/hipe_bif_list.m4 index ea4f68ef24..764b8d180c 100644 --- a/erts/emulator/hipe/hipe_bif_list.m4 +++ b/erts/emulator/hipe/hipe_bif_list.m4 @@ -1,7 +1,7 @@ /* * %CopyrightBegin% * - * Copyright Ericsson AB 2004-2011. All Rights Reserved. + * Copyright Ericsson AB 2004-2012. All Rights Reserved. * * The contents of this file are subject to the Erlang Public License, * Version 1.1, (the "License"); you may not use this file except in @@ -145,6 +145,7 @@ * Zero-arity BIFs that can fail. */ standard_bif_interface_0(nbif_processes_0, processes_0) +standard_bif_interface_0(nbif_ports_0, ports_0) /* * BIFs and primops that may do a GC (change heap limit and walk the native stack). diff --git a/erts/emulator/hipe/hipe_debug.c b/erts/emulator/hipe/hipe_debug.c index 37615bf718..f2e9d03607 100644 --- a/erts/emulator/hipe/hipe_debug.c +++ b/erts/emulator/hipe/hipe_debug.c @@ -189,11 +189,10 @@ void hipe_print_pcb(Process *p) U("old_head ", old_heap); U("min_heap_..", min_heap_size); U("rcount ", rcount); - U("id ", id); - U("prio ", prio); + U("id ", common.id); U("reds ", reds); - U("tracer_pr..", tracer_proc); - U("trace_fla..", trace_flags); + U("tracer_pr..", common.tracer_proc); + U("trace_fla..", common.trace_flags); U("group_lea..", group_leader); U("flags ", flags); U("fvalue ", fvalue); @@ -202,8 +201,8 @@ void hipe_print_pcb(Process *p) /*XXX: ErlTimer tm; */ U("next ", next); /*XXX: ErlOffHeap off_heap; */ - U("reg ", reg); - U("nlinks ", nlinks); + U("reg ", common.u.alive.reg); + U("nlinks ", common.u.alive.links); /*XXX: ErlMessageQueue msg; */ U("mbuf ", mbuf); U("mbuf_sz ", mbuf_sz); diff --git a/erts/emulator/hipe/hipe_mkliterals.c b/erts/emulator/hipe/hipe_mkliterals.c index cbbf1db2e5..0e287908b1 100644 --- a/erts/emulator/hipe/hipe_mkliterals.c +++ b/erts/emulator/hipe/hipe_mkliterals.c @@ -262,47 +262,6 @@ static const struct literal { const char *name; int value; } literals[] = { - /* Field offsets in a process struct */ - { "P_HP", offsetof(struct process, htop) }, - { "P_HP_LIMIT", offsetof(struct process, stop) }, - { "P_OFF_HEAP_FIRST", offsetof(struct process, off_heap.first) }, - { "P_MBUF", offsetof(struct process, mbuf) }, - { "P_ID", offsetof(struct process, id) }, - { "P_FLAGS", offsetof(struct process, flags) }, - { "P_FVALUE", offsetof(struct process, fvalue) }, - { "P_FREASON", offsetof(struct process, freason) }, - { "P_FTRACE", offsetof(struct process, ftrace) }, - { "P_FCALLS", offsetof(struct process, fcalls) }, - { "P_BEAM_IP", offsetof(struct process, i) }, - { "P_ARITY", offsetof(struct process, arity) }, - { "P_ARG0", offsetof(struct process, def_arg_reg[0]) }, - { "P_ARG1", offsetof(struct process, def_arg_reg[1]) }, - { "P_ARG2", offsetof(struct process, def_arg_reg[2]) }, - { "P_ARG3", offsetof(struct process, def_arg_reg[3]) }, - { "P_ARG4", offsetof(struct process, def_arg_reg[4]) }, - { "P_ARG5", offsetof(struct process, def_arg_reg[5]) }, -#ifdef HIPE - { "P_NSP", offsetof(struct process, hipe.nsp) }, - { "P_NCALLEE", offsetof(struct process, hipe.ncallee) }, - { "P_CLOSURE", offsetof(struct process, hipe.closure) }, -#if defined(__i386__) || defined(__x86_64__) - { "P_NSP_LIMIT", offsetof(struct process, hipe.nstack) }, - { "P_CSP", offsetof(struct process, hipe.ncsp) }, -#elif defined(__sparc__) || defined(__powerpc__) || defined(__ppc__) || defined(__powerpc64__) || defined(__arm__) - { "P_NSP_LIMIT", offsetof(struct process, hipe.nstack) }, - { "P_NRA", offsetof(struct process, hipe.nra) }, -#endif - { "P_NARITY", offsetof(struct process, hipe.narity) }, - { "P_FLOAT_RESULT", -# ifdef NO_FPE_SIGNALS - offsetof(struct process, hipe.float_result) -# endif - }, -# if defined(ERTS_ENABLE_LOCK_CHECK) && defined(ERTS_SMP) - { "P_BIF_CALLEE", offsetof(struct process, hipe.bif_callee) }, -# endif -#endif /* HIPE */ - /* process flags bits */ { "F_TIMO", F_TIMO }, @@ -380,8 +339,6 @@ static const struct literal { { "MS_SAVEOFFSET_SIZE", field_sizeof(struct erl_bin_match_struct, save_offset)}, /* messages */ - { "P_MSG_FIRST", offsetof(struct process, msg.first) }, - { "P_MSG_SAVE", offsetof(struct process, msg.save) }, { "MSG_NEXT", offsetof(struct erl_mesg, next) }, /* ARM */ @@ -460,12 +417,14 @@ static const struct atom_literal { * These depend on configuration options such as heap architecture. * The compiler accesses these through hipe_bifs:get_rts_param/1. */ -static const struct rts_param { +struct rts_param { unsigned int nr; const char *name; unsigned int is_defined; int value; -} rts_params[] = { +}; + +static const struct rts_param rts_params[] = { { 1, "P_OFF_HEAP_FUNS", 1, offsetof(struct process, off_heap.first) }, @@ -518,7 +477,53 @@ static const struct rts_param { { 19, "MSG_MESSAGE", 1, offsetof(struct erl_mesg, m[0]) }, - /* highest entry ever used == 21 */ + + /* Field offsets in a process struct */ + { 22, "P_HP", 1, offsetof(struct process, htop) }, + { 23, "P_HP_LIMIT", 1, offsetof(struct process, stop) }, + { 24, "P_OFF_HEAP_FIRST", 1, offsetof(struct process, off_heap.first) }, + { 25, "P_MBUF", 1, offsetof(struct process, mbuf) }, + { 26, "P_ID", 1, offsetof(struct process, common.id) }, + { 27, "P_FLAGS", 1, offsetof(struct process, flags) }, + { 28, "P_FVALUE", 1, offsetof(struct process, fvalue) }, + { 29, "P_FREASON", 1, offsetof(struct process, freason) }, + { 30, "P_FTRACE", 1, offsetof(struct process, ftrace) }, + { 31, "P_FCALLS", 1, offsetof(struct process, fcalls) }, + { 32, "P_BEAM_IP", 1, offsetof(struct process, i) }, + { 33, "P_ARITY", 1, offsetof(struct process, arity) }, + { 34, "P_ARG0", 1, offsetof(struct process, def_arg_reg[0]) }, + { 35, "P_ARG1", 1, offsetof(struct process, def_arg_reg[1]) }, + { 36, "P_ARG2", 1, offsetof(struct process, def_arg_reg[2]) }, + { 37, "P_ARG3", 1, offsetof(struct process, def_arg_reg[3]) }, + { 38, "P_ARG4", 1, offsetof(struct process, def_arg_reg[4]) }, + { 39, "P_ARG5", 1, offsetof(struct process, def_arg_reg[5]) }, + { 40, "P_NSP", 1, offsetof(struct process, hipe.nsp) }, + { 41, "P_NCALLEE", 1, offsetof(struct process, hipe.ncallee) }, + { 42, "P_CLOSURE", 1, offsetof(struct process, hipe.closure) }, + { 43, "P_NSP_LIMIT", 1, offsetof(struct process, hipe.nstack) }, + { 44, "P_CSP", +#if defined(__i386__) || defined(__x86_64__) + 1, offsetof(struct process, hipe.ncsp) +#endif + }, + { 45, "P_NRA", +#if defined(__sparc__) || defined(__powerpc__) || defined(__ppc__) || defined(__powerpc64__) || defined(__arm__) + 1, offsetof(struct process, hipe.nra) +#endif + }, + { 46, "P_NARITY", 1, offsetof(struct process, hipe.narity) }, + { 47, "P_FLOAT_RESULT", +#ifdef NO_FPE_SIGNALS + 1, offsetof(struct process, hipe.float_result) +#endif + }, + { 48, "P_BIF_CALLEE", +#if defined(ERTS_ENABLE_LOCK_CHECK) && defined(ERTS_SMP) + 1, offsetof(struct process, hipe.bif_callee) +#endif + }, + { 49, "P_MSG_FIRST", 1, offsetof(struct process, msg.first) }, + { 50, "P_MSG_SAVE", 1, offsetof(struct process, msg.save) }, }; #define NR_PARAMS ARRAY_SIZE(rts_params) diff --git a/erts/emulator/hipe/hipe_x86.c b/erts/emulator/hipe/hipe_x86.c index 24d232c968..4281730ae2 100644 --- a/erts/emulator/hipe/hipe_x86.c +++ b/erts/emulator/hipe/hipe_x86.c @@ -265,7 +265,7 @@ void *hipe_make_native_stub(void *beamAddress, unsigned int beamArity) void hipe_arch_print_pcb(struct hipe_process_state *p) { #define U(n,x) \ - printf(" % 4d | %s | 0x%08x | |\r\n", offsetof(struct hipe_process_state,x), n, (unsigned)p->x) + printf(" % 4d | %s | 0x%08x | |\r\n", (int)offsetof(struct hipe_process_state,x), n, (unsigned)p->x) U("ncsp ", ncsp); U("narity ", narity); #undef U diff --git a/erts/emulator/hipe/hipe_x86_bifs.m4 b/erts/emulator/hipe/hipe_x86_bifs.m4 index 8cc6340933..dd6980f555 100644 --- a/erts/emulator/hipe/hipe_x86_bifs.m4 +++ b/erts/emulator/hipe/hipe_x86_bifs.m4 @@ -2,7 +2,7 @@ changecom(`/*', `*/')dnl /* * %CopyrightBegin% * - * Copyright Ericsson AB 2001-2011. All Rights Reserved. + * Copyright Ericsson AB 2001-2012. All Rights Reserved. * * The contents of this file are subject to the Erlang Public License, * Version 1.1, (the "License"); you may not use this file except in diff --git a/erts/emulator/sys/common/erl_check_io.c b/erts/emulator/sys/common/erl_check_io.c index ce014c19c2..474408ae7c 100644 --- a/erts/emulator/sys/common/erl_check_io.c +++ b/erts/emulator/sys/common/erl_check_io.c @@ -200,17 +200,6 @@ static void event_large_fd_error(ErlDrvPort, ErtsSysFdType, ErlDrvEventData); #endif static void steal_pending_stop_select(erts_dsprintf_buf_t*, ErlDrvPort, ErtsDrvEventState*, int mode, int on); -static ERTS_INLINE Eterm -drvport2id(ErlDrvPort dp) -{ - Port *pp = erts_drvport2port(dp); - if (pp) - return pp->id; - else { - ASSERT(0); - return am_undefined; - } -} #ifdef ERTS_SMP ERTS_SCHED_PREF_QUICK_ALLOC_IMPL(removed_fd, struct removed_fd, 64, ERTS_ALC_T_FD_LIST) @@ -378,7 +367,7 @@ abort_task(Eterm id, ErtsPortTaskHandle *pthp, EventStateType type) || !erts_port_task_is_scheduled(pthp)); } else if (erts_port_task_is_scheduled(pthp)) { - erts_port_task_abort(id, pthp); + erts_port_task_abort(pthp); ASSERT(erts_is_port_alive(id)); } } @@ -492,7 +481,7 @@ ERTS_CIO_EXPORT(driver_select)(ErlDrvPort ix, int on) { void (*stop_select_fn)(ErlDrvEvent, void*) = NULL; - Eterm id = drvport2id(ix); + Eterm id = erts_drvport2id(ix); ErtsSysFdType fd = (ErtsSysFdType) e; ErtsPollEvents ctl_events = (ErtsPollEvents) 0; ErtsPollEvents new_events, old_events; @@ -503,8 +492,8 @@ ERTS_CIO_EXPORT(driver_select)(ErlDrvPort ix, DTRACE_CHARBUF(name, 64); #endif - ERTS_SMP_LC_ASSERT(erts_drvport2port(ix) - && erts_lc_is_port_locked(erts_drvport2port(ix))); + ERTS_SMP_LC_ASSERT(erts_drvport2port(ix, NULL) + && erts_lc_is_port_locked(erts_drvport2port(ix, NULL))); #ifdef ERTS_SYS_CONTINOUS_FD_NUMBERS if ((unsigned)fd >= (unsigned)erts_smp_atomic_read_nob(&drv_ev_state_len)) { @@ -530,9 +519,9 @@ ERTS_CIO_EXPORT(driver_select)(ErlDrvPort ix, if (!on && (mode&ERL_DRV_USE_NO_CALLBACK) == ERL_DRV_USE) { if (IS_FD_UNKNOWN(state)) { /* fast track to stop_select callback */ - stop_select_fn = erts_drvport2port(ix)->drv_ptr->stop_select; + stop_select_fn = erts_drvport2port(ix, NULL)->drv_ptr->stop_select; #ifdef USE_VM_PROBES - strncpy(name, erts_drvport2port(ix)->drv_ptr->name, sizeof(name)-1); + strncpy(name, erts_drvport2port(ix, NULL)->drv_ptr->name, sizeof(name)-1); name[sizeof(name)-1] = '\0'; #endif ret = 0; @@ -665,14 +654,14 @@ ERTS_CIO_EXPORT(driver_select)(ErlDrvPort ix, } } if ((mode & ERL_DRV_USE_NO_CALLBACK) == ERL_DRV_USE) { - erts_driver_t* drv_ptr = erts_drvport2port(ix)->drv_ptr; + erts_driver_t* drv_ptr = erts_drvport2port(ix, NULL)->drv_ptr; ASSERT(new_events==0); if (state->remove_cnt == 0 || !wake_poller) { /* Safe to close fd now as it is not in pollset or there was no need to eject fd (kernel poll) */ stop_select_fn = drv_ptr->stop_select; #ifdef USE_VM_PROBES - strncpy(name, erts_drvport2port(ix)->drv_ptr->name, sizeof(name)-1); + strncpy(name, erts_drvport2port(ix, NULL)->drv_ptr->name, sizeof(name)-1); name[sizeof(name)-1] = '\0'; #endif } @@ -719,13 +708,13 @@ ERTS_CIO_EXPORT(driver_event)(ErlDrvPort ix, ErtsPollEvents events; ErtsPollEvents add_events; ErtsPollEvents remove_events; - Eterm id = drvport2id(ix); + Eterm id = erts_drvport2id(ix); ErtsDrvEventState *state; int do_wake = 0; int ret; - ERTS_SMP_LC_ASSERT(erts_drvport2port(ix) - && erts_lc_is_port_locked(erts_drvport2port(ix))); + ERTS_SMP_LC_ASSERT(erts_drvport2port(ix, NULL) + && erts_lc_is_port_locked(erts_drvport2port(ix, NULL))); #ifdef ERTS_SYS_CONTINOUS_FD_NUMBERS if ((unsigned)fd >= (unsigned)erts_smp_atomic_read_nob(&drv_ev_state_len)) { @@ -960,7 +949,7 @@ static void print_select_op(erts_dsprintf_buf_t *dsbufp, ErlDrvPort ix, ErtsSysFdType fd, int mode, int on) { - Port *pp = erts_drvport2port(ix); + Port *pp = erts_drvport2port(ix, NULL); erts_dsprintf(dsbufp, "driver_select(%p, %d,%s%s%s%s, %d) " "by ", @@ -971,8 +960,8 @@ print_select_op(erts_dsprintf_buf_t *dsbufp, mode & ERL_DRV_USE ? " ERL_DRV_USE" : "", mode & (ERL_DRV_USE_NO_CALLBACK & ~ERL_DRV_USE) ? "_NO_CALLBACK" : "", on); - print_driver_name(dsbufp, pp->id); - erts_dsprintf(dsbufp, "driver %T ", pp ? pp->id : NIL); + print_driver_name(dsbufp, pp->common.id); + erts_dsprintf(dsbufp, "driver %T ", pp ? pp->common.id : NIL); } static void @@ -1031,7 +1020,7 @@ steal_pending_stop_select(erts_dsprintf_buf_t *dsbufp, ErlDrvPort ix, state->driver.drv_ptr = NULL; } else if ((mode & ERL_DRV_USE_NO_CALLBACK) == ERL_DRV_USE) { - erts_driver_t* drv_ptr = erts_drvport2port(ix)->drv_ptr; + erts_driver_t* drv_ptr = erts_drvport2port(ix, NULL)->drv_ptr; if (drv_ptr != state->driver.drv_ptr) { /* Some other driver wants the stop_select callback */ if (state->driver.drv_ptr->handle) { @@ -1053,7 +1042,7 @@ static void print_event_op(erts_dsprintf_buf_t *dsbufp, ErlDrvPort ix, ErtsSysFdType fd, ErlDrvEventData event_data) { - Port *pp = erts_drvport2port(ix); + Port *pp = erts_drvport2port(ix, NULL); erts_dsprintf(dsbufp, "driver_event(%p, %d, ", ix, (int) fd); if (!event_data) erts_dsprintf(dsbufp, "NULL"); @@ -1062,8 +1051,8 @@ print_event_op(erts_dsprintf_buf_t *dsbufp, (unsigned int) event_data->events, (unsigned int) event_data->revents); erts_dsprintf(dsbufp, ") by "); - print_driver_name(dsbufp, pp->id); - erts_dsprintf(dsbufp, "driver %T ", pp ? pp->id : NIL); + print_driver_name(dsbufp, pp->common.id); + erts_dsprintf(dsbufp, "driver %T ", pp ? pp->common.id : NIL); } static void @@ -1100,8 +1089,7 @@ iready(Eterm id, ErtsDrvEventState *state) if (erts_port_task_schedule(id, &state->driver.select->intask, ERTS_PORT_TASK_INPUT, - (ErlDrvEvent) state->fd, - NULL) != 0) { + (ErlDrvEvent) state->fd) != 0) { stale_drv_select(id, state, ERL_DRV_READ); } } @@ -1112,8 +1100,7 @@ oready(Eterm id, ErtsDrvEventState *state) if (erts_port_task_schedule(id, &state->driver.select->outtask, ERTS_PORT_TASK_OUTPUT, - (ErlDrvEvent) state->fd, - NULL) != 0) { + (ErlDrvEvent) state->fd) != 0) { stale_drv_select(id, state, ERL_DRV_WRITE); } } diff --git a/erts/emulator/sys/common/erl_mseg.c b/erts/emulator/sys/common/erl_mseg.c index db2854fa40..94f9f76a20 100644 --- a/erts/emulator/sys/common/erl_mseg.c +++ b/erts/emulator/sys/common/erl_mseg.c @@ -57,26 +57,48 @@ /* Implement some other way to get the real page size if needed! */ #endif -#define MAX_CACHE_SIZE 30 - #undef MIN #define MIN(X, Y) ((X) < (Y) ? (X) : (Y)) #undef MAX #define MAX(X, Y) ((X) > (Y) ? (X) : (Y)) -#undef PAGE_MASK -#define INV_PAGE_MASK ((Uint) (page_size - 1)) -#define PAGE_MASK (~INV_PAGE_MASK) -#define PAGE_FLOOR(X) ((X) & PAGE_MASK) -#define PAGE_CEILING(X) PAGE_FLOOR((X) + INV_PAGE_MASK) -#define PAGES(X) ((X) >> page_shift) +#define INV_ALIGNED_MASK ((UWord) ((MSEG_ALIGNED_SIZE) - 1)) +#define ALIGNED_MASK (~INV_ALIGNED_MASK) +#define ALIGNED_FLOOR(X) (((UWord)(X)) & ALIGNED_MASK) +#define ALIGNED_CEILING(X) ALIGNED_FLOOR((X) + INV_ALIGNED_MASK) +#define MAP_IS_ALIGNED(X) (((UWord)(X) & (MSEG_ALIGNED_SIZE - 1)) == 0) + +#define IS_2POW(X) ((X) && !((X) & ((X) - 1))) +static ERTS_INLINE Uint ceil_2pow(Uint x) { + int i = 1 << (4 + (sizeof(Uint) != 4 ? 1 : 0)); + x--; + do { x |= x >> i; } while(i >>= 1); + return x + 1; +} +static const int debruijn[32] = { + 0, 1, 28, 2, 29, 14, 24, 3, 30, 22, 20, 15, 25, 17, 4, 8, + 31, 27, 13, 23, 21, 19, 16, 7, 26, 12, 18, 6, 11, 5, 10, 9 +}; + +#define LOG2(X) (debruijn[((Uint32)(((X) & -(X)) * 0x077CB531U)) >> 27]) + +#define CACHE_AREAS (32 - MSEG_ALIGN_BITS) + +#define SIZE_TO_CACHE_AREA_IDX(S) (LOG2((S)) - MSEG_ALIGN_BITS) +#define MAX_CACHE_SIZE (30) + +#define MSEG_FLG_IS_2POW(X) ((X) & ERTS_MSEG_FLG_2POW) + +#ifdef DEBUG +#define DBG(F,...) fprintf(stderr, (F), __VA_ARGS__ ) +#else +#define DBG(F,...) do{}while(0) +#endif static int atoms_initialized; typedef struct mem_kind_t MemKind; -static void mseg_clear_cache(MemKind*); - #if HALFWORD_HEAP static int initialize_pmmap(void); static void *pmmap(size_t size); @@ -116,15 +138,6 @@ static int mmap_fd; #error "Not supported" #endif /* #if HAVE_MMAP */ -#if defined(ERTS_MSEG_FAKE_SEGMENTS) && HALFWORD_HEAP -# warning "ERTS_MSEG_FAKE_SEGMENTS will only be used for high memory segments" -#endif - -#if defined(ERTS_MSEG_FAKE_SEGMENTS) -#undef CAN_PARTLY_DESTROY -#define CAN_PARTLY_DESTROY 0 -#endif - const ErtsMsegOpt_t erts_mseg_default_opt = { 1, /* Use cache */ 1, /* Preserv data */ @@ -137,26 +150,17 @@ const ErtsMsegOpt_t erts_mseg_default_opt = { }; -typedef struct cache_desc_t_ { - void *seg; - Uint size; - struct cache_desc_t_ *next; - struct cache_desc_t_ *prev; -} cache_desc_t; - typedef struct { Uint32 giga_no; Uint32 no; } CallCounter; -static Uint page_size; -static Uint page_shift; - typedef struct { CallCounter alloc; CallCounter dealloc; CallCounter realloc; CallCounter create; + CallCounter create_resize; CallCounter destroy; #if HAVE_MSEG_RECREATE CallCounter recreate; @@ -165,17 +169,25 @@ typedef struct { CallCounter check_cache; } ErtsMsegCalls; +typedef struct cache_t_ cache_t; + +struct cache_t_ { + Uint size; + void *seg; + cache_t *next; +}; + + typedef struct ErtsMsegAllctr_t_ ErtsMsegAllctr_t; struct mem_kind_t { - cache_desc_t cache_descs[MAX_CACHE_SIZE]; - cache_desc_t *free_cache_descs; - cache_desc_t *cache; - cache_desc_t *cache_end; - - Uint cache_size; - Uint min_cached_seg_size; - Uint max_cached_seg_size; + + cache_t cache[MAX_CACHE_SIZE]; + cache_t *cache_unpowered; + cache_t *cache_area[CACHE_AREAS]; + cache_t *cache_free; + + Sint cache_size; Uint cache_hits; struct { @@ -320,8 +332,7 @@ static erts_mtx_t init_atoms_mutex; /* Also needed when !USE_THREADS */ static ERTS_INLINE void -schedule_cache_check(ErtsMsegAllctr_t *ma) -{ +schedule_cache_check(ErtsMsegAllctr_t *ma) { if (!ma->is_cache_check_scheduled && ma->is_init_done) { erts_set_aux_work_timeout(ma->ix, @@ -331,12 +342,45 @@ schedule_cache_check(ErtsMsegAllctr_t *ma) } } +/* remove ErtsMsegAllctr_t from arguments? + * only used for statistics + */ +static ERTS_INLINE void * +mmap_align(ErtsMsegAllctr_t *ma, void *addr, size_t length, int prot, int flags, int fd, off_t offset) { + + void *p, *q; + UWord d; + + p = mmap(addr, length, prot, flags, fd, offset); + + if (MAP_IS_ALIGNED(p) || p == MAP_FAILED) + return p; + + if (ma) + INC_CC(ma, create_resize); + + munmap(p, length); + + if ((p = mmap(addr, length + MSEG_ALIGNED_SIZE, prot, flags, fd, offset)) == MAP_FAILED) + return MAP_FAILED; + + q = (void *)ALIGNED_CEILING(p); + d = q - p; + + if (d > 0) + munmap(p, d); + + if (MSEG_ALIGNED_SIZE - d > 0) + munmap((void *) (q + length), MSEG_ALIGNED_SIZE - d); + + return q; +} + static ERTS_INLINE void * mseg_create(ErtsMsegAllctr_t *ma, MemKind* mk, Uint size) { void *seg; - - ASSERT(size % page_size == 0); + ASSERT(size % MSEG_ALIGNED_SIZE == 0); #if HALFWORD_HEAP if (mk == &ma->low_mem) { @@ -345,18 +389,17 @@ mseg_create(ErtsMsegAllctr_t *ma, MemKind* mk, Uint size) erts_fprintf(stderr,"Pointer mask failure (0x%08lx)\n",(unsigned long) seg); return NULL; } - } - else + } else #endif { -#if defined(ERTS_MSEG_FAKE_SEGMENTS) - seg = erts_sys_alloc(ERTS_ALC_N_INVALID, NULL, size); -#elif HAVE_MMAP +#if HAVE_MMAP { - seg = (void *) mmap((void *) 0, (size_t) size, + seg = (void *) mmap_align(ma, (void *) 0, (size_t) size, MMAP_PROT, MMAP_FLAGS, MMAP_FD, 0); if (seg == (void *) MAP_FAILED) seg = NULL; + + ASSERT(MAP_IS_ALIGNED(seg) || !seg); } #else # error "Missing mseg_create() implementation" @@ -369,38 +412,24 @@ mseg_create(ErtsMsegAllctr_t *ma, MemKind* mk, Uint size) } static ERTS_INLINE void -mseg_destroy(ErtsMsegAllctr_t *ma, MemKind* mk, void *seg, Uint size) -{ -#ifdef DEBUG - int res; -#endif +mseg_destroy(ErtsMsegAllctr_t *ma, MemKind* mk, void *seg, Uint size) { + ERTS_DECLARE_DUMMY(int res); #if HALFWORD_HEAP if (mk == &ma->low_mem) { -#ifdef DEBUG - res = -#endif - pmunmap((void *) seg, size); + res = pmunmap((void *) seg, size); } else #endif { -#ifdef ERTS_MSEG_FAKE_SEGMENTS - erts_sys_free(ERTS_ALC_N_INVALID, NULL, seg); -#ifdef DEBUG - res = 0; -#endif -#elif HAVE_MMAP -#ifdef DEBUG - res = -#endif - munmap((void *) seg, size); +#ifdef HAVE_MMAP + res = munmap((void *) seg, size); #else # error "Missing mseg_destroy() implementation" #endif } - ASSERT(size % page_size == 0); + ASSERT(size % MSEG_ALIGNED_SIZE == 0); ASSERT(res == 0); INC_CC(ma, destroy); @@ -408,14 +437,36 @@ mseg_destroy(ErtsMsegAllctr_t *ma, MemKind* mk, void *seg, Uint size) } #if HAVE_MSEG_RECREATE +#if defined(__NetBsd__) +#define MREMAP_FLAGS (0) +#else +#define MREMAP_FLAGS (MREMAP_MAYMOVE) +#endif + + +/* mseg_recreate + * May return *unaligned* segments as in address not aligned to MSEG_ALIGNMENT + * it is still page aligned + * + * This is fine for single block carriers as long as we don't cache misaligned + * segments (since multiblock carriers may use them) + * + * For multiblock carriers we *need* MSEG_ALIGNMENT but mbc's will never be + * reallocated. + * + * This should probably be fixed the following way: + * 1) Use an option to segment allocation - NEED_ALIGNMENT + * 2) Add mremap_align which takes care of aligning a new a mremaped area + * 3) Fix the cache to handle of aligned and unaligned segments + */ static ERTS_INLINE void * mseg_recreate(ErtsMsegAllctr_t *ma, MemKind* mk, void *old_seg, Uint old_size, Uint new_size) { void *new_seg; - ASSERT(old_size % page_size == 0); - ASSERT(new_size % page_size == 0); + ASSERT(old_size % MSEG_ALIGNED_SIZE == 0); + ASSERT(new_size % MSEG_ALIGNED_SIZE == 0); #if HALFWORD_HEAP if (mk == &ma->low_mem) { @@ -426,22 +477,12 @@ mseg_recreate(ErtsMsegAllctr_t *ma, MemKind* mk, void *old_seg, Uint old_size, U else #endif { -#if defined(ERTS_MSEG_FAKE_SEGMENTS) - new_seg = erts_sys_realloc(ERTS_ALC_N_INVALID, NULL, old_seg, new_size); -#elif HAVE_MREMAP - - #if defined(__NetBSD__) - new_seg = (void *) mremap((void *) old_seg, - (size_t) old_size, - NULL, - (size_t) new_size, - 0); - #else - new_seg = (void *) mremap((void *) old_seg, - (size_t) old_size, - (size_t) new_size, - MREMAP_MAYMOVE); - #endif +#if HAVE_MREMAP +#if defined(__NetBSD__) + new_seg = mremap(old_seg, (size_t)old_size, NULL, new_size, MREMAP_FLAGS); +#else + new_seg = mremap(old_seg, (size_t)old_size, (size_t)new_size, MREMAP_FLAGS); +#endif if (new_seg == (void *) MAP_FAILED) new_seg = NULL; #else @@ -475,151 +516,265 @@ do { \ #define ERTS_DBG_MK_CHK_THR_ACCESS(MK) #endif -static ERTS_INLINE cache_desc_t * -alloc_cd(MemKind* mk) -{ - cache_desc_t *cd = mk->free_cache_descs; +/* NEW CACHE interface */ + +static ERTS_INLINE cache_t *mseg_cache_alloc_descriptor(MemKind *mk) { + cache_t *c = mk->cache_free; + ERTS_DBG_MK_CHK_THR_ACCESS(mk); - if (cd) - mk->free_cache_descs = cd->next; - return cd; + if (c) + mk->cache_free = c->next; + + return c; } -static ERTS_INLINE void -free_cd(MemKind* mk, cache_desc_t *cd) -{ +static ERTS_INLINE void mseg_cache_free_descriptor(MemKind *mk, cache_t *c) { ERTS_DBG_MK_CHK_THR_ACCESS(mk); - cd->next = mk->free_cache_descs; - mk->free_cache_descs = cd; + ASSERT(c); + + c->seg = NULL; + c->size = 0; + c->next = mk->cache_free; + mk->cache_free = c; } +static ERTS_INLINE int cache_bless_segment(MemKind *mk, void *seg, Uint size) { -static ERTS_INLINE void -link_cd(MemKind* mk, cache_desc_t *cd) -{ + cache_t *c; ERTS_DBG_MK_CHK_THR_ACCESS(mk); - if (mk->cache) - mk->cache->prev = cd; - cd->next = mk->cache; - cd->prev = NULL; - mk->cache = cd; - - if (!mk->cache_end) { - ASSERT(!cd->next); - mk->cache_end = cd; + + if (mk->cache_free && MAP_IS_ALIGNED(seg)) { + if (IS_2POW(size)) { + int ix = SIZE_TO_CACHE_AREA_IDX(size); + + ASSERT(ix < CACHE_AREAS); + ASSERT((1 << (ix + MSEG_ALIGN_BITS)) == size); + + /* unlink from free cache list */ + c = mseg_cache_alloc_descriptor(mk); + + /* link to cache area */ + c->seg = seg; + c->size = size; + c->next = mk->cache_area[ix]; + + mk->cache_area[ix] = c; + mk->cache_size++; + + ASSERT(mk->cache_size <= mk->ma->max_cache_size); + + return 1; + } else { + /* unlink from free cache list */ + c = mseg_cache_alloc_descriptor(mk); + + /* link to cache area */ + c->seg = seg; + c->size = size; + c->next = mk->cache_unpowered; + + mk->cache_unpowered = c; + mk->cache_size++; + + ASSERT(mk->cache_size <= mk->ma->max_cache_size); + + return 1; + } } - mk->cache_size++; + return 0; } -#if CAN_PARTLY_DESTROY -static ERTS_INLINE void -end_link_cd(MemKind* mk, cache_desc_t *cd) -{ +static ERTS_INLINE void *cache_get_segment(MemKind *mk, Uint *size_p) { + + Uint size = *size_p; + ERTS_DBG_MK_CHK_THR_ACCESS(mk); - if (mk->cache_end) - mk->cache_end->next = cd; - cd->next = NULL; - cd->prev = mk->cache_end; - mk->cache_end = cd; - - if (!mk->cache) { - ASSERT(!cd->prev); - mk->cache = cd; - } + if (IS_2POW(size)) { + + int i, ix = SIZE_TO_CACHE_AREA_IDX(size); + void *seg; + cache_t *c; + Uint csize; + + for( i = ix; i < CACHE_AREAS; i++) { + + if ((c = mk->cache_area[i]) == NULL) + continue; + + ASSERT(IS_2POW(c->size)); + + /* unlink from cache area */ + csize = c->size; + seg = c->seg; + mk->cache_area[i] = c->next; + c->next = NULL; + mk->cache_size--; + mk->cache_hits++; + + /* link to free cache list */ + mseg_cache_free_descriptor(mk, c); + + ASSERT(!(mk->cache_size < 0)); + + /* divvy up the cache - if needed */ + while( i > ix) { + csize = csize >> 1; + /* try to cache half of it */ + if (!cache_bless_segment(mk, (char *)seg + csize, csize)) { + /* wouldn't cache .. destroy it instead */ + mseg_destroy(mk->ma, mk, (char *)seg + csize, csize); + } + i--; + } + ASSERT(csize == size); + return seg; + } + } + else if (mk->cache_unpowered) { + void *seg; + cache_t *c, *pc; + Uint csize; + Uint bad_max_abs = mk->ma->abs_max_cache_bad_fit; + Uint bad_max_rel = mk->ma->rel_max_cache_bad_fit; + + c = mk->cache_unpowered; + pc = c; + + while (c) { + csize = c->size; + if (csize >= size && + ((csize - size)*100 < bad_max_rel*size) && + (csize - size) < bad_max_abs ) { + + /* unlink from cache area */ + seg = c->seg; + + if (pc == c) { + mk->cache_unpowered = c->next; + } else { + pc->next = c->next; + } + + c->next = NULL; + mk->cache_size--; + mk->cache_hits++; + + /* link to free cache list */ + mseg_cache_free_descriptor(mk, c); + *size_p = csize; + + return seg; + } - mk->cache_size++; + pc = c; + c = c->next; + } + } + return NULL; } -#endif -static ERTS_INLINE void -unlink_cd(MemKind* mk, cache_desc_t *cd) -{ - ERTS_DBG_MK_CHK_THR_ACCESS(mk); - if (cd->next) - cd->next->prev = cd->prev; - else - mk->cache_end = cd->prev; - - if (cd->prev) - cd->prev->next = cd->next; - else - mk->cache = cd->next; - ASSERT(mk->cache_size > 0); +/* *_mseg_check_*_cache + * Slowly remove segments cached in the allocator by + * using callbacks from aux-work in the scheduler. + */ + +static ERTS_INLINE Uint mseg_drop_one_memkind_cache_size(MemKind *mk, cache_t **head) { + cache_t *c = NULL; + + c = *head; + + ASSERT( c != NULL ); + + *head = c->next; + + if (erts_mtrace_enabled) + erts_mtrace_crr_free(SEGTYPE, SEGTYPE, c->seg); + + mseg_destroy(mk->ma, mk, c->seg, c->size); + mseg_cache_free_descriptor(mk, c); + + mk->segments.current.watermark--; mk->cache_size--; -} -static ERTS_INLINE void -check_cache_limits(MemKind* mk) -{ - cache_desc_t *cd; - ERTS_DBG_MK_CHK_THR_ACCESS(mk); - mk->max_cached_seg_size = 0; - mk->min_cached_seg_size = ~((Uint) 0); - for (cd = mk->cache; cd; cd = cd->next) { - if (cd->size < mk->min_cached_seg_size) - mk->min_cached_seg_size = cd->size; - if (cd->size > mk->max_cached_seg_size) - mk->max_cached_seg_size = cd->size; - } + ASSERT( mk->cache_size >= 0 ); + + return mk->cache_size; } -static ERTS_INLINE void -adjust_cache_size(MemKind* mk, int force_check_limits) -{ - cache_desc_t *cd; - int check_limits = force_check_limits; - Sint max_cached = ((Sint) mk->segments.current.watermark - - (Sint) mk->segments.current.no); - ERTS_DBG_MK_CHK_THR_ACCESS(mk); - while (((Sint) mk->cache_size) > max_cached && ((Sint) mk->cache_size) > 0) { - ASSERT(mk->cache_end); - cd = mk->cache_end; - if (!check_limits && - !(mk->min_cached_seg_size < cd->size - && cd->size < mk->max_cached_seg_size)) { - check_limits = 1; - } +static ERTS_INLINE Uint mseg_drop_memkind_cache_size(MemKind *mk, cache_t **head) { + cache_t *c = NULL, *next = NULL; + + c = *head; + ASSERT( c != NULL ); + + while (c) { + + next = c->next; + if (erts_mtrace_enabled) - erts_mtrace_crr_free(SEGTYPE, SEGTYPE, cd->seg); - mseg_destroy(mk->ma, mk, cd->seg, cd->size); - unlink_cd(mk,cd); - free_cd(mk,cd); - } + erts_mtrace_crr_free(SEGTYPE, SEGTYPE, c->seg); - if (check_limits) - check_cache_limits(mk); -} + mseg_destroy(mk->ma, mk, c->seg, c->size); + mseg_cache_free_descriptor(mk, c); -static Uint -check_one_cache(MemKind* mk) -{ - if (mk->segments.current.watermark > mk->segments.current.no) mk->segments.current.watermark--; - adjust_cache_size(mk, 0); + mk->cache_size--; + + c = next; + } + + *head = NULL; + + ASSERT( mk->cache_size >= 0 ); - if (mk->cache_size) - schedule_cache_check(mk->ma); return mk->cache_size; } -static void do_cache_check(ErtsMsegAllctr_t *ma) -{ - int empty_cache = 1; +/* mseg_check_memkind_cache + * - Check if we can empty some cached segments in this + * MemKind. + */ + + +static Uint mseg_check_memkind_cache(MemKind *mk) { + int i; + + ERTS_DBG_MK_CHK_THR_ACCESS(mk); + + for (i = 0; i < CACHE_AREAS; i++) { + if (mk->cache_area[i] != NULL) + return mseg_drop_one_memkind_cache_size(mk, &(mk->cache_area[i])); + } + + if (mk->cache_unpowered) + return mseg_drop_one_memkind_cache_size(mk, &(mk->cache_unpowered)); + + return 0; +} + +/* mseg_cache_check + * - Check if we have some cache we can purge + * in any of the memkinds. + */ + +static void mseg_cache_check(ErtsMsegAllctr_t *ma) { MemKind* mk; + Uint empty_cache = 1; ERTS_MSEG_LOCK(ma); - for (mk=ma->mk_list; mk; mk=mk->next) { - if (check_one_cache(mk)) + for (mk = ma->mk_list; mk; mk = mk->next) { + if (mseg_check_memkind_cache(mk)) empty_cache = 0; } + /* If all MemKinds caches are empty, + * remove aux-work callback + */ if (empty_cache) { ma->is_cache_check_scheduled = 0; - erts_set_aux_work_timeout(ma->ix, - ERTS_SSI_AUX_WORK_MSEG_CACHE_CHECK, - 0); + erts_set_aux_work_timeout(ma->ix, ERTS_SSI_AUX_WORK_MSEG_CACHE_CHECK, 0); } INC_CC(ma, check_cache); @@ -627,27 +782,65 @@ static void do_cache_check(ErtsMsegAllctr_t *ma) ERTS_MSEG_UNLOCK(ma); } -void erts_mseg_cache_check(void) -{ - do_cache_check(ERTS_MSEG_ALLCTR_SS()); +/* erts_mseg_cache_check + * - This is a callback that is scheduled as aux-work from + * schedulers and is called at some interval if we have a cache + * on this mseg-allocator and memkind. + * - Purpose: Empty cache slowly so we don't collect mapped areas + * and bloat memory. + */ + +void erts_mseg_cache_check(void) { + mseg_cache_check(ERTS_MSEG_ALLCTR_SS()); } -static void -mseg_clear_cache(MemKind* mk) -{ - mk->segments.current.watermark = 0; - adjust_cache_size(mk, 1); +/* *_mseg_clear_*_cache + * Remove cached segments from the allocator completely + */ + +static void mseg_clear_memkind_cache(MemKind *mk) { + int i; + + /* drop pow2 caches */ + for (i = 0; i < CACHE_AREAS; i++) { + if (mk->cache_area[i] == NULL) + continue; + + mseg_drop_memkind_cache_size(mk, &(mk->cache_area[i])); + ASSERT(mk->cache_area[i] == NULL); + } + /* drop varied caches */ + if(mk->cache_unpowered) + mseg_drop_memkind_cache_size(mk, &(mk->cache_unpowered)); + + ASSERT(mk->cache_unpowered == NULL); + ASSERT(mk->cache_size == 0); +} + +static void mseg_clear_cache(ErtsMsegAllctr_t *ma) { + MemKind* mk; + + ERTS_MSEG_LOCK(ma); + ERTS_DBG_MA_CHK_THR_ACCESS(ma); + - ASSERT(!mk->cache); - ASSERT(!mk->cache_end); - ASSERT(!mk->cache_size); + for (mk = ma->mk_list; mk; mk = mk->next) { + mseg_clear_memkind_cache(mk); + } - mk->segments.current.watermark = mk->segments.current.no; + INC_CC(ma, clear_cache); - INC_CC(mk->ma, clear_cache); + ERTS_MSEG_UNLOCK(ma); } +void erts_mseg_clear_cache(void) { + mseg_clear_cache(ERTS_MSEG_ALLCTR_SS()); + mseg_clear_cache(ERTS_MSEG_ALLCTR_IX(0)); +} + + + static ERTS_INLINE MemKind* memkind(ErtsMsegAllctr_t *ma, const ErtsMsegOpt_t *opt) { @@ -660,116 +853,40 @@ static ERTS_INLINE MemKind* memkind(ErtsMsegAllctr_t *ma, static void * mseg_alloc(ErtsMsegAllctr_t *ma, ErtsAlcType_t atype, Uint *size_p, - const ErtsMsegOpt_t *opt) + Uint flags, const ErtsMsegOpt_t *opt) { - Uint max, min, diff_size, size; - cache_desc_t *cd, *cand_cd; + Uint size; void *seg; MemKind* mk = memkind(ma, opt); INC_CC(ma, alloc); - size = PAGE_CEILING(*size_p); + /* Carrier align */ + size = ALIGNED_CEILING(*size_p); + + /* Cache optim (if applicable) */ + if (MSEG_FLG_IS_2POW(flags) && !IS_2POW(size)) + size = ceil_2pow(size); #if CAN_PARTLY_DESTROY if (size < ma->min_seg_size) ma->min_seg_size = size; #endif + + if (opt->cache && mk->cache_size > 0 && (seg = cache_get_segment(mk, &size)) != NULL) + goto done; - if (!opt->cache) { - create_seg: - adjust_cache_size(mk,0); - seg = mseg_create(ma, mk, size); - if (!seg) { - mseg_clear_cache(mk); - seg = mseg_create(ma, mk, size); - if (!seg) - size = 0; - } - - *size_p = size; - if (seg) { - if (erts_mtrace_enabled) - erts_mtrace_crr_alloc(seg, atype, ERTS_MTRACE_SEGMENT_ID, size); - ERTS_MSEG_ALLOC_STAT(mk,size); - } - return seg; - } - - if (size > mk->max_cached_seg_size) - goto create_seg; - - if (size < mk->min_cached_seg_size) { - - diff_size = mk->min_cached_seg_size - size; - - if (diff_size > ma->abs_max_cache_bad_fit) - goto create_seg; - - if (100*PAGES(diff_size) > ma->rel_max_cache_bad_fit*PAGES(size)) - goto create_seg; - - } - - max = 0; - min = ~((Uint) 0); - cand_cd = NULL; - - for (cd = mk->cache; cd; cd = cd->next) { - if (cd->size >= size) { - if (!cand_cd) { - cand_cd = cd; - continue; - } - else if (cd->size < cand_cd->size) { - if (max < cand_cd->size) - max = cand_cd->size; - if (min > cand_cd->size) - min = cand_cd->size; - cand_cd = cd; - continue; - } - } - if (max < cd->size) - max = cd->size; - if (min > cd->size) - min = cd->size; - } - - mk->min_cached_seg_size = min; - mk->max_cached_seg_size = max; - - if (!cand_cd) - goto create_seg; - - diff_size = cand_cd->size - size; - - if (diff_size > ma->abs_max_cache_bad_fit - || 100*PAGES(diff_size) > ma->rel_max_cache_bad_fit*PAGES(size)) { - if (mk->max_cached_seg_size < cand_cd->size) - mk->max_cached_seg_size = cand_cd->size; - if (mk->min_cached_seg_size > cand_cd->size) - mk->min_cached_seg_size = cand_cd->size; - goto create_seg; - } - - mk->cache_hits++; - - size = cand_cd->size; - seg = cand_cd->seg; - - unlink_cd(mk,cand_cd); - free_cd(mk,cand_cd); + if ((seg = mseg_create(ma, mk, size)) == NULL) + size = 0; +done: *size_p = size; + if (seg) { + if (erts_mtrace_enabled) + erts_mtrace_crr_alloc(seg, atype, ERTS_MTRACE_SEGMENT_ID, size); - if (erts_mtrace_enabled) { - erts_mtrace_crr_free(SEGTYPE, SEGTYPE, seg); - erts_mtrace_crr_alloc(seg, atype, SEGTYPE, size); - } - - if (seg) ERTS_MSEG_ALLOC_STAT(mk,size); + } return seg; } @@ -780,73 +897,42 @@ mseg_dealloc(ErtsMsegAllctr_t *ma, ErtsAlcType_t atype, void *seg, Uint size, const ErtsMsegOpt_t *opt) { MemKind* mk = memkind(ma, opt); - cache_desc_t *cd; + ERTS_MSEG_DEALLOC_STAT(mk,size); - if (!opt->cache || ma->max_cache_size == 0) { - if (erts_mtrace_enabled) - erts_mtrace_crr_free(atype, SEGTYPE, seg); - mseg_destroy(ma, mk, seg, size); + if (opt->cache && cache_bless_segment(mk, seg, size)) { + schedule_cache_check(ma); + goto done; } - else { - int check_limits = 0; - - if (size < mk->min_cached_seg_size) - mk->min_cached_seg_size = size; - if (size > mk->max_cached_seg_size) - mk->max_cached_seg_size = size; - - if (!mk->free_cache_descs) { - cd = mk->cache_end; - if (!(mk->min_cached_seg_size < cd->size - && cd->size < mk->max_cached_seg_size)) { - check_limits = 1; - } - if (erts_mtrace_enabled) - erts_mtrace_crr_free(SEGTYPE, SEGTYPE, cd->seg); - mseg_destroy(ma, mk, cd->seg, cd->size); - unlink_cd(mk,cd); - free_cd(mk,cd); - } - cd = alloc_cd(mk); - ASSERT(cd); - cd->seg = seg; - cd->size = size; - link_cd(mk,cd); + if (erts_mtrace_enabled) + erts_mtrace_crr_free(atype, SEGTYPE, seg); - if (erts_mtrace_enabled) { - erts_mtrace_crr_free(atype, SEGTYPE, seg); - erts_mtrace_crr_alloc(seg, SEGTYPE, SEGTYPE, size); - } - - /* ASSERT(segments.current.watermark >= segments.current.no + cache_size); */ - - if (check_limits) - check_cache_limits(mk); + mseg_destroy(ma, mk, seg, size); - schedule_cache_check(ma); - - } +done: INC_CC(ma, dealloc); } static void * mseg_realloc(ErtsMsegAllctr_t *ma, ErtsAlcType_t atype, void *seg, - Uint old_size, Uint *new_size_p, const ErtsMsegOpt_t *opt) + Uint old_size, Uint *new_size_p, Uint flags, const ErtsMsegOpt_t *opt) { MemKind* mk; void *new_seg; Uint new_size; + /* Just allocate a new segment if we didn't have one before */ if (!seg || !old_size) { - new_seg = mseg_alloc(ma, atype, new_size_p, opt); + new_seg = mseg_alloc(ma, atype, new_size_p, flags, opt); DEC_CC(ma, alloc); return new_seg; } + + /* Dealloc old segment if new segment is of size 0 */ if (!(*new_size_p)) { mseg_dealloc(ma, atype, seg, old_size, opt); DEC_CC(ma, dealloc); @@ -855,7 +941,13 @@ mseg_realloc(ErtsMsegAllctr_t *ma, ErtsAlcType_t atype, void *seg, mk = memkind(ma, opt); new_seg = seg; - new_size = PAGE_CEILING(*new_size_p); + + /* Carrier align */ + new_size = ALIGNED_CEILING(*new_size_p); + + /* Cache optim (if applicable) */ + if (MSEG_FLG_IS_2POW(flags) && !IS_2POW(new_size)) + new_size = ceil_2pow(new_size); if (new_size == old_size) ; @@ -866,53 +958,27 @@ mseg_realloc(ErtsMsegAllctr_t *ma, ErtsAlcType_t atype, void *seg, if (new_size < ma->min_seg_size) ma->min_seg_size = new_size; #endif - + /* +M<S>rsbcst <ratio> */ if (shrink_sz < opt->abs_shrink_th - && 100*PAGES(shrink_sz) < opt->rel_shrink_th*PAGES(old_size)) { + && 100*shrink_sz < opt->rel_shrink_th*old_size) { new_size = old_size; } else { #if CAN_PARTLY_DESTROY - if (shrink_sz > ma->min_seg_size - && mk->free_cache_descs - && opt->cache) { - cache_desc_t *cd; - - cd = alloc_cd(mk); - ASSERT(cd); - cd->seg = ((char *) seg) + new_size; - cd->size = shrink_sz; - end_link_cd(mk,cd); - - if (erts_mtrace_enabled) { - erts_mtrace_crr_realloc(new_seg, - atype, - SEGTYPE, - seg, - new_size); - erts_mtrace_crr_alloc(cd->seg, SEGTYPE, SEGTYPE, cd->size); - } - schedule_cache_check(ma); - } - else { - if (erts_mtrace_enabled) - erts_mtrace_crr_realloc(new_seg, - atype, - SEGTYPE, - seg, - new_size); - mseg_destroy(ma, mk, ((char *) seg) + new_size, shrink_sz); - } + if (erts_mtrace_enabled) + erts_mtrace_crr_realloc(new_seg, atype, SEGTYPE, seg, new_size); -#elif HAVE_MSEG_RECREATE + mseg_destroy(ma, mk, ((char *) seg) + new_size, shrink_sz); +#elif HAVE_MSEG_RECREATE goto do_recreate; - #else + new_seg = mseg_alloc(ma, atype, &new_size, flags, opt); + + ASSERT(MAP_IS_ALIGNED(new_seg) || !new_seg); - new_seg = mseg_alloc(ma, atype, &new_size, opt); if (!new_seg) new_size = old_size; else { @@ -921,16 +987,15 @@ mseg_realloc(ErtsMsegAllctr_t *ma, ErtsAlcType_t atype, void *seg, MIN(new_size, old_size)); mseg_dealloc(ma, atype, seg, old_size, opt); } - #endif - } } else { if (!opt->preserv) { mseg_dealloc(ma, atype, seg, old_size, opt); - new_seg = mseg_alloc(ma, atype, &new_size, opt); + new_seg = mseg_alloc(ma, atype, &new_size, flags, opt); + ASSERT(MAP_IS_ALIGNED(new_seg) || !new_seg); } else { #if HAVE_MSEG_RECREATE @@ -938,18 +1003,23 @@ mseg_realloc(ErtsMsegAllctr_t *ma, ErtsAlcType_t atype, void *seg, do_recreate: #endif new_seg = mseg_recreate(ma, mk, (void *) seg, old_size, new_size); + /* ASSERT(MAP_IS_ALIGNED(new_seg) || !new_seg); + * will not always be aligned and it ok for now + */ + if (erts_mtrace_enabled) erts_mtrace_crr_realloc(new_seg, atype, SEGTYPE, seg, new_size); if (!new_seg) new_size = old_size; #else - new_seg = mseg_alloc(ma, atype, &new_size, opt); + new_seg = mseg_alloc(ma, atype, &new_size, flags, opt); + + ASSERT(MAP_IS_ALIGNED(new_seg) || !new_seg); + if (!new_seg) new_size = old_size; else { - sys_memcpy(((char *) new_seg), - ((char *) seg), - MIN(new_size, old_size)); + sys_memcpy(((char *) new_seg), ((char *) seg), MIN(new_size, old_size)); mseg_dealloc(ma, atype, seg, old_size, opt); } #endif @@ -958,6 +1028,7 @@ mseg_realloc(ErtsMsegAllctr_t *ma, ErtsAlcType_t atype, void *seg, INC_CC(ma, realloc); + ASSERT(!MSEG_FLG_IS_2POW(flags) || IS_2POW(new_size)); *new_size_p = new_size; ERTS_MSEG_REALLOC_STAT(mk, old_size, new_size); @@ -990,6 +1061,7 @@ static struct { Eterm mseg_dealloc; Eterm mseg_realloc; Eterm mseg_create; + Eterm mseg_create_resize; Eterm mseg_destroy; #if HAVE_MSEG_RECREATE Eterm mseg_recreate; @@ -1046,6 +1118,7 @@ init_atoms(ErtsMsegAllctr_t *ma) AM_INIT(mseg_dealloc); AM_INIT(mseg_realloc); AM_INIT(mseg_create); + AM_INIT(mseg_create_resize); AM_INIT(mseg_destroy); #if HAVE_MSEG_RECREATE AM_INIT(mseg_recreate); @@ -1065,14 +1138,12 @@ init_atoms(ErtsMsegAllctr_t *ma) erts_mtx_unlock(&init_atoms_mutex); } - #define bld_uint erts_bld_uint #define bld_cons erts_bld_cons #define bld_tuple erts_bld_tuple #define bld_string erts_bld_string #define bld_2tup_list erts_bld_2tup_list - /* * bld_unstable_uint() (instead of bld_uint()) is used when values may * change between size check and actual build. This because a value @@ -1116,6 +1187,7 @@ add_4tup(Uint **hpp, Uint *szp, Eterm *lp, *lp = bld_cons(hpp, szp, bld_tuple(hpp, szp, 4, el1, el2, el3, el4), *lp); } + static Eterm info_options(ErtsMsegAllctr_t *ma, char *prefix, @@ -1176,6 +1248,7 @@ info_calls(ErtsMsegAllctr_t *ma, int *print_to_p, void *print_to_arg, Uint **hpp PRINT_CC(to, arg, dealloc); PRINT_CC(to, arg, realloc); PRINT_CC(to, arg, create); + PRINT_CC(to, arg, create_resize); PRINT_CC(to, arg, destroy); #if HAVE_MSEG_RECREATE PRINT_CC(to, arg, recreate); @@ -1215,6 +1288,10 @@ info_calls(ErtsMsegAllctr_t *ma, int *print_to_p, void *print_to_arg, Uint **hpp bld_unstable_uint(hpp, szp, ma->calls.create.giga_no), bld_unstable_uint(hpp, szp, ma->calls.create.no)); + add_3tup(hpp, szp, &res, + am.mseg_create_resize, + bld_unstable_uint(hpp, szp, ma->calls.create_resize.giga_no), + bld_unstable_uint(hpp, szp, ma->calls.create_resize.no)); add_3tup(hpp, szp, &res, am.mseg_realloc, @@ -1401,21 +1478,21 @@ erts_mseg_info(int ix, } void * -erts_mseg_alloc_opt(ErtsAlcType_t atype, Uint *size_p, const ErtsMsegOpt_t *opt) +erts_mseg_alloc_opt(ErtsAlcType_t atype, Uint *size_p, Uint flags, const ErtsMsegOpt_t *opt) { ErtsMsegAllctr_t *ma = ERTS_MSEG_ALLCTR_OPT(opt); void *seg; ERTS_MSEG_LOCK(ma); ERTS_DBG_MA_CHK_THR_ACCESS(ma); - seg = mseg_alloc(ma, atype, size_p, opt); + seg = mseg_alloc(ma, atype, size_p, flags, opt); ERTS_MSEG_UNLOCK(ma); return seg; } void * -erts_mseg_alloc(ErtsAlcType_t atype, Uint *size_p) +erts_mseg_alloc(ErtsAlcType_t atype, Uint *size_p, Uint flags) { - return erts_mseg_alloc_opt(atype, size_p, &erts_mseg_default_opt); + return erts_mseg_alloc_opt(atype, size_p, flags, &erts_mseg_default_opt); } void @@ -1438,44 +1515,24 @@ erts_mseg_dealloc(ErtsAlcType_t atype, void *seg, Uint size) void * erts_mseg_realloc_opt(ErtsAlcType_t atype, void *seg, Uint old_size, Uint *new_size_p, + Uint flags, const ErtsMsegOpt_t *opt) { ErtsMsegAllctr_t *ma = ERTS_MSEG_ALLCTR_OPT(opt); void *new_seg; ERTS_MSEG_LOCK(ma); ERTS_DBG_MA_CHK_THR_ACCESS(ma); - new_seg = mseg_realloc(ma, atype, seg, old_size, new_size_p, opt); + new_seg = mseg_realloc(ma, atype, seg, old_size, new_size_p, flags, opt); ERTS_MSEG_UNLOCK(ma); return new_seg; } void * erts_mseg_realloc(ErtsAlcType_t atype, void *seg, - Uint old_size, Uint *new_size_p) + Uint old_size, Uint *new_size_p, Uint flags) { return erts_mseg_realloc_opt(atype, seg, old_size, new_size_p, - &erts_mseg_default_opt); -} - -void -erts_mseg_clear_cache(void) -{ - ErtsMsegAllctr_t *ma = ERTS_MSEG_ALLCTR_SS(); - MemKind* mk; - -start: - - ERTS_MSEG_LOCK(ma); - ERTS_DBG_MA_CHK_THR_ACCESS(ma); - for (mk=ma->mk_list; mk; mk=mk->next) { - mseg_clear_cache(mk); - } - ERTS_MSEG_UNLOCK(ma); - - if (ma->ix != 0) { - ma = ERTS_MSEG_ALLCTR_IX(0); - goto start; - } + flags, &erts_mseg_default_opt); } Uint @@ -1496,28 +1553,32 @@ erts_mseg_no(const ErtsMsegOpt_t *opt) Uint erts_mseg_unit_size(void) { - return page_size; + return MSEG_ALIGNED_SIZE; } static void mem_kind_init(ErtsMsegAllctr_t *ma, MemKind* mk, const char* name) { - unsigned i; + int i; - mk->cache = NULL; - mk->cache_end = NULL; - mk->max_cached_seg_size = 0; - mk->min_cached_seg_size = ~((Uint) 0); - mk->cache_size = 0; - mk->cache_hits = 0; + for (i = 0; i < CACHE_AREAS; i++) { + mk->cache_area[i] = NULL; + } + + mk->cache_free = NULL; - if (ma->max_cache_size > 0) { - for (i = 0; i < ma->max_cache_size - 1; i++) - mk->cache_descs[i].next = &mk->cache_descs[i + 1]; - mk->cache_descs[ma->max_cache_size - 1].next = NULL; - mk->free_cache_descs = &mk->cache_descs[0]; + ASSERT(ma->max_cache_size <= MAX_CACHE_SIZE); + + for (i = 0; i < ma->max_cache_size; i++) { + mk->cache[i].seg = NULL; + mk->cache[i].size = 0; + mk->cache[i].next = mk->cache_free; + mk->cache_free = &(mk->cache[i]); } - else - mk->free_cache_descs = NULL; + + mk->cache_unpowered = NULL; + + mk->cache_size = 0; + mk->cache_hits = 0; mk->segments.current.watermark = 0; mk->segments.current.no = 0; @@ -1570,15 +1631,10 @@ erts_mseg_init(ErtsMsegInit_t *init) initialize_pmmap(); #endif - page_size = GET_PAGE_SIZE; + if (!IS_2POW(GET_PAGE_SIZE)) + erl_exit(ERTS_ABORT_EXIT, "erts_mseg: Unexpected page_size %beu\n", GET_PAGE_SIZE); - page_shift = 1; - while ((page_size >> page_shift) != 1) { - if ((page_size & (1 << (page_shift - 1))) != 0) - erl_exit(ERTS_ABORT_EXIT, - "erts_mseg: Unexpected page_size %beu\n", page_size); - page_shift++; - } + ASSERT((MSEG_ALIGNED_SIZE % GET_PAGE_SIZE) == 0); for (i = 0; i < no_mseg_allocators; i++) { ErtsMsegAllctr_t *ma = ERTS_MSEG_ALLCTR_IX(i); @@ -1663,7 +1719,7 @@ erts_mseg_test(unsigned long op, case 0x400: /* Have erts_mseg */ return (unsigned long) 1; case 0x401: - return (unsigned long) erts_mseg_alloc(ERTS_ALC_A_INVALID, (Uint *) a1); + return (unsigned long) erts_mseg_alloc(ERTS_ALC_A_INVALID, (Uint *) a1, (Uint) 0); case 0x402: erts_mseg_dealloc(ERTS_ALC_A_INVALID, (void *) a1, (Uint) a2); return (unsigned long) 0; @@ -1671,7 +1727,8 @@ erts_mseg_test(unsigned long op, return (unsigned long) erts_mseg_realloc(ERTS_ALC_A_INVALID, (void *) a1, (Uint) a2, - (Uint *) a3); + (Uint *) a3, + (Uint) 0); case 0x404: erts_mseg_clear_cache(); return (unsigned long) 0; @@ -1707,7 +1764,40 @@ erts_mseg_test(unsigned long op, * mapping tricks. */ -/*#define HARDDEBUG 1*/ +/* #define HARDDEBUG 1 */ + +#ifdef HARDDEBUG +static void dump_freelist(void) +{ + FreeBlock *p = first; + + while (p) { + fprintf(stderr, "p = %p\r\np->num = %ld\r\np->next = %p\r\n\r\n", + (void *) p, (unsigned long) p->num, (void *) p->next); + p = p->next; + } +} + +#define HARDDEBUG_HW_INCOMPLETE_ALIGNMENT(PTR, SZ) \ + fprintf(stderr,"Mapping of address %p with size %ld " \ + "does not map complete pages (%s:%d)\r\n", \ + (void *) (PTR), (unsigned long) (SZ),__FILE__, __LINE__) + +#define HARDDEBUG_HW_UNALIGNED_ALIGNMENT(PTR, SZ) \ + fprintf(stderr,"Mapping of address %p with size %ld " \ + "is not page aligned (%s:%d)\r\n", \ + (void *) (PTR), (unsigned long) (SZ),__FILE__, __LINE__) + +#define HARDDEBUG_MAP_FAILED(PTR, SZ) \ + fprintf(stderr, "Could not actually map memory " \ + "at address %p with size %ld (%s:%d) ..\r\n", \ + (void *) (PTR), (unsigned long) (SZ),__FILE__, __LINE__) +#else +#define HARDDEBUG_HW_INCOMPLETE_ALIGNMENT(PTR, SZ) do{}while(0) +#define HARDDEBUG_HW_UNALIGNED_ALIGNMENT(PTR, SZ) do{}while(0) +#define HARDDEBUG_MAP_FAILED(PTR, SZ) do{}while(0) +#endif + #ifdef __APPLE__ #define MAP_ANONYMOUS MAP_ANON @@ -1726,49 +1816,20 @@ typedef struct _free_block { struct _free_block *next; } FreeBlock; -/* Assigned once and for all */ -static size_t pagsz; - /* Protect with lock */ static FreeBlock *first; -static size_t round_up_to_pagesize(size_t size) -{ - size_t x = size / pagsz; - - if ((size % pagsz)) { - ++x; - } - - return pagsz * x; -} - -static size_t round_down_to_pagesize(size_t size) -{ - size_t x = size / pagsz; - - return pagsz * x; -} - static void *do_map(void *ptr, size_t sz) { void *res; - if (round_up_to_pagesize(sz) != sz) { -#ifdef HARDDEBUG - fprintf(stderr,"Mapping of address %p with size %ld " - "does not map complete pages\r\n", - (void *) ptr, (unsigned long) sz); -#endif + if (ALIGNED_CEILING(sz) != sz) { + HARDDEBUG_HW_INCOMPLETE_ALIGNMENT(ptr, sz); return NULL; } - if (((unsigned long) ptr) % pagsz) { -#ifdef HARDDEBUG - fprintf(stderr,"Mapping of address %p with size %ld " - "is not page aligned\r\n", - (void *) ptr, (unsigned long) sz); -#endif + if (((unsigned long) ptr) % MSEG_ALIGNED_SIZE) { + HARDDEBUG_HW_UNALIGNED_ALIGNMENT(ptr, sz); return NULL; } @@ -1782,10 +1843,7 @@ static void *do_map(void *ptr, size_t sz) #endif if (res == MAP_FAILED) { -#ifdef HARDDEBUG - fprintf(stderr,"Mapping of address %p with size %ld failed!\r\n", - (void *) ptr, (unsigned long) sz); -#endif + HARDDEBUG_MAP_FAILED(ptr, sz); return NULL; } @@ -1796,35 +1854,22 @@ static int do_unmap(void *ptr, size_t sz) { void *res; - if (round_up_to_pagesize(sz) != sz) { -#ifdef HARDDEBUG - fprintf(stderr,"Mapping of address %p with size %ld " - "does not map complete pages\r\n", - (void *) ptr, (unsigned long) sz); -#endif + if (ALIGNED_CEILING(sz) != sz) { + HARDDEBUG_HW_INCOMPLETE_ALIGNMENT(ptr, sz); return 1; } - if (((unsigned long) ptr) % pagsz) { -#ifdef HARDDEBUG - fprintf(stderr,"Mapping of address %p with size %ld " - "is not page aligned\r\n", - (void *) ptr, (unsigned long) sz); -#endif + if (((unsigned long) ptr) % MSEG_ALIGNED_SIZE) { + HARDDEBUG_HW_UNALIGNED_ALIGNMENT(ptr, sz); return 1; } - res = mmap(ptr, sz, - PROT_NONE, MAP_PRIVATE | MAP_ANONYMOUS | MAP_NORESERVE - | MAP_FIXED, + PROT_NONE, MAP_PRIVATE | MAP_ANONYMOUS | MAP_NORESERVE | MAP_FIXED, -1 , 0); if (res == MAP_FAILED) { -#ifdef HARDDEBUG - fprintf(stderr,"Mapping of address %p with size %ld failed!\r\n", - (void *) ptr, (unsigned long) sz); -#endif + HARDDEBUG_MAP_FAILED(ptr, sz); return 1; } @@ -1862,8 +1907,6 @@ static int initialize_pmmap(void) size_t rsz; FreeBlock *initial; - - pagsz = getpagesize(); SET_RANGE_MIN(); if (sizeof(void *) != 8) { erl_exit(1,"Halfword emulator cannot be run in 32bit mode"); @@ -1872,15 +1915,15 @@ static int initialize_pmmap(void) p = (char *) RANGE_MIN; q = (char *) RANGE_MAX; - rsz = round_down_to_pagesize(q - p); + rsz = ALIGNED_FLOOR(q - p); - rptr = mmap((void *) p, rsz, + rptr = mmap_align(NULL, (void *) p, rsz, PROT_NONE, MAP_PRIVATE | MAP_ANONYMOUS | MAP_NORESERVE | EXTRA_MAP_FLAGS, -1 , 0); #ifdef HARDDEBUG printf("p=%p, rsz = %ld, pages = %ld, got range = %p -> %p\r\n", - p, (unsigned long) rsz, (unsigned long) (rsz / pagsz), + p, (unsigned long) rsz, (unsigned long) (rsz / MSEG_ALIGNED_SIZE), (void *) rptr, (void*)(rptr + rsz)); #endif if ((UWord)(rptr + rsz) > RANGE_MAX) { @@ -1892,39 +1935,27 @@ static int initialize_pmmap(void) munmap((void*)RANGE_MAX, rsz - rsz_trunc); rsz = rsz_trunc; } - if (!do_map(rptr,pagsz)) { + if (!do_map(rptr, MSEG_ALIGNED_SIZE)) { erl_exit(1,"Could not actually mmap first page for halfword emulator...\n"); } initial = (FreeBlock *) rptr; - initial->num = (rsz / pagsz); + initial->num = (rsz / MSEG_ALIGNED_SIZE); initial->next = NULL; first = initial; INIT_LOCK(); return 0; } -#ifdef HARDDEBUG -static void dump_freelist(void) -{ - FreeBlock *p = first; - - while (p) { - printf("p = %p\r\np->num = %ld\r\np->next = %p\r\n\r\n", - (void *) p, (unsigned long) p->num, (void *) p->next); - p = p->next; - } -} -#endif - - static void *pmmap(size_t size) { - size_t real_size = round_up_to_pagesize(size); - size_t num_pages = real_size / pagsz; + size_t real_size = ALIGNED_CEILING(size); + size_t num_pages = real_size / MSEG_ALIGNED_SIZE; FreeBlock **block; FreeBlock *tail; FreeBlock *res; + TAKE_LOCK(); + for (block = &first; *block != NULL && (*block)->num < num_pages; block = &((*block)->next)) @@ -1935,29 +1966,25 @@ static void *pmmap(size_t size) } if ((*block)->num == num_pages) { /* nice, perfect fit */ - res = *block; + res = *block; *block = (*block)->next; } else { tail = (FreeBlock *) (((char *) ((void *) (*block))) + real_size); - if (!do_map(tail,pagsz)) { -#ifdef HARDDEBUG - fprintf(stderr, "Could not actually allocate page at %p...\r\n", - (void *) tail); -#endif + if (!do_map(tail, MSEG_ALIGNED_SIZE)) { + HARDDEBUG_MAP_FAILED(tail, MSEG_ALIGNED_SIZE); RELEASE_LOCK(); return NULL; } - tail->num = (*block)->num - num_pages; + tail->num = (*block)->num - num_pages; tail->next = (*block)->next; res = *block; *block = tail; } + RELEASE_LOCK(); - if (!do_map(res,real_size)) { -#ifdef HARDDEBUG - fprintf(stderr, "Could not actually allocate %ld at %p...\r\n", - (unsigned long) real_size, (void *) res); -#endif + + if (!do_map(res, real_size)) { + HARDDEBUG_MAP_FAILED(res, real_size); return NULL; } @@ -1966,15 +1993,17 @@ static void *pmmap(size_t size) static int pmunmap(void *p, size_t size) { - size_t real_size = round_up_to_pagesize(size); - size_t num_pages = real_size / pagsz; + size_t real_size = ALIGNED_CEILING(size); + size_t num_pages = real_size / MSEG_ALIGNED_SIZE; + FreeBlock *block; FreeBlock *last; FreeBlock *nb = (FreeBlock *) p; ASSERT(((unsigned long)p & CHECK_POINTER_MASK)==0); - if (real_size > pagsz) { - if (do_unmap(((char *) p) + pagsz,real_size - pagsz)) { + + if (real_size > MSEG_ALIGNED_SIZE) { + if (do_unmap(((char *) p) + MSEG_ALIGNED_SIZE, real_size - MSEG_ALIGNED_SIZE)) { return 1; } } @@ -1993,7 +2022,7 @@ static int pmunmap(void *p, size_t size) /* Merge new free block with following */ nb->num = block->num + num_pages; nb->next = block->next; - if (do_unmap(block,pagsz)) { + if (do_unmap(block, MSEG_ALIGNED_SIZE)) { RELEASE_LOCK(); return 1; } @@ -2003,11 +2032,11 @@ static int pmunmap(void *p, size_t size) nb->next = block; } if (last != NULL) { - if (p == ((void *) (((char *) last) + (last->num * pagsz)))) { + if (p == ((void *) (((char *) last) + (last->num * MSEG_ALIGNED_SIZE)))) { /* Merge with previous */ last->num += nb->num; last->next = nb->next; - if (do_unmap(nb,pagsz)) { + if (do_unmap(nb, MSEG_ALIGNED_SIZE)) { RELEASE_LOCK(); return 1; } @@ -2024,10 +2053,10 @@ static int pmunmap(void *p, size_t size) static void *pmremap(void *old_address, size_t old_size, size_t new_size) { - size_t new_real_size = round_up_to_pagesize(new_size); - size_t new_num_pages = new_real_size / pagsz; - size_t old_real_size = round_up_to_pagesize(old_size); - size_t old_num_pages = old_real_size / pagsz; + size_t new_real_size = ALIGNED_CEILING(new_size); + size_t new_num_pages = new_real_size / MSEG_ALIGNED_SIZE; + size_t old_real_size = ALIGNED_CEILING(old_size); + size_t old_num_pages = old_real_size / MSEG_ALIGNED_SIZE; if (new_num_pages == old_num_pages) { return old_address; } else if (new_num_pages < old_num_pages) { /* Shrink */ @@ -2045,8 +2074,8 @@ static void *pmremap(void *old_address, size_t old_size, (*block) > ((FreeBlock *)(((char *) vnfb) + nfb_real_size))) { /* Normal link in */ if (nfb_pages > 1) { - if (do_unmap((void *)(((char *) vnfb) + pagsz), - (nfb_pages - 1)*pagsz)) { + if (do_unmap((void *)(((char *) vnfb) + MSEG_ALIGNED_SIZE), + (nfb_pages - 1)*MSEG_ALIGNED_SIZE)) { return NULL; } } @@ -2058,8 +2087,8 @@ static void *pmremap(void *old_address, size_t old_size, nfb->num = nfb_pages + (*block)->num; /* unmap also the first page of the next freeblock */ (*block) = nfb; - if (do_unmap((void *)(((char *) vnfb) + pagsz), - nfb_pages*pagsz)) { + if (do_unmap((void *)(((char *) vnfb) + MSEG_ALIGNED_SIZE), + nfb_pages*MSEG_ALIGNED_SIZE)) { return NULL; } } @@ -2094,9 +2123,9 @@ static void *pmremap(void *old_address, size_t old_size, size_t remaining_pages = (*block)->num - (new_num_pages - old_num_pages); if (!remaining_pages) { - void *p = (void *) (((char *) (*block)) + pagsz); + void *p = (void *) (((char *) (*block)) + MSEG_ALIGNED_SIZE); void *n = (*block)->next; - size_t x = ((*block)->num - 1) * pagsz; + size_t x = ((*block)->num - 1) * MSEG_ALIGNED_SIZE; if (x > 0) { if (do_map(p,x) == NULL) { RELEASE_LOCK(); @@ -2108,7 +2137,7 @@ static void *pmremap(void *old_address, size_t old_size, FreeBlock *nfb = (FreeBlock *) ((void *) (((char *) old_address) + new_real_size)); - void *p = (void *) (((char *) (*block)) + pagsz); + void *p = (void *) (((char *) (*block)) + MSEG_ALIGNED_SIZE); if (do_map(p,new_real_size - old_real_size) == NULL) { RELEASE_LOCK(); return NULL; @@ -2122,5 +2151,4 @@ static void *pmremap(void *old_address, size_t old_size, } } } - #endif /* HALFWORD_HEAP */ diff --git a/erts/emulator/sys/common/erl_mseg.h b/erts/emulator/sys/common/erl_mseg.h index 741080fb78..6f373f13f9 100644 --- a/erts/emulator/sys/common/erl_mseg.h +++ b/erts/emulator/sys/common/erl_mseg.h @@ -32,12 +32,35 @@ #if HAVE_MMAP # define HAVE_ERTS_MSEG 1 +# define HAVE_SUPER_ALIGNED_MB_CARRIERS 1 #else # define HAVE_ERTS_MSEG 0 +# define HAVE_SUPER_ALIGNED_MB_CARRIERS 0 +#endif + +#if HAVE_SUPER_ALIGNED_MB_CARRIERS +# define MSEG_ALIGN_BITS (18) + /* Affects hard limits for sbct and lmbcs documented in erts_alloc.xml */ +#else +/* If we don't use super aligned multiblock carriers + * we will mmap with page size alignment (and thus use corresponding + * align bits). + * + * Current implementation needs this to be a constant and + * only uses this for user dev testing so setting page size + * to 4096 (12 bits) is fine. + */ +# define MSEG_ALIGN_BITS (12) #endif #if HAVE_ERTS_MSEG +#define MSEG_ALIGNED_SIZE (1 << MSEG_ALIGN_BITS) + +#define ERTS_MSEG_FLG_NONE ((Uint)(0)) +#define ERTS_MSEG_FLG_2POW ((Uint)(1 << 0)) + + #define ERTS_MSEG_VSN_STR "0.9" typedef struct { @@ -68,13 +91,13 @@ typedef struct { extern const ErtsMsegOpt_t erts_mseg_default_opt; -void *erts_mseg_alloc(ErtsAlcType_t, Uint *); -void *erts_mseg_alloc_opt(ErtsAlcType_t, Uint *, const ErtsMsegOpt_t *); +void *erts_mseg_alloc(ErtsAlcType_t, Uint *, Uint); +void *erts_mseg_alloc_opt(ErtsAlcType_t, Uint *, Uint, const ErtsMsegOpt_t *); void erts_mseg_dealloc(ErtsAlcType_t, void *, Uint); void erts_mseg_dealloc_opt(ErtsAlcType_t, void *, Uint, const ErtsMsegOpt_t *); -void *erts_mseg_realloc(ErtsAlcType_t, void *, Uint, Uint *); +void *erts_mseg_realloc(ErtsAlcType_t, void *, Uint, Uint *, Uint); void *erts_mseg_realloc_opt(ErtsAlcType_t, void *, Uint, Uint *, - const ErtsMsegOpt_t *); + Uint, const ErtsMsegOpt_t *); void erts_mseg_clear_cache(void); void erts_mseg_cache_check(void); Uint erts_mseg_no( const ErtsMsegOpt_t *); diff --git a/erts/emulator/sys/unix/erl_unix_sys_ddll.c b/erts/emulator/sys/unix/erl_unix_sys_ddll.c index 336d9586c4..a35aec560a 100644 --- a/erts/emulator/sys/unix/erl_unix_sys_ddll.c +++ b/erts/emulator/sys/unix/erl_unix_sys_ddll.c @@ -101,7 +101,7 @@ void erl_sys_ddll_init(void) { /* * Open a shared object */ -int erts_sys_ddll_open2(char *full_name, void **handle, ErtsSysDdllError* err) +int erts_sys_ddll_open2(const char *full_name, void **handle, ErtsSysDdllError* err) { #if defined(HAVE_DLOPEN) char* dlname; @@ -153,7 +153,7 @@ int erts_sys_ddll_open_noext(char *dlname, void **handle, ErtsSysDdllError* err) /* * Find a symbol in the shared object */ -int erts_sys_ddll_sym2(void *handle, char *func_name, void **function, +int erts_sys_ddll_sym2(void *handle, const char *func_name, void **function, ErtsSysDdllError* err) { #if defined(HAVE_DLOPEN) diff --git a/erts/emulator/sys/unix/sys.c b/erts/emulator/sys/unix/sys.c index b485dbf784..0b96eded76 100644 --- a/erts/emulator/sys/unix/sys.c +++ b/erts/emulator/sys/unix/sys.c @@ -123,7 +123,8 @@ struct ErtsSysReportExit_ { /* This data is shared by these drivers - initialized by spawn_init() */ static struct driver_data { - int port_num, ofd, packet_bytes; + ErlDrvPort port_num; + int ofd, packet_bytes; ErtsSysReportExit *report_exit; int pid; int alive; @@ -686,7 +687,7 @@ static RETSIGTYPE break_handler(int sig) } #endif /* 0 */ -static ERTS_INLINE void +static ERTS_INLINE int prepare_crash_dump(int secs) { #define NUFBUF (3) @@ -698,24 +699,41 @@ prepare_crash_dump(int secs) Eterm *hp = heap; Eterm list = NIL; int heart_fd[2] = {-1,-1}; + int has_heart = 0; UseTmpHeapNoproc(NUFBUF); if (ERTS_PREPARED_CRASH_DUMP) - return; /* We have already been called */ + return 0; /* We have already been called */ heart_port = erts_get_heart_port(); + + /* Positive secs means an alarm must be set + * 0 or negative means no alarm + * + * Set alarm before we try to write to a port + * we don't want to hang on a port write with + * no alarm. + * + */ + + if (secs >= 0) { + alarm((unsigned int)secs); + } + if (heart_port) { /* hearts input fd * We "know" drv_data is the in_fd since the port is started with read|write */ heart_fd[0] = (int)heart_port->drv_data; heart_fd[1] = (int)driver_data[heart_fd[0]].ofd; + has_heart = 1; list = CONS(hp, make_small(8), list); hp += 2; /* send to heart port, CMD = 8, i.e. prepare crash dump =o */ - erts_write_to_port(ERTS_INVALID_PID, heart_port, list); + erts_port_output(NULL, ERTS_PORT_SIG_FLG_FORCE_IMM_CALL, heart_port, + heart_port->common.id, list, NULL); } /* Make sure we unregister at epmd (unknown fd) and get at least @@ -752,20 +770,14 @@ prepare_crash_dump(int secs) erts_silence_warn_unused_result(nice(nice_val)); } - /* Positive secs means an alarm must be set - * 0 or negative means no alarm - */ - if (secs > 0) { - alarm((unsigned int)secs); - } UnUseTmpHeapNoproc(NUFBUF); #undef NUFBUF + return has_heart; } -void -erts_sys_prepare_crash_dump(int secs) +int erts_sys_prepare_crash_dump(int secs) { - prepare_crash_dump(secs); + return prepare_crash_dump(secs); } static ERTS_INLINE void @@ -802,12 +814,22 @@ static RETSIGTYPE request_break(int signum) static ERTS_INLINE void sigusr1_exit(void) { - /* We do this at interrupt level, since the main reason for - wanting to generate a crash dump in this way is that the emulator - is hung somewhere, so it won't be able to poll any flag we set here. - */ + char env[21]; /* enough to hold any 64-bit integer */ + size_t envsz; + int i, secs = -1; + + /* We do this at interrupt level, since the main reason for + * wanting to generate a crash dump in this way is that the emulator + * is hung somewhere, so it won't be able to poll any flag we set here. + */ ERTS_SET_GOT_SIGUSR1; - prepare_crash_dump((int)0); + + envsz = sizeof(env); + if ((i = erts_sys_getenv_raw("ERL_CRASH_DUMP_SECONDS", env, &envsz)) >= 0) { + secs = i != 0 ? 0 : atoi(env); + } + + prepare_crash_dump(secs); erl_exit(1, "Received SIGUSR1\n"); } @@ -1162,7 +1184,7 @@ static RETSIGTYPE onchld(int signum) #endif } -static int set_driver_data(int port_num, +static int set_driver_data(ErlDrvPort port_num, int ifd, int ofd, int packet_bytes, @@ -1170,6 +1192,7 @@ static int set_driver_data(int port_num, int exit_status, int pid) { + Port *prt; ErtsSysReportExit *report_exit; if (!exit_status) @@ -1178,7 +1201,7 @@ static int set_driver_data(int port_num, report_exit = erts_alloc(ERTS_ALC_T_PRT_REP_EXIT, sizeof(ErtsSysReportExit)); report_exit->next = report_exit_list; - report_exit->port = erts_port[port_num].id; + report_exit->port = erts_drvport2id(port_num); report_exit->pid = pid; report_exit->ifd = read_write & DO_READ ? ifd : -1; report_exit->ofd = read_write & DO_WRITE ? ofd : -1; @@ -1188,7 +1211,9 @@ static int set_driver_data(int port_num, report_exit_list = report_exit; } - erts_port[port_num].os_pid = pid; + prt = erts_drvport2port(port_num, NULL); + if (prt) + prt->os_pid = pid; if (read_write & DO_READ) { driver_data[ifd].packet_bytes = packet_bytes; @@ -1261,7 +1286,7 @@ static void close_pipes(int ifd[2], int ofd[2], int read_write) } } -static void init_fd_data(int fd, int prt) +static void init_fd_data(int fd, ErlDrvPort port_num) { fd_data[fd].buf = NULL; fd_data[fd].cpos = NULL; @@ -1951,7 +1976,7 @@ static void clear_fd_data(int fd) fd_data[fd].psz = 0; } -static void nbio_stop_fd(int prt, int fd) +static void nbio_stop_fd(ErlDrvPort prt, int fd) { driver_select(prt,fd,DO_READ|DO_WRITE,0); clear_fd_data(fd); @@ -1999,7 +2024,8 @@ static ErlDrvData vanilla_start(ErlDrvPort port_num, char* name, static void stop(ErlDrvData fd) { - int prt, ofd; + ErlDrvPort prt; + int ofd; prt = driver_data[(int)(long)fd].port_num; nbio_stop_fd(prt, (int)(long)fd); @@ -2012,7 +2038,7 @@ static void stop(ErlDrvData fd) CHLD_STAT_LOCK; - /* Mark as unused. Maybe resetting the 'port_num' slot is better? */ + /* Mark as unused. */ driver_data[(int)(long)fd].pid = -1; CHLD_STAT_UNLOCK; @@ -2028,7 +2054,7 @@ static void stop(ErlDrvData fd) static void outputv(ErlDrvData e, ErlIOVec* ev) { int fd = (int)(long)e; - int ix = driver_data[fd].port_num; + ErlDrvPort ix = driver_data[fd].port_num; int pb = driver_data[fd].packet_bytes; int ofd = driver_data[fd].ofd; ssize_t n; @@ -2078,7 +2104,7 @@ static void outputv(ErlDrvData e, ErlIOVec* ev) static void output(ErlDrvData e, char* buf, ErlDrvSizeT len) { int fd = (int)(long)e; - int ix = driver_data[fd].port_num; + ErlDrvPort ix = driver_data[fd].port_num; int pb = driver_data[fd].packet_bytes; int ofd = driver_data[fd].ofd; ssize_t n; @@ -2129,7 +2155,7 @@ static void output(ErlDrvData e, char* buf, ErlDrvSizeT len) return; /* 0; */ } -static int port_inp_failure(int port_num, int ready_fd, int res) +static int port_inp_failure(ErlDrvPort port_num, int ready_fd, int res) /* Result: 0 (eof) or -1 (error) */ { int err = errno; @@ -2179,7 +2205,7 @@ static int port_inp_failure(int port_num, int ready_fd, int res) static void ready_input(ErlDrvData e, ErlDrvEvent ready_fd) { int fd = (int)(long)e; - int port_num; + ErlDrvPort port_num; int packet_bytes; int res; Uint h; @@ -2302,7 +2328,7 @@ static void ready_input(ErlDrvData e, ErlDrvEvent ready_fd) static void ready_output(ErlDrvData e, ErlDrvEvent ready_fd) { int fd = (int)(long)e; - int ix = driver_data[fd].port_num; + ErlDrvPort ix = driver_data[fd].port_num; int n; struct iovec* iv; int vsize; @@ -2611,19 +2637,20 @@ report_exit_status(ErtsSysReportExit *rep, int status) Port *pp; #ifdef ERTS_SMP CHLD_STAT_UNLOCK; -#endif + pp = erts_thr_id2port_sflgs(rep->port, + ERTS_PORT_SFLGS_INVALID_DRIVER_LOOKUP); + CHLD_STAT_LOCK; +#else pp = erts_id2port_sflgs(rep->port, NULL, 0, ERTS_PORT_SFLGS_INVALID_DRIVER_LOOKUP); -#ifdef ERTS_SMP - CHLD_STAT_LOCK; #endif if (pp) { if (rep->ifd >= 0) { driver_data[rep->ifd].alive = 0; driver_data[rep->ifd].status = status; - (void) driver_select((ErlDrvPort) internal_port_index(pp->id), + (void) driver_select((ErlDrvPort) pp, rep->ifd, (ERL_DRV_READ|ERL_DRV_USE), 1); @@ -2631,12 +2658,16 @@ report_exit_status(ErtsSysReportExit *rep, int status) if (rep->ofd >= 0) { driver_data[rep->ofd].alive = 0; driver_data[rep->ofd].status = status; - (void) driver_select((ErlDrvPort) internal_port_index(pp->id), + (void) driver_select((ErlDrvPort) pp, rep->ofd, (ERL_DRV_WRITE|ERL_DRV_USE), 1); } +#ifdef ERTS_SMP + erts_thr_port_release(pp); +#else erts_port_release(pp); +#endif } erts_free(ERTS_ALC_T_PRT_REP_EXIT, rep); } diff --git a/erts/emulator/sys/win32/erl_win32_sys_ddll.c b/erts/emulator/sys/win32/erl_win32_sys_ddll.c index d00eed932b..54991a610c 100644 --- a/erts/emulator/sys/win32/erl_win32_sys_ddll.c +++ b/erts/emulator/sys/win32/erl_win32_sys_ddll.c @@ -58,7 +58,7 @@ void erl_sys_ddll_init(void) { /* * Open a shared object */ -int erts_sys_ddll_open2(char *full_name, void **handle, ErtsSysDdllError* err) +int erts_sys_ddll_open2(const char *full_name, void **handle, ErtsSysDdllError* err) { int len; char dlname[MAXPATHLEN + 1]; @@ -92,7 +92,7 @@ int erts_sys_ddll_open_noext(char *dlname, void **handle, ErtsSysDdllError* err) /* * Find a symbol in the shared object */ -int erts_sys_ddll_sym2(void *handle, char *func_name, void **function, +int erts_sys_ddll_sym2(void *handle, const char *func_name, void **function, ErtsSysDdllError* err) { FARPROC proc; diff --git a/erts/emulator/sys/win32/erl_win_dyn_driver.h b/erts/emulator/sys/win32/erl_win_dyn_driver.h index ec5141838a..8b6be2b2f1 100644 --- a/erts/emulator/sys/win32/erl_win_dyn_driver.h +++ b/erts/emulator/sys/win32/erl_win_dyn_driver.h @@ -74,7 +74,9 @@ WDD_TYPEDEF(ErlDrvTermData, driver_mk_port,(ErlDrvPort)); WDD_TYPEDEF(ErlDrvTermData, driver_connected,(ErlDrvPort)); WDD_TYPEDEF(ErlDrvTermData, driver_caller,(ErlDrvPort)); WDD_TYPEDEF(ErlDrvTermData, driver_mk_term_nil,(void)); +WDD_TYPEDEF(int, erl_drv_output_term, (ErlDrvTermData, ErlDrvTermData*, int)); WDD_TYPEDEF(int, driver_output_term, (ErlDrvPort, ErlDrvTermData*, int)); +WDD_TYPEDEF(int, erl_drv_send_term, (ErlDrvTermData, ErlDrvTermData, ErlDrvTermData*, int)); WDD_TYPEDEF(int, driver_send_term, (ErlDrvPort, ErlDrvTermData, ErlDrvTermData*, int)); WDD_TYPEDEF(long, driver_async, (ErlDrvPort,unsigned int*,void (*)(void*),void*,void (*)(void*))); WDD_TYPEDEF(int, driver_async_cancel, (unsigned int)); @@ -187,7 +189,9 @@ typedef struct { WDD_FTYPE(driver_connected) *driver_connected; WDD_FTYPE(driver_caller) *driver_caller; WDD_FTYPE(driver_mk_term_nil) *driver_mk_term_nil; + WDD_FTYPE(erl_drv_output_term) *erl_drv_output_term; WDD_FTYPE(driver_output_term) *driver_output_term; + WDD_FTYPE(erl_drv_send_term) *erl_drv_send_term; WDD_FTYPE(driver_send_term) *driver_send_term; WDD_FTYPE(driver_async) *driver_async; WDD_FTYPE(driver_async_cancel) *driver_async_cancel; @@ -294,7 +298,9 @@ extern TWinDynDriverCallbacks WinDynDriverCallbacks; #define driver_connected (WinDynDriverCallbacks.driver_connected) #define driver_caller (WinDynDriverCallbacks.driver_caller) #define driver_mk_term_nil (WinDynDriverCallbacks.driver_mk_term_nil) +#define erl_drv_output_term (WinDynDriverCallbacks.erl_drv_output_term) #define driver_output_term (WinDynDriverCallbacks.driver_output_term) +#define erl_drv_send_term (WinDynDriverCallbacks.erl_drv_send_term) #define driver_send_term (WinDynDriverCallbacks.driver_send_term) #define driver_async (WinDynDriverCallbacks.driver_async) #define driver_async_cancel (WinDynDriverCallbacks.driver_async_cancel) @@ -425,7 +431,9 @@ do { \ ((W).driver_connected) = driver_connected; \ ((W).driver_caller) = driver_caller; \ ((W).driver_mk_term_nil) = driver_mk_term_nil; \ +((W).erl_drv_output_term) = erl_drv_output_term; \ ((W).driver_output_term) = driver_output_term; \ +((W).erl_drv_send_term) = erl_drv_send_term; \ ((W).driver_send_term) = driver_send_term; \ ((W).driver_async) = driver_async; \ ((W).driver_async_cancel) = driver_async_cancel; \ diff --git a/erts/emulator/sys/win32/sys.c b/erts/emulator/sys/win32/sys.c index 47d12ed5fe..1cd9072cea 100755 --- a/erts/emulator/sys/win32/sys.c +++ b/erts/emulator/sys/win32/sys.c @@ -87,9 +87,6 @@ static erts_smp_tsd_key_t win32_errstr_key; static erts_smp_atomic_t pipe_creation_counter; -static erts_smp_mtx_t sys_driver_data_lock; - - /* Results from application_type(_w) is one of */ #define APPL_NONE 0 #define APPL_DOS 1 @@ -97,7 +94,6 @@ static erts_smp_mtx_t sys_driver_data_lock; #define APPL_WIN32 3 static int driver_write(long, HANDLE, byte*, int); -static void common_stop(int); static int create_file_thread(struct async_io* aio, int mode); #ifdef ERTS_SMP static void close_active_handle(ErlDrvPort, HANDLE handle); @@ -115,9 +111,6 @@ BOOL WINAPI ctrl_handler(DWORD dwCtrlType); #define PORT_BUFSIZ 4096 -#define PORT_FREE (-1) -#define PORT_EXITING (-2) - #define DRV_BUF_ALLOC(SZ) \ erts_alloc_fnf(ERTS_ALC_T_DRV_DATA_BUF, (SZ)) #define DRV_BUF_REALLOC(P, SZ) \ @@ -255,8 +248,7 @@ void erl_sys_args(int* argc, char** argv) #endif } -void -erts_sys_prepare_crash_dump(int secs) +int erts_sys_prepare_crash_dump(int secs) { Port *heart_port; Eterm heap[3]; @@ -270,11 +262,16 @@ erts_sys_prepare_crash_dump(int secs) list = CONS(hp, make_small(8), list); hp += 2; /* send to heart port, CMD = 8, i.e. prepare crash dump =o */ - erts_write_to_port(NIL, heart_port, list); + erts_port_output(NULL, ERTS_PORT_SIG_FLG_FORCE_IMM_CALL, heart_port, + heart_port->common.id, list, NULL); + + return 1; } /* Windows - free file descriptors are hopefully available */ /* Alarm not used on windows */ + + return 0; } static void @@ -471,7 +468,7 @@ typedef struct driver_data { byte *inbuf; /* Buffer to use for overlapped read. */ int outBufSize; /* Size of output buffer. */ byte *outbuf; /* Buffer to use for overlapped write. */ - ErlDrvPort port_num; /* The port number. */ + ErlDrvPort port_num; /* The port handle. */ int packet_bytes; /* 0: continous stream, 1, 2, or 4: the number * of bytes in the packet header. */ @@ -481,8 +478,6 @@ typedef struct driver_data { int report_exit; /* Do report exit status for the port */ } DriverData; -static DriverData* driver_data; /* Pointer to array of driver data. */ - /* Driver interfaces */ static ErlDrvData spawn_start(ErlDrvPort, char*, SysDriverOpts*); static ErlDrvData fd_start(ErlDrvPort, char*, SysDriverOpts*); @@ -594,67 +589,53 @@ struct erl_drv_entry vanilla_driver_entry = { */ static DriverData* -new_driver_data(int port_num, int packet_bytes, int wait_objs_required, int use_threads) +new_driver_data(ErlDrvPort port_num, int packet_bytes, int wait_objs_required, int use_threads) { DriverData* dp; - - erts_smp_mtx_lock(&sys_driver_data_lock); - DEBUGF(("new_driver_data(port_num %d, pb %d)\n", - port_num, packet_bytes)); + DEBUGF(("new_driver_data(%p, pb %d)\n", port_num, packet_bytes)); + dp = driver_alloc(sizeof(DriverData)); + if (!dp) + return NULL; /* * We used to test first at all that there is enough room in the * array used by WaitForMultipleObjects(), but that is not necessary * any more, since driver_select() can't fail. */ - /* - * Search for a free slot. - */ + dp->bytesInBuffer = 0; + dp->totalNeeded = packet_bytes; + dp->inBufSize = PORT_BUFSIZ; + dp->inbuf = DRV_BUF_ALLOC(dp->inBufSize); + if (dp->inbuf == NULL) + goto buf_alloc_error; + erts_smp_atomic_add_nob(&sys_misc_mem_sz, dp->inBufSize); + dp->outBufSize = 0; + dp->outbuf = NULL; + dp->port_num = port_num; + dp->packet_bytes = packet_bytes; + dp->port_pid = INVALID_HANDLE_VALUE; + if (init_async_io(&dp->in, use_threads) == -1) + goto async_io_error1; + if (init_async_io(&dp->out, use_threads) == -1) + goto async_io_error2; - for (dp = driver_data; dp < driver_data+max_files; dp++) { - if (dp->port_num == PORT_FREE) { - dp->bytesInBuffer = 0; - dp->totalNeeded = packet_bytes; - dp->inBufSize = PORT_BUFSIZ; - dp->inbuf = DRV_BUF_ALLOC(dp->inBufSize); - if (dp->inbuf == NULL) { - erts_smp_mtx_unlock(&sys_driver_data_lock); - return NULL; - } - erts_smp_atomic_add_nob(&sys_misc_mem_sz, dp->inBufSize); - dp->outBufSize = 0; - dp->outbuf = NULL; - dp->port_num = port_num; - dp->packet_bytes = packet_bytes; - dp->port_pid = INVALID_HANDLE_VALUE; - if (init_async_io(&dp->in, use_threads) == -1) - break; - if (init_async_io(&dp->out, use_threads) == -1) - break; - erts_smp_mtx_unlock(&sys_driver_data_lock); - return dp; - } - } + return dp; - /* - * Error or no free driver data. - */ +async_io_error2: + release_async_io(&dp->in, dp->port_num); +async_io_error1: + release_async_io(&dp->out, dp->port_num); - if (dp < driver_data+max_files) { - release_async_io(&dp->in, dp->port_num); - release_async_io(&dp->out, dp->port_num); - } - erts_smp_mtx_unlock(&sys_driver_data_lock); +buf_alloc_error: + driver_free(dp); return NULL; } static void release_driver_data(DriverData* dp) { - erts_smp_mtx_lock(&sys_driver_data_lock); - #ifdef ERTS_SMP #ifdef USE_CANCELIOEX if (fpCancelIoEx != NULL) { @@ -738,8 +719,7 @@ release_driver_data(DriverData* dp) * the exit thread. */ - dp->port_num = PORT_FREE; - erts_smp_mtx_unlock(&sys_driver_data_lock); + driver_free(dp); } #ifdef ERTS_SMP @@ -834,7 +814,6 @@ threaded_handle_closer(LPVOID param) static ErlDrvData set_driver_data(DriverData* dp, HANDLE ifd, HANDLE ofd, int read_write, int report_exit) { - int index = dp - driver_data; int result; dp->in.fd = ifd; @@ -853,13 +832,12 @@ set_driver_data(DriverData* dp, HANDLE ifd, HANDLE ofd, int read_write, int repo ERL_DRV_WRITE|ERL_DRV_USE, 1); ASSERT(result != -1); } - return (ErlDrvData)index; + return (ErlDrvData) dp; } static ErlDrvData reuse_driver_data(DriverData *dp, HANDLE ifd, HANDLE ofd, int read_write, ErlDrvPort port_num) { - int index = dp - driver_data; int result; dp->port_num = port_num; @@ -878,7 +856,7 @@ reuse_driver_data(DriverData *dp, HANDLE ifd, HANDLE ofd, int read_write, ErlDrv ERL_DRV_WRITE|ERL_DRV_USE, 1); ASSERT(result != -1); } - return (ErlDrvData)index; + return (ErlDrvData) dp; } /* @@ -1151,12 +1129,6 @@ spawn_init(void) ((module != NULL) ? GetProcAddress(module,"CancelIoEx") : NULL); DEBUGF(("fpCancelIoEx = %p\r\n", fpCancelIoEx)); #endif - driver_data = (struct driver_data *) - erts_alloc(ERTS_ALC_T_DRV_TAB, max_files * sizeof(struct driver_data)); - erts_smp_atomic_add_nob(&sys_misc_mem_sz, - max_files*sizeof(struct driver_data)); - for (i = 0; i < max_files; i++) - driver_data[i].port_num = PORT_FREE; return 0; } @@ -1287,9 +1259,12 @@ spawn_start(ErlDrvPort port_num, char* name, SysDriverOpts* opts) #endif retval = set_driver_data(dp, hFromChild, hToChild, opts->read_write, opts->exit_status); - if (retval != ERL_DRV_ERROR_GENERAL && retval != ERL_DRV_ERROR_ERRNO) - /* We assume that this cannot generate a negative number */ - erts_port[port_num].os_pid = (SWord) pid; + if (retval != ERL_DRV_ERROR_GENERAL && retval != ERL_DRV_ERROR_ERRNO) { + Port *prt = erts_drvport2port_raw(port_num); + /* We assume that this cannot generate a negative number */ + ASSERT(prt); + prt->os_pid = (SWord) pid; + } } if (retval != ERL_DRV_ERROR_GENERAL && retval != ERL_DRV_ERROR_ERRNO) @@ -2278,12 +2253,10 @@ fd_start(ErlDrvPort port_num, char* name, SysDriverOpts* opts) **/ if (!create_file_thread(&dp->in, DO_READ)) { - dp->port_num = PORT_FREE; return ERL_DRV_ERROR_GENERAL; } if (!create_file_thread(&dp->out, DO_WRITE)) { - dp->port_num = PORT_FREE; return ERL_DRV_ERROR_GENERAL; } @@ -2303,10 +2276,9 @@ fd_start(ErlDrvPort port_num, char* name, SysDriverOpts* opts) } } -static void fd_stop(ErlDrvData d) +static void fd_stop(ErlDrvData data) { - int fd = (int)d; - DriverData* dp = driver_data+fd; + DriverData * dp = (DriverData *) data; /* * There's no way we can terminate an fd port in a consistent way. * Instead we let it live until it's opened again (which it is, @@ -2369,16 +2341,10 @@ vanilla_start(ErlDrvPort port_num, char* name, SysDriverOpts* opts) } static void -stop(ErlDrvData index) +stop(ErlDrvData data) { - common_stop((int)index); -} - -static void common_stop(int index) -{ - DriverData* dp = driver_data+index; - - DEBUGF(("common_stop(%d)\n", index)); + DriverData *dp = (DriverData *) data; + DEBUGF(("stop(%p)\n", dp)); if (dp->in.ov.hEvent != NULL) { (void) driver_select(dp->port_num, @@ -2400,7 +2366,6 @@ static void common_stop(int index) */ HANDLE thread; DWORD tid; - dp->port_num = PORT_EXITING; thread = (HANDLE *) _beginthreadex(NULL, 0, threaded_exiter, dp, 0, &tid); CloseHandle(thread); } @@ -2525,22 +2490,17 @@ threaded_exiter(LPVOID param) static void output(ErlDrvData drv_data, char* buf, ErlDrvSizeT len) -/* long drv_data; /* The slot to use in the driver data table. +/* ErlDrvData drv_data; /* The slot to use in the driver data table. * For Windows NT, this is *NOT* a file handle. * The handle is found in the driver data. */ /* char *buf; /* Pointer to data to write to the port program. */ /* ErlDrvSizeT len; /* Number of bytes to write. */ { - DriverData* dp; + DriverData* dp = (DriverData *) drv_data; int pb; /* The header size for this port. */ - int port_num; /* The actual port number (for diagnostics). */ char* current; - dp = driver_data + (int)drv_data; - if ((port_num = dp->port_num) == -1) - return ; /*-1;*/ - pb = dp->packet_bytes; if ((pb+len) == 0) @@ -2551,7 +2511,7 @@ output(ErlDrvData drv_data, char* buf, ErlDrvSizeT len) */ if ((pb == 2 && len > 65535) || (pb == 1 && len > 255)) { - driver_failure_posix(port_num, EINVAL); + driver_failure_posix(dp->port_num, EINVAL); return ; /* -1; */ } @@ -2565,7 +2525,7 @@ output(ErlDrvData drv_data, char* buf, ErlDrvSizeT len) ASSERT(!dp->outbuf); dp->outbuf = DRV_BUF_ALLOC(pb+len); if (!dp->outbuf) { - driver_failure_posix(port_num, ENOMEM); + driver_failure_posix(dp->port_num, ENOMEM); return ; /* -1; */ } @@ -2595,7 +2555,7 @@ output(ErlDrvData drv_data, char* buf, ErlDrvSizeT len) memcpy(current, buf, len); if (!async_write_file(&dp->out, dp->outbuf, pb+len)) { - set_busy_port(port_num, 1); + set_busy_port(dp->port_num, 1); } else { dp->out.ov.Offset += pb+len; /* For vanilla driver. */ /* XXX OffsetHigh should be changed too. */ @@ -2630,10 +2590,9 @@ ready_input(ErlDrvData drv_data, ErlDrvEvent ready_event) { int error = 0; /* The error code (assume initially no errors). */ DWORD bytesRead; /* Number of bytes read. */ - DriverData* dp; + DriverData* dp = (DriverData *) drv_data; int pb; - dp = driver_data+(int)drv_data; pb = dp->packet_bytes; #ifdef ERTS_SMP if(dp->in.thread == (HANDLE) -1) { @@ -2801,7 +2760,7 @@ static void ready_output(ErlDrvData drv_data, ErlDrvEvent ready_event) { DWORD bytesWritten; - DriverData* dp = driver_data + (int)drv_data; + DriverData *dp = (DriverData *) drv_data; int error; #ifdef ERTS_SMP @@ -2809,7 +2768,7 @@ ready_output(ErlDrvData drv_data, ErlDrvEvent ready_event) dp->out.async_io_active = 0; } #endif - DEBUGF(("ready_output(%d, 0x%x)\n", drv_data, ready_event)); + DEBUGF(("ready_output(%p, 0x%x)\n", drv_data, ready_event)); set_busy_port(dp->port_num, 0); if (!(dp->outbuf)) { /* Happens because event sometimes get signalled during a successful @@ -2864,7 +2823,7 @@ sys_init_io(void) can change our view of the number of open files possible. We estimate the number to twice the amount of ports. We really dont know on windows, do we? */ - max_files = 2*erts_max_ports; + max_files = 2*erts_ptab_max(&erts_port); } #ifdef ERTS_SMP @@ -3319,9 +3278,6 @@ void erl_sys_init(void) noinherit_std_handle(STD_INPUT_HANDLE); noinherit_std_handle(STD_ERROR_HANDLE); - - erts_smp_mtx_init(&sys_driver_data_lock, "sys_driver_data_lock"); - #ifdef ERTS_SMP erts_smp_tsd_key_create(&win32_errstr_key); InitializeCriticalSection(&htbc_lock); diff --git a/erts/emulator/test/alloc_SUITE_data/allocator_test.h b/erts/emulator/test/alloc_SUITE_data/allocator_test.h index cd4a91d34a..c0396ddb61 100644 --- a/erts/emulator/test/alloc_SUITE_data/allocator_test.h +++ b/erts/emulator/test/alloc_SUITE_data/allocator_test.h @@ -60,9 +60,9 @@ typedef void* erts_cond; #define IS_MMAP_C(C) ((Ulong) ALC_TEST1(0x00a, (C))) #define C_SZ(C) ((Ulong) ALC_TEST1(0x00b, (C))) #define SBC2BLK(A, C) ((Block_t *) ALC_TEST2(0x00c, (A), (C))) -#define BLK2SBC(A, B) ((Carrier_t *) ALC_TEST2(0x00d, (A), (B))) -#define MBC2FBLK(A, C) ((Block_t *) ALC_TEST2(0x00e, (A), (C))) -#define FBLK2MBC(A, B) ((Carrier_t *) ALC_TEST2(0x00f, (A), (B))) +#define BLK_TO_SBC(A, B) ((Carrier_t *) ALC_TEST2(0x00d, (A), (B))) +#define MBC_TO_FIRST_BLK(A, C) ((Block_t *) ALC_TEST2(0x00e, (A), (C))) +#define FIRST_BLK_TO_MBC(A, B) ((Carrier_t *) ALC_TEST2(0x00f, (A), (B))) #define FIRST_MBC(A) ((Carrier_t *) ALC_TEST1(0x010, (A))) #define LAST_MBC(A) ((Carrier_t *) ALC_TEST1(0x011, (A))) #define FIRST_SBC(A) ((Carrier_t *) ALC_TEST1(0x012, (A))) @@ -73,7 +73,7 @@ typedef void* erts_cond; #define MIN_BLK_SZ(A) ((Ulong) ALC_TEST1(0x017, (A))) #define NXT_BLK(B) ((Block_t *) ALC_TEST1(0x018, (B))) #define PREV_BLK(B) ((Block_t *) ALC_TEST1(0x019, (B))) -#define IS_FIRST_BLK(B) ((Ulong) ALC_TEST1(0x01a, (B))) +#define IS_MBC_FIRST_BLK(A,B) ((Ulong) ALC_TEST2(0x01a, (A), (B))) #define UNIT_SZ ((Ulong) ALC_TEST0(0x01b)) /* From erl_goodfit_alloc.c */ diff --git a/erts/emulator/test/alloc_SUITE_data/basic.c b/erts/emulator/test/alloc_SUITE_data/basic.c index 4a5e888161..0c27665712 100644 --- a/erts/emulator/test/alloc_SUITE_data/basic.c +++ b/erts/emulator/test/alloc_SUITE_data/basic.c @@ -44,7 +44,7 @@ testcase_run(TestCaseState_t *tcs) c = FIRST_MBC(a); ASSERT(tcs, !NEXT_C(c)); - blk = MBC2FBLK(a, c); + blk = MBC_TO_FIRST_BLK(a, c); ASSERT(tcs, IS_LAST_BLK(blk)); ASSERT(tcs, IS_FREE_BLK(blk)); diff --git a/erts/emulator/test/alloc_SUITE_data/bucket_mask.c b/erts/emulator/test/alloc_SUITE_data/bucket_mask.c index b214f87e4a..34979cacf1 100644 --- a/erts/emulator/test/alloc_SUITE_data/bucket_mask.c +++ b/erts/emulator/test/alloc_SUITE_data/bucket_mask.c @@ -22,7 +22,7 @@ #include "allocator_test.h" #include <stdio.h> -#ifdef __WIN32__ && SIZEOF_VOID_P == 8 +#if defined(__WIN32__) && SIZEOF_VOID_P == 8 /* Use larger threashold for win64 as block alignment is 16 bytes and not 8 */ #define SBCT ((1024*1024)) @@ -48,10 +48,16 @@ testcase_cleanup(TestCaseState_t *tcs) void testcase_run(TestCaseState_t *tcs) { - void *tmp; - void **fence; + typedef struct linked_block { + struct linked_block* next; + }Linked; + Linked* link; + Linked* fence_list; + Linked* pad_list; + void* tmp; void **blk; Ulong sz; + Ulong residue; Ulong smbcs; int i; int bi; @@ -73,7 +79,7 @@ testcase_run(TestCaseState_t *tcs) ASSERT(tcs, a); min_blk_sz = MIN_BLK_SZ(a); - smbcs = 2*(no_bkts*sizeof(void *) + min_blk_sz) + min_blk_sz; + smbcs = (no_bkts*sizeof(void *) + min_blk_sz) + min_blk_sz; for (i = 0; i < no_bkts; i++) { sz = BKT_MIN_SZ(a, i); if (sz >= sbct) @@ -98,26 +104,42 @@ testcase_run(TestCaseState_t *tcs) tcs->extra = (void *) a; ASSERT(tcs, a); + blk = (void **) ALLOC(a, no_bkts*sizeof(void *)); - fence = (void **) ALLOC(a, no_bkts*sizeof(void *)); - ASSERT(tcs, blk && fence); + ASSERT(tcs, blk); + fence_list = NULL; testcase_printf(tcs, "Allocating blocks and fences\n"); for (i = 0; i < bi_tests; i++) { sz = BKT_MIN_SZ(a, i); blk[i] = ALLOC(a, sz - ablk_hdr_sz); - fence[i] = ALLOC(a, 1); - ASSERT(tcs, blk[i] && fence[i]); + link = (Linked*) ALLOC(a, sizeof(Linked)); + ASSERT(tcs, blk[i] && link); + link->next = fence_list; + fence_list = link; } - tmp = (void *) UMEM2BLK(fence[bi_tests - 1]); - tmp = (void *) NXT_BLK((Block_t *) tmp); - ASSERT(tcs, IS_LAST_BLK(tmp)); - sz = BLK_SZ((Block_t *) tmp); - testcase_printf(tcs, "Allocating leftover size = %lu\n", sz); - tmp = ALLOC(a, sz - ablk_hdr_sz); - ASSERT(tcs, tmp); + pad_list = 0; + do { + tmp = (void *) UMEM2BLK(link); /* last allocated */ + tmp = (void *) NXT_BLK((Block_t *) tmp); + ASSERT(tcs, IS_LAST_BLK(tmp)); + sz = BLK_SZ((Block_t *) tmp); + if (sz >= sbct) { + residue = sz; + sz = sbct - min_blk_sz; + residue -= sz; + } + else { + residue = 0; + } + testcase_printf(tcs, "Allocating leftover size = %lu, residue = %lu\n", sz, residue); + link = (Linked*) ALLOC(a, sz - ablk_hdr_sz); + ASSERT(tcs, link); + link->next = pad_list; + pad_list = link; + } while (residue); bi = FIND_BKT(a, 0); ASSERT(tcs, bi < 0); @@ -135,16 +157,23 @@ testcase_run(TestCaseState_t *tcs) for (i = 0; i < bi_tests; i++) { FREE(a, blk[i]); - FREE(a, fence[i]); + } + while (fence_list) { + link = fence_list; + fence_list = link->next; + FREE(a, link); } FREE(a, (void *) blk); - FREE(a, (void *) fence); bi = FIND_BKT(a, 0); ASSERT(tcs, bi == no_bkts - 1); - FREE(a, tmp); + while (pad_list) { + link = pad_list; + pad_list = link->next; + FREE(a, link); + } bi = FIND_BKT(a, 0); ASSERT(tcs, bi < 0); diff --git a/erts/emulator/test/alloc_SUITE_data/coalesce.c b/erts/emulator/test/alloc_SUITE_data/coalesce.c index 6f35d3279b..981fa6d43e 100644 --- a/erts/emulator/test/alloc_SUITE_data/coalesce.c +++ b/erts/emulator/test/alloc_SUITE_data/coalesce.c @@ -54,7 +54,7 @@ setup_sequence(TestCaseState_t *tcs, Allctr_t *a, Ulong bsz, int no, no, bsz); c = FIRST_MBC(a); ASSERT(tcs, !NEXT_C(c)); - blk = MBC2FBLK(a, c); + blk = MBC_TO_FIRST_BLK(a, c); ASSERT(tcs, IS_LAST_BLK(blk)); for (i = 0; i < no; i++) @@ -266,7 +266,7 @@ testcase_name(void) void testcase_run(TestCaseState_t *tcs) { - char *argv_org[] = {"-tmmbcs1024", "-tsbct2048", "-trmbcmt100", "-tas", NULL, NULL}; + char *argv_org[] = {"-tsmbcs511","-tmmbcs511", "-tsbct512", "-trmbcmt100", "-tas", NULL, NULL}; char *alg[] = {"af", "gf", "bf", "aobf", "aoff", NULL}; int i; @@ -276,7 +276,7 @@ testcase_run(TestCaseState_t *tcs) char *argv[sizeof(argv_org)/sizeof(argv_org[0])]; memcpy((void *) argv, (void *) argv_org, sizeof(argv_org)); - argv[4] = alg[i]; + argv[5] = alg[i]; testcase_printf(tcs, " *** Starting \"%s\" allocator *** \n", alg[i]); a = START_ALC("coalesce_", 0, argv); ASSERT(tcs, a); diff --git a/erts/emulator/test/alloc_SUITE_data/mseg_clear_cache.c b/erts/emulator/test/alloc_SUITE_data/mseg_clear_cache.c index 0277616bd0..7d5608f890 100644 --- a/erts/emulator/test/alloc_SUITE_data/mseg_clear_cache.c +++ b/erts/emulator/test/alloc_SUITE_data/mseg_clear_cache.c @@ -52,10 +52,10 @@ testcase_run(TestCaseState_t *tcs) tcs->extra = &seg[0]; for (i = 0; i < MAX_SEGS; i++) { - seg[i].size = 1000; + seg[i].size = 1 << 18; seg[i].ptr = MSEG_ALLOC(&seg[i].size); ASSERT(tcs, seg[i].ptr); - ASSERT(tcs, seg[i].size >= 1000); + ASSERT(tcs, seg[i].size >= (1 << 18)); } n = MSEG_NO(); diff --git a/erts/emulator/test/beam_SUITE.erl b/erts/emulator/test/beam_SUITE.erl index 02c6e19686..3197a4c137 100644 --- a/erts/emulator/test/beam_SUITE.erl +++ b/erts/emulator/test/beam_SUITE.erl @@ -54,7 +54,7 @@ end_per_group(_GroupName, Config) -> %% Verify that apply(M, F, A) is really tail recursive. apply_last(Config) when is_list(Config) -> - Pid=spawn(?MODULE, applied, [self(), 10000]), + Pid = spawn(?MODULE, applied, [self(), 10000]), Size = receive {Pid, finished} -> @@ -94,32 +94,32 @@ apply_last_bif(Config) when is_list(Config) -> %% Test three high register numbers in a put_list instruction %% (to test whether packing works properly). packed_registers(Config) when is_list(Config) -> - ?line PrivDir = ?config(priv_dir, Config), - ?line Mod = packed_regs, - ?line Name = filename:join(PrivDir, atom_to_list(Mod) ++ ".erl"), + PrivDir = ?config(priv_dir, Config), + Mod = packed_regs, + Name = filename:join(PrivDir, atom_to_list(Mod) ++ ".erl"), %% Generate a module which generates a list of tuples. %% put_list(A) -> [{A, 600}, {A, 999}, ... {A, 0}]. - ?line Code = gen_packed_regs(600, ["-module("++atom_to_list(Mod)++").\n", + Code = gen_packed_regs(600, ["-module("++atom_to_list(Mod)++").\n", "-export([put_list/1]).\n", "put_list(A) ->\n["]), - ?line ok = file:write_file(Name, Code), + ok = file:write_file(Name, Code), %% Compile the module. - ?line io:format("Compiling: ~s\n", [Name]), - ?line CompRc = compile:file(Name, [{outdir, PrivDir}, report]), - ?line io:format("Result: ~p\n",[CompRc]), - ?line {ok, Mod} = CompRc, + io:format("Compiling: ~s\n", [Name]), + CompRc = compile:file(Name, [{outdir, PrivDir}, report]), + io:format("Result: ~p\n",[CompRc]), + {ok, Mod} = CompRc, %% Load it. - ?line io:format("Loading...\n",[]), - ?line LoadRc = code:load_abs(filename:join(PrivDir, atom_to_list(Mod))), - ?line {module,_Module} = LoadRc, + io:format("Loading...\n",[]), + LoadRc = code:load_abs(filename:join(PrivDir, atom_to_list(Mod))), + {module,_Module} = LoadRc, %% Call it and verify result. - ?line Term = {a, b}, - ?line L = Mod:put_list(Term), - ?line verify_packed_regs(L, Term, 600), + Term = {a, b}, + L = Mod:put_list(Term), + verify_packed_regs(L, Term, 600), ok. gen_packed_regs(0, Acc) -> @@ -131,11 +131,11 @@ verify_packed_regs([], _, -1) -> ok; verify_packed_regs([{Term, N}| T], Term, N) -> verify_packed_regs(T, Term, N-1); verify_packed_regs(L, Term, N) -> - ?line ok = io:format("Expected [{~p, ~p}|T]; got\n~p\n", [Term, N, L]), - ?line test_server:fail(). + ok = io:format("Expected [{~p, ~p}|T]; got\n~p\n", [Term, N, L]), + test_server:fail(). buildo_mucho(Config) when is_list(Config) -> - ?line buildo_mucho_1(), + buildo_mucho_1(), ok. buildo_mucho_1() -> @@ -206,20 +206,27 @@ buildo_mucho_1() -> {<<>>,1},{<<>>,1},{<<>>,1},{<<>>,1},{<<>>,1},{<<>>,1},{<<>>,1},{<<>>,1}]. heap_sizes(Config) when is_list(Config) -> - ?line Sizes = erlang:system_info(heap_sizes), - ?line io:format("~p heap sizes\n", [length(Sizes)]), - ?line io:format("~p\n", [Sizes]), + Sizes = erlang:system_info(heap_sizes), + io:format("~p heap sizes\n", [length(Sizes)]), + io:format("~p\n", [Sizes]), %% Verify that heap sizes increase monotonically. - ?line Largest = lists:foldl(fun(E, P) when is_integer(P), E > P -> E; + Largest = lists:foldl(fun(E, P) when is_integer(P), E > P -> E; (E, []) -> E end, [], Sizes), - %% Verify that the largest heap size consists of 31 or 63 bits. - ?line - case Largest bsr (erlang:system_info(wordsize)*8-2) of - R when R > 0 -> ok - end, + %% Verify that the largest heap size consists of + %% - 31 bits of bytes on 32 bits arch + %% - atleast 52 bits of bytes (48 is the maximum virtual address) + %% and at the most 63 bits on 64 bit archs + %% heap sizes are in words + case erlang:system_info(wordsize) of + 8 -> + 0 = (Largest*8) bsr 63, + true = (Largest*8) > (1 bsl 52); + 4 -> + 1 = (Largest*4) bsr 31 + end, ok. %% Thanks to Igor Goryachev. @@ -302,10 +309,10 @@ b() -> end. fconv(Config) when is_list(Config) -> - ?line do_fconv(atom), - ?line do_fconv(nil), - ?line do_fconv(tuple_literal), - ?line 3.0 = do_fconv(1.0, 2.0), + do_fconv(atom), + do_fconv(nil), + do_fconv(tuple_literal), + 3.0 = do_fconv(1.0, 2.0), ok. do_fconv(Type) -> @@ -325,9 +332,9 @@ do_fconv(tuple_literal, Float) when is_float(Float) -> Float + {a,b}. select_val(Config) when is_list(Config) -> - ?line zero = do_select_val(0), - ?line big = do_select_val(1 bsl 64), - ?line integer = do_select_val(42), + zero = do_select_val(0), + big = do_select_val(1 bsl 64), + integer = do_select_val(42), ok. do_select_val(X) -> diff --git a/erts/emulator/test/bs_bit_binaries_SUITE.erl b/erts/emulator/test/bs_bit_binaries_SUITE.erl index 1428387a65..7f2cd1e881 100644 --- a/erts/emulator/test/bs_bit_binaries_SUITE.erl +++ b/erts/emulator/test/bs_bit_binaries_SUITE.erl @@ -1,7 +1,7 @@ %% %% %CopyrightBegin% %% -%% Copyright Ericsson AB 2006-2011. All Rights Reserved. +%% Copyright Ericsson AB 2006-2012. All Rights Reserved. %% %% The contents of this file are subject to the Erlang Public License, %% Version 1.1, (the "License"); you may not use this file except in diff --git a/erts/emulator/test/bs_construct_SUITE.erl b/erts/emulator/test/bs_construct_SUITE.erl index 9c88803fea..123952d01d 100644 --- a/erts/emulator/test/bs_construct_SUITE.erl +++ b/erts/emulator/test/bs_construct_SUITE.erl @@ -547,11 +547,47 @@ huge_float_check({'EXIT',{badarg,_}}) -> ok. huge_binary(Config) when is_list(Config) -> ?line 16777216 = size(<<0:(id(1 bsl 26)),(-1):(id(1 bsl 26))>>), ?line garbage_collect(), - ?line id(<<0:((1 bsl 32)-1)>>), + {Shift,Return} = case free_mem() of + undefined -> {32,ok}; + Mb when Mb > 600 -> {32,ok}; + Mb when Mb > 300 -> {31,"Limit huge binaries to 256 Mb"}; + _ -> {30,"Limit huge binary to 128 Mb"} + end, ?line garbage_collect(), - ?line id(<<0:(id((1 bsl 32)-1))>>), + ?line id(<<0:((1 bsl Shift)-1)>>), ?line garbage_collect(), - ok. + ?line id(<<0:(id((1 bsl Shift)-1))>>), + ?line garbage_collect(), + case Return of + ok -> ok; + Comment -> {comment, Comment} + end. + +free_mem() -> + Cmd = "uname; free", + Output = string:tokens(os:cmd(Cmd), "\n"), + io:format("Output from command ~p\n~p\n",[Cmd,Output]), + case Output of + [OS, ColumnNames, Values | _] -> + case string:str(OS,"Linux") of + 0 -> + io:format("Unknown OS\n",[]), + undefined; + _ -> + case {string:tokens(ColumnNames, " \t"), + string:tokens(Values, " \t")} of + {[_,_,"free"|_],["Mem:",_,_,FreeKb|_]} -> + list_to_integer(FreeKb) div 1024; + _ -> + io:format("Failed to parse output from 'free':\n",[]), + undefined + end + end; + _ -> + io:format("Too few lines in output\n",[]), + undefined + end. + system_limit(Config) when is_list(Config) -> WordSize = erlang:system_info(wordsize), diff --git a/erts/emulator/test/busy_port_SUITE.erl b/erts/emulator/test/busy_port_SUITE.erl index 3a29fd4d68..32e907ca69 100644 --- a/erts/emulator/test/busy_port_SUITE.erl +++ b/erts/emulator/test/busy_port_SUITE.erl @@ -148,9 +148,9 @@ message_order(Config) when is_list(Config) -> send_to_busy_1(Parent) -> {Owner, Slave} = get_slave(), - Slave ! {Owner, {command, "set_me_busy"}}, - Slave ! {Owner, {command, "hello"}}, - Slave ! {Owner, {command, "hello again"}}, + (catch port_command(Slave, "set_me_busy")), + (catch port_command(Slave, "hello")), + (catch port_command(Slave, "hello again")), receive Message -> Parent ! {self(), Message} @@ -193,10 +193,10 @@ system_monitor(Config) when is_list(Config) -> ?line Busy = spawn_link( fun() -> - Slave ! {Owner,{command,"set busy"}}, + (catch port_command(Slave, "set busy")), receive {Parent,alpha} -> ok end, - Slave ! {Owner,{command,"busy"}}, - Slave ! {Owner,{command,"free"}}, + (catch port_command(Slave, "busy")), + (catch port_command(Slave, "free")), Parent ! {self(),alpha}, command(lock), receive {Parent,beta} -> ok end, @@ -212,7 +212,7 @@ system_monitor(Config) when is_list(Config) -> ?line Void = rec(Void), ?line Busy ! {self(), beta}, ?line {monitor,Owner,busy_port,Slave} = rec(Void), - ?line Master ! {Owner, {command, "u"}}, + ?line port_command(Master, "u"), ?line {Busy,beta} = rec(Void), ?line Void = rec(Void), ?line _NewMonitor = erlang:system_monitor(OldMonitor), @@ -296,9 +296,9 @@ no_trap_exit_process(ResultTo, Link, Config) -> linked -> ok; unlink -> unlink(Slave) end, - ?line Slave ! {self(), {command, "lock port"}}, + ?line (catch port_command(Slave, "lock port")), ?line ResultTo ! {self(), port_created, Slave}, - ?line Slave ! {self(), {command, "suspend me"}}, + ?line (catch port_command(Slave, "suspend me")), ok. %% Assuming the following scenario, @@ -339,9 +339,9 @@ busy_port_exit_process(ResultTo, Config) -> ?line load_busy_driver(Config), ?line _Master = open_port({spawn, "busy_drv master"}, [eof]), ?line Slave = open_port({spawn, "busy_drv slave"}, [eof]), - ?line Slave ! {self(), {command, "lock port"}}, + ?line (catch port_command(Slave, "lock port")), ?line ResultTo ! {self(), port_created, Slave}, - ?line Slave ! {self(), {command, "suspend me"}}, + ?line (catch port_command(Slave, "suspend me")), receive {'EXIT', Slave, die} -> ResultTo ! {self(), ok}; @@ -383,8 +383,8 @@ multiple_writers(Config) when is_list(Config) -> quick_writer() -> {Owner, Port} = get_slave(), - Port ! {Owner, {command, "port to busy"}}, - Port ! {Owner, {command, "lock me"}}, + (catch port_command(Port, "port to busy")), + (catch port_command(Port, "lock me")), ok. hard_busy_driver(Config) when is_list(Config) -> @@ -644,11 +644,11 @@ loop(Master, Slave) -> Pid ! {busy_drv_reply, {self(), Slave}}, loop(Master, Slave); {Pid, unlock} -> - Master ! {self(), {command, "u"}}, + port_command(Master, "u"), Pid ! {busy_drv_reply, ok}, loop(Master, Slave); {Pid, lock} -> - Master ! {self(), {command, "l"}}, + port_command(Master, "l"), Pid ! {busy_drv_reply, ok}, loop(Master, Slave); {Pid, {port_command,Data}} -> diff --git a/erts/emulator/test/ddll_SUITE.erl b/erts/emulator/test/ddll_SUITE.erl index 6e15c228cd..4675cab15c 100644 --- a/erts/emulator/test/ddll_SUITE.erl +++ b/erts/emulator/test/ddll_SUITE.erl @@ -136,8 +136,8 @@ delayed_unload_with_ports(Config) when is_list(Config) -> ?line {ok,pending_driver,Ref} = erl_ddll:try_unload(echo_drv,[{monitor, pending_driver}]), ?line ok = receive _ -> false after 0 -> ok end, ?line Port ! {self(), close}, - ?line 1 = erl_ddll:info(echo_drv, port_count), ?line ok = receive {Port,closed} -> ok after 1000 -> false end, + ?line 1 = erl_ddll:info(echo_drv, port_count), ?line Port2 ! {self(), close}, ?line ok = receive {Port2,closed} -> ok after 1000 -> false end, ?line ok = receive {'DOWN', Ref, driver, echo_drv, unloaded} -> ok after 1000 -> false end, diff --git a/erts/emulator/test/driver_SUITE.erl b/erts/emulator/test/driver_SUITE.erl index 643357263c..13f18b4563 100644 --- a/erts/emulator/test/driver_SUITE.erl +++ b/erts/emulator/test/driver_SUITE.erl @@ -77,7 +77,8 @@ thread_mseg_alloc_cache_clean/1, otp_9302/1, thr_free_drv/1, - async_blast/1]). + async_blast/1, + thr_msg_blast/1]). -export([bin_prefix/2]). @@ -147,7 +148,8 @@ all() -> thread_mseg_alloc_cache_clean, otp_9302, thr_free_drv, - async_blast]. + async_blast, + thr_msg_blast]. groups() -> [{timer, [], @@ -1136,7 +1138,9 @@ check_driver_system_info_result(Result) -> {{1, 1}, _} -> ?line ExpNs = lists:sort(?EXPECTED_SYSTEM_INFO_NAMES -- ?EXPECTED_SYSTEM_INFO_NAMES2), - ?line ExpNs = lists:sort(Ns) + ?line ExpNs = lists:sort(Ns); + {{2, 0}, _} -> + ?line [] = Ns end. chk_sis(SIs, Ns) -> @@ -2010,7 +2014,64 @@ async_blast(Config) when is_list(Config) -> ?line erlang:display({async_blast_time, AsyncBlastTime}), ?line ok. +thr_msg_blast_receiver(_Port, N, N) -> + ok; +thr_msg_blast_receiver(Port, N, Max) -> + receive + {Port, hi} -> + thr_msg_blast_receiver(Port, N+1, Max) + end. + +thr_msg_blast_receiver_proc(Port, Max, Parent, Done) -> + case port_control(Port, 0, "") of + "receiver" -> + spawn(fun () -> + thr_msg_blast_receiver_proc(Port, Max+1, Parent, Done) + end), + thr_msg_blast_receiver(Port, 0, Max); + "done" -> + Parent ! Done + end. +thr_msg_blast(Config) when is_list(Config) -> + case erlang:system_info(smp_support) of + false -> + {skipped, "Non-SMP emulator; nothing to test..."}; + true -> + Path = ?config(data_dir, Config), + erl_ddll:start(), + ok = load_driver(Path, thr_msg_blast_drv), + MemBefore = driver_alloc_size(), + Start = os:timestamp(), + Port = open_port({spawn, thr_msg_blast_drv}, []), + true = is_port(Port), + Done = make_ref(), + Me = self(), + spawn(fun () -> + thr_msg_blast_receiver_proc(Port, 1, Me, Done) + end), + receive + Done -> ok + end, + ok = thr_msg_blast_receiver(Port, 0, 32*10000), + port_close(Port), + End = os:timestamp(), + receive + Garbage -> + ?t:fail({received_garbage, Port, Garbage}) + after 2000 -> + ok + end, + MemAfter = driver_alloc_size(), + io:format("MemBefore=~p, MemAfter=~p~n", + [MemBefore, MemAfter]), + ThrMsgBlastTime = timer:now_diff(End,Start)/1000000, + io:format("ThrMsgBlastTime=~p~n", [ThrMsgBlastTime]), + MemBefore = MemAfter, + Res = {thr_msg_blast_time, ThrMsgBlastTime}, + erlang:display(Res), + Res + end. %%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%% %% Utilities diff --git a/erts/emulator/test/driver_SUITE_data/Makefile.src b/erts/emulator/test/driver_SUITE_data/Makefile.src index 9cc107cc66..b667dff6b6 100644 --- a/erts/emulator/test/driver_SUITE_data/Makefile.src +++ b/erts/emulator/test/driver_SUITE_data/Makefile.src @@ -14,7 +14,8 @@ MISC_DRVS = outputv_drv@dll@ \ thr_alloc_drv@dll@ \ otp_9302_drv@dll@ \ thr_free_drv@dll@ \ - async_blast_drv@dll@ + async_blast_drv@dll@ \ + thr_msg_blast_drv@dll@ SYS_INFO_DRVS = sys_info_base_drv@dll@ \ sys_info_prev_drv@dll@ \ diff --git a/erts/emulator/test/driver_SUITE_data/thr_msg_blast_drv.c b/erts/emulator/test/driver_SUITE_data/thr_msg_blast_drv.c new file mode 100644 index 0000000000..1070678d7b --- /dev/null +++ b/erts/emulator/test/driver_SUITE_data/thr_msg_blast_drv.c @@ -0,0 +1,178 @@ +/* + * %CopyrightBegin% + * + * Copyright Ericsson AB 2012. All Rights Reserved. + * + * The contents of this file are subject to the Erlang Public License, + * Version 1.1, (the "License"); you may not use this file except in + * compliance with the License. You should have received a copy of the + * Erlang Public License along with this software. If not, it can be + * retrieved online at http://www.erlang.org/. + * + * Software distributed under the License is distributed on an "AS IS" + * basis, WITHOUT WARRANTY OF ANY KIND, either express or implied. See + * the License for the specific language governing rights and limitations + * under the License. + * + * %CopyrightEnd% + */ + +#include "erl_driver.h" + +#define THR_MSG_BLAST_NO_PROCS 10 +#define THR_MSG_BLAST_NO_SENDS_PER_PROC 10000 + +#define THR_MSG_BLAST_THREADS 32 + +static void stop(ErlDrvData drv_data); +static ErlDrvData start(ErlDrvPort port, + char *command); +static ErlDrvSSizeT control(ErlDrvData drv_data, + unsigned int command, + char *buf, ErlDrvSizeT len, + char **rbuf, ErlDrvSizeT rlen); + +static ErlDrvEntry thr_msg_blast_drv_entry = { + NULL /* init */, + start, + stop, + NULL /* output */, + NULL /* ready_input */, + NULL /* ready_output */, + "thr_msg_blast_drv", + NULL /* finish */, + NULL /* handle */, + control, + NULL /* timeout */, + NULL /* outputv */, + NULL /* ready_async */, + NULL /* flush */, + NULL /* call */, + NULL /* event */, + ERL_DRV_EXTENDED_MARKER, + ERL_DRV_EXTENDED_MAJOR_VERSION, + ERL_DRV_EXTENDED_MINOR_VERSION, + ERL_DRV_FLAG_USE_PORT_LOCKING, + NULL /* handle2 */, + NULL /* handle_monitor */ +}; + +typedef struct { + ErlDrvPort port; + ErlDrvTermData td_port; + ErlDrvTermData hi; + ErlDrvTid tid[THR_MSG_BLAST_THREADS]; + int no_thrs; + ErlDrvTermData proc[THR_MSG_BLAST_NO_PROCS]; + int no_procs; +} thr_msg_blast_data_t; + + +DRIVER_INIT(thr_msg_blast_drv) +{ + return &thr_msg_blast_drv_entry; +} + +static void stop(ErlDrvData drv_data) +{ + int i; + thr_msg_blast_data_t *tmbd = (thr_msg_blast_data_t *) drv_data; + for (i = 0; i < tmbd->no_thrs; i++) + erl_drv_thread_join(tmbd->tid[i], NULL); + driver_free((void *) tmbd); +} + +static ErlDrvData start(ErlDrvPort port, + char *command) +{ + thr_msg_blast_data_t *tmbd; + + tmbd = driver_alloc(sizeof(thr_msg_blast_data_t)); + if (!tmbd) + return ERL_DRV_ERROR_GENERAL; + + tmbd->port = port; + tmbd->td_port = driver_mk_port(port); + tmbd->hi = driver_mk_atom("hi"); + tmbd->no_thrs = 0; + tmbd->no_procs = 1; + tmbd->proc[0] = driver_caller(port); + + return (ErlDrvData) tmbd; +} + +static void *thread(void *); + +static ErlDrvSSizeT control(ErlDrvData drv_data, + unsigned int command, + char *buf, ErlDrvSizeT len, + char **rbuf, ErlDrvSizeT rlen) +{ + thr_msg_blast_data_t *tmbd = (thr_msg_blast_data_t *) drv_data; + char *res_str = "error"; + + if (tmbd->no_procs >= THR_MSG_BLAST_NO_PROCS) { + int i; + for (i = 0; i < tmbd->no_thrs; i++) + erl_drv_thread_join(tmbd->tid[i], NULL); + tmbd->no_thrs = 0; + res_str = "done"; + } + else { + + tmbd->proc[tmbd->no_procs++] = driver_caller(tmbd->port); + + if (tmbd->no_procs == THR_MSG_BLAST_NO_PROCS) { + for (tmbd->no_thrs = 0; + tmbd->no_thrs < THR_MSG_BLAST_THREADS; + tmbd->no_thrs++) { + int res = erl_drv_thread_create("test", + &tmbd->tid[tmbd->no_thrs], + thread, + tmbd, + NULL); + if (res != 0) { + driver_failure_posix(tmbd->port, res); + goto done; + } + } + } + + res_str = "receiver"; + } + + done: { + ErlDrvSSizeT res_len = strlen(res_str); + if (res_len > rlen) { + char *abuf = driver_alloc(sizeof(char)*res_len); + if (!abuf) + return 0; + *rbuf = abuf; + } + + memcpy((void *) *rbuf, (void *) res_str, res_len); + + return res_len; + } +} + +static void *thread(void *varg) +{ + int s, p; + thr_msg_blast_data_t *tmbd = (thr_msg_blast_data_t *) varg; + ErlDrvTermData spec[] = { + ERL_DRV_PORT, tmbd->td_port, + ERL_DRV_ATOM, tmbd->hi, + ERL_DRV_TUPLE, 2 + }; + + for (s = 0; s < THR_MSG_BLAST_NO_SENDS_PER_PROC; s++) { + for (p = 0; p < THR_MSG_BLAST_NO_PROCS; p++) { + int res = driver_send_term(tmbd->port, tmbd->proc[p], + spec, sizeof(spec)/sizeof(spec[0])); + if (p == 0 && res <= 0) + abort(); /* Could not send to creator */ + } + } + return NULL; +} diff --git a/erts/emulator/test/fun_SUITE.erl b/erts/emulator/test/fun_SUITE.erl index ef06845cf2..36ba4e0f48 100644 --- a/erts/emulator/test/fun_SUITE.erl +++ b/erts/emulator/test/fun_SUITE.erl @@ -726,8 +726,8 @@ t_arity(Config) when is_list(Config) -> ok. t_is_function2(Config) when is_list(Config) -> - ?line true = is_function({a,b}, 0), - ?line true = is_function({a,b}, 234343434333433433), + false = is_function(id({a,b}), 0), + false = is_function(id({a,b}), 234343434333433433), ?line true = is_function(fun() -> ok end, 0), ?line true = is_function(fun(_) -> ok end, 1), ?line false = is_function(fun(_) -> ok end, 0), diff --git a/erts/emulator/test/match_spec_SUITE.erl b/erts/emulator/test/match_spec_SUITE.erl index 461773114e..d5cb4ee1b7 100644 --- a/erts/emulator/test/match_spec_SUITE.erl +++ b/erts/emulator/test/match_spec_SUITE.erl @@ -28,7 +28,7 @@ unary_plus/1, unary_minus/1, moving_labels/1]). -export([fpe/1]). -export([otp_9422/1]). - +-export([faulty_seq_trace/1, do_faulty_seq_trace/0]). -export([runner/2, loop_runner/3]). -export([f1/1, f2/2, f3/2, fn/1, fn/2, fn/3]). -export([do_boxed_and_small/0]). @@ -59,6 +59,7 @@ all() -> ms_trace3, boxed_and_small, destructive_in_test_bif, guard_exceptions, unary_plus, unary_minus, fpe, moving_labels, + faulty_seq_trace, otp_9422]; true -> [not_run] end. @@ -726,6 +727,19 @@ do_boxed_and_small() -> {ok, false, _, _} = erlang:match_spec_test({0,3},[{{make_ref(),'_'},[],['$_']}],table), ok. +faulty_seq_trace(doc) -> + ["Test that faulty seq_trace_call does not crash emulator"]; +faulty_seq_trace(suite) -> []; +faulty_seq_trace(Config) when is_list(Config) -> + ?line {ok, Node} = start_node(match_spec_suite_other), + ?line ok = rpc:call(Node,?MODULE,do_faulty_seq_trace,[]), + ?line stop_node(Node), + ok. + +do_faulty_seq_trace() -> + {ok,'EXIT',_,_} = erlang:match_spec_test([],[{'_',[],[{message,{set_seq_token,yxa,true}}]}],trace), + ok. + errchk(Pat) -> case catch erlang:trace_pattern({?MODULE, f2, 2}, Pat) of {'EXIT', {badarg, _}} -> diff --git a/erts/emulator/test/port_SUITE.erl b/erts/emulator/test/port_SUITE.erl index 873601ddd1..13aa0f4c00 100644 --- a/erts/emulator/test/port_SUITE.erl +++ b/erts/emulator/test/port_SUITE.erl @@ -90,7 +90,7 @@ mix_up_ports/1, otp_5112/1, otp_5119/1, otp_6224/1, exit_status_multi_scheduling_block/1, ports/1, spawn_driver/1, spawn_executable/1, close_deaf_port/1, - unregister_name/1]). + unregister_name/1, parallelism_option/1]). -export([]). @@ -114,7 +114,8 @@ all() -> stderr_to_stdout, otp_3906, otp_4389, win_massive, mix_up_ports, otp_5112, otp_5119, exit_status_multi_scheduling_block, ports, spawn_driver, - spawn_executable, close_deaf_port, unregister_name]. + spawn_executable, close_deaf_port, unregister_name, + parallelism_option]. groups() -> [{stream, [], [stream_small, stream_big]}, @@ -159,11 +160,11 @@ win_massive(Config) when is_list(Config) -> do_win_massive() -> Dog = test_server:timetrap(test_server:seconds(360)), SuiteDir = filename:dirname(code:which(?MODULE)), - Env = " -env ERL_MAX_PORTS 8192", + Ports = " +Q 8192", {ok, Node} = test_server:start_node(win_massive, slave, - [{args, " -pa " ++ SuiteDir ++ Env}]), + [{args, " -pa " ++ SuiteDir ++ Ports}]), ok = rpc:call(Node,?MODULE,win_massive_client,[3000]), test_server:stop_node(Node), test_server:timetrap_cancel(Dog), @@ -1298,6 +1299,43 @@ spawn_driver(Config) when is_list(Config) -> test_server:timetrap_cancel(Dog), ok. +parallelism_option(suite) -> + []; +parallelism_option(doc) -> + ["Test parallelism option of open_port"]; +parallelism_option(Config) when is_list(Config) -> + ?line Dog = test_server:timetrap(test_server:seconds(10)), + ?line Path = ?config(data_dir, Config), + ?line ok = load_driver(Path, "echo_drv"), + ?line Port = erlang:open_port({spawn_driver, "echo_drv"}, + [{parallelism, true}]), + ?line {parallelism, true} = erlang:port_info(Port, parallelism), + ?line Port ! {self(), {command, "Hello port!"}}, + ?line receive + {Port, {data, "Hello port!"}} = Msg1 -> + io:format("~p~n", [Msg1]), + ok; + Other -> + test_server:fail({unexpected, Other}) + end, + ?line Port ! {self(), close}, + ?line receive {Port, closed} -> ok end, + + ?line Port2 = erlang:open_port({spawn_driver, "echo_drv -Hello port?"}, + [{parallelism, false}]), + ?line {parallelism, false} = erlang:port_info(Port2, parallelism), + ?line receive + {Port2, {data, "Hello port?"}} = Msg2 -> + io:format("~p~n", [Msg2]), + ok; + Other2 -> + test_server:fail({unexpected2, Other2}) + end, + ?line Port2 ! {self(), close}, + ?line receive {Port2, closed} -> ok end, + ?line test_server:timetrap_cancel(Dog), + ok. + spawn_executable(suite) -> []; spawn_executable(doc) -> @@ -1566,6 +1604,7 @@ otp_5112(Config) when is_list(Config) -> ?t:format("Links1: ~p~n",[Links1]), true = lists:member(Port, Links1), Port ! {self(), {command, ""}}, + ?line wait_until(fun () -> lists:member(Port, erlang:ports()) == false end), {links, Links2} = process_info(self(),links), ?t:format("Links2: ~p~n",[Links2]), false = lists:member(Port, Links2), %% This used to fail @@ -1636,38 +1675,8 @@ otp_5119_fill_empty_port_tab(Ports) -> LastPort end. --define(DEF_MAX_PORTS, 1024). - -max_ports_env() -> - case os:getenv("ERL_MAX_PORTS") of - EMP when is_list(EMP) -> - case catch list_to_integer(EMP) of - Int when is_integer(Int) -> Int; - _ -> false - end; - _ -> false - end. - max_ports() -> - PreMaxPorts - = case max_ports_env() of - Env when is_integer(Env) -> Env; - _ -> - case os:type() of - {unix, _} -> - UlimStr = string:strip(os:cmd("ulimit -n") - -- "\n"), - case catch list_to_integer(UlimStr) of - Ulim when is_integer(Ulim) -> Ulim; - _ -> ?DEF_MAX_PORTS - end; - _ -> ?DEF_MAX_PORTS - end - end, - case PreMaxPorts > ?DEF_MAX_PORTS of - true -> PreMaxPorts; - false -> ?DEF_MAX_PORTS - end. + erlang:system_info(port_limit). port_ix(Port) when is_port(Port) -> ["#Port",_,PortIxStr] = string:tokens(erlang:port_to_list(Port), @@ -2270,5 +2279,12 @@ close_deaf_port_1(N, Cmd) -> _:eagain -> {comment, "Could not spawn more than " ++ integer_to_list(N) ++ " OS processes."} end. - +wait_until(Fun) -> + case catch Fun() of + true -> + ok; + _ -> + receive after 100 -> ok end, + wait_until(Fun) + end. diff --git a/erts/emulator/test/process_SUITE.erl b/erts/emulator/test/process_SUITE.erl index 6509871a7d..11dd88413f 100644 --- a/erts/emulator/test/process_SUITE.erl +++ b/erts/emulator/test/process_SUITE.erl @@ -118,15 +118,15 @@ fun_spawn(Fun) -> %% (unclear if this test case will actually prove anything on %% a modern computer with lots of memory). spawn_with_binaries(Config) when is_list(Config) -> - ?line L = lists:duplicate(2048, 42), - ?line TwoMeg = lists:duplicate(1024, L), - ?line Fun = fun() -> spawn(?MODULE, binary_owner, [list_to_binary(TwoMeg)]), + L = lists:duplicate(2048, 42), + TwoMeg = lists:duplicate(1024, L), + Fun = fun() -> spawn(?MODULE, binary_owner, [list_to_binary(TwoMeg)]), receive after 1 -> ok end end, - ?line Iter = case test_server:purify_is_running() of + Iter = case test_server:purify_is_running() of true -> 10; false -> 150 end, - ?line test_server:do_times(Iter, Fun), + test_server:do_times(Iter, Fun), ok. binary_owner(Bin) when is_binary(Bin) -> @@ -134,87 +134,87 @@ binary_owner(Bin) when is_binary(Bin) -> %% Tests exit/1 with a big message. t_exit_1(Config) when is_list(Config) -> - ?line start_spawner(), - ?line Dog = test_server:timetrap(test_server:seconds(20)), - ?line process_flag(trap_exit, true), - ?line test_server:do_times(10, fun t_exit_1/0), - ?line test_server:timetrap_cancel(Dog), - ?line stop_spawner(), + start_spawner(), + Dog = test_server:timetrap(test_server:seconds(20)), + process_flag(trap_exit, true), + test_server:do_times(10, fun t_exit_1/0), + test_server:timetrap_cancel(Dog), + stop_spawner(), ok. t_exit_1() -> - ?line Pid = fun_spawn(fun() -> exit(kb_128()) end), - ?line Garbage = kb_128(), - ?line receive + Pid = fun_spawn(fun() -> exit(kb_128()) end), + Garbage = kb_128(), + receive {'EXIT', Pid, Garbage} -> ok end. %% Tests exit/2 with a lot of data in the exit message. t_exit_2_other(Config) when is_list(Config) -> - ?line start_spawner(), - ?line Dog = test_server:timetrap(test_server:seconds(20)), - ?line process_flag(trap_exit, true), - ?line test_server:do_times(10, fun t_exit_2_other/0), - ?line test_server:timetrap_cancel(Dog), - ?line stop_spawner(), + start_spawner(), + Dog = test_server:timetrap(test_server:seconds(20)), + process_flag(trap_exit, true), + test_server:do_times(10, fun t_exit_2_other/0), + test_server:timetrap_cancel(Dog), + stop_spawner(), ok. t_exit_2_other() -> - ?line Pid = fun_spawn(fun() -> receive x -> ok end end), - ?line Garbage = kb_128(), - ?line exit(Pid, Garbage), - ?line receive + Pid = fun_spawn(fun() -> receive x -> ok end end), + Garbage = kb_128(), + exit(Pid, Garbage), + receive {'EXIT', Pid, Garbage} -> ok end. %% Tests that exit(Pid, normal) does not kill another process.; t_exit_2_other_normal(Config) when is_list(Config) -> - ?line Dog = test_server:timetrap(test_server:seconds(20)), - ?line process_flag(trap_exit, true), - ?line Pid = fun_spawn(fun() -> receive x -> ok end end), - ?line exit(Pid, normal), - ?line receive + Dog = test_server:timetrap(test_server:seconds(20)), + process_flag(trap_exit, true), + Pid = fun_spawn(fun() -> receive x -> ok end end), + exit(Pid, normal), + receive {'EXIT', Pid, Reason} -> - ?line test_server:fail({process_died, Reason}) + test_server:fail({process_died, Reason}) after 1000 -> ok end, - ?line case process_info(Pid) of + case process_info(Pid) of undefined -> test_server:fail(process_died_on_normal); List when is_list(List) -> ok end, exit(Pid, kill), - ?line test_server:timetrap_cancel(Dog), + test_server:timetrap_cancel(Dog), ok. %% Tests that we can trap an exit message sent with exit/2 from %% the same process. self_exit(Config) when is_list(Config) -> - ?line start_spawner(), - ?line Dog = test_server:timetrap(test_server:seconds(10)), - ?line process_flag(trap_exit, true), - ?line test_server:do_times(200, fun self_exit/0), - ?line test_server:timetrap_cancel(Dog), - ?line stop_spawner(), + start_spawner(), + Dog = test_server:timetrap(test_server:seconds(10)), + process_flag(trap_exit, true), + test_server:do_times(200, fun self_exit/0), + test_server:timetrap_cancel(Dog), + stop_spawner(), ok. self_exit() -> - ?line Garbage = eight_kb(), - ?line P = self(), - ?line true = exit(P, Garbage), - ?line receive + Garbage = eight_kb(), + P = self(), + true = exit(P, Garbage), + receive {'EXIT', P, Garbage} -> ok end. %% Tests exit(self(), normal) is equivalent to exit(normal) for a process %% that doesn't trap exits. normal_suicide_exit(Config) when is_list(Config) -> - ?line process_flag(trap_exit, true), - ?line Pid = fun_spawn(fun() -> exit(self(), normal) end), - ?line receive + process_flag(trap_exit, true), + Pid = fun_spawn(fun() -> exit(self(), normal) end), + receive {'EXIT', Pid, normal} -> ok; Other -> test_server:fail({bad_message, Other}) end. @@ -222,19 +222,19 @@ normal_suicide_exit(Config) when is_list(Config) -> %% Tests exit(self(), Term) is equivalent to exit(Term) for a process %% that doesn't trap exits."; abnormal_suicide_exit(Config) when is_list(Config) -> - ?line Garbage = eight_kb(), - ?line process_flag(trap_exit, true), - ?line Pid = fun_spawn(fun() -> exit(self(), Garbage) end), - ?line receive + Garbage = eight_kb(), + process_flag(trap_exit, true), + Pid = fun_spawn(fun() -> exit(self(), Garbage) end), + receive {'EXIT', Pid, Garbage} -> ok; Other -> test_server:fail({bad_message, Other}) end. %% Tests that exit(self(), die) cannot be catched. t_exit_2_catch(Config) when is_list(Config) -> - ?line process_flag(trap_exit, true), - ?line Pid = fun_spawn(fun() -> catch exit(self(), die) end), - ?line receive + process_flag(trap_exit, true), + Pid = fun_spawn(fun() -> catch exit(self(), die) end), + receive {'EXIT', Pid, normal} -> test_server:fail(catch_worked); {'EXIT', Pid, die} -> @@ -246,29 +246,29 @@ t_exit_2_catch(Config) when is_list(Config) -> %% Tests trapping of an 'EXIT' message generated by a bad argument to %% the abs/1 bif. The 'EXIT' message will intentionally be very big. trap_exit_badarg(Config) when is_list(Config) -> - ?line start_spawner(), - ?line Dog = test_server:timetrap(test_server:seconds(10)), - ?line process_flag(trap_exit, true), - ?line test_server:do_times(10, fun trap_exit_badarg/0), - ?line test_server:timetrap_cancel(Dog), - ?line stop_spawner(), + start_spawner(), + Dog = test_server:timetrap(test_server:seconds(10)), + process_flag(trap_exit, true), + test_server:do_times(10, fun trap_exit_badarg/0), + test_server:timetrap_cancel(Dog), + stop_spawner(), ok. trap_exit_badarg() -> - ?line Pid = fun_spawn(fun() -> bad_guy(kb_128()) end), - ?line Garbage = kb_128(), - ?line receive + Pid = fun_spawn(fun() -> bad_guy(kb_128()) end), + Garbage = kb_128(), + receive {'EXIT',Pid,{badarg,[{erlang,abs,[Garbage],Loc1}, {?MODULE,bad_guy,1,Loc2}|_]}} when is_list(Loc1), is_list(Loc2) -> ok; Other -> - ?line ok = io:format("Bad EXIT message: ~P", [Other, 30]), - ?line test_server:fail(bad_exit_message) + ok = io:format("Bad EXIT message: ~P", [Other, 30]), + test_server:fail(bad_exit_message) end. bad_guy(Arg) -> - ?line abs(Arg). + abs(Arg). kb_128() -> @@ -281,11 +281,11 @@ kb_128() -> eight_kb() -> B64 = lists:seq(1, 64), - ?line B512 = {<<1>>,B64,<<2,3>>,B64,make_unaligned_sub_binary(<<4,5,6,7,8,9>>), + B512 = {<<1>>,B64,<<2,3>>,B64,make_unaligned_sub_binary(<<4,5,6,7,8,9>>), B64,make_sub_binary([1,2,3,4,5,6]), B64,make_sub_binary(lists:seq(1, ?heap_binary_size+1)), B64,B64,B64,B64,big_binary()}, - ?line lists:duplicate(8, {B512,B512}). + lists:duplicate(8, {B512,B512}). big_binary() -> big_binary(10, [42]). @@ -296,19 +296,19 @@ big_binary(N, Acc) -> %% Test receiving an EXIT message when spawning a BIF with bad arguments. trap_exit_badarg_in_bif(Config) when is_list(Config) -> - ?line Dog = test_server:timetrap(test_server:seconds(10)), - ?line process_flag(trap_exit, true), - ?line test_server:do_times(10, fun trap_exit_badarg_bif/0), - ?line test_server:timetrap_cancel(Dog), + Dog = test_server:timetrap(test_server:seconds(10)), + process_flag(trap_exit, true), + test_server:do_times(10, fun trap_exit_badarg_bif/0), + test_server:timetrap_cancel(Dog), ok. trap_exit_badarg_bif() -> - ?line Pid = spawn_link(erlang, node, [1]), - ?line receive + Pid = spawn_link(erlang, node, [1]), + receive {'EXIT', Pid, {badarg, _}} -> ok; Other -> - ?line test_server:fail({unexpected, Other}) + test_server:fail({unexpected, Other}) end. %% The following sequences of events have crasched Beam. @@ -321,27 +321,27 @@ trap_exit_badarg_bif() -> %% 3) The process will crash the next time it executes 'receive'. exit_and_timeout(Config) when is_list(Config) -> - ?line Dog = test_server:timetrap(test_server:seconds(20)), + Dog = test_server:timetrap(test_server:seconds(20)), - ?line process_flag(trap_exit, true), - ?line Parent = self(), - ?line Low = fun_spawn(fun() -> eat_low(Parent) end), - ?line High = fun_spawn(fun() -> eat_high(Low) end), - ?line eat_wait_for(Low, High), + process_flag(trap_exit, true), + Parent = self(), + Low = fun_spawn(fun() -> eat_low(Parent) end), + High = fun_spawn(fun() -> eat_high(Low) end), + eat_wait_for(Low, High), - ?line test_server:timetrap_cancel(Dog), + test_server:timetrap_cancel(Dog), ok. eat_wait_for(Low, High) -> - ?line receive - {'EXIT', Low, {you, are, dead}} -> - ok; - {'EXIT', High, normal} -> - eat_wait_for(Low, High); - Other -> - test_server:fail({bad_message, Other}) - end. + receive + {'EXIT', Low, {you, are, dead}} -> + ok; + {'EXIT', High, normal} -> + eat_wait_for(Low, High); + Other -> + test_server:fail({bad_message, Other}) + end. eat_low(_Parent) -> receive @@ -374,27 +374,27 @@ loop(_, _) -> %% Tries to send two different exit messages to a process. %% (The second one should be ignored.) exit_twice(Config) when is_list(Config) -> - ?line Dog = test_server:timetrap(test_server:seconds(20)), + Dog = test_server:timetrap(test_server:seconds(20)), - ?line process_flag(trap_exit, true), - ?line Low = fun_spawn(fun etwice_low/0), - ?line High = fun_spawn(fun() -> etwice_high(Low) end), - ?line etwice_wait_for(Low, High), + process_flag(trap_exit, true), + Low = fun_spawn(fun etwice_low/0), + High = fun_spawn(fun() -> etwice_high(Low) end), + etwice_wait_for(Low, High), - ?line test_server:timetrap_cancel(Dog), + test_server:timetrap_cancel(Dog), ok. etwice_wait_for(Low, High) -> - ?line receive - {'EXIT', Low, first} -> - ok; - {'EXIT', Low, Other} -> - test_server:fail({wrong_exit_reason, Other}); - {'EXIT', High, normal} -> - etwice_wait_for(Low, High); - Other -> - test_server:fail({bad_message, Other}) - end. + receive + {'EXIT', Low, first} -> + ok; + {'EXIT', Low, Other} -> + test_server:fail({wrong_exit_reason, Other}); + {'EXIT', High, normal} -> + etwice_wait_for(Low, High); + Other -> + test_server:fail({bad_message, Other}) + end. etwice_low() -> etwice_low(). @@ -406,15 +406,15 @@ etwice_high(Low) -> %% Tests the process_info/2 BIF. t_process_info(Config) when is_list(Config) -> - ?line [] = process_info(self(), registered_name), - ?line register(my_name, self()), - ?line {registered_name, my_name} = process_info(self(), registered_name), - ?line {status, running} = process_info(self(), status), - ?line {min_heap_size, 233} = process_info(self(), min_heap_size), - ?line {min_bin_vheap_size, 46368} = process_info(self(), min_bin_vheap_size), - ?line {current_function,{?MODULE,t_process_info,1}} = + [] = process_info(self(), registered_name), + register(my_name, self()), + {registered_name, my_name} = process_info(self(), registered_name), + {status, running} = process_info(self(), status), + {min_heap_size, 233} = process_info(self(), min_heap_size), + {min_bin_vheap_size,46422} = process_info(self(), min_bin_vheap_size), + {current_function,{?MODULE,t_process_info,1}} = process_info(self(), current_function), - ?line {current_function,{?MODULE,t_process_info,1}} = + {current_function,{?MODULE,t_process_info,1}} = apply(erlang, process_info, [self(),current_function]), %% current_location and current_stacktrace @@ -425,9 +425,9 @@ t_process_info(Config) when is_list(Config) -> verify_loc(Line2, Res2), pi_stacktrace([{?MODULE,t_process_info,1,?LINE}]), - ?line Gleader = group_leader(), - ?line {group_leader, Gleader} = process_info(self(), group_leader), - ?line {'EXIT',{badarg,_Info}} = (catch process_info('not_a_pid')), + Gleader = group_leader(), + {group_leader, Gleader} = process_info(self(), group_leader), + {'EXIT',{badarg,_Info}} = (catch process_info('not_a_pid')), ok. pi_stacktrace(Expected0) -> @@ -509,50 +509,50 @@ process_info_looper(Parent) -> %% Tests the process_info/1 BIF on another process with messages. process_info_other_msg(Config) when is_list(Config) -> Self = self(), - ?line Pid = spawn_link(fun() -> other_process(Self) end), + Pid = spawn_link(fun() -> other_process(Self) end), receive {go_ahead,Pid} -> ok end, - ?line Own = {my,own,message}, + Own = {my,own,message}, - ?line {messages,[Own]} = process_info(Pid, messages), + {messages,[Own]} = process_info(Pid, messages), - ?line Garbage = kb_128(), - ?line MsgA = {a,Garbage}, - ?line MsgB = {b,Garbage}, - ?line MsgC = {c,Garbage}, - ?line MsgD = {d,Garbage}, - ?line MsgE = {e,Garbage}, - - ?line Pid ! MsgA, - ?line {messages,[Own,MsgA]} = process_info(Pid, messages), - ?line Pid ! MsgB, - ?line {messages,[Own,MsgA,MsgB]} = process_info(Pid, messages), - ?line Pid ! MsgC, - ?line {messages,[Own,MsgA,MsgB,MsgC]} = process_info(Pid, messages), - ?line Pid ! MsgD, - ?line {messages,[Own,MsgA,MsgB,MsgC,MsgD]} = process_info(Pid, messages), - ?line Pid ! MsgE, - ?line {messages,[Own,MsgA,MsgB,MsgC,MsgD,MsgE]=All} = process_info(Pid, messages), - ?line {memory,BytesOther} = process_info(Pid, memory), - ?line {memory,BytesSelf} = process_info(self(), memory), + Garbage = kb_128(), + MsgA = {a,Garbage}, + MsgB = {b,Garbage}, + MsgC = {c,Garbage}, + MsgD = {d,Garbage}, + MsgE = {e,Garbage}, + + Pid ! MsgA, + {messages,[Own,MsgA]} = process_info(Pid, messages), + Pid ! MsgB, + {messages,[Own,MsgA,MsgB]} = process_info(Pid, messages), + Pid ! MsgC, + {messages,[Own,MsgA,MsgB,MsgC]} = process_info(Pid, messages), + Pid ! MsgD, + {messages,[Own,MsgA,MsgB,MsgC,MsgD]} = process_info(Pid, messages), + Pid ! MsgE, + {messages,[Own,MsgA,MsgB,MsgC,MsgD,MsgE]=All} = process_info(Pid, messages), + {memory,BytesOther} = process_info(Pid, memory), + {memory,BytesSelf} = process_info(self(), memory), io:format("Memory ~p: ~p\n", [Pid,BytesOther]), io:format("Memory ~p (self): ~p\n", [self(),BytesSelf]), [Own,MsgA,MsgB,MsgC,MsgD,MsgE] = All, - ?line Pid ! {self(),empty}, - ?line receive + Pid ! {self(),empty}, + receive empty -> ok end, - ?line {messages,[]} = process_info(Pid, messages), + {messages,[]} = process_info(Pid, messages), - ?line {min_heap_size, 233} = process_info(Pid, min_heap_size), - ?line {min_bin_vheap_size, 46368} = process_info(Pid, min_bin_vheap_size), + {min_heap_size, 233} = process_info(Pid, min_heap_size), + {min_bin_vheap_size, 46422} = process_info(Pid, min_bin_vheap_size), - ?line Pid ! stop, + Pid ! stop, ok. process_info_other_dist_msg(Config) when is_list(Config) -> @@ -560,52 +560,51 @@ process_info_other_dist_msg(Config) when is_list(Config) -> %% Check that process_info can handle messages that have not been %% decoded yet. %% - ?line {ok, Node} = start_node(Config), - ?line Self = self(), - ?line Pid = spawn_link(fun() -> other_process(Self) end), - ?line receive {go_ahead,Pid} -> ok end, + {ok, Node} = start_node(Config), + Self = self(), + Pid = spawn_link(fun() -> other_process(Self) end), + receive {go_ahead,Pid} -> ok end, - ?line Own = {my,own,message}, + Own = {my,own,message}, - ?line {messages,[Own]} = process_info(Pid, messages), - ?line Garbage = kb_128(), - ?line MsgA = {a,self(),Garbage}, - ?line MsgB = {b,self(),Garbage}, - ?line MsgC = {c,self(),Garbage}, - ?line MsgD = {d,self(),Garbage}, - ?line MsgE = {e,self(),Garbage}, + {messages,[Own]} = process_info(Pid, messages), + Garbage = kb_128(), + MsgA = {a,self(),Garbage}, + MsgB = {b,self(),Garbage}, + MsgC = {c,self(),Garbage}, + MsgD = {d,self(),Garbage}, + MsgE = {e,self(),Garbage}, %% We don't want the other process to decode messages itself %% therefore we suspend it. - ?line true = erlang:suspend_process(Pid), - ?line spawn_link(Node, fun () -> - Pid ! MsgA, - Pid ! MsgB, - Pid ! MsgC, - Self ! check_abc - end), - ?line receive check_abc -> ok end, - ?line [{status,suspended}, - {messages,[Own,MsgA,MsgB,MsgC]}, - {status,suspended}]= process_info(Pid, [status,messages,status]), - ?line spawn_link(Node, fun () -> - Pid ! MsgD, - Pid ! MsgE, - Self ! check_de - end), - ?line receive check_de -> ok end, - ?line {messages,[Own,MsgA,MsgB,MsgC,MsgD,MsgE]=All} - = process_info(Pid, messages), - ?line true = erlang:resume_process(Pid), - ?line Pid ! {self(), get_all_messages}, - ?line receive + true = erlang:suspend_process(Pid), + spawn_link(Node, fun () -> + Pid ! MsgA, + Pid ! MsgB, + Pid ! MsgC, + Self ! check_abc + end), + receive check_abc -> ok end, + [{status,suspended}, + {messages,[Own,MsgA,MsgB,MsgC]}, + {status,suspended}]= process_info(Pid, [status,messages,status]), + spawn_link(Node, fun () -> + Pid ! MsgD, + Pid ! MsgE, + Self ! check_de + end), + receive check_de -> ok end, + {messages,[Own,MsgA,MsgB,MsgC,MsgD,MsgE]=All} = process_info(Pid, messages), + true = erlang:resume_process(Pid), + Pid ! {self(), get_all_messages}, + receive {all_messages, AllMsgs} -> - ?line All = AllMsgs + All = AllMsgs end, - ?line {messages,[]} = process_info(Pid, messages), - ?line Pid ! stop, - ?line stop_node(Node), - ?line ok. + {messages,[]} = process_info(Pid, messages), + Pid ! stop, + stop_node(Node), + ok. other_process(Parent) -> @@ -652,38 +651,36 @@ process_info_2_list(doc) -> process_info_2_list(suite) -> []; process_info_2_list(Config) when is_list(Config) -> - ?line Proc = spawn(fun () -> - receive after infinity -> ok end end), + Proc = spawn(fun () -> receive after infinity -> ok end end), register(process_SUITE_process_info_2_list1, self()), register(process_SUITE_process_info_2_list2, Proc), - ?line erts_debug:set_internal_state(available_internal_state,true), - ?line AllArgs = erts_debug:get_internal_state(process_info_args), - ?line A1 = lists:sort(AllArgs) ++ [status] ++ lists:reverse(AllArgs), + erts_debug:set_internal_state(available_internal_state,true), + AllArgs = erts_debug:get_internal_state(process_info_args), + A1 = lists:sort(AllArgs) ++ [status] ++ lists:reverse(AllArgs), %% Verify that argument is accepted as single atom - ?line lists:foreach(fun (A) -> - ?line {A, _} = process_info(Proc, A), - ?line {A, _} = process_info(self(), A) - end, - A1), + lists:foreach(fun (A) -> + {A, _} = process_info(Proc, A), + {A, _} = process_info(self(), A) + end, A1), %% Verify that order is preserved - ?line ok = chk_pi_order(process_info(self(), A1), A1), - ?line ok = chk_pi_order(process_info(Proc, A1), A1), + ok = chk_pi_order(process_info(self(), A1), A1), + ok = chk_pi_order(process_info(Proc, A1), A1), %% Small arg list - ?line A2 = [status, stack_size, trap_exit, priority], - ?line [{status, _}, {stack_size, _}, {trap_exit, _}, {priority, _}] + A2 = [status, stack_size, trap_exit, priority], + [{status, _}, {stack_size, _}, {trap_exit, _}, {priority, _}] = process_info(Proc, A2), - ?line [{status, _}, {stack_size, _}, {trap_exit, _}, {priority, _}] + [{status, _}, {stack_size, _}, {trap_exit, _}, {priority, _}] = process_info(self(), A2), %% Huge arg list (note values are shared) - ?line A3 = lists:duplicate(5000,backtrace), - ?line V3 = process_info(Proc, A3), - ?line 5000 = length(V3), - ?line lists:foreach(fun ({backtrace, _}) -> ok end, V3), - ?line ok. + A3 = lists:duplicate(5000,backtrace), + V3 = process_info(Proc, A3), + 5000 = length(V3), + lists:foreach(fun ({backtrace, _}) -> ok end, V3), + ok. process_info_lock_reschedule(doc) -> []; @@ -692,43 +689,37 @@ process_info_lock_reschedule(suite) -> process_info_lock_reschedule(Config) when is_list(Config) -> %% We need a process that is running and an item that requires %% process_info to take the main process lock. - ?line Target1 = spawn_link(fun tok_loop/0), - ?line Name1 = process_info_lock_reschedule_running, - ?line register(Name1, Target1), - ?line Target2 = spawn_link(fun () -> receive after infinity -> ok end end), - ?line Name2 = process_info_lock_reschedule_waiting, - ?line register(Name2, Target2), - ?line PI = fun(_) -> - ?line erlang:yield(), - ?line [{registered_name, Name1}] - = process_info(Target1, [registered_name]), - ?line [{registered_name, Name2}] - = process_info(Target2, [registered_name]), - ?line erlang:yield(), - ?line {registered_name, Name1} - = process_info(Target1, registered_name), - ?line {registered_name, Name2} - = process_info(Target2, registered_name), - ?line erlang:yield(), - ?line [{registered_name, Name1}| _] - = process_info(Target1), - ?line [{registered_name, Name2}| _] - = process_info(Target2) - end, - ?line lists:foreach(PI, lists:seq(1,1000)), + Target1 = spawn_link(fun tok_loop/0), + Name1 = process_info_lock_reschedule_running, + register(Name1, Target1), + Target2 = spawn_link(fun () -> receive after infinity -> ok end end), + Name2 = process_info_lock_reschedule_waiting, + register(Name2, Target2), + PI = fun(_) -> + erlang:yield(), + [{registered_name, Name1}] = process_info(Target1, [registered_name]), + [{registered_name, Name2}] = process_info(Target2, [registered_name]), + erlang:yield(), + {registered_name, Name1} = process_info(Target1, registered_name), + {registered_name, Name2} = process_info(Target2, registered_name), + erlang:yield(), + [{registered_name, Name1}| _] = process_info(Target1), + [{registered_name, Name2}| _] = process_info(Target2) + end, + lists:foreach(PI, lists:seq(1,1000)), %% Make sure Target1 still is willing to "tok loop" - ?line case process_info(Target1, status) of - {status, OkStatus} when OkStatus == runnable; - OkStatus == running; - OkStatus == garbage_collecting -> - ?line unlink(Target1), - ?line unlink(Target2), - ?line exit(Target1, bang), - ?line exit(Target2, bang), - ?line OkStatus; - {status, BadStatus} -> - ?line ?t:fail(BadStatus) - end. + case process_info(Target1, status) of + {status, OkStatus} when OkStatus == runnable; + OkStatus == running; + OkStatus == garbage_collecting -> + unlink(Target1), + unlink(Target2), + exit(Target1, bang), + exit(Target2, bang), + OkStatus; + {status, BadStatus} -> + ?t:fail(BadStatus) + end. pi_loop(_Name, _Pid, 0) -> ok; @@ -741,50 +732,50 @@ process_info_lock_reschedule2(doc) -> process_info_lock_reschedule2(suite) -> []; process_info_lock_reschedule2(Config) when is_list(Config) -> - ?line Parent = self(), - ?line Fun = fun () -> - receive {go, Name, Pid} -> ok end, - pi_loop(Name, Pid, 10000), - Parent ! {done, self()}, - receive after infinity -> ok end - end, - ?line P1 = spawn_link(Fun), - ?line N1 = process_info_lock_reschedule2_1, - ?line true = register(N1, P1), - ?line P2 = spawn_link(Fun), - ?line N2 = process_info_lock_reschedule2_2, - ?line true = register(N2, P2), - ?line P3 = spawn_link(Fun), - ?line N3 = process_info_lock_reschedule2_3, - ?line true = register(N3, P3), - ?line P4 = spawn_link(Fun), - ?line N4 = process_info_lock_reschedule2_4, - ?line true = register(N4, P4), - ?line P5 = spawn_link(Fun), - ?line N5 = process_info_lock_reschedule2_5, - ?line true = register(N5, P5), - ?line P6 = spawn_link(Fun), - ?line N6 = process_info_lock_reschedule2_6, - ?line true = register(N6, P6), - ?line P1 ! {go, N2, P2}, - ?line P2 ! {go, N1, P1}, - ?line P3 ! {go, N1, P1}, - ?line P4 ! {go, N1, P1}, - ?line P5 ! {go, N6, P6}, - ?line P6 ! {go, N5, P5}, - ?line receive {done, P1} -> ok end, - ?line receive {done, P2} -> ok end, - ?line receive {done, P3} -> ok end, - ?line receive {done, P4} -> ok end, - ?line receive {done, P5} -> ok end, - ?line receive {done, P6} -> ok end, - ?line unlink(P1), exit(P1, bang), - ?line unlink(P2), exit(P2, bang), - ?line unlink(P3), exit(P3, bang), - ?line unlink(P4), exit(P4, bang), - ?line unlink(P5), exit(P5, bang), - ?line unlink(P6), exit(P6, bang), - ?line ok. + Parent = self(), + Fun = fun () -> + receive {go, Name, Pid} -> ok end, + pi_loop(Name, Pid, 10000), + Parent ! {done, self()}, + receive after infinity -> ok end + end, + P1 = spawn_link(Fun), + N1 = process_info_lock_reschedule2_1, + true = register(N1, P1), + P2 = spawn_link(Fun), + N2 = process_info_lock_reschedule2_2, + true = register(N2, P2), + P3 = spawn_link(Fun), + N3 = process_info_lock_reschedule2_3, + true = register(N3, P3), + P4 = spawn_link(Fun), + N4 = process_info_lock_reschedule2_4, + true = register(N4, P4), + P5 = spawn_link(Fun), + N5 = process_info_lock_reschedule2_5, + true = register(N5, P5), + P6 = spawn_link(Fun), + N6 = process_info_lock_reschedule2_6, + true = register(N6, P6), + P1 ! {go, N2, P2}, + P2 ! {go, N1, P1}, + P3 ! {go, N1, P1}, + P4 ! {go, N1, P1}, + P5 ! {go, N6, P6}, + P6 ! {go, N5, P5}, + receive {done, P1} -> ok end, + receive {done, P2} -> ok end, + receive {done, P3} -> ok end, + receive {done, P4} -> ok end, + receive {done, P5} -> ok end, + receive {done, P6} -> ok end, + unlink(P1), exit(P1, bang), + unlink(P2), exit(P2, bang), + unlink(P3), exit(P3, bang), + unlink(P4), exit(P4, bang), + unlink(P5), exit(P5, bang), + unlink(P6), exit(P6, bang), + ok. many_args(0,_B,_C,_D,_E,_F,_G,_H,_I,_J) -> ok; @@ -802,120 +793,115 @@ process_info_lock_reschedule3(suite) -> process_info_lock_reschedule3(Config) when is_list(Config) -> %% We need a process that is running and an item that requires %% process_info to take the main process lock. - ?line Target1 = spawn_link(fun tok_loop/0), - ?line Name1 = process_info_lock_reschedule_running, - ?line register(Name1, Target1), - ?line Target2 = spawn_link(fun () -> receive after infinity -> ok end end), - ?line Name2 = process_info_lock_reschedule_waiting, - ?line register(Name2, Target2), - ?line PI = fun(N) -> - case N rem 10 of - 0 -> erlang:yield(); - _ -> ok - end, - ?line do_pi_msg_len({proc, Target1}, - {arg, message_queue_len}) - end, - ?line many_args(100000,1,2,3,4,5,6,7,8,9), - ?line lists:foreach(PI, lists:seq(1,1000000)), + Target1 = spawn_link(fun tok_loop/0), + Name1 = process_info_lock_reschedule_running, + register(Name1, Target1), + Target2 = spawn_link(fun () -> receive after infinity -> ok end end), + Name2 = process_info_lock_reschedule_waiting, + register(Name2, Target2), + PI = fun(N) -> + case N rem 10 of + 0 -> erlang:yield(); + _ -> ok + end, + do_pi_msg_len({proc, Target1}, + {arg, message_queue_len}) + end, + many_args(100000,1,2,3,4,5,6,7,8,9), + lists:foreach(PI, lists:seq(1,1000000)), %% Make sure Target1 still is willing to "tok loop" - ?line case process_info(Target1, status) of + case process_info(Target1, status) of {status, OkStatus} when OkStatus == runnable; OkStatus == running; OkStatus == garbage_collecting -> - ?line unlink(Target1), - ?line unlink(Target2), - ?line exit(Target1, bang), - ?line exit(Target2, bang), - ?line OkStatus; + unlink(Target1), + unlink(Target2), + exit(Target1, bang), + exit(Target2, bang), + OkStatus; {status, BadStatus} -> - ?line ?t:fail(BadStatus) + ?t:fail(BadStatus) end. process_status_exiting(Config) when is_list(Config) -> %% Make sure that erts_debug:get_internal_state({process_status,P}) %% returns exiting if it is in status P_EXITING. - ?line erts_debug:set_internal_state(available_internal_state,true), - ?line Prio = process_flag(priority, max), - ?line P = spawn_opt(fun () -> receive after infinity -> ok end end, + erts_debug:set_internal_state(available_internal_state,true), + Prio = process_flag(priority, max), + P = spawn_opt(fun () -> receive after infinity -> ok end end, [{priority, normal}]), - ?line erlang:yield(), + erlang:yield(), %% The tok_loop processes are here to make it hard for the exiting %% process to be scheduled in for exit... - ?line TokLoops = lists:map(fun (_) -> - spawn_opt(fun tok_loop/0, - [link,{priority, high}]) - end, - lists:seq(1, erlang:system_info(schedulers_online))), - ?line exit(P, boom), - ?line wait_until( - fun () -> - exiting =:= erts_debug:get_internal_state({process_status,P}) - end), - ?line lists:foreach(fun (Tok) -> unlink(Tok), exit(Tok,bang) end, TokLoops), - ?line process_flag(priority, Prio), - ?line ok. + TokLoops = lists:map(fun (_) -> + spawn_opt(fun tok_loop/0, + [link,{priority, high}]) + end, lists:seq(1, erlang:system_info(schedulers_online))), + exit(P, boom), + wait_until(fun() -> + exiting =:= erts_debug:get_internal_state({process_status,P}) + end), + lists:foreach(fun (Tok) -> unlink(Tok), exit(Tok,bang) end, TokLoops), + process_flag(priority, Prio), + ok. otp_4725(Config) when is_list(Config) -> - ?line Tester = self(), - ?line Ref1 = make_ref(), - ?line Pid1 = spawn_opt(fun () -> - Tester ! {Ref1, process_info(self())}, - receive - Ref1 -> bye - end - end, - [link, - {priority, max}, - {fullsweep_after, 600}]), - ?line receive - {Ref1, ProcInfo1A} -> - ?line ProcInfo1B = process_info(Pid1), - ?line Pid1 ! Ref1, - ?line check_proc_infos(ProcInfo1A, ProcInfo1B) - end, - ?line Ref2 = make_ref(), - ?line Pid2 = spawn_opt(fun () -> - Tester ! {Ref2, process_info(self())}, - receive - Ref2 -> bye - end - end, - []), - ?line receive - {Ref2, ProcInfo2A} -> - ?line ProcInfo2B = process_info(Pid2), - ?line Pid2 ! Ref2, - ?line check_proc_infos(ProcInfo2A, ProcInfo2B) - end, - ?line ok. + Tester = self(), + Ref1 = make_ref(), + Pid1 = spawn_opt(fun () -> + Tester ! {Ref1, process_info(self())}, + receive + Ref1 -> bye + end + end, [link, {priority, max}, {fullsweep_after, 600}]), + receive + {Ref1, ProcInfo1A} -> + ProcInfo1B = process_info(Pid1), + Pid1 ! Ref1, + check_proc_infos(ProcInfo1A, ProcInfo1B) + end, + Ref2 = make_ref(), + Pid2 = spawn_opt(fun () -> + Tester ! {Ref2, process_info(self())}, + receive + Ref2 -> bye + end + end, + []), + receive + {Ref2, ProcInfo2A} -> + ProcInfo2B = process_info(Pid2), + Pid2 ! Ref2, + check_proc_infos(ProcInfo2A, ProcInfo2B) + end, + ok. check_proc_infos(A, B) -> - ?line IC = lists:keysearch(initial_call, 1, A), - ?line IC = lists:keysearch(initial_call, 1, B), + IC = lists:keysearch(initial_call, 1, A), + IC = lists:keysearch(initial_call, 1, B), - ?line L = lists:keysearch(links, 1, A), - ?line L = lists:keysearch(links, 1, B), + L = lists:keysearch(links, 1, A), + L = lists:keysearch(links, 1, B), - ?line D = lists:keysearch(dictionary, 1, A), - ?line D = lists:keysearch(dictionary, 1, B), + D = lists:keysearch(dictionary, 1, A), + D = lists:keysearch(dictionary, 1, B), - ?line TE = lists:keysearch(trap_exit, 1, A), - ?line TE = lists:keysearch(trap_exit, 1, B), + TE = lists:keysearch(trap_exit, 1, A), + TE = lists:keysearch(trap_exit, 1, B), - ?line EH = lists:keysearch(error_handler, 1, A), - ?line EH = lists:keysearch(error_handler, 1, B), + EH = lists:keysearch(error_handler, 1, A), + EH = lists:keysearch(error_handler, 1, B), - ?line P = lists:keysearch(priority, 1, A), - ?line P = lists:keysearch(priority, 1, B), + P = lists:keysearch(priority, 1, A), + P = lists:keysearch(priority, 1, B), - ?line GL = lists:keysearch(group_leader, 1, A), - ?line GL = lists:keysearch(group_leader, 1, B), + GL = lists:keysearch(group_leader, 1, A), + GL = lists:keysearch(group_leader, 1, B), - ?line GC = lists:keysearch(garbage_collection, 1, A), - ?line GC = lists:keysearch(garbage_collection, 1, B), + GC = lists:keysearch(garbage_collection, 1, A), + GC = lists:keysearch(garbage_collection, 1, B), - ?line ok. + ok. %% Dummies. @@ -928,18 +914,18 @@ stop_spawner() -> %% Tests erlang:bump_reductions/1. bump_reductions(Config) when is_list(Config) -> - ?line erlang:garbage_collect(), - ?line receive after 1 -> ok end, % Clear reductions. - ?line {reductions,R1} = process_info(self(), reductions), - ?line true = erlang:bump_reductions(100), - ?line {reductions,R2} = process_info(self(), reductions), - ?line case R2-R1 of + erlang:garbage_collect(), + receive after 1 -> ok end, % Clear reductions. + {reductions,R1} = process_info(self(), reductions), + true = erlang:bump_reductions(100), + {reductions,R2} = process_info(self(), reductions), + case R2-R1 of Diff when Diff < 100 -> - ?line ok = io:format("R1 = ~w, R2 = ~w", [R1, R2]), - ?line test_server:fail({small_diff, Diff}); + ok = io:format("R1 = ~w, R2 = ~w", [R1, R2]), + test_server:fail({small_diff, Diff}); Diff when Diff > 110 -> - ?line ok = io:format("R1 = ~w, R2 = ~w", [R1, R2]), - ?line test_server:fail({big_diff, Diff}); + ok = io:format("R1 = ~w, R2 = ~w", [R1, R2]), + test_server:fail({big_diff, Diff}); Diff -> io:format("~p\n", [Diff]), ok @@ -949,11 +935,11 @@ bump_reductions(Config) when is_list(Config) -> bump_big(R2, 16#08000000). bump_big(Prev, Limit) -> - ?line true = erlang:bump_reductions(100000), %Limited to CONTEXT_REDUCTIONS. - ?line case process_info(self(), reductions) of + true = erlang:bump_reductions(100000), %Limited to CONTEXT_REDUCTIONS. + case process_info(self(), reductions) of {reductions,Big} when is_integer(Big), Big > Limit -> - ?line erlang:garbage_collect(), - ?line io:format("~p\n", [Big]); + erlang:garbage_collect(), + io:format("~p\n", [Big]); {reductions,R} when is_integer(R), R > Prev -> bump_big(R, Limit) end, @@ -964,34 +950,34 @@ bump_big(Prev, Limit) -> low_prio(Config) when is_list(Config) -> case erlang:system_info(schedulers_online) of 1 -> - ?line ok = low_prio_test(Config); + ok = low_prio_test(Config); _ -> - ?line erlang:system_flag(multi_scheduling, block), - ?line ok = low_prio_test(Config), - ?line erlang:system_flag(multi_scheduling, unblock), - ?line {comment, + erlang:system_flag(multi_scheduling, block), + ok = low_prio_test(Config), + erlang:system_flag(multi_scheduling, unblock), + {comment, "Test not written for SMP runtime system. " "Multi scheduling blocked during test."} end. low_prio_test(Config) when is_list(Config) -> - ?line process_flag(trap_exit, true), - ?line S = spawn_link(?MODULE, prio_server, [0, 0]), - ?line PCs = spawn_prio_clients(S, erlang:system_info(schedulers_online)), - ?line timer:sleep(2000), - ?line lists:foreach(fun (P) -> exit(P, kill) end, PCs), - ?line S ! exit, - ?line receive {'EXIT', S, {A, B}} -> check_prio(A, B) end, + process_flag(trap_exit, true), + S = spawn_link(?MODULE, prio_server, [0, 0]), + PCs = spawn_prio_clients(S, erlang:system_info(schedulers_online)), + timer:sleep(2000), + lists:foreach(fun (P) -> exit(P, kill) end, PCs), + S ! exit, + receive {'EXIT', S, {A, B}} -> check_prio(A, B) end, ok. check_prio(A, B) -> - ?line Prop = A/B, - ?line ok = io:format("Low=~p, High=~p, Prop=~p\n", [A, B, Prop]), + Prop = A/B, + ok = io:format("Low=~p, High=~p, Prop=~p\n", [A, B, Prop]), %% It isn't 1/8, it's more like 0.3, but let's check that %% the low-prio processes get some little chance to run at all. - ?line true = (Prop < 1.0), - ?line true = (Prop > 1/32). + true = (Prop < 1.0), + true = (Prop > 1/32). prio_server(A, B) -> receive @@ -1051,25 +1037,25 @@ yield(Config) when is_list(Config) -> end. yield_test() -> - ?line erlang:garbage_collect(), - ?line receive after 1 -> ok end, % Clear reductions. - ?line SC = schedcnt(start), - ?line {reductions, R1} = process_info(self(), reductions), - ?line {ok, true} = call_yield(middle), - ?line true = call_yield(final), - ?line true = call_yield(), - ?line true = apply(erlang, yield, []), - ?line {reductions, R2} = process_info(self(), reductions), - ?line Schedcnt = schedcnt(stop, SC), - ?line case {R2-R1, Schedcnt} of - {Diff, 4} when Diff < 30 -> - ?line ok = io:format("R1 = ~w, R2 = ~w, Schedcnt = ~w", - [R1, R2, Schedcnt]); - {Diff, _} -> - ?line ok = io:format("R1 = ~w, R2 = ~w, Schedcnt = ~w", - [R1, R2, Schedcnt]), - ?line test_server:fail({measurement_error, Diff, Schedcnt}) - end. + erlang:garbage_collect(), + receive after 1 -> ok end, % Clear reductions. + SC = schedcnt(start), + {reductions, R1} = process_info(self(), reductions), + {ok, true} = call_yield(middle), + true = call_yield(final), + true = call_yield(), + true = apply(erlang, yield, []), + {reductions, R2} = process_info(self(), reductions), + Schedcnt = schedcnt(stop, SC), + case {R2-R1, Schedcnt} of + {Diff, 4} when Diff < 30 -> + ok = io:format("R1 = ~w, R2 = ~w, Schedcnt = ~w", + [R1, R2, Schedcnt]); + {Diff, _} -> + ok = io:format("R1 = ~w, R2 = ~w, Schedcnt = ~w", + [R1, R2, Schedcnt]), + test_server:fail({measurement_error, Diff, Schedcnt}) + end. call_yield() -> erlang:yield(). @@ -1108,61 +1094,61 @@ schedcnt(stop, {Ref, Pid}) when is_reference(Ref), is_pid(Pid) -> yield2(doc) -> []; yield2(suite) -> []; yield2(Config) when is_list(Config) -> - ?line Me = self(), - ?line Go = make_ref(), - ?line RedDiff = make_ref(), - ?line Done = make_ref(), - ?line P = spawn(fun () -> - receive Go -> ok end, - {reductions, R1} = process_info(self(), reductions), - {ok, true} = call_yield(middle), - true = call_yield(final), - true = call_yield(), - true = apply(erlang, yield, []), - {reductions, R2} = process_info(self(), reductions), - Me ! {RedDiff, R2 - R1}, - exit(Done) - end), - ?line erlang:yield(), - - ?line 1 = erlang:trace(P, true, [running, procs, {tracer, self()}]), - - ?line P ! Go, + Me = self(), + Go = make_ref(), + RedDiff = make_ref(), + Done = make_ref(), + P = spawn(fun () -> + receive Go -> ok end, + {reductions, R1} = process_info(self(), reductions), + {ok, true} = call_yield(middle), + true = call_yield(final), + true = call_yield(), + true = apply(erlang, yield, []), + {reductions, R2} = process_info(self(), reductions), + Me ! {RedDiff, R2 - R1}, + exit(Done) + end), + erlang:yield(), + + 1 = erlang:trace(P, true, [running, procs, {tracer, self()}]), + + P ! Go, %% receive Go -> ok end, - ?line {trace, P, in, _} = next_tmsg(P), + {trace, P, in, _} = next_tmsg(P), %% {ok, true} = call_yield(middle), - ?line {trace, P, out, _} = next_tmsg(P), - ?line {trace, P, in, _} = next_tmsg(P), + {trace, P, out, _} = next_tmsg(P), + {trace, P, in, _} = next_tmsg(P), %% true = call_yield(final), - ?line {trace, P, out, _} = next_tmsg(P), - ?line {trace, P, in, _} = next_tmsg(P), + {trace, P, out, _} = next_tmsg(P), + {trace, P, in, _} = next_tmsg(P), %% true = call_yield(), - ?line {trace, P, out, _} = next_tmsg(P), - ?line {trace, P, in, _} = next_tmsg(P), + {trace, P, out, _} = next_tmsg(P), + {trace, P, in, _} = next_tmsg(P), %% true = apply(erlang, yield, []), - ?line {trace, P, out, _} = next_tmsg(P), - ?line {trace, P, in, _} = next_tmsg(P), + {trace, P, out, _} = next_tmsg(P), + {trace, P, in, _} = next_tmsg(P), %% exit(Done) - ?line {trace, P, exit, Done} = next_tmsg(P), + {trace, P, exit, Done} = next_tmsg(P), - ?line receive + receive {RedDiff, Reductions} when Reductions < 30, Reductions > 0 -> io:format("Reductions = ~p~n", [Reductions]), - ?line ok; + ok; {RedDiff, Reductions} -> - ?line ?t:fail({unexpected_reduction_count, Reductions}) + ?t:fail({unexpected_reduction_count, Reductions}) end, - ?line none = next_tmsg(P), + none = next_tmsg(P), - ?line ok. + ok. next_tmsg(Pid) -> receive @@ -1178,19 +1164,19 @@ next_tmsg(Pid) -> bad_register(Config) when is_list(Config) -> Name = a_long_and_unused_name, - ?line {'EXIT',{badarg,_}} = (catch register({bad,name}, self())), - ?line fail_register(undefined, self()), - ?line fail_register([bad,name], self()), + {'EXIT',{badarg,_}} = (catch register({bad,name}, self())), + fail_register(undefined, self()), + fail_register([bad,name], self()), - ?line {Dead,Mref} = spawn_monitor(fun() -> true end), + {Dead,Mref} = spawn_monitor(fun() -> true end), receive {'DOWN',Mref,process,Dead,_} -> ok end, - ?line fail_register(Name, Dead), - ?line fail_register(Name, make_ref()), - ?line fail_register(Name, []), - ?line fail_register(Name, {bad,process}), - ?line fail_register(Name, <<>>), + fail_register(Name, Dead), + fail_register(Name, make_ref()), + fail_register(Name, []), + fail_register(Name, {bad,process}), + fail_register(Name, <<>>), ok. fail_register(Name, Process) -> @@ -1201,50 +1187,50 @@ fail_register(Name, Process) -> garbage_collect(doc) -> []; garbage_collect(suite) -> []; garbage_collect(Config) when is_list(Config) -> - ?line Prio = process_flag(priority, high), - ?line true = erlang:garbage_collect(), - ?line TokLoopers = lists:map(fun (_) -> - spawn_opt(fun tok_loop/0, - [{priority, low}, link]) - end, - lists:seq(1, 10)), - ?line lists:foreach(fun (Pid) -> - ?line Mon = erlang:monitor(process, Pid), - ?line DownBefore = receive - {'DOWN', Mon, _, _, _} -> - ?line true - after 0 -> - ?line false - end, - ?line GC = erlang:garbage_collect(Pid), - ?line DownAfter = receive - {'DOWN', Mon, _, _, _} -> - ?line true - after 0 -> - ?line false - end, - ?line true = erlang:demonitor(Mon), - ?line case {DownBefore, DownAfter} of - {true, _} -> ?line false = GC; - {false, false} -> ?line true = GC; - _ -> ?line GC - end - end, - processes()), - ?line lists:foreach(fun (Pid) -> - unlink(Pid), - exit(Pid, bang) - end, TokLoopers), - ?line process_flag(priority, Prio), - ?line ok. + Prio = process_flag(priority, high), + true = erlang:garbage_collect(), + + TokLoopers = lists:map(fun (_) -> + spawn_opt(fun tok_loop/0, [{priority, low}, link]) + end, lists:seq(1, 10)), + + lists:foreach(fun (Pid) -> + Mon = erlang:monitor(process, Pid), + DownBefore = receive + {'DOWN', Mon, _, _, _} -> + true + after 0 -> + false + end, + GC = erlang:garbage_collect(Pid), + DownAfter = receive + {'DOWN', Mon, _, _, _} -> + true + after 0 -> + false + end, + true = erlang:demonitor(Mon), + case {DownBefore, DownAfter} of + {true, _} -> false = GC; + {false, false} -> true = GC; + _ -> GC + end + end, processes()), + + lists:foreach(fun (Pid) -> + unlink(Pid), + exit(Pid, bang) + end, TokLoopers), + process_flag(priority, Prio), + ok. process_info_messages(doc) -> ["This used to cause the nofrag emulator to dump core"]; process_info_messages(suite) -> []; process_info_messages(Config) when is_list(Config) -> - ?line process_info_messages_test(), - ?line ok. + process_info_messages_test(), + ok. process_info_messages_loop(0) -> ok; process_info_messages_loop(N) -> process_info_messages_loop(N-1). @@ -1259,43 +1245,42 @@ process_info_messages_send_my_msgs_to(Rcvr) -> end. process_info_messages_test() -> - ?line Go = make_ref(), - ?line Done = make_ref(), - ?line Rcvr = self(), - ?line Rcvr2 = spawn_link(fun () -> - receive {Go, Rcvr} -> ok end, - garbage_collect(), - Rcvr ! {Done, self()} - end), - ?line Sndrs = lists:map( - fun (_) -> - spawn_link(fun () -> - Rcvr ! {Go, self()}, - receive {Go, Rcvr} -> ok end, - BigData = lists:seq(1, 1000), - Rcvr ! BigData, - Rcvr ! BigData, - Rcvr ! BigData, - Rcvr ! {Done, self()} - end) - end, - lists:seq(1, 10)), - ?line lists:foreach(fun (Sndr) -> receive {Go, Sndr} -> ok end end, + Go = make_ref(), + Done = make_ref(), + Rcvr = self(), + Rcvr2 = spawn_link(fun () -> + receive {Go, Rcvr} -> ok end, + garbage_collect(), + Rcvr ! {Done, self()} + end), + Sndrs = lists:map( + fun (_) -> + spawn_link(fun () -> + Rcvr ! {Go, self()}, + receive {Go, Rcvr} -> ok end, + BigData = lists:seq(1, 1000), + Rcvr ! BigData, + Rcvr ! BigData, + Rcvr ! BigData, + Rcvr ! {Done, self()} + end) + end, lists:seq(1, 10)), + lists:foreach(fun (Sndr) -> receive {Go, Sndr} -> ok end end, Sndrs), - ?line garbage_collect(), - ?line erlang:yield(), - ?line lists:foreach(fun (Sndr) -> Sndr ! {Go, self()} end, Sndrs), - ?line process_info_messages_loop(100000000), - ?line Msgs = process_info(self(), messages), - ?line lists:foreach(fun (Sndr) -> receive {Done, Sndr} -> ok end end, + garbage_collect(), + erlang:yield(), + lists:foreach(fun (Sndr) -> Sndr ! {Go, self()} end, Sndrs), + process_info_messages_loop(100000000), + Msgs = process_info(self(), messages), + lists:foreach(fun (Sndr) -> receive {Done, Sndr} -> ok end end, Sndrs), - ?line garbage_collect(), - ?line Rcvr2 ! Msgs, - ?line process_info_messages_send_my_msgs_to(Rcvr2), - ?line Rcvr2 ! {Go, self()}, - ?line garbage_collect(), - ?line receive {Done, Rcvr2} -> ok end, - ?line Msgs. + garbage_collect(), + Rcvr2 ! Msgs, + process_info_messages_send_my_msgs_to(Rcvr2), + Rcvr2 ! {Go, self()}, + garbage_collect(), + receive {Done, Rcvr2} -> ok end, + Msgs. chk_badarg(Fun) -> try Fun(), exit(no_badarg) catch error:badarg -> ok end. @@ -1305,76 +1290,72 @@ process_flag_badarg(doc) -> process_flag_badarg(suite) -> []; process_flag_badarg(Config) when is_list(Config) -> - ?line chk_badarg(fun () -> process_flag(gurka, banan) end), - ?line chk_badarg(fun () -> process_flag(trap_exit, gurka) end), - ?line chk_badarg(fun () -> process_flag(error_handler, 1) end), - ?line chk_badarg(fun () -> process_flag(min_heap_size, gurka) end), - ?line chk_badarg(fun () -> process_flag(min_bin_vheap_size, gurka) end), - ?line chk_badarg(fun () -> process_flag(min_bin_vheap_size, -1) end), - ?line chk_badarg(fun () -> process_flag(priority, 4711) end), - ?line chk_badarg(fun () -> process_flag(save_calls, hmmm) end), - ?line P= spawn_link(fun () -> receive die -> ok end end), - ?line chk_badarg(fun () -> process_flag(P, save_calls, hmmm) end), - ?line chk_badarg(fun () -> process_flag(gurka, save_calls, hmmm) end), - ?line P ! die, - ?line ok. + chk_badarg(fun () -> process_flag(gurka, banan) end), + chk_badarg(fun () -> process_flag(trap_exit, gurka) end), + chk_badarg(fun () -> process_flag(error_handler, 1) end), + chk_badarg(fun () -> process_flag(min_heap_size, gurka) end), + chk_badarg(fun () -> process_flag(min_bin_vheap_size, gurka) end), + chk_badarg(fun () -> process_flag(min_bin_vheap_size, -1) end), + chk_badarg(fun () -> process_flag(priority, 4711) end), + chk_badarg(fun () -> process_flag(save_calls, hmmm) end), + P= spawn_link(fun () -> receive die -> ok end end), + chk_badarg(fun () -> process_flag(P, save_calls, hmmm) end), + chk_badarg(fun () -> process_flag(gurka, save_calls, hmmm) end), + P ! die, + ok. -include_lib("stdlib/include/ms_transform.hrl"). otp_6237(doc) -> []; otp_6237(suite) -> []; otp_6237(Config) when is_list(Config) -> - ?line Slctrs = lists:map(fun (_) -> - spawn_link(fun () -> - otp_6237_select_loop() - end) - end, - lists:seq(1,5)), - ?line lists:foreach(fun (_) -> otp_6237_test() end, lists:seq(1, 100)), - ?line lists:foreach(fun (S) -> unlink(S),exit(S, kill) end, Slctrs), - ?line ok. + Slctrs = lists:map(fun (_) -> + spawn_link(fun () -> + otp_6237_select_loop() + end) + end, + lists:seq(1,5)), + lists:foreach(fun (_) -> otp_6237_test() end, lists:seq(1, 100)), + lists:foreach(fun (S) -> unlink(S),exit(S, kill) end, Slctrs), + ok. otp_6237_test() -> - ?line Parent = self(), - ?line Inited = make_ref(), - ?line Die = make_ref(), - ?line Pid = spawn_link(fun () -> - register(otp_6237,self()), - otp_6237 = ets:new(otp_6237, - [named_table, - ordered_set]), - ets:insert(otp_6237, - [{I,I} - || I <- lists:seq(1, 100)]), - %% Inserting a lot of bif timers - %% increase the possibility that - %% the test will fail when the - %% original cleanup order is used - lists:foreach( - fun (_) -> - erlang:send_after(1000000, - self(), - {a,b,c}) - end, - lists:seq(1,1000)), - Parent ! Inited, - receive Die -> bye end - end), - ?line receive - Inited -> ?line ok - end, - ?line Pid ! Die, + Parent = self(), + Inited = make_ref(), + Die = make_ref(), + Pid = spawn_link(fun () -> + register(otp_6237,self()), + otp_6237 = ets:new(otp_6237, + [named_table, + ordered_set]), + ets:insert(otp_6237, + [{I,I} + || I <- lists:seq(1, 100)]), + %% Inserting a lot of bif timers + %% increase the possibility that + %% the test will fail when the + %% original cleanup order is used + lists:foreach( fun (_) -> + erlang:send_after(1000000, self(), {a,b,c}) + end, lists:seq(1,1000)), + Parent ! Inited, + receive Die -> bye end + end), + receive + Inited -> ok + end, + Pid ! Die, otp_6237_whereis_loop(). otp_6237_whereis_loop() -> - ?line case whereis(otp_6237) of + case whereis(otp_6237) of undefined -> - ?line otp_6237 = ets:new(otp_6237, + otp_6237 = ets:new(otp_6237, [named_table,ordered_set]), - ?line ets:delete(otp_6237), - ?line ok; + ets:delete(otp_6237), + ok; _ -> - ?line otp_6237_whereis_loop() + otp_6237_whereis_loop() end. otp_6237_select_loop() -> @@ -1382,9 +1363,8 @@ otp_6237_select_loop() -> otp_6237_select_loop(). - -define(NoTestProcs, 10000). --record(processes_bif_info, {min_start_reds, +-record(ptab_list_bif_info, {min_start_reds, tab_chunks, tab_chunks_size, tab_indices_per_red, @@ -1399,89 +1379,86 @@ processes_large_tab(doc) -> processes_large_tab(suite) -> []; processes_large_tab(Config) when is_list(Config) -> - ?line enable_internal_state(), - ?line MaxDbgLvl = 20, - ?line MinProcTabSize = 2*(1 bsl 15), - ?line ProcTabSize0 = 1000000, - ?line ProcTabSize1 = case {erlang:system_info(schedulers_online), - erlang:system_info(logical_processors)} of - {Schdlrs, Cpus} when is_integer(Cpus), - Schdlrs =< Cpus -> - ProcTabSize0; - _ -> - ProcTabSize0 div 4 - end, - ?line ProcTabSize2 = case erlang:system_info(debug_compiled) of - true -> ProcTabSize1 - 500000; - false -> ProcTabSize1 - end, + enable_internal_state(), + MaxDbgLvl = 20, + MinProcTabSize = 2*(1 bsl 15), + ProcTabSize0 = 1000000, + ProcTabSize1 = case {erlang:system_info(schedulers_online), + erlang:system_info(logical_processors)} of + {Schdlrs, Cpus} when is_integer(Cpus), + Schdlrs =< Cpus -> + ProcTabSize0; + _ -> + ProcTabSize0 div 4 + end, + ProcTabSize2 = case erlang:system_info(debug_compiled) of + true -> ProcTabSize1 - 500000; + false -> ProcTabSize1 + end, %% With high debug levels this test takes so long time that %% the connection times out; therefore, shrink the test on %% high debug levels. - ?line DbgLvl = case erts_debug:get_internal_state(processes_bif_info) of - #processes_bif_info{debug_level = Lvl} when Lvl > MaxDbgLvl -> + DbgLvl = case erts_debug:get_internal_state(processes_bif_info) of + #ptab_list_bif_info{debug_level = Lvl} when Lvl > MaxDbgLvl -> 20; - #processes_bif_info{debug_level = Lvl} when Lvl < 0 -> - ?line ?t:fail({debug_level, Lvl}); - #processes_bif_info{debug_level = Lvl} -> + #ptab_list_bif_info{debug_level = Lvl} when Lvl < 0 -> + ?t:fail({debug_level, Lvl}); + #ptab_list_bif_info{debug_level = Lvl} -> Lvl end, - ?line ProcTabSize3 = ProcTabSize2 - (1300000 * DbgLvl div MaxDbgLvl), - ?line ProcTabSize = case ProcTabSize3 < MinProcTabSize of + ProcTabSize3 = ProcTabSize2 - (1300000 * DbgLvl div MaxDbgLvl), + ProcTabSize = case ProcTabSize3 < MinProcTabSize of true -> MinProcTabSize; false -> ProcTabSize3 end, - ?line {ok, LargeNode} = start_node(Config, + {ok, LargeNode} = start_node(Config, "+P " ++ integer_to_list(ProcTabSize)), - ?line Res = rpc:call(LargeNode, ?MODULE, processes_bif_test, []), - ?line case rpc:call(LargeNode, + Res = rpc:call(LargeNode, ?MODULE, processes_bif_test, []), + case rpc:call(LargeNode, erts_debug, get_internal_state, [processes_bif_info]) of - #processes_bif_info{tab_chunks = Chunks} when is_integer(Chunks), + #ptab_list_bif_info{tab_chunks = Chunks} when is_integer(Chunks), Chunks > 1 -> ok; PBInfo -> ?t:fail(PBInfo) end, - ?line stop_node(LargeNode), - ?line chk_processes_bif_test_res(Res). + stop_node(LargeNode), + chk_processes_bif_test_res(Res). processes_default_tab(doc) -> []; processes_default_tab(suite) -> []; processes_default_tab(Config) when is_list(Config) -> - ?line {ok, DefaultNode} = start_node(Config, ""), - ?line Res = rpc:call(DefaultNode, ?MODULE, processes_bif_test, []), - ?line stop_node(DefaultNode), - ?line chk_processes_bif_test_res(Res). + {ok, DefaultNode} = start_node(Config, ""), + Res = rpc:call(DefaultNode, ?MODULE, processes_bif_test, []), + stop_node(DefaultNode), + chk_processes_bif_test_res(Res). processes_small_tab(doc) -> []; processes_small_tab(suite) -> []; processes_small_tab(Config) when is_list(Config) -> - ?line {ok, SmallNode} = start_node(Config, "+P 500"), - ?line Res = rpc:call(SmallNode, ?MODULE, processes_bif_test, []), - ?line PBInfo = rpc:call(SmallNode, - erts_debug, - get_internal_state, - [processes_bif_info]), - ?line stop_node(SmallNode), - ?line 1 = PBInfo#processes_bif_info.tab_chunks, - ?line chk_processes_bif_test_res(Res). + {ok, SmallNode} = start_node(Config, "+P 1024"), + Res = rpc:call(SmallNode, ?MODULE, processes_bif_test, []), + PBInfo = rpc:call(SmallNode, erts_debug, get_internal_state, [processes_bif_info]), + stop_node(SmallNode), + true = PBInfo#ptab_list_bif_info.tab_chunks < 10, + chk_processes_bif_test_res(Res). processes_this_tab(doc) -> []; processes_this_tab(suite) -> []; processes_this_tab(Config) when is_list(Config) -> - ?line chk_processes_bif_test_res(processes_bif_test()). + chk_processes_bif_test_res(processes_bif_test()). chk_processes_bif_test_res(ok) -> ok; chk_processes_bif_test_res({comment, _} = Comment) -> Comment; chk_processes_bif_test_res(Failure) -> ?t:fail(Failure). -print_processes_bif_info(#processes_bif_info{min_start_reds = MinStartReds, +print_processes_bif_info(#ptab_list_bif_info{min_start_reds = MinStartReds, tab_chunks = TabChunks, tab_chunks_size = TabChunksSize, tab_indices_per_red = TabIndPerRed, @@ -1575,26 +1552,26 @@ do_processes(WantReds) -> processes(). processes_bif_test() -> - ?line Tester = self(), - ?line enable_internal_state(), - ?line PBInfo = erts_debug:get_internal_state(processes_bif_info), - ?line print_processes_bif_info(PBInfo), - ?line WantReds = PBInfo#processes_bif_info.min_start_reds + 10, - ?line WillTrap = case PBInfo of - #processes_bif_info{tab_chunks = 1} -> - false; - #processes_bif_info{tab_chunks = Chunks, - tab_chunks_size = ChunksSize, - tab_indices_per_red = IndiciesPerRed - } -> - Chunks*ChunksSize >= IndiciesPerRed*WantReds - end, - ?line Processes = fun () -> - erts_debug:set_internal_state(reds_left,WantReds), - processes() - end, + Tester = self(), + enable_internal_state(), + PBInfo = erts_debug:get_internal_state(processes_bif_info), + print_processes_bif_info(PBInfo), + WantReds = PBInfo#ptab_list_bif_info.min_start_reds + 10, + WillTrap = case PBInfo of + #ptab_list_bif_info{tab_chunks = Chunks} when Chunks < 10 -> + false; %% Skip for small tables + #ptab_list_bif_info{tab_chunks = Chunks, + tab_chunks_size = ChunksSize, + tab_indices_per_red = IndiciesPerRed + } -> + Chunks*ChunksSize >= IndiciesPerRed*WantReds + end, + Processes = fun () -> + erts_debug:set_internal_state(reds_left,WantReds), + processes() + end, - ?line ok = do_processes_bif_test(WantReds, WillTrap, Processes), + ok = do_processes_bif_test(WantReds, WillTrap, Processes), case WillTrap of false -> @@ -1602,8 +1579,8 @@ processes_bif_test() -> true -> %% Do it again with a process suspended while %% in the processes/0 bif. - ?line erlang:system_flag(multi_scheduling, block), - ?line Suspendee = spawn_link(fun () -> + erlang:system_flag(multi_scheduling, block), + Suspendee = spawn_link(fun () -> Tester ! {suspend_me, self()}, Tester ! {self(), done, @@ -1613,179 +1590,160 @@ processes_bif_test() -> ok end end), - ?line receive {suspend_me, Suspendee} -> ok end, - ?line erlang:suspend_process(Suspendee), - ?line erlang:system_flag(multi_scheduling, unblock), + receive {suspend_me, Suspendee} -> ok end, + erlang:suspend_process(Suspendee), + erlang:system_flag(multi_scheduling, unblock), - ?line [{status,suspended}, - {current_function,{erlang,processes_trap,2}}] - = process_info(Suspendee, [status, current_function]), + [{status,suspended},{current_function,{erlang,ptab_list_continue,2}}] = + process_info(Suspendee, [status, current_function]), - ?line ok = do_processes_bif_test(WantReds, WillTrap, Processes), + ok = do_processes_bif_test(WantReds, WillTrap, Processes), - ?line erlang:resume_process(Suspendee), - ?line receive {Suspendee, done, _} -> ok end, - ?line unlink(Suspendee), - ?line exit(Suspendee, bang) + erlang:resume_process(Suspendee), + receive {Suspendee, done, _} -> ok end, + unlink(Suspendee), + exit(Suspendee, bang) end, case get(processes_bif_testcase_comment) of - undefined -> ?line ok; - Comment -> ?line {comment, Comment} + undefined -> ok; + Comment -> {comment, Comment} end. do_processes_bif_test(WantReds, DieTest, Processes) -> - ?line Tester = self(), - ?line SpawnProcesses = fun (Prio) -> - spawn_opt(?MODULE, - do_processes, - [WantReds], - [link, {priority, Prio}]) - end, - ?line Cleaner = spawn_link(fun () -> - process_flag(trap_exit, true), - Tester ! {cleaner_alive, self()}, - processes_bif_cleaner() - end), - ?line receive {cleaner_alive, Cleaner} -> ok end, + Tester = self(), + SpawnProcesses = fun (Prio) -> + spawn_opt(?MODULE, do_processes, [WantReds], [link, {priority, Prio}]) + end, + Cleaner = spawn_link(fun () -> + process_flag(trap_exit, true), + Tester ! {cleaner_alive, self()}, + processes_bif_cleaner() + end), + receive {cleaner_alive, Cleaner} -> ok end, try - ?line DoIt = make_ref(), - ?line GetGoing = make_ref(), - ?line {NoTestProcs, TestProcs} = spawn_initial_hangarounds(Cleaner), - ?line ?t:format("Testing with ~p processes~n", [NoTestProcs]), - ?line SpawnHangAround = fun () -> - spawn(?MODULE, - hangaround, - [Cleaner, new_hangaround]) - end, - ?line Killer = spawn_opt(fun () -> - Splt = NoTestProcs div 10, - {TP1, TP23} = lists:split(Splt, - TestProcs), - {TP2, TP3} = lists:split(Splt, TP23), - erlang:system_flag(multi_scheduling, - block), - Tester ! DoIt, - receive GetGoing -> ok end, - erlang:system_flag(multi_scheduling, - unblock), - SpawnProcesses(high), - lists:foreach( - fun (P) -> - SpawnHangAround(), - exit(P, bang) - end, - TP1), - SpawnProcesses(high), - erlang:yield(), - lists:foreach( - fun (P) -> - SpawnHangAround(), - exit(P, bang) - end, - TP2), - SpawnProcesses(high), - lists:foreach( - fun (P) -> - SpawnHangAround(), - exit(P, bang) - end, - TP3) - end, - [{priority, high}, link]), - ?line receive DoIt -> ok end, - ?line process_flag(priority, low), - ?line SpawnProcesses(low), - ?line erlang:yield(), - ?line process_flag(priority, normal), - ?line CorrectProcs0 = erts_debug:get_internal_state(processes), - ?line Killer ! GetGoing, - ?line erts_debug:set_internal_state(reds_left, WantReds), - ?line Procs0 = processes(), - ?line Procs = lists:sort(Procs0), - ?line CorrectProcs = lists:sort(CorrectProcs0), - ?line LengthCorrectProcs = length(CorrectProcs), - ?line ?t:format("~p = length(CorrectProcs)~n", [LengthCorrectProcs]), - ?line true = LengthCorrectProcs > NoTestProcs, - ?line case CorrectProcs =:= Procs of - true -> - ?line ok; - false -> - ?line processes_unexpected_result(CorrectProcs, Procs) - end, - ?line unlink(Killer), - ?line exit(Killer, bang) + DoIt = make_ref(), + GetGoing = make_ref(), + {NoTestProcs, TestProcs} = spawn_initial_hangarounds(Cleaner), + ?t:format("Testing with ~p processes~n", [NoTestProcs]), + SpawnHangAround = fun () -> + spawn(?MODULE, hangaround, [Cleaner, new_hangaround]) + end, + Killer = spawn_opt(fun () -> + Splt = NoTestProcs div 10, + {TP1, TP23} = lists:split(Splt, TestProcs), + {TP2, TP3} = lists:split(Splt, TP23), + erlang:system_flag(multi_scheduling, block), + Tester ! DoIt, + receive GetGoing -> ok end, + erlang:system_flag(multi_scheduling, unblock), + SpawnProcesses(high), + lists:foreach( fun (P) -> + SpawnHangAround(), + exit(P, bang) + end, TP1), + SpawnProcesses(high), + erlang:yield(), + lists:foreach( fun (P) -> + SpawnHangAround(), + exit(P, bang) + end, TP2), + SpawnProcesses(high), + lists:foreach( + fun (P) -> + SpawnHangAround(), + exit(P, bang) + end, TP3) + end, [{priority, high}, link]), + receive DoIt -> ok end, + process_flag(priority, low), + SpawnProcesses(low), + erlang:yield(), + process_flag(priority, normal), + CorrectProcs0 = erts_debug:get_internal_state(processes), + Killer ! GetGoing, + erts_debug:set_internal_state(reds_left, WantReds), + Procs0 = processes(), + Procs = lists:sort(Procs0), + CorrectProcs = lists:sort(CorrectProcs0), + LengthCorrectProcs = length(CorrectProcs), + ?t:format("~p = length(CorrectProcs)~n", [LengthCorrectProcs]), + true = LengthCorrectProcs > NoTestProcs, + case CorrectProcs =:= Procs of + true -> + ok; + false -> + processes_unexpected_result(CorrectProcs, Procs) + end, + unlink(Killer), + exit(Killer, bang) after unlink(Cleaner), exit(Cleaner, kill), %% Wait for the system to recover to a normal state... wait_until_system_recover() end, - ?line do_processes_bif_die_test(DieTest, Processes), - ?line ok. + do_processes_bif_die_test(DieTest, Processes), + ok. do_processes_bif_die_test(false, _Processes) -> - ?line ?t:format("Skipping test killing process executing processes/0~n",[]), - ?line ok; + ?t:format("Skipping test killing process executing processes/0~n",[]), + ok; do_processes_bif_die_test(true, Processes) -> - ?line do_processes_bif_die_test(5, Processes); + do_processes_bif_die_test(5, Processes); do_processes_bif_die_test(N, Processes) -> - ?line ?t:format("Doing test killing process executing processes/0~n",[]), + ?t:format("Doing test killing process executing processes/0~n",[]), try - ?line Tester = self(), - ?line Oooh_Nooooooo = make_ref(), - ?line {_, DieWhileDoingMon} = erlang:spawn_monitor( - fun () -> - Victim = self(), - spawn_opt( - fun () -> - exit(Victim, got_him) - end, - [link, - {priority, max}]), - Tester ! {Oooh_Nooooooo, - hd(Processes())}, - exit(ohhhh_nooooo) - end), - ?line receive - {'DOWN', DieWhileDoingMon, _, _, Reason} -> - case Reason of - got_him -> ok; - _ -> throw({kill_in_trap, Reason}) - end - end, - ?line receive - {Oooh_Nooooooo, _} -> - ?line throw({kill_in_trap, 'Oooh_Nooooooo'}) - after 0 -> - ?line ok - end, - ?line PrcsCllrsSeqLen = 2*erlang:system_info(schedulers_online), - ?line PrcsCllrsSeq = lists:seq(1, PrcsCllrsSeqLen), - ?line ProcsCallers = lists:map( - fun (_) -> - spawn_link( - fun () -> - Tester ! hd(Processes()) - end) - end, - PrcsCllrsSeq), - ?line erlang:yield(), + Tester = self(), + Oooh_Nooooooo = make_ref(), + {_, DieWhileDoingMon} = erlang:spawn_monitor( fun () -> + Victim = self(), + spawn_opt( + fun () -> + exit(Victim, got_him) + end, + [link, {priority, max}]), + Tester ! {Oooh_Nooooooo, + hd(Processes())}, + exit(ohhhh_nooooo) + end), + receive + {'DOWN', DieWhileDoingMon, _, _, Reason} -> + case Reason of + got_him -> ok; + _ -> throw({kill_in_trap, Reason}) + end + end, + receive + {Oooh_Nooooooo, _} -> + throw({kill_in_trap, 'Oooh_Nooooooo'}) + after 0 -> + ok + end, + PrcsCllrsSeqLen = 2*erlang:system_info(schedulers_online), + PrcsCllrsSeq = lists:seq(1, PrcsCllrsSeqLen), + ProcsCallers = lists:map( fun (_) -> + spawn_link( + fun () -> + Tester ! hd(Processes()) + end) + end, PrcsCllrsSeq), + erlang:yield(), {ProcsCallers1, ProcsCallers2} = lists:split(PrcsCllrsSeqLen div 2, ProcsCallers), - ?line process_flag(priority, high), - ?line lists:foreach( + process_flag(priority, high), + lists:foreach( fun (P) -> unlink(P), exit(P, bang) end, lists:reverse(ProcsCallers2) ++ ProcsCallers1), - ?line process_flag(priority, normal), - ?line ok + process_flag(priority, normal), + ok catch throw:{kill_in_trap, R} when N > 0 -> ?t:format("Failed to kill in trap: ~p~n", [R]), - ?t:format("Trying again~p~n", []), + ?t:format("Trying again~n", []), do_processes_bif_die_test(N-1, Processes) end. @@ -1844,23 +1802,23 @@ processes_last_call_trap(doc) -> processes_last_call_trap(suite) -> []; processes_last_call_trap(Config) when is_list(Config) -> - ?line enable_internal_state(), - ?line Processes = fun () -> processes() end, - ?line PBInfo = erts_debug:get_internal_state(processes_bif_info), - ?line print_processes_bif_info(PBInfo), - ?line WantReds = case PBInfo#processes_bif_info.min_start_reds of - R when R > 10 -> R - 1; - _R -> 9 - end, - ?line lists:foreach(fun (_) -> - ?line erts_debug:set_internal_state(reds_left, - WantReds), - Processes(), - ?line erts_debug:set_internal_state(reds_left, - WantReds), - my_processes() - end, - lists:seq(1,100)). + enable_internal_state(), + Processes = fun () -> processes() end, + PBInfo = erts_debug:get_internal_state(processes_bif_info), + print_processes_bif_info(PBInfo), + WantReds = case PBInfo#ptab_list_bif_info.min_start_reds of + R when R > 10 -> R - 1; + _R -> 9 + end, + lists:foreach(fun (_) -> + erts_debug:set_internal_state(reds_left, + WantReds), + Processes(), + erts_debug:set_internal_state(reds_left, + WantReds), + my_processes() + end, + lists:seq(1,100)). my_processes() -> processes(). @@ -1870,108 +1828,106 @@ processes_apply_trap(doc) -> processes_apply_trap(suite) -> []; processes_apply_trap(Config) when is_list(Config) -> - ?line enable_internal_state(), - ?line PBInfo = erts_debug:get_internal_state(processes_bif_info), - ?line print_processes_bif_info(PBInfo), - ?line WantReds = case PBInfo#processes_bif_info.min_start_reds of - R when R > 10 -> R - 1; - _R -> 9 - end, - ?line lists:foreach(fun (_) -> - ?line erts_debug:set_internal_state(reds_left, - WantReds), - ?line apply(erlang, processes, []) - end, - lists:seq(1,100)). + enable_internal_state(), + PBInfo = erts_debug:get_internal_state(processes_bif_info), + print_processes_bif_info(PBInfo), + WantReds = case PBInfo#ptab_list_bif_info.min_start_reds of + R when R > 10 -> R - 1; + _R -> 9 + end, + lists:foreach(fun (_) -> + erts_debug:set_internal_state(reds_left, + WantReds), + apply(erlang, processes, []) + end, lists:seq(1,100)). processes_gc_trap(doc) -> []; processes_gc_trap(suite) -> []; processes_gc_trap(Config) when is_list(Config) -> - ?line Tester = self(), - ?line enable_internal_state(), - ?line PBInfo = erts_debug:get_internal_state(processes_bif_info), - ?line print_processes_bif_info(PBInfo), - ?line WantReds = PBInfo#processes_bif_info.min_start_reds + 10, - ?line Processes = fun () -> - erts_debug:set_internal_state(reds_left,WantReds), - processes() - end, + Tester = self(), + enable_internal_state(), + PBInfo = erts_debug:get_internal_state(processes_bif_info), + print_processes_bif_info(PBInfo), + WantReds = PBInfo#ptab_list_bif_info.min_start_reds + 10, + Processes = fun () -> + erts_debug:set_internal_state(reds_left,WantReds), + processes() + end, - ?line erlang:system_flag(multi_scheduling, block), - ?line Suspendee = spawn_link(fun () -> + erlang:system_flag(multi_scheduling, block), + Suspendee = spawn_link(fun () -> Tester ! {suspend_me, self()}, Tester ! {self(), done, hd(Processes())}, receive after infinity -> ok end end), - ?line receive {suspend_me, Suspendee} -> ok end, - ?line erlang:suspend_process(Suspendee), - ?line erlang:system_flag(multi_scheduling, unblock), + receive {suspend_me, Suspendee} -> ok end, + erlang:suspend_process(Suspendee), + erlang:system_flag(multi_scheduling, unblock), - ?line [{status,suspended}, {current_function,{erlang,processes_trap,2}}] + [{status,suspended}, {current_function,{erlang,ptab_list_continue,2}}] = process_info(Suspendee, [status, current_function]), - ?line erlang:garbage_collect(Suspendee), - ?line erlang:garbage_collect(Suspendee), + erlang:garbage_collect(Suspendee), + erlang:garbage_collect(Suspendee), - ?line erlang:resume_process(Suspendee), - ?line receive {Suspendee, done, _} -> ok end, - ?line erlang:garbage_collect(Suspendee), - ?line erlang:garbage_collect(Suspendee), + erlang:resume_process(Suspendee), + receive {Suspendee, done, _} -> ok end, + erlang:garbage_collect(Suspendee), + erlang:garbage_collect(Suspendee), - ?line unlink(Suspendee), - ?line exit(Suspendee, bang), - ?line ok. + unlink(Suspendee), + exit(Suspendee, bang), + ok. process_flag_heap_size(doc) -> []; process_flag_heap_size(suite) -> []; process_flag_heap_size(Config) when is_list(Config) -> - HSize = 2584, % must be gc fib number - VHSize = 317811, % must be gc fib number - ?line OldHmin = erlang:process_flag(min_heap_size, HSize), - ?line {min_heap_size, HSize} = erlang:process_info(self(), min_heap_size), - ?line OldVHmin = erlang:process_flag(min_bin_vheap_size, VHSize), - ?line {min_bin_vheap_size, VHSize} = erlang:process_info(self(), min_bin_vheap_size), - ?line HSize = erlang:process_flag(min_heap_size, OldHmin), - ?line VHSize = erlang:process_flag(min_bin_vheap_size, OldVHmin), - ?line ok. + HSize = 2586, % must be gc fib+ number + VHSize = 318187, % must be gc fib+ number + OldHmin = erlang:process_flag(min_heap_size, HSize), + {min_heap_size, HSize} = erlang:process_info(self(), min_heap_size), + OldVHmin = erlang:process_flag(min_bin_vheap_size, VHSize), + {min_bin_vheap_size, VHSize} = erlang:process_info(self(), min_bin_vheap_size), + HSize = erlang:process_flag(min_heap_size, OldHmin), + VHSize = erlang:process_flag(min_bin_vheap_size, OldVHmin), + ok. spawn_opt_heap_size(doc) -> []; spawn_opt_heap_size(suite) -> []; spawn_opt_heap_size(Config) when is_list(Config) -> - HSize = 987, % must be gc fib number - VHSize = 46368, % must be gc fib number - ?line Pid = spawn_opt(fun () -> receive stop -> ok end end, + HSize = 987, % must be gc fib+ number + VHSize = 46422, % must be gc fib+ number + Pid = spawn_opt(fun () -> receive stop -> ok end end, [{min_heap_size, HSize},{ min_bin_vheap_size, VHSize}]), - ?line {min_heap_size, HSize} = process_info(Pid, min_heap_size), - ?line {min_bin_vheap_size, VHSize} = process_info(Pid, min_bin_vheap_size), - ?line Pid ! stop, - ?line ok. + {min_heap_size, HSize} = process_info(Pid, min_heap_size), + {min_bin_vheap_size, VHSize} = process_info(Pid, min_bin_vheap_size), + Pid ! stop, + ok. processes_term_proc_list(doc) -> []; processes_term_proc_list(suite) -> []; processes_term_proc_list(Config) when is_list(Config) -> - ?line Tester = self(), - ?line as_expected = processes_term_proc_list_test(false), - ?line {ok, Node} = start_node(Config, "+Mis true"), - ?line RT = spawn_link(Node, - fun () -> - receive after 1000 -> ok end, - processes_term_proc_list_test(false), - Tester ! {it_worked, self()} - end), - ?line receive {it_worked, RT} -> ok end, - ?line stop_node(Node), - ?line ok. + Tester = self(), + as_expected = processes_term_proc_list_test(false), + {ok, Node} = start_node(Config, "+Mis true"), + RT = spawn_link(Node, fun () -> + receive after 1000 -> ok end, + processes_term_proc_list_test(false), + Tester ! {it_worked, self()} + end), + receive {it_worked, RT} -> ok end, + stop_node(Node), + ok. -define(CHK_TERM_PROC_LIST(MC, XB), chk_term_proc_list(?LINE, MC, XB)). @@ -1982,8 +1938,8 @@ chk_term_proc_list(Line, MustChk, ExpectBlks) -> not_enabled; {_, MS} -> {value, - {processes_term_proc_el, - DL}} = lists:keysearch(processes_term_proc_el, 1, MS), + {ptab_list_deleted_el, + DL}} = lists:keysearch(ptab_list_deleted_el, 1, MS), case lists:keysearch(blocks, 1, DL) of {value, {blocks, ExpectBlks, _, _}} -> ok; @@ -1997,35 +1953,34 @@ chk_term_proc_list(Line, MustChk, ExpectBlks) -> ok. processes_term_proc_list_test(MustChk) -> - ?line Tester = self(), - ?line enable_internal_state(), - ?line PBInfo = erts_debug:get_internal_state(processes_bif_info), - ?line print_processes_bif_info(PBInfo), - ?line WantReds = PBInfo#processes_bif_info.min_start_reds + 10, - ?line #processes_bif_info{tab_chunks = Chunks, - tab_chunks_size = ChunksSize, - tab_indices_per_red = IndiciesPerRed - } = PBInfo, - ?line true = Chunks > 1, - ?line true = Chunks*ChunksSize >= IndiciesPerRed*WantReds, - ?line Processes = fun () -> - erts_debug:set_internal_state(reds_left, - WantReds), - processes() - end, - ?line Exit = fun (P) -> - unlink(P), - exit(P, bang), - wait_until( - fun () -> - not lists:member( - P, - erts_debug:get_internal_state( - processes)) - end) - end, - ?line SpawnSuspendProcessesProc - = fun () -> + Tester = self(), + enable_internal_state(), + PBInfo = erts_debug:get_internal_state(processes_bif_info), + print_processes_bif_info(PBInfo), + WantReds = PBInfo#ptab_list_bif_info.min_start_reds + 10, + #ptab_list_bif_info{tab_chunks = Chunks, + tab_chunks_size = ChunksSize, + tab_indices_per_red = IndiciesPerRed + } = PBInfo, + true = Chunks > 1, + true = Chunks*ChunksSize >= IndiciesPerRed*WantReds, + Processes = fun () -> + erts_debug:set_internal_state(reds_left, + WantReds), + processes() + end, + Exit = fun (P) -> + unlink(P), + exit(P, bang), + wait_until( + fun () -> + not lists:member( + P, + erts_debug:get_internal_state( + processes)) + end) + end, + SpawnSuspendProcessesProc = fun () -> erlang:system_flag(multi_scheduling, block), P = spawn_link(fun () -> Tester ! {suspend_me, self()}, @@ -2038,76 +1993,76 @@ processes_term_proc_list_test(MustChk) -> erlang:suspend_process(P), erlang:system_flag(multi_scheduling, unblock), [{status,suspended}, - {current_function,{erlang,processes_trap,2}}] + {current_function,{erlang,ptab_list_continue,2}}] = process_info(P, [status, current_function]), P end, - ?line ResumeProcessesProc = fun (P) -> + ResumeProcessesProc = fun (P) -> erlang:resume_process(P), receive {P, done, _} -> ok end end, - ?line ?CHK_TERM_PROC_LIST(MustChk, 0), - ?line HangAround = fun () -> receive after infinity -> ok end end, - ?line HA1 = spawn_link(HangAround), - ?line HA2 = spawn_link(HangAround), - ?line HA3 = spawn_link(HangAround), - ?line S1 = SpawnSuspendProcessesProc(), - ?line ?CHK_TERM_PROC_LIST(MustChk, 1), - ?line Exit(HA1), - ?line ?CHK_TERM_PROC_LIST(MustChk, 2), - ?line S2 = SpawnSuspendProcessesProc(), - ?line ?CHK_TERM_PROC_LIST(MustChk, 3), - ?line S3 = SpawnSuspendProcessesProc(), - ?line ?CHK_TERM_PROC_LIST(MustChk, 4), - ?line Exit(HA2), - ?line ?CHK_TERM_PROC_LIST(MustChk, 5), - ?line S4 = SpawnSuspendProcessesProc(), - ?line ?CHK_TERM_PROC_LIST(MustChk, 6), - ?line Exit(HA3), - ?line ?CHK_TERM_PROC_LIST(MustChk, 7), - ?line ResumeProcessesProc(S1), - ?line ?CHK_TERM_PROC_LIST(MustChk, 5), - ?line ResumeProcessesProc(S3), - ?line ?CHK_TERM_PROC_LIST(MustChk, 4), - ?line ResumeProcessesProc(S4), - ?line ?CHK_TERM_PROC_LIST(MustChk, 3), - ?line ResumeProcessesProc(S2), - ?line ?CHK_TERM_PROC_LIST(MustChk, 0), - ?line Exit(S1), - ?line Exit(S2), - ?line Exit(S3), - ?line Exit(S4), - - - ?line HA4 = spawn_link(HangAround), - ?line HA5 = spawn_link(HangAround), - ?line HA6 = spawn_link(HangAround), - ?line S5 = SpawnSuspendProcessesProc(), - ?line ?CHK_TERM_PROC_LIST(MustChk, 1), - ?line Exit(HA4), - ?line ?CHK_TERM_PROC_LIST(MustChk, 2), - ?line S6 = SpawnSuspendProcessesProc(), - ?line ?CHK_TERM_PROC_LIST(MustChk, 3), - ?line Exit(HA5), - ?line ?CHK_TERM_PROC_LIST(MustChk, 4), - ?line S7 = SpawnSuspendProcessesProc(), - ?line ?CHK_TERM_PROC_LIST(MustChk, 5), - ?line Exit(HA6), - ?line ?CHK_TERM_PROC_LIST(MustChk, 6), - ?line S8 = SpawnSuspendProcessesProc(), - ?line ?CHK_TERM_PROC_LIST(MustChk, 7), - - ?line erlang:system_flag(multi_scheduling, block), - ?line Exit(S8), - ?line ?CHK_TERM_PROC_LIST(MustChk, 7), - ?line Exit(S5), - ?line ?CHK_TERM_PROC_LIST(MustChk, 6), - ?line Exit(S7), - ?line ?CHK_TERM_PROC_LIST(MustChk, 6), - ?line Exit(S6), - ?line ?CHK_TERM_PROC_LIST(MustChk, 0), - ?line erlang:system_flag(multi_scheduling, unblock), - ?line as_expected. + ?CHK_TERM_PROC_LIST(MustChk, 0), + HangAround = fun () -> receive after infinity -> ok end end, + HA1 = spawn_link(HangAround), + HA2 = spawn_link(HangAround), + HA3 = spawn_link(HangAround), + S1 = SpawnSuspendProcessesProc(), + ?CHK_TERM_PROC_LIST(MustChk, 1), + Exit(HA1), + ?CHK_TERM_PROC_LIST(MustChk, 2), + S2 = SpawnSuspendProcessesProc(), + ?CHK_TERM_PROC_LIST(MustChk, 3), + S3 = SpawnSuspendProcessesProc(), + ?CHK_TERM_PROC_LIST(MustChk, 4), + Exit(HA2), + ?CHK_TERM_PROC_LIST(MustChk, 5), + S4 = SpawnSuspendProcessesProc(), + ?CHK_TERM_PROC_LIST(MustChk, 6), + Exit(HA3), + ?CHK_TERM_PROC_LIST(MustChk, 7), + ResumeProcessesProc(S1), + ?CHK_TERM_PROC_LIST(MustChk, 5), + ResumeProcessesProc(S3), + ?CHK_TERM_PROC_LIST(MustChk, 4), + ResumeProcessesProc(S4), + ?CHK_TERM_PROC_LIST(MustChk, 3), + ResumeProcessesProc(S2), + ?CHK_TERM_PROC_LIST(MustChk, 0), + Exit(S1), + Exit(S2), + Exit(S3), + Exit(S4), + + + HA4 = spawn_link(HangAround), + HA5 = spawn_link(HangAround), + HA6 = spawn_link(HangAround), + S5 = SpawnSuspendProcessesProc(), + ?CHK_TERM_PROC_LIST(MustChk, 1), + Exit(HA4), + ?CHK_TERM_PROC_LIST(MustChk, 2), + S6 = SpawnSuspendProcessesProc(), + ?CHK_TERM_PROC_LIST(MustChk, 3), + Exit(HA5), + ?CHK_TERM_PROC_LIST(MustChk, 4), + S7 = SpawnSuspendProcessesProc(), + ?CHK_TERM_PROC_LIST(MustChk, 5), + Exit(HA6), + ?CHK_TERM_PROC_LIST(MustChk, 6), + S8 = SpawnSuspendProcessesProc(), + ?CHK_TERM_PROC_LIST(MustChk, 7), + + erlang:system_flag(multi_scheduling, block), + Exit(S8), + ?CHK_TERM_PROC_LIST(MustChk, 7), + Exit(S5), + ?CHK_TERM_PROC_LIST(MustChk, 6), + Exit(S7), + ?CHK_TERM_PROC_LIST(MustChk, 6), + Exit(S6), + ?CHK_TERM_PROC_LIST(MustChk, 0), + erlang:system_flag(multi_scheduling, unblock), + as_expected. otp_7738_waiting(doc) -> @@ -2115,88 +2070,88 @@ otp_7738_waiting(doc) -> otp_7738_waiting(suite) -> []; otp_7738_waiting(Config) when is_list(Config) -> - ?line otp_7738_test(waiting). + otp_7738_test(waiting). otp_7738_suspended(doc) -> []; otp_7738_suspended(suite) -> []; otp_7738_suspended(Config) when is_list(Config) -> - ?line otp_7738_test(suspended). + otp_7738_test(suspended). otp_7738_resume(doc) -> []; otp_7738_resume(suite) -> []; otp_7738_resume(Config) when is_list(Config) -> - ?line otp_7738_test(resume). + otp_7738_test(resume). otp_7738_test(Type) -> - ?line T = self(), - ?line S = spawn_link(fun () -> - receive - {suspend, Suspendee} -> - erlang:suspend_process(Suspendee), - T ! {suspended, Suspendee}, - receive - after 10 -> - erlang:resume_process(Suspendee), - Suspendee ! wake_up - end; - {send, To, Msg} -> - receive after 10 -> ok end, - To ! Msg - end - end), - ?line R = spawn_link(fun () -> - X = lists:seq(1, 20000000), - T ! {initialized, self()}, - ?line case Type of - _ when Type == suspended; - Type == waiting -> - receive _ -> ok end; - _ when Type == resume -> - Receive = fun (F) -> - receive - _ -> - ok - after 0 -> - F(F) - end - end, - Receive(Receive) - end, - T ! {woke_up, self()}, - id(X) - end), - ?line receive {initialized, R} -> ok end, - ?line receive after 10 -> ok end, - ?line case Type of + T = self(), + S = spawn_link(fun () -> + receive + {suspend, Suspendee} -> + erlang:suspend_process(Suspendee), + T ! {suspended, Suspendee}, + receive + after 10 -> + erlang:resume_process(Suspendee), + Suspendee ! wake_up + end; + {send, To, Msg} -> + receive after 10 -> ok end, + To ! Msg + end + end), + R = spawn_link(fun () -> + X = lists:seq(1, 20000000), + T ! {initialized, self()}, + case Type of + _ when Type == suspended; + Type == waiting -> + receive _ -> ok end; + _ when Type == resume -> + Receive = fun (F) -> + receive + _ -> + ok + after 0 -> + F(F) + end + end, + Receive(Receive) + end, + T ! {woke_up, self()}, + id(X) + end), + receive {initialized, R} -> ok end, + receive after 10 -> ok end, + case Type of suspended -> - ?line erlang:suspend_process(R), - ?line S ! {send, R, wake_up}; + erlang:suspend_process(R), + S ! {send, R, wake_up}; waiting -> - ?line S ! {send, R, wake_up}; + S ! {send, R, wake_up}; resume -> - ?line S ! {suspend, R}, - ?line receive {suspended, R} -> ok end + S ! {suspend, R}, + receive {suspended, R} -> ok end end, - ?line erlang:garbage_collect(R), - ?line case Type of + erlang:garbage_collect(R), + case Type of suspended -> - ?line erlang:resume_process(R); + erlang:resume_process(R); _ -> - ?line ok + ok end, - ?line receive + receive {woke_up, R} -> - ?line ok + ok after 2000 -> - ?line I = process_info(R, [status, message_queue_len]), - ?line ?t:format("~p~n", [I]), - ?line ?t:fail(no_progress) + I = process_info(R, [status, message_queue_len]), + ?t:format("~p~n", [I]), + ?t:fail(no_progress) end, - ?line ok. + ok. gor(Reds, Stop) -> receive @@ -2210,28 +2165,28 @@ gor(Reds, Stop) -> end. garb_other_running(Config) when is_list(Config) -> - ?line Stop = make_ref(), - ?line {Pid, Mon} = spawn_monitor(fun () -> gor(0, Stop) end), - ?line Reds = lists:foldl(fun (_, OldReds) -> - ?line erlang:garbage_collect(Pid), - ?line receive after 1 -> ok end, - ?line Pid ! {self(), reds}, - ?line receive + Stop = make_ref(), + {Pid, Mon} = spawn_monitor(fun () -> gor(0, Stop) end), + Reds = lists:foldl(fun (_, OldReds) -> + erlang:garbage_collect(Pid), + receive after 1 -> ok end, + Pid ! {self(), reds}, + receive {reds, NewReds, Pid} -> - ?line true = (NewReds > OldReds), - ?line NewReds + true = (NewReds > OldReds), + NewReds end end, 0, lists:seq(1, 10000)), - ?line receive after 1 -> ok end, - ?line Pid ! {self(), Stop}, - ?line receive + receive after 1 -> ok end, + Pid ! {self(), Stop}, + receive {stopped, Stop, StopReds, Pid} -> - ?line true = (StopReds > Reds) + true = (StopReds > Reds) end, - ?line receive {'DOWN', Mon, process, Pid, normal} -> ok end, - ?line ok. + receive {'DOWN', Mon, process, Pid, normal} -> ok end, + ok. %% Internal functions @@ -2255,9 +2210,9 @@ start_node(Config) -> start_node(Config, ""). start_node(Config, Args) when is_list(Config) -> - ?line Pa = filename:dirname(code:which(?MODULE)), - ?line {A, B, C} = now(), - ?line Name = list_to_atom(atom_to_list(?MODULE) + Pa = filename:dirname(code:which(?MODULE)), + {A, B, C} = now(), + Name = list_to_atom(atom_to_list(?MODULE) ++ "-" ++ atom_to_list(?config(testcase, Config)) ++ "-" @@ -2266,7 +2221,7 @@ start_node(Config, Args) when is_list(Config) -> ++ integer_to_list(B) ++ "-" ++ integer_to_list(C)), - ?line ?t:start_node(Name, slave, [{args, "-pa "++Pa++" "++Args}]). + ?t:start_node(Name, slave, [{args, "-pa "++Pa++" "++Args}]). stop_node(Node) -> ?t:stop_node(Node). diff --git a/erts/emulator/test/smoke_test_SUITE.erl b/erts/emulator/test/smoke_test_SUITE.erl index 71186d4942..6f5c2080c0 100644 --- a/erts/emulator/test/smoke_test_SUITE.erl +++ b/erts/emulator/test/smoke_test_SUITE.erl @@ -1,7 +1,7 @@ %% %% %CopyrightBegin% %% -%% Copyright Ericsson AB 2011. All Rights Reserved. +%% Copyright Ericsson AB 2011-2012. All Rights Reserved. %% %% The contents of this file are subject to the Erlang Public License, %% Version 1.1, (the "License"); you may not use this file except in diff --git a/erts/emulator/test/tuple_SUITE.erl b/erts/emulator/test/tuple_SUITE.erl index bfc3910742..a3b2764a5d 100644 --- a/erts/emulator/test/tuple_SUITE.erl +++ b/erts/emulator/test/tuple_SUITE.erl @@ -20,6 +20,7 @@ -export([all/0, suite/0,groups/0,init_per_suite/1, end_per_suite/1, init_per_group/2,end_per_group/2, t_size/1, t_tuple_size/1, t_element/1, t_setelement/1, + t_insert_element/1, t_delete_element/1, t_list_to_tuple/1, t_tuple_to_list/1, t_make_tuple_2/1, t_make_tuple_3/1, t_append_element/1, build_and_match/1, tuple_with_case/1, tuple_in_guard/1]). @@ -41,6 +42,7 @@ all() -> [build_and_match, t_size, t_tuple_size, t_list_to_tuple, t_tuple_to_list, t_element, t_setelement, t_make_tuple_2, t_make_tuple_3, t_append_element, + t_insert_element, t_delete_element, tuple_with_case, tuple_in_guard]. groups() -> @@ -60,40 +62,40 @@ end_per_group(_GroupName, Config) -> build_and_match(Config) when is_list(Config) -> - ?line {} = id({}), - ?line {1} = id({1}), - ?line {1, 2} = id({1, 2}), - ?line {1, 2, 3} = id({1, 2, 3}), - ?line {1, 2, 3, 4} = id({1, 2, 3, 4}), - ?line {1, 2, 3, 4, 5} = id({1, 2, 3, 4, 5}), - ?line {1, 2, 3, 4, 5, 6} = id({1, 2, 3, 4, 5, 6}), - ?line {1, 2, 3, 4, 5, 6} = id({1, 2, 3, 4, 5, 6}), - ?line {1, 2, 3, 4, 5, 6, 7} = id({1, 2, 3, 4, 5, 6, 7}), - ?line {1, 2, 3, 4, 5, 6, 7, 8} = id({1, 2, 3, 4, 5, 6, 7, 8}), + {} = id({}), + {1} = id({1}), + {1, 2} = id({1, 2}), + {1, 2, 3} = id({1, 2, 3}), + {1, 2, 3, 4} = id({1, 2, 3, 4}), + {1, 2, 3, 4, 5} = id({1, 2, 3, 4, 5}), + {1, 2, 3, 4, 5, 6} = id({1, 2, 3, 4, 5, 6}), + {1, 2, 3, 4, 5, 6} = id({1, 2, 3, 4, 5, 6}), + {1, 2, 3, 4, 5, 6, 7} = id({1, 2, 3, 4, 5, 6, 7}), + {1, 2, 3, 4, 5, 6, 7, 8} = id({1, 2, 3, 4, 5, 6, 7, 8}), ok. %% Tests size(Tuple). t_size(Config) when is_list(Config) -> - ?line 0 = size({}), - ?line 1 = size({a}), - ?line 1 = size({{a}}), - ?line 2 = size({{a}, {b}}), - ?line 3 = size({1, 2, 3}), + 0 = size({}), + 1 = size({a}), + 1 = size({{a}}), + 2 = size({{a}, {b}}), + 3 = size({1, 2, 3}), ok. t_tuple_size(Config) when is_list(Config) -> - ?line 0 = tuple_size(id({})), - ?line 1 = tuple_size(id({a})), - ?line 1 = tuple_size(id({{a}})), - ?line 2 = tuple_size(id({{a},{b}})), - ?line 3 = tuple_size(id({1,2,3})), + 0 = tuple_size(id({})), + 1 = tuple_size(id({a})), + 1 = tuple_size(id({{a}})), + 2 = tuple_size(id({{a},{b}})), + 3 = tuple_size(id({1,2,3})), %% Error cases. - ?line {'EXIT',{badarg,_}} = (catch tuple_size([])), - ?line {'EXIT',{badarg,_}} = (catch tuple_size(<<1,2,3>>)), - ?line error = ludicrous_tuple_size({a,b,c}), - ?line error = ludicrous_tuple_size([a,b,c]), + {'EXIT',{badarg,_}} = (catch tuple_size([])), + {'EXIT',{badarg,_}} = (catch tuple_size(<<1,2,3>>)), + error = ludicrous_tuple_size({a,b,c}), + error = ludicrous_tuple_size([a,b,c]), ok. @@ -104,44 +106,44 @@ ludicrous_tuple_size(_) -> error. %% Tests element/2. t_element(Config) when is_list(Config) -> - ?line a = element(1, {a}), - ?line a = element(1, {a, b}), + a = element(1, {a}), + a = element(1, {a, b}), - ?line List = lists:seq(1, 4096), - ?line Tuple = list_to_tuple(lists:seq(1, 4096)), - ?line get_elements(List, Tuple, 1), + List = lists:seq(1, 4096), + Tuple = list_to_tuple(lists:seq(1, 4096)), + get_elements(List, Tuple, 1), - ?line {'EXIT', {badarg, _}} = (catch element(0, id({a,b}))), - ?line {'EXIT', {badarg, _}} = (catch element(3, id({a,b}))), - ?line {'EXIT', {badarg, _}} = (catch element(1, id({}))), - ?line {'EXIT', {badarg, _}} = (catch element(1, id([a,b]))), - ?line {'EXIT', {badarg, _}} = (catch element(1, id(42))), - ?line {'EXIT', {badarg, _}} = (catch element(id(1.5), id({a,b}))), + {'EXIT', {badarg, _}} = (catch element(0, id({a,b}))), + {'EXIT', {badarg, _}} = (catch element(3, id({a,b}))), + {'EXIT', {badarg, _}} = (catch element(1, id({}))), + {'EXIT', {badarg, _}} = (catch element(1, id([a,b]))), + {'EXIT', {badarg, _}} = (catch element(1, id(42))), + {'EXIT', {badarg, _}} = (catch element(id(1.5), id({a,b}))), ok. get_elements([Element|Rest], Tuple, Pos) -> - ?line Element = element(Pos, Tuple), - ?line get_elements(Rest, Tuple, Pos+1); + Element = element(Pos, Tuple), + get_elements(Rest, Tuple, Pos+1); get_elements([], _Tuple, _Pos) -> ok. %% Tests set_element/3. t_setelement(Config) when is_list(Config) -> - ?line {x} = setelement(1, id({1}), x), - ?line {x,2} = setelement(1, id({1,2}), x), - ?line {1,x} = setelement(2, id({1,2}), x), + {x} = setelement(1, id({1}), x), + {x,2} = setelement(1, id({1,2}), x), + {1,x} = setelement(2, id({1,2}), x), - ?line Tuple = list_to_tuple(lists:duplicate(2048, x)), - ?line NewTuple = set_all_elements(Tuple, 1), - ?line NewTuple = list_to_tuple(lists:seq(1+7, 2048+7)), + Tuple = list_to_tuple(lists:duplicate(2048, x)), + NewTuple = set_all_elements(Tuple, 1), + NewTuple = list_to_tuple(lists:seq(1+7, 2048+7)), - ?line {'EXIT', {badarg, _}} = (catch setelement(0, {a, b}, x)), - ?line {'EXIT', {badarg, _}} = (catch setelement(3, {a, b}, x)), - ?line {'EXIT', {badarg, _}} = (catch setelement(1, {}, x)), - ?line {'EXIT', {badarg, _}} = (catch setelement(1, [a, b], x)), - ?line {'EXIT', {badarg, _}} = (catch setelement(1.5, {a, b}, x)), + {'EXIT', {badarg, _}} = (catch setelement(0, {a, b}, x)), + {'EXIT', {badarg, _}} = (catch setelement(3, {a, b}, x)), + {'EXIT', {badarg, _}} = (catch setelement(1, {}, x)), + {'EXIT', {badarg, _}} = (catch setelement(1, [a, b], x)), + {'EXIT', {badarg, _}} = (catch setelement(1.5, {a, b}, x)), %% Nested setelement with literals. AnotherTuple = id({0,0,a,b,c}), @@ -159,52 +161,68 @@ set_all_elements(Tuple, Pos) when Pos > size(Tuple) -> %% Tests list_to_tuple/1. t_list_to_tuple(Config) when is_list(Config) -> - ?line {} = list_to_tuple([]), - ?line {a} = list_to_tuple([a]), - ?line {a, b} = list_to_tuple([a, b]), - ?line {a, b, c} = list_to_tuple([a, b, c]), - ?line {a, b, c, d} = list_to_tuple([a, b, c, d]), - ?line {a, b, c, d, e} = list_to_tuple([a, b, c, d, e]), - - ?line Size = 4096, - ?line Tuple = list_to_tuple(lists:seq(1, Size)), - ?line Size = size(Tuple), - - ?line {'EXIT', {badarg, _}} = (catch list_to_tuple(id({a,b}))), - ?line {'EXIT', {badarg, _}} = (catch list_to_tuple(id([a|b]))), - ?line {'EXIT', {badarg, _}} = (catch list_to_tuple(id([a|b]))), - + {} = list_to_tuple([]), + {a} = list_to_tuple([a]), + {a, b} = list_to_tuple([a, b]), + {a, b, c} = list_to_tuple([a, b, c]), + {a, b, c, d} = list_to_tuple([a, b, c, d]), + {a, b, c, d, e} = list_to_tuple([a, b, c, d, e]), + + Size = 4096, + Tuple = list_to_tuple(lists:seq(1, Size)), + Size = size(Tuple), + + {'EXIT', {badarg, _}} = (catch list_to_tuple(id({a,b}))), + {'EXIT', {badarg, _}} = (catch list_to_tuple(id([a|b]))), + {'EXIT', {badarg, _}} = (catch list_to_tuple(id([a|b]))), + + % test upper boundry, 16777215 elements + MaxSize = 1 bsl 24 - 1, + MaxTuple = list_to_tuple(lists:seq(1, MaxSize)), + MaxSize = size(MaxTuple), + + {'EXIT', {badarg,_}} = (catch list_to_tuple(lists:seq(1, 1 bsl 24))), ok. %% Tests tuple_to_list/1. t_tuple_to_list(Config) when is_list(Config) -> - ?line [] = tuple_to_list({}), - ?line [a] = tuple_to_list({a}), - ?line [a, b] = tuple_to_list({a, b}), - ?line [a, b, c] = tuple_to_list({a, b, c}), - ?line [a, b, c, d] = tuple_to_list({a, b, c, d}), - ?line [a, b, c, d] = tuple_to_list({a, b, c, d}), - - ?line Size = 4096, - ?line List = lists:seq(1, Size), - ?line Tuple = list_to_tuple(List), - ?line Size = size(Tuple), - ?line List = tuple_to_list(Tuple), - - ?line {'EXIT', {badarg,_}} = (catch tuple_to_list(id(a))), - ?line {'EXIT', {badarg,_}} = (catch tuple_to_list(id(42))), + [] = tuple_to_list({}), + [a] = tuple_to_list({a}), + [a, b] = tuple_to_list({a, b}), + [a, b, c] = tuple_to_list({a, b, c}), + [a, b, c, d] = tuple_to_list({a, b, c, d}), + [a, b, c, d] = tuple_to_list({a, b, c, d}), + + Size = 4096, + List = lists:seq(1, Size), + Tuple = list_to_tuple(List), + Size = size(Tuple), + List = tuple_to_list(Tuple), + + {'EXIT', {badarg,_}} = (catch tuple_to_list(id(a))), + {'EXIT', {badarg,_}} = (catch tuple_to_list(id(42))), ok. %% Tests the make_tuple/2 BIF. t_make_tuple_2(Config) when is_list(Config) -> - ?line t_make_tuple1([]), - ?line t_make_tuple1(42), - ?line t_make_tuple1(a), - ?line t_make_tuple1({}), - ?line t_make_tuple1({a}), - ?line t_make_tuple1(erlang:make_tuple(400, [])), + t_make_tuple1([]), + t_make_tuple1(42), + t_make_tuple1(a), + t_make_tuple1({}), + t_make_tuple1({a}), + t_make_tuple1(erlang:make_tuple(400, [])), + + % test upper boundry, 16777215 elements + t_make_tuple(1 bsl 24 - 1, a), + {'EXIT', {badarg,_}} = (catch erlang:make_tuple(1 bsl 24, a)), + + {'EXIT', {badarg,_}} = (catch erlang:make_tuple(-1, a)), + % 26 bits is the total header arity room (for now) + {'EXIT', {badarg,_}} = (catch erlang:make_tuple(1 bsl 26 + 3, a)), + % bignum + {'EXIT', {badarg,_}} = (catch erlang:make_tuple(1 bsl 65 + 3, a)), ok. t_make_tuple1(Element) -> @@ -222,29 +240,82 @@ t_make_tuple(Size, Element) -> %% Tests the erlang:make_tuple/3 BIF. t_make_tuple_3(Config) when is_list(Config) -> - ?line {} = erlang:make_tuple(0, def, []), - ?line {def} = erlang:make_tuple(1, def, []), - ?line {a} = erlang:make_tuple(1, def, [{1,a}]), - ?line {a,def,c,def,e} = erlang:make_tuple(5, def, [{5,e},{1,a},{3,c}]), - ?line {a,def,c,def,e} = erlang:make_tuple(5, def, - [{1,blurf},{5,e},{3,blurf}, - {1,a},{3,c}]), + {} = erlang:make_tuple(0, def, []), + {def} = erlang:make_tuple(1, def, []), + {a} = erlang:make_tuple(1, def, [{1,a}]), + + {a,def,c,def,e} = erlang:make_tuple(5, def, [{5,e},{1,a},{3,c}]), + {a,def,c,def,e} = erlang:make_tuple(5, def, [{1,blurf},{5,e},{3,blurf},{1,a},{3,c}]), + MaxSize = 1 bsl 16 - 1, + MaxTuple = erlang:make_tuple(MaxSize, def, [{1,blurf},{5,e},{3,blurf},{1,a},{3,c}]), + MaxSize = size(MaxTuple), + + %% Error cases. + {'EXIT',{badarg,_}} = (catch erlang:make_tuple(0, def, [{1,a}])), + {'EXIT',{badarg,_}} = (catch erlang:make_tuple(5, def, [{-1,a}])), + {'EXIT',{badarg,_}} = (catch erlang:make_tuple(5, def, [{0,a}])), + {'EXIT',{badarg,_}} = (catch erlang:make_tuple(5, def, [{6,z}])), + {'EXIT',{badarg,_}} = (catch erlang:make_tuple(a, def, [{6,z}])), + {'EXIT',{badarg,_}} = (catch erlang:make_tuple(5, def, [{1,a}|b])), + {'EXIT',{badarg,_}} = (catch erlang:make_tuple(5, def, [42])), + {'EXIT',{badarg,_}} = (catch erlang:make_tuple(5, def, [[a,b,c]])), + {'EXIT',{badarg,_}} = (catch erlang:make_tuple(5, def, non_list)), + {'EXIT',{badarg,_}} = (catch erlang:make_tuple(1 bsl 24, def, [{5,e},{1,a},{3,c}])), + + ok. + +%% Tests the erlang:insert_element/3 BIF. +t_insert_element(Config) when is_list(Config) -> + {a} = erlang:insert_element(1, {}, a), + {{b,b},a} = erlang:insert_element(1, {a}, {b,b}), + {a,b} = erlang:insert_element(2, {a}, b), + [b,def|_] = tuple_to_list(erlang:insert_element(1, erlang:make_tuple(1 bsl 20, def), b)), + [def,b|_] = tuple_to_list(erlang:insert_element(2, erlang:make_tuple(1 bsl 20, def), b)), + [def,b|_] = lists:reverse(tuple_to_list(erlang:insert_element(1 bsl 20, erlang:make_tuple(1 bsl 20, def), b))), + [b,def|_] = lists:reverse(tuple_to_list(erlang:insert_element((1 bsl 20) + 1, erlang:make_tuple(1 bsl 20, def), b))), + + %% Error cases. + {'EXIT',{badarg,_}} = (catch erlang:insert_element(1, [], a)), + {'EXIT',{badarg,_}} = (catch erlang:insert_element(1, a, a)), + {'EXIT',{badarg,_}} = (catch erlang:insert_element(0, {}, a)), + {'EXIT',{badarg,_}} = (catch erlang:insert_element(0, {b,b,b,b,b}, a)), + {'EXIT',{badarg,_}} = (catch erlang:insert_element(-1, {}, a)), + {'EXIT',{badarg,_}} = (catch erlang:insert_element(2, {}, a)), + {'EXIT',{badarg,_}} = (catch erlang:insert_element(6, {b,b,b,b}, a)), + {'EXIT',{badarg,_}} = (catch erlang:insert_element(1 bsl 20, {b,b,b,b}, a)), + ok. + +%% Tests the erlang:delete_element/3 BIF. +t_delete_element(Config) when is_list(Config) -> + {} = erlang:delete_element(1, {a}), + {{b,b},c} = erlang:delete_element(1, {a,{b,b},c}), + {a,b} = erlang:delete_element(2, {a,c,b}), + [2,3|_] = tuple_to_list(erlang:delete_element(1, list_to_tuple(lists:seq(1, 1 bsl 20)))), + [1,3|_] = tuple_to_list(erlang:delete_element(2, list_to_tuple(lists:seq(1, 1 bsl 20)))), + [(1 bsl 20) - 1, (1 bsl 20) - 2 |_] = lists:reverse(tuple_to_list(erlang:delete_element(1 bsl 20, list_to_tuple(lists:seq(1, 1 bsl 20))))), + [(1 bsl 20), (1 bsl 20) - 2 |_] = lists:reverse(tuple_to_list(erlang:delete_element((1 bsl 20) - 1, list_to_tuple(lists:seq(1, 1 bsl 20))))), %% Error cases. - ?line {'EXIT',{badarg,_}} = (catch erlang:make_tuple(0, def, [{1,a}])), - ?line {'EXIT',{badarg,_}} = (catch erlang:make_tuple(5, def, [{-1,a}])), - ?line {'EXIT',{badarg,_}} = (catch erlang:make_tuple(5, def, [{0,a}])), - ?line {'EXIT',{badarg,_}} = (catch erlang:make_tuple(5, def, [{6,z}])), - ?line {'EXIT',{badarg,_}} = (catch erlang:make_tuple(a, def, [{6,z}])), - ?line {'EXIT',{badarg,_}} = (catch erlang:make_tuple(5, def, [{1,a}|b])), - ?line {'EXIT',{badarg,_}} = (catch erlang:make_tuple(5, def, [42])), - ?line {'EXIT',{badarg,_}} = (catch erlang:make_tuple(5, def, [[a,b,c]])), - ?line {'EXIT',{badarg,_}} = (catch erlang:make_tuple(5, def, non_list)), + {'EXIT',{badarg,_}} = (catch erlang:delete_element(1, [])), + {'EXIT',{badarg,_}} = (catch erlang:delete_element(1, a)), + {'EXIT',{badarg,_}} = (catch erlang:delete_element(0, {})), + {'EXIT',{badarg,_}} = (catch erlang:delete_element(-1, {})), + {'EXIT',{badarg,_}} = (catch erlang:delete_element(1, {})), + {'EXIT',{badarg,_}} = (catch erlang:delete_element(0, {b,b,b,b,b})), + {'EXIT',{badarg,_}} = (catch erlang:delete_element(5, {b,b,b,b})), + {'EXIT',{badarg,_}} = (catch erlang:delete_element(1 bsl 20, {b,c,b,b,b})), ok. + %% Tests the append_element/2 BIF. t_append_element(Config) when is_list(Config) -> - t_append_element({}, 2048, 2048). + ok = t_append_element({}, 2048, 2048), + + % test upper boundry, 16777215 elements + MaxSize = 1 bsl 24 - 1, + MaxTuple = list_to_tuple(lists:seq(1, MaxSize)), + {'EXIT',{badarg,_}} = (catch erlang:append_element(MaxTuple, a)), + ok. t_append_element(_Tuple, 0, _High) -> ok; t_append_element(Tuple, N, High) -> @@ -261,7 +332,7 @@ verify_seq([High|T], High, Lower) -> %% (This is known to crash earlier versions of BEAM.) tuple_with_case(Config) when is_list(Config) -> - ?line {reply, true} = tuple_with_case(), + {reply, true} = tuple_with_case(), ok. tuple_with_case() -> @@ -280,21 +351,21 @@ foo() -> ignored. %% Test to build a tuple in a guard. tuple_in_guard(Config) when is_list(Config) -> - ?line Tuple1 = id({a,b}), - ?line Tuple2 = id({a,b,c}), - ?line if - Tuple1 == {element(1, Tuple2),element(2, Tuple2)} -> - ok; - true -> - ?line test_server:fail() - end, - ?line if - Tuple2 == {element(1, Tuple2),element(2, Tuple2), - element(3, Tuple2)} -> - ok; - true -> - ?line test_server:fail() - end, + Tuple1 = id({a,b}), + Tuple2 = id({a,b,c}), + if + Tuple1 == {element(1, Tuple2),element(2, Tuple2)} -> + ok; + true -> + test_server:fail() + end, + if + Tuple2 == {element(1, Tuple2),element(2, Tuple2), + element(3, Tuple2)} -> + ok; + true -> + test_server:fail() + end, ok. %% Use this function to avoid compile-time evaluation of an expression. diff --git a/erts/etc/common/erlexec.c b/erts/etc/common/erlexec.c index 04d3425fe0..50c61f50eb 100644 --- a/erts/etc/common/erlexec.c +++ b/erts/etc/common/erlexec.c @@ -127,6 +127,7 @@ static char *pluss_val_switches[] = { "wt", "ws", "ss", + "pp", NULL }; /* +h arguments with values */ @@ -799,7 +800,9 @@ int main(int argc, char **argv) case 'A': case 'b': case 'i': + case 'n': case 'P': + case 'Q': case 'S': case 't': case 'T': @@ -1103,7 +1106,8 @@ usage_aux(void) "[-make] [-man [manopts] MANPAGE] [-x] [-emu_args] " "[-args_file FILENAME] [+A THREADS] [+a SIZE] [+B[c|d|i]] [+c] " "[+h HEAP_SIZE_OPTION] [+K BOOLEAN] " - "[+l] [+M<SUBSWITCH> <ARGUMENT>] [+P MAX_PROCS] [+R COMPAT_REL] " + "[+l] [+M<SUBSWITCH> <ARGUMENT>] [+P MAX_PROCS] [+Q MAX_PORTS] " + "[+R COMPAT_REL] " "[+r] [+rg READER_GROUPS_LIMIT] [+s SCHEDULER_OPTION] " "[+S NO_SCHEDULERS:NO_SCHEDULERS_ONLINE] [+T LEVEL] [+V] [+v] " "[+W<i|w>] [+z MISC_OPTION] [args ...]\n"); diff --git a/erts/etc/unix/cerl.src b/erts/etc/unix/cerl.src index 651452e531..cc7d77fd9a 100644 --- a/erts/etc/unix/cerl.src +++ b/erts/etc/unix/cerl.src @@ -304,10 +304,17 @@ else ;; esac - # Set annotation level for gdb in emacs 22 and higher. - emacs_major=`$EMACS --version | head -1 | sed 's,^[^0-9]*\([0-9]*\).*,\1,g'` - if [ '!' -z "$emacs_major" -a $emacs_major -gt 21 ]; then - GDBARGS="--annotate=1 " + if [ "$EMACS_ANNOTATE_LEVEL" != "" ]; then + GDBARGS="--annotate=$EMACS_ANNOTATE_LEVEL" + else + # Set annotation level for gdb in emacs 22 and higher. Seems to + # be working with level 1 for emacs 22 and level 3 for emacs 23... + emacs_major=`$EMACS --version | head -1 | sed 's,^[^0-9]*\([0-9]*\).*,\1,g'` + if [ '!' -z "$emacs_major" -a $emacs_major -gt 22 ]; then + GDBARGS="--annotate=3 " + elif [ '!' -z "$emacs_major" -a $emacs_major -gt 21 ]; then + GDBARGS="--annotate=1 " + fi fi gdbcmd="$gdbcmd $GDBBP \ (insert-string \"source $ROOTDIR/erts/etc/unix/etp-commands\") \ diff --git a/erts/etc/unix/etp-commands b/erts/etc/unix/etp-commands index d98505c0ff..f059662271 100644 --- a/erts/etc/unix/etp-commands +++ b/erts/etc/unix/etp-commands @@ -705,8 +705,6 @@ define etp-ct-name-1 end end - - define etp-pid-1 # Args: Eterm pid # @@ -714,9 +712,17 @@ define etp-pid-1 # set $etp_pid_1 = (Eterm)($arg0) if ($etp_pid_1 & 0xF) == 0x3 + if (etp_arch_bits == 64 && etp_halfword == 0) + if (etp_big_endian) + set $etp_pid_data = (unsigned) ((((Uint64) $etp_pid_1) >> 36) & 0x0fffffff) + else + set $etp_pid_data = (unsigned) ((((Uint64) $etp_pid_1) >> 4) & 0x0fffffff) + end + else + set $etp_pid_data = (unsigned) (((((Uint32) $etp_pid_1) >> 4) & ~erts_proc.r.o.pix_mask) | ((((Uint32) $etp_pid_1) >> (erts_proc.r.o.pix_cl_shift + 4)) & erts_proc.r.o.pix_cl_mask) | (((((Uint32) $etp_pid_1) >> 4) & erts_proc.r.o.pix_cli_mask) << erts_proc.r.o.pix_cli_shift)) + end # Internal pid - printf "<0.%u.%u>", (unsigned) ($etp_pid_1>>4)&0x7fff, \ - (unsigned) ($etp_pid_1>>19)&0x1fff + printf "<0.%u.%u>", $etp_pid_data & 0x7fff, ($etp_pid_data >> 15) & 0x1fff else printf "#NotPid<%#x>", ($arg0) end @@ -759,7 +765,6 @@ define etp-extpid-1 end - define etp-port-1 # Args: Eterm port # @@ -767,8 +772,17 @@ define etp-port-1 # set $etp_port_1 = (Eterm)($arg0) if ($etp_port_1 & 0xF) == 0x7 + if (etp_arch_bits == 64 && etp_halfword == 0) + if (etp_big_endian) + set $etp_port_data = (unsigned) ((((Uint64) $etp_port_1) >> 36) & 0x0fffffff) + else + set $etp_port_data = (unsigned) ((((Uint64) $etp_port_1) >> 4) & 0x0fffffff) + end + else + set $etp_port_data = (unsigned) (((((Uint32) $etp_port_1) >> 4) & ~erts_port.r.o.pix_mask) | ((((Uint32) $etp_port_1) >> (erts_port.r.o.pix_cl_shift + 4)) & erts_port.r.o.pix_cl_mask) | (((((Uint32) $etp_port_1) >> 4) & erts_port.r.o.pix_cli_mask) << erts_port.r.o.pix_cli_shift)) + end # Internal port - printf "#Port<0.%u>", (unsigned) ($etp_port_1>>4)&0x3ffff + printf "#Port<0.%u>", $etp_port_data else printf "#NotPort<%#x>", ($arg0) end @@ -1264,23 +1278,32 @@ document etpf-stackdump %--------------------------------------------------------------------------- end +define etp-pid2pix-1 +# Args: Eterm +# + if (etp_arch_bits == 64 && etp_halfword == 0) + if (etp_big_endian) + set $etp_pix = (int) (((Uint64) $arg0) & 0x0fffffff) + else + set $etp_pix = (int) ((((Uint64) $arg0) >> 32) & 0x0fffffff) + end + else + set $etp_pix = (int) ((((Uint32) $arg0) >> 4) & erts_proc.r.o.pix_mask) + end +end + define etp-pix2proc # Args: Eterm # - set $proc = (Process *) *((UWord *) &erts_proc.tab[((int) $arg0)]) + set $proc = (Process *) *((UWord *) &erts_proc.r.o.tab[((int) $arg0)]) printf "(Process *) %p\n", $proc end define etp-pid2proc-1 # Args: Eterm # - set $pix = (int) ($arg0 >> 4) - - if (erts_proc.pix_cl_mask != 0) - set $proc = (Process *) *((UWord *) &erts_proc.tab[(((($pix) & erts_proc.pix_cl_mask) << erts_proc.pix_cl_shift) + ((($pix) >> erts_proc.pix_cli_shift) & erts_proc.pix_cli_mask))]) - else - set $proc =(Process *) *((UWord *) &erts_proc.tab[((($pix) % erts_proc.max) % erts_proc.tab_cache_lines + (($pix) % erts_proc.max) / erts_proc.tab_cache_lines)]) - end + etp-pid2pix-1 $arg0 + set $proc = (Process *) *((UWord *) &erts_proc.r.o.tab[$etp_pix]) end define etp-pid2proc @@ -1369,13 +1392,15 @@ define etp-process-info # Args: Process* # printf " Pid: " - etp-1 $arg0->id + etp-1 $arg0->common.id printf "\n State: " etp-proc-state $arg0 - if ($arg0->reg) - printf " Registered name: " - etp-1 $arg0->reg->name - printf "\n" + if (*(((Uint32 *) &(((Process *) $arg0)->state))) & 0x4) == 0 + if ($arg0->common.u.alive.reg) + printf " Registered name: " + etp-1 $arg0->common.u.alive.reg->name + printf "\n" + end end if ($arg0->current) printf " Current function: " @@ -1400,7 +1425,7 @@ define etp-process-info end printf " Mbuf size: %ld\n", $arg0->mbuf_sz if (etp_smp_compiled) - printf " Msgq len: %ld (inner=%ld, outer=%ld)\n", ($arg0->msg.len + $arg0->u.alive.msg_inq.len), $arg0->msg.len, $arg0->u.alive.msg_inq.len + printf " Msgq len: %ld (inner=%ld, outer=%ld)\n", ($arg0->msg.len + $arg0->msg_inq.len), $arg0->msg.len, $arg0->msg_inq.len else printf " Msgq len: %d\n", $arg0->msg.len end @@ -1422,9 +1447,9 @@ define etp-processes printf "No processes, since system isn't initialized!\n" else set $proc_ix = 0 - while $proc_ix < erts_proc.max - set $proc = (Process *) *((UWord *) &erts_proc.tab[$proc_ix]) - if ($proc != (Process *) 0) + while $proc_ix < erts_proc.r.o.max + set $proc = (Process *) *((UWord *) &erts_proc.r.o.tab[$proc_ix]) + if ($proc != ((Process *) 0) && $proc != &erts_invalid_process) printf "---\n" printf " Pix: %d\n", $proc_ix etp-process-info $proc @@ -1443,8 +1468,108 @@ document etp-processes %--------------------------------------------------------------------------- end +define etp-port-id2pix-1 +# Args: Eterm +# + if (etp_arch_bits == 64 && etp_halfword == 0) + if (etp_big_endian) + set $etp_pix = (int) (((Uint64) $arg0) & 0x0fffffff) + elser + set $etp_pix = (int) ((((Uint64) $arg0) >> 32) & 0x0fffffff) + end + else + set $etp_pix = (int) ((((Uint32) $arg0) >> 4) & erts_port.r.o.pix_mask) + end +end + +define etp-pix2port +# Args: Eterm +# + set $port = (Port *) *((UWord *) &erts_port.r.o.tab[((int) $arg0)]) + printf "(Port *) %p\n", $port +end -define etp-port-status-int +define etp-id2port-1 +# Args: Eterm +# + etp-port-id2pix-1 $arg0 + set $port = (Port *) *((UWord *) &erts_port.r.o.tab[((int) $etp_pix)]) +end + +define etp-id2port +# Args: Eterm +# + etp-id2port-1 $arg0 + printf "(Port *) %p\n", $port +end + +define etp-port-sched-flags-int +# Args: int +# + if ($arg0 & 0x1) + printf " in-run-queue" + end + if ($arg0 & 0x2) + printf " executing" + end + if ($arg0 & 0x4) + printf " have-tasks" + end + if ($arg0 & 0x8) + printf " exited" + end + if ($arg0 & 0x10) + printf " busy-port" + end + if ($arg0 & 0x20) + printf " busy-port-q" + end + if ($arg0 & 0x40) + printf " chk-unset-busy-port-q" + end + if ($arg0 & 0x80) + printf " have-busy-tasks" + end + if ($arg0 & 0x100) + printf " have-nosuspend-tasks" + end + if ($arg0 & 0x200) + printf " parallelism" + end + if ($arg0 & 0x400) + printf " force-sched" + end + if ($arg0 & 0xfffff800) + printf " GARBAGE" + end + printf "\n" +end + +document etp-port-sched-flags-int +%--------------------------------------------------------------------------- +% etp-proc-sched-flags-int int +% +% Print port sched-flags +%--------------------------------------------------------------------------- +end + + +define etp-port-sched-flags +# Args: Port* +# + set $sched_flags_int = *(((Uint32 *) &(((Port *) $arg0)->sched.flags))) + etp-port-sched-flags-int $sched_flags_int +end + +document etp-port-sched-flags +%--------------------------------------------------------------------------- +% etp-proc-sched-flags-int Port * +% +% Print port sched-flags +%--------------------------------------------------------------------------- +end + +define etp-port-state-int # Args: int # if ($arg0 & 0x1) @@ -1463,68 +1588,62 @@ define etp-port-status-int printf " soft-eof" end if ($arg0 & 0x20) - printf " port-busy" + printf " closing" end if ($arg0 & 0x40) - printf " closing" + printf " send-closed" end if ($arg0 & 0x80) - printf " send-closed" + printf " linebuf-io" end if ($arg0 & 0x100) - printf " linebuf-io" + printf " free" end if ($arg0 & 0x200) - printf " immortal" + printf " initializing" end if ($arg0 & 0x400) - printf " free" + printf " port-specific-lock" end if ($arg0 & 0x800) - printf " free-scheduled" + printf " invalid" end if ($arg0 & 0x1000) - printf " initializing" - end - if ($arg0 & 0x2000) - printf " port-specific-lock" - end - if ($arg0 & 0x4000) - printf " invalid" + printf " halt" end if (etp_debug_compiled) - if ($arg0 & 0x7fff8000) + if ($arg0 & 0x7fffe000) printf " GARBAGE" end else - if ($arg0 & 0xffff8000) + if ($arg0 & 0xffffe000) printf " GARBAGE" end end printf "\n" end -document etp-port-status-int +document etp-port-state-int %--------------------------------------------------------------------------- % etp-proc-state-int int % -% Print port status +% Print port state %--------------------------------------------------------------------------- end -define etp-port-status +define etp-port-state # Args: Port* # - set $status_int = *(((Uint32 *) &(((Port *) $arg0)->status))) - etp-port-status-int $status_int + set $state_int = *(((Uint32 *) &(((Port *) $arg0)->state))) + etp-port-state-int $state_int end -document etp-port-status +document etp-port-state %--------------------------------------------------------------------------- % etp-proc-state-int Port * % -% Print port status +% Print port state %--------------------------------------------------------------------------- end @@ -1532,17 +1651,22 @@ define etp-port-info # Args: Port* # printf " Port: " - etp-1 $arg0->id + etp-1 $arg0->common.id printf "\n Name: %s\n", $arg0->name - printf " Status:" - etp-port-status $arg0 - if ($arg0->reg) - printf " Registered name: " - etp-1 $arg0->reg->name - printf "\n" + printf " State:" + etp-port-state $arg0 + printf " Scheduler flags:" + etp-port-sched-flags $arg0 + if (*(((Uint32 *) &(((Port *) $arg0)->state))) & 0x5C00) == 0 + if ($arg0->common.u.alive.reg) + printf " Registered name: " + etp-1 $arg0->common.u.alive.reg->name + printf "\n" + end end printf " Connected: " - etp-1 $arg0->connected + set $connected = *(((Eterm *) &(((Port *) $arg0)->connected))) + etp-1 $connected printf "\n Pointer: (Port *) %p\n", $arg0 end @@ -1560,13 +1684,15 @@ define etp-ports printf "No ports, since system isn't initialized!\n" else set $port_ix = 0 - while $port_ix < erts_max_ports - set $port = &erts_port[$port_ix] - if ($port->status & 0x400) == 0 - # I.e, not free - printf "---\n" - printf " Pix: %d\n", $port_ix - etp-port-info $port + while $port_ix < erts_port.r.o.max + set $port = (Port *) *((UWord *) &erts_port.r.o.tab[$port_ix]) + if ($port != ((Port *) 0) && $port != &erts_invalid_port) + if (*(((Uint32 *) &(((Port *) $port)->state))) & 0x100) == 0 + # I.e, not free + printf "---\n" + printf " Pix: %d\n", $port_ix + etp-port-info $port + end end set $port_ix++ end @@ -1866,6 +1992,12 @@ define etp-system-info printf "ERTS version: %s\n", etp_erts_version printf "Compile date: %s\n", etp_compile_date printf "Arch: %s\n", etp_arch + printf "Endianess: " + if (etp_big_endian) + printf "Big\n" + else + printf "Little\n" + end printf "Word size: %d-bit\n", etp_arch_bits printf "Halfword: " if (etp_halfword) @@ -2113,8 +2245,8 @@ define etp-search-heaps-1 end set $etp_search_heaps_i = 0 set $etp_search_heaps_found = 0 - while $etp_search_heaps_i < erts_proc.max - set $proc = (Process *) *((UWord *) &erts_proc.tab[$proc_ix]) + while $etp_search_heaps_i < erts_proc.r.o.max + set $proc = (Process *) *((UWord *) &erts_proc.r.o.tab[$proc_ix]) if $proc if ($proc->heap <= ($arg0)) && \ (($arg0) < $proc->hend) diff --git a/erts/etc/win32/nsis/Makefile b/erts/etc/win32/nsis/Makefile index a53ac708f8..7bcecaa264 100644 --- a/erts/etc/win32/nsis/Makefile +++ b/erts/etc/win32/nsis/Makefile @@ -62,9 +62,15 @@ else endif REDIST_FILE=$(shell (sh ./find_redist.sh || echo "")) +ifeq ($(MSYSTEM),MINGW32) + NICEREDISTFILE=$(shell (msys2win_path.sh -m "$(REDIST_FILE)" 2>/dev/null || echo "")) +else + NICEREDISTFILE=$(shell (cygpath -d -m "$(REDIST_FILE)" 2>/dev/null || echo "")) +endif + REDIST_TARGET=$(shell (sh ./find_redist.sh -n || echo "")) -REDIST_DLL_VERSION=$(shell (sh ./dll_version_helper.sh || echo "")) -REDIST_DLL_NAME=$(shell (sh ./dll_version_helper.sh -n || echo "")) +REDIST_DLL_VERSION=$(shell (sh ./dll_version_helper.sh $(NICEREDISTFILE) || echo "")) +REDIST_DLL_NAME=$(shell (sh ./dll_version_helper.sh -n $(NICEREDISTFILE) || echo "")) release_spec: @NSIS_VER=`makensis /hdrinfo | head -1 | awk '{print $$2}'`; \ diff --git a/erts/etc/win32/nsis/dll_version_helper.sh b/erts/etc/win32/nsis/dll_version_helper.sh index 96e4532b7a..0d9ba4248d 100755 --- a/erts/etc/win32/nsis/dll_version_helper.sh +++ b/erts/etc/win32/nsis/dll_version_helper.sh @@ -25,6 +25,13 @@ # echo "8.0.50727.763" # exit 0 +if [ "$1" = "-n" ]; then + SWITCH=$1 + shift +else + SWITCH="" +fi + cat > hello.c <<EOF #include <windows.h> #include <stdio.h> @@ -42,11 +49,16 @@ if [ '!' -f hello.exe.manifest ]; then # need another way of getting the version DLLNAME=`dumpbin.exe -imports hello.exe | egrep MSVCR.*dll` DLLNAME=`echo $DLLNAME` + if [ '!' -z "$1" ]; then + FILETOLOOKIN=$1 + else + FILETOLOOKIN=$DLLNAME + fi cat > helper.c <<EOF #include <windows.h> #include <stdio.h> -#define REQ_MODULE "$DLLNAME" +#define REQ_MODULE "$FILETOLOOKIN" int main(void) { @@ -100,7 +112,7 @@ else NAME=`grep '<assemblyIdentity' hello.exe.manifest | sed 's,.*name=.[A-Za-z\.]*\([0-9]*\).*,msvcr\1.dll,g' | grep -v '<'` fi #rm -f hello.c hello.obj hello.exe hello.exe.manifest helper.c helper.obj helper.exe helper.exe.manifest -if [ "$1" = "-n" ]; then +if [ "$SWITCH" = "-n" ]; then ASKEDFOR=$NAME else ASKEDFOR=$VERSION diff --git a/erts/etc/win32/nsis/erlang20.nsi b/erts/etc/win32/nsis/erlang20.nsi index c5ada9e3b3..3333c4a9aa 100644 --- a/erts/etc/win32/nsis/erlang20.nsi +++ b/erts/etc/win32/nsis/erlang20.nsi @@ -35,6 +35,7 @@ !include "MUI.nsh"
!include "WordFunc.nsh"
+ !include "WinVer.nsh"
;--------------------------------
;Configuration
@@ -341,6 +342,9 @@ FunctionEnd Function .onInit
Var /GLOBAL archprefix
Var /GLOBAL sysnativedir
+ Var /GLOBAL winvermajor
+ Var /GLOBAL winverminor
+
SectionGetFlags 0 $MYTEMP
StrCmpS ${WINTYPE} "win64" +1 +4
StrCpy $archprefix "amd64"
@@ -348,9 +352,16 @@ Function .onInit Goto +3
StrCpy $archprefix "x86"
StrCpy $sysnativedir $SYSDIR
- ;MessageBox MB_YESNO "Found $sysnativedir\${REDIST_DLL_NAME}" IDYES FoundLbl
+ ${WinVerGetMajor} $0
+ ${WinVerGetMinor} $1
+ StrCpy $winvermajor $0
+ StrCpy $winverminor $1
IfFileExists $sysnativedir\${REDIST_DLL_NAME} MaybeFoundInSystemLbl
SearchSxSLbl:
+ IntCmp $winvermajor 6 WVCheckMinorLbl WVCheckDoneLbl NotFoundLbl
+ WVCheckMinorLbl:
+ IntCmp $winverminor 1 WVCheckDoneLbl WVCheckDoneLbl NotFoundLbl
+ WVCheckDoneLbl:
FindFirst $0 $1 $WINDIR\WinSxS\$archprefix*
LoopLbl:
StrCmp $1 "" NotFoundLbl
diff --git a/erts/include/internal/ethread.h b/erts/include/internal/ethread.h index e1885c627a..aef31e282a 100644 --- a/erts/include/internal/ethread.h +++ b/erts/include/internal/ethread.h @@ -1,7 +1,7 @@ /* * %CopyrightBegin% * - * Copyright Ericsson AB 2004-2011. All Rights Reserved. + * Copyright Ericsson AB 2004-2012. All Rights Reserved. * * The contents of this file are subject to the Erlang Public License, * Version 1.1, (the "License"); you may not use this file except in diff --git a/erts/lib_src/common/erl_misc_utils.c b/erts/lib_src/common/erl_misc_utils.c index 3b123063fa..1b49f69581 100644 --- a/erts/lib_src/common/erl_misc_utils.c +++ b/erts/lib_src/common/erl_misc_utils.c @@ -28,8 +28,6 @@ #include "erl_misc_utils.h" #if defined(__WIN32__) -#elif defined(VXWORKS) -# include <selectLib.h> #else /* UNIX */ # include <stdio.h> # include <sys/types.h> @@ -124,6 +122,12 @@ #include <sys/sysctl.h> #endif +/* Simplify include for static functions */ + +#if defined(__linux__) || defined(HAVE_KSTAT) || defined(__WIN32__) || defined(__FreeBSD__) +# define ERTS_CPU_TOPOLOGY_ENABLED (1) +#endif + static int read_topology(erts_cpu_info_t *cpuinfo); #if defined(ERTS_HAVE_MISC_UTIL_AFFINITY_MASK__) @@ -669,6 +673,7 @@ erts_unbind_from_cpu_str(char *str) } +#if defined(ERTS_CPU_TOPOLOGY_ENABLED) static int pn_cmp(const void *vx, const void *vy) { @@ -759,6 +764,7 @@ adjust_processor_nodes(erts_cpu_info_t *cpuinfo, int no_nodes) } } } +#endif #ifdef __linux__ diff --git a/erts/preloaded/ebin/erl_prim_loader.beam b/erts/preloaded/ebin/erl_prim_loader.beam Binary files differindex e46795e2b1..344fa3ecc0 100644 --- a/erts/preloaded/ebin/erl_prim_loader.beam +++ b/erts/preloaded/ebin/erl_prim_loader.beam diff --git a/erts/preloaded/ebin/erlang.beam b/erts/preloaded/ebin/erlang.beam Binary files differindex bf6d6e871b..bea586b11c 100644 --- a/erts/preloaded/ebin/erlang.beam +++ b/erts/preloaded/ebin/erlang.beam diff --git a/erts/preloaded/ebin/erts_internal.beam b/erts/preloaded/ebin/erts_internal.beam Binary files differnew file mode 100644 index 0000000000..af06c2a1cb --- /dev/null +++ b/erts/preloaded/ebin/erts_internal.beam diff --git a/erts/preloaded/ebin/init.beam b/erts/preloaded/ebin/init.beam Binary files differindex 1a250932da..9710204552 100644 --- a/erts/preloaded/ebin/init.beam +++ b/erts/preloaded/ebin/init.beam diff --git a/erts/preloaded/ebin/otp_ring0.beam b/erts/preloaded/ebin/otp_ring0.beam Binary files differindex 2fa007f683..0ba1cb8f73 100644 --- a/erts/preloaded/ebin/otp_ring0.beam +++ b/erts/preloaded/ebin/otp_ring0.beam diff --git a/erts/preloaded/ebin/prim_file.beam b/erts/preloaded/ebin/prim_file.beam Binary files differindex 19cf856754..ff44d38fa9 100644 --- a/erts/preloaded/ebin/prim_file.beam +++ b/erts/preloaded/ebin/prim_file.beam diff --git a/erts/preloaded/ebin/prim_inet.beam b/erts/preloaded/ebin/prim_inet.beam Binary files differindex b2d224e560..a1aca09d1d 100644 --- a/erts/preloaded/ebin/prim_inet.beam +++ b/erts/preloaded/ebin/prim_inet.beam diff --git a/erts/preloaded/ebin/prim_zip.beam b/erts/preloaded/ebin/prim_zip.beam Binary files differindex 55697cfe11..bec1f2cc21 100644 --- a/erts/preloaded/ebin/prim_zip.beam +++ b/erts/preloaded/ebin/prim_zip.beam diff --git a/erts/preloaded/ebin/zlib.beam b/erts/preloaded/ebin/zlib.beam Binary files differindex d2f30fd9cd..1ac89dd42a 100644 --- a/erts/preloaded/ebin/zlib.beam +++ b/erts/preloaded/ebin/zlib.beam diff --git a/erts/preloaded/src/Makefile b/erts/preloaded/src/Makefile index 5bcc2eb6e4..a224b6a5d4 100644 --- a/erts/preloaded/src/Makefile +++ b/erts/preloaded/src/Makefile @@ -40,7 +40,8 @@ PRE_LOADED_MODULES = \ zlib \ prim_zip \ otp_ring0 \ - erlang + erlang \ + erts_internal RELSYSDIR = $(RELEASE_PATH)/lib/erts-$(VSN) # not $(RELEASE_PATH)/erts-$(VSN)/preloaded diff --git a/erts/preloaded/src/erlang.erl b/erts/preloaded/src/erlang.erl index 9e814ae54b..061db72dd8 100644 --- a/erts/preloaded/src/erlang.erl +++ b/erts/preloaded/src/erlang.erl @@ -79,6 +79,7 @@ -export([bump_reductions/1, byte_size/1, call_on_load_function/1]). -export([cancel_timer/1, check_old_code/1, check_process_code/2, crc32/1]). -export([crc32/2, crc32_combine/3, date/0, decode_packet/3]). +-export([delete_element/2]). -export([delete_module/1, demonitor/1, demonitor/2, display/1]). -export([display_nl/0, display_string/1, dist_exit/3, erase/0, erase/1]). -export([error/1, error/2, exit/1, exit/2, external_size/1]). @@ -88,6 +89,7 @@ -export([garbage_collect_message_area/0, get/0, get/1, get_keys/1]). -export([get_module_info/1, get_stacktrace/0, group_leader/0]). -export([group_leader/2, halt/0, halt/1, halt/2, hash/2, hibernate/3]). +-export([insert_element/3]). -export([integer_to_list/1, iolist_size/1, iolist_to_binary/1]). -export([is_alive/0, is_builtin/3, is_process_alive/1, length/1, link/1]). -export([list_to_atom/1, list_to_binary/1, list_to_bitstr/1]). @@ -529,6 +531,14 @@ date() -> decode_packet(_Type, _Bin, _Options) -> erlang:nif_error(undefined). +%% delete_element/2 +-spec erlang:delete_element(Index, Tuple1) -> Tuple2 when + Index :: pos_integer(), + Tuple1 :: tuple(), + Tuple2 :: tuple(). +delete_element(_Index, _Tuple1) -> + erlang:nif_error(undefined). + %% delete_module/1 -spec delete_module(Module) -> true | undefined when Module :: module(). @@ -820,6 +830,15 @@ hash(_Term, _Range) -> hibernate(_Module, _Function, _Args) -> erlang:nif_error(undefined). +%% insert_element/3 +-spec erlang:insert_element(Index, Tuple1, Term) -> Tuple2 when + Index :: pos_integer(), + Tuple1 :: tuple(), + Tuple2 :: tuple(), + Term :: term(). +insert_element(_Index, _Tuple1, _Term) -> + erlang:nif_error(undefined). + %% integer_to_list/1 -spec integer_to_list(Integer) -> string() when Integer :: integer(). @@ -1075,57 +1094,6 @@ phash2(_Term, _Range) -> pid_to_list(_Pid) -> erlang:nif_error(undefined). -%% port_close/1 --spec port_close(Port) -> true when - Port :: port() | atom(). -port_close(_Port) -> - erlang:nif_error(undefined). - -%% port_command/2 --spec port_command(Port, Data) -> true when - Port :: port() | atom(), - Data :: iodata(). -port_command(_Port, _Data) -> - erlang:nif_error(undefined). - -%% port_command/3 --spec port_command(Port, Data, OptionList) -> boolean() when - Port :: port() | atom(), - Data :: iodata(), - OptionList :: [Option], - Option :: force | nosuspend. -port_command(_Port, _Data, _OptionList) -> - erlang:nif_error(undefined). - -%% port_connect/2 --spec port_connect(Port, Pid) -> true when - Port :: port() | atom(), - Pid :: pid(). -port_connect(_Port, _Pid) -> - erlang:nif_error(undefined). - -%% port_control/3 --spec port_control(Port, Operation, Data) -> Res when - Port :: port() | atom(), - Operation :: integer(), - Data :: iodata(), - Res :: string() | binary(). -port_control(_Port, _Operation, _Data) -> - erlang:nif_error(undefined). - -%% port_get_data/1 --spec erlang:port_get_data(P1) -> term() when - P1 :: port() | atom(). -port_get_data(_P1) -> - erlang:nif_error(undefined). - -%% port_set_data/2 --spec erlang:port_set_data(P1, P2) -> true when - P1 :: port() | atom(), - P2 :: term(). -port_set_data(_P1, _P2) -> - erlang:nif_error(undefined). - %% port_to_list/1 -spec erlang:port_to_list(Port) -> string() when Port :: port(). @@ -1700,60 +1668,11 @@ nodes(_Arg) -> | out | binary | eof + | {parallelism, Boolean :: boolean()} | hide. open_port(_PortName,_PortSettings) -> erlang:nif_error(undefined). -%% Shadowed by erl_bif_types: erlang:port_call/2 --spec erlang:port_call(Port, Data) -> term() when - Port :: port() | atom(), - Data :: term(). -port_call(_Port, _Data) -> - erlang:nif_error(undefined). - -%% Shadowed by erl_bif_types: erlang:port_call/3 --spec erlang:port_call(Port, Operation, Data) -> term() when - Port :: port() | atom(), - Operation :: integer(), - Data :: term(). -port_call(_Port, _Operation, _Data) -> - erlang:nif_error(undefined). - --type port_info_item() :: - registered_name | - id | - connected | - links | - name | - input | - output | - os_pid. - --type port_info_result_item() :: - {registered_name, RegName :: atom()} | - {id, Index :: non_neg_integer()} | - {connected, Pid :: pid()} | - {links, Pids :: [pid()]} | - {name, String :: string()} | - {input, Bytes :: non_neg_integer()} | - {output, Bytes :: non_neg_integer()} | - {os_pid, OsPid :: non_neg_integer() | 'undefined'}. - -%% Shadowed by erl_bif_types: erlang:port_info/1 --spec erlang:port_info(Port) -> Result when - Port :: port() | atom(), - Result :: [port_info_result_item()] | undefined. -port_info(_Result) -> - erlang:nif_error(undefined). - -%% Shadowed by erl_bif_types: erlang:port_info/2 --spec erlang:port_info(Port, Item) -> Result when - Port :: port() | atom(), - Item :: port_info_item(), - Result :: port_info_result_item() | undefined. -port_info(_Result, _Item) -> - erlang:nif_error(undefined). - -type priority_level() :: low | normal | high | max. @@ -1837,7 +1756,7 @@ process_flag(_Flag, _Value) -> {group_leader, GroupLeader :: pid()} | {heap_size, Size :: non_neg_integer()} | {initial_call, mfa()} | - {links, Pids :: [pid() | port()]} | + {links, PidsAndPorts :: [pid() | port()]} | {last_calls, false | (Calls :: [mfa()])} | {memory, Size :: non_neg_integer()} | {message_que_len, MessageQueueLen :: non_neg_integer()} | @@ -2531,6 +2450,216 @@ suspend_process(P) -> end. %% +%% Port BIFs +%% +%% Currently all port BIFs calls the corresponding +%% erts_internal:port_*() native function which perform +%% most of the actual work. These native functions should +%% *never* be called directly by other functionality. The +%% native functions may be changed, or removed without any +%% notice whatsoever! +%% +%% IMPORTANT NOTE: +%% When the erts_internal:port_*() native functions return +%% a reference, they have also internally prepared the +%% message queue of the caller for a receive that will +%% unconditionally wait for a message containing this +%% reference. If the erlang code calling these native +%% functions do not do this, subsequent receives will not +%% work as expected! That is, it is of *vital importance* +%% that the receive is performed as described above! +%% + +-spec port_command(Port, Data) -> 'true' when + Port :: port() | atom(), + Data :: iodata(). + +port_command(Port, Data) -> + case case erts_internal:port_command(Port, Data, []) of + Ref when erlang:is_reference(Ref) -> receive {Ref, Res} -> Res end; + Res -> Res + end of + true -> true; + Error -> erlang:error(Error, [Port, Data]) + end. + +-spec port_command(Port, Data, OptionList) -> boolean() when + Port :: port() | atom(), + Data :: iodata(), + Option :: force | nosuspend, + OptionList :: [Option]. + +port_command(Port, Data, Flags) -> + case case erts_internal:port_command(Port, Data, Flags) of + Ref when erlang:is_reference(Ref) -> receive {Ref, Res} -> Res end; + Res -> Res + end of + Bool when Bool == true; Bool == false -> Bool; + Error -> erlang:error(Error, [Port, Data, Flags]) + end. + +-spec port_connect(Port, Pid) -> 'true' when + Port :: port() | atom(), + Pid :: pid(). + +port_connect(Port, Pid) -> + case case erts_internal:port_connect(Port, Pid) of + Ref when erlang:is_reference(Ref) -> receive {Ref, Res} -> Res end; + Res -> Res + end of + true -> true; + Error -> erlang:error(Error, [Port, Pid]) + end. + +-spec port_close(Port) -> 'true' when + Port :: port() | atom(). + +port_close(Port) -> + case case erts_internal:port_close(Port) of + Ref when erlang:is_reference(Ref) -> receive {Ref, Res} -> Res end; + Res -> Res + end of + true -> true; + Error -> erlang:error(Error, [Port]) + end. + +-spec port_control(Port, Operation, Data) -> iodata() | binary() when + Port :: port() | atom(), + Operation :: integer(), + Data :: iodata(). + +port_control(Port, Operation, Data) -> + case case erts_internal:port_control(Port, Operation, Data) of + Ref when erlang:is_reference(Ref) -> receive {Ref, Res} -> Res end; + Res -> Res + end of + badarg -> erlang:error(badarg, [Port, Operation, Data]); + Result -> Result + end. + +-spec erlang:port_call(Port, Data) -> term() when + Port :: port() | atom(), + Data :: term(). + +port_call(Port, Data) -> + case case erts_internal:port_call(Port, 0, Data) of + Ref when erlang:is_reference(Ref) -> receive {Ref, Res} -> Res end; + Res -> Res + end of + {ok, Result} -> Result; + Error -> erlang:error(Error, [Port, Data]) + end. + +-spec erlang:port_call(Port, Operation, Data) -> term() when + Port :: port() | atom(), + Operation :: integer(), + Data :: term(). + +port_call(Port, Operation, Data) -> + case case erts_internal:port_call(Port, Operation, Data) of + Ref when erlang:is_reference(Ref) -> receive {Ref, Res} -> Res end; + Res -> Res + end of + {ok, Result} -> Result; + Error -> erlang:error(Error, [Port, Operation, Data]) + end. + +-spec erlang:port_info(Port) -> Result when + Port :: port() | atom(), + ResultItem :: {registered_name, RegisteredName :: atom()} + | {id, Index :: non_neg_integer()} + | {connected, Pid :: pid()} + | {links, Pids :: [pid()]} + | {name, String :: string()} + | {input, Bytes :: non_neg_integer()} + | {output, Bytes :: non_neg_integer()} + | {os_pid, OsPid :: non_neg_integer() | 'undefined'}, + Result :: [ResultItem] | 'undefined'. + +port_info(Port) -> + case case erts_internal:port_info(Port) of + Ref when erlang:is_reference(Ref) -> receive {Ref, Res} -> Res end; + Res -> Res + end of + badarg -> erlang:error(badarg, [Port]); + Result -> Result + end. + +-spec erlang:port_info(Port, connected) -> {connected, Pid} | 'undefined' when + Port :: port() | atom(), + Pid :: pid(); + (Port, id) -> {id, Index} | 'undefined' when + Port :: port() | atom(), + Index :: non_neg_integer(); + (Port, input) -> {input, Bytes} | 'undefined' when + Port :: port() | atom(), + Bytes :: non_neg_integer(); + (Port, links) -> {links, Pids} | 'undefined' when + Port :: port() | atom(), + Pids :: [pid()]; + (Port, locking) -> {locking, Locking} | 'undefined' when + Port :: port() | atom(), + Locking :: 'false' | 'port_level' | 'driver_level'; + (Port, memory) -> {memory, Bytes} | 'undefined' when + Port :: port() | atom(), + Bytes :: non_neg_integer(); + (Port, monitors) -> {monitors, Monitors} | 'undefined' when + Port :: port() | atom(), + Monitors :: [{process, pid()}]; + (Port, name) -> {name, Name} | 'undefined' when + Port :: port() | atom(), + Name :: string(); + (Port, os_pid) -> {os_pid, OsPid} | 'undefined' when + Port :: port() | atom(), + OsPid :: non_neg_integer() | 'undefined'; + (Port, output) -> {output, Bytes} | 'undefined' when + Port :: port() | atom(), + Bytes :: non_neg_integer(); + (Port, parallelism) -> {parallelism, Boolean} | 'undefined' when + Port :: port() | atom(), + Boolean :: boolean(); + (Port, queue_size) -> {queue_size, Bytes} | 'undefined' when + Port :: port() | atom(), + Bytes :: non_neg_integer(); + (Port, registered_name) -> {registered_name, RegisteredName} | [] | 'undefined' when + Port :: port() | atom(), + RegisteredName :: atom(). + +port_info(Port, Item) -> + case case erts_internal:port_info(Port, Item) of + Ref when erlang:is_reference(Ref) -> receive {Ref, Res} -> Res end; + Res -> Res + end of + badarg -> erlang:error(badarg, [Port, Item]); + Result -> Result + end. + +-spec erlang:port_set_data(Port, Data) -> 'true' when + Port :: port() | atom(), + Data :: term(). + +port_set_data(Port, Data) -> + case case erts_internal:port_set_data(Port, Data) of + Ref when erlang:is_reference(Ref) -> receive {Ref, Res} -> Res end; + Res -> Res + end of + badarg -> erlang:error(badarg, [Port, Data]); + Result -> Result + end. + +-spec erlang:port_get_data(Port) -> term() when + Port :: port() | atom(). + +port_get_data(Port) -> + case case erts_internal:port_get_data(Port) of + Ref when erlang:is_reference(Ref) -> receive {Ref, Res} -> Res end; + Res -> Res + end of + {ok, Data} -> Data; + Error -> erlang:error(Error, [Port]) + end. + +%% %% If the emulator wants to perform a distributed command and %% a connection is not established to the actual node the following %% functions are called in order to set up the connection and then diff --git a/erts/preloaded/src/erts_internal.erl b/erts/preloaded/src/erts_internal.erl new file mode 100644 index 0000000000..f1c83f4518 --- /dev/null +++ b/erts/preloaded/src/erts_internal.erl @@ -0,0 +1,155 @@ +%% +%% %CopyrightBegin% +%% +%% Copyright Ericsson AB 2012. All Rights Reserved. +%% +%% The contents of this file are subject to the Erlang Public License, +%% Version 1.1, (the "License"); you may not use this file except in +%% compliance with the License. You should have received a copy of the +%% Erlang Public License along with this software. If not, it can be +%% retrieved online at http://www.erlang.org/. +%% +%% Software distributed under the License is distributed on an "AS IS" +%% basis, WITHOUT WARRANTY OF ANY KIND, either express or implied. See +%% the License for the specific language governing rights and limitations +%% under the License. +%% +%% %CopyrightEnd% +%% + +%% +%% As the module name imply, this module is here for ERTS internal +%% functionality. As an application programmer you should *never* +%% call anything in this module directly. Functions exported by +%% this module may change behaviour or be removed at any time +%% without any notice whatsoever. Everything in this module is +%% intentionally left undocumented, and should remain so. +%% + +-module(erts_internal). + +-export([await_port_send_result/3]). + +-export([port_command/3, port_connect/2, port_close/1, + port_control/3, port_call/3, port_set_data/2, port_get_data/1, + port_info/1, port_info/2]). + +%% +%% Await result of send to port +%% + +await_port_send_result(Ref, Busy, Ok) -> + receive + {Ref, false} -> Busy; + {Ref, _} -> Ok + end. + +%% +%% Statically linked port NIFs +%% + +-spec erts_internal:port_command(Port, Data, OptionList) -> Result when + Port :: port() | atom(), + Data :: iodata(), + OptionList :: [Option], + Option :: force | nosuspend, + Result :: boolean() | reference() | badarg | notsup. +port_command(_Port, _Data, _OptionList) -> + erlang:nif_error(undefined). + +-spec erts_internal:port_connect(Port, Pid) -> Result when + Port :: port() | atom(), + Pid :: pid(), + Result :: true | reference() | badarg. +port_connect(_Port, _Pid) -> + erlang:nif_error(undefined). + +-spec erts_internal:port_close(Port) -> Result when + Port :: port() | atom(), + Result :: true | reference() | badarg. +port_close(_Port) -> + erlang:nif_error(undefined). + +-spec erts_internal:port_control(Port, Operation, Data) -> Result when + Port :: port() | atom(), + Operation :: integer(), + Data :: iodata(), + Result :: string() | binary() | reference() | badarg. +port_control(_Port, _Operation, _Data) -> + erlang:nif_error(undefined). + +-spec erts_internal:port_call(Port, Operation, Data) -> Result when + Port :: port() | atom(), + Operation :: integer(), + Data :: term(), + Result :: {ok, term()} | reference() | badarg. +port_call(_Port, _Operation, _Data) -> + erlang:nif_error(undefined). + +-spec erts_internal:port_get_data(P1) -> Result when + P1 :: port() | atom(), + Result :: {ok, term()} | reference() | badarg. +port_get_data(_P1) -> + erlang:nif_error(undefined). + +-spec erts_internal:port_set_data(P1, P2) -> Result when + P1 :: port() | atom(), + P2 :: term(), + Result :: true | reference() | badarg. +port_set_data(_P1, _P2) -> + erlang:nif_error(undefined). + +-type port_info_1_result_item() :: + {registered_name, RegName :: atom()} | + {id, Index :: non_neg_integer()} | + {connected, Pid :: pid()} | + {links, Pids :: [pid()]} | + {name, String :: string()} | + {input, Bytes :: non_neg_integer()} | + {output, Bytes :: non_neg_integer()} | + {os_pid, OsPid :: non_neg_integer() | 'undefined'}. + +-spec erts_internal:port_info(Port) -> Result when + Port :: port() | atom(), + Result :: [port_info_1_result_item()] | undefined | reference() | badarg | []. +port_info(_Result) -> + erlang:nif_error(undefined). + +-type port_info_2_item() :: + registered_name | + id | + connected | + links | + name | + input | + output | + os_pid | + monitors | + memory | + parallelism | + queue_size | + locking. + +-type port_info_2_result_item() :: + {registered_name, RegName :: atom()} | + [] | % No registered name + {id, Index :: non_neg_integer()} | + {connected, Pid :: pid()} | + {links, Pids :: [pid()]} | + {name, String :: string()} | + {input, Bytes :: non_neg_integer()} | + {output, Bytes :: non_neg_integer()} | + {os_pid, OsPid :: non_neg_integer() | 'undefined'} | + {monitors, Monitors :: [{process, pid()}]} | + {memory, MemSz :: non_neg_integer()} | + {parallelism, Boolean :: boolean()} | + {queue_size, QSz :: non_neg_integer()} | + {locking, Locking :: 'false' | 'port_level' | 'driver_level'}. + +-spec erts_internal:port_info(Port, Item) -> Result when + Port :: port() | atom(), + Item :: port_info_2_item(), + Result :: port_info_2_result_item() | undefined | reference() | badarg. + +port_info(_Result, _Item) -> + erlang:nif_error(undefined). diff --git a/erts/preloaded/src/prim_inet.erl b/erts/preloaded/src/prim_inet.erl index b9800a05da..21d23159f0 100644 --- a/erts/preloaded/src/prim_inet.erl +++ b/erts/preloaded/src/prim_inet.erl @@ -1072,6 +1072,8 @@ enc_opt(deliver) -> ?INET_LOPT_DELIVER; enc_opt(exit_on_close) -> ?INET_LOPT_EXITONCLOSE; enc_opt(high_watermark) -> ?INET_LOPT_TCP_HIWTRMRK; enc_opt(low_watermark) -> ?INET_LOPT_TCP_LOWTRMRK; +enc_opt(high_msgq_watermark) -> ?INET_LOPT_TCP_MSGQ_HIWTRMRK; +enc_opt(low_msgq_watermark) -> ?INET_LOPT_TCP_MSGQ_LOWTRMRK; enc_opt(send_timeout) -> ?INET_LOPT_TCP_SEND_TIMEOUT; enc_opt(send_timeout_close) -> ?INET_LOPT_TCP_SEND_TIMEOUT_CLOSE; enc_opt(delay_send) -> ?INET_LOPT_TCP_DELAY_SEND; @@ -1126,6 +1128,8 @@ dec_opt(?INET_LOPT_DELIVER) -> deliver; dec_opt(?INET_LOPT_EXITONCLOSE) -> exit_on_close; dec_opt(?INET_LOPT_TCP_HIWTRMRK) -> high_watermark; dec_opt(?INET_LOPT_TCP_LOWTRMRK) -> low_watermark; +dec_opt(?INET_LOPT_TCP_MSGQ_HIWTRMRK) -> high_msgq_watermark; +dec_opt(?INET_LOPT_TCP_MSGQ_LOWTRMRK) -> low_msgq_watermark; dec_opt(?INET_LOPT_TCP_SEND_TIMEOUT) -> send_timeout; dec_opt(?INET_LOPT_TCP_SEND_TIMEOUT_CLOSE) -> send_timeout_close; dec_opt(?INET_LOPT_TCP_DELAY_SEND) -> delay_send; @@ -1221,6 +1225,8 @@ type_opt_1(deliver) -> type_opt_1(exit_on_close) -> bool; type_opt_1(low_watermark) -> int; type_opt_1(high_watermark) -> int; +type_opt_1(low_msgq_watermark) -> int; +type_opt_1(high_msgq_watermark) -> int; type_opt_1(send_timeout) -> time; type_opt_1(send_timeout_close) -> bool; type_opt_1(delay_send) -> bool; diff --git a/erts/vsn.mk b/erts/vsn.mk index a420781e9f..a13326a02a 100644 --- a/erts/vsn.mk +++ b/erts/vsn.mk @@ -17,6 +17,7 @@ # %CopyrightEnd% # + VSN = 5.10 SYSTEM_VSN = R16B diff --git a/lib/asn1/doc/src/asn1_ug.xml b/lib/asn1/doc/src/asn1_ug.xml index 1b399fb641..6cb251c3e2 100644 --- a/lib/asn1/doc/src/asn1_ug.xml +++ b/lib/asn1/doc/src/asn1_ug.xml @@ -186,13 +186,21 @@ END </pre> The following shows how the compiler can be called from the Erlang shell:</p> <pre> -1><input>asn1ct:compile("People",[ber_bin]).</input> +1><input>asn1ct:compile("People", [ber]).</input> +ok +2> </pre> + + <p>The <c>verbose</c> option can be given to have information + about the generated files printed:</p> + <pre> +2><input>asn1ct:compile("People", [ber,verbose]).</input> Erlang ASN.1 compiling "People.asn" --{generated,"People.asn1db"}-- --{generated,"People.hrl"}-- --{generated,"People.erl"}-- ok -2> </pre> +3> </pre> + <p>The ASN.1 module People is now accepted and the abstract syntax tree is saved in the <c>People.asn1db</c> file, the generated Erlang code is compiled using the Erlang compiler and @@ -229,7 +237,7 @@ receive constructed and encoded using <c>'People':encode('Person',Answer)</c> which takes an instance of a defined ASN.1 type and transforms it to a - (possibly) nested list of bytes according to the BER or PER + binary according to the BER or PER encoding-rules. <br></br> The encoder and the decoder can also be run from @@ -239,24 +247,12 @@ The encoder and the decoder can also be run from <pre> 2> <input>Rockstar = {'Person',"Some Name",roving,50}.</input> {'Person',"Some Name",roving,50} -3> <input>{ok,Bytes} = asn1rt:encode('People','Person',Rockstar).</input> -{ok,[<<243>>, - [17], - [19,9,"Some Name"], - [2,1,[2]], - [2,1,"2"]]} -4> <input>Bin = list_to_binary(Bytes).</input> -<<243,17,19,9,83,111,109,101,32,78,97,109,101,2,1,2,2,1,50>> -5> <input>{ok,Person} = asn1rt:decode('People','Person',Bin).</input> +3> <input>{ok,Bin} = asn1rt:encode('People','Person',Rockstar).</input> +{ok,<<243,17,19,9,83,111,109,101,32,78,97,109,101,2,1,2, + 2,1,50>>} +4> <input>{ok,Person} = asn1rt:decode('People','Person',Bin).</input> {ok,{'Person',"Some Name",roving,50}} -6> </pre> - <p>Notice that the result from <c>encode</c> is a nested list which - must be turned into a binary before the call to <c>decode</c>. A - binary is necessary as input to decode since the module was compiled - with the <c>ber_bin</c> option - The reason for returning a nested list is that it is faster to produce - and the <c>list_to_binary</c> operation is - performed automatically when the list is sent via the Erlang port mechanism.</p> +5> </pre> </section> <section> @@ -305,17 +301,15 @@ The encoder and the decoder can also be run from ASN.1 compiler:</p> <pre> erlc Person.asn -erlc -bper_bin Person.asn -erlc -bber_bin +optimize ../Example.asn +erlc -bper Person.asn +erlc -bber ../Example.asn erlc -o ../asnfiles -I ../asnfiles -I /usr/local/standards/asn1 Person.asn </pre> <p>The useful options for the ASN.1 compiler are:</p> <taglist> - <tag><c>-b[ber | per | ber_bin | per_bin | uper_bin]</c></tag> + <tag><c>-b[ber | per | uper]</c></tag> <item> <p>Choice of encoding rules, if omitted <c>ber</c> is the - default. The <c>ber_bin</c> and <c>per_bin</c> options - allows for optimizations and are therefore recommended - instead of the <c>ber</c> and <c>per</c> options.</p> + default.</p> </item> <tag><c>-o OutDirectory</c></tag> <item> @@ -339,42 +333,12 @@ erlc -o ../asnfiles -I ../asnfiles -I /usr/local/standards/asn1 Person.asn </item> <tag><c>+der</c></tag> <item> - <p>DER encoding rule. Only when using <c>-ber</c> or - <c>-ber_bin</c> option.</p> - </item> - <tag><c>+optimize</c></tag> - <item> - <p>This flag has effect only when used together with one of - <c>per_bin</c> or <c>ber_bin</c> flags. It gives time optimized - code in the generated modules and it uses another runtime module. - In the <c>per_bin</c> case a nif is used. The - result from an encode is a binary.</p> - <p><em>When this flag is used you cannot use the old format</em><c>{TypeName,Value}</c> when you encode values. Since it is - an unnecessary construct it has been removed in favor of - performance. It - is neither admitted to construct SEQUENCE or SET component values - with the format <c>{ComponentName,Value}</c> since it also is - unnecessary. The only case were it is necessary is in a CHOICE, - were you have to pass values to the right component by specifying - <c>{ComponentName,Value}</c>. See also about - <seealso marker="#TypeNameValue">{Typename,Value}</seealso> below - and in the sections for each type.</p> - </item> - <tag><c>+driver</c></tag> - <item> - <p>As of R15B this means the same as the <c>nif</c> option. Kept for - backwards compatability reasons.</p> - </item> - <tag><c>+nif</c></tag> - <item> - <p>Together with the flags <c>ber_bin</c> - and <c>optimize</c> you choose to use a nif for considerable - faster encode and decode. </p> + <p>DER encoding rule. Only when using <c>-ber</c> option.</p> </item> <tag><c>+asn1config</c></tag> <item> <p>This functionality works together with the flags - <c>ber_bin</c> and <c>optimize</c>. You enables the + <c>ber</c>. It enables the specialized decodes, see the <seealso marker="asn1_spec">Specialized Decode</seealso> chapter. </p> </item> @@ -413,7 +377,6 @@ erlc -o ../asnfiles -I ../asnfiles -I /usr/local/standards/asn1 Person.asn </item> </taglist> <p>For a complete description of <c>erlc</c> see Erts Reference Manual.</p> - <p>For preferred option use see <seealso marker="#preferred option use">Preferred Option Use</seealso> section.</p> <p>The compiler and other compile-time functions can also be invoked from the Erlang shell. Below follows a brief description of the primary functions, for a @@ -429,9 +392,9 @@ asn1ct:compile("H323-MESSAGES.asn1"). </pre> <p>which equals:</p> <pre> asn1ct:compile("H323-MESSAGES.asn1",[ber]). </pre> - <p>If one wants PER encoding with optimizations:</p> + <p>If one wants PER encoding:</p> <pre> -asn1ct:compile("H323-MESSAGES.asn1",[per_bin,optimize]). </pre> +asn1ct:compile("H323-MESSAGES.asn1",[per]). </pre> <p>The generic encode and decode functions can be invoked like this:</p> <pre> asn1ct:encode('H323-MESSAGES','SomeChoiceType',{call,"octetstring"}). @@ -443,269 +406,6 @@ asn1ct:decode('H323-MESSAGES','SomeChoiceType',Bytes). </pre> </section> <section> - <marker id="preferred option use"></marker> - <title>Preferred Option Use</title> - <p> - It may not be obvious which compile options best fit a - situation. This section describes the format of the result of - encode and decode. It also gives some performance statistics - when using certain options. Finally there is a recommendation - which option combinations should be used. - </p> - <p> - The default option is <c>ber</c>. It is the same backend as - <c>ber_bin</c> except that the result of encode is transformed - to a flat list. Below is a table that gives the different - formats of input and output of encode and decode using the - <em>allowed combinations</em> of coding and optimization - options: (EAVF stands for how ASN1 values are represented in - Erlang which is described in the <seealso - marker="#ASN1Types">ASN1 Types chapter</seealso>) - </p> - <table> - <row> - <cell align="left" valign="middle"><em>Encoding Rule</em></cell> - <cell align="left" valign="middle"><em>Compile options, allowed combinations</em></cell> - <cell align="left" valign="middle"><em>encode input</em></cell> - <cell align="left" valign="middle"><em>encode output</em></cell> - <cell align="left" valign="middle"><em>decode input</em></cell> - <cell align="left" valign="middle"><em>decode output</em></cell> - </row> - <row> - <cell align="left" valign="middle">BER</cell> - <cell align="left" valign="middle">[ber] (default)</cell> - <cell align="left" valign="middle">EAVF</cell> - <cell align="left" valign="middle">flat list</cell> - <cell align="left" valign="middle">flat list / binary</cell> - <cell align="left" valign="middle">EAVF</cell> - </row> - <row> - <cell align="left" valign="middle">BER</cell> - <cell align="left" valign="middle">[ber_bin]</cell> - <cell align="left" valign="middle">EAVF</cell> - <cell align="left" valign="middle">iolist</cell> - <cell align="left" valign="middle">binary</cell> - <cell align="left" valign="middle">EAVF</cell> - </row> - <row> - <cell align="left" valign="middle">BER</cell> - <cell align="left" valign="middle"><em>[ber_bin, optimize]</em></cell> - <cell align="left" valign="middle">EAVF</cell> - <cell align="left" valign="middle">iolist</cell> - <cell align="left" valign="middle">binary</cell> - <cell align="left" valign="middle">EAVF</cell> - </row> - <row> - <cell align="left" valign="middle">BER</cell> - <cell align="left" valign="middle"><em>[ber_bin, optimize, nif]</em></cell> - <cell align="left" valign="middle">EAVF</cell> - <cell align="left" valign="middle">iolist</cell> - <cell align="left" valign="middle">iolist / binary</cell> - <cell align="left" valign="middle">EAVF</cell> - </row> - <row> - <cell align="left" valign="middle">PER aligned variant</cell> - <cell align="left" valign="middle">[per]</cell> - <cell align="left" valign="middle">EAVF</cell> - <cell align="left" valign="middle">flat list</cell> - <cell align="left" valign="middle">flat list</cell> - <cell align="left" valign="middle">EAVF</cell> - </row> - <row> - <cell align="left" valign="middle">PER aligned variant</cell> - <cell align="left" valign="middle"><em>[per_bin]</em></cell> - <cell align="left" valign="middle">EAVF</cell> - <cell align="left" valign="middle">iolist / binary</cell> - <cell align="left" valign="middle">binary</cell> - <cell align="left" valign="middle">EAVF</cell> - </row> - <row> - <cell align="left" valign="middle">PER aligned variant</cell> - <cell align="left" valign="middle"><em>[per_bin, optimize]</em></cell> - <cell align="left" valign="middle">EAVF</cell> - <cell align="left" valign="middle">binary</cell> - <cell align="left" valign="middle">binary</cell> - <cell align="left" valign="middle">EAVF</cell> - </row> - <row> - <cell align="left" valign="middle">PER unaligned variant</cell> - <cell align="left" valign="middle"><em>[uper_bin]</em></cell> - <cell align="left" valign="middle">EAVF</cell> - <cell align="left" valign="middle">binary</cell> - <cell align="left" valign="middle">binary</cell> - <cell align="left" valign="middle">EAVF</cell> - </row> - - <row> - <cell align="left" valign="middle">DER</cell> - <cell align="left" valign="middle">[(ber), der]</cell> - <cell align="left" valign="middle">EAVF</cell> - <cell align="left" valign="middle">flat list</cell> - <cell align="left" valign="middle">flat list / binary</cell> - <cell align="left" valign="middle">EAVF</cell> - </row> - <row> - <cell align="left" valign="middle">DER</cell> - <cell align="left" valign="middle">[ber_bin, der]</cell> - <cell align="left" valign="middle">EAVF</cell> - <cell align="left" valign="middle">iolist</cell> - <cell align="left" valign="middle">binary</cell> - <cell align="left" valign="middle">EAVF</cell> - </row> - <row> - <cell align="left" valign="middle">DER</cell> - <cell align="left" valign="middle"><em>[ber_bin, optimize, der]</em></cell> - <cell align="left" valign="middle">EAVF</cell> - <cell align="left" valign="middle">iolist</cell> - <cell align="left" valign="middle">binary</cell> - <cell align="left" valign="middle">EAVF</cell> - </row> - <row> - <cell align="left" valign="middle">DER</cell> - <cell align="left" valign="middle"><em>[ber_bin, optimize, nif, der]</em></cell> - <cell align="left" valign="middle">EAVF</cell> - <cell align="left" valign="middle">iolist</cell> - <cell align="left" valign="middle">binary</cell> - <cell align="left" valign="middle">EAVF</cell> - </row> - - - <tcaption>The output / input formats for different combinations of compile options.</tcaption> - </table> - <p> - Encode / decode speed comparison in one user case for the above - alternatives (except <c>DER</c>) is showed in the table below. The - <c>DER</c> alternatives are slower than their corresponding - <c>BER</c> alternative. - </p> - - <table> - <row> - <cell align="left" valign="middle"><em>compile options</em></cell> - <cell align="left" valign="middle"><em>encode time</em></cell> - <cell align="left" valign="middle"><em>decode time</em></cell> - </row> - <row> - <cell align="left" valign="middle">[ber]</cell> - <cell align="left" valign="middle">120</cell> - <cell align="left" valign="middle">162</cell> - </row> - <row> - <cell align="left" valign="middle">[ber_bin]</cell> - <cell align="left" valign="middle">124</cell> - <cell align="left" valign="middle">154</cell> - </row> - <row> - <cell align="left" valign="middle"><em>[ber_bin, optimize]</em></cell> - <cell align="left" valign="middle">50</cell> - <cell align="left" valign="middle">78</cell> - </row> - <row> - <cell align="left" valign="middle"><em>[ber_bin, optimize, driver]</em></cell> - <cell align="left" valign="middle">50</cell> - <cell align="left" valign="middle">62</cell> - </row> - <row> - <cell align="left" valign="middle">[per]</cell> - <cell align="left" valign="middle">141</cell> - <cell align="left" valign="middle">133</cell> - </row> - <row> - <cell align="left" valign="middle"><em>[per_bin]</em></cell> - <cell align="left" valign="middle">125</cell> - <cell align="left" valign="middle">123</cell> - </row> - <row> - <cell align="left" valign="middle"><em>[per_bin, optimize]</em></cell> - <cell align="left" valign="middle">77</cell> - <cell align="left" valign="middle">72</cell> - </row> - <row> - <cell align="left" valign="middle"><em>[uper_bin]</em></cell> - <cell align="left" valign="middle">97</cell> - <cell align="left" valign="middle">104</cell> - </row> - <tcaption> - One example of difference in speed for the compile option alternatives. - </tcaption> - </table> - - <p> - The compile options <c>ber</c>, <c>per</c> and - <c>driver</c> are kept for backwards compatibility and should not be - used in new code. The nif implementation which replaces the linked-in - driver has been shown to be about 5-15% faster. - </p> - <p> - You are strongly recommended to use the appropriate alternative - of the bold typed options. The <c>optimize</c> and - <c>nif</c> options does not affect the encode or decode - result, just the time spent in run-time. When <c>ber_bin</c> and - <c>nif</c> or <c>per_bin</c> and <c>optimize</c> is - combined the C-code nif is used in chosen parts of encode / - decode procedure. - </p> - <table> - <row> - <cell align="left" valign="middle"><em>Compile options, allowed combinations</em></cell> - <cell align="left" valign="middle"><em>use of nif</em></cell> - </row> - <row> - <cell align="left" valign="middle">[ber]</cell> - <cell align="left" valign="middle">no</cell> - </row> - <row> - <cell align="left" valign="middle">[ber_bin]</cell> - <cell align="left" valign="middle">no</cell> - </row> - <row> - <cell align="left" valign="middle"><em>[ber_bin, optimize]</em></cell> - <cell align="left" valign="middle">no</cell> - </row> - <row> - <cell align="left" valign="middle"><em>[ber_bin, optimize, nif]</em></cell> - <cell align="left" valign="middle">yes</cell> - </row> - <row> - <cell align="left" valign="middle">[per]</cell> - <cell align="left" valign="middle">no</cell> - </row> - <row> - <cell align="left" valign="middle"><em>[per_bin]</em></cell> - <cell align="left" valign="middle">no</cell> - </row> - <row> - <cell align="left" valign="middle"><em>[per_bin, optimize]</em></cell> - <cell align="left" valign="middle">yes</cell> - </row> - <row> - <cell align="left" valign="middle"><em>[uper_bin]</em></cell> - <cell align="left" valign="middle">no</cell> - </row> - - <row> - <cell align="left" valign="middle">[(ber), der]</cell> - <cell align="left" valign="middle">no</cell> - </row> - <row> - <cell align="left" valign="middle">[ber_bin, der]</cell> - <cell align="left" valign="middle">no</cell> - </row> - <row> - <cell align="left" valign="middle"><em>[ber_bin, optimize, der]</em></cell> - <cell align="left" valign="middle">no</cell> - </row> - <row> - <cell align="left" valign="middle"><em>[ber_bin, optimize, nif, der]</em></cell> - <cell align="left" valign="middle">yes</cell> - </row> - - - <tcaption>When the ASN1 nif is used.</tcaption> - </table> - - </section> - <section> <title>Run-time Functions</title> <p>A brief description of the major functions is given here. For a complete description of each function see @@ -719,9 +419,9 @@ asn1rt:decode('H323-MESSAGES','SomeChoiceType',Bytes). </pre> 'H323-MESSAGES':encode('SomeChoiceType',{call,"octetstring"}). 'H323-MESSAGES':decode('SomeChoiceType',Bytes). </pre> <p>The asn1 nif is enabled in two occasions: encoding of - asn1 values when the asn1 spec is compiled with <c>per_bin</c> and - <c>optimize</c> or decode of encoded asn1 values when the asn1 spec is - compiled with <c>ber_bin</c>, <c>optimize</c> and <c>nif</c>. In + asn1 values when the asn1 spec is compiled with <c>per</c> and + or decode of encoded asn1 values when the asn1 spec is + compiled with <c>ber</c>. In those cases the nif will be loaded automatically at the first call to <c>encode</c>/<c>decode</c>. If one doesn't want the performance overhead of the nif being loaded at the first call it is possible @@ -868,26 +568,6 @@ Operational ::= BOOLEAN --ASN.1 definition </pre> <pre> Val = true, {ok,Bytes}=asn1rt:encode(MyModule,'Operational',Val), </pre> - <p>For historical reasons it is also possible to assign ASN.1 values - in Erlang using a tuple notation - with type and value as this</p> - <pre> -Val = {'Operational',true} </pre> - <warning> - <marker id="warning"></marker> - <p>The tuple notation, <c>{Typename, Value}</c> is only kept - because of backward compatibility and may be withdrawn in a - future release. If the notation is used the <c>Typename</c> - element must be spelled correctly, otherwise a run-time error - will occur. - </p> - <p>If the ASN.1 module is compiled with the flags - <c>per_bin</c> or <c>ber_bin</c> and <c>optimize</c> it is not - allowed to use the tuple notation. That possibility has been - removed due to performance reasons. Neither is it allowed to - use the <c>{ComponentName,Value}</c> notation in case of a - SEQUENCE or SET type.</p> - </warning> <p>Below follows a description of how values of each type can be represented in Erlang. </p> @@ -1149,7 +829,7 @@ TextFileVal2 = [88,76,55,44,99,121 .......... a lot of characters here ....] The following example shows how it works:</p> <p>In a file <c>PrimStrings.asn1</c> the type <c>BMP</c> is defined as <br></br> -<c>BMP ::= BMPString</c> then using BER encoding (<c>ber_bin</c> +<c>BMP ::= BMPString</c> then using BER encoding (<c>ber</c> option)the input/output format will be:</p> <pre> 1> <input>{ok,Bytes1} = asn1rt:encode('PrimStrings','BMP',[{0,0,53,53},{0,0,45,56}]).</input> @@ -1174,9 +854,9 @@ TextFileVal2 = [88,76,55,44,99,121 .......... a lot of characters here ....] <c>utf8_list_to_binary</c>, are in the <c>asn1rt</c> module. In the example below we assume an asn1 definition <c>UTF ::= UTF8String</c> in a module <c>UTF.asn</c>:</p> <pre> -1> <input>asn1ct:compile('UTF',[ber_bin]).</input> +1> <input>asn1ct:compile('UTF',[ber]).</input> Erlang ASN.1 version "1.4.3.3" compiling "UTF.asn" -Compiler Options: [ber_bin] +Compiler Options: [ber] --{generated,"UTF.asn1db"}-- --{generated,"UTF.erl"}-- ok @@ -1287,14 +967,6 @@ Pdu ::= SEQUENCE { <p>Values can be assigned in Erlang as shown below:</p> <pre> MyPdu = #'Pdu'{a=22,b=77.99,c={0,1,2,3,4},d='NULL'}. </pre> -<note> - <p> - In very early versions of the asn1 compiler it was also possible to - specify the values of the components in - a SEQUENCE or a SET as a list of tuples <c>{ComponentName,Value}</c>. - This is no longer supported. - </p> -</note> <p>The decode functions will return a record as result when decoding a <c>SEQUENCE</c> or a <c>SET</c>. <marker id="DEFAULT"></marker> @@ -1739,12 +1411,9 @@ SS ::= SET { 1> <input>Val = 'Values':tt().</input> {'TT',77,["kalle","kula"]} 2> <input>{ok,Bytes} = 'Values':encode('TT',Val).</input> -{ok,["0", - [18], - [[[128],[1],"M"],["\\241","\\r",[[[4],[5],"kalle"],[[4],[4],"kula"]]]]]} -3> <input>FlatBytes = lists:flatten(Bytes).</input> -[48,18,128,1,77,161,13,4,5,107,97,108,108,101,4,4,107,117,108,97] -4> <input>'Values':decode('TT',FlatBytes).</input> +{ok,<<48,18,128,1,77,161,13,4,5,107,97,108,108,101,4,4, + 107,117,108,97>>} +4> <input>'Values':decode('TT',Bytes).</input> {ok,{'TT',77,["kalle","kula"]}} 5> </pre> diff --git a/lib/asn1/doc/src/asn1ct.xml b/lib/asn1/doc/src/asn1ct.xml index 3be58cbc8e..9a9bb7ec1b 100644 --- a/lib/asn1/doc/src/asn1ct.xml +++ b/lib/asn1/doc/src/asn1ct.xml @@ -41,6 +41,19 @@ encode/decode functions. There are also some generic functions which can be used in during development of applications which handles ASN.1 data (encoded as BER or PER).</p> + <note> + <p>In R16, the options have been simplified. The back-end is chosen + using one of the options <c>ber</c>, <c>per</c>, or <c>uper</c>. + The options <c>optimize</c>, <c>nif</c>, and <c>driver</c> options + are no longer necessary (and the ASN.1 compiler will print a + warning if they are used). The options <c>ber_bin</c>, <c>per_bin</c>, + and <c>uper_bin</c> options will still work, but will print a warning. + </p> + <p>Another change in R16 is that the generated <c>encode/2</c> + function (and <c>asn1rt:encode/3</c>) always returns a binary. + The <c>encode/2</c> function for the BER back-end used to return + an iolist.</p> + </note> </description> <funcs> <func> @@ -50,9 +63,9 @@ <type> <v>Asn1module = atom() | string()</v> <v>Options = [Option| OldOption]</v> - <v>Option = ber_bin | per_bin | uper_bin | der | compact_bit_string | - noobj | {n2n, EnumTypeName} |{outdir, Dir} | {i, IncludeDir} | optimize | - nif | asn1config | undec_rest | {inline, OutputName} | inline | + <v>Option = ber | per | uper | der | compact_bit_string | + noobj | {n2n, EnumTypeName} |{outdir, Dir} | {i, IncludeDir} | + asn1config | undec_rest | {inline, OutputName} | inline | {macro_name_prefix, Prefix} | {record_name_prefix, Prefix} | verbose | warnings_as_errors</v> <v>OldOption = ber | per</v> <v>Reason = term()</v> @@ -112,7 +125,7 @@ File3.asn </pre> section in users guide</seealso>. Available options are: </p> <taglist> - <tag><c>ber | ber_bin | per | per_bin | uper_bin</c></tag> + <tag><c>ber | per | uper</c></tag> <item> <p> The encoding rule to be used. The supported encoding rules @@ -120,23 +133,12 @@ File3.asn </pre> PER aligned (Packed Encoding Rules) and PER unaligned. If the encoding rule option is omitted <c>ber</c> is the default. - The <c>per_bin</c> option means the aligned - variant. To use the unaligned variant the <c>uper_bin</c> - option has to be used. </p> <p> The generated Erlang module always gets the same name as the ASN.1 module and as a consequence of this only one encoding rule per ASN.1 module can be used at runtime. </p> - <p> - The <c>ber_bin</c> and <c>per_bin</c> options are - equivalent with the <c>OldOptions</c> <c>ber</c> and <c>per</c> - with the difference that the generated encoding/decoding - functions take advantage of the bit syntax, which in most - cases increases the performance considerably. The result - from encoding is a binary or an iolist. - </p> </item> <tag><c>der</c></tag> <item> @@ -144,7 +146,7 @@ File3.asn </pre> By this option the Distinguished Encoding Rules (DER) is chosen. DER is regarded as a specialized variant of the BER encoding rule, therefore the <c>der</c> option only makes sense together - with the <c>ber</c> or <c>ber_bin</c> option. + with the <c>ber</c> option. This option sometimes adds sorting and value checks when encoding, which implies a slower encoding. The decoding routines are the same @@ -206,28 +208,6 @@ Binary = binary() shall be placed. If omitted the files are placed in the current directory.</p> </item> - <tag><c>optimize</c></tag> - <item> - <p>This option is only valid together with one of the - <c>per_bin</c> - or <c>ber_bin</c> option. It gives time optimized code - generated and it uses another runtime module and - in the <c>per_bin</c> case a nif. The result - in the <c>per_bin</c> case from an encode when compiled - with this option will be a binary.</p> - </item> - <tag><c>driver</c></tag> - <item> - <p>As of R15B this means the same as the <c>nif</c> option. Kept for - backwards compatability reasons.</p> - </item> - <tag><c>nif</c></tag> - <item> - <p>Option valid together with <c>ber_bin</c> and <c>optimize</c> - options. It enables the use of several nifs that gives faster - encode and decode. Nifs are only enabled by the explicit use of - the option <c>nif</c></p> - </item> <tag><c>asn1config</c></tag> <item> <p>When one of the specialized decodes, exclusive or @@ -270,10 +250,6 @@ Binary = binary() <c>{export, [atom()]}</c> or <c>{export_all, true}</c> option are provided. The list of atoms are names of chosen asn1 specs from the <c>.set.asn</c> file. </p> - <p>When used together with <c>nif</c> for <c>ber_bin</c>, the - asn1 nifs will be used if the <c>asn1rt_nif</c> module is - available. If it is not available, a slower erlang fallback - will be used.</p> </item> <tag><c>inline</c></tag> <item> @@ -327,13 +303,12 @@ Binary = binary() <type> <v>Module = Type = atom()</v> <v>Value = term()</v> - <v>Bytes = [Int] when integer(Int), Int >= 0, Int =< 255</v> + <v>Bytes = binary()</v> <v>Reason = term()</v> </type> <desc> <p>Encodes <c>Value</c> of <c>Type</c> defined in the ASN.1 module - <c>Module</c>. Returns a list of bytes if successful. To get as fast execution as - possible the + <c>Module</c>. To get as fast execution as possible the encode function only performs rudimentary tests that the input <c>Value</c> is a correct instance of <c>Type</c>. The length of strings is for example @@ -348,10 +323,10 @@ Binary = binary() <type> <v>Module = Type = atom()</v> <v>Value = Reason = term()</v> - <v>Bytes = [Int] when integer(Int), Int >= 0, Int =< 255</v> + <v>Bytes = binary()</v> </type> <desc> - <p>Decodes <c>Type</c> from <c>Module</c> from the list of bytes + <p>Decodes <c>Type</c> from <c>Module</c> from the binary <c>Bytes</c>. Returns <c>{ok, Value}</c> if successful.</p> </desc> </func> diff --git a/lib/asn1/doc/src/asn1rt.xml b/lib/asn1/doc/src/asn1rt.xml index 0c3c257189..f2cac0c9e7 100644 --- a/lib/asn1/doc/src/asn1rt.xml +++ b/lib/asn1/doc/src/asn1rt.xml @@ -47,35 +47,34 @@ <type> <v>Module = Type = atom()</v> <v>Value = Reason = term()</v> - <v>Bytes = binary | [Int] when integer(Int), Int >= 0, Int =< 255 | binary</v> + <v>Bytes = binary</v> </type> <desc> - <p>Decodes <c>Type</c> from <c>Module</c> from the list of bytes or - binary <c>Bytes</c>. If the module is compiled with <c>ber_bin</c> - or <c>per_bin</c> option <c>Bytes</c> must be a binary. + <p>Decodes <c>Type</c> from <c>Module</c> from the binary <c>Bytes</c>. Returns <c>{ok,Value}</c> if successful.</p> </desc> </func> <func> - <name>encode(Module,Type,Value)-> {ok,BinOrList} | {error,Reason}</name> + <name>encode(Module,Type,Value)-> {ok,Bytes} | {error,Reason}</name> <fsummary>Encode an ASN.1 value.</fsummary> <type> <v>Module = Type = atom()</v> <v>Value = term()</v> - <v>BinOrList = Bytes | binary()</v> - <v>Bytes = [Int|binary|Bytes] when integer(Int), Int >= 0, Int =< 255</v> + <v>Bytes = binary</v> <v>Reason = term()</v> </type> <desc> <p>Encodes <c>Value</c> of <c>Type</c> defined in the ASN.1 module <c>Module</c>. Returns a possibly nested list of bytes and or binaries - if successful. If <c>Module</c> was compiled with the options <c>per_bin</c> and <c>optimize</c> the result is a binary. To get as - fast execution as possible the + if successful. To get as fast execution as possible the encode function only performs rudimentary tests that the input <c>Value</c> is a correct instance of <c>Type</c>. The length of strings is for example not always checked. </p> + <note> + <p>Starting in R16, <c>Bytes</c> is always a binary.</p> + </note> </desc> </func> @@ -95,28 +94,6 @@ </func> <func> - <name>load_driver() -> ok | {error,Reason}</name> - <fsummary>Loads the linked-in driver. (deprecated)</fsummary> - <type> - <v>Reason = term()</v> - </type> - <desc> - <p>This function is obsolete and will be removed in R16A</p> - </desc> - </func> - - <func> - <name>unload_driver() -> ok | {error,Reason}</name> - <fsummary>Unloads the linked-in driver. (deprecated)</fsummary> - <type> - <v>Reason = term()</v> - </type> - <desc> - <p>This function is obsolete and will be removed in R16A</p> - </desc> - </func> - - <func> <name>utf8_binary_to_list(UTF8Binary) -> {ok,UnicodeList} | {error,Reason}</name> <fsummary>Transforms an utf8 encoded binary to a unicode list.</fsummary> <type> diff --git a/lib/asn1/doc/src/notes.xml b/lib/asn1/doc/src/notes.xml index 5ca86130a1..72b496caf7 100644 --- a/lib/asn1/doc/src/notes.xml +++ b/lib/asn1/doc/src/notes.xml @@ -483,7 +483,7 @@ ENUMERATION type, the compilation will be terminated with an error code.<br/> Below follows an example on how to use the option from the command line with <c>erlc</c>:<br/> - <c>erlc -bper_bin +optimize +driver +"{n2n,'CauseMisc'}" +"{n2n,'CausePcl'}" MyModyle.asn</c> + <c>erlc -bper+"{n2n,'CauseMisc'}" +"{n2n,'CausePcl'}" MyModyle.asn</c> </p> <p> Own Id: OTP-8136 Aux Id: seq11347 </p> diff --git a/lib/asn1/src/Makefile b/lib/asn1/src/Makefile index 4bd49aa93b..8d9422144e 100644 --- a/lib/asn1/src/Makefile +++ b/lib/asn1/src/Makefile @@ -52,10 +52,9 @@ CT_MODULES= \ asn1ct_gen_per_rt2ct \ asn1ct_name \ asn1ct_constructed_per \ - asn1ct_constructed_ber \ - asn1ct_gen_ber \ asn1ct_constructed_ber_bin_v2 \ asn1ct_gen_ber_bin_v2 \ + asn1ct_imm \ asn1ct_value \ asn1ct_tok \ asn1ct_parser2 \ @@ -63,7 +62,6 @@ CT_MODULES= \ RT_MODULES= \ asn1rt \ - asn1rt_per_bin \ asn1rt_ber_bin \ asn1rt_ber_bin_v2 \ asn1rt_per_bin_rt2ct \ @@ -166,11 +164,32 @@ release_spec: opt release_docs_spec: +# +# Dependencies +# - - - - - - - +$(EBIN)/asn1_app.beam: asn1_app.erl +$(EBIN)/asn1_db.beam: asn1_db.erl +$(EBIN)/asn1ct.beam: asn1ct.erl asn1_records.hrl +$(EBIN)/asn1ct_check.beam: asn1ct_check.erl asn1_records.hrl +$(EBIN)/asn1ct_constructed_ber_bin_v2.beam: asn1ct_constructed_ber_bin_v2.erl \ + asn1_records.hrl +$(EBIN)/asn1ct_constructed_per.beam: asn1ct_constructed_per.erl asn1_records.hrl +$(EBIN)/asn1ct_gen.beam: asn1ct_gen.erl asn1_records.hrl +$(EBIN)/asn1ct_gen_ber_bin_v2.beam: asn1ct_gen_ber_bin_v2.erl asn1_records.hrl +$(EBIN)/asn1ct_gen_per.beam: asn1ct_gen_per.erl asn1_records.hrl +$(EBIN)/asn1ct_gen_per_rt2ct.beam: asn1ct_gen_per_rt2ct.erl asn1_records.hrl +$(EBIN)/asn1ct_imm.beam: asn1ct_imm.erl +$(EBIN)/asn1ct_name.beam: asn1ct_name.erl +$(EBIN)/asn1ct_parser2.beam: asn1ct_parser2.erl asn1_records.hrl +$(EBIN)/asn1ct_pretty_format.beam: asn1ct_pretty_format.erl +$(EBIN)/asn1ct_table.beam: asn1ct_table.erl +$(EBIN)/asn1ct_tok.beam: asn1ct_tok.erl +$(EBIN)/asn1ct_value.beam: asn1ct_value.erl asn1_records.hrl +$(EBIN)/asn1rt.beam: asn1rt.erl +$(EBIN)/asn1rt_ber_bin.beam: asn1rt_ber_bin.erl asn1_records.hrl +$(EBIN)/asn1rt_ber_bin_v2.beam: asn1rt_ber_bin_v2.erl +$(EBIN)/asn1rt_check.beam: asn1rt_check.erl +$(EBIN)/asn1rt_nif.beam: asn1rt_nif.erl +$(EBIN)/asn1rt_per_bin_rt2ct.beam: asn1rt_per_bin_rt2ct.erl asn1_records.hrl +$(EBIN)/asn1rt_uper_bin.beam: asn1rt_uper_bin.erl asn1_records.hrl diff --git a/lib/asn1/src/asn1.app.src b/lib/asn1/src/asn1.app.src index 09144ba2f7..64b33a8a30 100644 --- a/lib/asn1/src/asn1.app.src +++ b/lib/asn1/src/asn1.app.src @@ -3,7 +3,6 @@ {vsn, "%VSN%"}, {modules, [ asn1rt, - asn1rt_per_bin, asn1rt_per_bin_rt2ct, asn1rt_uper_bin, asn1rt_ber_bin, diff --git a/lib/asn1/src/asn1ct.erl b/lib/asn1/src/asn1ct.erl index 8e971a1c76..90882462ac 100644 --- a/lib/asn1/src/asn1ct.erl +++ b/lib/asn1/src/asn1ct.erl @@ -85,14 +85,8 @@ compile(File) -> compile(File,[]). -compile(File,Options) when is_list(Options) -> - case lists:member(driver, Options) of %% remove me in R16A! - true -> - io:format("Warning: driver option is obsolete and will be removed in R16A, use nif instead!"); - false -> - ok - end, - Options1 = optimize_ber_bin(Options), +compile(File, Options0) when is_list(Options0) -> + Options1 = translate_options(Options0), Options2 = includes(File,Options1), Includes = strip_includes(Options2), in_process(fun() -> compile_proc(File, Includes, Options2) end). @@ -852,8 +846,8 @@ generate({true,{M,_Module,GenTOrV}},OutFile,EncodingRule,Options) -> debug_off(Options), put(compact_bit_string,false), erase(encoding_options), - erase(tlv_format), % used in ber_bin, optimize - erase(class_default_type),% used in ber_bin, optimize + erase(tlv_format), % used in ber + erase(class_default_type),% used in ber asn1ct_table:delete(check_functions), case Result of {error,_} -> @@ -876,14 +870,13 @@ parse_and_save(Module,S) -> Options = S#state.options, SourceDir = S#state.sourcedir, Includes = [I || {i,I} <-Options], - Options1 = optimize_ber_bin(Options), - + case get_input_file(Module,[SourceDir|Includes]) of %% search for asn1 source {file,SuffixedASN1source} -> - case dbfile_uptodate(SuffixedASN1source,Options1) of + case dbfile_uptodate(SuffixedASN1source,Options) of false -> - parse_and_save1(S,SuffixedASN1source,Options1,Includes); + parse_and_save1(S,SuffixedASN1source,Options,Includes); _ -> ok end; Err -> @@ -1065,9 +1058,9 @@ get_file_list1(Stream,Dir,Includes,Acc) -> end. get_rule(Options) -> - case [Rule ||Rule <-[per,ber,ber_bin,ber_bin_v2,per_bin,uper_bin], - Opt <- Options, - Rule==Opt] of + case [Rule || Rule <- [ber,per,uper], + Opt <- Options, + Rule =:= Opt] of [Rule] -> Rule; [Rule|_] -> @@ -1079,19 +1072,34 @@ get_rule(Options) -> get_runtime_mod(Options) -> RtMod1= case get_rule(Options) of - per -> ["asn1rt_per_bin.erl"]; - ber -> ["asn1rt_ber_bin.erl"]; - per_bin -> - case lists:member(optimize,Options) of - true -> ["asn1rt_per_bin_rt2ct.erl"]; - _ -> ["asn1rt_per_bin.erl"] - end; - ber_bin -> ["asn1rt_ber_bin.erl"]; - ber_bin_v2 -> ["asn1rt_ber_bin_v2.erl"]; - uper_bin -> ["asn1rt_uper_bin.erl"] + per -> "asn1rt_per_bin_rt2ct.erl"; + ber -> ["asn1rt_ber_bin_v2.erl"]; + uper -> ["asn1rt_uper_bin.erl"] end, RtMod1++["asn1rt_check.erl","asn1rt.erl"]. - + +%% translate_options(NewOptions) -> OldOptions +%% Translate the new option names to the old option name. +%% FIXME. We should rewrite all code to handle the new option names. + +translate_options([ber_bin|T]) -> + io:format("Warning: The option 'ber_bin' is now called 'ber'.\n"), + [ber|translate_options(T)]; +translate_options([per_bin|T]) -> + io:format("Warning: The option 'per_bin' is now called 'per'.\n"), + [per|translate_options(T)]; +translate_options([uper_bin|T]) -> + io:format("Warning: The option 'uper_bin' is now called 'uper'.\n"), + translate_options([uper|T]); +translate_options([nif|T]) -> + io:format("Warning: The option 'nif' is no longer needed.\n"), + translate_options(T); +translate_options([optimize|T]) -> + io:format("Warning: The option 'optimize' is no longer needed.\n"), + translate_options(T); +translate_options([H|T]) -> + [H|translate_options(T)]; +translate_options([]) -> []. erl_compile(OutFile,Options) -> % io:format("Options:~n~p~n",[Options]), @@ -1115,7 +1123,6 @@ remove_asn_flags(Options) -> X /= optimize, X /= compact_bit_string, X /= debug, - X /= keyed_list, X /= asn1config, X /= record_name_prefix]. @@ -1125,12 +1132,6 @@ debug_on(Options) -> put(asndebug,true); _ -> true - end, - case lists:member(keyed_list,Options) of - true -> - put(asn_keyed_list,true); - _ -> - true end. igorify_options(Options) -> @@ -1151,8 +1152,7 @@ generated_file(Name,Options) -> end. debug_off(_Options) -> - erase(asndebug), - erase(asn_keyed_list). + erase(asndebug). outfile(Base, Ext, Opts) -> @@ -1168,13 +1168,6 @@ outfile(Base, Ext, Opts) -> lists:concat([Obase,".",Ext]) end. -optimize_ber_bin(Options) -> - case {lists:member(optimize,Options),lists:member(ber_bin,Options)} of - {true,true} -> - [ber_bin_v2|Options--[ber_bin]]; - _ -> Options - end. - includes(File,Options) -> Options2 = include_append(".", Options), Options3 = include_append(filename:dirname(File), Options2), @@ -1284,12 +1277,7 @@ make_erl_options(Opts) -> Defines) ++ case OutputType of undefined -> [ber]; % temporary default (ber when it's ready) - ber -> [ber]; - ber_bin -> [ber_bin]; - ber_bin_v2 -> [ber_bin_v2]; - per -> [per]; - per_bin -> [per_bin]; - uper_bin -> [uper_bin] + _ -> [OutputType] % pass through end, Options++[errors, {cwd, Cwd}, {outdir, Outdir}| @@ -1400,8 +1388,7 @@ test_value(Module, Type, Value) -> in_process(fun() -> case catch encode(Module, Type, Value) of {ok, Bytes} -> - M = to_atom(Module), - NewBytes = prepare_bytes(M:encoding_rule(), Bytes), + NewBytes = prepare_bytes(Bytes), case decode(Module, Type, NewBytes) of {ok, Value} -> {ok, {Module, Type, Value}}; @@ -1452,18 +1439,8 @@ check(Module, Includes) -> end end. -to_atom(Term) when is_list(Term) -> list_to_atom(Term); -to_atom(Term) when is_atom(Term) -> Term. - -prepare_bytes(ber, Bytes) -> lists:flatten(Bytes); -prepare_bytes(ber_bin, Bytes) when is_binary(Bytes) -> Bytes; -prepare_bytes(ber_bin, Bytes) -> list_to_binary(Bytes); -prepare_bytes(ber_bin_v2, Bytes) when is_binary(Bytes) -> Bytes; -prepare_bytes(ber_bin_v2, Bytes) -> list_to_binary(Bytes); -prepare_bytes(per, Bytes) -> lists:flatten(Bytes); -prepare_bytes(per_bin, Bytes) when is_binary(Bytes) -> Bytes; -prepare_bytes(per_bin, Bytes) -> list_to_binary(Bytes); -prepare_bytes(uper_bin, Bytes) -> Bytes. +prepare_bytes(Bytes) when is_binary(Bytes) -> Bytes; +prepare_bytes(Bytes) -> list_to_binary(Bytes). vsn() -> ?vsn. @@ -1504,7 +1481,7 @@ specialized_decode_prepare(Erule,M,TsAndVs,Options) -> end. %% Reads the configuration file if it exists and stores information %% about partial decode and incomplete decode -partial_decode_prepare(ber_bin_v2,M,TsAndVs,Options) when is_tuple(TsAndVs) -> +partial_decode_prepare(ber,M,TsAndVs,Options) when is_tuple(TsAndVs) -> %% read configure file ModName = diff --git a/lib/asn1/src/asn1ct_check.erl b/lib/asn1/src/asn1ct_check.erl index 59e82b7a57..fe1b2e14a8 100644 --- a/lib/asn1/src/asn1ct_check.erl +++ b/lib/asn1/src/asn1ct_check.erl @@ -61,13 +61,13 @@ -define(TAG_PRIMITIVE(Num), case S#state.erule of - ber_bin_v2 -> + ber -> #tag{class='UNIVERSAL',number=Num,type='IMPLICIT',form=0}; _ -> [] end). -define(TAG_CONSTRUCTED(Num), case S#state.erule of - ber_bin_v2 -> + ber -> #tag{class='UNIVERSAL',number=Num,type='IMPLICIT',form=32}; _ -> [] end). @@ -3262,7 +3262,7 @@ check_type(S=#state{recordtopname=TopName},Type,Ts) when is_record(Ts,type) -> inlined=IsInlined}, TestFun = fun(Tref) -> - {_,MaybeChoice} = get_referenced_type(S,Tref), + MaybeChoice = get_non_typedef(S, Tref), case catch((MaybeChoice#typedef.typespec)#type.def) of {'CHOICE',_} -> maybe_illicit_implicit_tag(choice,Tag); @@ -3347,7 +3347,7 @@ check_type(S=#state{recordtopname=TopName},Type,Ts) when is_record(Ts,type) -> TempNewDef#newt{ type = check_externaltypereference(S,NewExt), tag = case S#state.erule of - ber_bin_v2 -> + ber -> merge_tags(Ct,RefType#type.tag); _ -> Ct @@ -3617,6 +3617,14 @@ check_type(S=#state{recordtopname=TopName},Type,Ts) when is_record(Ts,type) -> check_type(_S,Type,Ts) -> exit({error,{asn1,internal_error,Type,Ts}}). +get_non_typedef(S, Tref0) -> + case get_referenced_type(S, Tref0) of + {_,#typedef{typespec=#type{def=#'Externaltypereference'{}=Tref}}} -> + get_non_typedef(S, Tref); + {_,Type} -> + Type + end. + %% tablecinf_choose. A SEQUENCE or SET may be inserted in another %% SEQUENCE or SET by the COMPONENTS OF directive. If this inserted %% type is a referenced type that already has been checked it already @@ -5289,7 +5297,7 @@ iof_associated_type(S,[]) -> AssociateSeq = iof_associated_type1(S,[]), Tag = case S#state.erule of - ber_bin_v2 -> + ber -> [?TAG_CONSTRUCTED(?N_INSTANCE_OF)]; _ -> [] end, @@ -5320,7 +5328,7 @@ iof_associated_type1(S,C) -> end, {ObjIdTag,C1TypeTag}= case S#state.erule of - ber_bin_v2 -> + ber -> {[{'UNIVERSAL',8}], [#tag{class='UNIVERSAL', number=6, @@ -5551,8 +5559,9 @@ complist_as_tuple(_Per,[],Acc,Ext,_Acc2,ext) -> complist_as_tuple(_Per,[],Acc,Ext,Acc2,root2) -> {lists:reverse(Acc),lists:reverse(Ext),lists:reverse(Acc2)}. -is_erule_per(Erule) -> - lists:member(Erule,[per,per_bin,uper_bin]). +is_erule_per(per) -> true; +is_erule_per(uper) -> true; +is_erule_per(ber) -> false. expand_components(S, [{'COMPONENTS OF',Type}|T]) -> CompList = expand_components2(S,get_referenced_type(S,Type#type.def)), @@ -5641,7 +5650,7 @@ check_set(S,Type,Components) -> {true,_} -> {Sorted,SortedComponents} = sort_components(der,S,NewComponents), {Sorted,TableCInf,SortedComponents}; - {_,PER} when PER =:= per; PER =:= per_bin; PER =:= uper_bin -> + {_,PER} when PER =:= per; PER =:= uper -> {Sorted,SortedComponents} = sort_components(per,S,NewComponents), {Sorted,TableCInf,SortedComponents}; _ -> @@ -5765,7 +5774,7 @@ sort_universal_type(Components) -> decode_type(I) when is_integer(I) -> I; decode_type(T) -> - asn1ct_gen_ber:decode_type(T). + asn1ct_gen_ber_bin_v2:decode_type(T). untagged_choice(_S,[#'ComponentType'{typespec=#type{tag=[],def={'CHOICE',_}}}|_Rest]) -> true; @@ -6884,16 +6893,16 @@ get_taglist(S,{ObjCl,FieldNameList}) when is_record(ObjCl,objectclass), {TypeFieldName,_} when is_atom(TypeFieldName) -> []%should check if allowed end; get_taglist(S,Def) -> - case lists:member(S#state.erule,[ber_bin_v2]) of - false -> + case S#state.erule of + ber -> + []; + _ -> case Def of 'ASN1_OPEN_TYPE' -> % open_type has no UNIVERSAL tag as such []; _ -> [asn1ct_gen:def_to_tag(Def)] - end; - _ -> - [] + end end. get_taglist1(S,[#'ComponentType'{name=_Cname,tags=TagL}|Rest]) when is_list(TagL) -> diff --git a/lib/asn1/src/asn1ct_constructed_ber.erl b/lib/asn1/src/asn1ct_constructed_ber.erl deleted file mode 100644 index 360de77663..0000000000 --- a/lib/asn1/src/asn1ct_constructed_ber.erl +++ /dev/null @@ -1,1596 +0,0 @@ -%% -%% %CopyrightBegin% -%% -%% Copyright Ericsson AB 1997-2011. All Rights Reserved. -%% -%% The contents of this file are subject to the Erlang Public License, -%% Version 1.1, (the "License"); you may not use this file except in -%% compliance with the License. You should have received a copy of the -%% Erlang Public License along with this software. If not, it can be -%% retrieved online at http://www.erlang.org/. -%% -%% Software distributed under the License is distributed on an "AS IS" -%% basis, WITHOUT WARRANTY OF ANY KIND, either express or implied. See -%% the License for the specific language governing rights and limitations -%% under the License. -%% -%% %CopyrightEnd% -%% -%% --module(asn1ct_constructed_ber). - --export([gen_encode_sequence/3]). --export([gen_decode_sequence/3]). --export([gen_encode_set/3]). --export([gen_decode_set/3]). --export([gen_encode_sof/4]). --export([gen_decode_sof/4]). --export([gen_encode_choice/3]). --export([gen_decode_choice/3]). - -%%%% Application internal exports --export([match_tag/2]). - --include("asn1_records.hrl"). - --import(asn1ct_gen, [emit/1,demit/1,get_record_name_prefix/0]). - -% the encoding of class of tag bits 8 and 7 --define(UNIVERSAL, 0). --define(APPLICATION, 16#40). --define(CONTEXT, 16#80). --define(PRIVATE, 16#C0). - -% primitive or constructed encoding % bit 6 --define(PRIMITIVE, 0). --define(CONSTRUCTED, 2#00100000). - - - - -%%=============================================================================== -%%=============================================================================== -%%=============================================================================== -%% Encode/decode SEQUENCE -%%=============================================================================== -%%=============================================================================== -%%=============================================================================== - -gen_encode_sequence(Erules,Typename,D) when is_record(D,type) -> - asn1ct_name:start(), - asn1ct_name:new(term), - asn1ct_name:new(bytes), - - %% if EXTERNAL type the input value must be transformed to - %% ASN1 1990 format - case Typename of - ['EXTERNAL'] -> - emit([" NewVal = asn1rt_check:transform_to_EXTERNAL1990(Val),", - nl]); - _ -> - ok - end, - - {SeqOrSet,TableConsInfo,CompList0} = - case D#type.def of - #'SEQUENCE'{tablecinf=TCI,components=CL} -> - {'SEQUENCE',TCI,CL}; - #'SET'{tablecinf=TCI,components=CL} -> - {'SET',TCI,CL} - end, - %% filter away extensionAdditiongroup markers - CompList = filter_complist(CompList0), - Ext = extensible(CompList), - CompList1 = case CompList of - {Rl1,El,Rl2} -> Rl1 ++ El ++ Rl2; - {Rl,El} -> Rl ++ El; - _ -> CompList - end, - EncObj = - case TableConsInfo of - #simpletableattributes{usedclassfield=Used, - uniqueclassfield=Unique} when Used /= Unique -> - false; - %% ObjectSetRef, name of the object set in constraints - %% - %%{ObjectSetRef,AttrN,N,UniqueFieldName} - #simpletableattributes{objectsetname=ObjectSetRef, - c_name=AttrN, - c_index=N, - usedclassfield=UniqueFieldName, - uniqueclassfield=UniqueFieldName, - valueindex=ValueIndex - } -> - OSDef = - case ObjectSetRef of - {Module,OSName} -> - asn1_db:dbget(Module,OSName); - OSName -> - asn1_db:dbget(get(currmod),OSName) - end, -% io:format("currmod: ~p~nOSName: ~p~nAttrN: ~p~nN: ~p~nUniqueFieldName: ~p~n", -% [get(currmod),OSName,AttrN,N,UniqueFieldName]), - case (OSDef#typedef.typespec)#'ObjectSet'.gen of - true -> -% Val = lists:concat(["?RT_BER:cindex(", -% N+1,",Val,"]), - ObjectEncode = - asn1ct_gen:un_hyphen_var(lists:concat(['Obj', - AttrN])), - emit({ObjectEncode," = ",nl}), - {ObjSetMod,ObjSetName} = - case ObjectSetRef of - {M,O} -> - {{asis,M},O}; - O -> - {"?MODULE",O} - end, - emit({" ",ObjSetMod,":'getenc_",ObjSetName,"'(",{asis,UniqueFieldName}, - ", ",nl}), -% emit({indent(35),"?RT_BER:cindex(",N+1,", Val,", -% {asis,AttrN},")),",nl}), - Length = fun(X,_LFun) when is_atom(X) -> - length(atom_to_list(X)); - (X,_LFun) when is_list(X) -> - length(X); - ({X1,X2},LFun) -> - LFun(X1,LFun) + LFun(X2,LFun) - end, - emit([indent(10+Length(ObjectSetRef,Length)), - "value_match(",{asis,ValueIndex},",", - "?RT_BER:cindex(",N+1,",Val,", - {asis,AttrN},"))),",nl]), - notice_value_match(), - {AttrN,ObjectEncode}; - _ -> - false - end; - _ -> - case D#type.tablecinf of - [{objfun,_}|_] -> - %% when the simpletableattributes was at an - %% outer level and the objfun has been passed - %% through the function call - {"got objfun through args","ObjFun"}; - _ -> - false - end - end, - - gen_enc_sequence_call(Erules,Typename,CompList1,1,Ext,EncObj), - - MyTag = [X#tag{class=asn1ct_gen_ber:decode_class(X#tag.class)}|| X <- D#type.tag] - ++ - [#tag{class = asn1ct_gen_ber:decode_class('UNIVERSAL'), - number = asn1ct_gen_ber:decode_type(SeqOrSet), - form = ?CONSTRUCTED, - type = 'IMPLICIT'}], - emit([nl," BytesSoFar = "]), - case SeqOrSet of - 'SET' when (D#type.def)#'SET'.sorted == dynamic -> - emit("asn1rt_check:dynamicsort_SET_components(["), - mkvlist(asn1ct_name:all(encBytes)), - emit(["]),",nl]); - _ -> - emit("["), - mkvlist(asn1ct_name:all(encBytes)), - emit(["],",nl]) - end, - emit(" LenSoFar = "), - case asn1ct_name:all(encLen) of - [] -> emit("0"); - AllLengths -> - mkvplus(AllLengths) - end, - emit([",",nl]), -% emit(["{TagBytes,Len} = ?RT_BER:encode_tags(TagIn ++ ", - emit([" ?RT_BER:encode_tags(TagIn ++ ", - {asis,MyTag},", BytesSoFar, LenSoFar).",nl]). - - -gen_decode_sequence(Erules,Typename,D) when is_record(D,type) -> - asn1ct_name:start(), - asn1ct_name:new(tag), - #'SEQUENCE'{tablecinf=TableConsInfo,components=CList0} = D#type.def, - - %% filter away extensionAdditiongroup markers - CList = filter_complist(CList0), - - Ext = extensible(CList), - {CompList,CompList2} = case CList of - {Rl1,El,Rl2} -> {Rl1 ++ El ++ Rl2,CList}; - {Rl,El} -> {Rl ++ El, Rl ++ El}; - _ -> {CList,CList} - end, - - emit([" %%-------------------------------------------------",nl]), - emit([" %% decode tag and length ",nl]), - emit([" %%-------------------------------------------------",nl]), - - asn1ct_name:new(rb), - MyTag = [X#tag{class=asn1ct_gen_ber:decode_class(X#tag.class)}|| X <- D#type.tag] - ++ - [#tag{class = asn1ct_gen_ber:decode_class('UNIVERSAL'), - number = asn1ct_gen_ber:decode_type('SEQUENCE'), - form = ?CONSTRUCTED, - type = 'IMPLICIT'}], - emit([" {{_,",asn1ct_gen_ber:unused_var("Len",D#type.def),"},",{next,bytes},",",{curr,rb}, - "} = ?RT_BER:check_tags(TagIn ++ ",{asis,MyTag},", ", - {curr,bytes},", OptOrMand), ",nl]), - asn1ct_name:new(bytes), - asn1ct_name:new(len), - - case CompList of - [] -> true; - _ -> - emit({"{",{next,bytes}, - ",RemBytes} = ?RT_BER:split_list(", - {curr,bytes}, - ",", {prev,len},"),",nl}), - asn1ct_name:new(bytes) - end, - - {DecObjInf,UniqueFName,ValueIndex} = - case TableConsInfo of - #simpletableattributes{objectsetname=ObjectSet, - c_name=AttrN, - usedclassfield=UniqueFieldName, - uniqueclassfield=UniqueFieldName, - valueindex=ValIndex - } -> - F = fun(#'ComponentType'{typespec=CT})-> - case {asn1ct_gen:get_constraint(CT#type.constraint,componentrelation),CT#type.tablecinf} of -% case {CT#type.constraint,CT#type.tablecinf} of - {no,[{objfun,_}|_R]} -> true; - _ -> false - end - end, - case lists:any(F,CompList) of - %%AttributeName = asn1ct_gen:un_hyphen_var(AttrN), - true -> % when component relation constraint establish - %% relation from a component to another components - %% subtype component - {{AttrN,{deep,ObjectSet,UniqueFieldName, - ValIndex}}, - UniqueFieldName,ValIndex}; - false -> - {{AttrN,ObjectSet},UniqueFieldName,ValIndex} - end; - _ -> - {false,false,false} - end, - RecordName = lists:concat([get_record_name_prefix(),asn1ct_gen:list2rname(Typename)]), - case gen_dec_sequence_call(Erules,Typename,CompList2,Ext,DecObjInf) of - no_terms -> % an empty sequence - emit([nl,nl]), - demit({"Result = "}), %dbg - %% return value as record - asn1ct_name:new(rb), - emit([" {{'",RecordName,"'}, ",{curr,bytes},",",nl," "]), - asn1ct_gen_ber:add_removed_bytes(), - emit(["}.",nl]); - {LeadingAttrTerm,PostponedDecArgs} -> - emit([com,nl,nl]), - case {LeadingAttrTerm,PostponedDecArgs} of - {[],[]} -> - ok; - {_,[]} -> - ok; - {[{ObjSet,LeadingAttr,Term}],PostponedDecArgs} -> - DecObj = asn1ct_gen:un_hyphen_var(lists:concat(['DecObj',LeadingAttr,Term])), - ValueMatch = value_match(ValueIndex,Term), - {ObjSetMod,ObjSetName} = - case ObjSet of - {M,O} -> - {{asis,M},O}; - _ -> - {"?MODULE",ObjSet} - end, - emit([DecObj," =",nl," ",ObjSetMod,":'getdec_",ObjSetName,"'(", -% {asis,UniqueFName},", ",Term,"),",nl}), - {asis,UniqueFName},", ",ValueMatch,"),",nl]), - gen_dec_postponed_decs(DecObj,PostponedDecArgs) - end, - demit({"Result = "}), %dbg - %% return value as record - asn1ct_name:new(rb), - asn1ct_name:new(bytes), - ExtStatus = case Ext of - {ext,_,_} -> ext; - _ -> noext % noext | extensible - end, - emit([" {",{next,bytes},",",{curr,rb},"} = ?RT_BER:restbytes2(RemBytes, ", - {curr,bytes},",",ExtStatus,"),",nl]), - asn1ct_name:new(rb), - case Typename of - ['EXTERNAL'] -> - emit([" OldFormat={'",RecordName, - "', "]), - mkvlist(asn1ct_name:all(term)), - emit(["},",nl]), - emit([" ASN11994Format =",nl, - " asn1rt_check:transform_to_EXTERNAL1994", - "(OldFormat),",nl]), - emit([" {ASN11994Format,",{next,bytes},", "]); - _ -> - emit([" {{'",RecordName,"', "]), - mkvlist(asn1ct_name:all(term)), - emit(["}, ",{next,bytes},", "]) - end, - asn1ct_gen_ber:add_removed_bytes(), - emit(["}.",nl]) - end. - -gen_dec_postponed_decs(_,[]) -> - emit(nl); -gen_dec_postponed_decs(DecObj,[{_Cname,{FirstPFN,PFNList},Term,TmpTerm,_Tag,OptOrMand}|Rest]) -> -% asn1ct_name:new(term), - asn1ct_name:new(tmpterm), - asn1ct_name:new(reason), - - emit({"{",Term,", _, _} = ",nl}), - N = case OptOrMand of - mandatory -> 0; - 'OPTIONAL' -> - emit_opt_or_mand_check(asn1_NOVALUE,TmpTerm), - 6; - {'DEFAULT',Val} -> - emit_opt_or_mand_check(Val,TmpTerm), - 6 - end, - emit({indent(N+3),"case (catch ",DecObj,"(",{asis,FirstPFN}, -% ", ",TmpTerm,", ", {asis,Tag},", ",{asis,PFNList},")) of",nl}), - ", ",TmpTerm,", [], ",{asis,PFNList},")) of",nl}), - emit({indent(N+6),"{'EXIT', ",{curr,reason},"} ->",nl}), - emit({indent(N+9),"exit({'Type not compatible with table constraint',", - {curr,reason},"});",nl}), - emit({indent(N+6),{curr,tmpterm}," ->",nl}), - emit({indent(N+9),{curr,tmpterm},nl}), - - case OptOrMand of - mandatory -> emit([indent(N+3),"end,",nl]); - _ -> - emit([indent(N+3),"end",nl, - indent(3),"end,",nl]) - end, -% emit({indent(3),"end,",nl}), - gen_dec_postponed_decs(DecObj,Rest). - - -emit_opt_or_mand_check(Value,TmpTerm) -> - emit([indent(3),"case ",TmpTerm," of",nl, - indent(6),{asis,Value}," -> {",{asis,Value},",[],[]};",nl, - indent(6),"_ ->",nl]). - -%%============================================================================ -%% Encode/decode SET -%% -%%============================================================================ - -gen_encode_set(Erules,Typename,D) when is_record(D,type) -> - gen_encode_sequence(Erules,Typename,D). - -gen_decode_set(Erules,Typename,D) when is_record(D,type) -> - asn1ct_name:start(), - asn1ct_name:clear(), - asn1ct_name:new(term), - asn1ct_name:new(tag), - #'SET'{components=TCompList0} = D#type.def, - - %% filter away extensionAdditiongroup markers - TCompList = filter_complist(TCompList0), - Ext = extensible(TCompList), - ToOptional = fun(mandatory) -> - 'OPTIONAL'; - (X) -> X - end, - CompList = case TCompList of - {Rl1,El,Rl2} -> - Rl1 ++ [X#'ComponentType'{prop=ToOptional(Y)}||X = #'ComponentType'{prop=Y}<-El] ++ Rl2; - {Rl,El} -> Rl ++ El; - _ -> TCompList - end, - - emit([" %%-------------------------------------------------",nl]), - emit([" %% decode tag and length ",nl]), - emit([" %%-------------------------------------------------",nl]), - - asn1ct_name:new(rb), - MyTag = [X#tag{class=asn1ct_gen_ber:decode_class(X#tag.class)}|| X <- D#type.tag] - ++ - [#tag{class = asn1ct_gen_ber:decode_class('UNIVERSAL'), - number = asn1ct_gen_ber:decode_type('SET'), - form = ?CONSTRUCTED, - type = 'IMPLICIT'}], - emit([" {{_,Len},",{next,bytes},",",{curr,rb}, - "} = ?RT_BER:check_tags(TagIn ++ ",{asis,MyTag},", ", - {curr,bytes},", OptOrMand), ",nl]), - asn1ct_name:new(bytes), - asn1ct_name:new(len), - asn1ct_name:new(rb), - - emit([" {SetTerm, SetBytes, ",{curr,rb},"} = ?RT_BER:decode_set(0, Len, ", - {curr,bytes},", OptOrMand, ", - "fun 'dec_",asn1ct_gen:list2name(Typename),"_fun'/2, []),",nl]), - - asn1ct_name:new(rb), - {ExtFlatten1,ExtFlatten2} = - case Ext of - noext -> {"",""}; - _ -> {"lists:flatten(",")"} - end, - emit([" 'dec_",asn1ct_gen:list2name(Typename), - "__result__'(lists:sort(",ExtFlatten1,"SetTerm",ExtFlatten2,"), SetBytes, "]), - asn1ct_gen_ber:add_removed_bytes(), - emit([").",nl,nl,nl]), - - emit({"%%-------------------------------------------------",nl}), - emit({"%% Set loop fun for ",asn1ct_gen:list2name(Typename),nl}), - emit({"%%-------------------------------------------------",nl}), - - asn1ct_name:clear(), - asn1ct_name:new(term), - emit(["'dec_",asn1ct_gen:list2name(Typename),"_fun'(",{curr,bytes}, - ", OptOrMand) ->",nl]), - - asn1ct_name:new(bytes), - gen_dec_set(Erules,Typename,CompList,1,Ext), - - emit([" %% tag not found, if extensionmark we should skip bytes here",nl]), - emit([indent(6),"_ -> ",nl]), - case Ext of - noext -> - emit([indent(9),"{[], Bytes,0}",nl]); - _ -> - asn1ct_name:new(rbCho), - emit([indent(9),"{RestBytes, ",{curr,rbCho}, - "} = ?RT_BER:skipvalue(Bytes),",nl, - indent(9),"{[], RestBytes, ",{curr,rbCho},"}",nl]) - end, - emit([indent(3),"end.",nl,nl,nl]), - - - emit({"%%-------------------------------------------------",nl}), - emit({"%% Result ",asn1ct_gen:list2name(Typename),nl}), - emit({"%%-------------------------------------------------",nl}), - - asn1ct_name:clear(), - emit({"'dec_",asn1ct_gen:list2name(Typename),"__result__'(", - asn1ct_gen_ber:unused_var("TermList",D#type.def),", Bytes, Rb) ->",nl}), - RecordName = lists:concat([get_record_name_prefix(), - asn1ct_gen:list2rname(Typename)]), - case gen_dec_set_result(Erules,Typename,CompList) of - no_terms -> - %% return value as record - asn1ct_name:new(rb), - emit({" {{'",RecordName,"'}, Bytes, Rb}.",nl}); - _ -> - emit({nl," case ",{curr,termList}," of",nl}), - emit({" [] -> {{'",RecordName,"', "}), - mkvlist(asn1ct_name:all(term)), - emit({"}, Bytes, Rb};",nl}), - emit({" ExtraAtt -> exit({error,{asn1,{too_many_attributes, ExtraAtt}}})",nl}), - emit({" end.",nl}), - emit({nl,nl,nl}) - end. - - -%%=============================================================================== -%%=============================================================================== -%%=============================================================================== -%% Encode/decode SEQUENCE OF and SET OF -%%=============================================================================== -%%=============================================================================== -%%=============================================================================== - -gen_encode_sof(Erules,Typename,_InnerTypename,D) when is_record(D,type) -> - asn1ct_name:start(), - {SeqOrSetOf, Cont} = D#type.def, - - Objfun = case D#type.tablecinf of - [{objfun,_}|_R] -> - ", ObjFun"; - _ -> - "" - end, - - emit({" {EncBytes,EncLen} = 'enc_",asn1ct_gen:list2name(Typename), - "_components'(Val",Objfun,",[],0),",nl}), - - MyTag = [X#tag{class=asn1ct_gen_ber:decode_class(X#tag.class)}|| X <- D#type.tag] - ++ - [#tag{class = asn1ct_gen_ber:decode_class('UNIVERSAL'), - number = asn1ct_gen_ber:decode_type(SeqOrSetOf), - form = ?CONSTRUCTED, - type = 'IMPLICIT'}], -% gen_encode_tags(Erules,MyTag,"EncLen","EncBytes"), - emit([" ?RT_BER:encode_tags(TagIn ++ ", - {asis,MyTag},", EncBytes, EncLen).",nl,nl]), - - gen_encode_sof_components(Erules,Typename,SeqOrSetOf,Cont). -% gen_enc_line(Erules,Typename,TypeNameSuffix,Cont,"H",0, -% mandatory,"{EncBytes,EncLen} = "), - - -gen_decode_sof(Erules,Typename,_InnerTypename,D) when is_record(D,type) -> - asn1ct_name:start(), - asn1ct_name:clear(), - {SeqOrSetOf, TypeTag, Cont} = - case D#type.def of - {'SET OF',_Cont} -> {'SET OF','SET',_Cont}; - {'SEQUENCE OF',_Cont} -> {'SEQUENCE OF','SEQUENCE',_Cont} - end, - TypeNameSuffix = asn1ct_gen:constructed_suffix(SeqOrSetOf,Cont#type.def), - - emit({" %%-------------------------------------------------",nl}), - emit({" %% decode tag and length ",nl}), - emit({" %%-------------------------------------------------",nl}), - - asn1ct_name:new(rb), - MyTag = [X#tag{class=asn1ct_gen_ber:decode_class(X#tag.class)}|| X <- D#type.tag] - ++ - [#tag{class = asn1ct_gen_ber:decode_class('UNIVERSAL'), - number = asn1ct_gen_ber:decode_type(TypeTag), - form = ?CONSTRUCTED, - type = 'IMPLICIT'}], - emit([" {{_,Len},",{next,bytes},",",{curr,rb}, - "} = ?RT_BER:check_tags(TagIn ++ ",{asis,MyTag},", ", - {curr,bytes},", OptOrMand), ",nl]), - - emit([" ?RT_BER:decode_components(",{curr,rb}]), - InnerType = asn1ct_gen:get_inner(Cont#type.def), - ContName = case asn1ct_gen:type(InnerType) of - Atom when is_atom(Atom) -> Atom; - _ -> TypeNameSuffix - end, - emit([", Len, ",{next,bytes},", "]), -% NewCont = -% case Cont#type.def of -% {'ENUMERATED',_,Components}-> -% Cont#type{def={'ENUMERATED',Components}}; -% _ -> Cont -% end, - ObjFun = - case D#type.tablecinf of - [{objfun,_}|_R] -> - ", ObjFun"; - _ -> - [] - end, - gen_dec_line_sof(Erules,Typename,ContName,Cont,ObjFun), - emit([", []).",nl,nl,nl]). - - -gen_encode_sof_components(Erules,Typename,SeqOrSetOf,Cont) - when is_record(Cont,type)-> - - {Objfun,ObjFun_novar,EncObj} = - case Cont#type.tablecinf of - [{objfun,_}|_R] -> - {", ObjFun",", _",{no_attr,"ObjFun"}}; - _ -> - {"","",false} - end, - emit(["'enc_",asn1ct_gen:list2name(Typename), - "_components'([]",ObjFun_novar,", AccBytes, AccLen) -> ",nl]), - - case catch lists:member(der,get(encoding_options)) of - true when SeqOrSetOf=='SET OF' -> - emit([indent(3), - "{asn1rt_check:dynamicsort_SETOF(AccBytes),AccLen};",nl,nl]); - _ -> - emit([indent(3),"{lists:reverse(AccBytes),AccLen};",nl,nl]) - end, - emit(["'enc_",asn1ct_gen:list2name(Typename), - "_components'([H|T]",Objfun,",AccBytes, AccLen) ->",nl]), - TypeNameSuffix = asn1ct_gen:constructed_suffix(SeqOrSetOf,Cont#type.def), - gen_enc_line(Erules,Typename,TypeNameSuffix,Cont,"H",3, -% mandatory,"{EncBytes,EncLen} = ",EncObj), - mandatory,EncObj), - emit([",",nl]), - emit([indent(3),"'enc_",asn1ct_gen:list2name(Typename), - "_components'(T",Objfun,","]), - emit(["[EncBytes|AccBytes], AccLen + EncLen).",nl,nl]). - -%%============================================================================ -%% Encode/decode CHOICE -%% -%%============================================================================ - -gen_encode_choice(Erules,Typename,D) when is_record(D,type) -> - ChoiceTag = D#type.tag, - {'CHOICE',CompList} = D#type.def, - Ext = extensible(CompList), - CompList1 = case CompList of - {Rl1,El,Rl2} -> Rl1 ++ El ++ Rl2; - {Rl,El} -> Rl ++ El; - _ -> CompList - end, - gen_enc_choice(Erules,Typename,ChoiceTag,CompList1,Ext), - emit({nl,nl}). - -gen_decode_choice(Erules,Typename,D) when is_record(D,type) -> - asn1ct_name:start(), - asn1ct_name:new(bytes), - ChoiceTag = D#type.tag, - {'CHOICE',CompList} = D#type.def, - Ext = extensible(CompList), - CompList1 = case CompList of - {Rl1,El,Rl2} -> Rl1 ++ El ++Rl2; - {Rl,El} -> Rl ++ El; - _ -> CompList - end, - gen_dec_choice(Erules,Typename,ChoiceTag,CompList1,Ext), - emit({".",nl}). - - -%%============================================================================ -%% Encode SEQUENCE -%% -%%============================================================================ - -gen_enc_sequence_call(Erules,TopType,[#'ComponentType'{name=Cname,typespec=Type,prop=Prop,textual_order=Order}|Rest],Pos,Ext,EncObj) -> - asn1ct_name:new(encBytes), - asn1ct_name:new(encLen), - CindexPos = - case Order of - undefined -> - Pos; - _ -> Order % der - end, - Element = - case TopType of - ['EXTERNAL'] -> - io_lib:format("?RT_BER:cindex(~w,NewVal,~w)",[CindexPos+1,Cname]); - _ -> - io_lib:format("?RT_BER:cindex(~w,Val,~w)",[CindexPos+1,Cname]) - end, - InnerType = asn1ct_gen:get_inner(Type#type.def), - print_attribute_comment(InnerType,Pos,Prop), - gen_enc_line(Erules,TopType,Cname,Type,Element,3,Prop,EncObj), - case Rest of - [] -> - emit({com,nl}); - _ -> - emit({com,nl}), - gen_enc_sequence_call(Erules,TopType,Rest,Pos+1,Ext,EncObj) - end; - -gen_enc_sequence_call(_Erules,_TopType,[],_Num,_,_) -> - true. - -%%============================================================================ -%% Decode SEQUENCE -%% -%%============================================================================ - -gen_dec_sequence_call(Erules,TopType,CompList,Ext,DecObjInf) - when is_list(CompList) -> - gen_dec_sequence_call1(Erules,TopType, CompList, 1, Ext,DecObjInf,[],[]); -gen_dec_sequence_call(Erules,TopType,CList,Ext,DecObjInf) -> - gen_dec_sequence_call2(Erules,TopType,CList,Ext,DecObjInf). - -gen_dec_sequence_call1(Erules,TopType,[#'ComponentType'{name=Cname,typespec=Type,prop=Prop,tags=Tags}|Rest],Num,Ext,DecObjInf,LeadingAttrAcc,ArgsAcc) -> - {LA,PostponedDec} = - gen_dec_component(Erules,TopType,Cname,Tags,Type,Num,Prop, - Ext,DecObjInf), - case Rest of - [] -> - {LA ++ LeadingAttrAcc,PostponedDec ++ ArgsAcc}; - _ -> - emit({com,nl}), -% asn1ct_name:new(term), - asn1ct_name:new(bytes), - gen_dec_sequence_call1(Erules,TopType,Rest,Num+1,Ext,DecObjInf, - LA++LeadingAttrAcc,PostponedDec++ArgsAcc) - end; - -gen_dec_sequence_call1(_Erules,_TopType,[],1,_,_,_,_) -> - no_terms. - -gen_dec_sequence_call2(_Erules,_TopType,{[],[],[]},_Ext,_DecObjInf) -> - no_terms; -gen_dec_sequence_call2(Erules,TopType,{Root1,EList,Root2},_Ext,DecObjInf) -> - {LA,ArgsAcc} = - case gen_dec_sequence_call1(Erules,TopType,Root1++EList,1, - extensible({Root1,EList}),DecObjInf,[],[]) of - no_terms -> - {[],[]}; - Res -> Res - end, - %% TagList is the tags of Root2 elements from the first up to and - %% including the first mandatory element. - TagList = get_root2_taglist(Root2,[]), - emit({com,nl}), - asn1ct_name:new(bytes), - emit([" {",{next,bytes},", ",{next,rb}, - "} = ?RT_BER:skip_ExtensionAdditions(", - {curr,bytes},", ",{asis,TagList},"),",nl]), - asn1ct_name:new(rb), - asn1ct_name:new(bytes), - gen_dec_sequence_call1(Erules,TopType,Root2, - length(Root1)+length(EList),noext, - DecObjInf,LA,ArgsAcc). - -%% returns a list of tags of the elements in the component (second -%% root) list up to and including the first mandatory tag. See 24.6 in -%% X.680 (7/2002) -get_root2_taglist([],Acc) -> - lists:reverse(Acc); -get_root2_taglist([#'ComponentType'{prop=Prop,typespec=Type}|Rest],Acc) -> - FirstTag = fun([])->[]; - ([H|_T])->H#tag{class=asn1ct_gen_ber:decode_class(H#tag.class)} - end(Type#type.tag), - case Prop of - mandatory -> - %% match_tags/ may be used - %% this is the last tag of interest -> return - lists:reverse([FirstTag|Acc]); - _ -> - get_root2_taglist(Rest,[FirstTag|Acc]) - end. - - -%%---------------------------- -%%SEQUENCE mandatory -%%---------------------------- - -gen_dec_component(Erules,TopType,Cname,CTags,Type,Pos,Prop,Ext,DecObjInf) -> - InnerType = - case Type#type.def of - #'ObjectClassFieldType'{type=OCFTType} -> OCFTType; - _ -> asn1ct_gen:get_inner(Type#type.def) - end, - - Prop1 = case {Prop,Ext} of - {_,{ext,Epos,_Root2pos}} when Pos < Epos -> - Prop; - {mandatory,{ext,Epos,_}} when Pos >= Epos -> - 'OPTIONAL'; - _ -> - Prop - end, - print_attribute_comment(InnerType,Pos,Prop1), - emit(" "), - - case {InnerType,DecObjInf} of - {{typefield,_},NotFalse} when NotFalse /= false -> - asn1ct_name:new(term), - asn1ct_name:new(tmpterm), - emit({"{",{curr,tmpterm},", ",{next,bytes},",",{next,rb},"} = "}); - {{objectfield,_,_},_} -> - asn1ct_name:new(term), - asn1ct_name:new(tmpterm), - emit({"{",{curr,tmpterm},", ",{next,bytes},",",{next,rb},"} = "}); - _ -> - asn1ct_name:new(term), - emit({"{",{curr,term},",",{next,bytes},",",{next,rb},"} = "}) - end, - asn1ct_name:new(rb), - PostponedDec = - gen_dec_line(Erules,TopType,Cname,CTags,Type,Prop1,DecObjInf), - asn1ct_name:new(form), - PostponedDec. - - -%%------------------------------------- -%% Decode SET -%%------------------------------------- - -gen_dec_set(Erules,TopType,CompList,Pos,Ext) -> - ExtCatch = case Ext of - noext ->""; - _ -> " catch" - end, - TagList = get_all_choice_tags(CompList), - emit({indent(3), - {curr,tagList}," = ",{asis,TagList},",",nl}), - emit({indent(3), - "case",ExtCatch," ?RT_BER:check_if_valid_tag(Bytes, ", - {curr,tagList},", OptOrMand) of",nl}), - asn1ct_name:new(tagList), - asn1ct_name:new(rbCho), - asn1ct_name:new(choTags), - gen_dec_set_cases(Erules,TopType,CompList,TagList,Pos), - asn1ct_name:new(tag), - asn1ct_name:new(bytes). - - - -gen_dec_set_cases(_,_,[],_,_) -> - ok; -gen_dec_set_cases(Erules,TopType,[H|T],List,Pos) -> - Name = H#'ComponentType'.name, - Type = H#'ComponentType'.typespec, - - emit({indent(6),"'",Name,"' ->",nl}), - case Type#type.def of - {'CHOICE',_NewCompList} -> - gen_dec_set_cases_choice(Erules,TopType,H,Pos); - _ -> - gen_dec_set_cases_type(Erules,TopType,H,Pos) - end, - gen_dec_set_cases(Erules,TopType,T,List,Pos+1). - - - -gen_dec_set_cases_choice(_Erules,TopType,H,Pos) -> - Cname = H#'ComponentType'.name, - Tag = [X#tag{class=asn1ct_gen_ber:decode_class(X#tag.class)} - || X <- (H#'ComponentType'.typespec)#type.tag], - asn1ct_name:new(rbCho), - emit({indent(9),"{Dec, Rest, ",{curr,rbCho},"} = "}), - emit({"'dec_",asn1ct_gen:list2name([Cname|TopType]), - "'(Bytes,OptOrMand,",{asis,Tag},"),",nl}), - emit([" {{",Pos,",Dec}, Rest, ",{curr,rbCho},"}"]), - emit([";",nl,nl]). - - -gen_dec_set_cases_type(Erules,TopType,H,Pos) -> - Cname = H#'ComponentType'.name, - Type = H#'ComponentType'.typespec, - %% always use Prop = mandatory here Prop = H#'ComponentType'.prop, - - asn1ct_name:new(rbCho), - emit({indent(9),"{Dec, Rest, ",{curr,rbCho},"} = "}), - asn1ct_name:delete(bytes), - %% we have already seen the tag so now we must find the value - %% that why we always use 'mandatory' here - gen_dec_line(Erules,TopType,Cname,[],Type,mandatory,decObjInf), - asn1ct_name:new(bytes), - - emit([",",nl]), - emit(["{{",Pos,",Dec}, Rest, ",{curr,rbCho},"}"]), - emit([";",nl,nl]). - - -%%--------------------------------- -%% Decode SET result -%%--------------------------------- - -gen_dec_set_result(Erules,TopType,CompList) -> - gen_dec_set_result1(Erules,TopType, CompList, 1). - -gen_dec_set_result1(Erules,TopType, - [#'ComponentType'{name=Cname, - typespec=Type, - prop=Prop}|Rest],Num) -> - gen_dec_set_component(Erules,TopType,Cname,Type,Num,Prop), - case Rest of - [] -> - true; - _ -> - gen_dec_set_result1(Erules,TopType,Rest,Num+1) - end; - -gen_dec_set_result1(_Erules,_TopType,[],1) -> - no_terms; -gen_dec_set_result1(_Erules,_TopType,[],_Num) -> - true. - - -gen_dec_set_component(_Erules,_TopType,_Cname,Type,Pos,Prop) -> - InnerType = asn1ct_gen:get_inner(Type#type.def), - print_attribute_comment(InnerType,Pos,Prop), - emit({" {",{next,term},com,{next,termList},"} =",nl}), - emit({" case ",{curr,termList}," of",nl}), - emit({" [{",Pos,com,{curr,termTmp},"}|", - {curr,rest},"] -> "}), - emit({"{",{curr,termTmp},com, - {curr,rest},"};",nl}), - case Prop of - 'OPTIONAL' -> - emit([indent(10),"_ -> {asn1_NOVALUE, ",{curr,termList},"}",nl]); - {'DEFAULT', DefVal} -> - emit([indent(10), - "_ -> {",{asis,DefVal},", ",{curr,termList},"}",nl]); - mandatory -> - emit([indent(10), - "_ -> exit({error,{asn1,{mandatory_attribute_no, ", - Pos,", missing}}})",nl]) - end, - emit([indent(6),"end,",nl]), - asn1ct_name:new(rest), - asn1ct_name:new(term), - asn1ct_name:new(termList), - asn1ct_name:new(termTmp). - - -%%--------------------------------------------- -%% Encode CHOICE -%%--------------------------------------------- -%% for BER we currently do care (a little) if the choice has an EXTENSIONMARKER - - -gen_enc_choice(Erules,TopType,Tag,CompList,_Ext) -> - gen_enc_choice1(Erules,TopType,Tag,CompList,_Ext). - -gen_enc_choice1(Erules,TopType,Tag,CompList,_Ext) -> - asn1ct_name:clear(), - emit({" {EncBytes,EncLen} = case element(1,Val) of",nl}), - gen_enc_choice2(Erules,TopType,CompList), - emit([nl," end,",nl,nl]), - NewTag = [X#tag{class=asn1ct_gen_ber:decode_class(X#tag.class)}|| X <- Tag], -% gen_encode_tags(Erules,NewTag,"EncLen","EncBytes"). - emit(["?RT_BER:encode_tags(TagIn ++",{asis,NewTag},", EncBytes, EncLen).",nl]). - - - -gen_enc_choice2(Erules,TopType,[H1|T]) when is_record(H1,'ComponentType') -> - Cname = H1#'ComponentType'.name, - Type = H1#'ComponentType'.typespec, - emit({" ",{asis,Cname}," ->",nl}), - {Encobj,Assign} = -% case asn1ct_gen:get_constraint(Type#type.constraint, -% tableconstraint_info) of - case {Type#type.def,asn1ct_gen:get_constraint(Type#type.constraint, - componentrelation)} of - {#'ObjectClassFieldType'{},{componentrelation,_,_}} -> - asn1ct_name:new(tmpBytes), - asn1ct_name:new(encBytes), - asn1ct_name:new(encLen), - Emit = ["{",{curr,tmpBytes},", _} = "], - {{no_attr,"ObjFun"},Emit}; - _ -> - case Type#type.tablecinf of - [{objfun,_}] -> {{no_attr,"ObjFun"},[]}; - _-> {false,[]} - end - end, - gen_enc_line(Erules,TopType,Cname,Type,"element(2,Val)",9, - mandatory,Assign,Encobj), - case {Type#type.def,Encobj} of - {#'ObjectClassFieldType'{},{no_attr,"ObjFun"}} -> - emit({",",nl,indent(9),"{",{curr,encBytes},", ", - {curr,encLen},"}"}); - _ -> ok - end, - emit({";",nl}), - case T of - [] -> - emit([indent(6), "Else -> ",nl, - indent(9),"exit({error,{asn1,{invalid_choice_type,Else}}})"]); - _ -> - true - end, - gen_enc_choice2(Erules,TopType,T); - -gen_enc_choice2(_,_,[]) -> - true. - - - - -%%-------------------------------------------- -%% Decode CHOICE -%%-------------------------------------------- - -gen_dec_choice(Erules,TopType, ChTag, CompList, Ext) -> - asn1ct_name:delete(bytes), - Tags = [X#tag{class=asn1ct_gen_ber:decode_class(X#tag.class)}|| X <- ChTag], - - emit([" {{_,Len},",{next,bytes}, - ", RbExp} = ?RT_BER:check_tags(TagIn++", - {asis,Tags},", ", - {curr,bytes},", OptOrMand),",nl]), - asn1ct_name:new(bytes), - asn1ct_name:new(len), - gen_dec_choice_indef_funs(Erules), - case Erules of - ber_bin -> - emit([indent(3),"case ",{curr,bytes}," of",nl]); - ber -> - emit([indent(3), - "case (catch ?RT_BER:peek_tag(",{curr,bytes},")) of",nl]) - end, - asn1ct_name:new(tagList), - asn1ct_name:new(choTags), - gen_dec_choice_cases(Erules,TopType,CompList), - case Ext of - noext -> - emit([indent(6), {curr,else}," -> ",nl]), - emit([indent(9),"case OptOrMand of",nl, - indent(12),"mandatory ->","exit({error,{asn1,", - "{invalid_choice_tag,",{curr,else},"}}});",nl, - indent(12),"_ ->","exit({error,{asn1,{no_optional_tag,", - {curr,else},"}}})",nl, - indent(9),"end",nl]); - _ -> - emit([indent(6),"_ -> ",nl]), - emit([indent(9),"{{asn1_ExtAlt,",{curr,bytes},"},", - empty_lb(Erules),", RbExp}",nl]) - end, - emit([indent(3),"end"]), - asn1ct_name:new(tag), - asn1ct_name:new(else). - -gen_dec_choice_indef_funs(Erules) -> - emit({indent(3),"IndefEndBytes = fun(indefinite,",indefend_match(Erules,used_var), - ")-> R; (_,B)-> B end,",nl}), - emit({indent(3),"IndefEndRb = fun(indefinite,",indefend_match(Erules,unused_var), - ")-> 2; (_,_)-> 0 end,",nl}). - - -gen_dec_choice_cases(_,_, []) -> - ok; -gen_dec_choice_cases(Erules,TopType, [H|T]) -> - asn1ct_name:push(rbCho), - Name = H#'ComponentType'.name, - emit([nl,"%% '",Name,"'",nl]), - Fcases = fun([T1,T2|Tail],Fun) -> - emit([indent(6),match_tag(Erules,T1)," ->",nl]), - gen_dec_choice_cases_type(Erules,TopType, H), - Fun([T2|Tail],Fun); - ([T1],_) -> - emit([indent(6),match_tag(Erules,T1)," ->",nl]), - gen_dec_choice_cases_type(Erules,TopType, H) - end, - Fcases(H#'ComponentType'.tags,Fcases), - asn1ct_name:pop(rbCho), - gen_dec_choice_cases(Erules,TopType, T). - - - -gen_dec_choice_cases_type(Erules,TopType,H) -> - Cname = H#'ComponentType'.name, - Type = H#'ComponentType'.typespec, - Prop = H#'ComponentType'.prop, - emit({indent(9),"{Dec, Rest, ",{curr,rbCho},"} = "}), - gen_dec_line(Erules,TopType,Cname,[],Type,Prop,false), - emit([",",nl,indent(9),"{{",{asis,Cname}, - ", Dec}, IndefEndBytes(Len,Rest), RbExp + ", - {curr,rbCho}," + IndefEndRb(Len,Rest)};",nl,nl]). - -encode_tag_val(Erules,{Class,TagNo}) when is_integer(TagNo) -> - Rtmod = rtmod(Erules), - Rtmod:encode_tag_val({asn1ct_gen_ber:decode_class(Class), - 0,TagNo}); -encode_tag_val(Erules,{Class,TypeName}) -> - Rtmod = rtmod(Erules), - Rtmod:encode_tag_val({asn1ct_gen_ber:decode_class(Class), - 0,asn1ct_gen_ber:decode_type(TypeName)}). - - -match_tag(ber_bin,Arg) -> - match_tag_with_bitsyntax(Arg); -match_tag(Erules,Arg) -> - io_lib:format("~p",[encode_tag_val(Erules,Arg)]). - -match_tag_with_bitsyntax({Class,TagNo}) when is_integer(TagNo) -> - match_tag_with_bitsyntax1({asn1ct_gen_ber:decode_class(Class), - 0,TagNo}); -match_tag_with_bitsyntax({Class,TypeName}) -> - match_tag_with_bitsyntax1({asn1ct_gen_ber:decode_class(Class), - 0,asn1ct_gen_ber:decode_type(TypeName)}). - -match_tag_with_bitsyntax1({Class, _Form, TagNo}) when (TagNo =< 30) -> - io_lib:format("<<~p:2,_:1,~p:5,_/binary>>",[Class bsr 6,TagNo]); - -match_tag_with_bitsyntax1({Class, _Form, TagNo}) -> - {Octets,Len} = mk_object_val(TagNo), - OctForm = case Len of - 1 -> "~p"; - 2 -> "~p,~p"; - 3 -> "~p,~p,~p"; - 4 -> "~p,~p,~p,~p" - end, - io_lib:format("<<~p:2,_:1,31:5," ++ OctForm ++ ",_/binary>>", - [Class bsr 6] ++ Octets). - -%%%%%%%%%%% -%% mk_object_val(Value) -> {OctetList, Len} -%% returns a Val as a list of octets, the 8 bit is allways set to one except -%% for the last octet, where its 0 -%% - - -mk_object_val(Val) when Val =< 127 -> - {[255 band Val], 1}; -mk_object_val(Val) -> - mk_object_val(Val bsr 7, [Val band 127], 1). -mk_object_val(0, Ack, Len) -> - {Ack, Len}; -mk_object_val(Val, Ack, Len) -> - mk_object_val(Val bsr 7, [((Val band 127) bor 128) | Ack], Len + 1). - - -get_all_choice_tags(ComponentTypeList) -> - get_all_choice_tags(ComponentTypeList,[]). - -get_all_choice_tags([],TagList) -> - TagList; -get_all_choice_tags([H|T],TagList) -> - Tags = H#'ComponentType'.tags, - get_all_choice_tags(T, TagList ++ [{H#'ComponentType'.name, Tags}]). - - - -%%--------------------------------------- -%% Generate the encode/decode code -%%--------------------------------------- - -gen_enc_line(Erules,TopType,Cname, - Type=#type{constraint=C, - def=#'ObjectClassFieldType'{type={typefield,_}}}, - Element,Indent,OptOrMand=mandatory,EncObj) - when is_list(Element) -> - case asn1ct_gen:get_constraint(C,componentrelation) of - {componentrelation,_,_} -> - asn1ct_name:new(tmpBytes), - gen_enc_line(Erules,TopType,Cname,Type,Element,Indent,OptOrMand, - ["{",{curr,tmpBytes},",_} = "],EncObj); - _ -> - gen_enc_line(Erules,TopType,Cname,Type,Element,Indent,OptOrMand, - ["{",{curr,encBytes},",",{curr,encLen},"} = "], - EncObj) - end; - gen_enc_line(Erules,TopType,Cname,Type,Element,Indent,OptOrMand,EncObj) - when is_list(Element) -> - gen_enc_line(Erules,TopType,Cname,Type,Element,Indent,OptOrMand, - ["{",{curr,encBytes},",",{curr,encLen},"} = "],EncObj). - -gen_enc_line(Erules,TopType,Cname,Type,Element,Indent,OptOrMand,Assign,EncObj) - when is_list(Element) -> - IndDeep = indent(Indent), - - Tag = [X#tag{class=asn1ct_gen_ber:decode_class(X#tag.class)} - || X <- Type#type.tag], - InnerType = asn1ct_gen:get_inner(Type#type.def), - WhatKind = asn1ct_gen:type(InnerType), - emit(IndDeep), - emit(Assign), - gen_optormand_case(OptOrMand,Erules,TopType,Cname,Type,InnerType,WhatKind, - Element), - case {Type,asn1ct_gen:get_constraint(Type#type.constraint, - componentrelation)} of - {#type{def=#'ObjectClassFieldType'{type={typefield,_}, - fieldname=RefedFieldName}}, - {componentrelation,_,_}} -> - {_LeadingAttrName,Fun} = EncObj, - case RefedFieldName of - {Name,RestFieldNames} when is_atom(Name),Name =/= notype -> - case OptOrMand of - mandatory -> ok; - _ -> - emit(["{",{curr,tmpBytes},", _} = "]) - end, - emit({Fun,"(",{asis,Name},", ",Element,", [], ", - {asis,RestFieldNames},"),",nl}), - emit(IndDeep), - case OptOrMand of - mandatory -> - emit({"{",{curr,encBytes},", ",{curr,encLen},"} = "}), - emit({"?RT_BER:encode_open_type(",{curr,tmpBytes}, - ",",{asis,Tag},")"}); - _ -> - emit({"{",{next,tmpBytes},", ",{curr,tmpLen}, - "} = "}), - emit({"?RT_BER:encode_open_type(",{curr,tmpBytes}, - ",",{asis,Tag},"),",nl}), - emit(IndDeep), - emit({"{",{next,tmpBytes},", ",{curr,tmpLen},"}"}) - end; - Err -> - throw({asn1,{'internal error',Err}}) - end; - _ -> - case WhatKind of - {primitive,bif} -> - EncType = - case Type#type.def of - #'ObjectClassFieldType'{ - type={fixedtypevaluefield, - _,Btype}} -> - Btype; - _ -> - Type - end, - asn1ct_gen_ber:gen_encode_prim(ber,EncType,{asis,Tag}, - Element); - 'ASN1_OPEN_TYPE' -> - asn1ct_gen_ber:gen_encode_prim(ber,Type#type{def='ASN1_OPEN_TYPE'},{asis,Tag},Element); - _ -> - {EncFunName, _, _} = - mkfuncname(TopType,Cname,WhatKind,enc), - case {WhatKind,Type#type.tablecinf,EncObj} of - {{constructed,bif},[{objfun,_}|_R],{_,Fun}} -> - emit([EncFunName,"(",Element,", ",{asis,Tag}, - ", ",Fun,")"]); - _ -> - emit([EncFunName,"(",Element,", ",{asis,Tag},")"]) - end - end - end, - case OptOrMand of - mandatory -> true; - _ -> - emit({nl,indent(7),"end"}) - end. - - - -gen_optormand_case(mandatory,_,_,_,_,_,_, _) -> - ok; -gen_optormand_case('OPTIONAL',Erules,_,_,_,_,_,Element) -> - emit({" case ",Element," of",nl}), - emit({indent(9),"asn1_NOVALUE -> {", - empty_lb(Erules),",0};",nl}), - emit({indent(9),"_ ->",nl,indent(12)}); -gen_optormand_case({'DEFAULT',DefaultValue},Erules,TopType,Cname,Type, - InnerType,WhatKind,Element) -> - CurrMod = get(currmod), - case catch lists:member(der,get(encoding_options)) of - true -> - emit(" case catch "), - asn1ct_gen:gen_check_call(TopType,Cname,Type,InnerType, - WhatKind,{asis,DefaultValue}, - Element), - emit({" of",nl}), - emit({indent(12),"true -> {[],0};",nl}); - _ -> - emit({" case ",Element," of",nl}), - emit({indent(9),"asn1_DEFAULT -> {", - empty_lb(Erules), - ",0};",nl}), - case DefaultValue of - #'Externalvaluereference'{module=CurrMod, - value=V} -> - emit({indent(9),"?",{asis,V}," -> {", - empty_lb(Erules),",0};",nl}); - _ -> - emit({indent(9),{asis, - DefaultValue}," -> {", - empty_lb(Erules),",0};",nl}) - end - end, - emit({indent(9),"_ ->",nl,indent(12)}). - - - - -gen_dec_line_sof(Erules,TopType,Cname,Type,ObjFun) -> - - Tag = [X#tag{class=asn1ct_gen_ber:decode_class(X#tag.class)} - || X <- Type#type.tag], - InnerType = asn1ct_gen:get_inner(Type#type.def), - WhatKind = asn1ct_gen:type(InnerType), - case WhatKind of - {primitive,bif} -> - asn1ct_name:delete(len), - - asn1ct_name:new(len), - emit(["fun(FBytes,_,_)->",nl]), - EncType = case Type#type.def of - #'ObjectClassFieldType'{ - type={fixedtypevaluefield, - _,Btype}} -> - Btype; - _ -> - Type - end, - asn1ct_gen_ber:gen_dec_prim(ber,EncType,"FBytes",Tag, - [],no_length,?PRIMITIVE, - mandatory), - emit([nl,"end, []"]); - _ -> - case ObjFun of - [] -> - {DecFunName, _, _} = - mkfunname(Erules,TopType,Cname,WhatKind,dec,3), - emit([DecFunName,", ",{asis,Tag}]); - _ -> - {DecFunName, _, _} = - mkfunname(Erules,TopType,Cname,WhatKind,dec,4), - emit([DecFunName,", ",{asis,Tag},", ObjFun"]) - end - end. - - -gen_dec_line(Erules,TopType,Cname,CTags,Type,OptOrMand,DecObjInf) -> - BytesVar = asn1ct_gen:mk_var(asn1ct_name:curr(bytes)), - Tag = [X#tag{class=asn1ct_gen_ber:decode_class(X#tag.class)} - || X <- Type#type.tag], - InnerType = - case Type#type.def of - #'ObjectClassFieldType'{type=OCFTType} -> - OCFTType; - _ -> - asn1ct_gen:get_inner(Type#type.def) - end, - PostpDec = - case OptOrMand of - mandatory -> - gen_dec_call(InnerType,Erules,TopType,Cname,Type, - BytesVar,Tag,mandatory,", mandatory, ", - DecObjInf,OptOrMand); - _ -> %optional or default - case {CTags,Erules} of - {[CTag],ber_bin} when CTag =/= [] -> % R9C-0.patch-34 - emit(["case ",{curr,bytes}," of",nl]), - emit([match_tag(Erules,CTag)," ->",nl]), - PostponedDec = - gen_dec_call(InnerType,Erules,TopType,Cname,Type, - BytesVar,Tag,mandatory, - ", opt_or_default, ",DecObjInf, - OptOrMand), - emit([";",nl]), - emit(["_ ->",nl]), - case OptOrMand of - {'DEFAULT', Def} -> - emit(["{",{asis,Def},",", - BytesVar,", 0 }",nl]); - 'OPTIONAL' -> - emit(["{ asn1_NOVALUE, ", - BytesVar,", 0 }",nl]) - end, - emit("end"), - PostponedDec; - _ -> - emit("case (catch "), - PostponedDec = - gen_dec_call(InnerType,Erules,TopType,Cname,Type, - BytesVar,Tag,OptOrMand, - ", opt_or_default, ",DecObjInf, - OptOrMand), - emit([") of",nl]), - case OptOrMand of - {'DEFAULT', Def} -> - emit(["{'EXIT',{error,{asn1,{no_optional_tag,_}}}}", - " -> {",{asis,Def},",", - BytesVar,", 0 };",nl]); - 'OPTIONAL' -> - emit(["{'EXIT',{error,{asn1,{no_optional_tag,_}}}}", - " -> { asn1_NOVALUE, ", - BytesVar,", 0 };",nl]) - end, - asn1ct_name:new(casetmp), - emit([{curr,casetmp},"-> ",{curr,casetmp},nl,"end"]), - PostponedDec - end - end, - case DecObjInf of - {Cname,ObjSet} -> % this must be the component were an object is - %% choosen from the object set according to the table - %% constraint. - ObjSetName = case ObjSet of - {deep,OSName,_,_} -> - OSName; - _ -> ObjSet - end, - {[{ObjSetName,Cname,asn1ct_gen:mk_var(asn1ct_name:curr(term))}], - PostpDec}; - _ -> {[],PostpDec} - end. - - -gen_dec_call({typefield,_},Erules,_,_,Type,_,Tag,_,_,false,_) -> - %% this in case of a choice with typefield components - asn1ct_name:new(reason), - {FirstPFName,RestPFName} = -% asn1ct_gen:get_constraint(Type#type.constraint, -% tableconstraint_info), - (Type#type.def)#'ObjectClassFieldType'.fieldname, - emit([nl,indent(6),"begin",nl]), - emit([indent(9),"{OpenDec,TmpRest,TmpRbCho} =",nl,indent(12), - "?RT_BER:decode_open_type(",Erules,",",{curr,bytes},",", - {asis,Tag},"),",nl]), - emit([indent(9),"case (catch ObjFun(",{asis,FirstPFName}, - ", OpenDec, [], ",{asis,RestPFName}, - ")) of", nl]),%% ??? What about Tag - emit([indent(12),"{'EXIT',",{curr,reason},"} ->",nl]), -%% emit({indent(15),"throw({runtime_error,{'Type not ", -%% "compatible with tableconstraint', OpenDec}});",nl}), - emit([indent(15),"exit({'Type not ", - "compatible with table constraint', ",{curr,reason},"});",nl]), - emit([indent(12),"{TmpDec,_ ,_} ->",nl]), - emit([indent(15),"{TmpDec, TmpRest, TmpRbCho}",nl]), - emit([indent(9),"end",nl,indent(6),"end",nl]), - []; -gen_dec_call({typefield,_},_Erules,_,Cname,Type,_BytesVar,Tag,_,_, - _DecObjInf,OptOrMandComp) -> - emit(["?RT_BER:decode_open_type(",{curr,bytes},",",{asis,Tag},")"]), - RefedFieldName = - (Type#type.def)#'ObjectClassFieldType'.fieldname, -% asn1ct_gen:get_constraint(Type#type.constraint, -% tableconstraint_info), - [{Cname,RefedFieldName, - asn1ct_gen:mk_var(asn1ct_name:curr(term)), -% asn1ct_gen:mk_var(asn1ct_name:curr(tmpterm)),[],OptOrMandComp}]; - asn1ct_gen:mk_var(asn1ct_name:curr(tmpterm)),Tag,OptOrMandComp}]; -gen_dec_call({objectfield,PrimFieldName,PFNList},_Erules,_,Cname,_,_,Tag,_,_,_, - OptOrMandComp) -> - emit(["?RT_BER:decode_open_type(",{curr,bytes},",",{asis,Tag},")"]), - [{Cname,{PrimFieldName,PFNList}, - asn1ct_gen:mk_var(asn1ct_name:curr(term)), -% asn1ct_gen:mk_var(asn1ct_name:curr(tmpterm)),[],OptOrMandComp}]; - asn1ct_gen:mk_var(asn1ct_name:curr(tmpterm)),Tag,OptOrMandComp}]; -gen_dec_call(InnerType,Erules,TopType,Cname,Type,BytesVar,Tag,PrimOptOrMand, - OptOrMand,DecObjInf,_) -> - WhatKind = asn1ct_gen:type(InnerType), - gen_dec_call1(WhatKind,InnerType,Erules,TopType,Cname,Type,BytesVar,Tag, - PrimOptOrMand,OptOrMand), - case DecObjInf of - {Cname,{_,OSet,UniqueFName,ValIndex}} -> - Term = asn1ct_gen:mk_var(asn1ct_name:curr(term)), - ValueMatch = value_match(ValIndex,Term), - {ObjSetMod,ObjSetName} = - case OSet of - {M,O} -> - {{asis,M},O}; - _ -> - {"?MODULE",OSet} - end, - emit({",",nl,"ObjFun = ",ObjSetMod,":'getdec_",ObjSetName,"'(", - {asis,UniqueFName},", ",ValueMatch,")"}); - _ -> - ok - end, - []. -gen_dec_call1({primitive,bif},InnerType,Erules,_,_,Type,BytesVar, - Tag,OptOrMand,_) -> - case InnerType of - {fixedtypevaluefield,_,Btype} -> - asn1ct_gen_ber:gen_dec_prim(Erules,Btype,BytesVar,Tag,[],no_length, - ?PRIMITIVE,OptOrMand); - _ -> - asn1ct_gen_ber:gen_dec_prim(Erules,Type,BytesVar,Tag,[],no_length, - ?PRIMITIVE,OptOrMand) - end; -gen_dec_call1('ASN1_OPEN_TYPE',_InnerType,Erules,_,_,Type,BytesVar, - Tag,OptOrMand,_) -> - asn1ct_gen_ber:gen_dec_prim(Erules,Type#type{def='ASN1_OPEN_TYPE'}, - BytesVar,Tag,[],no_length, - ?PRIMITIVE,OptOrMand); -gen_dec_call1(WhatKind,_,_Erules,TopType,Cname,Type,_,Tag,_,OptOrMand) -> - {DecFunName,_,_} = - mkfuncname(TopType,Cname,WhatKind,dec), - case {WhatKind,Type#type.tablecinf} of - {{constructed,bif},[{objfun,_}|_R]} -> - emit({DecFunName,"(",{curr,bytes},OptOrMand,{asis,Tag},", ObjFun)"}); - _ -> - emit({DecFunName,"(",{curr,bytes},OptOrMand,{asis,Tag},")"}) - end. - - -%%------------------------------------------------------ -%% General and special help functions (not exported) -%%------------------------------------------------------ - - -indent(N) -> - lists:duplicate(N,32). % 32 = space - - -mkvlist([H,T1|T], Sep) -> % Sep is a string e.g ", " or "+ " - emit([{var,H},Sep]), - mkvlist([T1|T], Sep); -mkvlist([H|T], Sep) -> - emit([{var,H}]), - mkvlist(T, Sep); -mkvlist([], _) -> - true. - -mkvlist(L) -> - mkvlist(L,", "). - -mkvplus(L) -> - mkvlist(L," + "). - -extensible(CompList) when is_list(CompList) -> - noext; -extensible({RootList,ExtList}) -> - {ext,length(RootList)+1,length(ExtList)}; -extensible({_Rl1,_ExtL,_Rl2}) -> - extensible. -%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%% -%% filter away ExtensionAdditionGroup start and end marks since these -%% have no significance for the BER encoding -%% -filter_complist(CompList) when is_list(CompList) -> - lists:filter(fun(#'ExtensionAdditionGroup'{}) -> - false; - ('ExtensionAdditionGroupEnd') -> - false; - (_) -> - true - end, CompList); -filter_complist({Root,Ext}) -> - {Root,filter_complist(Ext)}; -filter_complist({Root1,Ext,Root2}) -> - {Root1,filter_complist(Ext),Root2}. - -print_attribute_comment(InnerType,Pos,Prop) -> - CommentLine = "%%-------------------------------------------------", - emit([nl,CommentLine]), - case InnerType of - {typereference,_,Name} -> - emit([nl,"%% attribute number ",Pos," with type ",Name]); - {'Externaltypereference',_,XModule,Name} -> - emit([nl,"%% attribute number ",Pos," External ",XModule,":",Name]); - _ -> - emit([nl,"%% attribute number ",Pos," with type ",InnerType]) - end, - case Prop of - mandatory -> - continue; - {'DEFAULT', Def} -> - emit([" DEFAULT = ",{asis,Def}]); - 'OPTIONAL' -> - emit([" OPTIONAL"]) - end, - emit([nl,CommentLine,nl]). - - -mkfuncname(TopType,Cname,WhatKind,DecOrEnc) -> - CurrMod = get(currmod), - case WhatKind of - #'Externaltypereference'{module=CurrMod,type=EType} -> - F = lists:concat(["'",DecOrEnc,"_",EType,"'"]), - {F, "?MODULE", F}; - #'Externaltypereference'{module=Mod,type=EType} -> - {lists:concat(["'",Mod,"':'",DecOrEnc,"_",EType,"'"]),Mod, - lists:concat(["'",DecOrEnc,"_",EType,"'"])}; - {constructed,bif} -> - F = lists:concat(["'",DecOrEnc,"_",asn1ct_gen:list2name([Cname|TopType]),"'"]), - {F, "?MODULE", F} - end. - -mkfunname(Erule,TopType,Cname,WhatKind,DecOrEnc,Arity) -> - CurrMod = get(currmod), - case WhatKind of - #'Externaltypereference'{module=CurrMod,type=EType} -> - F = lists:concat(["fun '",DecOrEnc,"_",EType,"'/",Arity]), - {F, "?MODULE", F}; - #'Externaltypereference'{module=Mod,type=EType} -> - {lists:concat(["fun '",Mod,"':'",DecOrEnc,"_",EType,"'/",Arity]),Mod, - lists:concat(["'",DecOrEnc,"_",EType,"'"])}; - {constructed,bif} -> - F = - lists:concat(["fun '",DecOrEnc,"_", - asn1ct_gen:list2name([Cname|TopType]),"'/", - Arity]), - {F, "?MODULE", F}; - 'ASN1_OPEN_TYPE' -> - case Arity of - 3 -> - F = lists:concat(["fun(A,_,C) -> ?RT_BER:decode_open_type(",Erule,",A,C) end"]), - {F, "?MODULE", F}; - 4 -> - F = lists:concat(["fun(A,_,C,_) -> ?RT_BER:decode_open_type(",Erule,",A,C) end"]), - {F, "?MODULE", F} - end - end. - -empty_lb(ber) -> - "[]"; -empty_lb(ber_bin) -> - "<<>>". - -rtmod(ber) -> - list_to_atom(?RT_BER_BIN); -rtmod(ber_bin) -> - list_to_atom(?RT_BER_BIN). - -indefend_match(ber,used_var) -> - "[0,0|R]"; -indefend_match(ber,unused_var) -> - "[0,0|_R]"; -indefend_match(ber_bin,used_var) -> - "<<0,0,R/binary>>"; -indefend_match(ber_bin,unused_var) -> - "<<0,0,_R/binary>>". - -notice_value_match() -> - Module = get(currmod), - put(value_match,{true,Module}). - -value_match(Index,Value) when is_atom(Value) -> - value_match(Index,atom_to_list(Value)); -value_match([],Value) -> - Value; -value_match([{VI,_Cname}|VIs],Value) -> - value_match1(Value,VIs,lists:concat(["element(",VI,","]),1). -value_match1(Value,[],Acc,Depth) -> - Acc ++ Value ++ lists:concat(lists:duplicate(Depth,")")); -value_match1(Value,[{VI,_Cname}|VIs],Acc,Depth) -> - value_match1(Value,VIs,Acc++lists:concat(["element(",VI,","]),Depth+1). diff --git a/lib/asn1/src/asn1ct_constructed_ber_bin_v2.erl b/lib/asn1/src/asn1ct_constructed_ber_bin_v2.erl index 2c4b44996d..78cb9297d8 100644 --- a/lib/asn1/src/asn1ct_constructed_ber_bin_v2.erl +++ b/lib/asn1/src/asn1ct_constructed_ber_bin_v2.erl @@ -32,7 +32,6 @@ -include("asn1_records.hrl"). -import(asn1ct_gen, [emit/1,demit/1,get_record_name_prefix/0]). --import(asn1ct_constructed_ber,[match_tag/2]). -define(ASN1CT_GEN_BER,asn1ct_gen_ber_bin_v2). @@ -913,7 +912,7 @@ gen_dec_choice_cases(Erules,TopType, [H|T]) -> [DecTag],Type}), asn1ct:update_gen_state(namelist,Names), emit([indent(4),{curr,res}," = ", - match_tag(ber_bin,{FirstT#tag.class,FirstT#tag.number}), + match_tag(FirstT#tag.class, FirstT#tag.number), " -> ",nl]), emit([indent(8),"{",{asis,Cname},", {'", asn1ct_gen:list2name([Cname|TopType]),"',", @@ -928,7 +927,25 @@ gen_dec_choice_cases(Erules,TopType, [H|T]) -> end, gen_dec_choice_cases(Erules,TopType, T). +match_tag(Class, TagNo) when is_integer(TagNo) -> + match_tag1(asn1ct_gen_ber_bin_v2:decode_class(Class), TagNo). +match_tag1(Class, TagNo) when TagNo =< 30 -> + io_lib:format("<<~p:2,_:1,~p:5,_/binary>>", [Class bsr 6,TagNo]); +match_tag1(Class, TagNo) -> + Octets = mk_object_val(TagNo), + io_lib:format("<<~p:2,_:1,31:5,~s,_/binary>>", [Class bsr 6,Octets]). + +mk_object_val(Val) when Val < 16#80 -> + integer_to_list(Val); +mk_object_val(Val) -> + mk_object_val(Val bsr 7, [integer_to_list(Val band 16#7F)]). + +mk_object_val(0, Acc) -> + Acc; +mk_object_val(Val, Acc) -> + I = integer_to_list((Val band 16#7F) bor 16#80), + mk_object_val(Val bsr 7, [I,","|Acc]). %%--------------------------------------- %% Generate the encode/decode code @@ -1493,10 +1510,6 @@ mkfuncname(TopType,Cname,WhatKind,Prefix,Suffix) -> end. empty_lb(ber) -> - "[]"; -empty_lb(ber_bin) -> - "<<>>"; -empty_lb(ber_bin_v2) -> "<<>>". value_match(Index,Value) when is_atom(Value) -> diff --git a/lib/asn1/src/asn1ct_constructed_per.erl b/lib/asn1/src/asn1ct_constructed_per.erl index 8de41a4dd4..b29a7b3048 100644 --- a/lib/asn1/src/asn1ct_constructed_per.erl +++ b/lib/asn1/src/asn1ct_constructed_per.erl @@ -76,13 +76,9 @@ gen_encode_constructed(Erule,Typename,D) when is_record(D,type) -> case {Optionals = optionals(to_textual_order(CompList)),CompList, is_optimized(Erule)} of {[],EmptyCL,_} when EmptyCL == {[],[],[]};EmptyCL == {[],[]};EmptyCL == [] -> - emit(["%%Variable setting just to eliminate ", - "compiler warning for unused vars!",nl, - "_Val = ",{curr,val},",",nl]); + ok; {[],_,_} -> - emit([{next,val}," = ?RT_PER:list_to_record("]), - emit(["'",asn1ct_gen:list2rname(Typename),"'"]), - emit([", ",{curr,val},"),",nl]); + emit([{next,val}," = ",{curr,val},",",nl]); {_,_,true} -> gen_fixoptionals(Optionals), FixOpts = param_map(fun(Var) -> @@ -155,7 +151,7 @@ gen_encode_constructed(Erule,Typename,D) when is_record(D,type) -> emit([ObjectEncode," = ",nl]), emit([" ",ObjSetMod,":'getenc_",ObjSetName,"'(", {asis,UniqueFieldName},", ",nl]), - El = make_element(N+1,asn1ct_gen:mk_var(asn1ct_name:curr(val)),AttrN), + El = make_element(N+1,asn1ct_gen:mk_var(asn1ct_name:curr(val))), Length = fun(X,_LFun) when is_atom(X) -> length(atom_to_list(X)); @@ -221,9 +217,74 @@ gen_decode_set(Erules,Typename,D) -> gen_decode_sequence(Erules,Typename,D) -> gen_decode_constructed(Erules,Typename,D). -gen_decode_constructed(Erules,Typename,D) when is_record(D,type) -> +gen_decode_constructed(Erule, Typename, #type{}=D) -> + Imm0 = gen_dec_constructed_imm(Erule, Typename, #type{}=D), + Imm = opt_imm(Imm0), asn1ct_name:start(), asn1ct_name:clear(), + emit_gen_dec_imm(Imm), + emit([".",nl,nl]). + +opt_imm(Imm0) -> + {Imm,_} = opt_imm_1(Imm0, unknown, []), + Imm. + +opt_imm_1([{imm,Imm0,F}|T], Al0, Acc) -> + {Imm,Al} = asn1ct_imm:optimize_alignment(Imm0, Al0), + opt_imm_1(T, Al, [{imm,Imm,F}|Acc]); +opt_imm_1([ignore|T], Al, Acc) -> + opt_imm_1(T, Al, Acc); +opt_imm_1([{ignore,_}=H|T], Al, Acc) -> + opt_imm_1(T, Al, [H|Acc]); +opt_imm_1([{safe,ignore}|T], Al, Acc) -> + opt_imm_1(T, Al, Acc); +opt_imm_1([{safe,_}=H|T], Al, Acc) -> + opt_imm_1(T, Al, [H|Acc]); +opt_imm_1([{group,G0}|T], Al0, Acc) -> + {G,Al} = opt_imm_1(G0, Al0, []), + opt_imm_1(T, Al, [{group,G}|Acc]); +opt_imm_1([Emit|T], _, Acc) when is_function(Emit, 1) -> + opt_imm_1(T, unknown, [Emit|Acc]); +opt_imm_1([], Al, Acc) -> + {lists:reverse(Acc),Al}. + +emit_gen_dec_imm(L) -> + emit_gen_dec_imm(L, "", []). + +emit_gen_dec_imm([{ignore,Fun}|T], Sep, St0) -> + St = Fun(St0), + emit_gen_dec_imm(T, Sep, St); +emit_gen_dec_imm([{group,L}|T], Sep, St0) -> + emit(Sep), + St = emit_gen_dec_imm_group(L, St0), + emit_gen_dec_imm(T, [com,nl], St); +emit_gen_dec_imm([{imm,Imm,Emit}|T], Sep, St0) -> + emit(Sep), + St = Emit(Imm, St0), + emit_gen_dec_imm(T, [com,nl], St); +emit_gen_dec_imm([{safe,Item}|T], Sep, St) -> + emit_gen_dec_imm([Item|T], Sep, St); +emit_gen_dec_imm([Emit|T], Sep, St0) -> + emit(Sep), + St = Emit(St0), + emit_gen_dec_imm(T, [com,nl], St); +emit_gen_dec_imm([], _, _) -> ok. + +emit_gen_dec_imm_group([H|T], St0) -> + St = emit_gen_dec_group_item(H, St0), + emit_gen_dec_imm_group(T, St); +emit_gen_dec_imm_group([], St) -> St. + +emit_gen_dec_group_item({ignore,Fun}, St) -> + Fun(St); +emit_gen_dec_group_item({imm,Imm,Fun}, St) -> + Fun(Imm, St); +emit_gen_dec_group_item({safe,Item}, St) -> + emit_gen_dec_group_item(Item, St); +emit_gen_dec_group_item(Emit, St) -> + Emit(St). + +gen_dec_constructed_imm(Erule, Typename, #type{}=D) -> {CompList,TableConsInfo} = case D#type.def of #'SEQUENCE'{tablecinf=TCI,components=CL} -> @@ -233,27 +294,19 @@ gen_decode_constructed(Erules,Typename,D) when is_record(D,type) -> {CL,TCI} % the textual order is already taken care of end, Ext = extensible_dec(CompList), - MaybeComma1 = case Ext of - {ext,_Pos,_NumExt} -> - gen_dec_extension_value("Bytes"), - {",",nl}; - _ -> - "" - end, + EmitExt = case Ext of + {ext,_Pos,_NumExt} -> + gen_dec_extension_value(); + _ -> ignore + end, Optionals = optionals(CompList), - MaybeComma2 = case Optionals of - [] -> MaybeComma1; - _ -> - Bcurr = asn1ct_name:curr(bytes), - Bnext = asn1ct_name:next(bytes), - emit(MaybeComma1), - GetoptCall = "} = ?RT_PER:getoptionals2(", - emit({"{Opt,",{var,Bnext},GetoptCall, - {var,Bcurr},",",{asis,length(Optionals)},")"}), - asn1ct_name:new(bytes), - ", " - end, - {DecObjInf,UniqueFName,ValueIndex} = + EmitOpt = case Optionals of + [] -> + ignore; + [_|_] -> + gen_dec_optionals(Optionals) + end, + ObjSetInfo = case TableConsInfo of %% {ObjectSet,AttrN,N,UniqueFieldName} ->%% N is index of attribute that determines constraint #simpletableattributes{objectsetname=ObjectSet, @@ -285,13 +338,19 @@ gen_decode_constructed(Erules,Typename,D) when is_record(D,type) -> {false,false,false} end end, -%% NewCompList = wrap_compList(CompList), - {AccTerm,AccBytes} = - gen_dec_components_call(Erules,Typename,CompList,MaybeComma2,DecObjInf,Ext,length(Optionals)), - case asn1ct_name:all(term) of - [] -> emit(MaybeComma2); % no components at all - _ -> emit({com,nl}) - end, + {DecObjInf,_,_} = ObjSetInfo, + EmitComp = gen_dec_components_call(Erule, Typename, CompList, + DecObjInf, Ext, length(Optionals)), + EmitRest = fun({AccTerm,AccBytes}) -> + gen_dec_constructed_imm_2(Typename, CompList, + ObjSetInfo, + AccTerm, AccBytes) + end, + [EmitExt,EmitOpt|EmitComp++[{safe,EmitRest}]]. + +gen_dec_constructed_imm_2(Typename, CompList, + ObjSetInfo, AccTerm, AccBytes) -> + {_,UniqueFName,ValueIndex} = ObjSetInfo, case {AccTerm,AccBytes} of {[],[]} -> ok; @@ -333,8 +392,7 @@ gen_decode_constructed(Erules,Typename,D) when is_record(D,type) -> mkvlist(textual_order(to_encoding_order(CompList),asn1ct_name:all(term))), emit("},") end, - emit({{curr,bytes},"}"}), - emit({".",nl,nl}). + emit({{curr,bytes},"}"}). textual_order([#'ComponentType'{textual_order=undefined}|_],TermList) -> TermList; @@ -516,10 +574,10 @@ gen_decode_sof(Erules,Typename,SeqOrSetOf,D) when is_record(D,type) -> _ -> "" end, - gen_decode_length(SizeConstraint, - is_optimized(Erules)), - emit({"'dec_",asn1ct_gen:list2name(Typename), - "_components'(Num, Bytes1, telltype",ObjFun,", []).",nl}), + {Num,Buf} = gen_decode_length(SizeConstraint, Erules), + emit([",",nl, + "'dec_",asn1ct_gen:list2name(Typename), + "_components'(",Num,", ",Buf,ObjFun,", []).",nl,nl]), NewComponentType = case ComponentType#type.def of {'ENUMERATED',_,Component}-> @@ -528,40 +586,13 @@ gen_decode_sof(Erules,Typename,SeqOrSetOf,D) when is_record(D,type) -> end, gen_decode_sof_components(Erules,Typename,SeqOrSetOf,NewComponentType). -%% Logic copied from asn1_per_bin_rt2ct:decode_constrained_number -gen_decode_length({Lb,Ub},true) when Ub =< 65535, Lb >= 0 -> - Range = Ub - Lb + 1, - Call = if - Range == 1 -> - "{0,Bytes}"; - Range == 2 -> - "?RT_PER:getbits(Bytes,1)"; - Range =< 4 -> - "?RT_PER:getbits(Bytes,2)"; - Range =< 8 -> - "?RT_PER:getbits(Bytes,3)"; - Range =< 16 -> - "?RT_PER:getbits(Bytes,4)"; - Range =< 32 -> - "?RT_PER:getbits(Bytes,5)"; - Range =< 64 -> - "?RT_PER:getbits(Bytes,6)"; - Range =< 128 -> - "?RT_PER:getbits(Bytes,7)"; - Range =< 255 -> - "?RT_PER:getbits(Bytes,8)"; - Range =< 256 -> - "?RT_PER:getoctets(Bytes,1)"; - Range =< 65536 -> - "?RT_PER:getoctets(Bytes,2)"; - true -> - ["exit({not_supported,{integer_range,",Range,"}}"] - end, - emit({nl,"{Val,Remain} = ",Call,",",nl}), - emit({nl,"{Num,Bytes1} = {Val+",Lb,",Remain},",nl}); -gen_decode_length(SizeConstraint,_) -> - emit({nl,"{Num,Bytes1} = ?RT_PER:decode_length(Bytes,", - {asis,SizeConstraint},"),",nl}). +is_aligned(per) -> true; +is_aligned(uper) -> false. + +gen_decode_length(Constraint, Erule) -> + emit(["%% Length with constraint ",{asis,Constraint},nl]), + Imm = asn1ct_imm:per_dec_length(Constraint, true, is_aligned(Erule)), + asn1ct_imm:dec_slim_cg(Imm, "Bytes"). gen_encode_sof_components(Erule,Typename,SeqOrSetOf,Cont) -> {ObjFun,ObjFun_Var} = @@ -583,8 +614,7 @@ gen_encode_sof_components(Erule,Typename,SeqOrSetOf,Cont) -> Conttype = asn1ct_gen:get_inner(Cont#type.def), Currmod = get(currmod), - Ctgenmod = list_to_atom(lists:concat(["asn1ct_gen_",per, - asn1ct_gen:rt2ct_suffix()])), + Ctgenmod = asn1ct_gen:ct_gen_module(Erule), case asn1ct_gen:type(Conttype) of {primitive,bif} -> gen_encode_prim_wrapper(Ctgenmod,Erule,Cont,false,"H"); @@ -614,16 +644,15 @@ gen_decode_sof_components(Erule,Typename,SeqOrSetOf,Cont) -> {"",""} end, emit({"'dec_",asn1ct_gen:list2name(Typename), - "_components'(0, Bytes, _",ObjFun_Var,", Acc) ->",nl, + "_components'(0, Bytes",ObjFun_Var,", Acc) ->",nl, indent(3),"{lists:reverse(Acc), Bytes};",nl}), emit({"'dec_",asn1ct_gen:list2name(Typename), - "_components'(Num, Bytes, _",ObjFun,", Acc) ->",nl}), + "_components'(Num, Bytes",ObjFun,", Acc) ->",nl}), emit({indent(3),"{Term,Remain} = "}), Constructed_Suffix = asn1ct_gen:constructed_suffix(SeqOrSetOf, Cont#type.def), Conttype = asn1ct_gen:get_inner(Cont#type.def), - Ctgenmod = list_to_atom(lists:concat(["asn1ct_gen_",per, - asn1ct_gen:rt2ct_suffix()])), + Ctgenmod = asn1ct_gen:ct_gen_module(Erule), CurrMod = get(currmod), case asn1ct_gen:type(Conttype) of {primitive,bif} -> @@ -647,7 +676,7 @@ gen_decode_sof_components(Erule,Typename,SeqOrSetOf,Cont) -> emit({"'dec_",Conttype,"'(Bytes,telltype),",nl}) end, emit({indent(3),"'dec_",asn1ct_gen:list2name(Typename), - "_components'(Num-1, Remain, telltype",ObjFun,", [Term|Acc]).",nl}). + "_components'(Num-1, Remain",ObjFun,", [Term|Acc]).",nl}). %%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%% @@ -718,10 +747,25 @@ extgrouppos([_|T],ActualPos,VirtualPos,Len,Acc) -> extgrouppos(T,ActualPos,VirtualPos,Len+1,Acc). - -gen_dec_extension_value(_) -> - emit({"{Ext,",{next,bytes},"} = ?RT_PER:getext(",{curr,bytes},")"}), - asn1ct_name:new(bytes). +gen_dec_extension_value() -> + Imm0 = {get_bits,1,[1]}, + E = fun(Imm, _) -> + emit(["{Ext,",{next,bytes},"} = "]), + BytesVar = asn1ct_gen:mk_var(asn1ct_name:curr(bytes)), + asn1ct_imm:dec_code_gen(Imm, BytesVar), + asn1ct_name:new(bytes) + end, + {imm,Imm0,E}. + +gen_dec_optionals(Optionals) -> + Imm0 = {get_bits,length(Optionals),[1]}, + E = fun(Imm, _) -> + BytesVar = asn1ct_gen:mk_var(asn1ct_name:curr(bytes)), + emit(["{Opt,",{next,bytes},"} = "]), + asn1ct_imm:dec_code_gen(Imm, BytesVar), + asn1ct_name:new(bytes) + end, + {imm,Imm0,E}. gen_fixoptionals([{Pos,Def}|R]) -> asn1ct_name:new(fixopt), @@ -889,7 +933,7 @@ gen_enc_components_call1(_Erule,_TopType,[],Pos,_,_,_) -> Pos. gen_enc_component_default(Erule,TopType,Cname,Type,Pos,DynamicEnc,Ext,DefaultVal) -> - Element = make_element(Pos+1,asn1ct_gen:mk_var(asn1ct_name:curr(val)),Cname), + Element = make_element(Pos+1,asn1ct_gen:mk_var(asn1ct_name:curr(val))), emit({"case ",Element," of",nl}), % emit({"asn1_DEFAULT -> [];",nl}), emit({"DFLT when DFLT == asn1_DEFAULT; DFLT == ",{asis,DefaultVal}," -> [];",nl}), @@ -909,7 +953,7 @@ gen_enc_component_optional(Erule,TopType,Cname, components=_ExtGroupCompList}}, Pos,DynamicEnc,Ext) when is_integer(Number) -> - Element = make_element(Pos+1,asn1ct_gen:mk_var(asn1ct_name:curr(val)),Cname), + Element = make_element(Pos+1,asn1ct_gen:mk_var(asn1ct_name:curr(val))), emit({"case ",Element," of",nl}), emit({"asn1_NOVALUE -> [];",nl}), @@ -922,7 +966,7 @@ gen_enc_component_optional(Erule,TopType,Cname, gen_enc_line(Erule,TopType,Cname,Type,NextElement, Pos,DynamicEnc,Ext), emit({nl,"end"}); gen_enc_component_optional(Erule,TopType,Cname,Type,Pos,DynamicEnc,Ext) -> - Element = make_element(Pos+1,asn1ct_gen:mk_var(asn1ct_name:curr(val)),Cname), + Element = make_element(Pos+1,asn1ct_gen:mk_var(asn1ct_name:curr(val))), emit({"case ",Element," of",nl}), emit({"asn1_NOVALUE -> [];",nl}), @@ -942,11 +986,10 @@ gen_enc_component_mandatory(Erule,TopType,Cname,Type,Pos,DynamicEnc,Ext) -> gen_enc_line(Erule,TopType,Cname,Type,[],Pos,DynamicEnc,Ext). gen_enc_line(Erule,TopType, Cname, Type, [], Pos,DynamicEnc,Ext) -> - Element = make_element(Pos+1,asn1ct_gen:mk_var(asn1ct_name:curr(val)),Cname), + Element = make_element(Pos+1,asn1ct_gen:mk_var(asn1ct_name:curr(val))), gen_enc_line(Erule,TopType,Cname,Type,Element, Pos,DynamicEnc,Ext); gen_enc_line(Erule,TopType,Cname,Type,Element, _Pos,DynamicEnc,Ext) -> - Ctgenmod = list_to_atom(lists:concat(["asn1ct_gen_",per, - asn1ct_gen:rt2ct_suffix()])), + Ctgenmod = asn1ct_gen:ct_gen_module(Erule), Atype = case Type of #type{def=#'ObjectClassFieldType'{type=InnerType}} -> @@ -1036,53 +1079,79 @@ gen_enc_line(Erule,TopType,Cname,Type,Element, _Pos,DynamicEnc,Ext) -> emit("))"); _ -> true end. -gen_dec_components_call(Erule,TopType,{Root,ExtList},MaybeComma, + +gen_dec_components_call(Erule, TopType, {Root,ExtList}, + DecInfObj, Ext, NumberOfOptionals) -> + gen_dec_components_call(Erule,TopType,{Root,ExtList,[]}, + DecInfObj,Ext,NumberOfOptionals); +gen_dec_components_call(Erule,TopType,CL={Root1,ExtList,Root2}, DecInfObj,Ext,NumberOfOptionals) -> - gen_dec_components_call(Erule,TopType,{Root,ExtList,[]},MaybeComma,DecInfObj,Ext,NumberOfOptionals); -gen_dec_components_call(Erule,TopType,CL={Root1,ExtList,Root2},MaybeComma,DecInfObj,Ext,NumberOfOptionals) -> %% The type has extensionmarker - OptTable = create_optionality_table(Root1++Root2), - {Rpos,AccTerm,AccBytes} = - gen_dec_components_call1(Erule,TopType, Root1++Root2, 1, OptTable, - MaybeComma,DecInfObj,noext,[],[], - NumberOfOptionals), - emit([",",nl,"{Extensions,",{next,bytes},"} = "]), - emit(["?RT_PER:getextension(Ext,",{curr,bytes},"),",nl]), - asn1ct_name:new(bytes), + Init = {ignore,fun(_) -> {[],[]} end}, + {EmitRoot,Tpos} = + gen_dec_comp_calls(Root1++Root2, Erule, TopType, OptTable, + DecInfObj, noext, NumberOfOptionals, + 1, []), + EmitGetExt = gen_dec_get_extension(Erule), {extgrouppos,ExtGroupPosLen} = extgroup_pos_and_length(CL), - NewExtList = wrap_extensionAdditionGroups(ExtList,ExtGroupPosLen), - {_Epos,AccTermE,AccBytesE} = - gen_dec_components_call1(Erule,TopType,NewExtList,Rpos, OptTable, - "",DecInfObj,Ext,[],[],NumberOfOptionals), - case ExtList of - [] -> true; - _ -> emit([",",nl]) - end, - emit([{next,bytes},"= ?RT_PER:skipextensions(",{curr,bytes},",", - length(ExtList)+1,",Extensions)",nl]), - asn1ct_name:new(bytes), - {AccTerm++AccTermE,AccBytes++AccBytesE}; - -gen_dec_components_call(Erule,TopType,CompList,MaybeComma,DecInfObj, - Ext,NumberOfOptionals) -> + NewExtList = wrap_extensionAdditionGroups(ExtList, ExtGroupPosLen), + {EmitExts,_} = gen_dec_comp_calls(NewExtList, Erule, TopType, OptTable, + DecInfObj, Ext, NumberOfOptionals, + Tpos, []), + Finish = + fun(St) -> + emit([{next,bytes},"= ?RT_PER:skipextensions(",{curr,bytes},",", + length(ExtList)+1,",Extensions)"]), + asn1ct_name:new(bytes), + St + end, + [Init] ++ EmitRoot ++ [EmitGetExt|EmitExts] ++ [Finish]; +gen_dec_components_call(Erule, TopType, CompList, DecInfObj, + Ext, NumberOfOptionals) -> %% The type has no extensionmarker OptTable = create_optionality_table(CompList), - {_,AccTerm,AccBytes} = - gen_dec_components_call1(Erule,TopType, CompList, 1, OptTable, - MaybeComma,DecInfObj,Ext,[],[], - NumberOfOptionals), - {AccTerm,AccBytes}. - - -gen_dec_components_call1(Erule,TopType, - [C=#'ComponentType'{name=Cname,typespec=Type,prop=Prop,textual_order=TextPos}|Rest], - Tpos,OptTable,MaybeComma,DecInfObj,Ext,AccTerm,AccBytes,NumberOfOptionals) -> + Init = {ignore,fun(_) -> {[],[]} end}, + {Cs,_} = gen_dec_comp_calls(CompList, Erule, TopType, OptTable, + DecInfObj, Ext, NumberOfOptionals, + 1, []), + [Init|Cs]. + +gen_dec_get_extension(Erule) -> + Imm0 = asn1ct_imm:per_dec_extension_map(is_aligned(Erule)), + E = fun(Imm, St) -> + emit([nl,"%% Extensions", + nl, + "{Extensions,",{next,bytes},"} = ", + "case Ext of",nl, + "0 -> {<<>>,",{curr,bytes},"};",nl, + "1 ->",nl]), + BytesVar = asn1ct_gen:mk_var(asn1ct_name:curr(bytes)), + {Dst,DstBuf} = asn1ct_imm:dec_slim_cg(Imm, BytesVar), + emit([com,nl, + "{",Dst,",",DstBuf,"}",nl, + "end"]), + asn1ct_name:new(bytes), + St + end, + {imm,Imm0,E}. + +gen_dec_comp_calls([C|Cs], Erule, TopType, OptTable, DecInfObj, + Ext, NumberOfOptionals, Tpos, Acc) -> + L = gen_dec_comp_call(C, Erule, TopType, Tpos, OptTable, DecInfObj, + Ext, NumberOfOptionals), + gen_dec_comp_calls(Cs, Erule, TopType, OptTable, DecInfObj, + Ext, NumberOfOptionals, Tpos+1, [L|Acc]); +gen_dec_comp_calls([], _, _, _, _, _, _, Tpos, Acc) -> + {lists:append(lists:reverse(Acc)),Tpos}. + +gen_dec_comp_call(Comp, Erule, TopType, Tpos, OptTable, DecInfObj, + Ext, NumberOfOptionals) -> + #'ComponentType'{typespec=Type,prop=Prop,textual_order=TextPos} = Comp, Pos = case Ext of noext -> Tpos; {ext,Epos,_Enum} -> Tpos - Epos + 1 end, - emit(MaybeComma), InnerType = case Type#type.def of #'ObjectClassFieldType'{type=InType} -> @@ -1091,109 +1160,128 @@ gen_dec_components_call1(Erule,TopType, asn1ct_gen:get_inner(Def) end, - case InnerType of - #'Externaltypereference'{type=T} -> - emit({nl,"%% attribute number ",TextPos," with type ", - T,nl}); - IT when is_tuple(IT) -> - emit({nl,"%% attribute number ",TextPos," with type ", - element(2,IT),nl}); - _ -> - emit({nl,"%% attribute number ",TextPos," with type ", - InnerType,nl}) - end, - - IsMandatoryAndPredefinedTableC = - fun(noext,mandatory,{"got objfun through args","ObjFun"}) -> - true; - (_,_,{"got objfun through args","ObjFun"}) -> - false; - (_,_,_) -> - true - end, - case {InnerType,IsMandatoryAndPredefinedTableC(Ext,Prop,DecInfObj)} of -%% {typefield,_} when Ext == noext, Prop == mandatory -> - {{typefield,_},true} -> - %% DecInfObj /= {"got objfun through args","ObjFun"} | - %% (DecInfObj == {"got objfun through args","ObjFun"} & - %% Ext == noext & Prop == mandatory) - asn1ct_name:new(term), - asn1ct_name:new(tmpterm), - emit({"{",{curr,tmpterm},", ",{next,bytes},"} = "}); + DispType = case InnerType of + #'Externaltypereference'{type=T} -> T; + IT when is_tuple(IT) -> element(2,IT); + _ -> InnerType + end, + Comment = fun(St) -> + emit([nl,"%% attribute number ",TextPos, + " with type ",DispType,nl]), + St + end, + + Preamble = + case {InnerType,is_mandatory_predef_tab_c(Ext, Prop, DecInfObj)} of + {{typefield,_},true} -> + %% DecInfObj /= {"got objfun through args","ObjFun"} | + %% (DecInfObj == {"got objfun through args","ObjFun"} & + %% Ext == noext & Prop == mandatory) + fun(St) -> + asn1ct_name:new(term), + asn1ct_name:new(tmpterm), + emit(["{",{curr,tmpterm},", ",{next,bytes},"} = "]), + St + end; %%{objectfield,_,_} when Ext == noext, Prop == mandatory -> - {{objectfield,_,_},true} -> - asn1ct_name:new(term), - asn1ct_name:new(tmpterm), - emit({"{",{curr,tmpterm},", ",{next,bytes},"} = "}); + {{objectfield,_,_},true} -> + fun(St) -> + asn1ct_name:new(term), + asn1ct_name:new(tmpterm), + emit(["{",{curr,tmpterm},", ",{next,bytes},"} = "]), + St + end; _ -> case Type of #type{def=#'SEQUENCE'{ extaddgroup=Number1, components=ExtGroupCompList1}} when is_integer(Number1)-> - emit({"{{_,"}), - emit_extaddgroupTerms(term,ExtGroupCompList1), - emit({"}"}); - _ -> - asn1ct_name:new(term), - emit({"{",{curr,term}}) - end, - emit({",",{next,bytes},"} = "}) - end, - - case {Ext,Prop,is_optimized(Erule)} of - {noext,mandatory,_} -> ok; % generate nothing - {noext,_,_} -> %% OPTIONAL or DEFAULT - OptPos = get_optionality_pos(TextPos,OptTable), - Element = io_lib:format("Opt band (1 bsl ~w)",[NumberOfOptionals - OptPos]), - emit(["case ",Element," of",nl]), - emit([" _Opt",TextPos," when _Opt",TextPos," > 0 ->"]); - {_,_,false} -> %% extension element, not bitstring - emit(["case Extensions of",nl]), - emit([" _ when size(Extensions) >= ",Pos,",element(",Pos,",Extensions) == 1 ->",nl]); - _ -> - emit(["case Extensions of",nl]), - emit([" <<_:",Pos-1,",1:1,_/bitstring>> when bit_size(Extensions) >= ",Pos," ->",nl]) - end, - put(component_type,{true,C}), - {TermVar,BytesVar} = gen_dec_line(Erule,TopType,Cname,Type,Tpos,DecInfObj,Ext,Prop), - erase(component_type), - case {Ext,Prop} of - {noext,mandatory} -> true; % generate nothing - {noext,_} -> - emit([";",nl,"0 ->"]), - emit(["{"]), - gen_dec_component_no_val(Ext,Prop), - emit({",",{curr,bytes},"}",nl}), - emit([nl,"end"]); - _ -> - emit([";",nl,"_ ->",nl]), - emit(["{"]), - case Type of - #type{def=#'SEQUENCE'{ - extaddgroup=Number2, - components=ExtGroupCompList2}} when is_integer(Number2)-> - emit({"{extAddGroup,"}), - gen_dec_extaddGroup_no_val(Ext,ExtGroupCompList2), - emit({"}"}); + fun(St) -> + emit(["{{_,"]), + emit_extaddgroupTerms(term,ExtGroupCompList1), + emit(["}"]), + emit([",",{next,bytes},"} = "]), + St + end; _ -> - gen_dec_component_no_val(Ext,Prop) - end, - emit({",",{curr,bytes},"}",nl}), - emit([nl,"end"]) - end, - asn1ct_name:new(bytes), - case Rest of - [] -> - {Tpos+1,AccTerm++TermVar,AccBytes++BytesVar}; - _ -> - emit({com,nl}), - gen_dec_components_call1(Erule,TopType,Rest,Tpos+1,OptTable, - "",DecInfObj,Ext, AccTerm++TermVar, - AccBytes++BytesVar,NumberOfOptionals) - end; + fun(St) -> + asn1ct_name:new(term), + emit(["{",{curr,term}]), + emit([",",{next,bytes},"} = "]), + St + end + end + end, -gen_dec_components_call1(_,_TopType,[],Pos,_OptTable,_,_,_,AccTerm,AccBytes,_NumberOfOptionals) -> - {Pos,AccTerm,AccBytes}. + OptOrDef = + case {Ext,Prop} of + {noext,mandatory} -> + ignore; + {noext,_} -> %% OPTIONAL or DEFAULT + OptPos = get_optionality_pos(TextPos, OptTable), + Element = io_lib:format("Opt band (1 bsl ~w)", + [NumberOfOptionals - OptPos]), + fun(St) -> + emit(["case ",Element," of",nl]), + emit([" _Opt",TextPos," when _Opt",TextPos," > 0 ->"]), + St + end; + {{ext,_,_},_} -> %Extension + fun(St) -> + emit(["case Extensions of",nl, + " <<_:",Pos-1,",1:1,_/bitstring>> ->",nl]), + St + end + end, + Lines = gen_dec_line_imm(Erule, TopType, Comp, Tpos, DecInfObj, Ext), + Postamble = + case {Ext,Prop} of + {noext,mandatory} -> + ignore; + {noext,_} -> + fun(St) -> + emit([";",nl,"0 ->"]), + emit(["{"]), + gen_dec_component_no_val(Ext,Prop), + emit({",",{curr,bytes},"}",nl}), + emit([nl,"end"]), + St + end; + _ -> + fun(St) -> + emit([";",nl,"_ ->",nl]), + emit(["{"]), + case Type of + #type{def=#'SEQUENCE'{ + extaddgroup=Number2, + components=ExtGroupCompList2}} + when is_integer(Number2)-> + emit({"{extAddGroup,"}), + gen_dec_extaddGroup_no_val(Ext,ExtGroupCompList2), + emit({"}"}); + _ -> + gen_dec_component_no_val(Ext, Prop) + end, + emit({",",{curr,bytes},"}",nl}), + emit([nl,"end"]), + St + end + end, + AdvBuffer = {ignore,fun(St) -> + asn1ct_name:new(bytes), + St + end}, + [{group,[{safe,Comment},{safe,Preamble}, + {safe,OptOrDef}|Lines]++ + [{safe,Postamble},{safe,AdvBuffer}]}]. + +is_mandatory_predef_tab_c(noext, mandatory, + {"got objfun through args","ObjFun"}) -> + true; +is_mandatory_predef_tab_c(_, _, {"got objfun through args","ObjFun"}) -> + false; +is_mandatory_predef_tab_c(_,_,_) -> + true. gen_dec_extaddGroup_no_val(Ext,[#'ComponentType'{prop=Prop}])-> gen_dec_component_no_val(Ext,Prop), @@ -1213,9 +1301,14 @@ gen_dec_component_no_val({ext,_,_},mandatory) -> emit({"asn1_NOVALUE"}). -gen_dec_line(Erule,TopType,Cname,Type,Pos,DecInfObj,Ext,Prop) -> - Ctgenmod = list_to_atom(lists:concat(["asn1ct_gen_",per, - asn1ct_gen:rt2ct_suffix()])), +gen_dec_line(Erule, TopType, Comp, Pos, DecInfObj, Ext) -> + Imm0 = gen_dec_line_imm(Erule, TopType, Comp, Pos, DecInfObj, Ext), + Init = {ignore,fun(_) -> {[],[]} end}, + Imm = [{group,[Init|Imm0]}], + emit_gen_dec_imm(Imm). + +gen_dec_line_imm(Erule, TopType, Comp, Pos, DecInfObj, Ext) -> + #'ComponentType'{name=Cname,typespec=Type} = Comp, Atype = case Type of #type{def=#'ObjectClassFieldType'{type=InnerType}} -> @@ -1224,175 +1317,235 @@ gen_dec_line(Erule,TopType,Cname,Type,Pos,DecInfObj,Ext,Prop) -> asn1ct_gen:get_inner(Type#type.def) end, - BytesVar0 = asn1ct_gen:mk_var(asn1ct_name:curr(bytes)), - BytesVar = case Ext of - {ext,Ep,_} when Pos >= Ep -> - emit(["begin",nl,"{TmpVal",Pos,",Trem",Pos, - "}=?RT_PER:decode_open_type(", - {curr,bytes},",[]),",nl, - "{TmpValx",Pos,",_}="]), - io_lib:format("TmpVal~p",[Pos]); - _ -> BytesVar0 - end, - SaveBytes = - case Atype of - {typefield,_} -> - case DecInfObj of - false -> % This is in a choice with typefield components - {Name,RestFieldNames} = - (Type#type.def)#'ObjectClassFieldType'.fieldname, - - asn1ct_name:new(tmpterm), - asn1ct_name:new(reason), - emit([indent(2),"{",{curr,tmpterm},", ",{next,bytes}, - "} = ?RT_PER:decode_open_type(",{curr,bytes}, - ", []),",nl]), - emit([indent(2),"case (catch ObjFun(", - {asis,Name},",",{curr,tmpterm},",telltype,", - {asis,RestFieldNames},")) of", nl]), - emit([indent(4),"{'EXIT',",{curr,reason},"} ->",nl]), - emit([indent(6),"exit({'Type not ", - "compatible with table constraint', ", - {curr,reason},"});",nl]), - asn1ct_name:new(tmpterm), - emit([indent(4),"{",{curr,tmpterm},", _} ->",nl]), - emit([indent(6),"{",{asis,Cname},", {",{curr,tmpterm},", ", - {next,bytes},"}}",nl]), - emit([indent(2),"end"]), - []; - {"got objfun through args","ObjFun"} -> - %% this is when the generated code gots the - %% objfun though arguments on function - %% invocation. - if - Ext == noext andalso Prop == mandatory -> - ok; - true -> - asn1ct_name:new(tmpterm), - asn1ct_name:new(tmpbytes), - emit([nl," {",{curr,tmpterm},", ",{curr,tmpbytes},"} ="]) - end, - {Name,RestFieldNames} = - (Type#type.def)#'ObjectClassFieldType'.fieldname, - emit(["?RT_PER:decode_open_type(",{curr,bytes}, - ", []),",nl]), - if - Ext == noext andalso Prop == mandatory -> - emit([{curr,term}," =",nl," "]); - true -> - emit([" {"]) - end, - emit(["case (catch ObjFun(",{asis,Name},",", - {curr,tmpterm},",telltype,", - {asis,RestFieldNames},")) of", nl]), - emit([" {'EXIT',",{curr,reason},"} ->",nl]), - emit([indent(6),"exit({'Type not ", - "compatible with table constraint', ", - {curr,reason},"});",nl]), - asn1ct_name:new(tmpterm), - emit([indent(4),"{",{curr,tmpterm},", _} ->",nl]), - emit([indent(6),{curr,tmpterm},nl]), - emit([indent(2),"end"]), - if - Ext == noext andalso Prop == mandatory -> - ok; - true -> - emit([",",nl,{curr,tmpbytes},"}"]) - end, - []; - _ -> - emit(["?RT_PER:decode_open_type(",{curr,bytes}, - ", [])"]), - RefedFieldName = - (Type#type.def)#'ObjectClassFieldType'.fieldname, - - [{Cname,RefedFieldName, - asn1ct_gen:mk_var(asn1ct_name:curr(term)), - asn1ct_gen:mk_var(asn1ct_name:curr(tmpterm)), - get_components_prop()}] - end; - {objectfield,PrimFieldName1,PFNList} -> - emit(["?RT_PER:decode_open_type(",{curr,bytes},", [])"]), - [{Cname,{PrimFieldName1,PFNList}, - asn1ct_gen:mk_var(asn1ct_name:curr(term)), - asn1ct_gen:mk_var(asn1ct_name:curr(tmpterm)), - get_components_prop()}]; - _ -> - CurrMod = get(currmod), - case asn1ct_gen:type(Atype) of - #'Externaltypereference'{module=CurrMod,type=EType} -> - emit({"'dec_",EType,"'(",BytesVar,",telltype)"}); - #'Externaltypereference'{module=Mod,type=EType} -> - emit({"'",Mod,"':'dec_",EType,"'(",BytesVar, - ",telltype)"}); - {primitive,bif} -> - case Atype of - {fixedtypevaluefield,_,Btype} -> - Ctgenmod:gen_dec_prim(Erule,Btype, - BytesVar); - _ -> - Ctgenmod:gen_dec_prim(Erule,Type, - BytesVar) - end; - 'ASN1_OPEN_TYPE' -> - case Type#type.def of - #'ObjectClassFieldType'{type=OpenType} -> - Ctgenmod:gen_dec_prim(Erule,#type{def=OpenType}, - BytesVar); - _ -> - Ctgenmod:gen_dec_prim(Erule,Type, - BytesVar) - end; - #typereference{val=Dname} -> - emit({"'dec_",Dname,"'(",BytesVar,",telltype)"}); - {notype,_} -> - emit({"'dec_",Atype,"'(",BytesVar,",telltype)"}); - {constructed,bif} -> - NewTypename = [Cname|TopType], - case Type#type.tablecinf of - [{objfun,_}|_R] -> - emit({"'dec_",asn1ct_gen:list2name(NewTypename), - "'(",BytesVar,", telltype, ObjFun)"}); - _ -> - emit({"'dec_",asn1ct_gen:list2name(NewTypename), - "'(",BytesVar,", telltype)"}) - end - end, + Pre = gen_dec_line_open_type(Erule, Ext, Pos), + Decode = gen_dec_line_special(Erule, Atype, TopType, Comp, DecInfObj, Ext), + Post = + fun({SaveBytes,Finish}) -> + {AccTerm,AccBytes} = Finish(), + #'ComponentType'{name=Cname} = Comp, case DecInfObj of - {Cname,{_,OSet,UniqueFName,ValIndex}} -> - Term = asn1ct_gen:mk_var(asn1ct_name:curr(term)), - ValueMatch = value_match(ValIndex,Term), - {ObjSetMod,ObjSetName} = - case OSet of - {M,O} -> {{asis,M},O}; - _ -> {"?MODULE",OSet} + {Cname,ObjSet} -> + ObjSetRef = + case ObjSet of + {deep,OSName,_,_} -> + OSName; + _ -> ObjSet end, - emit({",",nl,"ObjFun = ",ObjSetMod, - ":'getdec_",ObjSetName,"'(", - {asis,UniqueFName},", ",ValueMatch,")"}); + {AccTerm++[{ObjSetRef,Cname, + asn1ct_gen:mk_var(asn1ct_name:curr(term))}], + AccBytes++SaveBytes}; _ -> - ok - end, - [] + {AccTerm,AccBytes++SaveBytes} + end end, - case Ext of - {ext,Ep2,_} when Pos >= Ep2 -> - emit([", {TmpValx",Pos,",Trem",Pos,"}",nl,"end"]); - _ -> true - end, - %% Prepare return value + [Pre,Decode,{safe,Post}]. + +gen_dec_line_open_type(Erule, {ext,Ep,_}, Pos) when Pos >= Ep -> + Imm = asn1ct_imm:per_dec_open_type(is_aligned(Erule)), + {safe,fun(St) -> + emit(["begin",nl]), + BytesVar = asn1ct_gen:mk_var(asn1ct_name:curr(bytes)), + {Dst,DstBuf} = asn1ct_imm:dec_slim_cg(Imm, BytesVar), + emit([",",nl,"{TmpValx",Pos,",_} = "]), + {Dst, + fun() -> + emit([",",nl, + "{TmpValx",Pos,",",DstBuf,"}",nl, + "end"]), + St + end} + end}; +gen_dec_line_open_type(_, _, _) -> + {safe,fun(St) -> + {asn1ct_gen:mk_var(asn1ct_name:curr(bytes)), + fun() -> St end} + end}. + +gen_dec_line_special(_, {typefield,_}, _TopType, Comp, + DecInfObj, Ext) -> + #'ComponentType'{name=Cname,typespec=Type,prop=Prop} = Comp, + fun({_BytesVar,PrevSt}) -> + case DecInfObj of + false -> % This is in a choice with typefield components + {Name,RestFieldNames} = + (Type#type.def)#'ObjectClassFieldType'.fieldname, + + asn1ct_name:new(tmpterm), + asn1ct_name:new(reason), + emit([indent(2),"{",{curr,tmpterm},", ",{next,bytes}, + "} = ?RT_PER:decode_open_type(",{curr,bytes}, + ", []),",nl]), + emit([indent(2),"case (catch ObjFun(", + {asis,Name},",",{curr,tmpterm},",telltype,", + {asis,RestFieldNames},")) of", nl]), + emit([indent(4),"{'EXIT',",{curr,reason},"} ->",nl]), + emit([indent(6),"exit({'Type not ", + "compatible with table constraint', ", + {curr,reason},"});",nl]), + asn1ct_name:new(tmpterm), + emit([indent(4),"{",{curr,tmpterm},", _} ->",nl]), + emit([indent(6),"{",{asis,Cname},", {",{curr,tmpterm},", ", + {next,bytes},"}}",nl]), + emit([indent(2),"end"]), + {[],PrevSt}; + {"got objfun through args","ObjFun"} -> + %% this is when the generated code gots the + %% objfun though arguments on function + %% invocation. + if + Ext == noext andalso Prop == mandatory -> + ok; + true -> + asn1ct_name:new(tmpterm), + asn1ct_name:new(tmpbytes), + emit([nl," {",{curr,tmpterm},", ",{curr,tmpbytes},"} ="]) + end, + {Name,RestFieldNames} = + (Type#type.def)#'ObjectClassFieldType'.fieldname, + emit(["?RT_PER:decode_open_type(",{curr,bytes}, + ", []),",nl]), + if + Ext == noext andalso Prop == mandatory -> + emit([{curr,term}," =",nl," "]); + true -> + emit([" {"]) + end, + emit(["case (catch ObjFun(",{asis,Name},",", + {curr,tmpterm},",telltype,", + {asis,RestFieldNames},")) of", nl]), + emit([" {'EXIT',",{curr,reason},"} ->",nl]), + emit([indent(6),"exit({'Type not ", + "compatible with table constraint', ", + {curr,reason},"});",nl]), + asn1ct_name:new(tmpterm), + emit([indent(4),"{",{curr,tmpterm},", _} ->",nl]), + emit([indent(6),{curr,tmpterm},nl]), + emit([indent(2),"end"]), + if + Ext == noext andalso Prop == mandatory -> + ok; + true -> + emit([",",nl,{curr,tmpbytes},"}"]) + end, + {[],PrevSt}; + _ -> + emit(["?RT_PER:decode_open_type(",{curr,bytes}, + ", [])"]), + RefedFieldName = + (Type#type.def)#'ObjectClassFieldType'.fieldname, + + {[{Cname,RefedFieldName, + asn1ct_gen:mk_var(asn1ct_name:curr(term)), + asn1ct_gen:mk_var(asn1ct_name:curr(tmpterm)), + Prop}],PrevSt} + end + end; +gen_dec_line_special(_, {objectfield,PrimFieldName1,PFNList}, _TopType, + Comp, _DecInfObj, _Ext) -> + fun({_BytesVar,PrevSt}) -> + emit(["?RT_PER:decode_open_type(",{curr,bytes},", [])"]), + #'ComponentType'{name=Cname,prop=Prop} = Comp, + SaveBytes = [{Cname,{PrimFieldName1,PFNList}, + asn1ct_gen:mk_var(asn1ct_name:curr(term)), + asn1ct_gen:mk_var(asn1ct_name:curr(tmpterm)), + Prop}], + {SaveBytes,PrevSt} + end; +gen_dec_line_special(Erule, Atype, TopType, Comp, DecInfObj, _Ext) -> + case gen_dec_line_other(Erule, Atype, TopType, Comp) of + Fun when is_function(Fun, 1) -> + fun({BytesVar,PrevSt}) -> + Fun(BytesVar), + gen_dec_line_dec_inf(Comp, DecInfObj), + {[],PrevSt} + end; + Imm0 -> + {imm,Imm0, + fun(Imm, {BytesVar,PrevSt}) -> + asn1ct_imm:dec_code_gen(Imm, BytesVar), + gen_dec_line_dec_inf(Comp, DecInfObj), + {[],PrevSt} + end} + end. + +gen_dec_line_dec_inf(Comp, DecInfObj) -> + #'ComponentType'{name=Cname} = Comp, case DecInfObj of - {Cname,ObjSet} -> - ObjSetRef = - case ObjSet of - {deep,OSName,_,_} -> - OSName; - _ -> ObjSet + {Cname,{_,OSet,UniqueFName,ValIndex}} -> + Term = asn1ct_gen:mk_var(asn1ct_name:curr(term)), + ValueMatch = value_match(ValIndex,Term), + {ObjSetMod,ObjSetName} = + case OSet of + {M,O} -> {{asis,M},O}; + _ -> {"?MODULE",OSet} end, - {[{ObjSetRef,Cname,asn1ct_gen:mk_var(asn1ct_name:curr(term))}], - SaveBytes}; + emit({",",nl,"ObjFun = ",ObjSetMod, + ":'getdec_",ObjSetName,"'(", + {asis,UniqueFName},", ",ValueMatch,")"}); _ -> - {[],SaveBytes} + ok + end. + +gen_dec_line_other(Erule, Atype, TopType, Comp) -> + #'ComponentType'{name=Cname,typespec=Type} = Comp, + CurrMod = get(currmod), + Ctgenmod = asn1ct_gen:ct_gen_module(Erule), + case asn1ct_gen:type(Atype) of + #'Externaltypereference'{module=CurrMod,type=EType} -> + fun(BytesVar) -> + emit({"'dec_",EType,"'(",BytesVar,",telltype)"}) + end; + #'Externaltypereference'{module=Mod,type=EType} -> + fun(BytesVar) -> + emit({"'",Mod,"':'dec_",EType,"'(",BytesVar, + ",telltype)"}) + end; + {primitive,bif} -> + case Atype of + {fixedtypevaluefield,_,Btype} -> + gen_dec_prim(Ctgenmod, Erule, Btype); + _ -> + gen_dec_prim(Ctgenmod, Erule, Type) + end; + 'ASN1_OPEN_TYPE' -> + case Type#type.def of + #'ObjectClassFieldType'{type=OpenType} -> + gen_dec_prim(Ctgenmod, Erule, #type{def=OpenType}); + _ -> + gen_dec_prim(Ctgenmod, Erule, Type) + end; + #typereference{val=Dname} -> + fun(BytesVar) -> + emit({"'dec_",Dname,"'(",BytesVar,",telltype)"}) + end; + {notype,_} -> + fun(BytesVar) -> + emit({"'dec_",Atype,"'(",BytesVar,",telltype)"}) + end; + {constructed,bif} -> + NewTypename = [Cname|TopType], + case Type#type.tablecinf of + [{objfun,_}|_R] -> + fun(BytesVar) -> + emit({"'dec_",asn1ct_gen:list2name(NewTypename), + "'(",BytesVar,", telltype, ObjFun)"}) + end; + _ -> + fun(BytesVar) -> + emit({"'dec_",asn1ct_gen:list2name(NewTypename), + "'(",BytesVar,", telltype)"}) + end + end + end. + +gen_dec_prim(Ctgenmod, Erule, Type) -> + case asn1ct_gen_per:gen_dec_imm(Erule, Type) of + no -> + fun(BytesVar) -> + Ctgenmod:gen_dec_prim(Erule, Type, BytesVar) + end; + Imm -> + Imm end. gen_enc_choice(Erule,TopType,CompList,Ext) -> @@ -1578,22 +1731,17 @@ gen_encode_prim_wrapper(CtgenMod,Erule,Cont,DoTag,Value) -> make_elements(I,Val,ExtCnames) -> make_elements(I,Val,ExtCnames,[]). -make_elements(I,Val,[ExtCname],Acc)-> % the last one, no comma needed - Element = make_element(I,Val,ExtCname), +make_elements(I,Val,[_ExtCname],Acc)-> % the last one, no comma needed + Element = make_element(I, Val), make_elements(I+1,Val,[],[Element|Acc]); -make_elements(I,Val,[ExtCname|Rest],Acc)-> - Element = make_element(I,Val,ExtCname), +make_elements(I,Val,[_ExtCname|Rest],Acc)-> + Element = make_element(I, Val), make_elements(I+1,Val,Rest,[", ",Element|Acc]); make_elements(_I,_,[],Acc) -> lists:reverse(Acc). -make_element(I,Val,Cname) -> - case tuple_notation_allowed() of - true -> - io_lib:format("?RT_PER:cindex(~w,~s,~w)",[I,Val,Cname]); - _ -> - io_lib:format("element(~w,~s)",[I,Val]) - end. +make_element(I, Val) -> + io_lib:format("element(~w,~s)", [I,Val]). emit_extaddgroupTerms(VarSeries,[_]) -> asn1ct_name:new(VarSeries), @@ -1651,22 +1799,12 @@ wrap_extensionAdditionGroups([],_,Acc,_,_) -> lists:reverse(Acc). -tuple_notation_allowed() -> - Options = get(encoding_options), - not (lists:member(optimize,Options) orelse lists:member(uper_bin,Options)). - -wrap_gen_dec_line(Erule,C,TopType,Cname,Type,Pos,DIO,Ext) -> +wrap_gen_dec_line(Erule,C,TopType,_Cname,_Type,Pos,DIO,Ext) -> put(component_type,{true,C}), - gen_dec_line(Erule,TopType,Cname,Type,Pos,DIO,Ext,mandatory), + gen_dec_line(Erule, TopType, C#'ComponentType'{prop=mandatory}, + Pos, DIO, Ext), erase(component_type). -get_components_prop() -> - case get(component_type) of - undefined -> - mandatory; - {true,#'ComponentType'{prop=Prop}} -> Prop - end. - value_match(Index,Value) when is_atom(Value) -> value_match(Index,atom_to_list(Value)); @@ -1683,7 +1821,5 @@ notice_value_match() -> Module = get(currmod), put(value_match,{true,Module}). -is_optimized(per_bin) -> - lists:member(optimize,get(encoding_options)); -is_optimized(_Erule) -> - false. +is_optimized(per) -> true; +is_optimized(uper) -> false. diff --git a/lib/asn1/src/asn1ct_gen.erl b/lib/asn1/src/asn1ct_gen.erl index 64a3555f62..57b12ce186 100644 --- a/lib/asn1/src/asn1ct_gen.erl +++ b/lib/asn1/src/asn1ct_gen.erl @@ -37,8 +37,7 @@ gen_check_call/7, get_constraint/2, insert_once/2, - rt2ct_suffix/1, - rt2ct_suffix/0, + ct_gen_module/1, index2suffix/1, get_record_name_prefix/0]). -export([pgen/5, @@ -52,7 +51,7 @@ %% pgen(Outfile, Erules, Module, TypeOrVal, Options) %% Generate Erlang module (.erl) and (.hrl) file corresponding to an ASN.1 module %% .hrl file is only generated if necessary -%% Erules = per | ber | ber_bin | per_bin +%% Erules = per | ber %% Module = atom() %% TypeOrVal = {TypeList,ValueList} %% TypeList = ValueList = [atom()] @@ -83,7 +82,7 @@ pgen_module(OutFile,Erules,Module, pgen_exports(Erules,Module,TypeOrVal), pgen_dispatcher(Erules,Module,TypeOrVal), pgen_info(), - pgen_typeorval(wrap_ber(Erules),Module,N2nConvEnums,TypeOrVal), + pgen_typeorval(Erules,Module,N2nConvEnums,TypeOrVal), pgen_partial_incomplete_decode(Erules), % gen_vars(asn1_db:mod_to_vars(Module)), % gen_tag_table(AllTypes), @@ -92,8 +91,7 @@ pgen_module(OutFile,Erules,Module, pgen_typeorval(Erules,Module,N2nConvEnums,{Types,Values,_Ptypes,_Classes,Objects,ObjectSets}) -> - Rtmod = list_to_atom(lists:concat(["asn1ct_gen_",erule(Erules), - rt2ct_suffix(Erules)])), + Rtmod = ct_gen_module(Erules), pgen_types(Rtmod,Erules,N2nConvEnums,Module,Types), pgen_values(Erules,Module,Values), pgen_objects(Rtmod,Erules,Module,Objects), @@ -196,7 +194,7 @@ pgen_check_defaultval(Erules,Module) -> end, gen_check_defaultval(Erules,Module,CheckObjects). -pgen_partial_decode(Rtmod,Erule,Module) when Erule == ber_bin_v2 -> +pgen_partial_decode(Rtmod,Erule,Module) when Erule == ber -> pgen_partial_inc_dec(Rtmod,Erule,Module), pgen_partial_dec(Rtmod,Erule,Module); pgen_partial_decode(_,_,_) -> @@ -240,7 +238,7 @@ pgen_partial_inc_dec1(Rtmod,Erules,Module,[P|Ps]) -> pgen_partial_inc_dec1(_,_,_,[]) -> ok. -gen_partial_inc_dec_refed_funcs(Rtmod,Erule) when Erule == ber_bin_v2 -> +gen_partial_inc_dec_refed_funcs(Rtmod,Erule) when Erule == ber -> case asn1ct:next_refed_func() of [] -> ok; @@ -296,8 +294,7 @@ pgen_partial_types1(_,undefined) -> %% TypeList a decode function will be generated. traverse_type_structure(Erules,Type,[],FuncName,TopTypeName) -> %% this is the selected type - Ctmod = list_to_atom(lists:concat(["asn1ct_gen_",erule(Erules), - rt2ct_suffix(Erules)])), + Ctmod = ct_gen_module(Erules), TypeDef = case Type of #type{} -> @@ -457,7 +454,7 @@ pgen_partial_incomplete_decode(Erule) -> _ -> ok end. -pgen_partial_incomplete_decode1(ber_bin_v2) -> +pgen_partial_incomplete_decode1(ber) -> case asn1ct:read_config_data(partial_incomplete_decode) of undefined -> ok; @@ -552,20 +549,17 @@ gen_part_decode_funcs(WhatKind,_TypeName,{_,Directive,_,_}) -> gen_types(Erules,Tname,{RootL1,ExtList,RootL2}) when is_list(RootL1), is_list(RootL2) -> gen_types(Erules,Tname,RootL1), - Rtmod = list_to_atom(lists:concat(["asn1ct_gen_",erule(Erules), - rt2ct_suffix(Erules)])), + Rtmod = ct_gen_module(Erules), gen_types(Erules,Tname,Rtmod:extaddgroup2sequence(ExtList)), gen_types(Erules,Tname,RootL2); gen_types(Erules,Tname,{RootList,ExtList}) when is_list(RootList) -> gen_types(Erules,Tname,RootList), - Rtmod = list_to_atom(lists:concat(["asn1ct_gen_",erule(Erules), - rt2ct_suffix(Erules)])), + Rtmod = ct_gen_module(Erules), gen_types(Erules,Tname,Rtmod:extaddgroup2sequence(ExtList)); gen_types(Erules,Tname,[{'EXTENSIONMARK',_,_}|Rest]) -> gen_types(Erules,Tname,Rest); gen_types(Erules,Tname,[ComponentType|Rest]) -> - Rtmod = list_to_atom(lists:concat(["asn1ct_gen_",erule(Erules), - rt2ct_suffix(Erules)])), + Rtmod = ct_gen_module(Erules), asn1ct_name:clear(), Rtmod:gen_encode(Erules,Tname,ComponentType), asn1ct_name:clear(), @@ -574,8 +568,7 @@ gen_types(Erules,Tname,[ComponentType|Rest]) -> gen_types(_,_,[]) -> true; gen_types(Erules,Tname,Type) when is_record(Type,type) -> - Rtmod = list_to_atom(lists:concat(["asn1ct_gen_",erule(Erules), - rt2ct_suffix(Erules)])), + Rtmod = ct_gen_module(Erules), asn1ct_name:clear(), Rtmod:gen_encode(Erules,Tname,Type), asn1ct_name:clear(), @@ -754,8 +747,7 @@ gen_value(Value) when is_record(Value,valuedef) -> emit([{asis,V},".",nl,nl]). gen_encode_constructed(Erules,Typename,InnerType,D) when is_record(D,type) -> - - Rtmod = list_to_atom(lists:concat(["asn1ct_constructed_",erule(Erules)])), + Rtmod = ct_constructed_module(Erules), case InnerType of 'SET' -> Rtmod:gen_encode_set(Erules,Typename,D), @@ -787,7 +779,7 @@ gen_encode_constructed(Erules,Typename,InnerType,D) gen_encode_constructed(Erules,Typename,InnerType,D#typedef.typespec). gen_decode_constructed(Erules,Typename,InnerType,D) when is_record(D,type) -> - Rtmod = list_to_atom(lists:concat(["asn1ct_constructed_",erule(Erules)])), + Rtmod = ct_constructed_module(Erules), asn1ct:step_in_constructed(), %% updates namelist for exclusive decode case InnerType of 'SET' -> @@ -818,27 +810,11 @@ pgen_exports(Erules,_Module,{Types,Values,_,_,Objects,ObjectSets}) -> case Erules of ber -> gen_exports1(Types,"enc_",2); - ber_bin -> - gen_exports1(Types,"enc_",2); - ber_bin_v2 -> - gen_exports1(Types,"enc_",2); _ -> gen_exports1(Types,"enc_",1) end, emit({"-export([",nl}), - gen_exports1(Types,"dec_",2), - case Erules of - ber -> - emit({"-export([",nl}), - gen_exports1(Types,"dec_",3); - ber_bin -> - emit({"-export([",nl}), - gen_exports1(Types,"dec_",3); -% ber_bin_v2 -> -% emit({"-export([",nl}), -% gen_exports1(Types,"dec_",2); - _ -> ok - end + gen_exports1(Types,"dec_",2) end, case [X || {n2n,X} <- get(encoding_options)] of [] -> ok; @@ -863,16 +839,11 @@ pgen_exports(Erules,_Module,{Types,Values,_,_,Objects,ObjectSets}) -> gen_exports1(Objects,"enc_",3), emit({"-export([",nl}), gen_exports1(Objects,"dec_",4); - ber_bin_v2 -> + ber -> emit({"-export([",nl}), gen_exports1(Objects,"enc_",3), emit({"-export([",nl}), - gen_exports1(Objects,"dec_",3); - _ -> - emit({"-export([",nl}), - gen_exports1(Objects,"enc_",4), - emit({"-export([",nl}), - gen_exports1(Objects,"dec_",4) + gen_exports1(Objects,"dec_",3) end end, case ObjectSets of @@ -948,20 +919,17 @@ pgen_dispatcher(Erules,_Module,{Types,_Values,_,_,_Objects,_ObjectSets}) -> emit(["encoding_rule() ->",nl]), emit([" ",{asis,Erules},".",nl,nl]), NoFinalPadding = lists:member(no_final_padding,get(encoding_options)), - Call = case Erules of - per -> "?RT_PER:complete(encode_disp(Type,Data))"; - per_bin -> ["?RT_PER:complete(encode_disp(Type,Data))"]; - ber -> "encode_disp(Type,Data)"; - ber_bin -> "encode_disp(Type,Data)"; - ber_bin_v2 -> "encode_disp(Type,Data)"; - uper_bin when NoFinalPadding == true -> - "?RT_PER:complete_NFP(encode_disp(Type,Data))"; - uper_bin -> ["?RT_PER:complete(encode_disp(Type,Data))"] - end, - EncWrap = case Erules of - ber -> "wrap_encode(Bytes)"; - _ -> "Bytes" - end, + {Call,BytesAsBinary} = + case Erules of + per -> + {["?RT_PER:complete(encode_disp(Type,Data))"],"Bytes"}; + ber -> + {"encode_disp(Type,Data)","iolist_to_binary(Bytes)"}; + uper when NoFinalPadding == true -> + {"?RT_PER:complete_NFP(encode_disp(Type,Data))","Bytes"}; + uper -> + {["?RT_PER:complete(encode_disp(Type,Data))"],"Bytes"} + end, emit(["encode(Type,Data) ->",nl, "case catch ",Call," of",nl, " {'EXIT',{error,Reason}} ->",nl, @@ -969,42 +937,25 @@ pgen_dispatcher(Erules,_Module,{Types,_Values,_,_,_Objects,_ObjectSets}) -> " {'EXIT',Reason} ->",nl, " {error,{asn1,Reason}};",nl, " {Bytes,_Len} ->",nl, - " {ok,",EncWrap,"};",nl]), - case Erules of - per -> - emit([" Bytes when is_binary(Bytes) ->",nl, - " {ok,binary_to_list(Bytes)};",nl, - " Bytes ->",nl, - " {ok,binary_to_list(list_to_binary(Bytes))}",nl, - " end.",nl,nl]); - _ -> - emit([" Bytes ->",nl, - " {ok,",EncWrap,"}",nl, - "end.",nl,nl]) - end, - -% case Erules of -% ber_bin_v2 -> -% emit(["decode(Type,Data0) ->",nl]), -% emit(["{Data,_RestBin} = ?RT_BER:decode(Data0",nif_parameter(),"),",nl]); -% _ -> -% emit(["decode(Type,Data) ->",nl]) -% end, + " {ok,",BytesAsBinary,"};",nl, + " Bytes ->",nl, + " {ok,",BytesAsBinary,"}",nl, + "end.",nl,nl]), Return_rest = lists:member(undec_rest,get(encoding_options)), Data = case {Erules,Return_rest} of - {ber_bin_v2,true} -> "Data0"; + {ber,true} -> "Data0"; _ -> "Data" end, emit(["decode(Type,",Data,") ->",nl]), DecAnonymous = case {Erules,Return_rest} of - {ber_bin_v2,false} -> + {ber,false} -> io_lib:format("~s~s~s~n", ["element(1,?RT_BER:decode(Data", nif_parameter(),"))"]); - {ber_bin_v2,true} -> + {ber,true} -> emit(["{Data,Rest} = ?RT_BER:decode(Data0", nif_parameter(),"),",nl]), "Data"; @@ -1012,10 +963,8 @@ pgen_dispatcher(Erules,_Module,{Types,_Values,_,_,_Objects,_ObjectSets}) -> "Data" end, DecWrap = case Erules of - ber -> "wrap_decode(Data)"; - ber_bin_v2 -> + ber -> DecAnonymous; - per -> "list_to_binary(Data)"; _ -> "Data" end, @@ -1025,32 +974,18 @@ pgen_dispatcher(Erules,_Module,{Types,_Values,_,_,_Objects,_ObjectSets}) -> " {'EXIT',Reason} ->",nl, " {error,{asn1,Reason}};",nl]), case {Erules,Return_rest} of - {ber_bin_v2,false} -> + {ber,false} -> emit([" Result ->",nl, " {ok,Result}",nl]); - {ber_bin_v2,true} -> + {ber,true} -> emit([" Result ->",nl, " {ok,Result,Rest}",nl]); - {per,false} -> - emit([" {X,_Rest} ->",nl, - " {ok,if_binary2list(X)};",nl, - " {X,_Rest,_Len} ->",nl, - " {ok,if_binary2list(X)}",nl]); {_,false} -> emit([" {X,_Rest} ->",nl, " {ok,X};",nl, " {X,_Rest,_Len} ->",nl, " {ok,X}",nl]); - {per,true} -> - emit([" {X,{_,Rest}} ->",nl, - " {ok,if_binary2list(X),Rest};",nl, - " {X,{_,Rest},_Len} ->",nl, - " {ok,if_binary2list(X),Rest};",nl, - " {X,Rest} ->",nl, - " {ok,if_binary2list(X),Rest};",nl, - " {X,Rest,_Len} ->",nl, - " {ok,if_binary2list(X),Rest}",nl]); - {per_bin,true} -> + {per,true} -> emit([" {X,{_,Rest}} ->",nl, " {ok,X,Rest};",nl, " {X,{_,Rest},_Len} ->",nl, @@ -1059,7 +994,7 @@ pgen_dispatcher(Erules,_Module,{Types,_Values,_,_,_Objects,_ObjectSets}) -> " {ok,X,Rest};",nl, " {X,Rest,_Len} ->",nl, " {ok,X,Rest}",nl]); - {uper_bin,true} -> + {uper,true} -> emit([" {X,{_,Rest}} ->",nl, " {ok,X,Rest};",nl, " {X,{_,Rest},_Len} ->",nl, @@ -1067,34 +1002,14 @@ pgen_dispatcher(Erules,_Module,{Types,_Values,_,_,_Objects,_ObjectSets}) -> " {X,Rest} ->",nl, " {ok,X,Rest};",nl, " {X,Rest,_Len} ->",nl, - " {ok,X,Rest}",nl]); - _ -> - emit([" {X,Rest} ->",nl, - " {ok,X,Rest};",nl, - " {X,Rest,_Len} ->",nl, " {ok,X,Rest}",nl]) end, emit(["end.",nl,nl]), - case Erules of - per -> - emit(["if_binary2list(B) when is_binary(B) ->",nl, - " binary_to_list(B);",nl, - "if_binary2list(L) -> L.",nl,nl]); - _ -> - ok - end, - gen_decode_partial_incomplete(Erules), case Erules of ber -> - gen_dispatcher(Types,"encode_disp","enc_",",[]"), - gen_dispatcher(Types,"decode_disp","dec_",",mandatory"); - ber_bin -> - gen_dispatcher(Types,"encode_disp","enc_",",[]"), - gen_dispatcher(Types,"decode_disp","dec_",",mandatory"); - ber_bin_v2 -> gen_dispatcher(Types,"encode_disp","enc_",""), gen_dispatcher(Types,"decode_disp","dec_",""), gen_partial_inc_dispatcher(); @@ -1103,17 +1018,10 @@ pgen_dispatcher(Erules,_Module,{Types,_Values,_,_,_Objects,_ObjectSets}) -> gen_dispatcher(Types,"decode_disp","dec_",",mandatory") end, emit([nl]), - - case Erules of - ber -> - gen_wrapper(); - _ -> ok - end, emit({nl,nl}). -gen_decode_partial_incomplete(Erule) when Erule == ber;Erule==ber_bin; - Erule==ber_bin_v2 -> +gen_decode_partial_incomplete(ber) -> case {asn1ct:read_config_data(partial_incomplete_decode), asn1ct:get_gen_state_field(inc_type_pattern)} of {undefined,_} -> @@ -1121,34 +1029,35 @@ gen_decode_partial_incomplete(Erule) when Erule == ber;Erule==ber_bin; {_,undefined} -> ok; _ -> - case Erule of - ber_bin_v2 -> - EmitCaseClauses = - fun() -> - emit([" {'EXIT',{error,Reason}} ->",nl, - " {error,Reason};",nl, - " {'EXIT',Reason} ->",nl, - " {error,{asn1,Reason}};",nl, - " Result ->",nl, - " {ok,Result}",nl, - " end.",nl,nl]) - end, - emit(["decode_partial_incomplete(Type,Data0,", - "Pattern) ->",nl]), - emit([" {Data,_RestBin} =",nl, - " ?RT_BER:decode_primitive_", - "incomplete(Pattern,Data0),",nl, - " case catch decode_partial_inc_disp(Type,", - "Data) of",nl]), - EmitCaseClauses(), - emit(["decode_part(Type,Data0) ->",nl]), - emit([" case catch decode_inc_disp(Type,element(1," - "?RT_BER:decode(Data0",nif_parameter(),"))) of",nl]), -% " {Data,_RestBin} = ?RT_BER:decode(Data0),",nl, -% " case catch decode_inc_disp(Type,Data) of",nl]), - EmitCaseClauses(); - _ -> ok % add later - end + EmitCaseClauses = + fun() -> + emit([" {'EXIT',{error,Reason}} ->",nl, + " {error,Reason};",nl, + " {'EXIT',Reason} ->",nl, + " {error,{asn1,Reason}};",nl, + " Result ->",nl, + " {ok,Result}",nl, + " end"]) + end, + emit(["decode_partial_incomplete(Type,Data0,", + "Pattern) ->",nl]), + emit([" {Data,_RestBin} =",nl, + " ?RT_BER:decode_primitive_", + "incomplete(Pattern,Data0),",nl, + " case catch decode_partial_inc_disp(Type,", + "Data) of",nl]), + EmitCaseClauses(), + emit([".",nl,nl]), + emit(["decode_part(Type, Data0) " + "when is_binary(Data0) ->",nl]), + emit([" case catch decode_inc_disp(Type,element(1," + "?RT_BER:decode(Data0",nif_parameter(),"))) of",nl]), + EmitCaseClauses(), + emit([";",nl]), + emit(["decode_part(Type, Data0) ->",nl]), + emit([" case catch decode_inc_disp(Type, Data0) of",nl]), + EmitCaseClauses(), + emit([".",nl,nl]) end; gen_decode_partial_incomplete(_Erule) -> ok. @@ -1187,23 +1096,8 @@ gen_partial_inc_dispatcher([],_) -> " exit({error,{asn1,{undefined_type,Type}}}).",nl]). nif_parameter() -> - Options = get(encoding_options), - case {lists:member(driver,Options),lists:member(nif,Options)} of - {true,_} -> ",nif"; - {_,true} -> ",nif"; - _ -> "" - end. + ",nif". -gen_wrapper() -> - emit(["wrap_encode(Bytes) when is_list(Bytes) ->",nl, - " binary_to_list(list_to_binary(Bytes));",nl, - "wrap_encode(Bytes) when is_binary(Bytes) ->",nl, - " binary_to_list(Bytes);",nl, - "wrap_encode(Bytes) -> Bytes.",nl,nl]), - emit(["wrap_decode(Bytes) when is_list(Bytes) ->",nl, - " list_to_binary(Bytes);",nl, - "wrap_decode(Bytes) -> Bytes.",nl]). - gen_dispatcher([F1,F2|T],FuncName,Prefix,ExtraArg) -> emit([FuncName,"('",F1,"',Data) -> '",Prefix,F1,"'(Data",ExtraArg,")",";",nl]), gen_dispatcher([F2|T],FuncName,Prefix,ExtraArg); @@ -1213,19 +1107,16 @@ gen_dispatcher([Flast|_T],FuncName,Prefix,ExtraArg) -> pgen_info() -> emit(["info() ->",nl, - " case ?MODULE:module_info() of",nl, - " MI when is_list(MI) ->",nl, - " case lists:keysearch(attributes,1,MI) of",nl, - " {value,{_,Attributes}} when is_list(Attributes) ->",nl, - " case lists:keysearch(asn1_info,1,Attributes) of",nl, - " {value,{_,Info}} when is_list(Info) ->",nl, - " Info;",nl, - " _ ->",nl, - " []",nl, - " end;",nl, - " _ ->",nl, - " []",nl, - " end",nl, + " case ?MODULE:module_info(attributes) of",nl, + " Attributes when is_list(Attributes) ->",nl, + " case lists:keyfind(asn1_info, 1, Attributes) of",nl, + " {_,Info} when is_list(Info) ->",nl, + " Info;",nl, + " _ ->",nl, + " []",nl, + " end;",nl, + " _ ->",nl, + " []",nl, " end.",nl]). open_hrl(OutFile,Module) -> @@ -1496,32 +1387,15 @@ gen_head(Erules,Mod,Hrl) -> {Rtmac,Rtmod} = case Erules of per -> emit({"%% Generated by the Erlang ASN.1 PER-" - "compiler version:",asn1ct:vsn(),nl}), - {"RT_PER",?RT_PER_BIN}; - ber -> - emit({"%% Generated by the Erlang ASN.1 BER-" - "compiler version:",asn1ct:vsn(),nl}), - {"RT_BER",?RT_BER_BIN}; - per_bin -> - emit({"%% Generated by the Erlang ASN.1 BER-" - "compiler version, utilizing bit-syntax:", - asn1ct:vsn(),nl}), - %% temporary code to enable rt2ct optimization - case lists:member(optimize,Options) of - true -> {"RT_PER","asn1rt_per_bin_rt2ct"}; - _ -> {"RT_PER",?RT_PER_BIN} - end; - ber_bin -> - emit({"%% Generated by the Erlang ASN.1 BER-" "compiler version, utilizing bit-syntax:", asn1ct:vsn(),nl}), - {"RT_BER",?RT_BER_BIN}; - ber_bin_v2 -> + {"RT_PER","asn1rt_per_bin_rt2ct"}; + ber -> emit({"%% Generated by the Erlang ASN.1 BER_V2-" "compiler version, utilizing bit-syntax:", asn1ct:vsn(),nl}), {"RT_BER","asn1rt_ber_bin_v2"}; - uper_bin -> + uper -> emit(["%% Generated by the Erlang ASN.1 UNALIGNED" " PER-compiler version, utilizing" " bit-syntax:", @@ -1531,7 +1405,7 @@ gen_head(Erules,Mod,Hrl) -> emit({"%% Purpose: encoder and decoder to the types in mod ",Mod,nl,nl}), emit({"-module('",Mod,"').",nl}), put(currmod,Mod), - %emit({"-compile(export_all).",nl}), + emit({"-compile(nowarn_unused_vars).",nl}), case {Hrl,lists:member(inline,get(encoding_options))} of {0,_} -> true; {_,true} -> true; @@ -1541,7 +1415,7 @@ gen_head(Erules,Mod,Hrl) -> emit(["-define('",Rtmac,"',",Rtmod,").",nl]), emit(["-asn1_info([{vsn,'",asn1ct:vsn(),"'},",nl, " {module,'",Mod,"'},",nl, - " {options,",io_lib:format("~w",[Options]),"}]).",nl,nl]). + " {options,",io_lib:format("~p",[Options]),"}]).",nl,nl]). gen_hrlhead(Mod) -> @@ -2019,43 +1893,29 @@ constructed_suffix('SEQUENCE OF',_) -> constructed_suffix('SET OF',_) -> 'SETOF'. -erule(ber) -> - ber; -erule(ber_bin) -> - ber; -erule(ber_bin_v2) -> - ber_bin_v2; -erule(per) -> - per; -erule(per_bin) -> - per; -erule(uper_bin) -> - per. - -wrap_ber(ber) -> - ber_bin; -wrap_ber(Erule) -> - Erule. - -rt2ct_suffix() -> - Options = get(encoding_options), - case {lists:member(optimize,Options),lists:member(per_bin,Options)} of - {true,true} -> "_rt2ct"; - _ -> "" - end. -rt2ct_suffix(per_bin) -> - Options = get(encoding_options), - case lists:member(optimize,Options) of - true -> "_rt2ct"; - _ -> "" - end; -rt2ct_suffix(_) -> "". +erule(ber) -> ber; +erule(per) -> per; +erule(uper) -> per. index2suffix(0) -> ""; index2suffix(N) -> lists:concat(["_",N]). +ct_gen_module(ber) -> + asn1ct_gen_ber_bin_v2; +ct_gen_module(per) -> + asn1ct_gen_per_rt2ct; +ct_gen_module(uper) -> + asn1ct_gen_per. + +ct_constructed_module(ber) -> + asn1ct_constructed_ber_bin_v2; +ct_constructed_module(per) -> + asn1ct_constructed_per; +ct_constructed_module(uper) -> + asn1ct_constructed_per. + get_constraint(C,Key) -> case lists:keysearch(Key,1,C) of false -> diff --git a/lib/asn1/src/asn1ct_gen_ber.erl b/lib/asn1/src/asn1ct_gen_ber.erl deleted file mode 100644 index 491ebcb8fd..0000000000 --- a/lib/asn1/src/asn1ct_gen_ber.erl +++ /dev/null @@ -1,1749 +0,0 @@ -%% -%% %CopyrightBegin% -%% -%% Copyright Ericsson AB 1997-2010. All Rights Reserved. -%% -%% The contents of this file are subject to the Erlang Public License, -%% Version 1.1, (the "License"); you may not use this file except in -%% compliance with the License. You should have received a copy of the -%% Erlang Public License along with this software. If not, it can be -%% retrieved online at http://www.erlang.org/. -%% -%% Software distributed under the License is distributed on an "AS IS" -%% basis, WITHOUT WARRANTY OF ANY KIND, either express or implied. See -%% the License for the specific language governing rights and limitations -%% under the License. -%% -%% %CopyrightEnd% -%% -%% --module(asn1ct_gen_ber). - -%% Generate erlang module which handles (PER) encode and decode for -%% all types in an ASN.1 module - --include("asn1_records.hrl"). - --export([pgen/4]). --export([decode_class/1, decode_type/1]). --export([add_removed_bytes/0]). --export([gen_encode/2,gen_encode/3,gen_decode/2,gen_decode/3]). --export([gen_encode_prim/4]). --export([gen_dec_prim/8]). --export([gen_objectset_code/2, gen_obj_code/3]). --export([re_wrap_erule/1]). --export([unused_var/2]). --export([extaddgroup2sequence/1]). - --import(asn1ct_gen, [emit/1,demit/1]). - - % the encoding of class of tag bits 8 and 7 --define(UNIVERSAL, 0). --define(APPLICATION, 16#40). --define(CONTEXT, 16#80). --define(PRIVATE, 16#C0). - - % primitive or constructed encoding % bit 6 --define(PRIMITIVE, 0). --define(CONSTRUCTED, 2#00100000). - - --define(T_ObjectDescriptor, ?UNIVERSAL bor ?PRIMITIVE bor 7). - % restricted character string types --define(T_NumericString, ?UNIVERSAL bor ?PRIMITIVE bor 18). %can be constructed --define(T_PrintableString, ?UNIVERSAL bor ?PRIMITIVE bor 19). %can be constructed --define(T_TeletexString, ?UNIVERSAL bor ?PRIMITIVE bor 20). %can be constructed --define(T_VideotexString, ?UNIVERSAL bor ?PRIMITIVE bor 21). %can be constructed --define(T_IA5String, ?UNIVERSAL bor ?PRIMITIVE bor 22). %can be constructed --define(T_GraphicString, ?UNIVERSAL bor ?PRIMITIVE bor 25). %can be constructed --define(T_VisibleString, ?UNIVERSAL bor ?PRIMITIVE bor 26). %can be constructed --define(T_GeneralString, ?UNIVERSAL bor ?PRIMITIVE bor 27). %can be constructed - -%% pgen(Erules, Module, TypeOrVal) -%% Generate Erlang module (.erl) and (.hrl) file corresponding to an ASN.1 module -%% .hrl file is only generated if necessary -%% Erules = per | ber -%% Module = atom() -%% TypeOrVal = {TypeList,ValueList,PTypeList} -%% TypeList = ValueList = [atom()] - -pgen(OutFile,Erules,Module,TypeOrVal) -> - asn1ct_gen:pgen_module(OutFile,Erules,Module,TypeOrVal,[],true). - - -%%=============================================================================== -%%=============================================================================== -%%=============================================================================== -%% Generate ENCODING -%%=============================================================================== -%%=============================================================================== -%%=============================================================================== - -%%=============================================================================== -%% encode #{typedef, {pos, name, typespec}} -%%=============================================================================== - -gen_encode(Erules,Type) when is_record(Type,typedef) -> - gen_encode_user(Erules,Type). - -%%=============================================================================== -%% encode #{type, {tag, def, constraint}} -%%=============================================================================== - -gen_encode(Erules,Typename,Type) when is_record(Type,type) -> - InnerType = asn1ct_gen:get_inner(Type#type.def), - ObjFun = - case lists:keysearch(objfun,1,Type#type.tablecinf) of - {value,{_,_Name}} -> - ", ObjFun"; - false -> - "" - end, - case asn1ct_gen:type(InnerType) of - {constructed,bif} -> - emit([nl,nl,nl,"%%================================"]), - emit([nl,"%% ",asn1ct_gen:list2name(Typename)]), - emit([nl,"%%================================",nl]), - case lists:member(InnerType,['SET','SEQUENCE']) of - true -> - case get(asn_keyed_list) of - true -> - CompList = - case Type#type.def of - #'SEQUENCE'{components=Cl} -> Cl; - #'SET'{components=Cl} -> Cl - end, - emit([nl,"'enc_",asn1ct_gen:list2name(Typename), - "'(Val, TagIn",ObjFun, - ") when is_list(Val) ->",nl]), - emit([" 'enc_",asn1ct_gen:list2name(Typename), - "'(?RT_BER:fixoptionals(", - {asis,optionals(CompList)}, - ",Val), TagIn",ObjFun,");",nl,nl]); - _ -> true - end; - _ -> - emit([nl,"'enc_",asn1ct_gen:list2name(Typename), - "'({'",asn1ct_gen:list2name(Typename), - "',Val}, TagIn",ObjFun,") ->",nl]), - emit([" 'enc_",asn1ct_gen:list2name(Typename), - "'(Val, TagIn",ObjFun,");",nl,nl]) - end, - emit(["'enc_",asn1ct_gen:list2name(Typename), - "'(Val, TagIn",ObjFun,") ->",nl," "]), - asn1ct_gen:gen_encode_constructed(Erules,Typename,InnerType,Type); - _ -> - true - end; - -%%=============================================================================== -%% encode ComponentType -%%=============================================================================== - -gen_encode(Erules,Tname,#'ComponentType'{name=Cname,typespec=Type}) -> - NewTname = [Cname|Tname], - %% The tag is set to [] to avoid that it is - %% taken into account twice, both as a component/alternative (passed as - %% argument to the encode decode function and within the encode decode - %% function it self. - NewType = Type#type{tag=[]}, - gen_encode(Erules,NewTname,NewType). - -gen_encode_user(Erules,D) when is_record(D,typedef) -> - Typename = [D#typedef.name], - Type = D#typedef.typespec, - InnerType = asn1ct_gen:get_inner(Type#type.def), - OTag = Type#type.tag, - Tag = [X#tag{class=decode_class(X#tag.class)}|| X <- OTag], - emit([nl,nl,"%%================================"]), - emit([nl,"%% ",Typename]), - emit([nl,"%%================================",nl]), - case lists:member(InnerType,['SET','SEQUENCE']) of - true -> - case get(asn_keyed_list) of - true -> - CompList = - case Type#type.def of - #'SEQUENCE'{components=Cl} -> Cl; - #'SET'{components=Cl} -> Cl - end, - - emit([nl,"'enc_",asn1ct_gen:list2name(Typename), - "'(Val, TagIn) when is_list(Val) ->",nl]), - emit([" 'enc_",asn1ct_gen:list2name(Typename), - "'(?RT_BER:fixoptionals(", - {asis,optionals(CompList)}, - ",Val), TagIn);",nl,nl]); - _ -> true - end; - _ -> - emit({nl,"'enc_",asn1ct_gen:list2name(Typename), - "'({'",asn1ct_gen:list2name(Typename),"',Val}, TagIn) ->",nl}), - emit({" 'enc_",asn1ct_gen:list2name(Typename),"'(Val, TagIn);",nl,nl}) - end, - emit({"'enc_",asn1ct_gen:list2name(Typename),"'(", - unused_var("Val",Type#type.def),", TagIn) ->",nl}), - CurrentMod = get(currmod), - case asn1ct_gen:type(InnerType) of - {constructed,bif} -> - asn1ct_gen:gen_encode_constructed(Erules,Typename,InnerType,D); - {primitive,bif} -> - asn1ct_gen_ber:gen_encode_prim(ber,Type,["TagIn ++ ", - {asis,Tag}],"Val"), - emit([".",nl]); - #typereference{val=Ename} -> - emit([" 'enc_",Ename,"'(Val, TagIn ++ ",{asis,Tag},").",nl]); - #'Externaltypereference'{module=CurrentMod,type=Etype} -> - emit([" 'enc_",Etype,"'(Val, TagIn ++ ", - {asis,Tag},").",nl]); - #'Externaltypereference'{module=Emod,type=Etype} -> - emit([" '",Emod,"':'enc_",Etype,"'(Val, TagIn ++ ", - {asis,Tag},").",nl]); - 'ASN1_OPEN_TYPE' -> - emit(["%% OPEN TYPE",nl]), - asn1ct_gen_ber:gen_encode_prim(ber, - Type#type{def='ASN1_OPEN_TYPE'}, - ["TagIn ++ ", - {asis,Tag}],"Val"), - emit([".",nl]) - end. - -unused_var(Var,#'SEQUENCE'{components=Cl}) -> - unused_var1(Var,Cl); -unused_var(Var,#'SET'{components=Cl}) -> - unused_var1(Var,Cl); -unused_var(Var,_) -> - Var. -unused_var1(Var,Cs) when Cs == []; Cs == {[],[]} -> - lists:concat(["_",Var]); -unused_var1(Var,_) -> - Var. - -unused_optormand_var(Var,Def) -> - case asn1ct_gen:type(asn1ct_gen:get_inner(Def)) of - 'ASN1_OPEN_TYPE' -> - lists:concat(["_",Var]); - _ -> - Var - end. - - -gen_encode_prim(Erules,D,DoTag,Value) when is_record(D,type) -> - -%%% Currently not used for BER (except for BitString) and therefore replaced -%%% with [] as a placeholder - BitStringConstraint = D#type.constraint, - Constraint = [], - asn1ct_name:new(enumval), - case D#type.def of - 'BOOLEAN' -> - emit_encode_func('boolean',Value,DoTag); - 'INTEGER' -> - emit_encode_func('integer',Constraint,Value,DoTag); - {'INTEGER',NamedNumberList} -> - emit_encode_func('integer',Constraint,Value, - NamedNumberList,DoTag); - {'ENUMERATED',NamedNumberList={_,_}} -> - - emit(["case (case ",Value," of {_,",{curr,enumval},"}->", - {curr,enumval},";_->", Value," end) of",nl]), - asn1ct_name:new(enumval), - emit_enc_enumerated_cases(NamedNumberList,DoTag); - {'ENUMERATED',NamedNumberList} -> - - emit(["case (case ",Value," of {_,",{curr,enumval},"}->", - {curr,enumval},";_->", Value," end) of",nl]), - asn1ct_name:new(enumval), - emit_enc_enumerated_cases(NamedNumberList,DoTag); - - 'REAL' -> - emit_encode_func('real',Constraint,Value,DoTag); - - {'BIT STRING',NamedNumberList} -> - emit_encode_func('bit_string',BitStringConstraint,Value, - NamedNumberList,DoTag); - 'ANY' -> - emit_encode_func('open_type', Value,DoTag); - 'NULL' -> - emit_encode_func('null',Value,DoTag); - 'OBJECT IDENTIFIER' -> - emit_encode_func("object_identifier",Value,DoTag); - 'RELATIVE-OID' -> - emit_encode_func("relative_oid",Value,DoTag); - 'ObjectDescriptor' -> - emit_encode_func('restricted_string',Constraint,Value, - ?T_ObjectDescriptor,DoTag); - 'OCTET STRING' -> - emit_encode_func('octet_string',Constraint,Value,DoTag); - 'NumericString' -> - emit_encode_func('restricted_string',Constraint,Value, - ?T_NumericString,DoTag); - TString when TString == 'TeletexString'; - TString == 'T61String' -> - emit_encode_func('restricted_string',Constraint,Value, - ?T_TeletexString,DoTag); - 'VideotexString' -> - emit_encode_func('restricted_string',Constraint,Value, - ?T_VideotexString,DoTag); - 'GraphicString' -> - emit_encode_func('restricted_string',Constraint,Value, - ?T_GraphicString,DoTag); - 'VisibleString' -> - emit_encode_func('restricted_string',Constraint,Value, - ?T_VisibleString,DoTag); - 'GeneralString' -> - emit_encode_func('restricted_string',Constraint,Value, - ?T_GeneralString,DoTag); - 'PrintableString' -> - emit_encode_func('restricted_string',Constraint,Value, - ?T_PrintableString,DoTag); - 'IA5String' -> - emit_encode_func('restricted_string',Constraint,Value, - ?T_IA5String,DoTag); - 'UniversalString' -> - emit_encode_func('universal_string',Constraint,Value,DoTag); - 'UTF8String' -> - emit_encode_func('UTF8_string',Constraint,Value,DoTag); - 'BMPString' -> - emit_encode_func('BMP_string',Constraint,Value,DoTag); - 'UTCTime' -> - emit_encode_func('utc_time',Constraint,Value,DoTag); - 'GeneralizedTime' -> - emit_encode_func('generalized_time',Constraint,Value,DoTag); - 'ASN1_OPEN_TYPE' -> - emit_encode_func('open_type', Value,DoTag); - #'ObjectClassFieldType'{} -> - case asn1ct_gen:get_inner(D#type.def) of - {fixedtypevaluefield,_,InnerType} -> - gen_encode_prim(Erules,InnerType,DoTag,Value); - 'ASN1_OPEN_TYPE' -> - emit_encode_func('open_type', Value,DoTag); - XX -> - exit({'can not encode' ,XX}) - end; - XX -> - exit({'can not encode' ,XX}) - end. - - -emit_encode_func(Name,Value,Tags) when is_atom(Name) -> - emit_encode_func(atom_to_list(Name),Value,Tags); -emit_encode_func(Name,Value,Tags) -> - Fname = "?RT_BER:encode_" ++ Name, - emit([Fname,"(",Value,", ",Tags,")"]). - -emit_encode_func(Name,Constraint,Value,Tags) when is_atom(Name) -> - emit_encode_func(atom_to_list(Name),Constraint,Value,Tags); -emit_encode_func(Name,Constraint,Value,Tags) -> - Fname = "?RT_BER:encode_" ++ Name, - emit([Fname,"(",{asis,Constraint},", ",Value,", ",Tags,")"]). - -emit_encode_func(Name,Constraint,Value,Asis,Tags) when is_atom(Name) -> - emit_encode_func(atom_to_list(Name),Constraint,Value,Asis,Tags); -emit_encode_func(Name,Constraint,Value,Asis,Tags) -> - Fname = "?RT_BER:encode_" ++ Name, - emit([Fname,"(",{asis,Constraint},", ",Value, - ", ",{asis,Asis}, - ", ",Tags,")"]). - -emit_enc_enumerated_cases({L1,L2}, Tags) -> - emit_enc_enumerated_cases(L1++L2, Tags, ext); -emit_enc_enumerated_cases(L, Tags) -> - emit_enc_enumerated_cases(L, Tags, noext). - -emit_enc_enumerated_cases([{EnumName,EnumVal},H2|T], Tags, Ext) -> - emit([{asis,EnumName}," -> ?RT_BER:encode_enumerated(",EnumVal,",",Tags,");",nl]), -%% emit(["'",{asis,EnumName},"' -> ?RT_BER:encode_enumerated(",EnumVal,",",Tags,");",nl]), - emit_enc_enumerated_cases([H2|T], Tags, Ext); -emit_enc_enumerated_cases([{EnumName,EnumVal}], Tags, Ext) -> - emit([{asis,EnumName}," -> ?RT_BER:encode_enumerated(",EnumVal,",",Tags,")"]), -%% emit(["'",{asis,EnumName},"' -> ?RT_BER:encode_enumerated(",EnumVal,",",Tags,")"]), - case Ext of - noext -> emit([";",nl]); - ext -> -%% emit([";",nl,"{asn1_enum,",{curr,enumval},"} -> ", -%% "?RT_BER:encode_enumerated(",{curr,enumval},",",Tags,");",nl]), -%% asn1ct_name:new(enumval) - emit([";",nl]) - end, - emit([{curr,enumval}," -> exit({error,{asn1, {enumerated_not_in_range,",{curr, enumval},"}}})"]), - emit([nl,"end"]). - - -%%=============================================================================== -%%=============================================================================== -%%=============================================================================== -%% Generate DECODING -%%=============================================================================== -%%=============================================================================== -%%=============================================================================== - -%%=============================================================================== -%% decode #{typedef, {pos, name, typespec}} -%%=============================================================================== - -gen_decode(Erules,Type) when is_record(Type,typedef) -> - D = Type, - emit({nl,nl}), - emit({"'dec_",Type#typedef.name,"'(Bytes, OptOrMand) ->",nl}), - emit({" 'dec_",Type#typedef.name,"'(Bytes, OptOrMand, []).",nl,nl}), - emit({"'dec_",Type#typedef.name,"'(Bytes, ", - unused_optormand_var("OptOrMand",(Type#typedef.typespec)#type.def),", TagIn) ->",nl}), - dbdec(Type#typedef.name), - gen_decode_user(Erules,D). - - -%%=============================================================================== -%% decode #{type, {tag, def, constraint}} -%%=============================================================================== - -gen_decode(Erules,Tname,Type) when is_record(Type,type) -> - Typename = Tname, - InnerType = asn1ct_gen:get_inner(Type#type.def), - case asn1ct_gen:type(InnerType) of - {constructed,bif} -> - ObjFun = - case Type#type.tablecinf of - [{objfun,_}|_R] -> - ", ObjFun"; - _ -> - "" - end, - emit({"'dec_",asn1ct_gen:list2name(Typename),"'(Bytes, OptOrMand, TagIn",ObjFun,") ->",nl}), - dbdec(Typename), - asn1ct_gen:gen_decode_constructed(Erules,Typename,InnerType,Type); - _ -> - true - end; - - -%%=============================================================================== -%% decode ComponentType -%%=============================================================================== - -gen_decode(Erules,Tname,#'ComponentType'{name=Cname,typespec=Type}) -> - NewTname = [Cname|Tname], - %% The tag is set to [] to avoid that it is - %% taken into account twice, both as a component/alternative (passed as - %% argument to the encode decode function and within the encode decode - %% function it self. - NewType = Type#type{tag=[]}, - gen_decode(Erules,NewTname,NewType). - - -gen_decode_user(Erules,D) when is_record(D,typedef) -> - Typename = [D#typedef.name], - Def = D#typedef.typespec, - InnerType = asn1ct_gen:get_inner(Def#type.def), - InnerTag = Def#type.tag , - Tag = [X#tag{class=decode_class(X#tag.class)}|| X <- InnerTag], - case asn1ct_gen:type(InnerType) of - 'ASN1_OPEN_TYPE' -> - BytesVar = asn1ct_gen:mk_var(asn1ct_name:curr(bytes)), - asn1ct_name:new(len), - gen_dec_prim(Erules, Def#type{def='ASN1_OPEN_TYPE'}, - BytesVar, Tag, "TagIn",no_length, - ?PRIMITIVE,"OptOrMand"), - emit({".",nl,nl}); - {primitive,bif} -> - BytesVar = asn1ct_gen:mk_var(asn1ct_name:curr(bytes)), - asn1ct_name:new(len), - gen_dec_prim(Erules, Def, BytesVar, Tag, "TagIn",no_length, - ?PRIMITIVE,"OptOrMand"), - emit({".",nl,nl}); - {constructed,bif} -> - asn1ct_gen:gen_decode_constructed(Erules,Typename,InnerType,D); - TheType -> - DecFunName = mkfuncname(TheType,dec), - emit({DecFunName,"(",{curr,bytes}, - ", OptOrMand, TagIn++",{asis,Tag},")"}), - emit({".",nl,nl}) - end. - - -gen_dec_prim(Erules,Att,BytesVar,DoTag,TagIn,Length,Form,OptOrMand) -> - Typename = Att#type.def, -%% Currently not used for BER replaced with [] as place holder -%% Constraint = Att#type.constraint, -%% Constraint = [], - Constraint = - case get_constraint(Att#type.constraint,'SizeConstraint') of - no -> []; - Tc -> Tc - end, - ValueRange = - case get_constraint(Att#type.constraint,'ValueRange') of - no -> []; - Tv -> Tv - end, - SingleValue = - case get_constraint(Att#type.constraint,'SingleValue') of - no -> []; - Sv -> Sv - end, - AsBin = case get(binary_strings) of - true -> "_as_bin"; - _ -> "" - end, - NewTypeName = case Typename of - 'ANY' -> 'ASN1_OPEN_TYPE'; - _ -> Typename - end, - DoLength = - case NewTypeName of - 'BOOLEAN'-> - emit({"?RT_BER:decode_boolean(",BytesVar,","}), - false; - 'INTEGER' -> - emit({"?RT_BER:decode_integer(",BytesVar,",", - {asis,int_constr(SingleValue,ValueRange)},","}), - false; - {'INTEGER',NamedNumberList} -> - emit({"?RT_BER:decode_integer(",BytesVar,",", - {asis,int_constr(SingleValue,ValueRange)},",", - {asis,NamedNumberList},","}), - false; - {'ENUMERATED',NamedNumberList} -> - emit({"?RT_BER:decode_enumerated(",BytesVar,",", - {asis,Constraint},",", - {asis,NamedNumberList},","}), - false; - 'REAL' -> - emit({"?RT_BER:decode_real(",BytesVar,",", - {asis,Constraint},","}), - false; - {'BIT STRING',NamedNumberList} -> - case get(compact_bit_string) of - true -> - emit({"?RT_BER:decode_compact_bit_string(", - BytesVar,",",{asis,Constraint},",", - {asis,NamedNumberList},","}); - _ -> - emit({"?RT_BER:decode_bit_string(",BytesVar,",", - {asis,Constraint},",", - {asis,NamedNumberList},","}) - end, - true; - 'NULL' -> - emit({"?RT_BER:decode_null(",BytesVar,","}), - false; - 'OBJECT IDENTIFIER' -> - emit({"?RT_BER:decode_object_identifier(",BytesVar,","}), - false; - 'RELATIVE-OID' -> - emit({"?RT_BER:decode_relative_oid(",BytesVar,","}), - false; - 'ObjectDescriptor' -> - emit({"?RT_BER:decode_restricted_string(", - BytesVar,",",{asis,Constraint},",",{asis,?T_ObjectDescriptor},","}), - true; - 'OCTET STRING' -> - emit({"?RT_BER:decode_octet_string",AsBin,"(",BytesVar,",",{asis,Constraint},","}), - true; - 'NumericString' -> - emit({"?RT_BER:decode_restricted_string",AsBin,"(", - BytesVar,",",{asis,Constraint},",",{asis,?T_NumericString},","}),true; - TString when TString == 'TeletexString'; - TString == 'T61String' -> - emit({"?RT_BER:decode_restricted_string",AsBin,"(", - BytesVar,",",{asis,Constraint},",",{asis,?T_TeletexString},","}), - true; - 'VideotexString' -> - emit({"?RT_BER:decode_restricted_string",AsBin,"(", - BytesVar,",",{asis,Constraint},",",{asis,?T_VideotexString},","}), - true; - 'GraphicString' -> - emit({"?RT_BER:decode_restricted_string",AsBin,"(", - BytesVar,",",{asis,Constraint},",",{asis,?T_GraphicString},","}) - ,true; - 'VisibleString' -> - emit({"?RT_BER:decode_restricted_string",AsBin,"(", - BytesVar,",",{asis,Constraint},",",{asis,?T_VisibleString},","}), - true; - 'GeneralString' -> - emit({"?RT_BER:decode_restricted_string",AsBin,"(", - BytesVar,",",{asis,Constraint},",",{asis,?T_GeneralString},","}), - true; - 'PrintableString' -> - emit({"?RT_BER:decode_restricted_string",AsBin,"(", - BytesVar,",",{asis,Constraint},",",{asis,?T_PrintableString},","}), - true; - 'IA5String' -> - emit({"?RT_BER:decode_restricted_string",AsBin,"(", - BytesVar,",",{asis,Constraint},",",{asis,?T_IA5String},","}), - true; - 'UniversalString' -> - emit({"?RT_BER:decode_universal_string",AsBin,"(", - BytesVar,",",{asis,Constraint},","}), - true; - 'UTF8String' -> - emit({"?RT_BER:decode_UTF8_string",AsBin,"(", - BytesVar,","}), - false; - 'BMPString' -> - emit({"?RT_BER:decode_BMP_string",AsBin,"(", - BytesVar,",",{asis,Constraint},","}), - true; - 'UTCTime' -> - emit({"?RT_BER:decode_utc_time",AsBin,"(", - BytesVar,",",{asis,Constraint},","}), - true; - 'GeneralizedTime' -> - emit({"?RT_BER:decode_generalized_time",AsBin,"(", - BytesVar,",",{asis,Constraint},","}), - true; - 'ASN1_OPEN_TYPE' -> - emit(["?RT_BER:decode_open_type(",re_wrap_erule(Erules),",", - BytesVar,","]), - false; - #'ObjectClassFieldType'{} -> - case asn1ct_gen:get_inner(Att#type.def) of - {fixedtypevaluefield,_,InnerType} -> - gen_dec_prim(Erules,InnerType,BytesVar,DoTag,TagIn,Length,Form,OptOrMand), - false; - 'ASN1_OPEN_TYPE' -> - emit(["?RT_BER:decode_open_type(", - re_wrap_erule(Erules),",", - BytesVar,","]), - false; - XX -> - exit({'can not decode' ,XX}) - end; - Other -> - exit({'can not decode' ,Other}) - end, - - NewLength = case DoLength of - true -> [", ", Length]; - false -> "" - end, - NewOptOrMand = case OptOrMand of - _ when is_list(OptOrMand) -> OptOrMand; - mandatory -> {asis,mandatory}; - _ -> {asis,opt_or_default} - end, - case {TagIn,NewTypeName} of - {_,#'ObjectClassFieldType'{}} -> - case asn1ct_gen:get_inner(Att#type.def) of - 'ASN1_OPEN_TYPE' -> - emit([{asis,DoTag},")"]); - _ -> ok - end; - {[],'ASN1_OPEN_TYPE'} -> - emit([{asis,DoTag},")"]); - {_,'ASN1_OPEN_TYPE'} -> - emit([TagIn,"++",{asis,DoTag},")"]); - {[],_} -> - emit([{asis,DoTag},NewLength,", ",NewOptOrMand,")"]); - _ when is_list(TagIn) -> - emit([TagIn,"++",{asis,DoTag},NewLength,", ",NewOptOrMand,")"]) - end. - - -int_constr([],[]) -> - []; -int_constr([],ValueRange) -> - ValueRange; -int_constr(SingleValue,[]) -> - SingleValue; -int_constr(SV,VR) -> - [SV,VR]. - -%% Object code generating for encoding and decoding -%% ------------------------------------------------ - -gen_obj_code(Erules,_Module,Obj) when is_record(Obj,typedef) -> - ObjName = Obj#typedef.name, - Def = Obj#typedef.typespec, - #'Externaltypereference'{module=M,type=ClName} = Def#'Object'.classname, - Class = asn1_db:dbget(M,ClName), - - {object,_,Fields} = Def#'Object'.def, - emit({nl,nl,nl,"%%================================"}), - emit({nl,"%% ",ObjName}), - emit({nl,"%%================================",nl}), - EncConstructed = - gen_encode_objectfields(ClName,get_class_fields(Class), - ObjName,Fields,[]), - emit(nl), - gen_encode_constr_type(Erules,EncConstructed), - emit(nl), - DecConstructed = - gen_decode_objectfields(ClName,get_class_fields(Class), - ObjName,Fields,[]), - emit(nl), - gen_decode_constr_type(Erules,DecConstructed); -gen_obj_code(_Erules,_Module,Obj) when is_record(Obj,pobjectdef) -> - ok. - - -gen_encode_objectfields(ClassName,[{typefield,Name,OptOrMand}|Rest], - ObjName,ObjectFields,ConstrAcc) -> - EmitFuncClause = - fun(Args) -> - emit(["'enc_",ObjName,"'(",{asis,Name}, - ", ",Args,", _RestPrimFieldName) ->",nl]) - end, -% emit(["'enc_",ObjName,"'(",{asis,Name}, -% ", Val, TagIn, _RestPrimFieldName) ->",nl]), - MaybeConstr= - case {get_object_field(Name,ObjectFields),OptOrMand} of - {false,'MANDATORY'} -> %% this case is illegal - exit({error,{asn1,{"missing mandatory field in object", - ObjName}}}); - {false,'OPTIONAL'} -> %% OPTIONAL field in class - EmitFuncClause("Val, _"), %% Value must be anything - %% already encoded - emit([" {Val,0}"]), - []; - {false,{'DEFAULT',DefaultType}} -> - EmitFuncClause("Val, TagIn"), - gen_encode_default_call(ClassName,Name,DefaultType); - {{Name,TypeSpec},_} -> - %% A specified field owerwrites any 'DEFAULT' or - %% 'OPTIONAL' field in the class - EmitFuncClause("Val, TagIn"), - gen_encode_field_call(ObjName,Name,TypeSpec) - end, - case more_genfields(Rest) of - true -> - emit([";",nl]); - false -> - emit([".",nl]) - end, - gen_encode_objectfields(ClassName,Rest,ObjName,ObjectFields, - MaybeConstr++ConstrAcc); -gen_encode_objectfields(ClassName,[{objectfield,Name,_,_,OptOrMand}|Rest], - ObjName,ObjectFields,ConstrAcc) -> - CurrentMod = get(currmod), - EmitFuncClause = - fun(Args) -> - emit(["'enc_",ObjName,"'(",{asis,Name}, - ", ",Args,") ->",nl]) - end, -% emit(["'enc_",ObjName,"'(",{asis,Name}, -% ", Val, TagIn, [H|T]) ->",nl]), - case {get_object_field(Name,ObjectFields),OptOrMand} of - {false,'MANDATORY'} -> - exit({error,{asn1,{"missing mandatory field in object", - ObjName}}}); - {false,'OPTIONAL'} -> - EmitFuncClause("_,_,_"), - emit([" exit({error,{'use of missing field in object', ",{asis,Name}, - "}})"]); - {false,{'DEFAULT',_DefaultObject}} -> - exit({error,{asn1,{"not implemented yet",Name}}}); - {{Name,#'Externalvaluereference'{module=CurrentMod, - value=TypeName}},_} -> - EmitFuncClause(" Val, TagIn, [H|T]"), - emit({indent(3),"'enc_",TypeName,"'(H, Val, TagIn, T)"}); - {{Name,#'Externalvaluereference'{module=M,value=TypeName}},_} -> - EmitFuncClause(" Val, TagIn, [H|T]"), - emit({indent(3),"'",M,"':'enc_",TypeName,"'(H, Val, TagIn, T)"}); - {{Name,TypeSpec},_} -> - EmitFuncClause(" Val, TagIn, [H|T]"), - case TypeSpec#typedef.name of - {ExtMod,TypeName} -> - emit({indent(3),"'",ExtMod,"':'enc_",TypeName, - "'(H, Val, TagIn, T)"}); - TypeName -> - emit({indent(3),"'enc_",TypeName,"'(H, Val, TagIn, T)"}) - end - end, - case more_genfields(Rest) of - true -> - emit([";",nl]); - false -> - emit([".",nl]) - end, - gen_encode_objectfields(ClassName,Rest,ObjName,ObjectFields,ConstrAcc); -gen_encode_objectfields(ClassName,[_|Cs],O,OF,Acc) -> - gen_encode_objectfields(ClassName,Cs,O,OF,Acc); -gen_encode_objectfields(_,[],_,_,Acc) -> - Acc. - - -% gen_encode_objectfields(Class,ObjName,[{FieldName,Type}|Rest],ConstrAcc) -> -% Fields = Class#objectclass.fields, -% MaybeConstr= -% case is_typefield(Fields,FieldName) of -% true -> -% Def = Type#typedef.typespec, -% OTag = Def#type.tag, -% Tag = [X#tag{class=decode_class(X#tag.class)}|| X <- OTag], -% emit({"'enc_",ObjName,"'(",{asis,FieldName}, -% ", Val, TagIn, RestPrimFieldName) ->",nl}), -% CAcc= -% case Type#typedef.name of -% {primitive,bif} -> -% gen_encode_prim(ber,Def,["TagIn ++ ",{asis,Tag}], -% "Val"), -% []; -% {constructed,bif} -> -% %%InnerType = asn1ct_gen:get_inner(Def#type.def), -% %%asn1ct_gen:gen_encode_constructed(ber,[ObjName], -% %% InnerType,Def); -% emit({" 'enc_",ObjName,'_',FieldName, -% "'(Val, TagIn ++ ",{asis,Tag},")"}), -% [{['enc_',ObjName,'_',FieldName],Def}]; -% {ExtMod,TypeName} -> -% emit({" '",ExtMod,"':'enc_",TypeName, -% "'(Val, TagIn ++ ",{asis,Tag},")"}), -% []; -% TypeName -> -% emit({" 'enc_",TypeName,"'(Val, TagIn ++ ", -% {asis,Tag},")"}), -% [] -% end, -% case more_genfields(Fields,Rest) of -% true -> -% emit({";",nl}); -% false -> -% emit({".",nl}) -% end, -% CAcc; -% {false,objectfield} -> -% emit({"'enc_",ObjName,"'(",{asis,FieldName}, -% ", Val, TagIn, [H|T]) ->",nl}), -% case Type#typedef.name of -% {ExtMod,TypeName} -> -% emit({indent(3),"'",ExtMod,"':'enc_",TypeName, -% "'(H, Val, TagIn, T)"}); -% TypeName -> -% emit({indent(3),"'enc_",TypeName,"'(H, Val, TagIn, T)"}) -% end, -% case more_genfields(Fields,Rest) of -% true -> -% emit({";",nl}); -% false -> -% emit({".",nl}) -% end, -% []; -% {false,_} -> [] -% end, -% gen_encode_objectfields(Class,ObjName,Rest,MaybeConstr ++ ConstrAcc); -% gen_encode_objectfields(C,O,[H|T],Acc) -> -% gen_encode_objectfields(C,O,T,Acc); -% gen_encode_objectfields(_,_,[],Acc) -> -% Acc. - -% gen_encode_constr_type([{Name,Def}|Rest]) -> -% emit({Name,"(Val,TagIn) ->",nl}), -% InnerType = asn1ct_gen:get_inner(Def#type.def), -% asn1ct_gen:gen_encode_constructed(ber,Name,InnerType,Def), -% gen_encode_constr_type(Rest); -gen_encode_constr_type(Erules,[TypeDef|Rest]) when is_record(TypeDef,typedef) -> - case is_already_generated(enc,TypeDef#typedef.name) of - true -> ok; - _ -> gen_encode_user(Erules,TypeDef) - end, - gen_encode_constr_type(Erules,Rest); -gen_encode_constr_type(_,[]) -> - ok. - -gen_encode_field_call(_ObjName,_FieldName, - #'Externaltypereference'{module=M,type=T}) -> - CurrentMod = get(currmod), - TDef = asn1_db:dbget(M,T), - Def = TDef#typedef.typespec, - OTag = Def#type.tag, - Tag = [X#tag{class=decode_class(X#tag.class)}|| X <- OTag], - if - M == CurrentMod -> - emit({" 'enc_",T,"'(Val, TagIn ++ ",{asis,Tag},")"}), - []; - true -> - emit({" '",M,"':'enc_",T,"'(Val, TagIn ++ ",{asis,Tag},")"}), - [] - end; -gen_encode_field_call(ObjName,FieldName,Type) -> - Def = Type#typedef.typespec, - OTag = Def#type.tag, - Tag = [X#tag{class=decode_class(X#tag.class)}|| X <- OTag], - case Type#typedef.name of - {primitive,bif} -> %%tag should be the primitive tag - gen_encode_prim(ber,Def,["TagIn ++ ",{asis,Tag}], - "Val"), - []; - {constructed,bif} -> - emit({" 'enc_",ObjName,'_',FieldName, - "'(Val, TagIn ++",{asis,Tag},")"}), - [Type#typedef{name=list_to_atom(lists:concat([ObjName,'_',FieldName]))}]; - {ExtMod,TypeName} -> - emit({" '",ExtMod,"':'enc_",TypeName, - "'(Val, TagIn ++ ",{asis,Tag},")"}), - []; - TypeName -> - emit({" 'enc_",TypeName,"'(Val, TagIn ++ ",{asis,Tag},")"}), - [] - end. - -gen_encode_default_call(ClassName,FieldName,Type) -> - CurrentMod = get(currmod), - InnerType = asn1ct_gen:get_inner(Type#type.def), - OTag = Type#type.tag, - Tag = [X#tag{class=decode_class(X#tag.class)}|| X <- OTag], - case asn1ct_gen:type(InnerType) of - {constructed,bif} -> -%% asn1ct_gen:gen_encode_constructed(Erules,Typename,InnerType,Type); - emit([" 'enc_",ClassName,'_',FieldName,"'(Bytes, TagIn ++ ", - {asis,Tag},")"]), - [#typedef{name=list_to_atom(lists:concat([ClassName,'_',FieldName])), - typespec=Type}]; - {primitive,bif} -> - gen_encode_prim(ber,Type,["TagIn ++ ",{asis,Tag}],"Val"), - []; - #'Externaltypereference'{module=CurrentMod,type=Etype} -> - emit([" 'enc_",Etype,"'(Val, TagIn ++ ",{asis,Tag},")",nl]), - []; - #'Externaltypereference'{module=Emod,type=Etype} -> - emit([" '",Emod,"':'enc_",Etype,"'(Val, TagIn ++ ",{asis,Tag},")",nl]), - [] - end. - - - -gen_decode_objectfields(ClassName,[{typefield,Name,OptOrMand}|Rest], - ObjName,ObjectFields,ConstrAcc) -> - EmitFuncClause = - fun(Args) -> - emit(["'dec_",ObjName,"'(",{asis,Name}, - ", ",Args,"_) ->",nl]) - end, -% emit(["'dec_",ObjName,"'(",{asis,Name}, -% ", Bytes, TagIn, RestPrimFieldName) ->",nl]), - MaybeConstr= - case {get_object_field(Name,ObjectFields),OptOrMand} of - {false,'MANDATORY'} -> %% this case is illegal - exit({error,{asn1,{"missing mandatory field in object", - ObjName}}}); - {false,'OPTIONAL'} -> - EmitFuncClause("Bytes, _,"), -% emit([" asn1_NOVALUE"]), - emit([" {Bytes,[],0}"]), - []; - {false,{'DEFAULT',DefaultType}} -> - EmitFuncClause("Bytes, TagIn,"), - gen_decode_default_call(ClassName,Name,"Bytes",DefaultType); - {{Name,TypeSpec},_} -> - %% A specified field owerwrites any 'DEFAULT' or - %% 'OPTIONAL' field in the class - EmitFuncClause("Bytes, TagIn,"), - gen_decode_field_call(ObjName,Name,"Bytes",TypeSpec) - end, - case more_genfields(Rest) of - true -> - emit([";",nl]); - false -> - emit([".",nl]) - end, - gen_decode_objectfields(ClassName,Rest,ObjName,ObjectFields,MaybeConstr++ConstrAcc); -gen_decode_objectfields(ClassName,[{objectfield,Name,_,_,OptOrMand}|Rest], - ObjName,ObjectFields,ConstrAcc) -> - CurrentMod = get(currmod), - EmitFuncClause = - fun(Args) -> - emit(["'dec_",ObjName,"'(",{asis,Name}, - ", ",Args,") ->",nl]) - end, -% emit(["'dec_",ObjName,"'(",{asis,Name}, -% ", Bytes,TagIn,[H|T]) ->",nl]), - case {get_object_field(Name,ObjectFields),OptOrMand} of - {false,'MANDATORY'} -> - exit({error,{asn1,{"missing mandatory field in object", - ObjName}}}); - {false,'OPTIONAL'} -> - EmitFuncClause("_,_,_"), - emit([" exit({error,{'illegal use of missing field in object', ",{asis,Name}, - "}})"]); - {false,{'DEFAULT',_DefaultObject}} -> - exit({error,{asn1,{"not implemented yet",Name}}}); - {{Name,#'Externalvaluereference'{module=CurrentMod, - value=TypeName}},_} -> - EmitFuncClause("Bytes,TagIn,[H|T]"), - emit({indent(3),"'dec_",TypeName,"'(H, Bytes, TagIn, T)"}); - {{Name,#'Externalvaluereference'{module=M,value=TypeName}},_} -> - EmitFuncClause("Bytes,TagIn,[H|T]"), - emit({indent(3),"'",M,"':'dec_",TypeName, - "'(H, Bytes, TagIn, T)"}); - {{Name,TypeSpec},_} -> - EmitFuncClause("Bytes,TagIn,[H|T]"), - case TypeSpec#typedef.name of - {ExtMod,TypeName} -> - emit({indent(3),"'",ExtMod,"':'dec_",TypeName, - "'(H, Bytes, TagIn, T)"}); - TypeName -> - emit({indent(3),"'dec_",TypeName,"'(H, Bytes, TagIn, T)"}) - end - end, - case more_genfields(Rest) of - true -> - emit([";",nl]); - false -> - emit([".",nl]) - end, - gen_decode_objectfields(ClassName,Rest,ObjName,ObjectFields,ConstrAcc); -gen_decode_objectfields(CN,[_|Cs],O,OF,CAcc) -> - gen_decode_objectfields(CN,Cs,O,OF,CAcc); -gen_decode_objectfields(_,[],_,_,CAcc) -> - CAcc. - - -gen_decode_constr_type(Erules,[TypeDef|Rest]) when is_record(TypeDef,typedef) -> - case is_already_generated(dec,TypeDef#typedef.name) of - true -> ok; - _ -> - gen_decode(Erules,TypeDef) - end, - gen_decode_constr_type(Erules,Rest); -gen_decode_constr_type(_,[]) -> - ok. - -gen_decode_field_call(_ObjName,_FieldName,Bytes, - #'Externaltypereference'{module=M,type=T}) -> - CurrentMod = get(currmod), - TDef = asn1_db:dbget(M,T), - Def = TDef#typedef.typespec, - OTag = Def#type.tag, - Tag = [X#tag{class=decode_class(X#tag.class)}|| X <- OTag], - if - M == CurrentMod -> - emit({" 'dec_",T,"'(",Bytes, - ", opt_or_default,TagIn ++ ",{asis,Tag},")"}), - []; - true -> - emit({" '",M,"':'dec_",T, - "'(",Bytes,", opt_or_default,TagIn ++ ",{asis,Tag},")"}), - [] - end; -gen_decode_field_call(ObjName,FieldName,Bytes,Type) -> - Def = Type#typedef.typespec, - OTag = Def#type.tag, - Tag = [X#tag{class=decode_class(X#tag.class)}|| X <- OTag], - case Type#typedef.name of - {primitive,bif} -> %%tag should be the primitive tag - gen_dec_prim(ber,Def,Bytes,Tag,"TagIn",no_length, - ?PRIMITIVE,opt_or_default), - []; - {constructed,bif} -> - emit({" 'dec_",ObjName,'_',FieldName, - "'(",Bytes,",opt_or_default, TagIn ++ ",{asis,Tag},")"}), - [Type#typedef{name=list_to_atom(lists:concat([ObjName,'_',FieldName]))}]; - {ExtMod,TypeName} -> - emit({" '",ExtMod,"':'dec_",TypeName, - "'(",Bytes,", opt_or_default,TagIn ++ ",{asis,Tag},")"}), - []; - TypeName -> - emit({" 'dec_",TypeName,"'(",Bytes, - ", opt_or_default,TagIn ++ ",{asis,Tag},")"}), - [] - end. - -gen_decode_default_call(ClassName,FieldName,Bytes,Type) -> - CurrentMod = get(currmod), - InnerType = asn1ct_gen:get_inner(Type#type.def), - OTag = Type#type.tag, - Tag = [X#tag{class=decode_class(X#tag.class)}|| X <- OTag], - case asn1ct_gen:type(InnerType) of - {constructed,bif} -> - emit([" 'dec_",ClassName,'_',FieldName,"'(",Bytes, - ",opt_or_default, TagIn ++ ",{asis,Tag},")"]), - [#typedef{name=list_to_atom(lists:concat([ClassName,'_',FieldName])), - typespec=Type}]; - {primitive,bif} -> - gen_dec_prim(ber,Type,Bytes,Tag,"TagIn",no_length, - ?PRIMITIVE,opt_or_default), - []; - #'Externaltypereference'{module=CurrentMod,type=Etype} -> - emit([" 'dec_",Etype,"'(",Bytes, - " ,opt_or_default, TagIn ++ ",{asis,Tag},")",nl]), - []; - #'Externaltypereference'{module=Emod,type=Etype} -> - emit([" '",Emod,"':'dec_",Etype,"'(",Bytes, - ", opt_or_defualt, TagIn ++ ",{asis,Tag},")",nl]), - [] - end. - - -more_genfields([]) -> - false; -more_genfields([Field|Fields]) -> - case element(1,Field) of - typefield -> - true; - objectfield -> - true; - _ -> - more_genfields(Fields) - end. - - - -%% Object Set code generating for encoding and decoding -%% ---------------------------------------------------- -gen_objectset_code(Erules,ObjSet) -> - ObjSetName = ObjSet#typedef.name, - Def = ObjSet#typedef.typespec, -% {ClassName,ClassDef} = Def#'ObjectSet'.class, - #'Externaltypereference'{module=ClassModule, - type=ClassName} = Def#'ObjectSet'.class, - ClassDef = asn1_db:dbget(ClassModule,ClassName), - UniqueFName = Def#'ObjectSet'.uniquefname, - Set = Def#'ObjectSet'.set, - emit({nl,nl,nl,"%%================================"}), - emit({nl,"%% ",ObjSetName}), - emit({nl,"%%================================",nl}), - case ClassName of - {_Module,ExtClassName} -> - gen_objset_code(Erules,ObjSetName,UniqueFName,Set, - ExtClassName,ClassDef); - _ -> - gen_objset_code(Erules,ObjSetName,UniqueFName,Set, - ClassName,ClassDef) - end, - emit(nl). - -gen_objset_code(Erules,ObjSetName,UniqueFName,Set,ClassName,ClassDef)-> - ClassFields = (ClassDef#classdef.typespec)#objectclass.fields, - InternalFuncs=gen_objset_enc(ObjSetName,UniqueFName,Set,ClassName,ClassFields,1,[]), - gen_objset_dec(Erules,ObjSetName,UniqueFName,Set,ClassName,ClassFields,1), - gen_internal_funcs(Erules,InternalFuncs). - - -%% gen_objset_enc iterates over the objects of the object set -gen_objset_enc(_,{unique,undefined},_,_,_,_,_) -> - %% There is no unique field in the class of this object set - %% don't bother about the constraint - []; -gen_objset_enc(ObjSName,UniqueName,[{_,no_unique_value,_},T|Rest], - ClName,ClFields,NthObj,Acc) -> - %% No need to check that this class has property OPTIONAL for the - %% unique field, it was detected in the previous phase - gen_objset_enc(ObjSName,UniqueName,[T|Rest],ClName,ClFields,NthObj,Acc); -gen_objset_enc(ObjSetName,UniqueName,[{_,no_unique_value,_}], - _ClName,_ClFields,_NthObj,Acc) -> - %% No need to check that this class has property OPTIONAL for the - %% unique field, it was detected in the previous phase - emit_default_getenc(ObjSetName,UniqueName), - emit({".",nl,nl}), - Acc; -gen_objset_enc(ObjSName,UniqueName, - [{ObjName,Val,Fields},T|Rest],ClName,ClFields,NthObj,Acc)-> - emit({"'getenc_",ObjSName,"'(",{asis,UniqueName},",",{asis,Val},") ->",nl}), - CurrMod = get(currmod), - {InternalFunc,NewNthObj}= - case ObjName of - {no_mod,no_name} -> - gen_inlined_enc_funs(Fields,ClFields,ObjSName,NthObj); - {CurrMod,Name} -> - emit({" fun 'enc_",Name,"'/4"}), - {[],NthObj}; - {ModuleName,Name} -> - emit_ext_fun(enc,ModuleName,Name), -% emit([" {'",ModuleName,"', 'enc_",Name,"'}"]), - {[],NthObj}; - _Other -> - emit({" fun 'enc_",ObjName,"'/4"}), - {[],NthObj} - end, - emit({";",nl}), - gen_objset_enc(ObjSName,UniqueName,[T|Rest],ClName,ClFields, - NewNthObj,InternalFunc ++ Acc); -gen_objset_enc(ObjSetName,UniqueName, - [{ObjName,Val,Fields}],_ClName,ClFields,NthObj,Acc) -> - emit({"'getenc_",ObjSetName,"'(",{asis,UniqueName},",",{asis,Val},") ->",nl}), - CurrMod = get(currmod), - {InternalFunc,_}= - case ObjName of - {no_mod,no_name} -> - gen_inlined_enc_funs(Fields,ClFields,ObjSetName,NthObj); - {CurrMod,Name} -> - emit({" fun 'enc_",Name,"'/4"}), - {[],NthObj}; - {ModuleName,Name} -> - emit_ext_fun(enc,ModuleName,Name), -% emit([" {'",ModuleName,"', 'enc_",Name,"'}"]), - {[],NthObj}; - _Other -> - emit({" fun 'enc_",ObjName,"'/4"}), - {[],NthObj} - end, - emit([";",nl]), - emit_default_getenc(ObjSetName,UniqueName), - emit({".",nl,nl}), - InternalFunc ++ Acc; -%% See X.681 Annex E for the following case -gen_objset_enc(ObjSetName,_UniqueName,['EXTENSIONMARK'], - _ClName,_ClFields,_NthObj,Acc) -> - emit({"'getenc_",ObjSetName,"'(_, _) ->",nl}), - emit({indent(3),"fun(_Attr, Val, _TagIn, _RestPrimFieldName) ->",nl}), - emit({indent(6),"Len = case Val of",nl,indent(9), - "Bin when is_binary(Bin) -> size(Bin);",nl,indent(9), - "_ -> length(Val)",nl,indent(6),"end,"}), - emit({indent(6),"{Val,Len}",nl}), - emit({indent(3),"end.",nl,nl}), - Acc; -gen_objset_enc(ObjSetName,UniqueName,['EXTENSIONMARK','EXTENSIONMARK'|Rest], - ClName,ClFields,NthObj,Acc) -> - gen_objset_enc(ObjSetName,UniqueName,['EXTENSIONMARK'|Rest], - ClName,ClFields,NthObj,Acc); -gen_objset_enc(ObjSetName,UniqueName,['EXTENSIONMARK'|Rest], - ClName,ClFields,NthObj,Acc) -> - gen_objset_enc(ObjSetName,UniqueName,Rest++['EXTENSIONMARK'], - ClName,ClFields,NthObj,Acc); -gen_objset_enc(_,_,[],_,_,_,Acc) -> - Acc. - -emit_ext_fun(EncDec,ModuleName,Name) -> - emit([indent(3),"fun(T,V,_O1,_O2) -> '",ModuleName,"':'",EncDec,"_", - Name,"'(T,V,_O1,_O2) end"]). - -emit_default_getenc(ObjSetName,UniqueName) -> - emit(["'getenc_",ObjSetName,"'(",{asis,UniqueName},", ErrV) ->",nl]), - emit([indent(3),"fun(C,V,_,_) -> exit({'Type not compatible with table constraint',{component,C},{value,V}, {unique_name_and_value,",{asis,UniqueName},", ErrV}}) end"]). - -%% gen_inlined_enc_funs for each object iterates over all fields of a -%% class, and for each typefield it checks if the object has that -%% field and emits the proper code. -gen_inlined_enc_funs(Fields,[{typefield,Name,_}|Rest],ObjSetName, - NthObj) -> - CurrMod = get(currmod), - InternalDefFunName = asn1ct_gen:list2name([NthObj,Name,ObjSetName]), - case lists:keysearch(Name,1,Fields) of - {value,{_,Type}} when is_record(Type,type) -> - emit({indent(3),"fun(Type, Val, TagIn, _RestPrimFieldName) ->",nl, - indent(6),"case Type of",nl}), - {Ret,N} = emit_inner_of_fun(Type,InternalDefFunName), - gen_inlined_enc_funs1(Fields,Rest,ObjSetName,NthObj+N,Ret); - {value,{_,Type}} when is_record(Type,typedef) -> - emit({indent(3),"fun(Type, Val, TagIn, _RestPrimFieldName) ->",nl, - indent(6),"case Type of",nl}), - emit({indent(9),{asis,Name}," ->",nl}), - {Ret,N} = emit_inner_of_fun(Type,InternalDefFunName), - gen_inlined_enc_funs1(Fields,Rest,ObjSetName,NthObj+N,Ret); - {value,{_,#'Externaltypereference'{module=M,type=T}}} -> - #typedef{typespec=Type} = asn1_db:dbget(M,T), - OTag = Type#type.tag, - Tag = [X#tag{class=decode_class(X#tag.class)}|| X <- OTag], - emit([indent(3),"fun(Type, Val, TagIn, _RestPrimFieldName) ->",nl, - indent(6),"case Type of",nl]), - emit([indent(9),{asis,Name}," ->",nl]), - if - M == CurrMod -> - emit([indent(12),"'enc_",T,"'(Val, TagIn ++ ", - {asis,Tag},")"]); - true -> - emit([indent(12),"'",M,"':'enc_",T,"'(Val,TagIn ++", - {asis,Tag},")"]) - end, - gen_inlined_enc_funs1(Fields,Rest,ObjSetName,NthObj,[]); - false -> - %% This field was not present in the object thus there were no - %% type in the table and we therefore generate code that returns - %% the input for application treatment. - emit([indent(3),"fun(Type, Val, TagIn, _RestPrimFieldName) ->",nl, - indent(6),"case Type of",nl]), - emit([indent(9),{asis,Name}," ->",nl]), - emit([indent(12),"Len = case Val of",nl, - indent(15),"Bin when is_binary(Bin) -> size(Bin);",nl, - indent(15),"_ -> length(Val)",nl,indent(12),"end,",nl, - indent(12),"{Val,Len}"]), - gen_inlined_enc_funs1(Fields,Rest,ObjSetName,NthObj,[]) - end; -gen_inlined_enc_funs(Fields,[_H|Rest],ObjSetName,NthObj) -> - gen_inlined_enc_funs(Fields,Rest,ObjSetName,NthObj); -gen_inlined_enc_funs(_,[],_,NthObj) -> - {[],NthObj}. - -gen_inlined_enc_funs1(Fields,[{typefield,Name,_}|Rest],ObjSetName, - NthObj,Acc) -> - CurrMod = get(currmod), - InternalDefFunName = asn1ct_gen:list2name([NthObj,Name,ObjSetName]), - {Acc2,NAdd}= - case lists:keysearch(Name,1,Fields) of - {value,{_,Type}} when is_record(Type,type) -> - emit({";",nl}), - {Ret,N}=emit_inner_of_fun(Type,InternalDefFunName), - {Ret++Acc,N}; - {value,{_,Type}} when is_record(Type,typedef) -> - emit({";",nl,indent(9),{asis,Name}," ->",nl}), - {Ret,N}=emit_inner_of_fun(Type,InternalDefFunName), - {Ret++Acc,N}; - {value,{_,#'Externaltypereference'{module=M,type=T}}} -> - #typedef{typespec=Type} = asn1_db:dbget(M,T), - OTag = Type#type.tag, - Tag = [X#tag{class=decode_class(X#tag.class)}|| X <- OTag], - emit({";",nl,indent(9),{asis,Name}," ->",nl}), - if - M == CurrMod -> - emit([indent(12),"'enc_",T,"'(Val, TagIn ++ ", - {asis,Tag},")"]); - true -> - emit([indent(12),"'",M,"':'enc_",T,"'(Val,TagIn ++", - {asis,Tag},")"]) - end, - {Acc,0}; - false -> - %% This field was not present in the object thus there were no - %% type in the table and we therefore generate code that returns - %% the input for application treatment. - emit([";",nl,indent(9),{asis,Name}," ->",nl]), - emit([indent(12),"Len = case Val of",nl, - indent(15),"Bin when is_binary(Bin) -> size(Bin);",nl, - indent(15),"_ -> length(Val)",nl,indent(12),"end,",nl, - indent(12),"{Val,Len}"]), - {Acc,0} - end, - gen_inlined_enc_funs1(Fields,Rest,ObjSetName,NthObj+NAdd,Acc2); -gen_inlined_enc_funs1(Fields,[_H|Rest],ObjSetName,NthObj,Acc)-> - gen_inlined_enc_funs1(Fields,Rest,ObjSetName,NthObj,Acc); -gen_inlined_enc_funs1(_,[],_,NthObj,Acc) -> - emit({nl,indent(6),"end",nl}), - emit({indent(3),"end"}), - {Acc,NthObj}. - - -emit_inner_of_fun(TDef = #typedef{name={ExtMod,Name},typespec=Type}, - InternalDefFunName) -> - OTag = Type#type.tag, - Tag = [X#tag{class=decode_class(X#tag.class)}|| X <- OTag], - case {ExtMod,Name} of - {primitive,bif} -> - emit(indent(12)), - gen_encode_prim(ber,Type,["TagIn ++ ",{asis,Tag}],"Val"), - {[],0}; - {constructed,bif} -> - emit([indent(12),"'enc_", - InternalDefFunName,"'(Val,TagIn ++ ", - {asis,Tag},")"]), - {[TDef#typedef{name=InternalDefFunName}],1}; - _ -> - emit({indent(12),"'",ExtMod,"':'enc_",Name,"'(Val, TagIn ++ ", - {asis,Tag},")"}), - {[],0} - end; -emit_inner_of_fun(#typedef{name=Name,typespec=Type},_) -> - OTag = Type#type.tag, - Tag = [X#tag{class=decode_class(X#tag.class)}|| X <- OTag], - emit({indent(12),"'enc_",Name,"'(Val, TagIn ++ ",{asis,Tag},")"}), - {[],0}; -emit_inner_of_fun(Type,_) when is_record(Type,type) -> - CurrMod = get(currmod), - OTag = Type#type.tag, - Tag = [X#tag{class=decode_class(X#tag.class)}|| X <- OTag], - case Type#type.def of - Def when is_atom(Def) -> - emit({indent(9),Def," ->",nl,indent(12)}), - gen_encode_prim(ber,Type,["TagIn ++ ",{asis,Tag}],"Val"); - TRef when is_record(TRef,typereference) -> - T = TRef#typereference.val, - emit({indent(9),T," ->",nl,indent(12),"'enc_",T, - "'(Val, TagIn ++ ",{asis,Tag},")"}); - #'Externaltypereference'{module=CurrMod,type=T} -> - emit({indent(9),T," ->",nl,indent(12),"'enc_",T, - "'(Val, TagIn ++ ",{asis,Tag},")"}); - #'Externaltypereference'{module=ExtMod,type=T} -> - emit({indent(9),T," ->",nl,indent(12),ExtMod,":'enc_", - T,"'(Val, TagIn ++ ",{asis,Tag},")"}) - end, - {[],0}. - -indent(N) -> - lists:duplicate(N,32). % 32 = space - - -gen_objset_dec(_,_,{unique,undefined},_,_,_,_) -> - %% There is no unique field in the class of this object set - %% don't bother about the constraint - ok; -gen_objset_dec(Erules,ObjSName,UniqueName,[{_,no_unique_value,_},T|Rest], - ClName,ClFields,NthObj)-> - gen_objset_dec(Erules,ObjSName,UniqueName,[T|Rest],ClName,ClFields, - NthObj); -gen_objset_dec(_Erules,ObjSetName,UniqueName,[{_,no_unique_value,_}], - _ClName,_ClFields,_NthObj)-> - emit_default_getdec(ObjSetName,UniqueName), - emit({".",nl,nl}); -gen_objset_dec(Erules,ObjSName,UniqueName,[{ObjName,Val,Fields},T|Rest], - ClName,ClFields,NthObj)-> - emit({"'getdec_",ObjSName,"'(",{asis,UniqueName},",",{asis,Val}, - ") ->",nl}), - CurrMod = get(currmod), - NewNthObj= - case ObjName of - {no_mod,no_name} -> - gen_inlined_dec_funs(Erules,Fields,ClFields,ObjSName, - NthObj); - {CurrMod,Name} -> - emit({" fun 'dec_",Name,"'/4"}), - NthObj; - {ModName,Name} -> - emit_ext_fun(dec,ModName,Name), -% emit([" {'",ModName,"', 'dec_",Name,"'}"]), - NthObj; - _Other -> - emit({" fun 'dec_",ObjName,"'/4"}), - NthObj - end, - emit({";",nl}), - gen_objset_dec(Erules,ObjSName,UniqueName,[T|Rest],ClName,ClFields, - NewNthObj); -gen_objset_dec(Erules,ObjSetName,UniqueName,[{ObjName,Val,Fields}],_ClName, - ClFields,NthObj) -> - emit({"'getdec_",ObjSetName,"'(",{asis,UniqueName},",",{asis,Val},") ->",nl}), - CurrMod = get(currmod), - case ObjName of - {no_mod,no_name} -> - gen_inlined_dec_funs(Erules,Fields,ClFields,ObjSetName, - NthObj); - {CurrMod,Name} -> - emit({" fun 'dec_",Name,"'/4"}); - {ModName,Name} -> - emit_ext_fun(dec,ModName,Name); -% emit([" {'",ModName,"', 'dec_",Name,"'}"]); - _Other -> - emit({" fun 'dec_",ObjName,"'/4"}) - end, - emit({";",nl}), - emit_default_getdec(ObjSetName,UniqueName), - emit({".",nl,nl}); -gen_objset_dec(_,ObjSetName,_UniqueName,['EXTENSIONMARK'],_ClName,_ClFields, - _NthObj) -> - emit({"'getdec_",ObjSetName,"'(_, _) ->",nl}), - emit({indent(3),"fun(_, Bytes, _, _) ->",nl}), - emit({indent(6),"Len = case Bytes of",nl,indent(9), - "Bin when is_binary(Bin) -> size(Bin);",nl,indent(9), - "_ -> length(Bytes)",nl,indent(6),"end,"}), - emit({indent(6),"{Bytes,[],Len}",nl}), - emit({indent(3),"end.",nl,nl}), - ok; -gen_objset_dec(Erule,ObjSetName,UniqueName, - ['EXTENSIONMARK','EXTENSIONMARK'|Rest], - ClName,ClFields,NthObj) -> - gen_objset_dec(Erule,ObjSetName,UniqueName,['EXTENSIONMARK'|Rest], - ClName,ClFields,NthObj); -gen_objset_dec(Erule,ObjSetName,UniqueName,['EXTENSIONMARK'|Rest], - ClName,ClFields,NthObj) -> - gen_objset_dec(Erule,ObjSetName,UniqueName,Rest++['EXTENSIONMARK'], - ClName,ClFields,NthObj); -gen_objset_dec(_,_,_,[],_,_,_) -> - ok. - -emit_default_getdec(ObjSetName,UniqueName) -> - emit(["'getdec_",ObjSetName,"'(",{asis,UniqueName},", ErrV) ->",nl]), - emit([indent(3),"fun(C,V,_,_) -> exit({{component,C},{value,V}, {unique_name_and_value,",{asis,UniqueName},", ErrV}}) end"]). - -gen_inlined_dec_funs(Erules,Fields,[{typefield,Name,Prop}|Rest], - ObjSetName,NthObj) -> - DecProp = case Prop of - 'OPTIONAL' -> opt_or_default; - {'DEFAULT',_} -> opt_or_default; - _ -> mandatory - end, - CurrMod = get(currmod), - InternalDefFunName = [NthObj,Name,ObjSetName], - case lists:keysearch(Name,1,Fields) of - {value,{_,Type}} when is_record(Type,type) -> - emit({indent(3),"fun(Type, Bytes, TagIn, _RestPrimFieldName) ->", - nl,indent(6),"case Type of",nl}), - N=emit_inner_of_decfun(Erules,Type,DecProp,InternalDefFunName), - gen_inlined_dec_funs1(Erules,Fields,Rest,ObjSetName,NthObj+N); - {value,{_,Type}} when is_record(Type,typedef) -> - emit({indent(3),"fun(Type, Bytes, TagIn, _RestPrimFieldName) ->", - nl,indent(6),"case Type of",nl}), - emit({indent(9),{asis,Name}," ->",nl}), - N=emit_inner_of_decfun(Erules,Type,DecProp,InternalDefFunName), - gen_inlined_dec_funs1(Erules,Fields,Rest,ObjSetName,NthObj+N); - {value,{_,#'Externaltypereference'{module=M,type=T}}} -> - #typedef{typespec=Type} = asn1_db:dbget(M,T), - OTag = Type#type.tag, - Tag = [X#tag{class=decode_class(X#tag.class)}|| X <- OTag], - emit({indent(3),"fun(Type, Bytes, TagIn, _RestPrimFieldName) ->", - nl,indent(6),"case Type of",nl}), - emit({indent(9),{asis,Name}," ->",nl}), - if - M == CurrMod -> - emit([indent(12),"'dec_",T,"'(Bytes, ",DecProp, - ", TagIn ++ ",{asis,Tag},")"]); - true -> - emit([indent(12),"'",M,"':'dec_",T,"'(Bytes, ", - DecProp,", TagIn ++ ",{asis,Tag},")"]) - end, - gen_inlined_dec_funs1(Erules,Fields,Rest,ObjSetName,NthObj); - false -> - emit([indent(3),"fun(Type, Bytes, TagIn, _RestPrimFieldName) ->", - nl,indent(6),"case Type of",nl, - indent(9),{asis,Name}," ->",nl, - indent(12),"Len = case Bytes of",nl, - indent(15),"B when is_binary(B) -> size(B);",nl, - indent(15),"_ -> length(Bytes)",nl, - indent(12),"end,",nl, - indent(12),"{Bytes,[],Len}"]), - gen_inlined_dec_funs1(Erules,Fields,Rest,ObjSetName,NthObj) - end; -gen_inlined_dec_funs(Erules,Fields,[_H|Rest],ObjSetName,NthObj) -> - gen_inlined_dec_funs(Erules,Fields,Rest,ObjSetName,NthObj); -gen_inlined_dec_funs(_,_,[],_,NthObj) -> - NthObj. - -gen_inlined_dec_funs1(Erules,Fields,[{typefield,Name,Prop}|Rest], - ObjSetName,NthObj) -> - DecProp = case Prop of - 'OPTIONAL' -> opt_or_default; - {'DEFAULT',_} -> opt_or_default; - _ -> mandatory - end, - CurrMod = get(currmod), - InternalDefFunName = [NthObj,Name,ObjSetName], - N= - case lists:keysearch(Name,1,Fields) of - {value,{_,Type}} when is_record(Type,type) -> - emit({";",nl}), - emit_inner_of_decfun(Erules,Type,DecProp,InternalDefFunName); - {value,{_,Type}} when is_record(Type,typedef) -> - emit({";",nl,indent(9),{asis,Name}," ->",nl}), - emit_inner_of_decfun(Erules,Type,DecProp,InternalDefFunName); - {value,{_,#'Externaltypereference'{module=M,type=T}}} -> - #typedef{typespec=Type} = asn1_db:dbget(M,T), - OTag = Type#type.tag, - Tag = [X#tag{class=decode_class(X#tag.class)}|| X <- OTag], - emit({";",nl,indent(9),{asis,Name}," ->",nl}), - if - M == CurrMod -> - emit([indent(12),"'dec_",T,"'(Bytes, ",DecProp, - ", TagIn ++ ",{asis,Tag},")"]); - true -> - emit([indent(12),"'",M,"':'dec_",T,"'(Bytes, ", - DecProp,", TagIn ++ ",{asis,Tag},")"]) - end, - 0; - false -> - emit([";",nl, - indent(9),{asis,Name}," ->",nl, - indent(12),"Len = case Bytes of",nl, - indent(15),"B when is_binary(B) -> size(B);",nl, - indent(15),"_ -> length(Bytes)",nl, - indent(12),"end,",nl, - indent(12),"{Bytes,[],Len}"]), - 0 - end, - gen_inlined_dec_funs1(Erules,Fields,Rest,ObjSetName,NthObj+N); -gen_inlined_dec_funs1(Erules,Fields,[_H|Rest],ObjSetName,NthObj)-> - gen_inlined_dec_funs1(Erules,Fields,Rest,ObjSetName,NthObj); -gen_inlined_dec_funs1(_,_,[],_,NthObj) -> - emit({nl,indent(6),"end",nl}), - emit({indent(3),"end"}), - NthObj. - -emit_inner_of_decfun(Erules,#typedef{name={ExtName,Name},typespec=Type}, - Prop,InternalDefFunName) -> - OTag = Type#type.tag, - Tag = [X#tag{class=decode_class(X#tag.class)}|| X <- OTag], - case {ExtName,Name} of - {primitive,bif} -> - emit(indent(12)), - gen_dec_prim(Erules,Type,"Bytes",Tag,"TagIn",no_length, - ?PRIMITIVE,Prop), - 0; - {constructed,bif} -> - emit({indent(12),"'dec_", - asn1ct_gen:list2name(InternalDefFunName),"'(Bytes, ",Prop, - ", TagIn ++ ",{asis,Tag},")"}), - 1; - _ -> - emit({indent(12),"'",ExtName,"':'dec_",Name,"'(Bytes, ",Prop, - ", TagIn ++ ",{asis,Tag},")"}), - 0 - end; -emit_inner_of_decfun(_,#typedef{name=Name,typespec=Type},Prop,_) -> - OTag = Type#type.tag, - Tag = [X#tag{class=decode_class(X#tag.class)}|| X <- OTag], - emit({indent(12),"'dec_",Name,"'(Bytes, ",Prop,", TagIn ++ ", - {asis,Tag},")"}), - 0; -emit_inner_of_decfun(Erules,Type,Prop,_) when is_record(Type,type) -> - OTag = Type#type.tag, - Tag = [X#tag{class=decode_class(X#tag.class)}|| X <- OTag], - CurrMod = get(currmod), - Def = Type#type.def, - InnerType = asn1ct_gen:get_inner(Def), - WhatKind = asn1ct_gen:type(InnerType), - case WhatKind of - {primitive,bif} -> - emit({indent(9),Def," ->",nl,indent(12)}), - gen_dec_prim(Erules,Type,"Bytes",Tag,"TagIn",no_length, - ?PRIMITIVE,Prop); -% TRef when is_record(TRef,typereference) -> -% T = TRef#typereference.val, -% emit({indent(9),T," ->",nl,indent(12),"'dec_",T,"'(Val)"}); - #'Externaltypereference'{module=CurrMod,type=T} -> - emit({indent(9),T," ->",nl,indent(12),"'dec_",T, - "'(Bytes, ",Prop,", TagIn ++ ",{asis,Tag},")"}); - #'Externaltypereference'{module=ExtMod,type=T} -> - emit({indent(9),T," ->",nl,indent(12),ExtMod,":'dec_", - T,"'(Bytes, ",Prop,", TagIn ++ ",{asis,Tag},")"}) - end, - 0. - - -gen_internal_funcs(_,[]) -> - ok; -gen_internal_funcs(Erules,[TypeDef|Rest]) -> - gen_encode_user(Erules,TypeDef), - emit({"'dec_",TypeDef#typedef.name,"'(Bytes, ", - unused_optormand_var("OptOrMand",(TypeDef#typedef.typespec)#type.def),", TagIn) ->",nl}), - gen_decode_user(Erules,TypeDef), - gen_internal_funcs(Erules,Rest). - - -dbdec(Type) -> - demit({"io:format(\"decoding: ",{asis,Type},"~w~n\",[Bytes]),",nl}). - - -decode_class('UNIVERSAL') -> - ?UNIVERSAL; -decode_class('APPLICATION') -> - ?APPLICATION; -decode_class('CONTEXT') -> - ?CONTEXT; -decode_class('PRIVATE') -> - ?PRIVATE. - -decode_type('BOOLEAN') -> 1; -decode_type('INTEGER') -> 2; -decode_type('BIT STRING') -> 3; -decode_type('OCTET STRING') -> 4; -decode_type('NULL') -> 5; -decode_type('OBJECT IDENTIFIER') -> 6; -decode_type('ObjectDescriptor') -> 7; -decode_type('EXTERNAL') -> 8; -decode_type('REAL') -> 9; -decode_type('ENUMERATED') -> 10; -decode_type('EMBEDDED_PDV') -> 11; -decode_type('UTF8String') -> 12; -decode_type('RELATIVE-OID') -> 13; -decode_type('SEQUENCE') -> 16; -decode_type('SEQUENCE OF') -> 16; -decode_type('SET') -> 17; -decode_type('SET OF') -> 17; -decode_type('NumericString') -> 18; -decode_type('PrintableString') -> 19; -decode_type('TeletexString') -> 20; -decode_type('T61String') -> 20; -decode_type('VideotexString') -> 21; -decode_type('IA5String') -> 22; -decode_type('UTCTime') -> 23; -decode_type('GeneralizedTime') -> 24; -decode_type('GraphicString') -> 25; -decode_type('VisibleString') -> 26; -decode_type('GeneralString') -> 27; -decode_type('UniversalString') -> 28; -decode_type('BMPString') -> 30; -decode_type('CHOICE') -> 'CHOICE'; % choice gets the tag from the actual alternative -decode_type(Else) -> exit({error,{asn1,{unrecognized_type,Else}}}). - -add_removed_bytes() -> - asn1ct_name:delete(rb), - add_removed_bytes(asn1ct_name:all(rb)). - -add_removed_bytes([H,T1|T]) -> - emit({{var,H},"+"}), - add_removed_bytes([T1|T]); -add_removed_bytes([H|T]) -> - emit({{var,H}}), - add_removed_bytes(T); -add_removed_bytes([]) -> - true. - -mkfuncname(WhatKind,DecOrEnc) -> - case WhatKind of - #'Externaltypereference'{module=Mod,type=EType} -> - CurrMod = get(currmod), - case CurrMod of - Mod -> - lists:concat(["'",DecOrEnc,"_",EType,"'"]); - _ -> -% io:format("CurrMod: ~p, Mod: ~p~n",[CurrMod,Mod]), - lists:concat(["'",Mod,"':'",DecOrEnc,"_",EType,"'"]) - end; - #'typereference'{val=EType} -> - lists:concat(["'",DecOrEnc,"_",EType,"'"]); - 'ASN1_OPEN_TYPE' -> - lists:concat(["'",DecOrEnc,"_",WhatKind,"'"]) - - end. - -optionals(L) -> optionals(L,[],1). - -optionals([{'EXTENSIONMARK',_,_}|Rest],Acc,Pos) -> - optionals(Rest,Acc,Pos); % optionals in extension are currently not handled -optionals([#'ComponentType'{name=Name,prop='OPTIONAL'}|Rest],Acc,Pos) -> - optionals(Rest,[{Name,Pos}|Acc],Pos+1); -optionals([#'ComponentType'{name=Name,prop={'DEFAULT',_}}|Rest],Acc,Pos) -> - optionals(Rest,[{Name,Pos}|Acc],Pos+1); -optionals([#'ComponentType'{}|Rest],Acc,Pos) -> - optionals(Rest,Acc,Pos+1); -optionals([],Acc,_) -> - lists:reverse(Acc). - -get_constraint(C,Key) -> - case lists:keysearch(Key,1,C) of - false -> - no; - {value,{_,V}} -> - V - end. - -%% if the original option was ber and it has been wrapped to ber_bin -%% turn it back to ber -re_wrap_erule(ber_bin) -> - case get(encoding_options) of - Options when is_list(Options) -> - case lists:member(ber,Options) of - true -> ber; - _ -> ber_bin - end; - _ -> ber_bin - end; -re_wrap_erule(Erule) -> - Erule. - -is_already_generated(Operation,Name) -> - case get(class_default_type) of - undefined -> - put(class_default_type,[{Operation,Name}]), - false; - GeneratedList -> - case lists:member({Operation,Name},GeneratedList) of - true -> - true; - false -> - put(class_default_type,[{Operation,Name}|GeneratedList]), - false - end - end. - -get_class_fields(#classdef{typespec=ObjClass}) -> - ObjClass#objectclass.fields; -get_class_fields(#objectclass{fields=Fields}) -> - Fields; -get_class_fields(_) -> - []. - -get_object_field(Name,ObjectFields) -> - case lists:keysearch(Name,1,ObjectFields) of - {value,Field} -> Field; - false -> false - end. - -%% For BER the ExtensionAdditionGroup notation has no impact on the encoding/decoding -%% and therefore we only filter away the ExtensionAdditionGroup start and end markers -%% -extaddgroup2sequence(ExtList) when is_list(ExtList) -> - lists:filter(fun(#'ExtensionAdditionGroup'{}) -> - false; - ('ExtensionAdditionGroupEnd') -> - false; - (_) -> - true - end, ExtList). diff --git a/lib/asn1/src/asn1ct_gen_ber_bin_v2.erl b/lib/asn1/src/asn1ct_gen_ber_bin_v2.erl index 3ccfca3784..00c3dd98b2 100644 --- a/lib/asn1/src/asn1ct_gen_ber_bin_v2.erl +++ b/lib/asn1/src/asn1ct_gen_ber_bin_v2.erl @@ -116,22 +116,7 @@ gen_encode(Erules,Typename,Type) when is_record(Type,type) -> end, case lists:member(InnerType,['SET','SEQUENCE']) of true -> - case get(asn_keyed_list) of - true -> - CompList = - case Type#type.def of - #'SEQUENCE'{components=Cl} -> Cl; - #'SET'{components=Cl} -> Cl - end, - emit([nl,"'enc_",asn1ct_gen:list2name(Typename), - "'(Val, TagIn",ObjFun, - ") when is_list(Val) ->",nl]), - emit([" 'enc_",asn1ct_gen:list2name(Typename), - "'(?RT_BER:fixoptionals(", - {asis,optionals(CompList)}, - ",Val), TagIn",ObjFun,");",nl,nl]); - _ -> true - end; + true; _ -> emit([nl,"'enc_",asn1ct_gen:list2name(Typename), "'({'",asn1ct_gen:list2name(Typename), @@ -175,22 +160,7 @@ gen_encode_user(Erules,D) when is_record(D,typedef) -> case lists:member(InnerType,['SET','SEQUENCE']) of true -> - case get(asn_keyed_list) of - true -> - CompList = - case Type#type.def of - #'SEQUENCE'{components=Cl} -> Cl; - #'SET'{components=Cl} -> Cl - end, - - emit([nl,"'enc_",asn1ct_gen:list2name(Typename), - "'(Val, TagIn) when is_list(Val) ->",nl]), - emit([" 'enc_",asn1ct_gen:list2name(Typename), - "'(?RT_BER:fixoptionals(", - {asis,optionals(CompList)}, - ",Val), TagIn);",nl,nl]); - _ -> true - end; + true; _ -> emit({nl,"'enc_",asn1ct_gen:list2name(Typename), "'({'",asn1ct_gen:list2name(Typename),"',Val}, TagIn) ->",nl}), @@ -1504,7 +1474,7 @@ gen_objset_dec(Erules,ObjSetName,_UniqueName,['EXTENSIONMARK'],_ClName, emit([indent(2),"fun(_,Bytes, _RestPrimFieldName) ->",nl]), case Erules of - ber_bin_v2 -> + ber -> emit([indent(4),"case Bytes of",nl, indent(6),"Bin when is_binary(Bin) -> ",nl, indent(8),"Bin;",nl, @@ -1772,19 +1742,6 @@ mkfuncname(WhatKind,DecOrEnc) -> end. -optionals(L) -> optionals(L,[],1). - -optionals([{'EXTENSIONMARK',_,_}|Rest],Acc,Pos) -> - optionals(Rest,Acc,Pos); % optionals in extension are currently not handled -optionals([#'ComponentType'{name=Name,prop='OPTIONAL'}|Rest],Acc,Pos) -> - optionals(Rest,[{Name,Pos}|Acc],Pos+1); -optionals([#'ComponentType'{name=Name,prop={'DEFAULT',_}}|Rest],Acc,Pos) -> - optionals(Rest,[{Name,Pos}|Acc],Pos+1); -optionals([#'ComponentType'{}|Rest],Acc,Pos) -> - optionals(Rest,Acc,Pos+1); -optionals([],Acc,_) -> - lists:reverse(Acc). - get_constraint(C,Key) -> case lists:keysearch(Key,1,C) of false -> diff --git a/lib/asn1/src/asn1ct_gen_per.erl b/lib/asn1/src/asn1ct_gen_per.erl index bd5b81991d..eb4cfdccc6 100644 --- a/lib/asn1/src/asn1ct_gen_per.erl +++ b/lib/asn1/src/asn1ct_gen_per.erl @@ -25,6 +25,7 @@ -include("asn1_records.hrl"). %-compile(export_all). +-export([gen_dec_imm/2]). -export([pgen/4,gen_dec_prim/3,gen_encode_prim/4]). -export([gen_obj_code/3,gen_objectset_code/2]). -export([gen_decode/2, gen_decode/3]). @@ -146,16 +147,18 @@ gen_encode_prim(Erules,D,DoTag,Value) when is_record(D,type) -> case D#type.def of 'INTEGER' -> emit({"?RT_PER:encode_integer(", %fel - {asis,effective_constraint(integer,Constraint)},",",Value,")"}); + {asis,asn1ct_imm:effective_constraint(integer,Constraint)}, + ",",Value,")"}); {'INTEGER',NamedNumberList} -> emit({"?RT_PER:encode_integer(", - {asis,effective_constraint(integer,Constraint)},",",Value,",", + {asis,asn1ct_imm:effective_constraint(integer,Constraint)}, + ",",Value,",", {asis,NamedNumberList},")"}); {'ENUMERATED',{Nlist1,Nlist2}} -> NewList = lists:concat([[{0,X}||{X,_} <- Nlist1],['EXT_MARK'],[{1,X}||{X,_} <- Nlist2]]), NewC = [{'ValueRange',{0,length(Nlist1)-1}}], case Erules of - uper_bin -> + uper -> emit(["case ",Value," of",nl]); _ -> emit(["case (case ",Value," of {_,",{curr,enumval},"}-> ", @@ -168,7 +171,7 @@ gen_encode_prim(Erules,D,DoTag,Value) when is_record(D,type) -> NewList = [X||{X,_} <- NamedNumberList], NewC = [{'ValueRange',{0,length(NewList)-1}}], case Erules of - uper_bin -> + uper -> emit(["case ",Value," of",nl]); _ -> emit(["case (case ",Value," of {_,",{curr,enumval}, @@ -184,7 +187,7 @@ gen_encode_prim(Erules,D,DoTag,Value) when is_record(D,type) -> {asis,Constraint},",",Value,",", {asis,NamedNumberList},")"}); 'NULL' -> - emit({"?RT_PER:encode_null(",Value,")"}); + emit("[]"); 'OBJECT IDENTIFIER' -> emit({"?RT_PER:encode_object_identifier(",Value,")"}); 'RELATIVE-OID' -> @@ -274,7 +277,7 @@ emit_enc_enumerated_cases(Erule, C, [H1,H2|T], Count) -> -emit_enc_enumerated_case(uper_bin,_C, {asn1_enum,High}, _) -> +emit_enc_enumerated_case(uper,_C, {asn1_enum,High}, _) -> emit([ "{asn1_enum,EnumV} when is_integer(EnumV), EnumV > ",High," -> ", "[<<1:1>>,?RT_PER:encode_small_number(EnumV)]"]); @@ -284,118 +287,27 @@ emit_enc_enumerated_case(_Per,_C, {asn1_enum,High}, _) -> "[{bit,1},?RT_PER:encode_small_number(EnumV)]"]); emit_enc_enumerated_case(_Erule, _C, 'EXT_MARK', _Count) -> true; -emit_enc_enumerated_case(uper_bin,_C, {1,EnumName}, Count) -> +emit_enc_enumerated_case(uper,_C, {1,EnumName}, Count) -> emit(["'",EnumName,"' -> [<<1:1>>,?RT_PER:encode_small_number(",Count,")]"]); emit_enc_enumerated_case(_Per,_C, {1,EnumName}, Count) -> emit(["'",EnumName,"' -> [{bit,1},?RT_PER:encode_small_number(",Count,")]"]); -emit_enc_enumerated_case(uper_bin,C, {0,EnumName}, Count) -> +emit_enc_enumerated_case(uper,C, {0,EnumName}, Count) -> emit(["'",EnumName,"' -> [<<0:1>>,?RT_PER:encode_integer(",{asis,C},", ",Count,")]"]); emit_enc_enumerated_case(_Per,C, {0,EnumName}, Count) -> emit(["'",EnumName,"' -> [{bit,0},?RT_PER:encode_integer(",{asis,C},", ",Count,")]"]); emit_enc_enumerated_case(_Erule, C, EnumName, Count) -> emit(["'",EnumName,"' -> ?RT_PER:encode_integer(",{asis,C},", ",Count,")"]). -%% effective_constraint(Type,C) -%% Type = atom() -%% C = [C1,...] -%% C1 = {'SingleValue',SV} | {'ValueRange',VR} | {atom(),term()} -%% SV = integer() | [integer(),...] -%% VR = {Lb,Ub} -%% Lb = 'MIN' | integer() -%% Ub = 'MAX' | integer() -%% Returns a single value if C only has a single value constraint, and no -%% value range constraints, that constrains to a single value, otherwise -%% returns a value range that has the lower bound set to the lowest value -%% of all single values and lower bound values in C and the upper bound to -%% the greatest value. -effective_constraint(integer,[C={{_,_},_}|_Rest]) -> % extension - [C]; %% [C|effective_constraint(integer,Rest)]; XXX what is possible ??? -effective_constraint(integer,C) -> - SVs = get_constraints(C,'SingleValue'), - SV = effective_constr('SingleValue',SVs), - VRs = get_constraints(C,'ValueRange'), - VR = effective_constr('ValueRange',VRs), - greatest_common_range(SV,VR). - -effective_constr(_,[]) -> - []; -effective_constr('SingleValue',List) -> - SVList = lists:flatten(lists:map(fun(X)->element(2,X)end,List)), - %% Sort and remove duplicates before generating SingleValue or ValueRange - %% In case of ValueRange, also check for 'MIN and 'MAX' - case lists:usort(SVList) of - [N] -> - [{'SingleValue',N}]; - L when is_list(L) -> - [{'ValueRange',{least_Lb(L),greatest_Ub(L)}}] - end; -effective_constr('ValueRange',List) -> - LBs = lists:map(fun({_,{Lb,_}})-> Lb end,List), - UBs = lists:map(fun({_,{_,Ub}})-> Ub end,List), - Lb = least_Lb(LBs), - [{'ValueRange',{Lb,lists:max(UBs)}}]. - -greatest_common_range([],VR) -> - VR; -greatest_common_range(SV,[]) -> - SV; -greatest_common_range(SV,VR) -> - greatest_common_range2(mk_vr(SV),mk_vr(VR)). -greatest_common_range2({_,Int},{'MIN',Ub}) when is_integer(Int), - Int > Ub -> - [{'ValueRange',{'MIN',Int}}]; -greatest_common_range2({_,Int},{Lb,Ub}) when is_integer(Int), - Int < Lb -> - [{'ValueRange',{Int,Ub}}]; -greatest_common_range2({_,Int},VR={_Lb,_Ub}) when is_integer(Int) -> - [{'ValueRange',VR}]; -greatest_common_range2({_,L},{Lb,Ub}) when is_list(L) -> - Min = least_Lb([Lb|L]), - Max = greatest_Ub([Ub|L]), - [{'ValueRange',{Min,Max}}]; -greatest_common_range2({Lb1,Ub1},{Lb2,Ub2}) -> - Min = least_Lb([Lb1,Lb2]), - Max = greatest_Ub([Ub1,Ub2]), - [{'ValueRange',{Min,Max}}]. - -mk_vr([{Type,I}]) when is_atom(Type), is_integer(I) -> - {I,I}; -mk_vr([{Type,{Lb,Ub}}]) when is_atom(Type) -> - {Lb,Ub}; -mk_vr(Other) -> - Other. - -least_Lb(L) -> - case lists:member('MIN',L) of - true -> 'MIN'; - _ -> lists:min(L) - end. - -greatest_Ub(L) -> - case lists:member('MAX',L) of - true -> 'MAX'; - _ -> lists:max(L) - end. - - -get_constraints(L=[{Key,_}],Key) -> - L; -get_constraints([],_) -> - []; -get_constraints(C,Key) -> - {value,L} = keysearch_allwithkey(Key,1,C,[]), - L. - -keysearch_allwithkey(Key,Ix,C,Acc) -> - case lists:keysearch(Key,Ix,C) of - false -> - {value,Acc}; - {value,T} -> - RestC = lists:delete(T,C), - keysearch_allwithkey(Key,Ix,RestC,[T|Acc]) +get_constraint([{Key,V}], Key) -> + V; +get_constraint([], _) -> + no; +get_constraint(C, Key) -> + case lists:keyfind(Key, 1, C) of + false -> no; + {Key,V} -> V end. - %% Object code generating for encoding and decoding %% ------------------------------------------------ @@ -442,7 +354,7 @@ gen_encode_objectfields(Erule,ClassName,[{typefield,Name,OptOrMand}|Rest], {false,'OPTIONAL'} -> EmitFuncClause("Val"), case Erule of - uper_bin -> + uper -> emit(" Val"); _ -> emit(" [{octets,Val}]") @@ -833,7 +745,7 @@ gen_objset_enc(Erule,ObjSetName,_UniqueName,['EXTENSIONMARK'],_ClName, emit({"'getenc_",ObjSetName,"'(_, _) ->",nl}), emit({indent(3),"fun(_, Val, _) ->",nl}), case Erule of - uper_bin -> + uper -> emit([indent(6),"Val",nl]); _ -> emit([indent(6),"[{octets,Val}]",nl]) @@ -883,7 +795,7 @@ gen_inlined_enc_funs(Erule,Fields,[{typefield,Name,_}|Rest],ObjSetName,NthObj) - emit({indent(9),{asis,Name}," ->",nl}), emit([indent(12),"'",M,"'",":'enc_",T,"'(Val)"]), gen_inlined_enc_funs1(Erule,Fields,Rest,ObjSetName,NthObj,[]); - false when Erule == uper_bin -> + false when Erule =:= uper -> emit([indent(3),"fun(Type,Val,_) ->",nl, indent(6),"case Type of",nl, indent(9),{asis,Name}," -> Val",nl]), @@ -921,7 +833,7 @@ gen_inlined_enc_funs1(Erule,Fields,[{typefield,Name,_}|Rest],ObjSetName, emit({";",nl,indent(9),{asis,Name}," ->",nl}), emit([indent(12),"'",M,"'",":'enc_",T,"'(Val)"]), {Acc,0}; - false when Erule == uper_bin -> + false when Erule =:= uper -> emit([";",nl, indent(9),{asis,Name}," -> ",nl, "Val",nl]), @@ -1228,19 +1140,43 @@ gen_decode_user(Erules,D) when is_record(D,typedef) -> exit({error,{asn1,{unknown,Other}}}) end. +gen_dec_imm(Erule, #type{def=Name,constraint=C}) -> + Aligned = case Erule of + uper -> false; + per -> true + end, + gen_dec_imm_1(Name, C, Aligned). + +gen_dec_imm_1('BOOLEAN', _Constr, _Aligned) -> + asn1ct_imm:per_dec_boolean(); +gen_dec_imm_1({'ENUMERATED',{Base,Ext}}, _Constr, Aligned) -> + asn1ct_imm:per_dec_enumerated(Base, Ext, Aligned); +gen_dec_imm_1({'ENUMERATED',NamedNumberList}, _Constr, Aligned) -> + asn1ct_imm:per_dec_enumerated(NamedNumberList, Aligned); +gen_dec_imm_1('INTEGER', Constr, Aligned) -> + asn1ct_imm:per_dec_integer(Constr, Aligned); +gen_dec_imm_1({'INTEGER',NamedNumberList}, Constraint, Aligned) -> + asn1ct_imm:per_dec_named_integer(Constraint, + NamedNumberList, + Aligned); +gen_dec_imm_1('OCTET STRING', Constraint, Aligned) -> + SzConstr = get_constraint(Constraint, 'SizeConstraint'), + Imm = asn1ct_imm:per_dec_octet_string(SzConstr, Aligned), + {convert,binary_to_list,Imm}; +gen_dec_imm_1(_, _, _) -> no. + +gen_dec_prim(Erule, Type, BytesVar) -> + case gen_dec_imm(Erule, Type) of + no -> + gen_dec_prim_1(Erule, Type, BytesVar); + Imm -> + asn1ct_imm:dec_code_gen(Imm, BytesVar) + end. -gen_dec_prim(Erules,Att,BytesVar) -> - Typename = Att#type.def, - Constraint = Att#type.constraint, +gen_dec_prim_1(Erule, + #type{def=Typename,constraint=Constraint}=Att, + BytesVar) -> case Typename of - 'INTEGER' -> - emit({"?RT_PER:decode_integer(",BytesVar,",", - {asis,effective_constraint(integer,Constraint)},")"}); - {'INTEGER',NamedNumberList} -> - emit({"?RT_PER:decode_integer(",BytesVar,",", - {asis,effective_constraint(integer,Constraint)},",", - {asis,NamedNumberList},")"}); - 'REAL' -> emit({"?RT_PER:decode_real(",BytesVar,")"}); @@ -1256,8 +1192,7 @@ gen_dec_prim(Erules,Att,BytesVar) -> {asis,NamedNumberList},")"}) end; 'NULL' -> - emit({"?RT_PER:decode_null(", - BytesVar,")"}); + emit({"{'NULL',",BytesVar,"}"}); 'OBJECT IDENTIFIER' -> emit({"?RT_PER:decode_object_identifier(", BytesVar,")"}); @@ -1267,24 +1202,6 @@ gen_dec_prim(Erules,Att,BytesVar) -> 'ObjectDescriptor' -> emit({"?RT_PER:decode_ObjectDescriptor(", BytesVar,")"}); - {'ENUMERATED',{NamedNumberList1,NamedNumberList2}} -> - NewTup = {list_to_tuple([X||{X,_} <- NamedNumberList1]), - list_to_tuple([X||{X,_} <- NamedNumberList2])}, - NewC = [{'ValueRange',{0,size(element(1,NewTup))-1}}], - emit({"?RT_PER:decode_enumerated(",BytesVar,",", - {asis,NewC},",", - {asis,NewTup},")"}); - {'ENUMERATED',NamedNumberList} -> - NewTup = list_to_tuple([X||{X,_} <- NamedNumberList]), - NewC = [{'ValueRange',{0,size(NewTup)-1}}], - emit({"?RT_PER:decode_enumerated(",BytesVar,",", - {asis,NewC},",", - {asis,NewTup},")"}); - 'BOOLEAN'-> - emit({"?RT_PER:decode_boolean(",BytesVar,")"}); - 'OCTET STRING' -> - emit({"?RT_PER:decode_octet_string(",BytesVar,",", - {asis,Constraint},")"}); 'NumericString' -> emit({"?RT_PER:decode_NumericString(",BytesVar,",", {asis,Constraint},")"}); @@ -1323,7 +1240,7 @@ gen_dec_prim(Erules,Att,BytesVar) -> 'UTF8String' -> emit({"?RT_PER:decode_UTF8String(",BytesVar,")"}); 'ANY' -> - case Erules of + case Erule of per -> emit(["fun() -> {XTerm,YTermXBytes} = ?RT_PER:decode_open_type(",BytesVar,",",{asis,Constraint}, "), {binary_to_list(XTerm),XBytes} end ()"]); _ -> @@ -1345,7 +1262,7 @@ gen_dec_prim(Erules,Att,BytesVar) -> emit([" {YTerm,_} = dec_",Tname,"(XTerm,mandatory),",nl]), emit([" {YTerm,XBytes} end(",BytesVar,")"]); _ -> - case Erules of + case Erule of per -> emit(["fun() -> {XTerm,XBytes} = ?RT_PER:decode_open_type(",BytesVar,", []), {binary_to_list(XTerm),XBytes} end()"]); _ -> @@ -1353,11 +1270,11 @@ gen_dec_prim(Erules,Att,BytesVar) -> end end; #'ObjectClassFieldType'{} -> - case asn1ct_gen:get_inner(Att#type.def) of + case asn1ct_gen:get_inner(Typename) of {fixedtypevaluefield,_,InnerType} -> - gen_dec_prim(Erules,InnerType,BytesVar); + gen_dec_prim(Erule, InnerType, BytesVar); T -> - gen_dec_prim(Erules,Att#type{def=T},BytesVar) + gen_dec_prim(Erule, Att#type{def=T}, BytesVar) end; Other -> exit({'cant decode' ,Other}) diff --git a/lib/asn1/src/asn1ct_gen_per_rt2ct.erl b/lib/asn1/src/asn1ct_gen_per_rt2ct.erl index 16eec92847..ecd212c3e3 100644 --- a/lib/asn1/src/asn1ct_gen_per_rt2ct.erl +++ b/lib/asn1/src/asn1ct_gen_per_rt2ct.erl @@ -176,7 +176,7 @@ gen_encode_prim(Erules,D,DoTag,Value) when is_record(D,type) -> {asis,NamedNumberList},")"}) end; 'NULL' -> - emit({"?RT_PER:encode_null(",Value,")"}); + emit("[]"); 'OBJECT IDENTIFIER' -> emit({"?RT_PER:encode_object_identifier(",Value,")"}); 'RELATIVE-OID' -> @@ -450,17 +450,6 @@ emit_enc_octet_string(_Erules,Constraint,Value) -> emit({" ?RT_PER:encode_octet_string(",{asis,C},",false,",Value,")",nl}) end. -emit_dec_octet_string(Constraint,BytesVar) -> - case get_constraint(Constraint,'SizeConstraint') of - 0 -> - emit({" {[],",BytesVar,"}",nl}); - {_,0} -> - emit({" {[],",BytesVar,"}",nl}); - C -> - emit({" ?RT_PER:decode_octet_string(",BytesVar,",", - {asis,C},",false)",nl}) - end. - emit_enc_integer_case(Value) -> case get(component_type) of {true,#'ComponentType'{prop=Prop}} -> @@ -624,23 +613,6 @@ get_constraint(C,Key) -> V end. -get_constraints(L=[{Key,_}],Key) -> - L; -get_constraints([],_) -> - []; -get_constraints(C,Key) -> - {value,L} = keysearch_allwithkey(Key,1,C,[]), - L. - -keysearch_allwithkey(Key,Ix,C,Acc) -> - case lists:keysearch(Key,Ix,C) of - false -> - {value,Acc}; - {value,T} -> - RestC = lists:delete(T,C), - keysearch_allwithkey(Key,Ix,RestC,[T|Acc]) - end. - %% effective_constraint(Type,C) %% Type = atom() %% C = [C1,...] @@ -657,69 +629,9 @@ keysearch_allwithkey(Key,Ix,C,Acc) -> effective_constraint(integer,[C={{_,_},_}|_Rest]) -> % extension [C]; %% [C|effective_constraint(integer,Rest)]; XXX what is possible ??? effective_constraint(integer,C) -> - SVs = get_constraints(C,'SingleValue'), - SV = effective_constr('SingleValue',SVs), - VRs = get_constraints(C,'ValueRange'), - VR = effective_constr('ValueRange',VRs), - CRange = greatest_common_range(SV,VR), - pre_encode(integer,CRange); + pre_encode(integer, asn1ct_imm:effective_constraint(integer, C)); effective_constraint(bitstring,C) -> - get_constraint(C,'SizeConstraint'). - -effective_constr(_,[]) -> - []; -effective_constr('SingleValue',List) -> - SVList = lists:flatten(lists:map(fun(X)->element(2,X)end,List)), - %% Sort and remove duplicates before generating SingleValue or ValueRange - %% In case of ValueRange, also check for 'MIN and 'MAX' - case lists:usort(SVList) of - [N] -> - [{'SingleValue',N}]; - L when is_list(L) -> - [{'ValueRange',{least_Lb(L),greatest_Ub(L)}}] - end; -effective_constr('ValueRange',List) -> - LBs = lists:map(fun({_,{Lb,_}})-> Lb end,List), - UBs = lists:map(fun({_,{_,Ub}})-> Ub end,List), - Lb = least_Lb(LBs), - [{'ValueRange',{Lb,lists:max(UBs)}}]. - -greatest_common_range([],VR) -> - VR; -greatest_common_range(SV,[]) -> - SV; -greatest_common_range([{_,Int}],[{_,{'MIN',Ub}}]) when is_integer(Int), - Int > Ub -> - [{'ValueRange',{'MIN',Int}}]; -greatest_common_range([{_,Int}],[{_,{Lb,Ub}}]) when is_integer(Int), - Int < Lb -> - [{'ValueRange',{Int,Ub}}]; -greatest_common_range([{_,Int}],VR=[{_,{_Lb,_Ub}}]) when is_integer(Int) -> - VR; -greatest_common_range([{_,L}],[{_,{Lb,Ub}}]) when is_list(L) -> - Min = least_Lb([Lb|L]), - Max = greatest_Ub([Ub|L]), - [{'ValueRange',{Min,Max}}]; -greatest_common_range([{_,{Lb1,Ub1}}],[{_,{Lb2,Ub2}}]) -> - Min = least_Lb([Lb1,Lb2]), - Max = greatest_Ub([Ub1,Ub2]), - [{'ValueRange',{Min,Max}}]. - - -least_Lb(L) -> - case lists:member('MIN',L) of - true -> 'MIN'; - _ -> lists:min(L) - end. - -greatest_Ub(L) -> - case lists:member('MAX',L) of - true -> 'MAX'; - _ -> lists:max(L) - end. - - - + asn1ct_imm:effective_constraint(bitstring, C). pre_encode(integer,[]) -> []; @@ -1586,17 +1498,9 @@ gen_dec_prim(Erules,Att,BytesVar) -> Constraint = Att#type.constraint, case Typename of 'INTEGER' -> - EffectiveConstr = effective_constraint(integer,Constraint), - emit_dec_integer(EffectiveConstr,BytesVar); -% emit({"?RT_PER:decode_integer(",BytesVar,",", -% {asis,EffectiveConstr},")"}); - {'INTEGER',NamedNumberList} -> - EffectiveConstr = effective_constraint(integer,Constraint), - emit_dec_integer(EffectiveConstr,BytesVar,NamedNumberList); -% emit({"?RT_PER:decode_integer(",BytesVar,",", -% {asis,EffectiveConstr},",", -% {asis,NamedNumberList},")"}); - + asn1ct_gen_per:gen_dec_prim(Erules, Att, BytesVar); + {'INTEGER',_NamedNumberList} -> + asn1ct_gen_per:gen_dec_prim(Erules, Att, BytesVar); 'REAL' -> emit(["?RT_PER:decode_real(",BytesVar,")"]); @@ -1612,8 +1516,7 @@ gen_dec_prim(Erules,Att,BytesVar) -> {asis,NamedNumberList},")"}) end; 'NULL' -> - emit({"?RT_PER:decode_null(", - BytesVar,")"}); + emit({"{'NULL',",BytesVar,"}"}); 'OBJECT IDENTIFIER' -> emit({"?RT_PER:decode_object_identifier(", BytesVar,")"}); @@ -1623,23 +1526,13 @@ gen_dec_prim(Erules,Att,BytesVar) -> 'ObjectDescriptor' -> emit({"?RT_PER:decode_ObjectDescriptor(", BytesVar,")"}); - {'ENUMERATED',{NamedNumberList1,NamedNumberList2}} -> - NewTup = {list_to_tuple([X||{X,_} <- NamedNumberList1]), - list_to_tuple([X||{X,_} <- NamedNumberList2])}, - NewC = [{'ValueRange',{0,size(element(1,NewTup))-1}}], - emit({"?RT_PER:decode_enumerated(",BytesVar,",", - {asis,NewC},",", - {asis,NewTup},")"}); - {'ENUMERATED',NamedNumberList} -> - NewNNL = [X||{X,_} <- NamedNumberList], - NewC = effective_constraint(integer, - [{'ValueRange',{0,length(NewNNL)-1}}]), - emit_dec_enumerated(BytesVar,NewC,NewNNL); + {'ENUMERATED',_} -> + asn1ct_gen_per:gen_dec_prim(Erules, Att, BytesVar); 'BOOLEAN'-> - emit({"?RT_PER:decode_boolean(",BytesVar,")"}); + asn1ct_gen_per:gen_dec_prim(Erules, Att, BytesVar); 'OCTET STRING' -> - emit_dec_octet_string(Constraint,BytesVar); + asn1ct_gen_per:gen_dec_prim(Erules, Att, BytesVar); 'NumericString' -> emit_dec_known_multiplier_string('NumericString', @@ -1716,88 +1609,6 @@ gen_dec_prim(Erules,Att,BytesVar) -> exit({'cant decode' ,Other}) end. - -emit_dec_integer(C,BytesVar,NNL) -> - asn1ct_name:new(tmpterm), - asn1ct_name:new(buffer), - Tmpterm = asn1ct_gen:mk_var(asn1ct_name:curr(tmpterm)), - Buffer = asn1ct_gen:mk_var(asn1ct_name:curr(buffer)), - emit({" begin {",{curr,tmpterm},",",{curr,buffer},"} = ",nl}), - emit_dec_integer(C,BytesVar), - emit({",",nl," case ",Tmpterm," of",nl}), - lists:map(fun({Name,Int})->emit({" ",Int," -> {",{asis,Name},",", - Buffer,"};",nl}); - (_)-> exit({error,{asn1,{"error in named number list",NNL}}}) - end, - NNL), - emit({" _ -> {",Tmpterm,",",Buffer,"}",nl}), - emit({" end",nl}), % end of case - emit(" end"). % end of begin - -emit_dec_integer([{'SingleValue',Int}],BytesVar) when is_integer(Int) -> - emit(["{",Int,",",BytesVar,"}"]); -emit_dec_integer([{_,{Lb,_Ub},_Range,{BitsOrOctets,N}}],BytesVar) -> - GetBorO = - case BitsOrOctets of - bits -> "getbits"; - _ -> "getoctets" - end, - asn1ct_name:new(tmpterm), - asn1ct_name:new(tmpremain), - emit({" begin",nl," {",{curr,tmpterm},",",{curr,tmpremain},"}=", - "?RT_PER:",GetBorO,"(",BytesVar,",",N,"),",nl}), - emit({" {",{curr,tmpterm},"+",Lb,",",{curr,tmpremain},"}",nl, - " end"}); -emit_dec_integer([{_,{'MIN',_}}],BytesVar) -> - emit({"?RT_PER:decode_unconstrained_number(",BytesVar,")"}); -emit_dec_integer([{_,{Lb,'MAX'}}],BytesVar) -> - emit({"?RT_PER:decode_semi_constrained_number(",BytesVar,",",Lb,")"}); -emit_dec_integer([{'ValueRange',VR={Lb,Ub}}],BytesVar) -> - Range = Ub-Lb+1, - emit({"?RT_PER:decode_constrained_number(",BytesVar,",", - {asis,VR},",",Range,")"}); -emit_dec_integer(C=[{Rc,_}],BytesVar) when is_tuple(Rc) -> - emit({"?RT_PER:decode_integer(",BytesVar,",",{asis,C},")"}); -emit_dec_integer(_,BytesVar) -> - emit({"?RT_PER:decode_unconstrained_number(",BytesVar,")"}). - - -emit_dec_enumerated(BytesVar,C,NamedNumberList) -> - emit_dec_enumerated_begin(),% emits a begin if component - asn1ct_name:new(tmpterm), - Tmpterm = asn1ct_gen:mk_var(asn1ct_name:curr(tmpterm)), - asn1ct_name:new(tmpremain), - Tmpremain = asn1ct_gen:mk_var(asn1ct_name:curr(tmpremain)), - emit({" {",{curr,tmpterm},",",{curr,tmpremain},"} =",nl}), - emit_dec_integer(C,BytesVar), - emit({",",nl," case ",Tmpterm," of "}), - - Cases=lists:flatten(dec_enumerated_cases(NamedNumberList,Tmpremain,0)), - emit({Cases++"_->exit({error,{asn1,{decode_enumerated,{",Tmpterm, - ",",{asis,NamedNumberList},"}}}}) end",nl}), - emit_dec_enumerated_end(). - -emit_dec_enumerated_begin() -> - case get(component_type) of - {true,_} -> - emit({" begin",nl}); - _ -> ok - end. - -emit_dec_enumerated_end() -> - case get(component_type) of - {true,_} -> - emit(" end"); - _ -> ok - end. - - -dec_enumerated_cases([Name|Rest],Tmpremain,No) -> - io_lib:format("~w->{~w,~s};",[No,Name,Tmpremain])++ - dec_enumerated_cases(Rest,Tmpremain,No+1); -dec_enumerated_cases([],_,_) -> - "". - %% For PER the ExtensionAdditionGroup notation has significance for the encoding and decoding %% the components within the ExtensionAdditionGroup is treated in a similar way as if they %% have been specified within a SEQUENCE, therefore we construct a fake sequence type here diff --git a/lib/asn1/src/asn1ct_imm.erl b/lib/asn1/src/asn1ct_imm.erl new file mode 100644 index 0000000000..34bb0b8714 --- /dev/null +++ b/lib/asn1/src/asn1ct_imm.erl @@ -0,0 +1,626 @@ +%% +%% %CopyrightBegin% +%% +%% Copyright Ericsson AB 2012. All Rights Reserved. +%% +%% The contents of this file are subject to the Erlang Public License, +%% Version 1.1, (the "License"); you may not use this file except in +%% compliance with the License. You should have received a copy of the +%% Erlang Public License along with this software. If not, it can be +%% retrieved online at http://www.erlang.org/. +%% +%% Software distributed under the License is distributed on an "AS IS" +%% basis, WITHOUT WARRANTY OF ANY KIND, either express or implied. See +%% the License for the specific language governing rights and limitations +%% under the License. +%% +%% %CopyrightEnd% +%% +%% +-module(asn1ct_imm). +-export([per_dec_boolean/0,per_dec_enumerated/2,per_dec_enumerated/3, + per_dec_extension_map/1, + per_dec_integer/2,per_dec_length/3,per_dec_named_integer/3, + per_dec_octet_string/2,per_dec_open_type/1]). +-export([optimize_alignment/1,optimize_alignment/2, + dec_slim_cg/2,dec_code_gen/2]). +-export([effective_constraint/2]). +-import(asn1ct_gen, [emit/1]). + +-record(st, {var, + base}). + +dec_slim_cg(Imm0, BytesVar) -> + {Imm,_} = optimize_alignment(Imm0), + asn1ct_name:new(v), + [H|T] = atom_to_list(asn1ct_name:curr(v)) ++ "@", + VarBase = [H-($a-$A)|T], + St0 = #st{var=0,base=VarBase}, + {Res,Pre,_} = flatten(Imm, BytesVar, St0), + dcg_list_outside(Pre), + Res. + +dec_code_gen(Imm, BytesVar) -> + emit(["begin",nl]), + {Dst,DstBuf} = dec_slim_cg(Imm, BytesVar), + emit([",",nl, + "{",Dst,",",DstBuf,"}",nl, + "end"]), + ok. + +optimize_alignment(Imm) -> + opt_al(Imm, unknown). + +optimize_alignment(Imm, Al) -> + opt_al(Imm, Al). + + +per_dec_boolean() -> + {map,{get_bits,1,[1]},[{0,false},{1,true}]}. + +per_dec_enumerated(NamedList0, Aligned) -> + Constraint = [{'ValueRange',{0,length(NamedList0)-1}}], + NamedList = per_dec_enumerated_fix_list(NamedList0, [enum_error], 0), + Int = per_dec_integer(Constraint, Aligned), + {map,Int,NamedList}. + +per_dec_enumerated(BaseNamedList, NamedListExt0, Aligned) -> + Base = per_dec_enumerated(BaseNamedList, Aligned), + NamedListExt = per_dec_enumerated_fix_list(NamedListExt0, + [enum_default], 0), + Ext = {map,per_dec_normally_small_number(Aligned),NamedListExt}, + bit_case(Base, Ext). + +per_dec_extension_map(Aligned) -> + Len = {add,per_dec_normally_small_number(Aligned),1}, + {get_bits,Len,[1,bitstring]}. + +per_dec_integer(Constraint0, Aligned) -> + Constraint = effective_constraint(integer, Constraint0), + per_dec_integer_1(Constraint, Aligned). + +per_dec_length(SingleValue, _, _Aligned) when is_integer(SingleValue) -> + {value,SingleValue}; +per_dec_length({S,S}, _, _Aligned) when is_integer(S) -> + {value,S}; +per_dec_length({{_,_}=Constr,_}, AllowZero, Aligned) -> + bit_case(per_dec_length(Constr, AllowZero, Aligned), + per_dec_length(undefined, AllowZero, Aligned)); +per_dec_length({Lb,Ub}, _AllowZero, Aligned) when is_integer(Lb), + is_integer(Lb), + Ub =< 65535 -> + per_dec_constrained(Lb, Ub, Aligned); +per_dec_length({_,_}, AllowZero, Aligned) -> + decode_unconstrained_length(AllowZero, Aligned); +per_dec_length(undefined, AllowZero, Aligned) -> + decode_unconstrained_length(AllowZero, Aligned). + +per_dec_named_integer(Constraint, NamedList0, Aligned) -> + Int = per_dec_integer(Constraint, Aligned), + NamedList = [{K,V} || {V,K} <- NamedList0] ++ [integer_default], + {map,Int,NamedList}. + +per_dec_octet_string(Constraint, Aligned) -> + dec_string(Constraint, 8, Aligned). + +per_dec_open_type(Aligned) -> + {get_bits,decode_unconstrained_length(true, Aligned), + [8,binary,{align,Aligned}]}. + + +%%% +%%% Local functions. +%%% + +dec_string(Sv, U, _Aligned) when is_integer(Sv), U*Sv =< 16 -> + {get_bits,Sv,[U,binary]}; +dec_string(Sv, U, Aligned) when is_integer(Sv), Sv < 16#10000 -> + {get_bits,Sv,[U,binary,{align,Aligned}]}; +dec_string(C, U, Aligned) when is_list(C) -> + dec_string({hd(C),lists:max(C)}, U, Aligned); +dec_string({Sv,Sv}, U, Aligned) -> + dec_string(Sv, U, Aligned); +dec_string({{_,_}=C,_}, U, Aligned) -> + bit_case(dec_string(C, U, Aligned), + dec_string(no, U, Aligned)); +dec_string({Lb,Ub}, U, Aligned) when Ub < 16#10000 -> + Len = per_dec_constrained(Lb, Ub, Aligned), + {get_bits,Len,[U,binary,{align,Aligned}]}; +dec_string(_, U, Aligned) -> + Al = [{align,Aligned}], + DecRest = fun(V, Buf) -> + emit(["?RT_PER:decode_fragmented(",V,", ", + Buf,", ",U,")"]) + end, + {'case',[{test,{get_bits,1,[1|Al]},0, + {value,{get_bits, + {get_bits,7,[1]}, + [U,binary]}}}, + {test,{get_bits,1,[1|Al]},1, + {test,{get_bits,1,[1]},0, + {value,{get_bits, + {get_bits,14,[1]}, + [U,binary]}}}}, + {test,{get_bits,1,[1|Al]},1, + {test,{get_bits,1,[1]},1, + {value,{call,DecRest,{get_bits,6,[1]}}}}}]}. + +per_dec_enumerated_fix_list([{V,_}|T], Tail, N) -> + [{N,V}|per_dec_enumerated_fix_list(T, Tail, N+1)]; +per_dec_enumerated_fix_list([], Tail, _) -> Tail. + +per_dec_integer_1([{'SingleValue',Value}], _Aligned) -> + {value,Value}; +per_dec_integer_1([{'ValueRange',{Lb,'MAX'}}], Aligned) when is_integer(Lb) -> + per_dec_unconstrained(Aligned); +per_dec_integer_1([{'ValueRange',{Lb,Ub}}], Aligned) when is_integer(Lb), + is_integer(Ub) -> + per_dec_constrained(Lb, Ub, Aligned); +per_dec_integer_1([{{_,_}=Constr0,_}], Aligned) -> + Constr = effective_constraint(integer, [Constr0]), + bit_case(per_dec_integer(Constr, Aligned), + per_dec_unconstrained(Aligned)); +per_dec_integer_1([], Aligned) -> + per_dec_unconstrained(Aligned). + +per_dec_unconstrained(Aligned) -> + {get_bits,decode_unconstrained_length(false, Aligned),[8,signed]}. + +per_dec_constrained(Lb, Ub, false) -> + Range = Ub - Lb + 1, + Get = {get_bits,uper_num_bits(Range),[1]}, + add_lb(Lb, Get); +per_dec_constrained(Lb, Ub, true) -> + Range = Ub - Lb + 1, + Get = if + Range =< 255 -> + {get_bits,per_num_bits(Range),[1,unsigned]}; + Range == 256 -> + {get_bits,1,[8,unsigned,{align,true}]}; + Range =< 65536 -> + {get_bits,2,[8,unsigned,{align,true}]}; + true -> + RangeOctLen = byte_size(binary:encode_unsigned(Range - 1)), + {get_bits,per_dec_length({1,RangeOctLen}, false, true), + [8,unsigned,{align,true}]} + end, + add_lb(Lb, Get). + +add_lb(0, Get) -> Get; +add_lb(Lb, Get) -> {add,Get,Lb}. + +per_dec_normally_small_number(Aligned) -> + Small = {get_bits,6,[1]}, + Unlimited = per_decode_semi_constrained(0, Aligned), + bit_case(Small, Unlimited). + +per_decode_semi_constrained(Lb, Aligned) -> + add_lb(Lb, {get_bits,decode_unconstrained_length(false, Aligned),[8]}). + +bit_case(Base, Ext) -> + {'case',[{test,{get_bits,1,[1]},0,Base}, + {test,{get_bits,1,[1]},1,Ext}]}. + +decode_unconstrained_length(AllowZero, Aligned) -> + Al = [{align,Aligned}], + Zero = case AllowZero of + false -> [non_zero]; + true -> [] + end, + {'case',[{test,{get_bits,1,[1|Al]},0, + {value,{get_bits,7,[1|Zero]}}}, + {test,{get_bits,1,[1|Al]},1, + {test,{get_bits,1,[1]},0, + {value,{get_bits,14,[1|Zero]}}}}]}. + +uper_num_bits(N) -> + uper_num_bits(N, 1, 0). + +uper_num_bits(N, T, B) when N =< T -> B; +uper_num_bits(N, T, B) -> uper_num_bits(N, T bsl 1, B+1). + +per_num_bits(2) -> 1; +per_num_bits(N) when N =< 4 -> 2; +per_num_bits(N) when N =< 8 -> 3; +per_num_bits(N) when N =< 16 -> 4; +per_num_bits(N) when N =< 32 -> 5; +per_num_bits(N) when N =< 64 -> 6; +per_num_bits(N) when N =< 128 -> 7; +per_num_bits(N) when N =< 255 -> 8. + +%%% +%%% Remove unnecessary aligning to octet boundaries. +%%% + +opt_al({get_bits,E0,Opts0}, A0) -> + {E,A1} = opt_al(E0, A0), + Opts = opt_al_1(A1, Opts0), + A = update_al(A1, E, Opts), + {{get_bits,E,Opts},A}; +opt_al({call,Fun,E0}, A0) -> + {E,A} = opt_al(E0, A0), + {{call,Fun,E},A}; +opt_al({convert,Op,E0}, A0) -> + {E,A} = opt_al(E0, A0), + {{convert,Op,E},A}; +opt_al({value,E0}, A0) -> + {E,A} = opt_al(E0, A0), + {{value,E},A}; +opt_al({add,E0,I}, A0) when is_integer(I) -> + {E,A} = opt_al(E0, A0), + {{add,E,I},A}; +opt_al({test,E0,V,B0}, A0) -> + {E,A1} = opt_al(E0, A0), + {B,A2} = opt_al(B0, A1), + {{test,E,V,B},A2}; +opt_al({'case',Cs0}, A0) -> + {Cs,A} = opt_al_cs(Cs0, A0), + {{'case',Cs},A}; +opt_al({map,E0,Cs}, A0) -> + {E,A} = opt_al(E0, A0), + {{map,E,Cs},A}; +opt_al(I, A) when is_integer(I) -> + {I,A}. + +opt_al_cs([C0|Cs0], A0) -> + {C,A1} = opt_al(C0, A0), + {Cs,A2} = opt_al_cs(Cs0, A0), + {[C|Cs],merge_al(A1, A2)}; +opt_al_cs([], _) -> {[],none}. + +merge_al(unknown, _) -> unknown; +merge_al(Other, none) -> Other; +merge_al(_, unknown) -> unknown; +merge_al(I0, I1) -> + case {I0 rem 8,I1 rem 8} of + {I,I} -> I; + {_,_} -> unknown + end. + +opt_al_1(unknown, Opts) -> + Opts; +opt_al_1(A, Opts0) -> + case alignment(Opts0) of + none -> + Opts0; + full -> + case A rem 8 of + 0 -> + %% Already in alignment. + proplists:delete(align, Opts0); + Bits -> + %% Cheaper alignment with a constant padding. + Opts1 = proplists:delete(align, Opts0), + [{align,8-Bits }|Opts1] + end; + A -> %Assertion. + Opts0 + end. + +update_al(A0, E, Opts) -> + A = case alignment(Opts) of + none -> A0; + full -> 0; + Bits when is_integer(A0) -> + 0 = (A0 + Bits) rem 8; %Assertion. + _ -> + 0 + end, + [U] = [U || U <- Opts, is_integer(U)], + if + U rem 8 =:= 0 -> A; + is_integer(A), is_integer(E) -> A + U*E; + true -> unknown + end. + +%%% +%%% Flatten the intermediate format and assign temporaries. +%%% + +flatten({get_bits,I,U}, Buf0, St0) when is_integer(I) -> + {Dst,St} = new_var_pair(St0), + Gb = {get_bits,{I,Buf0},U,Dst}, + flatten_align(Gb, [], St); +flatten({get_bits,E0,U}, Buf0, St0) -> + {E,Pre,St1} = flatten(E0, Buf0, St0), + {Dst,St2} = new_var_pair(St1), + Gb = {get_bits,E,U,Dst}, + flatten_align(Gb, Pre, St2); +flatten({test,{get_bits,I,U},V,E0}, Buf0, St0) when is_integer(I) -> + {DstBuf0,St1} = new_var("Buf", St0), + Gb = {get_bits,{I,Buf0},U,{V,DstBuf0}}, + {{_Dst,DstBuf},Pre0,St2} = flatten_align(Gb, [], St1), + {E,Pre1,St3} = flatten(E0, DstBuf, St2), + {E,Pre0++Pre1,St3}; +flatten({add,E0,I}, Buf0, St0) -> + {{Src,Buf},Pre,St1} = flatten(E0, Buf0, St0), + {Dst,St} = new_var("Add", St1), + {{Dst,Buf},Pre++[{add,Src,I,Dst}],St}; +flatten({'case',Cs0}, Buf0, St0) -> + {Dst,St1} = new_var_pair(St0), + {Cs1,St} = flatten_cs(Cs0, Buf0, St1), + {Al,Cs2} = flatten_hoist_align(Cs1), + {Dst,Al++[{'case',Buf0,Cs2,Dst}],St}; +flatten({map,E0,Cs0}, Buf0, St0) -> + {{E,DstBuf},Pre,St1} = flatten(E0, Buf0, St0), + {Dst,St2} = new_var("Int", St1), + Cs = flatten_map_cs(Cs0, E), + {{Dst,DstBuf},Pre++[{'map',E,Cs,{Dst,DstBuf}}],St2}; +flatten({value,V0}, Buf0, St0) when is_integer(V0) -> + {{V0,Buf0},[],St0}; +flatten({value,V0}, Buf0, St0) -> + flatten(V0, Buf0, St0); +flatten({convert,Op,E0}, Buf0, St0) -> + {{E,Buf},Pre,St1} = flatten(E0, Buf0, St0), + {Dst,St2} = new_var("Conv", St1), + {{Dst,Buf},Pre++[{convert,Op,E,Dst}],St2}; +flatten({call,Fun,E0}, Buf0, St0) -> + {Src,Pre,St1} = flatten(E0, Buf0, St0), + {Dst,St2} = new_var_pair(St1), + {Dst,Pre++[{call,Fun,Src,Dst}],St2}. + +flatten_cs([C0|Cs0], Buf, St0) -> + {C,Pre,St1} = flatten(C0, Buf, St0), + {Cs,St2} = flatten_cs(Cs0, Buf, St0), + St3 = St2#st{var=max(St1#st.var, St2#st.var)}, + {[Pre++[{return,C}]|Cs],St3}; +flatten_cs([], _, St) -> {[],St}. + +flatten_map_cs(Cs, Var) -> + flatten_map_cs_1(Cs, {Var,Cs}). + +flatten_map_cs_1([{K,V}|Cs], DefData) -> + [{{asis,K},{asis,V}}|flatten_map_cs_1(Cs, DefData)]; +flatten_map_cs_1([integer_default], {Int,_}) -> + [{'_',Int}]; +flatten_map_cs_1([enum_default], {Int,_}) -> + [{'_',["{asn1_enum,",Int,"}"]}]; +flatten_map_cs_1([enum_error], {Var,Cs}) -> + Vs = [V || {_,V} <- Cs], + [{'_',["exit({error,{asn1,{decode_enumerated,{",Var,",", + {asis,Vs},"}}}})"]}]; +flatten_map_cs_1([], _) -> []. + +flatten_hoist_align([[{align_bits,_,_}=Ab|T]|Cs]) -> + flatten_hoist_align_1(Cs, Ab, [T]); +flatten_hoist_align(Cs) -> {[],Cs}. + +flatten_hoist_align_1([[Ab|T]|Cs], Ab, Acc) -> + flatten_hoist_align_1(Cs, Ab, [T|Acc]); +flatten_hoist_align_1([], Ab, Acc) -> + {[Ab],lists:reverse(Acc)}. + +flatten_align({get_bits,{SrcBits,SrcBuf},U,Dst}=Gb0, Pre, St0) -> + case alignment(U) of + none -> + flatten_align_1(U, Dst, Pre++[Gb0], St0); + full -> + {PadBits,St1} = new_var("Pad", St0), + {DstBuf,St2} = new_var("Buf", St1), + Ab = {align_bits,SrcBuf,PadBits}, + Agb = {get_bits,{PadBits,SrcBuf},[1],{'_',DstBuf}}, + Gb = {get_bits,{SrcBits,DstBuf},U,Dst}, + flatten_align_1(U, Dst, Pre++[Ab,Agb,Gb], St2); + PadBits when is_integer(PadBits), PadBits > 0 -> + {DstBuf,St1} = new_var("Buf", St0), + Agb = {get_bits,{PadBits,SrcBuf},[1],{'_',DstBuf}}, + Gb = {get_bits,{SrcBits,DstBuf},U,Dst}, + flatten_align_1(U, Dst, Pre++[Agb,Gb], St1) + end. + +flatten_align_1(U, {D,_}=Dst, Pre, St) -> + case is_non_zero(U) of + false -> + {Dst,Pre,St}; + true -> + {Dst,Pre++[{non_zero,D}],St} + end. + +new_var_pair(St0) -> + {Var,St1} = new_var("V", St0), + {Buf,St2} = new_var("Buf", St1), + {{Var,Buf},St2}. + +new_var(Tag, #st{base=VarBase,var=N}=St) -> + {VarBase++Tag++integer_to_list(N),St#st{var=N+1}}. + +alignment([{align,false}|_]) -> none; +alignment([{align,true}|_]) -> full; +alignment([{align,Bits}|_]) -> Bits; +alignment([_|T]) -> alignment(T); +alignment([]) -> none. + +is_non_zero(Fl) -> + lists:member(non_zero, Fl). + +%%% +%%% Generate Erlang code from the flattened intermediate format. +%%% + +dcg_list_outside([{align_bits,Buf,SzVar}|T]) -> + emit([SzVar," = bit_size(",Buf,") band 7"]), + iter_dcg_list_outside(T); +dcg_list_outside([{'case',Buf,Cs,Dst}|T]) -> + dcg_case(Buf, Cs, Dst), + iter_dcg_list_outside(T); +dcg_list_outside([{'map',Val,Cs,Dst}|T]) -> + dcg_map(Val, Cs, Dst), + iter_dcg_list_outside(T); +dcg_list_outside([{add,S1,S2,Dst}|T]) -> + emit([Dst," = ",S1," + ",S2]), + iter_dcg_list_outside(T); +dcg_list_outside([{return,{V,Buf}}|T]) -> + emit(["{",V,",",Buf,"}"]), + iter_dcg_list_outside(T); +dcg_list_outside([{call,Fun,{V,Buf},{Dst,DstBuf}}|T]) -> + emit(["{",Dst,",",DstBuf,"} = "]), + Fun(V, Buf), + iter_dcg_list_outside(T); +dcg_list_outside([{convert,Op,V,Dst}|T]) -> + emit([Dst," = ",Op,"(",V,")"]), + iter_dcg_list_outside(T); +dcg_list_outside([{get_bits,{_,Buf0},_,_}|_]=L0) -> + emit("<<"), + {L,Buf} = dcg_list_inside(L0, buf), + emit([Buf,"/bitstring>> = ",Buf0]), + iter_dcg_list_outside(L); +dcg_list_outside([]) -> + emit("ignore"), + ok. + +iter_dcg_list_outside([_|_]=T) -> + emit([",",nl]), + dcg_list_outside(T); +iter_dcg_list_outside([]) -> ok. + +dcg_case(Buf, Cs, {Dst,DstBuf}) -> + emit(["{",Dst,",",DstBuf,"} = case ",Buf," of",nl]), + dcg_case_cs(Cs), + emit("end"). + +dcg_case_cs([C|Cs]) -> + emit("<<"), + {T0,DstBuf} = dcg_list_inside(C, buf), + emit([DstBuf,"/bitstring>>"]), + T1 = dcg_guard(T0), + dcg_list_outside(T1), + case Cs of + [] -> emit([nl]); + [_|_] -> emit([";",nl]) + end, + dcg_case_cs(Cs); +dcg_case_cs([]) -> ok. + +dcg_guard([{non_zero,Src}|T]) -> + emit([" when ",Src," =/= 0 ->",nl]), + T; +dcg_guard(T) -> + emit([" ->",nl]), + T. + +dcg_map(Val, Cs, {Dst,_}) -> + emit([Dst," = case ",Val," of",nl]), + dcg_map_cs(Cs), + emit("end"). + +dcg_map_cs([{K,V}]) -> + emit([K," -> ",V,nl]); +dcg_map_cs([{K,V}|Cs]) -> + emit([K," -> ",V,";",nl]), + dcg_map_cs(Cs). + +dcg_list_inside([{get_bits,{Sz,_},Fl0,{Dst,DstBuf}}|T], _) -> + Fl = bit_flags(Fl0, []), + emit([mk_dest(Dst),":",Sz,Fl,","]), + dcg_list_inside(T, DstBuf); +dcg_list_inside(L, Dst) -> {L,Dst}. + +bit_flags([1|T], Acc) -> + bit_flags(T, Acc); +bit_flags([{align,_}|T], Acc) -> + bit_flags(T, Acc); +bit_flags([non_zero|T], Acc) -> + bit_flags(T, Acc); +bit_flags([U|T], Acc) when is_integer(U) -> + bit_flags(T, ["unit:"++integer_to_list(U)|Acc]); +bit_flags([H|T], Acc) -> + bit_flags(T, [atom_to_list(H)|Acc]); +bit_flags([], []) -> + ""; +bit_flags([], Acc) -> + "/" ++ bit_flags_1(Acc, ""). + +bit_flags_1([H|T], Sep) -> + Sep ++ H ++ bit_flags_1(T, "-"); +bit_flags_1([], _) -> []. + +mk_dest(I) when is_integer(I) -> + integer_to_list(I); +mk_dest(S) -> S. + +%% effective_constraint(Type,C) +%% Type = atom() +%% C = [C1,...] +%% C1 = {'SingleValue',SV} | {'ValueRange',VR} | {atom(),term()} +%% SV = integer() | [integer(),...] +%% VR = {Lb,Ub} +%% Lb = 'MIN' | integer() +%% Ub = 'MAX' | integer() +%% Returns a single value if C only has a single value constraint, and no +%% value range constraints, that constrains to a single value, otherwise +%% returns a value range that has the lower bound set to the lowest value +%% of all single values and lower bound values in C and the upper bound to +%% the greatest value. +effective_constraint(integer,[C={{_,_},_}|_Rest]) -> % extension + [C]; +effective_constraint(integer, C) -> + SVs = get_constraints(C, 'SingleValue'), + SV = effective_constr('SingleValue', SVs), + VRs = get_constraints(C, 'ValueRange'), + VR = effective_constr('ValueRange', VRs), + greatest_common_range(SV, VR); +effective_constraint(bitstring, C) -> + get_constraint(C, 'SizeConstraint'). + +effective_constr(_, []) -> []; +effective_constr('SingleValue', List) -> + SVList = lists:flatten(lists:map(fun(X) -> element(2, X) end, List)), + %% Sort and remove duplicates before generating SingleValue or ValueRange + %% In case of ValueRange, also check for 'MIN and 'MAX' + case lists:usort(SVList) of + [N] -> + [{'SingleValue',N}]; + [_|_]=L -> + [{'ValueRange',{least_Lb(L),greatest_Ub(L)}}] + end; +effective_constr('ValueRange', List) -> + LBs = lists:map(fun({_,{Lb,_}}) -> Lb end, List), + UBs = lists:map(fun({_,{_,Ub}}) -> Ub end, List), + Lb = least_Lb(LBs), + [{'ValueRange',{Lb,lists:max(UBs)}}]. + +greatest_common_range([], VR) -> + VR; +greatest_common_range(SV, []) -> + SV; +greatest_common_range([{_,Int}], [{_,{'MIN',Ub}}]) + when is_integer(Int), Int > Ub -> + [{'ValueRange',{'MIN',Int}}]; +greatest_common_range([{_,Int}],[{_,{Lb,Ub}}]) + when is_integer(Int), Int < Lb -> + [{'ValueRange',{Int,Ub}}]; +greatest_common_range([{_,Int}],VR=[{_,{_Lb,_Ub}}]) when is_integer(Int) -> + VR; +greatest_common_range([{_,L}],[{_,{Lb,Ub}}]) when is_list(L) -> + Min = least_Lb([Lb|L]), + Max = greatest_Ub([Ub|L]), + [{'ValueRange',{Min,Max}}]; +greatest_common_range([{_,{Lb1,Ub1}}], [{_,{Lb2,Ub2}}]) -> + Min = least_Lb([Lb1,Lb2]), + Max = greatest_Ub([Ub1,Ub2]), + [{'ValueRange',{Min,Max}}]. + + +least_Lb(L) -> + case lists:member('MIN', L) of + true -> 'MIN'; + false -> lists:min(L) + end. + +greatest_Ub(L) -> + case lists:member('MAX', L) of + true -> 'MAX'; + false -> lists:max(L) + end. + +get_constraint(C, Key) -> + case lists:keyfind(Key, 1, C) of + false -> no; + {_,V} -> V + end. + +get_constraints([{Key,_}=Pair|T], Key) -> + [Pair|get_constraints(T, Key)]; +get_constraints([_|T], Key) -> + get_constraints(T, Key); +get_constraints([], _) -> []. diff --git a/lib/asn1/src/asn1ct_value.erl b/lib/asn1/src/asn1ct_value.erl index 9013baef92..389642c446 100644 --- a/lib/asn1/src/asn1ct_value.erl +++ b/lib/asn1/src/asn1ct_value.erl @@ -54,7 +54,7 @@ from_type(M,Typename,Type) when is_record(Type,type) -> {notype,_} -> true; {primitive,bif} -> - from_type_prim(Type,get_encoding_rule(M)); + from_type_prim(Type); 'ASN1_OPEN_TYPE' -> case Type#type.constraint of [#'Externaltypereference'{type=TrefConstraint}] -> @@ -164,7 +164,7 @@ gen_list(_,_,_,0) -> gen_list(M,Typename,Oftype,N) -> [from_type(M,Typename,Oftype)|gen_list(M,Typename,Oftype,N-1)]. -from_type_prim(D,Erule) -> +from_type_prim(D) -> C = D#type.constraint, case D#type.def of 'INTEGER' -> @@ -303,12 +303,7 @@ from_type_prim(D,Erule) -> adjust_list(size_random(C),c_string(C,"BMPString")); 'UTF8String' -> {ok,Res}=asn1rt:utf8_list_to_binary(adjust_list(random(50),[$U,$T,$F,$8,$S,$t,$r,$i,$n,$g,16#ffff,16#fffffff,16#ffffff,16#fffff,16#fff])), - case Erule of - per -> - binary_to_list(Res); - _ -> - Res - end; + Res; 'UniversalString' -> adjust_list(size_random(C),c_string(C,"UniversalString")); XX -> @@ -440,20 +435,9 @@ get_encoding_rule(M) -> end. open_type_value(ber) -> - [4,9,111,112,101,110,95,116,121,112,101]; -open_type_value(ber_bin) -> -% [4,9,111,112,101,110,95,116,121,112,101]; - <<4,9,111,112,101,110,95,116,121,112,101>>; -open_type_value(ber_bin_v2) -> -% [4,9,111,112,101,110,95,116,121,112,101]; <<4,9,111,112,101,110,95,116,121,112,101>>; -open_type_value(per) -> - "\n\topen_type"; %octet string value "open_type" -open_type_value(per_bin) -> - <<"\n\topen_type">>; -% <<10,9,111,112,101,110,95,116,121,112,101>>; open_type_value(_) -> - [4,9,111,112,101,110,95,116,121,112,101]. + <<"\n\topen_type">>. %octet string value "open_type" to_textual_order({Root,Ext}) -> {to_textual_order(Root),Ext}; diff --git a/lib/asn1/src/asn1rt_ber_bin.erl b/lib/asn1/src/asn1rt_ber_bin.erl index 22f9f2ecfd..ec1549804b 100644 --- a/lib/asn1/src/asn1rt_ber_bin.erl +++ b/lib/asn1/src/asn1rt_ber_bin.erl @@ -19,337 +19,30 @@ %% -module(asn1rt_ber_bin). -%% encoding / decoding of BER - --export([decode/1]). --export([fixoptionals/2,split_list/2,cindex/3,restbytes2/3, - list_to_record/2, - encode_tag_val/1,decode_tag/1,peek_tag/1, - check_tags/3, encode_tags/3]). --export([encode_boolean/2,decode_boolean/3, - encode_integer/3,encode_integer/4, - decode_integer/4,decode_integer/5,encode_enumerated/2, - encode_enumerated/4,decode_enumerated/5, +-export([decode_length/1, encode_real/2, encode_real/3, decode_real/2, decode_real/4, - encode_bit_string/4,decode_bit_string/6, - decode_compact_bit_string/6, - encode_octet_string/3,decode_octet_string/5, - encode_null/2,decode_null/3, - encode_object_identifier/2,decode_object_identifier/3, - encode_relative_oid/2,decode_relative_oid/3, - encode_restricted_string/4,decode_restricted_string/6, - encode_universal_string/3,decode_universal_string/5, - encode_UTF8_string/3, decode_UTF8_string/3, - encode_BMP_string/3,decode_BMP_string/5, - encode_generalized_time/3,decode_generalized_time/5, - encode_utc_time/3,decode_utc_time/5, - encode_length/1,decode_length/1, - check_if_valid_tag/3, - decode_tag_and_length/1, decode_components/6, - decode_components/7, decode_set/6]). - --export([encode_open_type/1,encode_open_type/2,decode_open_type/1,decode_open_type/2,decode_open_type/3]). --export([skipvalue/1, skipvalue/2,skip_ExtensionAdditions/2]). + decode_tag/1]). -include("asn1_records.hrl"). -% the encoding of class of tag bits 8 and 7 +%% the encoding of class of tag bits 8 and 7 -define(UNIVERSAL, 0). --define(APPLICATION, 16#40). --define(CONTEXT, 16#80). --define(PRIVATE, 16#C0). %%% primitive or constructed encoding % bit 6 -define(PRIMITIVE, 0). -define(CONSTRUCTED, 2#00100000). %%% The tag-number for universal types --define(N_BOOLEAN, 1). --define(N_INTEGER, 2). --define(N_BIT_STRING, 3). --define(N_OCTET_STRING, 4). --define(N_NULL, 5). --define(N_OBJECT_IDENTIFIER, 6). --define(N_OBJECT_DESCRIPTOR, 7). --define(N_EXTERNAL, 8). -define(N_REAL, 9). --define(N_ENUMERATED, 10). --define(N_EMBEDDED_PDV, 11). --define(N_UTF8String, 12). --define('N_RELATIVE-OID',13). --define(N_SEQUENCE, 16). --define(N_SET, 17). --define(N_NumericString, 18). --define(N_PrintableString, 19). --define(N_TeletexString, 20). --define(N_VideotexString, 21). --define(N_IA5String, 22). --define(N_UTCTime, 23). --define(N_GeneralizedTime, 24). --define(N_GraphicString, 25). --define(N_VisibleString, 26). --define(N_GeneralString, 27). --define(N_UniversalString, 28). --define(N_BMPString, 30). - - -% the complete tag-word of built-in types --define(T_BOOLEAN, ?UNIVERSAL bor ?PRIMITIVE bor 1). --define(T_INTEGER, ?UNIVERSAL bor ?PRIMITIVE bor 2). --define(T_BIT_STRING, ?UNIVERSAL bor ?PRIMITIVE bor 3). % can be CONSTRUCTED --define(T_OCTET_STRING, ?UNIVERSAL bor ?PRIMITIVE bor 4). % can be CONSTRUCTED --define(T_NULL, ?UNIVERSAL bor ?PRIMITIVE bor 5). --define(T_OBJECT_IDENTIFIER,?UNIVERSAL bor ?PRIMITIVE bor 6). --define(T_OBJECT_DESCRIPTOR,?UNIVERSAL bor ?PRIMITIVE bor 7). --define(T_EXTERNAL, ?UNIVERSAL bor ?PRIMITIVE bor 8). --define(T_REAL, ?UNIVERSAL bor ?PRIMITIVE bor 9). --define(T_ENUMERATED, ?UNIVERSAL bor ?PRIMITIVE bor 10). --define(T_EMBEDDED_PDV, ?UNIVERSAL bor ?PRIMITIVE bor 11). --define(T_SEQUENCE, ?UNIVERSAL bor ?CONSTRUCTED bor 16). --define(T_SET, ?UNIVERSAL bor ?CONSTRUCTED bor 17). --define(T_NumericString, ?UNIVERSAL bor ?PRIMITIVE bor 18). %can be constructed --define(T_PrintableString, ?UNIVERSAL bor ?PRIMITIVE bor 19). %can be constructed --define(T_TeletexString, ?UNIVERSAL bor ?PRIMITIVE bor 20). %can be constructed --define(T_VideotexString, ?UNIVERSAL bor ?PRIMITIVE bor 21). %can be constructed --define(T_IA5String, ?UNIVERSAL bor ?PRIMITIVE bor 22). %can be constructed --define(T_UTCTime, ?UNIVERSAL bor ?PRIMITIVE bor 23). --define(T_GeneralizedTime, ?UNIVERSAL bor ?PRIMITIVE bor 24). --define(T_GraphicString, ?UNIVERSAL bor ?PRIMITIVE bor 25). %can be constructed --define(T_VisibleString, ?UNIVERSAL bor ?PRIMITIVE bor 26). %can be constructed --define(T_GeneralString, ?UNIVERSAL bor ?PRIMITIVE bor 27). %can be constructed --define(T_UniversalString, ?UNIVERSAL bor ?PRIMITIVE bor 28). %can be constructed --define(T_BMPString, ?UNIVERSAL bor ?PRIMITIVE bor 30). %can be constructed - - -decode(Bin) -> - decode_primitive(Bin). - -decode_primitive(Bin) -> - {Tlv = {Tag,Len,V},<<>>} = decode_tlv(Bin), - case element(2,Tag) of - ?CONSTRUCTED -> - {Tag,Len,decode_constructed(V)}; - _ -> - Tlv - end. - -decode_constructed(<<>>) -> - []; -decode_constructed(Bin) -> - {Tlv = {Tag,Len,V},Rest} = decode_tlv(Bin), - NewTlv = - case element(2,Tag) of - ?CONSTRUCTED -> - {Tag,Len,decode_constructed(V)}; - _ -> - Tlv - end, - [NewTlv|decode_constructed(Rest)]. - -decode_tlv(Bin) -> - {Tag,Bin1,_Rb1} = decode_tag(Bin), - {{Len,Bin2},_Rb2} = decode_length(Bin1), - <<V:Len/binary,Bin3/binary>> = Bin2, - {{Tag,Len,V},Bin3}. - - - -%%%%%%%%%%%%% -% split_list(List,HeadLen) -> {HeadList,TailList} -% -% splits List into HeadList (Length=HeadLen) and TailList -% if HeadLen == indefinite -> return {List,indefinite} -split_list(List,indefinite) -> - {List, indefinite}; -split_list(Bin, Len) when is_binary(Bin) -> - split_binary(Bin,Len); -split_list(List,Len) -> - {lists:sublist(List,Len),lists:nthtail(Len,List)}. - -%%% new function which fixes a bug regarding indefinite length decoding -restbytes2(indefinite,<<0,0,RemBytes/binary>>,_) -> - {RemBytes,2}; -restbytes2(indefinite,RemBytes,ext) -> - skipvalue(indefinite,RemBytes); -restbytes2(RemBytes,<<>>,_) -> - {RemBytes,0}; -restbytes2(_RemBytes,Bytes,noext) -> - exit({error,{asn1, {unexpected,Bytes}}}); -restbytes2(RemBytes,Bytes,ext) -> -%% {RemBytes,0}. - {RemBytes,byte_size(Bytes)}. - - - -%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%% -%% skipvalue(Length, Bytes) -> {RemainingBytes, RemovedNumberOfBytes} -%% -%% skips the one complete (could be nested) TLV from Bytes -%% handles both definite and indefinite length encodings -%% - -skipvalue(L, Bytes) -> - skipvalue(L, Bytes, 0). - -skipvalue(L, Bytes, Rb) -> - skipvalue(L, Bytes, Rb, 0). - -skipvalue(indefinite, Bytes, Rb, IndefLevel) -> - {T,Bytes2,R2} = decode_tag(Bytes), - {{L,Bytes3},R3} = decode_length(Bytes2), - case {T,L} of - {_,indefinite} -> - skipvalue(indefinite,Bytes3,Rb+R2+R3,IndefLevel+1); - {{0,0,0},0} when IndefLevel =:= 0 -> - %% See X690 8.1.5 NOTE, end of indefinite content - {Bytes3,Rb+2}; - {{0,0,0},0} -> - skipvalue(indefinite,Bytes3,Rb+2,IndefLevel - 1); - _ -> - <<_:L/binary, RestBytes/binary>> = Bytes3, - skipvalue(indefinite,RestBytes,Rb+R2+R3+L, IndefLevel) - %%{RestBytes, R2+R3+L} - end; -%% case Bytes4 of -%% <<0,0,Bytes5/binary>> -> -%% {Bytes5,Rb+Rb4+2}; -%% _ -> skipvalue(indefinite,Bytes4,Rb+Rb4) -%% end; -skipvalue(L, Bytes, Rb, _) -> -% <<Skip:L/binary, RestBytes/binary>> = Bytes, - <<_:L/binary, RestBytes/binary>> = Bytes, - {RestBytes,Rb+L}. - - -skipvalue(Bytes) -> - {_T,Bytes2,R2} = decode_tag(Bytes), - {{L,Bytes3},R3} = decode_length(Bytes2), - skipvalue(L,Bytes3,R2+R3). - - -cindex(Ix,Val,Cname) -> - case element(Ix,Val) of - {Cname,Val2} -> Val2; - X -> X - end. - -%%% -%% skips byte sequence of Bytes that do not match a tag in Tags -skip_ExtensionAdditions(Bytes,Tags) -> - skip_ExtensionAdditions(Bytes,Tags,0). -skip_ExtensionAdditions(<<>>,_Tags,RmB) -> - {<<>>,RmB}; -skip_ExtensionAdditions(Bytes,Tags,RmB) -> - case catch decode_tag(Bytes) of - {'EXIT',_Reason} -> - tag_error(no_data,Tags,Bytes,'OPTIONAL'); - {_T={Class,_Form,TagNo},_Bytes2,_R2} -> - case [X||X=#tag{class=Cl,number=TN} <- Tags,Cl==Class,TN==TagNo] of - [] -> - %% skip this TLV and continue with next - {Bytes3,R3} = skipvalue(Bytes), - skip_ExtensionAdditions(Bytes3,Tags,RmB+R3); - _ -> - {Bytes,RmB} - end - end. - -%%=============================================================================== -%%=============================================================================== -%%=============================================================================== -%% Optionals, preset not filled optionals with asn1_NOVALUE -%%=============================================================================== -%%=============================================================================== -%%=============================================================================== - -% converts a list to a record if necessary -list_to_record(Name,List) when is_list(List) -> - list_to_tuple([Name|List]); -list_to_record(_Name,Tuple) when is_tuple(Tuple) -> - Tuple. - - -fixoptionals(OptList,Val) when is_list(Val) -> - fixoptionals(OptList,Val,1,[],[]). - -fixoptionals([{Name,Pos}|Ot],[{Name,Val}|Vt],_Opt,Acc1,Acc2) -> - fixoptionals(Ot,Vt,Pos+1,[1|Acc1],[{Name,Val}|Acc2]); -fixoptionals([{_Name,Pos}|Ot],V,Pos,Acc1,Acc2) -> - fixoptionals(Ot,V,Pos+1,[0|Acc1],[asn1_NOVALUE|Acc2]); -fixoptionals(O,[Vh|Vt],Pos,Acc1,Acc2) -> - fixoptionals(O,Vt,Pos+1,Acc1,[Vh|Acc2]); -fixoptionals([],[Vh|Vt],Pos,Acc1,Acc2) -> - fixoptionals([],Vt,Pos+1,Acc1,[Vh|Acc2]); -fixoptionals([],[],_,_Acc1,Acc2) -> - % return Val as a record - list_to_tuple([asn1_RECORDNAME|lists:reverse(Acc2)]). - - -%%encode_tag(TagClass(?UNI, APP etc), Form (?PRIM etx), TagInteger) -> -%% 8bit Int | binary encode_tag_val({Class, Form, TagNo}) when (TagNo =< 30) -> <<(Class bsr 6):2,(Form bsr 5):1,TagNo:5>>; encode_tag_val({Class, Form, TagNo}) -> {Octets,_Len} = mk_object_val(TagNo), BinOct = list_to_binary(Octets), - <<(Class bsr 6):2, (Form bsr 5):1, 31:5,BinOct/binary>>; - -%% asumes whole correct tag bitpattern, multiple of 8 -encode_tag_val(Tag) when (Tag =< 255) -> Tag; %% anv�nds denna funktion??!! -%% asumes correct bitpattern of 0-5 -encode_tag_val(Tag) -> encode_tag_val2(Tag,[]). - -encode_tag_val2(Tag, OctAck) when (Tag =< 255) -> - [Tag | OctAck]; -encode_tag_val2(Tag, OctAck) -> - encode_tag_val2(Tag bsr 8, [255 band Tag | OctAck]). - - -%%%encode_tag(TagClass(?UNI, APP etc), Form (?PRIM etx), TagInteger) -> -%%% 8bit Int | [list of octets] -%encode_tag_val({Class, Form, TagNo}) when (TagNo =< 30) -> -%%% <<Class:2,Form:1,TagNo:5>>; -% [Class bor Form bor TagNo]; -%encode_tag_val({Class, Form, TagNo}) -> -% {Octets,L} = mk_object_val(TagNo), -% [Class bor Form bor 31 | Octets]; - - -%%============================================================================\%% Peek on the initial tag -%% peek_tag(Bytes) -> TagBytes -%% interprets the first byte and possible second, third and fourth byte as -%% a tag and returns all the bytes comprising the tag, the constructed/primitive bit (6:th bit of first byte) is normalised to 0 -%% - -peek_tag(<<B7_6:2,_:1,31:5,Buffer/binary>>) -> - Bin = peek_tag(Buffer, <<>>), - <<B7_6:2,31:6,Bin/binary>>; -%% single tag (tagno < 31) -peek_tag(<<B7_6:2,_:1,B4_0:5,_Buffer/binary>>) -> - <<B7_6:2,B4_0:6>>. - -peek_tag(<<0:1,PartialTag:7,_Buffer/binary>>, TagAck) -> - <<TagAck/binary,PartialTag>>; -peek_tag(<<PartialTag,Buffer/binary>>, TagAck) -> - peek_tag(Buffer,<<TagAck/binary,PartialTag>>); -peek_tag(_,TagAck) -> - exit({error,{asn1, {invalid_tag,TagAck}}}). -%%peek_tag([Tag|Buffer]) when (Tag band 31) =:= 31 -> -%% [Tag band 2#11011111 | peek_tag(Buffer,[])]; -%%%% single tag (tagno < 31) -%%peek_tag([Tag|Buffer]) -> -%% [Tag band 2#11011111]. - -%%peek_tag([PartialTag|Buffer], TagAck) when (PartialTag < 128 ) -> -%% lists:reverse([PartialTag|TagAck]); -%%peek_tag([PartialTag|Buffer], TagAck) -> -%% peek_tag(Buffer,[PartialTag|TagAck]); -%%peek_tag(Buffer,TagAck) -> -%% exit({error,{asn1, {invalid_tag,lists:reverse(TagAck)}}}). - + <<(Class bsr 6):2, (Form bsr 5):1, 31:5,BinOct/binary>>. %%=============================================================================== %% Decode a tag @@ -403,33 +96,11 @@ check_tags_i([Tag1|TagRest], Buffer, Rb, OptOrMand) -> _ -> check_tags_i(TagRest, Buffer2, Rb + Rb1, mandatory) end - end; - -check_tags_i([], Buffer, Rb, _) -> - {[],{{0,0},Buffer,Rb}}. + end. %%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%% %% This function is called from generated code -check_tags([Tag], Buffer, OptOrMand) -> % optimized very usual case - check_one_tag(Tag, Buffer, OptOrMand); -check_tags(Tags, Buffer, OptOrMand) -> - check_tags(Tags, Buffer, 0, OptOrMand). - -check_tags([Tag1,Tag2|TagRest], Buffer, Rb, OptOrMand) - when Tag1#tag.type == 'IMPLICIT' -> - check_tags([Tag1#tag{type=Tag2#tag.type}|TagRest], Buffer, Rb, OptOrMand); - -check_tags([Tag1|TagRest], Buffer, Rb, OptOrMand) -> - {Form_Length,Buffer2,Rb1} = check_one_tag(Tag1, Buffer, OptOrMand), - case TagRest of - [] -> {Form_Length, Buffer2, Rb + Rb1}; - _ -> check_tags(TagRest, Buffer2, Rb + Rb1, mandatory) - end; - -check_tags([], Buffer, Rb, _) -> - {{0,0},Buffer,Rb}. - check_one_tag(Tag=#tag{class=ExpectedClass,number=ExpectedNumber}, Buffer, OptOrMand) -> case catch decode_tag(Buffer) of {'EXIT',_Reason} -> @@ -491,382 +162,6 @@ encode_one_tag(#tag{class=Class,number=No,type=Type, form = Form}) -> Bytes = encode_tag_val({Class,NewForm,No}), {Bytes,size(Bytes)}. -%%=============================================================================== -%% Change the tag (used when an implicit tagged type has a reference to something else) -%% The constructed bit in the tag is taken from the tag to be replaced. -%% -%% change_tag(NewTag,[Tag,Buffer]) -> [NewTag,Buffer] -%%=============================================================================== - -%change_tag({NewClass,NewTagNr}, Buffer) -> -% {{OldClass, OldForm, OldTagNo}, Buffer1, RemovedBytes} = decode_tag(lists:flatten(Buffer)), -% [encode_tag_val({NewClass, OldForm, NewTagNr}) | Buffer1]. - - -%%=============================================================================== -%% -%% This comment is valid for all the encode/decode functions -%% -%% C = Constraint -> typically {'ValueRange',LowerBound,UpperBound} -%% used for PER-coding but not for BER-coding. -%% -%% Val = Value. If Val is an atom then it is a symbolic integer value -%% (i.e the atom must be one of the names in the NamedNumberList). -%% The NamedNumberList is used to translate the atom to an integer value -%% before encoding. -%% -%%=============================================================================== - -%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%% -%% encode_open_type(Value) -> CompleteList -%% Value = list of bytes of an already encoded value (the list must be flat) -%% | binary - -%% This version does not consider Explicit tagging of the open type. It -%% is only left because of backward compatibility. -encode_open_type(Val) when is_list(Val) -> - {Val, byte_size(list_to_binary(Val))}; -encode_open_type(Val) -> - {Val, byte_size(Val)}. - -%% -encode_open_type(Val, []) when is_list(Val) -> - {Val, byte_size(list_to_binary(Val))}; -encode_open_type(Val, []) -> - {Val, byte_size(Val)}; -encode_open_type(Val, Tag) when is_list(Val) -> - encode_tags(Tag, Val, byte_size(list_to_binary(Val))); -encode_open_type(Val, Tag) -> - encode_tags(Tag, Val, byte_size(Val)). - - -%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%% -%% decode_open_type(Buffer) -> Value -%% Bytes = [byte] with BER encoded data -%% Value = [byte] with decoded data (which must be decoded again as some type) -%% -decode_open_type(Bytes) -> -% {_Tag, Len, _RemainingBuffer, RemovedBytes} = decode_tag_and_length(Bytes), -% N = Len + RemovedBytes, - {_Tag, Len, RemainingBuffer, RemovedBytes} = decode_tag_and_length(Bytes), - {_RemainingBuffer2, RemovedBytes2} = skipvalue(Len, RemainingBuffer, RemovedBytes), - N = RemovedBytes2, - <<Val:N/binary, RemainingBytes/binary>> = Bytes, -% {Val, RemainingBytes, Len + RemovedBytes}. - {Val,RemainingBytes,N}. - -decode_open_type(<<>>,[]=ExplTag) -> % R9C-0.patch-40 - exit({error, {asn1,{no_optional_tag, ExplTag}}}); -decode_open_type(Bytes,ExplTag) -> - {Tag, Len, RemainingBuffer, RemovedBytes} = decode_tag_and_length(Bytes), - case {Tag,ExplTag} of -% {{Class,Form,32},[#tag{class=Class,number=No,form=32}]} -> -% {_Tag2, Len2, RemainingBuffer2, RemovedBytes2} = decode_tag_and_length(RemainingBuffer), -% {_RemainingBuffer3, RemovedBytes3} = skipvalue(Len2, RemainingBuffer2, RemovedBytes2), -% N = RemovedBytes3, -% <<_:RemovedBytes/unit:8,Val:N/binary,RemainingBytes/binary>> = Bytes, -% {Val, RemainingBytes, N + RemovedBytes}; - {{Class,Form,No},[#tag{class=Class,number=No,form=Form}]} -> - {_RemainingBuffer2, RemovedBytes2} = - skipvalue(Len, RemainingBuffer), - N = RemovedBytes2, - <<_:RemovedBytes/unit:8,Val:N/binary,RemainingBytes/binary>> = Bytes, - {Val, RemainingBytes, N + RemovedBytes}; - _ -> - {_RemainingBuffer2, RemovedBytes2} = - skipvalue(Len, RemainingBuffer, RemovedBytes), - N = RemovedBytes2, - <<Val:N/binary, RemainingBytes/binary>> = Bytes, - {Val, RemainingBytes, N} - end. - -decode_open_type(ber_bin,Bytes,ExplTag) -> - decode_open_type(Bytes,ExplTag); -decode_open_type(ber,Bytes,ExplTag) -> - {Val,RemBytes,Len}=decode_open_type(Bytes,ExplTag), - {binary_to_list(Val),RemBytes,Len}. - -%%=============================================================================== -%%=============================================================================== -%%=============================================================================== -%% Boolean, ITU_T X.690 Chapter 8.2 -%%=============================================================================== -%%=============================================================================== -%%=============================================================================== - -%%=============================================================================== -%% encode_boolean(Integer, tag | notag) -> [octet list] -%%=============================================================================== - -encode_boolean({Name, Val}, DoTag) when is_atom(Name) -> - dotag(DoTag, ?N_BOOLEAN, encode_boolean(Val)); -encode_boolean(true,[]) -> - {[1,1,16#FF],3}; -encode_boolean(false,[]) -> - {[1,1,0],3}; -encode_boolean(Val, DoTag) -> - dotag(DoTag, ?N_BOOLEAN, encode_boolean(Val)). - -%% encode_boolean(Boolean) -> [Len, Boolean] = [1, $FF | 0] -encode_boolean(true) -> {[16#FF],1}; -encode_boolean(false) -> {[0],1}; -encode_boolean(X) -> exit({error,{asn1, {encode_boolean, X}}}). - - -%%=============================================================================== -%% decode_boolean(BuffList, HasTag, TotalLen) -> {true, Remain, RemovedBytes} | -%% {false, Remain, RemovedBytes} -%%=============================================================================== - -decode_boolean(Buffer, Tags, OptOrMand) -> - NewTags = new_tags(Tags,#tag{class=?UNIVERSAL,number=?N_BOOLEAN}), - decode_boolean_notag(Buffer, NewTags, OptOrMand). - -decode_boolean_notag(Buffer, Tags, OptOrMand) -> - {RestTags, {FormLen,Buffer0,Rb0}} = - check_tags_i(Tags, Buffer, OptOrMand), - case FormLen of - {?CONSTRUCTED,Len} -> - {Buffer00,RestBytes} = split_list(Buffer0,Len), - {Val,Buffer1,Rb1} = decode_boolean_notag(Buffer00, RestTags, OptOrMand), - {Buffer2, Rb2} = restbytes2(RestBytes,Buffer1,noext), - {Val, Buffer2, Rb0+Rb1+Rb2}; - {_,_} -> - decode_boolean2(Buffer0, Rb0) - end. - -decode_boolean2(<<0:8, Buffer/binary>>, RemovedBytes) -> - {false, Buffer, RemovedBytes + 1}; -decode_boolean2(<<_:8, Buffer/binary>>, RemovedBytes) -> - {true, Buffer, RemovedBytes + 1}; -decode_boolean2(Buffer, _) -> - exit({error,{asn1, {decode_boolean, Buffer}}}). - - -%%=========================================================================== -%% Integer, ITU_T X.690 Chapter 8.3 - -%% encode_integer(Constraint, Value, Tag) -> [octet list] -%% encode_integer(Constraint, Name, NamedNumberList, Tag) -> [octet list] -%% Value = INTEGER | {Name,INTEGER} -%% Tag = tag | notag -%%=========================================================================== - -encode_integer(C, Val, []) when is_integer(Val) -> - {EncVal,Len} = encode_integer(C, Val), - dotag_universal(?N_INTEGER,EncVal,Len); -encode_integer(C, Val, Tag) when is_integer(Val) -> - dotag(Tag, ?N_INTEGER, encode_integer(C, Val)); -encode_integer(C,{Name,Val},Tag) when is_atom(Name) -> - encode_integer(C,Val,Tag); -encode_integer(_, Val, _) -> - exit({error,{asn1, {encode_integer, Val}}}). - - -encode_integer(C, Val, NamedNumberList, Tag) when is_atom(Val) -> - case lists:keyfind(Val, 1, NamedNumberList) of - {_, NewVal} -> - dotag(Tag, ?N_INTEGER, encode_integer(C, NewVal)); - _ -> - exit({error,{asn1, {encode_integer_namednumber, Val}}}) - end; -encode_integer(C,{_,Val},NamedNumberList,Tag) -> - encode_integer(C,Val,NamedNumberList,Tag); -encode_integer(C, Val, _NamedNumberList, Tag) -> - dotag(Tag, ?N_INTEGER, encode_integer(C, Val)). - - -encode_integer(_C, Val) -> - Bytes = - if - Val >= 0 -> - encode_integer_pos(Val, []); - true -> - encode_integer_neg(Val, []) - end, - {Bytes,length(Bytes)}. - -encode_integer_pos(0, L=[B|_Acc]) when B < 128 -> - L; -encode_integer_pos(N, Acc) -> - encode_integer_pos((N bsr 8), [N band 16#ff| Acc]). - -encode_integer_neg(-1, L=[B1|_T]) when B1 > 127 -> - L; -encode_integer_neg(N, Acc) -> - encode_integer_neg(N bsr 8, [N band 16#ff|Acc]). - -%%=============================================================================== -%% decode integer -%% (Buffer, Range, HasTag, TotalLen) -> {Integer, Remain, RemovedBytes} -%% (Buffer, Range, NamedNumberList, HasTag, TotalLen) -> {Integer, Remain, RemovedBytes} -%%=============================================================================== - -decode_integer(Buffer, Range, Tags, OptOrMand) -> - NewTags = new_tags(Tags,#tag{class=?UNIVERSAL,number=?N_INTEGER}), - decode_integer_notag(Buffer, Range, [], NewTags, OptOrMand). - -decode_integer(Buffer, Range, NamedNumberList, Tags, OptOrMand) -> - NewTags = new_tags(Tags,#tag{class=?UNIVERSAL,number=?N_INTEGER}), - decode_integer_notag(Buffer, Range, NamedNumberList, NewTags, OptOrMand). - -decode_integer_notag(Buffer, Range, NamedNumberList, NewTags, OptOrMand) -> - {RestTags, {FormLen, Buffer0, Rb0}} = - check_tags_i(NewTags, Buffer, OptOrMand), -% Result = {Val, Buffer2, RemovedBytes} = - case FormLen of - {?CONSTRUCTED,Len} -> - {Buffer00, RestBytes} = split_list(Buffer0,Len), - {Val01, Buffer01, Rb01} = - decode_integer_notag(Buffer00, Range, NamedNumberList, - RestTags, OptOrMand), - {Buffer02, Rb02} = restbytes2(RestBytes,Buffer01,noext), - {Val01, Buffer02, Rb0+Rb01+Rb02}; - {_, Len} -> - Result = - decode_integer2(Len,Buffer0,Rb0+Len), - Result2 = check_integer_constraint(Result,Range), - resolve_named_value(Result2,NamedNumberList) - end. - -resolve_named_value(Result={Val,Buffer,RemBytes},NamedNumberList) -> - case NamedNumberList of - [] -> Result; - _ -> - NewVal = case lists:keyfind(Val, 2, NamedNumberList) of - {NamedVal, _} -> - NamedVal; - _ -> - Val - end, - {NewVal, Buffer, RemBytes} - end. - -check_integer_constraint(Result={Val, _Buffer,_},Range) -> - case Range of - [] -> % No length constraint - Result; - {Lb,Ub} when Val >= Lb, Ub >= Val -> % variable length constraint - Result; - Val -> % fixed value constraint - Result; - {_,_} -> - exit({error,{asn1,{integer_range,Range,Val}}}); - SingleValue when is_integer(SingleValue) -> - exit({error,{asn1,{integer_range,Range,Val}}}); - _ -> % some strange constraint that we don't support yet - Result - end. - -%%============================================================================ -%% Enumerated value, ITU_T X.690 Chapter 8.4 - -%% encode enumerated value -%%============================================================================ -encode_enumerated(Val, []) when is_integer(Val) -> - {EncVal,Len} = encode_integer(false,Val), - dotag_universal(?N_ENUMERATED,EncVal,Len); -encode_enumerated(Val, DoTag) when is_integer(Val) -> - dotag(DoTag, ?N_ENUMERATED, encode_integer(false,Val)); -encode_enumerated({Name,Val}, DoTag) when is_atom(Name) -> - encode_enumerated(Val, DoTag). - -%% The encode_enumerated functions below this line can be removed when the -%% new code generation is stable. (the functions might have to be kept here -%% a while longer for compatibility reasons) - -encode_enumerated(C, Val, {NamedNumberList,ExtList}, DoTag) when is_atom(Val) -> - case catch encode_enumerated(C, Val, NamedNumberList, DoTag) of - {'EXIT',_} -> encode_enumerated(C, Val, ExtList, DoTag); - Result -> Result - end; - -encode_enumerated(C, Val, NamedNumberList, DoTag) when is_atom(Val) -> - case lists:keyfind(Val, 1, NamedNumberList) of - {_, NewVal} when DoTag =:= [] -> - {EncVal,Len} = encode_integer(C,NewVal), - dotag_universal(?N_ENUMERATED,EncVal,Len); - {_, NewVal} -> - dotag(DoTag, ?N_ENUMERATED, encode_integer(C, NewVal)); - _ -> - exit({error,{asn1, {enumerated_not_in_range, Val}}}) - end; - -encode_enumerated(C, {asn1_enum, Val}, {_,_}, DoTag) when is_integer(Val) -> - dotag(DoTag, ?N_ENUMERATED, encode_integer(C,Val)); - -encode_enumerated(C, {Name,Val}, NamedNumberList, DoTag) when is_atom(Name) -> - encode_enumerated(C, Val, NamedNumberList, DoTag); - -encode_enumerated(_, Val, _, _) -> - exit({error,{asn1, {enumerated_not_namednumber, Val}}}). - - - -%%============================================================================ -%% decode enumerated value -%% (Buffer, Range, NamedNumberList, HasTag, TotalLen) -> -%% {Value, RemainingBuffer, RemovedBytes} -%%=========================================================================== -decode_enumerated(Buffer, Range, NamedNumberList, Tags, OptOrMand) -> - NewTags = new_tags(Tags,#tag{class=?UNIVERSAL,number=?N_ENUMERATED}), - decode_enumerated_notag(Buffer, Range, NamedNumberList, - NewTags, OptOrMand). - -decode_enumerated_notag(Buffer, Range, NNList = {NamedNumberList,ExtList}, Tags, OptOrMand) -> - {RestTags, {FormLen, Buffer0, Rb0}} = - check_tags_i(Tags, Buffer, OptOrMand), - - case FormLen of - {?CONSTRUCTED,Len} -> - {Buffer00,RestBytes} = split_list(Buffer0,Len), - {Val01, Buffer01, Rb01} = - decode_enumerated_notag(Buffer00, Range, NNList, RestTags, OptOrMand), - {Buffer02, Rb02} = restbytes2(RestBytes,Buffer01,noext), - {Val01, Buffer02, Rb0+Rb01+Rb02}; - {_,Len} -> - {Val01, Buffer01, Rb01} = - decode_integer2(Len, Buffer0, Rb0+Len), - case decode_enumerated1(Val01, NamedNumberList) of - {asn1_enum,Val01} -> - {decode_enumerated1(Val01,ExtList), Buffer01, Rb01}; - Result01 -> - {Result01, Buffer01, Rb01} - end - end; - -decode_enumerated_notag(Buffer, Range, NNList, Tags, OptOrMand) -> - {RestTags, {FormLen, Buffer0, Rb0}} = - check_tags_i(Tags, Buffer, OptOrMand), - - case FormLen of - {?CONSTRUCTED,Len} -> - {Buffer00,RestBytes} = split_list(Buffer0,Len), - {Val01, Buffer01, Rb01} = - decode_enumerated_notag(Buffer00, Range, NNList, RestTags, OptOrMand), - {Buffer02, Rb02} = restbytes2(RestBytes,Buffer01,noext), - {Val01, Buffer02, Rb0+Rb01+Rb02}; - {_,Len} -> - {Val01, Buffer02, Rb02} = - decode_integer2(Len, Buffer0, Rb0+Len), - case decode_enumerated1(Val01, NNList) of - {asn1_enum,_} -> - exit({error,{asn1, {illegal_enumerated, Val01}}}); - Result01 -> - {Result01, Buffer02, Rb02} - end - end. - -decode_enumerated1(Val, NamedNumberList) -> - %% it must be a named integer - case lists:keyfind(Val, 2, NamedNumberList) of - {NamedVal, _} -> - NamedVal; - _ -> - {asn1_enum,Val} - end. - - %%============================================================================ %% %% Real value, ITU_T X.690 Chapter 8.5 @@ -1117,513 +412,15 @@ decode_real2(Buffer0, _C, Len, RemBytes1) -> {{Mantissa, Base, Exp}, Buffer4, RemBytes2+RemBytes3} end. +encode_integer_pos(0, L=[B|_Acc]) when B < 128 -> + L; +encode_integer_pos(N, Acc) -> + encode_integer_pos((N bsr 8), [N band 16#ff| Acc]). -%%============================================================================ -%% Bitstring value, ITU_T X.690 Chapter 8.6 -%% -%% encode bitstring value -%% -%% bitstring NamedBitList -%% Val can be of: -%% - [identifiers] where only named identifers are set to one, -%% the Constraint must then have some information of the -%% bitlength. -%% - [list of ones and zeroes] all bits -%% - integer value representing the bitlist -%% C is constrint Len, only valid when identifiers -%%============================================================================ - -encode_bit_string(C,Bin={Unused,BinBits},NamedBitList,DoTag) when is_integer(Unused), is_binary(BinBits) -> - encode_bin_bit_string(C,Bin,NamedBitList,DoTag); -encode_bit_string(C, [FirstVal | RestVal], NamedBitList, DoTag) when is_atom(FirstVal) -> - encode_bit_string_named(C, [FirstVal | RestVal], NamedBitList, DoTag); - -encode_bit_string(C, [{bit,X} | RestVal], NamedBitList, DoTag) -> - encode_bit_string_named(C, [{bit,X} | RestVal], NamedBitList, DoTag); - -encode_bit_string(C, [FirstVal| RestVal], NamedBitList, DoTag) when is_integer(FirstVal) -> - encode_bit_string_bits(C, [FirstVal | RestVal], NamedBitList, DoTag); - -encode_bit_string(_, 0, _, []) -> - {[?N_BIT_STRING,1,0],3}; - -encode_bit_string(_, 0, _, DoTag) -> - dotag(DoTag, ?N_BIT_STRING, {<<0>>,1}); - -encode_bit_string(_, [], _, []) -> - {[?N_BIT_STRING,1,0],3}; - -encode_bit_string(_, [], _, DoTag) -> - dotag(DoTag, ?N_BIT_STRING, {<<0>>,1}); - -encode_bit_string(C, IntegerVal, NamedBitList, DoTag) when is_integer(IntegerVal) -> - BitListVal = int_to_bitlist(IntegerVal), - encode_bit_string_bits(C, BitListVal, NamedBitList, DoTag); - -encode_bit_string(C, {Name,BitList}, NamedBitList, DoTag) when is_atom(Name) -> - encode_bit_string(C, BitList, NamedBitList, DoTag). - - - -int_to_bitlist(0) -> - []; -int_to_bitlist(Int) when is_integer(Int), Int >= 0 -> - [Int band 1 | int_to_bitlist(Int bsr 1)]. - - -%%================================================================= -%% Encode BIT STRING of the form {Unused,BinBits}. -%% Unused is the number of unused bits in the last byte in BinBits -%% and BinBits is a binary representing the BIT STRING. -%%================================================================= -encode_bin_bit_string(C,{Unused,BinBits},_NamedBitList,DoTag)-> - case get_constraint(C,'SizeConstraint') of - no -> - remove_unused_then_dotag(DoTag,?N_BIT_STRING,Unused,BinBits); - {_Min,Max} -> - BBLen = (size(BinBits)*8)-Unused, - if - BBLen > Max -> - exit({error,{asn1, - {bitstring_length, - {{was,BBLen},{maximum,Max}}}}}); - true -> - remove_unused_then_dotag(DoTag,?N_BIT_STRING, - Unused,BinBits) - end; - Size -> - case ((size(BinBits)*8)-Unused) of - BBSize when BBSize =< Size -> - remove_unused_then_dotag(DoTag,?N_BIT_STRING, - Unused,BinBits); - BBSize -> - exit({error,{asn1, - {bitstring_length, - {{was,BBSize},{should_be,Size}}}}}) - end - end. - -remove_unused_then_dotag(DoTag,StringType,Unused,BinBits) -> - case Unused of - 0 when (byte_size(BinBits) =:= 0), DoTag =:= [] -> - %% time optimization of next case - {[StringType,1,0],3}; - 0 when (byte_size(BinBits) =:= 0) -> - dotag(DoTag,StringType,{<<0>>,1}); - 0 when DoTag =:= [] -> % time optimization of next case - dotag_universal(StringType,[Unused|[BinBits]],size(BinBits)+1); -% {LenEnc,Len} = encode_legth(size(BinBits)+1), -% {[StringType,LenEnc,[Unused|BinBits]],size(BinBits)+1+Len+1}; - 0 -> - dotag(DoTag,StringType,<<Unused,BinBits/binary>>); - Num when DoTag =:= [] -> % time optimization of next case - N = byte_size(BinBits) - 1, - <<BBits:N/binary,LastByte>> = BinBits, - dotag_universal(StringType, - [Unused,BBits,(LastByte bsr Num) bsl Num], - byte_size(BinBits) + 1); -% {LenEnc,Len} = encode_legth(size(BinBits)+1), -% {[StringType,LenEnc,[Unused,BBits,(LastByte bsr Num) bsl Num], -% 1+Len+size(BinBits)+1}; - Num -> - N = byte_size(BinBits) - 1, - <<BBits:N/binary,LastByte>> = BinBits, - dotag(DoTag,StringType,{[Unused,binary_to_list(BBits) ++ - [(LastByte bsr Num) bsl Num]], - byte_size(BinBits) + 1}) - end. - - -%%================================================================= -%% Encode named bits -%%================================================================= - -encode_bit_string_named(C, [FirstVal | RestVal], NamedBitList, DoTag) -> - {Len,Unused,OctetList} = - case get_constraint(C,'SizeConstraint') of - no -> - ToSetPos = get_all_bitposes([FirstVal | RestVal], - NamedBitList, []), - BitList = make_and_set_list(lists:max(ToSetPos)+1, - ToSetPos, 0), - encode_bitstring(BitList); - {_Min,Max} -> - ToSetPos = get_all_bitposes([FirstVal | RestVal], - NamedBitList, []), - BitList = make_and_set_list(Max, ToSetPos, 0), - encode_bitstring(BitList); - Size -> - ToSetPos = get_all_bitposes([FirstVal | RestVal], - NamedBitList, []), - BitList = make_and_set_list(Size, ToSetPos, 0), - encode_bitstring(BitList) - end, - case DoTag of - [] -> - dotag_universal(?N_BIT_STRING,[Unused|OctetList],Len+1); -% {EncLen,LenLen} = encode_length(Len+1), -% {[?N_BIT_STRING,EncLen,Unused,OctetList],1+LenLen+Len+1}; - _ -> - dotag(DoTag, ?N_BIT_STRING, {[Unused|OctetList],Len+1}) - end. - - -%%---------------------------------------- -%% get_all_bitposes([list of named bits to set], named_bit_db, []) -> -%% [sorted_list_of_bitpositions_to_set] -%%---------------------------------------- - -get_all_bitposes([{bit,ValPos}|Rest], NamedBitList, Ack) -> - get_all_bitposes(Rest, NamedBitList, [ValPos | Ack ]); -get_all_bitposes([Val | Rest], NamedBitList, Ack) when is_atom(Val) -> - case lists:keyfind(Val, 1, NamedBitList) of - {_ValName, ValPos} -> - get_all_bitposes(Rest, NamedBitList, [ValPos | Ack]); - _ -> - exit({error,{asn1, {bitstring_namedbit, Val}}}) - end; -get_all_bitposes([], _NamedBitList, Ack) -> - lists:sort(Ack). - - -%%---------------------------------------- -%% make_and_set_list(Len of list to return, [list of positions to set to 1])-> -%% returns list of Len length, with all in SetPos set. -%% in positioning in list the first element is 0, the second 1 etc.., but -%% Len will make a list of length Len, not Len + 1. -%% BitList = make_and_set_list(C, ToSetPos, 0), -%%---------------------------------------- - -make_and_set_list(0, [], _) -> []; -make_and_set_list(0, _, _) -> - exit({error,{asn1,bitstring_sizeconstraint}}); -make_and_set_list(Len, [XPos|SetPos], XPos) -> - [1 | make_and_set_list(Len - 1, SetPos, XPos + 1)]; -make_and_set_list(Len, [Pos|SetPos], XPos) -> - [0 | make_and_set_list(Len - 1, [Pos | SetPos], XPos + 1)]; -make_and_set_list(Len, [], XPos) -> - [0 | make_and_set_list(Len - 1, [], XPos + 1)]. - - -%%================================================================= -%% Encode bit string for lists of ones and zeroes -%%================================================================= -encode_bit_string_bits(C, BitListVal, _NamedBitList, DoTag) when is_list(BitListVal) -> - {Len,Unused,OctetList} = - case get_constraint(C,'SizeConstraint') of - no -> - encode_bitstring(BitListVal); - Constr={Min,_Max} when is_integer(Min) -> - encode_constr_bit_str_bits(Constr,BitListVal,DoTag); - {Constr={_,_},[]} -> - %% constraint with extension mark - encode_constr_bit_str_bits(Constr,BitListVal,DoTag); - Constr={{_,_},{_,_}} ->%{{Min1,Max1},{Min2,Max2}} - %% constraint with extension mark - encode_constr_bit_str_bits(Constr,BitListVal,DoTag); - Size -> - case length(BitListVal) of - BitSize when BitSize =:= Size -> - encode_bitstring(BitListVal); - BitSize when BitSize < Size -> - PaddedList = - pad_bit_list(Size-BitSize,BitListVal), - encode_bitstring(PaddedList); - BitSize -> - exit({error, - {asn1, - {bitstring_length, - {{was,BitSize}, - {should_be,Size}}}}}) - end - end, - %%add unused byte to the Len - case DoTag of - [] -> - dotag_universal(?N_BIT_STRING,[Unused|OctetList],Len+1); -% {EncLen,LenLen}=encode_length(Len+1), -% {[?N_BIT_STRING,EncLen,Unused|OctetList],1+LenLen+Len+1}; - _ -> - dotag(DoTag, ?N_BIT_STRING, - {[Unused | OctetList],Len+1}) - end. - - -encode_constr_bit_str_bits({{_Min1,Max1},{Min2,Max2}},BitListVal,_DoTag) -> - BitLen = length(BitListVal), - case BitLen of - Len when Len > Max2 -> - exit({error,{asn1,{bitstring_length,{{was,BitLen}, - {maximum,Max2}}}}}); - Len when Len > Max1, Len < Min2 -> - exit({error,{asn1,{bitstring_length,{{was,BitLen}, - {not_allowed_interval, - Max1,Min2}}}}}); - _ -> - encode_bitstring(BitListVal) - end; -encode_constr_bit_str_bits({Min,Max},BitListVal,_DoTag) -> - BitLen = length(BitListVal), - if - BitLen > Max -> - exit({error,{asn1,{bitstring_length,{{was,BitLen}, - {maximum,Max}}}}}); - BitLen < Min -> - exit({error,{asn1,{bitstring_length,{{was,BitLen}, - {minimum,Min}}}}}); - true -> - encode_bitstring(BitListVal) - end. - - -%% returns a list of length Size + length(BitListVal), with BitListVal -%% as the most significant elements followed by padded zero elements -pad_bit_list(Size,BitListVal) -> - Tail = lists:duplicate(Size,0), - BitListVal ++ Tail. - -%%================================================================= -%% Do the actual encoding -%% ([bitlist]) -> {ListLen, UnusedBits, OctetList} -%%================================================================= - -encode_bitstring([B8, B7, B6, B5, B4, B3, B2, B1 | Rest]) -> - Val = (B8 bsl 7) bor (B7 bsl 6) bor (B6 bsl 5) bor (B5 bsl 4) bor - (B4 bsl 3) bor (B3 bsl 2) bor (B2 bsl 1) bor B1, - encode_bitstring(Rest, [Val], 1); -encode_bitstring(Val) -> - {Unused, Octet} = unused_bitlist(Val, 7, 0), - {1, Unused, [Octet]}. - -encode_bitstring([B8, B7, B6, B5, B4, B3, B2, B1 | Rest], Ack, Len) -> - Val = (B8 bsl 7) bor (B7 bsl 6) bor (B6 bsl 5) bor (B5 bsl 4) bor - (B4 bsl 3) bor (B3 bsl 2) bor (B2 bsl 1) bor B1, - encode_bitstring(Rest, [Ack | [Val]], Len + 1); -%%even multiple of 8 bits.. -encode_bitstring([], Ack, Len) -> - {Len, 0, Ack}; -%% unused bits in last octet -encode_bitstring(Rest, Ack, Len) -> -% io:format("uneven ~w ~w ~w~n",[Rest, Ack, Len]), - {Unused, Val} = unused_bitlist(Rest, 7, 0), - {Len + 1, Unused, [Ack | [Val]]}. - -%%%%%%%%%%%%%%%%%% -%% unused_bitlist([list of ones and zeros <= 7], 7, []) -> -%% {Unused bits, Last octet with bits moved to right} -unused_bitlist([], Trail, Ack) -> - {Trail + 1, Ack}; -unused_bitlist([Bit | Rest], Trail, Ack) -> -%% io:format("trail Bit: ~w Rest: ~w Trail: ~w Ack:~w~n",[Bit, Rest, Trail, Ack]), - unused_bitlist(Rest, Trail - 1, (Bit bsl Trail) bor Ack). - - -%%============================================================================ -%% decode bitstring value -%% (Buffer, Range, NamedNumberList, HasTag, TotalLen) -> {Integer, Remain, RemovedBytes} -%%============================================================================ - -decode_compact_bit_string(Buffer, Range, NamedNumberList, Tags, LenIn, OptOrMand) -> -% NewTags = new_tags(HasTag,#tag{class=?UNIVERSAL,number=?N_BIT_STRING}), - decode_restricted_string(Buffer, Range, ?N_BIT_STRING, Tags, LenIn, - NamedNumberList, OptOrMand,bin). - -decode_bit_string(Buffer, Range, NamedNumberList, Tags, LenIn, OptOrMand) -> -% NewTags = new_tags(HasTag,#tag{class=?UNIVERSAL,number=?N_BIT_STRING}), - decode_restricted_string(Buffer, Range, ?N_BIT_STRING, Tags, LenIn, - NamedNumberList, OptOrMand,old). - - -decode_bit_string2(1,<<0 ,Buffer/binary>>,_NamedNumberList,RemovedBytes,BinOrOld) -> - case BinOrOld of - bin -> - {{0,<<>>},Buffer,RemovedBytes}; - _ -> - {[], Buffer, RemovedBytes} - end; -decode_bit_string2(Len,<<Unused,Buffer/binary>>,NamedNumberList, - RemovedBytes,BinOrOld) -> - L = Len - 1, - <<Bits:L/binary,BufferTail/binary>> = Buffer, - case NamedNumberList of - [] -> - case BinOrOld of - bin -> - {{Unused,Bits},BufferTail,RemovedBytes}; - _ -> - BitString = decode_bitstring2(L, Unused, Buffer), - {BitString,BufferTail, RemovedBytes} - end; - _ -> - BitString = decode_bitstring2(L, Unused, Buffer), - {decode_bitstring_NNL(BitString,NamedNumberList), - BufferTail, - RemovedBytes} - end. - -%%---------------------------------------- -%% Decode the in buffer to bits -%%---------------------------------------- -decode_bitstring2(1,Unused,<<B7:1,B6:1,B5:1,B4:1,B3:1,B2:1,B1:1,B0:1,_/binary>>) -> - lists:sublist([B7,B6,B5,B4,B3,B2,B1,B0],8-Unused); -decode_bitstring2(Len, Unused, - <<B7:1,B6:1,B5:1,B4:1,B3:1,B2:1,B1:1,B0:1,Buffer/binary>>) -> - [B7, B6, B5, B4, B3, B2, B1, B0 | - decode_bitstring2(Len - 1, Unused, Buffer)]. - -%%decode_bitstring2(1, Unused, Buffer) -> -%% make_bits_of_int(hd(Buffer), 128, 8-Unused); -%%decode_bitstring2(Len, Unused, [BitVal | Buffer]) -> -%% [B7, B6, B5, B4, B3, B2, B1, B0] = make_bits_of_int(BitVal, 128, 8), -%% [B7, B6, B5, B4, B3, B2, B1, B0 | -%% decode_bitstring2(Len - 1, Unused, Buffer)]. - - -%%make_bits_of_int(_, _, 0) -> -%% []; -%%make_bits_of_int(BitVal, MaskVal, Unused) when Unused > 0 -> -%% X = case MaskVal band BitVal of -%% 0 -> 0 ; -%% _ -> 1 -%% end, -%% [X | make_bits_of_int(BitVal, MaskVal bsr 1, Unused - 1)]. - - - -%%---------------------------------------- -%% Decode the bitlist to names -%%---------------------------------------- - -decode_bitstring_NNL(BitList,NamedNumberList) -> - decode_bitstring_NNL(BitList,NamedNumberList,0,[]). - - -decode_bitstring_NNL([],_,_No,Result) -> - lists:reverse(Result); - -decode_bitstring_NNL([B|BitList],[{Name,No}|NamedNumberList],No,Result) -> - if - B =:= 0 -> - decode_bitstring_NNL(BitList,NamedNumberList,No+1,Result); - true -> - decode_bitstring_NNL(BitList,NamedNumberList,No+1,[Name|Result]) - end; -decode_bitstring_NNL([1|BitList],NamedNumberList,No,Result) -> - decode_bitstring_NNL(BitList,NamedNumberList,No+1,[{bit,No}|Result]); -decode_bitstring_NNL([0|BitList],NamedNumberList,No,Result) -> - decode_bitstring_NNL(BitList,NamedNumberList,No+1,Result). - - -%%============================================================================ -%% Octet string, ITU_T X.690 Chapter 8.7 -%% -%% encode octet string -%% The OctetList must be a flat list of integers in the range 0..255 -%% the function does not check this because it takes to much time -%%============================================================================ -encode_octet_string(_C, OctetList, []) when is_binary(OctetList) -> - dotag_universal(?N_OCTET_STRING,OctetList,byte_size(OctetList)); -encode_octet_string(_C, OctetList, DoTag) when is_binary(OctetList) -> - dotag(DoTag, ?N_OCTET_STRING, {OctetList,byte_size(OctetList)}); -encode_octet_string(_C, OctetList, DoTag) when is_list(OctetList) -> - case length(OctetList) of - Len when DoTag =:= [] -> - dotag_universal(?N_OCTET_STRING,OctetList,Len); - Len -> - dotag(DoTag, ?N_OCTET_STRING, {OctetList,Len}) - end; -%% encode_octet_string(C, OctetList, DoTag) when is_list(OctetList) -> -%% dotag(DoTag, ?N_OCTET_STRING, {OctetList,length(OctetList)}); -encode_octet_string(C, {Name,OctetList}, DoTag) when is_atom(Name) -> - encode_octet_string(C, OctetList, DoTag). - - -%%============================================================================ -%% decode octet string -%% (Buffer, Range, HasTag, TotalLen) -> {String, Remain, RemovedBytes} -%% -%% Octet string is decoded as a restricted string -%%============================================================================ -decode_octet_string(Buffer, Range, Tags, TotalLen, OptOrMand) -> -%% NewTags = new_tags(HasTag,#tag{class=?UNIVERSAL,number=?N_OCTET_STRING}), - decode_restricted_string(Buffer, Range, ?N_OCTET_STRING, - Tags, TotalLen, [], OptOrMand,old). - -%%============================================================================ -%% Null value, ITU_T X.690 Chapter 8.8 -%% -%% encode NULL value -%%============================================================================ - -encode_null(_, []) -> - {[?N_NULL,0],2}; -encode_null(_, DoTag) -> - dotag(DoTag, ?N_NULL, {[],0}). - -%%============================================================================ -%% decode NULL value -%% (Buffer, HasTag, TotalLen) -> {NULL, Remain, RemovedBytes} -%%============================================================================ -decode_null(Buffer, Tags, OptOrMand) -> - NewTags = new_tags(Tags,#tag{class=?UNIVERSAL,number=?N_NULL}), - decode_null_notag(Buffer, NewTags, OptOrMand). - -decode_null_notag(Buffer, Tags, OptOrMand) -> - {RestTags, {FormLen, Buffer0, Rb0}} = - check_tags_i(Tags, Buffer, OptOrMand), - - case FormLen of - {?CONSTRUCTED,Len} -> - {_Buffer00,RestBytes} = split_list(Buffer0,Len), - {Val01, Buffer01, Rb01} = decode_null_notag(Buffer0, RestTags, - OptOrMand), - {Buffer02, Rb02} = restbytes2(RestBytes,Buffer01,noext), - {Val01, Buffer02, Rb0+Rb01+Rb02}; - {_,0} -> - {'NULL', Buffer0, Rb0}; - {_,Len} -> - exit({error,{asn1,{invalid_length,'NULL',Len}}}) - end. - - -%%============================================================================ -%% Object identifier, ITU_T X.690 Chapter 8.19 -%% -%% encode Object Identifier value -%%============================================================================ - -encode_object_identifier({Name,Val}, DoTag) when is_atom(Name) -> - encode_object_identifier(Val, DoTag); -encode_object_identifier(Val, []) -> - {EncVal,Len} = e_object_identifier(Val), - dotag_universal(?N_OBJECT_IDENTIFIER,EncVal,Len); -encode_object_identifier(Val, DoTag) -> - dotag(DoTag, ?N_OBJECT_IDENTIFIER, e_object_identifier(Val)). - -e_object_identifier({'OBJECT IDENTIFIER', V}) -> - e_object_identifier(V); -e_object_identifier({Cname, V}) when is_atom(Cname), is_tuple(V) -> - e_object_identifier(tuple_to_list(V)); -e_object_identifier({Cname, V}) when is_atom(Cname), is_list(V) -> - e_object_identifier(V); -e_object_identifier(V) when is_tuple(V) -> - e_object_identifier(tuple_to_list(V)); - -%%%%%%%%%%%%%%% -%% e_object_identifier([List of Obect Identifiers]) -> -%% {[Encoded Octetlist of ObjIds], IntLength} -%% -e_object_identifier([E1, E2 | Tail]) -> - Head = 40*E1 + E2, % wow! - {H,Lh} = mk_object_val(Head), - {R,Lr} = enc_obj_id_tail(Tail, [], 0), - {[H|R], Lh+Lr}. - -enc_obj_id_tail([], Ack, Len) -> - {lists:reverse(Ack), Len}; -enc_obj_id_tail([H|T], Ack, Len) -> - {B, L} = mk_object_val(H), - enc_obj_id_tail(T, [B|Ack], Len+L). +encode_integer_neg(-1, L=[B1|_T]) when B1 > 127 -> + L; +encode_integer_neg(N, Acc) -> + encode_integer_neg(N bsr 8, [N band 16#ff|Acc]). %%%%%%%%%%% @@ -1643,476 +440,6 @@ mk_object_val(Val, Ack, Len) -> mk_object_val(Val bsr 7, [((Val band 127) bor 128) | Ack], Len + 1). - -%%============================================================================ -%% decode Object Identifier value -%% (Buffer, HasTag, TotalLen) -> {{ObjId}, Remain, RemovedBytes} -%%============================================================================ - -decode_object_identifier(Buffer, Tags, OptOrMand) -> - NewTags = new_tags(Tags,#tag{class=?UNIVERSAL, - number=?N_OBJECT_IDENTIFIER}), - decode_object_identifier_notag(Buffer, NewTags, OptOrMand). - -decode_object_identifier_notag(Buffer, Tags, OptOrMand) -> - {RestTags, {FormLen, Buffer0, Rb0}} = - check_tags_i(Tags, Buffer, OptOrMand), - - case FormLen of - {?CONSTRUCTED,Len} -> - {Buffer00,RestBytes} = split_list(Buffer0,Len), - {Val01, Buffer01, Rb01} = - decode_object_identifier_notag(Buffer00, - RestTags, OptOrMand), - {Buffer02, Rb02} = restbytes2(RestBytes,Buffer01,noext), - {Val01, Buffer02, Rb0+Rb01+Rb02}; - {_,Len} -> - {[AddedObjVal|ObjVals],Buffer01} = - dec_subidentifiers(Buffer0,0,[],Len), - {Val1, Val2} = if - AddedObjVal < 40 -> - {0, AddedObjVal}; - AddedObjVal < 80 -> - {1, AddedObjVal - 40}; - true -> - {2, AddedObjVal - 80} - end, - {list_to_tuple([Val1, Val2 | ObjVals]), Buffer01, - Rb0+Len} - end. - -dec_subidentifiers(Buffer,_Av,Al,0) -> - {lists:reverse(Al),Buffer}; -dec_subidentifiers(<<1:1,H:7,T/binary>>,Av,Al,Len) -> - dec_subidentifiers(T,(Av bsl 7) + H,Al,Len-1); -dec_subidentifiers(<<H,T/binary>>,Av,Al,Len) -> - dec_subidentifiers(T,0,[((Av bsl 7) + H)|Al],Len-1). - -%%============================================================================ -%% RELATIVE-OID, ITU_T X.690 Chapter 8.20 -%% -%% encode Relative Object Identifier -%%============================================================================ -encode_relative_oid({Name,Val},TagIn) when is_atom(Name) -> - encode_relative_oid(Val,TagIn); -encode_relative_oid(Val,TagIn) when is_tuple(Val) -> - encode_relative_oid(tuple_to_list(Val),TagIn); -encode_relative_oid(Val,[]) -> - {EncVal,Len} = enc_relative_oid(Val), - dotag_universal(?'N_RELATIVE-OID',EncVal,Len); -encode_relative_oid(Val, DoTag) -> - dotag(DoTag, ?'N_RELATIVE-OID', enc_relative_oid(Val)). - -enc_relative_oid(Val) -> - lists:mapfoldl(fun(X,AccIn) -> - {SO,L}=mk_object_val(X), - {SO,L+AccIn} - end - ,0,Val). - -%%============================================================================ -%% decode Relative Object Identifier value -%% (Buffer, HasTag, TotalLen) -> {{ObjId}, Remain, RemovedBytes} -%%============================================================================ -decode_relative_oid(Buffer, Tags, OptOrMand) -> - NewTags = new_tags(Tags,#tag{class=?UNIVERSAL, - number=?'N_RELATIVE-OID'}), - decode_relative_oid_notag(Buffer, NewTags, OptOrMand). - -decode_relative_oid_notag(Buffer, Tags, OptOrMand) -> - {_RestTags, {_FormLen={_,Len}, Buffer0, Rb0}} = - check_tags_i(Tags, Buffer, OptOrMand), - {ObjVals,Buffer01} = - dec_subidentifiers(Buffer0,0,[],Len), - {list_to_tuple(ObjVals), Buffer01, Rb0+Len}. - -%%============================================================================ -%% Restricted character string types, ITU_T X.690 Chapter 8.21 -%% -%% encode Numeric Printable Teletex Videotex Visible IA5 Graphic General strings -%%============================================================================ -encode_restricted_string(_C, OctetList, StringType, []) - when is_binary(OctetList) -> - dotag_universal(StringType, OctetList, byte_size(OctetList)); -encode_restricted_string(_C, OctetList, StringType, DoTag) - when is_binary(OctetList) -> - dotag(DoTag, StringType, {OctetList, byte_size(OctetList)}); -encode_restricted_string(_C, OctetList, StringType, []) - when is_list(OctetList) -> - dotag_universal(StringType, OctetList, length(OctetList)); -encode_restricted_string(_C, OctetList, StringType, DoTag) - when is_list(OctetList) -> - dotag(DoTag, StringType, {OctetList, length(OctetList)}); -encode_restricted_string(C,{Name,OctetL},StringType,DoTag) when is_atom(Name) -> - encode_restricted_string(C, OctetL, StringType, DoTag). - -%%============================================================================ -%% decode Numeric Printable Teletex Videotex Visible IA5 Graphic General strings -%% (Buffer, Range, StringType, HasTag, TotalLen) -> -%% {String, Remain, RemovedBytes} -%%============================================================================ - -decode_restricted_string(Buffer, Range, StringType, Tags, LenIn, OptOrMand) -> - {Val,Buffer2,Rb} = - decode_restricted_string_tag(Buffer, Range, StringType, Tags, - LenIn, [], OptOrMand,old), - {check_and_convert_restricted_string(Val,StringType,Range,[],old), - Buffer2,Rb}. - - -decode_restricted_string(Buffer, Range, StringType, Tags, LenIn, NNList, OptOrMand, BinOrOld ) -> - {Val,Buffer2,Rb} = - decode_restricted_string_tag(Buffer, Range, StringType, Tags, - LenIn, NNList, OptOrMand, BinOrOld), - {check_and_convert_restricted_string(Val,StringType,Range,NNList,BinOrOld), - Buffer2,Rb}. - -decode_restricted_string_tag(Buffer, Range, StringType, TagsIn, LenIn, NNList, OptOrMand, BinOrOld ) -> - NewTags = new_tags(TagsIn, #tag{class=?UNIVERSAL,number=StringType}), - decode_restricted_string_notag(Buffer, Range, StringType, NewTags, - LenIn, NNList, OptOrMand, BinOrOld). - - -check_and_convert_restricted_string(Val,StringType,Range,NamedNumberList,_BinOrOld) -> - {StrLen,NewVal} = case StringType of - ?N_BIT_STRING when NamedNumberList =/= [] -> - {no_check,Val}; - ?N_BIT_STRING when is_list(Val) -> - {length(Val),Val}; - ?N_BIT_STRING when is_tuple(Val) -> - {(size(element(2,Val))*8) - element(1,Val),Val}; - _ when is_binary(Val) -> - {byte_size(Val),binary_to_list(Val)}; - _ when is_list(Val) -> - {length(Val), Val} - end, - case Range of - _ when StrLen =:= no_check -> - NewVal; - [] -> % No length constraint - NewVal; - {Lb,Ub} when StrLen >= Lb, Ub >= StrLen -> % variable length constraint - NewVal; - {{Lb,_Ub},[]} when StrLen >= Lb -> - NewVal; - {{Lb,_Ub},_Ext=[MinExt|_]} when StrLen >= Lb; StrLen >= MinExt -> - NewVal; - {{Lb1,Ub1},{Lb2,Ub2}} when StrLen >= Lb1, StrLen =< Ub1; - StrLen =< Ub2, StrLen >= Lb2 -> - NewVal; - StrLen -> % fixed length constraint - NewVal; - {_,_} -> - exit({error,{asn1,{length,Range,Val}}}); - _Len when is_integer(_Len) -> - exit({error,{asn1,{length,Range,Val}}}); - _ -> % some strange constraint that we don't support yet - NewVal - end. - -%%============================================================================= -%% Common routines for several string types including bit string -%% handles indefinite length -%%============================================================================= - - -decode_restricted_string_notag(Buffer, _Range, StringType, TagsIn, - _, NamedNumberList, OptOrMand, BinOrOld) -> - %%----------------------------------------------------------- - %% Get inner (the implicit tag or no tag) and - %% outer (the explicit tag) lengths. - %%----------------------------------------------------------- - {RestTags, {FormLength={_,_Len01}, Buffer0, Rb0}} = - check_tags_i(TagsIn, Buffer, OptOrMand), - - case FormLength of - {?CONSTRUCTED,Len} -> - {Buffer00, RestBytes} = split_list(Buffer0,Len), - {Val01, Buffer01, Rb01} = - decode_restricted_parts(Buffer00, RestBytes, [], StringType, - RestTags, - Len, NamedNumberList, - OptOrMand, - BinOrOld, 0, []), - {Val01, Buffer01, Rb0+Rb01}; - {_, Len} -> - {Val01, Buffer01, Rb01} = - decode_restricted(Buffer0, Len, StringType, - NamedNumberList, BinOrOld), - {Val01, Buffer01, Rb0+Rb01} - end. - - -decode_restricted_parts(Buffer, RestBytes, [], StringType, RestTags, Len, NNList, - OptOrMand, BinOrOld, AccRb, AccVal) -> - DecodeFun = case RestTags of - [] -> fun decode_restricted_string_tag/8; - _ -> fun decode_restricted_string_notag/8 - end, - {Val, Buffer1, Rb} = - DecodeFun(Buffer, [], StringType, RestTags, - no_length, NNList, - OptOrMand, BinOrOld), - {Buffer2,More} = - case Buffer1 of - <<0,0,Buffer10/binary>> when Len == indefinite -> - {Buffer10,false}; - <<>> -> - {RestBytes,false}; - _ -> - {Buffer1,true} - end, - {NewVal, NewRb} = - case StringType of - ?N_BIT_STRING when BinOrOld == bin -> - {concat_bit_binaries(AccVal, Val), AccRb+Rb}; - _ when is_binary(Val),is_binary(AccVal) -> - {<<AccVal/binary,Val/binary>>,AccRb+Rb}; - _ when is_binary(Val), AccVal =:= [] -> - {Val,AccRb+Rb}; - _ -> - {AccVal++Val, AccRb+Rb} - end, - case More of - false -> - {NewVal, Buffer2, NewRb}; - true -> - decode_restricted_parts(Buffer2, RestBytes, [], StringType, RestTags, Len, NNList, - OptOrMand, BinOrOld, NewRb, NewVal) - end. - - - -decode_restricted(Buffer, InnerLen, StringType, NamedNumberList,BinOrOld) -> - - case StringType of - ?N_BIT_STRING -> - decode_bit_string2(InnerLen,Buffer,NamedNumberList,InnerLen,BinOrOld); - - ?N_UniversalString -> - <<PreBuff:InnerLen/binary,RestBuff/binary>> = Buffer,%%added for binary - UniString = mk_universal_string(binary_to_list(PreBuff)), - {UniString,RestBuff,InnerLen}; - ?N_BMPString -> - <<PreBuff:InnerLen/binary,RestBuff/binary>> = Buffer,%%added for binary - BMP = mk_BMP_string(binary_to_list(PreBuff)), - {BMP,RestBuff,InnerLen}; - _ -> - <<PreBuff:InnerLen/binary,RestBuff/binary>> = Buffer,%%added for binary - {PreBuff, RestBuff, InnerLen} - end. - - - -%%============================================================================ -%% encode Universal string -%%============================================================================ - -encode_universal_string(C, {Name, Universal}, DoTag) when is_atom(Name) -> - encode_universal_string(C, Universal, DoTag); -encode_universal_string(_C, Universal, []) -> - OctetList = mk_uni_list(Universal), - dotag_universal(?N_UniversalString,OctetList,length(OctetList)); -encode_universal_string(_C, Universal, DoTag) -> - OctetList = mk_uni_list(Universal), - dotag(DoTag, ?N_UniversalString, {OctetList,length(OctetList)}). - -mk_uni_list(In) -> - mk_uni_list(In,[]). - -mk_uni_list([],List) -> - lists:reverse(List); -mk_uni_list([{A,B,C,D}|T],List) -> - mk_uni_list(T,[D,C,B,A|List]); -mk_uni_list([H|T],List) -> - mk_uni_list(T,[H,0,0,0|List]). - -%%=========================================================================== -%% decode Universal strings -%% (Buffer, Range, StringType, HasTag, LenIn) -> -%% {String, Remain, RemovedBytes} -%%=========================================================================== - -decode_universal_string(Buffer, Range, Tags, LenIn, OptOrMand) -> -% NewTags = new_tags(HasTag, #tag{class=?UNIVERSAL,number=?N_UniversalString}), - decode_restricted_string(Buffer, Range, ?N_UniversalString, - Tags, LenIn, [], OptOrMand,old). - - -mk_universal_string(In) -> - mk_universal_string(In,[]). - -mk_universal_string([],Acc) -> - lists:reverse(Acc); -mk_universal_string([0,0,0,D|T],Acc) -> - mk_universal_string(T,[D|Acc]); -mk_universal_string([A,B,C,D|T],Acc) -> - mk_universal_string(T,[{A,B,C,D}|Acc]). - - -%%============================================================================ -%% encode UTF8 string -%%============================================================================ -encode_UTF8_string(_,UTF8String,[]) when is_binary(UTF8String) -> - dotag_universal(?N_UTF8String,UTF8String,byte_size(UTF8String)); -encode_UTF8_string(_,UTF8String,DoTag) when is_binary(UTF8String) -> - dotag(DoTag,?N_UTF8String,{UTF8String,byte_size(UTF8String)}); -encode_UTF8_string(_,UTF8String,[]) -> - dotag_universal(?N_UTF8String,UTF8String,length(UTF8String)); -encode_UTF8_string(_,UTF8String,DoTag) -> - dotag(DoTag,?N_UTF8String,{UTF8String,length(UTF8String)}). - -%%============================================================================ -%% decode UTF8 string -%%============================================================================ - -decode_UTF8_string(Buffer, Tags, OptOrMand) -> - NewTags = new_tags(Tags, #tag{class=?UNIVERSAL,number=?N_UTF8String}), - decode_UTF8_string_notag(Buffer, NewTags, OptOrMand). - -decode_UTF8_string_notag(Buffer, Tags, OptOrMand) -> - {RestTags, {FormLen, Buffer0, Rb0}} = - check_tags_i(Tags, Buffer, OptOrMand), - case FormLen of - {?CONSTRUCTED,Len} -> - %% an UTF8String may be encoded as a constructed type - {Buffer00,RestBytes} = split_list(Buffer0,Len), - {Val01, Buffer01, Rb01} = - decode_UTF8_string_notag(Buffer00,RestTags,OptOrMand), - {Buffer02, Rb02} = restbytes2(RestBytes,Buffer01,noext), - {Val01, Buffer02, Rb0+Rb01+Rb02}; - {_,Len} -> - <<Result:Len/binary,RestBuff/binary>> = Buffer0, - {Result,RestBuff,Rb0 + Len} - end. - - -%%============================================================================ -%% encode BMP string -%%============================================================================ - -encode_BMP_string(C, {Name,BMPString}, DoTag) when is_atom(Name) -> - encode_BMP_string(C, BMPString, DoTag); -encode_BMP_string(_C, BMPString, []) -> - OctetList = mk_BMP_list(BMPString), - dotag_universal(?N_BMPString,OctetList,length(OctetList)); -encode_BMP_string(_C, BMPString, DoTag) -> - OctetList = mk_BMP_list(BMPString), - dotag(DoTag, ?N_BMPString, {OctetList,length(OctetList)}). - -mk_BMP_list(In) -> - mk_BMP_list(In,[]). - -mk_BMP_list([],List) -> - lists:reverse(List); -mk_BMP_list([{0,0,C,D}|T],List) -> - mk_BMP_list(T,[D,C|List]); -mk_BMP_list([H|T],List) -> - mk_BMP_list(T,[H,0|List]). - -%%============================================================================ -%% decode (OctetList, Range(ignored), tag|notag) -> {ValList, RestList} -%% (Buffer, Range, StringType, HasTag, TotalLen) -> -%% {String, Remain, RemovedBytes} -%%============================================================================ -decode_BMP_string(Buffer, Range, Tags, LenIn, OptOrMand) -> -% NewTags = new_tags(HasTag, #tag{class=?UNIVERSAL,number=?N_BMPString}), - decode_restricted_string(Buffer, Range, ?N_BMPString, - Tags, LenIn, [], OptOrMand,old). - -mk_BMP_string(In) -> - mk_BMP_string(In,[]). - -mk_BMP_string([],US) -> - lists:reverse(US); -mk_BMP_string([0,B|T],US) -> - mk_BMP_string(T,[B|US]); -mk_BMP_string([C,D|T],US) -> - mk_BMP_string(T,[{0,0,C,D}|US]). - - -%%============================================================================ -%% Generalized time, ITU_T X.680 Chapter 39 -%% -%% encode Generalized time -%%============================================================================ - -encode_generalized_time(C, {Name,OctetList}, DoTag) when is_atom(Name) -> - encode_generalized_time(C, OctetList, DoTag); -encode_generalized_time(_C, OctetList, []) -> - dotag_universal(?N_GeneralizedTime,OctetList,length(OctetList)); -encode_generalized_time(_C, OctetList, DoTag) -> - dotag(DoTag, ?N_GeneralizedTime, {OctetList,length(OctetList)}). - -%%============================================================================ -%% decode Generalized time -%% (Buffer, Range, HasTag, TotalLen) -> {String, Remain, RemovedBytes} -%%============================================================================ - -decode_generalized_time(Buffer, Range, Tags, TotalLen, OptOrMand) -> - NewTags = new_tags(Tags,#tag{class=?UNIVERSAL, - number=?N_GeneralizedTime}), - decode_generalized_time_notag(Buffer, Range, NewTags, TotalLen, OptOrMand). - -decode_generalized_time_notag(Buffer, Range, Tags, TotalLen, OptOrMand) -> - {RestTags, {FormLen, Buffer0, Rb0}} = - check_tags_i(Tags, Buffer, OptOrMand), - - case FormLen of - {?CONSTRUCTED,Len} -> - {Buffer00,RestBytes} = split_list(Buffer0,Len), - {Val01, Buffer01, Rb01} = - decode_generalized_time_notag(Buffer00, Range, - RestTags, TotalLen, - OptOrMand), - {Buffer02, Rb02} = restbytes2(RestBytes,Buffer01,noext), - {Val01, Buffer02, Rb0+Rb01+Rb02}; - {_,Len} -> - <<PreBuff:Len/binary,RestBuff/binary>> = Buffer0, - {binary_to_list(PreBuff), RestBuff, Rb0+Len} - end. - -%%============================================================================ -%% Universal time, ITU_T X.680 Chapter 40 -%% -%% encode UTC time -%%============================================================================ - -encode_utc_time(C, {Name,OctetList}, DoTag) when is_atom(Name) -> - encode_utc_time(C, OctetList, DoTag); -encode_utc_time(_C, OctetList, []) -> - dotag_universal(?N_UTCTime, OctetList,length(OctetList)); -encode_utc_time(_C, OctetList, DoTag) -> - dotag(DoTag, ?N_UTCTime, {OctetList,length(OctetList)}). - -%%============================================================================ -%% decode UTC time -%% (Buffer, Range, HasTag, TotalLen) -> {String, Remain, RemovedBytes} -%%============================================================================ - -decode_utc_time(Buffer, Range, Tags, TotalLen, OptOrMand) -> - NewTags = new_tags(Tags,#tag{class=?UNIVERSAL,number=?N_UTCTime}), - decode_utc_time_notag(Buffer, Range, NewTags, TotalLen, OptOrMand). - -decode_utc_time_notag(Buffer, Range, Tags, TotalLen, OptOrMand) -> - {RestTags, {FormLen, Buffer0, Rb0}} = - check_tags_i(Tags, Buffer, OptOrMand), - - case FormLen of - {?CONSTRUCTED,Len} -> - {Buffer00,RestBytes} = split_list(Buffer0,Len), - {Val01, Buffer01, Rb01} = - decode_utc_time_notag(Buffer00, Range, - RestTags, TotalLen, - OptOrMand), - {Buffer02, Rb02} = restbytes2(RestBytes,Buffer01,noext), - {Val01, Buffer02, Rb0+Rb01+Rb02}; - {_,Len} -> - <<PreBuff:Len/binary,RestBuff/binary>> = Buffer0, - {binary_to_list(PreBuff), RestBuff, Rb0+Len} - end. - - %%============================================================================ %% Length handling %% @@ -2122,8 +449,6 @@ decode_utc_time_notag(Buffer, Range, Tags, TotalLen, OptOrMand) -> %% [<127]| [128 + Int (<127),OctetList] | [16#80] %%============================================================================ -encode_length(indefinite) -> - {[16#80],1}; % 128 encode_length(L) when L =< 16#7F -> {[L],1}; encode_length(L) -> @@ -2162,242 +487,12 @@ decode_length(<<1:1,LL:7,T/binary>>) -> <<Length:LL/unit:8,Rest/binary>> = T, {{Length,Rest}, LL+1}. -%decode_length([128 | T]) -> -% {{indefinite, T},1}; -%decode_length([H | T]) when H =< 127 -> -% {{H, T},1}; -%decode_length([H | T]) -> -% dec_long_length(H band 16#7F, T, 0, 1). - - -%%dec_long_length(0, Buffer, Acc, Len) -> -%% {{Acc, Buffer},Len}; -%%dec_long_length(Bytes, [H | T], Acc, Len) -> -%% dec_long_length(Bytes - 1, T, (Acc bsl 8) + H, Len+1). - -%%=========================================================================== -%% Decode tag and length -%% -%% decode_tag_and_length(Buffer) -> {Tag, Len, RemainingBuffer, RemovedBytes} -%% -%%=========================================================================== - -decode_tag_and_length(Buffer) -> - {Tag, Buffer2, RemBytesTag} = decode_tag(Buffer), - {{Len, Buffer3}, RemBytesLen} = decode_length(Buffer2), - {Tag, Len, Buffer3, RemBytesTag+RemBytesLen}. - - -%%============================================================================ -%% Check if valid tag -%% -%% check_if_valid_tag(Tag, List_of_valid_tags, OptOrMand) -> name of the tag -%%============================================================================ - -check_if_valid_tag(<<0,0,_/binary>>,_,_) -> - asn1_EOC; -check_if_valid_tag(<<>>, _, OptOrMand) -> - check_if_valid_tag2_error([], OptOrMand); -check_if_valid_tag(Bytes, ListOfTags, OptOrMand) when is_binary(Bytes) -> - {Tag, _, _} = decode_tag(Bytes), - check_if_valid_tag(Tag, ListOfTags, OptOrMand); - -%% This alternative should be removed in the near future -%% Bytes as input should be the only necessary call -check_if_valid_tag(Tag, ListOfTags, OptOrMand) -> - {Class, _Form, TagNo} = Tag, - C = code_class(Class), - T = case C of - 'UNIVERSAL' -> - code_type(TagNo); - _ -> - TagNo - end, - check_if_valid_tag2({C,T}, ListOfTags, Tag, OptOrMand). - -check_if_valid_tag2(_Class_TagNo, [], Tag, MandOrOpt) -> - check_if_valid_tag2_error(Tag,MandOrOpt); -check_if_valid_tag2(Class_TagNo, [{TagName,TagList}|T], Tag, OptOrMand) -> - case check_if_valid_tag_loop(Class_TagNo, TagList) of - true -> - TagName; - false -> - check_if_valid_tag2(Class_TagNo, T, Tag, OptOrMand) - end. - --spec check_if_valid_tag2_error(term(), atom()) -> no_return(). - -check_if_valid_tag2_error(Tag,mandatory) -> - exit({error,{asn1,{invalid_tag,Tag}}}); -check_if_valid_tag2_error(Tag,_) -> - exit({error,{asn1,{no_optional_tag,Tag}}}). - -check_if_valid_tag_loop(_Class_TagNo,[]) -> - false; -check_if_valid_tag_loop(Class_TagNo,[H|T]) -> - %% It is not possible to distinguish between SEQUENCE OF and SEQUENCE, and - %% between SET OF and SET because both are coded as 16 and 17, respectively. - H_without_OF = case H of - {C, 'SEQUENCE OF'} -> - {C, 'SEQUENCE'}; - {C, 'SET OF'} -> - {C, 'SET'}; - Else -> - Else - end, - - case H_without_OF of - Class_TagNo -> - true; - {_,_} -> - check_if_valid_tag_loop(Class_TagNo,T); - _ -> - check_if_valid_tag_loop(Class_TagNo,H), - check_if_valid_tag_loop(Class_TagNo,T) - end. - - - -code_class(0) -> 'UNIVERSAL'; -code_class(16#40) -> 'APPLICATION'; -code_class(16#80) -> 'CONTEXT'; -code_class(16#C0) -> 'PRIVATE'. - - -code_type(1) -> 'BOOLEAN'; -code_type(2) -> 'INTEGER'; -code_type(3) -> 'BIT STRING'; -code_type(4) -> 'OCTET STRING'; -code_type(5) -> 'NULL'; -code_type(6) -> 'OBJECT IDENTIFIER'; -code_type(7) -> 'ObjectDescriptor'; -code_type(8) -> 'EXTERNAL'; -code_type(9) -> 'REAL'; -code_type(10) -> 'ENUMERATED'; -code_type(11) -> 'EMBEDDED_PDV'; -code_type(16) -> 'SEQUENCE'; -% code_type(16) -> 'SEQUENCE OF'; -code_type(17) -> 'SET'; -% code_type(17) -> 'SET OF'; -code_type(18) -> 'NumericString'; -code_type(19) -> 'PrintableString'; -code_type(20) -> 'TeletexString'; -code_type(21) -> 'VideotexString'; -code_type(22) -> 'IA5String'; -code_type(23) -> 'UTCTime'; -code_type(24) -> 'GeneralizedTime'; -code_type(25) -> 'GraphicString'; -code_type(26) -> 'VisibleString'; -code_type(27) -> 'GeneralString'; -code_type(28) -> 'UniversalString'; -code_type(30) -> 'BMPString'; -code_type(Else) -> exit({error,{asn1,{unrecognized_type,Else}}}). - -%%------------------------------------------------------------------------- -%% decoding of the components of a SET -%%------------------------------------------------------------------------- - -decode_set(Rb, indefinite, <<0,0,Bytes/binary>>, _OptOrMand, _Fun3, Acc) -> - {lists:reverse(Acc),Bytes,Rb+2}; - -decode_set(Rb, indefinite, Bytes, OptOrMand, Fun3, Acc) -> - case Fun3(Bytes, OptOrMand) of - {_Term, _Remain, 0} -> - {lists:reverse(Acc),Bytes,Rb}; - {Term, Remain, Rb1} -> - Fun3(Bytes, OptOrMand), - decode_set(Rb+Rb1, indefinite, Remain, OptOrMand, Fun3, [Term|Acc]) - end; -%% {Term, Remain, Rb1} = Fun3(Bytes, OptOrMand), -%% decode_set(Rb+Rb1, indefinite, Remain, OptOrMand, Fun3, [Term|Acc]); - -decode_set(Rb, Num, Bytes, _OptOrMand, _Fun3, Acc) when Num == 0 -> - {lists:reverse(Acc), Bytes, Rb}; - -decode_set(_, Num, _, _, _, _) when Num < 0 -> - exit({error,{asn1,{length_error,'SET'}}}); - -decode_set(Rb, Num, Bytes, OptOrMand, Fun3, Acc) -> - case Fun3(Bytes, OptOrMand) of - {_Term, _Remain, 0} -> - {lists:reverse(Acc),Bytes,Rb}; - {Term, Remain, Rb1} -> - Fun3(Bytes, OptOrMand), - decode_set(Rb+Rb1, Num-Rb1, Remain, OptOrMand, Fun3, [Term|Acc]) - end. -%% {Term, Remain, Rb1} = Fun3(Bytes, OptOrMand), -%% decode_set(Rb+Rb1, Num-Rb1, Remain, OptOrMand, Fun3, [Term|Acc]). - - -%%------------------------------------------------------------------------- -%% decoding of SEQUENCE OF and SET OF -%%------------------------------------------------------------------------- - -decode_components(Rb, indefinite, <<0,0,Bytes/binary>>, _Fun3, _TagIn, Acc) -> - {lists:reverse(Acc),Bytes,Rb+2}; - -decode_components(Rb, indefinite, Bytes, Fun3, TagIn, Acc) -> - {Term, Remain, Rb1} = Fun3(Bytes, mandatory, TagIn), - decode_components(Rb+Rb1, indefinite, Remain, Fun3, TagIn, [Term|Acc]); - -decode_components(Rb, Num, Bytes, _Fun3, _TagIn, Acc) when Num == 0 -> - {lists:reverse(Acc), Bytes, Rb}; - -decode_components(_, Num, _, _, _, _) when Num < 0 -> - exit({error,{asn1,{length_error,'SET/SEQUENCE OF'}}}); - -decode_components(Rb, Num, Bytes, Fun3, TagIn, Acc) -> - {Term, Remain, Rb1} = Fun3(Bytes, mandatory, TagIn), - decode_components(Rb+Rb1, Num-Rb1, Remain, Fun3, TagIn, [Term|Acc]). - -%%decode_components(Rb, indefinite, [0,0|Bytes], _Fun3, _TagIn, Acc) -> -%% {lists:reverse(Acc),Bytes,Rb+2}; - -decode_components(Rb, indefinite, <<0,0,Bytes/binary>>, _Fun4, _TagIn, _Fun, Acc) -> - {lists:reverse(Acc),Bytes,Rb+2}; - -decode_components(Rb, indefinite, Bytes, _Fun4, TagIn, _Fun, Acc) -> - {Term, Remain, Rb1} = _Fun4(Bytes, mandatory, TagIn, _Fun), - decode_components(Rb+Rb1, indefinite, Remain, _Fun4, TagIn, _Fun, [Term|Acc]); - -decode_components(Rb, Num, Bytes, _Fun4, _TagIn, _Fun, Acc) when Num == 0 -> - {lists:reverse(Acc), Bytes, Rb}; - -decode_components(_, Num, _, _, _, _, _) when Num < 0 -> - exit({error,{asn1,{length_error,'SET/SEQUENCE OF'}}}); - -decode_components(Rb, Num, Bytes, _Fun4, TagIn, _Fun, Acc) -> - {Term, Remain, Rb1} = _Fun4(Bytes, mandatory, TagIn, _Fun), - decode_components(Rb+Rb1, Num-Rb1, Remain, _Fun4, TagIn, _Fun, [Term|Acc]). - - - -%%------------------------------------------------------------------------- -%% INTERNAL HELPER FUNCTIONS (not exported) -%%------------------------------------------------------------------------- - - -%%========================================================================== -%% Encode tag -%% -%% dotag(tag | notag, TagValpattern | TagValTuple, [Length, Value]) -> [Tag] -%% TagValPattern is a correct bitpattern for a tag -%% TagValTuple is a tuple of three bitpatterns, Class, Form and TagNo where -%% Class = UNIVERSAL | APPLICATION | CONTEXT | PRIVATE -%% Form = Primitive | Constructed -%% TagNo = Number of tag -%%========================================================================== - dotag([], Tag, {Bytes,Len}) -> dotag_universal(Tag,Bytes,Len); dotag(Tags, Tag, {Bytes,Len}) -> encode_tags(Tags ++ [#tag{class=?UNIVERSAL,number=Tag,form=?PRIMITIVE}], - Bytes, Len); - -dotag(Tags, Tag, Bytes) -> - encode_tags(Tags ++ [#tag{class=?UNIVERSAL,number=Tag,form=?PRIMITIVE}], - Bytes, size(Bytes)). + Bytes, Len). dotag_universal(UniversalTag,Bytes,Len) when Len =< 16#7F-> {[UniversalTag,Len,Bytes],2+Len}; @@ -2415,47 +510,6 @@ decode_integer2(Len,<<1:1,B2:7,Bs/binary>>,RemovedBytes) -> Int = N - (1 bsl (8 * Len - 1)), {Int,Buffer2,RemovedBytes}. -%%decode_integer2(Len,Buffer,Acc,RemovedBytes) when (hd(Buffer) band 16#FF) =< 16#7F -> -%% {decode_integer_pos(Buffer, 8 * (Len - 1)),skip(Buffer,Len),RemovedBytes}; -%%decode_integer2(Len,Buffer,Acc,RemovedBytes) -> -%% {decode_integer_neg(Buffer, 8 * (Len - 1)),skip(Buffer,Len),RemovedBytes}. - -%%decode_integer_pos([Byte|Tail], Shift) -> -%% (Byte bsl Shift) bor decode_integer_pos(Tail, Shift-8); -%%decode_integer_pos([], _) -> 0. - - -%%decode_integer_neg([Byte|Tail], Shift) -> -%% (-128 + (Byte band 127) bsl Shift) bor decode_integer_pos(Tail, Shift-8). - - -concat_bit_binaries([],Bin={_,_}) -> - Bin; -concat_bit_binaries({0,B1},{U2,B2}) -> - {U2,<<B1/binary,B2/binary>>}; -concat_bit_binaries({U1,B1},{U2,B2}) -> - S1 = (size(B1) * 8) - U1, - S2 = (size(B2) * 8) - U2, - PadBits = 8 - ((S1+S2) rem 8), - {PadBits, <<B1:S1/binary-unit:1,B2:S2/binary-unit:1,0:PadBits>>}; -concat_bit_binaries(L1,L2) when is_list(L1), is_list(L2) -> - %% this case occur when decoding with NNL - L1 ++ L2. - - -get_constraint(C,Key) -> - case lists:keyfind(Key,1,C) of - false -> - no; - {_, V} -> - V - end. - -%%skip(Buffer, 0) -> -%% Buffer; -%%skip([H | T], Len) -> -%% skip(T, Len-1). - new_tags([],LastTag) -> [LastTag]; new_tags(Tags = [#tag{type='IMPLICIT'}],_LastTag) -> diff --git a/lib/asn1/src/asn1rt_ber_bin_v2.erl b/lib/asn1/src/asn1rt_ber_bin_v2.erl index 9ff5017c68..92ca11cf89 100644 --- a/lib/asn1/src/asn1rt_ber_bin_v2.erl +++ b/lib/asn1/src/asn1rt_ber_bin_v2.erl @@ -22,8 +22,7 @@ %% encoding / decoding of BER -export([decode/1, decode/2, match_tags/2, encode/1, encode/2]). --export([fixoptionals/2, cindex/3, - list_to_record/2, +-export([fixoptionals/2, encode_tag_val/1, encode_tags/3, skip_ExtensionAdditions/2]). @@ -54,8 +53,6 @@ decode_open_type_as_binary/3]). -export([decode_primitive_incomplete/2,decode_selective/2]). - --export([is_nif_loadable/0]). % the encoding of class of tag bits 8 and 7 -define(UNIVERSAL, 0). @@ -135,12 +132,7 @@ encode(Tlv,_) when is_binary(Tlv) -> encode([Tlv],Method) -> encode(Tlv,Method); encode(Tlv, nif) -> - case is_nif_loadable() of - true -> - asn1rt_nif:encode_ber_tlv(Tlv); - false -> - encode_erl(Tlv) - end; + asn1rt_nif:encode_ber_tlv(Tlv); encode(Tlv, _) -> encode_erl(Tlv). @@ -178,36 +170,15 @@ decode(B) -> %% asn1-1.7 decode(B, nif) -> - case is_nif_loadable() of - true -> - case asn1rt_nif:decode_ber_tlv(B) of - {error, Reason} -> handle_error(Reason, B); - Else -> Else - end; - false -> - decode(B) + case asn1rt_nif:decode_ber_tlv(B) of + {error, Reason} -> handle_error(Reason, B); + Else -> Else end; decode(B,erlang) when is_binary(B) -> decode_primitive(B); decode(Tlv,erlang) -> {Tlv,<<>>}. -%% Have to check this since asn1 is not guaranteed to be available -is_nif_loadable() -> - case application:get_env(asn1, nif_loadable) of - {ok,R} -> - R; - undefined -> - case catch code:load_file(asn1rt_nif) of - {module, asn1rt_nif} -> - application:set_env(asn1, nif_loadable, true), - true; - _Else -> - application:set_env(asn1, nif_loadable, false), - false - end - end. - handle_error([],_)-> exit({error,{asn1,{"memory allocation problem"}}}); handle_error({$1,_},L) -> % error in nif @@ -612,13 +583,6 @@ match_tags(Tlv, []) -> Tlv; match_tags(Tlv = {Tag,_V},[T|_Tt]) -> exit({error,{asn1,{wrong_tag,{{expected,T},{got,Tag,Tlv}}}}}). - - -cindex(Ix,Val,Cname) -> - case element(Ix,Val) of - {Cname,Val2} -> Val2; - X -> X - end. %%% %% skips components that do not match a tag in Tags @@ -642,13 +606,6 @@ skip_ExtensionAdditions(TLV=[{Tag,_}|Rest],Tags) -> %%=============================================================================== %%=============================================================================== -% converts a list to a record if necessary -list_to_record(Name,List) when is_list(List) -> - list_to_tuple([Name|List]); -list_to_record(_Name,Tuple) when is_tuple(Tuple) -> - Tuple. - - fixoptionals(OptList,Val) when is_list(Val) -> fixoptionals(OptList,Val,1,[],[]). diff --git a/lib/asn1/src/asn1rt_per_bin.erl b/lib/asn1/src/asn1rt_per_bin.erl deleted file mode 100644 index 5772f09bf4..0000000000 --- a/lib/asn1/src/asn1rt_per_bin.erl +++ /dev/null @@ -1,2285 +0,0 @@ -%% -%% %CopyrightBegin% -%% -%% Copyright Ericsson AB 2001-2012. All Rights Reserved. -%% -%% The contents of this file are subject to the Erlang Public License, -%% Version 1.1, (the "License"); you may not use this file except in -%% compliance with the License. You should have received a copy of the -%% Erlang Public License along with this software. If not, it can be -%% retrieved online at http://www.erlang.org/. -%% -%% Software distributed under the License is distributed on an "AS IS" -%% basis, WITHOUT WARRANTY OF ANY KIND, either express or implied. See -%% the License for the specific language governing rights and limitations -%% under the License. -%% -%% %CopyrightEnd% -%% -%% --module(asn1rt_per_bin). -%% encoding / decoding of PER aligned - --include("asn1_records.hrl"). - --export([dec_fixup/3, cindex/3, list_to_record/2]). --export([setchoiceext/1, setext/1, fixoptionals/2, fixoptionals/3, - fixextensions/2, - getext/1, getextension/2, skipextensions/3, getbit/1, getchoice/3 ]). --export([getoptionals/2, getoptionals2/2, set_choice/3, encode_integer/2, encode_integer/3 ]). --export([decode_integer/2, decode_integer/3, encode_small_number/1, encode_boolean/1, - decode_boolean/1, encode_length/2, decode_length/1, decode_length/2, - encode_small_length/1, decode_small_length/1, - decode_compact_bit_string/3]). --export([decode_enumerated/3, - encode_bit_string/3, decode_bit_string/3 ]). --export([encode_octet_string/2, decode_octet_string/2, - encode_null/1, decode_null/1, - encode_object_identifier/1, decode_object_identifier/1, - encode_real/1, decode_real/1, - encode_relative_oid/1, decode_relative_oid/1, - complete/1]). - - --export([encode_open_type/2, decode_open_type/2]). - --export([encode_UniversalString/2, decode_UniversalString/2, - encode_PrintableString/2, decode_PrintableString/2, - encode_GeneralString/2, decode_GeneralString/2, - encode_GraphicString/2, decode_GraphicString/2, - encode_TeletexString/2, decode_TeletexString/2, - encode_VideotexString/2, decode_VideotexString/2, - encode_VisibleString/2, decode_VisibleString/2, - encode_UTF8String/1, decode_UTF8String/1, - encode_BMPString/2, decode_BMPString/2, - encode_IA5String/2, decode_IA5String/2, - encode_NumericString/2, decode_NumericString/2, - encode_ObjectDescriptor/2, decode_ObjectDescriptor/1 - ]). --export([complete_bytes/1, getbits/2, getoctets/2, minimum_bits/1]). - --define('16K',16384). --define('32K',32768). --define('64K',65536). - -dec_fixup(Terms,Cnames,RemBytes) -> - dec_fixup(Terms,Cnames,RemBytes,[]). - -dec_fixup([novalue|T],[_Hc|Tc],RemBytes,Acc) -> - dec_fixup(T,Tc,RemBytes,Acc); -dec_fixup([{_Name,novalue}|T],[_Hc|Tc],RemBytes,Acc) -> - dec_fixup(T,Tc,RemBytes,Acc); -dec_fixup([H|T],[Hc|Tc],RemBytes,Acc) -> - dec_fixup(T,Tc,RemBytes,[{Hc,H}|Acc]); -dec_fixup([],_Cnames,RemBytes,Acc) -> - {lists:reverse(Acc),RemBytes}. - -cindex(Ix,Val,Cname) -> - case element(Ix,Val) of - {Cname,Val2} -> Val2; - X -> X - end. - -%% converts a list to a record if necessary -list_to_record(_Name,Tuple) when is_tuple(Tuple) -> - Tuple; -list_to_record(Name,List) when is_list(List) -> - list_to_tuple([Name|List]). - -%%-------------------------------------------------------- -%% setchoiceext(InRootSet) -> [{bit,X}] -%% X is set to 1 when InRootSet==false -%% X is set to 0 when InRootSet==true -%% -setchoiceext(true) -> - [{debug,choiceext},{bits,1,0}]; -setchoiceext(false) -> - [{debug,choiceext},{bits,1,1}]. - -%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%% -%% setext(true|false) -> CompleteList -%% - -setext(false) -> - [{debug,ext},{bits,1,0}]; -setext(true) -> - [{debug,ext},{bits,1,1}]. - - -%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%% -%% This version of fixoptionals/2 are left only because of -%% backward compatibility with older generates - -fixoptionals(OptList,Val) when is_tuple(Val) -> - fixoptionals1(OptList,Val,[]); - -fixoptionals(OptList,Val) when is_list(Val) -> - fixoptionals1(OptList,Val,1,[],[]). - -fixoptionals1([],Val,Acc) -> - %% return {Val,Opt} - {Val,lists:reverse(Acc)}; -fixoptionals1([{_,Pos}|Ot],Val,Acc) -> - case element(Pos+1,Val) of - asn1_NOVALUE -> fixoptionals1(Ot,Val,[0|Acc]); - asn1_DEFAULT -> fixoptionals1(Ot,Val,[0|Acc]); - _ -> fixoptionals1(Ot,Val,[1|Acc]) - end. - - -fixoptionals1([{Name,Pos}|Ot],[{Name,Val}|Vt],_Opt,Acc1,Acc2) -> - fixoptionals1(Ot,Vt,Pos+1,[1|Acc1],[{Name,Val}|Acc2]); -fixoptionals1([{_Name,Pos}|Ot],V,Pos,Acc1,Acc2) -> - fixoptionals1(Ot,V,Pos+1,[0|Acc1],[asn1_NOVALUE|Acc2]); -fixoptionals1(O,[Vh|Vt],Pos,Acc1,Acc2) -> - fixoptionals1(O,Vt,Pos+1,Acc1,[Vh|Acc2]); -fixoptionals1([],[Vh|Vt],Pos,Acc1,Acc2) -> - fixoptionals1([],Vt,Pos+1,Acc1,[Vh|Acc2]); -fixoptionals1([],[],_,Acc1,Acc2) -> - % return {Val,Opt} - {list_to_tuple([asn1_RECORDNAME|lists:reverse(Acc2)]),lists:reverse(Acc1)}. - -%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%% -%% This is the new fixoptionals/3 which is used by the new generates -%% -fixoptionals(OptList,OptLength,Val) when is_tuple(Val) -> - Bits = fixoptionals(OptList,Val,0), - {Val,{bits,OptLength,Bits}}; - -fixoptionals([],_Val,Acc) -> - %% Optbits - Acc; -fixoptionals([{Pos,DefVal}|Ot],Val,Acc) -> - case element(Pos,Val) of - asn1_DEFAULT -> fixoptionals(Ot,Val,Acc bsl 1); - DefVal -> fixoptionals(Ot,Val,Acc bsl 1); - _ -> fixoptionals(Ot,Val,(Acc bsl 1) + 1) - end; -fixoptionals([Pos|Ot],Val,Acc) -> - case element(Pos,Val) of - asn1_NOVALUE -> fixoptionals(Ot,Val,Acc bsl 1); - asn1_DEFAULT -> fixoptionals(Ot,Val,Acc bsl 1); - _ -> fixoptionals(Ot,Val,(Acc bsl 1) + 1) - end. - - -getext(Bytes) when is_tuple(Bytes) -> - getbit(Bytes); -getext(Bytes) when is_binary(Bytes) -> - getbit({0,Bytes}). - -getextension(0, Bytes) -> - {{},Bytes}; -getextension(1, Bytes) -> - {Len,Bytes2} = decode_small_length(Bytes), - {Blist, Bytes3} = getbits_as_list(Len,Bytes2), - {list_to_tuple(Blist),Bytes3}. - -fixextensions({ext,ExtPos,ExtNum},Val) -> - case fixextensions(ExtPos,ExtNum+ExtPos,Val,0) of - 0 -> []; - ExtBits -> - [encode_small_length(ExtNum),{bits,ExtNum,ExtBits}] - end. - -fixextensions(Pos,MaxPos,_,Acc) when Pos >= MaxPos -> - Acc; -fixextensions(Pos,ExtPos,Val,Acc) -> - Bit = case catch(element(Pos+1,Val)) of - asn1_NOVALUE -> - 0; - asn1_NOEXTVALUE -> - 0; - {'EXIT',_} -> - 0; - _ -> - 1 - end, - fixextensions(Pos+1,ExtPos,Val,(Acc bsl 1)+Bit). - -skipextensions(Bytes,Nr,ExtensionBitPattern) -> - case (catch element(Nr,ExtensionBitPattern)) of - 1 -> - {_,Bytes2} = decode_open_type(Bytes,[]), - skipextensions(Bytes2, Nr+1, ExtensionBitPattern); - 0 -> - skipextensions(Bytes, Nr+1, ExtensionBitPattern); - {'EXIT',_} -> % badarg, no more extensions - Bytes - end. - - -getchoice(Bytes,1,0) -> % only 1 alternative is not encoded - {0,Bytes}; -getchoice(Bytes,_,1) -> - decode_small_number(Bytes); -getchoice(Bytes,NumChoices,0) -> - decode_constrained_number(Bytes,{0,NumChoices-1}). - -%% old version kept for backward compatibility with generates from R7B -getoptionals(Bytes,NumOpt) -> - {Blist,Bytes1} = getbits_as_list(NumOpt,Bytes), - {list_to_tuple(Blist),Bytes1}. - -%% new version used in generates from r8b_patch/3 and later -getoptionals2(Bytes,NumOpt) -> - getbits(Bytes,NumOpt). - - -%% getbits_as_binary(Num,Bytes) -> {{Unused,BinBits},RestBytes}, -%% Num = integer(), -%% Bytes = list() | tuple(), -%% Unused = integer(), -%% BinBits = binary(), -%% RestBytes = tuple() -getbits_as_binary(Num,Bytes) when is_binary(Bytes) -> - getbits_as_binary(Num,{0,Bytes}); -getbits_as_binary(0,Buffer) -> - {{0,<<>>},Buffer}; -getbits_as_binary(Num,{0,Bin}) when Num > 16 -> - Used = Num rem 8, - Pad = (8 - Used) rem 8, -% Nbytes = Num div 8, - <<Bits:Num,_:Pad,RestBin/binary>> = Bin, - {{Pad,<<Bits:Num,0:Pad>>},RestBin}; -getbits_as_binary(Num,Buffer={_Used,_Bin}) -> % Unaligned buffer - %% Num =< 16, - {Bits2,Buffer2} = getbits(Buffer,Num), - Pad = (8 - (Num rem 8)) rem 8, - {{Pad,<<Bits2:Num,0:Pad>>},Buffer2}. - - -% integer_from_list(Int,[],BigInt) -> -% BigInt; -% integer_from_list(Int,[H|T],BigInt) when Int < 8 -> -% (BigInt bsl Int) bor (H bsr (8-Int)); -% integer_from_list(Int,[H|T],BigInt) -> -% integer_from_list(Int-8,T,(BigInt bsl 8) bor H). - -getbits_as_list(Num,Bytes) when is_binary(Bytes) -> - getbits_as_list(Num,{0,Bytes},[]); -getbits_as_list(Num,Bytes) -> - getbits_as_list(Num,Bytes,[]). - -%% If buffer is empty and nothing more will be picked. -getbits_as_list(0, B, Acc) -> - {lists:reverse(Acc),B}; -%% If first byte in buffer is full and at least one byte will be picked, -%% then pick one byte. -getbits_as_list(N,{0,Bin},Acc) when N >= 8 -> - <<B7:1,B6:1,B5:1,B4:1,B3:1,B2:1,B1:1,B0:1,Rest/binary>> = Bin, - getbits_as_list(N-8,{0,Rest},[B0,B1,B2,B3,B4,B5,B6,B7|Acc]); -getbits_as_list(N,{Used,Bin},Acc) when N >= 4, Used =< 4 -> - NewUsed = Used + 4, - Rem = 8 - NewUsed, - <<_:Used,B3:1,B2:1,B1:1,B0:1,_:Rem, Rest/binary>> = Bin, - NewRest = case Rem of 0 -> Rest; _ -> Bin end, - getbits_as_list(N-4,{NewUsed rem 8,NewRest},[B0,B1,B2,B3|Acc]); -getbits_as_list(N,{Used,Bin},Acc) when N >= 2, Used =< 6 -> - NewUsed = Used + 2, - Rem = 8 - NewUsed, - <<_:Used,B1:1,B0:1,_:Rem, Rest/binary>> = Bin, - NewRest = case Rem of 0 -> Rest; _ -> Bin end, - getbits_as_list(N-2,{NewUsed rem 8,NewRest},[B0,B1|Acc]); -getbits_as_list(N,{Used,Bin},Acc) when Used =< 7 -> - NewUsed = Used + 1, - Rem = 8 - NewUsed, - <<_:Used,B0:1,_:Rem, Rest/binary>> = Bin, - NewRest = case Rem of 0 -> Rest; _ -> Bin end, - getbits_as_list(N-1,{NewUsed rem 8,NewRest},[B0|Acc]). - - -getbit({7,<<_:7,B:1,Rest/binary>>}) -> - {B,{0,Rest}}; -getbit({0,Buffer = <<B:1,_:7,_/binary>>}) -> - {B,{1,Buffer}}; -getbit({Used,Buffer}) -> - Unused = (8 - Used) - 1, - <<_:Used,B:1,_:Unused,_/binary>> = Buffer, - {B,{Used+1,Buffer}}; -getbit(Buffer) when is_binary(Buffer) -> - getbit({0,Buffer}). - - -getbits({0,Buffer},Num) when (Num rem 8) == 0 -> - <<Bits:Num,Rest/binary>> = Buffer, - {Bits,{0,Rest}}; -getbits({Used,Bin},Num) -> - NumPlusUsed = Num + Used, - NewUsed = NumPlusUsed rem 8, - Unused = (8-NewUsed) rem 8, - case Unused of - 0 -> - <<_:Used,Bits:Num,Rest/binary>> = Bin, - {Bits,{0,Rest}}; - _ -> - Bytes = NumPlusUsed div 8, - <<_:Used,Bits:Num,_UBits:Unused,_/binary>> = Bin, - <<_:Bytes/binary,Rest/binary>> = Bin, - {Bits,{NewUsed,Rest}} - end; -getbits(Bin,Num) when is_binary(Bin) -> - getbits({0,Bin},Num). - - - -% getoctet(Bytes) when is_list(Bytes) -> -% getoctet({0,Bytes}); -% getoctet(Bytes) -> -% %% io:format("getoctet:Buffer = ~p~n",[Bytes]), -% getoctet1(Bytes). - -% getoctet1({0,[H|T]}) -> -% {H,{0,T}}; -% getoctet1({Pos,[_,H|T]}) -> -% {H,{0,T}}. - -align({0,L}) -> - {0,L}; -align({_Pos,<<_H,T/binary>>}) -> - {0,T}; -align(Bytes) -> - {0,Bytes}. - -%% First align buffer, then pick the first Num octets. -%% Returns octets as an integer with bit significance as in buffer. -getoctets({0,Buffer},Num) -> - <<Val:Num/integer-unit:8,RestBin/binary>> = Buffer, - {Val,{0,RestBin}}; -getoctets({U,<<_Padding,Rest/binary>>},Num) when U /= 0 -> - getoctets({0,Rest},Num); -getoctets(Buffer,Num) when is_binary(Buffer) -> - getoctets({0,Buffer},Num). -% getoctets(Buffer,Num) -> -% %% io:format("getoctets:Buffer = ~p~nNum = ~p~n",[Buffer,Num]), -% getoctets(Buffer,Num,0). - -% getoctets(Buffer,0,Acc) -> -% {Acc,Buffer}; -% getoctets(Buffer,Num,Acc) -> -% {Oct,NewBuffer} = getoctet(Buffer), -% getoctets(NewBuffer,Num-1,(Acc bsl 8)+Oct). - -% getoctets_as_list(Buffer,Num) -> -% getoctets_as_list(Buffer,Num,[]). - -% getoctets_as_list(Buffer,0,Acc) -> -% {lists:reverse(Acc),Buffer}; -% getoctets_as_list(Buffer,Num,Acc) -> -% {Oct,NewBuffer} = getoctet(Buffer), -% getoctets_as_list(NewBuffer,Num-1,[Oct|Acc]). - -%% First align buffer, then pick the first Num octets. -%% Returns octets as a binary -getoctets_as_bin({0,Bin},Num)-> - <<Octets:Num/binary,RestBin/binary>> = Bin, - {Octets,{0,RestBin}}; -getoctets_as_bin({_U,Bin},Num) -> - <<_Padding,Octets:Num/binary,RestBin/binary>> = Bin, - {Octets,{0,RestBin}}; -getoctets_as_bin(Bin,Num) when is_binary(Bin) -> - getoctets_as_bin({0,Bin},Num). - -%% same as above but returns octets as a List -getoctets_as_list(Buffer,Num) -> - {Bin,Buffer2} = getoctets_as_bin(Buffer,Num), - {binary_to_list(Bin),Buffer2}. -%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%% -%% set_choice(Alt,Choices,Altnum) -> ListofBitSettings -%% Alt = atom() -%% Altnum = integer() | {integer(),integer()}% number of alternatives -%% Choices = [atom()] | {[atom()],[atom()]} -%% When Choices is a tuple the first list is the Rootset and the -%% second is the Extensions and then Altnum must also be a tuple with the -%% lengths of the 2 lists -%% -set_choice(Alt,{L1,L2},{Len1,_Len2}) -> - case set_choice_tag(Alt,L1) of - N when is_integer(N), Len1 > 1 -> - [{bits,1,0}, % the value is in the root set - encode_integer([{'ValueRange',{0,Len1-1}}],N)]; - N when is_integer(N) -> - [{bits,1,0}]; % no encoding if only 0 or 1 alternative - false -> - [{bits,1,1}, % extension value - case set_choice_tag(Alt,L2) of - N2 when is_integer(N2) -> - encode_small_number(N2); - false -> - unknown_choice_alt - end] - end; -set_choice(Alt,L,Len) -> - case set_choice_tag(Alt,L) of - N when is_integer(N), Len > 1 -> - encode_integer([{'ValueRange',{0,Len-1}}],N); - N when is_integer(N) -> - []; % no encoding if only 0 or 1 alternative - false -> - [unknown_choice_alt] - end. - -set_choice_tag(Alt,Choices) -> - set_choice_tag(Alt,Choices,0). - -set_choice_tag(Alt,[Alt|_Rest],Tag) -> - Tag; -set_choice_tag(Alt,[_H|Rest],Tag) -> - set_choice_tag(Alt,Rest,Tag+1); -set_choice_tag(_Alt,[],_Tag) -> - false. - -%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%% -%% decode_fragmented_XXX; decode of values encoded fragmented according -%% to ITU-T X.691 clause 10.9.3.8. The unit (XXX) is either bits, octets, -%% characters or number of components (in a choice,sequence or similar). -%% Buffer is a buffer {Used, Bin}. -%% C is the constrained length. -%% If the buffer is not aligned, this function does that. -decode_fragmented_bits({0,Buffer},C) -> - decode_fragmented_bits(Buffer,C,[]); -decode_fragmented_bits({_N,<<_,Bs/binary>>},C) -> - decode_fragmented_bits(Bs,C,[]). - -decode_fragmented_bits(<<3:2,Len:6,Bin/binary>>,C,Acc) -> - {Value,Bin2} = split_binary(Bin, Len * ?'16K'), - decode_fragmented_bits(Bin2,C,[Value,Acc]); -decode_fragmented_bits(<<0:1,0:7,Bin/binary>>,C,Acc) -> - BinBits = list_to_binary(lists:reverse(Acc)), - case C of - Int when is_integer(Int),C == size(BinBits) -> - {BinBits,{0,Bin}}; - Int when is_integer(Int) -> - exit({error,{asn1,{illegal_value,C,BinBits}}}) - end; -decode_fragmented_bits(<<0:1,Len:7,Bin/binary>>,C,Acc) -> - Result = {BinBits,{Used,_Rest}} = - case (Len rem 8) of - 0 -> - <<Value:Len/binary-unit:1,Bin2/binary>> = Bin, - {list_to_binary(lists:reverse([Value|Acc])),{0,Bin2}}; - Rem -> - Bytes = Len div 8, - U = 8 - Rem, - <<Value:Bytes/binary-unit:8,Bits1:Rem,Bits2:U,Bin2/binary>> = Bin, - {list_to_binary(lists:reverse([Bits1 bsl U,Value|Acc])), - {Rem,<<Bits2,Bin2/binary>>}} - end, - case C of - Int when is_integer(Int),C == (size(BinBits) - ((8 - Used) rem 8)) -> - Result; - Int when is_integer(Int) -> - exit({error,{asn1,{illegal_value,C,BinBits}}}) - end. - - -decode_fragmented_octets({0,Bin},C) -> - decode_fragmented_octets(Bin,C,[]). - -decode_fragmented_octets(<<3:2,Len:6,Bin/binary>>,C,Acc) -> - {Value,Bin2} = split_binary(Bin,Len * ?'16K'), - decode_fragmented_octets(Bin2,C,[Value,Acc]); -decode_fragmented_octets(<<0:1,0:7,Bin/binary>>,C,Acc) -> - Octets = list_to_binary(lists:reverse(Acc)), - case C of - Int when is_integer(Int), C == size(Octets) -> - {Octets,{0,Bin}}; - Int when is_integer(Int) -> - exit({error,{asn1,{illegal_value,C,Octets}}}) - end; -decode_fragmented_octets(<<0:1,Len:7,Bin/binary>>,C,Acc) -> - <<Value:Len/binary-unit:8,Bin2/binary>> = Bin, - BinOctets = list_to_binary(lists:reverse([Value|Acc])), - case C of - Int when is_integer(Int),size(BinOctets) == Int -> - {BinOctets,Bin2}; - Int when is_integer(Int) -> - exit({error,{asn1,{illegal_value,C,BinOctets}}}) - end. - - - -%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%% -%% encode_open_type(Constraint, Value) -> CompleteList -%% Value = list of bytes of an already encoded value (the list must be flat) -%% | binary -%% Contraint = not used in this version -%% -encode_open_type(_C, Val) when is_list(Val) -> - Bin = list_to_binary(Val), - [encode_length(undefined,size(Bin)),{octets,Bin}]; % octets implies align -encode_open_type(_C, Val) when is_binary(Val) -> - [encode_length(undefined,size(Val)),{octets,Val}]. % octets implies align -%% the binary_to_list is not optimal but compatible with the current solution - -%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%% -%% decode_open_type(Buffer,Constraint) -> Value -%% Constraint is not used in this version -%% Buffer = [byte] with PER encoded data -%% Value = [byte] with decoded data (which must be decoded again as some type) -%% -decode_open_type(Bytes, _C) -> - {Len,Bytes2} = decode_length(Bytes,undefined), - getoctets_as_bin(Bytes2,Len). - -%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%% -%% encode_integer(Constraint,Value,NamedNumberList) -> CompleteList -%% encode_integer(Constraint,Value) -> CompleteList -%% encode_integer(Constraint,{Name,Value}) -> CompleteList -%% -%% -encode_integer(C,V,NamedNumberList) when is_atom(V) -> - case lists:keysearch(V,1,NamedNumberList) of - {value,{_,NewV}} -> - encode_integer(C,NewV); - _ -> - exit({error,{asn1,{namednumber,V}}}) - end; -encode_integer(C,V,_NamedNumberList) when is_integer(V) -> - encode_integer(C,V); -encode_integer(C,{Name,V},NamedNumberList) when is_atom(Name) -> - encode_integer(C,V,NamedNumberList). - -encode_integer(C,{Name,Val}) when is_atom(Name) -> - encode_integer(C,Val); - -encode_integer([{Rc,_Ec}],Val) when is_tuple(Rc) -> % XXX when is this invoked? First argument most often a list,...Ok this is the extension case...but it doesn't work. - case (catch encode_integer([Rc],Val)) of - {'EXIT',{error,{asn1,_}}} -> - [{bits,1,1},encode_unconstrained_number(Val)]; - Encoded -> - [{bits,1,0},Encoded] - end; -encode_integer(C,Val ) when is_list(C) -> - case get_constraint(C,'SingleValue') of - no -> - encode_integer1(C,Val); - V when is_integer(V),V == Val -> - []; % a type restricted to a single value encodes to nothing - V when is_list(V) -> - case lists:member(Val,V) of - true -> - encode_integer1(C,Val); - _ -> - exit({error,{asn1,{illegal_value,Val}}}) - end; - _ -> - exit({error,{asn1,{illegal_value,Val}}}) - end. - -encode_integer1(C, Val) -> - case VR = get_constraint(C,'ValueRange') of - no -> - encode_unconstrained_number(Val); - {Lb,'MAX'} -> - encode_semi_constrained_number(Lb,Val); - %% positive with range - {Lb,Ub} when Val >= Lb, - Ub >= Val -> - encode_constrained_number(VR,Val); - _ -> - exit({error,{asn1,{illegal_value,VR,Val}}}) - end. - -decode_integer(Buffer,Range,NamedNumberList) -> - {Val,Buffer2} = decode_integer(Buffer,Range), - case lists:keysearch(Val,2,NamedNumberList) of - {value,{NewVal,_}} -> {NewVal,Buffer2}; - _ -> {Val,Buffer2} - end. - -decode_integer(Buffer,[{Rc,_Ec}]) when is_tuple(Rc) -> - {Ext,Buffer2} = getext(Buffer), - case Ext of - 0 -> decode_integer(Buffer2,[Rc]); - 1 -> decode_unconstrained_number(Buffer2) - end; -decode_integer(Buffer,undefined) -> - decode_unconstrained_number(Buffer); -decode_integer(Buffer,C) -> - case get_constraint(C,'SingleValue') of - V when is_integer(V) -> - {V,Buffer}; - V when is_list(V) -> - {Val,Buffer2} = decode_integer1(Buffer,C), - case lists:member(Val,V) of - true -> - {Val,Buffer2}; - _ -> - exit({error,{asn1,{illegal_value,Val}}}) - end; - _ -> - decode_integer1(Buffer,C) - end. - -decode_integer1(Buffer,C) -> - case VR = get_constraint(C,'ValueRange') of - no -> - decode_unconstrained_number(Buffer); - {Lb, 'MAX'} -> - decode_semi_constrained_number(Buffer,Lb); - {_,_} -> - decode_constrained_number(Buffer,VR) - end. - - % X.691:10.6 Encoding of a normally small non-negative whole number - % Use this for encoding of CHOICE index if there is an extension marker in - % the CHOICE -encode_small_number({Name,Val}) when is_atom(Name) -> - encode_small_number(Val); -encode_small_number(Val) when Val =< 63 -> -% [{bits,1,0},{bits,6,Val}]; - [{bits,7,Val}]; % same as above but more efficient -encode_small_number(Val) -> - [{bits,1,1},encode_semi_constrained_number(0,Val)]. - -decode_small_number(Bytes) -> - {Bit,Bytes2} = getbit(Bytes), - case Bit of - 0 -> - getbits(Bytes2,6); - 1 -> - decode_semi_constrained_number(Bytes2,0) - end. - -%% X.691:10.7 Encoding of a semi-constrained whole number -%% might be an optimization encode_semi_constrained_number(0,Val) -> -encode_semi_constrained_number(C,{Name,Val}) when is_atom(Name) -> - encode_semi_constrained_number(C,Val); -encode_semi_constrained_number({Lb,'MAX'},Val) -> - encode_semi_constrained_number(Lb,Val); -encode_semi_constrained_number(Lb,Val) -> - Val2 = Val - Lb, - Oct = eint_positive(Val2), - Len = length(Oct), - if - Len < 128 -> - {octets,[Len|Oct]}; % equiv with encode_length(undefined,Len) but faster - true -> - [encode_length(undefined,Len),{octets,Oct}] - end. - -decode_semi_constrained_number(Bytes,{Lb,_}) -> - decode_semi_constrained_number(Bytes,Lb); -decode_semi_constrained_number(Bytes,Lb) -> - {Len,Bytes2} = decode_length(Bytes,undefined), - {V,Bytes3} = getoctets(Bytes2,Len), - {V+Lb,Bytes3}. - -encode_constrained_number(Range,{Name,Val}) when is_atom(Name) -> - encode_constrained_number(Range,Val); -encode_constrained_number({Lb,Ub},Val) when Val >= Lb, Ub >= Val -> - Range = Ub - Lb + 1, - Val2 = Val - Lb, - if - Range == 1 -> - []; - Range == 2 -> - {bits,1,Val2}; - Range =< 4 -> - {bits,2,Val2}; - Range =< 8 -> - {bits,3,Val2}; - Range =< 16 -> - {bits,4,Val2}; - Range =< 32 -> - {bits,5,Val2}; - Range =< 64 -> - {bits,6,Val2}; - Range =< 128 -> - {bits,7,Val2}; - Range =< 255 -> - {bits,8,Val2}; - Range =< 256 -> - {octets,[Val2]}; - Range =< 65536 -> - {octets,<<Val2:16>>}; - Range =< (1 bsl (255*8)) -> - Octs = binary:encode_unsigned(Val2), - RangeOcts = binary:encode_unsigned(Range - 1), - OctsLen = erlang:byte_size(Octs), - RangeOctsLen = erlang:byte_size(RangeOcts), - LengthBitsNeeded = minimum_bits(RangeOctsLen - 1), - [{bits, LengthBitsNeeded, OctsLen - 1}, {octets, Octs}]; - true -> - exit({not_supported,{integer_range,Range}}) - end; -encode_constrained_number(Range,Val) -> - exit({error,{asn1,{integer_range,Range,value,Val}}}). - -%% For some reason the minimum bits needed in the length field in encoding of -%% constrained whole numbers must always be atleast 2? -minimum_bits(N) when N < 4 -> 2; -minimum_bits(N) when N < 8 -> 3; -minimum_bits(N) when N < 16 -> 4; -minimum_bits(N) when N < 32 -> 5; -minimum_bits(N) when N < 64 -> 6; -minimum_bits(N) when N < 128 -> 7; -minimum_bits(_N) -> 8. - -decode_constrained_number(Buffer,{Lb,Ub}) -> - Range = Ub - Lb + 1, - % Val2 = Val - Lb, - {Val,Remain} = - if - Range == 1 -> - {0,Buffer}; - Range == 2 -> - getbits(Buffer,1); - Range =< 4 -> - getbits(Buffer,2); - Range =< 8 -> - getbits(Buffer,3); - Range =< 16 -> - getbits(Buffer,4); - Range =< 32 -> - getbits(Buffer,5); - Range =< 64 -> - getbits(Buffer,6); - Range =< 128 -> - getbits(Buffer,7); - Range =< 255 -> - getbits(Buffer,8); - Range =< 256 -> - getoctets(Buffer,1); - Range =< 65536 -> - getoctets(Buffer,2); - Range =< (1 bsl (255*8)) -> - OList = binary:bin_to_list(binary:encode_unsigned(Range - 1)), - RangeOctLen = length(OList), - {Len, Bytes} = decode_length(Buffer, {1, RangeOctLen}), - {Octs, RestBytes} = getoctets_as_list(Bytes, Len), - {binary:decode_unsigned(binary:list_to_bin(Octs)), RestBytes}; - true -> - exit({not_supported,{integer_range,Range}}) - end, - {Val+Lb,Remain}. - -%% X.691:10.8 Encoding of an unconstrained whole number - -encode_unconstrained_number(Val) when Val >= 0 -> - Oct = eint(Val,[]), - Len = length(Oct), - if - Len < 128 -> - {octets,[Len|Oct]}; % equiv with encode_length(undefined,Len) but faster - true -> - [encode_length(undefined,Len),{octets,Oct}] - end; -encode_unconstrained_number(Val) -> % negative - Oct = enint(Val,[]), - Len = length(Oct), - if - Len < 128 -> - {octets,[Len|Oct]}; % equiv with encode_length(undefined,Len) but faster - true -> - [encode_length(undefined,Len),{octets,Oct}] - end. - - -%% used for positive Values which don't need a sign bit -%% returns a binary -eint_positive(Val) -> - case eint(Val,[]) of - [0,B1|T] -> - [B1|T]; - T -> - T - end. - - -eint(0, [B|Acc]) when B < 128 -> - [B|Acc]; -eint(N, Acc) -> - eint(N bsr 8, [N band 16#ff| Acc]). - -enint(-1, [B1|T]) when B1 > 127 -> - [B1|T]; -enint(N, Acc) -> - enint(N bsr 8, [N band 16#ff|Acc]). - -decode_unconstrained_number(Bytes) -> - {Len,Bytes2} = decode_length(Bytes,undefined), - {Ints,Bytes3} = getoctets_as_list(Bytes2,Len), - {dec_integer(Ints),Bytes3}. - -dec_integer(Ints) when hd(Ints) band 255 =< 127 -> %% Positive number - decpint(Ints, 8 * (length(Ints) - 1)); -dec_integer(Ints) -> %% Negative - decnint(Ints, 8 * (length(Ints) - 1)). - -decpint([Byte|Tail], Shift) -> - (Byte bsl Shift) bor decpint(Tail, Shift-8); -decpint([], _) -> 0. - -decnint([Byte|Tail], Shift) -> - (-128 + (Byte band 127) bsl Shift) bor decpint(Tail, Shift-8). - -% minimum_octets(Val) -> -% minimum_octets(Val,[]). - -% minimum_octets(Val,Acc) when Val > 0 -> -% minimum_octets((Val bsr 8),[Val band 16#FF|Acc]); -% minimum_octets(0,Acc) -> -% Acc. - - -%% X.691:10.9 Encoding of a length determinant -%%encode_small_length(undefined,Len) -> % null means no UpperBound -%% encode_small_number(Len). - -%% X.691:10.9.3.5 -%% X.691:10.9.3.7 -encode_length(undefined,Len) -> % un-constrained - if - Len < 128 -> - {octets,[Len]}; - Len < 16384 -> - {octets,<<2:2,Len:14>>}; - true -> % should be able to endode length >= 16384 - exit({error,{asn1,{encode_length,{nyi,above_16k}}}}) - end; - -encode_length({0,'MAX'},Len) -> - encode_length(undefined,Len); -encode_length(Vr={Lb,Ub},Len) when Ub =< 65535 ,Lb >= 0 -> % constrained - encode_constrained_number(Vr,Len); -encode_length({Lb,_Ub},Len) when is_integer(Lb), Lb >= 0 -> % Ub > 65535 - encode_length(undefined,Len); -encode_length({Vr={Lb,Ub},Ext},Len) - when Ub =< 65535 ,Lb >= 0, Len=<Ub, is_list(Ext) -> - %% constrained extensible - [{bits,1,0},encode_constrained_number(Vr,Len)]; -encode_length({{Lb,_Ub},Ext},Len) when is_list(Ext) -> - [{bits,1,1},encode_semi_constrained_number(Lb,Len)]; -encode_length(SingleValue,_Len) when is_integer(SingleValue) -> - []. - -%% X.691 10.9.3.4 (only used for length of bitmap that prefixes extension -%% additions in a sequence or set -encode_small_length(Len) when Len =< 64 -> -%% [{bits,1,0},{bits,6,Len-1}]; - {bits,7,Len-1}; % the same as above but more efficient -encode_small_length(Len) -> - [{bits,1,1},encode_length(undefined,Len)]. - -% decode_small_length({Used,<<_:Used,0:1,Num:6,_:((8-Used+1) rem 8),Rest/binary>>}) -> -% case Buffer of -% <<_:Used,0:1,Num:6,_:((8-Used+1) rem 8),Rest/binary>> -> -% {Num, -% case getbit(Buffer) of -% {0,Remain} -> -% {Bits,Remain2} = getbits(Remain,6), -% {Bits+1,Remain2}; -% {1,Remain} -> -% decode_length(Remain,undefined) -% end. - -decode_small_length(Buffer) -> - case getbit(Buffer) of - {0,Remain} -> - {Bits,Remain2} = getbits(Remain,6), - {Bits+1,Remain2}; - {1,Remain} -> - decode_length(Remain,undefined) - end. - -decode_length(Buffer) -> - decode_length(Buffer,undefined). - -decode_length(Buffer,undefined) -> % un-constrained - {0,Buffer2} = align(Buffer), - case Buffer2 of - <<0:1,Oct:7,Rest/binary>> -> - {Oct,{0,Rest}}; - <<2:2,Val:14,Rest/binary>> -> - {Val,{0,Rest}}; - <<3:2,_:14,_Rest/binary>> -> - %% this case should be fixed - exit({error,{asn1,{decode_length,{nyi,above_16k}}}}) - end; -%% {Bits,_} = getbits(Buffer2,2), -% case Bits of -% 2 -> -% {Val,Bytes3} = getoctets(Buffer2,2), -% {(Val band 16#3FFF),Bytes3}; -% 3 -> -% exit({error,{asn1,{decode_length,{nyi,above_16k}}}}); -% _ -> -% {Val,Bytes3} = getoctet(Buffer2), -% {Val band 16#7F,Bytes3} -% end; - -decode_length(Buffer,{Lb,Ub}) when Ub =< 65535 ,Lb >= 0 -> % constrained - decode_constrained_number(Buffer,{Lb,Ub}); -decode_length(Buffer,{Lb,_}) when is_integer(Lb), Lb >= 0 -> % Ub > 65535 - decode_length(Buffer,undefined); -decode_length(Buffer,{VR={_Lb,_Ub},Ext}) when is_list(Ext) -> - case getbit(Buffer) of - {0,Buffer2} -> - decode_length(Buffer2, VR); - {1,Buffer2} -> - decode_length(Buffer2, undefined) - end; -%% {0,Buffer2} = getbit(Buffer), -%% decode_length(Buffer2, VR); - - -%When does this case occur with {_,_Lb,Ub} ?? -% X.691:10.9.3.5 -decode_length({Used,Bin},{_,_Lb,_Ub}) -> %when Len =< 127 -> % Unconstrained or large Ub NOTE! this case does not cover case when Ub > 65535 - Unused = (8-Used) rem 8, - case Bin of - <<_:Used,0:1,Val:7,R:Unused,Rest/binary>> -> - {Val,{Used,<<R,Rest/binary>>}}; - <<_:Used,_:Unused,2:2,Val:14,Rest/binary>> -> - {Val, {0,Rest}}; - <<_:Used,_:Unused,3:2,_:14,_Rest/binary>> -> - exit({error,{asn1,{decode_length,{nyi,length_above_64K}}}}) - end; -% decode_length(Buffer,{_,_Lb,Ub}) -> %when Len =< 127 -> % Unconstrained or large Ub -% case getbit(Buffer) of -% {0,Remain} -> -% getbits(Remain,7); -% {1,Remain} -> -% {Val,Remain2} = getoctets(Buffer,2), -% {Val band 2#0111111111111111, Remain2} -% end; -decode_length(Buffer,SingleValue) when is_integer(SingleValue) -> - {SingleValue,Buffer}. - - - % X.691:11 -encode_boolean(true) -> - {bits,1,1}; -encode_boolean(false) -> - {bits,1,0}; -encode_boolean({Name,Val}) when is_atom(Name) -> - encode_boolean(Val); -encode_boolean(Val) -> - exit({error,{asn1,{encode_boolean,Val}}}). - -decode_boolean(Buffer) -> %when record(Buffer,buffer) - case getbit(Buffer) of - {1,Remain} -> {true,Remain}; - {0,Remain} -> {false,Remain} - end. - - -%% ENUMERATED with extension marker -decode_enumerated(Buffer,C,{Ntup1,Ntup2}) when is_tuple(Ntup1), is_tuple(Ntup2) -> - {Ext,Buffer2} = getext(Buffer), - case Ext of - 0 -> % not an extension value - {Val,Buffer3} = decode_integer(Buffer2,C), - case catch (element(Val+1,Ntup1)) of - NewVal when is_atom(NewVal) -> {NewVal,Buffer3}; - _Error -> exit({error,{asn1,{decode_enumerated,{Val,[Ntup1,Ntup2]}}}}) - end; - 1 -> % this an extension value - {Val,Buffer3} = decode_small_number(Buffer2), - case catch (element(Val+1,Ntup2)) of - NewVal when is_atom(NewVal) -> {NewVal,Buffer3}; - _ -> {{asn1_enum,Val},Buffer3} - end - end; - -decode_enumerated(Buffer,C,NamedNumberTup) when is_tuple(NamedNumberTup) -> - {Val,Buffer2} = decode_integer(Buffer,C), - case catch (element(Val+1,NamedNumberTup)) of - NewVal when is_atom(NewVal) -> {NewVal,Buffer2}; - _Error -> exit({error,{asn1,{decode_enumerated,{Val,NamedNumberTup}}}}) - end. - -%%=============================================================================== -%%=============================================================================== -%%=============================================================================== -%% Bitstring value, ITU_T X.690 Chapter 8.5 -%%=============================================================================== -%%=============================================================================== -%%=============================================================================== - -%%=============================================================================== -%% encode bitstring value -%%=============================================================================== - - - -%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%% -%% bitstring NamedBitList -%% Val can be of: -%% - [identifiers] where only named identifers are set to one, -%% the Constraint must then have some information of the -%% bitlength. -%% - [list of ones and zeroes] all bits -%% - integer value representing the bitlist -%% C is constraint Len, only valid when identifiers - - -%% when the value is a list of {Unused,BinBits}, where -%% Unused = integer(), -%% BinBits = binary(). - -encode_bit_string(C,Bin={Unused,BinBits},NamedBitList) when is_integer(Unused), - is_binary(BinBits) -> - encode_bin_bit_string(C,Bin,NamedBitList); - -%% when the value is a list of named bits -encode_bit_string(C, LoNB=[FirstVal | _RestVal], NamedBitList) when is_atom(FirstVal) -> - ToSetPos = get_all_bitposes(LoNB, NamedBitList, []), - BitList = make_and_set_list(ToSetPos,0), - encode_bit_string(C,BitList,NamedBitList); - -encode_bit_string(C, BL=[{bit,_No} | _RestVal], NamedBitList) -> - ToSetPos = get_all_bitposes(BL, NamedBitList, []), - BitList = make_and_set_list(ToSetPos,0), - encode_bit_string(C,BitList,NamedBitList); - -%% when the value is a list of ones and zeroes - -% encode_bit_string(C, BitListValue, NamedBitList) when is_list(BitListValue) -> -% Bl1 = -% case NamedBitList of -% [] -> % dont remove trailing zeroes -% BitListValue; -% _ -> % first remove any trailing zeroes -% lists:reverse(lists:dropwhile(fun(0)->true;(1)->false end, -% lists:reverse(BitListValue))) -% end, -% BitList = [{bit,X} || X <- Bl1], -% %% BListLen = length(BitList), -% case get_constraint(C,'SizeConstraint') of -% 0 -> % fixed length -% []; % nothing to encode -% V when is_integer(V),V=<16 -> % fixed length 16 bits or less -% pad_list(V,BitList); -% V when is_integer(V) -> % fixed length 16 bits or more -% [align,pad_list(V,BitList)]; % should be another case for V >= 65537 -% {Lb,Ub} when is_integer(Lb),is_integer(Ub) -> -% [encode_length({Lb,Ub},length(BitList)),align,BitList]; -% no -> -% [encode_length(undefined,length(BitList)),align,BitList]; -% Sc -> % extension marker -% [encode_length(Sc,length(BitList)),align,BitList] -% end; -encode_bit_string(C, BitListValue, NamedBitList) when is_list(BitListValue) -> - BitListToBinary = - %% fun that transforms a list of 1 and 0 to a tuple: - %% {UnusedBitsInLastByte, Binary} - fun([1|T],Acc,N,Fun) -> - Fun(T,(Acc bsl 1)+1,N+1,Fun); - ([0|T],Acc,N,Fun) -> - Fun(T,(Acc bsl 1),N+1,Fun); - ([_H|_T],_,_,_) -> - exit({error,{asn1,{bitstring_bitlist,BitListValue}}}); - ([],Acc,N,_) -> - Unused = (8 - (N rem 8)) rem 8, - {Unused,<<Acc:N,0:Unused>>} - end, - UnusedAndBin = - case NamedBitList of - [] -> % dont remove trailing zeroes - BitListToBinary(BitListValue,0,0,BitListToBinary); - _ -> - BitListToBinary(lists:reverse( - lists:dropwhile(fun(0)->true;(_)->false end, - lists:reverse(BitListValue))), - 0,0,BitListToBinary) - end, - encode_bin_bit_string(C,UnusedAndBin,NamedBitList); - -%% when the value is an integer -encode_bit_string(C, IntegerVal, NamedBitList) when is_integer(IntegerVal)-> - BitList = int_to_bitlist(IntegerVal), - encode_bit_string(C,BitList,NamedBitList); - -%% when the value is a tuple -encode_bit_string(C,{Name,Val}, NamedBitList) when is_atom(Name) -> - encode_bit_string(C,Val,NamedBitList). - - -%% encode_bin_bit_string/3, when value is a tuple of Unused and BinBits. -%% Unused = integer(),i.e. number unused bits in least sign. byte of -%% BinBits = binary(). - - -encode_bin_bit_string(C,UnusedAndBin={_Unused,_BinBits},NamedBitList) -> - Constr = get_constraint(C,'SizeConstraint'), - UnusedAndBin1 = {Unused1,Bin1} = - remove_trailing_bin(NamedBitList,UnusedAndBin,lower_bound(Constr)), - case Constr of - 0 -> - []; - V when is_integer(V),V=<16 -> - {Unused2,Bin2} = pad_list(V,UnusedAndBin1), - <<BitVal:V,_:Unused2>> = Bin2, - {bits,V,BitVal}; - V when is_integer(V) -> - [align, pad_list(V, UnusedAndBin1)]; - {Lb,Ub} when is_integer(Lb),is_integer(Ub) -> - [encode_length({Lb,Ub},size(Bin1)*8 - Unused1), - align,UnusedAndBin1]; - {{Fix,Fix},L} when is_integer(Fix),is_list(L) -> - %% X.691 � 15.6, the rest of this paragraph is covered by - %% the last, ie. Sc, clause in this case - case (size(Bin1)*8)-Unused1 of - Size when Size =< Fix, Fix =< 16 -> - {Unused2,Bin2} = pad_list(Fix,UnusedAndBin), - <<BitVal:Fix,_:Unused2>> = Bin2, - [{bits,1,0},{bits,Fix,BitVal}]; - Size when Size =< Fix -> - [{bits,1,0},align, pad_list(Fix, UnusedAndBin1)]; - Size -> - [{bits,1,1},encode_length(undefined,Size), - align,UnusedAndBin1] - end; - no -> - [encode_length(undefined,size(Bin1)*8 - Unused1), - align,UnusedAndBin1]; - Sc -> - [encode_length(Sc,size(Bin1)*8 - Unused1), - align,UnusedAndBin1] - end. - - -remove_trailing_bin([], {Unused,Bin},_) -> - {Unused,Bin}; -remove_trailing_bin(_NamedNumberList,{_Unused,<<>>},C) -> - case C of - Int when is_integer(Int),Int > 0 -> - %% this padding see OTP-4353 - pad_list(Int,{0,<<>>}); - _ -> {0,<<>>} - end; -remove_trailing_bin(NamedNumberList, {_Unused,Bin},C) -> - Size = size(Bin)-1, - <<Bfront:Size/binary, LastByte:8>> = Bin, - %% clear the Unused bits to be sure - Unused1 = trailingZeroesInNibble(LastByte band 15), - Unused2 = - case Unused1 of - 4 -> - 4 + trailingZeroesInNibble(LastByte bsr 4); - _ -> Unused1 - end, - case Unused2 of - 8 -> - remove_trailing_bin(NamedNumberList,{0,Bfront},C); - _ -> - case C of - Int when is_integer(Int),Int > ((size(Bin)*8)-Unused2) -> - %% this padding see OTP-4353 - pad_list(Int,{Unused2,Bin}); - _ -> {Unused2,Bin} - end - end. - - -trailingZeroesInNibble(0) -> - 4; -trailingZeroesInNibble(1) -> - 0; -trailingZeroesInNibble(2) -> - 1; -trailingZeroesInNibble(3) -> - 0; -trailingZeroesInNibble(4) -> - 2; -trailingZeroesInNibble(5) -> - 0; -trailingZeroesInNibble(6) -> - 1; -trailingZeroesInNibble(7) -> - 0; -trailingZeroesInNibble(8) -> - 3; -trailingZeroesInNibble(9) -> - 0; -trailingZeroesInNibble(10) -> - 1; -trailingZeroesInNibble(11) -> - 0; -trailingZeroesInNibble(12) -> %#1100 - 2; -trailingZeroesInNibble(13) -> - 0; -trailingZeroesInNibble(14) -> - 1; -trailingZeroesInNibble(15) -> - 0. - -lower_bound({{Lb,_},_}) when is_integer(Lb) -> - Lb; -lower_bound({Lb,_}) when is_integer(Lb) -> - Lb; -lower_bound(C) -> - C. - -%%%%%%%%%%%%%%% -%% The result is presented as a list of named bits (if possible) -%% else as a tuple {Unused,Bits}. Unused is the number of unused -%% bits, least significant bits in the last byte of Bits. Bits is -%% the BIT STRING represented as a binary. -%% -decode_compact_bit_string(Buffer, C, NamedNumberList) -> - case get_constraint(C,'SizeConstraint') of - 0 -> % fixed length - {{8,0},Buffer}; - V when is_integer(V),V=<16 -> %fixed length 16 bits or less - compact_bit_string(Buffer,V,NamedNumberList); - V when is_integer(V),V=<65536 -> %fixed length > 16 bits - Bytes2 = align(Buffer), - compact_bit_string(Bytes2,V,NamedNumberList); - V when is_integer(V) -> % V > 65536 => fragmented value - {Bin,Buffer2} = decode_fragmented_bits(Buffer,V), - case Buffer2 of - {0,_} -> {{0,Bin},Buffer2}; - {U,_} -> {{8-U,Bin},Buffer2} - end; - {Lb,Ub} when is_integer(Lb),is_integer(Ub) -> - %% This case may demand decoding of fragmented length/value - {Len,Bytes2} = decode_length(Buffer,{Lb,Ub}), - Bytes3 = align(Bytes2), - compact_bit_string(Bytes3,Len,NamedNumberList); - no -> - %% This case may demand decoding of fragmented length/value - {Len,Bytes2} = decode_length(Buffer,undefined), - Bytes3 = align(Bytes2), - compact_bit_string(Bytes3,Len,NamedNumberList); - {{Fix,Fix},L} = Sc when is_list(L), is_integer(Fix), Fix =< 16 -> - %% X.691 �15.6, special case of extension marker - case decode_length(Buffer,Sc) of - {Len,Bytes2} when Len > Fix -> - Bytes3 = align(Bytes2), - compact_bit_string(Bytes3,Len,NamedNumberList); - {Len,Bytes2} -> - compact_bit_string(Bytes2,Len,NamedNumberList) - end; - Sc -> - {Len,Bytes2} = decode_length(Buffer,Sc), - Bytes3 = align(Bytes2), - compact_bit_string(Bytes3,Len,NamedNumberList) - end. - - -%%%%%%%%%%%%%%% -%% The result is presented as a list of named bits (if possible) -%% else as a list of 0 and 1. -%% -decode_bit_string(Buffer, C, NamedNumberList) -> - case get_constraint(C,'SizeConstraint') of - {Lb,Ub} when is_integer(Lb),is_integer(Ub) -> - {Len,Bytes2} = decode_length(Buffer,{Lb,Ub}), - Bytes3 = align(Bytes2), - bit_list_or_named(Bytes3,Len,NamedNumberList); - no -> - {Len,Bytes2} = decode_length(Buffer,undefined), - Bytes3 = align(Bytes2), - bit_list_or_named(Bytes3,Len,NamedNumberList); - 0 -> % fixed length - {[],Buffer}; % nothing to encode - V when is_integer(V),V=<16 -> % fixed length 16 bits or less - bit_list_or_named(Buffer,V,NamedNumberList); - V when is_integer(V),V=<65536 -> - Bytes2 = align(Buffer), - bit_list_or_named(Bytes2,V,NamedNumberList); - V when is_integer(V) -> - Bytes2 = align(Buffer), - {BinBits,_} = decode_fragmented_bits(Bytes2,V), - bit_list_or_named(BinBits,V,NamedNumberList); - {{Fix,Fix},L} = Sc when is_list(L), is_integer(Fix), Fix =< 16 -> - %% X.691 �15.6, special case of extension marker - case decode_length(Buffer,Sc) of - {Len,Bytes2} when Len > Fix -> - Bytes3 = align(Bytes2), - bit_list_or_named(Bytes3,Len,NamedNumberList); - {Len,Bytes2} when Len > 16 -> - Bytes3 = align(Bytes2), - bit_list_or_named(Bytes3,Len,NamedNumberList); - {Len,Bytes2} -> - bit_list_or_named(Bytes2,Len,NamedNumberList) - end; - Sc -> %% X.691 �15.6, extension marker - {Len,Bytes2} = decode_length(Buffer,Sc), - Bytes3 = align(Bytes2), - bit_list_or_named(Bytes3,Len,NamedNumberList) - end. - - -%% if no named bits are declared we will return a -%% {Unused,Bits}. Unused = integer(), -%% Bits = binary(). -compact_bit_string(Buffer,Len,[]) -> - getbits_as_binary(Len,Buffer); % {{Unused,BinBits},NewBuffer} -compact_bit_string(Buffer,Len,NamedNumberList) -> - bit_list_or_named(Buffer,Len,NamedNumberList). - - -%% if no named bits are declared we will return a -%% BitList = [0 | 1] - -bit_list_or_named(Buffer,Len,[]) -> - getbits_as_list(Len,Buffer); - -%% if there are named bits declared we will return a named -%% BitList where the names are atoms and unnamed bits represented -%% as {bit,Pos} -%% BitList = [atom() | {bit,Pos}] -%% Pos = integer() - -bit_list_or_named(Buffer,Len,NamedNumberList) -> - {BitList,Rest} = getbits_as_list(Len,Buffer), - {bit_list_or_named1(0,BitList,NamedNumberList,[]), Rest}. - -bit_list_or_named1(Pos,[0|Bt],Names,Acc) -> - bit_list_or_named1(Pos+1,Bt,Names,Acc); -bit_list_or_named1(Pos,[1|Bt],Names,Acc) -> - case lists:keysearch(Pos,2,Names) of - {value,{Name,_}} -> - bit_list_or_named1(Pos+1,Bt,Names,[Name|Acc]); - _ -> - bit_list_or_named1(Pos+1,Bt,Names,[{bit,Pos}|Acc]) - end; -bit_list_or_named1(_,[],_,Acc) -> - lists:reverse(Acc). - - - -%%%%%%%%%%%%%%% -%% - -int_to_bitlist(Int) when is_integer(Int), Int > 0 -> - [Int band 1 | int_to_bitlist(Int bsr 1)]; -int_to_bitlist(0) -> - []. - - -%%%%%%%%%%%%%%%%%% -%% get_all_bitposes([list of named bits to set], named_bit_db, []) -> -%% [sorted_list_of_bitpositions_to_set] - -get_all_bitposes([{bit,ValPos}|Rest], NamedBitList, Ack) -> - get_all_bitposes(Rest, NamedBitList, [ValPos | Ack ]); - -get_all_bitposes([Val | Rest], NamedBitList, Ack) -> - case lists:keysearch(Val, 1, NamedBitList) of - {value, {_ValName, ValPos}} -> - get_all_bitposes(Rest, NamedBitList, [ValPos | Ack]); - _ -> - exit({error,{asn1, {bitstring_namedbit, Val}}}) - end; -get_all_bitposes([], _NamedBitList, Ack) -> - lists:sort(Ack). - -%%%%%%%%%%%%%%%%%% -%% make_and_set_list([list of positions to set to 1])-> -%% returns list with all in SetPos set. -%% in positioning in list the first element is 0, the second 1 etc.., but -%% - -make_and_set_list([XPos|SetPos], XPos) -> - [1 | make_and_set_list(SetPos, XPos + 1)]; -make_and_set_list([Pos|SetPos], XPos) -> - [0 | make_and_set_list([Pos | SetPos], XPos + 1)]; -make_and_set_list([], _) -> - []. - -%%%%%%%%%%%%%%%%% -%% pad_list(N,BitList) -> PaddedList -%% returns a padded (with trailing {bit,0} elements) list of length N -%% if Bitlist contains more than N significant bits set an exit asn1_error -%% is generated - -pad_list(N,In={Unused,Bin}) -> - pad_list(N, size(Bin)*8 - Unused, In). - -pad_list(N,Size,In={_,_}) when N < Size -> - exit({error,{asn1,{range_error,{bit_string,In}}}}); -pad_list(N,Size,{Unused,Bin}) when N > Size, Unused > 0 -> - pad_list(N,Size+1,{Unused-1,Bin}); -pad_list(N,Size,{_Unused,Bin}) when N > Size -> - pad_list(N,Size+1,{7,<<Bin/binary,0>>}); -pad_list(N,N,In={_,_}) -> - In. - -%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%% -%% X.691:16 -%% encode_octet_string(Constraint,ExtensionMarker,Val) -%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%% - -encode_octet_string(C,Val) -> - encode_octet_string2(C,Val). - -encode_octet_string2(C,{_Name,Val}) -> - encode_octet_string2(C,Val); -encode_octet_string2(C,Val) -> - case get_constraint(C,'SizeConstraint') of - 0 -> - []; - 1 -> - [V] = Val, - {bits,8,V}; - 2 -> - [V1,V2] = Val, - [{bits,8,V1},{bits,8,V2}]; - Sv when Sv =<65535, Sv == length(Val) -> % fixed length - {octets,Val}; - {Lb,Ub} -> - [encode_length({Lb,Ub},length(Val)),{octets,Val}]; - Sv when is_list(Sv) -> - [encode_length({hd(Sv),lists:max(Sv)},length(Val)),{octets,Val}]; - no -> - [encode_length(undefined,length(Val)),{octets,Val}] - end. - -decode_octet_string(Bytes,Range) -> - decode_octet_string(Bytes,Range,false). - -decode_octet_string(Bytes,C,false) -> - case get_constraint(C,'SizeConstraint') of - 0 -> - {[],Bytes}; - 1 -> - {B1,Bytes2} = getbits(Bytes,8), - {[B1],Bytes2}; - 2 -> - {Bs,Bytes2}= getbits(Bytes,16), - {binary_to_list(<<Bs:16>>),Bytes2}; - {_,0} -> - {[],Bytes}; - Sv when is_integer(Sv), Sv =<65535 -> % fixed length - getoctets_as_list(Bytes,Sv); - Sv when is_integer(Sv) -> % fragmented encoding - Bytes2 = align(Bytes), - decode_fragmented_octets(Bytes2,Sv); - {Lb,Ub} -> - {Len,Bytes2} = decode_length(Bytes,{Lb,Ub}), - getoctets_as_list(Bytes2,Len); - Sv when is_list(Sv) -> - {Len,Bytes2} = decode_length(Bytes,{hd(Sv),lists:max(Sv)}), - getoctets_as_list(Bytes2,Len); - no -> - {Len,Bytes2} = decode_length(Bytes,undefined), - getoctets_as_list(Bytes2,Len) - end. - -%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%% -%% Restricted char string types -%% (NumericString, PrintableString,VisibleString,IA5String,BMPString,UniversalString) -%% X.691:26 and X.680:34-36 -%%encode_restricted_string(aligned,'BMPString',Constraints,Extension,Val) - - -encode_restricted_string(aligned,{Name,Val}) when is_atom(Name) -> - encode_restricted_string(aligned,Val); - -encode_restricted_string(aligned,Val) when is_list(Val)-> - [encode_length(undefined,length(Val)),{octets,Val}]. - -encode_known_multiplier_string(aligned,StringType,C,_Ext,{Name,Val}) when is_atom(Name) -> - encode_known_multiplier_string(aligned,StringType,C,false,Val); - -encode_known_multiplier_string(aligned,StringType,C,_Ext,Val) -> - Result = chars_encode(C,StringType,Val), - NumBits = get_NumBits(C,StringType), - case get_constraint(C,'SizeConstraint') of - Ub when is_integer(Ub), Ub*NumBits =< 16 -> - Result; - 0 -> - []; - Ub when is_integer(Ub),Ub =<65535 -> % fixed length - [align,Result]; - {Ub,Lb} -> - [encode_length({Ub,Lb},length(Val)),align,Result]; - Vl when is_list(Vl) -> - [encode_length({lists:min(Vl),lists:max(Vl)},length(Val)),align,Result]; - no -> - [encode_length(undefined,length(Val)),align,Result] - end. - -decode_restricted_string(Bytes,aligned) -> - {Len,Bytes2} = decode_length(Bytes,undefined), - getoctets_as_list(Bytes2,Len). - -decode_known_multiplier_string(Bytes,aligned,StringType,C,_Ext) -> - NumBits = get_NumBits(C,StringType), - case get_constraint(C,'SizeConstraint') of - Ub when is_integer(Ub), Ub*NumBits =< 16 -> - chars_decode(Bytes,NumBits,StringType,C,Ub); - Ub when is_integer(Ub),Ub =<65535 -> % fixed length - Bytes1 = align(Bytes), - chars_decode(Bytes1,NumBits,StringType,C,Ub); - 0 -> - {[],Bytes}; - Vl when is_list(Vl) -> - {Len,Bytes1} = decode_length(Bytes,{hd(Vl),lists:max(Vl)}), - Bytes2 = align(Bytes1), - chars_decode(Bytes2,NumBits,StringType,C,Len); - no -> - {Len,Bytes1} = decode_length(Bytes,undefined), - Bytes2 = align(Bytes1), - chars_decode(Bytes2,NumBits,StringType,C,Len); - {Lb,Ub}-> - {Len,Bytes1} = decode_length(Bytes,{Lb,Ub}), - Bytes2 = align(Bytes1), - chars_decode(Bytes2,NumBits,StringType,C,Len) - end. - - -encode_NumericString(C,Val) -> - encode_known_multiplier_string(aligned,'NumericString',C,false,Val). -decode_NumericString(Bytes,C) -> - decode_known_multiplier_string(Bytes,aligned,'NumericString',C,false). - -encode_PrintableString(C,Val) -> - encode_known_multiplier_string(aligned,'PrintableString',C,false,Val). -decode_PrintableString(Bytes,C) -> - decode_known_multiplier_string(Bytes,aligned,'PrintableString',C,false). - -encode_VisibleString(C,Val) -> % equivalent with ISO646String - encode_known_multiplier_string(aligned,'VisibleString',C,false,Val). -decode_VisibleString(Bytes,C) -> - decode_known_multiplier_string(Bytes,aligned,'VisibleString',C,false). - -encode_IA5String(C,Val) -> - encode_known_multiplier_string(aligned,'IA5String',C,false,Val). -decode_IA5String(Bytes,C) -> - decode_known_multiplier_string(Bytes,aligned,'IA5String',C,false). - -encode_BMPString(C,Val) -> - encode_known_multiplier_string(aligned,'BMPString',C,false,Val). -decode_BMPString(Bytes,C) -> - decode_known_multiplier_string(Bytes,aligned,'BMPString',C,false). - -encode_UniversalString(C,Val) -> - encode_known_multiplier_string(aligned,'UniversalString',C,false,Val). -decode_UniversalString(Bytes,C) -> - decode_known_multiplier_string(Bytes,aligned,'UniversalString',C,false). - - -%% end of known-multiplier strings for which PER visible constraints are -%% applied - -encode_GeneralString(_C,Val) -> - encode_restricted_string(aligned,Val). -decode_GeneralString(Bytes,_C) -> - decode_restricted_string(Bytes,aligned). - -encode_GraphicString(_C,Val) -> - encode_restricted_string(aligned,Val). -decode_GraphicString(Bytes,_C) -> - decode_restricted_string(Bytes,aligned). - -encode_ObjectDescriptor(_C,Val) -> - encode_restricted_string(aligned,Val). -decode_ObjectDescriptor(Bytes) -> - decode_restricted_string(Bytes,aligned). - -encode_TeletexString(_C,Val) -> % equivalent with T61String - encode_restricted_string(aligned,Val). -decode_TeletexString(Bytes,_C) -> - decode_restricted_string(Bytes,aligned). - -encode_VideotexString(_C,Val) -> - encode_restricted_string(aligned,Val). -decode_VideotexString(Bytes,_C) -> - decode_restricted_string(Bytes,aligned). - - - - -%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%% -%% getBMPChars(Bytes,Len) ->{BMPcharList,RemainingBytes} -%% -getBMPChars(Bytes,1) -> - {O1,Bytes2} = getbits(Bytes,8), - {O2,Bytes3} = getbits(Bytes2,8), - if - O1 == 0 -> - {[O2],Bytes3}; - true -> - {[{0,0,O1,O2}],Bytes3} - end; -getBMPChars(Bytes,Len) -> - getBMPChars(Bytes,Len,[]). - -getBMPChars(Bytes,0,Acc) -> - {lists:reverse(Acc),Bytes}; -getBMPChars(Bytes,Len,Acc) -> - {Octs,Bytes1} = getoctets_as_list(Bytes,2), - case Octs of - [0,O2] -> - getBMPChars(Bytes1,Len-1,[O2|Acc]); - [O1,O2]-> - getBMPChars(Bytes1,Len-1,[{0,0,O1,O2}|Acc]) - end. - - -%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%% -%% chars_encode(C,StringType,Value) -> ValueList -%% -%% encodes chars according to the per rules taking the constraint PermittedAlphabet -%% into account. -%% This function does only encode the value part and NOT the length - -chars_encode(C,StringType,Value) -> - case {StringType,get_constraint(C,'PermittedAlphabet')} of - {'UniversalString',{_,_Sv}} -> - exit({error,{asn1,{'not implemented',"UniversalString with PermittedAlphabet constraint"}}}); - {'BMPString',{_,_Sv}} -> - exit({error,{asn1,{'not implemented',"BMPString with PermittedAlphabet constraint"}}}); - _ -> - {NumBits,CharOutTab} = {get_NumBits(C,StringType),get_CharOutTab(C,StringType)}, - chars_encode2(Value,NumBits,CharOutTab) - end. - -chars_encode2([H|T],NumBits,{Min,Max,notab}) when H =< Max, H >= Min -> - [{bits,NumBits,H-Min}|chars_encode2(T,NumBits,{Min,Max,notab})]; -chars_encode2([H|T],NumBits,{Min,Max,Tab}) when H =< Max, H >= Min -> - [{bits,NumBits,exit_if_false(H,element(H-Min+1,Tab))}|chars_encode2(T,NumBits,{Min,Max,Tab})]; -chars_encode2([{A,B,C,D}|T],NumBits,{Min,Max,notab}) -> - %% no value range check here (ought to be, but very expensive) -% [{bits,NumBits,(A*B*C*D)-Min}|chars_encode2(T,NumBits,{Min,Max,notab})]; - [{bits,NumBits,((((((A bsl 8)+B) bsl 8)+C) bsl 8)+D)-Min}|chars_encode2(T,NumBits,{Min,Max,notab})]; -chars_encode2([{A,B,C,D}|T],NumBits,{Min,Max,Tab}) -> - %% no value range check here (ought to be, but very expensive) -% [{bits,NumBits,element((A*B*C*D)-Min,Tab)}|chars_encode2(T,NumBits,{Min,Max,notab})]; - [{bits,NumBits,exit_if_false({A,B,C,D},element(((((((A bsl 8)+B) bsl 8)+C) bsl 8)+D)-Min,Tab))}|chars_encode2(T,NumBits,{Min,Max,notab})]; -chars_encode2([H|_T],_,{_,_,_}) -> - exit({error,{asn1,{illegal_char_value,H}}}); -chars_encode2([],_,_) -> - []. - -exit_if_false(V,false)-> - exit({error,{asn1,{"illegal value according to Permitted alphabet constraint",V}}}); -exit_if_false(_,V) ->V. - - -get_NumBits(C,StringType) -> - case get_constraint(C,'PermittedAlphabet') of - {'SingleValue',Sv} -> - charbits(length(Sv),aligned); - no -> - case StringType of - 'IA5String' -> - charbits(128,aligned); % 16#00..16#7F - 'VisibleString' -> - charbits(95,aligned); % 16#20..16#7E - 'PrintableString' -> - charbits(74,aligned); % [$\s,$',$(,$),$+,$,,$-,$.,$/,"0123456789",$:,$=,$?,$A..$Z,$a..$z - 'NumericString' -> - charbits(11,aligned); % $ ,"0123456789" - 'UniversalString' -> - 32; - 'BMPString' -> - 16 - end - end. - -%%Maybe used later -%%get_MaxChar(C,StringType) -> -%% case get_constraint(C,'PermittedAlphabet') of -%% {'SingleValue',Sv} -> -%% lists:nth(length(Sv),Sv); -%% no -> -%% case StringType of -%% 'IA5String' -> -%% 16#7F; % 16#00..16#7F -%% 'VisibleString' -> -%% 16#7E; % 16#20..16#7E -%% 'PrintableString' -> -%% $z; % [$\s,$',$(,$),$+,$,,$-,$.,$/,"0123456789",$:,$=,$?,$A..$Z,$a..$z -%% 'NumericString' -> -%% $9; % $ ,"0123456789" -%% 'UniversalString' -> -%% 16#ffffffff; -%% 'BMPString' -> -%% 16#ffff -%% end -%% end. - -%%Maybe used later -%%get_MinChar(C,StringType) -> -%% case get_constraint(C,'PermittedAlphabet') of -%% {'SingleValue',Sv} -> -%% hd(Sv); -%% no -> -%% case StringType of -%% 'IA5String' -> -%% 16#00; % 16#00..16#7F -%% 'VisibleString' -> -%% 16#20; % 16#20..16#7E -%% 'PrintableString' -> -%% $\s; % [$\s,$',$(,$),$+,$,,$-,$.,$/,"0123456789",$:,$=,$?,$A..$Z,$a..$z -%% 'NumericString' -> -%% $\s; % $ ,"0123456789" -%% 'UniversalString' -> -%% 16#00; -%% 'BMPString' -> -%% 16#00 -%% end -%% end. - -get_CharOutTab(C,StringType) -> - get_CharTab(C,StringType,out). - -get_CharInTab(C,StringType) -> - get_CharTab(C,StringType,in). - -get_CharTab(C,StringType,InOut) -> - case get_constraint(C,'PermittedAlphabet') of - {'SingleValue',Sv} -> - get_CharTab2(C,StringType,hd(Sv),lists:max(Sv),Sv,InOut); - no -> - case StringType of - 'IA5String' -> - {0,16#7F,notab}; - 'VisibleString' -> - get_CharTab2(C,StringType,16#20,16#7F,notab,InOut); - 'PrintableString' -> - Chars = lists:sort( - " '()+,-./0123456789:=?ABCDEFGHIJKLMNOPQRSTUVWXYZabcdefghijklmnopqrstuvwxyz"), - get_CharTab2(C,StringType,hd(Chars),lists:max(Chars),Chars,InOut); - 'NumericString' -> - get_CharTab2(C,StringType,16#20,$9," 0123456789",InOut); - 'UniversalString' -> - {0,16#FFFFFFFF,notab}; - 'BMPString' -> - {0,16#FFFF,notab} - end - end. - -get_CharTab2(C,StringType,Min,Max,Chars,InOut) -> - BitValMax = (1 bsl get_NumBits(C,StringType))-1, - if - Max =< BitValMax -> - {0,Max,notab}; - true -> - case InOut of - out -> - {Min,Max,create_char_tab(Min,Chars)}; - in -> - {Min,Max,list_to_tuple(Chars)} - end - end. - -create_char_tab(Min,L) -> - list_to_tuple(create_char_tab(Min,L,0)). -create_char_tab(Min,[Min|T],V) -> - [V|create_char_tab(Min+1,T,V+1)]; -create_char_tab(_Min,[],_V) -> - []; -create_char_tab(Min,L,V) -> - [false|create_char_tab(Min+1,L,V)]. - -%% This very inefficient and should be moved to compiletime -charbits(NumOfChars,aligned) -> - case charbits(NumOfChars) of - 1 -> 1; - 2 -> 2; - B when B =< 4 -> 4; - B when B =< 8 -> 8; - B when B =< 16 -> 16; - B when B =< 32 -> 32 - end. - -charbits(NumOfChars) when NumOfChars =< 2 -> 1; -charbits(NumOfChars) when NumOfChars =< 4 -> 2; -charbits(NumOfChars) when NumOfChars =< 8 -> 3; -charbits(NumOfChars) when NumOfChars =< 16 -> 4; -charbits(NumOfChars) when NumOfChars =< 32 -> 5; -charbits(NumOfChars) when NumOfChars =< 64 -> 6; -charbits(NumOfChars) when NumOfChars =< 128 -> 7; -charbits(NumOfChars) when NumOfChars =< 256 -> 8; -charbits(NumOfChars) when NumOfChars =< 512 -> 9; -charbits(NumOfChars) when NumOfChars =< 1024 -> 10; -charbits(NumOfChars) when NumOfChars =< 2048 -> 11; -charbits(NumOfChars) when NumOfChars =< 4096 -> 12; -charbits(NumOfChars) when NumOfChars =< 8192 -> 13; -charbits(NumOfChars) when NumOfChars =< 16384 -> 14; -charbits(NumOfChars) when NumOfChars =< 32768 -> 15; -charbits(NumOfChars) when NumOfChars =< 65536 -> 16; -charbits(NumOfChars) when is_integer(NumOfChars) -> - 16 + charbits1(NumOfChars bsr 16). - -charbits1(0) -> - 0; -charbits1(NumOfChars) -> - 1 + charbits1(NumOfChars bsr 1). - - -chars_decode(Bytes,_,'BMPString',C,Len) -> - case get_constraint(C,'PermittedAlphabet') of - no -> - getBMPChars(Bytes,Len); - _ -> - exit({error,{asn1, - {'not implemented', - "BMPString with PermittedAlphabet constraint"}}}) - end; -chars_decode(Bytes,NumBits,StringType,C,Len) -> - CharInTab = get_CharInTab(C,StringType), - chars_decode2(Bytes,CharInTab,NumBits,Len). - - -chars_decode2(Bytes,CharInTab,NumBits,Len) -> - chars_decode2(Bytes,CharInTab,NumBits,Len,[]). - -chars_decode2(Bytes,_CharInTab,_NumBits,0,Acc) -> - {lists:reverse(Acc),Bytes}; -chars_decode2(Bytes,{Min,Max,notab},NumBits,Len,Acc) when NumBits > 8 -> - {Char,Bytes2} = getbits(Bytes,NumBits), - Result = - if - Char < 256 -> Char; - true -> - list_to_tuple(binary_to_list(<<Char:32>>)) - end, - chars_decode2(Bytes2,{Min,Max,notab},NumBits,Len -1,[Result|Acc]); -% chars_decode2(Bytes,{Min,Max,notab},NumBits,Len,Acc) when NumBits > 8 -> -% {Char,Bytes2} = getbits(Bytes,NumBits), -% Result = case minimum_octets(Char+Min) of -% [NewChar] -> NewChar; -% [C1,C2] -> {0,0,C1,C2}; -% [C1,C2,C3] -> {0,C1,C2,C3}; -% [C1,C2,C3,C4] -> {C1,C2,C3,C4} -% end, -% chars_decode2(Bytes2,{Min,Max,notab},NumBits,Len -1,[Result|Acc]); -chars_decode2(Bytes,{Min,Max,notab},NumBits,Len,Acc) -> - {Char,Bytes2} = getbits(Bytes,NumBits), - chars_decode2(Bytes2,{Min,Max,notab},NumBits,Len -1,[Char+Min|Acc]); - -%% BMPString and UniversalString with PermittedAlphabet is currently not supported -chars_decode2(Bytes,{Min,Max,CharInTab},NumBits,Len,Acc) -> - {Char,Bytes2} = getbits(Bytes,NumBits), - chars_decode2(Bytes2,{Min,Max,CharInTab},NumBits,Len -1,[element(Char+1,CharInTab)|Acc]). - - -%% UTF8String -encode_UTF8String(Val) when is_binary(Val) -> - [encode_length(undefined,size(Val)),{octets,Val}]; -encode_UTF8String(Val) -> - Bin = list_to_binary(Val), - encode_UTF8String(Bin). - -decode_UTF8String(Bytes) -> - {Len,Bytes2} = decode_length(Bytes,undefined), - {Octs,Bytes3} = getoctets_as_list(Bytes2,Len), - {list_to_binary(Octs),Bytes3}. - - - % X.691:17 -encode_null(_) -> []. % encodes to nothing -%encode_null({Name,Val}) when is_atom(Name) -> -% encode_null(Val). - -decode_null(Bytes) -> - {'NULL',Bytes}. - -%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%% -%% encode_object_identifier(Val) -> CompleteList -%% encode_object_identifier({Name,Val}) -> CompleteList -%% Val -> {Int1,Int2,...,IntN} % N >= 2 -%% Name -> atom() -%% Int1 -> integer(0..2) -%% Int2 -> integer(0..39) when Int1 (0..1) else integer() -%% Int3-N -> integer() -%% CompleteList -> [{bits,8,Val}|{octets,Ol}|align|...] -%% -encode_object_identifier({Name,Val}) when is_atom(Name) -> - encode_object_identifier(Val); -encode_object_identifier(Val) -> - OctetList = e_object_identifier(Val), - Octets = list_to_binary(OctetList), % performs a flatten at the same time - [{debug,object_identifier},encode_length(undefined,size(Octets)),{octets,Octets}]. - -%% This code is copied from asn1_encode.erl (BER) and corrected and modified - -e_object_identifier({'OBJECT IDENTIFIER',V}) -> - e_object_identifier(V); -e_object_identifier({Cname,V}) when is_atom(Cname),is_tuple(V) -> - e_object_identifier(tuple_to_list(V)); -e_object_identifier({Cname,V}) when is_atom(Cname),is_list(V) -> - e_object_identifier(V); -e_object_identifier(V) when is_tuple(V) -> - e_object_identifier(tuple_to_list(V)); - -%% E1 = 0|1|2 and (E2 < 40 when E1 = 0|1) -e_object_identifier([E1,E2|Tail]) when E1 >= 0, E1 < 2, E2 < 40 ; E1==2 -> - Head = 40*E1 + E2, % weird - e_object_elements([Head|Tail],[]); -e_object_identifier(Oid=[_,_|_Tail]) -> - exit({error,{asn1,{'illegal_value',Oid}}}). - -e_object_elements([],Acc) -> - lists:reverse(Acc); -e_object_elements([H|T],Acc) -> - e_object_elements(T,[e_object_element(H)|Acc]). - -e_object_element(Num) when Num < 128 -> - [Num]; -e_object_element(Num) -> - [e_o_e(Num bsr 7)|[Num band 2#1111111]]. -e_o_e(Num) when Num < 128 -> - Num bor 2#10000000; -e_o_e(Num) -> - [e_o_e(Num bsr 7)|[(Num band 2#1111111) bor 2#10000000]]. - - - -%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%% -%% decode_object_identifier(Bytes) -> {ObjId,RemainingBytes} -%% ObjId -> {integer(),integer(),...} % at least 2 integers -%% RemainingBytes -> [integer()] when integer() (0..255) -decode_object_identifier(Bytes) -> - {Len,Bytes2} = decode_length(Bytes,undefined), - {Octs,Bytes3} = getoctets_as_list(Bytes2,Len), - [First|Rest] = dec_subidentifiers(Octs,0,[]), - Idlist = if - First < 40 -> - [0,First|Rest]; - First < 80 -> - [1,First - 40|Rest]; - true -> - [2,First - 80|Rest] - end, - {list_to_tuple(Idlist),Bytes3}. - -dec_subidentifiers([H|T],Av,Al) when H >=16#80 -> - dec_subidentifiers(T,(Av bsl 7) + (H band 16#7F),Al); -dec_subidentifiers([H|T],Av,Al) -> - dec_subidentifiers(T,0,[(Av bsl 7) + H |Al]); -dec_subidentifiers([],_Av,Al) -> - lists:reverse(Al). - -%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%% -%% encode_relative_oid(Val) -> CompleteList -%% encode_relative_oid({Name,Val}) -> CompleteList -encode_relative_oid({Name,Val}) when is_atom(Name) -> - encode_relative_oid(Val); -encode_relative_oid(Val) when is_tuple(Val) -> - encode_relative_oid(tuple_to_list(Val)); -encode_relative_oid(Val) when is_list(Val) -> - Octets = list_to_binary([e_object_element(X)||X <- Val]), - [encode_length(undefined,size(Octets)),{octets,Octets}]. - - -%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%% -%% decode_relative_oid(Val) -> {ROID,Rest} -%% decode_relative_oid({Name,Val}) -> {ROID,Rest} -decode_relative_oid(Bytes) -> - {Len,Bytes2} = decode_length(Bytes,undefined), - {Octs,Bytes3} = getoctets_as_list(Bytes2,Len), - ObjVals = dec_subidentifiers(Octs,0,[]), - {list_to_tuple(ObjVals),Bytes3}. - - -%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%% -%% encode_real(Val) -> CompleteList -%% encode_real({Name,Val}) -> CompleteList -encode_real({Name,Val}) when is_atom(Name) -> - encode_real(Val); -encode_real(Real) -> - {EncVal,Len} = ?RT_COMMON:encode_real([],Real), - [encode_length(undefined,Len),{octets,EncVal}]. - - -%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%% -%% decode_real(Val) -> {REALvalue,Rest} -%% decode_real({Name,Val}) -> {REALvalue,Rest} -decode_real(Bytes) -> - {Len,{0,Bytes2}} = decode_length(Bytes,undefined), - {RealVal,Rest,Len} = ?RT_COMMON:decode_real(Bytes2,Len), - {RealVal,{0,Rest}}. - - -get_constraint([{Key,V}],Key) -> - V; -get_constraint([],_Key) -> - no; -get_constraint(C,Key) -> - case lists:keysearch(Key,1,C) of - false -> - no; - {value,{_,V}} -> - V - end. - -%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%% -%% complete(InList) -> ByteList -%% Takes a coded list with bits and bytes and converts it to a list of bytes -%% Should be applied as the last step at encode of a complete ASN.1 type -%% - -% complete(L) -> -% case complete1(L) of -% {[],0} -> -% <<0>>; -% {Acc,0} -> -% lists:reverse(Acc); -% {[Hacc|Tacc],Acclen} -> % Acclen >0 -% Rest = 8 - Acclen, -% NewHacc = Hacc bsl Rest, -% lists:reverse([NewHacc|Tacc]) -% end. - - -% complete1(InList) when is_list(InList) -> -% complete1(InList,[]); -% complete1(InList) -> -% complete1([InList],[]). - -% complete1([{debug,_}|T], Acc) -> -% complete1(T,Acc); -% complete1([H|T],Acc) when is_list(H) -> -% {NewH,NewAcclen} = complete1(H,Acc), -% complete1(T,NewH,NewAcclen); - -% complete1([{0,Bin}|T],Acc,0) when is_binary(Bin) -> -% complete1(T,[Bin|Acc],0); -% complete1([{Unused,Bin}|T],Acc,0) when is_integer(Unused),is_binary(Bin) -> -% Size = size(Bin)-1, -% <<Bs:Size/binary,B>> = Bin, -% complete1(T,[(B bsr Unused),Bs|Acc],8-Unused); -% complete1([{Unused,Bin}|T],[Hacc|Tacc],Acclen) when is_integer(Unused),is_binary(Bin) -> -% Rest = 8 - Acclen, -% Used = 8 - Unused, -% case size(Bin) of -% 1 -> -% if -% Rest >= Used -> -% <<B:Used,_:Unused>> = Bin, -% complete1(T,[(Hacc bsl Used) + B|Tacc], -% (Acclen+Used) rem 8); -% true -> -% LeftOver = 8 - Rest - Unused, -% <<Val2:Rest,Val1:LeftOver,_:Unused>> = Bin, -% complete1(T,[Val1,(Hacc bsl Rest) + Val2|Tacc], -% (Acclen+Used) rem 8) -% end; -% N -> -% if -% Rest == Used -> -% N1 = N - 1, -% <<B:Rest,Bs:N1/binary,_:Unused>> = Bin, -% complete1(T,[Bs,(Hacc bsl Rest) + B|Tacc],0); -% Rest > Used -> -% N1 = N - 2, -% N2 = (8 - Rest) + Used, -% <<B1:Rest,Bytes:N1/binary,B2:N2,_:Unused>> = Bin, -% complete1(T,[B2,Bytes,(Hacc bsl Rest) + B1|Tacc], -% (Acclen + Used) rem 8); -% true -> % Rest < Used -% N1 = N - 1, -% N2 = Used - Rest, -% <<B1:Rest,Bytes:N1/binary,B2:N2,_:Unused>> = Bin, -% complete1(T,[B2,Bytes,(Hacc bsl Rest) + B1|Tacc], -% (Acclen + Used) rem 8) -% end -% end; - -% %complete1([{octets,N,Val}|T],Acc,Acclen) when N =< 4 ,is_integer(Val) -> -% % complete1([{octets,<<Val:N/unit:8>>}|T],Acc,Acclen); -% complete1([{octets,N,Val}|T],Acc,Acclen) when N =< 4 ,is_integer(Val) -> -% Newval = case N of -% 1 -> -% Val4 = Val band 16#FF, -% [Val4]; -% 2 -> -% Val3 = (Val bsr 8) band 16#FF, -% Val4 = Val band 16#FF, -% [Val3,Val4]; -% 3 -> -% Val2 = (Val bsr 16) band 16#FF, -% Val3 = (Val bsr 8) band 16#FF, -% Val4 = Val band 16#FF, -% [Val2,Val3,Val4]; -% 4 -> -% Val1 = (Val bsr 24) band 16#FF, -% Val2 = (Val bsr 16) band 16#FF, -% Val3 = (Val bsr 8) band 16#FF, -% Val4 = Val band 16#FF, -% [Val1,Val2,Val3,Val4] -% end, -% complete1([{octets,Newval}|T],Acc,Acclen); - -% complete1([{octets,Bin}|T],Acc,Acclen) when is_binary(Bin) -> -% Rest = 8 - Acclen, -% if -% Rest == 8 -> -% complete1(T,[Bin|Acc],0); -% true -> -% [Hacc|Tacc]=Acc, -% complete1(T,[Bin, Hacc bsl Rest|Tacc],0) -% end; - -% complete1([{octets,Oct}|T],Acc,Acclen) when is_list(Oct) -> -% Rest = 8 - Acclen, -% if -% Rest == 8 -> -% complete1(T,[list_to_binary(Oct)|Acc],0); -% true -> -% [Hacc|Tacc]=Acc, -% complete1(T,[list_to_binary(Oct), Hacc bsl Rest|Tacc],0) -% end; - -% complete1([{bit,Val}|T], Acc, Acclen) -> -% complete1([{bits,1,Val}|T],Acc,Acclen); -% complete1([{octet,Val}|T], Acc, Acclen) -> -% complete1([{octets,1,Val}|T],Acc,Acclen); - -% complete1([{bits,N,Val}|T], Acc, 0) when N =< 8 -> -% complete1(T,[Val|Acc],N); -% complete1([{bits,N,Val}|T], [Hacc|Tacc], Acclen) when N =< 8 -> -% Rest = 8 - Acclen, -% if -% Rest >= N -> -% complete1(T,[(Hacc bsl N) + Val|Tacc],(Acclen+N) rem 8); -% true -> -% Diff = N - Rest, -% NewHacc = (Hacc bsl Rest) + (Val bsr Diff), -% Mask = element(Diff,{1,3,7,15,31,63,127,255}), -% complete1(T,[(Val band Mask),NewHacc|Tacc],(Acclen+N) rem 8) -% end; -% complete1([{bits,N,Val}|T], Acc, Acclen) -> % N > 8 -% complete1([{bits,N-8,Val bsr 8},{bits,8,Val band 255}|T],Acc,Acclen); - -% complete1([align|T],Acc,0) -> -% complete1(T,Acc,0); -% complete1([align|T],[Hacc|Tacc],Acclen) -> -% Rest = 8 - Acclen, -% complete1(T,[Hacc bsl Rest|Tacc],0); -% complete1([{octets,N,Val}|T],Acc,Acclen) when is_list(Val) -> % no security check here -% complete1([{octets,Val}|T],Acc,Acclen); - -% complete1([],Acc,Acclen) -> -% {Acc,Acclen}. - - -%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%% -%% complete(InList) -> ByteList -%% Takes a coded list with bits and bytes and converts it to a list of bytes -%% Should be applied as the last step at encode of a complete ASN.1 type -%% - -complete(L) -> - case complete1(L) of - {[],[]} -> - <<0>>; - {Acc,[]} -> - Acc; - {Acc,Bacc} -> - [Acc|complete_bytes(Bacc)] - end. - -%% this function builds the ugly form of lists [E1|E2] to avoid having to reverse it at the end. -%% this is done because it is efficient and that the result always will be sent on a port or -%% converted by means of list_to_binary/1 -complete1(InList) when is_list(InList) -> - complete1(InList,[],[]); -complete1(InList) -> - complete1([InList],[],[]). - -complete1([],Acc,Bacc) -> - {Acc,Bacc}; -complete1([H|T],Acc,Bacc) when is_list(H) -> - {NewH,NewBacc} = complete1(H,Acc,Bacc), - complete1(T,NewH,NewBacc); - -complete1([{octets,Bin}|T],Acc,[]) -> - complete1(T,[Acc|Bin],[]); - -complete1([{octets,Bin}|T],Acc,Bacc) -> - complete1(T,[Acc|[complete_bytes(Bacc),Bin]],[]); - -complete1([{debug,_}|T], Acc,Bacc) -> - complete1(T,Acc,Bacc); - -complete1([{bits,N,Val}|T],Acc,Bacc) -> - complete1(T,Acc,complete_update_byte(Bacc,Val,N)); - -complete1([{bit,Val}|T],Acc,Bacc) -> - complete1(T,Acc,complete_update_byte(Bacc,Val,1)); - -complete1([align|T],Acc,[]) -> - complete1(T,Acc,[]); -complete1([align|T],Acc,Bacc) -> - complete1(T,[Acc|complete_bytes(Bacc)],[]); -complete1([{0,Bin}|T],Acc,[]) when is_binary(Bin) -> - complete1(T,[Acc|Bin],[]); -complete1([{Unused,Bin}|T],Acc,[]) when is_integer(Unused),is_binary(Bin) -> - Size = size(Bin)-1, - <<Bs:Size/binary,B>> = Bin, - NumBits = 8-Unused, - complete1(T,[Acc|Bs],[[B bsr Unused]|NumBits]); -complete1([{Unused,Bin}|T],Acc,Bacc) when is_integer(Unused),is_binary(Bin) -> - Size = size(Bin)-1, - <<Bs:Size/binary,B>> = Bin, - NumBits = 8 - Unused, - Bf = complete_bytes(Bacc), - complete1(T,[Acc|[Bf,Bs]],[[B bsr Unused]|NumBits]). - - -complete_update_byte([],Val,Len) -> - complete_update_byte([[0]|0],Val,Len); -complete_update_byte([[Byte|Bacc]|NumBits],Val,Len) when NumBits + Len == 8 -> - [[0,((Byte bsl Len) + Val) band 255|Bacc]|0]; -complete_update_byte([[Byte|Bacc]|NumBits],Val,Len) when NumBits + Len > 8 -> - Rem = 8 - NumBits, - Rest = Len - Rem, - complete_update_byte([[0,((Byte bsl Rem) + (Val bsr Rest)) band 255 |Bacc]|0],Val,Rest); -complete_update_byte([[Byte|Bacc]|NumBits],Val,Len) -> - [[((Byte bsl Len) + Val) band 255|Bacc]|NumBits+Len]. - - -complete_bytes([[_Byte|Bacc]|0]) -> - lists:reverse(Bacc); -complete_bytes([[Byte|Bacc]|NumBytes]) -> - lists:reverse([(Byte bsl (8-NumBytes)) band 255|Bacc]); -complete_bytes([]) -> - []. - -% complete_bytes(L) -> -% complete_bytes1(lists:reverse(L),[],[],0,0). - -% complete_bytes1([H={V,B}|T],Acc,ReplyAcc,NumBits,NumFields) when ((NumBits+B) rem 8) == 0 -> -% NewReplyAcc = [complete_bytes2([H|Acc],0)|ReplyAcc], -% complete_bytes1(T,[],NewReplyAcc,0,0); -% complete_bytes1([H={V,B}|T],Acc,ReplyAcc,NumBits,NumFields) when NumFields == 7; (NumBits+B) div 8 > 0 -> -% Rem = (NumBits+B) rem 8, -% NewReplyAcc = [complete_bytes2([{V bsr Rem,B - Rem}|Acc],0)|ReplyAcc], -% complete_bytes1([{V,Rem}|T],[],NewReplyAcc,0,0); -% complete_bytes1([H={V,B}|T],Acc,ReplyAcc,NumBits,NumFields) -> -% complete_bytes1(T,[H|Acc],ReplyAcc,NumBits+B,NumFields+1); -% complete_bytes1([],[],ReplyAcc,_,_) -> -% lists:reverse(ReplyAcc); -% complete_bytes1([],Acc,ReplyAcc,NumBits,_) -> -% PadBits = case NumBits rem 8 of -% 0 -> 0; -% Rem -> 8 - Rem -% end, -% lists:reverse([complete_bytes2(Acc,PadBits)|ReplyAcc]). - - -% complete_bytes2([{V1,B1}],PadBits) -> -% <<V1:B1,0:PadBits>>; -% complete_bytes2([{V2,B2},{V1,B1}],PadBits) -> -% <<V1:B1,V2:B2,0:PadBits>>; -% complete_bytes2([{V3,B3},{V2,B2},{V1,B1}],PadBits) -> -% <<V1:B1,V2:B2,V3:B3,0:PadBits>>; -% complete_bytes2([{V4,B4},{V3,B3},{V2,B2},{V1,B1}],PadBits) -> -% <<V1:B1,V2:B2,V3:B3,V4:B4,0:PadBits>>; -% complete_bytes2([{V5,B5},{V4,B4},{V3,B3},{V2,B2},{V1,B1}],PadBits) -> -% <<V1:B1,V2:B2,V3:B3,V4:B4,V5:B5,0:PadBits>>; -% complete_bytes2([{V6,B6},{V5,B5},{V4,B4},{V3,B3},{V2,B2},{V1,B1}],PadBits) -> -% <<V1:B1,V2:B2,V3:B3,V4:B4,V5:B5,V6:B6,0:PadBits>>; -% complete_bytes2([{V7,B7},{V6,B6},{V5,B5},{V4,B4},{V3,B3},{V2,B2},{V1,B1}],PadBits) -> -% <<V1:B1,V2:B2,V3:B3,V4:B4,V5:B5,V6:B6,V7:B7,0:PadBits>>; -% complete_bytes2([{V8,B8},{V7,B7},{V6,B6},{V5,B5},{V4,B4},{V3,B3},{V2,B2},{V1,B1}],PadBits) -> -% <<V1:B1,V2:B2,V3:B3,V4:B4,V5:B5,V6:B6,V7:B7,V8:B8,0:PadBits>>. - - - - - - diff --git a/lib/asn1/src/asn1rt_per_bin_rt2ct.erl b/lib/asn1/src/asn1rt_per_bin_rt2ct.erl index 1df757a47f..5997232f13 100644 --- a/lib/asn1/src/asn1rt_per_bin_rt2ct.erl +++ b/lib/asn1/src/asn1rt_per_bin_rt2ct.erl @@ -22,25 +22,21 @@ -include("asn1_records.hrl"). --export([dec_fixup/3, cindex/3, list_to_record/2]). +-export([decode_fragmented/3]). -export([setchoiceext/1, setext/1, fixoptionals/3, fixextensions/2, - getext/1, getextension/2, skipextensions/3, getbit/1, getchoice/3 ]). --export([getoptionals/2, getoptionals2/2, - set_choice/3, encode_integer/2, encode_integer/3 ]). --export([decode_integer/2, decode_integer/3, encode_small_number/1, - decode_boolean/1, encode_length/2, decode_length/1, decode_length/2, - encode_small_length/1, decode_small_length/1, + skipextensions/3, getbit/1, getchoice/3 ]). +-export([set_choice/3, encode_integer/2, encode_integer/3 ]). +-export([encode_small_number/1, + encode_length/2, + encode_small_length/1, decode_compact_bit_string/3]). --export([decode_enumerated/3, - encode_bit_string/3, decode_bit_string/3 ]). --export([encode_octet_string/2, decode_octet_string/2, - encode_null/1, decode_null/1, +-export([encode_bit_string/3, decode_bit_string/3 ]). +-export([encode_octet_string/2, encode_object_identifier/1, decode_object_identifier/1, encode_real/1, decode_real/1, encode_relative_oid/1, decode_relative_oid/1, complete/1]). - -export([encode_open_type/2, decode_open_type/2]). -export([encode_GeneralString/2, decode_GeneralString/2, @@ -51,19 +47,10 @@ encode_UTF8String/1,decode_UTF8String/1 ]). --export([decode_constrained_number/2, - decode_constrained_number/3, - decode_unconstrained_number/1, - decode_semi_constrained_number/2, - encode_unconstrained_number/1, - decode_constrained_number/4, +-export([encode_unconstrained_number/1, encode_octet_string/3, - decode_octet_string/3, encode_known_multiplier_string/5, - decode_known_multiplier_string/5, - getoctets/2, getbits/2 -% start_drv/1,start_drv2/1,init_drv/1 - ]). + decode_known_multiplier_string/5]). -export([eint_positive/1]). @@ -73,32 +60,6 @@ -define('32K',32768). -define('64K',65536). -%%-define(nodriver,true). - -dec_fixup(Terms,Cnames,RemBytes) -> - dec_fixup(Terms,Cnames,RemBytes,[]). - -dec_fixup([novalue|T],[_Hc|Tc],RemBytes,Acc) -> - dec_fixup(T,Tc,RemBytes,Acc); -dec_fixup([{_Name,novalue}|T],[_Hc|Tc],RemBytes,Acc) -> - dec_fixup(T,Tc,RemBytes,Acc); -dec_fixup([H|T],[Hc|Tc],RemBytes,Acc) -> - dec_fixup(T,Tc,RemBytes,[{Hc,H}|Acc]); -dec_fixup([],_Cnames,RemBytes,Acc) -> - {lists:reverse(Acc),RemBytes}. - -cindex(Ix,Val,Cname) -> - case element(Ix,Val) of - {Cname,Val2} -> Val2; - X -> X - end. - -%% converts a list to a record if necessary -list_to_record(_,Tuple) when is_tuple(Tuple) -> - Tuple; -list_to_record(Name,List) when is_list(List) -> - list_to_tuple([Name|List]). - %%-------------------------------------------------------- %% setchoiceext(InRootSet) -> [{bit,X}] %% X is set to 1 when InRootSet==false @@ -143,15 +104,6 @@ fixoptionals([Pos|Ot],Val,Acc) -> end. -getext(Bytes) when is_bitstring(Bytes) -> - getbit(Bytes). - -getextension(0, Bytes) -> - {<<>>,Bytes}; -getextension(1, Bytes) -> - {Len,Bytes2} = decode_small_length(Bytes), - getbits_as_binary(Len,Bytes2).% {Bin,Bytes3}. - fixextensions({ext,ExtPos,ExtNum},Val) -> case fixextensions(ExtPos,ExtNum+ExtPos,Val,0) of 0 -> []; @@ -194,14 +146,6 @@ getchoice(Bytes,_,1) -> getchoice(Bytes,NumChoices,0) -> decode_constrained_number(Bytes,{0,NumChoices-1}). -%% old version kept for backward compatibility with generates from R7B01 -getoptionals(Bytes,NumOpt) -> - getbits_as_binary(NumOpt,Bytes). - -%% new version used in generates from r8b_patch/3 and later -getoptionals2(Bytes,NumOpt) -> - {_,_} = getbits(Bytes,NumOpt). - %% getbits_as_binary(Num,Bytes) -> {Bin,Rest} %% Num = integer(), @@ -344,32 +288,6 @@ decode_fragmented_bits(<<0:1,Len:7,Bin/binary>>,C,Acc) -> exit({error,{asn1,{illegal_value,C,BinBits}}}) end. - -decode_fragmented_octets(Bin,C) -> - decode_fragmented_octets(Bin,C,[]). - -decode_fragmented_octets(<<3:2,Len:6,Bin/binary>>,C,Acc) -> - {Value,Bin2} = split_binary(Bin,Len * ?'16K'), - decode_fragmented_octets(Bin2,C,[Value|Acc]); -decode_fragmented_octets(<<0:1,0:7,Bin/binary>>,C,Acc) -> - Octets = list_to_binary(lists:reverse(Acc)), - case C of - Int when is_integer(Int), C == size(Octets) -> - {Octets,Bin}; - Int when is_integer(Int) -> - exit({error,{asn1,{illegal_value,C,Octets}}}) - end; -decode_fragmented_octets(<<0:1,Len:7,Bin/binary>>,C,Acc) -> - <<Value:Len/binary-unit:8,Bin2/binary>> = Bin, - BinOctets = list_to_binary(lists:reverse([Value|Acc])), - case C of - Int when is_integer(Int),size(BinOctets) == Int -> - {BinOctets,Bin2}; - Int when is_integer(Int) -> - exit({error,{asn1,{illegal_value,C,BinOctets}}}) - end. - - %%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%% %% encode_open_type(Constraint, Value) -> CompleteList @@ -454,40 +372,6 @@ encode_integer(_,Val) -> exit({error,{asn1,{illegal_value,Val}}}). - -decode_integer(Buffer,Range,NamedNumberList) -> - {Val,Buffer2} = decode_integer(Buffer,Range), - case lists:keysearch(Val,2,NamedNumberList) of - {value,{NewVal,_}} -> {NewVal,Buffer2}; - _ -> {Val,Buffer2} - end. - -decode_integer(Buffer,[{Rc,_Ec}]) when is_tuple(Rc) -> - {Ext,Buffer2} = getext(Buffer), - case Ext of - 0 -> decode_integer(Buffer2,[Rc]); - 1 -> decode_unconstrained_number(Buffer2) - end; -decode_integer(Buffer,undefined) -> - decode_unconstrained_number(Buffer); -decode_integer(Buffer,C) -> - case get_constraint(C,'SingleValue') of - V when is_integer(V) -> - {V,Buffer}; - _ -> - decode_integer1(Buffer,C) - end. - -decode_integer1(Buffer,C) -> - case VR = get_constraint(C,'ValueRange') of - no -> - decode_unconstrained_number(Buffer); - {Lb, 'MAX'} -> - decode_semi_constrained_number(Buffer,Lb); - {_Lb,_Ub} -> - decode_constrained_number(Buffer,VR) - end. - %% X.691:10.6 Encoding of a normally small non-negative whole number %% Use this for encoding of CHOICE index if there is an extension marker in %% the CHOICE @@ -507,7 +391,7 @@ decode_small_number(Bytes) -> 0 -> getbits(Bytes2,6); 1 -> - decode_semi_constrained_number(Bytes2,0) + decode_semi_constrained_number(Bytes2) end. %% X.691:10.7 Encoding of a semi-constrained whole number @@ -530,12 +414,10 @@ encode_semi_constrained_number(Lb,Val) -> [encode_length(undefined,Len),[21,<<Len:16>>,Oct]] end. -decode_semi_constrained_number(Bytes,{Lb,_}) -> - decode_semi_constrained_number(Bytes,Lb); -decode_semi_constrained_number(Bytes,Lb) -> +decode_semi_constrained_number(Bytes) -> {Len,Bytes2} = decode_length(Bytes,undefined), {V,Bytes3} = getoctets(Bytes2,Len), - {V+Lb,Bytes3}. + {V,Bytes3}. encode_constrained_number({Lb,_Ub},_Range,{bits,N},Val) -> Val2 = Val-Lb, @@ -609,7 +491,7 @@ encode_constrained_number({Lb,Ub},Val) when Val >= Lb, Ub >= Val -> RangeOcts = binary:encode_unsigned(Range - 1), OctsLen = erlang:byte_size(Octs), RangeOctsLen = erlang:byte_size(RangeOcts), - LengthBitsNeeded = asn1rt_per_bin:minimum_bits(RangeOctsLen - 1), + LengthBitsNeeded = minimum_bits(RangeOctsLen - 1), [10,LengthBitsNeeded,OctsLen-1,20,OctsLen,Octs]; true -> exit({not_supported,{integer_range,Range}}) @@ -621,13 +503,6 @@ decode_constrained_number(Buffer,VR={Lb,Ub}) -> Range = Ub - Lb + 1, decode_constrained_number(Buffer,VR,Range). -decode_constrained_number(Buffer,{Lb,_Ub},_Range,{bits,N}) -> - {Val,Remain} = getbits(Buffer,N), - {Val+Lb,Remain}; -decode_constrained_number(Buffer,{Lb,_Ub},_Range,{octets,N}) -> - {Val,Remain} = getoctets(Buffer,N), - {Val+Lb,Remain}. - decode_constrained_number(Buffer,{Lb,_Ub},Range) -> % Val2 = Val - Lb, {Val,Remain} = @@ -665,6 +540,16 @@ decode_constrained_number(Buffer,{Lb,_Ub},Range) -> end, {Val+Lb,Remain}. +%% For some reason the minimum bits needed in the length field in +%% the encoding of constrained whole numbers must always be at least 2? +minimum_bits(N) when N < 4 -> 2; +minimum_bits(N) when N < 8 -> 3; +minimum_bits(N) when N < 16 -> 4; +minimum_bits(N) when N < 32 -> 5; +minimum_bits(N) when N < 64 -> 6; +minimum_bits(N) when N < 128 -> 7; +minimum_bits(_N) -> 8. + %% X.691:10.8 Encoding of an unconstrained whole number encode_unconstrained_number(Val) when Val >= 0 -> @@ -718,23 +603,6 @@ enint(-1, [B1|T]) when B1 > 127 -> enint(N, Acc) -> enint(N bsr 8, [N band 16#ff|Acc]). -decode_unconstrained_number(Bytes) -> - {Len,Bytes2} = decode_length(Bytes,undefined), - {Ints,Bytes3} = getoctets_as_bin(Bytes2,Len), - {dec_integer(Ints),Bytes3}. - -dec_integer(Bin = <<0:1,_:7,_/binary>>) -> - decpint(Bin); -dec_integer(<<_:1,B:7,BitStr/bitstring>>) -> - Size = bit_size(BitStr), - <<I:Size>> = BitStr, - (-128 + B) bsl bit_size(BitStr) bor I. - -decpint(Bin) -> - Size = bit_size(Bin), - <<Int:Size>> = Bin, - Int. - %% X.691:10.9 Encoding of a length determinant %%encode_small_length(undefined,Len) -> % null means no UpperBound %% encode_small_number(Len). @@ -779,18 +647,6 @@ encode_small_length(Len) -> [1,encode_length(undefined,Len)]. -decode_small_length(Buffer) -> - case getbit(Buffer) of - {0,Remain} -> - {Bits,Remain2} = getbits(Remain,6), - {Bits+1,Remain2}; - {1,Remain} -> - decode_length(Remain,undefined) - end. - -decode_length(Buffer) -> - decode_length(Buffer,undefined). - decode_length(Buffer,undefined) -> % un-constrained case align(Buffer) of <<0:1,Oct:7,Rest/binary>> -> @@ -833,38 +689,6 @@ decode_length(Buffer,SingleValue) when is_integer(SingleValue) -> {SingleValue,Buffer}. - % X.691:11 -decode_boolean(Buffer) -> %when record(Buffer,buffer) - case getbit(Buffer) of - {1,Remain} -> {true,Remain}; - {0,Remain} -> {false,Remain} - end. - - -%% ENUMERATED with extension marker -decode_enumerated(Buffer,C,{Ntup1,Ntup2}) when is_tuple(Ntup1), is_tuple(Ntup2) -> - {Ext,Buffer2} = getext(Buffer), - case Ext of - 0 -> % not an extension value - {Val,Buffer3} = decode_integer(Buffer2,C), - case catch (element(Val+1,Ntup1)) of - NewVal when is_atom(NewVal) -> {NewVal,Buffer3}; - _Error -> exit({error,{asn1,{decode_enumerated,{Val,[Ntup1,Ntup2]}}}}) - end; - 1 -> % this an extension value - {Val,Buffer3} = decode_small_number(Buffer2), - case catch (element(Val+1,Ntup2)) of - NewVal when is_atom(NewVal) -> {NewVal,Buffer3}; - _ -> {{asn1_enum,Val},Buffer3} - end - end; - -decode_enumerated(Buffer,C,NamedNumberTup) when is_tuple(NamedNumberTup) -> - {Val,Buffer2} = decode_integer(Buffer,C), - case catch (element(Val+1,NamedNumberTup)) of - NewVal when is_atom(NewVal) -> {NewVal,Buffer2}; - _Error -> exit({error,{asn1,{decode_enumerated,{Val,NamedNumberTup}}}}) - end. %%=============================================================================== %%=============================================================================== @@ -1315,49 +1139,74 @@ encode_octet_string(SZ={_,_},false,Val) -> % [encode_length(SZ,length(Val)),align, % {octets,Val}]; Len = length(Val), - [encode_length(SZ,Len),2, - octets_to_complete(Len,Val)]; + try + [encode_length(SZ,Len),2, + octets_to_complete(Len,Val)] + catch + exit:{error,{asn1,{encode_length,_}}} -> + encode_fragmented_octet_string(Val) + end; encode_octet_string(SZ,false,Val) when is_list(SZ) -> Len = length(Val), - [encode_length({hd(SZ),lists:max(SZ)},Len),2, - octets_to_complete(Len,Val)]; + try + [encode_length({hd(SZ),lists:max(SZ)},Len),2, + octets_to_complete(Len,Val)] + catch + exit:{error,{asn1,{encode_length,_}}} -> + encode_fragmented_octet_string(Val) + end; +encode_octet_string(Sv,false,Val) when is_integer(Sv) -> + encode_fragmented_octet_string(Val); encode_octet_string(no,false,Val) -> Len = length(Val), - [encode_length(undefined,Len),2, - octets_to_complete(Len,Val)]; + try + [encode_length(undefined,Len),2, + octets_to_complete(Len,Val)] + catch + exit:{error,{asn1,{encode_length,_}}} -> + encode_fragmented_octet_string(Val) + end; encode_octet_string(C,_,_) -> exit({error,{not_implemented,C}}). - -decode_octet_string(Bytes,Range) -> - decode_octet_string(Bytes,Range,false). - -decode_octet_string(<<B1,Bytes/bitstring>>,1,false) -> -%% {B1,Bytes2} = getbits(Bytes,8), - {[B1],Bytes}; -decode_octet_string(<<B1,B2,Bytes/bitstring>>,2,false) -> -%% {Bs,Bytes2}= getbits(Bytes,16), -%% {binary_to_list(<<Bs:16>>),Bytes2}; - {[B1,B2],Bytes}; -decode_octet_string(Bytes,Sv,false) when is_integer(Sv),Sv=<65535 -> - %% Bytes2 = align(Bytes), - %% getoctets_as_list aligns buffer before it picks octets - getoctets_as_list(Bytes,Sv); -decode_octet_string(Bytes,Sv,false) when is_integer(Sv) -> - Bytes2 = align(Bytes), - decode_fragmented_octets(Bytes2,Sv); -decode_octet_string(Bytes,{Lb,Ub},false) -> - {Len,Bytes2} = decode_length(Bytes,{Lb,Ub}), -%% Bytes3 = align(Bytes2), - getoctets_as_list(Bytes2,Len); -decode_octet_string(Bytes,Sv,false) when is_list(Sv) -> - {Len,Bytes2} = decode_length(Bytes,{hd(Sv),lists:max(Sv)}), -%% Bytes3 = align(Bytes2), - getoctets_as_list(Bytes2,Len); -decode_octet_string(Bytes,no,false) -> - {Len,Bytes2} = decode_length(Bytes,undefined), -%% Bytes3 = align(Bytes2), - getoctets_as_list(Bytes2,Len). +encode_fragmented_octet_string(Val) -> + Bin = iolist_to_binary(Val), + efos_1(Bin). + +efos_1(<<B1:16#C000/binary,B2:16#4000/binary,T/binary>>) -> + [20,1,<<3:2,4:6>>, + octets_to_complete(16#C000, B1), + octets_to_complete(16#4000, B2)|efos_1(T)]; +efos_1(<<B:16#C000/binary,T/binary>>) -> + [20,1,<<3:2,3:6>>,octets_to_complete(16#C000, B)|efos_1(T)]; +efos_1(<<B:16#8000/binary,T/binary>>) -> + [20,1,<<3:2,2:6>>,octets_to_complete(16#8000, B)|efos_1(T)]; +efos_1(<<B:16#4000/binary,T/binary>>) -> + [20,1,<<3:2,1:6>>,octets_to_complete(16#4000, B)|efos_1(T)]; +efos_1(<<>>) -> + [20,1,0]; +efos_1(<<B/bitstring>>) -> + Len = byte_size(B), + [encode_length(undefined, Len),octets_to_complete(Len, B)]. + +decode_fragmented(SegSz0, Buf0, Unit) -> + SegSz = SegSz0 * Unit * ?'16K', + <<Res:SegSz/bitstring,Buf/bitstring>> = Buf0, + decode_fragmented_1(Buf, Unit, Res). + +decode_fragmented_1(<<0:1,N:7,Buf0/bitstring>>, Unit, Res) -> + Sz = N*Unit, + <<S:Sz/bitstring,Buf/bitstring>> = Buf0, + {<<Res/bitstring,S/bitstring>>,Buf}; +decode_fragmented_1(<<1:1,0:1,N:14,Buf0/bitstring>>, Unit, Res) -> + Sz = N*Unit, + <<S:Sz/bitstring,Buf/bitstring>> = Buf0, + {<<Res/bitstring,S/bitstring>>,Buf}; +decode_fragmented_1(<<1:1,1:1,SegSz0:6,Buf0/bitstring>>, Unit, Res0) -> + SegSz = SegSz0 * Unit * ?'16K', + <<Frag:SegSz/bitstring,Buf/bitstring>> = Buf0, + Res = <<Res0/bitstring,Frag/bitstring>>, + decode_fragmented_1(Buf, Unit, Res). %%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%% @@ -1545,16 +1394,6 @@ chars_decode2(Bytes,{Min,Max,CharInTab},NumBits,Len,Acc) -> chars_decode2(Bytes2,{Min,Max,CharInTab},NumBits,Len -1,[element(Char+1,CharInTab)|Acc]). - % X.691:17 -encode_null(_Val) -> []. % encodes to nothing -%encode_null({Name,Val}) when is_atom(Name) -> -% encode_null(Val). - -decode_null(Bytes) -> - {'NULL',Bytes}. - - - %%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%% %% encode_UTF8String(Val) -> CompleteList %% Val -> <<utf8encoded binary>> diff --git a/lib/asn1/src/asn1rt_uper_bin.erl b/lib/asn1/src/asn1rt_uper_bin.erl index abe178a69e..9410c3ef90 100644 --- a/lib/asn1/src/asn1rt_uper_bin.erl +++ b/lib/asn1/src/asn1rt_uper_bin.erl @@ -25,22 +25,20 @@ %%-compile(export_all). - -export([cindex/3, list_to_record/2]). - -export([setext/1, fixoptionals/3, +-export([decode_fragmented/3]). +-export([setext/1, fixoptionals/3, fixextensions/2, - getext/1, getextension/2, skipextensions/3, getbit/1, getchoice/3 ]). - -export([getoptionals2/2, set_choice/3, encode_integer/2, encode_integer/3 ]). - -export([decode_integer/2, decode_integer/3, encode_small_number/1, encode_boolean/1, - decode_boolean/1, encode_length/2, decode_length/1, decode_length/2, - encode_small_length/1, decode_small_length/1, + skipextensions/3, getbit/1, getchoice/3 ]). +-export([set_choice/3, encode_integer/2, encode_integer/3]). +-export([encode_small_number/1, encode_boolean/1, + encode_length/2, + encode_small_length/1, decode_compact_bit_string/3]). - -export([decode_enumerated/3, - encode_bit_string/3, decode_bit_string/3 ]). - -export([encode_octet_string/2, decode_octet_string/2, - encode_null/1, decode_null/1, - encode_relative_oid/1, decode_relative_oid/1, +-export([encode_bit_string/3, decode_bit_string/3 ]). +-export([encode_octet_string/2, + encode_relative_oid/1, decode_relative_oid/1, encode_object_identifier/1, decode_object_identifier/1, - encode_real/1, decode_real/1, + encode_real/1, decode_real/1, complete/1, complete_NFP/1]). @@ -65,19 +63,6 @@ -define('64K',65536). -cindex(Ix,Val,Cname) -> - case element(Ix,Val) of - {Cname,Val2} -> Val2; - X -> X - end. - -%% converts a list to a record if necessary -list_to_record(_Name,Tuple) when is_tuple(Tuple) -> - Tuple; -list_to_record(Name,List) when is_list(List) -> - list_to_tuple([Name|List]). - - %%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%% %% setext(true|false) -> CompleteList %% @@ -113,16 +98,6 @@ fixoptionals([Pos|Ot],Val,Acc) -> end. -getext(Bytes) when is_bitstring(Bytes) -> - getbit(Bytes). - -getextension(0, Bytes) -> - {{},Bytes}; -getextension(1, Bytes) -> - {Len,Bytes2} = decode_small_length(Bytes), - {Blist, Bytes3} = getbits_as_list(Len,Bytes2), - {list_to_tuple(Blist),Bytes3}. - fixextensions({ext,ExtPos,ExtNum},Val) -> case fixextensions(ExtPos,ExtNum+ExtPos,Val,0) of 0 -> []; @@ -165,11 +140,6 @@ getchoice(Bytes,NumChoices,0) -> decode_constrained_number(Bytes,{0,NumChoices-1}). -%%%%%%%%%%%%%%% -getoptionals2(Bytes,NumOpt) -> - getbits(Bytes,NumOpt). - - %% getbits_as_binary(Num,Bytes) -> {{Unused,BinBits},RestBytes}, %% Num = integer(), %% Bytes = list() | tuple(), @@ -289,33 +259,6 @@ decode_fragmented_bits(<<0:1,Len:7,BitStr/bitstring>>,C,Acc) -> exit({error,{asn1,{illegal_value,C,ResBitStr}}}) end. - -decode_fragmented_octets({0,Bin},C) -> - decode_fragmented_octets(Bin,C,[]). - -decode_fragmented_octets(<<3:2,Len:6,BitStr/bitstring>>,C,Acc) -> - FragLen = Len * ?'16K', - <<Value:FragLen/binary,Rest/bitstring>> = BitStr, - decode_fragmented_octets(Rest,C,[Value|Acc]); -decode_fragmented_octets(<<0:1,0:7,Bin/bitstring>>,C,Acc) -> - Octets = list_to_binary(lists:reverse(Acc)), - case C of - Int when is_integer(Int), C == size(Octets) -> - {Octets,Bin}; - Int when is_integer(Int) -> - exit({error,{asn1,{illegal_value,C,Octets}}}) - end; -decode_fragmented_octets(<<0:1,Len:7,BitStr/bitstring>>,C,Acc) -> - <<Value:Len/binary-unit:8,BitStr2/binary>> = BitStr, - BinOctets = list_to_binary(lists:reverse([Value|Acc])), - case C of - Int when is_integer(Int),size(BinOctets) == Int -> - {BinOctets,BitStr2}; - Int when is_integer(Int) -> - exit({error,{asn1,{illegal_value,C,BinOctets}}}) - end. - - %%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%% %% encode_open_type(Constraint, Value) -> CompleteList @@ -398,47 +341,6 @@ encode_integer1(C, Val) -> exit({error,{asn1,{illegal_value,VR,Val}}}) end. -decode_integer(Buffer,Range,NamedNumberList) -> - {Val,Buffer2} = decode_integer(Buffer,Range), - case lists:keysearch(Val,2,NamedNumberList) of - {value,{NewVal,_}} -> {NewVal,Buffer2}; - _ -> {Val,Buffer2} - end. - -decode_integer(Buffer,[{Rc,_Ec}]) when is_tuple(Rc) -> - {Ext,Buffer2} = getext(Buffer), - case Ext of - 0 -> decode_integer(Buffer2,[Rc]); %% Value in root of constraint - 1 -> decode_unconstrained_number(Buffer2) - end; -decode_integer(Buffer,undefined) -> - decode_unconstrained_number(Buffer); -decode_integer(Buffer,C) -> - case get_constraint(C,'SingleValue') of - V when is_integer(V) -> - {V,Buffer}; - V when is_list(V) -> - {Val,Buffer2} = decode_integer1(Buffer,C), - case lists:member(Val,V) of - true -> - {Val,Buffer2}; - _ -> - exit({error,{asn1,{illegal_value,Val}}}) - end; - _ -> - decode_integer1(Buffer,C) - end. - -decode_integer1(Buffer,C) -> - case VR = get_constraint(C,'ValueRange') of - no -> - decode_unconstrained_number(Buffer); - {Lb, 'MAX'} -> - decode_semi_constrained_number(Buffer,Lb); - {_,_} -> - decode_constrained_number(Buffer,VR) - end. - %% X.691:10.6 Encoding of a normally small non-negative whole number %% Use this for encoding of CHOICE index if there is an extension marker in %% the CHOICE @@ -455,7 +357,7 @@ decode_small_number(Bytes) -> 0 -> getbits(Bytes2,6); 1 -> - decode_semi_constrained_number(Bytes2,0) + decode_semi_constrained_number(Bytes2) end. %% X.691:10.7 Encoding of a semi-constrained whole number @@ -479,12 +381,10 @@ encode_semi_constrained_number(Lb,Val) -> [encode_length(undefined,Size),Bin] end. -decode_semi_constrained_number(Bytes,{Lb,_}) -> - decode_semi_constrained_number(Bytes,Lb); -decode_semi_constrained_number(Bytes,Lb) -> +decode_semi_constrained_number(Bytes) -> {Len,Bytes2} = decode_length(Bytes,undefined), {V,Bytes3} = getoctets(Bytes2,Len), - {V+Lb,Bytes3}. + {V,Bytes3}. encode_constrained_number(Range,{Name,Val}) when is_atom(Name) -> encode_constrained_number(Range,Val); @@ -566,23 +466,6 @@ enint(-1, [B1|T]) when B1 > 127 -> enint(N, Acc) -> enint(N bsr 8, [N band 16#ff|Acc]). -decode_unconstrained_number(Bytes) -> - {Len,Bytes2} = decode_length(Bytes,undefined), - {Ints,Bytes3} = getoctets_as_bin(Bytes2,Len), - {dec_integer(Ints),Bytes3}. - -dec_integer(Bin = <<0:1,_:7,_/bitstring>>) -> - decpint(Bin); -dec_integer(<<_:1,B:7,BitStr/bitstring>>) -> - Size = bit_size(BitStr), - <<I:Size>> = BitStr, - (-128 + B) bsl bit_size(BitStr) bor I. - -decpint(Bin) -> - Size = bit_size(Bin), - <<Int:Size>> = Bin, - Int. - %% X.691:10.9 Encoding of a length determinant %%encode_small_length(undefined,Len) -> % null means no UpperBound @@ -597,7 +480,7 @@ encode_length(undefined,Len) -> % un-constrained Len < 16384 -> <<2:2,Len:14>>; true -> % should be able to endode length >= 16384 - exit({error,{asn1,{encode_length,{nyi,above_16k}}}}) + error({error,{asn1,{encode_length,{nyi,above_16k}}}}) end; encode_length({0,'MAX'},Len) -> @@ -623,18 +506,6 @@ encode_small_length(Len) -> [<<1:1>>,encode_length(undefined,Len)]. -decode_small_length(Buffer) -> - case getbit(Buffer) of - {0,Remain} -> - {Bits,Remain2} = getbits(Remain,6), - {Bits+1,Remain2}; - {1,Remain} -> - decode_length(Remain,undefined) - end. - -decode_length(Buffer) -> - decode_length(Buffer,undefined). - %% un-constrained decode_length(<<0:1,Oct:7,Rest/bitstring>>,undefined) -> {Oct,Rest}; @@ -677,38 +548,6 @@ encode_boolean({Name,Val}) when is_atom(Name) -> encode_boolean(Val) -> exit({error,{asn1,{encode_boolean,Val}}}). -decode_boolean(Buffer) -> %when record(Buffer,buffer) - case getbit(Buffer) of - {1,Remain} -> {true,Remain}; - {0,Remain} -> {false,Remain} - end. - - -%% ENUMERATED with extension marker -decode_enumerated(Buffer,C,{Ntup1,Ntup2}) when is_tuple(Ntup1), is_tuple(Ntup2) -> - {Ext,Buffer2} = getext(Buffer), - case Ext of - 0 -> % not an extension value - {Val,Buffer3} = decode_integer(Buffer2,C), - case catch (element(Val+1,Ntup1)) of - NewVal when is_atom(NewVal) -> {NewVal,Buffer3}; - _Error -> exit({error,{asn1,{decode_enumerated,{Val,[Ntup1,Ntup2]}}}}) - end; - 1 -> % this an extension value - {Val,Buffer3} = decode_small_number(Buffer2), - case catch (element(Val+1,Ntup2)) of - NewVal when is_atom(NewVal) -> {NewVal,Buffer3}; - _ -> {{asn1_enum,Val},Buffer3} - end - end; - -decode_enumerated(Buffer,C,NamedNumberTup) when is_tuple(NamedNumberTup) -> - {Val,Buffer2} = decode_integer(Buffer,C), - case catch (element(Val+1,NamedNumberTup)) of - NewVal when is_atom(NewVal) -> {NewVal,Buffer2}; - _Error -> exit({error,{asn1,{decode_enumerated,{Val,NamedNumberTup}}}}) - end. - %%============================================================================ %%============================================================================ @@ -1067,36 +906,71 @@ encode_octet_string(C,Val) -> list_to_binary(Val); 2 -> list_to_binary(Val); - Sv when Sv =<65535, Sv == length(Val) -> % fixed length - list_to_binary(Val); - VR = {_,_} -> - [encode_length(VR,length(Val)),list_to_binary(Val)]; + {_,_}=VR -> + try + [encode_length(VR, length(Val)),list_to_binary(Val)] + catch + error:{error,{asn1,{encode_length,_}}} -> + encode_fragmented_octet_string(Val) + end; + Sv when is_integer(Sv), Sv =:= length(Val) -> % fixed length + if + Sv =< 65535 -> + list_to_binary(Val); + true -> + encode_fragmented_octet_string(Val) + end; Sv when is_list(Sv) -> - [encode_length({hd(Sv),lists:max(Sv)},length(Val)),list_to_binary(Val)]; + try + [encode_length({hd(Sv),lists:max(Sv)}, + length(Val)),list_to_binary(Val)] + catch + error:{error,{asn1,{encode_length,_}}} -> + encode_fragmented_octet_string(Val) + end; no -> - [encode_length(undefined,length(Val)),list_to_binary(Val)] + try + [encode_length(undefined, length(Val)),list_to_binary(Val)] + catch + error:{error,{asn1,{encode_length,_}}} -> + encode_fragmented_octet_string(Val) + end end. -decode_octet_string(Bytes,C) -> - decode_octet_string1(Bytes,get_constraint(C,'SizeConstraint')). -decode_octet_string1(<<B1,Bytes/bitstring>>,1) -> - {[B1],Bytes}; -decode_octet_string1(<<B1,B2,Bytes/bitstring>>,2) -> - {[B1,B2],Bytes}; -decode_octet_string1(Bytes,Sv) when is_integer(Sv),Sv=<65535 -> - getoctets_as_list(Bytes,Sv); -decode_octet_string1(Bytes,Sv) when is_integer(Sv) -> - decode_fragmented_octets(Bytes,Sv); -decode_octet_string1(Bytes,{Lb,Ub}) -> - {Len,Bytes2} = decode_length(Bytes,{Lb,Ub}), - getoctets_as_list(Bytes2,Len); -decode_octet_string1(Bytes,Sv) when is_list(Sv) -> - {Len,Bytes2} = decode_length(Bytes,{hd(Sv),lists:max(Sv)}), - getoctets_as_list(Bytes2,Len); -decode_octet_string1(Bytes,no) -> - {Len,Bytes2} = decode_length(Bytes,undefined), - getoctets_as_list(Bytes2,Len). - +encode_fragmented_octet_string(Val) -> + Bin = list_to_binary(Val), + efos_1(Bin). + +efos_1(<<B:16#10000/binary,T/binary>>) -> + [<<3:2,4:6>>,B|efos_1(T)]; +efos_1(<<B:16#C000/binary,T/binary>>) -> + [<<3:2,3:6>>,B|efos_1(T)]; +efos_1(<<B:16#8000/binary,T/binary>>) -> + [<<3:2,2:6>>,B|efos_1(T)]; +efos_1(<<B:16#4000/binary,T/binary>>) -> + [<<3:2,1:6>>,B|efos_1(T)]; +efos_1(<<B/bitstring>>) -> + Len = byte_size(B), + [encode_length(undefined, Len),B]. + +decode_fragmented(SegSz0, Buf0, Unit) -> + SegSz = SegSz0 * Unit * ?'16K', + <<Res:SegSz/bitstring,Buf/bitstring>> = Buf0, + decode_fragmented_1(Buf, Unit, Res). + +decode_fragmented_1(<<0:1,N:7,Buf0/bitstring>>, Unit, Res) -> + Sz = N*Unit, + <<S:Sz/bitstring,Buf/bitstring>> = Buf0, + {<<Res/bitstring,S/bitstring>>,Buf}; +decode_fragmented_1(<<1:1,0:1,N:14,Buf0/bitstring>>, Unit, Res) -> + Sz = N*Unit, + <<S:Sz/bitstring,Buf/bitstring>> = Buf0, + {<<Res/bitstring,S/bitstring>>,Buf}; +decode_fragmented_1(<<1:1,1:1,SegSz0:6,Buf0/bitstring>>, Unit, Res0) -> + SegSz = SegSz0 * Unit * ?'16K', + <<Frag:SegSz/bitstring,Buf/bitstring>> = Buf0, + Res = <<Res0/bitstring,Frag/bitstring>>, + decode_fragmented_1(Buf, Unit, Res). %%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%% @@ -1441,12 +1315,6 @@ decode_UTF8String(Bytes) -> getoctets_as_bin(Bytes2,Len). - % X.691:17 -encode_null(_) -> []. % encodes to nothing - -decode_null(Bytes) -> - {'NULL',Bytes}. - %%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%% %% encode_object_identifier(Val) -> CompleteList %% encode_object_identifier({Name,Val}) -> CompleteList diff --git a/lib/asn1/test/Makefile b/lib/asn1/test/Makefile index 6e6374baf1..1794d6bb71 100644 --- a/lib/asn1/test/Makefile +++ b/lib/asn1/test/Makefile @@ -83,6 +83,7 @@ MODULES= \ testInfObj \ testParameterizedInfObj \ testMergeCompile \ + testMultipleLevels \ testDeepTConstr \ testTimer \ testMegaco \ diff --git a/lib/asn1/test/asn1_SUITE.erl b/lib/asn1/test/asn1_SUITE.erl index 79c7bf1476..38ec72c473 100644 --- a/lib/asn1/test/asn1_SUITE.erl +++ b/lib/asn1/test/asn1_SUITE.erl @@ -21,27 +21,17 @@ -module(asn1_SUITE). -define(only_per(Func), - if Rule == per orelse Rule == per_bin -> Func; - true -> ok + if Rule =:= per -> Func; + true -> ok end). -define(only_ber(Func), - if Rule == ber orelse Rule == ber_bin orelse Rule == ber_bin_v2 -> Func; - true -> ok + if Rule =:= ber -> Func; + true -> ok end). -define(only_uper(Func), case Rule of - uper_bin -> Func; - _ -> ok - end). --define(only_per_nif(Func), - case {Rule, lists:member(optimize, Opts)} of - {per_bin, true} -> Func; - _ -> ok - end). --define(only_ber_nif(Func), - case {Rule, lists:member(nif, Opts)} of - {ber_bin_v2, true} -> Func; - _ -> ok + uper -> Func; + _ -> ok end). -compile(export_all). @@ -55,7 +45,8 @@ suite() -> [{ct_hooks, [ts_install_cth]}]. all() -> - [{group, parallel}, + [{group, compile}, + {group, parallel}, {group, app_test}, {group, appup_test}, @@ -76,16 +67,14 @@ groups() -> {ber, parallel([]), [ber_choiceinseq, % Uses 'SOpttest' - {group, [], [ber_optional, - ber_optional_keyed_list]}]}, + ber_optional]}, {app_test, [], [{asn1_app_test, all}]}, {appup_test, [], [{asn1_appup_test, all}]}, {parallel, parallel([]), - [{group, compile}, - {group, ber}, + [{group, ber}, % Uses 'P-Record', 'Constraints', 'MEDIA-GATEWAY-CONTROL'... {group, [], [parse, test_driver_load, @@ -116,6 +105,7 @@ groups() -> testChoTypeRefPrim, testChoTypeRefSeq, testChoTypeRefSet, + testMultipleLevels, testDef, testOpt, testSeqDefault, @@ -203,13 +193,8 @@ groups() -> {performance, [], [testTimer_ber, - testTimer_ber_bin, - testTimer_ber_bin_opt, - testTimer_ber_bin_opt_driver, testTimer_per, - testTimer_per_bin, - testTimer_per_bin_opt, - testTimer_uper_bin, + testTimer_uper, smp]}]. parallel(Options) -> @@ -257,14 +242,8 @@ end_per_testcase(_Func, Config) -> test(Config, TestF) -> test(Config, TestF, [per, - per_bin, - {per_bin, [optimize]}, - uper_bin, - ber, - ber_bin, - ber_bin_v2, - % TODO: {ber_bin_v2, [optimize, nif]} ? - {ber_bin_v2, [nif]}]). + uper, + ber]). test(Config, TestF, Rules) -> Fun = fun(C, R, O) -> @@ -344,10 +323,10 @@ testCompactBitString(Config, Rule, Opts) -> testCompactBitString:compact_bit_string(Rule), ?only_uper(testCompactBitString:bit_string_unnamed(Rule)), ?only_per(testCompactBitString:bit_string_unnamed(Rule)), - ?only_per_nif(testCompactBitString:ticket_7734(Rule)), - ?only_per_nif(asn1_test_lib:compile("Constraints", Config, + ?only_per(testCompactBitString:ticket_7734(Rule)), + ?only_per(asn1_test_lib:compile("Constraints", Config, [Rule, compact_bit_string|Opts])), - ?only_per_nif(testCompactBitString:otp_4869(Rule)). + ?only_per(testCompactBitString:otp_4869(Rule)). testPrimStrings(Config) -> test(Config, fun testPrimStrings/3). testPrimStrings(Config, Rule, Opts) -> @@ -371,10 +350,10 @@ testPrimExternal(Config, Rule, Opts) -> asn1_test_lib:compile_all(["External", "PrimExternal"], Config, [Rule|Opts]), testPrimExternal:external(Rule), - ?only_ber_nif(asn1_test_lib:compile_all(["PrimStrings", "BitStr"], Config, + ?only_ber(asn1_test_lib:compile_all(["PrimStrings", "BitStr"], Config, [Rule|Opts])), - ?only_ber_nif(testPrimStrings_cases(Rule)), - ?only_ber_nif(testPrimStrings:more_strings(Rule)). + ?only_ber(testPrimStrings_cases(Rule)), + ?only_ber(testPrimStrings:more_strings(Rule)). testChoPrim(Config) -> test(Config, fun testChoPrim/3). testChoPrim(Config, Rule, Opts) -> @@ -399,7 +378,7 @@ testChoOptional(Config, Rule, Opts) -> testChoOptionalImplicitTag(Config) -> test(Config, fun testChoOptionalImplicitTag/3, - [ber, ber_bin, ber_bin_v2]). + [ber]). testChoOptionalImplicitTag(Config, Rule, Opts) -> %% Only meaningful for ber & co asn1_test_lib:compile("ChoOptionalImplicitTag", Config, [Rule|Opts]), @@ -430,6 +409,11 @@ testChoTypeRefSet(Config, Rule, Opts) -> asn1_test_lib:compile("ChoTypeRefSet", Config, [Rule|Opts]), testChoTypeRefSet:set(Rule). +testMultipleLevels(Config) -> test(Config, fun testMultipleLevels/3). +testMultipleLevels(Config, Rule, Opts) -> + asn1_test_lib:compile("MultipleLevels", Config, [Rule|Opts]), + testMultipleLevels:main(Rule). + testDef(Config) -> test(Config, fun testDef/3). testDef(Config, Rule, Opts) -> asn1_test_lib:compile("Def", Config, [Rule|Opts]), @@ -515,8 +499,7 @@ testSeqOfCho(Config, Rule, Opts) -> testSeqOfCho:main(Rule). testSeqOfIndefinite(Config) -> - test(Config, fun testSeqOfIndefinite/3, - [ber, ber_bin, ber_bin_v2, {ber_bin_v2, [nif]}]). + test(Config, fun testSeqOfIndefinite/3, [ber]). testSeqOfIndefinite(Config, Rule, Opts) -> Files = ["Mvrasn-Constants-1", "Mvrasn-DataTypes-1", "Mvrasn-21-4", "Mvrasn-20-4", "Mvrasn-19-4", "Mvrasn-18-4", "Mvrasn-17-4", @@ -632,13 +615,12 @@ c_syntax(Config) -> "SeqBadComma"]]. c_string(Config) -> - test(Config, fun c_string/3, [per, per_bin, ber, ber_bin, ber_bin_v2]). + test(Config, fun c_string/3, [per, ber]). c_string(Config, Rule, Opts) -> asn1_test_lib:compile("String", Config, [Rule|Opts]). c_implicit_before_choice(Config) -> - test(Config, fun c_implicit_before_choice/3, - [ber, ber_bin, ber_bin_v2]). + test(Config, fun c_implicit_before_choice/3, [ber]). c_implicit_before_choice(Config, Rule, Opts) -> DataDir = ?config(data_dir, Config), CaseDir = ?config(case_dir, Config), @@ -649,12 +631,13 @@ parse(Config) -> [asn1_test_lib:compile(M, Config, [abs]) || M <- test_modules()]. per(Config) -> - test(Config, fun per/3, [per, per_bin, {per_bin, [optimize]}]). + test(Config, fun per/3, [per,uper]). per(Config, Rule, Opts) -> [module_test(M, Config, Rule, Opts) || M <- per_modules()]. ber_other(Config) -> - test(Config, fun ber_other/3, [ber, ber_bin, ber_bin_v2]). + test(Config, fun ber_other/3, [ber]). + ber_other(Config, Rule, Opts) -> [module_test(M, Config, Rule, Opts) || M <- ber_modules()]. @@ -669,12 +652,12 @@ module_test(M, Config, Rule, Opts) -> ber_choiceinseq(Config) -> - test(Config, fun ber_choiceinseq/3, [ber, ber_bin, ber_bin_v2]). + test(Config, fun ber_choiceinseq/3, [ber]). ber_choiceinseq(Config, Rule, Opts) -> asn1_test_lib:compile("ChoiceInSeq", Config, [Rule|Opts]). ber_optional(Config) -> - test(Config, fun ber_optional/3, [ber, ber_bin, ber_bin_v2]). + test(Config, fun ber_optional/3, [ber]). ber_optional(Config, Rule, Opts) -> asn1_test_lib:compile("SOpttest", Config, [Rule|Opts]), V = {'S', {'A', 10, asn1_NOVALUE, asn1_NOVALUE}, @@ -685,21 +668,6 @@ ber_optional(Config, Rule, Opts) -> V2 = asn1_wrapper:decode('SOpttest', 'S', Bytes), V = element(2, V2). -ber_optional_keyed_list(Config) -> - test(Config, fun ber_optional_keyed_list/3, [ber, ber_bin]). -ber_optional_keyed_list(Config, Rule, Opts) -> - asn1_test_lib:compile("SOpttest", Config, [Rule, keyed_list|Opts]), - Vrecord = {'S', {'A', 10, asn1_NOVALUE, asn1_NOVALUE}, - {'B', asn1_NOVALUE, asn1_NOVALUE, asn1_NOVALUE}, - {'C', asn1_NOVALUE, 111, asn1_NOVALUE}}, - V = [{a, [{scriptKey, 10}]}, - {b, []}, - {c, [{callingPartysCategory, 111}]}], - {ok, B} = asn1_wrapper:encode('SOpttest', 'S', V), - Bytes = lists:flatten(B), - V2 = asn1_wrapper:decode('SOpttest', 'S', Bytes), - Vrecord = element(2, V2). - %% records used by test-case default -record('Def1', {bool0, bool1 = asn1_DEFAULT, @@ -734,7 +702,7 @@ value_bad_enum_test(Config) -> end. constructed(Config) -> - test(Config, fun constructed/3, [ber, ber_bin, ber_bin_v2]). + test(Config, fun constructed/3, [ber]). constructed(Config, Rule, Opts) -> asn1_test_lib:compile("Constructed", Config, [Rule|Opts]), {ok, B} = asn1_wrapper:encode('Constructed', 'S', {'S', false}), @@ -745,7 +713,7 @@ constructed(Config, Rule, Opts) -> [136, 1, 10] = lists:flatten(B2). ber_decode_error(Config) -> - test(Config, fun ber_decode_error/3, [ber, ber_bin, ber_bin_v2]). + test(Config, fun ber_decode_error/3, [ber]). ber_decode_error(Config, Rule, Opts) -> asn1_test_lib:compile("Constructed", Config, [Rule|Opts]), ber_decode_error:run(Opts). @@ -758,14 +726,14 @@ h323test(Config, Rule, Opts) -> h323test:run(Rule). per_GeneralString(Config) -> - test(Config, fun per_GeneralString/3, [per, per_bin]). + test(Config, fun per_GeneralString/3, [per]). per_GeneralString(Config, Rule, Opts) -> asn1_test_lib:compile("MULTIMEDIA-SYSTEM-CONTROL", Config, [Rule|Opts]), UI = [109, 64, 1, 57], {ok, _V} = asn1_wrapper:decode('MULTIMEDIA-SYSTEM-CONTROL', 'MultimediaSystemControlMessage', UI). -per_open_type(Config) -> test(Config, fun per_open_type/3, [per, per_bin]). +per_open_type(Config) -> test(Config, fun per_open_type/3, [per]). per_open_type(Config, Rule, Opts) -> asn1_test_lib:compile("OpenType", Config, [Rule|Opts]), {ok, _} = asn1ct:test('OpenType', 'Ot', {'Stype', 10, true}). @@ -778,24 +746,24 @@ testConstraints(Config, Rule, Opts) -> testSeqIndefinite(Config) -> - test(Config, fun testSeqIndefinite/3, [ber, ber_bin, ber_bin_v2, - {ber_bin_v2, [nif]}]). + test(Config, fun testSeqIndefinite/3, [ber]). + testSeqIndefinite(Config, Rule, Opts) -> asn1_test_lib:compile("SeqSetIndefinite", Config, [Rule|Opts]), testSeqIndefinite:main(Rule). testSetIndefinite(Config) -> - test(Config, fun testSetIndefinite/3, [ber, ber_bin, ber_bin_v2, - {ber_bin_v2, [nif]}]). + test(Config, fun testSetIndefinite/3, [ber]). + testSetIndefinite(Config, Rule, Opts) -> asn1_test_lib:compile("SeqSetIndefinite", Config, [Rule|Opts]), testSetIndefinite:main(Rule). testChoiceIndefinite(Config) -> - test(Config, fun testChoiceIndefinite/3, [ber, ber_bin, ber_bin_v2, - {ber_bin_v2, [nif]}]). + test(Config, fun testChoiceIndefinite/3, [ber]). + testChoiceIndefinite(Config, Rule, Opts) -> asn1_test_lib:compile("ChoiceIndef", Config, [Rule|Opts]), testChoiceIndefinite:main(Rule). @@ -857,7 +825,7 @@ testExport(Config) -> end. testImport(Config) -> - test(Config, fun testImport/3, [ber, ber_bin, ber_bin_v2]). + test(Config, fun testImport/3, [ber]). testImport(Config, Rule, Opts) -> {error, _} = asn1ct:compile(filename:join(?config(data_dir, Config), "ImportsFrom"), @@ -906,8 +874,7 @@ duplicate_tags(Config) -> {skip, "Runs in asn1_SUITE only"} end. -rtUI(Config) -> test(Config, fun rtUI/3, [per, per_bin, ber, - ber_bin, ber_bin_v2]). +rtUI(Config) -> test(Config, fun rtUI/3, [per,ber]). rtUI(Config, Rule, Opts) -> asn1_test_lib:compile("Prim", Config, [Rule|Opts]), {ok, _} = asn1rt:info('Prim'). @@ -923,19 +890,19 @@ testINSTANCE_OF(Config, Rule, Opts) -> testINSTANCE_OF:main(Rule). testTCAP(Config) -> - test(Config, fun testTCAP/3, - [ber, ber_bin, ber_bin_v2, {ber_bin_v2, [nif]}]). + test(Config, fun testTCAP/3, [ber]). testTCAP(Config, Rule, Opts) -> testTCAP:compile(Config, [Rule|Opts]), testTCAP:test(Rule, Config), case Rule of - ber_bin_v2 -> testTCAP:compile_asn1config(Config, [Rule, asn1config]), - testTCAP:test_asn1config(); - _ -> ok + ber -> + testTCAP:compile_asn1config(Config, [Rule, asn1config]), + testTCAP:test_asn1config(); + _ -> ok end. testDER(Config) -> - test(Config, fun testDER/3, [ber, ber_bin, ber_bin_v2]). + test(Config, fun testDER/3, [ber]). testDER(Config, Rule, Opts) -> asn1_test_lib:compile("DERSpec", Config, [Rule, der|Opts]), testDER:test(), @@ -945,7 +912,7 @@ testDER(Config, Rule, Opts) -> testSeqSetDefaultVal:main(Rule). specialized_decodes(Config) -> - test(Config, fun specialized_decodes/3, [ber_bin_v2]). + test(Config, fun specialized_decodes/3, [ber]). specialized_decodes(Config, Rule, Opts) -> asn1_test_lib:compile_all(["PartialDecSeq.asn", "PartialDecSeq2.asn", @@ -953,13 +920,12 @@ specialized_decodes(Config, Rule, Opts) -> "PartialDecMyHTTP.asn", "MEDIA-GATEWAY-CONTROL.asn", "P-Record"], - Config, [Rule, optimize, asn1config|Opts]), + Config, [Rule, asn1config|Opts]), test_partial_incomplete_decode:test(Config), test_selective_decode:test(). special_decode_performance(Config) -> - test(Config, fun special_decode_performance/3, - [{ber_bin, [optimize]}, {ber_bin_v2, [optimize, nif]}]). + test(Config, fun special_decode_performance/3, [ber]). special_decode_performance(Config, Rule, Opts) -> Files = ["MEDIA-GATEWAY-CONTROL", "PartialDecSeq"], asn1_test_lib:compile_all(Files, Config, [Rule, asn1config|Opts]), @@ -967,19 +933,19 @@ special_decode_performance(Config, Rule, Opts) -> test_driver_load(Config) -> - test(Config, fun test_driver_load/3, [{per_bin, [optimize]}]). + test(Config, fun test_driver_load/3, [per]). test_driver_load(Config, Rule, Opts) -> asn1_test_lib:compile("P-Record", Config, [Rule|Opts]), test_driver_load:test(5). test_ParamTypeInfObj(Config) -> - asn1_test_lib:compile("IN-CS-1-Datatypes", Config, [ber_bin]). + asn1_test_lib:compile("IN-CS-1-Datatypes", Config, [ber]). test_WS_ParamClass(Config) -> - asn1_test_lib:compile("InformationFramework", Config, [ber_bin]). + asn1_test_lib:compile("InformationFramework", Config, [ber]). test_Defed_ObjectIdentifier(Config) -> - asn1_test_lib:compile("UsefulDefinitions", Config, [ber_bin]). + asn1_test_lib:compile("UsefulDefinitions", Config, [ber]). testSelectionType(Config) -> test(Config, fun testSelectionType/3). testSelectionType(Config, Rule, Opts) -> @@ -987,7 +953,7 @@ testSelectionType(Config, Rule, Opts) -> {ok, _} = testSelectionTypes:test(). testSSLspecs(Config) -> - test(Config, fun testSSLspecs/3, [ber, ber_bin, ber_bin_v2, {ber_bin_v2, [optimize]}]). + test(Config, fun testSSLspecs/3, [ber]). testSSLspecs(Config, Rule, Opts) -> ok = testSSLspecs:compile(Config, [Rule, compact_bit_string, der|Opts]), @@ -1011,12 +977,12 @@ test_undecoded_rest(Config, Rule, Opts) -> ok = test_undecoded_rest:test([], Config), asn1_test_lib:compile("P-Record", Config, [Rule,undec_rest|Opts]), case Rule of - ber_bin_v2 -> ok; + ber -> ok; _ -> test_undecoded_rest:test(undec_rest, Config) end. test_inline(Config) -> - test(Config, fun test_inline/3, [ber, ber_bin, ber_bin_v2]). + test(Config, fun test_inline/3, [ber]). test_inline(Config, Rule, Opts) -> case code:which(asn1ct) of cover_compiled -> @@ -1029,12 +995,11 @@ test_inline(Config, Rule, Opts) -> end. testTcapsystem(Config) -> - test(Config, fun testTcapsystem/3, [ber, ber_bin, ber_bin_v2]). + test(Config, fun testTcapsystem/3, [ber]). testTcapsystem(Config, Rule, Opts) -> testTcapsystem:compile(Config, [Rule|Opts]). -testNBAPsystem(Config) -> test(Config, fun testNBAPsystem/3, - [per, per_bin, {per_bin, [optimize]}]). +testNBAPsystem(Config) -> test(Config, fun testNBAPsystem/3, [per]). testNBAPsystem(Config, Rule, Opts) -> testNBAPsystem:compile(Config, [Rule|Opts]), testNBAPsystem:test(Rule, Config). @@ -1068,20 +1033,19 @@ test_modified_x420(Config) -> testX420() -> [{timetrap,{minutes,90}}]. testX420(Config) -> - test(Config, fun testX420/3, [ber, ber_bin, ber_bin_v2]). + test(Config, fun testX420/3, [ber]). testX420(Config, Rule, Opts) -> testX420:compile(Rule, [der|Opts], Config), ok = testX420:ticket7759(Rule, Config), testX420:compile(Rule, Opts, Config). test_x691(Config) -> - test(Config, fun test_x691/3, - [per, per_bin, uper_bin, {per_bin, [optimize]}]). + test(Config, fun test_x691/3, [per, uper]). test_x691(Config, Rule, Opts) -> Files = ["P-RecordA1", "P-RecordA2", "P-RecordA3"], asn1_test_lib:compile_all(Files, Config, [Rule|Opts]), test_x691:cases(Rule, case Rule of - uper_bin -> unaligned; + uper -> unaligned; _ -> aligned end), asn1_test_lib:ticket_7708(Config, []), @@ -1092,8 +1056,7 @@ ticket_6143(Config) -> testExtensionAdditionGroup(Config) -> %% FIXME problems with automatic tags [ber_bin], [ber_bin, optimize] - test(Config, fun testExtensionAdditionGroup/3, - [per_bin, {per_bin, [optimize]}, uper_bin]). + test(Config, fun testExtensionAdditionGroup/3, [per, uper]). testExtensionAdditionGroup(Config, Rule, Opts) -> asn1_test_lib:compile("Extension-Addition-Group", Config, [Rule|Opts]), asn1_test_lib:compile_erlang("extensionAdditionGroup", Config, @@ -1182,46 +1145,17 @@ timer_compile(Config, Rule, Opts) -> asn1_test_lib:compile_all(["H235-SECURITY-MESSAGES", "H323-MESSAGES"], Config, [Rule|Opts]). -testTimer_ber(suite) -> []; testTimer_ber(Config) -> timer_compile(Config,ber,[]), testTimer:go(Config,ber). -testTimer_ber_bin(suite) -> []; -testTimer_ber_bin(Config) -> - timer_compile(Config,ber_bin,[]), - testTimer:go(Config,ber_bin). - -testTimer_ber_bin_opt(suite) -> []; -testTimer_ber_bin_opt(Config) -> - timer_compile(Config,ber_bin,[optimize]), - testTimer:go(Config,ber_bin). - -testTimer_ber_bin_opt_driver(suite) -> []; -testTimer_ber_bin_opt_driver(Config) -> - timer_compile(Config,ber_bin,[optimize,driver]), - testTimer:go(Config,ber_bin). - -testTimer_per(suite) -> []; testTimer_per(Config) -> timer_compile(Config,per,[]), testTimer:go(Config,per). -testTimer_per_bin(suite) -> []; -testTimer_per_bin(Config) -> - timer_compile(Config,per_bin,[]), - testTimer:go(Config,per_bin). - -testTimer_per_bin_opt(suite) -> []; -testTimer_per_bin_opt(Config) -> - timer_compile(Config,per_bin,[optimize]), - testTimer:go(Config,per_bin). - - -testTimer_uper_bin(suite) -> []; -testTimer_uper_bin(Config) -> - timer_compile(Config,uper_bin,[]), - {comment,_} = testTimer:go(Config,uper_bin). +testTimer_uper(Config) -> + timer_compile(Config,uper,[]), + {comment,_} = testTimer:go(Config,uper). %% Test of multiple-line comment, OTP-8043 testComment(suite) -> []; @@ -1264,11 +1198,11 @@ testName2Number(Config) -> ok. ticket_7407(Config) -> - asn1_test_lib:compile("EUTRA-extract-7407", Config, [uper_bin]), + asn1_test_lib:compile("EUTRA-extract-7407", Config, [uper]), asn1_test_lib:ticket_7407_code(true), asn1_test_lib:compile("EUTRA-extract-7407", Config, - [uper_bin, no_final_padding]), + [uper, no_final_padding]), asn1_test_lib:ticket_7407_code(false). smp(suite) -> []; @@ -1279,7 +1213,7 @@ smp(Config) -> io:format("smp starting ~p workers\n",[NumOfProcs]), Msg = {initiatingMessage, testNBAPsystem:cell_setup_req_msg()}, - ok = testNBAPsystem:compile(Config, [per_bin, optimize]), + ok = testNBAPsystem:compile(Config, [per]), enc_dec(NumOfProcs,Msg,2), @@ -1288,7 +1222,7 @@ smp(Config) -> {Time1,ok} = timer:tc(?MODULE,enc_dec,[NumOfProcs,Msg, N]), {Time1S,ok} = timer:tc(?MODULE,enc_dec,[1, Msg, NumOfProcs * N]), - ok = testNBAPsystem:compile(Config, [ber_bin, optimize, nif]), + ok = testNBAPsystem:compile(Config, [ber]), {Time3,ok} = timer:tc(?MODULE,enc_dec,[NumOfProcs,Msg, N]), {Time3S,ok} = timer:tc(?MODULE,enc_dec,[1, Msg, NumOfProcs * N]), @@ -1309,10 +1243,8 @@ per_performance(Config) -> file:make_dir(NifDir),file:make_dir(ErlDir), Msg = {initiatingMessage, testNBAPsystem:cell_setup_req_msg()}, - ok = testNBAPsystem:compile([{priv_dir,NifDir}|Config], - [per_bin, optimize]), - ok = testNBAPsystem:compile([{priv_dir,ErlDir}|Config], - [per_bin]), + ok = testNBAPsystem:compile([{priv_dir,NifDir}|Config], [per]), + ok = testNBAPsystem:compile([{priv_dir,ErlDir}|Config], [per]), Modules = ['NBAP-CommonDataTypes', 'NBAP-Constants', @@ -1350,7 +1282,7 @@ per_performance(Config) -> ber_performance(Config) -> Msg = {initiatingMessage, testNBAPsystem:cell_setup_req_msg()}, - ok = testNBAPsystem:compile(Config, [ber_bin, optimize, nif]), + ok = testNBAPsystem:compile(Config, [ber]), BerFun = fun() -> @@ -1524,7 +1456,7 @@ pforeach(_Fun,[],[]) -> -record('Iu-ReleaseCommand',{first,second}). ticket7904(Config) -> - asn1_test_lib:compile("RANAPextract1", Config, [per_bin, optimize]), + asn1_test_lib:compile("RANAPextract1", Config, [per]), Val1 = #'InitiatingMessage'{procedureCode=1, criticality=ignore, diff --git a/lib/asn1/test/asn1_SUITE_data/MultipleLevels.asn b/lib/asn1/test/asn1_SUITE_data/MultipleLevels.asn new file mode 100644 index 0000000000..1824e1fa5a --- /dev/null +++ b/lib/asn1/test/asn1_SUITE_data/MultipleLevels.asn @@ -0,0 +1,19 @@ +MultipleLevels DEFINITIONS AUTOMATIC TAGS ::= + +BEGIN + +Top ::= SEQUENCE { + country CountryName, + region Name +} + +CountryName ::= CountryName0 + +CountryName0 ::= Name + +Name ::= CHOICE { + short PrintableString (SIZE (0..10)), + long PrintableString +} + +END diff --git a/lib/asn1/test/asn1_SUITE_data/PartialDecSeq.asn1config b/lib/asn1/test/asn1_SUITE_data/PartialDecSeq.asn1config index 19fa3c990e..d388f6cd02 100644 --- a/lib/asn1/test/asn1_SUITE_data/PartialDecSeq.asn1config +++ b/lib/asn1/test/asn1_SUITE_data/PartialDecSeq.asn1config @@ -17,6 +17,6 @@ {decode_D_incomplete,['D',[{a,undecoded}]]}, {decode_F_fb_exclusive2,['F',[{fb,[{b,parts},{d,[{da,parts}]}]}]]}, {decode_F_fb_exclusive3,['F',[{fb,[{b,parts},{d,[{da,parts},{dc,[{dcc,undecoded}]}]}]}]]}]}}. {module_name,'Seq.asn1'}. -{compile_options,[ber_bin,optimize,debug_info]}. +{compile_options,[ber,debug_info]}. {multifile_compile,['M1.asn','M2.asn']}. diff --git a/lib/asn1/test/asn1_SUITE_data/PrimStrings.asn1 b/lib/asn1/test/asn1_SUITE_data/PrimStrings.asn1 index d287840f30..7cb47e9792 100644 --- a/lib/asn1/test/asn1_SUITE_data/PrimStrings.asn1 +++ b/lib/asn1/test/asn1_SUITE_data/PrimStrings.asn1 @@ -55,6 +55,19 @@ BS1024 ::= BIT STRING (SIZE (1024)) OsExpCon ::= [60] EXPLICIT OCTET STRING OsExpPri ::= [PRIVATE 61] EXPLICIT OCTET STRING OsExpApp ::= [APPLICATION 62] EXPLICIT OCTET STRING + OsFrag ::= OCTET STRING (SIZE (0..100000)) + FixedOs65536 ::= OCTET STRING (SIZE (65536)) + FixedOs65537 ::= OCTET STRING (SIZE (65537)) + + OsAlignment ::= SEQUENCE { + b1 BOOLEAN, + s1 Os, + b2 BOOLEAN, + s2 OsFrag, + b3 BOOLEAN, + s3 FixedOs65536, + i INTEGER (0..63) + } Ns ::= NumericString NsCon ::= [70] NumericString diff --git a/lib/asn1/test/asn1_SUITE_data/testobj.erl b/lib/asn1/test/asn1_SUITE_data/testobj.erl index be7ceee7d1..80942f7e38 100644 --- a/lib/asn1/test/asn1_SUITE_data/testobj.erl +++ b/lib/asn1/test/asn1_SUITE_data/testobj.erl @@ -1420,24 +1420,7 @@ wrapper_encode(Module,Type,Value) -> Error end. -wrapper_decode(Module,Type,Bytes) -> - case Module:encoding_rule() of - ber -> - asn1rt:decode(Module,Type,Bytes); - ber_bin when binary(Bytes) -> - asn1rt:decode(Module,Type,Bytes); - ber_bin -> - asn1rt:decode(Module,Type,list_to_binary(Bytes)); - ber_bin_v2 when binary(Bytes) -> - asn1rt:decode(Module,Type,Bytes); - ber_bin_v2 -> - asn1rt:decode(Module,Type,list_to_binary(Bytes)); - per -> - asn1rt:decode(Module,Type,Bytes); - per_bin when binary(Bytes) -> - asn1rt:decode(Module,Type,Bytes); - per_bin -> - asn1rt:decode(Module,Type,list_to_binary(Bytes)); - uper_bin -> - asn1rt:decode(Module,Type,list_to_binary(Bytes)) - end. +wrapper_decode(Module, Type, Bytes) when is_binary(Bytes) -> + asn1rt:decode(Module, Type, Bytes); +wrapper_decode(Module, Type, Bytes) when is_list(Bytes) -> + asn1rt:decode(Module, Type, list_to_binary(Bytes)). diff --git a/lib/asn1/test/asn1_app_test.erl b/lib/asn1/test/asn1_app_test.erl index 2c31c3259d..9dbe1b50e0 100644 --- a/lib/asn1/test/asn1_app_test.erl +++ b/lib/asn1/test/asn1_app_test.erl @@ -138,7 +138,8 @@ check_asn1ct_modules(Extra) -> asn1ct_name,asn1ct_constructed_per,asn1ct_constructed_ber, asn1ct_gen_ber,asn1ct_constructed_ber_bin_v2, asn1ct_gen_ber_bin_v2,asn1ct_value, - asn1ct_tok,asn1ct_parser2,asn1ct_table], + asn1ct_tok,asn1ct_parser2,asn1ct_table, + asn1ct_imm], case Extra -- ASN1CTMods of [] -> ok; diff --git a/lib/asn1/test/asn1_test_lib.erl b/lib/asn1/test/asn1_test_lib.erl index 96c04a9436..fda635d0eb 100644 --- a/lib/asn1/test/asn1_test_lib.erl +++ b/lib/asn1/test/asn1_test_lib.erl @@ -87,14 +87,14 @@ ticket_7407_compile(Config,Option) -> ?line OutDir = ?config(priv_dir,Config), ?line ok = asn1ct:compile(DataDir ++ "EUTRA-extract-7407", - [uper_bin, {outdir,OutDir}]++Option). + [uper, {outdir,OutDir}]++Option). ticket_7708(Config,Option) -> ?line DataDir = ?config(data_dir,Config), ?line OutDir = ?config(priv_dir,Config), ?line ok = asn1ct:compile(DataDir ++ "EUTRA-extract-55", - [uper_bin, {outdir,OutDir}]++Option). + [uper, {outdir,OutDir}]++Option). ticket_7407_code(FinalPadding) -> @@ -154,7 +154,7 @@ ticket_7678(Config, Option) -> ?line OutDir = ?config(priv_dir,Config), ?line ok = asn1ct:compile(DataDir ++ "UPERDefault", - [uper_bin, {outdir,OutDir}]++Option), + [uper, {outdir,OutDir}]++Option), ?line Val = 'UPERDefault':seq(), ?line {ok,<<0,6,0>>} = 'UPERDefault':encode('Seq',Val), @@ -167,12 +167,12 @@ ticket_7763(Config) -> ?line OutDir = ?config(priv_dir,Config), ?line ok = asn1ct:compile(DataDir ++ "EUTRA-extract-55", - [uper_bin, {outdir,OutDir}]), + [uper, {outdir,OutDir}]), Val = {'Seq',15,lists:duplicate(8,0),[0],lists:duplicate(28,0),15,true}, ?line {ok,Bin} = 'EUTRA-extract-55':encode('Seq',Val), ?line ok = asn1ct:compile(DataDir ++ "EUTRA-extract-55", - [uper_bin,compact_bit_string,{outdir,OutDir}]), + [uper,compact_bit_string,{outdir,OutDir}]), CompactVal = {'Seq',15,{0,<<0>>},{7,<<0>>},{4,<<0,0,0,0>>},15,true}, {ok,CompactBin} = 'EUTRA-extract-55':encode('Seq',CompactVal), diff --git a/lib/asn1/test/asn1_wrapper.erl b/lib/asn1/test/asn1_wrapper.erl index d515b99ac2..e764d8b4ca 100644 --- a/lib/asn1/test/asn1_wrapper.erl +++ b/lib/asn1/test/asn1_wrapper.erl @@ -34,41 +34,16 @@ encode(Module,Type,Value) -> Error end. -decode(Module,Type,Bytes) -> - case Module:encoding_rule() of - ber -> - asn1rt:decode(Module,Type,Bytes); - ber_bin when is_binary(Bytes) -> - asn1rt:decode(Module,Type,Bytes); - ber_bin -> - asn1rt:decode(Module,Type,list_to_binary(Bytes)); - ber_bin_v2 when is_binary(Bytes) -> - asn1rt:decode(Module,Type,Bytes); - ber_bin_v2 -> - asn1rt:decode(Module,Type,list_to_binary(Bytes)); - per -> - asn1rt:decode(Module,Type,Bytes); - per_bin when is_binary(Bytes) -> - asn1rt:decode(Module,Type,Bytes); - per_bin -> - asn1rt:decode(Module,Type,list_to_binary(Bytes)); - uper_bin when is_binary(Bytes) -> - asn1rt:decode(Module,Type,Bytes); - uper_bin -> - asn1rt:decode(Module,Type,list_to_binary(Bytes)) - end. +decode(Module, Type, Bytes) when is_binary(Bytes) -> + asn1rt:decode(Module, Type, Bytes); +decode(Module, Type, Bytes) when is_list(Bytes) -> + asn1rt:decode(Module, Type, list_to_binary(Bytes)). erule(ber) -> ber; -erule(ber_bin) -> - ber; -erule(ber_bin_v2) -> - ber; erule(per) -> per; -erule(per_bin) -> - per; -erule(uper_bin) -> +erule(uper) -> per. diff --git a/lib/asn1/test/h323test.erl b/lib/asn1/test/h323test.erl index b7a7d6e4df..426ae16994 100644 --- a/lib/asn1/test/h323test.erl +++ b/lib/asn1/test/h323test.erl @@ -22,7 +22,6 @@ -export([run/1]). -include_lib("test_server/include/test_server.hrl"). -run(per_bin) -> run(); run(per) -> run(); run(_Rules) -> ok. diff --git a/lib/asn1/test/testChoiceIndefinite.erl b/lib/asn1/test/testChoiceIndefinite.erl index 630efcf27a..b5832c985a 100644 --- a/lib/asn1/test/testChoiceIndefinite.erl +++ b/lib/asn1/test/testChoiceIndefinite.erl @@ -23,12 +23,7 @@ -include_lib("test_server/include/test_server.hrl"). -main(per_bin) -> ok; main(per) -> ok; -main(ber_bin_v2) -> - main(ber); -main(ber_bin) -> - main(ber); main(ber) -> %% Test case related to OTP-4358 %% normal encoding diff --git a/lib/asn1/test/testCompactBitString.erl b/lib/asn1/test/testCompactBitString.erl index 9563a31bf3..db102a3bda 100644 --- a/lib/asn1/test/testCompactBitString.erl +++ b/lib/asn1/test/testCompactBitString.erl @@ -233,7 +233,7 @@ compact_bit_string(Rules) -> ok. -ticket_7734(per_bin) -> +ticket_7734(per) -> ?line BS = {0,list_to_binary(lists:duplicate(128,0))}, ?line {ok,BSEnc} = asn1_wrapper:encode('PrimStrings','BS1024',BS), ?line {ok,BS} = asn1_wrapper:decode('PrimStrings','BS1024',BSEnc). @@ -251,7 +251,7 @@ bit_string_unnamed(Rules) -> lists:flatten(Bytes1)) end. -otp_4869(per_bin) -> +otp_4869(per) -> ?line Val1={'IP',[0],{0,<<62,235,90,50,0,0,0,0,0,0,0,0,0,0,0,0>>},asn1_NOVALUE}, ?line Val2 = {'IP',[0],[0,0,1,1,1,1,1,0,1,1,1,0,1,0,1,1,0,1,0,1,1,0,1,0,0,0,1,1,0,0,1,0] ++ lists:duplicate(128 - 32,0),asn1_NOVALUE}, diff --git a/lib/asn1/test/testEnumExt.erl b/lib/asn1/test/testEnumExt.erl index c97116413a..bb975a1d13 100644 --- a/lib/asn1/test/testEnumExt.erl +++ b/lib/asn1/test/testEnumExt.erl @@ -23,8 +23,8 @@ -include_lib("test_server/include/test_server.hrl"). -main(Rules) when Rules == per; Rules == per_bin; Rules == uper_bin -> - io:format("main(~p)~n",[Rules]), +main(Rule) when Rule =:= per; Rule =:= uper -> + io:format("main(~p)~n",[Rule]), B32=[32],B64=[64], %% ENUMERATED with extensionmark (value is in root set) ?line {ok,B32} = asn1_wrapper:encode('EnumExt','Ext',red), @@ -42,10 +42,6 @@ main(Rules) when Rules == per; Rules == per_bin; Rules == uper_bin -> ?line {ok,red} = asn1_wrapper:decode('EnumExt','Noext',B64), ok; -main(ber_bin_v2) -> - main(ber); -main(ber_bin) -> - main(ber); main(ber) -> io:format("main(ber)~n",[]), %% ENUMERATED with extensionmark (value is in root set) diff --git a/lib/asn1/test/testINSTANCE_OF.erl b/lib/asn1/test/testINSTANCE_OF.erl index 5986a00ec5..ce411beb92 100644 --- a/lib/asn1/test/testINSTANCE_OF.erl +++ b/lib/asn1/test/testINSTANCE_OF.erl @@ -26,7 +26,7 @@ main(Erule) -> ?line {ok,Integer} = asn1_wrapper:encode('INSTANCEOF','Int',3), - Int = wrap(Erule,Integer), + Int = list_to_binary(Integer), ValotherName = {otherName,{'INSTANCE OF',{2,4},Int}}, VallastName1 = {lastName,{'GeneralName_lastName',{2,4},12}}, VallastName2 = {lastName,{'GeneralName_lastName',{2,3,4}, @@ -61,18 +61,3 @@ test_encdec(_Erule,{lastName,{'GeneralName_lastName',{2,3,4}, ok; test_encdec(Erule,Res) -> {error,{Erule,Res}}. - -wrap(ber,Int) when is_list(Int) -> - binary_to_list(list_to_binary(Int)); -wrap(per,Int) when is_list(Int) -> - binary_to_list(list_to_binary(Int)); -wrap(ber_bin,Int) when is_list(Int) -> - list_to_binary(Int); -wrap(ber_bin_v2,Int) when is_list(Int) -> - list_to_binary(Int); -wrap(per_bin,Int) when is_list(Int) -> - list_to_binary(Int); -wrap(uper_bin,Int) when is_list(Int) -> - list_to_binary(Int); -wrap(_,Int) -> - Int. diff --git a/lib/asn1/test/testInfObjectClass.erl b/lib/asn1/test/testInfObjectClass.erl index e639066246..98408502c6 100644 --- a/lib/asn1/test/testInfObjectClass.erl +++ b/lib/asn1/test/testInfObjectClass.erl @@ -37,15 +37,12 @@ main(Rule) -> {component,'ArgumentType'}, {value,_},_}}} = asn1_wrapper:encode('InfClass','Seq', {'Seq',12,13,1}), - Bytes2 = - if - Rule==per;Rule==per_bin -> - [1,12,1,11,1,1]; - Rule == uper_bin -> - <<1,12,1,11,1,1>>; - true -> - [48,9,2,1,12,2,1,11,2,1,1] - end, + Bytes2 = case Rule of + ber -> + <<48,9,2,1,12,2,1,11,2,1,1>>; + _ -> + <<1,12,1,11,1,1>> + end, ?line {error,{asn1,{'Type not compatible with table constraint', {{component,_}, {value,_B},_}}}} = diff --git a/lib/asn1/test/testMergeCompile.erl b/lib/asn1/test/testMergeCompile.erl index 31aa3518f6..d63df28c31 100644 --- a/lib/asn1/test/testMergeCompile.erl +++ b/lib/asn1/test/testMergeCompile.erl @@ -43,17 +43,11 @@ main(Erule) -> ?line EncVal = case Erule of per -> - [1,100]; - per_bin -> <<1,100>>; - uper_bin -> + uper -> <<1,100>>; ber -> - [2,1,1]; - ber_bin -> - <<2,1,1>>; - ber_bin_v2 -> - <<2,1,1>> + [2,1,1] end, ?line PEVal2 = [{dummy,1,ignore,EncVal},{dummy,2,reject,EncVal}], ?line Val2 = @@ -76,7 +70,7 @@ main(Erule) -> mvrasn(Erule) -> case Erule of - Ber when Ber == ber;Ber == ber_bin -> + ber -> ?line ok = test(isd), ?line ok = test(isd2), ?line ok = test(dsd), diff --git a/lib/compiler/test/compilation_SUITE_data/bad_functional_value.erl b/lib/asn1/test/testMultipleLevels.erl index 126a573e83..ff6d023440 100644 --- a/lib/compiler/test/compilation_SUITE_data/bad_functional_value.erl +++ b/lib/asn1/test/testMultipleLevels.erl @@ -1,28 +1,27 @@ %% %% %CopyrightBegin% -%% -%% Copyright Ericsson AB 2003-2009. All Rights Reserved. -%% +%% +%% Copyright Ericsson AB 2012. All Rights Reserved. +%% %% The contents of this file are subject to the Erlang Public License, %% Version 1.1, (the "License"); you may not use this file except in %% compliance with the License. You should have received a copy of the %% Erlang Public License along with this software. If not, it can be %% retrieved online at http://www.erlang.org/. -%% +%% %% Software distributed under the License is distributed on an "AS IS" %% basis, WITHOUT WARRANTY OF ANY KIND, either express or implied. See %% the License for the specific language governing rights and limitations %% under the License. -%% +%% %% %CopyrightEnd% %% --module(bad_functional_value). - --export([?MODULE/0,a/0]). - -?MODULE() -> - ok. +%% -a() -> - .list_to_atom("ok"). +-module(testMultipleLevels). +-export([main/1]). +main(_) -> + Data = {'Top',{short,"abc"},{long,"a long string follows here"}}, + {ok,B} = 'MultipleLevels':encode('Top', Data), + {ok,Data} = 'MultipleLevels':decode('Top', iolist_to_binary(B)). diff --git a/lib/asn1/test/testParameterizedInfObj.erl b/lib/asn1/test/testParameterizedInfObj.erl index 68faf08a61..6f53595132 100644 --- a/lib/asn1/test/testParameterizedInfObj.erl +++ b/lib/asn1/test/testParameterizedInfObj.erl @@ -98,7 +98,7 @@ ranap(_Erule) -> ok. -open_type(uper_bin,Val) when is_list(Val) -> +open_type(uper,Val) when is_list(Val) -> list_to_binary(Val); open_type(_,Val) -> Val. diff --git a/lib/asn1/test/testPrimStrings.erl b/lib/asn1/test/testPrimStrings.erl index b1c5172b95..9a640a2cb1 100644 --- a/lib/asn1/test/testPrimStrings.erl +++ b/lib/asn1/test/testPrimStrings.erl @@ -338,69 +338,24 @@ octet_string(Rules) -> ok end, + roundtrip('Os', [47,23,99,255,1]), + roundtrip('OsCon', [47,23,99,255,1]), + roundtrip('OsPri', [47,23,99,255,1]), + roundtrip('OsApp', [47,23,99,255,1]), - - ?line {ok,Bytes4} = - asn1_wrapper:encode('PrimStrings','Os',[47,23,99,255,1]), - ?line {ok,[47,23,99,255,1]} = asn1_wrapper:decode('PrimStrings','Os',lists:flatten(Bytes4)), - - ?line {ok,Bytes5} = - asn1_wrapper:encode('PrimStrings','OsCon',[47,23,99,255,1]), - ?line {ok,[47,23,99,255,1]} = asn1_wrapper:decode('PrimStrings','OsCon',lists:flatten(Bytes5)), - - ?line {ok,Bytes6} = - asn1_wrapper:encode('PrimStrings','OsPri',[47,23,99,255,1]), - ?line {ok,[47,23,99,255,1]} = asn1_wrapper:decode('PrimStrings','OsPri',lists:flatten(Bytes6)), - - ?line {ok,Bytes7} = - asn1_wrapper:encode('PrimStrings','OsApp',[47,23,99,255,1]), - ?line {ok,[47,23,99,255,1]} = asn1_wrapper:decode('PrimStrings','OsApp',lists:flatten(Bytes7)), - - ?line {ok,Bytes8} = - asn1_wrapper:encode('PrimStrings','OsExpCon',[47,23,99,255,1]), - ?line {ok,[47,23,99,255,1]} = asn1_wrapper:decode('PrimStrings','OsExpCon',lists:flatten(Bytes8)), - - ?line {ok,Bytes9} = - asn1_wrapper:encode('PrimStrings','OsExpPri',[47,23,99,255,1]), - ?line {ok,[47,23,99,255,1]} = asn1_wrapper:decode('PrimStrings','OsExpPri',lists:flatten(Bytes9)), - - ?line {ok,Bytes10} = - asn1_wrapper:encode('PrimStrings','OsExpApp',[47,23,99,255,1]), - ?line {ok,[47,23,99,255,1]} = asn1_wrapper:decode('PrimStrings','OsExpApp',lists:flatten(Bytes10)), - - ?line {ok,Bytes11} = - asn1_wrapper:encode('PrimStrings','Os',[]), - ?line {ok,[]} = asn1_wrapper:decode('PrimStrings','Os',lists:flatten(Bytes11)), - - ?line {ok,Bytes12} = - asn1_wrapper:encode('PrimStrings','OsApp',[]), - ?line {ok,[]} = asn1_wrapper:decode('PrimStrings','OsApp',lists:flatten(Bytes12)), - - ?line {ok,Bytes13} = - asn1_wrapper:encode('PrimStrings','OsExpApp',[]), - ?line {ok,[]} = asn1_wrapper:decode('PrimStrings','OsExpApp',lists:flatten(Bytes13)), - - - - - + roundtrip('OsExpCon', [47,23,99,255,1]), + roundtrip('OsExpPri', [47,23,99,255,1]), + roundtrip('OsExpApp', [47,23,99,255,1]), + roundtrip('Os', []), + roundtrip('OsApp', []), + roundtrip('OsExpApp',[]), OsR = "12345678901234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890", - ?line {ok,Bytes21} = - asn1_wrapper:encode('PrimStrings','Os',OsR), - ?line {ok,Os1} = asn1_wrapper:decode('PrimStrings','Os',lists:flatten(Bytes21)), - ?line Os1 = OsR, - ?line {ok,Bytes22} = - asn1_wrapper:encode('PrimStrings','OsCon',OsR), - ?line {ok,Os2} = asn1_wrapper:decode('PrimStrings','OsCon',lists:flatten(Bytes22)), - ?line Os2 = OsR, - ?line {ok,Bytes23} = - asn1_wrapper:encode('PrimStrings','OsExpApp',OsR), - ?line {ok,Os3} = asn1_wrapper:decode('PrimStrings','OsExpApp',lists:flatten(Bytes23)), - ?line Os3 = OsR, - + roundtrip('Os', OsR), + roundtrip('OsCon', OsR), + roundtrip('OsExpApp', OsR), ?line case asn1_wrapper:erule(Rules) of @@ -416,21 +371,85 @@ octet_string(Rules) -> ok end, - + fragmented_octet_string(Rules), ok. +fragmented_octet_string(Erules) -> + K16 = 1 bsl 14, + K32 = K16 + K16, + K48 = K32 + K16, + K64 = K48 + K16, + Lens = [0,1,14,15,16,17,127,128, + K16-1,K16,K16+1,K16+(1 bsl 7)-1,K16+(1 bsl 7),K16+(1 bsl 7)+1, + K32-1,K32,K32+1,K32+(1 bsl 7)-1,K32+(1 bsl 7),K32+(1 bsl 7)+1, + K48-1,K48,K48+1,K48+(1 bsl 7)-1,K48+(1 bsl 7),K48+(1 bsl 7)+1, + K64-1,K64,K64+1,K64+(1 bsl 7)-1,K64+(1 bsl 7),K64+(1 bsl 7)+1, + K64+K16-1,K64+K16,K64+K16+1], + Types = ['Os','OsFrag'], + [fragmented_octet_string(Erules, Types, L) || L <- Lens], + fragmented_octet_string(Erules, ['FixedOs65536'], 65536), + fragmented_octet_string(Erules, ['FixedOs65537'], 65537), + + %% Make sure that octet alignment works. + roundtrip('OsAlignment', + {'OsAlignment',false,make_value(70000),true,make_value(66666), + false,make_value(65536),42}), + roundtrip('OsAlignment', + {'OsAlignment',false,make_value(0),true,make_value(0), + false,make_value(65536),42}), + ok. + +fragmented_octet_string(Erules, Types, L) -> + Value = make_value(L), + [begin + Encoded = enc_frag(Erules, Type, Value), + {ok,Value} = 'PrimStrings':decode(Type, Encoded) + end || Type <- Types], + ok. +enc_frag(Erules, Type, Value) -> + {ok,Encoded} = 'PrimStrings':encode(Type, Value), + case Erules of + ber -> + Encoded; + _ -> + %% Validate encoding with our own encoder. + Encoded = enc_frag_1(<<>>, list_to_binary(Value)) + end. + +enc_frag_1(Res, Bin0) -> + K16 = 1 bsl 14, + Sz = byte_size(Bin0), + if + Sz >= K16 -> + F = min(Sz div K16, 4), + FragSize = F * K16, + <<Frag:FragSize/binary-unit:8,Bin/binary>> = Bin0, + enc_frag_1(<<Res/binary,3:2,F:6,Frag/binary>>, Bin); + Sz >= 128 -> + <<Res/binary,1:1,0:1,Sz:14,Bin0/binary>>; + true -> + <<Res/binary,0:1,Sz:7,Bin0/binary>> + end. + +make_value(L) -> + make_value(L, 0, []). + +make_value(0, _, Acc) -> + Acc; +make_value(N, Byte, Acc) when Byte =< 255 -> + make_value(N-1, Byte+7, [Byte|Acc]); +make_value(N, Byte, Acc) -> + make_value(N, Byte band 16#FF, Acc). - numeric_string(Rules) -> %%========================================================== %% Ns ::= NumericString %%========================================================== - ?line {ok,BytesNs2} = asn1_wrapper:encode('PrimStrings','Ns',[]), - ?line {ok,[]} = asn1_wrapper:decode('PrimStrings','Ns',lists:flatten(BytesNs2)), + roundtrip('Ns', []), ?line case asn1_wrapper:erule(Rules) of ber -> @@ -455,10 +474,7 @@ numeric_string(Rules) -> %% NsCon ::= [70] NumericString %%========================================================== - ?line {ok,BytesNs12} = asn1_wrapper:encode('PrimStrings','NsCon',[]), - ?line {ok,[]} = asn1_wrapper:decode('PrimStrings','NsCon',lists:flatten(BytesNs12)), - - + roundtrip('NsCon', []), ?line case asn1_wrapper:erule(Rules) of ber -> @@ -482,10 +498,7 @@ numeric_string(Rules) -> %% NsExpCon ::= [71] EXPLICIT NumericString %%========================================================== - ?line {ok,BytesNs22} = asn1_wrapper:encode('PrimStrings','NsExpCon',[]), - ?line {ok,[]} = asn1_wrapper:decode('PrimStrings','NsExpCon',lists:flatten(BytesNs22)), - - + roundtrip('NsExpCon', []), ?line case asn1_wrapper:erule(Rules) of ber -> @@ -507,9 +520,6 @@ numeric_string(Rules) -> ok. - - - other_strings(_Rules) -> @@ -517,49 +527,27 @@ other_strings(_Rules) -> %% Ps ::= PrintableString %%========================================================== - ?line {ok,BytesPs1} = asn1_wrapper:encode('PrimStrings','Ps',[47,23,99,75,47]), - ?line {ok,[47,23,99,75,47]} = - asn1_wrapper:decode('PrimStrings','Ps',lists:flatten(BytesPs1)), - - ?line {ok,BytesPs2} = asn1_wrapper:encode('PrimStrings','Ps',[]), - ?line {ok,[]} = asn1_wrapper:decode('PrimStrings','Ps',lists:flatten(BytesPs2)), - + roundtrip('Ps', [47,23,99,75,47]), + roundtrip('Ps', []), %%========================================================== %% Vis ::= VisibleString %%========================================================== - ?line {ok,BytesVis1} = asn1_wrapper:encode('PrimStrings','Vis',[47,23,99,75,47]), - ?line {ok,[47,23,99,75,47]} = - asn1_wrapper:decode('PrimStrings','Vis',lists:flatten(BytesVis1)), - - ?line {ok,BytesVis2} = asn1_wrapper:encode('PrimStrings','Vis',[]), - ?line {ok,[]} = asn1_wrapper:decode('PrimStrings','Vis',lists:flatten(BytesVis2)), - + roundtrip('Vis', [47,23,99,75,47]), + roundtrip('Vis', []), %%========================================================== %% IA5 ::= IA5String %%========================================================== - ?line {ok,BytesIA51} = asn1_wrapper:encode('PrimStrings','IA5',[47,23,99,75,47]), - ?line {ok,[47,23,99,75,47]} = - asn1_wrapper:decode('PrimStrings','IA5',lists:flatten(BytesIA51)), - - ?line {ok,BytesIA52} = asn1_wrapper:encode('PrimStrings','IA5',[]), - ?line {ok,[]} = asn1_wrapper:decode('PrimStrings','IA5',lists:flatten(BytesIA52)), + roundtrip('IA5', [47,23,99,75,47]), + roundtrip('IA5', []), - IA5_1 = "12345678901234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890", - - ?line {ok,BytesIA53} = asn1_wrapper:encode('PrimStrings','IA5',IA5_1), - ?line {ok,IA5_1r} = asn1_wrapper:decode('PrimStrings','IA5',lists:flatten(BytesIA53)), - ?line IA5_1 = IA5_1r, - - - - + roundtrip('IA5', IA5_1), ok. @@ -568,94 +556,60 @@ more_strings(_Rules) -> %% Ts ::= TeletexString %%========================================================== - ?line {ok,BytesTs1} = asn1_wrapper:encode('PrimStrings','Ts',[47,23,99,75,47]), - ?line {ok,[47,23,99,75,47]} = - asn1_wrapper:decode('PrimStrings','Ts',lists:flatten(BytesTs1)), - - ?line {ok,BytesTs2} = asn1_wrapper:encode('PrimStrings','Ts',[]), - ?line {ok,[]} = asn1_wrapper:decode('PrimStrings','Ts',lists:flatten(BytesTs2)), - + roundtrip('Ts', [47,23,99,75,47]), + roundtrip('Ts', []), %%========================================================== %% Vxs ::= VideotexString %%========================================================== - ?line {ok,BytesVxs1} = asn1_wrapper:encode('PrimStrings','Vxs',[47,23,99,75,47]), - ?line {ok,[47,23,99,75,47]} = - asn1_wrapper:decode('PrimStrings','Vxs',lists:flatten(BytesVxs1)), - - ?line {ok,BytesVxs2} = asn1_wrapper:encode('PrimStrings','Vxs',[]), - ?line {ok,[]} = asn1_wrapper:decode('PrimStrings','Vxs',lists:flatten(BytesVxs2)), - + roundtrip('Vxs', [47,23,99,75,47]), + roundtrip('Vxs', []), %%========================================================== %% Grs ::= GraphicString %%========================================================== - ?line {ok,BytesGrs1} = asn1_wrapper:encode('PrimStrings','Grs',[47,23,99,75,47]), - ?line {ok,[47,23,99,75,47]} = - asn1_wrapper:decode('PrimStrings','Grs',lists:flatten(BytesGrs1)), - - ?line {ok,BytesGrs2} = asn1_wrapper:encode('PrimStrings','Grs',[]), - ?line {ok,[]} = asn1_wrapper:decode('PrimStrings','Grs',lists:flatten(BytesGrs2)), + roundtrip('Grs',[47,23,99,75,47]), + roundtrip('Grs', []), %%========================================================== %% ODesc ::= ObjectDescriptor, test case for OTP-4161 %%========================================================== - ?line {ok,BytesODesc1} = asn1_wrapper:encode('PrimStrings','ODesc',[79,98,106,101,99,116,68,101,115,99,114,105,112,116,111,114]), - ?line {ok,[79,98,106,101,99,116,68,101,115,99,114,105,112,116,111,114]} = - asn1_wrapper:decode('PrimStrings','ODesc',lists:flatten(BytesODesc1)), - - ?line {ok,BytesODesc2} = asn1_wrapper:encode('PrimStrings','ODesc',[]), - ?line {ok,[]} = asn1_wrapper:decode('PrimStrings','ODesc',lists:flatten(BytesODesc2)), + roundtrip('ODesc', [79,98,106,101,99,116,68,101,115,99,114, + 105,112,116,111,114]), + roundtrip('ODesc', []), %%========================================================== %% Ges ::= GeneralString %%========================================================== - ?line {ok,BytesGes1} = asn1_wrapper:encode('PrimStrings','Ges',[47,23,99,75,47]), - ?line {ok,[47,23,99,75,47]} = - asn1_wrapper:decode('PrimStrings','Ges',lists:flatten(BytesGes1)), - - ?line {ok,BytesGes2} = asn1_wrapper:encode('PrimStrings','Ges',[]), - ?line {ok,[]} = asn1_wrapper:decode('PrimStrings','Ges',lists:flatten(BytesGes2)), - - ok. + roundtrip('Ges', [47,23,99,75,47]), + roundtrip('Ges', []), + ok. universal_string(Rules) -> - %%========================================================== %% Us ::= UniversalString %%========================================================== - ?line {ok,Bytes1} = - asn1_wrapper:encode('PrimStrings','Us',[{47,23,99,47},{0,0,55,66}]), - ?line {ok,[{47,23,99,47},{0,0,55,66}]} = - asn1_wrapper:decode('PrimStrings','Us',lists:flatten(Bytes1)), + roundtrip('Us', [{47,23,99,47},{0,0,55,66}]), ?line {ok,Bytes2} = asn1_wrapper:encode('PrimStrings','Us',[{47,23,99,255},{0,0,0,201}]), ?line {ok,[{47,23,99,255},201]} = asn1_wrapper:decode('PrimStrings','Us',lists:flatten(Bytes2)), - ?line {ok,Bytes3} = asn1_wrapper:encode('PrimStrings','Us',"Universal String"), - ?line {ok,"Universal String"} = - asn1_wrapper:decode('PrimStrings','Us',lists:flatten(Bytes3)), - - ?line {ok,Bytes4} = asn1_wrapper:encode('PrimStrings','Us',[]), - ?line {ok,[]} = asn1_wrapper:decode('PrimStrings','Us',lists:flatten(Bytes4)), - - ?line {ok,Bytes5} = asn1_wrapper:encode('PrimStrings','Us',[{47,23,99,47}]), - ?line {ok,[{47,23,99,47}]} = - asn1_wrapper:decode('PrimStrings','Us',lists:flatten(Bytes5)), - + roundtrip('Us', "Universal String"), + roundtrip('Us', []), + roundtrip('Us', [{47,23,99,47}]), ?line case asn1_wrapper:erule(Rules) of ber -> @@ -670,32 +624,22 @@ universal_string(Rules) -> Us1 = "12345678901234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890", - ?line {ok,Bytes15} = asn1_wrapper:encode('PrimStrings','IA5',Us1), - ?line {ok,Us1r} = asn1_wrapper:decode('PrimStrings','IA5',lists:flatten(Bytes15)), - ?line Us1 = Us1r, - + roundtrip('IA5', Us1), %%========================================================== %% UsCon ::= [70] UniversalString %%========================================================== - ?line {ok,Bytes11} = - asn1_wrapper:encode('PrimStrings','UsCon',[{47,23,99,255},{0,0,2,201}]), - ?line {ok,[{47,23,99,255},{0,0,2,201}]} = - asn1_wrapper:decode('PrimStrings','UsCon',lists:flatten(Bytes11)), + roundtrip('UsCon', [{47,23,99,255},{0,0,2,201}]), ?line {ok,Bytes12} = asn1_wrapper:encode('PrimStrings','UsCon',[{47,23,99,255},{0,0,0,201}]), ?line {ok,[{47,23,99,255},201]} = asn1_wrapper:decode('PrimStrings','UsCon',lists:flatten(Bytes12)), - ?line {ok,Bytes13} = asn1_wrapper:encode('PrimStrings','UsCon',"Universal String"), - ?line {ok,"Universal String"} = - asn1_wrapper:decode('PrimStrings','UsCon',lists:flatten(Bytes13)), - - ?line {ok,Bytes14} = asn1_wrapper:encode('PrimStrings','UsCon',[]), - ?line {ok,[]} = asn1_wrapper:decode('PrimStrings','UsCon',lists:flatten(Bytes14)), + roundtrip('UsCon', "Universal String"), + roundtrip('UsCon', []), ?line case asn1_wrapper:erule(Rules) of ber -> @@ -712,25 +656,15 @@ universal_string(Rules) -> %% UsExpCon ::= [71] EXPLICIT UniversalString %%========================================================== - ?line {ok,Bytes21} = - asn1_wrapper:encode('PrimStrings','UsExpCon',[{47,23,99,255},{0,0,2,201}]), - ?line {ok,[{47,23,99,255},{0,0,2,201}]} = - asn1_wrapper:decode('PrimStrings','UsExpCon',lists:flatten(Bytes21)), + roundtrip('UsExpCon', [{47,23,99,255},{0,0,2,201}]), ?line {ok,Bytes22} = asn1_wrapper:encode('PrimStrings','UsExpCon',[{47,23,99,255},{0,0,0,201}]), ?line {ok,[{47,23,99,255},201]} = asn1_wrapper:decode('PrimStrings','UsExpCon',lists:flatten(Bytes22)), - ?line {ok,Bytes23} = - asn1_wrapper:encode('PrimStrings','UsExpCon',"Universal String"), - ?line {ok,"Universal String"} = - asn1_wrapper:decode('PrimStrings','UsExpCon',lists:flatten(Bytes23)), - - ?line {ok,Bytes24} = - asn1_wrapper:encode('PrimStrings','UsExpCon',[]), - ?line {ok,[]} = - asn1_wrapper:decode('PrimStrings','UsExpCon',lists:flatten(Bytes24)), + roundtrip('UsExpCon', "Universal String"), + roundtrip('UsExpCon', []), ?line case asn1_wrapper:erule(Rules) of ber -> @@ -740,12 +674,8 @@ universal_string(Rules) -> asn1_wrapper:decode('PrimStrings','UsExpCon',lists:flatten([16#BF,16#47,16,60,16#80,28,4,47,23,99,255,28,4,0,0,2,201,0,0])); _ -> ok end, - - -ok. - - + ok. bmp_string(_Rules) -> @@ -754,29 +684,18 @@ bmp_string(_Rules) -> %% BMP ::= BMPString %%========================================================== - ?line {ok,Bytes1} = - asn1_wrapper:encode('PrimStrings','BMP',[{0,0,99,48},{0,0,2,201}]), - ?line {ok,[{0,0,99,48},{0,0,2,201}]} = - asn1_wrapper:decode('PrimStrings','BMP',lists:flatten(Bytes1)), + roundtrip('BMP', [{0,0,99,48},{0,0,2,201}]), ?line {ok,Bytes2} = asn1_wrapper:encode('PrimStrings','BMP',[{0,0,0,48},{0,0,2,201}]), ?line {ok,[48,{0,0,2,201}]} = asn1_wrapper:decode('PrimStrings','BMP',lists:flatten(Bytes2)), - - ?line {ok,Bytes3} = asn1_wrapper:encode('PrimStrings','BMP',"BMP String"), - ?line {ok,"BMP String"} = - asn1_wrapper:decode('PrimStrings','BMP',lists:flatten(Bytes3)), - - ?line {ok,Bytes4} = asn1_wrapper:encode('PrimStrings','BMP',[]), - ?line {ok,[]} = asn1_wrapper:decode('PrimStrings','BMP',lists:flatten(Bytes4)), + roundtrip('BMP', "BMP String"), + roundtrip('BMP', []), BMP1 = "12345678901234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890", - ?line {ok,Bytes5} = asn1_wrapper:encode('PrimStrings','BMP',BMP1), - ?line {ok,BMP1r} = asn1_wrapper:decode('PrimStrings','BMP',lists:flatten(Bytes5)), - ?line BMP1 = BMP1r, - + roundtrip('BMP', BMP1), ok. @@ -790,35 +709,17 @@ times(_Rules) -> %% Gt ::= GeneralizedTime %%========================================================== - ?line {ok,Bytes1} = asn1_wrapper:encode('PrimStrings','Gt',"19970923110723.2"), - ?line {ok,"19970923110723.2"} = - asn1_wrapper:decode('PrimStrings','Gt',lists:flatten(Bytes1)), + roundtrip('Gt', "19970923110723.2"), + roundtrip('Gt', "19970923110723.2Z"), + roundtrip('Gt', "19970923110723.2-0500"), - ?line {ok,Bytes2} = asn1_wrapper:encode('PrimStrings','Gt',"19970923110723.2Z"), - ?line {ok,"19970923110723.2Z"} = - asn1_wrapper:decode('PrimStrings','Gt',lists:flatten(Bytes2)), - - ?line {ok,Bytes3} = asn1_wrapper:encode('PrimStrings','Gt',"19970923110723.2-0500"), - ?line {ok,"19970923110723.2-0500"} = - asn1_wrapper:decode('PrimStrings','Gt',lists:flatten(Bytes3)), - - - - - - %%========================================================== %% UTC ::= UTCTime %%========================================================== - ?line {ok,Bytes11} = asn1_wrapper:encode('PrimStrings','UTC',"9709211107Z"), - ?line {ok,"9709211107Z"} = - asn1_wrapper:decode('PrimStrings','UTC',lists:flatten(Bytes11)), - - ?line {ok,Bytes12} = asn1_wrapper:encode('PrimStrings','UTC',"9709211107-0500"), - ?line {ok,"9709211107-0500"} = - asn1_wrapper:decode('PrimStrings','UTC',lists:flatten(Bytes12)), + roundtrip('UTC', "9709211107Z"), + roundtrip('UTC', "9709211107-0500"), ok. @@ -917,3 +818,8 @@ wrapper_utf8_binary_to_list(L) when is_list(L) -> asn1rt:utf8_binary_to_list(list_to_binary(L)); wrapper_utf8_binary_to_list(B) -> asn1rt:utf8_binary_to_list(B). + +roundtrip(Type, Value) -> + {ok,Encoded} = 'PrimStrings':encode(Type, Value), + {ok,Value} = 'PrimStrings':decode(Type, Encoded), + ok. diff --git a/lib/asn1/test/testSSLspecs.erl b/lib/asn1/test/testSSLspecs.erl index 51ef134e5f..45c5da50f0 100644 --- a/lib/asn1/test/testSSLspecs.erl +++ b/lib/asn1/test/testSSLspecs.erl @@ -42,11 +42,11 @@ compile(Config, Options) -> asn1_test_lib:compile_all(["PKIX1Explicit93", "PKIX1Implicit93"], Config, NewOptions). -compile_inline(Config, Rule) when Rule == ber_bin; Rule == ber_bin_v2 -> +compile_inline(Config, ber=Rule) -> DataDir = ?config(data_dir, Config), CaseDir = ?config(case_dir, Config), Options = [{i, CaseDir}, {i, DataDir}, Rule, - der, compact_bit_string, optimize, asn1config, inline], + der, compact_bit_string, asn1config, inline], ok = remove_db_file_inline(CaseDir), asn1_test_lib:compile("OTP-PKIX.set.asn", Config, Options); compile_inline(_Config, _Rule) -> @@ -73,7 +73,7 @@ remove_db_file_inline(Dir) -> ?line ok = remove_db_file(Dir ++ "PKIX1Explicit88.asn1db"), ?line ok = remove_db_file(Dir ++ "PKIX1Implicit88.asn1db"). -run(BER) when BER==ber_bin;BER==ber_bin_v2 -> +run(ber) -> run1(1); run(_) -> ok. @@ -100,20 +100,20 @@ transform1(ATAV) -> ?line {ok, ATAVEnc} = 'PKIX1Explicit88':encode('AttributeTypeAndValue', ATAV), ?line {ok, _ATAVDec} = 'SSL-PKIX':decode('AttributeTypeAndValue', - list_to_binary(ATAVEnc)). + ATAVEnc). transform2(ATAV) -> ?line {ok, ATAVEnc} = 'PKIX1Explicit88':encode('AttributeTypeAndValue', ATAV), ?line {ok, _ATAVDec} = 'PKIX1Explicit88':decode('AttributeTypeAndValue', - list_to_binary(ATAVEnc)). + ATAVEnc). transform4(ATAV) -> ?line {ok, ATAVEnc} = 'PKIX1Explicit88':encode('Attribute', ATAV), ?line {ok, _ATAVDec} = 'PKIX1Explicit88':decode('Attribute', - list_to_binary(ATAVEnc)). + ATAVEnc). ex(1) -> @@ -146,7 +146,7 @@ ex(7) -> {1,2,840,113549,1,9,1}, [[19,5,111,116,112,67,65]]}. -run_inline(Rule) when Rule==ber_bin;Rule==ber_bin_v2 -> +run_inline(ber) -> Cert = cert(), ?line {ok,{'CertificatePKIX1Explicit88',{Type,UnDec},_,_}} = 'OTP-PKIX':decode_TBSCert_exclusive(Cert), ?line {ok,_} = 'OTP-PKIX':decode_part(Type,UnDec), diff --git a/lib/asn1/test/testSeqIndefinite.erl b/lib/asn1/test/testSeqIndefinite.erl index 25742474bb..c7b8aba523 100644 --- a/lib/asn1/test/testSeqIndefinite.erl +++ b/lib/asn1/test/testSeqIndefinite.erl @@ -23,13 +23,7 @@ -include_lib("test_server/include/test_server.hrl"). - -main(per_bin) -> ok; main(per) -> ok; -main(ber_bin_v2) -> - main(ber); -main(ber_bin) -> - main(ber); main(ber) -> %% normal encoding diff --git a/lib/asn1/test/testSeqOf.erl b/lib/asn1/test/testSeqOf.erl index 0c0bbc3e66..1aa1eab26d 100644 --- a/lib/asn1/test/testSeqOf.erl +++ b/lib/asn1/test/testSeqOf.erl @@ -198,19 +198,10 @@ main(Rules) -> ?line {ok,Bytes51} = asn1_wrapper:encode('SeqOf','SeqEmp',#'SeqEmp'{seq1 = [#'Empty'{}]}), ?line {ok,{'SeqEmp',[{'Empty'}]}} = asn1_wrapper:decode('SeqOf','SeqEmp',lists:flatten(Bytes51)), - - case Rules of - ber -> - ?line {ok,Bytes52} = asn1_wrapper:encode('SeqOfEnum','SeqOfEnum', - {'SeqOfEnum',[{'Enum',a},{'Enum',b}]}), - ?line {ok,[a,b]} = asn1_wrapper:decode('SeqOfEnum','SeqOfEnum', - lists:flatten(Bytes52)); - _ -> ok - end, %% tests of OTP-4590 case Rules of - PER when PER == per; PER == per_bin -> + per -> DayNames = ["Monday","Tuesday","Wednesday","Thursday","Friday","Saturday","Sunday"], ?line {ok,Bytes60} = asn1_wrapper:encode('XSeqOf','DayNames2',DayNames), ?line {ok,Bytes60} = asn1_wrapper:encode('XSeqOf','DayNames4',DayNames), diff --git a/lib/asn1/test/testSetIndefinite.erl b/lib/asn1/test/testSetIndefinite.erl index d8e2b6a9cf..73006da62b 100644 --- a/lib/asn1/test/testSetIndefinite.erl +++ b/lib/asn1/test/testSetIndefinite.erl @@ -24,12 +24,7 @@ -include_lib("test_server/include/test_server.hrl"). -main(per_bin) -> ok; main(per) -> ok; -main(ber_bin_v2) -> - main(ber); -main(ber_bin) -> - main(ber); main(ber) -> %% normal encoding diff --git a/lib/asn1/test/testSetOptional.erl b/lib/asn1/test/testSetOptional.erl index 4692941524..cef90bc843 100644 --- a/lib/asn1/test/testSetOptional.erl +++ b/lib/asn1/test/testSetOptional.erl @@ -181,7 +181,7 @@ main(_Rules) -> ok. -ticket_7533(Ber) when Ber == ber; Ber == ber_bin -> +ticket_7533(Ber) when Ber == ber -> Val = #'SetOpt1'{bool1 = true,int1=12,set1=#'SetIn'{boolIn=false,intIn=13}}, ?line {ok,B} = asn1_wrapper:encode('SetOptional','SetOpt1',Val), ?line {ok,Val} = asn1_wrapper:decode('SetOptional','SetOpt1',B), diff --git a/lib/asn1/test/testTCAP.erl b/lib/asn1/test/testTCAP.erl index 878ce7c070..b723995e40 100644 --- a/lib/asn1/test/testTCAP.erl +++ b/lib/asn1/test/testTCAP.erl @@ -37,7 +37,7 @@ compile_asn1config(Config, Options) -> asn1_test_lib:compile_all(Files, Config, Options), asn1_test_lib:compile_erlang("TCAPPackage_msg", Config, []). -test(Erule,_Config) when Erule==ber;Erule==ber_bin;Erule==ber_bin_v2 -> +test(ber=Erule,_Config) -> % ?line OutDir = ?config(priv_dir,Config), %% testing OTP-4798, open type encoded with indefinite length ?line {ok,_Res} = asn1_wrapper:decode('TCAPMessages-simple','MessageType', val_OTP_4798(Erule)), @@ -81,7 +81,7 @@ test_asn1config() -> ?line Val2 = 'TCAPPackage_msg':val('TransactionPDU'), ?line {ok,B2} = 'TCAPPackage':encode('TransactionPDU',Val2), - ?line {ok,ExMsg2}='TCAPPackage':decode_TransactionPDU(list_to_binary(B2)), + {ok,ExMsg2}='TCAPPackage':decode_TransactionPDU(B2), ?line {_,_,_,{Key2,ExVal2}}=ExMsg2, ?line {ok,_Parts2}='TCAPPackage':decode_part(Key2,ExVal2), diff --git a/lib/asn1/test/testTimer.erl b/lib/asn1/test/testTimer.erl index 2d3b777558..cd7ceb5630 100644 --- a/lib/asn1/test/testTimer.erl +++ b/lib/asn1/test/testTimer.erl @@ -133,23 +133,7 @@ go(Config,Enc) -> Module = 'H323-MESSAGES', Type = 'H323-UserInformation', Value = val(), -%% ok = asn1ct:compile(HelpModule,[Enc]), - -%% ok = asn1ct:compile(Module,[Enc]), - ?line {ok,B} = asn1rt:encode(Module,Type,Value), - Bytes = case Enc of - ber_bin -> - list_to_binary(B); - per_bin when is_list(B) -> - list_to_binary(B); - per_bin -> - B; - uper_bin -> - B; - _ -> - %%lists:flatten(B) - list_to_binary(B) - end, + {ok,Bytes} = asn1rt:encode(Module,Type,Value), CompileOptions = compile_options(), @@ -181,35 +165,18 @@ encode(N, Module,Type,Value) -> decode(0, _Module,_Type,_Value,_Erule) -> done; decode(N, Module,Type,Value,Erule) -> - case Erule of - ber -> - ?line {ok,_B} = asn1rt:decode(Module,Type,binary_to_list(Value)); - per -> - ?line {ok,_B} = asn1rt:decode(Module,Type,binary_to_list(Value)); - _ -> - ?line {ok,_B} = asn1rt:decode(Module,Type,Value) - end, + {ok,_B} = asn1rt:decode(Module,Type,Value), decode(N-1, Module,Type,Value,Erule). compile_options() -> - ?line {ok,Info} = asn1rt:info('H323-MESSAGES'), - case lists:keysearch(options,1,Info) of - {_,{_,Opts}} -> - Opts2 = - case lists:member(ber_bin_v2,Opts) of - true -> - [ber_bin,optimize] ++ lists:delete(optimize,Opts); - _ -> - Opts - end, - Opts3 = [X||X <- Opts2, - (X == ber orelse - X == ber_bin orelse - X == per orelse - X == per_bin orelse - X == optimize orelse - X == driver)], - lists:flatten(io_lib:format("~p",[Opts3])); + {ok,Info} = asn1rt:info('H323-MESSAGES'), + case lists:keyfind(options, 1, Info) of + {_,Opts0} -> + Opts1 = [X || X <- Opts0, + (X =:= ber orelse + X =:= per orelse + X =:= uper)], + lists:flatten(io_lib:format("~p", [Opts1])); _ -> "[]" end. diff --git a/lib/asn1/test/testTypeValueNotation.erl b/lib/asn1/test/testTypeValueNotation.erl index cd5223ef23..59f7385f08 100644 --- a/lib/asn1/test/testTypeValueNotation.erl +++ b/lib/asn1/test/testTypeValueNotation.erl @@ -21,11 +21,9 @@ -export([main/2]). --include_lib("test_server/include/test_server.hrl"). - -record('Seq', {octstr, int, bool, enum, bitstr, null, oid, vstr}). -main(Rule, Option) -> +main(_Rule, _Option) -> Value1 = #'Seq'{octstr = [1, 2, 3, 4], int = 12, bool = true, @@ -34,28 +32,5 @@ main(Rule, Option) -> null = 'NULL', oid = {1, 2, 55}, vstr = "Hello World"}, - Value2 = #'Seq'{octstr = {'OctStr', [1, 2, 3, 4]}, - int = {'Int', 12}, - bool = {'Bool', true}, - enum = {'Enum', a}, - bitstr = {'BitStr', [1, 0, 1, 0]}, - null = {'Null', 'NULL'}, - oid = {'OId', {1, 2, 55}}, - vstr = {'VStr', "Hello World"}}, - main(Rule, Option, Value1, Value2). - -%% Value2 will fail for ber_bin_v2, per_bin with nifs (optimize) and uper_bin -main(ber_bin_v2, _, Value1, Value2) -> encode_fail(Value1, Value2); -main(per_bin, [optimize], Value1, Value2) -> encode_fail(Value1, Value2); -main(uper_bin, [], Value1, Value2) -> encode_fail(Value1, Value2); -main(_, _, Value1, Value2) -> encode_normal(Value1, Value2). - -encode_normal(Value1, Value2) -> - {ok, Bytes} = asn1_wrapper:encode('SeqTypeRefPrim', 'Seq', Value1), - {ok, Bytes} = asn1_wrapper:encode('SeqTypeRefPrim', 'Seq', Value2), - {ok, Value1} = asn1_wrapper:decode('SeqTypeRefPrim', 'Seq', Bytes). - -encode_fail(Value1, Value2) -> - {ok, Bytes} = asn1_wrapper:encode('SeqTypeRefPrim', 'Seq', Value1), - {error, _Reason} = asn1_wrapper:encode('SeqTypeRefPrim', 'Seq', Value2), - {ok, Value1} = asn1_wrapper:decode('SeqTypeRefPrim', 'Seq', Bytes). + {ok, Bytes} = asn1_wrapper:encode('SeqTypeRefPrim', 'Seq', Value1), + {ok, Value1} = asn1_wrapper:decode('SeqTypeRefPrim', 'Seq', Bytes). diff --git a/lib/asn1/test/testX420.erl b/lib/asn1/test/testX420.erl index abdbbfe536..52b20a2c70 100644 --- a/lib/asn1/test/testX420.erl +++ b/lib/asn1/test/testX420.erl @@ -34,7 +34,7 @@ compile(Erule, Options, Config) -> compile_loop(_Erule, [], _Options, _Config) -> ok; compile_loop(Erule, [Spec|Specs], Options, Config) - when Erule == ber; Erule == ber_bin; Erule == ber_bin_v2; Erule == per -> + when Erule =:= ber; Erule =:= per -> CaseDir = ?config(case_dir, Config), asn1_test_lib:compile(filename:join([x420, Spec]), Config, [Erule, {i, CaseDir}]), diff --git a/lib/asn1/test/test_compile_options.erl b/lib/asn1/test/test_compile_options.erl index 4e732308d8..b973c5fbcc 100644 --- a/lib/asn1/test/test_compile_options.erl +++ b/lib/asn1/test/test_compile_options.erl @@ -92,7 +92,8 @@ noobj(Config) -> file:delete(filename:join([OutDir,'P-Record.beam'])), file:delete(filename:join([OutDir,'p_record.erl'])), file:delete(filename:join([OutDir,'p_record.beam'])), - ?line ok=asn1ct:compile(filename:join([DataDir,"p_record.set.asn"]),[asn1config,ber_bin,optimize,noobj,{outdir,OutDir}]), + ok = asn1ct:compile(filename:join([DataDir,"p_record.set.asn"]), + [asn1config,ber,noobj,{outdir,OutDir}]), %% ?line false = code:is_loaded('P-Record'), %% ?line false = code:is_loaded('p_record'), ?line {error,enoent} = diff --git a/lib/asn1/test/test_inline.erl b/lib/asn1/test/test_inline.erl index 62625572e3..e03ad739f9 100644 --- a/lib/asn1/test/test_inline.erl +++ b/lib/asn1/test/test_inline.erl @@ -41,16 +41,16 @@ inline1(Config, Rule, Opt) -> asn1_test_lib:compile("P-Record", Config, [{inline, 'inlined_P_Record'}|Opt]), test_inline1(), - ok=remove_inlined_files2(CaseDir, ber_bin_v2), + ok=remove_inlined_files2(CaseDir, ber), case Rule of - ber_bin_v2 -> + ber -> asn1_test_lib:compile("P-Record", Config, - [ber_bin, inline, asn1config, optimize|Opt]), + [ber, inline, asn1config|Opt]), test_inline2(Rule, 'P-Record'), remove_inlined_files3(CaseDir, Rule), asn1_test_lib:compile("p_record.set.asn", Config, - [ber_bin, inline, asn1config, optimize|Opt]), + [ber, inline, asn1config|Opt]), test_inline2(Rule, 'p_record'), remove_inlined_files4(CaseDir, Rule); _ -> @@ -71,12 +71,12 @@ test_inline1() -> ?line {ok,_}=asn1_wrapper:decode('inlined_P_Record', 'PersonnelRecord',Bytes). -test_inline2(ber_bin_v2,Mod) -> +test_inline2(ber,Mod) -> PRecMsg = {'PersonnelRecord',{'Name',"Sven","S","Svensson"}, "manager",123,"20000202",{'Name',"Inga","K","Svensson"}, asn1_DEFAULT}, ?line {ok,Bytes} = Mod:encode('PersonnelRecord',PRecMsg), - ?line {ok,_} = Mod:sel_dec(list_to_binary(Bytes)); + {ok,_} = Mod:sel_dec(Bytes); test_inline2(_,_) -> ok. @@ -243,7 +243,7 @@ remove_inlined_files2(Dir,Rule) -> ?line ok=file:delete(X) end,[TargetErl,TargetBeam]), ok. -remove_inlined_files3(Dir,ber_bin_v2) -> +remove_inlined_files3(Dir,ber) -> Erl=filename:join([Dir,"P-Record.erl"]), Beam=filename:join([Dir,"P-Record.beam"]), Asn1DB=filename:join([Dir,"P-Record.asn1db"]), @@ -255,7 +255,7 @@ remove_inlined_files3(Dir,ber_bin_v2) -> remove_inlined_files3(_,_) -> ok. -remove_inlined_files4(Dir,ber_bin_v2) -> +remove_inlined_files4(Dir,ber) -> Erl=filename:join([Dir,"p_record.erl"]), Beam=filename:join([Dir,"p_record.beam"]), Asn1DB=filename:join([Dir,"p_record.asn1db"]), diff --git a/lib/asn1/test/test_special_decode_performance.erl b/lib/asn1/test/test_special_decode_performance.erl index 4ac0ff2b27..dd56d29b28 100644 --- a/lib/asn1/test/test_special_decode_performance.erl +++ b/lib/asn1/test/test_special_decode_performance.erl @@ -33,7 +33,7 @@ go(all) -> go(N,Mod) -> ?line Val = val(Mod), ?line {ok,B} = Mod:encode(element(1,Val),Val), - ?line go(Mod,list_to_binary(B),N). + ?line go(Mod,B,N). go(Mod,Bin,N) -> ?line FsS = get_selective_funcs(Mod), diff --git a/lib/common_test/doc/src/common_test_app.xml b/lib/common_test/doc/src/common_test_app.xml index b6d4a633cb..151159ad69 100644 --- a/lib/common_test/doc/src/common_test_app.xml +++ b/lib/common_test/doc/src/common_test_app.xml @@ -4,7 +4,7 @@ <erlref> <header> <copyright> - <year>2003</year><year>2012</year> + <year>2003</year><year>2013</year> <holder>Ericsson AB. All Rights Reserved.</holder> </copyright> <legalnotice> @@ -170,7 +170,9 @@ <v> UserData = term()</v> <v> Conns = [atom()]</v> <v> CSSFile = string()</v> - <v> CTHs = [CTHModule | {CTHModule, CTHInitArgs} | {CTHModule, CTHInitArgs, CTHPriority}]</v> + <v> CTHs = [CTHModule |</v> + <v> {CTHModule, CTHInitArgs} |</v> + <v> {CTHModule, CTHInitArgs, CTHPriority}]</v> <v> CTHModule = atom()</v> <v> CTHInitArgs = term()</v> </type> @@ -297,8 +299,9 @@ <v> UserData = term()</v> <v> Conns = [atom()]</v> <v> CSSFile = string()</v> - <v> CTHs = [CTHModule | {CTHModule, CTHInitArgs} | - {CTHModule, CTHInitArgs, CTHPriority}]</v> + <v> CTHs = [CTHModule |</v> + <v> {CTHModule, CTHInitArgs} |</v> + <v> {CTHModule, CTHInitArgs, CTHPriority}]</v> <v> CTHModule = atom()</v> <v> CTHInitArgs = term()</v> </type> diff --git a/lib/common_test/doc/src/ct_hooks_chapter.xml b/lib/common_test/doc/src/ct_hooks_chapter.xml index 86237f5fc1..fe871eb516 100644 --- a/lib/common_test/doc/src/ct_hooks_chapter.xml +++ b/lib/common_test/doc/src/ct_hooks_chapter.xml @@ -4,7 +4,7 @@ <chapter> <header> <copyright> - <year>2011</year><year>2012</year> + <year>2011</year><year>2013</year> <holder>Ericsson AB. All Rights Reserved.</holder> </copyright> <legalnotice> @@ -439,14 +439,14 @@ terminate(State) -> <table> <row> - <cell><em>CTH Name</em></cell> - <cell><em>Is Built-in</em></cell> - <cell><em>Description</em></cell> + <cell align="left"><em>CTH Name</em></cell> + <cell align="left"><em>Is Built-in</em></cell> + <cell align="left"><em>Description</em></cell> </row> <row> - <cell>cth_log_redirect</cell> - <cell>yes</cell> - <cell>Captures all error_logger and SASL logging events and prints them + <cell align="left">cth_log_redirect</cell> + <cell align="left">yes</cell> + <cell align="left">Captures all error_logger and SASL logging events and prints them to the current test case log. If an event can not be associated with a testcase it will be printed in the common test framework log. This will happen for testcases which are run in parallel and events which occur @@ -455,14 +455,29 @@ terminate(State) -> using the normal SASL mechanisms. </cell> </row> <row> - <cell>cth_surefire</cell> - <cell>no</cell> - <cell>Captures all test results and outputs them as surefire XML into - a file. The file which is created is by default called junit_report.xml. - The name can be by setting the path option for this hook. e.g. + <cell align="left">cth_surefire</cell> + <cell align="left">no</cell> + <cell align="left"><p>Captures all test results and outputs them as surefire + XML into a file. The file which is created is by default + called junit_report.xml. The file name can be changed by + setting the <c>path</c> option for this hook, e.g.</p> + <code>-ct_hooks cth_surefire [{path,"/tmp/report.xml"}]</code> - Surefire XML can forinstance be used by Jenkins to display test - results.</cell> + + <p>If the <c>url_base</c> option is set, an additional + attribute named <c>url</c> will be added to each + <c>testsuite</c> and <c>testcase</c> XML element. The value will + be constructed from the <c>url_base</c> and a relative path + to the test suite or test case log respectively, e.g.</p> + + <code>-ct_hooks cth_surefire [{url_base, "http://myserver.com/"}]</code> + <p>will give a url attribute value similar to</p> + + <code>"http://myserver.com/[email protected]_11.19.39/ +x86_64-unknown-linux-gnu.my_test.logs/run.2012-12-12_11.19.39/suite.log.html"</code> + + <p>Surefire XML can for instance be used by Jenkins to display test + results.</p></cell> </row> </table> diff --git a/lib/common_test/doc/src/ct_run.xml b/lib/common_test/doc/src/ct_run.xml index da18640df7..0750f560b3 100644 --- a/lib/common_test/doc/src/ct_run.xml +++ b/lib/common_test/doc/src/ct_run.xml @@ -90,7 +90,7 @@ <pre> ct_run [-dir TestDir1 TestDir2 .. TestDirN] | [[-dir TestDir] -suite Suite1 Suite2 .. SuiteN - [[-group Group1 Group2 .. GroupN] [-case Case1 Case2 .. CaseN]]] + [[-group Groups1 Groups2 .. GroupsN] [-case Case1 Case2 .. CaseN]]] [-step [config | keep_inactive]] [-config ConfigFile1 ConfigFile2 .. ConfigFileN] [-userconfig CallbackModule1 ConfigString1 and CallbackModule2 diff --git a/lib/common_test/doc/src/notes.xml b/lib/common_test/doc/src/notes.xml index abe8cb2041..8c3b13951d 100644 --- a/lib/common_test/doc/src/notes.xml +++ b/lib/common_test/doc/src/notes.xml @@ -32,6 +32,150 @@ <file>notes.xml</file> </header> +<section><title>Common_Test 1.6.3.1</title> + + <section><title>Known Bugs and Problems</title> + <list> + <item> + <p> + The following corrections/changes are done in the + cth_surefire hook:</p> + <p> + <list> <item> Earlier there would always be a + 'properties' element under the 'testsuites' element. This + would exist even if there were no 'property' element + inside it. This has been changed so if there are no + 'property' elements to display, then there will not be a + 'properties' element either. </item> <item> The XML file + will now (unless other is specified) be stored in the top + log directory. Earlier, the default directory would be + the current working directory for the erlang node, which + would mostly, but not always, be the top log directory. + </item> <item> The 'hostname' attribute in the + 'testsuite' element would earlier never have the correct + value. This has been corrected. </item> <item> The + 'errors' attribute in the 'testsuite' element would + earlier display the number of failed testcases. This has + been changed and will now always have the value 0, while + the 'failures' attribute will show the number of failed + testcases. </item> <item> A new attribute 'skipped' is + added to the 'testsuite' element. This will display the + number of skipped testcases. These would earlier be + included in the number of failed test cases. </item> + <item> The total number of tests displayed by the 'tests' + attribute in the 'testsuite' element would earlier + include init/end_per_suite and init/end_per_group. This + is no longer the case. The 'tests' attribute will now + only count "real" test cases. </item> <item> Earlier, + auto skipped test cases would have no value in the 'log' + attribute. This is now corrected. </item> <item> A new + attributes 'log' is added to the 'testsuite' element. + </item> <item> A new option named 'url_base' is added for + this hook. If this option is used, a new attribute named + 'url' will be added to the 'testcase' and 'testsuite' + elements. </item> </list></p> + <p> + Own Id: OTP-10589</p> + </item> + </list> + </section> + +</section> + +<section><title>Common_Test 1.6.3</title> + + <section><title>Fixed Bugs and Malfunctions</title> + <list> + <item> + <p> + The ct:run_test/1 option 'config' only worked with a + single config file, not a list of files. This has been + fixed.</p> + <p> + Own Id: OTP-10495</p> + </item> + <item> + <p> + ct_netconfc:close_session sometimes returned + {error,closed} because the ssh connection was closed + (from the server side) before the rpc-reply was received + by the client. This is normal and can not be helped. It + has been corrected so the return will be 'ok' in this + case. Other error situations will still give + {error,Reason}.</p> + <p> + Own Id: OTP-10510 Aux Id: kunagi-320 [231] </p> + </item> + <item> + <p> + ct_netconfc:close_session sometimes returned + {error,closed} or (if the connection was named) + {error,{process_down,Pid,normal}} because the ssh + connection was closed (from the server side) before the + rpc-reply was received by the client. This is normal and + can not be helped. It has been corrected so the return + will be 'ok' in this situation.</p> + <p> + Own Id: OTP-10570</p> + </item> + <item> + <p> + Fix bug where ct:require of same name with same config + would return name_in_use.</p> + <p> + Own Id: OTP-10572</p> + </item> + </list> + </section> + + + <section><title>Improvements and New Features</title> + <list> + <item> + <p> + A new test case group search functionality has been + implemented that makes Common Test search automatically + through the group definitions tree (the return value of + groups/0) and create tests for all paths of nested groups + that match the specification. It also allows for + specifying unique paths to sub groups in order to avoid + execution of unwanted tests. This new feature can be used + whenever starting a test run by means of the ct_run + program, the ct:run_test/1 API function, or a Test + Specification. Details can be found in the Test Case + Group Execution section in the Running Tests chapter.</p> + <p> + Own Id: OTP-10466 Aux Id: kunagi-276 [187] </p> + </item> + </list> + </section> + + + <section><title>Known Bugs and Problems</title> + <list> + <item> + <p> + Restore Config data if lost when test case fails.</p> + <p> + Own Id: OTP-10070 Aux Id: kunagi-175 [86] </p> + </item> + <item> + <p> + IO server error in test_server.</p> + <p> + Own Id: OTP-10125 Aux Id: OTP-10101, kunagi-177 [88] </p> + </item> + <item> + <p> + Faulty connection handling in common_test.</p> + <p> + Own Id: OTP-10126 Aux Id: kunagi-178 [89] </p> + </item> + </list> + </section> + +</section> + <section><title>Common_Test 1.6.2.1</title> <section><title>Fixed Bugs and Malfunctions</title> diff --git a/lib/common_test/doc/src/run_test_chapter.xml b/lib/common_test/doc/src/run_test_chapter.xml index b5b914d506..d5f5d89e05 100644 --- a/lib/common_test/doc/src/run_test_chapter.xml +++ b/lib/common_test/doc/src/run_test_chapter.xml @@ -4,7 +4,7 @@ <chapter> <header> <copyright> - <year>2003</year><year>2012</year> + <year>2003</year><year>2013</year> <holder>Ericsson AB. All Rights Reserved.</holder> </copyright> <legalnotice> @@ -119,7 +119,7 @@ <item><c><![CDATA[ct_run -userconfig <callbackmodulename> <configfilenames> -suite <suiteswithfullpath>]]></c> </item> <item><c><![CDATA[ct_run -config <configfilenames> -suite <suitewithfullpath> - -group <groupnames> -case <casenames>]]></c></item> + -group <groups> -case <casenames>]]></c></item> </list> <p>Examples:</p> <p><c>$ ct_run -config $CFGS/sys1.cfg $CFGS/sys2.cfg -dir $SYS1_TEST $SYS2_TEST</c></p> @@ -137,6 +137,8 @@ <p><c>$ ct_run -suite ./testdir/x_SUITE ./testdir/y_SUITE</c></p> + <p>For more details on <seealso marker="run_test_chapter#group_execution">test case group execution</seealso>, please see below.</p> + <p>Other flags that may be used with <c>ct_run</c>:</p> <list> <item><c><![CDATA[-logdir <dir>]]></c>, specifies where the HTML log files are to be written.</item> @@ -269,6 +271,163 @@ <c><seealso marker="ct#run_test-1">ct</seealso></c> manual page.</p> </section> + <marker id="group_execution"></marker> + <section> + <title>Test case group execution</title> + + <p>With the <c>ct_run</c> flag, or <c>ct:run_test/1</c> option <c>group</c>, + one or more test case groups can be specified, optionally in combination + with specific test cases. The syntax for specifying groups is as follows + (on the command line):</p> + + <pre> + <![CDATA[$ ct_run -group <group_names_or_paths> [-case <cases>]]]></pre> + <p>or (in the Erlang shell):</p> + <pre> + <![CDATA[1> ct:run_test([{group,GroupsNamesOrPaths}, {case,Cases}]).]]></pre> + + <p>The <c>group_names_or_paths</c> parameter specifies either one + or more group names and/or one or more group paths. At start up, + Common Test will search for matching groups in the group definitions + tree (i.e. the list returned from <c>Suite:groups/0</c>, please see the + <seealso marker="write_test_chapter#test_case_groups">Test case groups</seealso> + chapter for details). + Given a group name, say <c>g</c>, Common Test will search for all paths + that lead to <c>g</c>. By path here we mean a sequence of nested groups, + all of which have to be followed in order to get from the top level + group to <c>g</c>. Actually, what Common Test needs to do in order to + execute the test cases in group <c>g</c>, is to call the + <c>init_per_group/2</c> function for each group in the path to + <c>g</c>, as well as all corresponding <c>end_per_group/2</c> + functions afterwards. The obvious reason for this is that the configuration + of a test case in <c>g</c> (and its <c>Config</c> input data) depends on + <c>init_per_testcase(TestCase, Config)</c> and its return value, which + in turn depends on <c>init_per_group(g, Config)</c> and its return value, + which in turn depends on <c>init_per_group/2</c> of the group above + <c>g</c>, etc, all the way up to the top level group.</p> + + <p>As you may have already realized, this means that if there is more than + one way to locate a group (and its test cases) in a path, the result of the + group search operation is a number of tests, all of which will be performed. + Common Test actually interprets a group specification that consists of a + single name this way:</p> + + <p>"Search and find all paths in the group definitions tree that lead + to the specified group and, for each path, create a test which (1) executes + all configuration functions in the path to the specified group, then (2) + executes all - or all matching - test cases in this group, as well as (3) + all - or all matching - test cases in all sub groups of the group". + </p> + + <p>It is also possible for the user to specify a specific group path with + the <c>group_names_or_paths</c> parameter. With this type of specification it's + possible to avoid execution of unwanted groups (in otherwise matching paths), + and/or the execution of sub groups. The syntax of the group path is a list of + group names in the path, e.g. on the command line: + </p> + <p><c>$ ct_run -suite "./x_SUITE" -group [g1,g3,g4] -case tc1 tc5</c></p> + <p>or similarly in the Erlang shell (requires a list within the groups list):</p> + <p><c>1> ct:run_test([{suite,"./x_SUITE"}, {group,[[g1,g3,g4]]}, {testcase,[tc1,tc5]}]).</c></p> + + <p>The last group in the specified path will be the terminating group in + the test, i.e. no sub groups following this group will be executed. In the + example above, <c>g4</c> is the terminating group, hence Common Test will + execute a test that calls all init configuration functions in the path to + <c>g4</c>, i.e. <c>g1..g3..g4</c>. It will then call test cases <c>tc1</c> + and <c>tc5</c> in <c>g4</c> and finally all end configuration functions in order + <c>g4..g3..g1</c>.</p> + + <p>Note that the group path specification doesn't necessarily + have to include <em>all</em> groups in the path to the terminating group. + Common Test will search for all matching paths if given an incomplete group + path.</p> + + <p>Note also that it's possible to combine group names and group paths with the + <c>group_names_or_paths</c> parameter. Each element is treated as + an individual specification in combination with the <c>cases</c> parameter. + See examples below.</p> + + <p>Examples:</p> + <pre> + -module(x_SUITE). + ... + %% The group definitions: + groups() -> + [{top1,[],[tc11,tc12, + {sub11,[],[tc12,tc13]}, + {sub12,[],[tc14,tc15, + {sub121,[],[tc12,tc16]}]}]}, + + {top2,[],[{group,sub21},{group,sub22}]}, + {sub21,[],[tc21,{group,sub2X2}]}, + {sub22,[],[{group,sub221},tc21,tc22,{group,sub2X2}]}, + {sub221,[],[tc21,tc23]}, + {sub2X2,[],[tc21,tc24]}]. + </pre> + <br></br> + <p><c>$ ct_run -suite "x_SUITE" -group all</c></p> + <p><c>1> ct:run_test([{suite,"x_SUITE"}, {group,all}]).</c></p> + <p>Two tests will be executed, one for all cases and all sub groups under <c>top1</c>, + and one for all under <c>top2</c>. (We would get the same result with + <c>-group top1 top2</c>, or <c>{group,[top1,top2]}</c>.</p> + <br></br> + <p><c>$ ct_run -suite "x_SUITE" -group top1</c></p> + <p><c>1> ct:run_test([{suite,"x_SUITE"}, {group,[top1]}]).</c></p> + <p>This will execute one test for all cases and sub groups under <c>top1</c>.</p> + <br></br> + <p><c>$ ct_run -suite "x_SUITE" -group top1 -case tc12</c></p> + <p><c>1> ct:run_test([{suite,"x_SUITE"}, {group,[top1]}, {testcase,[tc12]}]).</c></p> + <p>This will run a test that executes <c>tc12</c> in <c>top1</c> and any sub group + under <c>top1</c> where it can be found (<c>sub11</c> and <c>sub121</c>).</p> + <br></br> + <p><c>$ ct_run -suite "x_SUITE" -group [top1] -case tc12</c></p> + <p><c>1> ct:run_test([{suite,"x_SUITE"}, {group,[[top1]]}, {testcase,[tc12]}]).</c></p> + <p>This will execute <c>tc12</c> <em>only</em> in group <c>top1</c>.</p> + <br></br> + <p><c>$ ct_run -suite "x_SUITE" -group top1 -case tc16</c></p> + <p><c>1> ct:run_test([{suite,"x_SUITE"}, {group,[top1]}, {testcase,[tc16]}]).</c></p> + <p>This will search <c>top1</c> and all its sub groups for <c>tc16</c> and the result + will be that this test case executes in group <c>sub121</c>. (The specific path: + <c>-group [sub121]</c> or <c>{group,[[sub121]]}</c>, would have given + us the same result in this example).</p> + <br></br> + <p><c>$ ct_run -suite "x_SUITE" -group sub12 [sub12]</c></p> + <p><c>1> ct:run_test([{suite,"x_SUITE"}, {group,[sub12,[sub12]]}]).</c></p> + <p>This will execute two tests, one that includes all cases and sub groups under + <c>sub12</c>, and one with <em>only</em> the test cases in <c>sub12</c>.</p> + <br></br> + <p><c>$ ct_run -suite "x_SUITE" -group sub2X2</c></p> + <p><c>1> ct:run_test([{suite,"x_SUITE"}, {group,[sub2X2]}]).</c></p> + <p>In this example, Common Test will find and execute two tests, one for the path from + <c>top2</c> to <c>sub2X2</c> via <c>sub21</c>, and one from <c>top2</c> to <c>sub2X2</c> + via <c>sub22</c>.</p> + <br></br> + <p><c>$ ct_run -suite "x_SUITE" -group [sub21,sub2X2]</c></p> + <p><c>1> ct:run_test([{suite,"x_SUITE"}, {group,[[sub21,sub2X2]]}]).</c></p> + <p>Here, by specifying the unique path: <c>top2 -> sub21 -> sub2X2</c>, only one test + is executed. The second possible path from <c>top2</c> to <c>sub2X2</c> (above) + will be discarded.</p> + <br></br> + <p><c>$ ct_run -suite "x_SUITE" -group [sub22] -case tc22 tc21</c></p> + <p><c>1> ct:run_test([{suite,"x_SUITE"}, {group,[[sub22]]}, {testcase,[tc22,tc21]}]).</c></p> + <p>In this example only the test cases for <c>sub22</c> will be executed, and in + reverse order compared to the group definition.</p> + <br></br> + + <p>If a test case that belongs to a group (according to the group definition), is executed + without a group specification, i.e. simply by means of (command line):</p> + <p><c>$ ct_run -suite "my_SUITE" -case my_tc</c></p> + <p>or (Erlang shell):</p> + <p><c>1> ct:run_test([{suite,"my_SUITE"}, {testcase,my_tc}]).</c></p> + <p>then Common Test ignores the group definition and executes the test case in the scope of the + test suite only (no group configuration functions are called).</p> + + <p>The group specification feature, exactly as it has been presented in this section, can also + be used in <seealso marker="run_test_chapter#test_specifications">Test + Specifications</seealso> (with some extra features added). Please see below.</p> + </section> + + <section> <title>Running the interactive shell mode</title> @@ -400,8 +559,8 @@ terms (e.g. log directory, label, style sheet, auto compilation).</p> <p>With test specification terms it is possible to state exactly which tests should run and in which order. A test term specifies - either one or more suites, one or more test case groups, or one - or more test cases in a group or suite.</p> + either one or more suites, one or more test case groups (possibly nested), + or one or more test cases in a group (or in multiple groups) or in a suite.</p> <p>An arbitrary number of test terms may be declared in sequence. Common Test will by default compile the terms into one or more tests to be performed in one resulting test run. Note that a term that @@ -420,28 +579,32 @@ are not executed and show up in the HTML log files as SKIPPED.</p> <p>When a test case group is specified, the resulting test - executes the - <c>init_per_group</c> function, followed by all test cases and - sub groups (including their configuration functions), and + executes the <c>init_per_group</c> function, followed by all test + cases and sub groups (including their configuration functions), and finally the <c>end_per_group</c> function. Also if particular test cases in a group are specified, <c>init_per_group</c> and <c>end_per_group</c> for the group in question are called. If a group which is defined (in <c>Suite:group/0</c>) to - be a sub group of another group, is specified (or particular test + be a sub group of another group, is specified (or if particular test cases of a sub group are), Common Test will call the configuration functions for the top level groups as well as for the sub group in question (making it possible to pass configuration data all the way from <c>init_per_suite</c> down to the test cases in the sub group).</p> - - <p>With the <c>GroupSpec</c> element (below) it's possible to specify - group execution properties that will override those specified in the + <p>The test specification utilizes the same mechanism for specifying + test case groups by means of names and paths, as explained in the + <seealso marker="run_test_chapter#group_execution">Group Execution</seealso> + section above, with the addition of the <c>GroupSpec</c> element + described next.</p> + <p>The <c>GroupSpec</c> element makes it possible to specify + group execution properties that will override those in the group definition (i.e. in <c>groups/0</c>). Execution properties for sub-groups may be overridden as well. This feature makes it possible to change properties of groups at the time of execution, - without even having to edit the test suite. More detailed documentation, - and examples, can be found in the - <seealso marker="write_test_chapter#test_case_groups"> + without even having to edit the test suite. The very same + feature is available for <c>group</c> elements in the <c>Suite:all/0</c> + list. Therefore, more detailed documentation, and examples, can be + found in the <seealso marker="write_test_chapter#test_case_groups"> Test case groups</seealso> chapter.</p> <p>Below is the test specification syntax. Test specifications can @@ -546,8 +709,8 @@ {groups, Dir, Suite, Groups}. {groups, NodeRefs, Dir, Suite, Groups}. - {groups, Dir, Suite, GroupSpec, {cases,Cases}}. - {groups, NodeRefs, Dir, Suite, GroupSpec, {cases,Cases}}. + {groups, Dir, Suite, Groups, {cases,Cases}}. + {groups, NodeRefs, Dir, Suite, Groups, {cases,Cases}}. {cases, Dir, Suite, Cases}. {cases, NodeRefs, Dir, Suite, Cases}. @@ -589,13 +752,16 @@ PrivDirOption = auto_per_run | auto_per_tc | manual_per_tc EventHandlers = atom() | [atom()] InitArgs = [term()] - CTHModules = [CTHModule | {CTHModule, CTHInitArgs} | {CTHModule, CTHInitArgs, CTHPriority}] + CTHModules = [CTHModule | + {CTHModule, CTHInitArgs} | + {CTHModule, CTHInitArgs, CTHPriority}] CTHModule = atom() CTHInitArgs = term() Dir = string() Suites = atom() | [atom()] | all Suite = atom() - Groups = GroupSpec | [GroupSpec] | all + Groups = GroupPath | [GroupPath] | GroupSpec | [GroupSpec] | all + GroupPath = [GroupName] GroupSpec = GroupName | {GroupName,Properties} | {GroupName,Properties,GroupSpec} GroupName = atom() GroupNames = GroupName | [GroupName] diff --git a/lib/common_test/doc/src/write_test_chapter.xml b/lib/common_test/doc/src/write_test_chapter.xml index 248d7de8b6..cc8d913994 100644 --- a/lib/common_test/doc/src/write_test_chapter.xml +++ b/lib/common_test/doc/src/write_test_chapter.xml @@ -4,7 +4,7 @@ <chapter> <header> <copyright> - <year>2003</year><year>2012</year> + <year>2003</year><year>2013</year> <holder>Ericsson AB. All Rights Reserved.</holder> </copyright> <legalnotice> @@ -982,38 +982,36 @@ <p>Example:</p> <pre> + Some printouts during test case execution: - Some printouts during test case execution: + io:format("1. Standard IO, importance = ~w~n", [?STD_IMPORTANCE]), + ct:log("2. Uncategorized, importance = ~w", [?STD_IMPORTANCE]), + ct:log(info, "3. Categorized info, importance = ~w", [?STD_IMPORTANCE]]), + ct:log(info, ?LOW_IMPORTANCE, "4. Categorized info, importance = ~w", [?LOW_IMPORTANCE]), + ct:log(error, "5. Categorized error, importance = ~w", [?HI_IMPORTANCE]), + ct:log(error, ?HI_IMPORTANCE, "6. Categorized error, importance = ~w", [?MAX_IMPORTANCE]), - io:format("1. Standard IO, importance = ~w~n", [?STD_IMPORTANCE]), - ct:log("2. Uncategorized, importance = ~w", [?STD_IMPORTANCE]), - ct:log(info, "3. Categorized info, importance = ~w", [?STD_IMPORTANCE]]), - ct:log(info, ?LOW_IMPORTANCE, "4. Categorized info, importance = ~w", [?LOW_IMPORTANCE]), - ct:log(error, "5. Categorized error, importance = ~w", [?HI_IMPORTANCE]), - ct:log(error, ?HI_IMPORTANCE, "6. Categorized error, importance = ~w", [?MAX_IMPORTANCE]), + If starting the test without specifying any verbosity levels: - If starting the test without specifying any verbosity levels: + $ ct_run ... - $ ct_run ... + the following gets printed: - the following gets printed: - - 1. Standard IO, importance = 50 - 2. Uncategorized, importance = 50 - 3. Categorized info, importance = 50 - 5. Categorized error, importance = 75 - 6. Categorized error, importance = 99 - - If starting the test with: - - $ ct_run -verbosity 1 and info 75 - - the following gets printed: + 1. Standard IO, importance = 50 + 2. Uncategorized, importance = 50 + 3. Categorized info, importance = 50 + 5. Categorized error, importance = 75 + 6. Categorized error, importance = 99 + + If starting the test with: + + $ ct_run -verbosity 1 and info 75 + + the following gets printed: - 3. Categorized info, importance = 50 - 4. Categorized info, importance = 25 - 6. Categorized error, importance = 99 - </pre> + 3. Categorized info, importance = 50 + 4. Categorized info, importance = 25 + 6. Categorized error, importance = 99</pre> <p>How categories can be mapped to CSS tags is documented in the <seealso marker="run_test_chapter#html_stylesheet">Running Tests</seealso> diff --git a/lib/common_test/src/ct.erl b/lib/common_test/src/ct.erl index 6ecd0a022c..8eafdff29f 100644 --- a/lib/common_test/src/ct.erl +++ b/lib/common_test/src/ct.erl @@ -161,7 +161,8 @@ run(TestDirs) -> %%% TestDirs = [string()] | string() %%% Suites = [string()] | [atom()] | string() | atom() %%% Cases = [atom()] | atom() -%%% Groups = [atom()] | atom() +%%% Groups = GroupNameOrPath | [GroupNameOrPath] +%%% GroupNameOrPath = [atom()] | atom() | all %%% TestSpecs = [string()] | string() %%% Label = string() | atom() %%% CfgFiles = [string()] | string() diff --git a/lib/common_test/src/ct_groups.erl b/lib/common_test/src/ct_groups.erl index 24ca3826a8..74ab5e5439 100644 --- a/lib/common_test/src/ct_groups.erl +++ b/lib/common_test/src/ct_groups.erl @@ -176,7 +176,7 @@ find(Mod, GrNames, all, [{M,TC} | Gs], Known, %% Check if test case should be saved find(Mod, GrNames, TCs, [TC | Gs], Known, Defs, FindAll) when is_atom(TC) orelse - ((size(TC) == 2) and (hd(TC) /= group)) -> + ((size(TC) == 2) and (element(1,TC) /= group)) -> Case = if is_atom(TC) -> Tuple = {Mod,TC}, diff --git a/lib/common_test/src/ct_netconfc.erl b/lib/common_test/src/ct_netconfc.erl index 294b82bff6..1ccbc86d8f 100644 --- a/lib/common_test/src/ct_netconfc.erl +++ b/lib/common_test/src/ct_netconfc.erl @@ -307,7 +307,7 @@ -type option() :: {ssh,host()} | {port,inet:port_number()} | {user,string()} | {password,string()} | {user_dir,string()} | {timeout,timeout()}. --type host() :: inet:host_name() | inet:ip_address(). +-type host() :: inet:hostname() | inet:ip_address(). -type notification() :: {notification, xml_attributes(), notification_content()}. -type notification_content() :: [event_time()|simple_xml()]. @@ -1073,7 +1073,8 @@ handle_msg({get_event_streams=Op,Streams,Timeout}, From, State) -> SimpleXml = encode_rpc_operation(get,[Filter]), do_send_rpc(Op, SimpleXml, Timeout, From, State). -handle_msg({ssh_cm, _CM, {data, _Ch, _Type, Data}}, State) -> +handle_msg({ssh_cm, CM, {data, Ch, _Type, Data}}, State) -> + ssh_connection:adjust_window(CM,Ch,size(Data)), handle_data(Data, State); handle_msg({ssh_cm, _CM, _SshCloseMsg}, State) -> %% _SshCloseMsg can probably be one of @@ -1805,7 +1806,8 @@ get_tag([]) -> %%% SSH stuff ssh_receive_data() -> receive - {ssh_cm, _CM, {data, _Ch, _Type, Data}} -> + {ssh_cm, CM, {data, Ch, _Type, Data}} -> + ssh_connection:adjust_window(CM,Ch,size(Data)), {ok, Data}; {ssh_cm, _CM, {Closed, _Ch}} = X when Closed == closed; Closed == eof -> {error,X}; diff --git a/lib/common_test/src/ct_slave.erl b/lib/common_test/src/ct_slave.erl index cb05423497..1fd8c04f8b 100644 --- a/lib/common_test/src/ct_slave.erl +++ b/lib/common_test/src/ct_slave.erl @@ -37,7 +37,7 @@ -record(options, {username, password, boot_timeout, init_timeout, startup_timeout, startup_functions, monitor_master, - kill_if_fail, erl_flags}). + kill_if_fail, erl_flags, env}). %%%----------------------------------------------------------------- %%% @spec start(Node) -> Result @@ -85,7 +85,8 @@ start(Host, Node) -> %%% {startup_functions, StartupFunctions} | %%% {monitor_master, Monitor} | %%% {kill_if_fail, KillIfFail} | -%%% {erl_flags, ErlangFlags} +%%% {erl_flags, ErlangFlags} | +%%% {env, [{EnvVar,Value}]} %%% Username = string() %%% Password = string() %%% BootTimeout = integer() @@ -99,6 +100,8 @@ start(Host, Node) -> %%% Monitor = bool() %%% KillIfFail = bool() %%% ErlangFlags = string() +%%% EnvVar = string() +%%% Value = string() %%% Result = {ok, NodeName} | {error, already_started, NodeName} | %%% {error, started_not_connected, NodeName} | %%% {error, boot_timeout, NodeName} | @@ -152,6 +155,9 @@ start(Host, Node) -> %%% <p>Option <code>erlang_flags</code> specifies, which flags will be added %%% to the parameters of the <code>erl</code> executable.</p> %%% +%%% <p>Option <code>env</code> specifies a list of environment variables +%%% that will extended the environment.</p> +%%% %%% <p>Special return values are: %%% <list> %%% <item><code>{error, already_started, NodeName}</code> - if the node with @@ -233,10 +239,12 @@ fetch_options(Options) -> Monitor = get_option_value(monitor_master, Options, false), KillIfFail = get_option_value(kill_if_fail, Options, true), ErlFlags = get_option_value(erl_flags, Options, []), + EnvVars = get_option_value(env, Options, []), #options{username=UserName, password=Password, boot_timeout=BootTimeout, init_timeout=InitTimeout, startup_timeout=StartupTimeout, startup_functions=StartupFunctions, - monitor_master=Monitor, kill_if_fail=KillIfFail, erl_flags=ErlFlags}. + monitor_master=Monitor, kill_if_fail=KillIfFail, + erl_flags=ErlFlags, env=EnvVars}. % send a message when slave node is started % @hidden @@ -306,6 +314,7 @@ do_start(Host, Node, Options) -> true-> spawn_remote_node(Host, Node, Options) end, + BootTimeout = Options#options.boot_timeout, InitTimeout = Options#options.init_timeout, StartupTimeout = Options#options.startup_timeout, @@ -372,9 +381,9 @@ get_cmd(Node, Flags) -> % spawn node locally spawn_local_node(Node, Options) -> - ErlFlags = Options#options.erl_flags, + #options{env=Env,erl_flags=ErlFlags} = Options, Cmd = get_cmd(Node, ErlFlags), - open_port({spawn, Cmd}, [stream]). + open_port({spawn, Cmd}, [stream,{env,Env}]). % start crypto and ssh if not yet started check_for_ssh_running() -> @@ -393,9 +402,10 @@ check_for_ssh_running() -> % spawn node remotely spawn_remote_node(Host, Node, Options) -> - Username = Options#options.username, - Password = Options#options.password, - ErlFlags = Options#options.erl_flags, + #options{username=Username, + password=Password, + erl_flags=ErlFlags, + env=Env} = Options, SSHOptions = case {Username, Password} of {[], []}-> []; @@ -407,8 +417,17 @@ spawn_remote_node(Host, Node, Options) -> check_for_ssh_running(), {ok, SSHConnRef} = ssh:connect(atom_to_list(Host), 22, SSHOptions), {ok, SSHChannelId} = ssh_connection:session_channel(SSHConnRef, infinity), + ssh_setenv(SSHConnRef, SSHChannelId, Env), ssh_connection:exec(SSHConnRef, SSHChannelId, get_cmd(Node, ErlFlags), infinity). + +ssh_setenv(SSHConnRef, SSHChannelId, [{Var, Value} | Vars]) + when is_list(Var), is_list(Value) -> + success = ssh_connection:setenv(SSHConnRef, SSHChannelId, + Var, Value, infinity), + ssh_setenv(SSHConnRef, SSHChannelId, Vars); +ssh_setenv(_SSHConnRef, _SSHChannelId, []) -> ok. + % call functions on a remote Erlang node call_functions(_Node, []) -> ok; @@ -430,15 +449,29 @@ wait_for_node_alive(Node, N) -> % call init:stop on a remote node do_stop(ENode) -> - case test_server:is_cover() of - true -> - MainCoverNode = cover:get_main_node(), - rpc:call(MainCoverNode,cover,flush,[ENode]); - false -> - ok + {Cover,MainCoverNode} = + case test_server:is_cover() of + true -> + Main = cover:get_main_node(), + rpc:call(Main,cover,flush,[ENode]), + {true,Main}; + false -> + {false,undefined} end, spawn(ENode, init, stop, []), - wait_for_node_dead(ENode, 5). + case wait_for_node_dead(ENode, 5) of + {ok,ENode} -> + if Cover -> + %% To avoid that cover is started again if a node + %% with the same name is started later. + rpc:call(MainCoverNode,cover,stop,[ENode]); + true -> + ok + end, + {ok,ENode}; + Error -> + Error + end. % wait N seconds until node is disconnected wait_for_node_dead(Node, 0) -> diff --git a/lib/common_test/src/cth_conn_log.erl b/lib/common_test/src/cth_conn_log.erl index 3af89db3a5..255f3ec78a 100644 --- a/lib/common_test/src/cth_conn_log.erl +++ b/lib/common_test/src/cth_conn_log.erl @@ -58,7 +58,7 @@ -spec init(Id, HookOpts) -> Result when Id :: term(), - HookOpts :: ct:hook_options(), + HookOpts :: ct_netconfc:hook_options(), Result :: {ok,[{ct_netconfc:conn_mod(), {ct_netconfc:log_type(),[ct_netconfc:key_or_name()]}}]}. init(_Id, HookOpts) -> diff --git a/lib/common_test/src/cth_surefire.erl b/lib/common_test/src/cth_surefire.erl index 76b0f0b5ea..e6eaad8d48 100644 --- a/lib/common_test/src/cth_surefire.erl +++ b/lib/common_test/src/cth_surefire.erl @@ -1,3 +1,22 @@ +%%-------------------------------------------------------------------- +%% %CopyrightBegin% +%% +%% Copyright Ericsson AB 2012. All Rights Reserved. +%% +%% The contents of this file are subject to the Erlang Public License, +%% Version 1.1, (the "License"); you may not use this file except in +%% compliance with the License. You should have received a copy of the +%% Erlang Public License along with this software. If not, it can be +%% retrieved online at http://www.erlang.org/. +%% +%% Software distributed under the License is distributed on an "AS IS" +%% basis, WITHOUT WARRANTY OF ANY KIND, either express or implied. See +%% the License for the specific language governing rights and limitations +%% under the License. +%% +%% %CopyrightEnd% +%%-------------------------------------------------------------------- + %%% @doc Common Test Framework functions handling test specifications. %%% %%% <p>This module creates a junit report of the test run if plugged in @@ -27,18 +46,28 @@ -export([terminate/1]). -record(state, { filepath, axis, properties, package, hostname, - curr_suite, curr_suite_ts, curr_group = [], curr_tc, - curr_log_dir, timer, tc_log, + curr_suite, curr_suite_ts, curr_group = [], + curr_log_dir, timer, tc_log, url_base, test_cases = [], test_suites = [] }). --record(testcase, { log, group, classname, name, time, failure, timestamp }). --record(testsuite, { errors, failures, hostname, name, tests, +-record(testcase, { log, url, group, classname, name, time, result, timestamp }). +-record(testsuite, { errors, failures, skipped, hostname, name, tests, time, timestamp, id, package, - properties, testcases }). + properties, testcases, log, url }). + +-define(default_report,"junit_report.xml"). +-define(suite_log,"suite.log.html"). + +%% Number of dirs from log root to testcase log file. +%% ct_run.<node>.<timestamp>/<test_name>/run.<timestamp>/<tc_log>.html +-define(log_depth,3). id(Opts) -> - filename:absname(proplists:get_value(path, Opts, "junit_report.xml")). + case proplists:get_value(path, Opts) of + undefined -> ?default_report; + Path -> filename:absname(Path) + end. init(Path, Opts) -> {ok, Host} = inet:gethostname(), @@ -47,10 +76,24 @@ init(Path, Opts) -> package = proplists:get_value(package,Opts), axis = proplists:get_value(axis,Opts,[]), properties = proplists:get_value(properties,Opts,[]), + url_base = proplists:get_value(url_base,Opts), timer = now() }. pre_init_per_suite(Suite,Config,#state{ test_cases = [] } = State) -> - {Config, init_tc(State#state{ curr_suite = Suite, curr_suite_ts = now() }, + TcLog = proplists:get_value(tc_logfile,Config), + CurrLogDir = filename:dirname(TcLog), + Path = + case State#state.filepath of + ?default_report -> + RootDir = get_test_root(TcLog), + filename:join(RootDir,?default_report); + P -> + P + end, + {Config, init_tc(State#state{ filepath = Path, + curr_suite = Suite, + curr_suite_ts = now(), + curr_log_dir = CurrLogDir}, Config) }; pre_init_per_suite(Suite,Config,State) -> %% Have to close the previous suite @@ -59,7 +102,8 @@ pre_init_per_suite(Suite,Config,State) -> post_init_per_suite(_Suite,Config, Result, State) -> {Result, end_tc(init_per_suite,Config,Result,State)}. -pre_end_per_suite(_Suite,Config,State) -> {Config, init_tc(State, Config)}. +pre_end_per_suite(_Suite,Config,State) -> + {Config, init_tc(State, Config)}. post_end_per_suite(_Suite,Config,Result,State) -> {Result, end_tc(end_per_suite,Config,Result,State)}. @@ -71,13 +115,15 @@ pre_init_per_group(Group,Config,State) -> post_init_per_group(_Group,Config,Result,State) -> {Result, end_tc(init_per_group,Config,Result,State)}. -pre_end_per_group(_Group,Config,State) -> {Config, init_tc(State, Config)}. +pre_end_per_group(_Group,Config,State) -> + {Config, init_tc(State, Config)}. post_end_per_group(_Group,Config,Result,State) -> NewState = end_tc(end_per_group, Config, Result, State), {Result, NewState#state{ curr_group = tl(NewState#state.curr_group)}}. -pre_init_per_testcase(_TC,Config,State) -> {Config, init_tc(State, Config)}. +pre_init_per_testcase(_TC,Config,State) -> + {Config, init_tc(State, Config)}. post_end_per_testcase(TC,Config,Result,State) -> {Result, end_tc(TC,Config, Result,State)}. @@ -88,11 +134,19 @@ on_tc_fail(_TC, Res, State) -> TCs = State#state.test_cases, TC = hd(TCs), NewTC = TC#testcase{ - failure = + result = {fail,lists:flatten(io_lib:format("~p",[Res]))} }, State#state{ test_cases = [NewTC | tl(TCs)]}. -on_tc_skip(Tc,{Type,_Reason} = Res, State) when Type == tc_auto_skip -> +on_tc_skip(Tc,{Type,_Reason} = Res, State0) when Type == tc_auto_skip -> + TcStr = atom_to_list(Tc), + State = + case State0#state.test_cases of + [#testcase{name=TcStr}|TCs] -> + State0#state{test_cases=TCs}; + _ -> + State0 + end, do_tc_skip(Res, end_tc(Tc,[],Res,init_tc(State,[]))); on_tc_skip(_Tc, _Res, State = #state{test_cases = []}) -> State; @@ -103,7 +157,7 @@ do_tc_skip(Res, State) -> TCs = State#state.test_cases, TC = hd(TCs), NewTC = TC#testcase{ - failure = + result = {skipped,lists:flatten(io_lib:format("~p",[Res]))} }, State#state{ test_cases = [NewTC | tl(TCs)]}. @@ -117,33 +171,52 @@ end_tc(Func, Config, Res, State) when is_atom(Func) -> end_tc(atom_to_list(Func), Config, Res, State); end_tc(Name, _Config, _Res, State = #state{ curr_suite = Suite, curr_group = Groups, - timer = TS, tc_log = Log } ) -> + curr_log_dir = CurrLogDir, + timer = TS, + tc_log = Log0, + url_base = UrlBase } ) -> + Log = + case Log0 of + "" -> + LowerSuiteName = string:to_lower(atom_to_list(Suite)), + filename:join(CurrLogDir,LowerSuiteName++"."++Name++".html"); + _ -> + Log0 + end, + Url = make_url(UrlBase,Log), ClassName = atom_to_list(Suite), PGroup = string:join([ atom_to_list(Group)|| Group <- lists:reverse(Groups)],"."), TimeTakes = io_lib:format("~f",[timer:now_diff(now(),TS) / 1000000]), State#state{ test_cases = [#testcase{ log = Log, + url = Url, timestamp = now_to_string(TS), classname = ClassName, group = PGroup, name = Name, time = TimeTakes, - failure = passed }| State#state.test_cases]}. + result = passed }| + State#state.test_cases], + tc_log = ""}. % so old tc_log is not set if next is on_tc_skip close_suite(#state{ test_cases = [] } = State) -> State; -close_suite(#state{ test_cases = TCs } = State) -> - Total = length(TCs), - Succ = length(lists:filter(fun(#testcase{ failure = F }) -> - F == passed - end,TCs)), - Fail = Total - Succ, +close_suite(#state{ test_cases = TCs, url_base = UrlBase } = State) -> + {Total,Fail,Skip} = count_tcs(TCs,0,0,0), TimeTaken = timer:now_diff(now(),State#state.curr_suite_ts) / 1000000, + SuiteLog = filename:join(State#state.curr_log_dir,?suite_log), + SuiteUrl = make_url(UrlBase,SuiteLog), Suite = #testsuite{ name = atom_to_list(State#state.curr_suite), package = State#state.package, + hostname = State#state.hostname, time = io_lib:format("~f",[TimeTaken]), timestamp = now_to_string(State#state.curr_suite_ts), - errors = Fail, tests = Total, - testcases = lists:reverse(TCs) }, + errors = 0, + failures = Fail, + skipped = Skip, + tests = Total, + testcases = lists:reverse(TCs), + log = SuiteLog, + url = SuiteUrl}, State#state{ test_cases = [], test_suites = [Suite | State#state.test_suites]}. @@ -159,14 +232,15 @@ terminate(State) -> -to_xml(#testcase{ group = Group, classname = CL, log = L, name = N, time = T, timestamp = TS, failure = F}) -> +to_xml(#testcase{ group = Group, classname = CL, log = L, url = U, name = N, time = T, timestamp = TS, result = R}) -> ["<testcase ", - [["group=\"",Group,"\""]||Group /= ""]," " + [["group=\"",Group,"\" "]||Group /= ""], "name=\"",N,"\" " "time=\"",T,"\" " - "timestamp=\"",TS,"\" " + "timestamp=\"",TS,"\" ", + [["url=\"",U,"\" "]||U /= undefined], "log=\"",L,"\">", - case F of + case R of passed -> []; {skipped,Reason} -> @@ -176,22 +250,29 @@ to_xml(#testcase{ group = Group, classname = CL, log = L, name = N, time = T, ti ["<failure message=\"Test ",N," in ",CL," failed!\" type=\"crash\">", sanitize(Reason),"</failure>"] end,"</testcase>"]; -to_xml(#testsuite{ package = P, hostname = H, errors = E, time = Time, - timestamp = TS, tests = T, name = N, testcases = Cases }) -> +to_xml(#testsuite{ package = P, hostname = H, errors = E, failures = F, + skipped = S, time = Time, timestamp = TS, tests = T, name = N, + testcases = Cases, log = Log, url = Url }) -> ["<testsuite ", [["package=\"",P,"\" "]||P /= undefined], - [["hostname=\"",P,"\" "]||H /= undefined], - [["name=\"",N,"\" "]||N /= undefined], - [["time=\"",Time,"\" "]||Time /= undefined], - [["timestamp=\"",TS,"\" "]||TS /= undefined], + "hostname=\"",H,"\" " + "name=\"",N,"\" " + "time=\"",Time,"\" " + "timestamp=\"",TS,"\" " "errors=\"",integer_to_list(E),"\" " - "tests=\"",integer_to_list(T),"\">", + "failures=\"",integer_to_list(F),"\" " + "skipped=\"",integer_to_list(S),"\" " + "tests=\"",integer_to_list(T),"\" ", + [["url=\"",Url,"\" "]||Url /= undefined], + "log=\"",Log,"\">", [to_xml(Case) || Case <- Cases], "</testsuite>"]; to_xml(#state{ test_suites = TestSuites, axis = Axis, properties = Props }) -> ["<testsuites>",properties_to_xml(Axis,Props), [to_xml(TestSuite) || TestSuite <- TestSuites],"</testsuites>"]. +properties_to_xml([],[]) -> + []; properties_to_xml(Axis,Props) -> ["<properties>", [["<property name=\"",Name,"\" axis=\"yes\" value=\"",Value,"\" />"] || {Name,Value} <- Axis], @@ -217,3 +298,37 @@ sanitize([]) -> now_to_string(Now) -> {{YY,MM,DD},{HH,Mi,SS}} = calendar:now_to_local_time(Now), io_lib:format("~p-~2..0B-~2..0BT~2..0B:~2..0B:~2..0B",[YY,MM,DD,HH,Mi,SS]). + +make_url(undefined,_) -> + undefined; +make_url(_,[]) -> + undefined; +make_url(UrlBase0,Log) -> + UrlBase = string:strip(UrlBase0,right,$/), + RelativeLog = get_relative_log_url(Log), + string:join([UrlBase,RelativeLog],"/"). + +get_test_root(Log) -> + LogParts = filename:split(Log), + filename:join(lists:sublist(LogParts,1,length(LogParts)-?log_depth)). + +get_relative_log_url(Log) -> + LogParts = filename:split(Log), + Start = length(LogParts)-?log_depth, + Length = ?log_depth+1, + string:join(lists:sublist(LogParts,Start,Length),"/"). + +count_tcs([#testcase{name=ConfCase}|TCs],Ok,F,S) + when ConfCase=="init_per_suite"; + ConfCase=="end_per_suite"; + ConfCase=="init_per_group"; + ConfCase=="end_per_group" -> + count_tcs(TCs,Ok,F,S); +count_tcs([#testcase{result=passed}|TCs],Ok,F,S) -> + count_tcs(TCs,Ok+1,F,S); +count_tcs([#testcase{result={fail,_}}|TCs],Ok,F,S) -> + count_tcs(TCs,Ok,F+1,S); +count_tcs([#testcase{result={skipped,_}}|TCs],Ok,F,S) -> + count_tcs(TCs,Ok,F,S+1); +count_tcs([],Ok,F,S) -> + {Ok+F+S,F,S}. diff --git a/lib/common_test/test/Makefile b/lib/common_test/test/Makefile index df816f9a61..d469d03e04 100644 --- a/lib/common_test/test/Makefile +++ b/lib/common_test/test/Makefile @@ -56,7 +56,8 @@ MODULES= \ ct_snmp_SUITE \ ct_group_leader_SUITE \ ct_cover_SUITE \ - ct_groups_search_SUITE + ct_groups_search_SUITE \ + ct_surefire_SUITE ERL_FILES= $(MODULES:%=%.erl) diff --git a/lib/common_test/test/ct_groups_search_SUITE_data/groups_search_dummy_1_SUITE.erl b/lib/common_test/test/ct_groups_search_SUITE_data/groups_search_dummy_1_SUITE.erl index 6f6922e686..1c5b572f92 100644 --- a/lib/common_test/test/ct_groups_search_SUITE_data/groups_search_dummy_1_SUITE.erl +++ b/lib/common_test/test/ct_groups_search_SUITE_data/groups_search_dummy_1_SUITE.erl @@ -1,7 +1,7 @@ %% %% %CopyrightBegin% %% -%% Copyright Ericsson AB 2009-2011. All Rights Reserved. +%% Copyright Ericsson AB 2009-2012. All Rights Reserved. %% %% The contents of this file are subject to the Erlang Public License, %% Version 1.1, (the "License"); you may not use this file except in diff --git a/lib/common_test/test/ct_groups_search_SUITE_data/groups_search_dummy_2_SUITE.erl b/lib/common_test/test/ct_groups_search_SUITE_data/groups_search_dummy_2_SUITE.erl index ac3c000079..060012de29 100644 --- a/lib/common_test/test/ct_groups_search_SUITE_data/groups_search_dummy_2_SUITE.erl +++ b/lib/common_test/test/ct_groups_search_SUITE_data/groups_search_dummy_2_SUITE.erl @@ -1,7 +1,7 @@ %% %% %CopyrightBegin% %% -%% Copyright Ericsson AB 2009-2011. All Rights Reserved. +%% Copyright Ericsson AB 2009-2012. All Rights Reserved. %% %% The contents of this file are subject to the Erlang Public License, %% Version 1.1, (the "License"); you may not use this file except in diff --git a/lib/common_test/test/ct_master_SUITE.erl b/lib/common_test/test/ct_master_SUITE.erl index 27243a0067..56a343a96f 100644 --- a/lib/common_test/test/ct_master_SUITE.erl +++ b/lib/common_test/test/ct_master_SUITE.erl @@ -117,14 +117,8 @@ ct_master_test(Config) when is_list(Config) -> reformat(Events, ?eh), PrivDir, []), - find_events(NodeNames, [{tc_start,{master_SUITE,init_per_suite}}, - {tc_start,{master_SUITE,first_testcase}}, - {tc_start,{master_SUITE,second_testcase}}, - {tc_start,{master_SUITE,third_testcase}}, - {tc_start,{master_SUITE,end_per_suite}}], - Events), - - ok. + TestEvents = events_to_check(ct_master_test), + ok = find_events(NodeNames, TestEvents, Events, Config). %%%----------------------------------------------------------------- %%% HELP FUNCTIONS @@ -153,13 +147,15 @@ make_spec(DataDir, FileName, NodeNames, Suites, Config) -> CM = [{config,master,filename:join(DataDir,"master/config.txt")}], + Env = [{"THIS_MUST_BE_SET","yes"}, + {"SO_MUST_THIS","value"}], NS = lists:map( fun(NodeName) -> {init,NodeName,[ {node_start,[{startup_functions,[]}, - {monitor_master,true}]}, - {eval,{erlang,nodes,[]}} - ] + {monitor_master,true}, + {env,Env}]}, + {eval,{erlang,nodes,[]}}] } end, NodeNames), @@ -199,7 +195,6 @@ run_test(_Name, FileName, Config) -> [{FileName,ok}] = ct_test_support:run({ct_master,run,[FileName]}, [{ct_master,basic_html,[true]}], Config), - timer:sleep(5000), [{FileName,ok}] = ct_test_support:run({ct_master,run,[FileName]}, [{ct_master,basic_html,[false]}], Config). @@ -210,28 +205,26 @@ reformat(Events, EH) -> %%%----------------------------------------------------------------- %%% TEST EVENTS %%%----------------------------------------------------------------- -find_events([], _CheckEvents, _) -> - ok; -find_events([NodeName|NodeNames],CheckEvents,AllEvents) -> - find_events(NodeNames, CheckEvents, - remove_events(add_host(NodeName),CheckEvents, AllEvents, [])). - -remove_events(Node,[{Name,Data} | RestChecks], - [{?eh,#event{ name = Name, node = Node, data = Data }}|RestEvs], - Acc) -> - remove_events(Node, RestChecks, RestEvs, Acc); -remove_events(Node, Checks, [Event|RestEvs], Acc) -> - remove_events(Node, Checks, RestEvs, [Event | Acc]); -remove_events(_Node, [], [], Acc) -> - lists:reverse(Acc); -remove_events(Node, Events, [], Acc) -> - test_server:format("Could not find events: ~p in ~p for node ~p", - [Events, lists:reverse(Acc), Node]), - exit(event_not_found). + +find_events(NodeNames, TestEvents, Events, Config) -> + [begin + Node = add_host(Node0), + io:format("Searching for events for node: ~s", [Node]), + ok = ct_test_support:verify_events(TestEvents, Events, Node, Config), + io:nl() + end || Node0 <- NodeNames], + ok. add_host(NodeName) -> {ok, HostName} = inet:gethostname(), list_to_atom(atom_to_list(NodeName)++"@"++HostName). -expected_events(_) -> - []. +events_to_check(_) -> + [{?eh,tc_start,{master_SUITE,first_testcase}}, + {?eh,tc_done,{master_SUITE,first_testcase,ok}}, + {?eh,tc_start,{master_SUITE,second_testcase}}, + {?eh,tc_done,{master_SUITE,second_testcase,ok}}, + {?eh,tc_start,{master_SUITE,third_testcase}}, + {?eh,tc_done,{master_SUITE,third_testcase,ok}}, + {?eh,tc_start,{master_SUITE,env_vars}}, + {?eh,tc_done,{master_SUITE,env_vars,ok}}]. diff --git a/lib/common_test/test/ct_master_SUITE_data/master/master_SUITE.erl b/lib/common_test/test/ct_master_SUITE_data/master/master_SUITE.erl index 032d69ad9f..8a5009ad62 100644 --- a/lib/common_test/test/ct_master_SUITE_data/master/master_SUITE.erl +++ b/lib/common_test/test/ct_master_SUITE_data/master/master_SUITE.erl @@ -39,7 +39,8 @@ init_per_suite(Config) -> end_per_suite(_) -> ok. -all() -> [first_testcase, second_testcase, third_testcase]. +all() -> [first_testcase, second_testcase, third_testcase, + env_vars]. init_per_testcase(_, Config) -> Config. @@ -56,3 +57,9 @@ second_testcase(_)-> third_testcase(_)-> A = 4, A = 2*2. + +env_vars(_) -> + io:format("~p\n", [os:getenv()]), + "yes" = os:getenv("THIS_MUST_BE_SET"), + "value" = os:getenv("SO_MUST_THIS"), + ok. diff --git a/lib/common_test/test/ct_surefire_SUITE.erl b/lib/common_test/test/ct_surefire_SUITE.erl new file mode 100644 index 0000000000..69e98cef48 --- /dev/null +++ b/lib/common_test/test/ct_surefire_SUITE.erl @@ -0,0 +1,351 @@ +%% +%% %CopyrightBegin% +%% +%% Copyright Ericsson AB 2012. All Rights Reserved. +%% +%% The contents of this file are subject to the Erlang Public License, +%% Version 1.1, (the "License"); you may not use this file except in +%% compliance with the License. You should have received a copy of the +%% Erlang Public License along with this software. If not, it can be +%% retrieved online at http://www.erlang.org/. +%% +%% Software distributed under the License is distributed on an "AS IS" +%% basis, WITHOUT WARRANTY OF ANY KIND, either express or implied. See +%% the License for the specific language governing rights and limitations +%% under the License. +%% +%% %CopyrightEnd% +%% + +%%%------------------------------------------------------------------- +%%% File: ct_surefire_SUITE +%%% +%%% Description: +%%% Test cth_surefire hook +%%% +%%%------------------------------------------------------------------- +-module(ct_surefire_SUITE). + +-compile(export_all). + +-include_lib("common_test/include/ct.hrl"). +-include_lib("common_test/include/ct_event.hrl"). + +-include_lib("xmerl/include/xmerl.hrl"). + +-define(eh, ct_test_support_eh). + +-define(url_base,"http://my.host.com/"). + +%%-------------------------------------------------------------------- +%% TEST SERVER CALLBACK FUNCTIONS +%%-------------------------------------------------------------------- + +%%-------------------------------------------------------------------- +%% Description: Since Common Test starts another Test Server +%% instance, the tests need to be performed on a separate node (or +%% there will be clashes with logging processes etc). +%%-------------------------------------------------------------------- +init_per_suite(Config) -> + Config1 = ct_test_support:init_per_suite(Config), + Config1. + +end_per_suite(Config) -> + ct_test_support:end_per_suite(Config). + +init_per_testcase(TestCase, Config) -> + ct_test_support:init_per_testcase(TestCase, Config). + +end_per_testcase(TestCase, Config) -> + ct_test_support:end_per_testcase(TestCase, Config). + +suite() -> [{ct_hooks,[ts_install_cth]}]. + +all() -> + [ + default, + absolute_path, + relative_path, + url, + logdir + ]. + +%%-------------------------------------------------------------------- +%% TEST CASES +%%-------------------------------------------------------------------- + +%%%----------------------------------------------------------------- +%%% +default(Config) when is_list(Config) -> + run(default,[cth_surefire],Config), + PrivDir = ?config(priv_dir,Config), + XmlRe = filename:join([PrivDir,"*","junit_report.xml"]), + check_xml(default,XmlRe). + +absolute_path(Config) when is_list(Config) -> + PrivDir = ?config(priv_dir,Config), + Path = filename:join(PrivDir,"abspath.xml"), + run(absolute_path,[{cth_surefire,[{path,Path}]}],Config), + check_xml(absolute_path,Path). + +relative_path(Config) when is_list(Config) -> + Path = "relpath.xml", + run(relative_path,[{cth_surefire,[{path,Path}]}],Config), + PrivDir = ?config(priv_dir,Config), + XmlRe = filename:join([PrivDir,"*",Path]), + check_xml(relative_path,XmlRe). + +url(Config) when is_list(Config) -> + Path = "url.xml", + run(url,[{cth_surefire,[{url_base,?url_base}, + {path,Path}]}],Config), + PrivDir = ?config(priv_dir,Config), + XmlRe = filename:join([PrivDir,"*",Path]), + check_xml(url,XmlRe). + +logdir(Config) when is_list(Config) -> + PrivDir = ?config(priv_dir,Config), + LogDir = filename:join(PrivDir,"specific_logdir"), + file:make_dir(LogDir), + Path = "logdir.xml", + run(logdir,[{cth_surefire,[{path,Path}]}],Config,[{logdir,LogDir}]), + PrivDir = ?config(priv_dir,Config), + XmlRe = filename:join([LogDir,"*",Path]), + check_xml(logdir,XmlRe). + +%%%----------------------------------------------------------------- +%%% HELP FUNCTIONS +%%%----------------------------------------------------------------- +run(Case,CTHs,Config) -> + run(Case,CTHs,Config,[]). +run(Case,CTHs,Config,ExtraOpts) -> + DataDir = ?config(data_dir, Config), + Suite = filename:join(DataDir, "surefire_SUITE"), + {Opts,ERPid} = setup([{suite,Suite},{ct_hooks,CTHs},{label,Case}|ExtraOpts], + Config), + ok = execute(Case, Opts, ERPid, Config). + +setup(Test, Config) -> + Opts0 = ct_test_support:get_opts(Config), + Opts1 = + case lists:keymember(logdir,1,Test) of + true -> lists:keydelete(logdir,1,Opts0); + false -> Opts0 + end, + Level = ?config(trace_level, Config), + EvHArgs = [{cbm,ct_test_support},{trace_level,Level}], + Opts = Opts1 ++ [{event_handler,{?eh,EvHArgs}}|Test], + ERPid = ct_test_support:start_event_receiver(Config), + {Opts,ERPid}. + +execute(Name, Opts, ERPid, Config) -> + ok = ct_test_support:run(Opts, Config), + Events = ct_test_support:get_events(ERPid, Config), + + ct_test_support:log_events(Name, + reformat(Events, ?eh), + ?config(priv_dir, Config), + Opts), + + TestEvents = events_to_check(Name), + ct_test_support:verify_events(TestEvents, Events, Config). + +reformat(Events, EH) -> + ct_test_support:reformat(Events, EH). + +%%%----------------------------------------------------------------- +%%% TEST EVENTS +%%%----------------------------------------------------------------- +events_to_check(Test) -> + %% 2 tests (ct:run_test + script_start) is default + events_to_check(Test, 2). + +events_to_check(_, 0) -> + []; +events_to_check(Test, N) -> + test_events(Test) ++ events_to_check(Test, N-1). + +test_events(_) -> + [{?eh,start_logging,'_'}, + {?eh,start_info,{1,1,9}}, + {?eh,tc_start,{surefire_SUITE,init_per_suite}}, + {?eh,tc_done,{surefire_SUITE,init_per_suite,ok}}, + {?eh,tc_start,{surefire_SUITE,tc_ok}}, + {?eh,tc_done,{surefire_SUITE,tc_ok,ok}}, + {?eh,test_stats,{1,0,{0,0}}}, + {?eh,tc_start,{surefire_SUITE,tc_fail}}, + {?eh,tc_done,{surefire_SUITE,tc_fail, + {failed,{error,{test_case_failed,"this test should fail"}}}}}, + {?eh,test_stats,{1,1,{0,0}}}, + {?eh,tc_start,{surefire_SUITE,tc_skip}}, + {?eh,tc_done,{surefire_SUITE,tc_skip,{skipped,"this test is skipped"}}}, + {?eh,test_stats,{1,1,{1,0}}}, + {?eh,tc_start,{surefire_SUITE,tc_autoskip_require}}, + {?eh,tc_done,{surefire_SUITE,tc_autoskip_require, + {skipped,{require_failed,'_'}}}}, + {?eh,test_stats,{1,1,{1,1}}}, + [{?eh,tc_start,{surefire_SUITE,{init_per_group,g,[]}}}, + {?eh,tc_done,{surefire_SUITE,{init_per_group,g,[]},ok}}, + {?eh,tc_start,{surefire_SUITE,tc_ok}}, + {?eh,tc_done,{surefire_SUITE,tc_ok,ok}}, + {?eh,test_stats,{2,1,{1,1}}}, + {?eh,tc_start,{surefire_SUITE,tc_fail}}, + {?eh,tc_done,{surefire_SUITE,tc_fail, + {failed,{error,{test_case_failed,"this test should fail"}}}}}, + {?eh,test_stats,{2,2,{1,1}}}, + {?eh,tc_start,{surefire_SUITE,tc_skip}}, + {?eh,tc_done,{surefire_SUITE,tc_skip,{skipped,"this test is skipped"}}}, + {?eh,test_stats,{2,2,{2,1}}}, + {?eh,tc_start,{surefire_SUITE,tc_autoskip_require}}, + {?eh,tc_done,{surefire_SUITE,tc_autoskip_require, + {skipped,{require_failed,'_'}}}}, + {?eh,test_stats,{2,2,{2,2}}}, + {?eh,tc_start,{surefire_SUITE,{end_per_group,g,[]}}}, + {?eh,tc_done,{surefire_SUITE,{end_per_group,g,[]},ok}}], + [{?eh,tc_start,{surefire_SUITE,{init_per_group,g_fail,[]}}}, + {?eh,tc_done,{surefire_SUITE,{init_per_group,g_fail,[]}, + {failed,{error,all_cases_should_be_skipped}}}}, + {?eh,tc_auto_skip,{surefire_SUITE,tc_ok, + {failed, + {surefire_SUITE,init_per_group, + {'EXIT',all_cases_should_be_skipped}}}}}, + {?eh,test_stats,{2,2,{2,3}}}, + {?eh,tc_auto_skip,{surefire_SUITE,end_per_group, + {failed, + {surefire_SUITE,init_per_group, + {'EXIT',all_cases_should_be_skipped}}}}}], + {?eh,tc_start,{surefire_SUITE,end_per_suite}}, + {?eh,tc_done,{surefire_SUITE,end_per_suite,ok}}, + {?eh,stop_logging,[]}]. + + +%%%----------------------------------------------------------------- +%%% Check generated xml log files +check_xml(Case,XmlRe) -> + case filelib:wildcard(XmlRe) of + [] -> + ct:fail("No xml files found with regexp ~p~n", [XmlRe]); + [_] = Xmls when Case==absolute_path -> + do_check_xml(Case,Xmls); + [_,_] = Xmls -> + do_check_xml(Case,Xmls) + end. + +%% Allowed structure: +%% <testsuites> +%% <testsuite> +%% <properties> +%% <property/> +%% ... +%% </properties> +%% <testcase> +%% [<failure/> | <error/> | <skipped/> ] +%% </testcase> +%% ... +%% </testsuite> +%% ... +%% </testsuites> +do_check_xml(Case,[Xml|Xmls]) -> + ct:log("Checking <a href=~p>~s</a>~n",[Xml,Xml]), + {E,_} = xmerl_scan:file(Xml), + Expected = events_to_result(lists:flatten(test_events(Case))), + ParseResult = testsuites(Case,E), + ct:log("Expecting: ~p~n",[[Expected]]), + ct:log("Actual : ~p~n",[ParseResult]), + [Expected] = ParseResult, + do_check_xml(Case,Xmls); +do_check_xml(_,[]) -> + ok. + +%% Scanning the XML to get the same type of result as events_to_result/1 +testsuites(Case,#xmlElement{name=testsuites,content=TS}) -> + %% OTP-10589 - move properties element to <testsuite> + false = lists:keytake(properties,#xmlElement.name,TS), + testsuite(Case,TS). + +testsuite(Case,[#xmlElement{name=testsuite,content=TC,attributes=A}|TS]) -> + {ET,EF,ES} = events_to_numbers(lists:flatten(test_events(Case))), + {T,E,F,S} = get_numbers_from_attrs(A,false,false,false,false), + ct:log("Expecting total:~p, error:~p, failure:~p, skipped:~p~n",[ET,0,EF,ES]), + ct:log("Actual total:~p, error:~p, failure:~p, skipped:~p~n",[T,E,F,S]), + {ET,0,EF,ES} = {T,E,F,S}, + + %% properties should only be there if given a options to hook + false = lists:keytake(properties,#xmlElement.name,TC), + %% system-out and system-err is not used by common_test + false = lists:keytake('system-out',#xmlElement.name,TC), + false = lists:keytake('system-err',#xmlElement.name,TC), + R=testcase(Case,TC), + [R|testsuite(Case,TS)]; +testsuite(_Case,[]) -> + []. + +testcase(url=Case,[#xmlElement{name=testcase,attributes=A,content=C}|TC]) -> + R = failed_or_skipped(C), + case R of + [s] -> + case lists:keyfind(url,#xmlAttribute.name,A) of + false -> ok; + #xmlAttribute{value=UrlAttr} -> + lists:keyfind(url,#xmlAttribute.name,A), + true = lists:prefix(?url_base,UrlAttr) + end; + _ -> + #xmlAttribute{value=UrlAttr} = + lists:keyfind(url,#xmlAttribute.name,A), + true = lists:prefix(?url_base,UrlAttr) + end, + [R|testcase(Case,TC)]; +testcase(Case,[#xmlElement{name=testcase,attributes=A,content=C}|TC]) -> + false = lists:keyfind(url,#xmlAttribute.name,A), + R = failed_or_skipped(C), + [R|testcase(Case,TC)]; +testcase(_Case,[]) -> + []. + +failed_or_skipped([#xmlElement{name=failure}|E]) -> + [f|failed_or_skipped(E)]; +failed_or_skipped([#xmlElement{name=error}|E]) -> + [e|failed_or_skipped(E)]; +failed_or_skipped([#xmlElement{name=skipped}|E]) -> + [s|failed_or_skipped(E)]; +failed_or_skipped([]) -> + []. + +%% Using the expected events to produce the expected result of the XML scanning. +%% The result is a list of test suites: +%% Testsuites = [Testsuite] +%% Testsuite = [Testcase] +%% Testcase = [] | [f] | [s], indicating ok, failed and skipped respectively +events_to_result([{?eh,tc_done,{_Suite,_Case,R}}|E]) -> + [result(R)|events_to_result(E)]; +events_to_result([{?eh,tc_auto_skip,_}|E]) -> + [[s]|events_to_result(E)]; +events_to_result([_|E]) -> + events_to_result(E); +events_to_result([]) -> + []. + +result(ok) ->[]; +result({skipped,_}) -> [s]; +result({failed,_}) -> [f]. + +%% Using the expected events' last test_stats element to produce the +%% expected number of totla, errors, failed and skipped testcases. +events_to_numbers(E) -> + RevE = lists:reverse(E), + {?eh,test_stats,{Ok,F,{US,AS}}} = lists:keyfind(test_stats,2,RevE), + {Ok+F+US+AS,F,US+AS}. + +get_numbers_from_attrs([#xmlAttribute{name=tests,value=X}|A],false,E,F,S) -> + get_numbers_from_attrs(A,list_to_integer(X),E,F,S); +get_numbers_from_attrs([#xmlAttribute{name=errors,value=X}|A],T,false,F,S) -> + get_numbers_from_attrs(A,T,list_to_integer(X),F,S); +get_numbers_from_attrs([#xmlAttribute{name=failures,value=X}|A],T,E,false,S) -> + get_numbers_from_attrs(A,T,E,list_to_integer(X),S); +get_numbers_from_attrs([#xmlAttribute{name=skipped,value=X}|A],T,E,F,false) -> + get_numbers_from_attrs(A,T,E,F,list_to_integer(X)); +get_numbers_from_attrs([_|A],T,E,F,S) -> + get_numbers_from_attrs(A,T,E,F,S); +get_numbers_from_attrs([],T,E,F,S) -> + {T,E,F,S}. diff --git a/lib/common_test/test/ct_surefire_SUITE_data/surefire_SUITE.erl b/lib/common_test/test/ct_surefire_SUITE_data/surefire_SUITE.erl new file mode 100644 index 0000000000..677aee46c5 --- /dev/null +++ b/lib/common_test/test/ct_surefire_SUITE_data/surefire_SUITE.erl @@ -0,0 +1,92 @@ +%%-------------------------------------------------------------------- +%% %CopyrightBegin% +%% +%% Copyright Ericsson AB 2012. All Rights Reserved. +%% +%% The contents of this file are subject to the Erlang Public License, +%% Version 1.1, (the "License"); you may not use this file except in +%% compliance with the License. You should have received a copy of the +%% Erlang Public License along with this software. If not, it can be +%% retrieved online at http://www.erlang.org/. +%% +%% Software distributed under the License is distributed on an "AS IS" +%% basis, WITHOUT WARRANTY OF ANY KIND, either express or implied. See +%% the License for the specific language governing rights and limitations +%% under the License. +%% +%% %CopyrightEnd% +%% +%%---------------------------------------------------------------------- +%% File: surefire_SUITE.erl +%% +%% Description: +%% This file contains the test cases for cth_surefire. +%% +%% @author Support +%% @doc Test of surefire support in common_test +%% @end +%%---------------------------------------------------------------------- +%%---------------------------------------------------------------------- +-module(surefire_SUITE). +-include_lib("common_test/include/ct.hrl"). + +-compile(export_all). + +%% Default timetrap timeout (set in init_per_testcase). +-define(default_timeout, ?t:minutes(1)). + +all() -> + testcases() ++ [{group,g},{group,g_fail}]. + +groups() -> + [{g,testcases()}, + {g_fail,[tc_ok]}]. + +testcases() -> + [tc_ok, + tc_fail, + tc_skip, + tc_autoskip_require]. + +init_per_suite(Config) -> + Config. + +end_per_suite(Config) -> + Config. + +init_per_group(g_fail, _Config) -> + exit(all_cases_should_be_skipped); +init_per_group(_, Config) -> + Config. + +end_per_group(_Group, Config) -> + Config. + +init_per_testcase(_Case, Config) -> + Dog = test_server:timetrap(?default_timeout), + [{watchdog, Dog}|Config]. + +end_per_testcase(_Case, Config) -> + Dog=?config(watchdog, Config), + test_server:timetrap_cancel(Dog), + ok. + +%%%----------------------------------------------------------------- +%%% Test cases +break(_Config) -> + test_server:break(""), + ok. + +tc_ok(_Config) -> + ok. + +tc_fail(_Config) -> + ct:fail("this test should fail"). + +tc_skip(_Config) -> + {skip,"this test is skipped"}. + +tc_autoskip_require() -> + [{require,whatever}]. +tc_autoskip_require(Config) -> + ct:fail("this test should never be executed - it should be autoskipped"). diff --git a/lib/common_test/test/ct_test_support.erl b/lib/common_test/test/ct_test_support.erl index 901e53c71d..fc572aa82f 100644 --- a/lib/common_test/test/ct_test_support.erl +++ b/lib/common_test/test/ct_test_support.erl @@ -32,7 +32,7 @@ run/2, run/3, run/4, get_opts/1, wait_for_ct_stop/1]). -export([handle_event/2, start_event_receiver/1, get_events/2, - verify_events/3, reformat/2, log_events/4, + verify_events/3, verify_events/4, reformat/2, log_events/4, join_abs_dirs/2]). -export([ct_test_halt/1]). @@ -117,11 +117,7 @@ end_per_suite(Config) -> CTNode = proplists:get_value(ct_node, Config), PrivDir = proplists:get_value(priv_dir, Config), true = rpc:call(CTNode, code, del_path, [filename:join(PrivDir,"")]), - case test_server:is_cover() of - true -> cover:flush(CTNode); - false -> ok - end, - slave:stop(CTNode), + slave_stop(CTNode), ok. %%%----------------------------------------------------------------- @@ -152,11 +148,7 @@ end_per_testcase(_TestCase, Config) -> case wait_for_ct_stop(CTNode) of %% Common test was not stopped to we restart node. false -> - case test_server:is_cover() of - true -> cover:flush(CTNode); - false -> ok - end, - slave:stop(CTNode), + slave_stop(CTNode), start_slave(Config,proplists:get_value(trace_level,Config)), {fail, "Could not stop common_test"}; true -> @@ -370,6 +362,14 @@ verify_events(TEvs, Evs, Config) -> ok end. +verify_events(TEvs, Evs, Node, Config) -> + case catch verify_events1(TEvs, Evs, Node, Config) of + {'EXIT',Reason} -> + Reason; + _ -> + ok + end. + verify_events1([TestEv|_], [{TEH,#event{name=stop_logging,node=Node,data=_}}|_], Node, _) when element(1,TestEv) == TEH, element(2,TestEv) =/= stop_logging -> test_server:format("Failed to find ~p in the list of events!~n", [TestEv]), @@ -1266,3 +1266,22 @@ rm_files([F | Fs]) -> rm_files([]) -> ok. +%%%----------------------------------------------------------------- +%%% +slave_stop(Node) -> + Cover = test_server:is_cover(), + if Cover-> cover:flush(Node); + true -> ok + end, + erlang:monitor_node(Node, true), + slave:stop(Node), + receive + {nodedown, Node} -> + if Cover -> cover:stop(Node); + true -> ok + end + after 5000 -> + erlang:monitor_node(Node, false), + receive {nodedown, Node} -> ok after 0 -> ok end %flush + end, + ok. diff --git a/lib/common_test/vsn.mk b/lib/common_test/vsn.mk index 5c9fdfc47e..f9bb22867e 100644 --- a/lib/common_test/vsn.mk +++ b/lib/common_test/vsn.mk @@ -1 +1 @@ -COMMON_TEST_VSN = 1.6.2.1 +COMMON_TEST_VSN = 1.6.3 diff --git a/lib/compiler/src/beam_bsm.erl b/lib/compiler/src/beam_bsm.erl index 37053e1cc4..02794a8e18 100644 --- a/lib/compiler/src/beam_bsm.erl +++ b/lib/compiler/src/beam_bsm.erl @@ -241,21 +241,45 @@ btb_reaches_match_2([{bif,_,{f,F},Ss,Dst}=I|Is], Regs0, D0) -> Regs = btb_kill([Dst], Regs0), D = btb_follow_branch(F, Regs, D0), btb_reaches_match_1(Is, Regs, D); -btb_reaches_match_2([{test,bs_start_match2,_,_,[Ctx,_],Ctx}|Is], Regs, D) -> - case btb_context_regs(Regs) of - [Ctx] -> - D; - CtxRegs -> - case member(Ctx, CtxRegs) of - false -> btb_reaches_match_2(Is, Regs, D); - true -> btb_error(unsuitable_bs_start_match) +btb_reaches_match_2([{test,bs_start_match2,{f,F},Live,[Ctx,_],Ctx}=I|Is], + Regs0, D0) -> + CtxRegs = btb_context_regs(Regs0), + case member(Ctx, CtxRegs) of + false -> + %% This bs_start_match2 instruction does not use "our" + %% match state. Therefore we can continue the search + %% for another bs_start_match2 instruction. + D = btb_follow_branch(F, Regs0, D0), + Regs = btb_kill_not_live(Live, Regs0), + btb_reaches_match_2(Is, Regs, D); + true -> + %% OK. This instruction will use "our" match state, + %% but we must make sure that all other copies of the + %% match state are killed in the code that follows + %% the instruction. (We know that the fail branch cannot + %% be taken in this case.) + OtherCtxRegs = CtxRegs -- [Ctx], + case btb_are_all_unused(OtherCtxRegs, Is, D0) of + false -> btb_error({OtherCtxRegs,not_all_unused_after,I}); + true -> D0 end end; -btb_reaches_match_2([{test,bs_start_match2,_,_,[Bin,_],Ctx}|Is], Regs, D) -> - CtxRegs = btb_context_regs(Regs), +btb_reaches_match_2([{test,bs_start_match2,{f,F},Live,[Bin,_],Ctx}|Is], + Regs0, D0) -> + CtxRegs = btb_context_regs(Regs0), case member(Bin, CtxRegs) orelse member(Ctx, CtxRegs) of - false -> btb_reaches_match_2(Is, Regs, D); - true -> btb_error(unsuitable_bs_start_match) + false -> + %% This bs_start_match2 does not reference any copy of the + %% match state. Therefore it can safely be passed on the + %% way to another (perhaps more suitable) bs_start_match2 + %% instruction. + D = btb_follow_branch(F, Regs0, D0), + Regs = btb_kill_not_live(Live, Regs0), + btb_reaches_match_2(Is, Regs, D); + true -> + %% This variant of the bs_start_match2 instruction does + %% not accept a match state as source. + btb_error(unsuitable_bs_start_match) end; btb_reaches_match_2([{test,_,{f,F},Ss}=I|Is], Regs, D0) -> btb_ensure_not_used(Ss, I, Regs), @@ -287,11 +311,11 @@ btb_reaches_match_2([{bs_restore2,Src,_}=I|Is], Regs0, D) -> btb_reaches_match_1(Is, Regs0, D); true -> %% Check that all other copies of the context registers - %% are killed by the following instructions. + %% are unused by the following instructions. Regs = btb_kill([Src], Regs0), CtxRegs = btb_context_regs(Regs), - case btb_are_all_killed(CtxRegs, Is, D) of - false -> btb_error({CtxRegs,not_all_killed_after,I}); + case btb_are_all_unused(CtxRegs, Is, D) of + false -> btb_error({CtxRegs,not_all_unused_after,I}); true -> D#btb{must_not_save=true} end end; @@ -301,11 +325,11 @@ btb_reaches_match_2([{bs_context_to_binary,Src}=I|Is], Regs0, D) -> btb_reaches_match_1(Is, Regs0, D); true -> %% Check that all other copies of the context registers - %% are killed by the following instructions. + %% are unused by the following instructions. Regs = btb_kill([Src], Regs0), CtxRegs = btb_context_regs(Regs), - case btb_are_all_killed(CtxRegs, Is, D) of - false -> btb_error({CtxRegs,not_all_killed_after,I}); + case btb_are_all_unused(CtxRegs, Is, D) of + false -> btb_error({CtxRegs,not_all_unused_after,I}); true -> D#btb{must_not_save=true} end end; @@ -343,7 +367,7 @@ btb_call(Arity, Lbl, Regs0, Is, D0) -> %% tucked away in a y register. RegList = btb_context_regs(Regs), YRegs = [R || {y,_}=R <- RegList], - case btb_are_all_killed(YRegs, Is, D) of + case btb_are_all_unused(YRegs, Is, D) of true -> D; false -> btb_error({multiple_uses,RegList}) end; @@ -426,11 +450,11 @@ btb_reaches_match_block([], Regs) -> Regs. %% btb_are_all_killed([Register], [Instruction], D) -> true|false -%% Test whether all of the register are killed in the instruction stream. +%% Test whether all of the register are unused in the instruction stream. -btb_are_all_killed(RegList, Is, #btb{index=Li}) -> +btb_are_all_unused(RegList, Is, #btb{index=Li}) -> all(fun(R) -> - beam_utils:is_killed(R, Is, Li) + beam_utils:is_not_used(R, Is, Li) end, RegList). %% btp_regs_from_list([Register]) -> RegisterSet. diff --git a/lib/compiler/src/beam_jump.erl b/lib/compiler/src/beam_jump.erl index b05d01b2a1..636c299e47 100644 --- a/lib/compiler/src/beam_jump.erl +++ b/lib/compiler/src/beam_jump.erl @@ -220,6 +220,8 @@ extract_seq([{line,_}=Line|Is], Acc) -> extract_seq(Is, [Line|Acc]); extract_seq([{block,_}=Bl|Is], Acc) -> extract_seq_1(Is, [Bl|Acc]); +extract_seq([{bs_context_to_binary,_}=I|Is], Acc) -> + extract_seq_1(Is, [I|Acc]); extract_seq([{label,_}|_]=Is, Acc) -> extract_seq_1(Is, Acc); extract_seq(_, _) -> no. diff --git a/lib/compiler/src/beam_utils.erl b/lib/compiler/src/beam_utils.erl index 8b661e6901..8af0447f63 100644 --- a/lib/compiler/src/beam_utils.erl +++ b/lib/compiler/src/beam_utils.erl @@ -263,19 +263,11 @@ check_liveness(R, [{test,_,{f,Fail},As}|Is], St0) -> {_,_}=Other -> Other end end; -check_liveness(R, [{test,_,{f,Fail},Live,Ss,_}|Is], St0) -> - case R of - {x,X} -> - case X < Live orelse member(R, Ss) of - true -> {used,St0}; - false -> check_liveness_at(R, Fail, St0) - end; - {y,_} -> - case check_liveness_at(R, Fail, St0) of - {killed,St} -> check_liveness(R, Is, St); - {_,_}=Other -> Other - end - end; +check_liveness(R, [{test,Op,Fail,Live,Ss,Dst}|Is], St) -> + %% Check this instruction as a block to get a less conservative + %% result if the caller is is_not_used/3. + Block = [{set,[Dst],Ss,{alloc,Live,{bif,Op,Fail}}}], + check_liveness(R, [{block,Block}|Is], St); check_liveness(R, [{select,_,R,_,_}|_], St) -> {used,St}; check_liveness(R, [{select,_,_,Fail,Branches}|_], St) -> diff --git a/lib/compiler/src/compile.erl b/lib/compiler/src/compile.erl index df1af36eeb..10e7f5e9ce 100644 --- a/lib/compiler/src/compile.erl +++ b/lib/compiler/src/compile.erl @@ -248,6 +248,7 @@ format_error({module_name,Mod,Filename}) -> abstract_code=[], %Abstract code for debugger. options=[] :: [option()], %Options for compilation mod_options=[] :: [option()], %Options for module_info + encoding=none :: none | epp:source_coding(), errors=[], warnings=[]}). @@ -734,8 +735,9 @@ collect_asm([X | Rest], R) -> beam_consult_asm(St) -> case file:consult(St#compile.ifile) of {ok, Forms0} -> + Encoding = epp:read_encoding(St#compile.ifile), {Module, Forms} = preprocess_asm_forms(Forms0), - {ok,St#compile{module=Module, code=Forms}}; + {ok,St#compile{module=Module, code=Forms, encoding=Encoding}}; {error,E} -> Es = [{St#compile.ifile,[{none,?MODULE,{open,E}}]}], {error,St#compile{errors=St#compile.errors ++ Es}} @@ -777,7 +779,8 @@ parse_module(St) -> R = epp:parse_file(St#compile.ifile, IncludePath, pre_defs(Opts)), case R of {ok,Forms} -> - {ok,St#compile{code=Forms}}; + Encoding = epp:read_encoding(St#compile.ifile), + {ok,St#compile{code=Forms,encoding=Encoding}}; {error,E} -> Es = [{St#compile.ifile,[{none,?MODULE,{epp,E}}]}], {error,St#compile{errors=St#compile.errors ++ Es}} @@ -1339,16 +1342,12 @@ save_binary(#compile{code=none}=St) -> {ok,St}; save_binary(#compile{module=Mod,ofile=Outfile, options=Opts}=St) -> %% Test that the module name and output file name match. - %% We must take care to not completely break a packaged module - %% (even though packages still is as an experimental, unsupported - %% feature) - so we will extract the last part of a packaged - %% module name and compare only that. case member(no_error_module_mismatch, Opts) of true -> save_binary_1(St); false -> Base = filename:rootname(filename:basename(Outfile)), - case lists:last(packages:split(Mod)) of + case atom_to_list(Mod) of Base -> save_binary_1(St); _ -> @@ -1424,28 +1423,28 @@ report_warnings(#compile{options=Opts,warnings=Ws0}) -> end. format_message(F, P, [{{Line,Column}=Loc,Mod,E}|Es]) -> - M = {{F,Loc},io_lib:format("~s:~w:~w ~s~s\n", + M = {{F,Loc},io_lib:format("~s:~w:~w ~s~ts\n", [F,Line,Column,P,Mod:format_error(E)])}, [M|format_message(F, P, Es)]; format_message(F, P, [{Line,Mod,E}|Es]) -> - M = {{F,{Line,0}},io_lib:format("~s:~w: ~s~s\n", + M = {{F,{Line,0}},io_lib:format("~s:~w: ~s~ts\n", [F,Line,P,Mod:format_error(E)])}, [M|format_message(F, P, Es)]; format_message(F, P, [{Mod,E}|Es]) -> - M = {none,io_lib:format("~s: ~s~s\n", [F,P,Mod:format_error(E)])}, + M = {none,io_lib:format("~s: ~s~ts\n", [F,P,Mod:format_error(E)])}, [M|format_message(F, P, Es)]; format_message(_, _, []) -> []. %% list_errors(File, ErrorDescriptors) -> ok list_errors(F, [{{Line,Column},Mod,E}|Es]) -> - io:fwrite("~s:~w:~w: ~s\n", [F,Line,Column,Mod:format_error(E)]), + io:fwrite("~s:~w:~w: ~ts\n", [F,Line,Column,Mod:format_error(E)]), list_errors(F, Es); list_errors(F, [{Line,Mod,E}|Es]) -> - io:fwrite("~s:~w: ~s\n", [F,Line,Mod:format_error(E)]), + io:fwrite("~s:~w: ~ts\n", [F,Line,Mod:format_error(E)]), list_errors(F, Es); list_errors(F, [{Mod,E}|Es]) -> - io:fwrite("~s: ~s\n", [F,Mod:format_error(E)]), + io:fwrite("~s: ~ts\n", [F,Mod:format_error(E)]), list_errors(F, Es); list_errors(_F, []) -> ok. @@ -1501,10 +1500,12 @@ src_listing(Ext, St) -> Ext, St). do_src_listing(Lf, Fs) -> - foreach(fun (F) -> io:put_chars(Lf, [erl_pp:form(F),"\n"]) end, + Opts = [lists:keyfind(encoding, 1, io:getopts(Lf))], + foreach(fun (F) -> io:put_chars(Lf, [erl_pp:form(F, Opts),"\n"]) end, Fs). -listing(Ext, St) -> +listing(Ext, St0) -> + St = St0#compile{encoding = none}, listing(fun(Lf, Fs) -> beam_listing:module(Lf, Fs) end, Ext, St). listing(LFun, Ext, St) -> @@ -1512,6 +1513,7 @@ listing(LFun, Ext, St) -> case file:open(Lfile, [write,delayed_write]) of {ok,Lf} -> Code = restore_expanded_types(Ext, St#compile.code), + output_encoding(Lf, St), LFun(Lf, Code), ok = file:close(Lf), {ok,St}; @@ -1520,6 +1522,12 @@ listing(LFun, Ext, St) -> {error,St#compile{errors=St#compile.errors ++ Es}} end. +output_encoding(F, #compile{encoding = none}) -> + ok = io:setopts(F, [{encoding, epp:default_encoding()}]); +output_encoding(F, #compile{encoding = Encoding}) -> + ok = io:setopts(F, [{encoding, Encoding}]), + ok = io:fwrite(F, <<"%% ~s\n">>, [epp:encoding_to_string(Encoding)]). + restore_expanded_types("P", Fs) -> epp:restore_typed_record_fields(Fs); restore_expanded_types("E", {M,I,Fs0}) -> diff --git a/lib/compiler/src/core_scan.erl b/lib/compiler/src/core_scan.erl index 5aab8ae855..0ca2f57dde 100644 --- a/lib/compiler/src/core_scan.erl +++ b/lib/compiler/src/core_scan.erl @@ -1,7 +1,8 @@ +%% -*- coding: utf-8 -*- %% %% %CopyrightBegin% %% -%% Copyright Ericsson AB 2000-2009. All Rights Reserved. +%% Copyright Ericsson AB 2000-2012. All Rights Reserved. %% %% The contents of this file are subject to the Erlang Public License, %% Version 1.1, (the "License"); you may not use this file except in @@ -31,16 +32,16 @@ %% 173 - 176 { - ~ punctuation %% 177 DEL control %% 200 - 237 control -%% 240 - 277 NBSP - � punctuation -%% 300 - 326 � - � uppercase -%% 327 � punctuation -%% 330 - 336 � - � uppercase -%% 337 - 366 � - � lowercase -%% 367 � punctuation -%% 370 - 377 � - � lowercase +%% 240 - 277 NBSP - ¿ punctuation +%% 300 - 326 À - Ö uppercase +%% 327 × punctuation +%% 330 - 336 Ø - Þ uppercase +%% 337 - 366 ß - ö lowercase +%% 367 ÷ punctuation +%% 370 - 377 ø - ÿ lowercase %% %% Many punctuation characters region have special meaning. Must -%% watch using � \327, bvery close to x \170 +%% watch using × \327, bvery close to x \170 -module(core_scan). @@ -239,11 +240,11 @@ scan1([C|Cs], Toks, Pos) when C >= $\200, C =< $\240 -> scan1(Cs, Toks, Pos); scan1([C|Cs], Toks, Pos) when C >= $a, C =< $z -> %Keywords scan_key_word(C, Cs, Toks, Pos); -scan1([C|Cs], Toks, Pos) when C >= $�, C =< $�, C /= $� -> +scan1([C|Cs], Toks, Pos) when C >= $ß, C =< $ÿ, C /= $÷ -> scan_key_word(C, Cs, Toks, Pos); scan1([C|Cs], Toks, Pos) when C >= $A, C =< $Z -> %Variables scan_variable(C, Cs, Toks, Pos); -scan1([C|Cs], Toks, Pos) when C >= $�, C =< $�, C /= $� -> +scan1([C|Cs], Toks, Pos) when C >= $À, C =< $Þ, C /= $× -> scan_variable(C, Cs, Toks, Pos); scan1([C|Cs], Toks, Pos) when C >= $0, C =< $9 -> %Numbers scan_number(C, Cs, Toks, Pos); @@ -308,9 +309,9 @@ scan_name([], Ncs) -> {Ncs,[]}. name_char(C) when C >= $a, C =< $z -> true; -name_char(C) when C >= $�, C =< $�, C /= $� -> true; +name_char(C) when C >= $ß, C =< $ÿ, C /= $÷ -> true; name_char(C) when C >= $A, C =< $Z -> true; -name_char(C) when C >= $�, C =< $�, C /= $� -> true; +name_char(C) when C >= $À, C =< $Þ, C /= $× -> true; name_char(C) when C >= $0, C =< $9 -> true; name_char($_) -> true; name_char($@) -> true; diff --git a/lib/compiler/src/sys_pre_expand.erl b/lib/compiler/src/sys_pre_expand.erl index e55fb2a037..a8c69c3cb1 100644 --- a/lib/compiler/src/sys_pre_expand.erl +++ b/lib/compiler/src/sys_pre_expand.erl @@ -35,10 +35,8 @@ -record(expand, {module=[], %Module name parameters=undefined, %Module parameters - package="", %Module package exports=[], %Exports imports=[], %Imports - mod_imports, %Module Imports compile=[], %Compile flags attributes=[], %Attributes callbacks=[], %Callbacks @@ -67,12 +65,8 @@ module(Fs0, Opts0) -> %% Set pre-defined exported functions. PreExp = [{module_info,0},{module_info,1}], - %% Set pre-defined module imports. - PreModImp = [{erlang,erlang},{packages,packages}], - %% Build initial expand record. St0 = #expand{exports=PreExp, - mod_imports=dict:from_list(PreModImp), compile=Opts, defined=PreExp, bitdefault = erl_bits:system_bitdefault(), @@ -242,14 +236,12 @@ forms([], St) -> {[],St}. %% Process an attribute, this just affects the state. attribute(module, {Module, As}, _L, St) -> - M = package_to_string(Module), - St#expand{module=list_to_atom(M), - package=packages:strip_last(M), + true = is_atom(Module), + St#expand{module=Module, parameters=As}; attribute(module, Module, _L, St) -> - M = package_to_string(Module), - St#expand{module=list_to_atom(M), - package=packages:strip_last(M)}; + true = is_atom(Module), + St#expand{module=Module}; attribute(export, Es, _L, St) -> St#expand{exports=union(from_list(Es), St#expand.exports)}; attribute(import, Is, _L, St) -> @@ -312,8 +304,6 @@ pattern({tuple,Line,Ps}, St0) -> %%pattern({struct,Line,Tag,Ps}, St0) -> %% {TPs,TPsvs,St1} = pattern_list(Ps, St0), %% {{tuple,Line,[{atom,Line,Tag}|TPs]},TPsvs,St1}; -pattern({record_field,_,_,_}=M, St) -> - {expand_package(M, St),St}; % must be a package name pattern({bin,Line,Es0}, St0) -> {Es1,St1} = pattern_bin(Es0, St0), {{bin,Line,Es1},St1}; @@ -404,8 +394,6 @@ expr({tuple,Line,Es0}, St0) -> %%expr({struct,Line,Tag,Es0}, Vs, St0) -> %% {Es1,Esvs,Esus,St1} = expr_list(Es0, Vs, St0), %% {{tuple,Line,[{atom,Line,Tag}|Es1]},Esvs,Esus,St1}; -expr({record_field,_,_,_}=M, St) -> - {expand_package(M, St),St}; % must be a package name expr({bin,Line,Es0}, St0) -> {Es1,St1} = expr_bin(Es0, St0), {{bin,Line,Es1},St1}; @@ -448,12 +436,9 @@ expr({call,Line,{atom,La,N}=Atom,As0}, St0) -> end end end; -expr({call,Line,{record_field,_,_,_}=M,As0}, St0) -> - expr({call,Line,expand_package(M, St0),As0}, St0); -expr({call,Line,{remote,Lr,M,F},As0}, St0) -> - M1 = expand_package(M, St0), - {[M2,F1|As1],St1} = expr_list([M1,F|As0], St0), - {{call,Line,{remote,Lr,M2,F1},As1},St1}; +expr({call,Line,{remote,Lr,M0,F},As0}, St0) -> + {[M1,F1|As1],St1} = expr_list([M0,F|As0], St0), + {{call,Line,{remote,Lr,M1,F1},As1},St1}; expr({call,Line,F,As0}, St0) -> {[Fun1|As1],St1} = expr_list([F|As0], St0), {{call,Line,Fun1,As1},St1}; @@ -666,32 +651,6 @@ string_to_conses(Line, Cs, Tail) -> foldr(fun (C, T) -> {cons,Line,{char,Line,C},T} end, Tail, Cs). -%% In syntax trees, module/package names are atoms or lists of atoms. - -package_to_string(A) when is_atom(A) -> atom_to_list(A); -package_to_string(L) when is_list(L) -> packages:concat(L). - -expand_package({atom,L,A} = M, St) -> - case dict:find(A, St#expand.mod_imports) of - {ok, A1} -> - {atom,L,A1}; - error -> - case packages:is_segmented(A) of - true -> - M; - false -> - M1 = packages:concat(St#expand.package, A), - {atom,L,list_to_atom(M1)} - end - end; -expand_package(M, _St) -> - case erl_parse:package_segments(M) of - error -> - M; - M1 -> - {atom,element(2,M),list_to_atom(package_to_string(M1))} - end. - %% import(Line, Imports, State) -> %% State' %% imported(Name, Arity, State) -> @@ -699,15 +658,10 @@ expand_package(M, _St) -> %% Handle import declarations and test for imported functions. No need to %% check when building imports as code is correct. -import({Mod0,Fs}, St) -> - Mod = list_to_atom(package_to_string(Mod0)), +import({Mod,Fs}, St) -> + true = is_atom(Mod), Mfs = from_list(Fs), - St#expand{imports=add_imports(Mod, Mfs, St#expand.imports)}; -import(Mod0, St) -> - Mod = package_to_string(Mod0), - Key = list_to_atom(packages:last(Mod)), - St#expand{mod_imports=dict:store(Key, list_to_atom(Mod), - St#expand.mod_imports)}. + St#expand{imports=add_imports(Mod, Mfs, St#expand.imports)}. add_imports(Mod, [F|Fs], Is) -> add_imports(Mod, Fs, orddict:store(F, Mod, Is)); diff --git a/lib/compiler/test/bs_match_SUITE.erl b/lib/compiler/test/bs_match_SUITE.erl index bcceb889ed..d63d2235d7 100644 --- a/lib/compiler/test/bs_match_SUITE.erl +++ b/lib/compiler/test/bs_match_SUITE.erl @@ -1,7 +1,7 @@ %% %% %CopyrightBegin% %% -%% Copyright Ericsson AB 2005-2011. All Rights Reserved. +%% Copyright Ericsson AB 2005-2012. All Rights Reserved. %% %% The contents of this file are subject to the Erlang Public License, %% Version 1.1, (the "License"); you may not use this file except in @@ -33,7 +33,7 @@ matching_meets_construction/1,simon/1,matching_and_andalso/1, otp_7188/1,otp_7233/1,otp_7240/1,otp_7498/1, match_string/1,zero_width/1,bad_size/1,haystack/1, - cover_beam_bool/1,matched_out_size/1]). + cover_beam_bool/1,matched_out_size/1,follow_fail_branch/1]). -export([coverage_id/1,coverage_external_ignore/2]). @@ -57,7 +57,7 @@ groups() -> matching_meets_construction,simon, matching_and_andalso,otp_7188,otp_7233,otp_7240, otp_7498,match_string,zero_width,bad_size,haystack, - cover_beam_bool,matched_out_size]}]. + cover_beam_bool,matched_out_size,follow_fail_branch]}]. init_per_suite(Config) -> @@ -1108,6 +1108,32 @@ mos_bin(<<L,Bin:L/binary,"abcdefghij">>) -> L = byte_size(Bin), Bin. +follow_fail_branch(_) -> + 42 = ffb_1(<<0,1>>, <<0>>), + 8 = ffb_1(<<0,1>>, [a]), + 42 = ffb_2(<<0,1>>, <<0>>, 17), + 8 = ffb_2(<<0,1>>, [a], 0), + ok. + +ffb_1(<<_,T/bitstring>>, List) -> + case List of + <<_>> -> + 42; + [_|_] -> + %% The fail branch of the bs_start_match2 instruction + %% pointing to here would be ignored, making the compiler + %% incorrectly assume that the delayed sub-binary + %% optimization was safe. + bit_size(T) + end. + +ffb_2(<<_,T/bitstring>>, List, A) -> + case List of + <<_>> when A =:= 17 -> 42; + [_|_] -> bit_size(T) + end. + + check(F, R) -> R = F(). diff --git a/lib/compiler/test/compilation_SUITE.erl b/lib/compiler/test/compilation_SUITE.erl index bec97b0199..f8f74e6f7a 100644 --- a/lib/compiler/test/compilation_SUITE.erl +++ b/lib/compiler/test/compilation_SUITE.erl @@ -44,7 +44,7 @@ groups() -> {group,vsn},otp_2380,otp_2141,otp_2173,otp_4790, const_list_256,bin_syntax_1,bin_syntax_2, bin_syntax_3,bin_syntax_4,bin_syntax_5,bin_syntax_6, - live_var,convopts,bad_functional_value, + live_var,convopts, catch_in_catch,redundant_case,long_string,otp_5076, complex_guard,otp_5092,otp_5151,otp_5235,otp_5244, trycatch_4,opt_crash,otp_5404,otp_5436,otp_5481, @@ -143,7 +143,6 @@ split({int, N}, <<N:16,B:N/binary,T/binary>>) -> ?comp(live_var). ?comp(trycatch_4). -?comp(bad_functional_value). ?comp(catch_in_catch). diff --git a/lib/compiler/test/compile_SUITE.erl b/lib/compiler/test/compile_SUITE.erl index 2cd75944f4..229e5a98a1 100644 --- a/lib/compiler/test/compile_SUITE.erl +++ b/lib/compiler/test/compile_SUITE.erl @@ -27,7 +27,7 @@ app_test/1, file_1/1, forms_2/1, module_mismatch/1, big_file/1, outdir/1, binary/1, makedep/1, cond_and_ifdef/1, listings/1, listings_big/1, - other_output/1, package_forms/1, encrypted_abstr/1, + other_output/1, encrypted_abstr/1, bad_record_use1/1, bad_record_use2/1, strict_record/1, missing_testheap/1, cover/1, env/1, core/1, asm/1, sys_pre_attributes/1]). @@ -44,7 +44,7 @@ all() -> test_lib:recompile(?MODULE), [app_test, file_1, forms_2, module_mismatch, big_file, outdir, binary, makedep, cond_and_ifdef, listings, listings_big, - other_output, package_forms, encrypted_abstr, + other_output, encrypted_abstr, {group, bad_record_use}, strict_record, missing_testheap, cover, env, core, asm, sys_pre_attributes]. @@ -410,32 +410,6 @@ other_output(Config) when is_list(Config) -> ?line test_server:timetrap_cancel(Dog), ok. -package_forms(Config) when is_list(Config) -> - Fs = [{attribute,1,file,{"./p.erl",1}}, - {attribute,1,module,[p,p]}, - {attribute,3,compile,export_all}, - {attribute,1,file, - {"/clearcase/otp/erts/lib/stdlib/include/qlc.hrl",1}}, - {attribute,6,file,{"./p.erl",6}}, - {function,7,q,0, - [{clause,7,[],[], - [{call,8, - {remote,8,{atom,8,qlc},{atom,8,q}}, - [{tuple,-8, - [{atom,-8,qlc_lc}, - {'fun',-8, - {clauses, - [{clause,-8,[],[], - [{tuple,-8, - [{atom,-8,simple_v1}, - {atom,-8,'X'}, - {'fun',-8,{clauses,[{clause,-8,[],[],[{nil,8}]}]}}, - {integer,-8,8}]}]}]}}, - {atom,-8,undefined}]}]}]}]}, - {eof,9}], - {ok,'p.p',_} = compile:forms(Fs, ['S',report]), - ok. - encrypted_abstr(Config) when is_list(Config) -> ?line Dog = test_server:timetrap(test_server:minutes(10)), ?line {Simple,Target} = files(Config, "encrypted_abstr"), diff --git a/lib/crypto/c_src/Makefile.in b/lib/crypto/c_src/Makefile.in index ffd556ca1a..e19d6617f3 100644 --- a/lib/crypto/c_src/Makefile.in +++ b/lib/crypto/c_src/Makefile.in @@ -59,8 +59,6 @@ TYPE_FLAGS = $(CFLAGS) endif endif -ALL_CFLAGS = $(TYPE_FLAGS) $(INCLUDES) - # ---------------------------------------------------- # Release directory specification # ---------------------------------------------------- @@ -69,13 +67,16 @@ RELSYSDIR = $(RELEASE_PATH)/lib/crypto-$(VSN) # ---------------------------------------------------- # Misc Macros # ---------------------------------------------------- -OBJS = $(OBJDIR)/crypto$(TYPEMARKER).o +CRYPTO_OBJS = $(OBJDIR)/crypto$(TYPEMARKER).o +CALLBACK_OBJS = $(OBJDIR)/crypto_callback$(TYPEMARKER).o NIF_MAKEFILE = $(PRIVDIR)/Makefile ifeq ($(findstring win32,$(TARGET)), win32) NIF_LIB = $(LIBDIR)/crypto$(TYPEMARKER).dll +CALLBACK_LIB = $(LIBDIR)/crypto_callback$(TYPEMARKER).dll else NIF_LIB = $(LIBDIR)/crypto$(TYPEMARKER).so +CALLBACK_LIB = $(LIBDIR)/crypto_callback$(TYPEMARKER).so endif ifeq ($(HOST_OS),) @@ -86,43 +87,69 @@ DYNAMIC_CRYPTO_LIB=@SSL_DYNAMIC_ONLY@ ifeq ($(DYNAMIC_CRYPTO_LIB),yes) SSL_DED_LD_RUNTIME_LIBRARY_PATH = @SSL_DED_LD_RUNTIME_LIBRARY_PATH@ CRYPTO_LINK_LIB=$(SSL_DED_LD_RUNTIME_LIBRARY_PATH) -L$(SSL_LIBDIR) -l$(SSL_CRYPTO_LIBNAME) +EXTRA_FLAGS = -DHAVE_DYNAMIC_CRYPTO_LIB else SSL_DED_LD_RUNTIME_LIBRARY_PATH= CRYPTO_LINK_LIB=$(SSL_LIBDIR)/lib$(SSL_CRYPTO_LIBNAME).a +EXTRA_FLAGS = +CRYPTO_OBJS := $(CRYPTO_OBJS) $(CALLBACK_OBJS) +CALLBACK_OBJS = +CALLBACK_LIB = endif +ALL_CFLAGS = $(TYPE_FLAGS) $(EXTRA_FLAGS) $(INCLUDES) + # ---------------------------------------------------- # Targets # ---------------------------------------------------- _create_dirs := $(shell mkdir -p $(OBJDIR) $(LIBDIR)) -debug opt valgrind: $(NIF_LIB) +debug opt valgrind: $(NIF_LIB) $(CALLBACK_LIB) $(OBJDIR)/%$(TYPEMARKER).o: %.c $(INSTALL_DIR) $(OBJDIR) $(CC) -c -o $@ $(ALL_CFLAGS) $< -$(LIBDIR)/crypto$(TYPEMARKER).so: $(OBJS) - $(INSTALL_DIR) $(LIBDIR) +$(LIBDIR)/crypto$(TYPEMARKER).so: $(CRYPTO_OBJS) + $(INSTALL_DIR) $(LIBDIR) $(LD) $(LDFLAGS) -o $@ $^ $(LDLIBS) $(CRYPTO_LINK_LIB) -$(LIBDIR)/crypto$(TYPEMARKER).dll: $(OBJS) +$(LIBDIR)/crypto$(TYPEMARKER).dll: $(CRYPTO_OBJS) + $(INSTALL_DIR) $(LIBDIR) + $(LD) $(LDFLAGS) -o $@ $(SSL_DED_LD_RUNTIME_LIBRARY_PATH) -L$(SSL_LIBDIR) $(CRYPTO_OBJS) -l$(SSL_CRYPTO_LIBNAME) -l$(SSL_SSL_LIBNAME) + +ifeq ($(DYNAMIC_CRYPTO_LIB),yes) +$(LIBDIR)/crypto_callback$(TYPEMARKER).so: $(CALLBACK_OBJS) + $(INSTALL_DIR) $(LIBDIR) + $(LD) $(LDFLAGS) -o $@ $^ $(LDLIBS) + +$(LIBDIR)/crypto_callback$(TYPEMARKER).dll: $(CALLBACK_OBJS) $(INSTALL_DIR) $(LIBDIR) - $(LD) $(LDFLAGS) -o $@ $(SSL_DED_LD_RUNTIME_LIBRARY_PATH) -L$(SSL_LIBDIR) $(OBJS) -l$(SSL_CRYPTO_LIBNAME) -l$(SSL_SSL_LIBNAME) + $(LD) $(LDFLAGS) -o $@ $(CALLBACK_OBJS) +endif + clean: ifeq ($(findstring win32,$(TARGET)), win32) rm -f $(LIBDIR)/crypto.dll rm -f $(LIBDIR)/crypto.debug.dll + rm -f $(LIBDIR)/crypto_callback.dll + rm -f $(LIBDIR)/crypto_callback.debug.dll else rm -f $(LIBDIR)/crypto.so rm -f $(LIBDIR)/crypto.debug.so rm -f $(LIBDIR)/crypto.valgrind.so + rm -f $(LIBDIR)/crypto_callback.so + rm -f $(LIBDIR)/crypto_callback.debug.so + rm -f $(LIBDIR)/crypto_callback.valgrind.so endif rm -f $(OBJDIR)/crypto.o rm -f $(OBJDIR)/crypto.debug.o rm -f $(OBJDIR)/crypto.valgrind.o + rm -f $(OBJDIR)/crypto_callback.o + rm -f $(OBJDIR)/crypto_callback.debug.o + rm -f $(OBJDIR)/crypto_callback.valgrind.o rm -f core *~ docs: @@ -136,8 +163,12 @@ release_spec: opt $(INSTALL_DIR) "$(RELSYSDIR)/priv/obj" $(INSTALL_DIR) "$(RELSYSDIR)/priv/lib" $(INSTALL_DATA) $(NIF_MAKEFILE) "$(RELSYSDIR)/priv/obj" - $(INSTALL_PROGRAM) $(OBJS) "$(RELSYSDIR)/priv/obj" + $(INSTALL_PROGRAM) $(CRYPTO_OBJS) "$(RELSYSDIR)/priv/obj" $(INSTALL_PROGRAM) $(NIF_LIB) "$(RELSYSDIR)/priv/lib" +ifeq ($(DYNAMIC_CRYPTO_LIB),yes) + $(INSTALL_PROGRAM) $(CALLBACK_OBJS) "$(RELSYSDIR)/priv/obj" + $(INSTALL_PROGRAM) $(CALLBACK_LIB) "$(RELSYSDIR)/priv/lib" +endif release_docs_spec: diff --git a/lib/crypto/c_src/crypto.c b/lib/crypto/c_src/crypto.c index 91ab244620..e77e5fb8f0 100644 --- a/lib/crypto/c_src/crypto.c +++ b/lib/crypto/c_src/crypto.c @@ -44,6 +44,7 @@ #include <openssl/md5.h> #include <openssl/md4.h> #include <openssl/sha.h> +#include <openssl/ripemd.h> #include <openssl/bn.h> #include <openssl/objects.h> #include <openssl/rc4.h> @@ -53,6 +54,8 @@ #include <openssl/evp.h> #include <openssl/hmac.h> +#include "crypto_callback.h" + #if OPENSSL_VERSION_NUMBER >= 0x00908000L && !defined(OPENSSL_NO_SHA224) && defined(NID_sha224)\ && !defined(OPENSSL_NO_SHA256) /* disabled like this in my sha.h (?) */ # define HAVE_SHA224 @@ -67,6 +70,9 @@ #if OPENSSL_VERSION_NUMBER >= 0x00908000L && !defined(OPENSSL_NO_SHA512) && defined(NID_sha512) # define HAVE_SHA512 #endif +#if OPENSSL_VERSION_NUMBER >= 0x0090705FL +# define HAVE_DES_ede3_cfb_encrypt +#endif #ifdef VALGRIND # include <valgrind/memcheck.h> @@ -125,7 +131,6 @@ /* NIF interface declarations */ static int load(ErlNifEnv* env, void** priv_data, ERL_NIF_TERM load_info); -static int reload(ErlNifEnv* env, void** priv_data, ERL_NIF_TERM load_info); static int upgrade(ErlNifEnv* env, void** priv_data, void** old_priv_data, ERL_NIF_TERM load_info); static void unload(ErlNifEnv* env, void* priv_data); @@ -135,6 +140,10 @@ static ERL_NIF_TERM md5(ErlNifEnv* env, int argc, const ERL_NIF_TERM argv[]); static ERL_NIF_TERM md5_init(ErlNifEnv* env, int argc, const ERL_NIF_TERM argv[]); static ERL_NIF_TERM md5_update(ErlNifEnv* env, int argc, const ERL_NIF_TERM argv[]); static ERL_NIF_TERM md5_final(ErlNifEnv* env, int argc, const ERL_NIF_TERM argv[]); +static ERL_NIF_TERM ripemd160(ErlNifEnv* env, int argc, const ERL_NIF_TERM argv[]); +static ERL_NIF_TERM ripemd160_init(ErlNifEnv* env, int argc, const ERL_NIF_TERM argv[]); +static ERL_NIF_TERM ripemd160_update(ErlNifEnv* env, int argc, const ERL_NIF_TERM argv[]); +static ERL_NIF_TERM ripemd160_final(ErlNifEnv* env, int argc, const ERL_NIF_TERM argv[]); static ERL_NIF_TERM sha(ErlNifEnv* env, int argc, const ERL_NIF_TERM argv[]); static ERL_NIF_TERM sha_init(ErlNifEnv* env, int argc, const ERL_NIF_TERM argv[]); static ERL_NIF_TERM sha_update(ErlNifEnv* env, int argc, const ERL_NIF_TERM argv[]); @@ -172,7 +181,7 @@ static ERL_NIF_TERM des_cbc_crypt(ErlNifEnv* env, int argc, const ERL_NIF_TERM a static ERL_NIF_TERM des_cfb_crypt(ErlNifEnv* env, int argc, const ERL_NIF_TERM argv[]); static ERL_NIF_TERM des_ecb_crypt(ErlNifEnv* env, int argc, const ERL_NIF_TERM argv[]); static ERL_NIF_TERM des_ede3_cbc_crypt(ErlNifEnv* env, int argc, const ERL_NIF_TERM argv[]); -static ERL_NIF_TERM des_ede3_cfb_crypt(ErlNifEnv* env, int argc, const ERL_NIF_TERM argv[]); +static ERL_NIF_TERM des_ede3_cfb_crypt_nif(ErlNifEnv* env, int argc, const ERL_NIF_TERM argv[]); static ERL_NIF_TERM aes_cfb_128_crypt(ErlNifEnv* env, int argc, const ERL_NIF_TERM argv[]); static ERL_NIF_TERM aes_ctr_encrypt(ErlNifEnv* env, int argc, const ERL_NIF_TERM argv[]); static ERL_NIF_TERM aes_ctr_stream_encrypt(ErlNifEnv* env, int argc, const ERL_NIF_TERM argv[]); @@ -204,17 +213,6 @@ static ERL_NIF_TERM bf_ecb_crypt(ErlNifEnv* env, int argc, const ERL_NIF_TERM ar static ERL_NIF_TERM blowfish_ofb64_encrypt(ErlNifEnv* env, int argc, const ERL_NIF_TERM argv[]); -/* openssl callbacks */ -#ifdef OPENSSL_THREADS -static void locking_function(int mode, int n, const char *file, int line); -static unsigned long id_function(void); -static struct CRYPTO_dynlock_value* dyn_create_function(const char *file, - int line); -static void dyn_lock_function(int mode, struct CRYPTO_dynlock_value* ptr, - const char *file, int line); -static void dyn_destroy_function(struct CRYPTO_dynlock_value *ptr, - const char *file, int line); -#endif /* OPENSSL_THREADS */ /* helpers */ static void init_digest_types(ErlNifEnv* env); @@ -253,6 +251,10 @@ static ErlNifFunc nif_funcs[] = { {"md5_init", 0, md5_init}, {"md5_update", 2, md5_update}, {"md5_final", 1, md5_final}, + {"ripemd160", 1, ripemd160}, + {"ripemd160_init", 0, ripemd160_init}, + {"ripemd160_update", 2, ripemd160_update}, + {"ripemd160_final", 1, ripemd160_final}, {"sha", 1, sha}, {"sha_init", 0, sha_init}, {"sha_update", 2, sha_update}, @@ -291,7 +293,7 @@ static ErlNifFunc nif_funcs[] = { {"des_cfb_crypt", 4, des_cfb_crypt}, {"des_ecb_crypt", 3, des_ecb_crypt}, {"des_ede3_cbc_crypt", 6, des_ede3_cbc_crypt}, - {"des_ede3_cfb_crypt", 6, des_ede3_cfb_crypt}, + {"des_ede3_cfb_crypt_nif", 6, des_ede3_cfb_crypt_nif}, {"aes_cfb_128_crypt", 4, aes_cfb_128_crypt}, {"aes_ctr_encrypt", 3, aes_ctr_encrypt}, {"aes_ctr_decrypt", 3, aes_ctr_encrypt}, @@ -325,7 +327,7 @@ static ErlNifFunc nif_funcs[] = { {"blowfish_ofb64_encrypt", 3, blowfish_ofb64_encrypt} }; -ERL_NIF_INIT(crypto,nif_funcs,load,reload,upgrade,unload) +ERL_NIF_INIT(crypto,nif_funcs,load,NULL,upgrade,unload) #define MD5_CTX_LEN (sizeof(MD5_CTX)) @@ -333,6 +335,8 @@ ERL_NIF_INIT(crypto,nif_funcs,load,reload,upgrade,unload) #define MD5_LEN_96 12 #define MD4_CTX_LEN (sizeof(MD4_CTX)) #define MD4_LEN 16 +#define RIPEMD160_CTX_LEN (sizeof(RIPEMD160_CTX)) +#define RIPEMD160_LEN 20 #define SHA_CTX_LEN (sizeof(SHA_CTX)) #define SHA_LEN 20 #define SHA_LEN_96 12 @@ -347,7 +351,6 @@ ERL_NIF_INIT(crypto,nif_funcs,load,reload,upgrade,unload) #define HMAC_OPAD 0x5c -static ErlNifRWLock** lock_vec = NULL; /* Static locks used by openssl */ static ERL_NIF_TERM atom_true; static ERL_NIF_TERM atom_false; static ERL_NIF_TERM atom_sha; @@ -374,55 +377,60 @@ static ERL_NIF_TERM atom_none; static ERL_NIF_TERM atom_notsup; static ERL_NIF_TERM atom_digest; +/* +#define PRINTF_ERR0(FMT) enif_fprintf(stderr, FMT "\n") +#define PRINTF_ERR1(FMT, A1) enif_fprintf(stderr, FMT "\n", A1) +*/ +#define PRINTF_ERR0(FMT) +#define PRINTF_ERR1(FMT,A1) -static int is_ok_load_info(ErlNifEnv* env, ERL_NIF_TERM load_info) +#ifdef HAVE_DYNAMIC_CRYPTO_LIB +static int change_basename(char* buf, int bufsz, const char* newfile) { - int i; - return enif_get_int(env,load_info,&i) && i == 101; -} -static void* crypto_alloc(size_t size) -{ - return enif_alloc(size); + char* p = strrchr(buf, '/'); + p = (p == NULL) ? buf : p + 1; + + if ((p - buf) + strlen(newfile) >= bufsz) { + PRINTF_ERR0("CRYPTO: lib name too long"); + return 0; + } + strcpy(p, newfile); + return 1; } -static void* crypto_realloc(void* ptr, size_t size) + +static void error_handler(void* null, const char* errstr) { - return enif_realloc(ptr, size); -} -static void crypto_free(void* ptr) -{ - enif_free(ptr); + PRINTF_ERR1("CRYPTO LOADING ERROR: '%s'", errstr); } +#endif /* HAVE_DYNAMIC_CRYPTO_LIB */ -static int load(ErlNifEnv* env, void** priv_data, ERL_NIF_TERM load_info) +static int init(ErlNifEnv* env, ERL_NIF_TERM load_info) { ErlNifSysInfo sys_info; - CRYPTO_set_mem_functions(crypto_alloc, crypto_realloc, crypto_free); - - if (!is_ok_load_info(env, load_info)) { - return -1; + get_crypto_callbacks_t* funcp; + struct crypto_callbacks* ccb; + int nlocks = 0; + int tpl_arity; + const ERL_NIF_TERM* tpl_array; + int vernum; + char lib_buf[1000]; + + /* load_info: {201, "/full/path/of/this/library"} */ + if (!enif_get_tuple(env, load_info, &tpl_arity, &tpl_array) + || tpl_arity != 2 + || !enif_get_int(env, tpl_array[0], &vernum) + || vernum != 201 + || enif_get_string(env, tpl_array[1], lib_buf, sizeof(lib_buf), ERL_NIF_LATIN1) <= 0) { + + PRINTF_ERR1("CRYPTO: Invalid load_info '%T'", load_info); + return 0; } - -#ifdef OPENSSL_THREADS - enif_system_info(&sys_info, sizeof(sys_info)); - - if (sys_info.scheduler_threads > 1) { - int i; - lock_vec = enif_alloc(CRYPTO_num_locks()*sizeof(*lock_vec)); - if (lock_vec==NULL) return -1; - memset(lock_vec,0,CRYPTO_num_locks()*sizeof(*lock_vec)); - - for (i=CRYPTO_num_locks()-1; i>=0; --i) { - lock_vec[i] = enif_rwlock_create("crypto_stat"); - if (lock_vec[i]==NULL) return -1; - } - CRYPTO_set_locking_callback(locking_function); - CRYPTO_set_id_callback(id_function); - CRYPTO_set_dynlock_create_callback(dyn_create_function); - CRYPTO_set_dynlock_lock_callback(dyn_lock_function); - CRYPTO_set_dynlock_destroy_callback(dyn_destroy_function); + if (library_refc > 0) { + /* Repeated loading of this library (module upgrade). + * Atoms and callbacks are already set, we are done. + */ + return 1; } - /* else no need for locks */ -#endif /* OPENSSL_THREADS */ atom_true = enif_make_atom(env,"true"); atom_false = enif_make_atom(env,"false"); @@ -451,37 +459,75 @@ static int load(ErlNifEnv* env, void** priv_data, ERL_NIF_TERM load_info) init_digest_types(env); - *priv_data = NULL; - library_refc++; - return 0; -} - -static int reload(ErlNifEnv* env, void** priv_data, ERL_NIF_TERM load_info) -{ - if (*priv_data != NULL) { - return -1; /* Don't know how to do that */ +#ifdef HAVE_DYNAMIC_CRYPTO_LIB + { + void* handle; + if (!change_basename(lib_buf, sizeof(lib_buf), "crypto_callback")) { + return 0; + } + if (!(handle = enif_dlopen(lib_buf, &error_handler, NULL))) { + return 0; + } + if (!(funcp = (get_crypto_callbacks_t*) enif_dlsym(handle, "get_crypto_callbacks", + &error_handler, NULL))) { + return 0; + } } - if (library_refc == 0) { - /* No support for real library upgrade. The tricky thing is to know - when to (re)set the callbacks for allocation and locking. */ - return -2; +#else /* !HAVE_DYNAMIC_CRYPTO_LIB */ + funcp = &get_crypto_callbacks; +#endif + +#ifdef OPENSSL_THREADS + enif_system_info(&sys_info, sizeof(sys_info)); + if (sys_info.scheduler_threads > 1) { + nlocks = CRYPTO_num_locks(); } - if (!is_ok_load_info(env, load_info)) { + /* else no need for locks */ +#endif + + ccb = (*funcp)(nlocks); + + if (!ccb || ccb->sizeof_me != sizeof(*ccb)) { + PRINTF_ERR0("Invalid 'crypto_callbacks'"); + return 0; + } + + CRYPTO_set_mem_functions(ccb->crypto_alloc, ccb->crypto_realloc, ccb->crypto_free); + +#ifdef OPENSSL_THREADS + if (nlocks > 0) { + CRYPTO_set_locking_callback(ccb->locking_function); + CRYPTO_set_id_callback(ccb->id_function); + CRYPTO_set_dynlock_create_callback(ccb->dyn_create_function); + CRYPTO_set_dynlock_lock_callback(ccb->dyn_lock_function); + CRYPTO_set_dynlock_destroy_callback(ccb->dyn_destroy_function); + } +#endif /* OPENSSL_THREADS */ + return 1; +} + +static int load(ErlNifEnv* env, void** priv_data, ERL_NIF_TERM load_info) +{ + if (!init(env, load_info)) { return -1; } - return 0; + + *priv_data = NULL; + library_refc++; + return 0; } static int upgrade(ErlNifEnv* env, void** priv_data, void** old_priv_data, ERL_NIF_TERM load_info) { - int i; if (*old_priv_data != NULL) { return -1; /* Don't know how to do that */ } - i = reload(env,priv_data,load_info); - if (i != 0) { - return i; + if (*priv_data != NULL) { + return -1; /* Don't know how to do that */ + } + if (!init(env, load_info)) { + return -1; } library_refc++; return 0; @@ -489,20 +535,7 @@ static int upgrade(ErlNifEnv* env, void** priv_data, void** old_priv_data, static void unload(ErlNifEnv* env, void* priv_data) { - if (--library_refc <= 0) { - CRYPTO_cleanup_all_ex_data(); - - if (lock_vec != NULL) { - int i; - for (i=CRYPTO_num_locks()-1; i>=0; --i) { - if (lock_vec[i] != NULL) { - enif_rwlock_destroy(lock_vec[i]); - } - } - enif_free(lock_vec); - } - } - /*else NIF library still used by other (new) module code */ + --library_refc; } static ERL_NIF_TERM info_lib(ErlNifEnv* env, int argc, const ERL_NIF_TERM argv[]) @@ -514,12 +547,21 @@ static ERL_NIF_TERM info_lib(ErlNifEnv* env, int argc, const ERL_NIF_TERM argv[] const char* ver = SSLeay_version(SSLEAY_VERSION); unsigned ver_sz = strlen(ver); ERL_NIF_TERM name_term, ver_term; + int ver_num = OPENSSL_VERSION_NUMBER; + /* R16: + * Ignore library version number from SSLeay() and instead show header + * version. Otherwise user might try to call a function that is implemented + * by a newer library but not supported by the headers used at compile time. + * Example: DES_ede3_cfb_encrypt in 0.9.7i but not in 0.9.7d. + * + * Version string is still from library though. + */ memcpy(enif_make_new_binary(env, name_sz, &name_term), libname, name_sz); memcpy(enif_make_new_binary(env, ver_sz, &ver_term), ver, ver_sz); return enif_make_list1(env, enif_make_tuple3(env, name_term, - enif_make_int(env, SSLeay()), + enif_make_int(env, ver_num), ver_term)); } @@ -569,6 +611,53 @@ static ERL_NIF_TERM md5_final(ErlNifEnv* env, int argc, const ERL_NIF_TERM argv[ return ret; } +static ERL_NIF_TERM ripemd160(ErlNifEnv* env, int argc, const ERL_NIF_TERM argv[]) +{/* (Data) */ + ErlNifBinary ibin; + ERL_NIF_TERM ret; + + if (!enif_inspect_iolist_as_binary(env, argv[0], &ibin)) { + return enif_make_badarg(env); + } + RIPEMD160((unsigned char *) ibin.data, ibin.size, + enif_make_new_binary(env,RIPEMD160_LEN, &ret)); + return ret; +} +static ERL_NIF_TERM ripemd160_init(ErlNifEnv* env, int argc, const ERL_NIF_TERM argv[]) +{/* () */ + ERL_NIF_TERM ret; + RIPEMD160_Init((RIPEMD160_CTX *) enif_make_new_binary(env, RIPEMD160_CTX_LEN, &ret)); + return ret; +} +static ERL_NIF_TERM ripemd160_update(ErlNifEnv* env, int argc, const ERL_NIF_TERM argv[]) +{/* (Context, Data) */ + RIPEMD160_CTX* new_ctx; + ErlNifBinary ctx_bin, data_bin; + ERL_NIF_TERM ret; + if (!enif_inspect_binary(env, argv[0], &ctx_bin) + || ctx_bin.size != RIPEMD160_CTX_LEN + || !enif_inspect_iolist_as_binary(env, argv[1], &data_bin)) { + return enif_make_badarg(env); + } + new_ctx = (RIPEMD160_CTX*) enif_make_new_binary(env,RIPEMD160_CTX_LEN, &ret); + memcpy(new_ctx, ctx_bin.data, RIPEMD160_CTX_LEN); + RIPEMD160_Update(new_ctx, data_bin.data, data_bin.size); + return ret; +} +static ERL_NIF_TERM ripemd160_final(ErlNifEnv* env, int argc, const ERL_NIF_TERM argv[]) +{/* (Context) */ + ErlNifBinary ctx_bin; + RIPEMD160_CTX ctx_clone; + ERL_NIF_TERM ret; + if (!enif_inspect_binary(env, argv[0], &ctx_bin) || ctx_bin.size != RIPEMD160_CTX_LEN) { + return enif_make_badarg(env); + } + memcpy(&ctx_clone, ctx_bin.data, RIPEMD160_CTX_LEN); /* writable */ + RIPEMD160_Final(enif_make_new_binary(env, RIPEMD160_LEN, &ret), &ctx_clone); + return ret; +} + + static ERL_NIF_TERM sha(ErlNifEnv* env, int argc, const ERL_NIF_TERM argv[]) {/* (Data) */ ErlNifBinary ibin; @@ -1199,8 +1288,9 @@ static ERL_NIF_TERM des_ede3_cbc_crypt(ErlNifEnv* env, int argc, const ERL_NIF_T return ret; } -static ERL_NIF_TERM des_ede3_cfb_crypt(ErlNifEnv* env, int argc, const ERL_NIF_TERM argv[]) +static ERL_NIF_TERM des_ede3_cfb_crypt_nif(ErlNifEnv* env, int argc, const ERL_NIF_TERM argv[]) {/* (Key1, Key2, Key3, IVec, Text/Cipher, IsEncrypt) */ +#ifdef HAVE_DES_ede3_cfb_encrypt ErlNifBinary key1, key2, key3, ivec, text; DES_key_schedule schedule1, schedule2, schedule3; DES_cblock ivec_clone; /* writable copy */ @@ -1222,6 +1312,9 @@ static ERL_NIF_TERM des_ede3_cfb_crypt(ErlNifEnv* env, int argc, const ERL_NIF_T 8, text.size, &schedule1, &schedule2, &schedule3, &ivec_clone, (argv[5] == atom_true)); return ret; +#else + return atom_notsup; +#endif } static ERL_NIF_TERM aes_cfb_128_crypt(ErlNifEnv* env, int argc, const ERL_NIF_TERM argv[]) @@ -2338,59 +2431,6 @@ static ERL_NIF_TERM blowfish_ofb64_encrypt(ErlNifEnv* env, int argc, const ERL_N -#ifdef OPENSSL_THREADS /* vvvvvvvvvvvvvvv OPENSSL_THREADS vvvvvvvvvvvvvvvv */ - -static INLINE void locking(int mode, ErlNifRWLock* lock) -{ - switch (mode) { - case CRYPTO_LOCK|CRYPTO_READ: - enif_rwlock_rlock(lock); - break; - case CRYPTO_LOCK|CRYPTO_WRITE: - enif_rwlock_rwlock(lock); - break; - case CRYPTO_UNLOCK|CRYPTO_READ: - enif_rwlock_runlock(lock); - break; - case CRYPTO_UNLOCK|CRYPTO_WRITE: - enif_rwlock_rwunlock(lock); - break; - default: - ASSERT(!"Invalid lock mode"); - } -} - -/* Callback from openssl for static locking - */ -static void locking_function(int mode, int n, const char *file, int line) -{ - ASSERT(n>=0 && n<CRYPTO_num_locks()); - - locking(mode, lock_vec[n]); -} - -/* Callback from openssl for thread id - */ -static unsigned long id_function(void) -{ - return(unsigned long) enif_thread_self(); -} - -/* Callbacks for dynamic locking, not used by current openssl version (0.9.8) - */ -static struct CRYPTO_dynlock_value* dyn_create_function(const char *file, int line) { - return(struct CRYPTO_dynlock_value*) enif_rwlock_create("crypto_dyn"); -} -static void dyn_lock_function(int mode, struct CRYPTO_dynlock_value* ptr,const char *file, int line) -{ - locking(mode, (ErlNifRWLock*)ptr); -} -static void dyn_destroy_function(struct CRYPTO_dynlock_value *ptr, const char *file, int line) -{ - enif_rwlock_destroy((ErlNifRWLock*)ptr); -} - -#endif /* ^^^^^^^^^^^^^^^^^^^^^^ OPENSSL_THREADS ^^^^^^^^^^^^^^^^^^^^^^ */ /* HMAC */ diff --git a/lib/crypto/c_src/crypto_callback.c b/lib/crypto/c_src/crypto_callback.c new file mode 100644 index 0000000000..81106b4cc2 --- /dev/null +++ b/lib/crypto/c_src/crypto_callback.c @@ -0,0 +1,165 @@ +/* + * %CopyrightBegin% + * + * Copyright Ericsson AB 2012. All Rights Reserved. + * + * The contents of this file are subject to the Erlang Public License, + * Version 1.1, (the "License"); you may not use this file except in + * compliance with the License. You should have received a copy of the + * Erlang Public License along with this software. If not, it can be + * retrieved online at http://www.erlang.org/. + * + * Software distributed under the License is distributed on an "AS IS" + * basis, WITHOUT WARRANTY OF ANY KIND, either express or implied. See + * the License for the specific language governing rights and limitations + * under the License. + * + * %CopyrightEnd% + */ + +#include <string.h> +#include <openssl/opensslconf.h> + +#include "erl_nif.h" +#include "crypto_callback.h" + +#ifdef DEBUG + # define ASSERT(e) \ + ((void) ((e) ? 1 : (fprintf(stderr,"Assert '%s' failed at %s:%d\n",\ + #e, __FILE__, __LINE__), abort(), 0))) +#else + # define ASSERT(e) ((void) 1) +#endif + +#ifdef __GNUC__ + # define INLINE __inline__ +#elif defined(__WIN32__) + # define INLINE __forceinline +#else + # define INLINE +#endif + +#ifdef __WIN32__ +# define DLLEXPORT __declspec(dllexport) +#else +# define DLLEXPORT +#endif + +/* to be dlsym'ed */ +DLLEXPORT struct crypto_callbacks* get_crypto_callbacks(int nlocks); + + +static ErlNifRWLock** lock_vec = NULL; /* Static locks used by openssl */ + +static void* crypto_alloc(size_t size) +{ + return enif_alloc(size); +} +static void* crypto_realloc(void* ptr, size_t size) +{ + return enif_realloc(ptr, size); +} +static void crypto_free(void* ptr) +{ + enif_free(ptr); +} + + +#ifdef OPENSSL_THREADS /* vvvvvvvvvvvvvvv OPENSSL_THREADS vvvvvvvvvvvvvvvv */ + +#include <openssl/crypto.h> + +static INLINE void locking(int mode, ErlNifRWLock* lock) +{ + switch (mode) { + case CRYPTO_LOCK|CRYPTO_READ: + enif_rwlock_rlock(lock); + break; + case CRYPTO_LOCK|CRYPTO_WRITE: + enif_rwlock_rwlock(lock); + break; + case CRYPTO_UNLOCK|CRYPTO_READ: + enif_rwlock_runlock(lock); + break; + case CRYPTO_UNLOCK|CRYPTO_WRITE: + enif_rwlock_rwunlock(lock); + break; + default: + ASSERT(!"Invalid lock mode"); + } +} + +static void locking_function(int mode, int n, const char *file, int line) +{ + ASSERT(n>=0 && n<CRYPTO_num_locks()); + + locking(mode, lock_vec[n]); +} + +static unsigned long id_function(void) +{ + return (unsigned long) enif_thread_self(); +} + +/* Dynamic locking, not used by current openssl version (0.9.8) + */ +static struct CRYPTO_dynlock_value* dyn_create_function(const char *file, int line) +{ + return (struct CRYPTO_dynlock_value*) enif_rwlock_create("crypto_dyn"); +} +static void dyn_lock_function(int mode, struct CRYPTO_dynlock_value* ptr,const char *file, int line) +{ + locking(mode, (ErlNifRWLock*)ptr); +} +static void dyn_destroy_function(struct CRYPTO_dynlock_value *ptr, const char *file, int line) +{ + enif_rwlock_destroy((ErlNifRWLock*)ptr); +} + +#endif /* ^^^^^^^^^^^^^^^^^^^^^^ OPENSSL_THREADS ^^^^^^^^^^^^^^^^^^^^^^ */ + +DLLEXPORT struct crypto_callbacks* get_crypto_callbacks(int nlocks) +{ + static int is_initialized = 0; + static struct crypto_callbacks the_struct = { + sizeof(struct crypto_callbacks), + + &crypto_alloc, + &crypto_realloc, + &crypto_free, + +#ifdef OPENSSL_THREADS + &locking_function, + &id_function, + &dyn_create_function, + &dyn_lock_function, + &dyn_destroy_function +#endif /* OPENSSL_THREADS */ + }; + + if (!is_initialized) { +#ifdef OPENSSL_THREADS + if (nlocks > 0) { + int i; + lock_vec = enif_alloc(nlocks*sizeof(*lock_vec)); + if (lock_vec==NULL) return NULL; + memset(lock_vec, 0, nlocks*sizeof(*lock_vec)); + + for (i=nlocks-1; i>=0; --i) { + lock_vec[i] = enif_rwlock_create("crypto_stat"); + if (lock_vec[i]==NULL) return NULL; + } + } +#endif + is_initialized = 1; + } + return &the_struct; +} + +#ifdef HAVE_DYNAMIC_CRYPTO_LIB +/* This is not really a NIF library, but we use ERL_NIF_INIT in order to + * get access to the erl_nif API (on Windows). + */ +ERL_NIF_INIT(dummy, (ErlNifFunc*)NULL , NULL, NULL, NULL, NULL) +#endif + diff --git a/lib/crypto/c_src/crypto_callback.h b/lib/crypto/c_src/crypto_callback.h new file mode 100644 index 0000000000..23ecba3e5d --- /dev/null +++ b/lib/crypto/c_src/crypto_callback.h @@ -0,0 +1,46 @@ +/* + * %CopyrightBegin% + * + * Copyright Ericsson AB 2012. All Rights Reserved. + * + * The contents of this file are subject to the Erlang Public License, + * Version 1.1, (the "License"); you may not use this file except in + * compliance with the License. You should have received a copy of the + * Erlang Public License along with this software. If not, it can be + * retrieved online at http://www.erlang.org/. + * + * Software distributed under the License is distributed on an "AS IS" + * basis, WITHOUT WARRANTY OF ANY KIND, either express or implied. See + * the License for the specific language governing rights and limitations + * under the License. + * + * %CopyrightEnd% + */ + +struct crypto_callbacks +{ + size_t sizeof_me; + + void* (*crypto_alloc)(size_t size); + void* (*crypto_realloc)(void* ptr, size_t size); + void (*crypto_free)(void* ptr); + + /* openssl callbacks */ + #ifdef OPENSSL_THREADS + void (*locking_function)(int mode, int n, const char *file, int line); + unsigned long (*id_function)(void); + struct CRYPTO_dynlock_value* (*dyn_create_function)(const char *file, + int line); + void (*dyn_lock_function)(int mode, struct CRYPTO_dynlock_value* ptr, + const char *file, int line); + void (*dyn_destroy_function)(struct CRYPTO_dynlock_value *ptr, + const char *file, int line); + #endif /* OPENSSL_THREADS */ +}; + +typedef struct crypto_callbacks* get_crypto_callbacks_t(int nlocks); + +#ifndef HAVE_DYNAMIC_CRYPTO_LIB +struct crypto_callbacks* get_crypto_callbacks(int nlocks); +#endif + diff --git a/lib/crypto/doc/src/crypto.xml b/lib/crypto/doc/src/crypto.xml index 4dcd6fc4ea..61e80d7d5f 100644..100755 --- a/lib/crypto/doc/src/crypto.xml +++ b/lib/crypto/doc/src/crypto.xml @@ -115,6 +115,12 @@ Mpint() = <![CDATA[<<ByteLen:32/integer-big, Bytes:ByteLen/binary>>]]> > <input>info_lib().</input> [{<<"OpenSSL">>,9469983,<<"OpenSSL 0.9.8a 11 Oct 2005">>}] </pre> + <note><p> + From OTP R16 the <em>numeric version</em> represents the version of the OpenSSL + <em>header files</em> (<c>openssl/opensslv.h</c>) used when crypto was compiled. + The text variant represents the OpenSSL library used at runtime. + In earlier OTP versions both numeric and text was taken from the library. + </p></note> </desc> </func> <func> @@ -259,24 +265,28 @@ Mpint() = <![CDATA[<<ByteLen:32/integer-big, Bytes:ByteLen/binary>>]]> <name>hash(Type, Data) -> Digest</name> <fsummary></fsummary> <type> - <v>Type = md4 | md5 | sha | sha224 | sha256 | sha384 | sha512</v> + <v>Type = md4 | md5 | ripemd160 | sha | sha224 | sha256 | sha384 | sha512</v> <v>Data = iodata()</v> <v>Digest = binary()</v> </type> <desc> <p>Computes a message digest of type <c>Type</c> from <c>Data</c>.</p> + <p>May throw exception <c>notsup</c> in case the chosen <c>Type</c> + is not supported by the underlying OpenSSL implementation.</p> </desc> </func> <func> <name>hash_init(Type) -> Context</name> <fsummary></fsummary> <type> - <v>Type = md4 | md5 | sha | sha224 | sha256 | sha384 | sha512</v> + <v>Type = md4 | md5 | ripemd160 | sha | sha224 | sha256 | sha384 | sha512</v> </type> <desc> <p>Initializes the context for streaming hash operations. <c>Type</c> determines which digest to use. The returned context should be used as argument to <seealso marker="#hash_update/2">hash_update</seealso>.</p> + <p>May throw exception <c>notsup</c> in case the chosen <c>Type</c> + is not supported by the underlying OpenSSL implementation.</p> </desc> </func> <func> @@ -548,6 +558,8 @@ Mpint() = <![CDATA[<<ByteLen:32/integer-big, Bytes:ByteLen/binary>>]]> keys, and <c>IVec</c> is an arbitrary initializing vector. The lengths of each of <c>Key1</c>, <c>Key2</c>, <c>Key3</c> and <c>IVec</c> must be 64 bits (8 bytes).</p> + <p>May throw exception <c>notsup</c> for old OpenSSL + versions (0.9.7) that does not support this encryption mode.</p> </desc> </func> <func> @@ -565,6 +577,8 @@ Mpint() = <![CDATA[<<ByteLen:32/integer-big, Bytes:ByteLen/binary>>]]> and <c>IVec</c> must have the same values as those used when encrypting. The lengths of <c>Key1</c>, <c>Key2</c>, <c>Key3</c>, and <c>IVec</c> must be 64 bits (8 bytes).</p> + <p>May throw exception <c>notsup</c> for old OpenSSL + versions (0.9.7) that does not support this encryption mode.</p> </desc> </func> diff --git a/lib/crypto/doc/src/crypto_app.xml b/lib/crypto/doc/src/crypto_app.xml index 1c01e3f099..6573a56f4c 100644 --- a/lib/crypto/doc/src/crypto_app.xml +++ b/lib/crypto/doc/src/crypto_app.xml @@ -62,7 +62,7 @@ <section> <title>OpenSSL libraries</title> <p>The current implementation of the Erlang Crypto application is - based on the <em>OpenSSL</em> package version 0.9.7 or higher. + based on the <em>OpenSSL</em> package version 0.9.8 or higher. There are source and binary releases on the web. </p> <p>Source releases of OpenSSL can be downloaded from the <url href="http://www.openssl.org">OpenSSL</url> project home page, diff --git a/lib/crypto/doc/src/notes.xml b/lib/crypto/doc/src/notes.xml index a9f8dd84af..4178ca2b08 100644 --- a/lib/crypto/doc/src/notes.xml +++ b/lib/crypto/doc/src/notes.xml @@ -4,7 +4,7 @@ <chapter> <header> <copyright> - <year>1999</year><year>2011</year> + <year>1999</year><year>2012</year> <holder>Ericsson AB. All Rights Reserved.</holder> </copyright> <legalnotice> diff --git a/lib/crypto/src/crypto.erl b/lib/crypto/src/crypto.erl index 0089e79a4f..f25d63fe3b 100644 --- a/lib/crypto/src/crypto.erl +++ b/lib/crypto/src/crypto.erl @@ -78,12 +78,11 @@ md5_mac, md5_mac_96, sha_mac, sha_mac_96, sha224_mac, sha256_mac, sha384_mac, sha512_mac, - sha_mac_init, sha_mac_update, sha_mac_final, des_cbc_encrypt, des_cbc_decrypt, des_cfb_encrypt, des_cfb_decrypt, des_ecb_encrypt, des_ecb_decrypt, - des_ede3_cbc_encrypt, des_ede3_cbc_decrypt, - des_ede3_cfb_encrypt, des_ede3_cfb_decrypt, + des3_cbc_encrypt, des3_cbc_decrypt, + des3_cfb_encrypt, des3_cfb_decrypt, aes_cfb_128_encrypt, aes_cfb_128_decrypt, rand_bytes, strong_rand_bytes, @@ -103,6 +102,13 @@ aes_cbc_256_encrypt, aes_cbc_256_decrypt, aes_ctr_encrypt, aes_ctr_decrypt, aes_ctr_stream_init, aes_ctr_stream_encrypt, aes_ctr_stream_decrypt, + aes_cbc_ivec, blowfish_cbc_encrypt, blowfish_cbc_decrypt, + blowfish_cfb64_encrypt, blowfish_cfb64_decrypt, + blowfish_ecb_encrypt, blowfish_ecb_decrypt, blowfish_ofb64_encrypt, + des_cbc_ivec, des_cfb_ivec, erlint, mpint, + hash, hash_init, hash_update, hash_final, + hmac_init, hmac_update, hmac_final, hmac_final_n, info, + rc2_cbc_encrypt, rc2_cbc_decrypt, info_lib]). -type rsa_digest_type() :: 'md5' | 'sha' | 'sha224' | 'sha256' | 'sha384' | 'sha512'. @@ -114,7 +120,7 @@ -on_load(on_load/0). --define(CRYPTO_NIF_VSN,101). +-define(CRYPTO_NIF_VSN,201). on_load() -> LibBaseName = "crypto", @@ -140,7 +146,7 @@ on_load() -> end end, Lib = filename:join([PrivDir, "lib", LibName]), - Status = case erlang:load_nif(Lib, ?CRYPTO_NIF_VSN) of + Status = case erlang:load_nif(Lib, {?CRYPTO_NIF_VSN,Lib}) of ok -> ok; {error, {load_failed, _}}=Error1 -> ArchLibDir = @@ -152,7 +158,7 @@ on_load() -> [] -> Error1; _ -> ArchLib = filename:join([ArchLibDir, LibName]), - erlang:load_nif(ArchLib, ?CRYPTO_NIF_VSN) + erlang:load_nif(ArchLib, {?CRYPTO_NIF_VSN,ArchLib}) end; Error1 -> Error1 end, @@ -191,43 +197,48 @@ version() -> ?CRYPTO_VSN. %% -spec hash(_, iodata()) -> binary(). -hash(md5, Data) -> md5(Data); -hash(md4, Data) -> md4(Data); -hash(sha, Data) -> sha(Data); -hash(sha224, Data) -> sha224(Data); -hash(sha256, Data) -> sha256(Data); -hash(sha384, Data) -> sha384(Data); -hash(sha512, Data) -> sha512(Data). - --spec hash_init('md5'|'md4'|'sha'|'sha224'|'sha256'|'sha384'|'sha512') -> any(). - -hash_init(md5) -> {md5, md5_init()}; -hash_init(md4) -> {md4, md4_init()}; -hash_init(sha) -> {sha, sha_init()}; -hash_init(sha224) -> {sha224, sha224_init()}; -hash_init(sha256) -> {sha256, sha256_init()}; -hash_init(sha384) -> {sha384, sha384_init()}; -hash_init(sha512) -> {sha512, sha512_init()}. +hash(md5, Data) -> md5(Data); +hash(md4, Data) -> md4(Data); +hash(sha, Data) -> sha(Data); +hash(ripemd160, Data) -> ripemd160(Data); +hash(sha224, Data) -> sha224(Data); +hash(sha256, Data) -> sha256(Data); +hash(sha384, Data) -> sha384(Data); +hash(sha512, Data) -> sha512(Data). + +-spec hash_init('md5'|'md4'|'ripemd160'| + 'sha'|'sha224'|'sha256'|'sha384'|'sha512') -> any(). + +hash_init(md5) -> {md5, md5_init()}; +hash_init(md4) -> {md4, md4_init()}; +hash_init(sha) -> {sha, sha_init()}; +hash_init(ripemd160) -> {ripemd160, ripemd160_init()}; +hash_init(sha224) -> {sha224, sha224_init()}; +hash_init(sha256) -> {sha256, sha256_init()}; +hash_init(sha384) -> {sha384, sha384_init()}; +hash_init(sha512) -> {sha512, sha512_init()}. -spec hash_update(_, iodata()) -> any(). -hash_update({md5,Context}, Data) -> {md5, md5_update(Context,Data)}; -hash_update({md4,Context}, Data) -> {md4, md4_update(Context,Data)}; -hash_update({sha,Context}, Data) -> {sha, sha_update(Context,Data)}; -hash_update({sha224,Context}, Data) -> {sha224, sha224_update(Context,Data)}; -hash_update({sha256,Context}, Data) -> {sha256, sha256_update(Context,Data)}; -hash_update({sha384,Context}, Data) -> {sha384, sha384_update(Context,Data)}; -hash_update({sha512,Context}, Data) -> {sha512, sha512_update(Context,Data)}. +hash_update({md5,Context}, Data) -> {md5, md5_update(Context,Data)}; +hash_update({md4,Context}, Data) -> {md4, md4_update(Context,Data)}; +hash_update({sha,Context}, Data) -> {sha, sha_update(Context,Data)}; +hash_update({ripemd160,Context}, Data) -> {ripemd160, ripemd160_update(Context,Data)}; +hash_update({sha224,Context}, Data) -> {sha224, sha224_update(Context,Data)}; +hash_update({sha256,Context}, Data) -> {sha256, sha256_update(Context,Data)}; +hash_update({sha384,Context}, Data) -> {sha384, sha384_update(Context,Data)}; +hash_update({sha512,Context}, Data) -> {sha512, sha512_update(Context,Data)}. -spec hash_final(_) -> binary(). -hash_final({md5,Context}) -> md5_final(Context); -hash_final({md4,Context}) -> md4_final(Context); -hash_final({sha,Context}) -> sha_final(Context); -hash_final({sha224,Context}) -> sha224_final(Context); -hash_final({sha256,Context}) -> sha256_final(Context); -hash_final({sha384,Context}) -> sha384_final(Context); -hash_final({sha512,Context}) -> sha512_final(Context). +hash_final({md5,Context}) -> md5_final(Context); +hash_final({md4,Context}) -> md4_final(Context); +hash_final({sha,Context}) -> sha_final(Context); +hash_final({ripemd160,Context}) -> ripemd160_final(Context); +hash_final({sha224,Context}) -> sha224_final(Context); +hash_final({sha256,Context}) -> sha256_final(Context); +hash_final({sha384,Context}) -> sha384_final(Context); +hash_final({sha512,Context}) -> sha512_final(Context). %% %% MD5 @@ -257,6 +268,20 @@ md4_update(_Context, _Data) -> ?nif_stub. md4_final(_Context) -> ?nif_stub. %% +%% RIPEMD160 +%% + +-spec ripemd160(iodata()) -> binary(). +-spec ripemd160_init() -> binary(). +-spec ripemd160_update(binary(), iodata()) -> binary(). +-spec ripemd160_final(binary()) -> binary(). + +ripemd160(_Data) -> ?nif_stub. +ripemd160_init() -> ?nif_stub. +ripemd160_update(_Context, _Data) -> ?nif_stub. +ripemd160_final(_Context) -> ?nif_stub. + +%% %% SHA %% -spec sha(iodata()) -> binary(). @@ -597,12 +622,12 @@ des_ecb_crypt(_Key, _Data, _IsEncrypt) -> ?nif_stub. binary(). des3_cbc_encrypt(Key1, Key2, Key3, IVec, Data) -> - des_ede3_cbc_encrypt(Key1, Key2, Key3, IVec, Data). + des_ede3_cbc_crypt(Key1, Key2, Key3, IVec, Data, true). des_ede3_cbc_encrypt(Key1, Key2, Key3, IVec, Data) -> des_ede3_cbc_crypt(Key1, Key2, Key3, IVec, Data, true). des3_cbc_decrypt(Key1, Key2, Key3, IVec, Data) -> - des_ede3_cbc_decrypt(Key1, Key2, Key3, IVec, Data). + des_ede3_cbc_crypt(Key1, Key2, Key3, IVec, Data, false). des_ede3_cbc_decrypt(Key1, Key2, Key3, IVec, Data) -> des_ede3_cbc_crypt(Key1, Key2, Key3, IVec, Data, false). @@ -617,16 +642,18 @@ des_ede3_cbc_crypt(_Key1, _Key2, _Key3, _IVec, _Data, _IsEncrypt) -> ?nif_stub. binary(). des3_cfb_encrypt(Key1, Key2, Key3, IVec, Data) -> - des_ede3_cfb_encrypt(Key1, Key2, Key3, IVec, Data). -des_ede3_cfb_encrypt(Key1, Key2, Key3, IVec, Data) -> des_ede3_cfb_crypt(Key1, Key2, Key3, IVec, Data, true). des3_cfb_decrypt(Key1, Key2, Key3, IVec, Data) -> - des_ede3_cfb_decrypt(Key1, Key2, Key3, IVec, Data). -des_ede3_cfb_decrypt(Key1, Key2, Key3, IVec, Data) -> des_ede3_cfb_crypt(Key1, Key2, Key3, IVec, Data, false). -des_ede3_cfb_crypt(_Key1, _Key2, _Key3, _IVec, _Data, _IsEncrypt) -> ?nif_stub. +des_ede3_cfb_crypt(Key1, Key2, Key3, IVec, Data, IsEncrypt) -> + case des_ede3_cfb_crypt_nif(Key1,Key2,Key3,IVec,Data,IsEncrypt) of + notsup -> erlang:error(notsup); + Bin -> Bin + end. + +des_ede3_cfb_crypt_nif(_Key1, _Key2, _Key3, _IVec, _Data, _IsEncrypt) -> ?nif_stub. %% %% Blowfish diff --git a/lib/crypto/test/crypto_SUITE.erl b/lib/crypto/test/crypto_SUITE.erl index 7ac693f371..019f5f9d83 100644 --- a/lib/crypto/test/crypto_SUITE.erl +++ b/lib/crypto/test/crypto_SUITE.erl @@ -39,6 +39,8 @@ hmac_update_md5_io/1, hmac_update_md5_n/1, hmac_rfc4231/1, + ripemd160/1, + ripemd160_update/1, sha256/1, sha256_update/1, sha512/1, @@ -86,11 +88,13 @@ groups() -> [{info, [sequence],[info, {group, rest}]}, {rest, [], [md5, md5_update, md4, md4_update, md5_mac, - md5_mac_io, sha, sha_update, + md5_mac_io, ripemd160, ripemd160_update, sha, sha_update, + sha256, sha256_update, sha512, sha512_update, hmac_update_sha, hmac_update_sha_n, hmac_update_sha256, hmac_update_sha512, hmac_update_md5_n, hmac_update_md5_io, hmac_update_md5, hmac_rfc4231, des_cbc, aes_cfb, aes_cbc, + des_cfb, des_cfb_iter, des3_cbc, des3_cfb, rc2_cbc, aes_cbc_iter, aes_ctr, aes_ctr_stream, des_cbc_iter, des_ecb, rand_uniform_test, strong_rand_test, rsa_verify_test, dsa_verify_test, rsa_sign_test, @@ -192,7 +196,16 @@ info(Config) when is_list(Config) -> {skip,"Missing crypto application"}; {_,_} -> ?line crypto:start(), - ?line crypto:info(), + ?line Info = crypto:info(), + ?line Exports = lists:usort([F || {F,_} <- crypto:module_info(exports)]), + ?line [] = Info -- Exports, + ?line NotInInfo = Exports -- Info, + io:format("NotInInfo = ~p\n", [NotInInfo]), + BlackList = lists:sort([des_ede3_cbc_decrypt, des_ede3_cbc_encrypt, + dh_check, dh_generate_parameters, + module_info, start, stop, version]), + ?line BlackList = NotInInfo, + ?line InfoLib = crypto:info_lib(), ?line [_|_] = InfoLib, F = fun([{Name,VerN,VerS}|T],Me) -> @@ -349,12 +362,8 @@ hmac_update_sha256(doc) -> hmac_update_sha256(suite) -> []; hmac_update_sha256(Config) when is_list(Config) -> - case openssl_version() of - V when V < 16#908000 -> - {skipped,"OpenSSL version too old"}; - _ -> - hmac_update_sha256_do() - end. + if_098(fun() -> hmac_update_sha256_do() end). + hmac_update_sha256_do() -> ?line Key = hexstr2bin("00010203101112132021222330313233" @@ -376,12 +385,7 @@ hmac_update_sha512(doc) -> hmac_update_sha512(suite) -> []; hmac_update_sha512(Config) when is_list(Config) -> - case openssl_version() of - V when V < 16#908000 -> - {skipped,"OpenSSL version too old"}; - _ -> - hmac_update_sha512_do() - end. + if_098(fun() -> hmac_update_sha512_do() end). hmac_update_sha512_do() -> ?line Key = hexstr2bin("00010203101112132021222330313233" @@ -422,12 +426,7 @@ hmac_rfc4231(doc) -> hmac_rfc4231(suite) -> []; hmac_rfc4231(Config) when is_list(Config) -> - case openssl_version() of - V when V < 16#908000 -> - {skipped,"OpenSSL version too old"}; - _ -> - hmac_rfc4231_do() - end. + if_098(fun() -> hmac_rfc4231_do() end). hmac_rfc4231_do() -> %% Test Case 1 @@ -722,7 +721,33 @@ hmac_update_md5_n(Config) when is_list(Config) -> ?line Mac = crypto:hmac_final_n(Ctx3, 12), ?line Exp = crypto:md5_mac_96(Key, lists:flatten([Data, Data2])), ?line m(Exp, Mac). - +%% +%% +ripemd160(doc) -> + ["Generate RIPEMD160 message digests and check the result."]; +ripemd160(suite) -> + []; +ripemd160(Config) when is_list(Config) -> + ?line m(crypto:hash(ripemd160,"abc"), + hexstr2bin("8EB208F7E05D987A9B044A8E98C6B087F15A0BFC")), + ?line m(crypto:hash(ripemd160,"abcdbcdecdefdefgefghfghighijhijkijkljklmklm" + "nlmnomnopnopq"), + hexstr2bin("12A053384A9C0C88E405A06C27DCF49ADA62EB2B")). + + +%% +%% +ripemd160_update(doc) -> + ["Generate RIPEMD160 message digests by using ripemd160_init," + "ripemd160_update, and ripemd160_final and check the result."]; +ripemd160_update(suite) -> + []; +ripemd160_update(Config) when is_list(Config) -> + ?line Ctx = crypto:hash_init(ripemd160), + ?line Ctx1 = crypto:hash_update(Ctx, "abcdbcdecdefdefgefghfghighi"), + ?line Ctx2 = crypto:hash_update(Ctx1, "jhijkijkljklmklmnlmnomnopnopq"), + ?line m(crypto:hash_final(Ctx2), + hexstr2bin("12A053384A9C0C88E405A06C27DCF49ADA62EB2B")). %% %% @@ -746,6 +771,9 @@ sha256(doc) -> sha256(suite) -> []; sha256(Config) when is_list(Config) -> + if_098(fun() -> sha256_do() end). + +sha256_do() -> ?line m(crypto:sha256("abc"), hexstr2bin("BA7816BF8F01CFEA4141" "40DE5DAE2223B00361A396177A9CB410FF61F20015AD")), @@ -762,6 +790,9 @@ sha256_update(doc) -> sha256_update(suite) -> []; sha256_update(Config) when is_list(Config) -> + if_098(fun() -> sha256_update_do() end). + +sha256_update_do() -> ?line Ctx = crypto:sha256_init(), ?line Ctx1 = crypto:sha256_update(Ctx, "abcdbcdecdefdefgefghfghighi"), ?line Ctx2 = crypto:sha256_update(Ctx1, "jhijkijkljklmklmnlmnomnopnopq"), @@ -778,6 +809,9 @@ sha512(doc) -> sha512(suite) -> []; sha512(Config) when is_list(Config) -> + if_098(fun() -> sha512_do() end). + +sha512_do() -> ?line m(crypto:sha512("abc"), hexstr2bin("DDAF35A193617ABACC417349AE20413112E6FA4E89A97EA2" "0A9EEEE64B55D39A2192992A274FC1A836BA3C23A3FEEBBD" @@ -796,6 +830,9 @@ sha512_update(doc) -> sha512_update(suite) -> []; sha512_update(Config) when is_list(Config) -> + if_098(fun() -> sha512_update_do() end). + +sha512_update_do() -> ?line Ctx = crypto:sha512_init(), ?line Ctx1 = crypto:sha512_update(Ctx, "abcdefghbcdefghicdefghijdefghijkefghijklfghijklmghijklmn"), ?line Ctx2 = crypto:sha512_update(Ctx1, "hijklmnoijklmnopjklmnopqklmnopqrlmnopqrsmnopqrstnopqrstu"), @@ -996,6 +1033,12 @@ des3_cfb(doc) -> des3_cfb(suite) -> []; des3_cfb(Config) when is_list(Config) -> + case openssl_version() of + V when V < 16#90705F -> {skipped,"OpenSSL version too old"}; + _ -> des3_cfb_do() + end. + +des3_cfb_do() -> ?line Key1 = hexstr2bin("0123456789abcdef"), ?line Key2 = hexstr2bin("fedcba9876543210"), ?line Key3 = hexstr2bin("0f2d4b6987a5c3e1"), @@ -1959,3 +2002,10 @@ openssl_version() -> undefined end. +if_098(Fun) -> + case openssl_version() of + V when V < 16#908000 -> + {skipped,"OpenSSL version too old"}; + _ -> + Fun() + end. diff --git a/lib/debugger/src/dbg_ieval.erl b/lib/debugger/src/dbg_ieval.erl index 2e88c35741..5e5948c965 100644 --- a/lib/debugger/src/dbg_ieval.erl +++ b/lib/debugger/src/dbg_ieval.erl @@ -1,7 +1,7 @@ %% %% %CopyrightBegin% %% -%% Copyright Ericsson AB 1998-2011. All Rights Reserved. +%% Copyright Ericsson AB 1998-2012. All Rights Reserved. %% %% The contents of this file are subject to the Erlang Public License, %% Version 1.1, (the "License"); you may not use this file except in @@ -1248,7 +1248,7 @@ if_clauses([], Bs, Ieval) -> exception(error, if_clause, Bs, Ieval). %% case_clauses(Value, Clauses, Bindings, Error, Ieval) -%% Error = try_clause � case_clause +%% Error = try_clause | case_clause case_clauses(Val, [{clause,_,[P],G,B}|Cs], Bs0, Error, Ieval) -> case match(P, Val, Bs0) of {match,Bs} -> diff --git a/lib/debugger/src/dbg_ui_trace.erl b/lib/debugger/src/dbg_ui_trace.erl index d318987f60..8017069c50 100644 --- a/lib/debugger/src/dbg_ui_trace.erl +++ b/lib/debugger/src/dbg_ui_trace.erl @@ -1,7 +1,7 @@ %% %% %CopyrightBegin% %% -%% Copyright Ericsson AB 1998-2009. All Rights Reserved. +%% Copyright Ericsson AB 1998-2012. All Rights Reserved. %% %% The contents of this file are subject to the Erlang Public License, %% Version 1.1, (the "License"); you may not use this file except in @@ -31,10 +31,10 @@ pid, % pid() Debugged process meta, % pid() Meta process - status, % {Status,Mod,Line} � {exit,Where,Reason} - % Status = init � idle | break - % | wait_break � wait_running - % � running + status, % {Status,Mod,Line} | {exit,Where,Reason} + % Status = init | idle | break + % | wait_break | wait_running + % | running % Where={Mod,Line} | null cm, % atom() | undefined Current module @@ -718,7 +718,7 @@ menus() -> %% enable(Status) -> [MenuItem] %% Status = init % when first message from Meta has arrived -%% | idle | break | exit | wait_break � wait_running | running +%% | idle | break | exit | wait_break | wait_running | running enable(init) -> []; enable(idle) -> ['Stop','Kill']; enable(break) -> ['Step','Next','Continue','Finish','Skip', diff --git a/lib/debugger/src/dbg_ui_trace_win.erl b/lib/debugger/src/dbg_ui_trace_win.erl index 1b439cbf18..beb3fbd71e 100644 --- a/lib/debugger/src/dbg_ui_trace_win.erl +++ b/lib/debugger/src/dbg_ui_trace_win.erl @@ -418,8 +418,8 @@ clear_breaks(WinInfo, Mod) -> %%-------------------------------------------------------------------- %% display(Arg) %% Arg = idle | {Status,Mod,Line} | {running,Mod} -%% � {exit,Where,Reason} | {text,Text} -%% Status = break | wait � Level +%% | {exit,Where,Reason} | {text,Text} +%% Status = break | wait | Level %% Level = int() %% Mod = atom() %% Line = integer() diff --git a/lib/debugger/src/dbg_wx_settings.erl b/lib/debugger/src/dbg_wx_settings.erl index 3be93c495c..94ecc5781d 100644 --- a/lib/debugger/src/dbg_wx_settings.erl +++ b/lib/debugger/src/dbg_wx_settings.erl @@ -49,31 +49,35 @@ save(Win, Pos, SFile) -> open_win(Win, Pos, SFile, Str, What) -> {SDir, SFileName} = - if - %% If settings are saved for the first time, and to - %% the default directory HOME/erlang.tools/debugger, - %% make sure the directory exists, or create it if - %% desired and possible - SFile==default -> {default_settings_dir(Win), "NoName.state"}; - true -> {filename:dirname(SFile), filename:basename(SFile)} - end, - + if + %% If settings are saved for the first time, and to + %% the default directory HOME/erlang.tools/debugger, + %% make sure the directory exists, or create it if + %% desired and possible + SFile==default -> {default_settings_dir(Win), "NoName.state"}; + true -> {filename:dirname(SFile), filename:basename(SFile)} + end, + FD = wxFileDialog:new(Win, [{message,Str},{pos, Pos}, - {defaultDir,SDir}, - {defaultFile,SFileName}, - {wildCard, "*.state"}, - {style,What}]), + {defaultDir,SDir}, + {defaultFile,SFileName}, + {wildCard, "*.state"}, + {style,What}]), case wxFileDialog:showModal(FD) of - ?wxID_OK -> - File = wxFileDialog:getFilename(FD), - Dir = wxFileDialog:getDirectory(FD), - wxFileDialog:destroy(FD), - {ok, filename:join(Dir,File)}; - _ -> - wxFileDialog:destroy(FD), - cancel + ?wxID_OK -> + case wxFileDialog:getPaths(FD) of + [NewFile] -> + wxFileDialog:destroy(FD), + {ok, NewFile}; + _ -> + wxFileDialog:destroy(FD), + cancel + end; + _ -> + wxFileDialog:destroy(FD), + cancel end. - + default_settings_dir(Win) -> {ok, [[Home]]} = init:get_argument(home), DefDir = filename:join([Home, ".erlang_tools", "debugger"]), diff --git a/lib/debugger/src/dbg_wx_trace.erl b/lib/debugger/src/dbg_wx_trace.erl index 2fdf39ba5a..f61df93aec 100644 --- a/lib/debugger/src/dbg_wx_trace.erl +++ b/lib/debugger/src/dbg_wx_trace.erl @@ -1,7 +1,7 @@ %% %% %CopyrightBegin% %% -%% Copyright Ericsson AB 2008-2011. All Rights Reserved. +%% Copyright Ericsson AB 2008-2012. All Rights Reserved. %% %% The contents of this file are subject to the Erlang Public License, %% Version 1.1, (the "License"); you may not use this file except in @@ -32,10 +32,10 @@ pid, % pid() Debugged process meta, % pid() Meta process - status, % {Status,Mod,Line} � {exit,Where,Reason} - % Status = init � idle | break - % | wait_break � wait_running - % � running + status, % {Status,Mod,Line} | {exit,Where,Reason} + % Status = init | idle | break + % | wait_break | wait_running + % | running % Where={Mod,Line} | null cm, % atom() | undefined Current module @@ -740,7 +740,7 @@ menus() -> %% enable(Status) -> [MenuItem] %% Status = init % when first message from Meta has arrived -%% | idle | break | exit | wait_break � wait_running | running +%% | idle | break | exit | wait_break | wait_running | running enable(init) -> []; enable(idle) -> ['Stop','Kill']; enable(break) -> ['Step','Next','Continue','Finish','Skip', diff --git a/lib/debugger/src/dbg_wx_trace_win.erl b/lib/debugger/src/dbg_wx_trace_win.erl index 68e8e0b844..40d110f071 100644 --- a/lib/debugger/src/dbg_wx_trace_win.erl +++ b/lib/debugger/src/dbg_wx_trace_win.erl @@ -431,8 +431,8 @@ clear_breaks(WinInfo, Mod) -> %%-------------------------------------------------------------------- %% display(Arg) %% Arg = idle | {Status,Mod,Line} | {running,Mod} -%% � {exit,Where,Reason} | {text,Text} -%% Status = break | wait � Level +%% | {exit,Where,Reason} | {text,Text} +%% Status = break | wait | Level %% Level = int() %% Mod = atom() %% Line = integer() diff --git a/lib/debugger/src/int.erl b/lib/debugger/src/int.erl index b3a8a07f03..1c9f2eddd1 100644 --- a/lib/debugger/src/int.erl +++ b/lib/debugger/src/int.erl @@ -626,18 +626,18 @@ find_src(Beam) -> find_beam(Mod, Src) -> SrcDir = filename:dirname(Src), - BeamFile = packages:last(Mod) ++ code:objfile_extension(), + BeamFile = atom_to_list(Mod) ++ code:objfile_extension(), File = filename:join(SrcDir, BeamFile), case is_file(File) of true -> File; - false -> find_beam_1(Mod, SrcDir) + false -> find_beam_1(Mod, BeamFile, SrcDir) end. -find_beam_1(Mod, SrcDir) -> - RootDir = find_root_dir(SrcDir, packages:first(Mod)), +find_beam_1(Mod, BeamFile, SrcDir) -> + RootDir = filename:dirname(SrcDir), EbinDir = filename:join(RootDir, "ebin"), CodePath = [EbinDir | code:get_path()], - BeamFile = to_path(Mod) ++ code:objfile_extension(), + BeamFile = atom_to_list(Mod) ++ code:objfile_extension(), lists:foldl(fun(_, Beam) when is_list(Beam) -> Beam; (Dir, error) -> File = filename:join(Dir, BeamFile), @@ -649,14 +649,6 @@ find_beam_1(Mod, SrcDir) -> error, CodePath). -to_path(X) -> - filename:join(packages:split(X)). - -find_root_dir(Dir, [_|Ss]) -> - find_root_dir(filename:dirname(Dir), Ss); -find_root_dir(Dir, []) -> - filename:dirname(Dir). - check_beam(BeamBin) when is_binary(BeamBin) -> case beam_lib:chunks(BeamBin, [abstract_code,exports]) of {ok,{_Mod,[{abstract_code,no_abstract_code}|_]}} -> @@ -711,14 +703,11 @@ scan_module_name_2(_, _) -> scan_module_name_3(Ts) -> case erl_parse:parse_form(Ts) of - {ok, {attribute,_,module,{M,_}}} -> module_atom(M); - {ok, {attribute,_,module,M}} -> module_atom(M); + {ok, {attribute,_,module,{M,_}}} -> M; + {ok, {attribute,_,module,M}} -> M; _ -> error end. -module_atom(A) when is_atom(A) -> A; -module_atom(L) when is_list(L) -> list_to_atom(packages:concat(L)). - %%--Stop interpreting modules----------------------------------------- del_mod(AbsMod, Dist) -> diff --git a/lib/dialyzer/doc/src/dialyzer.xml b/lib/dialyzer/doc/src/dialyzer.xml index 4080dfdf77..5e0c9b51e3 100644 --- a/lib/dialyzer/doc/src/dialyzer.xml +++ b/lib/dialyzer/doc/src/dialyzer.xml @@ -4,7 +4,7 @@ <erlref> <header> <copyright> - <year>2006</year><year>2011</year> + <year>2006</year><year>2013</year> <holder>Ericsson AB. All Rights Reserved.</holder> </copyright> <legalnotice> @@ -288,7 +288,11 @@ Option :: {files, [Filename :: string()]} | {include_dirs, [DirName :: string()]} | {output_file, FileName :: string()} | {output_plt, FileName :: string()} - | {analysis_type, 'succ_typings' | 'plt_add' | 'plt_build' | 'plt_check' | 'plt_remove'} + | {analysis_type, 'succ_typings' | + 'plt_add' | + 'plt_build' | + 'plt_check' | + 'plt_remove'} | {warnings, [WarnOpts]} | {get_warnings, bool()} diff --git a/lib/dialyzer/doc/src/dialyzer_chapter.xml b/lib/dialyzer/doc/src/dialyzer_chapter.xml index d15069991e..be75f1cc65 100644 --- a/lib/dialyzer/doc/src/dialyzer_chapter.xml +++ b/lib/dialyzer/doc/src/dialyzer_chapter.xml @@ -4,7 +4,7 @@ <chapter> <header> <copyright> - <year>2006</year><year>2009</year> + <year>2006</year><year>2013</year> <holder>Ericsson AB. All Rights Reserved.</holder> </copyright> <legalnotice> @@ -140,7 +140,9 @@ <code type="none"> - dialyzer --build_plt -r $ERL_TOP/lib/stdlib/ebin $ERL_TOP/lib/kernel/ebin $ERL_TOP/lib/mnesia/ebin + dialyzer --build_plt -r $ERL_TOP/lib/stdlib/ebin\ + $ERL_TOP/lib/kernel/ebin \ + $ERL_TOP/lib/mnesia/ebin </code> <p>Dialyzer will look if there is an environment variable called diff --git a/lib/dialyzer/doc/src/notes.xml b/lib/dialyzer/doc/src/notes.xml index 0c1556308c..f485e4d03b 100644 --- a/lib/dialyzer/doc/src/notes.xml +++ b/lib/dialyzer/doc/src/notes.xml @@ -31,6 +31,24 @@ <p>This document describes the changes made to the Dialyzer application.</p> +<section><title>Dialyzer 2.5.3</title> + + <section><title>Fixed Bugs and Malfunctions</title> + <list> + <item> + <p>Fix a crash in race condition detection</p> <p>Remove + old untested experimental extension</p> <p>Respect + {plt_check,false} option when using dialyzer:run/1</p> + <p>Fix handling of tuple set remote types appearing in + tuple sets</p> + <p> + Own Id: OTP-10464</p> + </item> + </list> + </section> + +</section> + <section><title>Dialyzer 2.5.2</title> <section><title>Fixed Bugs and Malfunctions</title> diff --git a/lib/dialyzer/test/options1_SUITE_data/src/compiler/core_scan.erl b/lib/dialyzer/test/options1_SUITE_data/src/compiler/core_scan.erl index a97270b9f3..f4e609bf5b 100644 --- a/lib/dialyzer/test/options1_SUITE_data/src/compiler/core_scan.erl +++ b/lib/dialyzer/test/options1_SUITE_data/src/compiler/core_scan.erl @@ -1,3 +1,4 @@ +%% -*- coding: utf-8 -*- %% ``The contents of this file are subject to the Erlang Public License, %% Version 1.1, (the "License"); you may not use this file except in %% compliance with the License. You should have received a copy of the @@ -30,16 +31,16 @@ %% 173 - 176 { - ~ punctuation %% 177 DEL control %% 200 - 237 control -%% 240 - 277 NBSP - � punctuation -%% 300 - 326 � - � uppercase -%% 327 � punctuation -%% 330 - 336 � - � uppercase -%% 337 - 366 � - � lowercase -%% 367 � punctuation -%% 370 - 377 � - � lowercase +%% 240 - 277 NBSP - ¿ punctuation +%% 300 - 326 À - Ö uppercase +%% 327 × punctuation +%% 330 - 336 Ø - Þ uppercase +%% 337 - 366 ß - ö lowercase +%% 367 ÷ punctuation +%% 370 - 377 ø - ÿ lowercase %% %% Many punctuation characters region have special meaning. Must -%% watch using � \327, bvery close to x \170 +%% watch using × \327, bvery close to x \170 -module(core_scan). @@ -266,11 +267,11 @@ scan1([C|Cs], Toks, Pos) when C >= $\200, C =< $\240 -> scan1(Cs, Toks, Pos); scan1([C|Cs], Toks, Pos) when C >= $a, C =< $z -> %Keywords scan_key_word(C, Cs, Toks, Pos); -scan1([C|Cs], Toks, Pos) when C >= $�, C =< $�, C /= $� -> +scan1([C|Cs], Toks, Pos) when C >= $ß, C =< $ÿ, C /= $÷ -> scan_key_word(C, Cs, Toks, Pos); scan1([C|Cs], Toks, Pos) when C >= $A, C =< $Z -> %Variables scan_variable(C, Cs, Toks, Pos); -scan1([C|Cs], Toks, Pos) when C >= $�, C =< $�, C /= $� -> +scan1([C|Cs], Toks, Pos) when C >= $À, C =< $Þ, C /= $× -> scan_variable(C, Cs, Toks, Pos); scan1([C|Cs], Toks, Pos) when C >= $0, C =< $9 -> %Numbers scan_number(C, Cs, Toks, Pos); @@ -335,9 +336,9 @@ scan_name([], Ncs) -> {Ncs,[]}. name_char(C) when C >= $a, C =< $z -> true; -name_char(C) when C >= $�, C =< $�, C /= $� -> true; +name_char(C) when C >= $ß, C =< $ÿ, C /= $÷ -> true; name_char(C) when C >= $A, C =< $Z -> true; -name_char(C) when C >= $�, C =< $�, C /= $� -> true; +name_char(C) when C >= $À, C =< $Þ, C /= $× -> true; name_char(C) when C >= $0, C =< $9 -> true; name_char($_) -> true; name_char($@) -> true; diff --git a/lib/dialyzer/test/r9c_SUITE_data/src/asn1/asn1ct_check.erl b/lib/dialyzer/test/r9c_SUITE_data/src/asn1/asn1ct_check.erl index 2f0ada122e..4335d8efae 100644 --- a/lib/dialyzer/test/r9c_SUITE_data/src/asn1/asn1ct_check.erl +++ b/lib/dialyzer/test/r9c_SUITE_data/src/asn1/asn1ct_check.erl @@ -4599,7 +4599,7 @@ get_simple_table_info1(S,#'ComponentType'{typespec=TS},[],Path) -> %% o.w. the asn1 code is wrong. #type{def=OCFT,constraint=Cnstr} = TS, case Cnstr of - [{simpletable,_OSRef}]�-> + [{simpletable,_OSRef}] -> #'ObjectClassFieldType'{classname=ClRef, class=ObjectClass, fieldname=FieldName} = OCFT, diff --git a/lib/dialyzer/test/r9c_SUITE_data/src/asn1/asn1ct_gen.erl b/lib/dialyzer/test/r9c_SUITE_data/src/asn1/asn1ct_gen.erl index 5d2f7a13bd..32d4a22645 100644 --- a/lib/dialyzer/test/r9c_SUITE_data/src/asn1/asn1ct_gen.erl +++ b/lib/dialyzer/test/r9c_SUITE_data/src/asn1/asn1ct_gen.erl @@ -148,7 +148,7 @@ pgen_partial_inc_dec(Erules,Module) -> % io:format("Start partial incomplete decode gen?~n"), case asn1ct:get_gen_state_field(inc_type_pattern) of undefined -> -% io:format("Partial incomplete decode gen not started:�~w~n",[asn1ct:get_gen_state_field(active)]), +% io:format("Partial incomplete decode gen not started: ~w~n",[asn1ct:get_gen_state_field(active)]), ok; % [] -> % ok; diff --git a/lib/dialyzer/test/r9c_SUITE_data/src/asn1/asn1rt_ber_bin.erl b/lib/dialyzer/test/r9c_SUITE_data/src/asn1/asn1rt_ber_bin.erl index 6064515a7e..b0786200fc 100644 --- a/lib/dialyzer/test/r9c_SUITE_data/src/asn1/asn1rt_ber_bin.erl +++ b/lib/dialyzer/test/r9c_SUITE_data/src/asn1/asn1rt_ber_bin.erl @@ -276,7 +276,7 @@ encode_tag_val({Class, Form, TagNo}) -> <<(Class bsr 6):2, (Form bsr 5):1, 31:5,BinOct/binary>>; %% asumes whole correct tag bitpattern, multiple of 8 -encode_tag_val(Tag) when (Tag =< 255) -> Tag; %% anv�nds denna funktion??!! +encode_tag_val(Tag) when (Tag =< 255) -> Tag; %% is this function used??!! %% asumes correct bitpattern of 0-5 encode_tag_val(Tag) -> encode_tag_val2(Tag,[]). diff --git a/lib/dialyzer/test/r9c_SUITE_data/src/asn1/asn1rt_ber_bin_v2.erl b/lib/dialyzer/test/r9c_SUITE_data/src/asn1/asn1rt_ber_bin_v2.erl index 50a91cf201..2f25e35cd3 100644 --- a/lib/dialyzer/test/r9c_SUITE_data/src/asn1/asn1rt_ber_bin_v2.erl +++ b/lib/dialyzer/test/r9c_SUITE_data/src/asn1/asn1rt_ber_bin_v2.erl @@ -531,7 +531,7 @@ encode_tag_val({Class, Form, TagNo}) -> <<(Class bsr 6):2, (Form bsr 5):1, 31:5,BinOct/binary>>; %% asumes whole correct tag bitpattern, multiple of 8 -encode_tag_val(Tag) when (Tag =< 255) -> Tag; %% anv�nds denna funktion??!! +encode_tag_val(Tag) when (Tag =< 255) -> Tag; %% is this function used??!! %% asumes correct bitpattern of 0-5 encode_tag_val(Tag) -> encode_tag_val2(Tag,[]). diff --git a/lib/dialyzer/test/r9c_SUITE_data/src/asn1/asn1rt_per_bin_rt2ct.erl b/lib/dialyzer/test/r9c_SUITE_data/src/asn1/asn1rt_per_bin_rt2ct.erl index 9f02ad4466..4781c81955 100644 --- a/lib/dialyzer/test/r9c_SUITE_data/src/asn1/asn1rt_per_bin_rt2ct.erl +++ b/lib/dialyzer/test/r9c_SUITE_data/src/asn1/asn1rt_per_bin_rt2ct.erl @@ -1198,7 +1198,7 @@ encode_bin_bit_string(C,UnusedAndBin={_,_},NamedBitList) -> % case get_constraint(C,'SizeConstraint') of % 0 -> -% []; % borde avg�ras i compile-time +% []; % should be dont in compile time % V when integer(V),V=<16 -> % {Unused2,Bin2} = pad_list(V,UnusedAndBin1), % <<BitVal:V,_:Unused2>> = Bin2, diff --git a/lib/dialyzer/test/small_SUITE_data/src/comm_layer/comm_connection.erl b/lib/dialyzer/test/small_SUITE_data/src/comm_layer/comm_connection.erl index 48cc50ae21..5a8f9710d6 100644 --- a/lib/dialyzer/test/small_SUITE_data/src/comm_layer/comm_connection.erl +++ b/lib/dialyzer/test/small_SUITE_data/src/comm_layer/comm_connection.erl @@ -1,4 +1,5 @@ -% Copyright 2008 Konrad-Zuse-Zentrum f�r Informationstechnik Berlin +%% -*- coding: utf-8 -*- +% Copyright 2008 Konrad-Zuse-Zentrum für Informationstechnik Berlin % % Licensed under the Apache License, Version 2.0 (the "License"); % you may not use this file except in compliance with the License. @@ -21,7 +22,7 @@ %%% Created : 18 Apr 2008 by Thorsten Schuett <[email protected]> %%%------------------------------------------------------------------- %% @author Thorsten Schuett <[email protected]> -%% @copyright 2008 Konrad-Zuse-Zentrum f�r Informationstechnik Berlin +%% @copyright 2008 Konrad-Zuse-Zentrum für Informationstechnik Berlin %% @version $Id $ -module(comm_layer_dir.comm_connection). diff --git a/lib/dialyzer/test/small_SUITE_data/src/comm_layer/comm_port.erl b/lib/dialyzer/test/small_SUITE_data/src/comm_layer/comm_port.erl index e8169b4673..d9fcb5e625 100644 --- a/lib/dialyzer/test/small_SUITE_data/src/comm_layer/comm_port.erl +++ b/lib/dialyzer/test/small_SUITE_data/src/comm_layer/comm_port.erl @@ -1,4 +1,5 @@ -% Copyright 2008 Konrad-Zuse-Zentrum f�r Informationstechnik Berlin +%% -*- coding: utf-8 -*- +% Copyright 2008 Konrad-Zuse-Zentrum für Informationstechnik Berlin % % Licensed under the Apache License, Version 2.0 (the "License"); % you may not use this file except in compliance with the License. @@ -20,7 +21,7 @@ %%% Created : 18 Apr 2008 by Thorsten Schuett <[email protected]> %%%------------------------------------------------------------------- %% @author Thorsten Schuett <[email protected]> -%% @copyright 2008 Konrad-Zuse-Zentrum f�r Informationstechnik Berlin +%% @copyright 2008 Konrad-Zuse-Zentrum für Informationstechnik Berlin %% @version $Id $ -module(comm_layer_dir.comm_port). diff --git a/lib/dialyzer/vsn.mk b/lib/dialyzer/vsn.mk index 64d05156b1..f344fe0604 100644 --- a/lib/dialyzer/vsn.mk +++ b/lib/dialyzer/vsn.mk @@ -1 +1 @@ -DIALYZER_VSN = 2.5.2 +DIALYZER_VSN = 2.5.3 diff --git a/lib/diameter/doc/src/diameter.xml b/lib/diameter/doc/src/diameter.xml index bc42b75c7a..b7669b760b 100644 --- a/lib/diameter/doc/src/diameter.xml +++ b/lib/diameter/doc/src/diameter.xml @@ -1,5 +1,11 @@ <?xml version="1.0" encoding="latin1" ?> <!DOCTYPE erlref SYSTEM "erlref.dtd" [ + <!ENTITY make_ref + '<seealso marker="erts:erlang#make_ref-0">erlang:make_ref/0</seealso>'> + <!ENTITY transport_module + '<seealso marker="diameter_transport">transport module</seealso>'> + <!ENTITY dictionary + '<seealso marker="diameter_dict">dictionary</seealso>'> <!ENTITY % also SYSTEM "seealso.ent" > <!ENTITY % here SYSTEM "seehere.ent" > %also; @@ -50,7 +56,7 @@ under the License. <p> This module provides the interface with which a user can implement a Diameter node that sends and receives messages using the -Diameter protocol as defined in RFC 3588.</p> +Diameter protocol as defined in &the_rfc;.</p> <p> Basic usage consists of creating a representation of a @@ -67,7 +73,7 @@ Beware the difference between <em>diameter</em> (not capitalised) and <em>Diameter</em> (capitalised). The former refers to the Erlang application named diameter whose main api is defined here, the latter to Diameter protocol in the sense of -RFC 3588.</p> +&the_rfc;.</p> <p> The diameter application must be started before calling most functions @@ -91,7 +97,7 @@ in this module.</p> <tag><c>UTF8String()</c></tag> <item> <p> -Types corresponding to RFC 3588 AVP Data Formats. +Types corresponding to &the_rfc; AVP Data Formats. Defined in &dict_data_types;.</p> <marker id="application_alias"/> @@ -298,7 +304,7 @@ Has one of the following types.</p> <item> <p> An address list is available to the start function of a -<seealso marker="diameter_transport">transport module</seealso>, which +&transport_module;, which can return a new list for use in the subsequent CER or CEA. Host-IP-Address need not be specified if the transport start function returns an address list.</p> @@ -312,7 +318,7 @@ returns an address list.</p> Origin-State-Id is optional but will be included in outgoing messages sent by diameter itself: CER/CEA, DWR/DWA and DPR/DPA. Setting a value of <c>0</c> (zero) is equivalent to not setting a -value as documented in RFC 3588. +value as documented in &the_rfc;. The function &origin_state_id; can be used as to retrieve a value that is computed when the diameter application is started.</p> @@ -464,7 +470,7 @@ that matches no peer.</p> <p> The <c>host</c> and <c>realm</c> filters examine the outgoing request as passed to &call;, -assuming that this is a record- or list-valued <c>&app_message;</c>, +assuming that this is a record- or list-valued <c>&codec_message;</c>, and that the message contains at most one of each AVP. If this is not the case then the <c>{host|realm, &dict_DiameterIdentity;}</c> filters must be used to achieve the desired result. @@ -529,8 +535,7 @@ connectivity.</p> <p> Note that a single <c>up</c>/<c>down</c> event for a given peer -corresponds to one -<seealso marker="diameter_app#Mod:peer_up-3">peer_up/peer_down</seealso> +corresponds to one &app_peer_up;/&app_peer_down; callback for each of the Diameter applications negotiated during capablilities exchange. That is, the event communicates connectivity with the @@ -677,7 +682,7 @@ info fields of forms other than the above.</p> The name of a service as passed to &start_service; and with which the service is identified. There can be at most one service with a given name on a given node. -Note that <seealso marker="erts:erlang#make_ref-0">erlang:make_ref/0</seealso> +Note that &make_ref; can be used to generate a service name that is somewhat unique.</p> <marker id="service_opt"/> @@ -703,8 +708,7 @@ For an outgoing Diameter request, the relevant <c>&application_alias;</c> is passed to &call;, while for an incoming request the application identifier in the message header determines the application, the identifier being specified in -the application's <seealso marker="diameter_dict">dictionary</seealso> -file.</p> +the application's &dictionary; file.</p> </item> <tag><c>{restrict_connections, false @@ -749,7 +753,7 @@ as follows.</p> (H bsl N) bor (Id band ((1 bsl N) - 1)) </pre> <p> -Note that RFC 3588 requires that End-to-End identifiers remain unique +Note that &the_rfc; requires that End-to-End identifiers remain unique for a period of at least 4 minutes and that this and the call rate places a lower bound on the appropriate values of <c>N</c>: at a rate of <c>R</c> requests per second an <c>N</c>-bit counter @@ -951,7 +955,7 @@ Tc = &dict_Unsigned32; </pre> <p> -For a connecting transport, the RFC 3588 Tc timer, in milliseconds. +For a connecting transport, the &the_rfc; Tc timer, in milliseconds. Note that this timer determines the frequency with which a transport will attempt to establish a connection with its peer only <em>before</em> an initial connection is established: once there is an initial @@ -1125,7 +1129,7 @@ its transports.</p> <type> <v>SvcName = &service_name;</v> <v>App = &application_alias;</v> -<v>Request = &app_message;</v> +<v>Request = &codec_message;</v> <v>Answer = term()</v> <v>Opt = &call_opt;</v> </type> @@ -1618,7 +1622,7 @@ Return the list of started services.</p> Return a value for a Session-Id AVP.</p> <p> -The value has the form required by section 8.8 of RFC 3588. +The value has the form required by section 8.8 of &the_rfc;. Ident should be the Origin-Host of the peer from which the message containing the returned value will be sent.</p> diff --git a/lib/diameter/doc/src/diameter_app.xml b/lib/diameter/doc/src/diameter_app.xml index 304c69ebda..f4db625c71 100644 --- a/lib/diameter/doc/src/diameter_app.xml +++ b/lib/diameter/doc/src/diameter_app.xml @@ -1,5 +1,8 @@ <?xml version="1.0" encoding="latin1" ?> <!DOCTYPE erlref SYSTEM "erlref.dtd" [ + <!ENTITY message '<seealso marker="#message">message()</seealso>'> + <!ENTITY dict + '<seealso marker="diameter_dict#MESSAGE_RECORDS">diameter_dict(4)</seealso>'> <!ENTITY % also SYSTEM "seealso.ent" > <!ENTITY % here SYSTEM "seehere.ent" > %also; @@ -124,39 +127,17 @@ mandatory values as the bare value.</p> <marker id="message"/> -<tag><c>message() = record() | list()</c></tag> +<tag><c>message() = &codec_message;</c></tag> <item> <p> The representation of a Diameter message as passed to -&mod_call;. -The record representation is as outlined in -<seealso -marker="diameter_dict#MESSAGE_RECORDS">diameter_dict(4)</seealso>: -a message as defined in a dictionary file is encoded as a record with -one field for each component AVP. -Equivalently, a message can also be encoded as a list whose head is -the atom-valued message name (the record name minus any -prefix specified in the relevant dictionary file) and whose tail is a -list of <c>{FieldName, FieldValue}</c> pairs.</p> - -<p> -A third representation allows a message to be specified as a list -whose head is a <c>#diameter_header{}</c> record and whose tail is a list -of <c>#diameter_avp{}</c> records. -This representation is used by diameter itself when relaying requests -as directed by the return value of a -&handle_request; -callback. -It differs from the other other two in that it bypasses the checks for -messages that do not agree with their definitions in the dictionary in -question (since relays agents must handle arbitrary request): messages -are sent exactly as specified.</p> +&mod_call; or returned from a &handle_request; callback.</p> </item> <marker id="packet"/> -<tag><c>packet() = #diameter_packet{}</c></tag> +<tag><c>packet() = &codec_packet;</c></tag> <item> <p> A container for incoming and outgoing Diameter messages that's passed @@ -505,8 +486,7 @@ The application in which the callback takes place (that is, the callback module as configured with &mod_start_service;) is determined by the Application Identifier in the header of the incoming request message, the selected module being the one -whose corresponding <seealso -marker="diameter_dict#MESSAGE_RECORDS">dictionary</seealso> declares +whose corresponding dictionary declares itself as defining either the application in question or the Relay application.</p> @@ -526,8 +506,7 @@ The argument &packet; has the following signature.</p> The <c>msg</c> field will be <c>undefined</c> in case the request has been received in the relay application. Otherwise it contains the record representing the request as outlined -in <seealso -marker="diameter_dict#MESSAGE_RECORDS">diameter_dict(4)</seealso>.</p> +in &dict;.</p> <p> The <c>errors</c> field specifies any Result-Code's identifying errors @@ -579,7 +558,7 @@ where <c>Avps</c> sets the Origin-Host, Origin-Realm, the specified Result-Code and (if the request sent one) Session-Id AVP's.</p> <p> -Note that RFC 3588 mandates that only answers with a 3xxx series +Note that &the_rfc; mandates that only answers with a 3xxx series Result-Code (protocol errors) may set the E bit. Returning a non-3xxx value in a <c>protocol_error</c> tuple will cause the request process in question to fail.</p> diff --git a/lib/diameter/doc/src/diameter_codec.xml b/lib/diameter/doc/src/diameter_codec.xml new file mode 100644 index 0000000000..4a77d5435b --- /dev/null +++ b/lib/diameter/doc/src/diameter_codec.xml @@ -0,0 +1,389 @@ +<?xml version="1.0" encoding="latin1" ?> +<!DOCTYPE erlref SYSTEM "erlref.dtd" [ + <!ENTITY records + '<seealso marker="diameter_dict#MESSAGE_RECORDS">diameter_dict(4)</seealso>'> + <!ENTITY types + '<seealso marker="diameter_dict#DATA_TYPES">diameter_dict(4)</seealso>'> + <!ENTITY % also SYSTEM "seealso.ent" > + <!ENTITY % here SYSTEM "seehere.ent" > + %also; + %here; +]> + +<erlref> +<header> +<copyright> +<year>2012</year> +<holder>Ericsson AB. All Rights Reserved.</holder> +</copyright> +<legalnotice> +The contents of this file are subject to the Erlang Public License, +Version 1.1, (the "License"); you may not use this file except in +compliance with the License. You should have received a copy of the +Erlang Public License along with this software. If not, it can be +retrieved online at http://www.erlang.org/. + +Software distributed under the License is distributed on an "AS IS" +basis, WITHOUT WARRANTY OF ANY KIND, either express or implied. See +the License for the specific language governing rights and limitations +under the License. + +</legalnotice> + +<title>diameter_codec(3)</title> +<prepared>Anders Svensson</prepared> +<responsible></responsible> +<docno></docno> +<approved></approved> +<checked></checked> +<date></date> +<rev></rev> +<file>diameter_codec.xml</file> +</header> + +<module>diameter_codec</module> +<modulesummary>Decode and encode of Diameter messages.</modulesummary> + +<description> + +<p> +Incoming Diameter messages are decoded from binary() before being +communicated to &man_app; callbacks. +Similarly, outgoing Diameter messages are encoded into binary() before +being passed to the appropriate &man_transport; module for +transmission. +The functions in this module implement this encode/decode.</p> + +<note> +<p> +Calls to this module are made by diameter itself as a consequence of +configuration passed to &mod_start_service;. +The encode/decode functions may also be useful for other purposes (eg. +test) but the diameter user does not need to call them explicitly when +sending and receiving messages using &mod_call; and the callback +interface documented in &man_app;.</p> +</note> + +<p> +The &header; and &packet; records below +are defined in diameter.hrl, which can be included as follows.</p> + +<pre> +-include_lib("diameter/include/diameter.hrl"). +</pre> + +<p> +Application-specific records are definied in the hrl +files resulting from dictionary file compilation.</p> + +</description> + +<!-- ===================================================================== --> + +<section> +<title>DATA TYPES</title> + +<p></p> + +<taglist> + +<marker id="integers"/> + +<tag><c>uint8() = 0..255</c></tag> +<tag><c>uint24() = 0..16777215</c></tag> +<tag><c>uint32() = 0..4294967295</c></tag> +<item> +<p> +8-bit, 24-bit and 32-bit integers occurring in Diameter and AVP +headers.</p> +</item> + +<marker id="avp"/> + +<tag><c>avp() = #diameter_avp{}</c></tag> +<item> +<p> +The application-neutral representation of an AVP. +Primarily intended for use by relay applications that need to handle +arbitrary Diameter applications. +A service implementing a specific Diameter application +(for which it configures a dictionary) can manipulate values of type +&message; instead.</p> + +<p> +Fields have the following types.</p> + +<taglist> + +<tag><c>code = uint32()</c></tag> +<tag><c>is_mandatory = boolean()</c></tag> +<tag><c>need_encryption = boolean()</c></tag> +<tag><c>vendor_id = uint32() | undefined</c></tag> +<item> +<p> +Values in the AVP header, corresponding to AVP Code, the M flag, P +flags and Vendor-ID respectivelty. +A Vendor-ID other than <c>undefined</c> implies a set V flag.</p> +</item> + +<tag><c>data = iolist()</c></tag> +<item> +<p> +The data bytes of the AVP.</p> +</item> + +<tag><c>name = atom()</c></tag> +<item> +<p> +The name of the AVP as defined in the dictionary file in question, or +<c>undefined</c> if the AVP is unknown to the dictionary file in +question.</p> +</item> + +<tag><c>value = term()</c></tag> +<item> +<p> +The decoded value of an AVP. +Will be <c>undefined</c> on decode if the data bytes could +not be decoded or the AVP is unknown. +The type of a decoded value is as document in &types;.</p> +</item> + +<tag><c>type = atom()</c></tag> +<item> +<p> +The type of the AVP as specified in the dictionary file in question +(or one it inherits). +Possible types are <c>undefined</c> and the Diameter types: +<c>OctetString</c>, <c>Integer32</c>, <c>Integer64</c>, +<c>Unsigned32</c>, <c>Unsigned64</c>, <c>Float32</c>, <c>Float64</c>, +<c>Grouped</c>, <c>Enumerated</c>, <c>Address</c>, <c>Time</c>, +<c>UTF8String</c>, <c>DiameterIdentity</c>, <c>DiameterURI</c>, +<c>IPFilterRule</c> and <c>QoSFilterRule</c>.</p> +</item> + +</taglist> + +</item> + +<marker id="dictionary"/> + +<tag><c>dictionary() = module()</c></tag> +<item> + +<p> +The name of a generated dictionary module as generated by &man_compile; +or &make_codec;. +The interface provided by a dictionary module is an +implementation detail that may change.</p> +</item> + +<marker id="header"/> + +<tag><c>header() = #diameter_header{}</c></tag> +<item> +<p> +The record representation of the Diameter header. +Values in a &packet; returned by &decode; are as extracted from the +incoming message. +Values set in an &packet; passed to &encode; are preserved in the +encoded binary(), with the exception of <c>length</c>, <c>cmd_code</c> +and <c>application_id</c>, all of which are determined by the +&dictionary; in question.</p> + +<note> +<p> +It is not necessary to set header fields explicitly in outgoing +messages as diameter itself will set appropriate values. +Setting inappropriate values can be useful for test purposes.</p> +</note> + +<p> +Fields have the following types.</p> + +<taglist> + +<tag><c>version = uint8()</c></tag> +<tag><c>length = uint24()</c></tag> +<tag><c>cmd_code = uint24()</c></tag> +<tag><c>application_id = uint32()</c></tag> +<tag><c>hop_by_hop_id = uint32()</c></tag> +<tag><c>end_to_end_id = uint32()</c></tag> +<item> +<p> +Values of the Version, Message Length, Command-Code, Application-ID, +Hop-by-Hop Identifier and End-to-End Identifier fields of the Diameter +header.</p> +</item> + +<tag><c>is_request = boolean()</c></tag> +<tag><c>is_proxiable = boolean()</c></tag> +<tag><c>is_error = boolean()</c></tag> +<tag><c>is_retransmitted = boolean()</c></tag> +<item> +<p> +Values correspoding to the R(equest), P(roxiable), E(rror) +and T(Potentially re-transmitted message) flags of the Diameter +header.</p> +</item> + +</taglist> + +</item> + +<marker id="message"/> + +<tag><c>message() = record() | list()</c></tag> +<item> +<p> +The representation of a Diameter message as passed to +&mod_call; or returned from a &app_handle_request; callback. +The record representation is as outlined in &records;: +a message as defined in a dictionary file is encoded as a record with +one field for each component AVP. +Equivalently, a message can also be encoded as a list whose head is +the atom-valued message name (as specified in the relevant dictionary +file) and whose tail is a list of <c>{AvpName, AvpValue}</c> pairs.</p> + +<p> +Another list-valued representation allows a message to be specified +as a list whose head is a &header; and whose tail is an &avp; list. +This representation is used by diameter itself when relaying requests +as directed by the return value of a &app_handle_request; callback. +It differs from the other other two in that it bypasses the checks for +messages that do not agree with their definitions in the dictionary in +question: messages are sent exactly as specified.</p> + +</item> + +<marker id="packet"/> + +<tag><c>packet() = #diameter_packet{}</c></tag> +<item> +<p> +A container for incoming and outgoing Diameter messages. +Fields have the following types.</p> + +<taglist> + +<tag><c>header = &header; | undefined</c></tag> +<item> +<p> +The Diameter header of the message. +Can be (and typically should be) <c>undefined</c> for an outgoing +message in a non-relay application, in which case diameter provides +appropriate values.</p> +</item> + +<tag><c>avps = [&avp;] | undefined</c></tag> +<item> +<p> +The AVPs of the message. +Ignored for an outgoing message if the <c>msg</c> field is set to a +value other than <c>undefined</c>.</p> +</item> + +<tag><c>msg = &message; | undefined</c></tag> +<item> +<p> +The incoming/outgoing message. +For an incoming message, a record if the message can be +decoded in a non-relay application, <c>undefined</c> otherwise. +For an outgoing message, setting a <c>[&header; | &avp;]</c> list is +equivalent to setting the <c>header</c> and <c>avps</c> fields to the +corresponding values.</p> + +<warning> +<p> +A record-valued <c>msg</c> field does <b>not</b> imply an absence of +decode errors. +The <c>errors</c> field should also be examined.</p> +</warning> + +</item> + +<tag><c>bin = binary()</c></tag> +<item> +<p> +The incoming message prior to encode or the outgoing message after +encode.</p> +</item> + +<tag><c>errors = [5000..5999 | {5000..5999, avp()}]</c></tag> +<item> +<p> +Errors detected at decode of an incoming message, as identified by +a corresponding 5xxx series Result-Code (Permanent Failures). +For an incoming request, these should be used to formulate an +appropriate answer as documented for the &app_handle_request; +callback in &man_app;. +For an incoming answer, the &mod_application_opt; +<c>answer_errors</c> determines the behaviour.</p> +</item> + +<tag><c>transport_data = term()</c></tag> +<item> +<p> +An arbitrary term of meaning only to the transport process in +question, as documented in &man_transport;.</p> +</item> + +</taglist> + +</item> + +</taglist> + +</section> + +<!-- ===================================================================== --> + +<funcs> + +<func> +<name>decode(Mod, Bin) -> Pkt</name> +<fsummary>Decode a Diameter message.</fsummary> +<type> +<v>Mod = &dictionary;</v> +<v>Bin = binary()</v> +<v>Pkt = &packet;</v> +</type> +<desc> + +<p> +Decode a Diameter message.</p> + +</desc> +</func> + +<func> +<name>encode(Mod, Msg) -> Pkt</name> +<fsummary>Encode a Diameter message.</fsummary> +<type> +<v>Mod = &dictionary;</v> +<v>Msg = &message; | &packet;</v> +<v>Pkt = &packet;</v> +</type> +<desc> + +<p> +Encode a Diameter message. +</p> + +</desc> +</func> + +</funcs> + +<!-- ===================================================================== --> +<!-- ===================================================================== --> + +<section> +<title>SEE ALSO</title> + +<p> +&man_compile;, &man_app;, &man_dict;, &man_make;</p> + +</section> + +</erlref> diff --git a/lib/diameter/doc/src/diameter_compile.xml b/lib/diameter/doc/src/diameter_compile.xml index eb6de80c11..0bd7ad1789 100644 --- a/lib/diameter/doc/src/diameter_compile.xml +++ b/lib/diameter/doc/src/diameter_compile.xml @@ -1,5 +1,7 @@ <?xml version="1.0" encoding="iso-8859-1" ?> <!DOCTYPE comref SYSTEM "comref.dtd" [ + <!ENTITY dictionary + '<seealso marker="diameter_dict">dictionary file</seealso>'> <!ENTITY % also SYSTEM "seealso.ent" > <!ENTITY % here SYSTEM "seehere.ent" > %also; @@ -36,12 +38,14 @@ supplied. <description> <p> -The diameterc utility is used to compile diameter -<seealso marker="diameter_dict">dictionary files</seealso> -into Erlang source. -The resulting source implements the interface diameter requires +The diameterc utility is used to compile a diameter +&dictionary; into Erlang source. +The resulting source implements the interface diameter required to encode and decode the dictionary's messages and AVP's.</p> +<p> +The module &man_make; provides an alternate compilation interface.</p> + </description> <section> @@ -52,53 +56,52 @@ to encode and decode the dictionary's messages and AVP's.</p> <tag><![CDATA[diameterc [<options>] <file>]]></tag> <item> <p> -Compiles a single dictionary file. Valid options are as follows.</p> - -<!-- Leave -h/d/v undocumented, except for the usage message from the - utility itself. --> - -<taglist> -<tag><![CDATA[-o <dir>]]></tag> -<item> -<p> -Specifies the directory into which the generated source should be written. -Defaults to the current working directory.</p> -</item> +Compile a single dictionary file to Erlang source. +Valid options are as follows.</p> <tag><![CDATA[-i <dir>]]></tag> <item> <p> -Specifies a directory to add to the code path. +Prepend the specified directory to the code path. Use to point at beam files compiled from inherited dictionaries, -<c>@inherits</c> in a dictionary file creating a beam dependency, not -an erl/hrl dependency.</p> +<c>&dict_inherits;</c> in a dictionary file creating a beam +dependency, not an erl/hrl dependency.</p> <p> Multiple <c>-i</c> options can be specified.</p> </item> +<taglist> +<tag><![CDATA[-o <dir>]]></tag> +<item> +<p> +Write generated source to the specified directory. +Defaults to the current working directory.</p> +</item> + <tag><![CDATA[-E]]></tag> <tag><![CDATA[-H]]></tag> <item> <p> -Supresses erl and hrl generation, respectively.</p> +Supress erl and hrl generation, respectively.</p> </item> <tag><![CDATA[--name <name>]]></tag> <tag><![CDATA[--prefix <prefix>]]></tag> <item> <p> -Set <c>@name</c> and <c>@prefix</c> in the dictionary, -respectively. +Set <c>&dict_name;</c> or <c>&dict_prefix;</c> to the specified +string. Overrides any setting in the file itself.</p> </item> <tag><![CDATA[--inherits <dict>]]></tag> <item> <p> -Append an <c>@inherits</c> to the dictionary before compiling. -Specifying <c>'-'</c> as the dictionary has the effect of clearing any -previous inherits, causing them to be ignored.</p> +Append &dict_inherits; of the specified module. +Specifying <c>"-"</c> has the effect of discarding clearing any +previous inherits, both in the dictionary file and on the options +list.</p> <p> Multiple <c>--inherits</c> options can be specified.</p> @@ -127,7 +130,7 @@ Returns 0 on success, non-zero on failure.</p> <title>SEE ALSO</title> <p> -&man_dict;</p> +&man_make;, &man_dict;</p> </section> diff --git a/lib/diameter/doc/src/diameter_dict.xml b/lib/diameter/doc/src/diameter_dict.xml index 4a6cccc276..8b0687a22e 100644 --- a/lib/diameter/doc/src/diameter_dict.xml +++ b/lib/diameter/doc/src/diameter_dict.xml @@ -1,5 +1,11 @@ <?xml version="1.0" encoding="latin1" ?> <!DOCTYPE erlref SYSTEM "fileref.dtd" [ + <!ENTITY format + '<seealso marker="#FILE_FORMAT">FILE FORMAT</seealso>'> + <!ENTITY records + '<seealso marker="#MESSAGE_RECORDS">MESSAGE RECORDS</seealso>'> + <!ENTITY types + '<seealso marker="#DATA_TYPES">DATA TYPES</seealso>'> <!ENTITY % also SYSTEM "seealso.ent" > <!ENTITY % here SYSTEM "seehere.ent" > %also; @@ -45,34 +51,33 @@ under the License. <description> <p> -A diameter service as configured with &mod_start_service; +A diameter service, as configured with &mod_start_service;, specifies one or more supported Diameter applications. Each Diameter application specifies a dictionary module that knows how to encode and decode its messages and AVPs. The dictionary module is in turn generated from a file that defines these messages and AVPs. -The format of such a file is defined in -<seealso marker="#FILE_FORMAT">FILE FORMAT</seealso> below. +The format of such a file is defined in &format; below. Users add support for their specific applications by creating dictionary files, compiling them to Erlang modules using -<seealso marker="diameterc">diameterc</seealso> and configuring the +either &man_compile; or &man_make; and configuring the resulting dictionaries modules on a service.</p> <p> -The codec generation also results in a hrl file that defines records -for the messages and grouped AVPs defined for the application, these -records being what a user of the diameter application sends and receives. -(Modulo other available formats as discussed in &man_app;.) +Dictionary module generation also results in a hrl file that defines +records for the messages and Grouped AVPs defined by the +dictionary, these records being what a user of the diameter +application sends and receives, modulo other possible formats as +discussed in &man_app;. These records and the underlying Erlang data types corresponding to -Diameter data formats are discussed in <seealso -marker="#MESSAGE_RECORDS">MESSAGE RECORDS</seealso> and <seealso -marker="#DATA_TYPES">DATA TYPES</seealso> respectively. -The generated hrl also contains defines for the possible values of +Diameter data formats are discussed in &records; and &types; +respectively. +The generated hrl also contains macro definitions for the possible values of AVPs of type Enumerated.</p> <p> The diameter application includes three dictionary modules -corresponding to applications defined in section 2.4 of RFC 3588: +corresponding to applications defined in section 2.4 of &the_rfc;: <c>diameter_gen_base_rfc3588</c> for the Diameter Common Messages application with application identifier 0, <c>diameter_gen_accounting</c> for the Diameter Base Accounting @@ -111,6 +116,8 @@ The order in which sections are specified is unimportant.</p> <taglist> +<marker id="id"/> + <tag><c>@id Number</c></tag> <item> <p> @@ -134,6 +141,8 @@ Example:</p> </item> +<marker id="name"/> + <tag><c>@name Mod</c></tag> <item> <p> @@ -155,6 +164,8 @@ Example:</p> </item> +<marker id="prefix"/> + <tag><c>@prefix Name</c></tag> <item> <p> @@ -178,6 +189,8 @@ Example:</p> </item> +<marker id="vendor"/> + <tag><c>@vendor Number Name</c></tag> <item> <p> @@ -198,6 +211,8 @@ Example:</p> </item> +<marker id="avp_vendor_id"/> + <tag><c>@avp_vendor_id Number</c></tag> <item> <p> @@ -218,6 +233,8 @@ Region-Set </item> +<marker id="inherits"/> + <tag><c>@inherits Mod</c></tag> <item> <p> @@ -241,7 +258,7 @@ is equivalent to using <c>@avp_vendor_id</c> with a copy of the dictionary's definitions but the former makes for easier reuse.</p> <p> -All dictionaries should typically inherit RFC3588 AVPs from +All dictionaries should typically inherit &the_rfc; AVPs from <c>diameter_gen_base_rfc3588</c>.</p> <p> @@ -253,6 +270,8 @@ Example:</p> </item> +<marker id="avp_types"/> + <tag><c>@avp_types</c></tag> <item> <p> @@ -263,7 +282,7 @@ The section consists of definitions of the form</p> <p> where Code is the integer AVP code, Type identifies an AVP Data Format -as defined in <seealso marker="#DATA_TYPES">DATA TYPES</seealso> below, +as defined in section &types; below, and Flags is a string of V, M and P characters indicating the flags to be set on an outgoing AVP or a single <c>'-'</c> (minus) character if none are to be set.</p> @@ -280,13 +299,13 @@ Requested-Information 353 Enumerated V <warning> <p> -The P flag has been deprecated by the Diameter Maintenance -and Extensions Working Group of the IETF and should be omitted -to conform to the current draft standard.</p> +The P flag has been deprecated by &the_rfc;.</p> </warning> </item> +<marker id="custom_types"/> + <tag><c>@custom_types Mod</c></tag> <item> <p> @@ -308,6 +327,8 @@ Framed-IP-Address </pre> </item> +<marker id="codecs"/> + <tag><c>@codecs Mod</c></tag> <item> <p> @@ -325,12 +346,14 @@ Framed-IP-Address </pre> </item> +<marker id="messages"/> + <tag><c>@messages</c></tag> <item> <p> Defines the messages of the application. The section content consists of definitions of the form specified in -section 3.2 of RFC 3588, "Command Code ABNF specification".</p> +section 3.2 of &the_rfc;, "Command Code Format Specification".</p> <!-- RFC 4740 RTR/RTA --> <pre> @@ -370,13 +393,15 @@ RTA ::= < Diameter Header: 287, PXY > </item> +<marker id="grouped"/> + <tag><c>@grouped</c></tag> <item> <p> Defines the contents of the AVPs of the application having type Grouped. The section content consists of definitions of the form specified in -section 4.4 of RFC 3588, "Grouped AVP Values".</p> +section 4.4 of &the_rfc;, "Grouped AVP Values".</p> <p> Example:</p> @@ -395,6 +420,8 @@ Specifying a Vendor-Id in the definition of a grouped AVP is equivalent to specifying it with <c>@avp_vendor_id</c>.</p> </item> +<marker id="enum"/> + <tag><c>@enum Name</c></tag> <item> <p> @@ -421,6 +448,8 @@ REMOVE_SIP_SERVER 3 </pre> </item> +<marker id="end"/> + <tag><c>@end</c></tag> <item> <p> @@ -447,8 +476,8 @@ The hrl generated from a dictionary specification defines records for the messages and grouped AVPs defined in <c>@messages</c> and <c>@grouped</c> sections. For each message or grouped AVP definition, a record is defined whose -name is the message or AVP name prefixed with any dictionary prefix -defined with <c>@prefix</c> and whose fields are the names of the AVPs +name is the message or AVP name, prefixed with any dictionary prefix +defined with <c>@prefix</c>, and whose fields are the names of the AVPs contained in the message or grouped AVP in the order specified in the definition in question. For example, the grouped AVP</p> @@ -476,7 +505,7 @@ type and number of times the AVP can occur. In particular, an AVP which is specified as occurring exactly once is encoded as a value of the AVP's type while an AVP with any other specification is encoded as a list of values of the AVP's type. -The AVP's type is as specified in the AVP definition, the RFC 3588 +The AVP's type is as specified in the AVP definition, the &the_rfc; types being described below.</p> <marker id="DATA_TYPES"/> @@ -489,7 +518,7 @@ types being described below.</p> <p> The data formats defined in sections 4.2 ("Basic AVP Data -Formats") and 4.3 ("Derived AVP Data Formats") of RFC 3588 are encoded +Formats") and 4.3 ("Derived AVP Data Formats") of &the_rfc; are encoded as values of the types defined here. Values are passed to &mod_call; in a request record when sending a request, returned in a resulting @@ -564,8 +593,8 @@ where <p> Additionally, values that can be encoded are -limited by way of their encoding as four octets as required by RFC -3588 with the required extension from RFC 2030. +limited by way of their encoding as four octets as required by +&the_rfc; with the required extension from RFC 2030. In particular, only values between <c>{{1968,1,20},{3,14,8}}</c> and <c>{{2104,2,26},{9,42,23}}</c> (both inclusive) can be encoded.</p> @@ -609,7 +638,7 @@ where On encode, fields port, transport and protocol default to 3868, sctp and diameter respectively. The grammar of an OctetString-valued DiameterURI() is as specified in -section 4.3 of RFC 3588. +section 4.3 of &the_rfc;. The record representation is used on decode.</p> <marker id="Enumerated"/> @@ -640,7 +669,7 @@ Values of these types are not currently parsed by diameter.</p> <title>SEE ALSO</title> <p> -&man_compile;, &man_main;, &man_app;</p> +&man_compile;, &man_main;, &man_app;, &man_codec;, &man_make;</p> </section> diff --git a/lib/diameter/doc/src/diameter_intro.xml b/lib/diameter/doc/src/diameter_intro.xml index bc2afbd453..fd578ccf45 100644 --- a/lib/diameter/doc/src/diameter_intro.xml +++ b/lib/diameter/doc/src/diameter_intro.xml @@ -1,5 +1,8 @@ <?xml version="1.0" encoding="latin1" ?> -<!DOCTYPE chapter SYSTEM "chapter.dtd"> +<!DOCTYPE chapter SYSTEM "chapter.dtd" [ + <!ENTITY % also SYSTEM "seealso.ent"> + %also; +]> <chapter> <header> @@ -35,7 +38,7 @@ under the License. <p> The diameter application is an implementation of the Diameter protocol -as defined by RFC 3588. +as defined by &the_rfc;. It supports arbitrary Diameter applications by way of a <em>dictionary</em> interface that allows messages and AVP's to be defined and input into diameter as configuration. @@ -86,9 +89,9 @@ dictionary module that provide encode/decode functionality for outgoing/incoming Diameter messages belonging to the application. A dictionary module is generated from a <seealso -marker="diameter_dict">specification file</seealso> using the <seealso +marker="diameter_dict">dictionary file</seealso> using the <seealso marker="diameterc">diameterc</seealso> utility. -Dictionaries for the RFC 3588 Diameter Common Messages, Base +Dictionaries for the &the_rfc; Diameter Common Messages, Base Accounting and Relay applications are provided with the diameter application.</p> diff --git a/lib/diameter/doc/src/diameter_make.xml b/lib/diameter/doc/src/diameter_make.xml new file mode 100644 index 0000000000..da6124310e --- /dev/null +++ b/lib/diameter/doc/src/diameter_make.xml @@ -0,0 +1,144 @@ +<?xml version="1.0" encoding="latin1" ?> +<!DOCTYPE erlref SYSTEM "erlref.dtd" [ + <!ENTITY filename + '<seealso marker="kernel:file#type-name">file:name()</seealso>'> + <!ENTITY dictionary + '<seealso marker="diameter_dict">dictionary file</seealso>'> + <!ENTITY % also SYSTEM "seealso.ent" > + <!ENTITY % here SYSTEM "seehere.ent" > + %also; + %here; +]> + +<erlref> +<header> +<copyright> +<year>2012</year> +<holder>Ericsson AB. All Rights Reserved.</holder> +</copyright> +<legalnotice> +The contents of this file are subject to the Erlang Public License, +Version 1.1, (the "License"); you may not use this file except in +compliance with the License. You should have received a copy of the +Erlang Public License along with this software. If not, it can be +retrieved online at http://www.erlang.org/. + +Software distributed under the License is distributed on an "AS IS" +basis, WITHOUT WARRANTY OF ANY KIND, either express or implied. See +the License for the specific language governing rights and limitations +under the License. + +</legalnotice> + +<title>diameter_make(3)</title> +<prepared>Anders Svensson</prepared> +<responsible></responsible> +<docno></docno> +<approved></approved> +<checked></checked> +<date></date> +<rev></rev> +<file>diameter_make.xml</file> +</header> + +<module>diameter_make</module> +<modulesummary>Diameter dictionary compilation.</modulesummary> + +<description> + +<p> +The function &codec; is used to compile a diameter +&dictionary; into Erlang source. +The resulting source implements the interface diameter required +to encode and decode the dictionary's messages and AVP's.</p> + +<p> +The utility &man_compile; provides an alternate compilation +interface.</p> + +</description> + +<!-- ===================================================================== --> + +<funcs> + +<func> +<name>codec(Path::string(), [Opt]) -> ok | {error, Reason}</name> +<fsummary>Compile a dictionary file into Erlang source.</fsummary> +<desc> + +<p> +Compile a single dictionary file to Erlang source. +<c>Opt</c> can have the following types.</p> + +<taglist> + +<tag><c>{include, Dir::string()}</c></tag> +<item> +<p> +Prepend the specified directory to the code path. +Use to point at beam files compiled from inherited dictionaries, +<c>&dict_inherits;</c> in a dictionary file creating a beam +dependency, not an erl/hrl dependency.</p> + +<p> +Multiple <c>include</c> options can be specified.</p> +</item> + +<tag><c>{outdir, Dir::string()}</c></tag> +<item> +<p> +Write generated source to the specified directory. +Defaults to the current working directory.</p> +</item> + +<tag><c>{name|prefix, string()}</c></tag> +<item> +<p> +Set <c>&dict_name;</c> or <c>&dict_prefix;</c> to the specified +string. +Overrides any setting in the file itself.</p> +</item> + +<tag><c>{inherits, Mod::string()}</c></tag> +<item> +<p> +Append &dict_inherits; of the specified module. +Specifying <c>"-"</c> has the effect of discarding clearing any +previous inherits, both in the dictionary file and on the options +list.</p> + +<p> +Multiple <c>inherits</c> options can be specified.</p> +</item> + +</taglist> + +</desc> +</func> + +</funcs> + +<!-- ===================================================================== --> + +<section> +<title>BUGS</title> + +<p> +All options are string-valued. +In particular, it is not currently possible to +an &dict_inherits; module as an atom() or a path as a &filename;</p> + +</section> + +<!-- ===================================================================== --> + +<section> +<title>SEE ALSO</title> + +<p> +&man_compile;, &man_dict;</p> + +</section> + +</erlref> diff --git a/lib/diameter/doc/src/diameter_sctp.xml b/lib/diameter/doc/src/diameter_sctp.xml index a023a9bc08..5e3fd5eaf1 100644 --- a/lib/diameter/doc/src/diameter_sctp.xml +++ b/lib/diameter/doc/src/diameter_sctp.xml @@ -1,5 +1,11 @@ <?xml version="1.0" encoding="latin1" ?> <!DOCTYPE erlref SYSTEM "erlref.dtd" [ + <!ENTITY gen_sctp '<seealso marker="kernel:gen_sctp">gen_sctp(3)</seealso>'> + <!ENTITY gen_sctp_open1 + '<seealso marker="kernel:gen_sctp#open-1">gen_sctp:open/1</seealso>'> + <!ENTITY ip_address + '<seealso marker="kernel:inet#type-ip_address">inet:ip_address()</seealso>'> + <!ENTITY inet '<seealso marker="kernel:inet">inet(3)</seealso>'> <!ENTITY % also SYSTEM "seealso.ent" > <!ENTITY % here SYSTEM "seehere.ent" > %also; @@ -43,8 +49,7 @@ under the License. <description> <p> -This module implements diameter transport over SCTP using <seealso -marker="kernel:gen_sctp">gen_sctp</seealso>. +This module implements diameter transport over SCTP using &gen_sctp;. It can be specified as the value of a transport_module option to &mod_add_transport; and implements the behaviour documented in @@ -65,9 +70,9 @@ and implements the behaviour documented in <v>Type = connect | accept</v> <v>Ref = &mod_transport_ref;</v> <v>Svc = #diameter_service{}</v> -<v>Opt = {raddr, <seealso marker="kernel:inet#type-ip_address">inet:ip_address()</seealso>} | {rport, integer()} | term()</v> +<v>Opt = {raddr, &ip_address;} | {rport, integer()} | term()</v> <v>Pid = pid()</v> -<v>LAddr = <seealso marker="kernel:inet#type-ip_address">inet:ip_address()</seealso></v> +<v>LAddr = &ip_address;</v> <v>Reason = term()</v> </type> <desc> @@ -84,8 +89,7 @@ unspecified. More than one <c>raddr</c> option can be specified, in which case the connecting transport in question attempts each in sequence until an association is established. -Remaining options are any accepted by <seealso -marker="kernel:gen_sctp#open-1">gen_sctp:open/1</seealso>, with the exception +Remaining options are any accepted by &gen_sctp_open1;, with the exception of options <c>mode</c>, <c>binary</c>, <c>list</c>, <c>active</c> and <c>sctp_events</c>. Note that options <c>ip</c> and <c>port</c> specify the local address @@ -102,14 +106,12 @@ connecting transport.</p> <warning> <p> An insufficiently large receive buffer may result in a peer having to -resend incoming messages: set the <seealso -marker="kernel:inet">inet(3)</seealso> option <c>recbuf</c> to increase +resend incoming messages: set the &inet; option <c>recbuf</c> to increase the buffer size.</p> <p> An insufficiently large send buffer may result in outgoing messages -being discarded: set the <seealso -marker="kernel:inet">inet(3)</seealso> option <c>sndbuf</c> to increase +being discarded: set the &inet; option <c>sndbuf</c> to increase the buffer size.</p> </warning> @@ -145,10 +147,7 @@ outbound streams.</p> <title>SEE ALSO</title> <p> -&man_main;, -&man_transport;, -<seealso marker="kernel:gen_sctp">gen_sctp(3)</seealso>, -<seealso marker="kernel:inet">inet(3)</seealso></p> +&man_main;, &man_transport;, &gen_sctp;, &inet;</p> </section> diff --git a/lib/diameter/doc/src/diameter_soc.xml b/lib/diameter/doc/src/diameter_soc.xml index 6b9ef9f756..16f6b9d5bb 100644 --- a/lib/diameter/doc/src/diameter_soc.xml +++ b/lib/diameter/doc/src/diameter_soc.xml @@ -1,11 +1,15 @@ <?xml version="1.0" encoding="latin1" ?> -<!DOCTYPE chapter SYSTEM "chapter.dtd"> +<!DOCTYPE chapter SYSTEM "chapter.dtd" [ + <!ENTITY % also SYSTEM "seealso.ent" > + %also; +]> <chapter> <header> <copyright> <year>2011</year> +<year>2012</year> <holder>Ericsson AB. All Rights Reserved.</holder> </copyright> @@ -41,38 +45,20 @@ Known points of questionable or non-compliance.</p> <!-- ===================================================================== --> <section> -<title>RFC 3588</title> +<title>&the_rfc;</title> <list> <item> <p> -The End-to-End Security framework (section 2.9) isn't implemented -since it is largely unspecified. -The document that was to describe it -(reference [AAACMS]) was abandoned in an uncompleted state several -years ago and the current draft RFC deprecates the framework, -including the P Flag in the AVP header.</p> -</item> - -<item> -<p> -There is no TLS support over SCTP. -RFC 3588 requires that a Diameter server support TLS but in -practise this seems to mean TLS over SCTP since there are limitations -with running over SCTP: see RFC 6083 (DTLS over SCTP), which is a -response to RFC 3436 (TLS over SCTP). -The current RFC 3588 draft acknowledges this by equating -TLS with TLS/TCP and DTLS/SCTP but we do not yet support DTLS.</p> +There is no support for DTLS over SCTP.</p> </item> <item> <p> There is no explicit support for peer discovery (section 5.2). It can possibly be implemented on top of diameter as is but this is -probably something that diameter should do. -The current draft deprecates portions of the original RFC's mechanisms -however.</p> +probably something that diameter should do.</p> </item> <item> diff --git a/lib/diameter/doc/src/diameter_tcp.xml b/lib/diameter/doc/src/diameter_tcp.xml index be8a938115..e3b8c733b7 100644 --- a/lib/diameter/doc/src/diameter_tcp.xml +++ b/lib/diameter/doc/src/diameter_tcp.xml @@ -1,5 +1,22 @@ <?xml version="1.0" encoding="latin1" ?> <!DOCTYPE erlref SYSTEM "erlref.dtd" [ + <!ENTITY gen_tcp_connect3 + '<seealso marker="kernel:gen_tcp#connect-3">gen_tcp:connect/3</seealso>'> + <!ENTITY gen_tcp_listen2 + '<seealso marker="kernel:gen_tcp#listen-2">gen_tcp:listen/2</seealso>'> + <!ENTITY ip_address + '<seealso marker="kernel:inet#type-ip_address">inet:ip_address()</seealso>'> + <!ENTITY ssl_connect2 + '<seealso marker="ssl:ssl#connect-2">ssl:connect/2</seealso>'> + <!ENTITY ssl_connect3 + '<seealso marker="ssl:ssl#connect-3">ssl:connect/3</seealso>'> + <!ENTITY ssl_accept2 + '<seealso marker="ssl:ssl#ssl_accept-2">ssl:ssl_accept/2</seealso>'> + <!ENTITY ssl_listen2 + '<seealso marker="ssl:ssl#listen-2">ssl:listen/2</seealso>'> + <!ENTITY gen_tcp '<seealso marker="kernel:gen_tcp">gen_tcp(3)</seealso>'> + <!ENTITY inet '<seealso marker="kernel:inet">inet(3)</seealso>'> + <!ENTITY ssl '<seealso marker="ssl:ssl">ssl(3)</seealso>'> <!ENTITY % also SYSTEM "seealso.ent" > <!ENTITY % here SYSTEM "seehere.ent" > %also; @@ -43,14 +60,13 @@ under the License. <description> <p> -This module implements diameter transport over TCP using <seealso -marker="kernel:gen_tcp">gen_tcp</seealso>. +This module implements diameter transport over TCP using &gen_tcp;. It can be specified as the value of a <c>transport_module</c> option to &mod_add_transport; and implements the behaviour documented in &man_transport;. TLS security is supported, both as an upgrade following -capabilities exchange as specified by RFC 3588 and +capabilities exchange as specified by &the_rfc; and at connection establishment as in the current draft standard.</p> <p> @@ -74,9 +90,9 @@ before configuring TLS capability on diameter transports.</p> <v>Svc = #diameter_service{}</v> <v>Opt = OwnOpt | SslOpt | TcpOpt</v> <v>Pid = pid()</v> -<v>LAddr = <seealso marker="kernel:inet#type-ip_address">inet:ip_address()</seealso></v> +<v>LAddr = &ip_address;</v> <v>Reason = term()</v> -<v>OwnOpt = {raddr, <seealso marker="kernel:inet#type-ip_address">inet:ip_address()</seealso>} +<v>OwnOpt = {raddr, &ip_address;} | {rport, integer()} | {port, integer()}</v> <v>SslOpt = {ssl_options, true | list()}</v> @@ -95,16 +111,12 @@ transport. Option <c>ssl_options</c> must be specified for a transport that should support TLS: a value of <c>true</c> results in a TLS handshake immediately upon connection establishment while -<c>list()</c> specifies options to be passed to <seealso -marker="ssl:ssl#connect-2">ssl:connect/2</seealso> or -<seealso marker="ssl:ssl#ssl_accept-2">ssl:ssl_accept/2</seealso> +<c>list()</c> specifies options to be passed to &ssl_connect2; or +&ssl_accept2; after capabilities exchange if TLS is negotiated. -Remaining options are any accepted by <seealso -marker="ssl:ssl#connect-3">ssl:connect/3</seealso> or <seealso -marker="kernel:gen_tcp#connect-3">gen_tcp:connect/3</seealso> for -a connecting transport, or <seealso -marker="ssl:ssl#listen-2">ssl:listen/2</seealso> or <seealso -marker="kernel:gen_tcp#listen-2">gen_tcp:listen/2</seealso> for +Remaining options are any accepted by &ssl_connect3; or +&gen_tcp_connect3; for +a connecting transport, or &ssl_listen2; or &gen_tcp_listen2; for a listening transport, depending on whether or not <c>{ssl_options, true}</c> has been specified. Options <c>binary</c>, @@ -150,11 +162,7 @@ The returned local address list has length one.</p> <title>SEE ALSO</title> <p> -&man_main;, -&man_transport;, -<seealso marker="kernel:gen_tcp">gen_tcp(3)</seealso>, -<seealso marker="kernel:inet">inet(3)</seealso>, -<seealso marker="ssl:ssl">ssl(3)</seealso></p> +&man_main;, &man_transport;, &gen_tcp;, &inet;, &ssl;</p> </section> diff --git a/lib/diameter/doc/src/diameter_transport.xml b/lib/diameter/doc/src/diameter_transport.xml index 0507af63a8..55b531155f 100644 --- a/lib/diameter/doc/src/diameter_transport.xml +++ b/lib/diameter/doc/src/diameter_transport.xml @@ -1,5 +1,8 @@ <?xml version="1.0" encoding="latin1" ?> <!DOCTYPE erlref SYSTEM "erlref.dtd" [ + <!ENTITY message '<seealso marker="#message">message()</seealso>'> + <!ENTITY ip_address + '<seealso marker="kernel:inet#type-ip_address">inet:ip_address()</seealso>'> <!ENTITY % also SYSTEM "seealso.ent" > <!ENTITY % here SYSTEM "seehere.ent" > %also; @@ -50,16 +53,49 @@ diameter starts a transport process and a message interface with which the transport process communicates with the process that starts it (aka its parent).</p> -<marker id="start"/> </description> <!-- ===================================================================== --> +<section> +<title>DATA TYPES</title> + +<taglist> + +<marker id="message"/> + +<tag><c>message() = binary() | &codec_packet;</c></tag> +<item> +<p> +A Diameter message as passed over the transport interface.</p> + +<p> +For an inbound message from a transport process, a &codec_packet; must +contain the received message in its <c>bin</c> field. +In the case of an inbound request, any value set in the +<c>transport_data</c> field will passed back to the transport module +in the corresponding answer message, unless the sender supplies +another value.</p> + +<p> +For an outbound message to a transport process, a &codec_packet; has a +value other than <c>undefined</c> in its <c>transport_data</c> field +and has the binary() to send in its <c>bin</c> field.</p> +</item> + +</taglist> + +</section> + +<!-- ===================================================================== --> + <funcs> <func> <name>Mod:start({Type, Ref}, Svc, Config) - -> {ok, Pid} | {ok, Pid, LAddrs} | {error, Reason}</name> + -> {ok, Pid} + | {ok, Pid, LAddrs} + | {error, Reason}</name> <fsummary>Start a transport process.</fsummary> <type> <v>Type = connect | accept</v> @@ -67,7 +103,7 @@ parent).</p> <v>Svc = #diameter_service{}</v> <v>Config = term()</v> <v>Pid = pid()</v> -<v>LAddrs = [<seealso marker="kernel:inet#type-ip_address">inet:ip_address()</seealso>]</v> +<v>LAddrs = [&ip_address;]</v> <v>Reason = term()</v> </type> <desc> @@ -79,8 +115,8 @@ A transport process maintains a connection with a single remote peer.</p> <p> <c>Type</c> indicates whether the transport process in question -is being started for a connecting (<c>connect</c>) or listening -(<c>accept</c>) transport. +is being started for a connecting (<c>Type=connect</c>) or listening +(<c>Type=accept</c>) transport. In the latter case, transport processes are started as required to accept connections from multiple peers.</p> @@ -90,13 +126,12 @@ that has lead to starting of a transport process.</p> <p> <c>Svc</c> contains the capabilities passed to &mod_start_service; and -&mod_add_transport;, -values passed to the latter overriding those passed to the former.</p> +&mod_add_transport;, values passed to the latter overriding those +passed to the former.</p> <p> <c>Config</c> is as passed in <c>transport_config</c> tuple in the -&mod_transport_opt; -list passed to &mod_add_transport;.</p> +&mod_transport_opt; list passed to &mod_add_transport;.</p> <p> The start function should use the <c>Host-IP-Address</c> list and/or @@ -114,13 +149,13 @@ it dies. It should not link to the parent. It should exit if its transport connection with its peer is lost.</p> -<marker id="MESSAGES"/> </desc> </func> </funcs> <!-- ===================================================================== --> +<marker id="MESSAGES"/> <section> <title>MESSAGES</title> @@ -130,19 +165,15 @@ All messages sent over the transport interface are of the form <c>{diameter, term()}</c>.</p> <p> -A transport process can expect the following messages from -diameter.</p> +A transport process can expect messages of the following types from +its parent.</p> <taglist> -<tag><c>{diameter, {send, Packet}}</c></tag> +<tag><c>{diameter, {send, &message;}}</c></tag> <item> <p> -An outbound Diameter message. -<c>Packet</c> can be either binary() (the message to be sent) -or a <c>#diameter_packet{}</c> record whose <c>transport_data</c> -field contains a value other than undefined and whose <c>bin</c> field -contains the binary to send.</p> +An outbound Diameter message.</p> </item> <tag><c>{diameter, {close, Pid}}</c></tag> @@ -185,7 +216,7 @@ TLS.</p> </taglist> <p> -A transport process should send the following messages +A transport process should send messages of the following types to its parent.</p> <taglist> @@ -208,19 +239,10 @@ Not sent if the transport process has <c>Type=accept</c>. endpoint to which the transport has connected.</p> </item> -<tag><c>{diameter, {recv, Packet}}</c></tag> +<tag><c>{diameter, {recv, &message;}}</c></tag> <item> <p> -An inbound Diameter message. -<c>Packet</c> can be either binary() (the received message) -or a <c>#diameter_packet{}</c> record -whose <c>bin</c> field contains the received binary(). -Any value (other than <c>undefined</c>) set in -the <c>transport_data</c> field will be passed back with a -corresponding answer message in the case that the inbound message is a -request unless the sender sets another value. -How <c>transport_data</c> is used/interpreted is up to the -transport module.</p> +An inbound Diameter message.</p> </item> <tag><c>{diameter, {tls, Ref}}</c></tag> diff --git a/lib/diameter/doc/src/files.mk b/lib/diameter/doc/src/files.mk index 89ec1031e6..00ced3d91e 100644 --- a/lib/diameter/doc/src/files.mk +++ b/lib/diameter/doc/src/files.mk @@ -26,6 +26,8 @@ XML_REF1_FILES = \ XML_REF3_FILES = \ diameter.xml \ diameter_app.xml \ + diameter_codec.xml \ + diameter_make.xml \ diameter_transport.xml \ diameter_tcp.xml \ diameter_sctp.xml diff --git a/lib/diameter/doc/src/notes.xml b/lib/diameter/doc/src/notes.xml index b89d84a4f6..d448b01eba 100644 --- a/lib/diameter/doc/src/notes.xml +++ b/lib/diameter/doc/src/notes.xml @@ -42,6 +42,163 @@ first.</p> <!-- ===================================================================== --> +<section><title>Diameter 1.3</title> + + <section><title>Fixed Bugs and Malfunctions</title> + <list> + <item> + <p> + Fix faulty handling of Origin-State-Id and faulty config + values.</p> + <p> + The former was expected in a list despite the + documentation requiring (correctly) an integer. A bare + value for a list-valued capability was not handled.</p> + <p> + Own Id: OTP-10440</p> + </item> + <item> + <p> + Fix timing of up/down events.</p> + <p> + Previously, a call to diameter:call/4 following a peer_up + callback might incorrectly return {error, no_connection}, + depending on timing. Both events now follow the + corresponding callbacks.</p> + <p> + Own Id: OTP-10459</p> + </item> + <item> + <p> + Make diameter:service_info/2 usable in peer_up, peer_down + and pick_peer callbacks.</p> + <p> + Except for in pick_peer when {call_mutates_state, false}, + it would previously hang indefinitely.</p> + <p> + Own Id: OTP-10460</p> + </item> + <item> + <p> + Verify that End-to-End and Hop-by-Hop Identifiers in an + incoming CEA/DPA match those sent in the corresponding + CER/DPR.</p> + <p> + The values were previously ignored. Answers whose + identifiers do not match are handled as unexpected.</p> + <p> + Own Id: OTP-10565</p> + </item> + <item> + <p> + Fix formatting problems in PDF documentation.</p> + <p> + In particular, text corresponding to links in HTML was + omitted in preformatted blocks. There are still issues + with indentation but this is not diameter-specific.</p> + <p> + Own Id: OTP-10583</p> + </item> + </list> + </section> + + + <section><title>Improvements and New Features</title> + <list> + <item> + <p> + Let prepare_request, prepare_retransmit and + handle_request callbacks return a function to be invoked + on outgoing messages after encode.</p> + <p> + This allows encoded messages to be logged for example.</p> + <p> + Own Id: OTP-10441</p> + </item> + <item> + <p> + Add service_opt() 'restrict_connections' to allow + multiple transport connections with the same peer.</p> + <p> + Own Id: OTP-10443</p> + </item> + <item> + <p> + Add service_opt() 'sequence' to allow the masking of a + constant onto the topmost bits of End-to-End and + Hop-by-Hop identifiers.</p> + <p> + This allows the same service on different nodes to use + distinct values in outgoing request messages.</p> + <p> + Own Id: OTP-10445</p> + </item> + <item> + <p> + Add diameter:service_info(PeerRef) to return the + transport_ref() and transport_opt() list of the + corresponding transport.</p> + <p> + This allows easy access to these from diameter_app + callbacks that only get peer_ref() as an argument.</p> + <p> + Own Id: OTP-10470</p> + </item> + <item> + <p> + Add reference pages diameter_codec(3) and + diameter_make(3).</p> + <p> + Own Id: OTP-10471</p> + </item> + <item> + <p> + Add events for service start and stop.</p> + <p> + Own Id: OTP-10492</p> + </item> + <item> + <p> + Add transport_opt() 'disconnect_cb' to make the sending + of DPR configurable.</p> + <p> + Whether or not DPR should be sent at application stop, + service stop or transport removal is determined by the + value returned by the callback, as is the + Disconnect-Cause and timeout if DPA is not received.</p> + <p> + Own Id: OTP-10493</p> + </item> + <item> + <p> + Add transport_opt() 'capx_timeout' for the timeout + associated with non-reception of CER/CEA.</p> + <p> + Own Id: OTP-10554</p> + </item> + <item> + <p> + Allow a handle_request callback to return a + #diameter_packet{}.</p> + <p> + This allows an answer to set transport_data and header + fields.</p> + <p> + Own Id: OTP-10566</p> + </item> + <item> + <p> + Update documentation for RFC 6733.</p> + <p> + RFC 3588 is now obsolete.</p> + <p> + Own Id: OTP-10568</p> + </item> + </list> + </section> + +</section> + <section><title>Diameter 1.2</title> <section><title>Fixed Bugs and Malfunctions</title> @@ -428,7 +585,7 @@ Known issues or limitations:</p> Some agent-related functionality is not entirely complete. In particular, support for proxy agents, that advertise specific Diameter applications but otherwise relay messages in much the same -way as relay agents (for which a &handle_request; +way as relay agents (for which a handle_request callback can return a <c>relay</c> tuple), will be completed in an upcoming release. There may also be more explicit support for redirect agents, although @@ -460,7 +617,7 @@ could likely be expanded upon.</p> <item> <p> -The function &service_info; +The function diameter:service_info/2 can be used to retrieve information about a started service (statistics, information about connected peers, etc) but this is not yet documented and both the input and output may change diff --git a/lib/diameter/doc/src/ref_man.xml b/lib/diameter/doc/src/ref_man.xml index 137ce79094..4b99fe716d 100644 --- a/lib/diameter/doc/src/ref_man.xml +++ b/lib/diameter/doc/src/ref_man.xml @@ -6,6 +6,7 @@ <header> <copyright> <year>2011</year> +<year>2012</year> <holder>Ericsson AB. All Rights Reserved.</holder> </copyright> <legalnotice> @@ -40,7 +41,9 @@ applications on top of the Diameter protocol. </p> <xi:include href="diameter.xml"/> <xi:include href="diameter_compile.xml"/> <xi:include href="diameter_app.xml"/> +<xi:include href="diameter_codec.xml"/> <xi:include href="diameter_dict.xml"/> +<xi:include href="diameter_make.xml"/> <xi:include href="diameter_transport.xml"/> <xi:include href="diameter_tcp.xml"/> <xi:include href="diameter_sctp.xml"/> diff --git a/lib/diameter/doc/src/seealso.ent b/lib/diameter/doc/src/seealso.ent index 6f67630220..4647c42f85 100644 --- a/lib/diameter/doc/src/seealso.ent +++ b/lib/diameter/doc/src/seealso.ent @@ -32,6 +32,8 @@ significant. --> +<!ENTITY the_rfc 'RFC 6733'> + <!-- diameter --> <!ENTITY mod_add_transport '<seealso marker="diameter#add_transport-2">diameter:add_transport/2</seealso>'> @@ -77,12 +79,21 @@ significant. <!ENTITY app_prepare_request '<seealso marker="diameter_app#Mod:prepare_request-3">prepare_request/3</seealso>'> <!ENTITY app_capabilities '<seealso marker="diameter_app#capabilities">diameter_app:capabilities()</seealso>'> -<!ENTITY app_message '<seealso marker="diameter_app#message">diameter_app:message()</seealso>'> -<!ENTITY app_packet '<seealso marker="diameter_app#packet">diameter_app:packet()</seealso>'> <!ENTITY app_peer '<seealso marker="diameter_app#peer">diameter_app:peer()</seealso>'> <!ENTITY app_peer_ref '<seealso marker="diameter_app#peer_ref">diameter_app:peer_ref()</seealso>'> <!ENTITY app_state '<seealso marker="diameter_app#state">diameter_app:state()</seealso>'> +<!-- diameter_codec --> + +<!ENTITY codec_encode '<seealso marker="diameter_codec#encode-2">diameter_codec:encode/2</seealso>'> +<!ENTITY codec_decode '<seealso marker="diameter_codec#decode-2">diameter_codec:decode/2</seealso>'> + +<!ENTITY codec_avp '<seealso marker="diameter_codec#avp">diameter_codec:avp()</seealso>'> +<!ENTITY codec_header '<seealso marker="diameter_codec#header">diameter_codec:header()</seealso>'> +<!ENTITY codec_dictionary '<seealso marker="diameter_codec#dictionary">diameter_codec:dictionary()</seealso>'> +<!ENTITY codec_message '<seealso marker="diameter_codec#message">diameter_codec:message()</seealso>'> +<!ENTITY codec_packet '<seealso marker="diameter_codec#packet">diameter_codec:packet()</seealso>'> + <!-- diameter_dict --> <!ENTITY dict_data_types '<seealso marker="diameter_dict#DATA_TYPES">diameter_dict(4)</seealso>'> @@ -95,6 +106,14 @@ significant. <!ENTITY dict_UTF8String '<seealso marker="diameter_dict#DATA_TYPES">UTF8String()</seealso>'> <!ENTITY dict_Unsigned32 '<seealso marker="diameter_dict#DATA_TYPES">Unsigned32()</seealso>'> +<!ENTITY dict_name '<seealso marker="diameter_dict#name">@name</seealso>'> +<!ENTITY dict_prefix '<seealso marker="diameter_dict#prefix">@prefix</seealso>'> +<!ENTITY dict_inherits '<seealso marker="diameter_dict#inherits">@inherits</seealso>'> + +<!-- diameter_make --> + +<!ENTITY make_codec '<seealso marker="diameter_make#codec-2">diameter_make:codec/2</seealso>'> + <!-- diameter_transport --> <!ENTITY transport_start @@ -105,8 +124,9 @@ significant. <!ENTITY man_compile '<seealso marker="diameterc">diameterc(1)</seealso>'> <!ENTITY man_main '<seealso marker="diameter">diameter(3)</seealso>'> <!ENTITY man_app '<seealso marker="diameter_app">diameter_app(3)</seealso>'> +<!ENTITY man_codec '<seealso marker="diameter_codec">diameter_codec(3)</seealso>'> <!ENTITY man_dict '<seealso marker="diameter_dict">diameter_dict(4)</seealso>'> -<!ENTITY man_transport - '<seealso marker="diameter_transport">diameter_transport(3)</seealso>'> +<!ENTITY man_make '<seealso marker="diameter_make">diameter_make(3)</seealso>'> +<!ENTITY man_transport '<seealso marker="diameter_transport">diameter_transport(3)</seealso>'> <!ENTITY man_sctp '<seealso marker="diameter_sctp">diameter_sctp(3)</seealso>'> <!ENTITY man_tcp '<seealso marker="diameter_tcp">diameter_tcp(3)</seealso>'> diff --git a/lib/diameter/doc/standard/draft-ietf-dime-capablities-update-07.txt b/lib/diameter/doc/standard/draft-ietf-dime-capablities-update-07.txt deleted file mode 100644 index bb7ec2d08c..0000000000 --- a/lib/diameter/doc/standard/draft-ietf-dime-capablities-update-07.txt +++ /dev/null @@ -1,392 +0,0 @@ -
-
-
-Network Working Group K. Jiao
-Internet-Draft Huawei
-Intended status: Standards Track G. Zorn
-Expires: April 27, 2011 Network Zen
- October 24, 2010
-
-
- The Diameter Capabilities Update Application
- draft-ietf-dime-capablities-update-07
-
-Abstract
-
- This document defines a new Diameter application and associated
- command codes. The Capabilities Update application is intended to
- allow the dynamic update of certain Diameter peer capabilities while
- the peer-to-peer connection is in the open state.
-
-Status of this Memo
-
- This Internet-Draft is submitted in full conformance with the
- provisions of BCP 78 and BCP 79.
-
- Internet-Drafts are working documents of the Internet Engineering
- Task Force (IETF). Note that other groups may also distribute
- working documents as Internet-Drafts. The list of current Internet-
- Drafts is at http://datatracker.ietf.org/drafts/current/.
-
- Internet-Drafts are draft documents valid for a maximum of six months
- and may be updated, replaced, or obsoleted by other documents at any
- time. It is inappropriate to use Internet-Drafts as reference
- material or to cite them other than as "work in progress."
-
- This Internet-Draft will expire on April 27, 2011.
-
-Copyright Notice
-
- Copyright (c) 2010 IETF Trust and the persons identified as the
- document authors. All rights reserved.
-
- This document is subject to BCP 78 and the IETF Trust's Legal
- Provisions Relating to IETF Documents
- (http://trustee.ietf.org/license-info) in effect on the date of
- publication of this document. Please review these documents
- carefully, as they describe your rights and restrictions with respect
- to this document. Code Components extracted from this document must
- include Simplified BSD License text as described in Section 4.e of
- the Trust Legal Provisions and are provided without warranty as
- described in the Simplified BSD License.
-
-
-
-Jiao & Zorn Expires April 27, 2011 [Page 1]
-
-Internet-Draft Diameter Capabilities Update October 2010
-
-
-Table of Contents
-
- 1. Introduction . . . . . . . . . . . . . . . . . . . . . . . . . 3
- 2. Specification of Requirements . . . . . . . . . . . . . . . . . 3
- 3. Diameter Protocol Considerations . . . . . . . . . . . . . . . 3
- 4. Capabilities Update . . . . . . . . . . . . . . . . . . . . . . 3
- 4.1. Command-Code Values . . . . . . . . . . . . . . . . . . . . 4
- 4.1.1. Capabilities-Update-Request . . . . . . . . . . . . . . 5
- 4.1.2. Capabilities-Update-Answer . . . . . . . . . . . . . . 5
- 5. Security Considerations . . . . . . . . . . . . . . . . . . . . 6
- 6. IANA Considerations . . . . . . . . . . . . . . . . . . . . . . 6
- 6.1. Application Identifier . . . . . . . . . . . . . . . . . . 6
- 6.2. Command Codes . . . . . . . . . . . . . . . . . . . . . . . 6
- 7. Contributors . . . . . . . . . . . . . . . . . . . . . . . . . 6
- 8. Acknowledgements . . . . . . . . . . . . . . . . . . . . . . . 6
- 9. References . . . . . . . . . . . . . . . . . . . . . . . . . . 6
- 9.1. Normative References . . . . . . . . . . . . . . . . . . . 6
- 9.2. Informative References . . . . . . . . . . . . . . . . . . 7
- Authors' Addresses . . . . . . . . . . . . . . . . . . . . . . . . 7
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-Jiao & Zorn Expires April 27, 2011 [Page 2]
-
-Internet-Draft Diameter Capabilities Update October 2010
-
-
-1. Introduction
-
- Capabilities exchange is an important component of the Diameter Base
- Protocol [I-D.ietf-dime-rfc3588bis], allowing peers to exchange
- identities and Diameter capabilities (protocol version number,
- supported Diameter applications, security mechanisms, etc.). As
- defined in RFC 3588, however, the capabilities exchange process takes
- place only once, at the inception of a transport connection between a
- given pair of peers. Therefore, if a peer's capabilities change (due
- to software update, for example), the existing connection(s) must be
- torn down (along with all of the associated user sessions) and
- restarted before the modified capabilities can be advertised.
-
- This document defines a new Diameter application intended to allow
- the dynamic update of a subset of Diameter peer capabilities over an
- existing connection. Because the Capabilities Update application
- specified herein operates over an existing transport connection,
- modification of certain capabilities is prohibited. Specifically,
- modifying the security mechanism in use is not allowed; if the
- security method used between a pair of peers is changed the affected
- connection MUST be restarted.
-
-
-2. Specification of Requirements
-
- The key words "MUST", "MUST NOT", "REQUIRED", "SHALL", "SHALL NOT",
- "SHOULD", "SHOULD NOT", "RECOMMENDED", "MAY", and "OPTIONAL" in this
- document are to be interpreted as described in RFC 2119 [RFC2119].
-
-
-3. Diameter Protocol Considerations
-
- This section details the relationship of the Diameter Capabilities
- Update application to the Diameter Base Protocol.
-
- This document specifies Diameter Application-ID <TBD1>. Diameter
- nodes conforming to this specification MUST advertise support by
- including the value <TBD1> in the Auth-Application-Id of the
- Capabilities-Exchange-Request (CER) and Capabilities-Exchange-Answer
- (CEA) commands [I-D.ietf-dime-rfc3588bis].
-
-
-4. Capabilities Update
-
- When the capabilities of a Diameter node conforming to this
- specification change, it MUST notify all of the nodes with which it
- has an open transport connection and which have also advertised
- support for the Capabilities Update application using the
-
-
-
-Jiao & Zorn Expires April 27, 2011 [Page 3]
-
-Internet-Draft Diameter Capabilities Update October 2010
-
-
- Capabilities-Update-Request (CUR) message (Section 4.1.1). This
- message allows the update of a peer's capabilities (supported
- Diameter applications, etc.).
-
- A Diameter node only issues a given command to those peers that have
- advertised support for the Diameter application that defines the
- command; a Diameter node must cache the supported applications in
- order to ensure that unrecognized commands and/or Attribute-Value
- Pairs (AVPs) are not unnecessarily sent to a peer.
-
- The receiver of the CUR MUST determine common applications by
- computing the intersection of its own set of supported Application Id
- against all of the application identifier AVPs (Auth-Application-Id,
- Acct-Application-Id and Vendor-Specific- Application-Id) present in
- the CUR. The value of the Vendor-Id AVP in the Vendor-Specific-
- Application-Id MUST NOT be used during computation.
-
- If the receiver of a CUR does not have any applications in common
- with the sender then it MUST return a Capabilities-Update-Answer
- (CUA) (Section 4.1.2) with the Result-Code AVP set to
- DIAMETER_NO_COMMON_APPLICATION [I-D.ietf-dime-rfc3588bis], and SHOULD
- disconnect the transport layer connection; however, if active
- sessions are using the connection, peers MAY delay disconnection
- until the sessions can be redirected or gracefully terminated. Note
- that receiving a CUA from a peer advertising itself as a Relay (see
- [I-D.ietf-dime-rfc3588bis], Section 2.4) MUST be interpreted as
- having common applications with the peer.
-
- As for CER/CEA messages, the CUR and CUA messages MUST NOT be
- proxied, redirected or relayed.
-
- Even though the CUR/CUA messages cannot be proxied, it is still
- possible for an upstream agent to receive a message for which there
- are no peers available to handle the application that corresponds to
- the Command-Code. This could happen if, for example, the peers are
- too busy or down. In such instances, the 'E' bit MUST be set in the
- answer message with the Result-Code AVP set to
- DIAMETER_UNABLE_TO_DELIVER to inform the downstream peer to take
- action (e.g., re-routing requests to an alternate peer).
-
-4.1. Command-Code Values
-
- This section defines Command-Code [I-D.ietf-dime-rfc3588bis] values
- that MUST be supported by all Diameter implementations conforming to
- this specification. The following Command Codes are defined (using
- modified ABNF [I-D.ietf-dime-rfc3588bis]) in this document:
- Capabilities-Update-Request (CUR, Section 4.1.1) and Capabilities-
- Update-Answer (CUA, Section 4.1.2).
-
-
-
-Jiao & Zorn Expires April 27, 2011 [Page 4]
-
-Internet-Draft Diameter Capabilities Update October 2010
-
-
-4.1.1. Capabilities-Update-Request
-
- The Capabilities-Update-Request (CUR), indicated by the Command-Code
- set to <CUCC> and the Command Flags' 'R' bit set, is sent to update
- local capabilities. Upon detection of a transport failure, this
- message MUST NOT be sent to an alternate peer.
-
- When Diameter is run over the Stream Control Transmission Protocol
- [RFC4960], which allows connections to span multiple interfaces and
- multiple IP addresses, the Capabilities-Update-Request message MUST
- contain one Host-IP-Address AVP for each potential IP address that
- may be locally used when transmitting Diameter messages.
-
- Message Format
-
- <CUR> ::= < Diameter Header: <CUCC>, REQ >
- { Origin-Host }
- { Origin-Realm }
- 1* { Host-IP-Address }
- { Vendor-Id }
- { Product-Name }
- [ Origin-State-Id ]
- * [ Supported-Vendor-Id ]
- * [ Auth-Application-Id ]
- * [ Acct-Application-Id ]
- * [ Vendor-Specific-Application-Id ]
- [ Firmware-Revision ]
- * [ AVP ]
-
-4.1.2. Capabilities-Update-Answer
-
- The Capabilities-Update-Answer, indicated by the Command-Code set to
- <CUCC> and the Command Flags' 'R' bit cleared, is sent in response to
- a CUR message.
-
- Message Format
-
- <CUA> ::= < Diameter Header: <CUCC> >
- { Origin-Host }
- { Origin-Realm }
- { Result-Code }
- [ Error-Message ]
- * [ AVP ]
-
-
-
-
-
-
-
-
-Jiao & Zorn Expires April 27, 2011 [Page 5]
-
-Internet-Draft Diameter Capabilities Update October 2010
-
-
-5. Security Considerations
-
- The security considerations applicable to the Diameter Base Protocol
- [I-D.ietf-dime-rfc3588bis] are also applicable to this document.
-
-
-6. IANA Considerations
-
- This section explains the criteria to be used by the IANA for
- assignment of numbers within namespaces used within this document.
-
-6.1. Application Identifier
-
- This specification assigns the value <TBD1> from the Application
- Identifiers namespace [I-D.ietf-dime-rfc3588bis]. See Section 3 for
- the assignment of the namespace in this specification.
-
-6.2. Command Codes
-
- This specification assigns the value <CUCC> from the Command Codes
- namespace [I-D.ietf-dime-rfc3588bis]. See Section 4.1 for the
- assignment of the namespace in this specification.
-
-
-7. Contributors
-
- This document is based upon work done by Tina Tsou.
-
-
-8. Acknowledgements
-
- Thanks to Sebastien Decugis, Niklas Neumann, Subash Comerica, Lionel
- Morand, Dan Romascanu, Dan Harkins and Ravi for helpful review and
- discussion.
-
-
-9. References
-
-9.1. Normative References
-
- [I-D.ietf-dime-rfc3588bis]
- Fajardo, V., Arkko, J., Loughney, J., and G. Zorn,
- "Diameter Base Protocol", draft-ietf-dime-rfc3588bis-25
- (work in progress), September 2010.
-
- [RFC2119] Bradner, S., "Key words for use in RFCs to Indicate
- Requirement Levels", BCP 14, RFC 2119, March 1997.
-
-
-
-
-Jiao & Zorn Expires April 27, 2011 [Page 6]
-
-Internet-Draft Diameter Capabilities Update October 2010
-
-
-9.2. Informative References
-
- [RFC4960] Stewart, R., "Stream Control Transmission Protocol",
- RFC 4960, September 2007.
-
-
-Authors' Addresses
-
- Jiao Kang
- Huawei Technologies
- Section B1, Huawei Industrial Base
- Bantian, Longgang District
- Shenzhen 518129
- P.R. China
-
- Phone: +86 755 2878-6690
- Email: [email protected]
-
-
- Glen Zorn
- Network Zen
- 227/358 Thanon Sanphawut
- Bang Na, Bangkok 10260
- Thailand
-
- Phone: +66 (0) 87-040-4617
- Email: [email protected]
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-Jiao & Zorn Expires April 27, 2011 [Page 7]
-
diff --git a/lib/diameter/doc/standard/draft-ietf-dime-rfc3588bis-26.txt b/lib/diameter/doc/standard/rfc6733.txt index 87b9562f93..2f5a477347 100644 --- a/lib/diameter/doc/standard/draft-ietf-dime-rfc3588bis-26.txt +++ b/lib/diameter/doc/standard/rfc6733.txt @@ -1,62 +1,72 @@ -DIME V. Fajardo, Ed. -Internet-Draft Telcordia Technologies -Obsoletes: 3588 (if approved) J. Arkko -Intended status: Standards Track Ericsson Research -Expires: July 24, 2011 J. Loughney + + + +Internet Engineering Task Force (IETF) V. Fajardo, Ed. +Request for Comments: 6733 Telcordia Technologies +Obsoletes: 3588, 5719 J. Arkko +Category: Standards Track Ericsson Research +ISSN: 2070-1721 J. Loughney Nokia Research Center - G. Zorn + G. Zorn, Ed. Network Zen - January 20, 2011 + October 2012 Diameter Base Protocol - draft-ietf-dime-rfc3588bis-26.txt Abstract The Diameter base protocol is intended to provide an Authentication, - Authorization and Accounting (AAA) framework for applications such as - network access or IP mobility in both local and roaming situations. - This document specifies the message format, transport, error - reporting, accounting and security services used by all Diameter - applications. The Diameter base protocol as defined in this document - must be supported by all Diameter implementations. + Authorization, and Accounting (AAA) framework for applications such + as network access or IP mobility in both local and roaming + situations. This document specifies the message format, transport, + error reporting, accounting, and security services used by all + Diameter applications. The Diameter base protocol as defined in this + document obsoletes RFC 3588 and RFC 5719, and it must be supported by + all new Diameter implementations. -Status of this Memo +Status of This Memo - This Internet-Draft is submitted in full conformance with the - provisions of BCP 78 and BCP 79. + This is an Internet Standards Track document. - Internet-Drafts are working documents of the Internet Engineering - Task Force (IETF). Note that other groups may also distribute - working documents as Internet-Drafts. The list of current Internet- - Drafts is at http://datatracker.ietf.org/drafts/current/. + This document is a product of the Internet Engineering Task Force + (IETF). It represents the consensus of the IETF community. It has + received public review and has been approved for publication by the + Internet Engineering Steering Group (IESG). Further information on + Internet Standards is available in Section 2 of RFC 5741. + + Information about the current status of this document, any errata, + and how to provide feedback on it may be obtained at + http://www.rfc-editor.org/info/rfc6733. - Internet-Drafts are draft documents valid for a maximum of six months - and may be updated, replaced, or obsoleted by other documents at any - time. It is inappropriate to use Internet-Drafts as reference - material or to cite them other than as "work in progress." - This Internet-Draft will expire on July 24, 2011. -Copyright Notice - Copyright (c) 2011 IETF Trust and the persons identified as the - document authors. All rights reserved. - This document is subject to BCP 78 and the IETF Trust's Legal - Provisions Relating to IETF Documents -Fajardo, et al. Expires July 24, 2011 [Page 1] + + + + + + +Fajardo, et al. Standards Track [Page 1] -Internet-Draft Diameter Base Protocol January 2011 +RFC 6733 Diameter Base Protocol October 2012 +Copyright Notice + + Copyright (c) 2012 IETF Trust and the persons identified as the + document authors. All rights reserved. + + This document is subject to BCP 78 and the IETF Trust's Legal + Provisions Relating to IETF Documents (http://trustee.ietf.org/license-info) in effect on the date of publication of this document. Please review these documents carefully, as they describe your rights and restrictions with respect @@ -65,256 +75,303 @@ Internet-Draft Diameter Base Protocol January 2011 the Trust Legal Provisions and are provided without warranty as described in the Simplified BSD License. + This document may contain material from IETF Documents or IETF + Contributions published or made publicly available before November + 10, 2008. The person(s) controlling the copyright in some of this + material may not have granted the IETF Trust the right to allow + modifications of such material outside the IETF Standards Process. + Without obtaining an adequate license from the person(s) controlling + the copyright in such materials, this document may not be modified + outside the IETF Standards Process, and derivative works of it may + not be created outside the IETF Standards Process, except to format + it for publication as an RFC or to translate it into languages other + than English. Table of Contents - 1. Introduction . . . . . . . . . . . . . . . . . . . . . . . . 6 - 1.1. Diameter Protocol . . . . . . . . . . . . . . . . . . . . 8 - 1.1.1. Description of the Document Set . . . . . . . . . . . 9 - 1.1.2. Conventions Used in This Document . . . . . . . . . . 10 - 1.1.3. Changes from RFC3588 . . . . . . . . . . . . . . . . 10 - 1.2. Terminology . . . . . . . . . . . . . . . . . . . . . . . 12 - 1.3. Approach to Extensibility . . . . . . . . . . . . . . . . 17 - 1.3.1. Defining New AVP Values . . . . . . . . . . . . . . . 18 - 1.3.2. Creating New AVPs . . . . . . . . . . . . . . . . . . 18 - 1.3.3. Creating New Commands . . . . . . . . . . . . . . . . 18 - 1.3.4. Creating New Diameter Applications . . . . . . . . . 19 - 2. Protocol Overview . . . . . . . . . . . . . . . . . . . . . . 21 - 2.1. Transport . . . . . . . . . . . . . . . . . . . . . . . . 22 - 2.1.1. SCTP Guidelines . . . . . . . . . . . . . . . . . . . 23 - 2.2. Securing Diameter Messages . . . . . . . . . . . . . . . 24 - 2.3. Diameter Application Compliance . . . . . . . . . . . . . 24 - 2.4. Application Identifiers . . . . . . . . . . . . . . . . . 24 - 2.5. Connections vs. Sessions . . . . . . . . . . . . . . . . 25 - 2.6. Peer Table . . . . . . . . . . . . . . . . . . . . . . . 26 - 2.7. Routing Table . . . . . . . . . . . . . . . . . . . . . . 27 - 2.8. Role of Diameter Agents . . . . . . . . . . . . . . . . . 28 - 2.8.1. Relay Agents . . . . . . . . . . . . . . . . . . . . 29 - 2.8.2. Proxy Agents . . . . . . . . . . . . . . . . . . . . 30 - 2.8.3. Redirect Agents . . . . . . . . . . . . . . . . . . . 31 - 2.8.4. Translation Agents . . . . . . . . . . . . . . . . . 32 - 2.9. Diameter Path Authorization . . . . . . . . . . . . . . . 33 - 3. Diameter Header . . . . . . . . . . . . . . . . . . . . . . . 35 - 3.1. Command Codes . . . . . . . . . . . . . . . . . . . . . . 38 - 3.2. Command Code ABNF specification . . . . . . . . . . . . . 38 - 3.3. Diameter Command Naming Conventions . . . . . . . . . . . 41 - 4. Diameter AVPs . . . . . . . . . . . . . . . . . . . . . . . . 42 - 4.1. AVP Header . . . . . . . . . . . . . . . . . . . . . . . 42 - 4.1.1. Optional Header Elements . . . . . . . . . . . . . . 43 - 4.2. Basic AVP Data Formats . . . . . . . . . . . . . . . . . 44 - 4.3. Derived AVP Data Formats . . . . . . . . . . . . . . . . 45 - 4.3.1. Common Derived AVPs . . . . . . . . . . . . . . . . . 45 - 4.4. Grouped AVP Values . . . . . . . . . . . . . . . . . . . 52 - - - -Fajardo, et al. Expires July 24, 2011 [Page 2] - -Internet-Draft Diameter Base Protocol January 2011 - - - 4.4.1. Example AVP with a Grouped Data type . . . . . . . . 53 - 4.5. Diameter Base Protocol AVPs . . . . . . . . . . . . . . . 56 - 5. Diameter Peers . . . . . . . . . . . . . . . . . . . . . . . 59 - 5.1. Peer Connections . . . . . . . . . . . . . . . . . . . . 59 - 5.2. Diameter Peer Discovery . . . . . . . . . . . . . . . . . 60 - 5.3. Capabilities Exchange . . . . . . . . . . . . . . . . . . 61 - 5.3.1. Capabilities-Exchange-Request . . . . . . . . . . . . 63 - 5.3.2. Capabilities-Exchange-Answer . . . . . . . . . . . . 63 - 5.3.3. Vendor-Id AVP . . . . . . . . . . . . . . . . . . . . 64 - 5.3.4. Firmware-Revision AVP . . . . . . . . . . . . . . . . 64 - 5.3.5. Host-IP-Address AVP . . . . . . . . . . . . . . . . . 64 - 5.3.6. Supported-Vendor-Id AVP . . . . . . . . . . . . . . . 65 - 5.3.7. Product-Name AVP . . . . . . . . . . . . . . . . . . 65 - 5.4. Disconnecting Peer connections . . . . . . . . . . . . . 65 - 5.4.1. Disconnect-Peer-Request . . . . . . . . . . . . . . . 66 - 5.4.2. Disconnect-Peer-Answer . . . . . . . . . . . . . . . 66 - 5.4.3. Disconnect-Cause AVP . . . . . . . . . . . . . . . . 66 - 5.5. Transport Failure Detection . . . . . . . . . . . . . . . 67 - 5.5.1. Device-Watchdog-Request . . . . . . . . . . . . . . . 67 - 5.5.2. Device-Watchdog-Answer . . . . . . . . . . . . . . . 67 - 5.5.3. Transport Failure Algorithm . . . . . . . . . . . . . 68 - 5.5.4. Failover and Failback Procedures . . . . . . . . . . 68 - 5.6. Peer State Machine . . . . . . . . . . . . . . . . . . . 69 - 5.6.1. Incoming connections . . . . . . . . . . . . . . . . 71 - 5.6.2. Events . . . . . . . . . . . . . . . . . . . . . . . 72 - 5.6.3. Actions . . . . . . . . . . . . . . . . . . . . . . . 73 - 5.6.4. The Election Process . . . . . . . . . . . . . . . . 75 - 6. Diameter message processing . . . . . . . . . . . . . . . . . 76 - 6.1. Diameter Request Routing Overview . . . . . . . . . . . . 76 - 6.1.1. Originating a Request . . . . . . . . . . . . . . . . 77 - 6.1.2. Sending a Request . . . . . . . . . . . . . . . . . . 77 - 6.1.3. Receiving Requests . . . . . . . . . . . . . . . . . 78 - 6.1.4. Processing Local Requests . . . . . . . . . . . . . . 78 - 6.1.5. Request Forwarding . . . . . . . . . . . . . . . . . 78 - 6.1.6. Request Routing . . . . . . . . . . . . . . . . . . . 78 - 6.1.7. Predictive Loop Avoidance . . . . . . . . . . . . . . 79 - 6.1.8. Redirecting Requests . . . . . . . . . . . . . . . . 79 - 6.1.9. Relaying and Proxying Requests . . . . . . . . . . . 80 - 6.2. Diameter Answer Processing . . . . . . . . . . . . . . . 82 - 6.2.1. Processing received Answers . . . . . . . . . . . . . 82 - 6.2.2. Relaying and Proxying Answers . . . . . . . . . . . . 82 - 6.3. Origin-Host AVP . . . . . . . . . . . . . . . . . . . . . 83 - 6.4. Origin-Realm AVP . . . . . . . . . . . . . . . . . . . . 83 - 6.5. Destination-Host AVP . . . . . . . . . . . . . . . . . . 83 - 6.6. Destination-Realm AVP . . . . . . . . . . . . . . . . . . 84 - 6.7. Routing AVPs . . . . . . . . . . . . . . . . . . . . . . 84 - 6.7.1. Route-Record AVP . . . . . . . . . . . . . . . . . . 84 - 6.7.2. Proxy-Info AVP . . . . . . . . . . . . . . . . . . . 84 - - - -Fajardo, et al. Expires July 24, 2011 [Page 3] - -Internet-Draft Diameter Base Protocol January 2011 - - - 6.7.3. Proxy-Host AVP . . . . . . . . . . . . . . . . . . . 85 - 6.7.4. Proxy-State AVP . . . . . . . . . . . . . . . . . . . 85 - 6.8. Auth-Application-Id AVP . . . . . . . . . . . . . . . . . 85 - 6.9. Acct-Application-Id AVP . . . . . . . . . . . . . . . . . 85 - 6.10. Inband-Security-Id AVP . . . . . . . . . . . . . . . . . 85 - 6.11. Vendor-Specific-Application-Id AVP . . . . . . . . . . . 86 - 6.12. Redirect-Host AVP . . . . . . . . . . . . . . . . . . . . 87 - 6.13. Redirect-Host-Usage AVP . . . . . . . . . . . . . . . . . 87 - 6.14. Redirect-Max-Cache-Time AVP . . . . . . . . . . . . . . . 88 - 7. Error Handling . . . . . . . . . . . . . . . . . . . . . . . 90 - 7.1. Result-Code AVP . . . . . . . . . . . . . . . . . . . . . 92 - 7.1.1. Informational . . . . . . . . . . . . . . . . . . . . 92 - 7.1.2. Success . . . . . . . . . . . . . . . . . . . . . . . 93 - 7.1.3. Protocol Errors . . . . . . . . . . . . . . . . . . . 93 - 7.1.4. Transient Failures . . . . . . . . . . . . . . . . . 94 - 7.1.5. Permanent Failures . . . . . . . . . . . . . . . . . 95 - 7.2. Error Bit . . . . . . . . . . . . . . . . . . . . . . . . 98 - 7.3. Error-Message AVP . . . . . . . . . . . . . . . . . . . . 99 - 7.4. Error-Reporting-Host AVP . . . . . . . . . . . . . . . . 99 - 7.5. Failed-AVP AVP . . . . . . . . . . . . . . . . . . . . . 99 - 7.6. Experimental-Result AVP . . . . . . . . . . . . . . . . . 100 - 7.7. Experimental-Result-Code AVP . . . . . . . . . . . . . . 100 - 8. Diameter User Sessions . . . . . . . . . . . . . . . . . . . 101 - 8.1. Authorization Session State Machine . . . . . . . . . . . 102 - 8.2. Accounting Session State Machine . . . . . . . . . . . . 107 - 8.3. Server-Initiated Re-Auth . . . . . . . . . . . . . . . . 112 - 8.3.1. Re-Auth-Request . . . . . . . . . . . . . . . . . . . 112 - 8.3.2. Re-Auth-Answer . . . . . . . . . . . . . . . . . . . 113 - 8.4. Session Termination . . . . . . . . . . . . . . . . . . . 114 - 8.4.1. Session-Termination-Request . . . . . . . . . . . . . 115 - 8.4.2. Session-Termination-Answer . . . . . . . . . . . . . 115 - 8.5. Aborting a Session . . . . . . . . . . . . . . . . . . . 116 - 8.5.1. Abort-Session-Request . . . . . . . . . . . . . . . . 116 - 8.5.2. Abort-Session-Answer . . . . . . . . . . . . . . . . 117 - 8.6. Inferring Session Termination from Origin-State-Id . . . 118 - 8.7. Auth-Request-Type AVP . . . . . . . . . . . . . . . . . . 118 - 8.8. Session-Id AVP . . . . . . . . . . . . . . . . . . . . . 119 - 8.9. Authorization-Lifetime AVP . . . . . . . . . . . . . . . 120 - 8.10. Auth-Grace-Period AVP . . . . . . . . . . . . . . . . . . 121 - 8.11. Auth-Session-State AVP . . . . . . . . . . . . . . . . . 121 - 8.12. Re-Auth-Request-Type AVP . . . . . . . . . . . . . . . . 121 - 8.13. Session-Timeout AVP . . . . . . . . . . . . . . . . . . . 122 - 8.14. User-Name AVP . . . . . . . . . . . . . . . . . . . . . . 122 - 8.15. Termination-Cause AVP . . . . . . . . . . . . . . . . . . 122 - 8.16. Origin-State-Id AVP . . . . . . . . . . . . . . . . . . . 124 - 8.17. Session-Binding AVP . . . . . . . . . . . . . . . . . . . 124 - 8.18. Session-Server-Failover AVP . . . . . . . . . . . . . . . 125 - 8.19. Multi-Round-Time-Out AVP . . . . . . . . . . . . . . . . 126 - - - -Fajardo, et al. Expires July 24, 2011 [Page 4] - -Internet-Draft Diameter Base Protocol January 2011 - - - 8.20. Class AVP . . . . . . . . . . . . . . . . . . . . . . . . 126 - 8.21. Event-Timestamp AVP . . . . . . . . . . . . . . . . . . . 126 - 9. Accounting . . . . . . . . . . . . . . . . . . . . . . . . . 127 - 9.1. Server Directed Model . . . . . . . . . . . . . . . . . . 127 - 9.2. Protocol Messages . . . . . . . . . . . . . . . . . . . . 128 - 9.3. Accounting Application Extension and Requirements . . . . 128 - 9.4. Fault Resilience . . . . . . . . . . . . . . . . . . . . 129 - 9.5. Accounting Records . . . . . . . . . . . . . . . . . . . 129 - 9.6. Correlation of Accounting Records . . . . . . . . . . . . 130 - 9.7. Accounting Command-Codes . . . . . . . . . . . . . . . . 131 - 9.7.1. Accounting-Request . . . . . . . . . . . . . . . . . 131 - 9.7.2. Accounting-Answer . . . . . . . . . . . . . . . . . . 132 - 9.8. Accounting AVPs . . . . . . . . . . . . . . . . . . . . . 133 - 9.8.1. Accounting-Record-Type AVP . . . . . . . . . . . . . 133 - 9.8.2. Acct-Interim-Interval AVP . . . . . . . . . . . . . . 134 - 9.8.3. Accounting-Record-Number AVP . . . . . . . . . . . . 135 - 9.8.4. Acct-Session-Id AVP . . . . . . . . . . . . . . . . . 135 - 9.8.5. Acct-Multi-Session-Id AVP . . . . . . . . . . . . . . 135 - 9.8.6. Accounting-Sub-Session-Id AVP . . . . . . . . . . . . 135 - 9.8.7. Accounting-Realtime-Required AVP . . . . . . . . . . 136 - 10. AVP Occurrence Table . . . . . . . . . . . . . . . . . . . . 137 - 10.1. Base Protocol Command AVP Table . . . . . . . . . . . . . 137 - 10.2. Accounting AVP Table . . . . . . . . . . . . . . . . . . 138 - 11. IANA Considerations . . . . . . . . . . . . . . . . . . . . . 140 - 11.1. Changes to AVP Header Allocation . . . . . . . . . . . . 140 - 11.2. Diameter Header . . . . . . . . . . . . . . . . . . . . . 140 - 11.3. AVP Values . . . . . . . . . . . . . . . . . . . . . . . 140 - 11.3.1. Experimental-Result-Code AVP . . . . . . . . . . . . 140 - 11.4. Diameter TCP, SCTP, TLS/TCP and DTLS/SCTP Port Numbers . 141 - 11.5. S-NAPTR Parameters . . . . . . . . . . . . . . . . . . . 141 - 12. Diameter protocol related configurable parameters . . . . . . 142 - 13. Security Considerations . . . . . . . . . . . . . . . . . . . 143 - 13.1. TLS/TCP and DTLS/SCTP Usage . . . . . . . . . . . . . . . 143 - 13.2. Peer-to-Peer Considerations . . . . . . . . . . . . . . . 144 - 14. References . . . . . . . . . . . . . . . . . . . . . . . . . 145 - 14.1. Normative References . . . . . . . . . . . . . . . . . . 145 - 14.2. Informational References . . . . . . . . . . . . . . . . 147 - Appendix A. Acknowledgements . . . . . . . . . . . . . . . . . . 149 - A.1. RFC3588bis . . . . . . . . . . . . . . . . . . . . . . . 149 - A.2. RFC3588 . . . . . . . . . . . . . . . . . . . . . . . . . 150 - Appendix B. S-NAPTR Example . . . . . . . . . . . . . . . . . . 151 - Appendix C. Duplicate Detection . . . . . . . . . . . . . . . . 152 - Appendix D. Internationalized Domain Names . . . . . . . . . . . 154 - Authors' Addresses . . . . . . . . . . . . . . . . . . . . . . . 155 - - - - - - - -Fajardo, et al. Expires July 24, 2011 [Page 5] - -Internet-Draft Diameter Base Protocol January 2011 + 1. Introduction ....................................................7 + 1.1. Diameter Protocol ..........................................9 + 1.1.1. Description of the Document Set ....................10 + 1.1.2. Conventions Used in This Document ..................11 + 1.1.3. Changes from RFC 3588 ..............................11 + 1.2. Terminology ...............................................12 + 1.3. Approach to Extensibility .................................17 + 1.3.1. Defining New AVP Values ............................18 + 1.3.2. Creating New AVPs ..................................18 + 1.3.3. Creating New Commands ..............................18 + 1.3.4. Creating New Diameter Applications .................19 + 2. Protocol Overview ..............................................20 + 2.1. Transport .................................................22 + 2.1.1. SCTP Guidelines ....................................23 + 2.2. Securing Diameter Messages ................................24 + 2.3. Diameter Application Compliance ...........................24 + 2.4. Application Identifiers ...................................24 + 2.5. Connections vs. Sessions ..................................25 + 2.6. Peer Table ................................................26 + + + +Fajardo, et al. Standards Track [Page 2] + +RFC 6733 Diameter Base Protocol October 2012 + + + 2.7. Routing Table .............................................27 + 2.8. Role of Diameter Agents ...................................28 + 2.8.1. Relay Agents .......................................30 + 2.8.2. Proxy Agents .......................................31 + 2.8.3. Redirect Agents ....................................31 + 2.8.4. Translation Agents .................................32 + 2.9. Diameter Path Authorization ...............................33 + 3. Diameter Header ................................................34 + 3.1. Command Codes .............................................37 + 3.2. Command Code Format Specification .........................38 + 3.3. Diameter Command Naming Conventions .......................40 + 4. Diameter AVPs ..................................................40 + 4.1. AVP Header ................................................41 + 4.1.1. Optional Header Elements ...........................42 + 4.2. Basic AVP Data Formats ....................................43 + 4.3. Derived AVP Data Formats ..................................44 + 4.3.1. Common Derived AVP Data Formats ....................44 + 4.4. Grouped AVP Values ........................................51 + 4.4.1. Example AVP with a Grouped Data Type ...............52 + 4.5. Diameter Base Protocol AVPs ...............................55 + 5. Diameter Peers .................................................58 + 5.1. Peer Connections ..........................................58 + 5.2. Diameter Peer Discovery ...................................59 + 5.3. Capabilities Exchange .....................................60 + 5.3.1. Capabilities-Exchange-Request ......................62 + 5.3.2. Capabilities-Exchange-Answer .......................63 + 5.3.3. Vendor-Id AVP ......................................63 + 5.3.4. Firmware-Revision AVP ..............................64 + 5.3.5. Host-IP-Address AVP ................................64 + 5.3.6. Supported-Vendor-Id AVP ............................64 + 5.3.7. Product-Name AVP ...................................64 + 5.4. Disconnecting Peer Connections ............................64 + 5.4.1. Disconnect-Peer-Request ............................65 + 5.4.2. Disconnect-Peer-Answer .............................65 + 5.4.3. Disconnect-Cause AVP ...............................66 + 5.5. Transport Failure Detection ...............................66 + 5.5.1. Device-Watchdog-Request ............................67 + 5.5.2. Device-Watchdog-Answer .............................67 + 5.5.3. Transport Failure Algorithm ........................67 + 5.5.4. Failover and Failback Procedures ...................67 + 5.6. Peer State Machine ........................................68 + 5.6.1. Incoming Connections ...............................71 + 5.6.2. Events .............................................71 + 5.6.3. Actions ............................................72 + 5.6.4. The Election Process ...............................74 + + + + + + +Fajardo, et al. Standards Track [Page 3] + +RFC 6733 Diameter Base Protocol October 2012 + + + 6. Diameter Message Processing ....................................74 + 6.1. Diameter Request Routing Overview .........................74 + 6.1.1. Originating a Request ..............................75 + 6.1.2. Sending a Request ..................................76 + 6.1.3. Receiving Requests .................................76 + 6.1.4. Processing Local Requests ..........................76 + 6.1.5. Request Forwarding .................................77 + 6.1.6. Request Routing ....................................77 + 6.1.7. Predictive Loop Avoidance ..........................77 + 6.1.8. Redirecting Requests ...............................78 + 6.1.9. Relaying and Proxying Requests .....................79 + 6.2. Diameter Answer Processing ................................80 + 6.2.1. Processing Received Answers ........................81 + 6.2.2. Relaying and Proxying Answers ......................81 + 6.3. Origin-Host AVP ...........................................81 + 6.4. Origin-Realm AVP ..........................................82 + 6.5. Destination-Host AVP ......................................82 + 6.6. Destination-Realm AVP .....................................82 + 6.7. Routing AVPs ..............................................83 + 6.7.1. Route-Record AVP ...................................83 + 6.7.2. Proxy-Info AVP .....................................83 + 6.7.3. Proxy-Host AVP .....................................83 + 6.7.4. Proxy-State AVP ....................................83 + 6.8. Auth-Application-Id AVP ...................................83 + 6.9. Acct-Application-Id AVP ...................................84 + 6.10. Inband-Security-Id AVP ...................................84 + 6.11. Vendor-Specific-Application-Id AVP .......................84 + 6.12. Redirect-Host AVP ........................................85 + 6.13. Redirect-Host-Usage AVP ..................................85 + 6.14. Redirect-Max-Cache-Time AVP ..............................87 + 7. Error Handling .................................................87 + 7.1. Result-Code AVP ...........................................89 + 7.1.1. Informational ......................................90 + 7.1.2. Success ............................................90 + 7.1.3. Protocol Errors ....................................90 + 7.1.4. Transient Failures .................................92 + 7.1.5. Permanent Failures .................................92 + 7.2. Error Bit .................................................95 + 7.3. Error-Message AVP .........................................96 + 7.4. Error-Reporting-Host AVP ..................................96 + 7.5. Failed-AVP AVP ............................................96 + 7.6. Experimental-Result AVP ...................................97 + 7.7. Experimental-Result-Code AVP ..............................97 + 8. Diameter User Sessions .........................................98 + 8.1. Authorization Session State Machine .......................99 + 8.2. Accounting Session State Machine .........................104 + + + + + +Fajardo, et al. Standards Track [Page 4] + +RFC 6733 Diameter Base Protocol October 2012 + + + 8.3. Server-Initiated Re-Auth .................................110 + 8.3.1. Re-Auth-Request ...................................110 + 8.3.2. Re-Auth-Answer ....................................110 + 8.4. Session Termination ......................................111 + 8.4.1. Session-Termination-Request .......................112 + 8.4.2. Session-Termination-Answer ........................113 + 8.5. Aborting a Session .......................................113 + 8.5.1. Abort-Session-Request .............................114 + 8.5.2. Abort-Session-Answer ..............................114 + 8.6. Inferring Session Termination from Origin-State-Id .......115 + 8.7. Auth-Request-Type AVP ....................................116 + 8.8. Session-Id AVP ...........................................116 + 8.9. Authorization-Lifetime AVP ...............................117 + 8.10. Auth-Grace-Period AVP ...................................118 + 8.11. Auth-Session-State AVP ..................................118 + 8.12. Re-Auth-Request-Type AVP ................................118 + 8.13. Session-Timeout AVP .....................................119 + 8.14. User-Name AVP ...........................................119 + 8.15. Termination-Cause AVP ...................................120 + 8.16. Origin-State-Id AVP .....................................120 + 8.17. Session-Binding AVP .....................................120 + 8.18. Session-Server-Failover AVP .............................121 + 8.19. Multi-Round-Time-Out AVP ................................122 + 8.20. Class AVP ...............................................122 + 8.21. Event-Timestamp AVP .....................................122 + 9. Accounting ....................................................123 + 9.1. Server Directed Model ....................................123 + 9.2. Protocol Messages ........................................124 + 9.3. Accounting Application Extension and Requirements ........124 + 9.4. Fault Resilience .........................................125 + 9.5. Accounting Records .......................................125 + 9.6. Correlation of Accounting Records ........................126 + 9.7. Accounting Command Codes .................................127 + 9.7.1. Accounting-Request ................................127 + 9.7.2. Accounting-Answer .................................128 + 9.8. Accounting AVPs ..........................................129 + 9.8.1. Accounting-Record-Type AVP ........................129 + 9.8.2. Acct-Interim-Interval AVP .........................130 + 9.8.3. Accounting-Record-Number AVP ......................131 + 9.8.4. Acct-Session-Id AVP ...............................131 + 9.8.5. Acct-Multi-Session-Id AVP .........................131 + 9.8.6. Accounting-Sub-Session-Id AVP .....................131 + 9.8.7. Accounting-Realtime-Required AVP ..................132 + 10. AVP Occurrence Tables ........................................132 + 10.1. Base Protocol Command AVP Table .........................133 + 10.2. Accounting AVP Table ....................................134 + + + + + +Fajardo, et al. Standards Track [Page 5] + +RFC 6733 Diameter Base Protocol October 2012 + + + 11. IANA Considerations ..........................................135 + 11.1. AVP Header ..............................................135 + 11.1.1. AVP Codes ........................................136 + 11.1.2. AVP Flags ........................................136 + 11.2. Diameter Header .........................................136 + 11.2.1. Command Codes ....................................136 + 11.2.2. Command Flags ....................................137 + 11.3. AVP Values ..............................................137 + 11.3.1. Experimental-Result-Code AVP .....................137 + 11.3.2. Result-Code AVP Values ...........................137 + 11.3.3. Accounting-Record-Type AVP Values ................137 + 11.3.4. Termination-Cause AVP Values .....................137 + 11.3.5. Redirect-Host-Usage AVP Values ...................137 + 11.3.6. Session-Server-Failover AVP Values ...............137 + 11.3.7. Session-Binding AVP Values .......................137 + 11.3.8. Disconnect-Cause AVP Values ......................138 + 11.3.9. Auth-Request-Type AVP Values .....................138 + 11.3.10. Auth-Session-State AVP Values ...................138 + 11.3.11. Re-Auth-Request-Type AVP Values .................138 + 11.3.12. Accounting-Realtime-Required AVP Values .........138 + 11.3.13. Inband-Security-Id AVP (code 299) ...............138 + 11.4. _diameters Service Name and Port Number Registration ....138 + 11.5. SCTP Payload Protocol Identifiers .......................139 + 11.6. S-NAPTR Parameters ......................................139 + 12. Diameter Protocol-Related Configurable Parameters ............139 + 13. Security Considerations ......................................140 + 13.1. TLS/TCP and DTLS/SCTP Usage .............................140 + 13.2. Peer-to-Peer Considerations .............................141 + 13.3. AVP Considerations ......................................141 + 14. References ...................................................142 + 14.1. Normative References ....................................142 + 14.2. Informative References ..................................144 + Appendix A. Acknowledgements .....................................147 + A.1. This Document .............................................147 + A.2. RFC 3588 ..................................................148 + Appendix B. S-NAPTR Example ......................................148 + Appendix C. Duplicate Detection ..................................149 + Appendix D. Internationalized Domain Names .......................151 + + + + + + + + + + + + + +Fajardo, et al. Standards Track [Page 6] + +RFC 6733 Diameter Base Protocol October 2012 1. Introduction - Authentication, Authorization and Accounting (AAA) protocols such as + Authentication, Authorization, and Accounting (AAA) protocols such as TACACS [RFC1492] and RADIUS [RFC2865] were initially deployed to provide dial-up PPP [RFC1661] and terminal server access. Over time, AAA support was needed on many new access technologies, the scale and complexity of AAA networks grew, and AAA was also used on new - applications (such as voice over IP). This lead to new demands on - AAA protocols. + applications (such as voice over IP). This led to new demands on AAA + protocols. Network access requirements for AAA protocols are summarized in - [RFC2989]. These include: - + Aboba, et al. [RFC2989]. These include: Failover - [RFC2865] does not define failover mechanisms, and as a result, + [RFC2865] does not define failover mechanisms and, as a result, failover behavior differs between implementations. In order to provide well-defined failover behavior, Diameter supports - application-layer acknowledgements, and defines failover - algorithms and the associated state machine. This is described in - Section 5.5 and [RFC3539]. + application-layer acknowledgements and defines failover algorithms + and the associated state machine. Transmission-level security - [RFC2865] defines an application-layer authentication and - integrity scheme that is required only for use with Response + RADIUS [RFC2865] defines an application-layer authentication and + integrity scheme that is required only for use with response packets. While [RFC2869] defines an additional authentication and integrity mechanism, use is only required during Extensible - Authentication Protocol (EAP) sessions. While attribute-hiding is - supported, [RFC2865] does not provide support for per-packet - confidentiality. In accounting, [RFC2866] assumes that replay - protection is provided by the backend billing server, rather than - within the protocol itself. + Authentication Protocol (EAP) [RFC3748] sessions. While attribute + hiding is supported, [RFC2865] does not provide support for per- + packet confidentiality. In accounting, [RFC2866] assumes that + replay protection is provided by the backend billing server rather + than within the protocol itself. While [RFC3162] defines the use of IPsec with RADIUS, support for IPsec is not required. In order to provide universal support for @@ -322,50 +379,48 @@ Internet-Draft Diameter Base Protocol January 2011 domain AAA deployments, Diameter provides support for TLS/TCP and DTLS/SCTP. Security is discussed in Section 13. - Reliable transport - RADIUS runs over UDP, and does not define retransmission behavior; as a result, reliability varies between implementations. As described in [RFC2975], this is a major issue in accounting, where + packet loss may translate directly into revenue loss. In order to -Fajardo, et al. Expires July 24, 2011 [Page 6] - -Internet-Draft Diameter Base Protocol January 2011 - packet loss may translate directly into revenue loss. In order to - provide well defined transport behavior, Diameter runs over - reliable transport mechanisms (TCP, SCTP) as defined in [RFC3539]. +Fajardo, et al. Standards Track [Page 7] + +RFC 6733 Diameter Base Protocol October 2012 - Agent support - [RFC2865] does not provide for explicit support for agents, - including Proxies, Redirects and Relays. Since the expected - behavior is not defined, it varies between implementations. - Diameter defines agent behavior explicitly; this is described in - Section 2.8. + provide well-defined transport behavior, Diameter runs over + reliable transport mechanisms (TCP, Stream Control Transmission + Protocol (SCTP)) as defined in [RFC3539]. + + Agent support + RADIUS does not provide for explicit support for agents, including + proxies, redirects, and relays. Since the expected behavior is + not defined, it varies between implementations. Diameter defines + agent behavior explicitly; this is described in Section 2.8. Server-initiated messages - While RADIUS server-initiated messages are defined in [RFC5176], + While server-initiated messages are defined in RADIUS [RFC5176], support is optional. This makes it difficult to implement features such as unsolicited disconnect or re-authentication/ re-authorization on demand across a heterogeneous deployment. To - tackle this issue, support for server-initiated messages is + address this issue, support for server-initiated messages is mandatory in Diameter. - Transition support While Diameter does not share a common protocol data unit (PDU) with RADIUS, considerable effort has been expended in enabling - backward compatibility with RADIUS, so that the two protocols may + backward compatibility with RADIUS so that the two protocols may be deployed in the same network. Initially, it is expected that Diameter will be deployed within new network devices, as well as within gateways enabling communication between legacy RADIUS @@ -373,9 +428,8 @@ Internet-Draft Diameter Base Protocol January 2011 support to be added to legacy networks, by addition of a gateway or server speaking both RADIUS and Diameter. - In addition to addressing the above requirements, Diameter also - provides support for the following: - + In addition to addressing the above requirements, Diameter also + provides support for the following: Capability negotiation @@ -383,19 +437,19 @@ Internet-Draft Diameter Base Protocol January 2011 a mandatory/non-mandatory flag for attributes. Since RADIUS clients and servers are not aware of each other's capabilities, they may not be able to successfully negotiate a mutually - acceptable service, or in some cases, even be aware of what + acceptable service or, in some cases, even be aware of what service has been implemented. Diameter includes support for error + handling (Section 7), capability negotiation (Section 5.3), and + mandatory/non-mandatory Attribute-Value Pairs (AVPs) + (Section 4.1). -Fajardo, et al. Expires July 24, 2011 [Page 7] - -Internet-Draft Diameter Base Protocol January 2011 - handling (Section 7), capability negotiation (Section 5.3), and - mandatory/non-mandatory Attribute-Value Pairs (AVPs) (Section - 4.1). +Fajardo, et al. Standards Track [Page 8] + +RFC 6733 Diameter Base Protocol October 2012 Peer discovery and configuration @@ -403,7 +457,7 @@ Internet-Draft Diameter Base Protocol January 2011 RADIUS implementations typically require that the name or address of servers or clients be manually configured, along with the corresponding shared secrets. This results in a large - administrative burden, and creates the temptation to reuse the + administrative burden and creates the temptation to reuse the RADIUS shared secret, which can result in major security vulnerabilities if the Request Authenticator is not globally and temporally unique as required in [RFC2865]. Through DNS, Diameter @@ -411,7 +465,6 @@ Internet-Draft Diameter Base Protocol January 2011 of dynamic session keys is enabled via transmission-level security. - Over time, the capabilities of Network Access Server (NAS) devices have increased substantially. As a result, while Diameter is a considerably more sophisticated protocol than RADIUS, it remains @@ -427,42 +480,41 @@ Internet-Draft Diameter Base Protocol January 2011 o Error notification - o Extensibility, through addition of new applications, commands and - AVPs (required in [RFC2989]). + o Extensibility, required in [RFC2989], through addition of new + applications, commands, and AVPs - o Basic services necessary for applications, such as handling of + o Basic services necessary for applications, such as the handling of user sessions or accounting All data delivered by the protocol is in the form of AVPs. Some of these AVP values are used by the Diameter protocol itself, while others deliver data associated with particular applications that employ Diameter. AVPs may be arbitrarily added to Diameter messages, - the only restriction being that the Augmented Backus-Naur Form (ABNF, - [RFC5234]) Command Code syntax specification (Section 3.2) is - satisfied. AVPs are used by the base Diameter protocol to support - the following required features: + the only restriction being that the Command Code Format (CCF) + specification (Section 3.2) be satisfied. AVPs are used by the base + Diameter protocol to support the following required features: + o Transporting of user authentication information, for the purposes + of enabling the Diameter server to authenticate the user + o Transporting of service-specific authorization information, + between client and servers, allowing the peers to decide whether a + user's access request should be granted -Fajardo, et al. Expires July 24, 2011 [Page 8] - -Internet-Draft Diameter Base Protocol January 2011 - o Transporting of user authentication information, for the purposes - of enabling the Diameter server to authenticate the user. +Fajardo, et al. Standards Track [Page 9] + +RFC 6733 Diameter Base Protocol October 2012 - o Transporting of service-specific authorization information, - between client and servers, allowing the peers to decide whether a - user's access request should be granted. o Exchanging resource usage information, which may be used for accounting purposes, capacity planning, etc. - o Routing, relaying, proxying and redirecting of Diameter messages - through a server hierarchy. + o Routing, relaying, proxying, and redirecting of Diameter messages + through a server hierarchy - The Diameter base protocol satisfies the minimum requirements for an + The Diameter base protocol satisfies the minimum requirements for a AAA protocol, as specified by [RFC2989]. The base protocol may be used by itself for accounting purposes only, or it may be used with a Diameter application, such as Mobile IPv4 [RFC4004], or network @@ -473,16 +525,16 @@ Internet-Draft Diameter Base Protocol January 2011 many applications might provide functionality not provided by Diameter. Therefore, it is imperative that the designers of new applications understand their requirements before using Diameter. - See Section 2.4 for more information on Diameter applications. + See Section 1.3.4 for more information on Diameter applications. Any node can initiate a request. In that sense, Diameter is a peer- - to-peer protocol. In this document, a Diameter Client is a device at + to-peer protocol. In this document, a Diameter client is a device at the edge of the network that performs access control, such as a Network Access Server (NAS) or a Foreign Agent (FA). A Diameter client generates Diameter messages to request authentication, authorization, and accounting services for the user. A Diameter agent is a node that does not provide local user authentication or - authorization services; agents include proxies, redirects and relay + authorization services; agents include proxies, redirects, and relay agents. A Diameter server performs authentication and/or authorization of the user. A Diameter node may act as an agent for certain requests while acting as a server for others. @@ -494,33 +546,33 @@ Internet-Draft Diameter Base Protocol January 2011 The Diameter specification consists of an updated version of the base protocol specification (this document) and the Transport Profile - [RFC3539]. This document obsoletes RFC 3588. A summary of the base - protocol updates included in this document can be found in - Section 1.1.3. - - - -Fajardo, et al. Expires July 24, 2011 [Page 9] - -Internet-Draft Diameter Base Protocol January 2011 - + [RFC3539]. This document obsoletes both RFC 3588 and RFC 5719. A + summary of the base protocol updates included in this document can be + found in Section 1.1.3. This document defines the base protocol specification for AAA, which includes support for accounting. There are also a myriad of applications documents describing applications that use this base - specification for Authentication, Authorization and Accounting. + specification for Authentication, Authorization, and Accounting. These application documents specify how to use the Diameter protocol within the context of their application. + + +Fajardo, et al. Standards Track [Page 10] + +RFC 6733 Diameter Base Protocol October 2012 + + The Transport Profile document [RFC3539] discusses transport layer issues that arise with AAA protocols and recommendations on how to overcome these issues. This document also defines the Diameter failover algorithm and state machine. - Clarifications on the Routing of Diameter Request based on Username - and the Realm [RFC5729] defines specific behavior on how to route - request based on the content of the User-Name AVP (Attribute Value - Pair). + "Clarifications on the Routing of Diameter Request Based on the + Username and the Realm" [RFC5729] defines specific behavior on how to + route requests based on the content of the User-Name AVP (Attribute + Value Pair). 1.1.2. Conventions Used in This Document @@ -528,100 +580,100 @@ Internet-Draft Diameter Base Protocol January 2011 "SHOULD", "SHOULD NOT", "RECOMMENDED", "MAY", and "OPTIONAL" in this document are to be interpreted as described in [RFC2119]. -1.1.3. Changes from RFC3588 +1.1.3. Changes from RFC 3588 This document obsoletes RFC 3588 but is fully backward compatible with that document. The changes introduced in this document focus on - fixing issues that have surfaced during implementation of [RFC3588]. - An overview of some the major changes are given below. - - o Deprecated the use of Inband-Security AVP for negotiating - transport layer security. It has been generally considered that - bootstrapping of TLS via Inband-Security AVP creates certain - security risk because it does not completely protect the - information carried in the CER (Capabilities Exchange Request)/CEA - (Capabilities Exchange Answer). This version of Diameter adopted - a common approach of defining a well-known secured port that peers - should use when communicating via TLS/TCP and DTLS/SCTP. This new - approach augments the existing Inband-Security negotiation but - does not completely replace it. The old method is kept for - backwards compatibility reasons. + fixing issues that have surfaced during the implementation of + Diameter (RFC 3588). An overview of some the major changes are given + below. + + o Deprecated the use of the Inband-Security AVP for negotiating + Transport Layer Security (TLS) [RFC5246]. It has been generally + considered that bootstrapping of TLS via Inband-Security AVP + creates certain security risks because it does not completely + protect the information carried in the CER/CEA (Capabilities- + Exchange-Request/Capabilities-Exchange-Answer). This version of + Diameter adopts the common approach of defining a well-known + secured port that peers should use when communicating via TLS/TCP + and DTLS/SCTP. This new approach augments the existing in-band + security negotiation, but it does not completely replace it. The + old method is kept for backward compatibility reasons. o Deprecated the exchange of CER/CEA messages in the open state. - This feature was implied in the peer state machine table of - [RFC3588] but it was not clearly defined anywhere else in that + This feature was implied in the peer state machine table of RFC + 3588, but it was not clearly defined anywhere else in that document. As work on this document progressed, it became clear - that the multiplicity of meaning and use of Application Id AVPs in + that the multiplicity of meaning and use of Application-Id AVPs in the CER/CEA messages (and the messages themselves) is seen as an + abuse of the Diameter extensibility rules and thus required + simplification. Capabilities exchange in the open state has been + re-introduced in a separate specification [RFC6737], which clearly + defines new commands for this feature. -Fajardo, et al. Expires July 24, 2011 [Page 10] - -Internet-Draft Diameter Base Protocol January 2011 - abuse of the Diameter extensibility rules and thus required - simplification. It is assumed that the capabilities exchange in - the open state will be re-introduced in a separate specification - which clearly defines new commands for this feature. +Fajardo, et al. Standards Track [Page 11] + +RFC 6733 Diameter Base Protocol October 2012 + - o Simplified Security Requirements. The use of a secured transport + o Simplified security requirements. The use of a secured transport for exchanging Diameter messages remains mandatory. However, TLS/ - TCP and DTLS/SCTP has become the primary method of securing - Diameter and IPsec is a secondary alternative. See Section 13 for - details. The support for the End-to-End security framework - (E2ESequence AVP and 'P'-bit in the AVP header) has also been + TCP and DTLS/SCTP have become the primary methods of securing + Diameter with IPsec as a secondary alternative. See Section 13 + for details. The support for the End-to-End security framework + (E2E-Sequence AVP and 'P'-bit in the AVP header) has also been deprecated. - o Diameter Extensibility Changes. This includes fixes to the + o Changed Diameter extensibility. This includes fixes to the Diameter extensibility description (Section 1.3 and others) to better aid Diameter application designers; in addition, the new specification relaxes the policy with respect to the allocation of - command codes for vendor-specific uses. - - o Application Id Usage. Clarify the proper use of Application Id - information which can be found in multiple places within a - Diameter message. This includes correlating Application Ids found - in the message headers and AVPs. These changes also clearly - specify the proper Application Id value to use for specific base - protocol messages (ASR/ASA, STR/STA) as well as clarifying the - content and use of Vendor-Specific-Application-Id. - - o Routing Fixes. This document more clearly specifies what - information (AVPs and Application Id) can be used for making + Command Codes for vendor-specific uses. + + o Clarified Application Id usage. Clarify the proper use of + Application Id information, which can be found in multiple places + within a Diameter message. This includes correlating Application + Ids found in the message headers and AVPs. These changes also + clearly specify the proper Application Id value to use for + specific base protocol messages (ASR/ASA, STR/STA) as well as + clarify the content and use of Vendor-Specific-Application-Id. + + o Clarified routing fixes. This document more clearly specifies + what information (AVPs and Application Ids) can be used for making general routing decisions. A rule for the prioritization of redirect routing criteria when multiple route entries are found - via redirects has also been added (See Section 6.13 for details). + via redirects has also been added (see Section 6.13). - o Simplification of Diameter Peer Discovery. The Diameter discovery + o Simplified Diameter peer discovery. The Diameter discovery process now supports only widely used discovery schemes; the rest have been deprecated (see Section 5.2 for details). - There are many other many miscellaneous fixes that have been - introduced in this document that may not be considered significant - but they are important nonetheless. Examples are removal of obsolete - types, fixes to command ABNFs, fixes to the state machine, - clarification of the election process, message validation, fixes to - Failed-AVP and Result-Code AVP values, etc. A comprehensive list of - changes is not shown here for practical reasons. - - - + There are many other miscellaneous fixes that have been introduced in + this document that may not be considered significant, but they have + value nonetheless. Examples are removal of obsolete types, fixes to + the state machine, clarification of the election process, message + validation, fixes to Failed-AVP and Result-Code AVP values, etc. All + of the errata filed against RFC 3588 prior to the publication of this + document have been addressed. A comprehensive list of changes is not + shown here for practical reasons. +1.2. Terminology + AAA + Authentication, Authorization, and Accounting. -Fajardo, et al. Expires July 24, 2011 [Page 11] - -Internet-Draft Diameter Base Protocol January 2011 -1.2. Terminology - AAA - Authentication, Authorization and Accounting. +Fajardo, et al. Standards Track [Page 12] + +RFC 6733 Diameter Base Protocol October 2012 ABNF @@ -631,14 +683,12 @@ Internet-Draft Diameter Base Protocol January 2011 is used to define message exchanges in a bi-directional communications protocol. - Accounting The act of collecting information on resource usage for the - purpose of capacity planning, auditing, billing or cost + purpose of capacity planning, auditing, billing, or cost allocation. - Accounting Record An accounting record represents a summary of the resource @@ -647,114 +697,102 @@ Internet-Draft Diameter Base Protocol January 2011 accounting events or accounting events from several devices serving the same user. - Authentication The act of verifying the identity of an entity (subject). - Authorization The act of determining whether a requesting entity (subject) will be allowed access to a resource (object). - - AVP + Attribute-Value Pair (AVP) The Diameter protocol consists of a header followed by one or more Attribute-Value-Pairs (AVPs). An AVP includes a header and is used to encapsulate protocol-specific data (e.g., routing - information) as well as authentication, authorization or + information) as well as authentication, authorization, or + accounting information. + Command Code Format (CCF) + A modified form of ABNF used to define Diameter commands (see + Section 3.2). -Fajardo, et al. Expires July 24, 2011 [Page 12] - -Internet-Draft Diameter Base Protocol January 2011 + Diameter Agent + A Diameter Agent is a Diameter node that provides relay, proxy, + redirect, or translation services. - accounting information. - Diameter Agent - A Diameter Agent is a Diameter Node that provides either relay, - proxy, redirect or translation services. +Fajardo, et al. Standards Track [Page 13] + +RFC 6733 Diameter Base Protocol October 2012 Diameter Client - A Diameter Client is a Diameter Node that supports Diameter client - applications as well as the base protocol. Diameter Clients are + A Diameter client is a Diameter node that supports Diameter client + applications as well as the base protocol. Diameter clients are often implemented in devices situated at the edge of a network and provide access control services for that network. Typical - examples of Diameter Clients include the Network Access Server + examples of Diameter clients include the Network Access Server (NAS) and the Mobile IP Foreign Agent (FA). - Diameter Node - A Diameter Node is a host process that implements the Diameter - protocol, and acts either as a Client, Agent or Server. - + A Diameter node is a host process that implements the Diameter + protocol and acts as either a client, an agent, or a server. Diameter Peer - If a Diameter Node shares a direct transport connection with - another Diameter Node, it is a Diameter Peer to that Diameter - Node. - + Two Diameter nodes sharing a direct TCP or SCTP transport + connection are called Diameter peers. Diameter Server - A Diameter Server is a Diameter Node that handles authentication, - authorization and accounting requests for a particular realm. By - its very nature, a Diameter Server must support Diameter server + A Diameter server is a Diameter node that handles authentication, + authorization, and accounting requests for a particular realm. By + its very nature, a Diameter server must support Diameter server applications in addition to the base protocol. - Downstream Downstream is used to identify the direction of a particular - Diameter message from the Home Server towards the Diameter Client. - - - - - - - -Fajardo, et al. Expires July 24, 2011 [Page 13] - -Internet-Draft Diameter Base Protocol January 2011 - + Diameter message from the home server towards the Diameter client. Home Realm A Home Realm is the administrative domain with which the user maintains an account relationship. - Home Server - A Diameter Server which serves the Home Realm. - + A Diameter server that serves the Home Realm. - Interim accounting + Interim Accounting An interim accounting message provides a snapshot of usage during - a user's session. It is typically implemented in order to provide - for partial accounting of a user's session in the case a device - reboot or other network problem prevents the delivery of a session - summary message or session record. + a user's session. Typically, it is implemented in order to + provide for partial accounting of a user's session in case a + device reboot or other network problem prevents the delivery of a + session summary message or session record. + + + + +Fajardo, et al. Standards Track [Page 14] + +RFC 6733 Diameter Base Protocol October 2012 Local Realm A local realm is the administrative domain providing services to a user. An administrative domain may act as a local realm for - certain users, while being a home realm for others. - + certain users while being a home realm for others. Multi-session @@ -764,60 +802,56 @@ Internet-Draft Diameter Base Protocol January 2011 leg of the bundle would be a session while the entire bundle would be a multi-session. - Network Access Identifier The Network Access Identifier, or NAI [RFC4282], is used in the Diameter protocol to extract a user's identity and realm. The identity is used to identify the user during authentication and/or - authorization, while the realm is used for message routing + authorization while the realm is used for message routing purposes. - - - - - - - -Fajardo, et al. Expires July 24, 2011 [Page 14] - -Internet-Draft Diameter Base Protocol January 2011 - - Proxy Agent or Proxy In addition to forwarding requests and responses, proxies make policy decisions relating to resource usage and provisioning. - This is typically accomplished by tracking the state of NAS - devices. While proxies typically do not respond to client - Requests prior to receiving a Response from the server, they may - originate Reject messages in cases where policies are violated. - As a result, proxies need to understand the semantics of the - messages passing through them, and may not support all Diameter + Typically, this is accomplished by tracking the state of NAS + devices. While proxies usually do not respond to client requests + prior to receiving a response from the server, they may originate + Reject messages in cases where policies are violated. As a + result, proxies need to understand the semantics of the messages + passing through them, and they may not support all Diameter applications. - Realm The string in the NAI that immediately follows the '@' character. - NAI realm names are required to be unique, and are piggybacked on + NAI realm names are required to be unique and are piggybacked on the administration of the DNS namespace. Diameter makes use of the realm, also loosely referred to as domain, to determine - whether messages can be satisfied locally, or whether they must be + whether messages can be satisfied locally or whether they must be routed or redirected. In RADIUS, realm names are not necessarily piggybacked on the DNS namespace but may be independent of it. - Real-time Accounting + + + + + + +Fajardo, et al. Standards Track [Page 15] + +RFC 6733 Diameter Base Protocol October 2012 + + + Real-Time Accounting Real-time accounting involves the processing of information on - resource usage within a defined time window. Time constraints are - typically imposed in order to limit financial risk. The Diameter - Credit Control Application [RFC4006] is an example of an + resource usage within a defined time window. Typically, time + constraints are imposed in order to limit financial risk. The + Diameter Credit-Control Application [RFC4006] is an example of an application that defines real-time accounting functionality. - Relay Agent or Relay Relays forward requests and responses based on routing-related @@ -827,20 +861,9 @@ Internet-Draft Diameter Base Protocol January 2011 the semantics of messages or non-routing AVPs, and are capable of handling any Diameter application or message type. Since relays make decisions based on information in routing AVPs and realm - forwarding tables they do not keep state on NAS resource usage or + forwarding tables, they do not keep state on NAS resource usage or sessions in progress. - - - - - - -Fajardo, et al. Expires July 24, 2011 [Page 15] - -Internet-Draft Diameter Base Protocol January 2011 - - Redirect Agent Rather than forwarding requests and responses between clients and @@ -850,11 +873,10 @@ Internet-Draft Diameter Base Protocol January 2011 client and server. Redirect agents do not originate messages and are capable of handling any message type, although they may be configured only to redirect messages of certain types, while - acting as relay or proxy agents for other types. As with proxy + acting as relay or proxy agents for other types. As with relay agents, redirect agents do not keep state with respect to sessions or NAS resources. - Session A session is a related progression of events devoted to a @@ -863,14 +885,19 @@ Internet-Draft Diameter Base Protocol January 2011 packets with the same Session-Id are considered to be part of the same session. - Stateful Agent A stateful agent is one that maintains session state information, by keeping track of all authorized active sessions. Each authorized session is bound to a particular service, and its state - is considered active either until it is notified otherwise, or by - expiration. + is considered active either until it is notified otherwise or + until expiration. + + + +Fajardo, et al. Standards Track [Page 16] + +RFC 6733 Diameter Base Protocol October 2012 Sub-session @@ -881,53 +908,33 @@ Internet-Draft Diameter Base Protocol January 2011 during the same session) or serially. These changes in sessions are tracked with the Accounting-Sub-Session-Id. - - Transaction state + Transaction State The Diameter protocol requires that agents maintain transaction state, which is used for failover purposes. Transaction state - implies that upon forwarding a request, the Hop-by-Hop identifier + implies that upon forwarding a request, the Hop-by-Hop Identifier is saved; the field is replaced with a locally unique identifier, which is restored to its original value when the corresponding - - - -Fajardo, et al. Expires July 24, 2011 [Page 16] - -Internet-Draft Diameter Base Protocol January 2011 - - answer is received. The request's state is released upon receipt of the answer. A stateless agent is one that only maintains transaction state. - Translation Agent - A translation agent is a stateful Diameter node that performs - protocol translation between Diameter and another AAA protocol, - such as RADIUS. - - - Transport Connection - - A transport connection is a TCP or SCTP connection existing - directly between two Diameter peers, otherwise known as a Peer-to- - Peer Connection. - + A translation agent (TLA in Figure 4) is a stateful Diameter node + that performs protocol translation between Diameter and another + AAA protocol, such as RADIUS. Upstream Upstream is used to identify the direction of a particular - Diameter message from the Diameter Client towards the Home Server. - + Diameter message from the Diameter client towards the home server. User The entity or device requesting or using some resource, in support of which a Diameter client has generated a request. - 1.3. Approach to Extensibility The Diameter protocol is designed to be extensible, using several @@ -941,107 +948,108 @@ Internet-Draft Diameter Base Protocol January 2011 o Creating new applications - From the point of view of extensibility Diameter authentication, - authorization and accounting applications are treated in the same - way. - -Fajardo, et al. Expires July 24, 2011 [Page 17] +Fajardo, et al. Standards Track [Page 17] -Internet-Draft Diameter Base Protocol January 2011 +RFC 6733 Diameter Base Protocol October 2012 - Note: Protocol designers should try to re-use existing functionality, + From the point of view of extensibility, Diameter authentication, + authorization, and accounting applications are treated in the same + way. + + Note: Protocol designers should try to reuse existing functionality, namely AVP values, AVPs, commands, and Diameter applications. Reuse simplifies standardization and implementation. To avoid potential - interoperability issues it is important to ensure that the semantics - of the re-used features are well understood. Given that Diameter can - also carry RADIUS attributes as Diameter AVPs, such re-use - considerations apply also to existing RADIUS attributes that may be + interoperability issues, it is important to ensure that the semantics + of the reused features are well understood. Given that Diameter can + also carry RADIUS attributes as Diameter AVPs, such reuse + considerations also apply to existing RADIUS attributes that may be useful in a Diameter application. -1.3.1. Defining New AVP Values +1.3.1. Defining New AVP Values In order to allocate a new AVP value for AVPs defined in the Diameter - Base protocol, the IETF needs to approve a new RFC that describes the + base protocol, the IETF needs to approve a new RFC that describes the AVP value. IANA considerations for these AVP values are discussed in - Section 11.4. + Section 11.3. The allocation of AVP values for other AVPs is guided by the IANA considerations of the document that defines those AVPs. Typically, - allocation of new values for an AVP defined in an IETF RFC should - require IETF Review [RFC5226], whereas values for vendor-specific - AVPs can be allocated by the vendor. + allocation of new values for an AVP defined in an RFC would require + IETF Review [RFC5226], whereas values for vendor-specific AVPs can be + allocated by the vendor. -1.3.2. Creating New AVPs +1.3.2. Creating New AVPs A new AVP being defined MUST use one of the data types listed in - Section 4.2 or Section 4.3. If an appropriate derived data type is - already defined, it SHOULD be used instead of a base data type to - encourage reusability and good design practice. + Sections 4.2 or 4.3. If an appropriate derived data type is already + defined, it SHOULD be used instead of a base data type to encourage + reusability and good design practice. In the event that a logical grouping of AVPs is necessary, and multiple "groups" are possible in a given command, it is recommended that a Grouped AVP be used (see Section 4.4). The creation of new AVPs can happen in various ways. The recommended - approach is to define a new general-purpose AVP in a standards track - RFC approved by the IETF. However, as described in Section 11.1.1 - there are also other mechanisms. + approach is to define a new general-purpose AVP in a Standards Track + RFC approved by the IETF. However, as described in Section 11.1.1, + there are other mechanisms. -1.3.3. Creating New Commands +1.3.3. Creating New Commands A new Command Code MUST be allocated when required AVPs (those - indicated as {AVP} in the ABNF definition) are added to, deleted from + indicated as {AVP} in the CCF definition) are added to, deleted from, or redefined in (for example, by changing a required AVP into an optional one) an existing command. - Furthermore, if the transport characteristics of a command are - changed (for example, with respect to the number of round trips - required) a new Command Code MUST be registered. - -Fajardo, et al. Expires July 24, 2011 [Page 18] +Fajardo, et al. Standards Track [Page 18] -Internet-Draft Diameter Base Protocol January 2011 +RFC 6733 Diameter Base Protocol October 2012 + + Furthermore, if the transport characteristics of a command are + changed (for example, with respect to the number of round trips + required), a new Command Code MUST be registered. - A change to the ABNF of a command, such as described above, MUST + A change to the CCF of a command, such as described above, MUST result in the definition of a new Command Code. This subsequently - leads to the need to define a new Diameter Application for any - application that will use that new Command. - - The IANA considerations for commands are discussed in Section 11.2.1. - -1.3.4. Creating New Diameter Applications - - Every Diameter application specification MUST have an IANA assigned - Application Id (see Section 2.4 and Section 11.3). The managed - Application Id space is flat and there is no relationship between - different Diameter applications with respect to their Application - Ids. As such, there is no versioning support provided by these - application Ids itself; every Diameter application is a standalone - application. If the application has a relationship with other - Diameter applications, such a relationship is not known to Diameter. - - Before describing the rules for creating new Diameter applications it - is important to discuss the semantics of the AVPs occurrences as - stated in the ABNF and the M-bit flag (Section 4.1) for an AVP. - There is no relationship imposed between the two; they are set + leads to the need to define a new Diameter application for any + application that will use that new command. + + The IANA considerations for Command Codes are discussed in + Section 3.1. + +1.3.4. Creating New Diameter Applications + + Every Diameter application specification MUST have an IANA-assigned + Application Id (see Section 2.4). The managed Application ID space + is flat, and there is no relationship between different Diameter + applications with respect to their Application Ids. As such, there + is no versioning support provided by these Application Ids + themselves; every Diameter application is a standalone application. + If the application has a relationship with other Diameter + applications, such a relationship is not known to Diameter. + + Before describing the rules for creating new Diameter applications, + it is important to discuss the semantics of the AVP occurrences as + stated in the CCF and the M-bit flag (Section 4.1) for an AVP. There + is no relationship imposed between the two; they are set independently. - o The ABNF indicates what AVPs are placed into a Diameter Command by - the sender of that Command. Often, since there are multiple modes - of protocol interactions many of the AVPs are indicated as + o The CCF indicates what AVPs are placed into a Diameter command by + the sender of that command. Often, since there are multiple modes + of protocol interactions, many of the AVPs are indicated as optional. o The M-bit allows the sender to indicate to the receiver whether or not understanding the semantics of an AVP and its content is mandatory. If the M-bit is set by the sender and the receiver - does not understand the AVP or the values carried within that AVP + does not understand the AVP or the values carried within that AVP, then a failure is generated (see Section 7). It is the decision of the protocol designer when to develop a new @@ -1050,77 +1058,48 @@ Internet-Draft Diameter Base Protocol January 2011 of the following criteria are met: - M-bit Setting - An AVP with the M-bit in the MUST column of the AVP flag table is - added to an existing Command/Application. - An AVP with the M-bit in the MAY column of the AVP flag table is - added to an existing Command/Application. -Fajardo, et al. Expires July 24, 2011 [Page 19] +Fajardo, et al. Standards Track [Page 19] -Internet-Draft Diameter Base Protocol January 2011 +RFC 6733 Diameter Base Protocol October 2012 + + M-bit Setting + + An AVP with the M-bit in the MUST column of the AVP flag table is + added to an existing Command/Application. An AVP with the M-bit + in the MAY column of the AVP flag table is added to an existing + Command/Application. Note: The M-bit setting for a given AVP is relevant to an - Application and each command within that application which - includes the AVP. That is, if an AVP appears in two commands for + Application and each command within that application that includes + the AVP. That is, if an AVP appears in two commands for application Foo and the M-bit settings are different in each command, then there should be two AVP flag tables describing when to set the M-bit. Commands - A new command is used within the existing application either - because an additional command is added, an existing command has + A new command is used within the existing application because + either an additional command is added, an existing command has been modified so that a new Command Code had to be registered, or a command has been deleted. - If the ABNF definition of a command allows it, an implementation may + AVP Flag bits + + If an existing application changes the meaning/semantics of its + AVP Flags or adds new flag bits, then a new Diameter application + MUST be created. + + If the CCF definition of a command allows it, an implementation may add arbitrary optional AVPs with the M-bit cleared (including vendor- specific AVPs) to that command without needing to define a new application. Please refer to Section 11.1.1 for details. - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - -Fajardo, et al. Expires July 24, 2011 [Page 20] - -Internet-Draft Diameter Base Protocol January 2011 - - 2. Protocol Overview The base Diameter protocol concerns itself with establishing @@ -1137,9 +1116,17 @@ Internet-Draft Diameter Base Protocol January 2011 The initial request for authentication and/or authorization of a user would include the Session-Id AVP. The Session-Id is then used in all subsequent messages to identify the user's session (see Section 8 for - more information). The communicating party may accept the request, - or reject it by returning an answer message with the Result-Code AVP - set to indicate an error occurred. The specific behavior of the + + + +Fajardo, et al. Standards Track [Page 20] + +RFC 6733 Diameter Base Protocol October 2012 + + + more information). The communicating party may accept the request or + reject it by returning an answer message with the Result-Code AVP set + to indicate that an error occurred. The specific behavior of the Diameter server or client receiving a request depends on the Diameter application employed. @@ -1153,30 +1140,23 @@ Internet-Draft Diameter Base Protocol January 2011 applications. For authentication and authorization, it is always extended for a particular application. - Diameter Clients MUST support the base protocol, which includes + Diameter clients MUST support the base protocol, which includes accounting. In addition, they MUST fully support each Diameter application that is needed to implement the client's service, e.g., - NASREQ and/or Mobile IPv4. A Diameter Client MUST be referred to as - "Diameter X Client" where X is the application which it supports, and - not a "Diameter Client". + Network Access Server Requirements (NASREQ) [RFC2881] and/or Mobile + IPv4. A Diameter client MUST be referred to as "Diameter X Client" + where X is the application that it supports and not a "Diameter + Client". - Diameter Servers MUST support the base protocol, which includes + Diameter servers MUST support the base protocol, which includes accounting. In addition, they MUST fully support each Diameter application that is needed to implement the intended service, e.g., - NASREQ and/or Mobile IPv4. A Diameter Server MUST be referred to as - "Diameter X Server" where X is the application which it supports, and + NASREQ and/or Mobile IPv4. A Diameter server MUST be referred to as + "Diameter X Server" where X is the application that it supports, and not a "Diameter Server". - Diameter Relays and redirect agents are transparent to the Diameter - applications but they MUST support the Diameter base protocol, which - - - -Fajardo, et al. Expires July 24, 2011 [Page 21] - -Internet-Draft Diameter Base Protocol January 2011 - - + Diameter relays and redirect agents are transparent to the Diameter + applications, but they MUST support the Diameter base protocol, which includes accounting, and all Diameter applications. Diameter proxies MUST support the base protocol, which includes @@ -1186,55 +1166,75 @@ Internet-Draft Diameter Base Protocol January 2011 "Diameter X Proxy" where X is the application which it supports, and not a "Diameter Proxy". + + + + + + + + + +Fajardo, et al. Standards Track [Page 21] + +RFC 6733 Diameter Base Protocol October 2012 + + 2.1. Transport The Diameter Transport profile is defined in [RFC3539]. - The base Diameter protocol is run on port 3868 for both TCP [RFC793] - and SCTP [RFC4960]. For TLS [RFC5246] and DTLS [RFC4347], a Diameter - node that initiate a connection prior to any message exchanges MUST - run on port [TBD]. It is assumed that TLS is run on top of TCP when - it is used and DTLS is run on top of SCTP when it is used. + The base Diameter protocol is run on port 3868 for both TCP [RFC0793] + and SCTP [RFC4960]. For TLS [RFC5246] and Datagram Transport Layer + Security (DTLS) [RFC6347], a Diameter node that initiates a + connection prior to any message exchanges MUST run on port 5658. It + is assumed that TLS is run on top of TCP when it is used, and DTLS is + run on top of SCTP when it is used. If the Diameter peer does not support receiving TLS/TCP and DTLS/SCTP - connections on port [TBD], i.e. the peer complies only with - [RFC3588], then the initiator MAY revert to using TCP or SCTP and on - port 3868. Note that this scheme is kept for the purpose of - backwards compatibility only and that there are inherent security - vulnerabilities when the initial CER/CEA messages are sent un- - protected (see Section 5.6). + connections on port 5658 (i.e., the peer complies only with RFC + 3588), then the initiator MAY revert to using TCP or SCTP on port + 3868. Note that this scheme is kept only for the purpose of backward + compatibility and that there are inherent security vulnerabilities + when the initial CER/CEA messages are sent unprotected (see + Section 5.6). - Diameter clients MUST support either TCP or SCTP, while agents and - servers SHOULD support both. + Diameter clients MUST support either TCP or SCTP; agents and servers + SHOULD support both. A Diameter node MAY initiate connections from a source port other than the one that it declares it accepts incoming connections on, and - MUST be prepared to receive connections on port 3868 for TCP or SCTP - and port [TBD] for TLS/TCP and DTLS/SCTP connections. A given - Diameter instance of the peer state machine MUST NOT use more than - one transport connection to communicate with a given peer, unless - multiple instances exist on the peer in which case a separate - connection per process is allowed. + it MUST always be prepared to receive connections on port 3868 for + TCP or SCTP and port 5658 for TLS/TCP and DTLS/SCTP connections. + When DNS-based peer discovery (Section 5.2) is used, the port numbers + received from SRV records take precedence over the default ports + (3868 and 5658). + + A given Diameter instance of the peer state machine MUST NOT use more + than one transport connection to communicate with a given peer, + unless multiple instances exist on the peer, in which, case a + separate connection per process is allowed. When no transport connection exists with a peer, an attempt to - connect SHOULD be periodically made. This behavior is handled via + connect SHOULD be made periodically. This behavior is handled via the Tc timer (see Section 12 for details), whose recommended value is 30 seconds. There are certain exceptions to this rule, such as when a peer has terminated the transport connection stating that it does not wish to communicate. When connecting to a peer and either zero or more transports are - specified, TLS SHOULD be tried first, followed by DTLS, then by TCP + specified, TLS SHOULD be tried first, followed by DTLS, then by TCP, + and finally by SCTP. See Section 5.2 for more information on peer + discovery. -Fajardo, et al. Expires July 24, 2011 [Page 22] - -Internet-Draft Diameter Base Protocol January 2011 - and finally by SCTP. See Section 5.2 for more information on peer - discovery. +Fajardo, et al. Standards Track [Page 22] + +RFC 6733 Diameter Base Protocol October 2012 + Diameter implementations SHOULD be able to interpret ICMP protocol port unreachable messages as explicit indications that the server is @@ -1253,18 +1253,18 @@ Internet-Draft Diameter Base Protocol January 2011 Diameter messages SHOULD be mapped into SCTP streams in a way that avoids head-of-the-line (HOL) blocking. Among different ways of performing the mapping that fulfill this requirement it is - RECOMMENDED that a Diameter node sends every Diameter message - (request or response) over the stream zero with the unordered flag - set. However, Diameter nodes MAY select and implement other design - alternatives for avoiding HOL blocking such as using multiple streams - with the unordered flag cleared (as originally instructed in - RFC3588). On the receiving side, a Diameter entity MUST be ready to - receive Diameter messages over any stream and it is free to return - responses over a different stream. This way, both sides manage the - available streams in the sending direction, independently of the - streams chosen by the other side to send a particular Diameter - message. These messages can be out-of-order and belong to different - Diameter sessions. + RECOMMENDED that a Diameter node send every Diameter message (request + or response) over stream zero with the unordered flag set. However, + Diameter nodes MAY select and implement other design alternatives for + avoiding HOL blocking such as using multiple streams with the + unordered flag cleared (as originally instructed in RFC 3588). On + the receiving side, a Diameter entity MUST be ready to receive + Diameter messages over any stream, and it is free to return responses + over a different stream. This way, both sides manage the available + streams in the sending direction, independently of the streams chosen + by the other side to send a particular Diameter message. These + messages can be out-of-order and belong to different Diameter + sessions. Out-of-order delivery has special concerns during a connection establishment and termination. When a connection is established, the @@ -1275,30 +1275,37 @@ Internet-Draft Diameter Base Protocol January 2011 the connection. In order to avoid this race condition, the receiver side SHOULD NOT use out-of-order delivery methods until the first message has been received from the initiator, proving that it has - moved to I-Open state. To trigger such message, the receiver side - could send a DWR immediatly after sending CEA. Upon reception of the - corresponding DWA, the receiver side should start using out-of-order - delivery methods to counter the HOL blocking. + moved to I-Open state. To trigger such a message, the receiver side + could send a DWR immediately after sending a CEA. Upon reception of + the corresponding DWA, the receiver side should start using out-of- + order delivery methods to counter the HOL blocking. Another race condition may occur when DPR and DPA messages are used. + Both DPR and DPA are small in size; thus, they may be delivered to + the peer faster than application messages when an out-of-order + delivery mechanism is used. Therefore, it is possible that a DPR/DPA -Fajardo, et al. Expires July 24, 2011 [Page 23] +Fajardo, et al. Standards Track [Page 23] -Internet-Draft Diameter Base Protocol January 2011 +RFC 6733 Diameter Base Protocol October 2012 + + exchange completes while application messages are still in transit, + resulting in a loss of these messages. An implementation could + mitigate this race condition, for example, using timers, and wait for + a short period of time for pending application level messages to + arrive before proceeding to disconnect the transport connection. + Eventually, lost messages are handled by the retransmission mechanism + described in Section 5.5.4. - Both DPR and DPA are small in size, thus they may be delivered faster - to the peer than application messages when out-of-order delivery - mechanism is used. Therefore, it is possible that a DPR/DPA exchange - completes while application messages are still in transit, resulting - to a loss of these messages. An implementation could mitigate this - race condition, for example, using timers and wait for a short period - of time for pending application level messages to arrive before - proceeding to disconnect the transport connection. Eventually, lost - messages are handled by the retransmission mechanism described in - Section 5.5.4. + A Diameter agent SHOULD use dedicated payload protocol identifiers + (PPIDs) for clear text and encrypted SCTP DATA chunks instead of only + using the unspecified payload protocol identifier (value 0). For + this purpose, two PPID values are allocated: the PPID value 46 is for + Diameter messages in clear text SCTP DATA chunks, and the PPID value + 47 is for Diameter messages in protected DTLS/SCTP DATA chunks. 2.2. Securing Diameter Messages @@ -1307,7 +1314,7 @@ Internet-Draft Diameter Base Protocol January 2011 the use of TLS/TCP and DTLS/SCTP. If desired, alternative security mechanisms that are independent of Diameter, such as IPsec [RFC4301], can be deployed to secure connections between peers. The Diameter - protocol MUST NOT be used without any security mechanism. + protocol MUST NOT be used without one of TLS, DTLS, or IPsec. 2.3. Diameter Application Compliance @@ -1318,37 +1325,40 @@ Internet-Draft Diameter Base Protocol January 2011 Implementations MAY add arbitrary optional AVPs with the M-bit cleared (including vendor-specific AVPs) to a command defined in an - application, but only if the command's ABNF syntax specification + application, but only if the command's CCF syntax specification allows for it. Please refer to Section 11.1.1 for details. 2.4. Application Identifiers - Each Diameter application MUST have an IANA assigned Application Id - (see Section 11.3). The base protocol does not require an - Application Id since its support is mandatory. During the - capabilities exchange, Diameter nodes inform their peers of locally - supported applications. Furthermore, all Diameter messages contain - an Application Id, which is used in the message forwarding process. + Each Diameter application MUST have an IANA-assigned Application ID. + The base protocol does not require an Application Id since its + support is mandatory. During the capabilities exchange, Diameter + nodes inform their peers of locally supported applications. + Furthermore, all Diameter messages contain an Application Id, which + is used in the message forwarding process. - The following Application Id values are defined: - Diameter Common Messages 0 - Diameter Base Accounting 3 - Relay 0xffffffff - Relay and redirect agents MUST advertise the Relay Application -Fajardo, et al. Expires July 24, 2011 [Page 24] + +Fajardo, et al. Standards Track [Page 24] -Internet-Draft Diameter Base Protocol January 2011 +RFC 6733 Diameter Base Protocol October 2012 + + The following Application Id values are defined: + + Diameter common message 0 + Diameter base accounting 3 + Relay 0xffffffff - Identifier, while all other Diameter nodes MUST advertise locally - supported applications. The receiver of a Capabilities Exchange - message advertising Relay service MUST assume that the sender - supports all current and future applications. + Relay and redirect agents MUST advertise the Relay Application ID, + while all other Diameter nodes MUST advertise locally supported + applications. The receiver of a Capabilities Exchange message + advertising relay service MUST assume that the sender supports all + current and future applications. Diameter relay and proxy agents are responsible for finding an upstream server that supports the application of a particular @@ -1358,16 +1368,15 @@ Internet-Draft Diameter Base Protocol January 2011 2.5. Connections vs. Sessions This section attempts to provide the reader with an understanding of - the difference between connection and session, which are terms used - extensively throughout this document. + the difference between "connection" and "session", which are terms + used extensively throughout this document. - A connection refers to a transport level connection between two peers + A connection refers to a transport-level connection between two peers that is used to send and receive Diameter messages. A session is a - logical concept at the application layer existing between the + logical concept at the application layer that exists between the Diameter client and the Diameter server; it is identified via the Session-Id AVP. - +--------+ +-------+ +--------+ | Client | | Relay | | Server | +--------+ +-------+ +--------+ @@ -1377,100 +1386,93 @@ Internet-Draft Diameter Base Protocol January 2011 <-----------------------------> User session x - Figure 1: Diameter connections and sessions + Figure 1: Diameter Connections and Sessions In the example provided in Figure 1, peer connection A is established - between the Client and the Relay. Peer connection B is established - between the Relay and the Server. User session X spans from the - Client via the Relay to the Server. Each "user" of a service causes + between the client and the relay. Peer connection B is established + between the relay and the server. User session X spans from the + client via the relay to the server. Each "user" of a service causes an auth request to be sent, with a unique session identifier. Once accepted by the server, both the client and the server are aware of the session. - It is important to note that there is no relationship between a - connection and a session, and that Diameter messages for multiple - sessions are all multiplexed through a single connection. Also note - that Diameter messages pertaining to the session, both application - specific and those that are defined in this document such as ASR/ASA, - RAR/RAA and STR/STA MUST carry the Application Id of the application. -Fajardo, et al. Expires July 24, 2011 [Page 25] +Fajardo, et al. Standards Track [Page 25] -Internet-Draft Diameter Base Protocol January 2011 +RFC 6733 Diameter Base Protocol October 2012 - Diameter messages pertaining to peer connection establishment and - maintenance such as CER/CEA, DWR/DWA and DPR/DPA MUST carry an - Application Id of zero (0). + It is important to note that there is no relationship between a + connection and a session, and that Diameter messages for multiple + sessions are all multiplexed through a single connection. Also, note + that Diameter messages pertaining to the session, both application- + specific and those that are defined in this document such as ASR/ASA, + RAR/RAA, and STR/STA, MUST carry the Application Id of the + application. Diameter messages pertaining to peer connection + establishment and maintenance such as CER/CEA, DWR/DWA, and DPR/DPA + MUST carry an Application Id of zero (0). 2.6. Peer Table - The Diameter Peer Table is used in message forwarding, and referenced - by the Routing Table. A Peer Table entry contains the following - fields: + The Diameter peer table is used in message forwarding and is + referenced by the routing table. A peer table entry contains the + following fields: - Host identity + Host Identity - Following the conventions described for the DiameterIdentity - derived AVP data format in Section 4.3. This field contains the + Following the conventions described for the DiameterIdentity- + derived AVP data format in Section 4.3.1, this field contains the contents of the Origin-Host (Section 6.3) AVP found in the CER or CEA message. - StatusT - This is the state of the peer entry, and MUST match one of the + This is the state of the peer entry, and it MUST match one of the values listed in Section 5.6. - Static or Dynamic Specifies whether a peer entry was statically configured or dynamically discovered. - - Expiration time + Expiration Time Specifies the time at which dynamically discovered peer table - entries are to be either refreshed, or expired. - + entries are to be either refreshed or expired. If public key + certificates are used for Diameter security (e.g., with TLS), this + value MUST NOT be greater than the expiry times in the relevant + certificates. TLS/TCP and DTLS/SCTP Enabled Specifies whether TLS/TCP and DTLS/SCTP is to be used when communicating with the peer. - Additional security information, when needed (e.g., keys, - certificates) - - + certificates). - - -Fajardo, et al. Expires July 24, 2011 [Page 26] +Fajardo, et al. Standards Track [Page 26] -Internet-Draft Diameter Base Protocol January 2011 +RFC 6733 Diameter Base Protocol October 2012 2.7. Routing Table All Realm-Based routing lookups are performed against what is - commonly known as the Routing Table (see Section 12). A Routing - Table Entry contains the following fields: + commonly known as the routing table (see Section 12). Each routing + table entry contains the following fields: Realm Name - This is the field that is MUST be used as a primary key in the + This is the field that MUST be used as a primary key in the routing table lookups. Note that some implementations perform their lookups based on longest-match-from-the-right on the realm rather than requiring an exact match. - Application Identifier An application is identified by an Application Id. A route entry @@ -1478,21 +1480,19 @@ Internet-Draft Diameter Base Protocol January 2011 the message header. This field MUST be used as a secondary key field in routing table lookups. - Local Action The Local Action field is used to identify how a message should be treated. The following actions are supported: - - 1. LOCAL - Diameter messages that can be satisfied locally, and - do not need to be routed to another Diameter entity. + 1. LOCAL - Diameter messages that can be satisfied locally and do + not need to be routed to another Diameter entity. 2. RELAY - All Diameter messages that fall within this category - MUST be routed to a next hop Diameter entity that is indicated + MUST be routed to a next-hop Diameter entity that is indicated by the identifier described below. Routing is done without modifying any non-routing AVPs. See Section 6.1.9 for - relaying guidelines + relaying guidelines. 3. PROXY - All Diameter messages that fall within this category MUST be routed to a next Diameter entity that is indicated by @@ -1504,47 +1504,53 @@ Internet-Draft Diameter Base Protocol January 2011 4. REDIRECT - Diameter messages that fall within this category MUST have the identity of the home Diameter server(s) appended, and returned to the sender of the message. See - Section 6.1.8 for redirect guidelines. + Section 6.1.8 for redirection guidelines. + + + -Fajardo, et al. Expires July 24, 2011 [Page 27] +Fajardo, et al. Standards Track [Page 27] -Internet-Draft Diameter Base Protocol January 2011 +RFC 6733 Diameter Base Protocol October 2012 Server Identifier - One or more servers to which the message is to be routed. These - servers MUST also be present in the Peer table. When the Local - Action is set to RELAY or PROXY, this field contains the identity - of the server(s) the message MUST be routed to. When the Local + The identity of one or more servers to which the message is to be + routed. This identity MUST also be present in the Host Identity + field of the peer table (Section 2.6). When the Local Action is + set to RELAY or PROXY, this field contains the identity of the + server(s) to which the message MUST be routed. When the Local Action field is set to REDIRECT, this field contains the identity - of one or more servers the message MUST be redirected to. + of one or more servers to which the message MUST be redirected. Static or Dynamic Specifies whether a route entry was statically configured or dynamically discovered. - Expiration time + Expiration Time Specifies the time at which a dynamically discovered route table - entry expires. + entry expires. If public key certificates are used for Diameter + security (e.g., with TLS), this value MUST NOT be greater than the + expiry time in the relevant certificates. It is important to note that Diameter agents MUST support at least - one of the LOCAL, RELAY, PROXY or REDIRECT modes of operation. + one of the LOCAL, RELAY, PROXY, or REDIRECT modes of operation. Agents do not need to support all modes of operation in order to - conform with the protocol specification, but MUST follow the protocol - compliance guidelines in Section 2. Relay agents and proxies MUST - NOT reorder AVPs. + conform with the protocol specification, but they MUST follow the + protocol compliance guidelines in Section 2. Relay agents and + proxies MUST NOT reorder AVPs. The routing table MAY include a default entry that MUST be used for any requests not matching any of the other entries. The routing table MAY consist of only such an entry. When a request is routed, the target server MUST have advertised the - Application Id (see Section 2.4) for the given message, or have + Application Id (see Section 2.4) for the given message or have advertised itself as a relay or proxy agent. Otherwise, an error is returned with the Result-Code AVP set to DIAMETER_UNABLE_TO_DELIVER. @@ -1552,23 +1558,24 @@ Internet-Draft Diameter Base Protocol January 2011 In addition to clients and servers, the Diameter protocol introduces relay, proxy, redirect, and translation agents, each of which is - defined in Section 1.3. These Diameter agents are useful for several + defined in Section 1.2. Diameter agents are useful for several reasons: o They can distribute administration of systems to a configurable grouping, including the maintenance of security associations. - o They can be used for concentration of requests from an number of - co-located or distributed NAS equipment sets to a set of like user - groups. -Fajardo, et al. Expires July 24, 2011 [Page 28] +Fajardo, et al. Standards Track [Page 28] -Internet-Draft Diameter Base Protocol January 2011 +RFC 6733 Diameter Base Protocol October 2012 + o They can be used for concentration of requests from a number of + co-located or distributed NAS equipment sets to a set of like user + groups. + o They can do value-added processing to the requests or responses. o They can be used for load balancing. @@ -1578,7 +1585,7 @@ Internet-Draft Diameter Base Protocol January 2011 The Diameter protocol requires that agents maintain transaction state, which is used for failover purposes. Transaction state - implies that upon forwarding a request, its Hop-by-Hop identifier is + implies that upon forwarding a request, its Hop-by-Hop Identifier is saved; the field is replaced with a locally unique identifier, which is restored to its original value when the corresponding answer is received. The request's state is released upon receipt of the @@ -1593,7 +1600,7 @@ Internet-Draft Diameter Base Protocol January 2011 A stateful agent is one that maintains session state information by keeping track of all authorized active sessions. Each authorized session is bound to a particular service, and its state is considered - active either until the agent is notified otherwise, or the session + active until either the agent is notified otherwise or the session expires. Each authorized session has an expiration, which is communicated by Diameter servers via the Session-Timeout AVP. @@ -1604,40 +1611,43 @@ Internet-Draft Diameter Base Protocol January 2011 o Limiting resources authorized to a particular user - o Per user or transaction auditing + o Per-user or per-transaction auditing A Diameter agent MAY act in a stateful manner for some requests and be stateless for others. A Diameter implementation MAY act as one - type of agent for some requests, and as another type of agent for + type of agent for some requests and as another type of agent for others. -2.8.1. Relay Agents - Relay Agents are Diameter agents that accept requests and route - messages to other Diameter nodes based on information found in the - messages (e.g., Destination-Realm). This routing decision is - performed using a list of supported realms, and known peers. This is -Fajardo, et al. Expires July 24, 2011 [Page 29] + +Fajardo, et al. Standards Track [Page 29] -Internet-Draft Diameter Base Protocol January 2011 +RFC 6733 Diameter Base Protocol October 2012 + +2.8.1. Relay Agents - known as the Routing Table, as is defined further in Section 2.7. + Relay agents are Diameter agents that accept requests and route + messages to other Diameter nodes based on information found in the + messages (e.g., the value of the Destination-Realm AVP Section 6.6). + This routing decision is performed using a list of supported realms + and known peers. This is known as the routing table, as is defined + further in Section 2.7. Relays may, for example, be used to aggregate requests from multiple Network Access Servers (NASes) within a common geographical area - (POP). The use of Relays is advantageous since it eliminates the - need for NASes to be configured with the necessary security - information they would otherwise require to communicate with Diameter - servers in other realms. Likewise, this reduces the configuration - load on Diameter servers that would otherwise be necessary when NASes - are added, changed or deleted. + (Point of Presence, POP). The use of relays is advantageous since it + eliminates the need for NASes to be configured with the necessary + security information they would otherwise require to communicate with + Diameter servers in other realms. Likewise, this reduces the + configuration load on Diameter servers that would otherwise be + necessary when NASes are added, changed, or deleted. Relays modify Diameter messages by inserting and removing routing - information, but do not modify any other portion of a message. + information, but they do not modify any other portion of a message. Relays SHOULD NOT maintain session state but MUST maintain transaction state. @@ -1650,44 +1660,46 @@ Internet-Draft Diameter Base Protocol January 2011 Figure 2: Relaying of Diameter messages - The example provided in Figure 2 depicts a request issued from NAS, + The example provided in Figure 2 depicts a request issued from a NAS, which is an access device, for the user [email protected]. Prior to - issuing the request, NAS performs a Diameter route lookup, using + issuing the request, the NAS performs a Diameter route lookup, using "example.com" as the key, and determines that the message is to be - relayed to DRL, which is a Diameter Relay. DRL performs the same - route lookup as NAS, and relays the message to HMS, which is - example.com's Home Diameter Server. HMS identifies that the request - can be locally supported (via the realm), processes the + relayed to a DRL, which is a Diameter relay. The DRL performs the + same route lookup as the NAS, and relays the message to the HMS, + which is example.com's home server. The HMS identifies that the + request can be locally supported (via the realm), processes the authentication and/or authorization request, and replies with an - answer, which is routed back to NAS using saved transaction state. - - Since Relays do not perform any application level processing, they - provide relaying services for all Diameter applications, and - therefore MUST advertise the Relay Application Id. + answer, which is routed back to the NAS using saved transaction + state. -2.8.2. Proxy Agents + Since relays do not perform any application-level processing, they + provide relaying services for all Diameter applications; therefore, + they MUST advertise the Relay Application Id. - Similarly to relays, proxy agents route Diameter messages using the - Diameter Routing Table. However, they differ since they modify - messages to implement policy enforcement. This requires that proxies - maintain the state of their downstream peers (e.g., access devices) - to enforce resource usage, provide admission control, and - provisioning. -Fajardo, et al. Expires July 24, 2011 [Page 30] +Fajardo, et al. Standards Track [Page 30] -Internet-Draft Diameter Base Protocol January 2011 +RFC 6733 Diameter Base Protocol October 2012 + +2.8.2. Proxy Agents + + Similar to relays, proxy agents route Diameter messages using the + Diameter routing table. However, they differ since they modify + messages to implement policy enforcement. This requires that proxies + maintain the state of their downstream peers (e.g., access devices) + to enforce resource usage, provide admission control, and provide + provisioning. Proxies may, for example, be used in call control centers or access - ISPs that provide outsourced connections, they can monitor the number - and types of ports in use, and make allocation and admission - decisions according to their configuration. + ISPs that provide outsourced connections; they can monitor the number + and type of ports in use and make allocation and admission decisions + according to their configuration. Since enforcing policies requires an understanding of the service - being provided, Proxies MUST only advertise the Diameter applications + being provided, proxies MUST only advertise the Diameter applications they support. 2.8.3. Redirect Agents @@ -1709,18 +1721,12 @@ Internet-Draft Diameter Base Protocol January 2011 The example provided in Figure 3 depicts a request issued from the access device, NAS, for the user [email protected]. The message is forwarded by the NAS to its relay, DRL, which does not have a routing - entry in its Diameter Routing Table for example.com. DRL has a + entry in its Diameter routing table for example.com. The DRL has a default route configured to DRD, which is a redirect agent that - returns a redirect notification to DRL, as well as HMS' contact - information. Upon receipt of the redirect notification, DRL - establishes a transport connection with HMS, if one doesn't already - exist, and forwards the request to it. - - - - - - + returns a redirect notification to DRL, as well as the HMS' contact + information. Upon receipt of the redirect notification, the DRL + establishes a transport connection with the HMS, if one doesn't + already exist, and forwards the request to it. @@ -1729,12 +1735,9 @@ Internet-Draft Diameter Base Protocol January 2011 - - - -Fajardo, et al. Expires July 24, 2011 [Page 31] +Fajardo, et al. Standards Track [Page 31] -Internet-Draft Diameter Base Protocol January 2011 +RFC 6733 Diameter Base Protocol October 2012 +------+ @@ -1755,10 +1758,10 @@ Internet-Draft Diameter Base Protocol January 2011 Figure 3: Redirecting a Diameter Message - Since redirect agents do not perform any application level + Since redirect agents do not perform any application-level processing, they provide relaying services for all Diameter - applications, and therefore MUST advertise the Relay Application - Identifier. + applications; therefore, they MUST advertise the Relay Application + ID. 2.8.4. Translation Agents @@ -1773,7 +1776,7 @@ Internet-Draft Diameter Base Protocol January 2011 MUST maintain transaction state. Translation of messages can only occur if the agent recognizes the - application of a particular request, and therefore translation agents + application of a particular request; therefore, translation agents MUST only advertise their locally supported applications. +------+ ---------> +------+ ---------> +------+ @@ -1788,23 +1791,24 @@ Internet-Draft Diameter Base Protocol January 2011 -Fajardo, et al. Expires July 24, 2011 [Page 32] +Fajardo, et al. Standards Track [Page 32] -Internet-Draft Diameter Base Protocol January 2011 +RFC 6733 Diameter Base Protocol October 2012 2.9. Diameter Path Authorization - As noted in Section 2.2, Diameter provides transmission level + As noted in Section 2.2, Diameter provides transmission-level security for each connection using TLS/TCP and DTLS/SCTP. Therefore, - each connection can be authenticated, replay and integrity protected. + each connection can be authenticated and can be replay and integrity + protected. - In addition to authenticating each connection, each connection as - well as the entire session MUST also be authorized. Before - initiating a connection, a Diameter Peer MUST check that its peers - are authorized to act in their roles. For example, a Diameter peer - may be authentic, but that does not mean that it is authorized to act - as a Diameter Server advertising a set of Diameter applications. + In addition to authenticating each connection, the entire session + MUST also be authorized. Before initiating a connection, a Diameter + peer MUST check that its peers are authorized to act in their roles. + For example, a Diameter peer may be authentic, but that does not mean + that it is authorized to act as a Diameter server advertising a set + of Diameter applications. Prior to bringing up a connection, authorization checks are performed at each connection along the path. Diameter capabilities negotiation @@ -1815,7 +1819,7 @@ Internet-Draft Diameter Base Protocol January 2011 As noted in Section 6.1.9, a relay or proxy agent MUST append a Route-Record AVP to all requests forwarded. The AVP contains the - identity of the peer the request was received from. + identity of the peer from which the request was received. The home Diameter server, prior to authorizing a session, MUST check the Route-Record AVPs to make sure that the route traversed by the @@ -1823,7 +1827,7 @@ Internet-Draft Diameter Base Protocol January 2011 realm may not wish to honor requests that have been routed through an untrusted realm. By authorizing a request, the home Diameter server is implicitly indicating its willingness to engage in the business - transaction as specified by the contractual relationship between the + transaction as specified by any contractual relationship between the server and the previous hop. A DIAMETER_AUTHORIZATION_REJECTED error message (see Section 7.1.5) is sent if the route traversed by the request is unacceptable. @@ -1839,72 +1843,24 @@ Internet-Draft Diameter Base Protocol January 2011 willingness to take on financial risk relative to the session. A local realm may wish to limit this exposure, for example, by establishing credit limits for intermediate realms and refusing to - accept responses which would violate those limits. By issuing an - accounting request corresponding to the authorization response, the + accept responses that would violate those limits. By issuing an -Fajardo, et al. Expires July 24, 2011 [Page 33] +Fajardo, et al. Standards Track [Page 33] -Internet-Draft Diameter Base Protocol January 2011 +RFC 6733 Diameter Base Protocol October 2012 + accounting request corresponding to the authorization response, the local realm implicitly indicates its agreement to provide the service indicated in the authorization response. If the service cannot be provided by the local realm, then a DIAMETER_UNABLE_TO_COMPLY error message MUST be sent within the accounting request; a Diameter client receiving an authorization response for a service that it cannot - perform MUST NOT substitute an alternate service, and then send + perform MUST NOT substitute an alternate service and then send accounting requests for the alternate service instead. - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - -Fajardo, et al. Expires July 24, 2011 [Page 34] - -Internet-Draft Diameter Base Protocol January 2011 - - 3. Diameter Header A summary of the Diameter header format is shown below. The fields @@ -1915,7 +1871,7 @@ Internet-Draft Diameter Base Protocol January 2011 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ | Version | Message Length | +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ - | command flags | Command-Code | + | Command Flags | Command Code | +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ | Application-ID | +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ @@ -1935,13 +1891,23 @@ Internet-Draft Diameter Base Protocol January 2011 The Message Length field is three octets and indicates the length of the Diameter message including the header fields and the padded - AVPs. Thus the message length field is always a multiple of 4. + AVPs. Thus, the Message Length field is always a multiple of 4. Command Flags The Command Flags field is eight bits. The following bits are assigned: + + + + + +Fajardo, et al. Standards Track [Page 34] + +RFC 6733 Diameter Base Protocol October 2012 + + 0 1 2 3 4 5 6 7 +-+-+-+-+-+-+-+-+ |R P E T r r r r| @@ -1952,127 +1918,113 @@ Internet-Draft Diameter Base Protocol January 2011 If set, the message is a request. If cleared, the message is an answer. - - - - -Fajardo, et al. Expires July 24, 2011 [Page 35] - -Internet-Draft Diameter Base Protocol January 2011 - - P(roxiable) - If set, the message MAY be proxied, relayed or redirected. If + If set, the message MAY be proxied, relayed, or redirected. If cleared, the message MUST be locally processed. - E(rror) If set, the message contains a protocol error, and the message - will not conform to the ABNF described for this command. + will not conform to the CCF described for this command. Messages with the 'E' bit set are commonly referred to as error - messages. This bit MUST NOT be set in request messages. See - Section 7.2. + messages. This bit MUST NOT be set in request messages (see + Section 7.2). - - T(Potentially re-transmitted message) + T(Potentially retransmitted message) This flag is set after a link failover procedure, to aid the removal of duplicate requests. It is set when resending requests not yet acknowledged, as an indication of a possible duplicate due to a link failure. This bit MUST be cleared when - sending a request for the first time, otherwise the sender MUST - set this flag. Diameter agents only need to be concerned about - the number of requests they send based on a single received - request; retransmissions by other entities need not be tracked. - Diameter agents that receive a request with the T flag set, - MUST keep the T flag set in the forwarded request. This flag - MUST NOT be set if an error answer message (e.g., a protocol - error) has been received for the earlier message. It can be - set only in cases where no answer has been received from the - server for a request and the request is sent again. This flag - MUST NOT be set in answer messages. - + sending a request for the first time; otherwise, the sender + MUST set this flag. Diameter agents only need to be concerned + about the number of requests they send based on a single + received request; retransmissions by other entities need not be + tracked. Diameter agents that receive a request with the T + flag set, MUST keep the T flag set in the forwarded request. + This flag MUST NOT be set if an error answer message (e.g., a + protocol error) has been received for the earlier message. It + can be set only in cases where no answer has been received from + the server for a request, and the request has been sent again. + This flag MUST NOT be set in answer messages. r(eserved) - These flag bits are reserved for future use, and MUST be set to - zero, and ignored by the receiver. + These flag bits are reserved for future use; they MUST be set + to zero and ignored by the receiver. - Command-Code - The Command-Code field is three octets, and is used in order to - communicate the command associated with the message. The 24-bit - address space is managed by IANA (see Section 11.2.1). - Command-Code values 16,777,214 and 16,777,215 (hexadecimal values - FFFFFE -FFFFFF) are reserved for experimental use (See Section - 11.3). -Fajardo, et al. Expires July 24, 2011 [Page 36] +Fajardo, et al. Standards Track [Page 35] -Internet-Draft Diameter Base Protocol January 2011 +RFC 6733 Diameter Base Protocol October 2012 + + + Command Code + The Command Code field is three octets and is used in order to + communicate the command associated with the message. The 24-bit + address space is managed by IANA (see Section 3.1). Command Code + values 16,777,214 and 16,777,215 (hexadecimal values FFFFFE- + FFFFFF) are reserved for experimental use (see Section 11.2). - Application-ID + Application-ID - Application-ID is four octets and is used to identify to which - application the message is applicable for. The application can be - an authentication application, an accounting application or a - vendor specific application. See Section 11.3 for the possible - values that the application-id may use. + Application-ID is four octets and is used to identify for which + application the message is applicable. The application can be an + authentication application, an accounting application, or a + vendor-specific application. - The value of the application-id field in the header MUST be the - same as any relevant application-id AVPs contained in the message. + The value of the Application-ID field in the header MUST be the + same as any relevant Application-Id AVPs contained in the message. Hop-by-Hop Identifier The Hop-by-Hop Identifier is an unsigned 32-bit integer field (in - network byte order) and aids in matching requests and replies. - The sender MUST ensure that the Hop-by-Hop identifier in a request - is unique on a given connection at any given time, and MAY attempt - to ensure that the number is unique across reboots. The sender of - an Answer message MUST ensure that the Hop-by-Hop Identifier field - contains the same value that was found in the corresponding - request. The Hop-by-Hop identifier is normally a monotonically - increasing number, whose start value was randomly generated. An - answer message that is received with an unknown Hop-by-Hop - Identifier MUST be discarded. - + network byte order) that aids in matching requests and replies. + The sender MUST ensure that the Hop-by-Hop Identifier in a request + is unique on a given connection at any given time, and it MAY + attempt to ensure that the number is unique across reboots. The + sender of an answer message MUST ensure that the Hop-by-Hop + Identifier field contains the same value that was found in the + corresponding request. The Hop-by-Hop Identifier is normally a + monotonically increasing number, whose start value was randomly + generated. An answer message that is received with an unknown + Hop-by-Hop Identifier MUST be discarded. End-to-End Identifier The End-to-End Identifier is an unsigned 32-bit integer field (in - network byte order) and is used to detect duplicate messages. - Upon reboot implementations MAY set the high order 12 bits to + network byte order) that is used to detect duplicate messages. + Upon reboot, implementations MAY set the high order 12 bits to contain the low order 12 bits of current time, and the low order 20 bits to a random value. Senders of request messages MUST insert a unique identifier on each message. The identifier MUST remain locally unique for a period of at least 4 minutes, even - across reboots. The originator of an Answer message MUST ensure + across reboots. The originator of an answer message MUST ensure that the End-to-End Identifier field contains the same value that was found in the corresponding request. The End-to-End Identifier MUST NOT be modified by Diameter agents of any kind. The - combination of the Origin-Host (see Section 6.3) and this field is + combination of the Origin-Host AVP (Section 6.3) and this field is used to detect duplicates. Duplicate requests SHOULD cause the - same answer to be transmitted (modulo the hop-by-hop Identifier - field and any routing AVPs that may be present), and MUST NOT - affect any state that was set when the original request was - processed. Duplicate answer messages that are to be locally - consumed (see Section 6.2) SHOULD be silently discarded. - - + same answer to be transmitted (modulo the Hop-by-Hop Identifier -Fajardo, et al. Expires July 24, 2011 [Page 37] +Fajardo, et al. Standards Track [Page 36] -Internet-Draft Diameter Base Protocol January 2011 +RFC 6733 Diameter Base Protocol October 2012 + field and any routing AVPs that may be present), and they MUST NOT + affect any state that was set when the original request was + processed. Duplicate answer messages that are to be locally + consumed (see Section 6.2) SHOULD be silently discarded. + AVPs AVPs are a method of encapsulating information relevant to the @@ -2080,17 +2032,17 @@ Internet-Draft Diameter Base Protocol January 2011 3.1. Command Codes - Each command Request/Answer pair is assigned a command code, and the + Each command Request/Answer pair is assigned a Command Code, and the sub-type (i.e., request or answer) is identified via the 'R' bit in the Command Flags field of the Diameter header. - - Every Diameter message MUST contain a command code in its header's - Command-Code field, which is used to determine the action that is to + Every Diameter message MUST contain a Command Code in its header's + Command Code field, which is used to determine the action that is to be taken for a particular message. The following Command Codes are defined in the Diameter base protocol: - Command-Name Abbrev. Code Reference + Section + Command Name Abbrev. Code Reference -------------------------------------------------------- Abort-Session-Request ASR 274 8.5.1 Abort-Session-Answer ASA 274 8.5.2 @@ -2111,140 +2063,142 @@ Internet-Draft Diameter Base Protocol January 2011 Session-Termination- STA 275 8.4.2 Answer -3.2. Command Code ABNF specification - Every Command Code defined MUST include a corresponding ABNF - specification, which is used to define the AVPs that MUST or MAY be - present when sending the message. The following format is used in - the definition: - command-def = <command-name> "::=" diameter-message - command-name = diameter-name -Fajardo, et al. Expires July 24, 2011 [Page 38] - -Internet-Draft Diameter Base Protocol January 2011 - diameter-name = ALPHA *(ALPHA / DIGIT / "-") +Fajardo, et al. Standards Track [Page 37] + +RFC 6733 Diameter Base Protocol October 2012 - diameter-message = header [ *fixed] [ *required] [ *optional] - header = "<" "Diameter Header:" command-id - [r-bit] [p-bit] [e-bit] [application-id] ">" +3.2. Command Code Format Specification - application-id = 1*DIGIT + Every Command Code defined MUST include a corresponding Command Code + Format (CCF) specification, which is used to define the AVPs that + MUST or MAY be present when sending the message. The following ABNF + specifies the CCF used in the definition: - command-id = 1*DIGIT - ; The Command Code assigned to the command + command-def = "<" command-name ">" "::=" diameter-message - r-bit = ", REQ" - ; If present, the 'R' bit in the Command - ; Flags is set, indicating that the message - ; is a request, as opposed to an answer. + command-name = diameter-name - p-bit = ", PXY" - ; If present, the 'P' bit in the Command - ; Flags is set, indicating that the message - ; is proxiable. + diameter-name = ALPHA *(ALPHA / DIGIT / "-") - e-bit = ", ERR" - ; If present, the 'E' bit in the Command - ; Flags is set, indicating that the answer - ; message contains a Result-Code AVP in - ; the "protocol error" class. + diameter-message = header *fixed *required *optional - fixed = [qual] "<" avp-spec ">" - ; Defines the fixed position of an AVP + header = "<Diameter-Header:" command-id + [r-bit] [p-bit] [e-bit] [application-id]">" - required = [qual] "{" avp-spec "}" - ; The AVP MUST be present and can appear - ; anywhere in the message. + application-id = 1*DIGIT + command-id = 1*DIGIT + ; The Command Code assigned to the command. - optional = [qual] "[" avp-name "]" - ; The avp-name in the 'optional' rule cannot - ; evaluate to any AVP Name which is included - ; in a fixed or required rule. The AVP can - ; appear anywhere in the message. - ; - ; NOTE: "[" and "]" have a slightly different - ; meaning than in ABNF (RFC 5234]). These braces - ; cannot be used to express optional fixed rules - ; (such as an optional ICV at the end). To do this, - ; the convention is '0*1fixed'. + r-bit = ", REQ" + ; If present, the 'R' bit in the Command + ; Flags is set, indicating that the message + ; is a request as opposed to an answer. + p-bit = ", PXY" + ; If present, the 'P' bit in the Command + ; Flags is set, indicating that the message + ; is proxiable. + e-bit = ", ERR" + ; If present, the 'E' bit in the Command + ; Flags is set, indicating that the answer + ; message contains a Result-Code AVP in + ; the "protocol error" class. + fixed = [qual] "<" avp-spec ">" + ; Defines the fixed position of an AVP. -Fajardo, et al. Expires July 24, 2011 [Page 39] - -Internet-Draft Diameter Base Protocol January 2011 + required = [qual] "{" avp-spec "}" + ; The AVP MUST be present and can appear + ; anywhere in the message. - qual = [min] "*" [max] - ; See ABNF conventions, RFC 5234 Section 4. - ; The absence of any qualifiers depends on - ; whether it precedes a fixed, required, or - ; optional rule. If a fixed or required rule has - ; no qualifier, then exactly one such AVP MUST - ; be present. If an optional rule has no - ; qualifier, then 0 or 1 such AVP may be - ; present. If an optional rule has a qualifier, - ; then the value of min MUST be 0 if present. - min = 1*DIGIT - ; The minimum number of times the element may - ; be present. If absent, the default value is zero - ; for fixed and optional rules and one for required - ; rules. The value MUST be at least one for for - ; required rules. - max = 1*DIGIT - ; The maximum number of times the element may - ; be present. If absent, the default value is - ; infinity. A value of zero implies the AVP MUST - ; NOT be present. - avp-spec = diameter-name - ; The avp-spec has to be an AVP Name, defined - ; in the base or extended Diameter - ; specifications. - - avp-name = avp-spec / "AVP" - ; The string "AVP" stands for *any* arbitrary AVP - ; Name, not otherwise listed in that command code - ; definition. Addition this AVP is recommended for - ; all command ABNFs to allow for extensibility. +Fajardo, et al. Standards Track [Page 38] + +RFC 6733 Diameter Base Protocol October 2012 - The following is a definition of a fictitious command code: + optional = [qual] "[" avp-name "]" + ; The avp-name in the 'optional' rule cannot + ; evaluate to any AVP Name that is included + ; in a fixed or required rule. The AVP can + ; appear anywhere in the message. + ; + ; NOTE: "[" and "]" have a slightly different + ; meaning than in ABNF. These braces + ; cannot be used to express optional fixed rules + ; (such as an optional ICV at the end). To do + ; this, the convention is '0*1fixed'. - Example-Request ::= < Diameter Header: 9999999, REQ, PXY > - { User-Name } - * { Origin-Host } - * [ AVP ] + qual = [min] "*" [max] + ; See ABNF conventions, RFC 5234, Section 4. + ; The absence of any qualifier depends on + ; whether it precedes a fixed, required, or + ; optional rule. If a fixed or required rule has + ; no qualifier, then exactly one such AVP MUST + ; be present. If an optional rule has no + ; qualifier, then 0 or 1 such AVP may be + ; present. If an optional rule has a qualifier, + ; then the value of min MUST be 0 if present. + min = 1*DIGIT + ; The minimum number of times the element may + ; be present. If absent, the default value is 0 + ; for fixed and optional rules and 1 for + ; required rules. The value MUST be at least 1 + ; for required rules. + max = 1*DIGIT + ; The maximum number of times the element may + ; be present. If absent, the default value is + ; infinity. A value of 0 implies the AVP MUST + ; NOT be present. + avp-spec = diameter-name + ; The avp-spec has to be an AVP Name, defined + ; in the base or extended Diameter + ; specifications. + avp-name = avp-spec / "AVP" + ; The string "AVP" stands for *any* arbitrary AVP + ; Name, not otherwise listed in that Command Code + ; definition. The inclusion of this string + ; is recommended for all CCFs to allow for + ; extensibility. -Fajardo, et al. Expires July 24, 2011 [Page 40] +Fajardo, et al. Standards Track [Page 39] -Internet-Draft Diameter Base Protocol January 2011 +RFC 6733 Diameter Base Protocol October 2012 + + + The following is a definition of a fictitious Command Code: + Example-Request ::= < Diameter Header: 9999999, REQ, PXY > + { User-Name } + 1* { Origin-Host } + * [ AVP ] 3.3. Diameter Command Naming Conventions Diameter command names typically includes one or more English words - followed by the verb Request or Answer. Each English word is + followed by the verb "Request" or "Answer". Each English word is delimited by a hyphen. A three-letter acronym for both the request and answer is also normally provided. @@ -2253,10 +2207,10 @@ Internet-Draft Diameter Base Protocol January 2011 the acronyms are STR and STA, respectively. Both the request and the answer for a given command share the same - command code. The request is identified by the R(equest) bit in the + Command Code. The request is identified by the R(equest) bit in the Diameter header set to one (1), to ask that a particular action be performed, such as authorizing a user or terminating a session. Once - the receiver has completed the request it issues the corresponding + the receiver has completed the request, it issues the corresponding answer, which includes a result code that communicates one of the following: @@ -2274,38 +2228,25 @@ Internet-Draft Diameter Base Protocol January 2011 Additional information, encoded within AVPs, may also be included in answer messages. +4. Diameter AVPs + Diameter AVPs carry specific authentication, accounting, + authorization, and routing information as well as configuration + details for the request and reply. - - - - - - - - - - - -Fajardo, et al. Expires July 24, 2011 [Page 41] +Fajardo, et al. Standards Track [Page 40] -Internet-Draft Diameter Base Protocol January 2011 - - -4. Diameter AVPs +RFC 6733 Diameter Base Protocol October 2012 - Diameter AVPs carry specific authentication, accounting, - authorization and routing information as well as configuration - details for the request and reply. Each AVP of type OctetString MUST be padded to align on a 32-bit boundary, while other AVP types align naturally. A number of zero- - valued bytes are added to the end of the AVP Data field till a word + valued bytes are added to the end of the AVP Data field until a word boundary is reached. The length of the padding is not reflected in the AVP Length field. @@ -2330,92 +2271,96 @@ Internet-Draft Diameter Base Protocol January 2011 The AVP Code, combined with the Vendor-Id field, identifies the attribute uniquely. AVP numbers 1 through 255 are reserved for - re-use of RADIUS attributes, without setting the Vendor-Id field. + reuse of RADIUS attributes, without setting the Vendor-Id field. AVP numbers 256 and above are used for Diameter, which are - allocated by IANA (see Section 11.1). - + allocated by IANA (see Section 11.1.1). AVP Flags The AVP Flags field informs the receiver how each attribute must - be handled. The 'r' (reserved) bits are unused and SHOULD be set - to 0. Note that subsequent Diameter applications MAY define - additional bits within the AVP Header, and an unrecognized bit - SHOULD be considered an error. The 'P' bit has been reserved for - future usage of end-to-end security. At the time of writing there - are no end-to-end security mechanisms specified therefore the 'P' - bit SHOULD be set to 0. + be handled. New Diameter applications SHOULD NOT define + additional AVP Flag bits. However, note that new Diameter + applications MAY define additional bits within the AVP header, and + an unrecognized bit SHOULD be considered an error. The sender of + the AVP MUST set 'R' (reserved) bits to 0 and the receiver SHOULD + ignore all 'R' (reserved) bits. The 'P' bit has been reserved for + future usage of end-to-end security. At the time of writing, + there are no end-to-end security mechanisms specified; therefore, + the 'P' bit SHOULD be set to 0. + + The 'M' bit, known as the Mandatory bit, indicates whether the + receiver of the AVP MUST parse and understand the semantics of the + AVP including its content. The receiving entity MUST return an + appropriate error message if it receives an AVP that has the M-bit -Fajardo, et al. Expires July 24, 2011 [Page 42] +Fajardo, et al. Standards Track [Page 41] -Internet-Draft Diameter Base Protocol January 2011 +RFC 6733 Diameter Base Protocol October 2012 - The 'M' Bit, known as the Mandatory bit, indicates whether the - receiver of the AVP MUST parse and understand the semantic of the - AVP including its content. The receiving entity MUST return an - appropriate error message if it receives an AVP that has the M-bit set but does not understand it. An exception applies when the AVP is embedded within a Grouped AVP. See Section 4.4 for details. - Diameter Relay and redirect agents MUST NOT reject messages with + Diameter relay and redirect agents MUST NOT reject messages with unrecognized AVPs. The 'M' bit MUST be set according to the rules defined in the - application specification which introduces or re-uses this AVP. - Within a given application, the M-bit setting for an AVP is either - defined for all command types or for each command type. + application specification that introduces or reuses this AVP. + Within a given application, the M-bit setting for an AVP is + defined either for all command types or for each command type. - AVPs with the 'M' bit cleared are informational only and a - receiver that receives a message with such an AVP that is not - supported, or whose value is not supported, MAY simply ignore the - AVP. + AVPs with the 'M' bit cleared are informational only; a receiver + that receives a message with such an AVP that is not supported, or + whose value is not supported, MAY simply ignore the AVP. The 'V' bit, known as the Vendor-Specific bit, indicates whether the optional Vendor-ID field is present in the AVP header. When - set the AVP Code belongs to the specific vendor code address + set, the AVP Code belongs to the specific vendor code address space. AVP Length The AVP Length field is three octets, and indicates the number of - octets in this AVP including the AVP Code, AVP Length, AVP Flags, - Vendor-ID field (if present) and the AVP data. If a message is - received with an invalid attribute length, the message MUST be - rejected. + octets in this AVP including the AVP Code field, AVP Length field, + AVP Flags field, Vendor-ID field (if present), and the AVP Data + field. If a message is received with an invalid attribute length, + the message MUST be rejected. 4.1.1. Optional Header Elements - The AVP Header contains one optional field. This field is only + The AVP header contains one optional field. This field is only present if the respective bit-flag is enabled. - Vendor-ID The Vendor-ID field is present if the 'V' bit is set in the AVP Flags field. The optional four-octet Vendor-ID field contains the - IANA assigned "SMI Network Management Private Enterprise Codes" - [RFC3232] value, encoded in network byte order. Any vendor or - standardization organization that are also treated like vendors in - the IANA managed "SMI Network Management Private Enterprise Codes" - space wishing to implement a vendor-specific Diameter AVP MUST use - their own Vendor-ID along with their privately managed AVP address + IANA-assigned "SMI Network Management Private Enterprise Codes" + [ENTERPRISE] value, encoded in network byte order. Any vendors or + standardization organizations that are also treated like vendors + in the IANA-managed "SMI Network Management Private Enterprise + Codes" space wishing to implement a vendor-specific Diameter AVP + MUST use their own Vendor-ID along with their privately managed + AVP address space, guaranteeing that they will not collide with + any other vendor's vendor-specific AVP(s) or with future IETF + AVPs. -Fajardo, et al. Expires July 24, 2011 [Page 43] - -Internet-Draft Diameter Base Protocol January 2011 - space, guaranteeing that they will not collide with any other - vendor's vendor-specific AVP(s), nor with future IETF AVPs. - A vendor ID value of zero (0) corresponds to the IETF adopted AVP - values, as managed by the IANA. Since the absence of the vendor - ID field implies that the AVP in question is not vendor specific, - implementations MUST NOT use the zero (0) vendor ID. +Fajardo, et al. Standards Track [Page 42] + +RFC 6733 Diameter Base Protocol October 2012 + + + A Vendor-ID value of zero (0) corresponds to the IETF-adopted AVP + values, as managed by IANA. Since the absence of the Vendor-ID + field implies that the AVP in question is not vendor specific, + implementations MUST NOT use the value of zero (0) for the + Vendor-ID field. 4.2. Basic AVP Data Formats @@ -2426,49 +2371,45 @@ Internet-Draft Diameter Base Protocol January 2011 type derived from the base data types. In the event that a new Basic AVP Data Format is needed, a new version of this RFC MUST be created. - OctetString The data contains arbitrary data of variable length. Unless otherwise noted, the AVP Length field MUST be set to at least 8 - (12 if the 'V' bit is enabled). AVP Values of this type that are - not a multiple of four-octets in length is followed by the - necessary padding so that the next AVP (if any) will start on a - 32-bit boundary. - + (12 if the 'V' bit is enabled). AVP values of this type that are + not a multiple of 4 octets in length are followed by the necessary + padding so that the next AVP (if any) will start on a 32-bit + boundary. Integer32 - 32 bit signed value, in network byte order. The AVP Length field + 32-bit signed value, in network byte order. The AVP Length field MUST be set to 12 (16 if the 'V' bit is enabled). - Integer64 - 64 bit signed value, in network byte order. The AVP Length field + 64-bit signed value, in network byte order. The AVP Length field MUST be set to 16 (20 if the 'V' bit is enabled). - Unsigned32 - 32 bit unsigned value, in network byte order. The AVP Length + 32-bit unsigned value, in network byte order. The AVP Length field MUST be set to 12 (16 if the 'V' bit is enabled). + Unsigned64 + 64-bit unsigned value, in network byte order. The AVP Length + field MUST be set to 16 (20 if the 'V' bit is enabled). -Fajardo, et al. Expires July 24, 2011 [Page 44] - -Internet-Draft Diameter Base Protocol January 2011 - Unsigned64 - 64 bit unsigned value, in network byte order. The AVP Length - field MUST be set to 16 (20 if the 'V' bit is enabled). +Fajardo, et al. Standards Track [Page 43] + +RFC 6733 Diameter Base Protocol October 2012 Float32 @@ -2478,7 +2419,6 @@ Internet-Draft Diameter Base Protocol January 2011 network byte order. The AVP Length field MUST be set to 12 (16 if the 'V' bit is enabled). - Float64 This represents floating point values of double precision as @@ -2486,73 +2426,69 @@ Internet-Draft Diameter Base Protocol January 2011 network byte order. The AVP Length field MUST be set to 16 (20 if the 'V' bit is enabled). - Grouped - The Data field is specified as a sequence of AVPs. Each of these - AVPs follows - in the order in which they are specified - - including their headers and padding. The AVP Length field is set - to 8 (12 if the 'V' bit is enabled) plus the total length of all - included AVPs, including their headers and padding. Thus the AVP - length field of an AVP of type Grouped is always a multiple of 4. - + The Data field is specified as a sequence of AVPs. These AVPs are + concatenated -- including their headers and padding -- in the + order in which they are specified and the result encapsulated in + the Data field. The AVP Length field is set to 8 (12 if the 'V' + bit is enabled) plus the total length of all included AVPs, + including their headers and padding. Thus, the AVP Length field + of an AVP of type Grouped is always a multiple of 4. 4.3. Derived AVP Data Formats In addition to using the Basic AVP Data Formats, applications may define data formats derived from the Basic AVP Data Formats. An application that defines new Derived AVP Data Formats MUST include - them in a section entitled "Derived AVP Data Formats", using the same + them in a section titled "Derived AVP Data Formats", using the same format as the definitions below. Each new definition MUST be either defined or listed with a reference to the RFC that defines the format. -4.3.1. Common Derived AVPs +4.3.1. Common Derived AVP Data Formats The following are commonly used Derived AVP Data Formats. + Address + The Address format is derived from the OctetString Basic AVP + Format. It is a discriminated union representing, for example, a + 32-bit (IPv4) [RFC0791] or 128-bit (IPv6) [RFC4291] address, most + significant octet first. The first two octets of the Address AVP + represent the AddressType, which contains an Address Family, + defined in [IANAADFAM]. The AddressType is used to discriminate + the content and format of the remaining octets. -Fajardo, et al. Expires July 24, 2011 [Page 45] +Fajardo, et al. Standards Track [Page 44] -Internet-Draft Diameter Base Protocol January 2011 - - - Address - - The Address format is derived from the OctetString AVP Base - Format. It is a discriminated union, representing, for example a - 32-bit (IPv4) [RFC791] or 128-bit (IPv6) [RFC4291] address, most - significant octet first. The first two octets of the Address AVP - represents the AddressType, which contains an Address Family - defined in [IANAADFAM]. The AddressType is used to discriminate - the content and format of the remaining octets. +RFC 6733 Diameter Base Protocol October 2012 Time - The Time format is derived from the OctetString AVP Base Format. + The Time format is derived from the OctetString Basic AVP Format. The string MUST contain four octets, in the same format as the first four bytes are in the NTP timestamp format. The NTP - Timestamp format is defined in Chapter 3 of [RFC5905]. + timestamp format is defined in Section 3 of [RFC5905]. This represents the number of seconds since 0h on 1 January 1900 with respect to the Coordinated Universal Time (UTC). - On 6h 28m 16s UTC, 7 February 2036 the time value will overflow. - SNTP [RFC5905] describes a procedure to extend the time to 2104. - This procedure MUST be supported by all Diameter nodes. - + On 6h 28m 16s UTC, 7 February 2036, the time value will overflow. + Simple Network Time Protocol (SNTP) [RFC5905] describes a + procedure to extend the time to 2104. This procedure MUST be + supported by all Diameter nodes. UTF8String - The UTF8String format is derived from the OctetString AVP Base - Format. This is a human readable string represented using the + The UTF8String format is derived from the OctetString Basic AVP + Format. This is a human-readable string represented using the ISO/IEC IS 10646-1 character set, encoded as an OctetString using - the UTF-8 [RFC3629] transformation format described in RFC 3629. + the UTF-8 transformation format [RFC3629]. Since additional code points are added by amendments to the 10646 standard from time to time, implementations MUST be prepared to @@ -2570,78 +2506,79 @@ Internet-Draft Diameter Base Protocol January 2011 or software, an alternative means of entry and display, such as hexadecimal, MAY be provided. - - -Fajardo, et al. Expires July 24, 2011 [Page 46] - -Internet-Draft Diameter Base Protocol January 2011 - - For information encoded in 7-bit US-ASCII, the UTF-8 charset is identical to the US-ASCII charset. UTF-8 may require multiple bytes to represent a single character / - code point; thus the length of an UTF8String in octets may be + code point; thus, the length of a UTF8String in octets may be different from the number of characters encoded. Note that the AVP Length field of an UTF8String is measured in - octets, not characters. + octets not characters. + + + + +Fajardo, et al. Standards Track [Page 45] + +RFC 6733 Diameter Base Protocol October 2012 + DiameterIdentity - The DiameterIdentity format is derived from the OctetString AVP - Base Format. + The DiameterIdentity format is derived from the OctetString Basic + AVP Format. DiameterIdentity = FQDN/Realm - - DiameterIdentity value is used to uniquely identify either: + The DiameterIdentity value is used to uniquely identify either: * A Diameter node for purposes of duplicate connection and routing loop detection. - * A Realm to determine whether messages can be satisfied locally, + * A Realm to determine whether messages can be satisfied locally or whether they must be routed or redirected. + When a DiameterIdentity value is used to identify a Diameter node, + the contents of the string MUST be the Fully Qualified Domain Name + (FQDN) of the Diameter node. If multiple Diameter nodes run on + the same host, each Diameter node MUST be assigned a unique + DiameterIdentity. If a Diameter node can be identified by several + FQDNs, a single FQDN should be picked at startup and used as the + only DiameterIdentity for that node, whatever the connection on + which it is sent. In this document, note that DiameterIdentity is + in ASCII form in order to be compatible with existing DNS + infrastructure. See Appendix D for interactions between the + Diameter protocol and Internationalized Domain Names (IDNs). - When a DiameterIdentity is used to identify a Diameter node the - contents of the string MUST be the FQDN of the Diameter node. If - multiple Diameter nodes run on the same host, each Diameter node - MUST be assigned a unique DiameterIdentity. If a Diameter node - can be identified by several FQDNs, a single FQDN should be picked - at startup, and used as the only DiameterIdentity for that node, - whatever the connection it is sent on. Note that in this - document, DiameterIdentity is in ASCII form in order to be - compatible with existing DNS infrastructure. See Appendix D for - interactions between the Diameter protocol and Internationalized - Domain Name (IDNs). + DiameterURI + The DiameterURI MUST follow the Uniform Resource Identifiers (RFC + 3986) syntax [RFC3986] rules specified below: - DiameterURI + "aaa://" FQDN [ port ] [ transport ] [ protocol ] - The DiameterURI MUST follow the Uniform Resource Identifiers (URI) - syntax [RFC3986] rules specified below: + ; No transport security + "aaas://" FQDN [ port ] [ transport ] [ protocol ] + ; Transport security used + FQDN = < Fully Qualified Domain Name > -Fajardo, et al. Expires July 24, 2011 [Page 47] - -Internet-Draft Diameter Base Protocol January 2011 - "aaa://" FQDN [ port ] [ transport ] [ protocol ] - ; No transport security - "aaas://" FQDN [ port ] [ transport ] [ protocol ] - ; Transport security used - FQDN = Fully Qualified Host Name +Fajardo, et al. Standards Track [Page 46] + +RFC 6733 Diameter Base Protocol October 2012 + port = ":" 1*DIGIT @@ -2649,8 +2586,9 @@ Internet-Draft Diameter Base Protocol January 2011 ; incoming connections. ; If absent, the default Diameter port ; (3868) is assumed if no transport - ; security is used and port (TBD) when - ; transport security (TLS/TCP and DTLS/SCTP) is used. + ; security is used and port 5658 when + ; transport security (TLS/TCP and DTLS/SCTP) + ; is used. transport = ";transport=" transport-protocol @@ -2660,46 +2598,47 @@ Internet-Draft Diameter Base Protocol January 2011 ; UDP MUST NOT be used when the aaa-protocol ; field is set to diameter. - transport-protocol = ( "tcp" / "sctp" / "udp" ) + transport-protocol = ( "tcp" / "sctp" / "udp" ) - protocol = ";protocol=" aaa-protocol + protocol = ";protocol=" aaa-protocol ; If absent, the default AAA protocol ; is Diameter. - aaa-protocol = ( "diameter" / "radius" / "tacacs+" ) + aaa-protocol = ( "diameter" / "radius" / "tacacs+" ) - The following are examples of valid Diameter host identities: + The following are examples of valid Diameter host identities: - aaa://host.example.com;transport=tcp - aaa://host.example.com:6666;transport=tcp - aaa://host.example.com;protocol=diameter - aaa://host.example.com:6666;protocol=diameter - aaa://host.example.com:6666;transport=tcp;protocol=diameter - aaa://host.example.com:1813;transport=udp;protocol=radius + aaa://host.example.com;transport=tcp + aaa://host.example.com:6666;transport=tcp + aaa://host.example.com;protocol=diameter + aaa://host.example.com:6666;protocol=diameter + aaa://host.example.com:6666;transport=tcp;protocol=diameter + aaa://host.example.com:1813;transport=udp;protocol=radius + Enumerated + The Enumerated format is derived from the Integer32 Basic AVP + Format. The definition contains a list of valid values and their + interpretation and is described in the Diameter application + introducing the AVP. -Fajardo, et al. Expires July 24, 2011 [Page 48] - -Internet-Draft Diameter Base Protocol January 2011 - Enumerated - Enumerated is derived from the Integer32 AVP Base Format. The - definition contains a list of valid values and their - interpretation and is described in the Diameter application - introducing the AVP. + +Fajardo, et al. Standards Track [Page 47] + +RFC 6733 Diameter Base Protocol October 2012 IPFilterRule - The IPFilterRule format is derived from the OctetString AVP Base + The IPFilterRule format is derived from the OctetString Basic AVP Format and uses the ASCII charset. The rule syntax is a modified subset of ipfw(8) from FreeBSD. Packets may be filtered based on the following information that is associated with it: @@ -2713,13 +2652,13 @@ Internet-Draft Diameter Base Protocol January 2011 IP options ICMP types - Rules for the appropriate direction are evaluated in order, with - the first matched rule terminating the evaluation. Each packet is - evaluated once. If no rule matches, the packet is dropped if the - last rule evaluated was a permit, and passed if the last rule was - a deny. + Rules for the appropriate direction are evaluated in order, with the + first matched rule terminating the evaluation. Each packet is + evaluated once. If no rule matches, the packet is dropped if the + last rule evaluated was a permit, and passed if the last rule was a + deny. - IPFilterRule filters MUST follow the format: + IPFilterRule filters MUST follow the format: action dir proto from src to dst [options] @@ -2737,22 +2676,28 @@ Internet-Draft Diameter Base Protocol January 2011 The <address/mask> may be specified as: ipno An IPv4 or IPv6 number in dotted- quad or canonical IPv6 form. Only + this exact IP number will match the + rule. + + + + + + -Fajardo, et al. Expires July 24, 2011 [Page 49] +Fajardo, et al. Standards Track [Page 48] -Internet-Draft Diameter Base Protocol January 2011 +RFC 6733 Diameter Base Protocol October 2012 - this exact IP number will match the - rule. ipno/bits An IP number as above with a mask width of the form 192.0.2.10/24. In this case, all IP numbers from 192.0.2.0 to 192.0.2.255 will match. The bit width MUST be valid for the - IP version and the IP number MUST + IP version, and the IP number MUST NOT have bits set beyond the mask. For a match to occur, the same IP version must be present in the @@ -2765,7 +2710,7 @@ Internet-Draft Diameter Base Protocol January 2011 is the address or set of addresses assigned to the terminal. For IPv4, a typical first rule is often "deny - in ip! assigned" + in ip! assigned". The sense of the match can be inverted by preceding an address with the not modifier (!), @@ -2773,7 +2718,7 @@ Internet-Draft Diameter Base Protocol January 2011 instead. This does not affect the selection of port numbers. - With the TCP, UDP and SCTP protocols, optional + With the TCP, UDP, and SCTP protocols, optional ports may be specified as: {port/port-port}[,ports[,...]] @@ -2796,33 +2741,35 @@ Internet-Draft Diameter Base Protocol January 2011 -Fajardo, et al. Expires July 24, 2011 [Page 50] + + +Fajardo, et al. Standards Track [Page 49] -Internet-Draft Diameter Base Protocol January 2011 +RFC 6733 Diameter Base Protocol October 2012 - ipoptions spec - Match if the IP header contains the comma - separated list of options specified in spec. The + ipoptions spec + Match if the IP header contains the comma-separated + list of options specified in spec. The supported IP options are: ssrr (strict source route), lsrr (loose source - route), rr (record packet route) and ts + route), rr (record packet route), and ts (timestamp). The absence of a particular option may be denoted with a '!'. tcpoptions spec - Match if the TCP header contains the comma - separated list of options specified in spec. The + Match if the TCP header contains the comma-separated + list of options specified in spec. The supported TCP options are: mss (maximum segment size), window (tcp window advertisement), sack (selective ack), ts (rfc1323 - timestamp) and cc (rfc1644 t/tcp connection + timestamp), and cc (rfc1644 t/tcp connection count). The absence of a particular option may be denoted with a '!'. - established + established TCP packets only. Match packets that have the RST or ACK bits set. @@ -2832,10 +2779,10 @@ Internet-Draft Diameter Base Protocol January 2011 tcpflags spec TCP packets only. Match if the TCP header - contains the comma separated list of flags + contains the comma-separated list of flags specified in spec. The supported TCP flags are: - fin, syn, rst, psh, ack and urg. The absence of a + fin, syn, rst, psh, ack, and urg. The absence of a particular flag may be denoted with a '!'. A rule that contains a tcpflags specification can never match a fragmented packet that has a non-zero @@ -2852,9 +2799,9 @@ Internet-Draft Diameter Base Protocol January 2011 -Fajardo, et al. Expires July 24, 2011 [Page 51] +Fajardo, et al. Standards Track [Page 50] -Internet-Draft Diameter Base Protocol January 2011 +RFC 6733 Diameter Base Protocol October 2012 echo reply (0), destination unreachable (3), @@ -2863,21 +2810,20 @@ Internet-Draft Diameter Base Protocol January 2011 solicitation (10), time-to-live exceeded (11), IP header bad (12), timestamp request (13), timestamp reply (14), information request (15), - information reply (16), address mask request (17) + information reply (16), address mask request (17), and address mask reply (18). - There is one kind of packet that the access device MUST always - discard, that is an IP fragment with a fragment offset of one. - This is a valid packet, but it only has one use, to try to - circumvent firewalls. - - An access device that is unable to interpret or apply a deny rule - MUST terminate the session. An access device that is unable to - interpret or apply a permit rule MAY apply a more restrictive - rule. An access device MAY apply deny rules of its own before the - supplied rules, for example to protect the access device owner's - infrastructure. + There is one kind of packet that the access device MUST always + discard, that is an IP fragment with a fragment offset of one. This + is a valid packet, but it only has one use, to try to circumvent + firewalls. + An access device that is unable to interpret or apply a deny rule + MUST terminate the session. An access device that is unable to + interpret or apply a permit rule MAY apply a more restrictive rule. + An access device MAY apply deny rules of its own before the supplied + rules, for example to protect the access device owner's + infrastructure. 4.4. Grouped AVP Values @@ -2885,75 +2831,76 @@ Internet-Draft Diameter Base Protocol January 2011 implies that the Data field is actually a sequence of AVPs. It is possible to include an AVP with a Grouped type within a Grouped type, that is, to nest them. AVPs within an AVP of type Grouped have the - same padding requirements as non-Grouped AVPs, as defined in Section - 4. + same padding requirements as non-Grouped AVPs, as defined in + Section 4.4. The AVP Code numbering space of all AVPs included in a Grouped AVP is - the same as for non-grouped AVPs. Receivers of a Grouped AVP that + the same as for non-Grouped AVPs. Receivers of a Grouped AVP that does not have the 'M' (mandatory) bit set and one or more of the encapsulated AVPs within the group has the 'M' (mandatory) bit set MAY simply be ignored if the Grouped AVP itself is unrecognized. The rule applies even if the encapsulated AVP with its 'M' (mandatory) - bit set is further encapsulated within other sub-groups; i.e. other + bit set is further encapsulated within other sub-groups, i.e., other Grouped AVPs embedded within the Grouped AVP. - Every Grouped AVP defined MUST include a corresponding grammar, using - ABNF [RFC5234] (with modifications), as defined below. - + Every Grouped AVP definition MUST include a corresponding grammar, + using ABNF [RFC5234] (with modifications), as defined below. + grouped-avp-def = "<" name ">" "::=" avp + name-fmt = ALPHA *(ALPHA / DIGIT / "-") -Fajardo, et al. Expires July 24, 2011 [Page 52] +Fajardo, et al. Standards Track [Page 51] -Internet-Draft Diameter Base Protocol January 2011 +RFC 6733 Diameter Base Protocol October 2012 - grouped-avp-def = <name> "::=" avp - - name-fmt = ALPHA *(ALPHA / DIGIT / "-") - name = name-fmt ; The name has to be the name of an AVP, ; defined in the base or extended Diameter ; specifications. - avp = header [ *fixed] [ *required] [ *optional] + avp = header *fixed *required *optional header = "<" "AVP-Header:" avpcode [vendor] ">" avpcode = 1*DIGIT - ; The AVP Code assigned to the Grouped AVP + ; The AVP Code assigned to the Grouped AVP. vendor = 1*DIGIT ; The Vendor-ID assigned to the Grouped AVP. ; If absent, the default value of zero is ; used. -4.4.1. Example AVP with a Grouped Data type +4.4.1. Example AVP with a Grouped Data Type The Example-AVP (AVP Code 999999) is of type Grouped and is used to clarify how Grouped AVP values work. The Grouped Data field has the - following ABNF grammar: - - - - - - - - + following CCF grammar: + Example-AVP ::= < AVP Header: 999999 > + { Origin-Host } + 1*{ Session-Id } + *[ AVP ] + An Example-AVP with Grouped Data follows. + The Origin-Host AVP (Section 6.3) is required. In this case: + Origin-Host = "example.com". + One or more Session-Ids must follow. Here there are two: + Session-Id = + "grump.example.com:33041;23432;893;0AF3B81" + Session-Id = + "grump.example.com:33054;23561;2358;0AF3B82" @@ -2964,31 +2911,11 @@ Internet-Draft Diameter Base Protocol January 2011 -Fajardo, et al. Expires July 24, 2011 [Page 53] +Fajardo, et al. Standards Track [Page 52] -Internet-Draft Diameter Base Protocol January 2011 - - - Example-AVP ::= < AVP Header: 999999 > - { Origin-Host } - 1*{ Session-Id } - *[ AVP ] - - An Example-AVP with Grouped Data follows. - - The Origin-Host AVP is required (Section 6.3). In this case: - - Origin-Host = "example.com". +RFC 6733 Diameter Base Protocol October 2012 - One or more Session-Ids must follow. Here there are two: - - Session-Id = - "grump.example.com:33041;23432;893;0AF3B81" - - Session-Id = - "grump.example.com:33054;23561;2358;0AF3B82" - optional AVPs included are Recovery-Policy = <binary> @@ -3007,25 +2934,42 @@ Internet-Draft Diameter Base Protocol January 2011 41d018d56fe938f3cbf089aac12a912a2f0d1923a9390e5f789cb2e5067 d3427475e49968f841 - The data for the optional AVPs is represented in hex since the format - of these AVPs is neither known at the time of definition of the - Example-AVP group, nor (likely) at the time when the example instance - of this AVP is interpreted - except by Diameter implementations which - support the same set of AVPs. The encoding example illustrates how - padding is used and how length fields are calculated. Also note that - AVPs may be present in the Grouped AVP value which the receiver - cannot interpret (here, the Recover-Policy and Futuristic-Acct-Record - AVPs). The length of the Example-AVP is the sum of all the length of - the member AVPs including their padding plus the Example-AVP header + The data for the optional AVPs is represented in hexadecimal form + since the format of these AVPs is not known at the time of definition + of the Example-AVP group nor (likely) at the time when the example + instance of this AVP is interpreted -- except by Diameter + implementations that support the same set of AVPs. The encoding + example illustrates how padding is used and how length fields are + calculated. Also, note that AVPs may be present in the Grouped AVP + value that the receiver cannot interpret (here, the Recover-Policy + and Futuristic-Acct-Record AVPs). The length of the Example-AVP is + the sum of all the length of the member AVPs, including their + padding, plus the Example-AVP header size. + + + + + + + + + + + + + + + + -Fajardo, et al. Expires July 24, 2011 [Page 54] - -Internet-Draft Diameter Base Protocol January 2011 - size. + +Fajardo, et al. Standards Track [Page 53] + +RFC 6733 Diameter Base Protocol October 2012 This AVP would be encoded as follows: @@ -3076,15 +3020,18 @@ Internet-Draft Diameter Base Protocol January 2011 -Fajardo, et al. Expires July 24, 2011 [Page 55] + + + +Fajardo, et al. Standards Track [Page 54] -Internet-Draft Diameter Base Protocol January 2011 +RFC 6733 Diameter Base Protocol October 2012 4.5. Diameter Base Protocol AVPs The following table describes the Diameter AVPs defined in the base - protocol, their AVP Code values, types, possible flag values. + protocol, their AVP Code values, types, and possible flag values. Due to space constraints, the short form DiamIdent is used to represent DiameterIdentity. @@ -3132,9 +3079,9 @@ Internet-Draft Diameter Base Protocol January 2011 -Fajardo, et al. Expires July 24, 2011 [Page 56] +Fajardo, et al. Standards Track [Page 55] -Internet-Draft Diameter Base Protocol January 2011 +RFC 6733 Diameter Base Protocol October 2012 +----------+ @@ -3154,7 +3101,7 @@ Internet-Draft Diameter Base Protocol January 2011 Record-Number | | | Accounting- 480 9.8.1 Enumerated | M | V | Record-Type | | | - Accounting- 44 9.8.4 OctetString| M | V | + Acct- 44 9.8.4 OctetString| M | V | Session-Id | | | Accounting- 287 9.8.6 Unsigned64 | M | V | Sub-Session-Id | | | @@ -3188,9 +3135,9 @@ Internet-Draft Diameter Base Protocol January 2011 -Fajardo, et al. Expires July 24, 2011 [Page 57] +Fajardo, et al. Standards Track [Page 56] -Internet-Draft Diameter Base Protocol January 2011 +RFC 6733 Diameter Base Protocol October 2012 +----------+ @@ -3244,9 +3191,9 @@ Internet-Draft Diameter Base Protocol January 2011 -Fajardo, et al. Expires July 24, 2011 [Page 58] +Fajardo, et al. Standards Track [Page 57] -Internet-Draft Diameter Base Protocol January 2011 +RFC 6733 Diameter Base Protocol October 2012 5. Diameter Peers @@ -3258,25 +3205,25 @@ Internet-Draft Diameter Base Protocol January 2011 Connections between diameter peers are established using their valid DiameterIdentity. A Diameter node initiating a connection to a peer - MUST know the peers DiameterIdentity. Methods for discovering a + MUST know the peer's DiameterIdentity. Methods for discovering a Diameter peer can be found in Section 5.2. - Although a Diameter node may have many possible peers that it is able - to communicate with, it may not be economical to have an established - connection to all of them. At a minimum, a Diameter node SHOULD have - an established connection with two peers per realm, known as the - primary and secondary peers. Of course, a node MAY have additional - connections, if it is deemed necessary. Typically, all messages for - a realm are sent to the primary peer, but in the event that failover - procedures are invoked, any pending requests are sent to the - secondary peer. However, implementations are free to load balance - requests between a set of peers. - - Note that a given peer MAY act as a primary for a given realm, while + Although a Diameter node may have many possible peers with which it + is able to communicate, it may not be economical to have an + established connection to all of them. At a minimum, a Diameter node + SHOULD have an established connection with two peers per realm, known + as the primary and secondary peers. Of course, a node MAY have + additional connections, if it is deemed necessary. Typically, all + messages for a realm are sent to the primary peer but, in the event + that failover procedures are invoked, any pending requests are sent + to the secondary peer. However, implementations are free to load + balance requests between a set of peers. + + Note that a given peer MAY act as a primary for a given realm while acting as a secondary for another realm. When a peer is deemed suspect, which could occur for various reasons, - including not receiving a DWA within an allotted timeframe, no new + including not receiving a DWA within an allotted time frame, no new requests should be forwarded to the peer, but failover procedures are invoked. When an active peer is moved to this mode, additional connections SHOULD be established to ensure that the necessary number @@ -3284,15 +3231,14 @@ Internet-Draft Diameter Base Protocol January 2011 There are two ways that a peer is removed from the suspect peer list: - 1. The peer is no longer reachable, causing the transport connection - to be shutdown. The peer is moved to the closed state. + to be shut down. The peer is moved to the closed state. - 2. Three watchdog messages are exchanged with accepted round trip + 2. Three watchdog messages are exchanged with accepted round-trip times, and the connection to the peer is considered stabilized. In the event the peer being removed is either the primary or - secondary, an alternate peer SHOULD replace the deleted peer, and + secondary, an alternate peer SHOULD replace the deleted peer and assume the role of either primary or secondary. @@ -3300,99 +3246,106 @@ Internet-Draft Diameter Base Protocol January 2011 -Fajardo, et al. Expires July 24, 2011 [Page 59] + +Fajardo, et al. Standards Track [Page 58] -Internet-Draft Diameter Base Protocol January 2011 +RFC 6733 Diameter Base Protocol October 2012 5.2. Diameter Peer Discovery - Allowing for dynamic Diameter agent discovery will make it possible - for simpler and more robust deployment of Diameter services. In - order to promote interoperable implementations of Diameter peer - discovery, the following mechanisms are described. These are based - on existing IETF standards. The first option (manual configuration) - MUST be supported by all Diameter nodes, while the latter option - (DNS) MAY be supported. + Allowing for dynamic Diameter agent discovery makes possible simpler + and more robust deployment of Diameter services. In order to promote + interoperable implementations of Diameter peer discovery, the + following mechanisms (manual configuration and DNS) are described. + These are based on existing IETF standards. Both mechanisms MUST be + supported by all Diameter implementations; either MAY be used. There are two cases where Diameter peer discovery may be performed. The first is when a Diameter client needs to discover a first-hop Diameter agent. The second case is when a Diameter agent needs to - discover another agent - for further handling of a Diameter - operation. In both cases, the following 'search order' is - recommended: - + discover another agent for further handling of a Diameter operation. + In both cases, the following 'search order' is recommended: - 1. The Diameter implementation consults its list of static + 1. The Diameter implementation consults its list of statically (manually) configured Diameter agent locations. These will be used if they exist and respond. - 2. The Diameter implementation performs a NAPTR query for a server - in a particular realm. The Diameter implementation has to know - in advance which realm to look for a Diameter agent. This could - be deduced, for example, from the 'realm' in a NAI that a - Diameter implementation needed to perform a Diameter operation - on. + in a particular realm. The Diameter implementation has to know, + in advance, in which realm to look for a Diameter agent. This + could be deduced, for example, from the 'realm' in an NAI on + which a Diameter implementation needed to perform a Diameter + operation. The NAPTR usage in Diameter follows the S-NAPTR DDDS application [RFC3958] in which the SERVICE field includes tags for the desired application and supported application protocol. The application service tag for a Diameter application is 'aaa' and the supported application protocol tags are 'diameter.tcp', - 'diameter.sctp', 'diameter.dtls' or 'diameter.tls.tcp'. + 'diameter.sctp', 'diameter.dtls', or 'diameter.tls.tcp' + [RFC6408]. The client can follow the resolution process defined by the - S-NAPTR DDDS [RFC3958] application to find a matching SRV, A or + S-NAPTR DDDS [RFC3958] application to find a matching SRV, A, or AAAA record of a suitable peer. The domain suffixes in the NAPTR replacement field SHOULD match the domain of the original query. An example can be found in Appendix B. 3. If no NAPTR records are found, the requester directly queries for - SRV records '_diameter._sctp'.realm, '_diameter._dtls'.realm, - '_diameter._tcp'.realm and '_diameter._tls'.realm depending on - the requesters network protocol capabilities. If SRV records are - found then the requester can perform address record query (A RR's + one of the following SRV records: for Diameter over TCP, use + "_diameter._tcp.realm"; for Diameter over TLS, use + "_diameters._tcp.realm"; for Diameter over SCTP, use + "_diameter._sctp.realm"; for Diameter over DTLS, use + "_diameters._sctp.realm". If SRV records are found, then the + requester can perform address record query (A RR's and/or AAAA + -Fajardo, et al. Expires July 24, 2011 [Page 60] +Fajardo, et al. Standards Track [Page 59] -Internet-Draft Diameter Base Protocol January 2011 +RFC 6733 Diameter Base Protocol October 2012 - and/or AAAA RR's) for the target hostname specified in the SRV - records. If no SRV records are found, the requester gives up. + RR's) for the target hostname specified in the SRV records + following the rules given in [RFC2782]. If no SRV records are + found, the requester gives up. If the server is using a site certificate, the domain name in the NAPTR query and the domain name in the replacement field MUST both be valid based on the site certificate handed out by the server in the - TLS/TCP and DTLS/SCTP or IKE exchange. Similarly, the domain name in - the SRV query and the domain name in the target in the SRV record - MUST both be valid based on the same site certificate. Otherwise, an - attacker could modify the DNS records to contain replacement values - in a different domain, and the client could not validate that this - was the desired behavior, or the result of an attack. - - Also, the Diameter Peer MUST check to make sure that the discovered + TLS/TCP and DTLS/SCTP or Internet Key Exchange Protocol (IKE) + exchange. Similarly, the domain name in the SRV query and the domain + name in the target in the SRV record MUST both be valid based on the + same site certificate. Otherwise, an attacker could modify the DNS + records to contain replacement values in a different domain, and the + client could not validate whether this was the desired behavior or + the result of an attack. + + Also, the Diameter peer MUST check to make sure that the discovered peers are authorized to act in its role. Authentication via IKE or TLS/TCP and DTLS/SCTP, or validation of DNS RRs via DNSSEC is not sufficient to conclude this. For example, a web server may have obtained a valid TLS/TCP and DTLS/SCTP certificate, and secured RRs may be included in the DNS, but this does not imply that it is - authorized to act as a Diameter Server. - - Authorization can be achieved for example, by configuration of a - Diameter Server CA. Alternatively this can be achieved by definition - of OIDs within TLS/TCP and DTLS/SCTP or IKE certificates so as to - signify Diameter Server authorization. - - A dynamically discovered peer causes an entry in the Peer Table (see + authorized to act as a Diameter server. + + Authorization can be achieved, for example, by the configuration of a + Diameter server Certification Authority (CA). The server CA issues a + certificate to the Diameter server, which includes an Object + Identifier (OID) to indicate the subject is a Diameter server in the + Extended Key Usage extension [RFC5280]. This certificate is then + used during TLS/TCP, DTLS/SCTP, or IKE security negotiation. + However, note that, at the time of writing, no Diameter server + Certification Authorities exist. + + A dynamically discovered peer causes an entry in the peer table (see Section 2.6) to be created. Note that entries created via DNS MUST - expire (or be refreshed) within the DNS TTL. If a peer is discovered - outside of the local realm, a routing table entry (see Section 2.7) - for the peer's realm is created. The routing table entry's - expiration MUST match the peer's expiration value. + expire (or be refreshed) within the DNS Time to Live (TTL). If a + peer is discovered outside of the local realm, a routing table entry + (see Section 2.7) for the peer's realm is created. The routing table + entry's expiration MUST match the peer's expiration value. 5.3. Capabilities Exchange @@ -3400,35 +3353,36 @@ Internet-Draft Diameter Base Protocol January 2011 exchange the Capabilities Exchange messages, as specified in the peer state machine (see Section 5.6). This message allows the discovery of a peer's identity and its capabilities (protocol version number, - supported Diameter applications, security mechanisms, etc.) + the identifiers of supported Diameter applications, security + mechanisms, etc.). - The receiver only issues commands to its peers that have advertised - support for the Diameter application that defines the command. A - Diameter node MUST cache the supported applications in order to - ensure that unrecognized commands and/or AVPs are not unnecessarily - sent to a peer. - - A receiver of a Capabilities-Exchange-Req (CER) message that does not -Fajardo, et al. Expires July 24, 2011 [Page 61] +Fajardo, et al. Standards Track [Page 60] -Internet-Draft Diameter Base Protocol January 2011 +RFC 6733 Diameter Base Protocol October 2012 - have any applications in common with the sender MUST return a + The receiver only issues commands to its peers that have advertised + support for the Diameter application that defines the command. A + Diameter node MUST cache the supported Application Ids in order to + ensure that unrecognized commands and/or AVPs are not unnecessarily + sent to a peer. + + A receiver of a Capabilities-Exchange-Request (CER) message that does + not have any applications in common with the sender MUST return a Capabilities-Exchange-Answer (CEA) with the Result-Code AVP set to - DIAMETER_NO_COMMON_APPLICATION, and SHOULD disconnect the transport + DIAMETER_NO_COMMON_APPLICATION and SHOULD disconnect the transport layer connection. Note that receiving a CER or CEA from a peer - advertising itself as a Relay (see Section 2.4) MUST be interpreted + advertising itself as a relay (see Section 2.4) MUST be interpreted as having common applications with the peer. The receiver of the Capabilities-Exchange-Request (CER) MUST determine common applications by computing the intersection of its - own set of supported Application Id against all of the application - identifier AVPs (Auth-Application-Id, Acct-Application-Id and Vendor- - Specific-Application-Id) present in the CER. The value of the + own set of supported Application Ids against all of the + Application-Id AVPs (Auth-Application-Id, Acct-Application-Id, and + Vendor-Specific-Application-Id) present in the CER. The value of the Vendor-Id AVP in the Vendor-Specific-Application-Id MUST NOT be used during computation. The sender of the Capabilities-Exchange-Answer (CEA) SHOULD include all of its supported applications as a hint to @@ -3438,12 +3392,13 @@ Internet-Draft Diameter Base Protocol January 2011 and DTLS/SCTP connection prior to the CER/CEA exchange. This protects the capabilities information of both peers. To support older Diameter implementations that do not fully conform to this - document, the transport security MAY still be negotiated via Inband- - Security AVP. In this case, the receiver of a Capabilities-Exchange- - Req (CER) message that does not have any security mechanisms in - common with the sender MUST return a Capabilities-Exchange-Answer - (CEA) with the Result-Code AVP set to DIAMETER_NO_COMMON_SECURITY, - and SHOULD disconnect the transport layer connection. + document, the transport security MAY still be negotiated via an + Inband-Security AVP. In this case, the receiver of a Capabilities- + Exchange-Request (CER) message that does not have any security + mechanisms in common with the sender MUST return a Capabilities- + Exchange-Answer (CEA) with the Result-Code AVP set to + DIAMETER_NO_COMMON_SECURITY and SHOULD disconnect the transport layer + connection. CERs received from unknown peers MAY be silently discarded, or a CEA MAY be issued with the Result-Code AVP set to DIAMETER_UNKNOWN_PEER. @@ -3455,34 +3410,34 @@ Internet-Draft Diameter Base Protocol January 2011 failure, all the pending transactions destined to the unknown peer can be discarded. - The CER and CEA messages MUST NOT be proxied, redirected or relayed. + The CER and CEA messages MUST NOT be proxied, redirected, or relayed. - Since the CER/CEA messages cannot be proxied, it is still possible - that an upstream agent receives a message for which it has no - available peers to handle the application that corresponds to the - Command-Code. In such instances, the 'E' bit is set in the answer - message (see Section 7.) with the Result-Code AVP set to - DIAMETER_UNABLE_TO_DELIVER to inform the downstream to take action - (e.g., re-routing request to an alternate peer). - -Fajardo, et al. Expires July 24, 2011 [Page 62] +Fajardo, et al. Standards Track [Page 61] -Internet-Draft Diameter Base Protocol January 2011 +RFC 6733 Diameter Base Protocol October 2012 + Since the CER/CEA messages cannot be proxied, it is still possible + that an upstream agent will receive a message for which it has no + available peers to handle the application that corresponds to the + Command Code. In such instances, the 'E' bit is set in the answer + message (Section 7) with the Result-Code AVP set to + DIAMETER_UNABLE_TO_DELIVER to inform the downstream agent to take + action (e.g., re-routing request to an alternate peer). + With the exception of the Capabilities-Exchange-Request message, a message of type Request that includes the Auth-Application-Id or Acct-Application-Id AVPs, or a message with an application-specific - command code, MAY only be forwarded to a host that has explicitly + Command Code MAY only be forwarded to a host that has explicitly advertised support for the application (or has advertised the Relay Application Id). 5.3.1. Capabilities-Exchange-Request - The Capabilities-Exchange-Request (CER), indicated by the Command- + The Capabilities-Exchange-Request (CER), indicated by the Command Code set to 257 and the Command Flags' 'R' bit set, is sent to exchange local capabilities. Upon detection of a transport failure, this message MUST NOT be sent to an alternate peer. @@ -3490,7 +3445,7 @@ Internet-Draft Diameter Base Protocol January 2011 When Diameter is run over SCTP [RFC4960] or DTLS/SCTP [RFC6083], which allow for connections to span multiple interfaces and multiple IP addresses, the Capabilities-Exchange-Request message MUST contain - one Host-IP- Address AVP for each potential IP address that MAY be + one Host-IP-Address AVP for each potential IP address that MAY be locally used when transmitting Diameter messages. Message Format @@ -3510,9 +3465,20 @@ Internet-Draft Diameter Base Protocol January 2011 [ Firmware-Revision ] * [ AVP ] + + + + + + +Fajardo, et al. Standards Track [Page 62] + +RFC 6733 Diameter Base Protocol October 2012 + + 5.3.2. Capabilities-Exchange-Answer - The Capabilities-Exchange-Answer (CEA), indicated by the Command-Code + The Capabilities-Exchange-Answer (CEA), indicated by the Command Code set to 257 and the Command Flags' 'R' bit cleared, is sent in response to a CER message. @@ -3522,13 +3488,6 @@ Internet-Draft Diameter Base Protocol January 2011 one Host-IP-Address AVP for each potential IP address that MAY be locally used when transmitting Diameter messages. - - -Fajardo, et al. Expires July 24, 2011 [Page 63] - -Internet-Draft Diameter Base Protocol January 2011 - - Message Format <CEA> ::= < Diameter Header: 257 > @@ -3552,22 +3511,34 @@ Internet-Draft Diameter Base Protocol January 2011 5.3.3. Vendor-Id AVP The Vendor-Id AVP (AVP Code 266) is of type Unsigned32 and contains - the IANA "SMI Network Management Private Enterprise Codes" [RFC3232] - value assigned to the vendor of the Diameter device. It is + the IANA "SMI Network Management Private Enterprise Codes" + [ENTERPRISE] value assigned to the Diameter Software vendor. It is envisioned that the combination of the Vendor-Id, Product-Name - (Section 5.3.7) and the Firmware-Revision (Section 5.3.4) AVPs may + (Section 5.3.7), and Firmware-Revision (Section 5.3.4) AVPs may provide useful debugging information. - A Vendor-Id value of zero in the CER or CEA messages is reserved and + A Vendor-Id value of zero in the CER or CEA message is reserved and indicates that this field is ignored. + + + + + + + +Fajardo, et al. Standards Track [Page 63] + +RFC 6733 Diameter Base Protocol October 2012 + + 5.3.4. Firmware-Revision AVP The Firmware-Revision AVP (AVP Code 267) is of type Unsigned32 and is used to inform a Diameter peer of the firmware revision of the issuing device. - For devices that do not have a firmware revision (general purpose + For devices that do not have a firmware revision (general-purpose computers running Diameter software modules, for instance), the revision of the Diameter software module may be reported instead. @@ -3577,77 +3548,70 @@ Internet-Draft Diameter Base Protocol January 2011 to inform a Diameter peer of the sender's IP address. All source addresses that a Diameter node expects to use with SCTP [RFC4960] or DTLS/SCTP [RFC6083] MUST be advertised in the CER and CEA messages by - - - -Fajardo, et al. Expires July 24, 2011 [Page 64] - -Internet-Draft Diameter Base Protocol January 2011 - - including a Host-IP-Address AVP for each address. 5.3.6. Supported-Vendor-Id AVP The Supported-Vendor-Id AVP (AVP Code 265) is of type Unsigned32 and contains the IANA "SMI Network Management Private Enterprise Codes" - [RFC3232] value assigned to a vendor other than the device vendor but - including the application vendor. This is used in the CER and CEA - messages in order to inform the peer that the sender supports (a - subset of) the vendor-specific AVPs defined by the vendor identified + [ENTERPRISE] value assigned to a vendor other than the device vendor + but including the application vendor. This is used in the CER and + CEA messages in order to inform the peer that the sender supports (a + subset of) the Vendor-Specific AVPs defined by the vendor identified in this AVP. The value of this AVP MUST NOT be set to zero. Multiple instances of this AVP containing the same value SHOULD NOT be sent. 5.3.7. Product-Name AVP - The Product-Name AVP (AVP Code 269) is of type UTF8String, and - contains the vendor assigned name for the product. The Product-Name + The Product-Name AVP (AVP Code 269) is of type UTF8String and + contains the vendor-assigned name for the product. The Product-Name AVP SHOULD remain constant across firmware revisions for the same product. -5.4. Disconnecting Peer connections +5.4. Disconnecting Peer Connections When a Diameter node disconnects one of its transport connections, - its peer cannot know the reason for the disconnect, and will most - likely assume that a connectivity problem occurred, or that the peer + its peer cannot know the reason for the disconnect and will most + likely assume that a connectivity problem occurred or that the peer has rebooted. In these cases, the peer may periodically attempt to reconnect, as stated in Section 2.1. In the event that the - disconnect was a result of either a shortage of internal resources, - or simply that the node in question has no intentions of forwarding - any Diameter messages to the peer in the foreseeable future, a - periodic connection request would not be welcomed. The - Disconnection-Reason AVP contains the reason the Diameter node issued - the Disconnect-Peer-Request message. + disconnect was a result of either a shortage of internal resources or + simply that the node in question has no intentions of forwarding any + Diameter messages to the peer in the foreseeable future, a periodic - The Disconnect-Peer-Request message is used by a Diameter node to - inform its peer of its intent to disconnect the transport layer, and - that the peer shouldn't reconnect unless it has a valid reason to do - so (e.g., message to be forwarded). Upon receipt of the message, the - Disconnect-Peer-Answer is returned, which SHOULD contain an error if - messages have recently been forwarded, and are likely in flight, - which would otherwise cause a race condition. - The receiver of the Disconnect-Peer-Answer initiates the transport - disconnect. The sender of the Disconnect-Peer-Answer should be able - to detect the transport closure and cleanup the connection. +Fajardo, et al. Standards Track [Page 64] + +RFC 6733 Diameter Base Protocol October 2012 + connection request would not be welcomed. The Disconnection-Reason + AVP contains the reason the Diameter node issued the Disconnect-Peer- + Request message. -Fajardo, et al. Expires July 24, 2011 [Page 65] - -Internet-Draft Diameter Base Protocol January 2011 + The Disconnect-Peer-Request message is used by a Diameter node to + inform its peer of its intent to disconnect the transport layer and + that the peer shouldn't reconnect unless it has a valid reason to do + so (e.g., message to be forwarded). Upon receipt of the message, the + Disconnect-Peer-Answer message is returned, which SHOULD contain an + error if messages have recently been forwarded, and are likely in + flight, which would otherwise cause a race condition. + The receiver of the Disconnect-Peer-Answer message initiates the + transport disconnect. The sender of the Disconnect-Peer-Answer + message should be able to detect the transport closure and clean up + the connection. 5.4.1. Disconnect-Peer-Request - The Disconnect-Peer-Request (DPR), indicated by the Command-Code set + The Disconnect-Peer-Request (DPR), indicated by the Command Code set to 282 and the Command Flags' 'R' bit set, is sent to a peer to - inform its intentions to shutdown the transport connection. Upon - detection of a transport failure, this message MUST NOT be sent to an - alternate peer. + inform it of its intentions to shut down the transport connection. + Upon detection of a transport failure, this message MUST NOT be sent + to an alternate peer. Message Format @@ -3659,28 +3623,14 @@ Internet-Draft Diameter Base Protocol January 2011 5.4.2. Disconnect-Peer-Answer - The Disconnect-Peer-Answer (DPA), indicated by the Command-Code set + The Disconnect-Peer-Answer (DPA), indicated by the Command Code set to 282 and the Command Flags' 'R' bit cleared, is sent as a response to the Disconnect-Peer-Request message. Upon receipt of this - message, the transport connection is shutdown. + message, the transport connection is shut down. - Message Format - <DPA> ::= < Diameter Header: 282 > - { Result-Code } - { Origin-Host } - { Origin-Realm } - [ Error-Message ] - [ Failed-AVP ] - * [ AVP ] -5.4.3. Disconnect-Cause AVP - The Disconnect-Cause AVP (AVP Code 273) is of type Enumerated. A - Diameter node MUST include this AVP in the Disconnect-Peer-Request - message to inform the peer of the reason for its intention to - shutdown the transport connection. The following values are - supported: @@ -3689,29 +3639,44 @@ Internet-Draft Diameter Base Protocol January 2011 +Fajardo, et al. Standards Track [Page 65] + +RFC 6733 Diameter Base Protocol October 2012 + Message Format + + <DPA> ::= < Diameter Header: 282 > + { Result-Code } + { Origin-Host } + { Origin-Realm } + [ Error-Message ] + [ Failed-AVP ] + * [ AVP ] -Fajardo, et al. Expires July 24, 2011 [Page 66] - -Internet-Draft Diameter Base Protocol January 2011 +5.4.3. Disconnect-Cause AVP + + The Disconnect-Cause AVP (AVP Code 273) is of type Enumerated. A + Diameter node MUST include this AVP in the Disconnect-Peer-Request + message to inform the peer of the reason for its intention to shut + down the transport connection. The following values are supported: - REBOOTING 0 - A scheduled reboot is imminent. Receiver of DPR with above result - code MAY attempt reconnection. + REBOOTING 0 + A scheduled reboot is imminent. A receiver of a DPR with + above result code MAY attempt reconnection. - BUSY 1 - The peer's internal resources are constrained, and it has - determined that the transport connection needs to be closed. - Receiver of DPR with above result code SHOULD NOT attempt - reconnection. + BUSY 1 + The peer's internal resources are constrained, and it has + determined that the transport connection needs to be closed. + A receiver of a DPR with above result code SHOULD NOT attempt + reconnection. - DO_NOT_WANT_TO_TALK_TO_YOU 2 - The peer has determined that it does not see a need for the - transport connection to exist, since it does not expect any - messages to be exchanged in the near future. Receiver of DPR - with above result code SHOULD NOT attempt reconnection. + DO_NOT_WANT_TO_TALK_TO_YOU 2 + The peer has determined that it does not see a need for the + transport connection to exist, since it does not expect any + messages to be exchanged in the near future. A receiver of a + DPR with above result code SHOULD NOT attempt reconnection. 5.5. Transport Failure Detection @@ -3723,9 +3688,21 @@ Internet-Draft Diameter Base Protocol January 2011 Watchdog-Answer messages, defined in this section, are used to pro- actively detect transport failures. + + + + + + + +Fajardo, et al. Standards Track [Page 66] + +RFC 6733 Diameter Base Protocol October 2012 + + 5.5.1. Device-Watchdog-Request - The Device-Watchdog-Request (DWR), indicated by the Command-Code set + The Device-Watchdog-Request (DWR), indicated by the Command Code set to 280 and the Command Flags' 'R' bit set, is sent to a peer when no traffic has been exchanged between two peers (see Section 5.5.3). Upon detection of a transport failure, this message MUST NOT be sent @@ -3741,18 +3718,10 @@ Internet-Draft Diameter Base Protocol January 2011 5.5.2. Device-Watchdog-Answer - The Device-Watchdog-Answer (DWA), indicated by the Command-Code set + The Device-Watchdog-Answer (DWA), indicated by the Command Code set to 280 and the Command Flags' 'R' bit cleared, is sent as a response to the Device-Watchdog-Request message. - - - -Fajardo, et al. Expires July 24, 2011 [Page 67] - -Internet-Draft Diameter Base Protocol January 2011 - - Message Format <DWA> ::= < Diameter Header: 280 > @@ -3767,7 +3736,7 @@ Internet-Draft Diameter Base Protocol January 2011 5.5.3. Transport Failure Algorithm The transport failure algorithm is defined in [RFC3539]. All - Diameter implementations MUST support the algorithm defined in the + Diameter implementations MUST support the algorithm defined in that specification in order to be compliant to the Diameter base protocol. 5.5.4. Failover and Failback Procedures @@ -3775,7 +3744,17 @@ Internet-Draft Diameter Base Protocol January 2011 In the event that a transport failure is detected with a peer, it is necessary for all pending request messages to be forwarded to an alternate agent, if possible. This is commonly referred to as - failover. + "failover". + + + + + + +Fajardo, et al. Standards Track [Page 67] + +RFC 6733 Diameter Base Protocol October 2012 + In order for a Diameter node to perform failover procedures, it is necessary for the node to maintain a pending message queue for a @@ -3783,16 +3762,16 @@ Internet-Draft Diameter Base Protocol January 2011 request is removed from the queue. The Hop-by-Hop Identifier field is used to match the answer with the queued request. - When a transport failure is detected, if possible all messages in the - queue are sent to an alternate agent with the T flag set. On booting - a Diameter client or agent, the T flag is also set on any records - still remaining to be transmitted in non-volatile storage. An - example of a case where it is not possible to forward the message to - an alternate server is when the message has a fixed destination, and - the unavailable peer is the message's final destination (see - Destination-Host AVP). Such an error requires that the agent return - an answer message with the 'E' bit set and the Result-Code AVP set to - DIAMETER_UNABLE_TO_DELIVER. + When a transport failure is detected, if possible, all messages in + the queue are sent to an alternate agent with the T flag set. On + booting a Diameter client or agent, the T flag is also set on any + remaining records in non-volatile storage that are still waiting to + be transmitted. An example of a case where it is not possible to + forward the message to an alternate server is when the message has a + fixed destination, and the unavailable peer is the message's final + destination (see Destination-Host AVP). Such an error requires that + the agent return an answer message with the 'E' bit set and the + Result-Code AVP set to DIAMETER_UNABLE_TO_DELIVER. It is important to note that multiple identical requests or answers MAY be received as a result of a failover. The End-to-End Identifier @@ -3801,17 +3780,9 @@ Internet-Draft Diameter Base Protocol January 2011 As described in Section 2.1, a connection request should be periodically attempted with the failed peer in order to re-establish - - - -Fajardo, et al. Expires July 24, 2011 [Page 68] - -Internet-Draft Diameter Base Protocol January 2011 - - the transport connection. Once a connection has been successfully established, messages can once again be forwarded to the peer. This - is commonly referred to as failback. + is commonly referred to as "failback". 5.6. Peer State Machine @@ -3824,16 +3795,24 @@ Internet-Draft Diameter Base Protocol January 2011 This state machine is closely coupled with the state machine described in [RFC3539], which is used to open, close, failover, - probe, and reopen transport connections. Note in particular that + probe, and reopen transport connections. In particular, note that [RFC3539] requires the use of watchdog messages to probe connections. For Diameter, DWR and DWA messages are to be used. - I- is used to represent the initiator (connecting) connection, while - the R- is used to represent the responder (listening) connection. - The lack of a prefix indicates that the event or action is the same - regardless of the connection on which the event occurred. + The I- prefix is used to represent the initiator (connecting) + connection, while the R- prefix is used to represent the responder + (listening) connection. The lack of a prefix indicates that the + event or action is the same regardless of the connection on which the + event occurred. + + + +Fajardo, et al. Standards Track [Page 68] + +RFC 6733 Diameter Base Protocol October 2012 + - The stable states that a state machine may be in are Closed, I-Open + The stable states that a state machine may be in are Closed, I-Open, and R-Open; all other states are intermediate. Note that I-Open and R-Open are equivalent except for whether the initiator or responder transport connection is used for communication. @@ -3848,23 +3827,15 @@ Internet-Draft Diameter Base Protocol January 2011 the results of an election on one peer are guaranteed to be the inverse of the results on the other. - For TLS/TCP and DTLS/SCTP usage, TLS/TCP and DTLS/SCTP handshake + For TLS/TCP and DTLS/SCTP usage, a TLS/TCP and DTLS/SCTP handshake SHOULD begin when both ends are in the closed state prior to any Diameter message exchanges. The TLS/TCP and DTLS/SCTP connection SHOULD be established before sending any CER or CEA message to secure and protect the capabilities information of both peers. The TLS/TCP and DTLS/SCTP connection SHOULD be disconnected when the state machine moves to the closed state. When connecting to responders - that do not conform to this document (i.e. older Diameter + that do not conform to this document (i.e., older Diameter implementations that are not prepared to received TLS/TCP and DTLS/ - - - -Fajardo, et al. Expires July 24, 2011 [Page 69] - -Internet-Draft Diameter Base Protocol January 2011 - - SCTP connections in the closed state), the initial TLS/TCP and DTLS/ SCTP connection attempt will fail. The initiator MAY then attempt to connect via TCP or SCTP and initiate the TLS/TCP and DTLS/SCTP @@ -3878,6 +3849,25 @@ Internet-Draft Diameter Base Protocol January 2011 Any implementation that produces equivalent results is considered compliant. + + + + + + + + + + + + + + +Fajardo, et al. Standards Track [Page 69] + +RFC 6733 Diameter Base Protocol October 2012 + + state event action next state ----------------------------------------------------------------- Closed Start I-Snd-Conn-Req Wait-Conn-Ack @@ -3914,13 +3904,6 @@ Internet-Draft Diameter Base Protocol January 2011 R-Conn-CER R-Reject Wait-Returns Timeout Error Closed - - -Fajardo, et al. Expires July 24, 2011 [Page 70] - -Internet-Draft Diameter Base Protocol January 2011 - - R-Open Send-Message R-Snd-Message R-Open R-Rcv-Message Process R-Open R-Rcv-DWR Process-DWR, R-Open @@ -3928,10 +3911,19 @@ Internet-Draft Diameter Base Protocol January 2011 R-Rcv-DWA Process-DWA R-Open R-Conn-CER R-Reject R-Open Stop R-Snd-DPR Closing - R-Rcv-DPR R-Snd-DPA, Closed - R-Disc + R-Rcv-DPR R-Snd-DPA Closing R-Peer-Disc R-Disc Closed + + + + + +Fajardo, et al. Standards Track [Page 70] + +RFC 6733 Diameter Base Protocol October 2012 + + I-Open Send-Message I-Snd-Message I-Open I-Rcv-Message Process I-Open I-Rcv-DWR Process-DWR, I-Open @@ -3939,8 +3931,7 @@ Internet-Draft Diameter Base Protocol January 2011 I-Rcv-DWA Process-DWA I-Open R-Conn-CER R-Reject I-Open Stop I-Snd-DPR Closing - I-Rcv-DPR I-Snd-DPA, Closed - I-Disc + I-Rcv-DPR I-Snd-DPA Closing I-Peer-Disc I-Disc Closed Closing I-Rcv-DPA I-Disc Closed @@ -3949,88 +3940,44 @@ Internet-Draft Diameter Base Protocol January 2011 I-Peer-Disc I-Disc Closed R-Peer-Disc R-Disc Closed -5.6.1. Incoming connections +5.6.1. Incoming Connections When a connection request is received from a Diameter peer, it is not, in the general case, possible to know the identity of that peer until a CER is received from it. This is because host and port - determine the identity of a Diameter peer; and the source port of an - incoming connection is arbitrary. Upon receipt of CER, the identity - of the connecting peer can be uniquely determined from Origin-Host. + determine the identity of a Diameter peer; the source port of an + incoming connection is arbitrary. Upon receipt of a CER, the + identity of the connecting peer can be uniquely determined from the + Origin-Host. For this reason, a Diameter peer must employ logic separate from the state machine to receive connection requests, accept them, and await - CER. Once CER arrives on a new connection, the Origin-Host that - identifies the peer is used to locate the state machine associated - with that peer, and the new connection and CER are passed to the - state machine as an R-Conn-CER event. + the CER. Once the CER arrives on a new connection, the Origin-Host + that identifies the peer is used to locate the state machine + associated with that peer, and the new connection and CER are passed + to the state machine as an R-Conn-CER event. The logic that handles incoming connections SHOULD close and discard - the connection if any message other than CER arrives, or if an + the connection if any message other than a CER arrives or if an implementation-defined timeout occurs prior to receipt of CER. - - - -Fajardo, et al. Expires July 24, 2011 [Page 71] - -Internet-Draft Diameter Base Protocol January 2011 - - Because handling of incoming connections up to and including receipt - of CER requires logic, separate from that of any individual state + of a CER requires logic, separate from that of any individual state machine associated with a particular peer, it is described separately in this section rather than in the state machine above. 5.6.2. Events Transitions and actions in the automaton are caused by events. In - this section, we will ignore the -I and -R prefix, since the actual - event would be identical, but would occur on one of two possible + this section, we will ignore the I- and R- prefixes, since the actual + event would be identical, but it would occur on one of two possible connections. - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - -Fajardo, et al. Expires July 24, 2011 [Page 72] +Fajardo, et al. Standards Track [Page 71] -Internet-Draft Diameter Base Protocol January 2011 +RFC 6733 Diameter Base Protocol October 2012 Start The Diameter application has signaled that a @@ -4053,7 +4000,8 @@ Internet-Draft Diameter Base Protocol January 2011 Rcv-CEA A CEA message from the peer was received. - Rcv-Non-CEA A message other than CEA from the peer was received. + Rcv-Non-CEA A message, other than a CEA, from the peer was + received. Peer-Disc A disconnection indication from the peer was received. @@ -4066,7 +4014,7 @@ Internet-Draft Diameter Base Protocol January 2011 Send-Message A message is to be sent. - Rcv-Message A message other than CER, CEA, DPR, DPA, DWR or DWA + Rcv-Message A message other than CER, CEA, DPR, DPA, DWR, or DWA was received. Stop The Diameter application has signaled that a @@ -4077,16 +4025,15 @@ Internet-Draft Diameter Base Protocol January 2011 Actions in the automaton are caused by events and typically indicate the transmission of packets and/or an action to be taken on the - connection. In this section we will ignore the I- and R-prefix, - since the actual action would be identical, but would occur on one of - two possible connections. - + connection. In this section, we will ignore the I- and R- prefixes, + since the actual action would be identical, but it would occur on one + of two possible connections. -Fajardo, et al. Expires July 24, 2011 [Page 73] +Fajardo, et al. Standards Track [Page 72] -Internet-Draft Diameter Base Protocol January 2011 +RFC 6733 Diameter Base Protocol October 2012 Snd-Conn-Req A transport connection is initiated with the peer. @@ -4098,11 +4045,12 @@ Internet-Draft Diameter Base Protocol January 2011 is disconnected. Process-CER The CER associated with the R-Conn-CER is processed. + Snd-CER A CER message is sent to the peer. Snd-CEA A CEA message is sent to the peer. - Cleanup If necessary, the connection is shutdown, and any + Cleanup If necessary, the connection is shut down, and any local resources are freed. Error The transport layer connection is disconnected, @@ -4139,10 +4087,9 @@ Internet-Draft Diameter Base Protocol January 2011 - -Fajardo, et al. Expires July 24, 2011 [Page 74] +Fajardo, et al. Standards Track [Page 73] -Internet-Draft Diameter Base Protocol January 2011 +RFC 6733 Diameter Base Protocol October 2012 5.6.4. The Election Process @@ -4150,134 +4097,97 @@ Internet-Draft Diameter Base Protocol January 2011 The election is performed on the responder. The responder compares the Origin-Host received in the CER with its own Origin-Host as two streams of octets. If the local Origin-Host lexicographically - succeeds the received Origin-Host a Win-Election event is issued - locally. Diameter identities are in ASCII form therefore the lexical - comparison is consistent with DNS case insensitivity where octets - that fall in the ASCII range 'a' through 'z' MUST compare equally to - their upper-case counterparts between 'A' and 'Z'. See Appendix D - for interactions between the Diameter protocol and Internationalized - Domain Name (IDNs). + succeeds the received Origin-Host, a Win-Election event is issued + locally. Diameter identities are in ASCII form; therefore, the + lexical comparison is consistent with DNS case insensitivity, where + octets that fall in the ASCII range 'a' through 'z' MUST compare + equally to their uppercase counterparts between 'A' and 'Z'. See + Appendix D for interactions between the Diameter protocol and + Internationalized Domain Name (IDNs). The winner of the election MUST close the connection it initiated. Historically, maintaining the responder side of a connection was more efficient than maintaining the initiator side. However, current practices makes this distinction irrelevant. - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - -Fajardo, et al. Expires July 24, 2011 [Page 75] - -Internet-Draft Diameter Base Protocol January 2011 - - -6. Diameter message processing +6. Diameter Message Processing This section describes how Diameter requests and answers are created and processed. 6.1. Diameter Request Routing Overview - A request is sent towards its final destination using a combination - of the Destination-Realm and Destination-Host AVPs, in one of these - three combinations: + A request is sent towards its final destination using one of the + following three combinations of the Destination-Realm and + Destination-Host AVPs: - o a request that is not able to be proxied (such as CER) MUST NOT + o A request that is not able to be proxied (such as a CER) MUST NOT contain either Destination-Realm or Destination-Host AVPs. - o a request that needs to be sent to a home server serving a + o A request that needs to be sent to a home server serving a specific realm, but not to a specific server (such as the first - request of a series of round-trips), MUST contain a Destination- - Realm AVP, but MUST NOT contain a Destination-Host AVP. For + request of a series of round trips), MUST contain a Destination- + Realm AVP but MUST NOT contain a Destination-Host AVP. For Diameter clients, the value of the Destination-Realm AVP MAY be extracted from the User-Name AVP, or other methods. - o otherwise, a request that needs to be sent to a specific home - server among those serving a given realm, MUST contain both the + o Otherwise, a request that needs to be sent to a specific home + server among those serving a given realm MUST contain both the Destination-Realm and Destination-Host AVPs. The Destination-Host AVP is used as described above when the destination of the request is fixed, which includes: - o Authentication requests that span multiple round trips + o Authentication requests that span multiple round trips. + + + + +Fajardo, et al. Standards Track [Page 74] + +RFC 6733 Diameter Base Protocol October 2012 + o A Diameter message that uses a security mechanism that makes use of a pre-established session key shared between the source and the final destination of the message. - o Server initiated messages that MUST be received by a specific + o Server-initiated messages that MUST be received by a specific Diameter client (e.g., access device), such as the Abort-Session- Request message, which is used to request that a particular user's session be terminated. - Note that an agent can forward a request to a host described in the - Destination-Host AVP only if the host in question is included in its - peer table (see Section 2.7). Otherwise, the request is routed based - on the Destination-Realm only (see Sections 6.1.6). + Note that an agent can only forward a request to a host described in + the Destination-Host AVP if the host in question is included in its + peer table (see Section 2.6). Otherwise, the request is routed based + on the Destination-Realm only (see Section 6.1.6). When a message is received, the message is processed in the following order: - - - - -Fajardo, et al. Expires July 24, 2011 [Page 76] - -Internet-Draft Diameter Base Protocol January 2011 - - o If the message is destined for the local host, the procedures listed in Section 6.1.4 are followed. o If the message is intended for a Diameter peer with whom the local host is able to directly communicate, the procedures listed in - Section 6.1.5 are followed. This is known as Request Forwarding. + Section 6.1.5 are followed. This is known as "Request + Forwarding". - o The procedures listed in Section 6.1.6 are followed, which is - known as Request Routing. + o The procedure listed in Section 6.1.6 is followed, which is known + as "Request Routing". - o If none of the above is successful, an answer is returned with the - Result-Code set to DIAMETER_UNABLE_TO_DELIVER, with the E-bit set. + o If none of the above are successful, an answer is returned with + the Result-Code set to DIAMETER_UNABLE_TO_DELIVER, with the 'E' + bit set. For routing of Diameter messages to work within an administrative domain, all Diameter nodes within the realm MUST be peers. - Note the processing rules contained in this section are intended to - be used as general guidelines to Diameter developers. Certain - implementations MAY use different methods than the ones described - here, and still comply with the protocol specification. See Section - 7 for more detail on error handling. + The overview contained in this section (6.1) is intended to provide + general guidelines to Diameter developers. Implementations are free + to use different methods than the ones described here as long as they + conform to the requirements specified in Sections 6.1.1 through + 6.1.9. See Section 7 for more details on error handling. 6.1.1. Originating a Request @@ -4285,33 +4195,35 @@ Internet-Draft Diameter Base Protocol January 2011 described in the application definition for that specific request, the following procedures MUST be followed: - o the Command-Code is set to the appropriate value - o the 'R' bit is set - o the End-to-End Identifier is set to a locally unique value - o the Origin-Host and Origin-Realm AVPs MUST be set to the - appropriate values, used to identify the source of the message - o the Destination-Host and Destination-Realm AVPs MUST be set to the - appropriate values as described in Section 6.1. +Fajardo, et al. Standards Track [Page 75] + +RFC 6733 Diameter Base Protocol October 2012 -6.1.2. Sending a Request - When sending a request, originated either locally, or as the result - of a forwarding or routing operation, the following procedures SHOULD - be followed: + o the Command Code is set to the appropriate value; - o The Hop-by-Hop Identifier SHOULD be set to a locally unique value. + o the 'R' bit is set; + o the End-to-End Identifier is set to a locally unique value; + o the Origin-Host and Origin-Realm AVPs MUST be set to the + appropriate values, used to identify the source of the message; + and + o the Destination-Host and Destination-Realm AVPs MUST be set to the + appropriate values, as described in Section 6.1. -Fajardo, et al. Expires July 24, 2011 [Page 77] - -Internet-Draft Diameter Base Protocol January 2011 +6.1.2. Sending a Request + When sending a request, originated either locally or as the result of + a forwarding or routing operation, the following procedures SHOULD be + followed: + + o The Hop-by-Hop Identifier SHOULD be set to a locally unique value. o The message SHOULD be saved in the list of pending requests. @@ -4328,25 +4240,34 @@ Internet-Draft Diameter Base Protocol January 2011 6.1.4. Processing Local Requests A request is known to be for local consumption when one of the - following conditions occur: + following conditions occurs: - o The Destination-Host AVP contains the local host's identity, + o The Destination-Host AVP contains the local host's identity; o The Destination-Host AVP is not present, the Destination-Realm AVP contains a realm the server is configured to process locally, and - the Diameter application is locally supported, or + the Diameter application is locally supported; or o Both the Destination-Host and the Destination-Realm are not present. + + + + +Fajardo, et al. Standards Track [Page 76] + +RFC 6733 Diameter Base Protocol October 2012 + + When a request is locally processed, the rules in Section 6.2 should be used to generate the corresponding answer. 6.1.5. Request Forwarding - Request forwarding is done using the Diameter Peer Table. The - Diameter peer table contains all of the peers that the local node is - able to directly communicate with. + Request forwarding is done using the Diameter peer table. The + Diameter peer table contains all of the peers with which the local + node is able to directly communicate. When a request is received, and the host encoded in the Destination- Host AVP is one that is present in the peer table, the message SHOULD @@ -4354,29 +4275,21 @@ Internet-Draft Diameter Base Protocol January 2011 6.1.6. Request Routing - Diameter request message routing is done via realms and application - identifiers. A Diameter message that may be forwarded by Diameter - agents (proxies, redirect or relay agents) MUST include the target + Diameter request message routing is done via realms and Application + Ids. A Diameter message that may be forwarded by Diameter agents + (proxies, redirect agents, or relay agents) MUST include the target realm in the Destination-Realm AVP. Request routing SHOULD rely on the Destination-Realm AVP and the Application Id present in the request message header to aid in the routing decision. The realm MAY be retrieved from the User-Name AVP, which is in the form of a - - - -Fajardo, et al. Expires July 24, 2011 [Page 78] - -Internet-Draft Diameter Base Protocol January 2011 - - Network Access Identifier (NAI). The realm portion of the NAI is inserted in the Destination-Realm AVP. Diameter agents MAY have a list of locally supported realms and - applications, and MAY have a list of externally supported realms and - applications. When a request is received that includes a realm + applications, and they MAY have a list of externally supported realms + and applications. When a request is received that includes a realm and/or application that is not locally supported, the message is - routed to the peer configured in the Routing Table (see Section 2.7). + routed to the peer configured in the routing table (see Section 2.7). Realm names and Application Ids are the minimum supported routing criteria, additional information may be needed to support redirect @@ -4385,13 +4298,23 @@ Internet-Draft Diameter Base Protocol January 2011 6.1.7. Predictive Loop Avoidance Before forwarding or routing a request, Diameter agents, in addition - to processing done in Section 6.1.3, SHOULD check for the presence of - candidate route's peer identity in any of the Route-Record AVPs. In - an event of the agent detecting the presence of a candidate route's - peer identity in a Route-Record AVP, the agent MUST ignore such route - for the Diameter request message and attempt alternate routes if any. - In case all the candidate routes are eliminated by the above - criteria, the agent SHOULD return DIAMETER_UNABLE_TO_DELIVER message. + to performing the processing described in Section 6.1.3, SHOULD check + for the presence of a candidate route's peer identity in any of the + Route-Record AVPs. In the event of the agent detecting the presence + of a candidate route's peer identity in a Route-Record AVP, the agent + MUST ignore such a route for the Diameter request message and attempt + alternate routes if any exist. In case all the candidate routes are + eliminated by the above criteria, the agent SHOULD return a + DIAMETER_UNABLE_TO_DELIVER message. + + + + + +Fajardo, et al. Standards Track [Page 77] + +RFC 6733 Diameter Base Protocol October 2012 + 6.1.8. Redirecting Requests @@ -4399,7 +4322,7 @@ Internet-Draft Diameter Base Protocol January 2011 to REDIRECT, it MUST reply with an answer message with the 'E' bit set, while maintaining the Hop-by-Hop Identifier in the header, and include the Result-Code AVP to DIAMETER_REDIRECT_INDICATION. Each of - the servers associated with the routing entry are added in separate + the servers associated with the routing entry are added in a separate Redirect-Host AVP. +------------------+ @@ -4417,21 +4340,13 @@ Internet-Draft Diameter Base Protocol January 2011 | Agent |<-------------| Server | +-------------+ 4. Answer +-------------+ - - - -Fajardo, et al. Expires July 24, 2011 [Page 79] - -Internet-Draft Diameter Base Protocol January 2011 - - Figure 5: Diameter Redirect Agent - The receiver of the answer message with the 'E' bit set, and the - Result-Code AVP set to DIAMETER_REDIRECT_INDICATION uses the hop-by- - hop field in the Diameter header to identify the request in the - pending message queue (see Section 5.3) that is to be redirected. If - no transport connection exists with the new agent, one is created, + The receiver of an answer message with the 'E' bit set and the + Result-Code AVP set to DIAMETER_REDIRECT_INDICATION uses the Hop-by- + Hop Identifier in the Diameter header to identify the request in the + pending message queue (see Section 5.5.4) that is to be redirected. + If no transport connection exists with the new peer, one is created, and the request is sent directly to it. Multiple Redirect-Host AVPs are allowed. The receiver of the answer @@ -4440,17 +4355,26 @@ Internet-Draft Diameter Base Protocol January 2011 When the Redirect-Host-Usage AVP included in the answer message has a non-zero value, a route entry for the redirect indications is created - and cached by the receiver. The redirect usage for such route entry - is set by the value of Redirect-Host-Usage AVP and the lifetime of - the cached route entry is set by Redirect-Max-Cache-Time AVP value. + and cached by the receiver. The redirect usage for such a route + entry is set by the value of Redirect-Host-Usage AVP and the lifetime + of the cached route entry is set by Redirect-Max-Cache-Time AVP + value. It is possible that multiple redirect indications can create multiple cached route entries differing only in their redirect usage and the peer to forward messages to. As an example, two(2) route entries that are created by two(2) redirect indications results in two(2) + + + +Fajardo, et al. Standards Track [Page 78] + +RFC 6733 Diameter Base Protocol October 2012 + + cached routes for the same realm and Application Id. However, one - has a redirect usage of ALL_SESSION where matching request will be - forwarded to one peer and the other has a redirect usage of ALL_REALM + has a redirect usage of ALL_SESSION, where matching requests will be + forwarded to one peer; the other has a redirect usage of ALL_REALM, where request are forwarded to another peer. Therefore, an incoming request that matches the realm and Application Id of both routes will need additional resolution. In such a case, a routing precedence @@ -4460,32 +4384,24 @@ Internet-Draft Diameter Base Protocol January 2011 6.1.9. Relaying and Proxying Requests A relay or proxy agent MUST append a Route-Record AVP to all requests - forwarded. The AVP contains the identity of the peer the request was - received from. + forwarded. The AVP contains the identity of the peer from which the + request was received. - The Hop-by-Hop identifier in the request is saved, and replaced with - a locally unique value. The source of the request is also saved, - which includes the IP address, port and protocol. + The Hop-by-Hop Identifier in the request is saved and replaced with a + locally unique value. The source of the request is also saved, which + includes the IP address, port, and protocol. A relay or proxy agent MAY include the Proxy-Info AVP in requests if it requires access to any local state information when the corresponding response is received. The Proxy-Info AVP has security - implications as state information is distribute to other entities. - As such, it is RECOMMMENDED to protect the content of the Proxy-Info - AVP with cryptographic mechanisms, for example by using a keyed - - - -Fajardo, et al. Expires July 24, 2011 [Page 80] - -Internet-Draft Diameter Base Protocol January 2011 - - - message digest. Such a mechanism, however, requires the management - of keys, although only locally at the Diameter server. Still, a full - description of the management of the keys used to protect the Proxy- - Info AVP is beyond the scope of this document. Below is a list of - commonly recommended: + implications as state information is distributed to other entities. + As such, it is RECOMMENDED that the content of the Proxy-Info AVP be + protected with cryptographic mechanisms, for example, by using a + keyed message digest such as HMAC-SHA1 [RFC2104]. Such a mechanism, + however, requires the management of keys, although only locally at + the Diameter server. Still, a full description of the management of + the keys used to protect the Proxy-Info AVP is beyond the scope of + this document. Below is a list of common recommendations: o The keys should be generated securely following the randomness recommendations in [RFC4086]. @@ -4494,32 +4410,39 @@ Internet-Draft Diameter Base Protocol January 2011 least 128 bits in strength. o The keys should not be used for any other purpose than generating - and verifying tickets. + and verifying instances of the Proxy-Info AVP. o The keys should be changed regularly. - o The keys should be changed if the ticket format or cryptographic + o The keys should be changed if the AVP format or cryptographic protection algorithms change. The message is then forwarded to the next hop, as identified in the - Routing Table. + routing table. + + + + +Fajardo, et al. Standards Track [Page 79] + +RFC 6733 Diameter Base Protocol October 2012 + Figure 6 provides an example of message routing using the procedures listed in these sections. - (Origin-Host=nas.example.net) (Origin-Host=nas.example.net) - (Origin-Realm=example.net) (Origin-Realm=example.net) - (Destination-Realm=example.com) (Destination- - Realm=example.com) + (Origin-Host=nas.example.net) (Origin-Host=nas.example.net) + (Origin-Realm=example.net) (Origin-Realm=example.net) + (Destination-Realm=example.com) (Destination-Realm=example.com) (Route-Record=nas.example.net) - +------+ ------> +------+ ------> +------+ - | | (Request) | | (Request) | | - | NAS +-------------------+ DRL +-------------------+ HMS | - | | | | | | - +------+ <------ +------+ <------ +------+ - example.net (Answer) example.net (Answer) example.com - (Origin-Host=hms.example.com) (Origin-Host=hms.example.com) - (Origin-Realm=example.com) (Origin-Realm=example.com) + +------+ ------> +------+ ------> +------+ + | | (Request) | | (Request) | | + | NAS +-------------------+ DRL +-------------------+ HMS | + | | | | | | + +------+ <------ +------+ <------ +------+ + example.net (Answer) example.net (Answer) example.com + (Origin-Host=hms.example.com) (Origin-Host=hms.example.com) + (Origin-Realm=example.com) (Origin-Realm=example.com) Figure 6: Routing of Diameter messages @@ -4527,15 +4450,7 @@ Internet-Draft Diameter Base Protocol January 2011 incoming messages. At a minimum, validation of the message header and relevant routing AVPs has to be done when relaying messages. Proxy agents may optionally perform more in-depth message validation - for applications it is interested in. - - - - -Fajardo, et al. Expires July 24, 2011 [Page 81] - -Internet-Draft Diameter Base Protocol January 2011 - + for applications in which it is interested. 6.2. Diameter Answer Processing @@ -4544,7 +4459,7 @@ Internet-Draft Diameter Base Protocol January 2011 additional procedures that MAY be discussed in the Diameter application defining the command: - o The same Hop-by-Hop identifier in the request is used in the + o The same Hop-by-Hop Identifier in the request is used in the answer. o The local host's identity is encoded in the Origin-Host AVP. @@ -4561,15 +4476,23 @@ Internet-Draft Diameter Base Protocol January 2011 o Any Proxy-Info AVPs in the request MUST be added to the answer message, in the same order they were present in the request. + + + +Fajardo, et al. Standards Track [Page 80] + +RFC 6733 Diameter Base Protocol October 2012 + + o The 'P' bit is set to the same value as the one in the request. o The same End-to-End identifier in the request is used in the answer. - Note that the error messages (see Section 7.3) are also subjected to + Note that the error messages (see Section 7) are also subjected to the above processing rules. -6.2.1. Processing received Answers +6.2.1. Processing Received Answers A Diameter client or proxy MUST match the Hop-by-Hop Identifier in an answer received against the list of pending requests. The @@ -4579,31 +4502,23 @@ Internet-Draft Diameter Base Protocol January 2011 6.2.2. Relaying and Proxying Answers - If the answer is for a request which was proxied or relayed, the - agent MUST restore the original value of the Diameter header's Hop- - by-Hop Identifier field. + If the answer is for a request that was proxied or relayed, the agent + MUST restore the original value of the Diameter header's Hop-by-Hop + Identifier field. If the last Proxy-Info AVP in the message is targeted to the local Diameter server, the AVP MUST be removed before the answer is - - - -Fajardo, et al. Expires July 24, 2011 [Page 82] - -Internet-Draft Diameter Base Protocol January 2011 - - forwarded. If a relay or proxy agent receives an answer with a Result-Code AVP indicating a failure, it MUST NOT modify the contents of the AVP. - Any additional local errors detected SHOULD be logged, but not + Any additional local errors detected SHOULD be logged but not reflected in the Result-Code AVP. If the agent receives an answer message with a Result-Code AVP indicating success, and it wishes to modify the AVP to indicate an error, it MUST modify the Result-Code AVP to contain the appropriate error in the message destined towards - the access device as well as include the Error-Reporting-Host AVP and - it MUST issue an STR on behalf of the access device towards the + the access device as well as include the Error-Reporting-Host AVP; it + MUST also issue an STR on behalf of the access device towards the Diameter server. The agent MUST then send the answer to the host that it received the @@ -4612,10 +4527,19 @@ Internet-Draft Diameter Base Protocol January 2011 6.3. Origin-Host AVP The Origin-Host AVP (AVP Code 264) is of type DiameterIdentity, and - MUST be present in all Diameter messages. This AVP identifies the + it MUST be present in all Diameter messages. This AVP identifies the endpoint that originated the Diameter message. Relay agents MUST NOT modify this AVP. + + + + +Fajardo, et al. Standards Track [Page 81] + +RFC 6733 Diameter Base Protocol October 2012 + + The value of the Origin-Host AVP is guaranteed to be unique within a single host. @@ -4638,17 +4562,9 @@ Internet-Draft Diameter Base Protocol January 2011 The Destination-Host AVP (AVP Code 293) is of type DiameterIdentity. This AVP MUST be present in all unsolicited agent initiated messages, - MAY be present in request messages, and MUST NOT be present in Answer + MAY be present in request messages, and MUST NOT be present in answer messages. - - - -Fajardo, et al. Expires July 24, 2011 [Page 83] - -Internet-Draft Diameter Base Protocol January 2011 - - The absence of the Destination-Host AVP will cause a message to be sent to any Diameter server supporting the application within the realm specified in Destination-Realm AVP. @@ -4658,20 +4574,28 @@ Internet-Draft Diameter Base Protocol January 2011 6.6. Destination-Realm AVP - The Destination-Realm AVP (AVP Code 283) is of type DiameterIdentity, - and contains the realm the message is to be routed to. The - Destination-Realm AVP MUST NOT be present in Answer messages. - Diameter Clients insert the realm portion of the User-Name AVP. + The Destination-Realm AVP (AVP Code 283) is of type DiameterIdentity + and contains the realm to which the message is to be routed. The + Destination-Realm AVP MUST NOT be present in answer messages. + Diameter clients insert the realm portion of the User-Name AVP. Diameter servers initiating a request message use the value of the Origin-Realm AVP from a previous message received from the intended target host (unless it is known a priori). When present, the Destination-Realm AVP is used to perform message routing decisions. - An ABNF for a request message that includes the Destination-Realm AVP + The CCF for a request message that includes the Destination-Realm AVP SHOULD list the Destination-Realm AVP as a required AVP (an AVP - indicated as {AVP}) otherwise the message is inherently a non- + indicated as {AVP}); otherwise, the message is inherently a non- routable message. + + + +Fajardo, et al. Standards Track [Page 82] + +RFC 6733 Diameter Base Protocol October 2012 + + This AVP SHOULD be placed as close to the Diameter header as possible. @@ -4692,19 +4616,11 @@ Internet-Draft Diameter Base Protocol January 2011 The Proxy-Info AVP (AVP Code 284) is of type Grouped. This AVP contains the identity and local state information of the Diameter node that creates and adds it to a message. The Grouped Data field - has the following ABNF grammar: + has the following CCF grammar: Proxy-Info ::= < AVP Header: 284 > { Proxy-Host } { Proxy-State } - - - -Fajardo, et al. Expires July 24, 2011 [Page 84] - -Internet-Draft Diameter Base Protocol January 2011 - - * [ AVP ] 6.7.3. Proxy-Host AVP @@ -4728,10 +4644,18 @@ Internet-Draft Diameter Base Protocol January 2011 Application-Id AVP MUST match the Application Id present in the Diameter message header. + + + +Fajardo, et al. Standards Track [Page 83] + +RFC 6733 Diameter Base Protocol October 2012 + + 6.9. Acct-Application-Id AVP The Acct-Application-Id AVP (AVP Code 259) is of type Unsigned32 and - is used in order to advertise support of the Accounting portion of an + is used in order to advertise support of the accounting portion of an application (see Section 2.4). If present in a message other than CER and CEA, the value of the Acct-Application-Id AVP MUST match the Application Id present in the Diameter message header. @@ -4740,10 +4664,10 @@ Internet-Draft Diameter Base Protocol January 2011 The Inband-Security-Id AVP (AVP Code 299) is of type Unsigned32 and is used in order to advertise support of the security portion of the - application. The use of this AVP in CER and CEA messages is no - longer recommended. Instead, discovery of a Diameter entities - security capabilities can be done either through static configuration - or via Diameter Peer Discovery described in Section 5.2. + application. The use of this AVP in CER and CEA messages is NOT + RECOMMENDED. Instead, discovery of a Diameter entity's security + capabilities can be done either through static configuration or via + Diameter Peer Discovery as described in Section 5.2. The following values are supported: @@ -4753,36 +4677,37 @@ Internet-Draft Diameter Base Protocol January 2011 This peer does not support TLS/TCP and DTLS/SCTP. This is the default value, if the AVP is omitted. - - - -Fajardo, et al. Expires July 24, 2011 [Page 85] - -Internet-Draft Diameter Base Protocol January 2011 - - TLS 1 - This node supports TLS/TCP and DTLS/SCTP security, as defined by - [RFC5246]. + This node supports TLS/TCP [RFC5246] and DTLS/SCTP [RFC6083] + security. 6.11. Vendor-Specific-Application-Id AVP The Vendor-Specific-Application-Id AVP (AVP Code 260) is of type Grouped and is used to advertise support of a vendor-specific - Diameter Application. Exactly one instance of either Auth- + Diameter application. Exactly one instance of either Auth- Application-Id or Acct-Application-Id AVP MUST be present. The Application Id carried by either Auth-Application-Id or Acct- - Application-Id AVP MUST comply with vendor specific Application Id - assignment described in Sec 11.3. It MUST also match the Application - Id present in the Diameter header except when used in a CER or CEA - message. + Application-Id AVP MUST comply with vendor-specific Application Id + assignment described in Section 11.3. It MUST also match the + Application Id present in the Diameter header except when used in a + CER or CEA message. The Vendor-Id AVP is an informational AVP pertaining to the vendor who may have authorship of the vendor-specific Diameter application. It MUST NOT be used as a means of defining a completely separate vendor-specific Application Id space. + + + + +Fajardo, et al. Standards Track [Page 84] + +RFC 6733 Diameter Base Protocol October 2012 + + The Vendor-Specific-Application-Id AVP SHOULD be placed as close to the Diameter header as possible. @@ -4795,9 +4720,9 @@ Internet-Draft Diameter Base Protocol January 2011 A Vendor-Specific-Application-Id AVP MUST contain exactly one of either Auth-Application-Id or Acct-Application-Id. If a Vendor- - Specific-Application-Id is received without any of these two AVPs, + Specific-Application-Id is received without one of these two AVPs, then the recipient SHOULD issue an answer with a Result-Code set to - DIAMETER_MISSING_AVP. The answer SHOULD also include a Failed-AVP + DIAMETER_MISSING_AVP. The answer SHOULD also include a Failed-AVP, which MUST contain an example of an Auth-Application-Id AVP and an Acct-Application-Id AVP. @@ -4805,22 +4730,13 @@ Internet-Draft Diameter Base Protocol January 2011 Auth-Application-Id and Acct-Application-Id, then the recipient MUST issue an answer with Result-Code set to DIAMETER_AVP_OCCURS_TOO_MANY_TIMES. The answer MUST also include a - Failed-AVP which MUST contain the received Auth-Application-Id AVP + Failed-AVP, which MUST contain the received Auth-Application-Id AVP and Acct-Application-Id AVP. - - - - -Fajardo, et al. Expires July 24, 2011 [Page 86] - -Internet-Draft Diameter Base Protocol January 2011 - - 6.12. Redirect-Host AVP The Redirect-Host AVP (AVP Code 292) is of type DiameterURI. One or - more of instances of this AVP MUST be present if the answer message's + more instances of this AVP MUST be present if the answer message's 'E' bit is set and the Result-Code AVP is set to DIAMETER_REDIRECT_INDICATION. @@ -4836,63 +4752,55 @@ Internet-Draft Diameter Base Protocol January 2011 This AVP MAY be present in answer messages whose 'E' bit is set and the Result-Code AVP is set to DIAMETER_REDIRECT_INDICATION. - When present, this AVP provides a hints about how the routing entry + When present, this AVP provides hints about how the routing entry resulting from the Redirect-Host is to be used. The following values are supported: + + +Fajardo, et al. Standards Track [Page 85] + +RFC 6733 Diameter Base Protocol October 2012 + + DONT_CACHE 0 The host specified in the Redirect-Host AVP SHOULD NOT be cached. This is the default value. - ALL_SESSION 1 All messages within the same session, as defined by the same value of the Session-ID AVP SHOULD be sent to the host specified in the Redirect-Host AVP. - ALL_REALM 2 All messages destined for the realm requested SHOULD be sent to the host specified in the Redirect-Host AVP. - REALM_AND_APPLICATION 3 All messages for the application requested to the realm specified SHOULD be sent to the host specified in the Redirect-Host AVP. - - - -Fajardo, et al. Expires July 24, 2011 [Page 87] - -Internet-Draft Diameter Base Protocol January 2011 - - ALL_APPLICATION 4 All messages for the application requested SHOULD be sent to the host specified in the Redirect-Host AVP. - ALL_HOST 5 All messages that would be sent to the host that generated the Redirect-Host SHOULD be sent to the host specified in the - Redirect- Host AVP. - + Redirect-Host AVP. ALL_USER 6 All messages for the user requested SHOULD be sent to the host specified in the Redirect-Host AVP. - - When multiple cached routes are created by redirect indications and they differ only in redirect usage and peers to forward requests to (see Section 6.1.8), a precedence rule MUST be applied to the @@ -4902,6 +4810,16 @@ Internet-Draft Diameter Base Protocol January 2011 other as they appear. The order is as follows: + + + + + +Fajardo, et al. Standards Track [Page 86] + +RFC 6733 Diameter Base Protocol October 2012 + + 1. ALL_SESSION 2. ALL_USER @@ -4917,58 +4835,31 @@ Internet-Draft Diameter Base Protocol January 2011 6.14. Redirect-Max-Cache-Time AVP The Redirect-Max-Cache-Time AVP (AVP Code 262) is of type Unsigned32. - This AVP MUST be present in answer messages whose 'E' bit is set, the - Result-Code AVP is set to DIAMETER_REDIRECT_INDICATION and the - Redirect-Host-Usage AVP set to a non-zero value. - - - - -Fajardo, et al. Expires July 24, 2011 [Page 88] - -Internet-Draft Diameter Base Protocol January 2011 - + This AVP MUST be present in answer messages whose 'E' bit is set, + whose Result-Code AVP is set to DIAMETER_REDIRECT_INDICATION, and + whose Redirect-Host-Usage AVP set to a non-zero value. This AVP contains the maximum number of seconds the peer and route table entries, created as a result of the Redirect-Host, SHOULD be cached. Note that once a host is no longer reachable, any associated - cache, peer and routing table entries MUST be deleted. - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - + cache, peer, and routing table entries MUST be deleted. +7. Error Handling + There are two different types of errors in Diameter; protocol errors + and application errors. A protocol error is one that occurs at the + base protocol level and MAY require per-hop attention (e.g., a + message routing error). Application errors, on the other hand, + generally occur due to a problem with a function specified in a + Diameter application (e.g., user authentication, missing AVP). + Result-Code AVP values that are used to report protocol errors MUST + only be present in answer messages whose 'E' bit is set. When a + request message is received that causes a protocol error, an answer + message is returned with the 'E' bit set, and the Result-Code AVP is + set to the appropriate protocol error value. As the answer is sent + back towards the originator of the request, each proxy or relay agent + MAY take action on the message. @@ -4980,27 +4871,10 @@ Internet-Draft Diameter Base Protocol January 2011 -Fajardo, et al. Expires July 24, 2011 [Page 89] +Fajardo, et al. Standards Track [Page 87] -Internet-Draft Diameter Base Protocol January 2011 - - -7. Error Handling +RFC 6733 Diameter Base Protocol October 2012 - There are two different types of errors in Diameter; protocol and - application errors. A protocol error is one that occurs at the base - protocol level, and MAY require per hop attention (e.g., message - routing error). Application errors, on the other hand, generally - occur due to a problem with a function specified in a Diameter - application (e.g., user authentication, missing AVP). - - Result-Code AVP values that are used to report protocol errors MUST - only be present in answer messages whose 'E' bit is set. When a - request message is received that causes a protocol error, an answer - message is returned with the 'E' bit set, and the Result-Code AVP is - set to the appropriate protocol error value. As the answer is sent - back towards the originator of the request, each proxy or relay agent - MAY take action on the message. 1. Request +---------+ Link Broken +-------------------------->|Diameter |----///----+ @@ -5014,7 +4888,7 @@ Internet-Draft Diameter Base Protocol January 2011 | Relay 3 |-----------+ +---------+ - Figure 7: Example of Protocol Error causing answer message + Figure 7: Example of Protocol Error Causing Answer Message Figure 7 provides an example of a message forwarded upstream by a Diameter relay. When the message is received by Relay 2, and it @@ -5032,40 +4906,40 @@ Internet-Draft Diameter Base Protocol January 2011 +---------+ 4. Answer +---------+ 3. Answer +---------+ (Missing AVP) (Missing AVP) - Figure 8: Example of Application Error Answer message - - - -Fajardo, et al. Expires July 24, 2011 [Page 90] - -Internet-Draft Diameter Base Protocol January 2011 - + Figure 8: Example of Application Error Answer Message Figure 8 provides an example of a Diameter message that caused an application error. When application errors occur, the Diameter - entity reporting the error clears the 'R' bit in the Command Flags, + entity reporting the error clears the 'R' bit in the Command Flags and adds the Result-Code AVP with the proper value. Application - errors do not require any proxy or relay agent involvement, and - therefore the message would be forwarded back to the originator of + errors do not require any proxy or relay agent involvement; + therefore, the message would be forwarded back to the originator of the request. In the case where the answer message itself contains errors, any related session SHOULD be terminated by sending an STR or ASR message. The Termination-Cause AVP in the STR MAY be filled with the appropriate value to indicate the cause of the error. An application - MAY also send an application-specific request instead of STR or ASR - to signal the error in the case where no state is maintained or to - allow for some form of error recovery with the corresponding Diameter - entity. + MAY also send an application-specific request instead of an STR or + ASR message to signal the error in the case where no state is + maintained or to allow for some form of error recovery with the + corresponding Diameter entity. + + + +Fajardo, et al. Standards Track [Page 88] + +RFC 6733 Diameter Base Protocol October 2012 + There are certain Result-Code AVP application errors that require additional AVPs to be present in the answer. In these cases, the Diameter node that sets the Result-Code AVP to indicate the error - MUST add the AVPs. Examples are: + MUST add the AVPs. Examples are as follows: o A request with an unrecognized AVP is received with the 'M' bit - (Mandatory bit) set, causes an answer to be sent with the Result- - Code AVP set to DIAMETER_AVP_UNSUPPORTED, and the Failed-AVP AVP + (Mandatory bit) set causes an answer to be sent with the Result- + Code AVP set to DIAMETER_AVP_UNSUPPORTED and the Failed-AVP AVP containing the offending AVP. o A request with an AVP that is received with an unrecognized value @@ -5073,46 +4947,46 @@ Internet-Draft Diameter Base Protocol January 2011 DIAMETER_INVALID_AVP_VALUE, with the Failed-AVP AVP containing the AVP causing the error. - o A received command which is missing AVP(s) that are defined as - required in the commands ABNF; examples are AVPs indicated as + o A received command that is missing AVPs that are defined as + required in the commands CCF; examples are AVPs indicated as {AVP}. The receiver issues an answer with the Result-Code set to - DIAMETER_MISSING_AVP, and creates an AVP with the AVP Code and + DIAMETER_MISSING_AVP and creates an AVP with the AVP Code and other fields set as expected in the missing AVP. The created AVP - is then added to the Failed- AVP AVP. + is then added to the Failed-AVP AVP. The Result-Code AVP describes the error that the Diameter node encountered in its processing. In case there are multiple errors, the Diameter node MUST report only the first error it encountered - (detected possibly in some implementation dependent order). The + (detected possibly in some implementation-dependent order). The specific errors that can be described by this AVP are described in the following section. +7.1. Result-Code AVP + The Result-Code AVP (AVP Code 268) is of type Unsigned32 and + indicates whether a particular request was completed successfully or + an error occurred. All Diameter answer messages in IETF-defined + Diameter application specifications MUST include one Result-Code AVP. + A non-successful Result-Code AVP (one containing a non-2xxx value + other than DIAMETER_REDIRECT_INDICATION) MUST include the Error- + Reporting-Host AVP if the host setting the Result-Code AVP is + different from the identity encoded in the Origin-Host AVP. + The Result-Code data field contains an IANA-managed 32-bit address + space representing errors (see Section 11.3.2). Diameter provides + the following classes of errors, all identified by the thousands + digit in the decimal notation: -Fajardo, et al. Expires July 24, 2011 [Page 91] - -Internet-Draft Diameter Base Protocol January 2011 -7.1. Result-Code AVP - The Result-Code AVP (AVP Code 268) is of type Unsigned32 and - indicates whether a particular request was completed successfully or - whether an error occurred. All Diameter answer messages in IETF - defined Diameter application specification MUST include one Result- - Code AVP. A non-successful Result-Code AVP (one containing a non - 2xxx value other than DIAMETER_REDIRECT_INDICATION) MUST include the - Error-Reporting-Host AVP if the host setting the Result-Code AVP is - different from the identity encoded in the Origin-Host AVP. +Fajardo, et al. Standards Track [Page 89] + +RFC 6733 Diameter Base Protocol October 2012 - The Result-Code data field contains an IANA-managed 32-bit address - space representing errors (see Section 11.4). Diameter provides the - following classes of errors, all identified by the thousands digit in - the decimal notation: o 1xxx (Informational) @@ -5124,7 +4998,7 @@ Internet-Draft Diameter Base Protocol January 2011 o 5xxx (Permanent Failure) - A non-recognized class (one whose first digit is not defined in this + An unrecognized class (one whose first digit is not defined in this section) MUST be handled as a permanent failure. 7.1.1. Informational @@ -5133,7 +5007,6 @@ Internet-Draft Diameter Base Protocol January 2011 requester that a request could not be satisfied, and additional action is required on its part before access is granted. - DIAMETER_MULTI_ROUND_AUTH 1001 This informational error is returned by a Diameter server to @@ -5141,24 +5014,11 @@ Internet-Draft Diameter Base Protocol January 2011 used requires multiple round trips, and a subsequent request needs to be issued in order for access to be granted. - - - - - - - -Fajardo, et al. Expires July 24, 2011 [Page 92] - -Internet-Draft Diameter Base Protocol January 2011 - - 7.1.2. Success Errors that fall within the Success category are used to inform a peer that a request has been successfully completed. - DIAMETER_SUCCESS 2001 The request was successfully completed. @@ -5174,27 +5034,33 @@ Internet-Draft Diameter Base Protocol January 2011 Errors that fall within the Protocol Error category SHOULD be treated on a per-hop basis, and Diameter proxies MAY attempt to correct the error, if it is possible. Note that these errors MUST only be used - in answer messages whose 'E' bit is set. This document omits some - error codes defined in [RFC3588]. To provide backward compatibility - with [RFC3588] implementations these error code values are not re- - used and hence the error codes values enumerated below are non- - sequential. + in answer messages whose 'E' bit is set. + + +Fajardo, et al. Standards Track [Page 90] + +RFC 6733 Diameter Base Protocol October 2012 + + + DIAMETER_COMMAND_UNSUPPORTED 3001 + + This error code is used when a Diameter entity receives a message + with a Command Code that it does not support. + DIAMETER_UNABLE_TO_DELIVER 3002 - This error is given when Diameter can not deliver the message to + This error is given when Diameter cannot deliver the message to the destination, either because no host within the realm supporting the required application was available to process the - request, or because Destination-Host AVP was given without the + request or because the Destination-Host AVP was given without the associated Destination-Realm AVP. - DIAMETER_REALM_NOT_SERVED 3003 The intended realm of the request is not recognized. - DIAMETER_TOO_BUSY 3004 When returned, a Diameter node SHOULD attempt to send the message @@ -5202,13 +5068,6 @@ Internet-Draft Diameter Base Protocol January 2011 specific server is requested, and it cannot provide the requested service. - - -Fajardo, et al. Expires July 24, 2011 [Page 93] - -Internet-Draft Diameter Base Protocol January 2011 - - DIAMETER_LOOP_DETECTED 3005 An agent detected a loop while trying to get the message to the @@ -5216,121 +5075,110 @@ Internet-Draft Diameter Base Protocol January 2011 if one is available, but the peer reporting the error has identified a configuration problem. - DIAMETER_REDIRECT_INDICATION 3006 A redirect agent has determined that the request could not be - satisfied locally and the initiator of the request SHOULD direct + satisfied locally, and the initiator of the request SHOULD direct the request directly to the server, whose contact information has been added to the response. When set, the Redirect-Host AVP MUST be present. - DIAMETER_APPLICATION_UNSUPPORTED 3007 A request was sent for an application that is not supported. + DIAMETER_INVALID_HDR_BITS 3008 - DIAMETER_INVALID_BIT_IN_HEADER 3011 + A request was received whose bits in the Diameter header were set + either to an invalid combination or to a value that is + inconsistent with the Command Code's definition. - This error is returned when a reserved bit in the Diameter header - is set to one (1) or the bits in the Diameter header defined in - Section 3 are set incorrectly. - DIAMETER_INVALID_MESSAGE_LENGTH 3012 +Fajardo, et al. Standards Track [Page 91] + +RFC 6733 Diameter Base Protocol October 2012 - This error is returned when a request is received with an invalid - message length. + DIAMETER_INVALID_AVP_BITS 3009 + + A request was received that included an AVP whose flag bits are + set to an unrecognized value or that is inconsistent with the + AVP's definition. + + DIAMETER_UNKNOWN_PEER 3010 + + A CER was received from an unknown peer. 7.1.4. Transient Failures Errors that fall within the transient failures category are used to inform a peer that the request could not be satisfied at the time it - was received, but MAY be able to satisfy the request in the future. + was received but MAY be able to satisfy the request in the future. Note that these errors MUST be used in answer messages whose 'E' bit is not set. - DIAMETER_AUTHENTICATION_REJECTED 4001 The authentication process for the user failed, most likely due to an invalid password used by the user. Further attempts MUST only - - - -Fajardo, et al. Expires July 24, 2011 [Page 94] - -Internet-Draft Diameter Base Protocol January 2011 - - be tried after prompting the user for a new password. - DIAMETER_OUT_OF_SPACE 4002 A Diameter node received the accounting request but was unable to commit it to stable storage due to a temporary lack of space. - ELECTION_LOST 4003 The peer has determined that it has lost the election process and has therefore disconnected the transport connection. - 7.1.5. Permanent Failures Errors that fall within the permanent failures category are used to - inform the peer that the request failed, and should not be attempted + inform the peer that the request failed and should not be attempted again. Note that these errors SHOULD be used in answer messages whose 'E' bit is not set. In error conditions where it is not - possible or efficient to compose application-specific answer grammar - then answer messages with E-bit set and complying to the grammar - described in 7.2 MAY also be used for permanent errors. + possible or efficient to compose application-specific answer grammar, + answer messages with the 'E' bit set and which comply to the grammar + described in Section 7.2 MAY also be used for permanent errors. - To provide backward compatibility with existing implementations that - follow [RFC3588], some of the error values that have previously been - used in this category by [RFC3588] will not be re-used. Therefore - the error values enumerated here may be non-sequential. - DIAMETER_AVP_UNSUPPORTED 5001 - The peer received a message that contained an AVP that is not - recognized or supported and was marked with the Mandatory bit. A - Diameter message with this error MUST contain one or more Failed- - AVP AVP containing the AVPs that caused the failure. - DIAMETER_UNKNOWN_SESSION_ID 5002 - The request contained an unknown Session-Id. +Fajardo, et al. Standards Track [Page 92] + +RFC 6733 Diameter Base Protocol October 2012 - DIAMETER_AUTHORIZATION_REJECTED 5003 - A request was received for which the user could not be authorized. - This error could occur if the service requested is not permitted + DIAMETER_AVP_UNSUPPORTED 5001 + The peer received a message that contained an AVP that is not + recognized or supported and was marked with the 'M' (Mandatory) + bit. A Diameter message with this error MUST contain one or more + Failed-AVP AVPs containing the AVPs that caused the failure. + DIAMETER_UNKNOWN_SESSION_ID 5002 -Fajardo, et al. Expires July 24, 2011 [Page 95] - -Internet-Draft Diameter Base Protocol January 2011 + The request contained an unknown Session-Id. + DIAMETER_AUTHORIZATION_REJECTED 5003 + A request was received for which the user could not be authorized. + This error could occur if the service requested is not permitted to the user. - DIAMETER_INVALID_AVP_VALUE 5004 The request contained an AVP with an invalid value in its data portion. A Diameter message indicating this error MUST include the offending AVPs within a Failed-AVP AVP. - DIAMETER_MISSING_AVP 5005 The request did not contain an AVP that is required by the Command @@ -5340,21 +5188,28 @@ Internet-Draft Diameter Base Protocol January 2011 Vendor-Id if applicable. The value field of the missing AVP should be of correct minimum length and contain zeroes. - DIAMETER_RESOURCES_EXCEEDED 5006 A request was received that cannot be authorized because the user has already expended allowed resources. An example of this error - condition is a user that is restricted to one dial-up PPP port, - attempts to establish a second PPP connection. - + condition is when a user that is restricted to one dial-up PPP + port attempts to establish a second PPP connection. DIAMETER_CONTRADICTING_AVPS 5007 The Home Diameter server has detected AVPs in the request that - contradicted each other, and is not willing to provide service to - the user. The Failed-AVP AVPs MUST be present which contains the - AVPs that contradicted each other. + contradicted each other, and it is not willing to provide service + to the user. The Failed-AVP AVP MUST be present, which contain + the AVPs that contradicted each other. + + + + + + +Fajardo, et al. Standards Track [Page 93] + +RFC 6733 Diameter Base Protocol October 2012 DIAMETER_AVP_NOT_ALLOWED 5008 @@ -5363,22 +5218,12 @@ Internet-Draft Diameter Base Protocol January 2011 Failed-AVP AVP MUST be included and contain a copy of the offending AVP. - DIAMETER_AVP_OCCURS_TOO_MANY_TIMES 5009 A message was received that included an AVP that appeared more often than permitted in the message definition. The Failed-AVP AVP MUST be included and contain a copy of the first instance of - - - -Fajardo, et al. Expires July 24, 2011 [Page 96] - -Internet-Draft Diameter Base Protocol January 2011 - - - the offending AVP that exceeded the maximum number of occurrences - + the offending AVP that exceeded the maximum number of occurrences. DIAMETER_NO_COMMON_APPLICATION 5010 @@ -5386,18 +5231,21 @@ Internet-Draft Diameter Base Protocol January 2011 whereby no applications are common between the CER sending peer and the CER receiving peer. - DIAMETER_UNSUPPORTED_VERSION 5011 This error is returned when a request was received, whose version number is unsupported. - DIAMETER_UNABLE_TO_COMPLY 5012 This error is returned when a request is rejected for unspecified reasons. + DIAMETER_INVALID_BIT_IN_HEADER 5013 + + This error is returned when a reserved bit in the Diameter header + is set to one (1) or the bits in the Diameter header are set + incorrectly. DIAMETER_INVALID_AVP_LENGTH 5014 @@ -5407,56 +5255,41 @@ Internet-Draft Diameter Base Protocol January 2011 value exceeds the message length or is less than the minimum AVP header length, it is sufficient to include the offending AVP header and a zero filled payload of the minimum required length - for the payloads data type. If the AVP is a grouped AVP, the - grouped AVP header with an empty payload would be sufficient to + for the payloads data type. If the AVP is a Grouped AVP, the + Grouped AVP header with an empty payload would be sufficient to indicate the offending AVP. In the case where the offending AVP header cannot be fully decoded when the AVP length is less than - the minimum AVP header length, it is sufficient to include an - offending AVP header that is formulated by padding the incomplete - AVP header with zero up to the minimum AVP header length. - - - DIAMETER_NO_COMMON_SECURITY 5017 - - This error is returned when a CER message is received, and there - are no common security mechanisms supported between the peers. A - Capabilities-Exchange-Answer (CEA) MUST be returned with the - Result-Code AVP set to DIAMETER_NO_COMMON_SECURITY. - - -Fajardo, et al. Expires July 24, 2011 [Page 97] +Fajardo, et al. Standards Track [Page 94] -Internet-Draft Diameter Base Protocol January 2011 +RFC 6733 Diameter Base Protocol October 2012 - DIAMETER_UNKNOWN_PEER 5018 - - A CER was received from an unknown peer. - - - DIAMETER_COMMAND_UNSUPPORTED 5019 - - This error code is used when a Diameter entity receives a message - with a Command Code that it does not support. - + the minimum AVP header length, it is sufficient to include an + offending AVP header that is formulated by padding the incomplete + AVP header with zero up to the minimum AVP header length. - DIAMETER_INVALID_HDR_BITS 5020 + DIAMETER_INVALID_MESSAGE_LENGTH 5015 - A request was received whose bits in the Diameter header were - either set to an invalid combination, or to a value that is - inconsistent with the command code's definition. + This error is returned when a request is received with an invalid + message length. + DIAMETER_INVALID_AVP_BIT_COMBO 5016 - DIAMETER_INVALID_AVP_BITS 5021 + The request contained an AVP with which is not allowed to have the + given value in the AVP Flags field. A Diameter message indicating + this error MUST include the offending AVPs within a Failed-AVP + AVP. - A request was received that included an AVP whose flag bits are - set to an unrecognized value, or that is inconsistent with the - AVP's definition. + DIAMETER_NO_COMMON_SECURITY 5017 + This error is returned when a CER message is received, and there + are no common security mechanisms supported between the peers. A + Capabilities-Exchange-Answer (CEA) message MUST be returned with + the Result-Code AVP set to DIAMETER_NO_COMMON_SECURITY. 7.2. Error Bit @@ -5465,12 +5298,12 @@ Internet-Draft Diameter Base Protocol January 2011 the 'E' bit MUST NOT be sent as a response to an answer message. Note that a message with the 'E' bit set is still subjected to the processing rules defined in Section 6.2. When set, the answer - message will not conform to the ABNF specification for the command, - and will instead conform to the following ABNF: + message will not conform to the CCF specification for the command; + instead, it and will conform to the following CCF: Message Format - <answer-message> ::= < Diameter Header: code, ERR [PXY] > + <answer-message> ::= < Diameter Header: code, ERR [, PXY] > 0*1< Session-Id > { Origin-Host } { Origin-Realm } @@ -5481,16 +5314,16 @@ Internet-Draft Diameter Base Protocol January 2011 [ Failed-AVP ] [ Experimental-Result ] * [ Proxy-Info ] + * [ AVP ] -Fajardo, et al. Expires July 24, 2011 [Page 98] + +Fajardo, et al. Standards Track [Page 95] -Internet-Draft Diameter Base Protocol January 2011 +RFC 6733 Diameter Base Protocol October 2012 - * [ AVP ] - Note that the code used in the header is the same than the one found in the request message, but with the 'R' bit cleared and the 'E' bit set. The 'P' bit in the header is set to the same value as the one @@ -5499,10 +5332,10 @@ Internet-Draft Diameter Base Protocol January 2011 7.3. Error-Message AVP The Error-Message AVP (AVP Code 281) is of type UTF8String. It MAY - accompany a Result-Code AVP as a human readable error message. The + accompany a Result-Code AVP as a human-readable error message. The Error-Message AVP is not intended to be useful in an environment where error messages are processed automatically. It SHOULD NOT be - expected that the content of this AVP is parsed by network entities. + expected that the content of this AVP be parsed by network entities. 7.4. Error-Reporting-Host AVP @@ -5511,8 +5344,8 @@ Internet-Draft Diameter Base Protocol January 2011 host that sent the Result-Code AVP to a value other than 2001 (Success), only if the host setting the Result-Code is different from the one encoded in the Origin-Host AVP. This AVP is intended to be - used for troubleshooting purposes, and MUST be set when the Result- - Code AVP indicates a failure. + used for troubleshooting purposes, and it MUST be set when the + Result-Code AVP indicates a failure. 7.5. Failed-AVP AVP @@ -5520,43 +5353,44 @@ Internet-Draft Diameter Base Protocol January 2011 debugging information in cases where a request is rejected or not fully processed due to erroneous information in a specific AVP. The value of the Result-Code AVP will provide information on the reason - for the Failed-AVP AVP. A Diameter message SHOULD contain only one - Failed-AVP that corresponds to the error indicated by the Result-Code - AVP. For practical purposes, this Failed-AVP would typically refer - to the first AVP processing error that a Diameter node encounters. + for the Failed-AVP AVP. A Diameter answer message SHOULD contain an + instance of the Failed-AVP AVP that corresponds to the error + indicated by the Result-Code AVP. For practical purposes, this + Failed-AVP would typically refer to the first AVP processing error + that a Diameter node encounters. The possible reasons for this AVP are the presence of an improperly constructed AVP, an unsupported or unrecognized AVP, an invalid AVP value, the omission of a required AVP, the presence of an explicitly - excluded AVP (see tables in Section 10), or the presence of two or - more occurrences of an AVP which is restricted to 0, 1, or 0-1 + excluded AVP (see tables in Section 10) or the presence of two or + more occurrences of an AVP that is restricted to 0, 1, or 0-1 occurrences. A Diameter message SHOULD contain one Failed-AVP AVP, containing the entire AVP that could not be processed successfully. If the failure reason is omission of a required AVP, an AVP with the missing AVP - code, the missing vendor id, and a zero filled payload of the minimum + code, the missing Vendor-Id, and a zero-filled payload of the minimum required length for the omitted AVP will be added. If the failure + reason is an invalid AVP length where the reported length is less -Fajardo, et al. Expires July 24, 2011 [Page 99] +Fajardo, et al. Standards Track [Page 96] -Internet-Draft Diameter Base Protocol January 2011 +RFC 6733 Diameter Base Protocol October 2012 - reason is an invalid AVP length where the reported length is less than the minimum AVP header length or greater than the reported - message length, a copy of the offending AVP header and a zero filled + message length, a copy of the offending AVP header and a zero-filled payload of the minimum required length SHOULD be added. - In the case where the offending AVP is embedded within a grouped AVP, - the Failed-AVP MAY contain the grouped AVP which in turn contains the - single offending AVP. The same method MAY be employed if the grouped - AVP itself is embedded in yet another grouped AVP and so on. In this - case, the Failed-AVP MAY contain the grouped AVP hierarchy up to the - single offending AVP. This enables the recipient to detect the - location of the offending AVP when embedded in a group. + In the case where the offending AVP is embedded within a Grouped AVP, + the Failed-AVP MAY contain the grouped AVP, which in turn contains + the single offending AVP. The same method MAY be employed if the + grouped AVP itself is embedded in yet another grouped AVP and so on. + In this case, the Failed-AVP MAY contain the grouped AVP hierarchy up + to the single offending AVP. This enables the recipient to detect + the location of the offending AVP when embedded in a group. AVP Format @@ -5577,7 +5411,7 @@ Internet-Draft Diameter Base Protocol January 2011 { Experimental-Result-Code } The Vendor-Id AVP (see Section 5.3.3) in this grouped AVP identifies - the vendor responsible for the assignment of the result code which + the vendor responsible for the assignment of the result code that follows. All Diameter answer messages defined in vendor-specific applications MUST include either one Result-Code AVP or one Experimental-Result AVP. @@ -5590,23 +5424,24 @@ Internet-Draft Diameter Base Protocol January 2011 It is recommended that vendor-specific result codes follow the same conventions given for the Result-Code AVP regarding the different - types of result codes and the handling of errors (for non 2xxx + types of result codes and the handling of errors (for non-2xxx values). -Fajardo, et al. Expires July 24, 2011 [Page 100] + +Fajardo, et al. Standards Track [Page 97] -Internet-Draft Diameter Base Protocol January 2011 +RFC 6733 Diameter Base Protocol October 2012 8. Diameter User Sessions In general, Diameter can provide two different types of services to applications. The first involves authentication and authorization, - and can optionally make use of accounting. The second only makes use - of accounting. + and it can optionally make use of accounting. The second only makes + use of accounting. When a service makes use of the authentication and/or authorization portion of an application, and a user requests access to the network, @@ -5614,27 +5449,28 @@ Internet-Draft Diameter Base Protocol January 2011 auth request is defined in a service-specific Diameter application (e.g., NASREQ). The request contains a Session-Id AVP, which is used in subsequent messages (e.g., subsequent authorization, accounting, - etc) relating to the user's session. The Session-Id AVP is a means + etc.) relating to the user's session. The Session-Id AVP is a means for the client and servers to correlate a Diameter message with a user session. - When a Diameter server authorizes a user to use network resources for - a finite amount of time, and it is willing to extend the - authorization via a future request, it MUST add the Authorization- - Lifetime AVP to the answer message. The Authorization-Lifetime AVP - defines the maximum number of seconds a user MAY make use of the - resources before another authorization request is expected by the - server. The Auth-Grace-Period AVP contains the number of seconds - following the expiration of the Authorization-Lifetime, after which - the server will release all state information related to the user's - session. Note that if payment for services is expected by the - serving realm from the user's home realm, the Authorization-Lifetime - AVP, combined with the Auth-Grace-Period AVP, implies the maximum - length of the session the home realm is willing to be fiscally - responsible for. Services provided past the expiration of the - Authorization-Lifetime and Auth-Grace-Period AVPs are the - responsibility of the access device. Of course, the actual cost of - services rendered is clearly outside the scope of the protocol. + When a Diameter server authorizes a user to implement network + resources for a finite amount of time, and it is willing to extend + the authorization via a future request, it MUST add the + Authorization- Lifetime AVP to the answer message. The + Authorization-Lifetime AVP defines the maximum number of seconds a + user MAY make use of the resources before another authorization + request is expected by the server. The Auth-Grace-Period AVP + contains the number of seconds following the expiration of the + Authorization-Lifetime, after which the server will release all state + information related to the user's session. Note that if payment for + services is expected by the serving realm from the user's home realm, + the Authorization-Lifetime AVP, combined with the Auth-Grace-Period + AVP, implies the maximum length of the session for which the home + realm is willing to be fiscally responsible. Services provided past + the expiration of the Authorization-Lifetime and Auth-Grace-Period + AVPs are the responsibility of the access device. Of course, the + actual cost of services rendered is clearly outside the scope of the + protocol. An access device that does not expect to send a re-authorization or a session termination request to the server MAY include the Auth- @@ -5648,32 +5484,33 @@ Internet-Draft Diameter Base Protocol January 2011 value NO_STATE_MAINTAINED MUST NOT be set in subsequent re- authorization requests and answers. - The base protocol does not include any authorization request -Fajardo, et al. Expires July 24, 2011 [Page 101] +Fajardo, et al. Standards Track [Page 98] -Internet-Draft Diameter Base Protocol January 2011 +RFC 6733 Diameter Base Protocol October 2012 + The base protocol does not include any authorization request messages, since these are largely application-specific and are defined in a Diameter application document. However, the base protocol does define a set of messages that are used to terminate user sessions. These are used to allow servers that maintain state information to free resources. - When a service only makes use of the Accounting portion of the + When a service only makes use of the accounting portion of the Diameter protocol, even in combination with an application, the Session-Id is still used to identify user sessions. However, the session termination messages are not used, since a session is signaled as being terminated by issuing an accounting stop message. Diameter may also be used for services that cannot be easily - categorized as authentication, authorization or accounting (e.g., - certain 3GPP IMS interfaces). In such cases, the finite state + categorized as authentication, authorization, or accounting (e.g., + certain Third Generation Partnership Project Internet Multimedia + System (3GPP IMS) interfaces). In such cases, the finite state machine defined in subsequent sections may not be applicable. - Therefore, the applications itself MAY need to define its own finite + Therefore, the application itself MAY need to define its own finite state machine. However, such application-specific state machines SHOULD follow the general state machine framework outlined in this document such as the use of Session-Id AVPs and the use of STR/STA, @@ -5681,12 +5518,12 @@ Internet-Draft Diameter Base Protocol January 2011 8.1. Authorization Session State Machine - This section contains a set of finite state machines, representing - the life cycle of Diameter sessions, and which MUST be observed by - all Diameter implementations that make use of the authentication - and/or authorization portion of a Diameter application. The term - Service-Specific below refers to a message defined in a Diameter - application (e.g., Mobile IPv4, NASREQ). + This section contains a set of finite state machines, which represent + the life cycle of Diameter sessions and which MUST be observed by all + Diameter implementations that make use of the authentication and/or + authorization portion of a Diameter application. The term "Service- + Specific" below refers to a message defined in a Diameter application + (e.g., Mobile IPv4, NASREQ). There are four different authorization session state machines supported in the Diameter base protocol. The first two describe a @@ -5700,26 +5537,25 @@ Internet-Draft Diameter Base Protocol January 2011 When a session is moved to the Idle state, any resources that were allocated for the particular session must be released. Any event not - listed in the state machines MUST be considered as an error - condition, and an answer, if applicable, MUST be returned to the - originator of the message. + listed in the state machines MUST be considered an error condition, + and an answer, if applicable, MUST be returned to the originator of + the message. - In the case that an application does not support re-auth, the state - -Fajardo, et al. Expires July 24, 2011 [Page 102] +Fajardo, et al. Standards Track [Page 99] -Internet-Draft Diameter Base Protocol January 2011 +RFC 6733 Diameter Base Protocol October 2012 - transitions related to server-initiated re-auth when both client and - server session maintains state (e.g., Send RAR, Pending, Receive RAA) - MAY be ignored. + In the case that an application does not support re-auth, the state + transitions related to server-initiated re-auth, when both client and + server sessions maintain state (e.g., Send RAR, Pending, Receive + RAA), MAY be ignored. - In the state table, the event 'Failure to send X' means that the + In the state table, the event "Failure to send X" means that the Diameter agent is unable to send command X to the desired - destination. This could be due to the peer being down, or due to the + destination. This could be due to the peer being down or due to the peer sending back a transient failure or temporary protocol error notification DIAMETER_TOO_BUSY or DIAMETER_LOOP_DETECTED in the Result-Code AVP of the corresponding Answer command. The event 'X @@ -5731,8 +5567,8 @@ Internet-Draft Diameter Base Protocol January 2011 CLIENT, STATEFUL State Event Action New State --------------------------------------------------------------- - Idle Client or Device Requests Send Pending - access service + Idle Client or device requests Send Pending + access service- specific auth req @@ -5748,39 +5584,38 @@ Internet-Draft Diameter Base Protocol January 2011 UNKNOWN_ SESSION_ID - Pending Successful Service-specific Grant Open + Pending Successful service-specific Grant Open authorization answer Access received with default Auth-Session-State value - Pending Successful Service-specific Sent STR Discon - authorization answer received + Pending Successful service-specific Sent STR Discon + authorization answer received, but service not provided Pending Error processing successful Sent STR Discon - Service-specific authorization + service-specific authorization answer - -Fajardo, et al. Expires July 24, 2011 [Page 103] +Fajardo, et al. Standards Track [Page 100] -Internet-Draft Diameter Base Protocol January 2011 +RFC 6733 Diameter Base Protocol October 2012 - Pending Failed Service-specific Cleanup Idle + Pending Failed service-specific Clean up Idle authorization answer received Open User or client device Send Open - requests access to service service + requests access to service service- specific auth req - Open Successful Service-specific Provide Open - authorization answer received Service + Open Successful service-specific Provide Open + authorization answer received service - Open Failed Service-specific Discon. Idle + Open Failed service-specific Discon. Idle authorization answer user/device received. @@ -5796,10 +5631,10 @@ Internet-Draft Diameter Base Protocol January 2011 Discon. user/device - Open Session-Timeout Expires on Send STR Discon - Access Device + Open Session-Timeout expires on Send STR Discon + access device - Open ASR Received, Send ASA Discon + Open ASR received, Send ASA Discon client will comply with with request to end the Result-Code = session = SUCCESS, @@ -5814,17 +5649,18 @@ Internet-Draft Diameter Base Protocol January 2011 Auth-Grace-Period expires on access device - Discon ASR Received Send ASA Discon + Discon ASR received Send ASA Discon + - Discon STA Received Discon. Idle -Fajardo, et al. Expires July 24, 2011 [Page 104] +Fajardo, et al. Standards Track [Page 101] -Internet-Draft Diameter Base Protocol January 2011 +RFC 6733 Diameter Base Protocol October 2012 + Discon STA received Discon. Idle user/device The following state machine is observed by a server when it is @@ -5835,31 +5671,34 @@ Internet-Draft Diameter Base Protocol January 2011 --------------------------------------------------------------- Idle Service-specific authorization Send Open request received, and successful - user is authorized serv. + user is authorized service- specific answer Idle Service-specific authorization Send Idle - request received, and failed serv. - user is not authorized specific + request received, and failed + user is not authorized service- + specific answer Open Service-specific authorization Send Open request received, and user successful - is authorized serv. specific + is authorized service- + specific answer Open Service-specific authorization Send Idle - request received, and user failed serv. - is not authorized specific + request received, and user failed + is not authorized service- + specific answer, - Cleanup + Clean up Open Home server wants to confirm Send RAR Pending authentication and/or authorization of the user - Pending Received RAA with a failed Cleanup Idle + Pending Received RAA with a failed Clean up Idle Result-Code Pending Received RAA with Result-Code Update Open @@ -5868,32 +5707,33 @@ Internet-Draft Diameter Base Protocol January 2011 Open Home server wants to Send ASR Discon terminate the service - Open Authorization-Lifetime (and Cleanup Idle - Auth-Grace-Period) expires - on home server. - Open Session-Timeout expires on Cleanup Idle -Fajardo, et al. Expires July 24, 2011 [Page 105] +Fajardo, et al. Standards Track [Page 102] -Internet-Draft Diameter Base Protocol January 2011 +RFC 6733 Diameter Base Protocol October 2012 + Open Authorization-Lifetime (and Clean up Idle + Auth-Grace-Period) expires + on home server + + Open Session-Timeout expires on Clean up Idle home server Discon Failure to send ASR Wait, Discon resend ASR - Discon ASR successfully sent and Cleanup Idle + Discon ASR successfully sent and Clean up Idle ASA Received with Result-Code - Not ASA Received None No Change. + Not ASA Received None No Change Discon Any STR Received Send STA, Idle - Cleanup. + Clean up The following state machine is observed by a client when state is not maintained on the server: @@ -5901,48 +5741,47 @@ Internet-Draft Diameter Base Protocol January 2011 CLIENT, STATELESS State Event Action New State --------------------------------------------------------------- - Idle Client or Device Requests Send Pending - access service + Idle Client or device requests Send Pending + access service- specific auth req - Pending Successful Service-specific Grant Open - authorization answer Access + Pending Successful service-specific Grant Open + authorization answer access received with Auth-Session- State set to NO_STATE_MAINTAINED - Pending Failed Service-specific Cleanup Idle + Pending Failed service-specific Clean up Idle authorization answer received - Open Session-Timeout Expires on Discon. Idle - Access Device user/device + Open Session-Timeout expires on Discon. Idle + access device user/device Open Service to user is terminated Discon. Idle user/device - The following state machine is observed by a server when it is not - maintaining state for the session: - - - -Fajardo, et al. Expires July 24, 2011 [Page 106] +Fajardo, et al. Standards Track [Page 103] -Internet-Draft Diameter Base Protocol January 2011 +RFC 6733 Diameter Base Protocol October 2012 + + The following state machine is observed by a server when it is not + maintaining state for the session: SERVER, STATELESS State Event Action New State --------------------------------------------------------------- - Idle Service-specific authorization Send serv. Idle - request received, and specific - successfully processed answer + Idle Service-specific authorization Send Idle + request received, and service- + successfully processed specific + answer 8.2. Accounting Session State Machine @@ -5960,8 +5799,8 @@ Internet-Draft Diameter Base Protocol January 2011 machine in this section described below. The default server side state machine requires the reception of - accounting records in any order and at any time, and does not place - any standards requirement on the processing of these records. + accounting records in any order and at any time, and it does not + place any standards requirement on the processing of these records. Implementations of Diameter may perform checking, ordering, correlation, fraud detection, and other tasks based on these records. AVPs may need to be inspected as a part of these tasks. The tasks @@ -5970,7 +5809,7 @@ Internet-Draft Diameter Base Protocol January 2011 or even policy dependent, they are not standardized by the Diameter specifications. Applications MAY define requirements on when to accept accounting records based on the used value of Accounting- - Realtime-Required AVP, credit limits checks, and so on. + Realtime-Required AVP, credit-limit checks, and so on. However, the Diameter base protocol defines one optional server side state machine that MAY be followed by applications that require @@ -5978,31 +5817,31 @@ Internet-Draft Diameter Base Protocol January 2011 that such tracking is incompatible with the ability to sustain long duration connectivity problems. Therefore, the use of this state machine is recommended only in applications where the value of the - Accounting-Realtime-Required AVP is DELIVER_AND_GRANT, and hence + Accounting-Realtime-Required AVP is DELIVER_AND_GRANT; hence, accounting connectivity problems are required to cause the serviced user to be disconnected. Otherwise, records produced by the client - may be lost by the server which no longer accepts them after the - connectivity is re-established. This state machine is the third - state machine in this section. The state machine is supervised by a - supervision session timer Ts, which the value should be reasonably -Fajardo, et al. Expires July 24, 2011 [Page 107] +Fajardo, et al. Standards Track [Page 104] -Internet-Draft Diameter Base Protocol January 2011 +RFC 6733 Diameter Base Protocol October 2012 - higher than the Acct_Interim_Interval value. Ts MAY be set to two - times the value of the Acct_Interim_Interval so as to avoid the - accounting session in the Diameter server to change to Idle state in - case of short transient network failure. + may be lost by the server, which no longer accepts them after the + connectivity is re-established. This state machine is the third + state machine in this section. The state machine is supervised by a + supervision session timer Ts, whose value should be reasonably higher + than the Acct_Interim_Interval value. Ts MAY be set to two times the + value of the Acct_Interim_Interval so as to avoid the accounting + session in the Diameter server to change to Idle state in case of + short transient network failure. Any event not listed in the state machines MUST be considered as an error condition, and a corresponding answer, if applicable, MUST be returned to the originator of the message. - In the state table, the event 'Failure to send' means that the + In the state table, the event "Failure to send" means that the Diameter client is unable to communicate with the desired destination. This could be due to the peer being down, or due to the peer sending back a transient failure or temporary protocol error @@ -6010,17 +5849,17 @@ Internet-Draft Diameter Base Protocol January 2011 DIAMETER_LOOP_DETECTED in the Result-Code AVP of the Accounting Answer command. - The event 'Failed answer' means that the Diameter client received a + The event "Failed answer" means that the Diameter client received a non-transient failure notification in the Accounting Answer command. - Note that the action 'Disconnect user/dev' MUST have an effect also - to the authorization session state table, e.g., cause the STR message + Note that the action "Disconnect user/dev" MUST also have an effect + on the authorization session state table, e.g., cause the STR message to be sent, if the given application has both authentication/ authorization and accounting portions. - The states PendingS, PendingI, PendingL, PendingE and PendingB stand + The states PendingS, PendingI, PendingL, PendingE, and PendingB stand for pending states to wait for an answer to an accounting request - related to a Start, Interim, Stop, Event or buffered record, + related to a Start, Interim, Stop, Event, or buffered record, respectively. CLIENT, ACCOUNTING @@ -6037,36 +5876,36 @@ Internet-Draft Diameter Base Protocol January 2011 Idle Records in storage Send PendingB record - PendingS Successful accounting Open - start answer received - - PendingS Failure to send and buffer Store Open -Fajardo, et al. Expires July 24, 2011 [Page 108] +Fajardo, et al. Standards Track [Page 105] -Internet-Draft Diameter Base Protocol January 2011 +RFC 6733 Diameter Base Protocol October 2012 - space available and realtime Start + PendingS Successful accounting Open + start answer received + + PendingS Failure to send and buffer Store Open + space available and real time Start not equal to DELIVER_AND_GRANT Record PendingS Failure to send and no buffer Open - space available and realtime + space available and real time equal to GRANT_AND_LOSE PendingS Failure to send and no Disconnect Idle buffer space available and user/dev - realtime not equal to + real time not equal to GRANT_AND_LOSE PendingS Failed accounting start answer Open - received and realtime equal + received and real time equal to GRANT_AND_LOSE PendingS Failed accounting start answer Disconnect Idle - received and realtime not user/dev + received and real time not user/dev equal to GRANT_AND_LOSE PendingS User service terminated Store PendingS @@ -6077,6 +5916,7 @@ Internet-Draft Diameter Base Protocol January 2011 accounting interim record + Open User service terminated Send PendingL accounting stop req. @@ -6087,34 +5927,35 @@ Internet-Draft Diameter Base Protocol January 2011 PendingI Failure to send and (buffer Store Open space available or old interim record can be overwritten) record - and realtime not equal to + and real time not equal to DELIVER_AND_GRANT - PendingI Failure to send and no buffer Open - space available and realtime - equal to GRANT_AND_LOSE - PendingI Failure to send and no Disconnect Idle - buffer space available and user/dev -Fajardo, et al. Expires July 24, 2011 [Page 109] +Fajardo, et al. Standards Track [Page 106] -Internet-Draft Diameter Base Protocol January 2011 +RFC 6733 Diameter Base Protocol October 2012 + + PendingI Failure to send and no buffer Open + space available and real time + equal to GRANT_AND_LOSE - realtime not equal to + PendingI Failure to send and no Disconnect Idle + buffer space available and user/dev + real time not equal to GRANT_AND_LOSE PendingI Failed accounting interim Open - answer received and realtime + answer received and real time equal to GRANT_AND_LOSE PendingI Failed accounting interim Disconnect Idle answer received and user/dev - realtime not equal to + real time not equal to GRANT_AND_LOSE PendingI User service terminated Store PendingI @@ -6148,6 +5989,13 @@ Internet-Draft Diameter Base Protocol January 2011 space available stop record + + +Fajardo, et al. Standards Track [Page 107] + +RFC 6733 Diameter Base Protocol October 2012 + + PendingL Failure to send and no buffer Idle space available @@ -6155,124 +6003,133 @@ Internet-Draft Diameter Base Protocol January 2011 received - -Fajardo, et al. Expires July 24, 2011 [Page 110] - -Internet-Draft Diameter Base Protocol January 2011 - - SERVER, STATELESS ACCOUNTING State Event Action New State --------------------------------------------------------------- Idle Accounting start request Send Idle - received, and successfully accounting + received and successfully accounting processed. start answer Idle Accounting event request Send Idle - received, and successfully accounting + received and successfully accounting processed. event answer - Idle Interim record received, Send Idle + Idle Interim record received Send Idle and successfully processed. accounting interim answer Idle Accounting stop request Send Idle - received, and successfully accounting + received and successfully accounting processed stop answer - Idle Accounting request received, Send Idle + Idle Accounting request received; Send Idle no space left to store accounting - records answer, + records answer; Result-Code = OUT_OF_ SPACE + + + + + + + + + + + + + + +Fajardo, et al. Standards Track [Page 108] + +RFC 6733 Diameter Base Protocol October 2012 + + SERVER, STATEFUL ACCOUNTING State Event Action New State --------------------------------------------------------------- Idle Accounting start request Send Open - received, and successfully accounting + received and successfully accounting processed. start - answer, + answer; Start Ts Idle Accounting event request Send Idle - received, and successfully accounting + received and successfully accounting processed. event answer - - Idle Accounting request received, Send Idle + Idle Accounting request received; Send Idle no space left to store accounting - records answer, - - - -Fajardo, et al. Expires July 24, 2011 [Page 111] - -Internet-Draft Diameter Base Protocol January 2011 - - + records answer; Result-Code = OUT_OF_ SPACE - Open Interim record received, Send Open + Open Interim record received Send Open and successfully processed. accounting interim - answer, + answer; Restart Ts Open Accounting stop request Send Idle - received, and successfully accounting - processed stop answer, + received and successfully accounting + processed stop answer; Stop Ts - Open Accounting request received, Send Idle + Open Accounting request received; Send Idle no space left to store accounting - records answer, + records answer; Result-Code = OUT_OF_ - SPACE, + SPACE; Stop Ts Open Session supervision timer Ts Stop Ts Idle expired + + + + + + + + +Fajardo, et al. Standards Track [Page 109] + +RFC 6733 Diameter Base Protocol October 2012 + + 8.3. Server-Initiated Re-Auth A Diameter server may initiate a re-authentication and/or re- authorization service for a particular session by issuing a Re-Auth- Request (RAR). - For example, for pre-paid services, the Diameter server that + For example, for prepaid services, the Diameter server that originally authorized a session may need some confirmation that the user is still using the services. - An access device that receives a RAR message with Session-Id equal to - a currently active session MUST initiate a re-auth towards the user, - if the service supports this particular feature. Each Diameter - application MUST state whether server-initiated re-auth is supported, - since some applications do not allow access devices to prompt the - user for re-auth. + An access device that receives an RAR message with the Session-Id + equal to a currently active session MUST initiate a re-auth towards + the user, if the service supports this particular feature. Each + Diameter application MUST state whether server-initiated re-auth is + supported, since some applications do not allow access devices to + prompt the user for re-auth. 8.3.1. Re-Auth-Request - The Re-Auth-Request (RAR), indicated by the Command-Code set to 258 + The Re-Auth-Request (RAR), indicated by the Command Code set to 258 and the message flags' 'R' bit set, may be sent by any server to the access device that is providing session service, to request that the - - - -Fajardo, et al. Expires July 24, 2011 [Page 112] - -Internet-Draft Diameter Base Protocol January 2011 - - user be re-authenticated and/or re-authorized. @@ -6294,15 +6151,22 @@ Internet-Draft Diameter Base Protocol January 2011 8.3.2. Re-Auth-Answer - The Re-Auth-Answer (RAA), indicated by the Command-Code set to 258 + The Re-Auth-Answer (RAA), indicated by the Command Code set to 258 and the message flags' 'R' bit clear, is sent in response to the RAR. - The Result-Code AVP MUST be present, and indicates the disposition of - the request. + The Result-Code AVP MUST be present, and it indicates the disposition + of the request. + + + + +Fajardo, et al. Standards Track [Page 110] + +RFC 6733 Diameter Base Protocol October 2012 + A successful RAA message MUST be followed by an application-specific authentication and/or authorization message. - Message Format <RAA> ::= < Diameter Header: 258, PXY > @@ -6321,14 +6185,6 @@ Internet-Draft Diameter Base Protocol January 2011 * [ Proxy-Info ] * [ AVP ] - - - -Fajardo, et al. Expires July 24, 2011 [Page 113] - -Internet-Draft Diameter Base Protocol January 2011 - - 8.4. Session Termination It is necessary for a Diameter server that authorized a session, for @@ -6356,6 +6212,14 @@ Internet-Draft Diameter Base Protocol January 2011 It is also possible that a session that was authorized is never actually started due to action of a proxy. For example, a proxy may + + + +Fajardo, et al. Standards Track [Page 111] + +RFC 6733 Diameter Base Protocol October 2012 + + modify an authorization answer, converting the result from success to failure, prior to forwarding the message to the access device. If the answer did not contain an Auth-Session-State AVP with the value @@ -6366,36 +6230,27 @@ Internet-Draft Diameter Base Protocol January 2011 A Diameter server that receives an STR message MUST clean up resources (e.g., session state) associated with the Session-Id - specified in the STR, and return a Session-Termination-Answer. + specified in the STR and return a Session-Termination-Answer. A Diameter server also MUST clean up resources when the Session- Timeout expires, or when the Authorization-Lifetime and the Auth- - Grace-Period AVPs expires without receipt of a re-authorization + Grace-Period AVPs expire without receipt of a re-authorization request, regardless of whether an STR for that session is received. The access device is not expected to provide service beyond the expiration of these timers; thus, expiration of either of these timers implies that the access device may have unexpectedly shut down. - - - -Fajardo, et al. Expires July 24, 2011 [Page 114] - -Internet-Draft Diameter Base Protocol January 2011 - - 8.4.1. Session-Termination-Request - The Session-Termination-Request (STR), indicated by the Command-Code + The Session-Termination-Request (STR), indicated by the Command Code set to 275 and the Command Flags' 'R' bit set, is sent by a Diameter - client or by a Diameter proxy to inform the Diameter Server that an + client or by a Diameter proxy to inform the Diameter server that an authenticated and/or authorized session is being terminated. + Message Format - Message Format - - <STR> ::= < Diameter Header: 275, REQ, PXY > + <STR> ::= < Diameter Header: 275, REQ, PXY > < Session-Id > { Origin-Host } { Origin-Realm } @@ -6410,40 +6265,33 @@ Internet-Draft Diameter Base Protocol January 2011 * [ Route-Record ] * [ AVP ] -8.4.2. Session-Termination-Answer - - The Session-Termination-Answer (STA), indicated by the Command-Code - set to 275 and the message flags' 'R' bit clear, is sent by the - Diameter Server to acknowledge the notification that the session has - been terminated. The Result-Code AVP MUST be present, and MAY - contain an indication that an error occurred while servicing the STR. - - Upon sending or receipt of the STA, the Diameter Server MUST release - all resources for the session indicated by the Session-Id AVP. Any - intermediate server in the Proxy-Chain MAY also release any - resources, if necessary. - - - - - +Fajardo, et al. Standards Track [Page 112] + +RFC 6733 Diameter Base Protocol October 2012 +8.4.2. Session-Termination-Answer -Fajardo, et al. Expires July 24, 2011 [Page 115] - -Internet-Draft Diameter Base Protocol January 2011 + The Session-Termination-Answer (STA), indicated by the Command Code + set to 275 and the message flags' 'R' bit clear, is sent by the + Diameter server to acknowledge the notification that the session has + been terminated. The Result-Code AVP MUST be present, and it MAY + contain an indication that an error occurred while servicing the STR. + Upon sending or receipt of the STA, the Diameter server MUST release + all resources for the session indicated by the Session-Id AVP. Any + intermediate server in the Proxy-Chain MAY also release any + resources, if necessary. - Message Format + Message Format - <STA> ::= < Diameter Header: 275, PXY > + <STA> ::= < Diameter Header: 275, PXY > < Session-Id > { Result-Code } { Origin-Host } @@ -6473,31 +6321,29 @@ Internet-Draft Diameter Base Protocol January 2011 An access device that receives an ASR with Session-ID equal to a currently active session MAY stop the session. Whether the access - device stops the session or not is implementation- and/or - configuration-dependent. For example, an access device may honor + device stops the session or not is implementation and/or + configuration dependent. For example, an access device may honor ASRs from certain agents only. In any case, the access device MUST + + + +Fajardo, et al. Standards Track [Page 113] + +RFC 6733 Diameter Base Protocol October 2012 + + respond with an Abort-Session-Answer, including a Result-Code AVP to indicate what action it took. 8.5.1. Abort-Session-Request - The Abort-Session-Request (ASR), indicated by the Command-Code set to + The Abort-Session-Request (ASR), indicated by the Command Code set to 274 and the message flags' 'R' bit set, may be sent by any Diameter server or any Diameter proxy to the access device that is providing session service, to request that the session identified by the Session-Id be stopped. - - - - - -Fajardo, et al. Expires July 24, 2011 [Page 116] - -Internet-Draft Diameter Base Protocol January 2011 - - - Message Format + Message Format <ASR> ::= < Diameter Header: 274, REQ, PXY > < Session-Id > @@ -6514,20 +6360,35 @@ Internet-Draft Diameter Base Protocol January 2011 8.5.2. Abort-Session-Answer - The Abort-Session-Answer (ASA), indicated by the Command-Code set to + The Abort-Session-Answer (ASA), indicated by the Command Code set to 274 and the message flags' 'R' bit clear, is sent in response to the - ASR. The Result-Code AVP MUST be present, and indicates the + ASR. The Result-Code AVP MUST be present and indicates the disposition of the request. If the session identified by Session-Id in the ASR was successfully - terminated, Result-Code is set to DIAMETER_SUCCESS. If the session - is not currently active, Result-Code is set to + terminated, the Result-Code is set to DIAMETER_SUCCESS. If the + session is not currently active, the Result-Code is set to DIAMETER_UNKNOWN_SESSION_ID. If the access device does not stop the - session for any other reason, Result-Code is set to + session for any other reason, the Result-Code is set to DIAMETER_UNABLE_TO_COMPLY. - Message Format + + + + + + + + + + +Fajardo, et al. Standards Track [Page 114] + +RFC 6733 Diameter Base Protocol October 2012 + + + Message Format <ASA> ::= < Diameter Header: 274, PXY > < Session-Id > @@ -6545,14 +6406,6 @@ Internet-Draft Diameter Base Protocol January 2011 * [ Proxy-Info ] * [ AVP ] - - - -Fajardo, et al. Expires July 24, 2011 [Page 117] - -Internet-Draft Diameter Base Protocol January 2011 - - 8.6. Inferring Session Termination from Origin-State-Id The Origin-State-Id is used to allow detection of terminated sessions @@ -6560,94 +6413,96 @@ Internet-Draft Diameter Base Protocol January 2011 shutdown of an access device. A Diameter client or access device increments the value of the - Origin-State-Id every time it is started or powered-up. The new + Origin-State-Id every time it is started or powered up. The new Origin-State-Id is then sent in the CER/CEA message immediately upon connection to the server. The Diameter server receiving the new Origin-State-Id can determine whether the sending Diameter client had - abruptly shutdown by comparing the old value of the Origin-State-Id + abruptly shut down by comparing the old value of the Origin-State-Id it has kept for that specific client is less than the new value and whether it has un-terminated sessions originating from that client. An access device can also include the Origin-State-Id in request - messages other than CER if there are relays or proxies in between the - access device and the server. In this case, however, the server + messages other than the CER if there are relays or proxies in between + the access device and the server. In this case, however, the server cannot discover that the access device has been restarted unless and - until it receives a new request from it. Therefore this mechanism is - more opportunistic across proxies and relays. + until it receives a new request from it. Therefore, this mechanism + is more opportunistic across proxies and relays. The Diameter server may assume that all sessions that were active prior to detection of a client restart have been terminated. The Diameter server MAY clean up all session state associated with such - lost sessions, and MAY also issues STRs for all such lost sessions + lost sessions, and it MAY also issue STRs for all such lost sessions that were authorized on upstream servers, to allow session state to be cleaned up globally. + + + + +Fajardo, et al. Standards Track [Page 115] + +RFC 6733 Diameter Base Protocol October 2012 + + 8.7. Auth-Request-Type AVP The Auth-Request-Type AVP (AVP Code 274) is of type Enumerated and is included in application-specific auth requests to inform the peers - whether a user is to be authenticated only, authorized only or both. + whether a user is to be authenticated only, authorized only, or both. Note any value other than both MAY cause RADIUS interoperability issues. The following values are defined: - AUTHENTICATE_ONLY 1 - The request being sent is for authentication only, and MUST - contain the relevant application specific authentication AVPs that + The request being sent is for authentication only, and it MUST + contain the relevant application-specific authentication AVPs that are needed by the Diameter server to authenticate the user. - - - - - - - -Fajardo, et al. Expires July 24, 2011 [Page 118] - -Internet-Draft Diameter Base Protocol January 2011 - - AUTHORIZE_ONLY 2 - The request being sent is for authorization only, and MUST contain - the application-specific authorization AVPs that are necessary to - identify the service being requested/offered. - + The request being sent is for authorization only, and it MUST + contain the application-specific authorization AVPs that are + necessary to identify the service being requested/offered. AUTHORIZE_AUTHENTICATE 3 The request contains a request for both authentication and authorization. The request MUST include both the relevant - application-specific authentication information, and authorization + application-specific authentication information and authorization information necessary to identify the service being requested/ offered. - 8.8. Session-Id AVP The Session-Id AVP (AVP Code 263) is of type UTF8String and is used to identify a specific session (see Section 8). All messages - pertaining to a specific session MUST include only one Session-Id AVP - and the same value MUST be used throughout the life of a session. - When present, the Session-Id SHOULD appear immediately following the - Diameter Header (see Section 3). + pertaining to a specific session MUST include only one Session-Id + AVP, and the same value MUST be used throughout the life of a + session. When present, the Session-Id SHOULD appear immediately + following the Diameter header (see Section 3). The Session-Id MUST be globally and eternally unique, as it is meant to uniquely identify a user session without reference to any other - information, and may be needed to correlate historical authentication - information with accounting information. The Session-Id includes a - mandatory portion and an implementation-defined portion; a - recommended format for the implementation-defined portion is outlined - below. + information, and it may be needed to correlate historical + authentication information with accounting information. The + Session-Id includes a mandatory portion and an implementation-defined + portion; a recommended format for the implementation-defined portion + is outlined below. The Session-Id MUST begin with the sender's identity encoded in the - DiameterIdentity type (see Section 4.4). The remainder of the - Session-Id is delimited by a ";" character, and MAY be any sequence - that the client can guarantee to be eternally unique; however, the - following format is recommended, (square brackets [] indicate an - optional element): + DiameterIdentity type (see Section 4.3.1). The remainder of the + Session-Id is delimited by a ";" character, and it MAY be any + + + +Fajardo, et al. Standards Track [Page 116] + +RFC 6733 Diameter Base Protocol October 2012 + + + sequence that the client can guarantee to be eternally unique; + however, the following format is recommended, (square brackets [] + indicate an optional element): <DiameterIdentity>;<high 32 bits>;<low 32 bits>[;<optional value>] @@ -6657,22 +6512,14 @@ Internet-Draft Diameter Base Protocol January 2011 processors. At startup, the high 32 bits of the 64-bit value MAY be initialized to the time in NTP format [RFC5905], and the low 32 bits MAY be initialized to zero. This will for practical purposes - - - -Fajardo, et al. Expires July 24, 2011 [Page 119] - -Internet-Draft Diameter Base Protocol January 2011 - - eliminate the possibility of overlapping Session-Ids after a reboot, assuming the reboot process takes longer than a second. Alternatively, an implementation MAY keep track of the increasing value in non-volatile memory. - <optional value> is implementation specific but may include a modem's - device Id, a layer 2 address, timestamp, etc. + <optional value> is implementation specific, but it may include a + modem's device Id, a Layer 2 address, timestamp, etc. Example, in which there is no optional value: @@ -6683,45 +6530,44 @@ Internet-Draft Diameter Base Protocol January 2011 accesspoint7.example.com;1876543210;523;[email protected] The Session-Id is created by the Diameter application initiating the - session, which in most cases is done by the client. Note that a - Session-Id MAY be used for both the authentication, authorization and - accounting commands of a given application. + session, which, in most cases, is done by the client. Note that a + Session-Id MAY be used for both the authentication, authorization, + and accounting commands of a given application. 8.9. Authorization-Lifetime AVP The Authorization-Lifetime AVP (AVP Code 291) is of type Unsigned32 and contains the maximum number of seconds of service to be provided to the user before the user is to be re-authenticated and/or re- - authorized. Care should be taken when the Authorization- Lifetime - value is determined, since a low, non-zero, value could create + authorized. Care should be taken when the Authorization-Lifetime + value is determined, since a low, non-zero value could create significant Diameter traffic, which could congest both the network and the agents. A value of zero (0) means that immediate re-auth is necessary by the access device. The absence of this AVP, or a value of all ones - (meaning all bits in the 32 bit field are set to one) means no re- + (meaning all bits in the 32-bit field are set to one) means no re- auth is expected. + + +Fajardo, et al. Standards Track [Page 117] + +RFC 6733 Diameter Base Protocol October 2012 + + If both this AVP and the Session-Timeout AVP are present in a message, the value of the latter MUST NOT be smaller than the Authorization-Lifetime AVP. An Authorization-Lifetime AVP MAY be present in re-authorization - messages, and contains the number of seconds the user is authorized - to receive service from the time the re-auth answer message is - received by the access device. + messages, and it contains the number of seconds the user is + authorized to receive service from the time the re-auth answer + message is received by the access device. This AVP MAY be provided by the client as a hint of the maximum lifetime that it is willing to accept. The server MUST return a - - - -Fajardo, et al. Expires July 24, 2011 [Page 120] - -Internet-Draft Diameter Base Protocol January 2011 - - - value that is equal to, or smaller, than the one provided by the + value that is equal to, or smaller than, the one provided by the client. 8.10. Auth-Grace-Period AVP @@ -6739,7 +6585,6 @@ Internet-Draft Diameter Base Protocol January 2011 the value in the server's answer message is binding. The following values are supported: - STATE_MAINTAINED 0 This value is used to specify that session state is being @@ -6747,46 +6592,42 @@ Internet-Draft Diameter Base Protocol January 2011 message when service to the user is terminated. This is the default value. - NO_STATE_MAINTAINED 1 This value is used to specify that no session termination messages will be sent by the access device upon expiration of the Authorization-Lifetime. - 8.12. Re-Auth-Request-Type AVP The Re-Auth-Request-Type AVP (AVP Code 285) is of type Enumerated and is included in application-specific auth answers to inform the client of the action expected upon expiration of the Authorization-Lifetime. - If the answer message contains an Authorization-Lifetime AVP with a - positive value, the Re-Auth-Request-Type AVP MUST be present in an - answer message. The following values are defined: - AUTHORIZE_ONLY 0 - An authorization only re-auth is expected upon expiration of the - Authorization-Lifetime. This is the default value if the AVP is +Fajardo, et al. Standards Track [Page 118] + +RFC 6733 Diameter Base Protocol October 2012 -Fajardo, et al. Expires July 24, 2011 [Page 121] - -Internet-Draft Diameter Base Protocol January 2011 + If the answer message contains an Authorization-Lifetime AVP with a + positive value, the Re-Auth-Request-Type AVP MUST be present in an + answer message. The following values are defined: + AUTHORIZE_ONLY 0 + An authorization only re-auth is expected upon expiration of the + Authorization-Lifetime. This is the default value if the AVP is not present in answer messages that include the Authorization- Lifetime. - AUTHORIZE_AUTHENTICATE 1 An authentication and authorization re-auth is expected upon expiration of the Authorization-Lifetime. - 8.13. Session-Timeout AVP The Session-Timeout AVP (AVP Code 27) [RFC2865] is of type Unsigned32 @@ -6799,10 +6640,10 @@ Internet-Draft Diameter Base Protocol January 2011 A session that terminates on an access device due to the expiration of the Session-Timeout MUST cause an STR to be issued, unless both the access device and the home server had previously agreed that no - session termination messages would be sent (see Section 8.11). + session termination messages would be sent (see Section 8). A Session-Timeout AVP MAY be present in a re-authorization answer - message, and contains the remaining number of seconds from the + message, and it contains the remaining number of seconds from the beginning of the re-auth. A value of zero, or the absence of this AVP, means that this session @@ -6810,7 +6651,7 @@ Internet-Draft Diameter Base Protocol January 2011 This AVP MAY be provided by the client as a hint of the maximum timeout that it is willing to accept. However, the server MAY return - a value that is equal to, or smaller, than the one provided by the + a value that is equal to, or smaller than, the one provided by the client. 8.14. User-Name AVP @@ -6819,83 +6660,29 @@ Internet-Draft Diameter Base Protocol January 2011 contains the User-Name, in a format consistent with the NAI specification [RFC4282]. -8.15. Termination-Cause AVP - - The Termination-Cause AVP (AVP Code 295) is of type Enumerated, and - is used to indicate the reason why a session was terminated on the - access device. The following values are defined: - -Fajardo, et al. Expires July 24, 2011 [Page 122] +Fajardo, et al. Standards Track [Page 119] -Internet-Draft Diameter Base Protocol January 2011 - - - DIAMETER_LOGOUT 1 - - The user initiated a disconnect - - - DIAMETER_SERVICE_NOT_PROVIDED 2 - - This value is used when the user disconnected prior to the receipt - of the authorization answer message. - - - DIAMETER_BAD_ANSWER 3 - - This value indicates that the authorization answer received by the - access device was not processed successfully. - - - DIAMETER_ADMINISTRATIVE 4 - - The user was not granted access, or was disconnected, due to - administrative reasons, such as the receipt of a Abort-Session- - Request message. - - - DIAMETER_LINK_BROKEN 5 - - The communication to the user was abruptly disconnected. - - - DIAMETER_AUTH_EXPIRED 6 - - The user's access was terminated since its authorized session time - has expired. - - - DIAMETER_USER_MOVED 7 - - The user is receiving services from another access device. - - - DIAMETER_SESSION_TIMEOUT 8 +RFC 6733 Diameter Base Protocol October 2012 - The user's session has timed out, and service has been terminated. +8.15. Termination-Cause AVP - - - - - - -Fajardo, et al. Expires July 24, 2011 [Page 123] - -Internet-Draft Diameter Base Protocol January 2011 - + The Termination-Cause AVP (AVP Code 295) is of type Enumerated, and + is used to indicate the reason why a session was terminated on the + access device. The currently assigned values for this AVP can be + found in the IANA registry for Termination-Cause AVP Values + [IANATCV]. 8.16. Origin-State-Id AVP The Origin-State-Id AVP (AVP Code 278), of type Unsigned32, is a monotonically increasing value that is advanced whenever a Diameter - entity restarts with loss of previous state, for example upon reboot. - Origin-State-Id MAY be included in any Diameter message, including - CER. + entity restarts with loss of previous state, for example, upon + reboot. Origin-State-Id MAY be included in any Diameter message, + including CER. A Diameter entity issuing this AVP MUST create a higher value for this AVP each time its state is reset. A Diameter entity MAY set @@ -6911,20 +6698,34 @@ Internet-Draft Diameter Base Protocol January 2011 message, it allows other Diameter entities to infer that sessions associated with a lower Origin-State-Id are no longer active. If an access device does not intend for such inferences to be made, it MUST - either not include Origin-State-Id in any message, or set its value - to 0. + either not include Origin-State-Id in any message or set its value to + 0. 8.17. Session-Binding AVP - The Session-Binding AVP (AVP Code 270) is of type Unsigned32, and MAY - be present in application-specific authorization answer messages. If - present, this AVP MAY inform the Diameter client that all future + The Session-Binding AVP (AVP Code 270) is of type Unsigned32, and it + MAY be present in application-specific authorization answer messages. + If present, this AVP MAY inform the Diameter client that all future application-specific re-auth and Session-Termination-Request messages for this session MUST be sent to the same authorization server. - This field is a bit mask, and the following bits have been defined: + + + + + + + + +Fajardo, et al. Standards Track [Page 120] + +RFC 6733 Diameter Base Protocol October 2012 + + + This field is a bit mask, and the following bits have been defined: + RE_AUTH 1 When set, future re-auth messages for this session MUST NOT @@ -6932,23 +6733,13 @@ Internet-Draft Diameter Base Protocol January 2011 value, the Destination-Host AVP MUST be present in all re-auth messages for this session. - STR 2 When set, the STR message for this session MUST NOT include the Destination-Host AVP. When cleared, the default value, the - - - -Fajardo, et al. Expires July 24, 2011 [Page 124] - -Internet-Draft Diameter Base Protocol January 2011 - - Destination-Host AVP MUST be present in the STR message for this session. - ACCOUNTING 4 When set, all accounting messages for this session MUST NOT @@ -6956,10 +6747,9 @@ Internet-Draft Diameter Base Protocol January 2011 value, the Destination-Host AVP, if known, MUST be present in all accounting messages for this session. - 8.18. Session-Server-Failover AVP - The Session-Server-Failover AVP (AVP Code 271) is of type Enumerated, + The Session-Server-Failover AVP (AVP Code 271) is of type Enumerated and MAY be present in application-specific authorization answer messages that either do not include the Session-Binding AVP or include the Session-Binding AVP with any of the bits set to a zero @@ -6970,11 +6760,24 @@ Internet-Draft Diameter Base Protocol January 2011 The following values are supported: - REFUSE_SERVICE 0 If either the re-auth or the STR message delivery fails, terminate - service with the user, and do not attempt any subsequent attempts. + service with the user and do not attempt any subsequent attempts. + + + + + + + + + + + +Fajardo, et al. Standards Track [Page 121] + +RFC 6733 Diameter Base Protocol October 2012 TRY_AGAIN 1 @@ -6982,34 +6785,23 @@ Internet-Draft Diameter Base Protocol January 2011 If either the re-auth or the STR message delivery fails, resend the failed message without the Destination-Host AVP present. - ALLOW_SERVICE 2 If re-auth message delivery fails, assume that re-authorization succeeded. If STR message delivery fails, terminate the session. - TRY_AGAIN_ALLOW_SERVICE 3 If either the re-auth or the STR message delivery fails, resend the failed message without the Destination-Host AVP present. If - - - -Fajardo, et al. Expires July 24, 2011 [Page 125] - -Internet-Draft Diameter Base Protocol January 2011 - - the second delivery fails for re-auth, assume re-authorization succeeded. If the second delivery fails for STR, terminate the session. - 8.19. Multi-Round-Time-Out AVP - The Multi-Round-Time-Out AVP (AVP Code 272) is of type Unsigned32, - and SHOULD be present in application-specific authorization answer + The Multi-Round-Time-Out AVP (AVP Code 272) is of type Unsigned32 and + SHOULD be present in application-specific authorization answer messages whose Result-Code AVP is set to DIAMETER_MULTI_ROUND_AUTH. This AVP contains the maximum number of seconds that the access device MUST provide the user in responding to an authentication @@ -7031,37 +6823,24 @@ Internet-Draft Diameter Base Protocol January 2011 8.21. Event-Timestamp AVP - The Event-Timestamp (AVP Code 55) is of type Time, and MAY be - included in an Accounting-Request and Accounting-Answer messages to - record the time that the reported event occurred, in seconds since - January 1, 1900 00:00 UTC. - - + The Event-Timestamp (AVP Code 55) is of type Time and MAY be included + in an Accounting-Request and Accounting-Answer messages to record the + time that the reported event occurred, in seconds since January 1, + 1900 00:00 UTC. - - - - - - - - - - - -Fajardo, et al. Expires July 24, 2011 [Page 126] +Fajardo, et al. Standards Track [Page 122] -Internet-Draft Diameter Base Protocol January 2011 +RFC 6733 Diameter Base Protocol October 2012 9. Accounting This accounting protocol is based on a server directed model with capabilities for real-time delivery of accounting information. - Several fault resilience methods [RFC2975] have been built in to the + Several fault resilience methods [RFC2975] have been built into the protocol in order minimize loss of accounting data in various fault situations and under different assumptions about the capabilities of the used devices. @@ -7075,15 +6854,16 @@ Internet-Draft Diameter Base Protocol January 2011 timeliness requirements. As discussed in [RFC2975], real-time transfer of accounting records - is a requirement, such as the need to perform credit limit checks and + is a requirement, such as the need to perform credit-limit checks and fraud detection. Note that batch accounting is not a requirement, and is therefore not supported by Diameter. Should batched accounting be required in the future, a new Diameter application will need to be created, or it could be handled using another protocol. - Note, however, that even if at the Diameter layer accounting requests - are processed one by one, transport protocols used under Diameter - typically batch several requests in the same packet under heavy - traffic conditions. This may be sufficient for many applications. + Note, however, that even if at the Diameter layer, accounting + requests are processed one by one; transport protocols used under + Diameter typically batch several requests in the same packet under + heavy traffic conditions. This may be sufficient for many + applications. The authorization server (chain) directs the selection of proper transfer strategy, based on its knowledge of the user and @@ -7097,7 +6877,7 @@ Internet-Draft Diameter Base Protocol January 2011 records from the Diameter client is delayed or unsuccessful. The Diameter accounting server MAY override the interim interval or - the realtime requirements by including the Acct-Interim-Interval or + the real-time requirements by including the Acct-Interim-Interval or Accounting-Realtime-Required AVP in the Accounting-Answer message. When one of these AVPs is present, the latest value received SHOULD be used in further accounting activities for the same session. @@ -7107,16 +6887,15 @@ Internet-Draft Diameter Base Protocol January 2011 - -Fajardo, et al. Expires July 24, 2011 [Page 127] +Fajardo, et al. Standards Track [Page 123] -Internet-Draft Diameter Base Protocol January 2011 +RFC 6733 Diameter Base Protocol October 2012 9.2. Protocol Messages A Diameter node that receives a successful authentication and/or - authorization messages from the Diameter server SHOULD collect + authorization message from the Diameter server SHOULD collect accounting information for the session. The Accounting-Request message is used to transmit the accounting information to the Diameter server, which MUST reply with the Accounting-Answer message @@ -7129,12 +6908,12 @@ Internet-Draft Diameter Base Protocol January 2011 9.3. Accounting Application Extension and Requirements - Each Diameter application (e.g., NASREQ, MobileIP), SHOULD define - their Service-Specific AVPs that MUST be present in the Accounting- - Request message in a section entitled "Accounting AVPs". The - application MUST assume that the AVPs described in this document will - be present in all Accounting messages, so only their respective - service-specific AVPs need to be defined in that section. + Each Diameter application (e.g., NASREQ, Mobile IP) SHOULD define its + service-specific AVPs that MUST be present in the Accounting-Request + message in a section titled "Accounting AVPs". The application MUST + assume that the AVPs described in this document will be present in + all Accounting messages, so only their respective service-specific + AVPs need to be defined in that section. Applications have the option of using one or both of the following accounting application extension models: @@ -7150,40 +6929,37 @@ Internet-Draft Diameter Base Protocol January 2011 advertise the Diameter base accounting Application Id during capabilities exchange. - Coupled Accounting Service - The accounting messages will carry the Application Id of the + The accounting message will carry the Application Id of the application that is using it. The application itself will process the received accounting records or forward them to an accounting server. There is no accounting application advertisement required - during capabilities exchange and the accounting messages will be - routed the same as any of the other application messages. - + during capabilities exchange, and the accounting messages will be + routed the same way as any of the other application messages. + In cases where an application does not define its own accounting + service, it is preferred that the split accounting model be used. -Fajardo, et al. Expires July 24, 2011 [Page 128] +Fajardo, et al. Standards Track [Page 124] -Internet-Draft Diameter Base Protocol January 2011 +RFC 6733 Diameter Base Protocol October 2012 - In cases where an application does not define its own accounting - service, it is preferred that the split accounting model be used. - 9.4. Fault Resilience - Diameter Base protocol mechanisms are used to overcome small message - loss and network faults of temporary nature. + Diameter base protocol mechanisms are used to overcome small message + loss and network faults of a temporary nature. Diameter peers acting as clients MUST implement the use of failover to guard against server failures and certain network failures. Diameter peers acting as agents or related off-line processing systems MUST detect duplicate accounting records caused by the - sending of same record to several servers and duplication of messages - in transit. This detection MUST be based on the inspection of the - Session-Id and Accounting-Record-Number AVP pairs. Appendix C + sending of the same record to several servers and duplication of + messages in transit. This detection MUST be based on the inspection + of the Session-Id and Accounting-Record-Number AVP pairs. Appendix C discusses duplicate detection needs and implementation issues. Diameter clients MAY have non-volatile memory for the safe storage of @@ -7191,23 +6967,23 @@ Internet-Draft Diameter Base Protocol January 2011 partitions, and server failures. If such memory is available, the client SHOULD store new accounting records there as soon as the records are created and until a positive acknowledgement of their - reception from the Diameter Server has been received. Upon a reboot, - the client MUST starting sending the records in the non-volatile - memory to the accounting server with appropriate modifications in + reception from the Diameter server has been received. Upon a reboot, + the client MUST start sending the records in the non-volatile memory + to the accounting server with the appropriate modifications in termination cause, session length, and other relevant information in the records. A further application of this protocol may include AVPs to control - how many accounting records may at most be stored in the Diameter - client without committing them to the non-volatile memory or + the maximum number of accounting records that may be stored in the + Diameter client without committing them to the non-volatile memory or transferring them to the Diameter server. The client SHOULD NOT remove the accounting data from any of its memory areas before the correct Accounting-Answer has been received. - The client MAY remove oldest, undelivered or yet unacknowledged - accounting data if it runs out of resources such as memory. It is an - implementation dependent matter for the client to accept new sessions - under this condition. + The client MAY remove the oldest, undelivered, or as yet + unacknowledged accounting data if it runs out of resources such as + memory. It is an implementation-dependent matter for the client to + accept new sessions under this condition. 9.5. Accounting Records @@ -7217,15 +6993,17 @@ Internet-Draft Diameter Base Protocol January 2011 Different types of accounting records are sent depending on the actual type of accounted service and the authorization server's + directions for interim accounting. If the accounted service is a + + -Fajardo, et al. Expires July 24, 2011 [Page 129] +Fajardo, et al. Standards Track [Page 125] -Internet-Draft Diameter Base Protocol January 2011 +RFC 6733 Diameter Base Protocol October 2012 - directions for interim accounting. If the accounted service is a one-time event, meaning that the start and stop of the event are simultaneous, then the Accounting-Record-Type AVP MUST be present and set to the value EVENT_RECORD. @@ -7251,11 +7029,11 @@ Internet-Draft Diameter Base Protocol January 2011 session. A particular value of Accounting-Sub-Session-Id MUST appear only in - one sequence of accounting records from a DIAMETER client, except for + one sequence of accounting records from a Diameter client, except for the purposes of retransmission. The one sequence that is sent MUST be either one record with Accounting-Record-Type AVP set to the value - EVENT_RECORD, or several records starting with one having the value - START_RECORD, followed by zero or more INTERIM_RECORD and a single + EVENT_RECORD or several records starting with one having the value + START_RECORD, followed by zero or more INTERIM_RECORDs and a single STOP_RECORD. A particular Diameter application specification MUST define the type of sequences that MUST be used. @@ -7265,20 +7043,21 @@ Internet-Draft Diameter Base Protocol January 2011 accounting records with a specific application session by using the Session-Id of the particular application session in the accounting messages. Accounting messages MAY also use a different Session-Id - from that of the application sessions in which case other session + from that of the application sessions, in which case, other session- related information is needed to perform correlation. In cases where an application requires multiple accounting sub- - session, an Accounting-Sub-Session-Id AVP is used to differentiate + sessions, an Accounting-Sub-Session-Id AVP is used to differentiate each sub-session. The Session-Id would remain constant for all sub- - sessions and is be used to correlate all the sub-sessions to a + sessions and is used to correlate all the sub-sessions to a particular application session. Note that receiving a STOP_RECORD -Fajardo, et al. Expires July 24, 2011 [Page 130] + +Fajardo, et al. Standards Track [Page 126] -Internet-Draft Diameter Base Protocol January 2011 +RFC 6733 Diameter Base Protocol October 2012 with no Accounting-Sub-Session-Id AVP when sub-sessions were @@ -7290,36 +7069,36 @@ Internet-Draft Diameter Base Protocol January 2011 record may span multiple different Diameter applications and sessions used by the same user at a given time. In such cases, the Acct- Multi-Session-Id AVP is used. The Acct-Multi-Session-Id AVP SHOULD - be signaled by the server to the access device (typically during + be signaled by the server to the access device (typically, during authorization) when it determines that a request belongs to an existing session. The access device MUST then include the Acct- Multi-Session-Id AVP in all subsequent accounting messages. The Acct-Multi-Session-Id AVP MAY include the value of the original - Session-Id. It's contents are implementation specific, but MUST be - globally unique across other Acct-Multi-Session-Id, and MUST NOT + Session-Id. Its contents are implementation specific, but the MUST + be globally unique across other Acct-Multi-Session-Ids and MUST NOT change during the life of a session. A Diameter application document MUST define the exact concept of a - session that is being accounted, and MAY define the concept of a + session that is being accounted, and it MAY define the concept of a multi-session. For instance, the NASREQ DIAMETER application treats - a single PPP connection to a Network Access Server as one session, - and a set of Multilink PPP sessions as one multi-session. + a single PPP connection to a Network Access Server as one session and + a set of Multilink PPP sessions as one multi-session. -9.7. Accounting Command-Codes +9.7. Accounting Command Codes - This section defines Command-Code values that MUST be supported by - all Diameter implementations that provide Accounting services. + This section defines Command Code values that MUST be supported by + all Diameter implementations that provide accounting services. 9.7.1. Accounting-Request - The Accounting-Request (ACR) command, indicated by the Command-Code + The Accounting-Request (ACR) command, indicated by the Command Code field set to 271 and the Command Flags' 'R' bit set, is sent by a Diameter node, acting as a client, in order to exchange accounting information with a peer. - The AVP listed below SHOULD include service-specific accounting AVPs, - as described in Section 9.3. + In addition to the AVPs listed below, Accounting-Request messages + SHOULD include service-specific accounting AVPs. @@ -7332,9 +7111,9 @@ Internet-Draft Diameter Base Protocol January 2011 -Fajardo, et al. Expires July 24, 2011 [Page 131] +Fajardo, et al. Standards Track [Page 127] -Internet-Draft Diameter Base Protocol January 2011 +RFC 6733 Diameter Base Protocol October 2012 Message Format @@ -7363,16 +7142,16 @@ Internet-Draft Diameter Base Protocol January 2011 9.7.2. Accounting-Answer - The Accounting-Answer (ACA) command, indicated by the Command-Code + The Accounting-Answer (ACA) command, indicated by the Command Code field set to 271 and the Command Flags' 'R' bit cleared, is used to acknowledge an Accounting-Request command. The Accounting-Answer command contains the same Session-Id as the corresponding request. - Only the target Diameter Server, known as the home Diameter Server, + Only the target Diameter server, known as the home Diameter server, SHOULD respond with the Accounting-Answer command. - The AVP listed below SHOULD include service-specific accounting AVPs, - as described in Section 9.3. + In addition to the AVPs listed below, Accounting-Answer messages + SHOULD include service-specific accounting AVPs. @@ -7388,9 +7167,9 @@ Internet-Draft Diameter Base Protocol January 2011 -Fajardo, et al. Expires July 24, 2011 [Page 132] +Fajardo, et al. Standards Track [Page 128] -Internet-Draft Diameter Base Protocol January 2011 +RFC 6733 Diameter Base Protocol October 2012 Message Format @@ -7429,13 +7208,12 @@ Internet-Draft Diameter Base Protocol January 2011 and contains the type of accounting record being sent. The following values are currently defined for the Accounting-Record-Type AVP: - EVENT_RECORD 1 An Accounting Event Record is used to indicate that a one-time event has occurred (meaning that the start and end of the event are simultaneous). This record contains all information relevant - to the service, and is the only record of the service. + to the service, and it is the only record of the service. @@ -7444,20 +7222,20 @@ Internet-Draft Diameter Base Protocol January 2011 -Fajardo, et al. Expires July 24, 2011 [Page 133] + +Fajardo, et al. Standards Track [Page 129] -Internet-Draft Diameter Base Protocol January 2011 +RFC 6733 Diameter Base Protocol October 2012 START_RECORD 2 - An Accounting Start, Interim, and Stop Records are used to - indicate that a service of a measurable length has been given. An - Accounting Start Record is used to initiate an accounting session, + Accounting Start, Interim, and Stop Records are used to indicate + that a service of a measurable length has been given. An + Accounting Start Record is used to initiate an accounting session and contains accounting information that is relevant to the initiation of the session. - INTERIM_RECORD 3 An Interim Accounting Record contains cumulative accounting @@ -7468,14 +7246,12 @@ Internet-Draft Diameter Base Protocol January 2011 applications. The selection of whether to use INTERIM_RECORD records is done by the Acct-Interim-Interval AVP. - STOP_RECORD 4 An Accounting Stop Record is sent to terminate an accounting session and contains cumulative accounting information relevant to the existing session. - 9.8.2. Acct-Interim-Interval AVP The Acct-Interim-Interval AVP (AVP Code 85) is of type Unsigned32 and @@ -7483,31 +7259,31 @@ Internet-Draft Diameter Base Protocol January 2011 client. The client uses information in this AVP to decide how and when to produce accounting records. With different values in this AVP, service sessions can result in one, two, or two+N accounting - records, based on the needs of the home-organization. The following + records, based on the needs of the home organization. The following accounting record production behavior is directed by the inclusion of this AVP: - 1. The omission of the Acct-Interim-Interval AVP or its inclusion with Value field set to 0 means that EVENT_RECORD, START_RECORD, and STOP_RECORD are produced, as appropriate for the service. - 2. The inclusion of the AVP with Value field set to a non-zero value means that INTERIM_RECORD records MUST be produced between the START_RECORD and STOP_RECORD records. The Value field of this AVP is the nominal interval between these records in seconds. + The Diameter node that originates the accounting information, + known as the client, MUST produce the first INTERIM_RECORD record + roughly at the time when this nominal interval has elapsed from + + -Fajardo, et al. Expires July 24, 2011 [Page 134] +Fajardo, et al. Standards Track [Page 130] -Internet-Draft Diameter Base Protocol January 2011 +RFC 6733 Diameter Base Protocol October 2012 - The Diameter node that originates the accounting information, - known as the client, MUST produce the first INTERIM_RECORD record - roughly at the time when this nominal interval has elapsed from the START_RECORD, the next one again as the interval has elapsed once more, and so on until the session ends and a STOP_RECORD record is produced. @@ -7522,10 +7298,10 @@ Internet-Draft Diameter Base Protocol January 2011 The Accounting-Record-Number AVP (AVP Code 485) is of type Unsigned32 and identifies this record within one session. As Session-Id AVPs are globally unique, the combination of Session-Id and Accounting- - Record-Number AVPs is also globally unique, and can be used in + Record-Number AVPs is also globally unique and can be used in matching accounting records with confirmations. An easy way to produce unique numbers is to set the value to 0 for records of type - EVENT_RECORD and START_RECORD, and set the value to 1 for the first + EVENT_RECORD and START_RECORD and set the value to 1 for the first INTERIM_RECORD, 2 for the second, and so on until the value for STOP_RECORD is one more than for the last INTERIM_RECORD. @@ -7539,10 +7315,10 @@ Internet-Draft Diameter Base Protocol January 2011 The Acct-Multi-Session-Id AVP (AVP Code 50) is of type UTF8String, following the format specified in Section 8.8. The Acct-Multi- - Session-Id AVP is used to link together multiple related accounting - sessions, where each session would have a unique Session-Id, but the - same Acct-Multi-Session-Id AVP. This AVP MAY be returned by the - Diameter server in an authorization answer, and MUST be used in all + Session-Id AVP is used to link multiple related accounting sessions, + where each session would have a unique Session-Id but the same Acct- + Multi-Session-Id AVP. This AVP MAY be returned by the Diameter + server in an authorization answer, and it MUST be used in all accounting messages for the given session. 9.8.6. Accounting-Sub-Session-Id AVP @@ -7553,18 +7329,17 @@ Internet-Draft Diameter Base Protocol January 2011 session, and the value of this AVP MUST be monotonically increased by one for all new sub-sessions. The absence of this AVP implies no sub-sessions are in use, with the exception of an Accounting-Request + whose Accounting-Record-Type is set to STOP_RECORD. A STOP_RECORD + message with no Accounting-Sub-Session-Id AVP present will signal the + termination of all sub-sessions for a given Session-Id. -Fajardo, et al. Expires July 24, 2011 [Page 135] +Fajardo, et al. Standards Track [Page 131] -Internet-Draft Diameter Base Protocol January 2011 +RFC 6733 Diameter Base Protocol October 2012 - whose Accounting-Record-Type is set to STOP_RECORD. A STOP_RECORD - message with no Accounting-Sub-Session-Id AVP present will signal the - termination of all sub-sessions for a given Session-Id. - 9.8.7. Accounting-Realtime-Required AVP The Accounting-Realtime-Required AVP (AVP Code 483) is of type @@ -7574,7 +7349,6 @@ Internet-Draft Diameter Base Protocol January 2011 if the sending of accounting records to the accounting server has been temporarily prevented due to, for instance, a network problem. - DELIVER_AND_GRANT 1 The AVP with Value field set to DELIVER_AND_GRANT means that the @@ -7584,7 +7358,6 @@ Internet-Draft Diameter Base Protocol January 2011 the accounting record stream to a backup server is not a reason to discontinue the service to the user. - GRANT_AND_STORE 2 The AVP with Value field set to GRANT_AND_STORE means that service @@ -7594,60 +7367,51 @@ Internet-Draft Diameter Base Protocol January 2011 This is the default behavior if the AVP isn't included in the reply from the authorization server. - GRANT_AND_LOSE 3 The AVP with Value field set to GRANT_AND_LOSE means that service SHOULD be granted even if the records cannot be delivered or stored. +10. AVP Occurrence Tables + The following tables present the AVPs defined in this document and + specify in which Diameter messages they MAY or MAY NOT be present. + AVPs that occur only inside a Grouped AVP are not shown in these + tables. + The tables use the following symbols: + 0 The AVP MUST NOT be present in the message. + 0+ Zero or more instances of the AVP MAY be present in the + message. - - -Fajardo, et al. Expires July 24, 2011 [Page 136] +Fajardo, et al. Standards Track [Page 132] -Internet-Draft Diameter Base Protocol January 2011 - - -10. AVP Occurrence Table - - The following tables presents the AVPs defined in this document, and - specifies in which Diameter messages they MAY be present or not. - AVPs that occur only inside a Grouped AVP are not shown in this - table. +RFC 6733 Diameter Base Protocol October 2012 - The table uses the following symbols: + 0-1 Zero or one instance of the AVP MAY be present in the message. + It is considered an error if there are more than one instance + of the AVP. - 0 The AVP MUST NOT be present in the message. + 1 One instance of the AVP MUST be present in the message. - 0+ Zero or more instances of the AVP MAY be present in the - message. - - 0-1 Zero or one instance of the AVP MAY be present in the message. - It is considered an error if there are more than one instance of - the AVP. - - 1 One instance of the AVP MUST be present in the message. - - 1+ At least one instance of the AVP MUST be present in the - message. + 1+ At least one instance of the AVP MUST be present in the + message. 10.1. Base Protocol Command AVP Table - The table in this section is limited to the non-accounting Command + The table in this section is limited to the non-Accounting Command Codes defined in this specification. +-----------------------------------------------+ - | Command-Code | + | Command Code | +---+---+---+---+---+---+---+---+---+---+---+---+ Attribute Name |CER|CEA|DPR|DPA|DWR|DWA|RAR|RAA|ASR|ASA|STR|STA| --------------------+---+---+---+---+---+---+---+---+---+---+---+---+ @@ -7665,22 +7429,29 @@ Internet-Draft Diameter Base Protocol January 2011 Class |0 |0 |0 |0 |0 |0 |0 |0 |0 |0 |0+ |0+ | Destination-Host |0 |0 |0 |0 |0 |0 |1 |0 |1 |0 |0-1|0 | Destination-Realm |0 |0 |0 |0 |0 |0 |1 |0 |1 |0 |1 |0 | - - - -Fajardo, et al. Expires July 24, 2011 [Page 137] - -Internet-Draft Diameter Base Protocol January 2011 - - Disconnect-Cause |0 |0 |1 |0 |0 |0 |0 |0 |0 |0 |0 |0 | Error-Message |0 |0-1|0 |0-1|0 |0-1|0 |0-1|0 |0-1|0 |0-1| Error-Reporting-Host|0 |0 |0 |0 |0 |0 |0 |0-1|0 |0-1|0 |0-1| - Failed-AVP |0 |0+ |0 |0+ |0 |0+ |0 |0+ |0 |0+ |0 |0+ | + Failed-AVP |0 |0-1|0 |0-1|0 |0-1|0 |0-1|0 |0-1|0 |0-1| Firmware-Revision |0-1|0-1|0 |0 |0 |0 |0 |0 |0 |0 |0 |0 | Host-IP-Address |1+ |1+ |0 |0 |0 |0 |0 |0 |0 |0 |0 |0 | Inband-Security-Id |0 |0 |0 |0 |0 |0 |0 |0 |0 |0 |0 |0 | Multi-Round-Time-Out|0 |0 |0 |0 |0 |0 |0 |0 |0 |0 |0 |0 | + + + + + + + + + + +Fajardo, et al. Standards Track [Page 133] + +RFC 6733 Diameter Base Protocol October 2012 + + Origin-Host |1 |1 |1 |1 |1 |1 |1 |1 |1 |1 |1 |1 | Origin-Realm |1 |1 |1 |1 |1 |1 |1 |1 |1 |1 |1 |1 | Origin-State-Id |0-1|0-1|0 |0 |0-1|0-1|0-1|0-1|0-1|0-1|0-1|0-1| @@ -7724,9 +7495,17 @@ Internet-Draft Diameter Base Protocol January 2011 -Fajardo, et al. Expires July 24, 2011 [Page 138] + + + + + + + + +Fajardo, et al. Standards Track [Page 134] -Internet-Draft Diameter Base Protocol January 2011 +RFC 6733 Diameter Base Protocol October 2012 +-----------+ @@ -7749,6 +7528,7 @@ Internet-Draft Diameter Base Protocol January 2011 Destination-Realm | 1 | 0 | Error-Reporting-Host | 0 | 0+ | Event-Timestamp | 0-1 | 0-1 | + Failed-AVP | 0 | 0-1 | Origin-Host | 1 | 1 | Origin-Realm | 1 | 1 | Proxy-Info | 0+ | 0+ | @@ -7760,144 +7540,220 @@ Internet-Draft Diameter Base Protocol January 2011 Vendor-Specific-Application-Id| 0-1 | 0-1 | ------------------------------+-----+-----+ +11. IANA Considerations + This section provides guidance to the Internet Assigned Numbers + Authority (IANA) regarding registration of values related to the + Diameter protocol, in accordance with [RFC5226]. Existing IANA + registries and assignments put in place by RFC 3588 remain the same + unless explicitly updated or deprecated in this section. +11.1. AVP Header + As defined in Section 4, the AVP header contains three fields that + require IANA namespace management: the AVP Code, Vendor-ID, and Flags + fields. - - - - - - - - - - - -Fajardo, et al. Expires July 24, 2011 [Page 139] +Fajardo, et al. Standards Track [Page 135] -Internet-Draft Diameter Base Protocol January 2011 +RFC 6733 Diameter Base Protocol October 2012 -11. IANA Considerations +11.1.1. AVP Codes - This section provides guidance to the Internet Assigned Numbers - Authority (IANA) regarding registration of values related to the - Diameter protocol, in accordance with BCP 26 [RFC5226]. The policies - and procedures for the IANA put in place by [RFC3588] applies here. - The criteria used by the IANA for assignment of numbers within this - namespace remains the same unless otherwise stated in this section. - Existing assignments remains the same unless explicitly updated or - deprecated in this secion. + There are multiple namespaces. Vendors can have their own AVP Codes + namespace that will be identified by their Vendor-ID (also known as + Enterprise-Number), and they control the assignments of their vendor- + specific AVP Codes within their own namespace. The absence of a + Vendor-ID or a Vendor-ID value of zero (0) identifies the IETF AVP + Codes namespace, which is under IANA control. The AVP Codes and + sometimes possible values in an AVP are controlled and maintained by + IANA. AVP Code 0 is not used. AVP Codes 1-255 are managed + separately as RADIUS Attribute Types. Where a Vendor-Specific AVP is + implemented by more than one vendor, allocation of global AVPs should + be encouraged instead. + + AVPs may be allocated following Expert Review (by a Designated + Expert) with Specification Required [RFC5226]. A block allocation + (release of more than three AVPs at a time for a given purpose) + requires IETF Review [RFC5226]. -11.1. Changes to AVP Header Allocation +11.1.2. AVP Flags - For AVP Headers, the only change is the AVP code block allocations. - Block allocation (release of more than 3 at a time for a given - purpose) now only require IETF Review as opposed to an IETF - Consensus. + Section 4.1 describes the existing AVP Flags. The remaining bits can + only be assigned via a Standards Action [RFC5226]. 11.2. Diameter Header - For the Diameter Header, the command code namespace allocation has +11.2.1. Command Codes + + For the Diameter header, the Command Code namespace allocation has changed. The new allocation rules are as follows: - The command code values 256 - 8,388,607 (0x100 to 0x7fffff) are + The Command Code values 256 - 8,388,607 (0x100 to 0x7fffff) are for permanent, standard commands, allocated by IETF Review [RFC5226]. The values 8,388,608 - 16,777,213 (0x800000 - 0xfffffd) are - reserved for vendor-specific command codes, to be allocated on a + reserved for vendor-specific Command Codes, to be allocated on a First Come, First Served basis by IANA [RFC5226]. The request to IANA for a Vendor-Specific Command Code SHOULD include a reference - to a publicly available specification which documents the command + to a publicly available specification that documents the command in sufficient detail to aid in interoperability between independent implementations. If the specification cannot be made - publicly available, the request for a vendor-specific command code + publicly available, the request for a vendor-specific Command Code MUST include the contact information of persons and/or entities responsible for authoring and maintaining the command. + + + + + +Fajardo, et al. Standards Track [Page 136] + +RFC 6733 Diameter Base Protocol October 2012 + + + The values 16,777,214 and 16,777,215 (hexadecimal values 0xfffffe + - 0xffffff) are reserved for experimental commands. As these + codes are only for experimental and testing purposes, no guarantee + is made for interoperability between Diameter peers using + experimental commands. + +11.2.2. Command Flags + + Section 3 describes the existing Command Flags field. The remaining + bits can only be assigned via a Standards Action [RFC5226]. + 11.3. AVP Values For AVP values, the Experimental-Result-Code AVP value allocation has - been added. The new rule is as follows: + been added; see Section 11.3.1. The old AVP value allocation rule, + IETF Consensus, has been updated to IETF Review as per [RFC5226], and + affected AVPs are listed as reminders. 11.3.1. Experimental-Result-Code AVP Values for this AVP are purely local to the indicated vendor, and no IANA registry is maintained for them. +11.3.2. Result-Code AVP Values + New values are available for assignment via IETF Review [RFC5226]. +11.3.3. Accounting-Record-Type AVP Values -Fajardo, et al. Expires July 24, 2011 [Page 140] - -Internet-Draft Diameter Base Protocol January 2011 + New values are available for assignment via IETF Review [RFC5226]. + +11.3.4. Termination-Cause AVP Values + New values are available for assignment via IETF Review [RFC5226]. -11.4. Diameter TCP, SCTP, TLS/TCP and DTLS/SCTP Port Numbers +11.3.5. Redirect-Host-Usage AVP Values - Updated port number assignments are described in this section. The - IANA has assigned port number 3868 for TCP and SCTP. The port number - [TBD] has been assigned for TLS/TCP and DTLS/SCTP. + New values are available for assignment via IETF Review [RFC5226]. -11.5. S-NAPTR Parameters +11.3.6. Session-Server-Failover AVP Values - This document registers a new S-NAPTR Application Service Tag value - of "aaa". + New values are available for assignment via IETF Review [RFC5226]. - This document also registers the following S-NAPTR Application - Protocol Tags: +11.3.7. Session-Binding AVP Values - Tag | Protocol - -------------------|--------- - diameter.tcp | TCP - diameter.sctp | SCTP - diameter.tls.tcp | TLS/TCP - diameter.dtls.sctp | DTLS/SCTP + New values are available for assignment via IETF Review [RFC5226]. +Fajardo, et al. Standards Track [Page 137] + +RFC 6733 Diameter Base Protocol October 2012 + + +11.3.8. Disconnect-Cause AVP Values + New values are available for assignment via IETF Review [RFC5226]. +11.3.9. Auth-Request-Type AVP Values + New values are available for assignment via IETF Review [RFC5226]. +11.3.10. Auth-Session-State AVP Values + New values are available for assignment via IETF Review [RFC5226]. +11.3.11. Re-Auth-Request-Type AVP Values + New values are available for assignment via IETF Review [RFC5226]. +11.3.12. Accounting-Realtime-Required AVP Values + New values are available for assignment via IETF Review [RFC5226]. +11.3.13. Inband-Security-Id AVP (code 299) + The use of this AVP has been deprecated. +11.4. _diameters Service Name and Port Number Registration + IANA has registered the "_diameters" service name and assigned port + numbers for TLS/TCP and DTLS/SCTP according to the guidelines given + in [RFC6335]. + Service Name: _diameters + Transport Protocols: TCP, SCTP + Assignee: IESG <[email protected]> + Contact: IETF Chair <[email protected]> + Description: Diameter over TLS/TCP and DTLS/SCTP + Reference: RFC 6733 + Port Number: 5868, from the User Range -Fajardo, et al. Expires July 24, 2011 [Page 141] + + +Fajardo, et al. Standards Track [Page 138] -Internet-Draft Diameter Base Protocol January 2011 +RFC 6733 Diameter Base Protocol October 2012 + + +11.5. SCTP Payload Protocol Identifiers + + Two SCTP payload protocol identifiers have been registered in the + SCTP Payload Protocol Identifiers registry: + + + Value | SCTP Payload Protocol Identifier + -------|----------------------------------- + 46 | Diameter in a SCTP DATA chunk + 47 | Diameter in a DTLS/SCTP DATA chunk + +11.6. S-NAPTR Parameters -12. Diameter protocol related configurable parameters + The following tag has been registered in the S-NAPTR Application + Protocol Tags registry: + + Tag | Protocol + -------------------|--------- + diameter.dtls.sctp | DTLS/SCTP + +12. Diameter Protocol-Related Configurable Parameters This section contains the configurable parameters that are found throughout this document: @@ -7915,8 +7771,8 @@ Internet-Draft Diameter Base Protocol January 2011 A Diameter proxy server routes messages based on the realm portion of a Network Access Identifier (NAI). The server MUST have a table of Realm Names, and the address of the peer to which the - message must be forwarded to. The routing table MAY also include - a "default route", which is typically used for all messages that + message must be forwarded. The routing table MAY also include a + "default route", which is typically used for all messages that cannot be locally processed. Tc timer @@ -7927,30 +7783,9 @@ Internet-Draft Diameter Base Protocol January 2011 - - - - - - - - - - - - - - - - - - - - - -Fajardo, et al. Expires July 24, 2011 [Page 142] +Fajardo, et al. Standards Track [Page 139] -Internet-Draft Diameter Base Protocol January 2011 +RFC 6733 Diameter Base Protocol October 2012 13. Security Considerations @@ -7959,34 +7794,34 @@ Internet-Draft Diameter Base Protocol January 2011 [RFC5246] or DTLS/SCTP [RFC6083]. Additional security mechanisms such as IPsec [RFC4301] MAY also be deployed to secure connections between peers. However, all Diameter base protocol implementations - MUST support the use of TLS/TCP and DTLS/SCTP and the Diameter - protocol MUST NOT be used without any security mechanism. + MUST support the use of TLS/TCP and DTLS/SCTP, and the Diameter + protocol MUST NOT be used without one of TLS, DTLS, or IPsec. If a Diameter connection is to be protected via TLS/TCP and DTLS/SCTP or IPsec, then TLS/TCP and DTLS/SCTP or IPsec/IKE SHOULD begin prior to any Diameter message exchange. All security parameters for TLS/ TCP and DTLS/SCTP or IPsec are configured independent of the Diameter - protocol. All Diameter message will be sent through the TLS/TCP and + protocol. All Diameter messages will be sent through the TLS/TCP and DTLS/SCTP or IPsec connection after a successful setup. For TLS/TCP and DTLS/SCTP connections to be established in the open state, the CER/CEA exchange MUST include an Inband-Security-ID AVP with a value of TLS/TCP and DTLS/SCTP. The TLS/TCP and DTLS/SCTP - handshake will begin when both ends successfully reached the open + handshake will begin when both ends successfully reach the open state, after completion of the CER/CEA exchange. If the TLS/TCP and DTLS/SCTP handshake is successful, all further messages will be sent - via TLS/TCP and DTLS/SCTP. If the handshake fails, both ends move to - the closed state. See Sections 13.1 for more details. + via TLS/TCP and DTLS/SCTP. If the handshake fails, both ends MUST + move to the closed state. See Section 13.1 for more details. 13.1. TLS/TCP and DTLS/SCTP Usage Diameter nodes using TLS/TCP and DTLS/SCTP for security MUST mutually authenticate as part of TLS/TCP and DTLS/SCTP session establishment. In order to ensure mutual authentication, the Diameter node acting as - TLS/TCP and DTLS/SCTP server MUST request a certificate from the + the TLS/TCP and DTLS/SCTP server MUST request a certificate from the Diameter node acting as TLS/TCP and DTLS/SCTP client, and the - Diameter node acting as TLS/TCP and DTLS/SCTP client MUST be prepared - to supply a certificate on request. + Diameter node acting as the TLS/TCP and DTLS/SCTP client MUST be + prepared to supply a certificate on request. Diameter nodes MUST be able to negotiate the following TLS/TCP and DTLS/SCTP cipher suites: @@ -8000,16 +7835,24 @@ Internet-Draft Diameter Base Protocol January 2011 TLS_RSA_WITH_AES_128_CBC_SHA - Diameter nodes MAY negotiate other TLS/TCP and DTLS/SCTP cipher -Fajardo, et al. Expires July 24, 2011 [Page 143] + +Fajardo, et al. Standards Track [Page 140] -Internet-Draft Diameter Base Protocol January 2011 +RFC 6733 Diameter Base Protocol October 2012 + + Note that it is quite possible that support for the + TLS_RSA_WITH_AES_128_CBC_SHA cipher suite will be REQUIRED at some + future date. Diameter nodes MAY negotiate other TLS/TCP and DTLS/ + SCTP cipher suites. - suites. + If public key certificates are used for Diameter security (for + example, with TLS), the value of the expiration times in the routing + and peer tables MUST NOT be greater than the expiry time in the + relevant certificates. 13.2. Peer-to-Peer Considerations @@ -8027,67 +7870,105 @@ Internet-Draft Diameter Base Protocol January 2011 peers may not be known beforehand and therefore peer discovery may be required. +13.3. AVP Considerations + Diameter AVPs often contain security-sensitive data; for example, + user passwords and location data, network addresses and cryptographic + keys. The following AVPs defined in this document are considered to + be security-sensitive: + o Acct-Interim-Interval + o Accounting-Realtime-Required + o Acct-Multi-Session-Id + o Accounting-Record-Number + o Accounting-Record-Type + o Accounting-Session-Id + o Accounting-Sub-Session-Id + o Class +Fajardo, et al. Standards Track [Page 141] + +RFC 6733 Diameter Base Protocol October 2012 + o Session-Id + o Session-Binding + o Session-Server-Failover + o User-Name + Diameter messages containing these or any other AVPs considered to be + security-sensitive MUST only be sent protected via mutually + authenticated TLS or IPsec. In addition, those messages MUST NOT be + sent via intermediate nodes unless there is end-to-end security + between the originator and recipient or the originator has locally + trusted configuration that indicates that end-to-end security is not + needed. For example, end-to-end security may not be required in the + case where an intermediary node is known to be operated as part of + the same administrative domain as the endpoints so that an ability to + successfully compromise the intermediary would imply a high + probability of being able to compromise the endpoints as well. Note + that no end-to-end security mechanism is specified in this document. +14. References +14.1. Normative References + [FLOATPOINT] + Institute of Electrical and Electronics Engineers, "IEEE + Standard for Binary Floating-Point Arithmetic, ANSI/IEEE + Standard 754-1985", August 1985. + [IANAADFAM] + IANA, "Address Family Numbers", + <http://www.iana.org/assignments/address-family-numbers>. + [RFC0791] Postel, J., "Internet Protocol", STD 5, RFC 791, + September 1981. + [RFC0793] Postel, J., "Transmission Control Protocol", STD 7, + RFC 793, September 1981. + [RFC2119] Bradner, S., "Key words for use in RFCs to Indicate + Requirement Levels", BCP 14, RFC 2119, March 1997. + [RFC3492] Costello, A., "Punycode: A Bootstring encoding of Unicode + for Internationalized Domain Names in Applications + (IDNA)", RFC 3492, March 2003. - - -Fajardo, et al. Expires July 24, 2011 [Page 144] +Fajardo, et al. Standards Track [Page 142] -Internet-Draft Diameter Base Protocol January 2011 - - -14. References - -14.1. Normative References - - [FLOATPOINT] - Institute of Electrical and Electronics Engineers, "IEEE - Standard for Binary Floating-Point Arithmetic, ANSI/IEEE - Standard 754-1985", August 1985. +RFC 6733 Diameter Base Protocol October 2012 - [IANAADFAM] - IANA,, "Address Family Numbers", - http://www.iana.org/assignments/address-family-numbers. - [RADTYPE] IANA,, "RADIUS Types", - http://www.iana.org/assignments/radius-types. + [RFC3539] Aboba, B. and J. Wood, "Authentication, Authorization and + Accounting (AAA) Transport Profile", RFC 3539, June 2003. - [RFC791] Postel, J., "Internet Protocol", RFC 791, September 1981. + [RFC3629] Yergeau, F., "UTF-8, a transformation format of ISO + 10646", STD 63, RFC 3629, November 2003. - [RFC793] Postel, J., "Transmission Control Protocol", RFC 793, - January 1981. + [RFC3958] Daigle, L. and A. Newton, "Domain-Based Application + Service Location Using SRV RRs and the Dynamic Delegation + Discovery Service (DDDS)", RFC 3958, January 2005. - [RFC3539] Aboba, B. and J. Wood, "Authentication, Authorization and - Accounting (AAA) Transport Profile", RFC 3539, June 2003. + [RFC3986] Berners-Lee, T., Fielding, R., and L. Masinter, "Uniform + Resource Identifier (URI): Generic Syntax", STD 66, + RFC 3986, January 2005. [RFC4004] Calhoun, P., Johansson, T., Perkins, C., Hiller, T., and P. McCann, "Diameter Mobile IPv4 Application", RFC 4004, @@ -8101,51 +7982,44 @@ Internet-Draft Diameter Base Protocol January 2011 Loughney, "Diameter Credit-Control Application", RFC 4006, August 2005. - [RFC5234] Crocker, D. and P. Overell, "Augmented BNF for Syntax - Specifications: ABNF", STD 68, RFC 5234, January 2008. - - [RFC3588] Calhoun, P., Loughney, J., Guttman, E., Zorn, G., and J. - Arkko, "Diameter Base Protocol", RFC 3588, September 2003. + [RFC4086] Eastlake, D., Schiller, J., and S. Crocker, "Randomness + Requirements for Security", BCP 106, RFC 4086, June 2005. - [RFC5226] Narten, T. and H. Alvestrand, "Guidelines for Writing an - IANA Considerations Section in RFCs", BCP 26, RFC 5226, - May 2008. + [RFC4282] Aboba, B., Beadles, M., Arkko, J., and P. Eronen, "The + Network Access Identifier", RFC 4282, December 2005. [RFC4291] Hinden, R. and S. Deering, "IP Version 6 Addressing Architecture", RFC 4291, February 2006. + [RFC4960] Stewart, R., "Stream Control Transmission Protocol", + RFC 4960, September 2007. + [RFC5226] Narten, T. and H. Alvestrand, "Guidelines for Writing an + IANA Considerations Section in RFCs", BCP 26, RFC 5226, + May 2008. -Fajardo, et al. Expires July 24, 2011 [Page 145] - -Internet-Draft Diameter Base Protocol January 2011 - + [RFC5234] Crocker, D. and P. Overell, "Augmented BNF for Syntax + Specifications: ABNF", STD 68, RFC 5234, January 2008. - [RFC2119] Bradner, S., "Key words for use in RFCs to Indicate - Requirement Levels", BCP 14, RFC 2119, March 1997. + [RFC5246] Dierks, T. and E. Rescorla, "The Transport Layer Security + (TLS) Protocol Version 1.2", RFC 5246, August 2008. - [RFC4282] Aboba, B., Beadles, M., Arkko, J., and P. Eronen, "The - Network Access Identifier", RFC 4282, December 2005. - [RFC4086] Eastlake, D., Schiller, J., and S. Crocker, "Randomness - Requirements for Security", BCP 106, RFC 4086, June 2005. - [RFC4960] Stewart, R., "Stream Control Transmission Protocol", - RFC 4960, September 2007. - [RFC3958] Daigle, L. and A. Newton, "Domain-Based Application - Service Location Using SRV RRs and the Dynamic Delegation - Discovery Service (DDDS)", RFC 3958, January 2005. +Fajardo, et al. Standards Track [Page 143] + +RFC 6733 Diameter Base Protocol October 2012 - [RFC5246] Dierks, T. and E. Rescorla, "The Transport Layer Security - (TLS) Protocol Version 1.2", RFC 5246, August 2008. - [RFC3986] Berners-Lee, T., Fielding, R., and L. Masinter, "Uniform - Resource Identifier (URI): Generic Syntax", STD 66, - RFC 3986, January 2005. + [RFC5280] Cooper, D., Santesson, S., Farrell, S., Boeyen, S., + Housley, R., and W. Polk, "Internet X.509 Public Key + Infrastructure Certificate and Certificate Revocation List + (CRL) Profile", RFC 5280, May 2008. - [RFC3629] Yergeau, F., "UTF-8, a transformation format of ISO - 10646", STD 63, RFC 3629, November 2003. + [RFC5729] Korhonen, J., Jones, M., Morand, L., and T. Tsou, + "Clarifications on the Routing of Diameter Requests Based + on the Username and the Realm", RFC 5729, December 2009. [RFC5890] Klensin, J., "Internationalized Domain Names for Applications (IDNA): Definitions and Document Framework", @@ -8154,103 +8028,120 @@ Internet-Draft Diameter Base Protocol January 2011 [RFC5891] Klensin, J., "Internationalized Domain Names in Applications (IDNA): Protocol", RFC 5891, August 2010. - [RFC3492] Costello, A., "Punycode: A Bootstring encoding of Unicode - for Internationalized Domain Names in Applications - (IDNA)", RFC 3492, March 2003. - - [RFC5729] Korhonen, J., Jones, M., Morand, L., and T. Tsou, - "Clarifications on the Routing of Diameter Requests Based - on the Username and the Realm", RFC 5729, December 2009. - - [RFC4347] Rescorla, E. and N. Modadugu, "Datagram Transport Layer - Security", RFC 4347, April 2006. - [RFC6083] Tuexen, M., Seggelmann, R., and E. Rescorla, "Datagram Transport Layer Security (DTLS) for Stream Control Transmission Protocol (SCTP)", RFC 6083, January 2011. + [RFC6347] Rescorla, E. and N. Modadugu, "Datagram Transport Layer + Security Version 1.2", RFC 6347, January 2012. + [RFC6408] Jones, M., Korhonen, J., and L. Morand, "Diameter + Straightforward-Naming Authority Pointer (S-NAPTR) Usage", + RFC 6408, November 2011. +14.2. Informative References -Fajardo, et al. Expires July 24, 2011 [Page 146] - -Internet-Draft Diameter Base Protocol January 2011 + [ENTERPRISE] IANA, "SMI Network Management Private Enterprise + Codes", + <http://www.iana.org/assignments/enterprise-numbers>. + [IANATCV] IANA, "Termination-Cause AVP Values (code 295)", + <http://www.iana.org/assignments/aaa-parameters/ + aaa-parameters.xml#aaa-parameters-16>. -14.2. Informational References + [RFC1492] Finseth, C., "An Access Control Protocol, Sometimes + Called TACACS", RFC 1492, July 1993. - [RFC2989] Aboba, B., Calhoun, P., Glass, S., Hiller, T., McCann, P., - Shiino, H., Walsh, P., Zorn, G., Dommety, G., Perkins, C., - Patil, B., Mitton, D., Manning, S., Beadles, M., Chen, X., - Sivalingham, S., Hameed, A., Munson, M., Jacobs, S., Lim, - B., Hirschman, B., Hsu, R., Koo, H., Lipford, M., - Campbell, E., Xu, Y., Baba, S., and E. Jaques, "Criteria - for Evaluating AAA Protocols for Network Access", - RFC 2989, November 2000. + [RFC1661] Simpson, W., "The Point-to-Point Protocol (PPP)", + STD 51, RFC 1661, July 1994. - [RFC2975] Aboba, B., Arkko, J., and D. Harrington, "Introduction to - Accounting Management", RFC 2975, October 2000. + [RFC2104] Krawczyk, H., Bellare, M., and R. Canetti, "HMAC: + Keyed-Hashing for Message Authentication", RFC 2104, + February 1997. - [RFC3232] Reynolds, J., "Assigned Numbers: RFC 1700 is Replaced by - an On-line Database", RFC 3232, January 2002. - [RFC5176] Chiba, M., Dommety, G., Eklund, M., Mitton, D., and B. - Aboba, "Dynamic Authorization Extensions to Remote - Authentication Dial In User Service (RADIUS)", RFC 5176, - January 2008. - [RFC1661] Simpson, W., "The Point-to-Point Protocol (PPP)", STD 51, - RFC 1661, July 1994. - [RFC2866] Rigney, C., "RADIUS Accounting", RFC 2866, June 2000. - [RFC2869] Rigney, C., Willats, W., and P. Calhoun, "RADIUS - Extensions", RFC 2869, June 2000. +Fajardo, et al. Standards Track [Page 144] + +RFC 6733 Diameter Base Protocol October 2012 - [RFC2865] Rigney, C., Willens, S., Rubens, A., and W. Simpson, - "Remote Authentication Dial In User Service (RADIUS)", - RFC 2865, June 2000. - [RFC3162] Aboba, B., Zorn, G., and D. Mitton, "RADIUS and IPv6", - RFC 3162, August 2001. + [RFC2782] Gulbrandsen, A., Vixie, P., and L. Esibov, "A DNS RR + for specifying the location of services (DNS SRV)", + RFC 2782, February 2000. - [RFC4301] Kent, S. and K. Seo, "Security Architecture for the - Internet Protocol", RFC 4301, December 2005. + [RFC2865] Rigney, C., Willens, S., Rubens, A., and W. Simpson, + "Remote Authentication Dial In User Service (RADIUS)", + RFC 2865, June 2000. - [RFC5905] Mills, D., Martin, J., Burbank, J., and W. Kasch, "Network - Time Protocol Version 4: Protocol and Algorithms - Specification", RFC 5905, June 2010. + [RFC2866] Rigney, C., "RADIUS Accounting", RFC 2866, June 2000. - [RFC1492] Finseth, C., "An Access Control Protocol, Sometimes Called - TACACS", RFC 1492, July 1993. + [RFC2869] Rigney, C., Willats, W., and P. Calhoun, "RADIUS + Extensions", RFC 2869, June 2000. - [RFC4690] Klensin, J., Faltstrom, P., Karp, C., and IAB, "Review and + [RFC2881] Mitton, D. and M. Beadles, "Network Access Server + Requirements Next Generation (NASREQNG) NAS Model", + RFC 2881, July 2000. + [RFC2975] Aboba, B., Arkko, J., and D. Harrington, "Introduction + to Accounting Management", RFC 2975, October 2000. + [RFC2989] Aboba, B., Calhoun, P., Glass, S., Hiller, T., McCann, + P., Shiino, H., Walsh, P., Zorn, G., Dommety, G., + Perkins, C., Patil, B., Mitton, D., Manning, S., + Beadles, M., Chen, X., Sivalingham, S., Hameed, A., + Munson, M., Jacobs, S., Lim, B., Hirschman, B., Hsu, + R., Koo, H., Lipford, M., Campbell, E., Xu, Y., Baba, + S., and E. Jaques, "Criteria for Evaluating AAA + Protocols for Network Access", RFC 2989, November 2000. -Fajardo, et al. Expires July 24, 2011 [Page 147] - -Internet-Draft Diameter Base Protocol January 2011 + [RFC3162] Aboba, B., Zorn, G., and D. Mitton, "RADIUS and IPv6", + RFC 3162, August 2001. + [RFC3748] Aboba, B., Blunk, L., Vollbrecht, J., Carlson, J., and + H. Levkowetz, "Extensible Authentication Protocol + (EAP)", RFC 3748, June 2004. - Recommendations for Internationalized Domain Names - (IDNs)", RFC 4690, September 2006. + [RFC4301] Kent, S. and K. Seo, "Security Architecture for the + Internet Protocol", RFC 4301, December 2005. - [RFC5461] Gont, F., "TCP's Reaction to Soft Errors", RFC 5461, - February 2009. + [RFC4690] Klensin, J., Faltstrom, P., Karp, C., and IAB, "Review + and Recommendations for Internationalized Domain Names + (IDNs)", RFC 4690, September 2006. - [RFC5927] Gont, F., "ICMP Attacks against TCP", RFC 5927, July 2010. + [RFC5176] Chiba, M., Dommety, G., Eklund, M., Mitton, D., and B. + Aboba, "Dynamic Authorization Extensions to Remote + Authentication Dial In User Service (RADIUS)", + RFC 5176, January 2008. - [RFC3692] Narten, T., "Assigning Experimental and Testing Numbers - Considered Useful", BCP 82, RFC 3692, January 2004. +Fajardo, et al. Standards Track [Page 145] + +RFC 6733 Diameter Base Protocol October 2012 + + [RFC5461] Gont, F., "TCP's Reaction to Soft Errors", RFC 5461, + February 2009. + [RFC5905] Mills, D., Martin, J., Burbank, J., and W. Kasch, + "Network Time Protocol Version 4: Protocol and + Algorithms Specification", RFC 5905, June 2010. + [RFC5927] Gont, F., "ICMP Attacks against TCP", RFC 5927, + July 2010. + [RFC6335] Cotton, M., Eggert, L., Touch, J., Westerlund, M., and + S. Cheshire, "Internet Assigned Numbers Authority + (IANA) Procedures for the Management of the Service + Name and Transport Protocol Port Number Registry", + BCP 165, RFC 6335, August 2011. + [RFC6737] Kang, J. and G. Zorn, "The Diameter Capabilities Update + Application", RFC 6737, October 2012. @@ -8284,80 +8175,83 @@ Internet-Draft Diameter Base Protocol January 2011 -Fajardo, et al. Expires July 24, 2011 [Page 148] +Fajardo, et al. Standards Track [Page 146] -Internet-Draft Diameter Base Protocol January 2011 +RFC 6733 Diameter Base Protocol October 2012 Appendix A. Acknowledgements -A.1. RFC3588bis +A.1. This Document The authors would like to thank the following people that have provided proposals and contributions to this document: To Vishnu Ram and Satendra Gera for their contributions on - Capabilities Updates, Predictive Loop Avoidance as well as many other - technical proposals. To Tolga Asveren for his insights and + capabilities updates, predictive loop avoidance, as well as many + other technical proposals. To Tolga Asveren for his insights and contributions on almost all of the proposed solutions incorporated - into this document. To Timothy Smith for helping on the Capabilities - Updates and other topics. To Tony Zhang for providing fixes to loop - holes on composing Failed-AVPs as well as many other issues and + into this document. To Timothy Smith for helping on the capabilities + Update and other topics. To Tony Zhang for providing fixes to + loopholes on composing Failed-AVPs as well as many other issues and topics. To Jan Nordqvist for clearly stating the usage of - Application Ids. To Anders Kristensen for providing needed technical + Application Ids. To Anders Kristensen for providing needed technical opinions. To David Frascone for providing invaluable review of the document. To Mark Jones for providing clarifying text on vendor - command codes and other vendor specific indicators. + command codes and other vendor-specific indicators. To Victor + Pascual and Sebastien Decugis for new text and recommendations on + SCTP/DTLS. To Jouni Korhonen for taking over the editing task and + resolving last bits from versions 27 through 29. - Special thanks to the Diameter extensibility design team which helped - resolve the tricky question of mandatory AVPs and ABNF semantics. - The members of this team are as follows: + Special thanks to the Diameter extensibility design team, which + helped resolve the tricky question of mandatory AVPs and ABNF + semantics. The members of this team are as follows: Avi Lior, Jari Arkko, Glen Zorn, Lionel Morand, Mark Jones, Tolga - Asveren Jouni Korhonen, Glenn McGregor. + Asveren, Jouni Korhonen, and Glenn McGregor. Special thanks also to people who have provided invaluable comments and inputs especially in resolving controversial issues: - Glen Zorn, Yoshihiro Ohba, Marco Stura, and Pasi Eronen. + Glen Zorn, Yoshihiro Ohba, Marco Stura, Stephen Farrel, Pete Resnick, + Peter Saint-Andre, Robert Sparks, Krishna Prasad, Sean Turner, Barry + Leiba, and Pasi Eronen. Finally, we would like to thank the original authors of this document: - Pat Calhoun, John Loughney, Jari Arkko, Erik Guttman and Glen Zorn. + Pat Calhoun, John Loughney, Jari Arkko, Erik Guttman, and Glen Zorn. Their invaluable knowledge and experience has given us a robust and flexible AAA protocol that many people have seen great value in adopting. We greatly appreciate their support and stewardship for the continued improvements of Diameter as a protocol. We would also like to extend our gratitude to folks aside from the authors who have - assisted and contributed to the original version of this document. - Their efforts significantly contributed to the success of Diameter. - - - -Fajardo, et al. Expires July 24, 2011 [Page 149] +Fajardo, et al. Standards Track [Page 147] -Internet-Draft Diameter Base Protocol January 2011 +RFC 6733 Diameter Base Protocol October 2012 -A.2. RFC3588 + assisted and contributed to the original version of this document. + Their efforts significantly contributed to the success of Diameter. + +A.2. RFC 3588 The authors would like to thank Nenad Trifunovic, Tony Johansson and Pankaj Patel for their participation in the pre-IETF Document Reading - Party. Allison Mankin, Jonathan Wood and Bernard Aboba provided - invaluable assistance in working out transport issues, and similarly - with Steven Bellovin in the security area. + Party. Allison Mankin, Jonathan Wood, and Bernard Aboba provided + invaluable assistance in working out transport issues and this was + also the case with Steven Bellovin in the security area. Paul Funk and David Mitton were instrumental in getting the Peer State Machine correct, and our deep thanks go to them for their time. Text in this document was also provided by Paul Funk, Mark Eklund, - Mark Jones and Dave Spence. Jacques Caron provided many great + Mark Jones, and Dave Spence. Jacques Caron provided many great comments as a result of a thorough review of the spec. The authors would also like to acknowledge the following people for @@ -8367,96 +8261,57 @@ A.2. RFC3588 David Frascone, Daniel C. Fox, Lol Grant, Ignacio Goyret, Nancy Greene, Peter Heitman, Fredrik Johansson, Mark Jones, Martin Julien, Bob Kopacz, Paul Krumviede, Fergal Ladley, Ryan Moats, Victor Muslin, - Kenneth Peirce, John Schnizlein, Sumit Vakil, John R. Vollbrecht and + Kenneth Peirce, John Schnizlein, Sumit Vakil, John R. Vollbrecht, and Jeff Weisberg. Finally, Pat Calhoun would like to thank Sun Microsystems since most of the effort put into this document was done while he was in their employ. - - - - - - - - - - - - - - - - - - - - - - -Fajardo, et al. Expires July 24, 2011 [Page 150] - -Internet-Draft Diameter Base Protocol January 2011 - - Appendix B. S-NAPTR Example As an example, consider a client that wishes to resolve aaa: - example1.com. The client performs a NAPTR query for that domain, and - the following NAPTR records are returned: + ex1.example.com. The client performs a NAPTR query for that domain, + and the following NAPTR records are returned: ;; order pref flags service regexp replacement IN NAPTR 50 50 "s" "aaa:diameter.tls.tcp" "" - _diameter._tls.example1.com + _diameter._tls.ex1.example.com IN NAPTR 100 50 "s" "aaa:diameter.tcp" "" - _aaa._tcp.example1.com + _aaa._tcp.ex1.example.com IN NAPTR 150 50 "s" "aaa:diameter.sctp" "" - _diameter._sctp.example1.com + _diameter._sctp.ex1.example.com - This indicates that the server supports TLS, TCP and SCTP in that + This indicates that the server supports TLS, TCP, and SCTP in that order. If the client supports TLS, TLS will be used, targeted to a - host determined by an SRV lookup of _diameter._tls.example1.com. + + + +Fajardo, et al. Standards Track [Page 148] + +RFC 6733 Diameter Base Protocol October 2012 + + + host determined by an SRV lookup of _diameter._tls.ex1.example.com. That lookup would return: ;; Priority Weight Port Target - IN SRV 0 1 5060 server1.example1.com - IN SRV 0 2 5060 server2.example1.com + IN SRV 0 1 5060 server1.ex1.example.com + IN SRV 0 2 5060 server2.ex1.example.com As an alternative example, a client that wishes to resolve aaa: - example2.com. The client performs a NAPTR query for that domain, and - the following NAPTR records are returned: + ex2.example.com. The client performs a NAPTR query for that domain, + and the following NAPTR records are returned: ;; order pref flags service regexp replacement IN NAPTR 150 50 "a" "aaa:diameter.tls.tcp" "" - server1.example2.com + server1.ex2.example.com IN NAPTR 150 50 "a" "aaa:diameter.tls.tcp" "" - server2.example2.com + server2.ex2.example.com This indicates that the server supports TCP available at the returned host names. - - - - - - - - - - - - - - - -Fajardo, et al. Expires July 24, 2011 [Page 151] - -Internet-Draft Diameter Base Protocol January 2011 - - Appendix C. Duplicate Detection As described in Section 9.4, accounting record duplicate detection is @@ -8464,14 +8319,14 @@ Appendix C. Duplicate Detection reasons: o Failover to an alternate server. Where close to real-time - performance is required, failover thresholds need to be kept low - and this may lead to an increased likelihood of duplicates. - Failover can occur at the client or within Diameter agents. - - o Failure of a client or agent after sending of a record from non- - volatile memory, but prior to receipt of an application layer ACK - and deletion of the record. record to be sent. This will result - in retransmission of the record soon after the client or agent has + performance is required, failover thresholds need to be kept low. + This may lead to an increased likelihood of duplicates. Failover + can occur at the client or within Diameter agents. + + o Failure of a client or agent after sending a record from non- + volatile memory, but prior to receipt of an application-layer ACK + and deletion of the record to be sent. This will result in + retransmission of the record soon after the client or agent has rebooted. o Duplicates received from RADIUS gateways. Since the @@ -8481,48 +8336,48 @@ Appendix C. Duplicate Detection o Implementation problems and misconfiguration. - The T flag is used as an indication of an application layer + The T flag is used as an indication of an application-layer retransmission event, e.g., due to failover to an alternate server. It is defined only for request messages sent by Diameter clients or agents. For instance, after a reboot, a client may not know whether + + + +Fajardo, et al. Standards Track [Page 149] + +RFC 6733 Diameter Base Protocol October 2012 + + it has already tried to send the accounting records in its non- volatile memory before the reboot occurred. Diameter servers MAY use the T flag as an aid when processing requests and detecting duplicate messages. However, servers that do this MUST ensure that duplicates are found even when the first transmitted request arrives at the server after the retransmitted request. It can be used only in cases - where no answer has been received from the Server for a request and + where no answer has been received from the server for a request and the request is sent again, (e.g., due to a failover to an alternate peer, due to a recovered primary peer or due to a client re-sending a stored record from non-volatile memory such as after reboot of a client or agent). - In some cases the Diameter accounting server can delay the duplicate + In some cases, the Diameter accounting server can delay the duplicate detection and accounting record processing until a post-processing phase takes place. At that time records are likely to be sorted according to the included User-Name and duplicate elimination is easy - in this case. In other situations it may be necessary to perform + in this case. In other situations, it may be necessary to perform real-time duplicate detection, such as when credit limits are imposed or real-time fraud detection is desired. - - - -Fajardo, et al. Expires July 24, 2011 [Page 152] - -Internet-Draft Diameter Base Protocol January 2011 - - In general, only generation of duplicates due to failover or re- sending of records in non-volatile storage can be reliably detected - by Diameter clients or agents. In such cases the Diameter client or - agents can mark the message as possible duplicate by setting the T + by Diameter clients or agents. In such cases, the Diameter client or + agents can mark the message as a possible duplicate by setting the T flag. Since the Diameter server is responsible for duplicate - detection, it can choose to make use of the T flag or not, in order - to optimize duplicate detection. Since the T flag does not affect - interoperability, and may not be needed by some servers, generation - of the T flag is REQUIRED for Diameter clients and agents, but MAY be - implemented by Diameter servers. + detection, it can choose whether or not to make use of the T flag, in + order to optimize duplicate detection. Since the T flag does not + affect interoperability, and it may not be needed by some servers, + generation of the T flag is REQUIRED for Diameter clients and agents, + but it MAY be implemented by Diameter servers. As an example, it can be usually be assumed that duplicates appear within a time window of longest recorded network partition or device @@ -8535,40 +8390,39 @@ Internet-Draft Diameter Base Protocol January 2011 The following is an example of how the T flag may be used by the server to detect duplicate requests. - A Diameter server MAY check the T flag of the received message to determine if the record is a possible duplicate. If the T flag is set in the request message, the server searches for a duplicate within a configurable duplication time window backward and forward. This limits database searching to those records where - the T flag is set. In a well run network, network partitions and + the T flag is set. In a well-run network, network partitions and + + + +Fajardo, et al. Standards Track [Page 150] + +RFC 6733 Diameter Base Protocol October 2012 + + device faults will presumably be rare events, so this approach represents a substantial optimization of the duplicate detection process. During failover, it is possible for the original record - to be received after the T flag marked record, due to differences + to be received after the T-flag-marked record, due to differences in network delays experienced along the path by the original and duplicate transmissions. The likelihood of this occurring increases as the failover interval is decreased. In order to be - able to detect out of order duplicates, the Diameter server should - use backward and forward time windows when performing duplicate - checking for the T flag marked request. For example, in order to - allow time for the original record to exit the network and be - recorded by the accounting server, the Diameter server can delay - processing records with the T flag set until a time period - TIME_WAIT + RECORD_PROCESSING_TIME has elapsed after the closing - of the original transport connection. After this time period has - expired, then it may check the T flag marked records against the + able to detect duplicates that are out of order, the Diameter + server should use backward and forward time windows when + performing duplicate checking for the T-flag-marked request. For + example, in order to allow time for the original record to exit + the network and be recorded by the accounting server, the Diameter + server can delay processing records with the T flag set until a + time period TIME_WAIT + RECORD_PROCESSING_TIME has elapsed after + the closing of the original transport connection. After this time + period, it may check the T-flag-marked records against the database with relative assurance that the original records, if sent, have been received and recorded. - - - -Fajardo, et al. Expires July 24, 2011 [Page 153] - -Internet-Draft Diameter Base Protocol January 2011 - - Appendix D. Internationalized Domain Names To be compatible with the existing DNS infrastructure and simplify @@ -8579,14 +8433,7 @@ Appendix D. Internationalized Domain Names recommendations in [RFC4690] and [RFC5890]. Applications that provide support for IDNs outside of the Diameter protocol but interacting with it SHOULD use the representation and conversion - framework described in [RFC5890], [RFC5891] and [RFC3492]. - - - - - - - + framework described in [RFC5890], [RFC5891], and [RFC3492]. @@ -8608,21 +8455,9 @@ Appendix D. Internationalized Domain Names - - - - - - - - - - - - -Fajardo, et al. Expires July 24, 2011 [Page 154] +Fajardo, et al. Standards Track [Page 151] -Internet-Draft Diameter Base Protocol January 2011 +RFC 6733 Diameter Base Protocol October 2012 Authors' Addresses @@ -8634,7 +8469,7 @@ Authors' Addresses USA Phone: +1-908-421-1845 - Email: [email protected] + EMail: [email protected] Jari Arkko @@ -8643,7 +8478,7 @@ Authors' Addresses Finland Phone: +358 40 5079256 - Email: [email protected] + EMail: [email protected] John Loughney @@ -8653,17 +8488,17 @@ Authors' Addresses US Phone: +1-650-283-8068 - Email: [email protected] + EMail: [email protected] - Glenn Zorn + Glen Zorn (editor) Network Zen - 1310 East Thomas Street - Seattle, WA 98102 - US + 227/358 Thanon Sanphawut + Bang Na, Bangkok 10260 + Thailand - Phone: - Email: [email protected] + Phone: +66 (0) 87-0404617 + EMail: [email protected] @@ -8676,6 +8511,5 @@ Authors' Addresses -Fajardo, et al. Expires July 24, 2011 [Page 155] +Fajardo, et al. Standards Track [Page 152] - diff --git a/lib/diameter/doc/standard/rfc6737.txt b/lib/diameter/doc/standard/rfc6737.txt new file mode 100644 index 0000000000..50aa33e98f --- /dev/null +++ b/lib/diameter/doc/standard/rfc6737.txt @@ -0,0 +1,339 @@ + + + + + + +Internet Engineering Task Force (IETF) K. Jiao +Request for Comments: 6737 Huawei +Category: Standards Track G. Zorn +ISSN: 2070-1721 Network Zen + October 2012 + + + The Diameter Capabilities Update Application + +Abstract + + This document defines a new Diameter application and associated + Command Codes. The Capabilities Update application is intended to + allow the dynamic update of certain Diameter peer capabilities while + the peer-to-peer connection is in the open state. + +Status of This Memo + + This is an Internet Standards Track document. + + This document is a product of the Internet Engineering Task Force + (IETF). It represents the consensus of the IETF community. It has + received public review and has been approved for publication by the + Internet Engineering Steering Group (IESG). Further information on + Internet Standards is available in Section 2 of RFC 5741. + + Information about the current status of this document, any errata, + and how to provide feedback on it may be obtained at + http://www.rfc-editor.org/info/rfc6737. + +Copyright Notice + + Copyright (c) 2012 IETF Trust and the persons identified as the + document authors. All rights reserved. + + This document is subject to BCP 78 and the IETF Trust's Legal + Provisions Relating to IETF Documents + (http://trustee.ietf.org/license-info) in effect on the date of + publication of this document. Please review these documents + carefully, as they describe your rights and restrictions with respect + to this document. Code Components extracted from this document must + include Simplified BSD License text as described in Section 4.e of + the Trust Legal Provisions and are provided without warranty as + described in the Simplified BSD License. + + + + + + + +Jiao & Zorn Standards Track [Page 1] + +RFC 6737 Diameter Capabilities Update October 2012 + + +Table of Contents + + 1. Introduction . . . . . . . . . . . . . . . . . . . . . . . . . 2 + 2. Specification of Requirements . . . . . . . . . . . . . . . . . 2 + 3. Diameter Protocol Considerations . . . . . . . . . . . . . . . 3 + 4. Capabilities Update . . . . . . . . . . . . . . . . . . . . . . 3 + 4.1. Command Code Values . . . . . . . . . . . . . . . . . . . . 4 + 4.1.1. Capabilities-Update-Request . . . . . . . . . . . . . . 4 + 4.1.2. Capabilities-Update-Answer . . . . . . . . . . . . . . 5 + 5. Security Considerations . . . . . . . . . . . . . . . . . . . . 5 + 6. IANA Considerations . . . . . . . . . . . . . . . . . . . . . . 5 + 6.1. Application Identifier . . . . . . . . . . . . . . . . . . 5 + 6.2. Command Codes . . . . . . . . . . . . . . . . . . . . . . . 5 + 7. Contributors . . . . . . . . . . . . . . . . . . . . . . . . . 5 + 8. Acknowledgements . . . . . . . . . . . . . . . . . . . . . . . 5 + 9. References . . . . . . . . . . . . . . . . . . . . . . . . . . 6 + 9.1. Normative References . . . . . . . . . . . . . . . . . . . 6 + 9.2. Informative References . . . . . . . . . . . . . . . . . . 6 + +1. Introduction + + Capabilities exchange is an important component of the Diameter base + protocol [RFC6733], allowing peers to exchange identities and + Diameter capabilities (protocol version number, supported Diameter + applications, security mechanisms, etc.). As defined in RFC 3588, + however, the capabilities exchange process takes place only once, at + the inception of a transport connection between a given pair of + peers. Therefore, if a peer's capabilities change (due to a software + update, for example), the existing connection(s) must be torn down + (along with all of the associated user sessions) and restarted before + the modified capabilities can be advertised. + + This document defines a new Diameter application intended to allow + the dynamic update of a subset of Diameter peer capabilities over an + existing connection. Because the Capabilities Update application + specified herein operates over an existing transport connection, + modification of certain capabilities is prohibited. Specifically, + modifying the security mechanism in use is not allowed; if the + security method used between a pair of peers is changed, the affected + connection MUST be restarted. + +2. Specification of Requirements + + The key words "MUST", "MUST NOT", "REQUIRED", "SHALL", "SHALL NOT", + "SHOULD", "SHOULD NOT", "RECOMMENDED", "MAY", and "OPTIONAL" in this + document are to be interpreted as described in RFC 2119 [RFC2119]. + + + + + +Jiao & Zorn Standards Track [Page 2] + +RFC 6737 Diameter Capabilities Update October 2012 + + +3. Diameter Protocol Considerations + + This section details the relationship of the Diameter Capabilities + Update application to the Diameter base protocol. + + This document specifies Diameter Application-Id 10. Diameter nodes + conforming to this specification MUST advertise support by including + the value 10 in the Auth-Application-Id of the Capabilities-Exchange- + Request (CER) and Capabilities-Exchange-Answer (CEA) commands + [RFC6733]. + +4. Capabilities Update + + When the capabilities of a Diameter node conforming to this + specification change, the node MUST notify all of the nodes with + which it has an open transport connection and which have also + advertised support for the Capabilities Update application using the + Capabilities-Update-Request (CUR) message (Section 4.1.1). This + message allows the update of a peer's capabilities (supported + Diameter applications, etc.). + + A Diameter node only issues a given command to those peers that have + advertised support for the Diameter application that defines the + command; a Diameter node must cache the supported applications in + order to ensure that unrecognized commands and/or Attribute-Value + Pairs (AVPs) are not unnecessarily sent to a peer. + + The receiver of the CUR MUST determine common applications by + computing the intersection of its own set of supported Application + Ids against all of the Application-Id AVPs (Auth-Application-Id, + Acct-Application-Id, and Vendor-Specific-Application-Id) present in + the CUR. The value of the Vendor-Id AVP in the Vendor-Specific- + Application-Id MUST NOT be used during computation. + + If the receiver of a CUR does not have any applications in common + with the sender, then it MUST return a Capabilities-Update-Answer + (CUA) (Section 4.1.2) with the Result-Code AVP set to + DIAMETER_NO_COMMON_APPLICATION [RFC6733], and it SHOULD disconnect + the transport-layer connection. However, if active sessions are + using the connection, peers MAY delay disconnection until the + sessions can be redirected or gracefully terminated. Note that + receiving a CUA from a peer advertising itself as a relay (see + [RFC6733], Section 2.4) MUST be interpreted as having common + applications with the peer. + + As for CER/CEA messages, the CUR and CUA messages MUST NOT be + proxied, redirected, or relayed. + + + + +Jiao & Zorn Standards Track [Page 3] + +RFC 6737 Diameter Capabilities Update October 2012 + + + Even though the CUR/CUA messages cannot be proxied, it is still + possible for an upstream agent to receive a message for which there + are no peers available to handle the application that corresponds to + the Command Code. This could happen if, for example, the peers are + too busy or down. In such instances, the 'E' bit MUST be set in the + answer message with the Result-Code AVP set to + DIAMETER_UNABLE_TO_DELIVER to inform the downstream peer to take + action (e.g., re-routing requests to an alternate peer). + +4.1. Command Code Values + + This section defines Command Code [RFC6733] values that MUST be + supported by all Diameter implementations conforming to this + specification. The following Command Codes are defined in this + document: Capabilities-Update-Request (CUR, Section 4.1.1), and + Capabilities-Update-Answer (CUA, Section 4.1.2). The Diameter + Command Code Format (CCF) ([RFC6733], Section 3.2) is used in the + definitions. + +4.1.1. Capabilities-Update-Request + + The Capabilities-Update-Request (CUR), indicated by the Command Code + set to 328 and the Command Flags' 'R' bit set, is sent to update + local capabilities. Upon detection of a transport failure, this + message MUST NOT be sent to an alternate peer. + + When Diameter is run over the Stream Control Transmission Protocol + (SCTP) [RFC4960], which allows connections to span multiple + interfaces and multiple IP addresses, the Capabilities-Update-Request + message MUST contain one Host-IP-Address AVP for each potential IP + address that may be locally used when transmitting Diameter messages. + + Message Format + + <CUR> ::= < Diameter Header: 328, REQ > + { Origin-Host } + { Origin-Realm } + 1* { Host-IP-Address } + { Vendor-Id } + { Product-Name } + [ Origin-State-Id ] + * [ Supported-Vendor-Id ] + * [ Auth-Application-Id ] + * [ Acct-Application-Id ] + * [ Vendor-Specific-Application-Id ] + [ Firmware-Revision ] + * [ AVP ] + + + + +Jiao & Zorn Standards Track [Page 4] + +RFC 6737 Diameter Capabilities Update October 2012 + + +4.1.2. Capabilities-Update-Answer + + The Capabilities-Update-Answer, indicated by the Command Code set to + 328 and the Command Flags' 'R' bit cleared, is sent in response to a + CUR message. + + Message Format + + <CUA> ::= < Diameter Header: 328 > + { Origin-Host } + { Origin-Realm } + { Result-Code } + [ Error-Message ] + * [ AVP ] + +5. Security Considerations + + The security considerations applicable to the Diameter base protocol + [RFC6733] are also applicable to this document. + +6. IANA Considerations + + This section explains the criteria to be used by the IANA for + assignment of numbers within namespaces used within this document. + +6.1. Application Identifier + + This specification assigns the value 10 (Diameter Capabilities + Update) from the Application Identifiers namespace [RFC6733]. See + Section 3 for the assignment of the namespace in this specification. + +6.2. Command Codes + + This specification assigns the value 328 (Capabilities-Update- + Request/Capabilities-Update-Answer (CUR/CUA)) from the Command Codes + namespace [RFC6733]. See Section 4.1 for the assignment of the + namespace in this specification. + +7. Contributors + + This document is based upon work done by Tina Tsou. + +8. Acknowledgements + + Thanks to Sebastien Decugis, Niklas Neumann, Subash Comerica, Lionel + Morand, Dan Romascanu, Dan Harkins, and Ravi for helpful review and + discussion. + + + + +Jiao & Zorn Standards Track [Page 5] + +RFC 6737 Diameter Capabilities Update October 2012 + + +9. References + +9.1. Normative References + + [RFC2119] Bradner, S., "Key words for use in RFCs to Indicate + Requirement Levels", BCP 14, RFC 2119, March 1997. + + [RFC6733] Fajardo, V., Arkko, J., Loughney, J., and G. Zorn, + "Diameter Base Protocol", RFC 6733, October 2012. + +9.2. Informative References + + [RFC4960] Stewart, R., "Stream Control Transmission Protocol", + RFC 4960, September 2007. + +Authors' Addresses + + Jiao Kang + Huawei Technologies + Section F1, Huawei Industrial Base + Bantian, Longgang District + Shenzhen 518129 + P.R. China + + EMail: [email protected] + + + Glen Zorn + Network Zen + 227/358 Thanon Sanphawut + Bang Na, Bangkok 10260 + Thailand + + Phone: +66 (0) 909-201060 + EMail: [email protected] + + + + + + + + + + + + + + + + +Jiao & Zorn Standards Track [Page 6] + diff --git a/lib/diameter/src/base/diameter_capx.erl b/lib/diameter/src/base/diameter_capx.erl index 190d37262b..c6c3d2934d 100644 --- a/lib/diameter/src/base/diameter_capx.erl +++ b/lib/diameter/src/base/diameter_capx.erl @@ -1,7 +1,7 @@ %% %% %CopyrightBegin% %% -%% Copyright Ericsson AB 2010-2011. All Rights Reserved. +%% Copyright Ericsson AB 2010-2012. All Rights Reserved. %% %% The contents of this file are subject to the Erlang Public License, %% Version 1.1, (the "License"); you may not use this file except in diff --git a/lib/diameter/src/base/diameter_service.erl b/lib/diameter/src/base/diameter_service.erl index a4a0b80348..91384b8b91 100644 --- a/lib/diameter/src/base/diameter_service.erl +++ b/lib/diameter/src/base/diameter_service.erl @@ -1703,16 +1703,20 @@ send(Pid, Pkt) -> %% retransmit/4 retransmit({TPid, Caps, #diameter_app{alias = Alias} = App} = T, - #request{app = Alias, packet = Pkt} + #request{app = Alias, packet = Pkt0} = Req, SvcName, Timeout) -> - have_request(Pkt, TPid) %% Don't failover to a peer we've + have_request(Pkt0, TPid) %% Don't failover to a peer we've andalso ?THROW(timeout), %% already sent to. + #diameter_packet{header = Hdr0} = Pkt0, + Hdr = Hdr0#diameter_header{is_retransmitted = true}, + Pkt = Pkt0#diameter_packet{header = Hdr}, + resend_req(cb(App, prepare_retransmit, [Pkt, SvcName, {TPid, Caps}]), T, - Req, + Req#request{packet = Pkt}, Timeout, []). diff --git a/lib/diameter/test/diameter_failover_SUITE.erl b/lib/diameter/test/diameter_failover_SUITE.erl index ed31670031..bb820a8bf2 100644 --- a/lib/diameter/test/diameter_failover_SUITE.erl +++ b/lib/diameter/test/diameter_failover_SUITE.erl @@ -18,19 +18,19 @@ %% %% -%% Tests of traffic between six Diameter nodes in three realms, +%% Tests of traffic between seven Diameter nodes in four realms, %% connected as follows. %% -%% ----- SERVER1.REALM2 -%% / -%% / ----- SERVER2.REALM2 -%% | / -%% CLIENT.REALM1 ------ SERVER3.REALM2 -%% | \ -%% | \ -%% \ ---- SERVER1.REALM3 -%% \ -%% ----- SERVER2.REALM3 +%% ----- SERVER1.REALM2 ----- +%% / \ +%% / ----- SERVER2.REALM2 ----- \ +%% | / \ | +%% CLIENT.REALM1 ------ SERVER3.REALM2 ------ CLIENT.REALM4 +%% | \ / | +%% | \ / | +%% \ ---- SERVER1.REALM3 ----- / +%% \ / +%% ----- SERVER2.REALM3 ----- %% -module(diameter_failover_SUITE). @@ -44,12 +44,16 @@ connect/1, send_ok/1, send_nok/1, + send_discard_1/1, + send_discard_2/1, stop_services/1, stop/1]). %% diameter callbacks -export([pick_peer/4, prepare_request/3, + prepare_retransmit/3, + handle_error/4, handle_answer/4, handle_request/3]). @@ -62,14 +66,18 @@ -define(ADDR, {127,0,0,1}). --define(CLIENT, "CLIENT.REALM1"). +-define(CLIENT1, "CLIENT.REALM1"). +-define(CLIENT2, "CLIENT.REALM4"). -define(SERVER1, "SERVER1.REALM2"). -define(SERVER2, "SERVER2.REALM2"). -define(SERVER3, "SERVER3.REALM2"). -define(SERVER4, "SERVER1.REALM3"). -define(SERVER5, "SERVER2.REALM3"). --define(SERVICES, [?CLIENT, ?SERVER1, ?SERVER2, ?SERVER3, ?SERVER4, ?SERVER5]). +-define(IS_CLIENT(Svc), Svc == ?CLIENT1; Svc == ?CLIENT2). + +-define(CLIENTS, [?CLIENT1, ?CLIENT2]). +-define(SERVERS, [?SERVER1, ?SERVER2, ?SERVER3, ?SERVER4, ?SERVER5]). -define(DICT_COMMON, ?DIAMETER_DICT_COMMON). @@ -77,26 +85,27 @@ -define(APP_ID, ?DICT_COMMON:id()). %% Config for diameter:start_service/2. --define(SERVICE(Host, Dict), +-define(SERVICE(Host), [{'Origin-Host', Host}, {'Origin-Realm', realm(Host)}, {'Host-IP-Address', [?ADDR]}, {'Vendor-Id', 12345}, {'Product-Name', "OTP/diameter"}, - {'Acct-Application-Id', [Dict:id()]}, + {'Acct-Application-Id', [?APP_ID]}, {application, [{alias, ?APP_ALIAS}, - {dictionary, Dict}, + {dictionary, ?DICT_COMMON}, {module, #diameter_callback {peer_up = false, peer_down = false, - handle_error = false, - prepare_retransmit = false, default = ?MODULE}}, {answer_errors, callback}]}]). -define(SUCCESS, 2001). --define(LOGOUT, ?'DIAMETER_BASE_TERMINATION-CAUSE_DIAMETER_LOGOUT'). +%% Value of Termination-Cause determines client/server behaviour. +-define(LOGOUT, ?'DIAMETER_BASE_TERMINATION-CAUSE_DIAMETER_LOGOUT'). +-define(MOVED, ?'DIAMETER_BASE_TERMINATION-CAUSE_DIAMETER_USER_MOVED'). +-define(TIMEOUT, ?'DIAMETER_BASE_TERMINATION-CAUSE_DIAMETER_SESSION_TIMEOUT'). %% =========================================================================== @@ -109,6 +118,8 @@ all() -> connect, send_ok, send_nok, + send_discard_1, + send_discard_2, stop_services, stop]. @@ -119,19 +130,20 @@ start(_Config) -> ok = diameter:start(). start_services(_Config) -> - S = [server(N, ?DICT_COMMON) || N <- tl(?SERVICES)], - - ok = diameter:start_service(?CLIENT, ?SERVICE(?CLIENT, ?DICT_COMMON)), + Servers = [server(N) || N <- ?SERVERS], + [] = [T || C <- ?CLIENTS, + T <- [diameter:start_service(C, ?SERVICE(C))], + T /= ok], - {save_config, [{?CLIENT, S}]}. + {save_config, Servers}. connect(Config) -> - {_, Conns} = proplists:get_value(saved_config, Config), + {start_services, Servers} = proplists:get_value(saved_config, Config), - lists:foreach(fun({CN,Ss}) -> connect(CN, Ss) end, Conns). + lists:foreach(fun(C) -> connect(C, Servers) end, ?CLIENTS). stop_services(_Config) -> - [] = [{H,T} || H <- ?SERVICES, + [] = [{H,T} || H <- ?CLIENTS ++ ?SERVERS, T <- [diameter:stop_service(H)], T /= ok]. @@ -140,8 +152,8 @@ stop(_Config) -> %% ---------------------------------------- -server(Name, Dict) -> - ok = diameter:start_service(Name, ?SERVICE(Name, Dict)), +server(Name) -> + ok = diameter:start_service(Name, ?SERVICE(Name)), {Name, ?util:listen(Name, tcp)}. connect(Name, Refs) -> @@ -153,30 +165,39 @@ connect(Name, Refs) -> %% Send an STR and expect success after SERVER3 answers after a couple %% of failovers. send_ok(_Config) -> - Req = ['STR', {'Destination-Realm', realm(?SERVER1)}, - {'Termination-Cause', ?LOGOUT}, - {'Auth-Application-Id', ?APP_ID}], + Req = #diameter_base_STR{'Destination-Realm' = realm(?SERVER1), + 'Termination-Cause' = ?LOGOUT, + 'Auth-Application-Id' = ?APP_ID}, #diameter_base_STA{'Result-Code' = ?SUCCESS, 'Origin-Host' = ?SERVER3} - = call(Req, [{filter, realm}]). + = call(?CLIENT1, Req). %% Send an STR and expect failure when both servers fail. send_nok(_Config) -> - Req = ['STR', {'Destination-Realm', realm(?SERVER4)}, - {'Termination-Cause', ?LOGOUT}, - {'Auth-Application-Id', ?APP_ID}], - {error, failover} = call(Req, [{filter, realm}]). + Req = #diameter_base_STR{'Destination-Realm' = realm(?SERVER4), + 'Termination-Cause' = ?LOGOUT, + 'Auth-Application-Id' = ?APP_ID}, + {failover, ?LOGOUT} = call(?CLIENT1, Req). + +%% Send an STR and have prepare_retransmit discard it. +send_discard_1(_Config) -> + Req = #diameter_base_STR{'Destination-Realm' = realm(?SERVER1), + 'Termination-Cause' = ?TIMEOUT, + 'Auth-Application-Id' = ?APP_ID}, + {rejected, ?TIMEOUT} = call(?CLIENT2, Req). +send_discard_2(_Config) -> + Req = #diameter_base_STR{'Destination-Realm' = realm(?SERVER4), + 'Termination-Cause' = ?MOVED, + 'Auth-Application-Id' = ?APP_ID}, + {discarded, ?MOVED} = call(?CLIENT2, Req). %% =========================================================================== realm(Host) -> tl(lists:dropwhile(fun(C) -> C /= $. end, Host)). -call(Req, Opts) -> - diameter:call(?CLIENT, ?APP_ALIAS, Req, Opts). - -set([H|T], Vs) -> - [H | Vs ++ T]. +call(Svc, Req) -> + diameter:call(Svc, ?APP_ALIAS, Req, [{filter, realm}]). %% =========================================================================== %% diameter callbacks @@ -184,7 +205,8 @@ set([H|T], Vs) -> %% pick_peer/4 %% Choose a server other than SERVER3 or SERVER5 if possible. -pick_peer(Peers, _, ?CLIENT, _State) -> +pick_peer(Peers, _, Svc, _State) + when ?IS_CLIENT(Svc) -> case lists:partition(fun({_, #diameter_caps{origin_host = {_, OH}}}) -> OH /= ?SERVER3 andalso OH /= ?SERVER5 end, @@ -198,20 +220,41 @@ pick_peer(Peers, _, ?CLIENT, _State) -> %% prepare_request/3 -prepare_request(Pkt, ?CLIENT, {_Ref, Caps}) -> +prepare_request(Pkt, Svc, {_Ref, Caps}) + when ?IS_CLIENT(Svc) -> {send, prepare(Pkt, Caps)}. prepare(#diameter_packet{msg = Req}, Caps) -> #diameter_caps{origin_host = {OH, _}, origin_realm = {OR, _}} = Caps, - set(Req, [{'Session-Id', diameter:session_id(OH)}, - {'Origin-Host', OH}, - {'Origin-Realm', OR}]). + Req#diameter_base_STR{'Origin-Host' = OH, + 'Origin-Realm' = OR, + 'Session-Id' = diameter:session_id(OH)}. + +%% prepare_retransmit/3 + +prepare_retransmit(#diameter_packet{header = H} = P, Svc, {_,_}) + when ?IS_CLIENT(Svc) -> + #diameter_header{is_retransmitted = true} = H, %% assert + prepare(P). + +prepare(#diameter_packet{msg = M} = P) -> + case M#diameter_base_STR.'Termination-Cause' of + ?LOGOUT -> {send, P}; + ?MOVED -> discard; + ?TIMEOUT -> {discard, rejected} + end. + +%% handle_error/4 + +handle_error(Reason, Req, _, _) -> + {Reason, Req#diameter_base_STR.'Termination-Cause'}. %% handle_answer/4 -handle_answer(Pkt, _Req, ?CLIENT, _Peer) -> +handle_answer(Pkt, _Req, Svc, _Peer) + when ?IS_CLIENT(Svc) -> #diameter_packet{msg = Rec, errors = []} = Pkt, Rec. @@ -219,8 +262,8 @@ handle_answer(Pkt, _Req, ?CLIENT, _Peer) -> %% Only SERVER3 actually answers. handle_request(Pkt, ?SERVER3, {_, Caps}) -> - #diameter_packet{msg = #diameter_base_STR{'Session-Id' = SId, - 'Origin-Host' = ?CLIENT}} + #diameter_packet{header = #diameter_header{is_retransmitted = true}, + msg = #diameter_base_STR{'Session-Id' = SId}} = Pkt, #diameter_caps{origin_host = {OH, _}, origin_realm = {OR, _}} diff --git a/lib/edoc/doc/src/notes.xml b/lib/edoc/doc/src/notes.xml index e01a6d6675..472b3bfc34 100644 --- a/lib/edoc/doc/src/notes.xml +++ b/lib/edoc/doc/src/notes.xml @@ -4,7 +4,7 @@ <chapter> <header> <copyright> - <year>2007</year><year>2011</year> + <year>2007</year><year>2012</year> <holder>Ericsson AB. All Rights Reserved.</holder> </copyright> <legalnotice> diff --git a/lib/edoc/priv/edoc.dtd b/lib/edoc/priv/edoc.dtd index 6a332cf22f..ba4ac0db28 100644 --- a/lib/edoc/priv/edoc.dtd +++ b/lib/edoc/priv/edoc.dtd @@ -4,7 +4,8 @@ <!ELEMENT overview (title, description?, author*, copyright?, version?, since?, see*, reference*, todo?, packages, modules)> <!ATTLIST overview - root CDATA #IMPLIED> + root CDATA #IMPLIED + encoding CDATA #IMPLIED> <!ELEMENT title (#PCDATA)> @@ -25,6 +26,7 @@ name CDATA #REQUIRED private (yes | no) #IMPLIED hidden (yes | no) #IMPLIED + encoding CDATA #IMPLIED root CDATA #IMPLIED> <!ELEMENT description (briefDescription, fullDescription?)> diff --git a/lib/edoc/src/edoc.erl b/lib/edoc/src/edoc.erl index 544465b14a..599036f380 100644 --- a/lib/edoc/src/edoc.erl +++ b/lib/edoc/src/edoc.erl @@ -120,7 +120,8 @@ file(Name, Options) -> Suffix = proplists:get_value(file_suffix, Options, ?DEFAULT_FILE_SUFFIX), Dir = proplists:get_value(dir, Options, filename:dirname(Name)), - edoc_lib:write_file(Text, Dir, BaseName ++ Suffix). + Encoding = [{encoding, edoc_lib:read_encoding(Name, [])}], + edoc_lib:write_file(Text, Dir, BaseName ++ Suffix, '', Encoding). %% TODO: better documentation of files/1/2, packages/1/2, application/1/2/3 @@ -455,14 +456,14 @@ expand_sources(Ss, Opts) -> end, expand_sources(Ss1, Suffix, sets:new(), [], []). -expand_sources([{P, F, D} | Fs], Suffix, S, As, Ms) -> - M = list_to_atom(packages:concat(P, filename:rootname(F, Suffix))), +expand_sources([{'', F, D} | Fs], Suffix, S, As, Ms) -> + M = list_to_atom(filename:rootname(F, Suffix)), case sets:is_element(M, S) of true -> expand_sources(Fs, Suffix, S, As, Ms); false -> S1 = sets:add_element(M, S), - expand_sources(Fs, Suffix, S1, [{M, P, F, D} | As], + expand_sources(Fs, Suffix, S1, [{M, '', F, D} | As], [M | Ms]) end; expand_sources([], _Suffix, _S, As, Ms) -> diff --git a/lib/edoc/src/edoc.hrl b/lib/edoc/src/edoc.hrl index 98debba4ab..44c5d6fef4 100644 --- a/lib/edoc/src/edoc.hrl +++ b/lib/edoc/src/edoc.hrl @@ -48,7 +48,8 @@ %% functions = ordset(function_name()), %% exports = ordset(function_name()), %% attributes = ordset({atom(), term()}), -%% records = [{atom(), [{atom(), term()}]}]} +%% records = [{atom(), [{atom(), term()}]}], +%% encoding = epp:source_encoding()} %% ordset(T) = sets:ordset(T) %% function_name(T) = {atom(), integer()} @@ -57,7 +58,8 @@ functions = [], exports = [], attributes = [], - records = [] + records = [], + encoding = latin1 }). %% Environment for generating documentation data diff --git a/lib/edoc/src/edoc_data.erl b/lib/edoc/src/edoc_data.erl index 624f9177a2..f88ba05f4b 100644 --- a/lib/edoc/src/edoc_data.erl +++ b/lib/edoc/src/edoc_data.erl @@ -83,7 +83,8 @@ module(Module, Entries, Env, Opts) -> AllTags = get_all_tags(Entries), Functions = function_filter(Entries, Opts), Out = {module, ([{name, Name}, - {root, Env#env.root}] + {root, Env#env.root}, + {encoding, Module#module.encoding}] ++ case is_private(HeaderTags) of true -> [{private, "yes"}]; false -> [] diff --git a/lib/edoc/src/edoc_doclet.erl b/lib/edoc/src/edoc_doclet.erl index 385d20e9ae..a0c1ae1c0f 100644 --- a/lib/edoc/src/edoc_doclet.erl +++ b/lib/edoc/src/edoc_doclet.erl @@ -192,8 +192,9 @@ source({M, P, Name, Path}, Dir, Suffix, Env, Set, Private, Hidden, andalso ((not is_hidden(Doc)) orelse Hidden) of true -> Text = edoc:layout(Doc, Options), - Name1 = packages:last(M) ++ Suffix, - edoc_lib:write_file(Text, Dir, Name1, P), + Name1 = atom_to_list(M) ++ Suffix, + Encoding = [{encoding,encoding(Doc)}], + edoc_lib:write_file(Text, Dir, Name1, P, Encoding), {sets:add_element(Module, Set), Error}; false -> {Set, Error} @@ -204,9 +205,9 @@ source({M, P, Name, Path}, Dir, Suffix, Env, Set, Private, Hidden, end. check_name(M, M0, P0, File) -> - P = list_to_atom(packages:strip_last(M)), - N = packages:last(M), - N0 = packages:last(M0), + P = '', + N = M, + N0 = M0, case N of [$? | _] -> %% A module name of the form '?...' is assumed to be caused @@ -359,14 +360,19 @@ xhtml_1(Title, CSS, Body) -> overview(Dir, Title, Env, Opts) -> File = proplists:get_value(overview, Opts, filename:join(Dir, ?OVERVIEW_FILE)), + Encoding = edoc_lib:read_encoding(File, [{in_comment_only, false}]), Tags = read_file(File, overview, Env, Opts), - Data = edoc_data:overview(Title, Tags, Env, Opts), + Data0 = edoc_data:overview(Title, Tags, Env, Opts), + EncodingAttribute = #xmlAttribute{name = encoding, + value = atom_to_list(Encoding)}, + #xmlElement{attributes = As} = Data0, + Data = Data0#xmlElement{attributes = [EncodingAttribute | As]}, F = fun (M) -> M:overview(Data, Opts) end, Text = edoc_lib:run_layout(F, Opts), - edoc_lib:write_file(Text, Dir, ?OVERVIEW_SUMMARY). - + EncOpts = [{encoding,Encoding}], + edoc_lib:write_file(Text, Dir, ?OVERVIEW_SUMMARY, '', EncOpts). copy_image(Dir) -> case code:priv_dir(?EDOC_APP) of @@ -441,6 +447,12 @@ is_hidden(E) -> _ -> false end. +encoding(E) -> + case get_attrval(encoding, E) of + "latin1" -> latin1; + _ -> utf8 + end. + get_attrval(Name, #xmlElement{attributes = As}) -> case get_attr(Name, As) of [#xmlAttribute{value = V}] -> diff --git a/lib/edoc/src/edoc_extract.erl b/lib/edoc/src/edoc_extract.erl index 5a79e127f6..67a95e80aa 100644 --- a/lib/edoc/src/edoc_extract.erl +++ b/lib/edoc/src/edoc_extract.erl @@ -121,7 +121,7 @@ source1(Tree, File0, Env, Opts, TypeDocs) -> Module = get_module_info(Tree, File), {Header, Footer, Entries} = collect(Forms, Module), Name = Module#module.name, - Package = list_to_atom(packages:strip_last(Name)), + Package = '', Env1 = Env#env{module = Name, package = Package, root = edoc_refs:relative_package_path('', Package)}, @@ -226,7 +226,7 @@ add_macro_defs(Defs0, Opts, Env) -> %% lines of text before the first tag are ignored. `Env' is an %% environment created by {@link edoc_lib:get_doc_env/4}. Upon error, %% `Reason' is an atom returned from the call to {@link -%% //kernel/file:read_file/1}. +%% //kernel/file:read_file/1} or the atom 'invalid_unicode'. %% %% See {@link text/4} for options. @@ -235,7 +235,13 @@ add_macro_defs(Defs0, Opts, Env) -> file(File, Context, Env, Opts) -> case file:read_file(File) of {ok, Bin} -> - {ok, text(binary_to_list(Bin), Context, Env, Opts, File)}; + Enc = edoc_lib:read_encoding(File,[{in_comment_only, false}]), + case catch unicode:characters_to_list(Bin, Enc) of + String when is_list(String) -> + {ok, text(String, Context, Env, Opts, File)}; + _ -> + {error, invalid_unicode} + end; {error, _} = Error -> Error end. @@ -306,12 +312,14 @@ get_module_info(Forms, File) -> Exports = ordsets:from_list(get_list_keyval(exports, L)), Attributes = ordsets:from_list(get_list_keyval(attributes, L)), Records = get_list_keyval(records, L), + Encoding = edoc_lib:read_encoding(File, []), #module{name = Name, parameters = Vars, functions = Functions, exports = ordsets:intersection(Exports, Functions), attributes = Attributes, - records = Records}. + records = Records, + encoding = Encoding}. get_list_keyval(Key, L) -> case lists:keyfind(Key, 1, L) of diff --git a/lib/edoc/src/edoc_layout.erl b/lib/edoc/src/edoc_layout.erl index 951cec121c..7bd0615f5c 100644 --- a/lib/edoc/src/edoc_layout.erl +++ b/lib/edoc/src/edoc_layout.erl @@ -210,7 +210,8 @@ layout_module(#xmlElement{name = module, content = Es}=E, Opts) -> ++ [hr, ?NL] ++ navigation("bottom") ++ timestamp()), - xhtml(Title, stylesheet(Opts), Body). + Encoding = get_attrval(encoding, E), + xhtml(Title, stylesheet(Opts), Body, Encoding). module_params(Es) -> As = [{get_text(argName, Es1), @@ -956,10 +957,17 @@ local_label(R) -> "#" ++ R. xhtml(Title, CSS, Body) -> + xhtml(Title, CSS, Body, "latin1"). + +xhtml(Title, CSS, Body, Encoding) -> + EncString = case Encoding of + "latin1" -> "ISO-8859-1"; + _ -> "UTF-8" + end, [{html, [?NL, {head, [?NL, {meta, [{'http-equiv',"Content-Type"}, - {content, "text/html; charset=ISO-8859-1"}], + {content, "text/html; charset="++EncString}], []}, ?NL, {title, Title}, @@ -1021,7 +1029,8 @@ overview(E=#xmlElement{name = overview, content = Es}, Options) -> ++ [?NL, hr] ++ navigation("bottom") ++ timestamp()), - XML = xhtml(Title, stylesheet(Opts), Body), + Encoding = get_attrval(encoding, E), + XML = xhtml(Title, stylesheet(Opts), Body, Encoding), xmerl:export_simple(XML, ?HTML_EXPORT, []). %%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%% NYTT diff --git a/lib/edoc/src/edoc_lib.erl b/lib/edoc/src/edoc_lib.erl index 90fb8a679c..3d193c4bfa 100644 --- a/lib/edoc/src/edoc_lib.erl +++ b/lib/edoc/src/edoc_lib.erl @@ -1,3 +1,4 @@ +%% -*- coding: utf-8 -*- %% ===================================================================== %% This library is free software; you can redistribute it and/or modify %% it under the terms of the GNU Lesser General Public License as @@ -30,10 +31,10 @@ parse_contact/2, escape_uri/1, join_uri/2, is_relative_uri/1, is_name/1, to_label/1, find_doc_dirs/0, find_sources/2, find_sources/3, find_file/3, try_subdir/2, unique/1, - write_file/3, write_file/4, write_info_file/4, + write_file/3, write_file/4, write_file/5, write_info_file/4, read_info_file/1, get_doc_env/1, get_doc_env/4, copy_file/2, uri_get/1, run_doclet/2, run_layout/2, - simplify_path/1, timestr/1, datestr/1]). + simplify_path/1, timestr/1, datestr/1, read_encoding/2]). -import(edoc_report, [report/2, warning/2]). @@ -57,6 +58,13 @@ datestr({Y,M,D}) -> lists:flatten(io_lib:fwrite("~s ~w ~w",[lists:nth(M, Ms),D,Y])). %% @private +read_encoding(File, Options) -> + case epp:read_encoding(File, Options) of + none -> epp:default_encoding(); + Encoding -> Encoding + end. + +%% @private count(X, Xs) -> count(X, Xs, 0). @@ -228,13 +236,13 @@ end_of_sentence_1(_, false, _) -> %% 173 - 176 { - ~ punctuation %% 177 DEL control %% 200 - 237 control -%% 240 - 277 NBSP - � punctuation -%% 300 - 326 � - � uppercase -%% 327 � punctuation -%% 330 - 336 � - � uppercase -%% 337 - 366 � - � lowercase -%% 367 � punctuation -%% 370 - 377 � - � lowercase +%% 240 - 277 NBSP - ¿ punctuation +%% 300 - 326 À - Ö uppercase +%% 327 × punctuation +%% 330 - 336 Ø - Þ uppercase +%% 337 - 366 ß - ö lowercase +%% 367 ÷ punctuation +%% 370 - 377 ø - ÿ lowercase %% Names must begin with a lowercase letter and contain only %% alphanumerics and underscores. @@ -261,6 +269,10 @@ is_name_1(_) -> false. to_atom(A) when is_atom(A) -> A; to_atom(S) when is_list(S) -> list_to_atom(S). + +to_list(A) when is_atom(A) -> atom_to_list(A); +to_list(S) when is_list(S) -> S. + %% @private unique([X | Xs]) -> [X | unique(Xs, X)]; @@ -677,7 +689,6 @@ try_subdir(Dir, Subdir) -> write_file(Text, Dir, Name) -> write_file(Text, Dir, Name, ''). - %% @spec (Text::deep_string(), Dir::edoc:filename(), %% Name::edoc:filename(), Package::atom()|string()) -> ok %% @doc Like {@link write_file/3}, but adds path components to the target @@ -685,10 +696,12 @@ write_file(Text, Dir, Name) -> %% @private write_file(Text, Dir, Name, Package) -> - Dir1 = filename:join([Dir | packages:split(Package)]), - File = filename:join(Dir1, Name), + write_file(Text, Dir, Name, Package, [{encoding,latin1}]). + +write_file(Text, Dir, Name, Package, Options) -> + File = filename:join([Dir, to_list(Package), Name]), ok = filelib:ensure_dir(File), - case file:open(File, [write]) of + case file:open(File, [write] ++ Options) of {ok, FD} -> io:put_chars(FD, Text), ok = file:close(FD); @@ -705,8 +718,9 @@ write_info_file(App, Packages, Modules, Dir) -> Ts1 = if App =:= ?NO_APP -> Ts; true -> [{application, App} | Ts] end, - S = [io_lib:fwrite("~p.\n", [T]) || T <- Ts1], - write_file(S, Dir, ?INFO_FILE). + S0 = [io_lib:fwrite("~p.\n", [T]) || T <- Ts1], + S = ["%% encoding: UTF-8\n" | S0], + write_file(S, Dir, ?INFO_FILE, '', [{encoding,unicode}]). %% @spec (Name::edoc:filename()) -> {ok, string()} | {error, Reason} %% @@ -714,7 +728,14 @@ write_info_file(App, Packages, Modules, Dir) -> read_file(File) -> case file:read_file(File) of - {ok, Bin} -> {ok, binary_to_list(Bin)}; + {ok, Bin} -> + Enc = edoc_lib:read_encoding(File, []), + case catch unicode:characters_to_list(Bin, Enc) of + String when is_list(String) -> + {ok, String}; + _ -> + {error, invalid_unicode} + end; {error, Reason} -> {error, Reason} end. @@ -818,7 +839,7 @@ find_sources(Path, Pkg, Rec, Ext, Opts) -> lists:flatten(find_sources_1(Path, to_atom(Pkg), Rec, Ext, Skip)). find_sources_1([P | Ps], Pkg, Rec, Ext, Skip) -> - Dir = filename:join(P, filename:join(packages:split(Pkg))), + Dir = filename:join(P, atom_to_list(Pkg)), Fs1 = find_sources_1(Ps, Pkg, Rec, Ext, Skip), case filelib:is_dir(Dir) of true -> @@ -844,9 +865,9 @@ find_sources_2(Dir, Pkg, Rec, Ext, Skip) -> [] end. -find_sources_3(Es, Dir, Pkg, Rec, Ext, Skip) -> +find_sources_3(Es, Dir, '', Rec, Ext, Skip) -> [find_sources_2(filename:join(Dir, E), - to_atom(packages:concat(Pkg, E)), Rec, Ext, Skip) + to_atom(E), Rec, Ext, Skip) || E <- Es, is_package_dir(E, Dir)]. is_source_file(Name, Ext) -> @@ -858,16 +879,16 @@ is_package_dir(Name, Dir) -> andalso filelib:is_dir(filename:join(Dir, Name)). %% @private -find_file([P | Ps], Pkg, Name) -> - Dir = filename:join(P, filename:join(packages:split(Pkg))), - File = filename:join(Dir, Name), +find_file([P | Ps], []=Pkg, Name) -> + Pkg = [], + File = filename:join(P, Name), case filelib:is_file(File) of true -> File; false -> find_file(Ps, Pkg, Name) end; -find_file([], _Pkg, _Name) -> +find_file([], [], _Name) -> "". %% @private diff --git a/lib/edoc/src/edoc_macros.erl b/lib/edoc/src/edoc_macros.erl index 70fb38bf0a..08686c4fb5 100644 --- a/lib/edoc/src/edoc_macros.erl +++ b/lib/edoc/src/edoc_macros.erl @@ -88,13 +88,13 @@ link_macro(S, Line, Env) -> true -> " target=\"_top\""; % note the initial space false -> "" end, - lists:flatten(io_lib:fwrite("<a href=\"~s\"~s>~s</a>", + lists:flatten(io_lib:fwrite("<a href=\"~s\"~s>~ts</a>", [URI, Target, Txt])). section_macro(S, _Line, _Env) -> S1 = lists:reverse(edoc_lib:strip_space( lists:reverse(edoc_lib:strip_space(S)))), - lists:flatten(io_lib:format("<a href=\"#~s\">~s</a>", + lists:flatten(io_lib:format("<a href=\"#~ts\">~ts</a>", [edoc_lib:to_label(S1), S1])). type_macro(S, Line, Env) -> diff --git a/lib/edoc/src/edoc_parser.yrl b/lib/edoc/src/edoc_parser.yrl index d7c1b1c045..cf1a2d6b11 100644 --- a/lib/edoc/src/edoc_parser.yrl +++ b/lib/edoc/src/edoc_parser.yrl @@ -1,6 +1,7 @@ +%% -*- coding: utf-8 -*- %% ========================== -*-Erlang-*- ============================= %% EDoc type specification grammar for the Yecc parser generator, -%% adapted from Sven-Olof Nystr�m's type specification parser. +%% adapted from Sven-Olof Nyström's type specification parser. %% %% Also contains entry points for parsing things like typedefs, %% references, and throws-declarations. @@ -317,10 +318,7 @@ tok_val(T) -> element(3, T). tok_line(T) -> element(2, T). -qname([A]) -> - A; % avoid unnecessary call to packages:concat/1. -qname(List) -> - list_to_atom(packages:concat(lists:reverse(List))). +qname([A]) -> A. union(Ts) -> case Ts of diff --git a/lib/edoc/src/edoc_refs.erl b/lib/edoc/src/edoc_refs.erl index 1f578a3b83..ea439490ed 100644 --- a/lib/edoc/src/edoc_refs.erl +++ b/lib/edoc/src/edoc_refs.erl @@ -126,7 +126,7 @@ abs_uri({package, P}, Env) -> module_ref(M, Env) -> case (Env#env.modules)(M) of "" -> - File = packages:last(M) ++ Env#env.file_suffix, + File = atom_to_list(M) ++ Env#env.file_suffix, Path = relative_module_path(M, Env#env.package), join_uri(Path, escape_uri(File)); Base -> @@ -134,8 +134,7 @@ module_ref(M, Env) -> end. module_absref(M, Env) -> - join_segments(packages:split(M)) - ++ escape_uri(Env#env.file_suffix). + escape_uri(atom_to_list(M)) ++ escape_uri(Env#env.file_suffix). package_ref(P, Env) -> case (Env#env.packages)(P) of @@ -147,7 +146,7 @@ package_ref(P, Env) -> end. package_absref(P, Env) -> - join_uri(join_segments(packages:split(P)), + join_uri(escape_uri(atom_to_list(P)), escape_uri(Env#env.package_summary)). app_ref(A, Env) -> @@ -179,14 +178,11 @@ join_segments([S | Ss]) -> %% The empty string is returned if the To module has only one segment, %% implying a local reference. -relative_module_path(To, From) -> - case first(packages:split(To)) of - [] -> ""; - P -> relative_path(P, packages:split(From)) - end. +relative_module_path(_To, _From) -> + "". relative_package_path(To, From) -> - relative_path(packages:split(To), packages:split(From)). + relative_path([atom_to_list(To)], [atom_to_list(From)]). %% This takes two lists of path segments (From, To). Note that an empty %% string will be returned if the paths are the same. Empty leading @@ -210,6 +206,3 @@ relative_path_2([], []) -> ""; relative_path_2([], Ts) -> join_segments(Ts). - -first([H | T]) when T /= [] -> [H | first(T)]; -first(_) -> []. diff --git a/lib/edoc/src/edoc_wiki.erl b/lib/edoc/src/edoc_wiki.erl index 5c71658af5..cc0529d2a9 100644 --- a/lib/edoc/src/edoc_wiki.erl +++ b/lib/edoc/src/edoc_wiki.erl @@ -80,6 +80,7 @@ parse_xml(Data, Line) -> parse_xml_1(Text, Line) -> Text1 = "<doc>" ++ Text ++ "</doc>", + %% Any coding except "utf-8". Opts = [{line, Line}, {encoding, 'iso-8859-1'}], case catch {ok, xmerl_scan:string(Text1, Opts)} of {ok, {E, _}} -> @@ -174,7 +175,7 @@ expand_heading_1(Cs, N, L, As) -> expand_heading_2(Ts, Cs, N, L, As) -> H = ?BASE_HEADING + N, - Ts1 = io_lib:format("<h~w><a name=\"~s\">~s</a></h~w>\n", + Ts1 = io_lib:format("<h~w><a name=\"~ts\">~ts</a></h~w>\n", [H, make_label(Ts), Ts, H]), expand_new_line(Cs, L + 1, lists:reverse(lists:flatten(Ts1), As)). diff --git a/lib/eldap/src/Makefile b/lib/eldap/src/Makefile index 39a41d08e2..46fb805bcc 100644 --- a/lib/eldap/src/Makefile +++ b/lib/eldap/src/Makefile @@ -88,7 +88,7 @@ $(TARGET_FILES): $(HRL_FILES) # Special Build Targets # ---------------------------------------------------- $(ASN1_HRL): ../asn1/$(ASN1_FILES) - $(ERLC) -o $(EBIN) -bber_bin +optimize +nif $(ERL_COMPILE_FLAGS) ../asn1/ELDAPv3.asn1 + $(ERLC) -o $(EBIN) -bber $(ERL_COMPILE_FLAGS) ../asn1/ELDAPv3.asn1 # ---------------------------------------------------- # Release Target diff --git a/lib/eldap/src/eldap.erl b/lib/eldap/src/eldap.erl index 699dfc8791..b3249d4f56 100644 --- a/lib/eldap/src/eldap.erl +++ b/lib/eldap/src/eldap.erl @@ -700,20 +700,22 @@ recv_response(S, Data) -> Error -> throw(Error) end; {error,Reason} -> - throw({gen_tcp_error, Reason}); - Error -> - throw(Error) + throw({gen_tcp_error, Reason}) end. %%% Sanity check of received packet check_tag(Data) -> - case asn1rt_ber_bin:decode_tag(l2b(Data)) of - {_Tag, Data1, _Rb} -> - case asn1rt_ber_bin:decode_length(l2b(Data1)) of - {{_Len, _Data2}, _Rb2} -> ok; - _ -> throw({error,decoded_tag_length}) - end; - _ -> throw({error,decoded_tag}) + try + {_Tag, Data1, _Rb} = asn1rt_ber_bin:decode_tag(l2b(Data)), + try + {{_Len, _Data2}, _Rb2} = asn1rt_ber_bin:decode_length(l2b(Data1)), + ok + catch + _ -> throw({error,decoded_tag_length}) + end + catch + _ -> + throw({error, decoded_tag}) end. %%% Check for expected kind of reply diff --git a/lib/eldap/vsn.mk b/lib/eldap/vsn.mk index c9d6e4e324..4d05d3d1e3 100644 --- a/lib/eldap/vsn.mk +++ b/lib/eldap/vsn.mk @@ -1 +1 @@ -ELDAP_VSN = 1.0 +ELDAP_VSN = 1.0.1 diff --git a/lib/erl_docgen/doc/src/doc-build.xml b/lib/erl_docgen/doc/src/doc-build.xml index 08410a1539..ae1b17dff5 100644 --- a/lib/erl_docgen/doc/src/doc-build.xml +++ b/lib/erl_docgen/doc/src/doc-build.xml @@ -4,7 +4,7 @@ <chapter> <header> <copyright> - <year>1997</year><year>2011</year> + <year>1997</year><year>2013</year> <holder>Ericsson AB. All Rights Reserved.</holder> </copyright> <legalnotice> @@ -56,7 +56,8 @@ </p> <code> - 1> escript $(ERL_TOP)/lib/erl_docgen/priv/bin/codeline_preprocessing.escript ex1.xmlsrc ex1.xml + 1> escript $(ERL_TOP)/lib/erl_docgen/priv/bin/codeline_preprocessing.escript \ + ex1.xmlsrc ex1.xml </code> </section> </section> diff --git a/lib/erl_docgen/doc/src/notes.xml b/lib/erl_docgen/doc/src/notes.xml index ee62de86b1..9775909d76 100644 --- a/lib/erl_docgen/doc/src/notes.xml +++ b/lib/erl_docgen/doc/src/notes.xml @@ -30,7 +30,23 @@ </header> <p>This document describes the changes made to the <em>erl_docgen</em> application.</p> - <section><title>Erl_Docgen 0.3.2</title> + <section><title>Erl_Docgen 0.3.3</title> + + <section><title>Improvements and New Features</title> + <list> + <item> + <p> A possibility to configure erl_docgen so it can + generate documentation for other products than + Erlang/OTP. </p> + <p> + Own Id: OTP-9040</p> + </item> + </list> + </section> + +</section> + +<section><title>Erl_Docgen 0.3.2</title> <section><title>Fixed Bugs and Malfunctions</title> <list> diff --git a/lib/erl_docgen/priv/bin/xml_from_edoc.escript b/lib/erl_docgen/priv/bin/xml_from_edoc.escript index 2cb81be1be..65a580dca2 100755 --- a/lib/erl_docgen/priv/bin/xml_from_edoc.escript +++ b/lib/erl_docgen/priv/bin/xml_from_edoc.escript @@ -2,7 +2,7 @@ %% -*- erlang -*- %% %CopyrightBegin% %% -%% Copyright Ericsson AB 2010-2011. All Rights Reserved. +%% Copyright Ericsson AB 2010-2012. All Rights Reserved. %% %% The contents of this file are subject to the Erlang Public License, %% Version 1.1, (the "License"); you may not use this file except in @@ -100,10 +100,12 @@ module(File, Args) -> users_guide(File, Args) -> case filelib:is_regular(File) of true -> + Enc = epp:read_encoding(File, [{in_comment_only, false}]), + Encoding = [{encoding, Enc} || Enc =/= none], Opts = [{def, Args#args.def}, {app_default, "OTPROOT"}, {file_suffix, Args#args.suffix}, - {layout, Args#args.layout}], + {layout, Args#args.layout}] ++ Encoding, Env = edoc_lib:get_doc_env(Opts), @@ -115,7 +117,7 @@ users_guide(File, Args) -> Text = edoc_lib:run_layout(F, Opts), OutFile = "chapter" ++ Args#args.suffix, - edoc_lib:write_file(Text, ".", OutFile); + edoc_lib:write_file(Text, ".", OutFile, '', Encoding); false -> io:format("~s: not a regular file\n", [File]), usage() diff --git a/lib/erl_docgen/priv/fop.xconf b/lib/erl_docgen/priv/fop.xconf new file mode 100644 index 0000000000..70ecd608c3 --- /dev/null +++ b/lib/erl_docgen/priv/fop.xconf @@ -0,0 +1,45 @@ +<?xml version="1.0"?> +<!-- + # + # %CopyrightBegin% + # + # Copyright Ericsson AB 2009-2012. All Rights Reserved. + # + # The contents of this file are subject to the Erlang Public License, + # Version 1.1, (the "License"); you may not use this file except in + # compliance with the License. You should have received a copy of the + # Erlang Public License along with this software. If not, it can be + # retrieved online at http://www.erlang.org/. + # + # Software distributed under the License is distributed on an "AS IS" + # basis, WITHOUT WARRANTY OF ANY KIND, either express or implied. See + # the License for the specific language governing rights and limitations + # under the License. + # + # %CopyrightEnd% + + --> +<!-- NOTE: This is the version of the configuration --> +<fop version="1.0"> + <!-- The substitutions of DejaVu 700 are there because FOP outputs + warnings about doing the substitutions otherwise --> + <fonts> + <substitutions> + <substitution> + <from font-family="DejaVuSans" font-weight="700"/> + <to font-family="DejaVuSans" font-weight="400"/> + </substitution> + <substitution> + <from font-family="DejaVuSansMono" font-weight="700"/> + <to font-family="DejaVuSansMono" font-weight="400"/> + </substitution> + </substitutions> + </fonts> + <renderers> + <renderer mime="application/pdf"> + <fonts> + <auto-detect/> + </fonts> + </renderer> + </renderers> +</fop> diff --git a/lib/erl_docgen/priv/xsl/db_pdf_params.xsl b/lib/erl_docgen/priv/xsl/db_pdf_params.xsl index 4e61f1f476..2e3b22acf4 100644 --- a/lib/erl_docgen/priv/xsl/db_pdf_params.xsl +++ b/lib/erl_docgen/priv/xsl/db_pdf_params.xsl @@ -3,7 +3,7 @@ # # %CopyrightBegin% # - # Copyright Ericsson AB 2009-2011. All Rights Reserved. + # Copyright Ericsson AB 2009-2012. All Rights Reserved. # # The contents of this file are subject to the Erlang Public License, # Version 1.1, (the "License"); you may not use this file except in @@ -52,7 +52,7 @@ <!-- XSL-FO properties --> <xsl:attribute-set name="caption"> - <xsl:attribute name="font-family">sans-serif</xsl:attribute> + <xsl:attribute name="font-family">DejaVuSans, sans-serif</xsl:attribute> <xsl:attribute name="font-size">0.8em</xsl:attribute> <xsl:attribute name="font-weight">bold</xsl:attribute> <xsl:attribute name="keep-with-previous.within-page">always</xsl:attribute> @@ -61,7 +61,7 @@ </xsl:attribute-set> <xsl:attribute-set name="pre"> - <xsl:attribute name="font-family">monospace</xsl:attribute> + <xsl:attribute name="font-family">DejaVuSansMono, monospace</xsl:attribute> <xsl:attribute name="font-size">0.8em</xsl:attribute> <xsl:attribute name="keep-together.within-page">auto</xsl:attribute> <xsl:attribute name="linefeed-treatment">preserve</xsl:attribute> @@ -162,7 +162,7 @@ <xsl:attribute name="border-after-width">1pt</xsl:attribute> <xsl:attribute name="border-color"><xsl:value-of select="$pdfcolor"/></xsl:attribute> <xsl:attribute name="break-before">page</xsl:attribute> - <xsl:attribute name="font-family">sans-serif</xsl:attribute> + <xsl:attribute name="font-family">DejaVuSans, sans-serif</xsl:attribute> <xsl:attribute name="font-size">1.83em</xsl:attribute> <xsl:attribute name="font-weight">normal</xsl:attribute> <xsl:attribute name="space-after">1em</xsl:attribute> @@ -171,7 +171,7 @@ </xsl:attribute-set> <xsl:attribute-set name="h2"> - <xsl:attribute name="font-family">sans-serif</xsl:attribute> + <xsl:attribute name="font-family">DejaVuSans, sans-serif</xsl:attribute> <xsl:attribute name="font-size">1.5em</xsl:attribute> <xsl:attribute name="font-weight">normal</xsl:attribute> <xsl:attribute name="keep-with-next.within-page">always</xsl:attribute> @@ -180,7 +180,7 @@ </xsl:attribute-set> <xsl:attribute-set name="h3"> - <xsl:attribute name="font-family">sans-serif</xsl:attribute> + <xsl:attribute name="font-family">DejaVuSans, sans-serif</xsl:attribute> <xsl:attribute name="font-size">1.33em</xsl:attribute> <xsl:attribute name="font-weight">normal</xsl:attribute> <xsl:attribute name="keep-with-next.within-page">always</xsl:attribute> @@ -189,7 +189,7 @@ </xsl:attribute-set> <xsl:attribute-set name="h4"> - <xsl:attribute name="font-family">sans-serif</xsl:attribute> + <xsl:attribute name="font-family">DejaVuSans, sans-serif</xsl:attribute> <xsl:attribute name="font-size">1.17em</xsl:attribute> <xsl:attribute name="font-weight">normal</xsl:attribute> <xsl:attribute name="keep-with-next.within-page">always</xsl:attribute> @@ -198,7 +198,7 @@ </xsl:attribute-set> <xsl:attribute-set name="h5"> - <xsl:attribute name="font-family">sans-serif</xsl:attribute> + <xsl:attribute name="font-family">DejaVuSans, sans-serif</xsl:attribute> <xsl:attribute name="font-size">1em</xsl:attribute> <xsl:attribute name="font-weight">bold</xsl:attribute> <xsl:attribute name="keep-with-next.within-page">always</xsl:attribute> @@ -207,7 +207,7 @@ </xsl:attribute-set> <xsl:attribute-set name="h6"> - <xsl:attribute name="font-family">sans-serif</xsl:attribute> + <xsl:attribute name="font-family">DejaVuSans, sans-serif</xsl:attribute> <xsl:attribute name="font-size">0.83em</xsl:attribute> <xsl:attribute name="font-weight">bold</xsl:attribute> <xsl:attribute name="keep-with-next.within-page">always</xsl:attribute> @@ -227,20 +227,20 @@ <xsl:attribute name="border-after-style">solid</xsl:attribute> <xsl:attribute name="border-after-width">2pt</xsl:attribute> <xsl:attribute name="border-color"><xsl:value-of select="$pdfcolor"/></xsl:attribute> - <xsl:attribute name="font-family">sans-serif</xsl:attribute> + <xsl:attribute name="font-family">DejaVuSans, sans-serif</xsl:attribute> <xsl:attribute name="font-size">0.9em</xsl:attribute> <xsl:attribute name="font-weight">bold</xsl:attribute> </xsl:attribute-set> <xsl:attribute-set name="page-footer"> - <xsl:attribute name="font-family">sans-serif</xsl:attribute> + <xsl:attribute name="font-family">DejaVuSans, sans-serif</xsl:attribute> <xsl:attribute name="font-size">0.9em</xsl:attribute> <xsl:attribute name="font-weight">bold</xsl:attribute> </xsl:attribute-set> <xsl:attribute-set name="code"> <xsl:attribute name="background-color">#e0e0ff</xsl:attribute> - <xsl:attribute name="font-family">monospace</xsl:attribute> + <xsl:attribute name="font-family">DejaVuSansMono, monospace</xsl:attribute> <xsl:attribute name="font-size">0.8em</xsl:attribute> <xsl:attribute name="keep-together.within-page">auto</xsl:attribute> <xsl:attribute name="linefeed-treatment">preserve</xsl:attribute> @@ -303,7 +303,7 @@ <xsl:attribute-set name="module-name"> <xsl:attribute name="border-after-style">solid</xsl:attribute> <xsl:attribute name="border-after-width">1pt</xsl:attribute> - <xsl:attribute name="font-family">sans-serif</xsl:attribute> + <xsl:attribute name="font-family">DejaVuSans, sans-serif</xsl:attribute> <xsl:attribute name="font-size">1.5em</xsl:attribute> <xsl:attribute name="font-weight">normal</xsl:attribute> <xsl:attribute name="keep-with-next.within-page">always</xsl:attribute> @@ -313,7 +313,7 @@ <xsl:attribute-set name="function-name"> <xsl:attribute name="font-weight">bold</xsl:attribute> - <xsl:attribute name="font-family">monospace</xsl:attribute> + <xsl:attribute name="font-family">DejaVuSansMono, monospace</xsl:attribute> <!-- xsl:attribute name="font-size">0.8em</xsl:attribute --> <xsl:attribute name="keep-with-next.within-page">always</xsl:attribute> <xsl:attribute name="space-after">0.25em</xsl:attribute> @@ -401,7 +401,7 @@ </xsl:attribute-set> <xsl:attribute-set name="caption"> - <xsl:attribute name="font-family">sans-serif</xsl:attribute> + <xsl:attribute name="font-family">DejaVuSans, sans-serif</xsl:attribute> <xsl:attribute name="font-size">0.8em</xsl:attribute> <xsl:attribute name="font-weight">bold</xsl:attribute> <xsl:attribute name="keep-with-previous.within-page">always</xsl:attribute> diff --git a/lib/erl_docgen/src/docgen_edoc_xml_cb.erl b/lib/erl_docgen/src/docgen_edoc_xml_cb.erl index cbaa93a15d..e3cc354206 100644 --- a/lib/erl_docgen/src/docgen_edoc_xml_cb.erl +++ b/lib/erl_docgen/src/docgen_edoc_xml_cb.erl @@ -1,19 +1,20 @@ -%% ``The contents of this file are subject to the Erlang Public License, +%% +%% %CopyrightBegin% +%% +%% Copyright Ericsson AB 2001-2012. All Rights Reserved. +%% +%% The contents of this file are subject to the Erlang Public License, %% Version 1.1, (the "License"); you may not use this file except in %% compliance with the License. You should have received a copy of the %% Erlang Public License along with this software. If not, it can be -%% retrieved via the world wide web at http://www.erlang.org/. +%% retrieved online at http://www.erlang.org/. %% %% Software distributed under the License is distributed on an "AS IS" -%% basis, WITHOUT WARRANTY OF ANY KIND, either expressed or implied. See -%% the Licence for the specific language governing rights and limitations +%% basis, WITHOUT WARRANTY OF ANY KIND, either express or implied. See +%% the License for the specific language governing rights and limitations %% under the License. %% -%% The Initial Developer of the Original Code is Ericsson AB. -%% Portions created by Ericsson are Copyright 1999-2006, Ericsson AB. -%% All Rights Reserved.�� -%% -%% $Id$ +%% %CopyrightEnd% %% -module(docgen_edoc_xml_cb). @@ -39,12 +40,14 @@ module(Element, Opts) -> SortP = proplists:get_value(sort_functions, Opts, true), XML = layout_module(Element, SortP), - xmerl:export_simple([XML], docgen_xmerl_xml_cb, []). + RootAttributes = root_attributes(Element, Opts), + xmerl:export_simple([XML], docgen_xmerl_xml_cb, RootAttributes). %% CHAPTER -overview(Element, _Opts) -> +overview(Element, Opts) -> XML = layout_chapter(Element), - xmerl:export_simple([XML], docgen_xmerl_xml_cb, []). + RootAttributes = root_attributes(Element, Opts), + xmerl:export_simple([XML], docgen_xmerl_xml_cb, RootAttributes). %%--Internal functions-------------------------------------------------- @@ -99,6 +102,16 @@ layout_module(#xmlElement{name = module, content = Es}=E, SortP) -> ?NL,Authors] }. +root_attributes(Element, Opts) -> + Encoding = case get_attrval(encoding, Element) of + "" -> + DefaultEncoding = epp:default_encoding(), + proplists:get_value(encoding, Opts, DefaultEncoding); + Enc -> + Enc + end, + [#xmlAttribute{name=encoding, value=Encoding}]. + layout_chapter(#xmlElement{name=overview, content=Es}) -> Title = get_text(title, Es), Header = {header, [ diff --git a/lib/erl_docgen/src/docgen_xmerl_xml_cb.erl b/lib/erl_docgen/src/docgen_xmerl_xml_cb.erl index 884932ed12..d713b61c0a 100644 --- a/lib/erl_docgen/src/docgen_xmerl_xml_cb.erl +++ b/lib/erl_docgen/src/docgen_xmerl_xml_cb.erl @@ -1,19 +1,20 @@ -%% ``The contents of this file are subject to the Erlang Public License, +%% +%% %CopyrightBegin% +%% +%% Copyright Ericsson AB 2001-2012. All Rights Reserved. +%% +%% The contents of this file are subject to the Erlang Public License, %% Version 1.1, (the "License"); you may not use this file except in %% compliance with the License. You should have received a copy of the %% Erlang Public License along with this software. If not, it can be -%% retrieved via the world wide web at http://www.erlang.org/. +%% retrieved online at http://www.erlang.org/. %% %% Software distributed under the License is distributed on an "AS IS" -%% basis, WITHOUT WARRANTY OF ANY KIND, either expressed or implied. See -%% the Licence for the specific language governing rights and limitations +%% basis, WITHOUT WARRANTY OF ANY KIND, either express or implied. See +%% the License for the specific language governing rights and limitations %% under the License. %% -%% The Initial Developer of the Original Code is Ericsson AB. -%% Portions created by Ericsson are Copyright 1999-2006, Ericsson AB. -%% All Rights Reserved.�� -%% -%% $Id$ +%% %CopyrightEnd% %% -module(docgen_xmerl_xml_cb). @@ -35,9 +36,14 @@ '#xml-inheritance#'() -> [xmerl_xml]. -'#root#'(Data, _Attrs, [], _E) -> +'#root#'(Data, Attrs, [], _E) -> + Encoding = + case [E || #xmlAttribute{name = encoding, value = E} <- Attrs] of + [E] -> E; + _ -> atom_to_list(epp:default_encoding()) + end, ["<",DTD,">"] = hd(hd(Data)), - ["<?xml version=\"1.0\" encoding=\"latin1\" ?>\n", + ["<?xml version=\"1.0\" encoding=\"",Encoding,"\" ?>\n", "<!DOCTYPE "++DTD++" SYSTEM \""++DTD++".dtd\">\n", Data]. diff --git a/lib/erl_interface/aclocal.m4 b/lib/erl_interface/aclocal.m4 index b1cf1fe404..9578cd35c4 100644 --- a/lib/erl_interface/aclocal.m4 +++ b/lib/erl_interface/aclocal.m4 @@ -740,11 +740,16 @@ dnl Try to find POSIX threads dnl The usual pthread lib... AC_CHECK_LIB(pthread, pthread_create, THR_LIBS="-lpthread") -dnl FreeBSD has pthreads in special c library, c_r... +dnl Very old versions of FreeBSD have pthreads in special c library, c_r... if test "x$THR_LIBS" = "x"; then AC_CHECK_LIB(c_r, pthread_create, THR_LIBS="-lc_r") fi +dnl QNX has pthreads in standard C library + if test "x$THR_LIBS" = "x"; then + AC_CHECK_FUNC(pthread_create, THR_LIBS="none_needed") + fi + dnl On ofs1 the '-pthread' switch should be used if test "x$THR_LIBS" = "x"; then AC_MSG_CHECKING([if the '-pthread' switch can be used]) @@ -765,6 +770,9 @@ dnl On ofs1 the '-pthread' switch should be used if test "x$THR_LIBS" != "x"; then THR_DEFS="$THR_DEFS -D_THREAD_SAFE -D_REENTRANT -DPOSIX_THREADS" THR_LIB_NAME=pthread + if test "x$THR_LIBS" = "xnone_needed"; then + THR_LIBS= + fi case $host_os in solaris*) THR_DEFS="$THR_DEFS -D_POSIX_PTHREAD_SEMANTICS" ;; diff --git a/lib/erl_interface/configure.in b/lib/erl_interface/configure.in index c958f80065..97f1cff345 100644 --- a/lib/erl_interface/configure.in +++ b/lib/erl_interface/configure.in @@ -1,7 +1,7 @@ # -*- Autoconf -*- # %CopyrightBegin% # -# Copyright Ericsson AB 2000-2011. All Rights Reserved. +# Copyright Ericsson AB 2000-2012. All Rights Reserved. # # The contents of this file are subject to the Erlang Public License, # Version 1.1, (the "License"); you may not use this file except in @@ -79,7 +79,7 @@ AC_ARG_ENABLE(threads, no) threads_disabled=yes ;; *) threads_disabled=no ;; esac ], -[ threads_disabled=no ]) +[ threads_disabled=maybe ]) dnl ---------------------------------------------------------------------- dnl Checks for programs @@ -237,12 +237,16 @@ AC_SUBST(THR_DEFS) AC_SUBST(EI_THREADS) case "$threads_disabled" in - no) + no|maybe) LM_CHECK_THR_LIB case "$THR_LIB_NAME" in "") EI_THREADS="false" + # Fail if --enable-threads given and no threads found + if test "x$threads_disabled" = "xno"; then + AC_MSG_ERROR(No threads support found) + fi ;; win32_threads) EI_THREADS="true" diff --git a/lib/erl_interface/doc/src/notes.xml b/lib/erl_interface/doc/src/notes.xml index 6791c39c7d..f0a9b336ff 100644 --- a/lib/erl_interface/doc/src/notes.xml +++ b/lib/erl_interface/doc/src/notes.xml @@ -30,6 +30,24 @@ </header> <p>This document describes the changes made to the Erl_interface application.</p> +<section><title>Erl_Interface 3.7.9</title> + + <section><title>Improvements and New Features</title> + <list> + <item> + <p>Teach lib/erl_interface/configure.in to look for + pthreads support in libc (where it can be found on + QNX)</p> <p>A minor tweak such that this configure + *fails* if you pass --enable-threads and no pthreads + support can be found.</p> (Thanks to Per Hedeland) + <p> + Own Id: OTP-10581</p> + </item> + </list> + </section> + +</section> + <section><title>Erl_Interface 3.7.8</title> <section><title>Improvements and New Features</title> diff --git a/lib/erl_interface/src/connect/ei_resolve.c b/lib/erl_interface/src/connect/ei_resolve.c index ba8f8fbce3..79d259b92d 100644 --- a/lib/erl_interface/src/connect/ei_resolve.c +++ b/lib/erl_interface/src/connect/ei_resolve.c @@ -186,11 +186,11 @@ static int verify_dns_configuration(void) * advance: increment buf by n bytes, reduce len by same amount . */ #if defined SIZEOF_VOID_P -#define ALIGNBYTES (SIZEOF_VOID_P - 1) +#define EI_ALIGNBYTES (SIZEOF_VOID_P - 1) #else -#define ALIGNBYTES (sizeof(void*) - 1) +#define EI_ALIGNBYTES (sizeof(void*) - 1) #endif -#define align_buf(buf,len) for (;(((unsigned)buf) & ALIGNBYTES); (buf)++, len--) +#define align_buf(buf,len) for (;(((unsigned)buf) & EI_ALIGNBYTES); (buf)++, len--) #define advance_buf(buf,len,n) ((buf)+=(n),(len)-=(n)) /* "and now the tricky part..." */ @@ -282,6 +282,8 @@ static int copy_hostent(struct hostent *dest, const struct hostent *src, char *b return 0; } +#undef EI_ALIGNBYTES + /* This function is a pseudo-reentrant version of gethostbyname(). It * uses locks to serialize the call to the regular (non-reentrant) * gethostbyname() and then copies the data into the user-provided diff --git a/lib/erl_interface/test/ei_decode_SUITE.erl b/lib/erl_interface/test/ei_decode_SUITE.erl index bb44b78854..2c4b6e5541 100644 --- a/lib/erl_interface/test/ei_decode_SUITE.erl +++ b/lib/erl_interface/test/ei_decode_SUITE.erl @@ -1,7 +1,8 @@ +%% -*- coding: utf-8 -*- %% %% %CopyrightBegin% %% -%% Copyright Ericsson AB 2004-2011. All Rights Reserved. +%% Copyright Ericsson AB 2004-2012. All Rights Reserved. %% %% The contents of this file are subject to the Erlang Public License, %% Version 1.1, (the "License"); you may not use this file except in @@ -204,15 +205,15 @@ test_ei_decode_misc(Config) when is_list(Config) -> ?line send_term_as_binary(P,foo), ?line send_term_as_binary(P,''), - ?line send_term_as_binary(P,'������'), + ?line send_term_as_binary(P,'ÅÄÖåäö'), ?line send_term_as_binary(P,"foo"), ?line send_term_as_binary(P,""), - ?line send_term_as_binary(P,"������"), + ?line send_term_as_binary(P,"ÅÄÖåäö"), ?line send_term_as_binary(P,<<"foo">>), ?line send_term_as_binary(P,<<>>), - ?line send_term_as_binary(P,<<"������">>), + ?line send_term_as_binary(P,<<"ÅÄÖåäö">>), % ?line send_term_as_binary(P,{}), % ?line send_term_as_binary(P,[]), diff --git a/lib/erl_interface/test/ei_encode_SUITE.erl b/lib/erl_interface/test/ei_encode_SUITE.erl index cefd33e5f6..537e9cb01c 100644 --- a/lib/erl_interface/test/ei_encode_SUITE.erl +++ b/lib/erl_interface/test/ei_encode_SUITE.erl @@ -1,7 +1,8 @@ +%% -*- coding: utf-8 -*- %% %% %CopyrightBegin% %% -%% Copyright Ericsson AB 2004-2011. All Rights Reserved. +%% Copyright Ericsson AB 2004-2012. All Rights Reserved. %% %% The contents of this file are subject to the Erlang Public License, %% Version 1.1, (the "License"); you may not use this file except in @@ -213,19 +214,19 @@ test_ei_encode_misc(Config) when is_list(Config) -> ?line {<<100,0,3,"foo">>,foo} = get_buf_and_term(P), ?line {<<100,0,0,"">>,''} = get_buf_and_term(P), ?line {<<100,0,0,"">>,''} = get_buf_and_term(P), - ?line {<<100,0,6,"������">>,'������'} = get_buf_and_term(P), - ?line {<<100,0,6,"������">>,'������'} = get_buf_and_term(P), + ?line {<<100,0,6,"ÅÄÖåäö">>,'ÅÄÖåäö'} = get_buf_and_term(P), + ?line {<<100,0,6,"ÅÄÖåäö">>,'ÅÄÖåäö'} = get_buf_and_term(P), ?line {<<107,0,3,"foo">>,"foo"} = get_buf_and_term(P), ?line {<<107,0,3,"foo">>,"foo"} = get_buf_and_term(P), ?line {<<106>>,""} = get_buf_and_term(P), ?line {<<106>>,""} = get_buf_and_term(P), - ?line {<<107,0,6,"������">>,"������"} = get_buf_and_term(P), - ?line {<<107,0,6,"������">>,"������"} = get_buf_and_term(P), + ?line {<<107,0,6,"ÅÄÖåäö">>,"ÅÄÖåäö"} = get_buf_and_term(P), + ?line {<<107,0,6,"ÅÄÖåäö">>,"ÅÄÖåäö"} = get_buf_and_term(P), ?line {<<109,0,0,0,3,"foo">>,<<"foo">>} = get_buf_and_term(P), ?line {<<109,0,0,0,0,"">>,<<>>} = get_buf_and_term(P), - ?line {<<109,0,0,0,6,"������">>,<<"������">>} = get_buf_and_term(P), + ?line {<<109,0,0,0,6,"ÅÄÖåäö">>,<<"ÅÄÖåäö">>} = get_buf_and_term(P), ?line {<<104,0>>,{}} = get_buf_and_term(P), % Tuple header for {} ?line {<<106>>,[]} = get_buf_and_term(P), % Empty list [] diff --git a/lib/erl_interface/test/ei_print_SUITE.erl b/lib/erl_interface/test/ei_print_SUITE.erl index 2a3ed81f53..6305302e28 100644 --- a/lib/erl_interface/test/ei_print_SUITE.erl +++ b/lib/erl_interface/test/ei_print_SUITE.erl @@ -1,7 +1,7 @@ %% %% %CopyrightBegin% %% -%% Copyright Ericsson AB 2001-2011. All Rights Reserved. +%% Copyright Ericsson AB 2001-2012. All Rights Reserved. %% %% The contents of this file are subject to the Erlang Public License, %% Version 1.1, (the "License"); you may not use this file except in @@ -136,7 +136,7 @@ lists(Config) when is_list(Config) -> %% ?line {term, "[{name, 'Madonna'}, {age, 21}, {data, [{addr, "E-street", 42}]}]"} = %% get_term(P), - %% kanske regexp i st�llet? + %% maybe regexp instead? ?line {term, "[{pi, 3.141500}, {'cos(70)', 0.342020}]"} = get_term(P), ?line {term, "[[pi, 3.141500], ['cos(70)', 0.342020]]"} = get_term(P), diff --git a/lib/erl_interface/vsn.mk b/lib/erl_interface/vsn.mk index 4e401c874e..1718f38069 100644 --- a/lib/erl_interface/vsn.mk +++ b/lib/erl_interface/vsn.mk @@ -1 +1 @@ -EI_VSN = 3.7.8 +EI_VSN = 3.7.9 diff --git a/lib/et/src/et.erl b/lib/et/src/et.erl index e2cd8564c3..c9ba4f6816 100644 --- a/lib/et/src/et.erl +++ b/lib/et/src/et.erl @@ -1,7 +1,7 @@ %% %% %CopyrightBegin% %% -%% Copyright Ericsson AB 2000-2010. All Rights Reserved. +%% Copyright Ericsson AB 2000-2012. All Rights Reserved. %% %% The contents of this file are subject to the Erlang Public License, %% Version 1.1, (the "License"); you may not use this file except in @@ -102,7 +102,7 @@ %% From = actor() %% To = actor() %% FromTo = actor() -%% Label = atom() | string() |�term() +%% Label = atom() | string() | term() %% Contents = [{Key, Value}] | term() %% %% actor() = term() diff --git a/lib/eunit/doc/src/notes.xml b/lib/eunit/doc/src/notes.xml index 814580e228..b797be0ccb 100644 --- a/lib/eunit/doc/src/notes.xml +++ b/lib/eunit/doc/src/notes.xml @@ -5,7 +5,7 @@ <header> <copyright> <year>2008</year> - <year>2011</year> + <year>2012</year> <holder>Ericsson AB, All Rights Reserved</holder> </copyright> <legalnotice> diff --git a/lib/eunit/include/eunit.hrl b/lib/eunit/include/eunit.hrl index fba840c3bd..8ebdb6ba16 100644 --- a/lib/eunit/include/eunit.hrl +++ b/lib/eunit/include/eunit.hrl @@ -1,3 +1,4 @@ +%% -*- coding: utf-8 -*- %% This library is free software; you can redistribute it and/or modify %% it under the terms of the GNU Lesser General Public License as %% published by the Free Software Foundation; either version 2 of the @@ -13,7 +14,7 @@ %% Foundation, Inc., 59 Temple Place, Suite 330, Boston, MA 02111-1307 %% USA %% -%% Copyright (C) 2004-2006 Micka�l R�mond, Richard Carlsson +%% Copyright (C) 2004-2006 Mickaël Rémond, Richard Carlsson %% Including this file turns on testing and defines TEST, unless NOTEST %% is defined before the file is included. If both NOTEST and TEST are @@ -124,8 +125,8 @@ -ifndef(UNDER_EUNIT). -define(UNDER_EUNIT, (?MATCHES({current_function,{eunit_proc,_,_}}, - .erlang:process_info(.erlang:group_leader(), - current_function)))). + erlang:process_info(erlang:group_leader(), + current_function)))). -endif. %% The plain assert macro should be defined to do nothing if this file @@ -142,14 +143,14 @@ ((fun () -> case (BoolExpr) of true -> ok; - __V -> .erlang:error({assertion_failed, - [{module, ?MODULE}, - {line, ?LINE}, - {expression, (??BoolExpr)}, - {expected, true}, - {value, case __V of false -> __V; - _ -> {not_a_boolean,__V} - end}]}) + __V -> erlang:error({assertion_failed, + [{module, ?MODULE}, + {line, ?LINE}, + {expression, (??BoolExpr)}, + {expected, true}, + {value, case __V of false -> __V; + _ -> {not_a_boolean,__V} + end}]}) end end)())). -endif. @@ -170,12 +171,12 @@ ((fun () -> case (Expr) of Guard -> ok; - __V -> .erlang:error({assertMatch_failed, - [{module, ?MODULE}, - {line, ?LINE}, - {expression, (??Expr)}, - {pattern, (??Guard)}, - {value, __V}]}) + __V -> erlang:error({assertMatch_failed, + [{module, ?MODULE}, + {line, ?LINE}, + {expression, (??Expr)}, + {pattern, (??Guard)}, + {value, __V}]}) end end)())). -endif. @@ -189,12 +190,12 @@ ((fun () -> __V = (Expr), case __V of - Guard -> .erlang:error({assertNotMatch_failed, - [{module, ?MODULE}, - {line, ?LINE}, - {expression, (??Expr)}, - {pattern, (??Guard)}, - {value, __V}]}); + Guard -> erlang:error({assertNotMatch_failed, + [{module, ?MODULE}, + {line, ?LINE}, + {expression, (??Expr)}, + {pattern, (??Guard)}, + {value, __V}]}); _ -> ok end end)())). @@ -210,12 +211,12 @@ ((fun (__X) -> case (Expr) of __X -> ok; - __V -> .erlang:error({assertEqual_failed, - [{module, ?MODULE}, - {line, ?LINE}, - {expression, (??Expr)}, - {expected, __X}, - {value, __V}]}) + __V -> erlang:error({assertEqual_failed, + [{module, ?MODULE}, + {line, ?LINE}, + {expression, (??Expr)}, + {expected, __X}, + {value, __V}]}) end end)(Expect))). -endif. @@ -228,11 +229,11 @@ -define(assertNotEqual(Unexpected, Expr), ((fun (__X) -> case (Expr) of - __X -> .erlang:error({assertNotEqual_failed, - [{module, ?MODULE}, - {line, ?LINE}, - {expression, (??Expr)}, - {value, __X}]}); + __X -> erlang:error({assertNotEqual_failed, + [{module, ?MODULE}, + {line, ?LINE}, + {expression, (??Expr)}, + {value, __X}]}); _ -> ok end end)(Unexpected))). @@ -248,7 +249,7 @@ -define(assertException(Class, Term, Expr), ((fun () -> try (Expr) of - __V -> .erlang:error({assertException_failed, + __V -> erlang:error({assertException_failed, [{module, ?MODULE}, {line, ?LINE}, {expression, (??Expr)}, @@ -259,16 +260,16 @@ catch Class:Term -> ok; __C:__T -> - .erlang:error({assertException_failed, - [{module, ?MODULE}, - {line, ?LINE}, - {expression, (??Expr)}, - {pattern, - "{ "++(??Class)++" , "++(??Term) - ++" , [...] }"}, - {unexpected_exception, - {__C, __T, - .erlang:get_stacktrace()}}]}) + erlang:error({assertException_failed, + [{module, ?MODULE}, + {line, ?LINE}, + {expression, (??Expr)}, + {pattern, + "{ "++(??Class)++" , "++(??Term) + ++" , [...] }"}, + {unexpected_exception, + {__C, __T, + erlang:get_stacktrace()}}]}) end end)())). -endif. @@ -299,17 +300,17 @@ Class -> case __T of Term -> - .erlang:error({assertNotException_failed, - [{module, ?MODULE}, - {line, ?LINE}, - {expression, (??Expr)}, - {pattern, - "{ "++(??Class)++" , " - ++(??Term)++" , [...] }"}, - {unexpected_exception, - {__C, __T, - .erlang:get_stacktrace() - }}]}); + erlang:error({assertNotException_failed, + [{module, ?MODULE}, + {line, ?LINE}, + {expression, (??Expr)}, + {pattern, + "{ "++(??Class)++" , " + ++(??Term)++" , [...] }"}, + {unexpected_exception, + {__C, __T, + erlang:get_stacktrace() + }}]}); _ -> ok end; _ -> ok @@ -324,17 +325,17 @@ %% require EUnit to be present at runtime, or at least eunit_lib.) %% these can be used for simply running commands in a controlled way --define(_cmd_(Cmd), (.eunit_lib:command(Cmd))). +-define(_cmd_(Cmd), (eunit_lib:command(Cmd))). -define(cmdStatus(N, Cmd), ((fun () -> case ?_cmd_(Cmd) of {(N), __Out} -> __Out; - {__N, _} -> .erlang:error({command_failed, - [{module, ?MODULE}, - {line, ?LINE}, - {command, (Cmd)}, - {expected_status,(N)}, - {status,__N}]}) + {__N, _} -> erlang:error({command_failed, + [{module, ?MODULE}, + {line, ?LINE}, + {command, (Cmd)}, + {expected_status,(N)}, + {status,__N}]}) end end)())). -define(_cmdStatus(N, Cmd), ?_test(?cmdStatus(N, Cmd))). @@ -350,12 +351,12 @@ ((fun () -> case ?_cmd_(Cmd) of {(N), _} -> ok; - {__N, _} -> .erlang:error({assertCmd_failed, - [{module, ?MODULE}, - {line, ?LINE}, - {command, (Cmd)}, - {expected_status,(N)}, - {status,__N}]}) + {__N, _} -> erlang:error({assertCmd_failed, + [{module, ?MODULE}, + {line, ?LINE}, + {command, (Cmd)}, + {expected_status,(N)}, + {status,__N}]}) end end)())). -endif. @@ -368,12 +369,12 @@ ((fun () -> case ?_cmd_(Cmd) of {_, (T)} -> ok; - {_, __T} -> .erlang:error({assertCmdOutput_failed, - [{module, ?MODULE}, - {line, ?LINE}, - {command,(Cmd)}, - {expected_output,(T)}, - {output,__T}]}) + {_, __T} -> erlang:error({assertCmdOutput_failed, + [{module, ?MODULE}, + {line, ?LINE}, + {command,(Cmd)}, + {expected_output,(T)}, + {output,__T}]}) end end)())). -endif. @@ -394,12 +395,12 @@ -else. -define(debugMsg(S), (begin - .io:fwrite(user, <<"~s:~w:~w: ~s\n">>, - [?FILE, ?LINE, self(), S]), + io:fwrite(user, <<"~s:~w:~w: ~s\n">>, + [?FILE, ?LINE, self(), S]), ok end)). -define(debugHere, (?debugMsg("<-"))). --define(debugFmt(S, As), (?debugMsg(.io_lib:format((S), (As))))). +-define(debugFmt(S, As), (?debugMsg(io_lib:format((S), (As))))). -define(debugVal(E), ((fun (__V) -> ?debugFmt(<<"~s = ~P">>, [(??E), __V, 15]), diff --git a/lib/eunit/src/eunit.erl b/lib/eunit/src/eunit.erl index 51846d73b3..5763949519 100644 --- a/lib/eunit/src/eunit.erl +++ b/lib/eunit/src/eunit.erl @@ -1,3 +1,4 @@ +%% -*- coding: utf-8 -*- %% This library is free software; you can redistribute it and/or modify %% it under the terms of the GNU Lesser General Public License as %% published by the Free Software Foundation; either version 2 of the @@ -13,8 +14,8 @@ %% Foundation, Inc., 59 Temple Place, Suite 330, Boston, MA 02111-1307 %% USA %% -%% @copyright 2004-2009 Micka�l R�mond, Richard Carlsson -%% @author Micka�l R�mond <[email protected]> +%% @copyright 2004-2009 Mickaël Rémond, Richard Carlsson +%% @author Mickaël Rémond <[email protected]> %% [http://www.process-one.net/] %% @author Richard Carlsson <[email protected]> %% @version {@version}, {@date} {@time} diff --git a/lib/eunit/src/eunit_autoexport.erl b/lib/eunit/src/eunit_autoexport.erl index 099bcb222e..36ae3b71d7 100644 --- a/lib/eunit/src/eunit_autoexport.erl +++ b/lib/eunit/src/eunit_autoexport.erl @@ -80,10 +80,9 @@ rewrite([F | Fs], As, Module, Test) -> rewrite(Fs, [F | As], Module, Test); rewrite([], As, Module, Test) -> {if Test -> - EUnit = {record_field,0,{atom,0,''},{atom,0,eunit}}, [{function,0,test,0, [{clause,0,[],[], - [{call,0,{remote,0,EUnit,{atom,0,test}}, + [{call,0,{remote,0,{atom,0,eunit},{atom,0,test}}, [{atom,0,Module}]}]}]} | As]; true -> @@ -92,9 +91,7 @@ rewrite([], As, Module, Test) -> Test}. module_decl(Name, M, Fs, Exports) -> - Module = if is_atom(Name) -> Name; - true -> list_to_atom(packages:concat(Name)) - end, + Module = Name, {Fs1, Test} = rewrite(Fs, [], Module, true), Es = if Test -> [{test,0} | Exports]; true -> Exports diff --git a/lib/eunit/src/eunit_lib.erl b/lib/eunit/src/eunit_lib.erl index ea9e944d7e..809cb7ab7b 100644 --- a/lib/eunit/src/eunit_lib.erl +++ b/lib/eunit/src/eunit_lib.erl @@ -1,3 +1,4 @@ +%% -*- coding: utf-8 -*- %% This library is free software; you can redistribute it and/or modify %% it under the terms of the GNU Lesser General Public License as %% published by the Free Software Foundation; either version 2 of the @@ -13,8 +14,8 @@ %% Foundation, Inc., 59 Temple Place, Suite 330, Boston, MA 02111-1307 %% USA %% -%% @copyright 2004-2007 Micka�l R�mond, Richard Carlsson -%% @author Micka�l R�mond <[email protected]> +%% @copyright 2004-2007 Mickaël Rémond, Richard Carlsson +%% @author Mickaël Rémond <[email protected]> %% [http://www.process-one.net/] %% @author Richard Carlsson <[email protected]> %% @private diff --git a/lib/eunit/src/eunit_surefire.erl b/lib/eunit/src/eunit_surefire.erl index 46b8c8b503..cc021625d5 100644 --- a/lib/eunit/src/eunit_surefire.erl +++ b/lib/eunit/src/eunit_surefire.erl @@ -1,3 +1,4 @@ +%% -*- coding: utf-8 -*- %% This library is free software; you can redistribute it and/or modify %% it under the terms of the GNU Lesser General Public License as %% published by the Free Software Foundation; either version 2 of the @@ -13,8 +14,8 @@ %% Foundation, Inc., 59 Temple Place, Suite 330, Boston, MA 02111-1307 %% USA %% -%% @author Micka�l R�mond <[email protected]> -%% @copyright 2009 Micka�l R�mond, Paul Guyot +%% @author Mickaël Rémond <[email protected]> +%% @copyright 2009 Mickaël Rémond, Paul Guyot %% @see eunit %% @doc Surefire reports for EUnit (Format used by Maven and Atlassian %% Bamboo for example to integrate test results). Based on initial code diff --git a/lib/gs/contribs/mandel/mandel.erl b/lib/gs/contribs/mandel/mandel.erl index 8ecd649532..a7a786ce98 100644 --- a/lib/gs/contribs/mandel/mandel.erl +++ b/lib/gs/contribs/mandel/mandel.erl @@ -1,3 +1,4 @@ +%% -*- coding: utf-8 -*- %% %% %CopyrightBegin% %% @@ -34,7 +35,7 @@ %%%----------------------------------------------------------------- %%% Distributed Mandelbrot program. %%% Originally written i C++/rpc/lwp/interviews by Klas Eriksson.(1200 lines) -%%% Rewritten in Erlang by Klas Eriksson and Martin Bj�rklund. +%%% Rewritten in Erlang by Klas Eriksson and Martin Björklund. %%%----------------------------------------------------------------- %% unix>erl -sname foo (all nodes will get the same name) diff --git a/lib/gs/src/gstk_editor.erl b/lib/gs/src/gstk_editor.erl index e918d93147..cb422aef95 100644 --- a/lib/gs/src/gstk_editor.erl +++ b/lib/gs/src/gstk_editor.erl @@ -90,9 +90,9 @@ %% type %% -%.t tag names 2.7 -> red blue (blue �r f�rgen) -%.t tag add blue 2.1 2.10 tagga text -%.t tag configure blue -foregr blue skapa tag +%.t tag names 2.7 -> red blue (blue is the colour) +%.t tag add blue 2.1 2.10 tag the text +%.t tag configure blue -foregr blue create tag % .t index end -> MaxRows.cols % .t yview moveto (Row-1)/MaxRows diff --git a/lib/hipe/amd64/hipe_amd64_encode.erl b/lib/hipe/amd64/hipe_amd64_encode.erl index ee68dfb3b8..cbdab25b25 100644 --- a/lib/hipe/amd64/hipe_amd64_encode.erl +++ b/lib/hipe/amd64/hipe_amd64_encode.erl @@ -1,7 +1,7 @@ %%% %%% %CopyrightBegin% %%% -%%% Copyright Ericsson AB 2004-2009. All Rights Reserved. +%%% Copyright Ericsson AB 2004-2012. All Rights Reserved. %%% %%% The contents of this file are subject to the Erlang Public License, %%% Version 1.1, (the "License"); you may not use this file except in @@ -373,16 +373,16 @@ sse2_arith_binop_encode(Prefix, Opcode, {{xmm, XMM64}, {rm64fp, RM64}}) -> [Prefix, 16#0F, Opcode | encode_rm(RM64, XMM64, [])]. sse2_cvtsi2sd_encode({{xmm,XMM64}, {rm64,RM64}}) -> - [rex([{w, 1}]), 16#F2, 16#0F, 16#2A�| encode_rm(RM64, XMM64, [])]. + [rex([{w, 1}]), 16#F2, 16#0F, 16#2A | encode_rm(RM64, XMM64, [])]. sse2_mov_encode(Opnds) -> case Opnds of {{xmm, XMM64}, {rm64fp, RM64}} -> % movsd - [16#F2, 16#0F, 16#10�| encode_rm(RM64, XMM64, [])]; + [16#F2, 16#0F, 16#10 | encode_rm(RM64, XMM64, [])]; {{rm64fp, RM64}, {xmm, XMM64}} -> % movsd - [16#F2, 16#0F, 16#11�| encode_rm(RM64, XMM64, [])] + [16#F2, 16#0F, 16#11 | encode_rm(RM64, XMM64, [])] % {{xmm, XMM64}, {rm64, RM64}} -> % cvtsi2sd -% [rex([{w, 1}]), 16#F2, 16#0F, 16#2A�| encode_rm(RM64, XMM64, [])] +% [rex([{w, 1}]), 16#F2, 16#0F, 16#2A | encode_rm(RM64, XMM64, [])] end. %% arith_binop_sizeof(Opnds) -> diff --git a/lib/hipe/cerl/erl_bif_types.erl b/lib/hipe/cerl/erl_bif_types.erl index 776e336aea..98d65abba1 100644 --- a/lib/hipe/cerl/erl_bif_types.erl +++ b/lib/hipe/cerl/erl_bif_types.erl @@ -766,37 +766,6 @@ type(erlang, node, 0, _) -> t_node(); type(erlang, node, 1, Xs) -> strict(arg_types(erlang, node, 1), Xs, fun (_) -> t_node() end); type(erlang, nodes, 0, _) -> t_list(t_node()); -type(erlang, port_call, Arity, Xs) when Arity =:= 2; Arity =:= 3 -> - strict(arg_types(erlang, port_call, Arity), Xs, fun (_) -> t_any() end); -type(erlang, port_info, 1, Xs) -> - strict(arg_types(erlang, port_info, 1), Xs, - fun (_) -> t_sup(t_atom('undefined'), t_list()) end); -type(erlang, port_info, 2, Xs) -> - strict(arg_types(erlang, port_info, 2), Xs, - fun ([_Port, Item]) -> - t_sup(t_atom('undefined'), - case t_atom_vals(Item) of - ['connected'] -> t_tuple([Item, t_pid()]); - ['id'] -> t_tuple([Item, t_integer()]); - ['input'] -> t_tuple([Item, t_integer()]); - ['links'] -> t_tuple([Item, t_list(t_pid())]); - ['name'] -> t_tuple([Item, t_string()]); - ['output'] -> t_tuple([Item, t_integer()]); - ['os_pid'] -> t_tuple([Item, t_sup(t_non_neg_integer(),t_atom('undefined'))]); - ['registered_name'] -> t_tuple([Item, t_atom()]); - List when is_list(List) -> - t_tuple([t_sup([t_atom(A) || A <- List]), - t_sup([t_atom(), t_integer(), - t_pid(), t_list(t_pid()), - t_string()])]); - unknown -> - [_, PosItem] = arg_types(erlang, port_info, 2), - t_tuple([PosItem, - t_sup([t_atom(), t_integer(), - t_pid(), t_list(t_pid()), - t_string()])]) - end) - end); %% Guard bif, needs to be here. type(erlang, round, 1, Xs) -> strict(arg_types(erlang, round, 1), Xs, fun (_) -> t_integer() end); @@ -922,6 +891,14 @@ type(erlang, system_info, 1, Xs) -> t_non_neg_fixnum(), t_non_neg_fixnum()]), t_string()); + ['otp_release'] -> + t_string(); + ['port_parallelism'] -> + t_boolean(); + ['port_count'] -> + t_non_neg_fixnum(); + ['port_limit'] -> + t_non_neg_fixnum(); ['process_count'] -> t_non_neg_fixnum(); ['process_limit'] -> @@ -2275,16 +2252,6 @@ arg_types(erlang, node, 1) -> [t_identifier()]; arg_types(erlang, nodes, 0) -> []; -arg_types(erlang, port_call, 2) -> - [t_sup(t_port(), t_atom()), t_any()]; -arg_types(erlang, port_call, 3) -> - [t_sup(t_port(), t_atom()), t_integer(), t_any()]; -arg_types(erlang, port_info, 1) -> - [t_sup(t_port(), t_atom())]; -arg_types(erlang, port_info, 2) -> - [t_sup(t_port(), t_atom()), - t_atoms(['registered_name', 'id', 'connected', - 'links', 'name', 'input', 'output', 'os_pid'])]; %% Guard bif, needs to be here. arg_types(erlang, round, 1) -> [t_number()]; diff --git a/lib/hipe/doc/src/notes.xml b/lib/hipe/doc/src/notes.xml index 1841c7d9de..cfd22a9d8d 100644 --- a/lib/hipe/doc/src/notes.xml +++ b/lib/hipe/doc/src/notes.xml @@ -30,6 +30,22 @@ </header> <p>This document describes the changes made to HiPE.</p> +<section><title>Hipe 3.9.3</title> + + <section><title>Fixed Bugs and Malfunctions</title> + <list> + <item> + <p> + A faulty spec for process_info/2 could cause false + dialyzer warnings. The spec is corrected.</p> + <p> + Own Id: OTP-10584</p> + </item> + </list> + </section> + +</section> + <section><title>Hipe 3.9.2</title> <section><title>Fixed Bugs and Malfunctions</title> diff --git a/lib/hipe/flow/hipe_dominators.erl b/lib/hipe/flow/hipe_dominators.erl index 17357461a5..1f2c830eaf 100644 --- a/lib/hipe/flow/hipe_dominators.erl +++ b/lib/hipe/flow/hipe_dominators.erl @@ -1,8 +1,8 @@ -%% -*- erlang-indent-level: 2 -*- +%% -*- coding: utf-8; erlang-indent-level: 2 -*- %% %% %CopyrightBegin% %% -%% Copyright Ericsson AB 2004-2010. All Rights Reserved. +%% Copyright Ericsson AB 2004-2013. All Rights Reserved. %% %% The contents of this file are subject to the Erlang Public License, %% Version 1.1, (the "License"); you may not use this file except in @@ -19,7 +19,7 @@ %% %%------------------------------------------------------------------------ %% File : hipe_dominators.erl -%% Author : Christoffer Vikstr�m <[email protected]> +%% Author : Christoffer Vikström <[email protected]> %% Daniel Deogun <[email protected]> %% Jesper Bengtsson <[email protected]> %% Created : 18 Mar 2002 diff --git a/lib/hipe/icode/hipe_icode_mulret.erl b/lib/hipe/icode/hipe_icode_mulret.erl index a3cae621ab..0bf9f89994 100644 --- a/lib/hipe/icode/hipe_icode_mulret.erl +++ b/lib/hipe/icode/hipe_icode_mulret.erl @@ -1,8 +1,8 @@ -%% -*- erlang-indent-level: 2 -*- +%% -*- coding: utf-8; erlang-indent-level: 2 -*- %% %% %CopyrightBegin% %% -%% Copyright Ericsson AB 2005-2011. All Rights Reserved. +%% Copyright Ericsson AB 2005-2012. All Rights Reserved. %% %% The contents of this file are subject to the Erlang Public License, %% Version 1.1, (the "License"); you may not use this file except in @@ -19,9 +19,9 @@ %% %%---------------------------------------------------------------------- %% File : hipe_icode_mulret.erl -%% Author : Christoffer Vikstr�m <[email protected]> +%% Author : Christoffer Vikström <[email protected]> %% Purpose : -%% Created : 23 Jun 2004 by Christoffer Vikstr�m <[email protected]> +%% Created : 23 Jun 2004 by Christoffer Vikström <[email protected]> %%---------------------------------------------------------------------- -module(hipe_icode_mulret). @@ -890,7 +890,7 @@ removeUnElems([I|Code], [OldVar] = OldVars, DstLst, Res, Def, Lab) -> %% [I|Res], Def, Lab) %% end; false -> - io:format("Borde aldrig kunna hamna h�r!", []), + io:format("Borde aldrig kunna hamna här!", []), removeUnElems(Code, OldVars, DstLst, [I|Res], Def, Lab) end end; @@ -1159,8 +1159,8 @@ printCallList([]) -> io:format("~n"). %% % Purpose : %% % Arguments : %% % Return : -%% % Notes : Fixa s� att funktionen anv�nder defines(I) ist�llet och -%% % selektorer ist�llet f�r att matcha p� #call{}. L�tt gjort. +%% % Notes : Fixa så att funktionen använder defines(I) istället och +%% % selektorer istället för att matcha på #call{}. Lätt gjort. %% %%>----------------------------------------------------------------------< %% removeUnElems(List, Var) -> removeUnElems(List, Var, []). %% removeUnElems([#icode_call{'fun'={unsafe_element,_}, args=Var}|List], Var, Res) -> diff --git a/lib/hipe/regalloc/hipe_coalescing_regalloc.erl b/lib/hipe/regalloc/hipe_coalescing_regalloc.erl index 5a4b017c71..7169dd18f3 100644 --- a/lib/hipe/regalloc/hipe_coalescing_regalloc.erl +++ b/lib/hipe/regalloc/hipe_coalescing_regalloc.erl @@ -1,8 +1,8 @@ -%% -*- erlang-indent-level: 2 -*- +%% -*- coding: utf-8; erlang-indent-level: 2 -*- %% %% %CopyrightBegin% %% -%% Copyright Ericsson AB 2001-2009. All Rights Reserved. +%% Copyright Ericsson AB 2001-2012. All Rights Reserved. %% %% The contents of this file are subject to the Erlang Public License, %% Version 1.1, (the "License"); you may not use this file except in @@ -20,8 +20,8 @@ %%----------------------------------------------------------------------- %% File : hipe_coalescing_regalloc.erl %% Authors : Andreas Wallin <[email protected]> -%% Thorild Sel�n <[email protected]> -%% Ingemar �berg <[email protected]> +%% Thorild Selén <[email protected]> +%% Ingemar Åberg <[email protected]> %% Purpose : Play paintball with registers on a target machine. We win %% if they are all colored. This is an iterated coalescing %% register allocator. diff --git a/lib/hipe/regalloc/hipe_optimistic_regalloc.erl b/lib/hipe/regalloc/hipe_optimistic_regalloc.erl index 183ec1994c..fc3718cbc0 100644 --- a/lib/hipe/regalloc/hipe_optimistic_regalloc.erl +++ b/lib/hipe/regalloc/hipe_optimistic_regalloc.erl @@ -1,8 +1,8 @@ -%% -*- erlang-indent-level: 2 -*- +%% -*- coding: utf-8; erlang-indent-level: 2 -*- %% %% %CopyrightBegin% %% -%% Copyright Ericsson AB 2005-2009. All Rights Reserved. +%% Copyright Ericsson AB 2005-2012. All Rights Reserved. %% %% The contents of this file are subject to the Erlang Public License, %% Version 1.1, (the "License"); you may not use this file except in @@ -1657,9 +1657,9 @@ findPrimitiveNodes(Node, N, Alias, PrimitiveNodes) -> % %?debug_msg("Node ~p~n", [Node]), % NextNode = Node - 1, % Coalesced_to = hipe_reg_worklists:member_coalesced_to(NextNode, Worklists), -% ?debug_msg("��-- member coalesced: ~p~n", [Coalesced_to]), +% ?debug_msg("³³-- member coalesced: ~p~n", [Coalesced_to]), % {Primitives, Alias1} = undoCoalescing(NextNode, No_temporaries, Alias), -% ?debug_msg("��-- primitivenodes ~w\n", [Primitives]), +% ?debug_msg("½½-- primitivenodes ~w\n", [Primitives]), % case (Coalesced_to) of % true -> printAlias(Alias1); % _ -> true @@ -1683,9 +1683,9 @@ findPrimitiveNodes(Node, N, Alias, PrimitiveNodes) -> fixAdj(N, SavedAdj, IG, Target) -> %Saved = hipe_vectors:get(SavedAdj, N), Saved = hipe_adj_list:edges(N, SavedAdj), - ?debug_msg("��--adj to ~p: ~p~n", [N, Saved]), + ?debug_msg("§§--adj to ~p: ~p~n", [N, Saved]), Adj = hipe_ig:node_adj_list(N, IG), - ?debug_msg("��--adj to ~p: ~p~n", [N, Adj]), + ?debug_msg("««--adj to ~p: ~p~n", [N, Adj]), New = findNew(Adj, Saved), ?debug_msg("++--new adj to ~p: ~p~n", [N, New]), removeAdj(New, N, IG, Target), diff --git a/lib/hipe/regalloc/hipe_reg_worklists.erl b/lib/hipe/regalloc/hipe_reg_worklists.erl index 67a5788c7c..e22cc8dc07 100644 --- a/lib/hipe/regalloc/hipe_reg_worklists.erl +++ b/lib/hipe/regalloc/hipe_reg_worklists.erl @@ -1,8 +1,8 @@ -%%% -*- erlang-indent-level: 2 -*- +%%% -*- coding: utf-8; erlang-indent-level: 2 -*- %%% %%% %CopyrightBegin% %%% -%%% Copyright Ericsson AB 2001-2009. All Rights Reserved. +%%% Copyright Ericsson AB 2001-2012. All Rights Reserved. %%% %%% The contents of this file are subject to the Erlang Public License, %%% Version 1.1, (the "License"); you may not use this file except in @@ -28,7 +28,7 @@ %%%---------------------------------------------------------------------- -module(hipe_reg_worklists). --author(['Andreas Wallin', 'Thorild Sel�n']). +-author(['Andreas Wallin', 'Thorild Selén']). -export([new/5, % only used by optimistic allocator new/6, simplify/1, diff --git a/lib/hipe/rtl/hipe_rtl_arith.inc b/lib/hipe/rtl/hipe_rtl_arith.inc index e608506234..7b587e882d 100644 --- a/lib/hipe/rtl/hipe_rtl_arith.inc +++ b/lib/hipe/rtl/hipe_rtl_arith.inc @@ -1,9 +1,9 @@ %% -*- Erlang -*- -%% -*- erlang-indent-level: 2 -*- +%% -*- coding: utf-8; erlang-indent-level: 2 -*- %% %% %CopyrightBegin% %% -%% Copyright Ericsson AB 2004-2011. All Rights Reserved. +%% Copyright Ericsson AB 2004-2012. All Rights Reserved. %% %% The contents of this file are subject to the Erlang Public License, %% Version 1.1, (the "License"); you may not use this file except in @@ -114,7 +114,7 @@ eval_alu(Op, Arg1, Arg2) eval_alu(Op, Arg1, Arg2) -> ?EXIT({argument_overflow,Op,Arg1,Arg2}). -%% Bj�rn & Bjarni: +%% Björn & Bjarni: %% We need to be able to do evaluations based only on the bits, since %% there are cases where we can evaluate a subset of the bits, but can %% not do a full eval-alub call (eg. a + 0 gives no carry) diff --git a/lib/hipe/rtl/hipe_rtl_mk_switch.erl b/lib/hipe/rtl/hipe_rtl_mk_switch.erl index e5175217d6..d859c50b7d 100644 --- a/lib/hipe/rtl/hipe_rtl_mk_switch.erl +++ b/lib/hipe/rtl/hipe_rtl_mk_switch.erl @@ -1,8 +1,8 @@ -%% -*- erlang-indent-level: 2 -*- +%% -*- coding: utf-8; erlang-indent-level: 2 -*- %% %% %CopyrightBegin% %% -%% Copyright Ericsson AB 2001-2009. All Rights Reserved. +%% Copyright Ericsson AB 2001-2012. All Rights Reserved. %% %% The contents of this file are subject to the Erlang Public License, %% Version 1.1, (the "License"); you may not use this file except in @@ -31,7 +31,7 @@ %% History : * 2001-02-28 Erik Johansson ([email protected]): %% Created. %% * 2001-04-01 Erik Trulsson ([email protected]): -%% Stefan Lindstr�m ([email protected]): +%% Stefan Lindström ([email protected]): %% Added clustering and inlined binary search trees. %% * 2001-07-30 EJ ([email protected]): %% Fixed some bugs and started cleanup. diff --git a/lib/hipe/rtl/hipe_rtl_primops.erl b/lib/hipe/rtl/hipe_rtl_primops.erl index 53aaa72aa6..d9d08356ce 100644 --- a/lib/hipe/rtl/hipe_rtl_primops.erl +++ b/lib/hipe/rtl/hipe_rtl_primops.erl @@ -2,7 +2,7 @@ %% %% %CopyrightBegin% %% -%% Copyright Ericsson AB 2001-2011. All Rights Reserved. +%% Copyright Ericsson AB 2001-2012. All Rights Reserved. %% %% The contents of this file are subject to the Erlang Public License, %% Version 1.1, (the "License"); you may not use this file except in @@ -738,7 +738,7 @@ gen_mkfun([Dst], {_Mod, _FunId, _Arity} = MFidA, MagicNr, Index, FreeVars) -> %% Tag the thing and increase the heap_pointer. %% make_fun(funp); - WordSize�= hipe_rtl_arch:word_size(), + WordSize = hipe_rtl_arch:word_size(), HeapNeed = (?ERL_FUN_SIZE + NumFree) * WordSize, TagCode = [hipe_tagscheme:tag_fun(Dst, HP), %% AdjustHPCode @@ -829,7 +829,7 @@ load_struct_field(Dest, StructP, Offset, int32) -> gen_free_vars(Vars, HPReg) -> HPVar = hipe_rtl:mk_new_var(), - WordSize�= hipe_rtl_arch:word_size(), + WordSize = hipe_rtl_arch:word_size(), [hipe_rtl:mk_alu(HPVar, HPReg, add, hipe_rtl:mk_imm(?EFT_ENV)) | gen_free_vars(Vars, HPVar, 0, WordSize, [])]. diff --git a/lib/hipe/rtl/hipe_rtl_ssa_const_prop.erl b/lib/hipe/rtl/hipe_rtl_ssa_const_prop.erl index 194cf29b64..1c900d767e 100644 --- a/lib/hipe/rtl/hipe_rtl_ssa_const_prop.erl +++ b/lib/hipe/rtl/hipe_rtl_ssa_const_prop.erl @@ -1,8 +1,8 @@ -%% -*- erlang-indent-level: 2 -*- +%% -*- coding: utf-8; erlang-indent-level: 2 -*- %% %% %CopyrightBegin% %% -%% Copyright Ericsson AB 2004-2011. All Rights Reserved. +%% Copyright Ericsson AB 2004-2012. All Rights Reserved. %% %% The contents of this file are subject to the Erlang Public License, %% Version 1.1, (the "License"); you may not use this file except in @@ -190,7 +190,7 @@ set_to(Dst, Val, Env) -> %% Returns : { FlowWorkList, SSAWorkList, NewEnvironment} %%----------------------------------------------------------------------------- -visit_branch(Inst, Env) -> %% Titta ocks� p� exekverbarflagga +visit_branch(Inst, Env) -> %% Titta också på exekverbarflagga Val1 = lookup_lattice_value(hipe_rtl:branch_src1(Inst), Env), Val2 = lookup_lattice_value(hipe_rtl:branch_src2(Inst), Env), CFGWL = case evaluate_relop(Val1, hipe_rtl:branch_cond(Inst), Val2) of diff --git a/lib/hipe/rtl/hipe_rtl_ssapre.erl b/lib/hipe/rtl/hipe_rtl_ssapre.erl index a9e92e5688..34897ba4b7 100644 --- a/lib/hipe/rtl/hipe_rtl_ssapre.erl +++ b/lib/hipe/rtl/hipe_rtl_ssapre.erl @@ -1,8 +1,8 @@ -%% -*- erlang-indent-level: 2 -*- +%% -*- coding: utf-8; erlang-indent-level: 2 -*- %% %% %CopyrightBegin% %% -%% Copyright Ericsson AB 2005-2009. All Rights Reserved. +%% Copyright Ericsson AB 2005-2012. All Rights Reserved. %% %% The contents of this file are subject to the Erlang Public License, %% Version 1.1, (the "License"); you may not use this file except in @@ -19,7 +19,7 @@ %% %%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%% %% File : hipe_rtl_ssapre.erl -%% Author : He Bingwen and Fr�d�ric Haziza +%% Author : He Bingwen and Frédéric Haziza %% Description : Performs Partial Redundancy Elimination on SSA form. %%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%% %% @doc @@ -552,7 +552,7 @@ emend_with_processed_xsis(E, [I|Rest], Pred, XsiGraph) -> true -> %% It's a computation of E! case xsi_arg(Xsi,Pred) of undetermined_operand -> - exit({?MODULE,check_operand_sharing,"######## �h Dear, we trusted Kostis !!!!!!!!! #############"}); + exit({?MODULE,check_operand_sharing,"######## Ôh Dear, we trusted Kostis !!!!!!!!! #############"}); XsiOp -> {sharing_operand,XsiOp} %% They share operands end; @@ -571,7 +571,7 @@ emend_with_processed_xsis(E, [I|Rest], Pred, XsiGraph) -> NewE = emend(E,Def,A#eop.var), emend_with_processed_xsis(NewE,Rest,Pred,XsiGraph); undetermined_operand -> - exit({?MODULE,emend_with_processed_xsis,"######## �h Dear, we trusted Kostis, again !!!!!!!!! #############"}); + exit({?MODULE,emend_with_processed_xsis,"######## Ôh Dear, we trusted Kostis, again !!!!!!!!! #############"}); XsiOp -> NewE = emend(E,Def,XsiOp), emend_with_processed_xsis(NewE,Rest,Pred,XsiGraph) diff --git a/lib/hipe/ssa/hipe_ssa.inc b/lib/hipe/ssa/hipe_ssa.inc index d15b5ddd56..e766a83c41 100644 --- a/lib/hipe/ssa/hipe_ssa.inc +++ b/lib/hipe/ssa/hipe_ssa.inc @@ -1,8 +1,8 @@ -%% -*- erlang-indent-level: 2 -*- +%% -*- coding: utf-8; erlang-indent-level: 2 -*- %% %% %CopyrightBegin% %% -%% Copyright Ericsson AB 2002-2009. All Rights Reserved. +%% Copyright Ericsson AB 2002-2012. All Rights Reserved. %% %% The contents of this file are subject to the Erlang Public License, %% Version 1.1, (the "License"); you may not use this file except in @@ -19,7 +19,7 @@ %% %%---------------------------------------------------------------------- %% File : hipe_ssa.inc -%% Authors : Christoffer Vikstr�m, Daniel Deogun, and Jesper Bengtsson +%% Authors : Christoffer Vikström, Daniel Deogun, and Jesper Bengtsson %% Created : March 2002 %% Purpose : Provides code which converts the code of a CFG into SSA %% (Static Single Assignment) form and back. diff --git a/lib/hipe/vsn.mk b/lib/hipe/vsn.mk index 144167f5d1..f3e2e695b5 100644 --- a/lib/hipe/vsn.mk +++ b/lib/hipe/vsn.mk @@ -1 +1 @@ -HIPE_VSN = 3.9.2 +HIPE_VSN = 3.9.3 diff --git a/lib/hipe/x86/hipe_x86_assemble.erl b/lib/hipe/x86/hipe_x86_assemble.erl index 4e65736db3..7878c7219d 100644 --- a/lib/hipe/x86/hipe_x86_assemble.erl +++ b/lib/hipe/x86/hipe_x86_assemble.erl @@ -2,7 +2,7 @@ %%% %%% %CopyrightBegin% %%% -%%% Copyright Ericsson AB 2001-2009. All Rights Reserved. +%%% Copyright Ericsson AB 2001-2012. All Rights Reserved. %%% %%% The contents of this file are subject to the Erlang Public License, %%% Version 1.1, (the "License"); you may not use this file except in @@ -698,7 +698,7 @@ mem_to_ea_common(#x86_mem{base=#x86_temp{reg=Base}, off=#x86_imm{value=Off}}) -> %% jmp_switch -ifdef(HIPE_AMD64). -resolve_jmp_switch_arg(I,�_Context) -> +resolve_jmp_switch_arg(I, _Context) -> Base = hipe_x86:temp_reg(hipe_x86:jmp_switch_jtab(I)), Index = hipe_x86:temp_reg(hipe_x86:jmp_switch_temp(I)), SINDEX = hipe_amd64_encode:sindex(3, Index), diff --git a/lib/hipe/x86/hipe_x86_postpass.erl b/lib/hipe/x86/hipe_x86_postpass.erl index 34e3d7a11b..c0918c4f89 100644 --- a/lib/hipe/x86/hipe_x86_postpass.erl +++ b/lib/hipe/x86/hipe_x86_postpass.erl @@ -1,8 +1,8 @@ -%%% -*- erlang-indent-level: 2 -*- +%%% -*- coding: utf-8; erlang-indent-level: 2 -*- %%% %%% %CopyrightBegin% %%% -%%% Copyright Ericsson AB 2003-2009. All Rights Reserved. +%%% Copyright Ericsson AB 2003-2012. All Rights Reserved. %%% %%% The contents of this file are subject to the Erlang Public License, %%% Version 1.1, (the "License"); you may not use this file except in @@ -19,9 +19,9 @@ %%% %%%---------------------------------------------------------------------- %%% File : hipe_x86_postpass.erl -%%% Author : Christoffer Vikstr�m <[email protected]> +%%% Author : Christoffer Vikström <[email protected]> %%% Purpose : Contain postpass optimisations for x86-assembler code. -%%% Created : 5 Aug 2003 by Christoffer Vikstr�m <[email protected]> +%%% Created : 5 Aug 2003 by Christoffer Vikström <[email protected]> %%%---------------------------------------------------------------------- -ifndef(HIPE_X86_POSTPASS). diff --git a/lib/hipe/x86/hipe_x86_ra_postconditions.erl b/lib/hipe/x86/hipe_x86_ra_postconditions.erl index 0b70764daf..6d7e90df43 100644 --- a/lib/hipe/x86/hipe_x86_ra_postconditions.erl +++ b/lib/hipe/x86/hipe_x86_ra_postconditions.erl @@ -2,7 +2,7 @@ %% %% %CopyrightBegin% %% -%% Copyright Ericsson AB 2001-2009. All Rights Reserved. +%% Copyright Ericsson AB 2001-2012. All Rights Reserved. %% %% The contents of this file are subject to the Erlang Public License, %% Version 1.1, (the "License"); you may not use this file except in @@ -224,7 +224,7 @@ do_byte_move(Src0, Dst0, TempMap, Strategy) -> do_move64(I, TempMap, Strategy) -> #move64{dst=Dst} = I, - case�is_spilled(Dst, TempMap) of + case is_spilled(Dst, TempMap) of false -> {[I], false}; true -> diff --git a/lib/ic/test/java_client_erl_server_SUITE.erl b/lib/ic/test/java_client_erl_server_SUITE.erl index c2bec94697..9e49305c3c 100644 --- a/lib/ic/test/java_client_erl_server_SUITE.erl +++ b/lib/ic/test/java_client_erl_server_SUITE.erl @@ -62,9 +62,14 @@ init_per_suite(Config) when is_list(Config) -> case case code:priv_dir(jinterface) of {error,bad_name} -> false; - P -> - filelib:is_dir(P) - end + P -> + case filelib:wildcard(filename:join(P, "*.jar")) of + [_|_] -> + true; + [] -> + false + end + end of true -> case find_executable(["java"]) of diff --git a/lib/ic/test/java_client_erl_server_SUITE_data/Makefile.src b/lib/ic/test/java_client_erl_server_SUITE_data/Makefile.src index ac8f2e619f..3143ab427b 100644 --- a/lib/ic/test/java_client_erl_server_SUITE_data/Makefile.src +++ b/lib/ic/test/java_client_erl_server_SUITE_data/Makefile.src @@ -68,7 +68,7 @@ EBINS = $(ERL_FILES:.erl=.@EMULATOR@) @IFEQ@ (@jinterface_classpath@,) all: -@ELSE +@ELSE@ all: $(CLASS_FILES) $(EBINS) @ENDIF@ diff --git a/lib/inets/doc/src/http_uri.xml b/lib/inets/doc/src/http_uri.xml index bd31ae42d2..d9e8587bbf 100644 --- a/lib/inets/doc/src/http_uri.xml +++ b/lib/inets/doc/src/http_uri.xml @@ -4,7 +4,7 @@ <erlref> <header> <copyright> - <year>2012</year><year>2012</year> + <year>2012</year><year>2013</year> <holder>Ericsson AB. All Rights Reserved.</holder> </copyright> <legalnotice> @@ -55,7 +55,8 @@ string() = list of ASCII characters <p>For more information about URI, see RFC 3986. </p> <code type="none"><![CDATA[ -uri() = string() - Syntax according to the URI definition in rfc 3986, ex: "http://www.erlang.org/" +uri() = string() - Syntax according to the URI definition in rfc 3986, + e.g.: "http://www.erlang.org/" user_info() = string() scheme() = atom() - Example: http, https host() = string() diff --git a/lib/inets/doc/src/httpd.xml b/lib/inets/doc/src/httpd.xml index 8497d91549..3fced5dfcd 100644 --- a/lib/inets/doc/src/httpd.xml +++ b/lib/inets/doc/src/httpd.xml @@ -4,7 +4,7 @@ <erlref> <header> <copyright> - <year>1997</year><year>2011</year> + <year>1997</year><year>2012</year> <holder>Ericsson AB. All Rights Reserved.</holder> </copyright> <legalnotice> diff --git a/lib/inets/doc/src/notes.xml b/lib/inets/doc/src/notes.xml index 3aae1ff70a..e0d6ae3454 100644 --- a/lib/inets/doc/src/notes.xml +++ b/lib/inets/doc/src/notes.xml @@ -33,7 +33,27 @@ </header> - <section> + <section><title>Inets 5.9.2</title> + + <section><title>Improvements and New Features</title> + <list> + <item> + <p> + Minimum bytes per second</p> + <p> + New option to http server, {minimum_bytes_per_second, + integer()}, for a connection, if it is not reached the + socket will close for that specific connection. Can be + used to prevent hanging requests from faulty clients.</p> + <p> + Own Id: OTP-10392</p> + </item> + </list> + </section> + +</section> + +<section> <title>Inets 5.9.1</title> <section> diff --git a/lib/inets/src/http_server/httpd_request_handler.erl b/lib/inets/src/http_server/httpd_request_handler.erl index 5e0bd39cb3..0f47d785ef 100644 --- a/lib/inets/src/http_server/httpd_request_handler.erl +++ b/lib/inets/src/http_server/httpd_request_handler.erl @@ -1,7 +1,7 @@ %% %% %CopyrightBegin% %% -%% Copyright Ericsson AB 1997-2011. All Rights Reserved. +%% Copyright Ericsson AB 1997-2012. All Rights Reserved. %% %% The contents of this file are subject to the Erlang Public License, %% Version 1.1, (the "License"); you may not use this file except in diff --git a/lib/inets/test/erl_make_certs.erl b/lib/inets/test/erl_make_certs.erl index 254aa6d2f9..d6bdd05d01 100644 --- a/lib/inets/test/erl_make_certs.erl +++ b/lib/inets/test/erl_make_certs.erl @@ -137,10 +137,10 @@ decode_key(PemBin, Pw) -> encode_key(Key = #'RSAPrivateKey'{}) -> {ok, Der} = 'OTP-PUB-KEY':encode('RSAPrivateKey', Key), - {'RSAPrivateKey', list_to_binary(Der), not_encrypted}; + {'RSAPrivateKey', Der, not_encrypted}; encode_key(Key = #'DSAPrivateKey'{}) -> {ok, Der} = 'OTP-PUB-KEY':encode('DSAPrivateKey', Key), - {'DSAPrivateKey', list_to_binary(Der), not_encrypted}. + {'DSAPrivateKey', Der, not_encrypted}. make_tbs(SubjectKey, Opts) -> Version = list_to_atom("v"++integer_to_list(proplists:get_value(version, Opts, 3))), diff --git a/lib/inets/test/inets_app_test.erl b/lib/inets/test/inets_app_test.erl index d32f7e290b..eabfa69f7c 100644 --- a/lib/inets/test/inets_app_test.erl +++ b/lib/inets/test/inets_app_test.erl @@ -1,7 +1,7 @@ %% %% %CopyrightBegin% %% -%% Copyright Ericsson AB 2002-2011. All Rights Reserved. +%% Copyright Ericsson AB 2002-2012. All Rights Reserved. %% %% The contents of this file are subject to the Erlang Public License, %% Version 1.1, (the "License"); you may not use this file except in diff --git a/lib/jinterface/test/jitu.erl b/lib/jinterface/test/jitu.erl index fb262cf9d7..571a2dc9c7 100644 --- a/lib/jinterface/test/jitu.erl +++ b/lib/jinterface/test/jitu.erl @@ -1,7 +1,7 @@ %% %% %CopyrightBegin% %% -%% Copyright Ericsson AB 2004-2010. All Rights Reserved. +%% Copyright Ericsson AB 2004-2012. All Rights Reserved. %% %% The contents of this file are subject to the Erlang Public License, %% Version 1.1, (the "License"); you may not use this file except in diff --git a/lib/jinterface/test/nc_SUITE.erl b/lib/jinterface/test/nc_SUITE.erl index d5388e54f4..c91c743498 100644 --- a/lib/jinterface/test/nc_SUITE.erl +++ b/lib/jinterface/test/nc_SUITE.erl @@ -1,7 +1,8 @@ +%% -*- coding: utf-8 -*- %% %% %CopyrightBegin% %% -%% Copyright Ericsson AB 2004-2011. All Rights Reserved. +%% Copyright Ericsson AB 2004-2012. All Rights Reserved. %% %% The contents of this file are subject to the Erlang Public License, %% Version 1.1, (the "License"); you may not use this file except in @@ -361,8 +362,8 @@ unicode(doc) -> []; unicode(suite) -> []; unicode(Config) when is_list(Config) -> S1 = "plain ascii", - S2 = "iso-latin ��� �", - S3 = "Codepoints... ��� \x{1000}", + S2 = "iso-latin åäö ñ", + S3 = "Codepoints... åäö \x{1000}", S4 = [0,1,31,32,63,64,127,128,255], S5 = [0,1,127,128,255,256,16#d7ff, 16#e000,16#fffd,16#10000,16#10ffff], diff --git a/lib/kernel/doc/src/Makefile b/lib/kernel/doc/src/Makefile index 5e04bff0c1..de3ca1e176 100644 --- a/lib/kernel/doc/src/Makefile +++ b/lib/kernel/doc/src/Makefile @@ -58,7 +58,6 @@ XML_REF3_FILES = application.xml \ net_adm.xml \ net_kernel.xml \ os.xml \ - packages.xml \ pg2.xml \ rpc.xml \ seq_trace.xml \ diff --git a/lib/kernel/doc/src/file.xml b/lib/kernel/doc/src/file.xml index b2a259080d..536b98b5f5 100644 --- a/lib/kernel/doc/src/file.xml +++ b/lib/kernel/doc/src/file.xml @@ -110,7 +110,7 @@ <desc> <p>As returned by <seealso marker="#open/2">file:open/2</seealso>, - a process handling IO protocols.</p> + a process handling I/O-protocols.</p> </desc> </datatype> <datatype> @@ -261,6 +261,9 @@ {person, "pelle", 30}.</code> <pre>1> <input>file:consult("f.txt").</input> {ok,[{person,"kalle",25},{person,"pelle",30}]}</pre> + <p>The encoding of of <c><anno>Filename</anno></c> can be set + by a comment as described in <seealso + marker="stdlib:epp#encoding">epp(3)</seealso>.</p> </desc> </func> <func> @@ -399,6 +402,9 @@ of the error.</p> </item> </taglist> + <p>The encoding of of <c><anno>Filename</anno></c> can be set + by a comment as described in <seealso + marker="stdlib:epp#encoding">epp(3)</seealso>.</p> </desc> </func> <func> @@ -610,7 +616,7 @@ <name name="open" arity="2"/> <fsummary>Open a file</fsummary> <desc> - <p>Opens the file <c><anno>Filename</anno></c> in the mode determined + <p>Opens the file <c><anno>File</anno></c> in the mode determined by <c><anno>Modes</anno></c>, which may contain one or more of the following items:</p> <taglist> @@ -767,6 +773,10 @@ <p>The Encoding can be changed for a file "on the fly" by using the <seealso marker="stdlib:io#setopts/2">io:setopts/2</seealso> function, why a file can be analyzed in latin1 encoding for i.e. a BOM, positioned beyond the BOM and then be set for the right encoding before further reading.See the <seealso marker="stdlib:unicode">unicode(3)</seealso> module for functions identifying BOM's.</p> <p>This option is not allowed on <c>raw</c> files.</p> </item> + <tag><c>ram</c></tag> + <item> + <p><c>File</c> must be <c>iodata()</c>. Returns an <c>fd()</c> which lets the <c>file</c> module operate on the data in-memory as if it is a file.</p> + </item> </taglist> <p>Returns:</p> <taglist> @@ -861,6 +871,9 @@ the error.</p> </item> </taglist> + <p>The encoding of of <c><anno>Filename</anno></c> can be set + by a comment as described in <seealso + marker="stdlib:epp#encoding">epp(3)</seealso>.</p> </desc> </func> <func> @@ -902,6 +915,9 @@ of the error.</p> </item> </taglist> + <p>The encoding of of <c><anno>Filename</anno></c> can be set + by a comment as described in <seealso + marker="stdlib:epp#encoding">epp(3)</seealso>.</p> </desc> </func> <func> @@ -971,7 +987,10 @@ of the error.</p> </item> </taglist> - </desc> + <p>The encoding of of <c><anno>Filename</anno></c> can be set + by a comment as described in <seealso + marker="stdlib:epp#encoding">epp(3)</seealso>.</p> + </desc> </func> <func> <name name="path_script" arity="3"/> @@ -1502,6 +1521,9 @@ of the error.</p> </item> </taglist> + <p>The encoding of of <c><anno>Filename</anno></c> can be set + by a comment as described in <seealso + marker="stdlib:epp#encoding">epp(3)</seealso>.</p> </desc> </func> <func> diff --git a/lib/kernel/doc/src/heart.xml b/lib/kernel/doc/src/heart.xml index 2826d3d00a..2856d84dcf 100644 --- a/lib/kernel/doc/src/heart.xml +++ b/lib/kernel/doc/src/heart.xml @@ -4,7 +4,7 @@ <erlref> <header> <copyright> - <year>1996</year><year>2011</year> + <year>1996</year><year>2012</year> <holder>Ericsson AB. All Rights Reserved.</holder> </copyright> <legalnotice> diff --git a/lib/kernel/doc/src/inet.xml b/lib/kernel/doc/src/inet.xml index f498a88c5e..3d929a772e 100644 --- a/lib/kernel/doc/src/inet.xml +++ b/lib/kernel/doc/src/inet.xml @@ -323,8 +323,11 @@ fe80::204:acff:fe17:bf38 <type name="stat_option"/> <desc> <p>Gets one or more statistic options for a socket.</p> + <p><c>getstat(<anno>Socket</anno>)</c> is equivalent to - <c>getstat(<anno>Socket</anno>, [recv_avg, recv_cnt, recv_dvi, recv_max, recv_oct, send_avg, send_cnt, send_dvi, send_max, send_oct])</c></p> + <c>getstat(<anno>Socket</anno>, [recv_avg, recv_cnt, recv_dvi, + recv_max, recv_oct, send_avg, send_cnt, send_dvi, send_max, + send_oct])</c>.</p> <p>The following options are available:</p> <taglist> <tag><c>recv_avg</c></tag> @@ -371,7 +374,51 @@ fe80::204:acff:fe17:bf38 </taglist> </desc> </func> - + <func> + <name name="parse_ipv4_address" arity="1" /> + <fsummary>Parse an IPv4 address</fsummary> + <desc> + <p>Parses an IPv4 address string and returns an <a href="#type-ip4_address">ip4_address()</a>. + Accepts a shortened IPv4 shortened address string.</p> + </desc> + </func> + <func> + <name name="parse_ipv4strict_address" arity="1" /> + <fsummary>Parse an IPv4 address strict.</fsummary> + <desc> + <p>Parses an IPv4 address string containing four fields, i.e <b>not</b> shortened, and returns an <a href="#type-ip4_adress">ip4_address()</a>.</p> + </desc> + </func> + <func> + <name name="parse_ipv6_address" arity="1" /> + <fsummary>Parse an IPv6 address</fsummary> + <desc> + <p>Parses an IPv6 address string and returns an <a href="#type-ip6_address">ip6_address()</a>. + If an IPv4 address string is passed, an IPv4-mapped IPv6 address is returned.</p> + </desc> + </func> + <func> + <name name="parse_ipv6strict_address" arity="1" /> + <fsummary>Parse an IPv6 address strict.</fsummary> + <desc> + <p>Parses an IPv6 address string and returns an <a href="#type-ip6_address">ip6_address()</a>. + Does <b>not</b> accept IPv4 adresses.</p> + </desc> + </func> + <func> + <name name="parse_address" arity="1" /> + <fsummary>Parse an IPv4 or IPv6 address.</fsummary> + <desc> + <p>Parses an IPv4 or IPv6 address string and returns an <a href="#type-ip4_address">ip4_address()</a> or <a href="#type-ip6_address">ip6_address()</a>. Accepts a shortened IPv4 address string.</p> + </desc> + </func> + <func> + <name name="parse_strict_address" arity="1" /> + <fsummary>Parse an IPv4 or IPv6 address strict.</fsummary> + <desc> + <p>Parses an IPv4 or IPv6 address string and returns an <a href="#type-ip4_address">ip4_address()</a> or <a href="#type-ip6_adress">ip6_address()</a>. Does <b>not</b> accept a shortened IPv4 address string.</p> + </desc> + </func> <func> <name name="peername" arity="1"/> <fsummary>Return the address and port for the other end of a connection</fsummary> @@ -512,13 +559,37 @@ fe80::204:acff:fe17:bf38 <c>[Byte1,Byte2|Binary]</c>.</p> </item> - <tag><c>{high_watermark, Size}</c></tag> - <item> <p> - Sender is forced busy if sent and enqueued data - reaches the highwater mark. - <br /> Default: 8192 kB. - </p> - </item> + <tag><c>{high_msgq_watermark, Size}</c> (TCP/IP sockets)</tag> + <item> + <p>The socket message queue will be set into a busy + state when the amount of data queued on the message + queue reaches this limit. Note that this limit only + concerns data that have not yet reached the ERTS internal + socket implementation. Default value used is 8 kB.</p> + <p>Senders of data to the socket will be suspended if + either the socket message queue is busy, or the socket + itself is busy.</p> + <p>For more information see the <c>low_msgq_watermark</c>, + <c>high_watermark</c>, and <c>low_watermark</c> options.</p> + <p>Note that distribution sockets will disable the use of + <c>high_msgq_watermark</c> and <c>low_msgq_watermark</c>, + and will instead use the + <seealso marker="erts:erlang#system_info_dist_buf_busy_limit">distribution + buffer busy limit</seealso> which is a similar feature.</p> + </item> + + <tag><c>{high_watermark, Size}</c> (TCP/IP sockets)</tag> + <item> + <p>The socket will be set into a busy state when the amount + of data queued internally by the ERTS socket implementation + reaches this limit. Default value used is 8 kB.</p> + <p>Senders of data to the socket will be suspended if + either the socket message queue is busy, or the socket + itself is busy.</p> + <p>For more information see the <c>low_watermark</c>, + <c>high_msgq_watermark</c>, and <c>low_msqg_watermark</c> + options.</p> + </item> <tag><c>{ipv6_v6only, Boolean}</c></tag> <item> @@ -603,14 +674,41 @@ fe80::204:acff:fe17:bf38 the flushing time-out in seconds.</p> </item> - <tag><c>{low_watermark, Size}</c></tag> - <item> <p> - If the port has reached its <c>high_watermark</c> it will - force busy onto senders. When the port data queue reaches the - <c>low_watermark</c> callers are no longer forced busy. - <br /> Default: 4096 kB. - </p> - </item> + <tag><c>{low_msgq_watermark, Size}</c> (TCP/IP sockets)</tag> + <item> + <p>If the socket message queue is in a busy state, the + socket message queue will be set in a not busy state when + the amount of data queued in the message queue falls + below this limit. Note that this limit only concerns data + that have not yet reached the ERTS internal socket + implementation. Default value used is 4 kB.</p> + <p>Senders that have been suspended due to either a + busy message queue or a busy socket, will be resumed + when neither the socket message queue, nor the socket + are busy.</p> + <p>For more information see the <c>high_msgq_watermark</c>, + <c>high_watermark</c>, and <c>low_watermark</c> options.</p> + <p>Note that distribution sockets will disable the use of + <c>high_msgq_watermark</c> and <c>low_msgq_watermark</c>, + and will instead use the + <seealso marker="erts:erlang#system_info_dist_buf_busy_limit">distribution + buffer busy limit</seealso> which is a similar feature.</p> + </item> + + <tag><c>{low_watermark, Size}</c> (TCP/IP sockets)</tag> + <item> + <p>If the socket is in a busy state, the socket will + be set in a not busy state when the amount of data + queued internally by the ERTS socket implementation + falls below this limit. Default value used is 4 kB.</p> + <p>Senders that have been suspended due to either a + busy message queue or a busy socket, will be resumed + when neither the socket message queue, nor the socket + are busy.</p> + <p>For more information see the <c>high_watermark</c>, + <c>high_msgq_watermark</c>, and <c>low_msgq_watermark</c> + options.</p> + </item> <tag><c>{mode, Mode :: binary | list}</c></tag> <item> diff --git a/lib/kernel/doc/src/notes.xml b/lib/kernel/doc/src/notes.xml index 8e911a406b..78bc533464 100644 --- a/lib/kernel/doc/src/notes.xml +++ b/lib/kernel/doc/src/notes.xml @@ -30,6 +30,44 @@ </header> <p>This document describes the changes made to the Kernel application.</p> +<section><title>Kernel 2.15.3</title> + + <section><title>Fixed Bugs and Malfunctions</title> + <list> + <item> + <p> + Ensure 'erl_crash.dump' when asked for it. This will + change erl_crash.dump behaviour.</p> + <p> + * Not setting ERL_CRASH_DUMP_SECONDS will now terminate + beam immediately on a crash without writing a crash dump + file.</p> + <p> + * Setting ERL_CRASH_DUMP_SECONDS to 0 will also terminate + beam immediately on a crash without writing a crash dump + file, i.e. same as not setting ERL_CRASH_DUMP_SECONDS + environment variable.</p> + <p> + * Setting ERL_CRASH_DUMP_SECONDS to a negative value will + let the beam wait indefinitely on the crash dump file + being written.</p> + <p> + * Setting ERL_CRASH_DUMP_SECONDS to a positive value will + let the beam wait that many seconds on the crash dump + file being written.</p> + <p> + A positive value will set an alarm/timeout for restart + both in beam and in heart if heart is running.</p> + <p> + *** POTENTIAL INCOMPATIBILITY ***</p> + <p> + Own Id: OTP-10422 Aux Id: kunagi-250 [161] </p> + </item> + </list> + </section> + +</section> + <section><title>Kernel 2.15.2</title> <section><title>Fixed Bugs and Malfunctions</title> diff --git a/lib/kernel/src/Makefile b/lib/kernel/src/Makefile index c76ff9e2f0..60291bbce6 100644 --- a/lib/kernel/src/Makefile +++ b/lib/kernel/src/Makefile @@ -107,7 +107,6 @@ MODULES = \ net_adm \ net_kernel \ os \ - packages \ pg2 \ ram_file \ rpc \ diff --git a/lib/kernel/src/application_controller.erl b/lib/kernel/src/application_controller.erl index 68cd26ec10..75ce852001 100644 --- a/lib/kernel/src/application_controller.erl +++ b/lib/kernel/src/application_controller.erl @@ -1960,5 +1960,5 @@ to_string(Term) -> true -> Term; false -> - lists:flatten(io_lib:write(Term)) + lists:flatten(io_lib:format("~134217728p", [Term])) end. diff --git a/lib/kernel/src/code.erl b/lib/kernel/src/code.erl index c808ac7cb7..361f2bdf8a 100644 --- a/lib/kernel/src/code.erl +++ b/lib/kernel/src/code.erl @@ -359,7 +359,6 @@ load_code_server_prerequisites() -> hipe_unified_loader, lists, os, - packages, unicode], [M = M:module_info(module) || M <- Needed], ok. @@ -413,7 +412,7 @@ which(Module) when is_atom(Module) -> end. which2(Module) -> - Base = to_path(Module), + Base = atom_to_list(Module), File = filename:basename(Base) ++ objfile_extension(), Path = get_path(), which(File, filename:dirname(Base), Path). @@ -547,9 +546,6 @@ has_ext(Ext, Extlen, File) -> _ -> false end. -to_path(X) -> - filename:join(packages:split(X)). - -spec load_native_code_for_all_loaded() -> ok. load_native_code_for_all_loaded() -> Architecture = erlang:system_info(hipe_architecture), diff --git a/lib/kernel/src/code_server.erl b/lib/kernel/src/code_server.erl index 00ad923466..b2d2c19f78 100644 --- a/lib/kernel/src/code_server.erl +++ b/lib/kernel/src/code_server.erl @@ -1229,7 +1229,7 @@ load_abs(File, Mod0, Caller, St) -> end. try_load_module(Mod, Dir, Caller, St) -> - File = filename:append(Dir, to_path(Mod) ++ + File = filename:append(Dir, to_list(Mod) ++ objfile_extension()), case erl_prim_loader:get_file(File) of error -> @@ -1347,7 +1347,7 @@ load_file_1(Mod, Caller, #state{cache=Cache}=St0) -> end. mod_to_bin([Dir|Tail], Mod) -> - File = filename:append(Dir, to_path(Mod) ++ objfile_extension()), + File = filename:append(Dir, to_list(Mod) ++ objfile_extension()), case erl_prim_loader:get_file(File) of error -> mod_to_bin(Tail, Mod); @@ -1356,7 +1356,7 @@ mod_to_bin([Dir|Tail], Mod) -> end; mod_to_bin([], Mod) -> %% At last, try also erl_prim_loader's own method - File = to_path(Mod) ++ objfile_extension(), + File = to_list(Mod) ++ objfile_extension(), case erl_prim_loader:get_file(File) of error -> error; % No more alternatives ! @@ -1570,6 +1570,3 @@ to_list(X) when is_atom(X) -> atom_to_list(X). to_atom(X) when is_atom(X) -> X; to_atom(X) when is_list(X) -> list_to_atom(X). - -to_path(X) -> - filename:join(packages:split(X)). diff --git a/lib/kernel/src/dist_util.erl b/lib/kernel/src/dist_util.erl index f0d54a2f3e..e3511988a6 100644 --- a/lib/kernel/src/dist_util.erl +++ b/lib/kernel/src/dist_util.erl @@ -757,7 +757,8 @@ setup_timer(Pid, Timeout) -> end. reset_timer(Timer) -> - Timer ! {self(), reset}. + Timer ! {self(), reset}, + ok. cancel_timer(Timer) -> unlink(Timer), diff --git a/lib/kernel/src/file.erl b/lib/kernel/src/file.erl index 22af38c598..de3eaad5a1 100644 --- a/lib/kernel/src/file.erl +++ b/lib/kernel/src/file.erl @@ -397,9 +397,10 @@ raw_write_file_info(Name, #file_info{} = Info) -> %% Contemporary mode specification - list of options --spec open(Filename, Modes) -> {ok, IoDevice} | {error, Reason} when +-spec open(File, Modes) -> {ok, IoDevice} | {error, Reason} when + File :: Filename | iodata(), Filename :: name(), - Modes :: [mode()], + Modes :: [mode() | ram], IoDevice :: io_device(), Reason :: posix() | badarg | system_limit. @@ -1314,6 +1315,7 @@ sendfile_send(Sock, Data, Old) -> %%% Helpers consult_stream(Fd) -> + _ = epp:set_encoding(Fd), consult_stream(Fd, 1, []). consult_stream(Fd, Line, Acc) -> @@ -1327,6 +1329,7 @@ consult_stream(Fd, Line, Acc) -> end. eval_stream(Fd, Handling, Bs) -> + _ = epp:set_encoding(Fd), eval_stream(Fd, Handling, 1, undefined, [], Bs). eval_stream(Fd, H, Line, Last, E, Bs) -> diff --git a/lib/kernel/src/file_io_server.erl b/lib/kernel/src/file_io_server.erl index 0bff56cf46..acaffe1e41 100644 --- a/lib/kernel/src/file_io_server.erl +++ b/lib/kernel/src/file_io_server.erl @@ -40,6 +40,8 @@ format_error({_Line, ?MODULE, Reason}) -> io_lib:format("~w", [Reason]); format_error({_Line, Mod, Reason}) -> Mod:format_error(Reason); +format_error(invalid_unicode) -> + io_lib:format("cannot translate from UTF-8", []); format_error(ErrorId) -> erl_posix_msg:message(ErrorId). @@ -549,7 +551,7 @@ get_chars_notempty(Mod, Func, XtraArg, S, OutEnc, <<>> -> get_chars_apply(Mod, Func, XtraArg, S, OutEnc, State, eof); _ -> - {stop,invalid_unicode,{error,invalid_unicode},State} + {stop,invalid_unicode,invalid_unicode_error(Mod, Func, XtraArg, S),State} end; {error,Reason}=Error -> {stop,Reason,Error,State} @@ -616,12 +618,22 @@ get_chars_apply(Mod, Func, XtraArg, S0, OutEnc, end catch exit:ExReason -> - {stop,ExReason,{error,err_func(Mod, Func, XtraArg)},State}; + {stop,ExReason,invalid_unicode_error(Mod, Func, XtraArg, S0),State}; error:ErrReason -> {stop,ErrReason,{error,err_func(Mod, Func, XtraArg)},State} end. - +%% A hack that tries to inform the caller about the position where the +%% error occured. +invalid_unicode_error(Mod, Func, XtraArg, S) -> + try + {erl_scan,tokens,_Args} = XtraArg, + Location = erl_scan:continuation_location(S), + {error,{Location, ?MODULE, invalid_unicode},Location} + catch + _:_ -> + {error,err_func(Mod, Func, XtraArg)} + end. %% Convert error code to make it look as before err_func(io_lib, get_until, {_,F,_}) -> diff --git a/lib/kernel/src/gen_tcp.erl b/lib/kernel/src/gen_tcp.erl index 22e6aa5bc8..ec13ab6d2e 100644 --- a/lib/kernel/src/gen_tcp.erl +++ b/lib/kernel/src/gen_tcp.erl @@ -37,9 +37,11 @@ {dontroute, boolean()} | {exit_on_close, boolean()} | {header, non_neg_integer()} | + {high_msgq_watermark, pos_integer()} | {high_watermark, non_neg_integer()} | {keepalive, boolean()} | {linger, {boolean(), non_neg_integer()}} | + {low_msgq_watermark, pos_integer()} | {low_watermark, non_neg_integer()} | {mode, list | binary} | list | binary | {nodelay, boolean()} | @@ -67,9 +69,11 @@ dontroute | exit_on_close | header | + high_msgq_watermark | high_watermark | keepalive | linger | + low_msgq_watermark | low_watermark | mode | nodelay | diff --git a/lib/kernel/src/group.erl b/lib/kernel/src/group.erl index f92c6f7208..4d2e31a429 100644 --- a/lib/kernel/src/group.erl +++ b/lib/kernel/src/group.erl @@ -424,7 +424,7 @@ get_password_chars(Drv,Buf) -> end. get_chars(Prompt, M, F, Xa, Drv, Buf, Encoding) -> - Pbs = prompt_bytes(Prompt), + Pbs = prompt_bytes(Prompt, Encoding), get_chars_loop(Pbs, M, F, Xa, Drv, Buf, start, Encoding). get_chars_loop(Pbs, M, F, Xa, Drv, Buf0, State, Encoding) -> @@ -688,9 +688,9 @@ edit_password([Char|Cs],Chars) -> edit_password(Cs,[Char|Chars]). %% prompt_bytes(Prompt) -%% Return a flat list of bytes for the Prompt. -prompt_bytes(Prompt) -> - lists:flatten(io_lib:format_prompt(Prompt)). +%% Return a flat list of characters for the Prompt. +prompt_bytes(Prompt, Encoding) -> + lists:flatten(io_lib:format_prompt(Prompt, Encoding)). cast(L, binary,latin1) when is_list(L) -> list_to_binary(L); diff --git a/lib/kernel/src/heart.erl b/lib/kernel/src/heart.erl index de287bfa43..87cb9d7f51 100644 --- a/lib/kernel/src/heart.erl +++ b/lib/kernel/src/heart.erl @@ -46,6 +46,7 @@ -define(TIMEOUT, 5000). -define(CYCLE_TIMEOUT, 10000). +-define(HEART_PORT_NAME, heart_port). %%--------------------------------------------------------------------- @@ -132,7 +133,7 @@ start_portprogram() -> case wait_ack(Port) of ok -> %% register port so the vm can find it if need be - register(heart_port, Port), + register(?HEART_PORT_NAME, Port), {ok, Port}; {error, Reason} -> report_problem({{port_problem, Reason}, @@ -228,6 +229,7 @@ no_reboot_shutdown(Port) -> end. do_cycle_port_program(Caller, Parent, Port, Cmd) -> + unregister(?HEART_PORT_NAME), case catch start_portprogram() of {ok, NewPort} -> send_shutdown(Port), diff --git a/lib/kernel/src/inet.erl b/lib/kernel/src/inet.erl index 92c1802a86..9670271b2e 100644 --- a/lib/kernel/src/inet.erl +++ b/lib/kernel/src/inet.erl @@ -30,7 +30,9 @@ ifget/3, ifget/2, ifset/3, ifset/2, getstat/1, getstat/2, ip/1, stats/0, options/0, - pushf/3, popf/1, close/1, gethostname/0, gethostname/1]). + pushf/3, popf/1, close/1, gethostname/0, gethostname/1, + parse_ipv4_address/1, parse_ipv6_address/1, parse_ipv4strict_address/1, + parse_ipv6strict_address/1, parse_address/1, parse_strict_address/1]). -export([connect_options/2, listen_options/2, udp_options/2, sctp_options/2]). @@ -527,6 +529,48 @@ getservbyname(Name, Protocol) when is_atom(Name) -> Error -> Error end. +-spec parse_ipv4_address(Address) -> + {ok, IPv4Address} | {error, einval} when + Address :: string(), + IPv4Address :: ip_address(). +parse_ipv4_address(Addr) -> + inet_parse:ipv4_address(Addr). + +-spec parse_ipv6_address(Address) -> + {ok, IPv6Address} | {error, einval} when + Address :: string(), + IPv6Address :: ip_address(). +parse_ipv6_address(Addr) -> + inet_parse:ipv6_address(Addr). + +-spec parse_ipv4strict_address(Address) -> + {ok, IPv4Address} | {error, einval} when + Address :: string(), + IPv4Address :: ip_address(). +parse_ipv4strict_address(Addr) -> + inet_parse:ipv4strict_address(Addr). + +-spec parse_ipv6strict_address(Address) -> + {ok, IPv6Address} | {error, einval} when + Address :: string(), + IPv6Address :: ip_address(). +parse_ipv6strict_address(Addr) -> + inet_parse:ipv6strict_address(Addr). + +-spec parse_address(Address) -> + {ok, IPAddress} | {error, einval} when + Address :: string(), + IPAddress :: ip_address(). +parse_address(Addr) -> + inet_parse:address(Addr). + +-spec parse_strict_address(Address) -> + {ok, IPAddress} | {error, einval} when + Address :: string(), + IPAddress :: ip_address(). +parse_strict_address(Addr) -> + inet_parse:strict_address(Addr). + %% Return a list of available options options() -> [ @@ -535,6 +579,7 @@ options() -> buffer, header, active, packet, deliver, mode, multicast_if, multicast_ttl, multicast_loop, exit_on_close, high_watermark, low_watermark, + high_msgq_watermark, low_msgq_watermark, send_timeout, send_timeout_close ]. @@ -552,8 +597,8 @@ stats() -> connect_options() -> [tos, priority, reuseaddr, keepalive, linger, sndbuf, recbuf, nodelay, header, active, packet, packet_size, buffer, mode, deliver, - exit_on_close, high_watermark, low_watermark, send_timeout, - send_timeout_close, delay_send,raw]. + exit_on_close, high_watermark, low_watermark, high_msgq_watermark, + low_msgq_watermark, send_timeout, send_timeout_close, delay_send, raw]. connect_options(Opts, Family) -> BaseOpts = @@ -608,8 +653,9 @@ con_add(Name, Val, R, Opts, AllOpts) -> listen_options() -> [tos, priority, reuseaddr, keepalive, linger, sndbuf, recbuf, nodelay, header, active, packet, buffer, mode, deliver, backlog, ipv6_v6only, - exit_on_close, high_watermark, low_watermark, send_timeout, - send_timeout_close, delay_send, packet_size,raw]. + exit_on_close, high_watermark, low_watermark, high_msgq_watermark, + low_msgq_watermark, send_timeout, send_timeout_close, delay_send, + packet_size, raw]. listen_options(Opts, Family) -> BaseOpts = diff --git a/lib/kernel/src/inet_int.hrl b/lib/kernel/src/inet_int.hrl index 6d808b54cd..000119bc74 100644 --- a/lib/kernel/src/inet_int.hrl +++ b/lib/kernel/src/inet_int.hrl @@ -141,6 +141,8 @@ -define(INET_LOPT_READ_PACKETS, 33). -define(INET_OPT_RAW, 34). -define(INET_LOPT_TCP_SEND_TIMEOUT_CLOSE, 35). +-define(INET_LOPT_TCP_MSGQ_HIWTRMRK, 36). +-define(INET_LOPT_TCP_MSGQ_LOWTRMRK, 37). % Specific SCTP options: separate range: -define(SCTP_OPT_RTOINFO, 100). -define(SCTP_OPT_ASSOCINFO, 101). diff --git a/lib/kernel/src/inet_parse.erl b/lib/kernel/src/inet_parse.erl index ba62a59068..3551e701b6 100644 --- a/lib/kernel/src/inet_parse.erl +++ b/lib/kernel/src/inet_parse.erl @@ -36,7 +36,7 @@ -export([ipv4_address/1, ipv6_address/1]). -export([ipv4strict_address/1, ipv6strict_address/1]). --export([address/1]). +-export([address/1, strict_address/1]). -export([visible_string/1, domain/1]). -export([ntoa/1, dots/1]). -export([split_line/1]). @@ -456,6 +456,17 @@ address(Cs) when is_list(Cs) -> address(_) -> {error, einval}. +%%Parse ipv4 strict address or ipv6 strict address +strict_address(Cs) when is_list(Cs) -> + case ipv4strict_address(Cs) of + {ok,IP} -> + {ok,IP}; + _ -> + ipv6strict_address(Cs) + end; +strict_address(Cs) -> + {error, einval}. + %% %% Parse IPv4 address: %% d1.d2.d3.d4 diff --git a/lib/kernel/src/kernel.app.src b/lib/kernel/src/kernel.app.src index 17ab84c177..9a20baf8d0 100644 --- a/lib/kernel/src/kernel.app.src +++ b/lib/kernel/src/kernel.app.src @@ -28,7 +28,6 @@ application_starter, auth, code, - packages, code_server, dist_util, erl_boot_server, diff --git a/lib/kernel/src/packages.erl b/lib/kernel/src/packages.erl deleted file mode 100644 index e0b1f36b85..0000000000 --- a/lib/kernel/src/packages.erl +++ /dev/null @@ -1,158 +0,0 @@ -%% -%% %CopyrightBegin% -%% -%% Copyright Ericsson AB 2002-2009. All Rights Reserved. -%% -%% The contents of this file are subject to the Erlang Public License, -%% Version 1.1, (the "License"); you may not use this file except in -%% compliance with the License. You should have received a copy of the -%% Erlang Public License along with this software. If not, it can be -%% retrieved online at http://www.erlang.org/. -%% -%% Software distributed under the License is distributed on an "AS IS" -%% basis, WITHOUT WARRANTY OF ANY KIND, either express or implied. See -%% the License for the specific language governing rights and limitations -%% under the License. -%% -%% %CopyrightEnd% -%% --module(packages). - --export([to_string/1, concat/1, concat/2, is_valid/1, is_segmented/1, - split/1, last/1, first/1, strip_last/1, find_modules/1, - find_modules/2]). - -%% A package name (or a package-qualified module name) may be an atom or -%% a string (list of nonnegative integers) - not a deep list, and not a -%% list containing atoms. A name may be empty, but may not contain two -%% consecutive period (`.') characters or end with a period character. - --type package_name() :: atom() | string(). - --spec to_string(package_name()) -> string(). -to_string(Name) when is_atom(Name) -> - atom_to_list(Name); -to_string(Name) -> - Name. - -%% `concat' does not insert a leading period if the first segment is -%% empty. However, the result may contain leading, consecutive or -%% dangling period characters, if any of the segments after the first -%% are empty. Use 'is_valid' to check the result if necessary. - --spec concat(package_name(), package_name()) -> string(). -concat(A, B) -> - concat([A, B]). - --spec concat([package_name()]) -> string(). -concat([H | T]) when is_atom(H) -> - concat([atom_to_list(H) | T]); -concat(["" | T]) -> - concat_1(T); -concat(L) -> - concat_1(L). - -concat_1([H | T]) when is_atom(H) -> - concat_1([atom_to_list(H) | T]); -concat_1([H]) -> - H; -concat_1([H | T]) -> - H ++ "." ++ concat_1(T); -concat_1([]) -> - ""; -concat_1(Name) -> - erlang:error({badarg, Name}). - --spec is_valid(package_name()) -> boolean(). -is_valid(Name) when is_atom(Name) -> - is_valid_1(atom_to_list(Name)); -is_valid([$. | _]) -> - false; -is_valid(Name) -> - is_valid_1(Name). - -is_valid_1([$.]) -> false; -is_valid_1([$., $. | _]) -> false; -is_valid_1([H | T]) when is_integer(H), H >= 0 -> - is_valid_1(T); -is_valid_1([]) -> true; -is_valid_1(_) -> false. - --spec split(package_name()) -> [string()]. -split(Name) when is_atom(Name) -> - split_1(atom_to_list(Name), []); -split(Name) -> - split_1(Name, []). - -split_1([$. | T], Cs) -> - [lists:reverse(Cs) | split_1(T, [])]; -split_1([H | T], Cs) when is_integer(H), H >= 0 -> - split_1(T, [H | Cs]); -split_1([], Cs) -> - [lists:reverse(Cs)]; -split_1(_, _) -> - erlang:error(badarg). - -%% This is equivalent to testing if `split(Name)' yields a list of -%% length larger than one (i.e., if the name can be split into two or -%% more segments), but is cheaper. - --spec is_segmented(package_name()) -> boolean(). -is_segmented(Name) when is_atom(Name) -> - is_segmented_1(atom_to_list(Name)); -is_segmented(Name) -> - is_segmented_1(Name). - -is_segmented_1([$. | _]) -> true; -is_segmented_1([H | T]) when is_integer(H), H >= 0 -> - is_segmented_1(T); -is_segmented_1([]) -> false; -is_segmented_1(_) -> - erlang:error(badarg). - --spec last(package_name()) -> string(). -last(Name) -> - last_1(split(Name)). - -last_1([H]) -> H; -last_1([_ | T]) -> last_1(T). - --spec first(package_name()) -> [string()]. -first(Name) -> - first_1(split(Name)). - -first_1([H | T]) when T =/= [] -> [H | first_1(T)]; -first_1(_) -> []. - --spec strip_last(package_name()) -> string(). -strip_last(Name) -> - concat(first(Name)). - -%% This finds all modules available for a given package, using the -%% current code server search path. (There is no guarantee that the -%% modules are loadable; only that the object files exist.) - --spec find_modules(package_name()) -> [string()]. -find_modules(P) -> - find_modules(P, code:get_path()). - --spec find_modules(package_name(), [string()]) -> [string()]. -find_modules(P, Paths) -> - P1 = filename:join(packages:split(P)), - find_modules(P1, Paths, code:objfile_extension(), sets:new()). - -find_modules(P, [Path | Paths], Ext, S0) -> - case file:list_dir(filename:join(Path, P)) of - {ok, Fs} -> - Fs1 = [F || F <- Fs, filename:extension(F) =:= Ext], - S1 = lists:foldl(fun (F, S) -> - F1 = filename:rootname(F, Ext), - sets:add_element(F1, S) - end, - S0, Fs1), - find_modules(P, Paths, Ext, S1); - _ -> - find_modules(P, Paths, Ext, S0) - end; -find_modules(_P, [], _Ext, S) -> - sets:to_list(S). diff --git a/lib/kernel/src/user.erl b/lib/kernel/src/user.erl index 88f32df20b..d6449d9e5e 100644 --- a/lib/kernel/src/user.erl +++ b/lib/kernel/src/user.erl @@ -81,7 +81,7 @@ server(PortName,PortSettings) -> run(P) -> put(read_mode,list), - put(unicode,false), + put(encoding,latin1), case init:get_argument(noshell) of %% non-empty list -> noshell {ok, [_|_]} -> @@ -191,39 +191,27 @@ do_io_request(Req, From, ReplyAs, Port, Q0) -> %% New in R13B %% Encoding option (unicode/latin1) io_request({put_chars,unicode,Chars}, Port, Q) -> % Binary new in R9C - put_chars(wrap_characters_to_binary(Chars,unicode, - case get(unicode) of - true -> unicode; - _ -> latin1 - end), Port, Q); + put_chars(wrap_characters_to_binary(Chars,unicode, get(encoding)), Port, Q); io_request({put_chars,unicode,Mod,Func,Args}, Port, Q) -> Result = case catch apply(Mod,Func,Args) of Data when is_list(Data); is_binary(Data) -> - wrap_characters_to_binary(Data,unicode, - case get(unicode) of - true -> unicode; - _ -> latin1 - end); + wrap_characters_to_binary(Data,unicode,get(encoding)); Undef -> Undef end, put_chars(Result, Port, Q); io_request({put_chars,latin1,Chars}, Port, Q) -> % Binary new in R9C - Data = case get(unicode) of - true -> + Data = case get(encoding) of + unicode -> unicode:characters_to_binary(Chars,latin1,unicode); - false -> + latin1 -> erlang:iolist_to_binary(Chars) end, put_chars(Data, Port, Q); io_request({put_chars,latin1,Mod,Func,Args}, Port, Q) -> Result = case catch apply(Mod,Func,Args) of Data when is_list(Data); is_binary(Data) -> - unicode:characters_to_binary(Data,latin1, - case get(unicode) of - true -> unicode; - _ -> latin1 - end); + unicode:characters_to_binary(Data,latin1,get(encoding)); Undef -> Undef end, @@ -351,9 +339,9 @@ check_valid_opts(_) -> do_setopts(Opts, _Port, Q) -> case proplists:get_value(encoding,Opts) of Valid when Valid =:= unicode; Valid =:= utf8 -> - put(unicode,true); + put(encoding,unicode); latin1 -> - put(unicode,false); + put(encoding,latin1); undefined -> ok end, @@ -370,12 +358,7 @@ do_setopts(Opts, _Port, Q) -> getopts(_Port,Q) -> Bin = {binary, get(read_mode) =:= binary}, - Uni = {encoding, case get(unicode) of - true -> - unicode; - _ -> - latin1 - end}, + Uni = {encoding, get(encoding)}, {ok,[Bin,Uni],Q}. @@ -575,31 +558,32 @@ binrev(L, T) -> %% end %% end %% end. -%% get_chars(Prompt, Module, Function, XtraArg, Port, Queue) +%% get_chars(Prompt, Module, Function, XtraArg, Port, Queue, Encoding) %% Gets characters from the input port until the applied function %% returns {stop,Result,RestBuf}. Does not block output until input -%% has been received. +%% has been received. Encoding is the encoding of the data sent to +%% the client and to Function. %% Returns: %% {Status,Result,NewQueue} %% {exit,Reason} %% Entry function. -get_chars(Prompt, M, F, Xa, Port, Q, Fmt) -> +get_chars(Prompt, M, F, Xa, Port, Q, Enc) -> prompt(Port, Prompt), case {get(eof),queue:is_empty(Q)} of {true,true} -> {ok,eof,Q}; _ -> - get_chars(Prompt, M, F, Xa, Port, Q, start, Fmt) + get_chars(Prompt, M, F, Xa, Port, Q, start, Enc) end. %% First loop. Wait for port data. Respond to output requests. -get_chars(Prompt, M, F, Xa, Port, Q, State, Fmt) -> +get_chars(Prompt, M, F, Xa, Port, Q, State, Enc) -> case queue:is_empty(Q) of true -> receive {Port,{data,Bytes}} -> - get_chars_bytes(State, M, F, Xa, Port, Q, Bytes, Fmt); + get_chars_bytes(State, M, F, Xa, Port, Q, Bytes, Enc); {Port, eof} -> put(eof, true), {ok, eof, []}; @@ -610,41 +594,41 @@ get_chars(Prompt, M, F, Xa, Port, Q, State, Fmt) -> do_io_request(Req, From, ReplyAs, Port, queue:new()), %Keep Q over this call %% No prompt. - get_chars(Prompt, M, F, Xa, Port, Q, State, Fmt); + get_chars(Prompt, M, F, Xa, Port, Q, State, Enc); {io_request,From,ReplyAs,Request} when is_pid(From) -> get_chars_req(Prompt, M, F, Xa, Port, Q, State, - Request, From, ReplyAs, Fmt); + Request, From, ReplyAs, Enc); {'EXIT',From,What} when node(From) =:= node() -> {exit,What} end; false -> - get_chars_apply(State, M, F, Xa, Port, Q, Fmt) + get_chars_apply(State, M, F, Xa, Port, Q, Enc) end. get_chars_req(Prompt, M, F, XtraArg, Port, Q, State, - Req, From, ReplyAs, Fmt) -> + Req, From, ReplyAs, Enc) -> do_io_request(Req, From, ReplyAs, Port, queue:new()), %Keep Q over this call prompt(Port, Prompt), - get_chars(Prompt, M, F, XtraArg, Port, Q, State, Fmt). + get_chars(Prompt, M, F, XtraArg, Port, Q, State, Enc). %% Second loop. Pass data to client as long as it wants more. %% A ^G in data interrupts loop if 'noshell' is not undefined. -get_chars_bytes(State, M, F, Xa, Port, Q, Bytes, Fmt) -> +get_chars_bytes(State, M, F, Xa, Port, Q, Bytes, Enc) -> case get(shell) of noshell -> - get_chars_apply(State, M, F, Xa, Port, queue:snoc(Q, Bytes),Fmt); + get_chars_apply(State, M, F, Xa, Port, queue:snoc(Q, Bytes),Enc); _ -> case contains_ctrl_g_or_ctrl_c(Bytes) of false -> get_chars_apply(State, M, F, Xa, Port, - queue:snoc(Q, Bytes),Fmt); + queue:snoc(Q, Bytes),Enc); _ -> throw(new_shell) end end. -get_chars_apply(State0, M, F, Xa, Port, Q, Fmt) -> - case catch M:F(State0, cast(queue:head(Q),Fmt), Fmt, Xa) of +get_chars_apply(State0, M, F, Xa, Port, Q, Enc) -> + case catch M:F(State0, cast(queue:head(Q),Enc), Enc, Xa) of {stop,Result,<<>>} -> {ok,Result,queue:tail(Q)}; {stop,Result,[]} -> @@ -653,32 +637,32 @@ get_chars_apply(State0, M, F, Xa, Port, Q, Fmt) -> {ok,Result,queue:tail(Q)}; {stop,Result,Buf} -> {ok,Result,queue:cons(Buf, queue:tail(Q))}; - {'EXIT',_} -> + {'EXIT',_Why} -> {error,{error,err_func(M, F, Xa)},queue:new()}; State1 -> - get_chars_more(State1, M, F, Xa, Port, queue:tail(Q), Fmt) + get_chars_more(State1, M, F, Xa, Port, queue:tail(Q), Enc) end. -get_chars_more(State, M, F, Xa, Port, Q, Fmt) -> +get_chars_more(State, M, F, Xa, Port, Q, Enc) -> case queue:is_empty(Q) of true -> case get(eof) of undefined -> receive {Port,{data,Bytes}} -> - get_chars_bytes(State, M, F, Xa, Port, Q, Bytes, Fmt); + get_chars_bytes(State, M, F, Xa, Port, Q, Bytes, Enc); {Port,eof} -> put(eof, true), get_chars_apply(State, M, F, Xa, Port, - queue:snoc(Q, eof), Fmt); + queue:snoc(Q, eof), Enc); {'EXIT',From,What} when node(From) =:= node() -> {exit,What} end; _ -> - get_chars_apply(State, M, F, Xa, Port, queue:snoc(Q, eof), Fmt) + get_chars_apply(State, M, F, Xa, Port, queue:snoc(Q, eof), Enc) end; false -> - get_chars_apply(State, M, F, Xa, Port, Q, Fmt) + get_chars_apply(State, M, F, Xa, Port, Q, Enc) end. @@ -689,11 +673,10 @@ get_chars_more(State, M, F, Xa, Port, Q, Fmt) -> prompt(_Port, '') -> ok; prompt(Port, Prompt) -> - put_port(wrap_characters_to_binary(io_lib:format_prompt(Prompt),unicode, - case get(unicode) of - true -> unicode; - _ -> latin1 - end), Port). + Encoding = get(encoding), + put_port(wrap_characters_to_binary(io_lib:format_prompt(Prompt, Encoding), + unicode, Encoding), + Port). %% Convert error code to make it look as before err_func(io_lib, get_until, {_,F,_}) -> @@ -710,56 +693,65 @@ contains_ctrl_g_or_ctrl_c(BinOrList)-> end. %% Convert a buffer between list and binary -cast(Data, _Format) when is_atom(Data) -> +cast(Data, _Encoding) when is_atom(Data) -> Data; -cast(Data, Format) -> - cast(Data, get(read_mode), Format, get(unicode)). +cast(Data, Encoding) -> + IoEncoding = get(encoding), + cast(Data, get(read_mode), IoEncoding, Encoding). -cast(B, binary, latin1, false) when is_binary(B) -> +cast(B, binary, latin1, latin1) when is_binary(B) -> B; -cast(B, binary, latin1, true) when is_binary(B) -> - unicode:characters_to_binary(B, unicode, latin1); -cast(L, binary, latin1, false) -> - erlang:iolist_to_binary(L); -cast(L, binary, latin1, true) -> - case unicode:characters_to_binary( - erlang:iolist_to_binary(L),unicode,latin1) of % may fail - {error,_,_} -> exit({no_translation, unicode, latin1}); - Else -> Else +cast(L, binary, latin1, latin1) -> + case catch erlang:iolist_to_binary(L) of + Bin when is_binary(Bin) -> Bin; + _ -> exit({no_translation, latin1, latin1}) + end; +cast(Data, binary, unicode, latin1) when is_binary(Data); is_list(Data) -> + case catch unicode:characters_to_binary(Data, unicode, latin1) of + Bin when is_binary(Bin) -> Bin; + _ -> exit({no_translation, unicode, latin1}) + end; +cast(Data, binary, latin1, unicode) when is_binary(Data); is_list(Data) -> + case catch unicode:characters_to_binary(Data, latin1, unicode) of + Bin when is_binary(Bin) -> Bin; + _ -> exit({no_translation, latin1, unicode}) end; -cast(B, binary, unicode, true) when is_binary(B) -> +cast(B, binary, unicode, unicode) when is_binary(B) -> B; -cast(B, binary, unicode, false) when is_binary(B) -> - unicode:characters_to_binary(B,latin1,unicode); -cast(L, binary, unicode, true) -> - % possibly a list containing UTF-8 encoded characters - unicode:characters_to_binary(erlang:iolist_to_binary(L)); -cast(L, binary, unicode, false) -> - unicode:characters_to_binary(L, latin1, unicode); -cast(L, list, latin1, UniTerm) -> - case UniTerm of - true -> % Convert input characters to protocol format (i.e latin1) - case unicode:characters_to_list( - erlang:iolist_to_binary(L),unicode) of % may fail - {error,_,_} -> exit({no_translation, unicode, latin1}); - Else -> [ case X of - High when High > 255 -> - exit({no_translation, unicode, latin1}); - Low -> - Low - end || X <- Else ] - end; - _ -> - binary_to_list(erlang:iolist_to_binary(L)) +cast(L, binary, unicode, unicode) -> + case catch unicode:characters_to_binary(L, unicode) of + Bin when is_binary(Bin) -> Bin; + _ -> exit({no_translation, unicode, unicode}) end; -cast(L, list, unicode, UniTerm) -> - unicode:characters_to_list(erlang:iolist_to_binary(L), - case UniTerm of - true -> unicode; - _ -> latin1 - end); -cast(Other, _, _,_) -> - Other. +cast(B, list, latin1, latin1) when is_binary(B) -> + binary_to_list(B); +cast(L, list, latin1, latin1) -> + case catch erlang:iolist_to_binary(L) of + Bin when is_binary(Bin) -> binary_to_list(Bin); + _ -> exit({no_translation, latin1, latin1}) + end; +cast(Data, list, unicode, latin1) when is_binary(Data); is_list(Data) -> + case catch unicode:characters_to_list(Data, unicode) of + Chars when is_list(Chars) -> + [ case X of + High when High > 255 -> + exit({no_translation, unicode, latin1}); + Low -> + Low + end || X <- Chars ]; + _ -> + exit({no_translation, unicode, latin1}) + end; +cast(Data, list, latin1, unicode) when is_binary(Data); is_list(Data) -> + case catch unicode:characters_to_list(Data, latin1) of + Chars when is_list(Chars) -> Chars; + _ -> exit({no_translation, latin1, unicode}) + end; +cast(Data, list, unicode, unicode) when is_binary(Data); is_list(Data) -> + case catch unicode:characters_to_list(Data, unicode) of + Chars when is_list(Chars) -> Chars; + _ -> exit({no_translation, unicode, unicode}) + end. wrap_characters_to_binary(Chars,unicode,latin1) -> case unicode:characters_to_binary(Chars,unicode,latin1) of diff --git a/lib/kernel/test/Makefile b/lib/kernel/test/Makefile index 8eca37029d..7fd3afe93c 100644 --- a/lib/kernel/test/Makefile +++ b/lib/kernel/test/Makefile @@ -73,6 +73,7 @@ MODULES= \ seq_trace_SUITE \ wrap_log_reader_SUITE \ cleanup \ + ignore_cores \ zlib_SUITE \ loose_node \ sendfile_SUITE diff --git a/lib/kernel/test/error_logger_warn_SUITE.erl b/lib/kernel/test/error_logger_warn_SUITE.erl index 265e1ae4c8..2bf467610e 100644 --- a/lib/kernel/test/error_logger_warn_SUITE.erl +++ b/lib/kernel/test/error_logger_warn_SUITE.erl @@ -1,7 +1,7 @@ %% %% %CopyrightBegin% %% -%% Copyright Ericsson AB 2003-2011. All Rights Reserved. +%% Copyright Ericsson AB 2003-2012. All Rights Reserved. %% %% The contents of this file are subject to the Erlang Public License, %% Version 1.1, (the "License"); you may not use this file except in @@ -310,7 +310,7 @@ nice_stop_node(Name) -> {nodedown,Name} -> ok end. -%rensa rd() f�re varje rapport-test s� man bara f�r en fil... +%clean out rd() before each report test in order to get only one file... clean_rd() -> {ok,L} = file:list_dir(rd()), lists:foreach(fun(F) -> diff --git a/lib/kernel/test/file_name_SUITE.erl b/lib/kernel/test/file_name_SUITE.erl index 3aa010a708..40bde8a736 100644 --- a/lib/kernel/test/file_name_SUITE.erl +++ b/lib/kernel/test/file_name_SUITE.erl @@ -1,4 +1,5 @@ -module(file_name_SUITE). +%% -*- coding: utf-8 -*- %% %% %CopyrightBegin% %% @@ -336,7 +337,7 @@ check_normal(Mod) -> check_icky(Mod) -> {ok,Dir} = Mod:get_cwd(), try - ?line true=(length("���") =:= 3), + ?line true=(length("åäö") =:= 3), ?line UniMode = file:native_name_encoding() =/= latin1, ?line make_icky_dir(Mod), ?line {ok, L0} = Mod:list_dir("."), @@ -357,42 +358,42 @@ check_icky(Mod) -> || {T,S,Targ} <- icky_dir(), T =:= symlink ], ?line [ {ok, Cont} = Mod:read_file(SymL) || {SymL,_,Cont} <- Syms ], ?line [ {ok, Targ} = fixlink(Mod:read_link(SymL)) || {SymL,Targ,_} <- Syms ], - ?line chk_cre_dir(Mod,[{directory,"���_dir",icky_dir()}]), + ?line chk_cre_dir(Mod,[{directory,"åäö_dir",icky_dir()}]), ?line {ok,BeginAt} = Mod:get_cwd(), ?line true = is_list(BeginAt), - ?line {error,enoent} = Mod:set_cwd("��_dir"), - ?line ok = Mod:set_cwd("���_dir"), + ?line {error,enoent} = Mod:set_cwd("åä_dir"), + ?line ok = Mod:set_cwd("åäö_dir"), ?line {ok, NowAt} = Mod:get_cwd(), ?line true = is_list(NowAt), ?line true = BeginAt =/= NowAt, ?line ok = Mod:set_cwd(".."), ?line {ok,BeginAt} = Mod:get_cwd(), - ?line rm_r2(Mod,"���_dir"), + ?line rm_r2(Mod,"åäö_dir"), {OS,TYPE} = os:type(), % Check that treat_icky really converts to the same as the OS case UniMode of true -> - ?line chk_cre_dir(Mod,[{directory,"���_dir",[]}]), - ?line ok = Mod:set_cwd("���_dir"), - ?line ok = Mod:write_file(<<"���">>,<<"hello">>), - ?line Treated = treat_icky(<<"���">>), + ?line chk_cre_dir(Mod,[{directory,"åäö_dir",[]}]), + ?line ok = Mod:set_cwd("åäö_dir"), + ?line ok = Mod:write_file(<<"ååå">>,<<"hello">>), + ?line Treated = treat_icky(<<"ååå">>), ?line {ok,[Treated]} = Mod:list_dir("."), - ?line ok = Mod:delete(<<"���">>), + ?line ok = Mod:delete(<<"ååå">>), ?line {ok,[]} = Mod:list_dir("."), ?line ok = Mod:set_cwd(".."), - ?line rm_r2(Mod,"���_dir"); + ?line rm_r2(Mod,"åäö_dir"); false -> ok end, - ?line chk_cre_dir(Mod,[{directory,treat_icky(<<"���_dir">>),icky_dir()}]), + ?line chk_cre_dir(Mod,[{directory,treat_icky(<<"åäö_dir">>),icky_dir()}]), if UniMode and (OS =/= win32) -> - ?line {error,enoent} = Mod:set_cwd("���_dir"); + ?line {error,enoent} = Mod:set_cwd("åäö_dir"); true -> ok end, - ?line ok = Mod:set_cwd(treat_icky(<<"���_dir">>)), + ?line ok = Mod:set_cwd(treat_icky(<<"åäö_dir">>)), ?line {ok, NowAt2} = Mod:get_cwd(), io:format("~p~n",[NowAt2]), % Cannot create raw unicode-breaking filenames on windows or macos @@ -400,22 +401,22 @@ check_icky(Mod) -> ?line true = BeginAt =/= NowAt2, ?line ok = Mod:set_cwd(".."), ?line {ok,BeginAt} = Mod:get_cwd(), - ?line rm_r2(Mod,conv(treat_icky(<<"���_dir">>))), + ?line rm_r2(Mod,conv(treat_icky(<<"åäö_dir">>))), case has_links() of true -> - ?line ok = Mod:make_link("fil1","nisse�"), - ?line {ok, <<"fil1">>} = Mod:read_file("nisse�"), - ?line {ok, #file_info{type = regular}} = Mod:read_link_info("nisse�"), - ?line ok = Mod:delete("nisse�"), - ?line ok = Mod:make_link("fil1",treat_icky(<<"nisse�">>)), - ?line {ok, <<"fil1">>} = Mod:read_file(treat_icky(<<"nisse�">>)), - ?line {ok, #file_info{type = regular}} = Mod:read_link_info(treat_icky(<<"nisse�">>)), - ?line ok = Mod:delete(treat_icky(<<"nisse�">>)), + ?line ok = Mod:make_link("fil1","nisseö"), + ?line {ok, <<"fil1">>} = Mod:read_file("nisseö"), + ?line {ok, #file_info{type = regular}} = Mod:read_link_info("nisseö"), + ?line ok = Mod:delete("nisseö"), + ?line ok = Mod:make_link("fil1",treat_icky(<<"nisseö">>)), + ?line {ok, <<"fil1">>} = Mod:read_file(treat_icky(<<"nisseö">>)), + ?line {ok, #file_info{type = regular}} = Mod:read_link_info(treat_icky(<<"nisseö">>)), + ?line ok = Mod:delete(treat_icky(<<"nisseö">>)), ?line {ok, <<"fil1">>} = Mod:read_file("fil1"), - ?line {error,enoent} = Mod:read_file("nisse�"), - ?line {error,enoent} = Mod:read_link_info("nisse�"), - ?line {error,enoent} = Mod:read_file(treat_icky(<<"nisse�">>)), - ?line {error,enoent} = Mod:read_link_info(treat_icky(<<"nisse�">>)); + ?line {error,enoent} = Mod:read_file("nisseö"), + ?line {error,enoent} = Mod:read_link_info("nisseö"), + ?line {error,enoent} = Mod:read_file(treat_icky(<<"nisseö">>)), + ?line {error,enoent} = Mod:read_link_info(treat_icky(<<"nisseö">>)); false -> ok end, @@ -430,42 +431,42 @@ check_icky(Mod) -> ?line {ok, BC} = Mod:read(FD,1024), ?line ok = file:close(FD) end || {regular,Name,Content} <- icky_dir() ], - ?line Mod:rename("���2","���_fil1"), - ?line {ok, <<"���2">>} = Mod:read_file("���_fil1"), - ?line {error,enoent} = Mod:read_file("���2"), - ?line Mod:rename("���_fil1","���2"), - ?line {ok, <<"���2">>} = Mod:read_file("���2"), - ?line {error,enoent} = Mod:read_file("���_fil1"), + ?line Mod:rename("åäö2","åäö_fil1"), + ?line {ok, <<"åäö2">>} = Mod:read_file("åäö_fil1"), + ?line {error,enoent} = Mod:read_file("åäö2"), + ?line Mod:rename("åäö_fil1","åäö2"), + ?line {ok, <<"åäö2">>} = Mod:read_file("åäö2"), + ?line {error,enoent} = Mod:read_file("åäö_fil1"), - ?line Mod:rename("���2",treat_icky(<<"���_fil1">>)), - ?line {ok, <<"���2">>} = Mod:read_file(treat_icky(<<"���_fil1">>)), + ?line Mod:rename("åäö2",treat_icky(<<"åäö_fil1">>)), + ?line {ok, <<"åäö2">>} = Mod:read_file(treat_icky(<<"åäö_fil1">>)), if UniMode and (OS =/= win32) -> - {error,enoent} = Mod:read_file("���_fil1"); + {error,enoent} = Mod:read_file("åäö_fil1"); true -> ok end, - ?line {error,enoent} = Mod:read_file("���2"), - ?line Mod:rename(treat_icky(<<"���_fil1">>),"���2"), - ?line {ok, <<"���2">>} = Mod:read_file("���2"), - ?line {error,enoent} = Mod:read_file("���_fil1"), - ?line {error,enoent} = Mod:read_file(treat_icky(<<"���_fil1">>)), + ?line {error,enoent} = Mod:read_file("åäö2"), + ?line Mod:rename(treat_icky(<<"åäö_fil1">>),"åäö2"), + ?line {ok, <<"åäö2">>} = Mod:read_file("åäö2"), + ?line {error,enoent} = Mod:read_file("åäö_fil1"), + ?line {error,enoent} = Mod:read_file(treat_icky(<<"åäö_fil1">>)), - ?line {ok,FI} = Mod:read_file_info("���2"), + ?line {ok,FI} = Mod:read_file_info("åäö2"), ?line NewMode = FI#file_info.mode band (bnot 8#333), ?line NewMode2 = NewMode bor 8#222, ?line true = NewMode2 =/= NewMode, - ?line ok = Mod:write_file_info("���2",FI#file_info{mode = NewMode}), - ?line {ok,#file_info{mode = NewMode}} = Mod:read_file_info("���2"), - ?line ok = Mod:write_file_info("���2",FI#file_info{mode = NewMode2}), - ?line {ok,#file_info{mode = NewMode2}} = Mod:read_file_info("���2"), + ?line ok = Mod:write_file_info("åäö2",FI#file_info{mode = NewMode}), + ?line {ok,#file_info{mode = NewMode}} = Mod:read_file_info("åäö2"), + ?line ok = Mod:write_file_info("åäö2",FI#file_info{mode = NewMode2}), + ?line {ok,#file_info{mode = NewMode2}} = Mod:read_file_info("åäö2"), - ?line {ok,FII} = Mod:read_file_info(treat_icky(<<"���5">>)), + ?line {ok,FII} = Mod:read_file_info(treat_icky(<<"åäö5">>)), ?line true = NewMode2 =/= NewMode, - ?line ok = Mod:write_file_info(treat_icky(<<"���5">>),FII#file_info{mode = NewMode}), - ?line {ok,#file_info{mode = NewMode}} = Mod:read_file_info(treat_icky(<<"���5">>)), - ?line ok = Mod:write_file_info(<<"���5">>,FII#file_info{mode = NewMode2}), - ?line {ok,#file_info{mode = NewMode2}} = Mod:read_file_info(treat_icky(<<"���5">>)), + ?line ok = Mod:write_file_info(treat_icky(<<"åäö5">>),FII#file_info{mode = NewMode}), + ?line {ok,#file_info{mode = NewMode}} = Mod:read_file_info(treat_icky(<<"åäö5">>)), + ?line ok = Mod:write_file_info(<<"åäö5">>,FII#file_info{mode = NewMode2}), + ?line {ok,#file_info{mode = NewMode2}} = Mod:read_file_info(treat_icky(<<"åäö5">>)), ok after Mod:set_cwd(Dir), @@ -475,7 +476,7 @@ check_icky(Mod) -> check_very_icky(Mod) -> {ok,Dir} = Mod:get_cwd(), try - ?line true=(length("���") =:= 3), + ?line true=(length("åäö") =:= 3), ?line UniMode = file:native_name_encoding() =/= latin1, if not UniMode -> @@ -497,7 +498,7 @@ check_very_icky(Mod) -> ?line chk_cre_dir(Mod,[{directory,[1088,1079,1091]++"_dir",very_icky_dir()}]), ?line {ok,BeginAt} = Mod:get_cwd(), ?line true = is_list(BeginAt), - ?line {error,enoent} = Mod:set_cwd("��_dir"), + ?line {error,enoent} = Mod:set_cwd("åä_dir"), ?line ok = Mod:set_cwd([1088,1079,1091]++"_dir"), ?line {ok, NowAt} = Mod:get_cwd(), ?line true = is_list(NowAt), @@ -514,16 +515,16 @@ check_very_icky(Mod) -> ?line {ok, #file_info{type = regular}} = Mod:read_link_info("nisse"++[1088,1079,1091]), ?line ok = Mod:delete("nisse"++[1088,1079,1091]), - ?line ok = Mod:make_link("fil1",<<"nisse�">>), - ?line {ok, <<"fil1">>} = Mod:read_file(<<"nisse�">>), + ?line ok = Mod:make_link("fil1",<<"nisseö">>), + ?line {ok, <<"fil1">>} = Mod:read_file(<<"nisseö">>), ?line {ok, #file_info{type = regular}} = - Mod:read_link_info(<<"nisse�">>), - ?line ok = Mod:delete(<<"nisse�">>), + Mod:read_link_info(<<"nisseö">>), + ?line ok = Mod:delete(<<"nisseö">>), ?line {ok, <<"fil1">>} = Mod:read_file("fil1"), ?line {error,enoent} = Mod:read_file("nisse"++[1088,1079,1091]), ?line {error,enoent} = Mod:read_link_info("nisse"++[1088,1079,1091]), - ?line {error,enoent} = Mod:read_file(<<"nisse�">>), - ?line {error,enoent} = Mod:read_link_info(<<"nisse�">>); + ?line {error,enoent} = Mod:read_file(<<"nisseö">>), + ?line {error,enoent} = Mod:read_link_info(<<"nisseö">>); false -> ok end, @@ -540,10 +541,10 @@ check_very_icky(Mod) -> end || {regular,Name,Content} <- very_icky_dir() ], ?line Mod:rename([956,965,963,954,959,49], [956,965,963,954,959]++"_fil1"), - ?line {ok, <<"���2">>} = Mod:read_file([956,965,963,954,959]++"_fil1"), + ?line {ok, <<"åäö2">>} = Mod:read_file([956,965,963,954,959]++"_fil1"), ?line {error,enoent} = Mod:read_file([956,965,963,954,959,49]), ?line Mod:rename([956,965,963,954,959]++"_fil1",[956,965,963,954,959,49]), - ?line {ok, <<"���2">>} = Mod:read_file([956,965,963,954,959,49]), + ?line {ok, <<"åäö2">>} = Mod:read_file([956,965,963,954,959,49]), ?line {error,enoent} = Mod:read_file([956,965,963,954,959]++"_fil1"), ?line {ok,FI} = Mod:read_file_info([956,965,963,954,959,49]), @@ -574,9 +575,9 @@ check_very_icky(Mod) -> end, ?line {NumOK,NumNOK} = filelib:fold_files(".",".*",true,fun(_F,{N,M}) when is_list(_F) -> io:format("~ts~n",[_F]),{N+1,M}; (_F,{N,M}) -> io:format("~p~n",[_F]),{N,M+1} end,{0,0}), ?line ok = filelib:fold_files(".",[1076,1089,1072,124,46,42],true,fun(_F,_) -> ok end,false), - ?line SF3 = unicode:characters_to_binary("���subfil3", + ?line SF3 = unicode:characters_to_binary("åäösubfil3", file:native_name_encoding()), - ?line SF2 = case treat_icky(<<"���subfil2">>) of + ?line SF2 = case treat_icky(<<"åäösubfil2">>) of LF2 when is_list(LF2) -> unicode:characters_to_binary(LF2, file:native_name_encoding()); @@ -584,7 +585,7 @@ check_very_icky(Mod) -> BF2 end, ?line Sorted = lists:sort([SF3,SF2]), - ?line Sorted = lists:sort(filelib:wildcard("*",<<"���subdir2">>)), + ?line Sorted = lists:sort(filelib:wildcard("*",<<"åäösubdir2">>)), ok catch throw:need_unicode_mode -> @@ -744,26 +745,26 @@ hopeless_darwin() -> icky_dir() -> [{regular,"fil1","fil1"}, - {regular,"���2","���2"}] ++ + {regular,"åäö2","åäö2"}] ++ case has_links() of true -> - [{regular,"���3","���2"}, - {symlink,"���4","���2"}]; + [{regular,"åäö3","åäö2"}, + {symlink,"åäö4","åäö2"}]; false -> [] end ++ - [{regular,treat_icky(<<"���5">>),"���5"}] ++ + [{regular,treat_icky(<<"åäö5">>),"åäö5"}] ++ case has_links() of true -> - [{symlink,treat_icky(<<"���6">>),treat_icky(<<"���5">>)}]; + [{symlink,treat_icky(<<"åäö6">>),treat_icky(<<"åäö5">>)}]; false -> [] end ++ - [{directory,treat_icky(<<"���subdir2">>), - [{regular,treat_icky(<<"���subfil2">>),"���subfil12"}, - {regular,"���subfil3","���subfil13"}]}, - {directory,"���subdir", - [{regular,"���subfil1","���subfil1"}]}]. + [{directory,treat_icky(<<"åäösubdir2">>), + [{regular,treat_icky(<<"åäösubfil2">>),"åäösubfil12"}, + {regular,"åäösubfil3","åäösubfil13"}]}, + {directory,"åäösubdir", + [{regular,"åäösubfil1","åäösubfil1"}]}]. make_very_icky_dir(Mod) -> rm_rf(Mod,"very_icky_dir"), @@ -774,26 +775,26 @@ make_very_icky_dir(Mod) -> very_icky_dir() -> [{regular,"fil1","fil1"}, - {regular,[956,965,963,954,959,49],"���2"}] ++ + {regular,[956,965,963,954,959,49],"åäö2"}] ++ case has_links() of true -> - [{regular,[956,965,963,954,959,50],"���2"}, + [{regular,[956,965,963,954,959,50],"åäö2"}, {symlink,[956,965,963,954,959,51],[956,965,963,954,959,49]}]; false -> [] end ++ - [{regular,treat_icky(<<"���5">>),"���5"}] ++ + [{regular,treat_icky(<<"åäö5">>),"åäö5"}] ++ case has_links() of true -> - [{symlink,treat_icky(<<"���6">>),treat_icky(<<"���5">>)}]; + [{symlink,treat_icky(<<"åäö6">>),treat_icky(<<"åäö5">>)}]; false -> [] end ++ - [{directory,treat_icky(<<"���subdir2">>), - [{regular,treat_icky(<<"���subfil2">>),"���subfil12"}, - {regular,"���subfil3","���subfil13"}]}, + [{directory,treat_icky(<<"åäösubdir2">>), + [{regular,treat_icky(<<"åäösubfil2">>),"åäösubfil12"}, + {regular,"åäösubfil3","åäösubfil13"}]}, {directory,[956,965,963,954,959]++"subdir1", - [{regular,[956,965,963,954,959]++"subfil1","���subfil1"}]}]. + [{regular,[956,965,963,954,959]++"subfil1","åäösubfil1"}]}]. %% Some OS'es simply do not allow non UTF8 filenames treat_icky(Bin) -> diff --git a/lib/kernel/test/gen_udp_SUITE.erl b/lib/kernel/test/gen_udp_SUITE.erl index 2354f8accd..cd768813cf 100644 --- a/lib/kernel/test/gen_udp_SUITE.erl +++ b/lib/kernel/test/gen_udp_SUITE.erl @@ -1,7 +1,8 @@ +%% -*- coding: utf-8 -*- %% %% %CopyrightBegin% %% -%% Copyright Ericsson AB 1998-2011. All Rights Reserved. +%% Copyright Ericsson AB 1998-2012. All Rights Reserved. %% %% The contents of this file are subject to the Erlang Public License, %% Version 1.1, (the "License"); you may not use this file except in @@ -442,7 +443,7 @@ open_fd(suite) -> open_fd(doc) -> ["Test that the 'fd' option works"]; open_fd(Config) when is_list(Config) -> - Msg = "Det g�r ont n�r knoppar brista. Varf�r skulle annars v�ren tveka?", + Msg = "Det gör ont när knoppar brista. Varför skulle annars våren tveka?", Addr = {127,0,0,1}, {ok,S1} = gen_udp:open(0), {ok,P2} = inet:port(S1), diff --git a/lib/kernel/test/global_SUITE.erl b/lib/kernel/test/global_SUITE.erl index 6eb2134644..b40c50f79f 100644 --- a/lib/kernel/test/global_SUITE.erl +++ b/lib/kernel/test/global_SUITE.erl @@ -1,7 +1,7 @@ %% %% %CopyrightBegin% %% -%% Copyright Ericsson AB 1997-2011. All Rights Reserved. +%% Copyright Ericsson AB 1997-2012. All Rights Reserved. %% %% The contents of this file are subject to the Erlang Public License, %% Version 1.1, (the "License"); you may not use this file except in @@ -3821,7 +3821,7 @@ start_node_rel(Name0, Rel, Config) -> Name = node_name(Name0, Config), {Release, Compat} = case Rel of this -> - {[this], "+R8"}; + {[this], ""}; Rel when is_atom(Rel) -> {[{release, atom_to_list(Rel)}], ""}; RelList -> diff --git a/lib/kernel/test/heart_SUITE.erl b/lib/kernel/test/heart_SUITE.erl index 970a03cfd5..320b23bea1 100644 --- a/lib/kernel/test/heart_SUITE.erl +++ b/lib/kernel/test/heart_SUITE.erl @@ -1,7 +1,7 @@ %% %% %CopyrightBegin% %% -%% Copyright Ericsson AB 1996-2011. All Rights Reserved. +%% Copyright Ericsson AB 1996-2012. All Rights Reserved. %% %% The contents of this file are subject to the Erlang Public License, %% Version 1.1, (the "License"); you may not use this file except in @@ -83,10 +83,10 @@ init_per_suite(Config) when is_list(Config) -> {win32, windows} -> {skipped, "No use to run on Windows 95/98"}; _ -> - Config + ignore_cores:init(Config) end. end_per_suite(Config) when is_list(Config) -> - Config. + ignore_cores:fini(Config). start_check(Type, Name) -> @@ -188,8 +188,20 @@ reboot(Config) when is_list(Config) -> %% Check that a node is up and running after a crash. %% This test exhausts the atom table on the remote node. %% ERL_CRASH_DUMP_SECONDS=0 will force beam not to dump an erl_crash.dump. +%% May currently dump core in beam debug build due to lock-order violation +%% This should be removed when a non-lockad information retriever is implemented +%% for crash dumps node_start_immediately_after_crash(suite) -> {req, [{time, 10}]}; node_start_immediately_after_crash(Config) when is_list(Config) -> + Config2 = ignore_cores:setup(?MODULE, node_start_immediately_after_crash, Config, true), + try + node_start_immediately_after_crash_test(Config2) + after + ignore_cores:restore(Config2) + end. + + +node_start_immediately_after_crash_test(Config) when is_list(Config) -> {ok, Node} = start_check(loose, heart_test_imm, [{"ERL_CRASH_DUMP_SECONDS", "0"}]), ok = rpc:call(Node, heart, set_cmd, @@ -228,8 +240,19 @@ node_start_immediately_after_crash(Config) when is_list(Config) -> %% This test exhausts the atom table on the remote node. %% ERL_CRASH_DUMP_SECONDS=10 will force beam %% to only dump an erl_crash.dump for 10 seconds. +%% May currently dump core in beam debug build due to lock-order violation +%% This should be removed when a non-lockad information retriever is implemented +%% for crash dumps node_start_soon_after_crash(suite) -> {req, [{time, 10}]}; node_start_soon_after_crash(Config) when is_list(Config) -> + Config2 = ignore_cores:setup(?MODULE, node_start_soon_after_crash, Config, true), + try + node_start_soon_after_crash_test(Config2) + after + ignore_cores:restore(Config2) + end. + +node_start_soon_after_crash_test(Config) when is_list(Config) -> {ok, Node} = start_check(loose, heart_test_soon, [{"ERL_CRASH_DUMP_SECONDS", "10"}]), ok = rpc:call(Node, heart, set_cmd, diff --git a/lib/kernel/test/ignore_cores.erl b/lib/kernel/test/ignore_cores.erl new file mode 100644 index 0000000000..8b1ac0fe6c --- /dev/null +++ b/lib/kernel/test/ignore_cores.erl @@ -0,0 +1,158 @@ +%% +%% %CopyrightBegin% +%% +%% Copyright Ericsson AB 2008-2010. All Rights Reserved. +%% +%% The contents of this file are subject to the Erlang Public License, +%% Version 1.1, (the "License"); you may not use this file except in +%% compliance with the License. You should have received a copy of the +%% Erlang Public License along with this software. If not, it can be +%% retrieved online at http://www.erlang.org/. +%% +%% Software distributed under the License is distributed on an "AS IS" +%% basis, WITHOUT WARRANTY OF ANY KIND, either express or implied. See +%% the License for the specific language governing rights and limitations +%% under the License. +%% +%% %CopyrightEnd% +%% + +%%%------------------------------------------------------------------- +%%% File : ignore_cores.erl +%%% Author : Rickard Green <[email protected]> +%%% Description : +%%% +%%% Created : 11 Feb 2008 by Rickard Green <[email protected]> +%%%------------------------------------------------------------------- + +-module(ignore_cores). + +-include_lib("test_server/include/test_server.hrl"). + +-export([init/1, fini/1, setup/3, setup/4, restore/1, dir/1]). + +-record(ignore_cores, {org_cwd, + org_path, + org_pwd_env, + ign_dir = false, + cores_dir = false}). + +%% +%% Takes a testcase config +%% + +init(Config) -> + {ok, OrgCWD} = file:get_cwd(), + [{ignore_cores, + #ignore_cores{org_cwd = OrgCWD, + org_path = code:get_path(), + org_pwd_env = os:getenv("PWD")}} + | lists:keydelete(ignore_cores, 1, Config)]. + +fini(Config) -> + #ignore_cores{org_cwd = OrgCWD, + org_path = OrgPath, + org_pwd_env = OrgPWD} = ?config(ignore_cores, Config), + ok = file:set_cwd(OrgCWD), + true = code:set_path(OrgPath), + case OrgPWD of + false -> ok; + _ -> true = os:putenv("PWD", OrgPWD) + end, + lists:keydelete(ignore_cores, 1, Config). + +setup(Suite, Testcase, Config) -> + setup(Suite, Testcase, Config, false). + +setup(Suite, Testcase, Config, SetCwd) when is_atom(Suite), + is_atom(Testcase), + is_list(Config) -> + #ignore_cores{org_cwd = OrgCWD, + org_path = OrgPath, + org_pwd_env = OrgPWD} = ?config(ignore_cores, Config), + Path = lists:map(fun (".") -> OrgCWD; (Dir) -> Dir end, OrgPath), + true = code:set_path(Path), + PrivDir = ?config(priv_dir, Config), + IgnDir = filename:join([PrivDir, + atom_to_list(Suite) + ++ "_" + ++ atom_to_list(Testcase) + ++ "_wd"]), + ok = file:make_dir(IgnDir), + case SetCwd of + false -> + ok; + _ -> + ok = file:set_cwd(IgnDir), + OrgPWD = case os:getenv("PWD") of + false -> false; + PWD -> + os:putenv("PWD", IgnDir), + PWD + end + end, + ok = file:write_file(filename:join([IgnDir, "ignore_core_files"]), <<>>), + %% cores are dumped in /cores on MacOS X + CoresDir = case {?t:os_type(), filelib:is_dir("/cores")} of + {{unix,darwin}, true} -> + filelib:fold_files("/cores", + "^core.*$", + false, + fun (C,Cs) -> [C|Cs] end, + []); + _ -> + false + end, + lists:keyreplace(ignore_cores, + 1, + Config, + {ignore_cores, + #ignore_cores{org_cwd = OrgCWD, + org_path = OrgPath, + org_pwd_env = OrgPWD, + ign_dir = IgnDir, + cores_dir = CoresDir}}). + +restore(Config) -> + #ignore_cores{org_cwd = OrgCWD, + org_path = OrgPath, + org_pwd_env = OrgPWD, + ign_dir = IgnDir, + cores_dir = CoresDir} = ?config(ignore_cores, Config), + try + case CoresDir of + false -> + ok; + _ -> + %% Move cores dumped by these testcases in /cores + %% to cwd. + lists:foreach(fun (C) -> + case lists:member(C, CoresDir) of + true -> ok; + _ -> + Dst = filename:join( + [IgnDir, + filename:basename(C)]), + {ok, _} = file:copy(C, Dst), + file:delete(C) + end + end, + filelib:fold_files("/cores", + "^core.*$", + false, + fun (C,Cs) -> [C|Cs] end, + [])) + end + after + catch file:set_cwd(OrgCWD), + catch code:set_path(OrgPath), + case OrgPWD of + false -> ok; + _ -> catch os:putenv("PWD", OrgPWD) + end + end. + + +dir(Config) -> + #ignore_cores{ign_dir = Dir} = ?config(ignore_cores, Config), + Dir. diff --git a/lib/kernel/test/inet_SUITE.erl b/lib/kernel/test/inet_SUITE.erl index ce138b6804..e5e1794514 100644 --- a/lib/kernel/test/inet_SUITE.erl +++ b/lib/kernel/test/inet_SUITE.erl @@ -37,7 +37,8 @@ gethostnative_soft_restart/0, gethostnative_soft_restart/1, gethostnative_debug_level/0, gethostnative_debug_level/1, getif/1, - getif_ifr_name_overflow/1,getservbyname_overflow/1, getifaddrs/1]). + getif_ifr_name_overflow/1,getservbyname_overflow/1, getifaddrs/1, + parse_strict_address/1]). -export([get_hosts/1, get_ipv6_hosts/1, parse_hosts/1, parse_address/1, kill_gethost/0, parallell_gethost/0]). @@ -52,7 +53,7 @@ all() -> t_gethostnative, gethostnative_parallell, cname_loop, gethostnative_debug_level, gethostnative_soft_restart, getif, getif_ifr_name_overflow, getservbyname_overflow, - getifaddrs]. + getifaddrs, parse_strict_address]. groups() -> [{parse, [], [parse_hosts, parse_address]}]. @@ -582,16 +583,16 @@ parse_address(Config) when is_list(Config) -> "fe80::198.168.0.", "fec0::fFfF:127.0.0.1."], t_parse_address - (ipv6_address, + (parse_ipv6_address, V6Strict++V6Sloppy++V6Err++V4Err), t_parse_address - (ipv6strict_address, + (parse_ipv6strict_address, V6Strict++V6Err++V4Err++[S || {_,S} <- V6Sloppy]), t_parse_address - (ipv4_address, + (parse_ipv4_address, V4Strict++V4Sloppy++V4Err++V6Err++[S || {_,S} <- V6Strict]), t_parse_address - (ipv4strict_address, + (parse_ipv4strict_address, V4Strict++V4Err++V6Err++[S || {_,S} <- V4Sloppy++V6Strict]). t_parse_address(Func, []) -> @@ -599,14 +600,16 @@ t_parse_address(Func, []) -> ok; t_parse_address(Func, [{Addr,String}|L]) -> io:format("~p = ~p.~n", [Addr,String]), - {ok,Addr} = inet_parse:Func(String), + {ok,Addr} = inet:Func(String), t_parse_address(Func, L); t_parse_address(Func, [String|L]) -> io:format("~p.~n", [String]), - {error,einval} = inet_parse:Func(String), + {error,einval} = inet:Func(String), t_parse_address(Func, L). - +parse_strict_address(Config) when is_list(Config) -> + {ok, Ipv4} = inet:parse_strict_address("127.0.0.1"), + {ok, Ipv6} = inet:parse_strict_address("c11:0c22:5c33:c440:55c0:c66c:77:0088"). t_gethostnative(suite) ->[]; t_gethostnative(doc) ->[]; diff --git a/lib/kernel/test/ram_file_SUITE.erl b/lib/kernel/test/ram_file_SUITE.erl index ab95a3ff5f..5c4437d4d3 100644 --- a/lib/kernel/test/ram_file_SUITE.erl +++ b/lib/kernel/test/ram_file_SUITE.erl @@ -1,7 +1,8 @@ +%% -*- coding: utf-8 -*- %% %% %CopyrightBegin% %% -%% Copyright Ericsson AB 2001-2011. All Rights Reserved. +%% Copyright Ericsson AB 2001-2012. All Rights Reserved. %% %% The contents of this file are subject to the Erlang Public License, %% Version 1.1, (the "License"); you may not use this file except in @@ -167,7 +168,7 @@ pread_pwrite(suite) -> pread_pwrite(doc) -> ["Test that pread/2,3 and pwrite/2,3 works."]; pread_pwrite(Config) when is_list(Config) -> - ?line Str = "Flygande b�ckaziner s�ka hwila p� mjuqa tuvor x", + ?line Str = "Flygande bäckaziner söka hwila på mjuqa tuvor x", ?line Bin = list_to_binary(Str), %% pread_pwrite_test(?FILE_MODULE, Str, [ram, read, write]), @@ -206,7 +207,7 @@ position(suite) -> position(doc) -> ["Test that position/2 works."]; position(Config) when is_list(Config) -> - ?line Str = "Att vara eller icke vara, det �r fr�gan. ", + ?line Str = "Att vara eller icke vara, det är frågan. ", ?line Bin = list_to_binary(Str), %% position_test(?FILE_MODULE, Str, [ram, read]), @@ -287,8 +288,8 @@ truncate(suite) -> truncate(doc) -> ["Test that truncate/1 works."]; truncate(Config) when is_list(Config) -> - ?line Str = "M�n �dlare att lida och f�rdraga " - ++ "ett bittert �des stygn av pilar, ", + ?line Str = "Mån ädlare att lida och fördraga " + ++ "ett bittert ödes stygn av pilar, ", ?line Bin = list_to_binary(Str), %% ok = truncate_test(?FILE_MODULE, Str, [ram, read, write]), @@ -331,7 +332,7 @@ sync(suite) -> sync(doc) -> ["Test that sync/1 at least does not crash."]; sync(Config) when is_list(Config) -> - ?line Str = "�n att ta till vapen mot ett hav av kval. ", + ?line Str = "än att ta till vapen mot ett hav av kval. ", ?line Bin = list_to_binary(Str), %% sync_test(?FILE_MODULE, Str, [ram, read, write]), @@ -365,8 +366,8 @@ get_set_file(doc) -> ["Tests get_file/1, set_file/2, get_file_close/1 and get_size/1."]; get_set_file(Config) when is_list(Config) -> %% These two strings should not be of equal length. - ?line Str = "N�r h�gan nord blir sn�bet�ckt, ", - ?line Str2 = "f�r alla harar byta dr�kt. ", + ?line Str = "När högan nord blir snöbetäckt, ", + ?line Str2 = "får alla harar byta dräkt. ", ?line Bin = list_to_binary(Str), ?line Bin2 = list_to_binary(Str2), %% diff --git a/lib/kernel/test/wrap_log_reader_SUITE.erl b/lib/kernel/test/wrap_log_reader_SUITE.erl index 6c47fda9c5..16b3a7cc1e 100644 --- a/lib/kernel/test/wrap_log_reader_SUITE.erl +++ b/lib/kernel/test/wrap_log_reader_SUITE.erl @@ -1,7 +1,7 @@ %% %% %CopyrightBegin% %% -%% Copyright Ericsson AB 1998-2011. All Rights Reserved. +%% Copyright Ericsson AB 1998-2012. All Rights Reserved. %% %% The contents of this file are subject to the Erlang Public License, %% Version 1.1, (the "License"); you may not use this file except in diff --git a/lib/megaco/doc/src/megaco_encode.xml b/lib/megaco/doc/src/megaco_encode.xml index 410e4f3b31..4a9d63c2e3 100644 --- a/lib/megaco/doc/src/megaco_encode.xml +++ b/lib/megaco/doc/src/megaco_encode.xml @@ -224,22 +224,11 @@ messages.</p> </item> <item> - <p>megaco_ber_bin_encoder - encode/decode ASN.1 BER - messages. This encoder uses ASN.1 ber_bin which - has been optimized using the bit syntax.</p> - </item> - <item> <p>megaco_per_encoder - encode/decode ASN.1 PER messages. N.B. that this format is not included in the Megaco standard.</p> </item> <item> - <p>megaco_per_bin_encoder - encode/decode ASN.1 PER - messages. N.B. that this format is not included in the - Megaco standard. This encoder uses ASN.1 per_bin which - has been optimized using the bit syntax.</p> - </item> - <item> <p>megaco_erl_dist_encoder - encodes messages into Erlangs distribution format. It is rather verbose but encoding and decoding is blinding fast. N.B. that this format is not @@ -370,18 +359,6 @@ needs to be specified.</p> <list type="bulleted"> <item> - <p><c><![CDATA[[driver|_]]]></c> - make use of the asn1 driver for decode - (ber_bin) and encode (per_bin). This option is only available for - encoding modules: <c><![CDATA[megaco_binary_encoder]]></c>, - <c><![CDATA[megaco_ber_bin_encoder]]></c> and <c><![CDATA[megaco_per_bin_encoder]]></c>.</p> - <p>If this option is present in the encoding config, it <em>must</em> - to be the <em>first</em>, unless the - <seealso marker="#handling_versions">version3</seealso> encoding - config is present, in which case it must come second, after - the version3 encoding config, - e.g. <c><![CDATA[[{version3,prev3b},driver]]]></c>.</p> - </item> - <item> <p><c><![CDATA[[native]]]></c> - skips the transformation phase, i.e. the decoded message(s) will not be transformed into our internal form.</p> diff --git a/lib/megaco/examples/meas/megaco_codec_meas.erl b/lib/megaco/examples/meas/megaco_codec_meas.erl index 51ee396338..65b986ccd1 100644 --- a/lib/megaco/examples/meas/megaco_codec_meas.erl +++ b/lib/megaco/examples/meas/megaco_codec_meas.erl @@ -176,7 +176,7 @@ display_megaco_info() -> io:format("Megaco version: ~s (~s)~n", [Ver, FlexStr]). display_asn1_info() -> - AI = megaco_ber_bin_drv_media_gateway_control_v1:info(), + AI = megaco_ber_media_gateway_control_v1:info(), Vsn = case lists:keysearch(vsn, 1, AI) of {value, {vsn, V}} when is_atom(V) -> @@ -281,15 +281,11 @@ expand_codec(Codec) -> [{Codec, megaco_compact_text_encoder, [flex_scanner], 3000}, {Codec, megaco_compact_text_encoder, [], 1500}]; ber -> - [{Codec, megaco_ber_bin_encoder, [driver,native], 4000}, - {Codec, megaco_ber_bin_encoder, [native], 3000}, - {Codec, megaco_ber_bin_encoder, [driver], 3000}, - {Codec, megaco_ber_bin_encoder, [], 1000}]; + [{Codec, megaco_ber_encoder, [native], 3000}, + {Codec, megaco_ber_encoder, [], 1000}]; per -> - [{Codec, megaco_per_bin_encoder, [driver,native], 4000}, - {Codec, megaco_per_bin_encoder, [native], 3000}, - {Codec, megaco_per_bin_encoder, [driver], 3000}, - {Codec, megaco_per_bin_encoder, [], 1000}]; + [{Codec, megaco_per_encoder, [native], 3000}, + {Codec, megaco_per_encoder, [], 1000}]; erlang -> [ {Codec, megaco_erl_dist_encoder, [megaco_compressed,compressed], 500}, diff --git a/lib/megaco/examples/meas/megaco_codec_mstone_lib.erl b/lib/megaco/examples/meas/megaco_codec_mstone_lib.erl index 9af88d9f50..b527ff2e89 100644 --- a/lib/megaco/examples/meas/megaco_codec_mstone_lib.erl +++ b/lib/megaco/examples/meas/megaco_codec_mstone_lib.erl @@ -268,7 +268,7 @@ display_megaco_info() -> io:format("Megaco version: ~s~n", [Ver]). display_asn1_info() -> - AI = megaco_ber_bin_drv_media_gateway_control_v1:info(), + AI = megaco_ber__media_gateway_control_v1:info(), Vsn = case lists:keysearch(vsn, 1, AI) of {value, {vsn, V}} when is_atom(V) -> @@ -361,15 +361,11 @@ expand_codec(Codec, only_drv) -> [{Codec, megaco_compact_text_encoder, [flex_scanner]}, {Codec, megaco_compact_text_encoder, [flex_scanner]}]; ber -> - [{Codec, megaco_ber_bin_encoder, [driver,native]}, - {Codec, megaco_ber_bin_encoder, [driver]}, - {Codec, megaco_ber_bin_encoder, [driver,native]}, - {Codec, megaco_ber_bin_encoder, [driver]}]; + [{Codec, megaco_ber_encoder, [native]}, + {Codec, megaco_ber_encoder, []}]; per -> - [{Codec, megaco_per_bin_encoder, [driver,native]}, - {Codec, megaco_per_bin_encoder, [native]}, - {Codec, megaco_per_bin_encoder, [driver,native]}, - {Codec, megaco_per_bin_encoder, [native]}]; + [{Codec, megaco_per_encoder, [native]}, + {Codec, megaco_per_encoder, []}]; erlang -> Encoder = megaco_erl_dist_encoder, [ @@ -390,15 +386,11 @@ expand_codec(Codec, no_drv) -> [{Codec, megaco_compact_text_encoder, []}, {Codec, megaco_compact_text_encoder, []}]; ber -> - [{Codec, megaco_ber_bin_encoder, [native]}, - {Codec, megaco_ber_bin_encoder, []}, - {Codec, megaco_ber_bin_encoder, [native]}, - {Codec, megaco_ber_bin_encoder, []}]; + [{Codec, megaco_ber_encoder, [native]}, + {Codec, megaco_ber_encoder, []}]; per -> - [{Codec, megaco_per_bin_encoder, [native]}, - {Codec, megaco_per_bin_encoder, []}, - {Codec, megaco_per_bin_encoder, [native]}, - {Codec, megaco_per_bin_encoder, []}]; + [{Codec, megaco_per_encoder, [native]}, + {Codec, megaco_per_encoder, []}]; erlang -> Encoder = megaco_erl_dist_encoder, [ @@ -419,15 +411,11 @@ expand_codec(Codec, _) -> [{Codec, megaco_compact_text_encoder, [flex_scanner]}, {Codec, megaco_compact_text_encoder, []}]; ber -> - [{Codec, megaco_ber_bin_encoder, [driver,native]}, - {Codec, megaco_ber_bin_encoder, [native]}, - {Codec, megaco_ber_bin_encoder, [driver]}, - {Codec, megaco_ber_bin_encoder, []}]; + [{Codec, megaco_ber_encoder, [native]}, + {Codec, megaco_ber_encoder, []}]; per -> - [{Codec, megaco_per_bin_encoder, [driver,native]}, - {Codec, megaco_per_bin_encoder, [native]}, - {Codec, megaco_per_bin_encoder, [driver]}, - {Codec, megaco_per_bin_encoder, []}]; + [{Codec, megaco_per_encoder, [native]}, + {Codec, megaco_per_encoder, []}]; erlang -> Encoder = megaco_erl_dist_encoder, [ diff --git a/lib/megaco/examples/meas/megaco_codec_transform.erl b/lib/megaco/examples/meas/megaco_codec_transform.erl index cfe832ff26..15db165566 100644 --- a/lib/megaco/examples/meas/megaco_codec_transform.erl +++ b/lib/megaco/examples/meas/megaco_codec_transform.erl @@ -213,11 +213,11 @@ decode_message(compact, BinMsg) -> Conf = [{version3,?V3}], do_decode(Mod, Conf, BinMsg); decode_message(ber, BinMsg) -> - Mod = megaco_ber_bin_encoder, + Mod = megaco_ber_encoder, Conf = [{version3,?V3}], do_decode(Mod, Conf, BinMsg); decode_message(per, BinMsg) -> - Mod = megaco_per_bin_encoder, + Mod = megaco_per_encoder, Conf = [{version3,?V3}], do_decode(Mod, Conf, BinMsg); decode_message(erlang, BinMsg) -> diff --git a/lib/megaco/src/app/megaco.app.src b/lib/megaco/src/app/megaco.app.src index c0d8218ac8..0ba2a866f9 100644 --- a/lib/megaco/src/app/megaco.app.src +++ b/lib/megaco/src/app/megaco.app.src @@ -23,19 +23,6 @@ {modules, [ megaco, - megaco_ber_bin_encoder, - megaco_ber_bin_drv_media_gateway_control_v1, - megaco_ber_bin_drv_media_gateway_control_v2, - megaco_ber_bin_drv_media_gateway_control_prev3a, - megaco_ber_bin_drv_media_gateway_control_prev3b, - megaco_ber_bin_drv_media_gateway_control_prev3c, - megaco_ber_bin_drv_media_gateway_control_v3, - megaco_ber_bin_media_gateway_control_v1, - megaco_ber_bin_media_gateway_control_v2, - megaco_ber_bin_media_gateway_control_prev3a, - megaco_ber_bin_media_gateway_control_prev3b, - megaco_ber_bin_media_gateway_control_prev3c, - megaco_ber_bin_media_gateway_control_v3, megaco_ber_encoder, megaco_ber_media_gateway_control_v1, megaco_ber_media_gateway_control_v2, @@ -87,19 +74,6 @@ megaco_per_media_gateway_control_prev3b, megaco_per_media_gateway_control_prev3c, megaco_per_media_gateway_control_v3, - megaco_per_bin_encoder, - megaco_per_bin_drv_media_gateway_control_v1, - megaco_per_bin_drv_media_gateway_control_v2, - megaco_per_bin_drv_media_gateway_control_prev3a, - megaco_per_bin_drv_media_gateway_control_prev3b, - megaco_per_bin_drv_media_gateway_control_prev3c, - megaco_per_bin_drv_media_gateway_control_v3, - megaco_per_bin_media_gateway_control_v1, - megaco_per_bin_media_gateway_control_v2, - megaco_per_bin_media_gateway_control_prev3a, - megaco_per_bin_media_gateway_control_prev3b, - megaco_per_bin_media_gateway_control_prev3c, - megaco_per_bin_media_gateway_control_v3, megaco_pretty_text_encoder, megaco_pretty_text_encoder_v1, megaco_pretty_text_encoder_v2, diff --git a/lib/megaco/src/binary/Makefile b/lib/megaco/src/binary/Makefile index 695599b9dc..660713605e 100644 --- a/lib/megaco/src/binary/Makefile +++ b/lib/megaco/src/binary/Makefile @@ -50,46 +50,22 @@ ASN1_SPECS = $(ASN1_V1_SPEC) \ ASN1_FILES = $(ASN1_SPECS:%=%.asn) V1_SPECS = $(BER_ASN1_V1_SPEC) \ - $(BER_BIN_ASN1_V1_SPEC) \ - $(BER_BIN_DRV_ASN1_V1_SPEC) \ - $(PER_ASN1_V1_SPEC) \ - $(PER_BIN_ASN1_V1_SPEC) \ - $(PER_BIN_DRV_ASN1_V1_SPEC) + $(PER_ASN1_V1_SPEC) V2_SPECS = $(BER_ASN1_V2_SPEC) \ - $(BER_BIN_ASN1_V2_SPEC) \ - $(BER_BIN_DRV_ASN1_V2_SPEC) \ - $(PER_ASN1_V2_SPEC) \ - $(PER_BIN_ASN1_V2_SPEC) \ - $(PER_BIN_DRV_ASN1_V2_SPEC) + $(PER_ASN1_V2_SPEC) PREV3A_SPECS = $(BER_ASN1_PREV3A_SPEC) \ - $(BER_BIN_ASN1_PREV3A_SPEC) \ - $(BER_BIN_DRV_ASN1_PREV3A_SPEC) \ - $(PER_ASN1_PREV3A_SPEC) \ - $(PER_BIN_ASN1_PREV3A_SPEC) \ - $(PER_BIN_DRV_ASN1_PREV3A_SPEC) + $(PER_ASN1_PREV3A_SPEC) PREV3B_SPECS = $(BER_ASN1_PREV3B_SPEC) \ - $(BER_BIN_ASN1_PREV3B_SPEC) \ - $(BER_BIN_DRV_ASN1_PREV3B_SPEC) \ - $(PER_ASN1_PREV3B_SPEC) \ - $(PER_BIN_ASN1_PREV3B_SPEC) \ - $(PER_BIN_DRV_ASN1_PREV3B_SPEC) + $(PER_ASN1_PREV3B_SPEC) PREV3C_SPECS = $(BER_ASN1_PREV3C_SPEC) \ - $(BER_BIN_ASN1_PREV3C_SPEC) \ - $(BER_BIN_DRV_ASN1_PREV3C_SPEC) \ - $(PER_ASN1_PREV3C_SPEC) \ - $(PER_BIN_ASN1_PREV3C_SPEC) \ - $(PER_BIN_DRV_ASN1_PREV3C_SPEC) + $(PER_ASN1_PREV3C_SPEC) V3_SPECS = $(BER_ASN1_V3_SPEC) \ - $(BER_BIN_ASN1_V3_SPEC) \ - $(BER_BIN_DRV_ASN1_V3_SPEC) \ $(PER_ASN1_V3_SPEC) \ - $(PER_BIN_ASN1_V3_SPEC) \ - $(PER_BIN_DRV_ASN1_V3_SPEC) \ $(PREV3A_SPECS) $(PREV3B_SPECS) $(PREV3C_SPECS) SPECS = $(V1_SPECS) $(V2_SPECS) $(V3_SPECS) diff --git a/lib/megaco/src/binary/depend.mk b/lib/megaco/src/binary/depend.mk index c9ca34bcf6..a1318079be 100644 --- a/lib/megaco/src/binary/depend.mk +++ b/lib/megaco/src/binary/depend.mk @@ -19,17 +19,8 @@ # Flag description: # -# +optimize -# For ber_bin this means "optimize" (whatever that is), -# but for per_bin it means that a stage in the encode -# is done in the asn1 driver. -# -# +nif -# For ber_bin this means that part of the decode is done -# in the asn1 nif. -# # +asn1config -# This is only used by the ber_bin, and means that +# This is only used by the ber, and means that # some partial decode functions will be created # (as described by the asn1config file). # @@ -43,42 +34,18 @@ ifeq ($(MEGACO_INLINE_ASN1_RT),true) ASN1_CT_OPTS += +inline endif -BER_V1_FLAGS = $(ASN1_CT_OPTS) -BER_BIN_V1_FLAGS = $(ASN1_CT_OPTS) +asn1config +optimize -BER_BIN_DRV_V1_FLAGS = $(ASN1_CT_OPTS) +asn1config +optimize +nif -BER_V2_FLAGS = $(ASN1_CT_OPTS) -BER_BIN_V2_FLAGS = $(ASN1_CT_OPTS) +asn1config +optimize -BER_BIN_DRV_V2_FLAGS = $(ASN1_CT_OPTS) +asn1config +optimize +nif -BER_PREV3A_FLAGS = $(ASN1_CT_OPTS) -BER_BIN_PREV3A_FLAGS = $(ASN1_CT_OPTS) +asn1config +optimize -BER_BIN_DRV_PREV3A_FLAGS = $(ASN1_CT_OPTS) +asn1config +optimize +nif -BER_PREV3B_FLAGS = $(ASN1_CT_OPTS) -BER_BIN_PREV3B_FLAGS = $(ASN1_CT_OPTS) +asn1config +optimize -BER_BIN_DRV_PREV3B_FLAGS = $(ASN1_CT_OPTS) +asn1config +optimize +nif -BER_PREV3C_FLAGS = $(ASN1_CT_OPTS) -BER_BIN_PREV3C_FLAGS = $(ASN1_CT_OPTS) +asn1config +optimize -BER_BIN_DRV_PREV3C_FLAGS = $(ASN1_CT_OPTS) +asn1config +optimize +nif -BER_V3_FLAGS = $(ASN1_CT_OPTS) -BER_BIN_V3_FLAGS = $(ASN1_CT_OPTS) +asn1config +optimize -BER_BIN_DRV_V3_FLAGS = $(ASN1_CT_OPTS) +asn1config +optimize +nif +BER_V1_FLAGS = $(ASN1_CT_OPTS) +asn1config +BER_V2_FLAGS = $(ASN1_CT_OPTS) +asn1config +BER_PREV3A_FLAGS = $(ASN1_CT_OPTS) +asn1config +BER_PREV3B_FLAGS = $(ASN1_CT_OPTS) +asn1config +BER_PREV3C_FLAGS = $(ASN1_CT_OPTS) +asn1config +BER_V3_FLAGS = $(ASN1_CT_OPTS) +asn1config PER_V1_FLAGS = $(ASN1_CT_OPTS) -PER_BIN_V1_FLAGS = $(ASN1_CT_OPTS) -PER_BIN_DRV_V1_FLAGS = $(ASN1_CT_OPTS) +optimize PER_V2_FLAGS = $(ASN1_CT_OPTS) -PER_BIN_V2_FLAGS = $(ASN1_CT_OPTS) -PER_BIN_DRV_V2_FLAGS = $(ASN1_CT_OPTS) +optimize PER_PREV3A_FLAGS = $(ASN1_CT_OPTS) -PER_BIN_PREV3A_FLAGS = $(ASN1_CT_OPTS) -PER_BIN_DRV_PREV3A_FLAGS = $(ASN1_CT_OPTS) +optimize PER_PREV3B_FLAGS = $(ASN1_CT_OPTS) -PER_BIN_PREV3B_FLAGS = $(ASN1_CT_OPTS) -PER_BIN_DRV_PREV3B_FLAGS = $(ASN1_CT_OPTS) +optimize PER_PREV3C_FLAGS = $(ASN1_CT_OPTS) -PER_BIN_PREV3C_FLAGS = $(ASN1_CT_OPTS) -PER_BIN_DRV_PREV3C_FLAGS = $(ASN1_CT_OPTS) +optimize PER_V3_FLAGS = $(ASN1_CT_OPTS) -PER_BIN_V3_FLAGS = $(ASN1_CT_OPTS) -PER_BIN_DRV_V3_FLAGS = $(ASN1_CT_OPTS) +optimize # --- Version 1 --- @@ -92,26 +59,6 @@ $(BER_ASN1_V1_SPEC).erl: \ $(EBIN)/$(BER_ASN1_V1_SPEC).$(EMULATOR): \ $(BER_ASN1_V1_SPEC).erl -$(BER_BIN_ASN1_V1_SPEC).erl: \ - $(BER_BIN_ASN1_V1_SPEC).set.asn \ - $(BER_BIN_ASN1_V1_SPEC).asn1config \ - $(ASN1_V1_SPEC).asn - @echo "$(BER_BIN_ASN1_V1_SPEC):" - $(ERLC) -bber_bin $(BER_BIN_V1_FLAGS) $(BER_BIN_ASN1_V1_SPEC).set.asn - -$(EBIN)/$(BER_BIN_ASN1_V1_SPEC).$(EMULATOR): \ - $(BER_BIN_ASN1_V1_SPEC).erl - -$(BER_BIN_DRV_ASN1_V1_SPEC).erl: \ - $(BER_BIN_DRV_ASN1_V1_SPEC).set.asn \ - $(BER_BIN_DRV_ASN1_V1_SPEC).asn1config \ - $(ASN1_V1_SPEC).asn - @echo "$(BER_BIN_DRV_ASN1_V1_SPEC):" - $(ERLC) -bber_bin $(BER_BIN_DRV_V1_FLAGS) $(BER_BIN_DRV_ASN1_V1_SPEC).set.asn - -$(EBIN)/$(BER_BIN_DRV_ASN1_V1_SPEC).$(EMULATOR): \ - $(BER_BIN_DRV_ASN1_V1_SPEC).erl - $(PER_ASN1_V1_SPEC).erl: \ $(PER_ASN1_V1_SPEC).set.asn \ $(ASN1_V1_SPEC).asn @@ -121,24 +68,6 @@ $(PER_ASN1_V1_SPEC).erl: \ $(EBIN)/$(PER_ASN1_V1_SPEC).$(EMULATOR): \ $(PER_ASN1_V1_SPEC).erl -$(PER_BIN_ASN1_V1_SPEC).erl: \ - $(PER_BIN_ASN1_V1_SPEC).set.asn \ - $(ASN1_V1_SPEC).asn - @echo "$(PER_BIN_ASN1_V1_SPEC):" - $(ERLC) -bper_bin $(PER_BIN_V1_FLAGS) $(PER_BIN_ASN1_V1_SPEC).set.asn - -$(EBIN)/$(PER_BIN_ASN1_V1_SPEC).$(EMULATOR): \ - $(PER_BIN_ASN1_V1_SPEC).erl - -$(PER_BIN_DRV_ASN1_V1_SPEC).erl: \ - $(PER_BIN_DRV_ASN1_V1_SPEC).set.asn \ - $(ASN1_V1_SPEC).asn - @echo "$(PER_BIN_DRV_ASN1_V1_SPEC):" - $(ERLC) -bper_bin $(PER_BIN_DRV_V1_FLAGS) $(PER_BIN_DRV_ASN1_V1_SPEC).set.asn - -$(EBIN)/$(PER_BIN_DRV_ASN1_V1_SPEC).$(EMULATOR): \ - $(PER_BIN_DRV_ASN1_V1_SPEC).erl - # --- Version 2 --- @@ -151,26 +80,6 @@ $(BER_ASN1_V2_SPEC).erl: \ $(EBIN)/$(BER_ASN1_V2_SPEC).$(EMULATOR): \ $(BER_ASN1_V2_SPEC).erl -$(BER_BIN_ASN1_V2_SPEC).erl: \ - $(BER_BIN_ASN1_V2_SPEC).set.asn \ - $(BER_BIN_ASN1_V2_SPEC).asn1config \ - $(ASN1_V2_SPEC).asn - @echo "$(BER_BIN_ASN1_V2_SPEC):" - $(ERLC) -bber_bin $(BER_BIN_V2_FLAGS) $(BER_BIN_ASN1_V2_SPEC).set.asn - -$(EBIN)/$(BER_BIN_ASN1_V2_SPEC).$(EMULATOR): \ - $(BER_BIN_ASN1_V2_SPEC).erl - -$(BER_BIN_DRV_ASN1_V2_SPEC).erl: \ - $(BER_BIN_DRV_ASN1_V2_SPEC).set.asn \ - $(BER_BIN_DRV_ASN1_V2_SPEC).asn1config \ - $(ASN1_V2_SPEC).asn - @echo "$(BER_BIN_DRV_ASN1_V2_SPEC):" - $(ERLC) -bber_bin $(BER_BIN_DRV_V2_FLAGS) $(BER_BIN_DRV_ASN1_V2_SPEC).set.asn - -$(EBIN)/$(BER_BIN_DRV_ASN1_V2_SPEC).$(EMULATOR): \ - $(BER_BIN_DRV_ASN1_V2_SPEC).erl - $(PER_ASN1_V2_SPEC).erl: \ $(PER_ASN1_V2_SPEC).set.asn \ $(ASN1_V2_SPEC).asn @@ -180,25 +89,6 @@ $(PER_ASN1_V2_SPEC).erl: \ $(EBIN)/$(PER_ASN1_V2_SPEC).$(EMULATOR): \ $(PER_ASN1_V2_SPEC).erl -$(PER_BIN_ASN1_V2_SPEC).erl: \ - $(PER_BIN_ASN1_V2_SPEC).set.asn \ - $(ASN1_V2_SPEC).asn - @echo "$(PER_BIN_ASN1_V2_SPEC):" - $(ERLC) -bper_bin $(PER_BIN_V2_FLAGS) $(PER_BIN_ASN1_V2_SPEC).set.asn - -$(EBIN)/$(PER_BIN_ASN1_V2_SPEC).$(EMULATOR): \ - $(PER_BIN_ASN1_V2_SPEC).erl - -$(PER_BIN_DRV_ASN1_V2_SPEC).erl: \ - $(PER_BIN_DRV_ASN1_V2_SPEC).set.asn \ - $(ASN1_V2_SPEC).asn - @echo "$(PER_BIN_DRV_ASN1_V2_SPEC):" - $(ERLC) -bper_bin $(PER_BIN_DRV_V2_FLAGS) $(PER_BIN_DRV_ASN1_V2_SPEC).set.asn - -$(EBIN)/$(PER_BIN_DRV_ASN1_V2_SPEC).$(EMULATOR): \ - $(PER_BIN_DRV_ASN1_V2_SPEC).erl - - # --- Version 3 --- # -- (prev3a) -- @@ -212,26 +102,6 @@ $(BER_ASN1_PREV3A_SPEC).erl: \ $(EBIN)/$(BER_ASN1_PREV3A_SPEC).$(EMULATOR): \ $(BER_ASN1_PREV3A_SPEC).erl -$(BER_BIN_ASN1_PREV3A_SPEC).erl: \ - $(BER_BIN_ASN1_PREV3A_SPEC).set.asn \ - $(BER_BIN_ASN1_PREV3A_SPEC).asn1config \ - $(ASN1_PREV3A_SPEC).asn - @echo "$(BER_BIN_ASN1_PREV3A_SPEC):" - $(ERLC) -bber_bin $(BER_BIN_PREV3A_FLAGS) $(BER_BIN_ASN1_PREV3A_SPEC).set.asn - -$(EBIN)/$(BER_BIN_ASN1_PREV3A_SPEC).$(EMULATOR): \ - $(BER_BIN_ASN1_PREV3A_SPEC).erl - -$(BER_BIN_DRV_ASN1_PREV3A_SPEC).erl: \ - $(BER_BIN_DRV_ASN1_PREV3A_SPEC).set.asn \ - $(BER_BIN_DRV_ASN1_PREV3A_SPEC).asn1config \ - $(ASN1_PREV3A_SPEC).asn - @echo "$(BER_BIN_DRV_ASN1_PREV3A_SPEC):" - $(ERLC) -bber_bin $(BER_BIN_DRV_PREV3A_FLAGS) $(BER_BIN_DRV_ASN1_PREV3A_SPEC).set.asn - -$(EBIN)/$(BER_BIN_DRV_ASN1_PREV3A_SPEC).$(EMULATOR): \ - $(BER_BIN_DRV_ASN1_PREV3A_SPEC).erl - $(PER_ASN1_PREV3A_SPEC).erl: \ $(PER_ASN1_PREV3A_SPEC).set.asn \ $(ASN1_PREV3A_SPEC).asn @@ -241,23 +111,6 @@ $(PER_ASN1_PREV3A_SPEC).erl: \ $(EBIN)/$(PER_ASN1_PREV3A_SPEC).$(EMULATOR): \ $(PER_ASN1_PREV3A_SPEC).erl -$(PER_BIN_ASN1_PREV3A_SPEC).erl: \ - $(PER_BIN_ASN1_PREV3A_SPEC).set.asn \ - $(ASN1_PREV3A_SPEC).asn - @echo "$(PER_BIN_ASN1_PREV3A_SPEC):" - $(ERLC) -bper_bin $(PER_BIN_PREV3A_FLAGS) $(PER_BIN_ASN1_PREV3A_SPEC).set.asn - -$(EBIN)/$(PER_BIN_ASN1_PREV3A_SPEC).$(EMULATOR): \ - $(PER_BIN_ASN1_PREV3A_SPEC).erl - -$(PER_BIN_DRV_ASN1_PREV3A_SPEC).erl: \ - $(PER_BIN_DRV_ASN1_PREV3A_SPEC).set.asn \ - $(ASN1_PREV3A_SPEC).asn - @echo "$(PER_BIN_DRV_ASN1_PREV3A_SPEC):" - $(ERLC) -bper_bin $(PER_BIN_DRV_PREV3A_FLAGS) $(PER_BIN_DRV_ASN1_PREV3A_SPEC).set.asn - -$(EBIN)/$(PER_BIN_DRV_ASN1_PREV3A_SPEC).$(EMULATOR): \ - $(PER_BIN_DRV_ASN1_PREV3A_SPEC).erl # -- (prev3b) -- @@ -270,26 +123,6 @@ $(BER_ASN1_PREV3B_SPEC).erl: \ $(EBIN)/$(BER_ASN1_PREV3B_SPEC).$(EMULATOR): \ $(BER_ASN1_PREV3B_SPEC).erl -$(BER_BIN_ASN1_PREV3B_SPEC).erl: \ - $(BER_BIN_ASN1_PREV3B_SPEC).set.asn \ - $(BER_BIN_ASN1_PREV3B_SPEC).asn1config \ - $(ASN1_PREV3B_SPEC).asn - @echo "$(BER_BIN_ASN1_PREV3B_SPEC):" - $(ERLC) -bber_bin $(BER_BIN_PREV3B_FLAGS) $(BER_BIN_ASN1_PREV3B_SPEC).set.asn - -$(EBIN)/$(BER_BIN_ASN1_PREV3B_SPEC).$(EMULATOR): \ - $(BER_BIN_ASN1_PREV3B_SPEC).erl - -$(BER_BIN_DRV_ASN1_PREV3B_SPEC).erl: \ - $(BER_BIN_DRV_ASN1_PREV3B_SPEC).set.asn \ - $(BER_BIN_DRV_ASN1_PREV3B_SPEC).asn1config \ - $(ASN1_PREV3B_SPEC).asn - @echo "$(BER_BIN_DRV_ASN1_PREV3B_SPEC):" - $(ERLC) -bber_bin $(BER_BIN_DRV_PREV3B_FLAGS) $(BER_BIN_DRV_ASN1_PREV3B_SPEC).set.asn - -$(EBIN)/$(BER_BIN_DRV_ASN1_PREV3B_SPEC).$(EMULATOR): \ - $(BER_BIN_DRV_ASN1_PREV3B_SPEC).erl - $(PER_ASN1_PREV3B_SPEC).erl: \ $(PER_ASN1_PREV3B_SPEC).set.asn \ $(ASN1_PREV3B_SPEC).asn @@ -299,24 +132,6 @@ $(PER_ASN1_PREV3B_SPEC).erl: \ $(EBIN)/$(PER_ASN1_PREV3B_SPEC).$(EMULATOR): \ $(PER_ASN1_PREV3B_SPEC).erl -$(PER_BIN_ASN1_PREV3B_SPEC).erl: \ - $(PER_BIN_ASN1_PREV3B_SPEC).set.asn \ - $(ASN1_PREV3B_SPEC).asn - @echo "$(PER_BIN_ASN1_PREV3B_SPEC):" - $(ERLC) -bper_bin $(PER_BIN_PREV3B_FLAGS) $(PER_BIN_ASN1_PREV3B_SPEC).set.asn - -$(EBIN)/$(PER_BIN_ASN1_PREV3B_SPEC).$(EMULATOR): \ - $(PER_BIN_ASN1_PREV3B_SPEC).erl - -$(PER_BIN_DRV_ASN1_PREV3B_SPEC).erl: \ - $(PER_BIN_DRV_ASN1_PREV3B_SPEC).set.asn \ - $(ASN1_PREV3B_SPEC).asn - @echo "$(PER_BIN_DRV_ASN1_PREV3B_SPEC):" - $(ERLC) -bper_bin $(PER_BIN_DRV_PREV3B_FLAGS) $(PER_BIN_DRV_ASN1_PREV3B_SPEC).set.asn - -$(EBIN)/$(PER_BIN_DRV_ASN1_PREV3B_SPEC).$(EMULATOR): \ - $(PER_BIN_DRV_ASN1_PREV3B_SPEC).erl - # -- (prev3c) -- @@ -329,26 +144,6 @@ $(BER_ASN1_PREV3C_SPEC).erl: \ $(EBIN)/$(BER_ASN1_PREV3C_SPEC).$(EMULATOR): \ $(BER_ASN1_PREV3C_SPEC).erl -$(BER_BIN_ASN1_PREV3C_SPEC).erl: \ - $(BER_BIN_ASN1_PREV3C_SPEC).set.asn \ - $(BER_BIN_ASN1_PREV3C_SPEC).asn1config \ - $(ASN1_PREV3C_SPEC).asn - @echo "$(BER_BIN_ASN1_PREV3C_SPEC):" - $(ERLC) -bber_bin $(BER_BIN_PREV3C_FLAGS) $(BER_BIN_ASN1_PREV3C_SPEC).set.asn - -$(EBIN)/$(BER_BIN_ASN1_PREV3C_SPEC).$(EMULATOR): \ - $(BER_BIN_ASN1_PREV3C_SPEC).erl - -$(BER_BIN_DRV_ASN1_PREV3C_SPEC).erl: \ - $(BER_BIN_DRV_ASN1_PREV3C_SPEC).set.asn \ - $(BER_BIN_DRV_ASN1_PREV3C_SPEC).asn1config \ - $(ASN1_PREV3C_SPEC).asn - @echo "$(BER_BIN_DRV_ASN1_PREV3C_SPEC):" - $(ERLC) -bber_bin $(BER_BIN_DRV_PREV3C_FLAGS) $(BER_BIN_DRV_ASN1_PREV3C_SPEC).set.asn - -$(EBIN)/$(BER_BIN_DRV_ASN1_PREV3C_SPEC).$(EMULATOR): \ - $(BER_BIN_DRV_ASN1_PREV3C_SPEC).erl - $(PER_ASN1_PREV3C_SPEC).erl: \ $(PER_ASN1_PREV3C_SPEC).set.asn \ $(ASN1_PREV3C_SPEC).asn @@ -358,24 +153,6 @@ $(PER_ASN1_PREV3C_SPEC).erl: \ $(EBIN)/$(PER_ASN1_PREV3C_SPEC).$(EMULATOR): \ $(PER_ASN1_PREV3C_SPEC).erl -$(PER_BIN_ASN1_PREV3C_SPEC).erl: \ - $(PER_BIN_ASN1_PREV3C_SPEC).set.asn \ - $(ASN1_PREV3C_SPEC).asn - @echo "$(PER_BIN_ASN1_PREV3C_SPEC):" - $(ERLC) -bper_bin $(PER_BIN_PREV3C_FLAGS) $(PER_BIN_ASN1_PREV3C_SPEC).set.asn - -$(EBIN)/$(PER_BIN_ASN1_PREV3C_SPEC).$(EMULATOR): \ - $(PER_BIN_ASN1_PREV3C_SPEC).erl - -$(PER_BIN_DRV_ASN1_PREV3C_SPEC).erl: \ - $(PER_BIN_DRV_ASN1_PREV3C_SPEC).set.asn \ - $(ASN1_PREV3C_SPEC).asn - @echo "$(PER_BIN_DRV_ASN1_PREV3C_SPEC):" - $(ERLC) -bper_bin $(PER_BIN_DRV_PREV3C_FLAGS) $(PER_BIN_DRV_ASN1_PREV3C_SPEC).set.asn - -$(EBIN)/$(PER_BIN_DRV_ASN1_PREV3C_SPEC).$(EMULATOR): \ - $(PER_BIN_DRV_ASN1_PREV3C_SPEC).erl - # -- (v3) -- @@ -388,26 +165,6 @@ $(BER_ASN1_V3_SPEC).erl: \ $(EBIN)/$(BER_ASN1_V3_SPEC).$(EMULATOR): \ $(BER_ASN1_V3_SPEC).erl -$(BER_BIN_ASN1_V3_SPEC).erl: \ - $(BER_BIN_ASN1_V3_SPEC).set.asn \ - $(BER_BIN_ASN1_V3_SPEC).asn1config \ - $(ASN1_V3_SPEC).asn - @echo "$(BER_BIN_ASN1_V3_SPEC):" - $(ERLC) -bber_bin $(BER_BIN_V3_FLAGS) $(BER_BIN_ASN1_V3_SPEC).set.asn - -$(EBIN)/$(BER_BIN_ASN1_V3_SPEC).$(EMULATOR): \ - $(BER_BIN_ASN1_V3_SPEC).erl - -$(BER_BIN_DRV_ASN1_V3_SPEC).erl: \ - $(BER_BIN_DRV_ASN1_V3_SPEC).set.asn \ - $(BER_BIN_DRV_ASN1_V3_SPEC).asn1config \ - $(ASN1_V3_SPEC).asn - @echo "$(BER_BIN_DRV_ASN1_V3_SPEC):" - $(ERLC) -bber_bin $(BER_BIN_DRV_V3_FLAGS) $(BER_BIN_DRV_ASN1_V3_SPEC).set.asn - -$(EBIN)/$(BER_BIN_DRV_ASN1_V3_SPEC).$(EMULATOR): \ - $(BER_BIN_DRV_ASN1_V3_SPEC).erl - $(PER_ASN1_V3_SPEC).erl: \ $(PER_ASN1_V3_SPEC).set.asn \ $(ASN1_V3_SPEC).asn @@ -417,39 +174,15 @@ $(PER_ASN1_V3_SPEC).erl: \ $(EBIN)/$(PER_ASN1_V3_SPEC).$(EMULATOR): \ $(PER_ASN1_V3_SPEC).erl -$(PER_BIN_ASN1_V3_SPEC).erl: \ - $(PER_BIN_ASN1_V3_SPEC).set.asn \ - $(ASN1_V3_SPEC).asn - @echo "$(PER_BIN_ASN1_V3_SPEC):" - $(ERLC) -bper_bin $(PER_BIN_V3_FLAGS) $(PER_BIN_ASN1_V3_SPEC).set.asn - -$(EBIN)/$(PER_BIN_ASN1_V3_SPEC).$(EMULATOR): \ - $(PER_BIN_ASN1_V3_SPEC).erl - -$(PER_BIN_DRV_ASN1_V3_SPEC).erl: \ - $(PER_BIN_DRV_ASN1_V3_SPEC).set.asn \ - $(ASN1_V3_SPEC).asn - @echo "$(PER_BIN_DRV_ASN1_V3_SPEC):" - $(ERLC) -bper_bin $(PER_BIN_DRV_V3_FLAGS) $(PER_BIN_DRV_ASN1_V3_SPEC).set.asn - -$(EBIN)/$(PER_BIN_DRV_ASN1_V3_SPEC).$(EMULATOR): \ - $(PER_BIN_DRV_ASN1_V3_SPEC).erl - # ------------- $(EBIN)/megaco_ber_encoder.$(EMULATOR): megaco_ber_encoder.erl \ $(MEGACO_ENGINEDIR)/megaco_message_internal.hrl -$(EBIN)/megaco_ber_bin_encoder.$(EMULATOR): megaco_ber_bin_encoder.erl \ - $(MEGACO_ENGINEDIR)/megaco_message_internal.hrl - $(EBIN)/megaco_per_encoder.$(EMULATOR): megaco_per_encoder.erl \ $(MEGACO_ENGINEDIR)/megaco_message_internal.hrl -$(EBIN)/megaco_per_bin_encoder.$(EMULATOR): megaco_per_bin_encoder.erl \ - $(MEGACO_ENGINEDIR)/megaco_message_internal.hrl - $(EBIN)/megaco_binary_encoder_lib.$(EMULATOR): megaco_binary_encoder_lib.erl \ $(MEGACO_ENGINEDIR)/megaco_message_internal.hrl diff --git a/lib/megaco/src/binary/megaco_ber_bin_drv_media_gateway_control_prev3a.asn1config b/lib/megaco/src/binary/megaco_ber_bin_drv_media_gateway_control_prev3a.asn1config deleted file mode 100644 index 179473717d..0000000000 --- a/lib/megaco/src/binary/megaco_ber_bin_drv_media_gateway_control_prev3a.asn1config +++ /dev/null @@ -1,43 +0,0 @@ -{exclusive_decode, - {'megaco_ber_bin_drv_media_gateway_control_prev3a', - [ - {decode_message_trans_partial, - [ - 'MegacoMessage',[{mess,[{messageBody,[{transactions,parts}]}]}] - ] - }, - {decode_message_acts_partial, - ['Transaction', - [ - {transactionRequest, - [ - {actions,parts} - ] - }, - {transactionReply, - [ - {transactionResult, [{actionReplies,parts}]} - ] - } - ] - ] - }, - {decode_message_version, - ['MegacoMessage', - [ - {authHeader,undecoded}, - {mess,[{mId,undecoded},{messageBody,undecoded}]} - ] - ] - }, - {decode_message_mId, - ['MegacoMessage', - [ - {authHeader,undecoded}, - {mess,[{messageBody,undecoded}]} - ] - ] - } - ] - } -}. diff --git a/lib/megaco/src/binary/megaco_ber_bin_drv_media_gateway_control_prev3a.set.asn b/lib/megaco/src/binary/megaco_ber_bin_drv_media_gateway_control_prev3a.set.asn deleted file mode 100644 index b9ba7ffdb4..0000000000 --- a/lib/megaco/src/binary/megaco_ber_bin_drv_media_gateway_control_prev3a.set.asn +++ /dev/null @@ -1 +0,0 @@ -MEDIA-GATEWAY-CONTROL-prev3a.asn diff --git a/lib/megaco/src/binary/megaco_ber_bin_drv_media_gateway_control_prev3b.asn1config b/lib/megaco/src/binary/megaco_ber_bin_drv_media_gateway_control_prev3b.asn1config deleted file mode 100644 index ceda97fbd1..0000000000 --- a/lib/megaco/src/binary/megaco_ber_bin_drv_media_gateway_control_prev3b.asn1config +++ /dev/null @@ -1,43 +0,0 @@ -{exclusive_decode, - {'megaco_ber_bin_drv_media_gateway_control_prev3b', - [ - {decode_message_trans_partial, - [ - 'MegacoMessage',[{mess,[{messageBody,[{transactions,parts}]}]}] - ] - }, - {decode_message_acts_partial, - ['Transaction', - [ - {transactionRequest, - [ - {actions,parts} - ] - }, - {transactionReply, - [ - {transactionResult, [{actionReplies,parts}]} - ] - } - ] - ] - }, - {decode_message_version, - ['MegacoMessage', - [ - {authHeader,undecoded}, - {mess,[{mId,undecoded},{messageBody,undecoded}]} - ] - ] - }, - {decode_message_mId, - ['MegacoMessage', - [ - {authHeader,undecoded}, - {mess,[{messageBody,undecoded}]} - ] - ] - } - ] - } -}. diff --git a/lib/megaco/src/binary/megaco_ber_bin_drv_media_gateway_control_prev3b.set.asn b/lib/megaco/src/binary/megaco_ber_bin_drv_media_gateway_control_prev3b.set.asn deleted file mode 100644 index 0437bde310..0000000000 --- a/lib/megaco/src/binary/megaco_ber_bin_drv_media_gateway_control_prev3b.set.asn +++ /dev/null @@ -1 +0,0 @@ -MEDIA-GATEWAY-CONTROL-prev3b.asn diff --git a/lib/megaco/src/binary/megaco_ber_bin_drv_media_gateway_control_prev3c.asn1config b/lib/megaco/src/binary/megaco_ber_bin_drv_media_gateway_control_prev3c.asn1config deleted file mode 100644 index d181ef44bd..0000000000 --- a/lib/megaco/src/binary/megaco_ber_bin_drv_media_gateway_control_prev3c.asn1config +++ /dev/null @@ -1,43 +0,0 @@ -{exclusive_decode, - {'megaco_ber_bin_drv_media_gateway_control_prev3c', - [ - {decode_message_trans_partial, - [ - 'MegacoMessage',[{mess,[{messageBody,[{transactions,parts}]}]}] - ] - }, - {decode_message_acts_partial, - ['Transaction', - [ - {transactionRequest, - [ - {actions,parts} - ] - }, - {transactionReply, - [ - {transactionResult, [{actionReplies,parts}]} - ] - } - ] - ] - }, - {decode_message_version, - ['MegacoMessage', - [ - {authHeader,undecoded}, - {mess,[{mId,undecoded},{messageBody,undecoded}]} - ] - ] - }, - {decode_message_mId, - ['MegacoMessage', - [ - {authHeader,undecoded}, - {mess,[{messageBody,undecoded}]} - ] - ] - } - ] - } -}. diff --git a/lib/megaco/src/binary/megaco_ber_bin_drv_media_gateway_control_prev3c.set.asn b/lib/megaco/src/binary/megaco_ber_bin_drv_media_gateway_control_prev3c.set.asn deleted file mode 100644 index e78055fbad..0000000000 --- a/lib/megaco/src/binary/megaco_ber_bin_drv_media_gateway_control_prev3c.set.asn +++ /dev/null @@ -1 +0,0 @@ -MEDIA-GATEWAY-CONTROL-prev3c.asn diff --git a/lib/megaco/src/binary/megaco_ber_bin_drv_media_gateway_control_v1.set.asn b/lib/megaco/src/binary/megaco_ber_bin_drv_media_gateway_control_v1.set.asn deleted file mode 100644 index 0f5a92dba1..0000000000 --- a/lib/megaco/src/binary/megaco_ber_bin_drv_media_gateway_control_v1.set.asn +++ /dev/null @@ -1 +0,0 @@ -MEDIA-GATEWAY-CONTROL-v1.asn diff --git a/lib/megaco/src/binary/megaco_ber_bin_drv_media_gateway_control_v2.asn1config b/lib/megaco/src/binary/megaco_ber_bin_drv_media_gateway_control_v2.asn1config deleted file mode 100644 index 3d0cb9a019..0000000000 --- a/lib/megaco/src/binary/megaco_ber_bin_drv_media_gateway_control_v2.asn1config +++ /dev/null @@ -1,43 +0,0 @@ -{exclusive_decode, - {'megaco_ber_bin_drv_media_gateway_control_v2', - [ - {decode_message_trans_partial, - [ - 'MegacoMessage',[{mess,[{messageBody,[{transactions,parts}]}]}] - ] - }, - {decode_message_acts_partial, - ['Transaction', - [ - {transactionRequest, - [ - {actions,parts} - ] - }, - {transactionReply, - [ - {transactionResult, [{actionReplies,parts}]} - ] - } - ] - ] - }, - {decode_message_version, - ['MegacoMessage', - [ - {authHeader,undecoded}, - {mess,[{mId,undecoded},{messageBody,undecoded}]} - ] - ] - }, - {decode_message_mId, - ['MegacoMessage', - [ - {authHeader,undecoded}, - {mess,[{messageBody,undecoded}]} - ] - ] - } - ] - } -}. diff --git a/lib/megaco/src/binary/megaco_ber_bin_drv_media_gateway_control_v2.set.asn b/lib/megaco/src/binary/megaco_ber_bin_drv_media_gateway_control_v2.set.asn deleted file mode 100644 index 7fc82b127f..0000000000 --- a/lib/megaco/src/binary/megaco_ber_bin_drv_media_gateway_control_v2.set.asn +++ /dev/null @@ -1 +0,0 @@ -MEDIA-GATEWAY-CONTROL-v2.asn diff --git a/lib/megaco/src/binary/megaco_ber_bin_drv_media_gateway_control_v3.asn1config b/lib/megaco/src/binary/megaco_ber_bin_drv_media_gateway_control_v3.asn1config deleted file mode 100644 index cc662c0145..0000000000 --- a/lib/megaco/src/binary/megaco_ber_bin_drv_media_gateway_control_v3.asn1config +++ /dev/null @@ -1,43 +0,0 @@ -{exclusive_decode, - {'megaco_ber_bin_drv_media_gateway_control_v3', - [ - {decode_message_trans_partial, - [ - 'MegacoMessage',[{mess,[{messageBody,[{transactions,parts}]}]}] - ] - }, - {decode_message_acts_partial, - ['Transaction', - [ - {transactionRequest, - [ - {actions,parts} - ] - }, - {transactionReply, - [ - {transactionResult, [{actionReplies,parts}]} - ] - } - ] - ] - }, - {decode_message_version, - ['MegacoMessage', - [ - {authHeader,undecoded}, - {mess,[{mId,undecoded},{messageBody,undecoded}]} - ] - ] - }, - {decode_message_mId, - ['MegacoMessage', - [ - {authHeader,undecoded}, - {mess,[{messageBody,undecoded}]} - ] - ] - } - ] - } -}. diff --git a/lib/megaco/src/binary/megaco_ber_bin_drv_media_gateway_control_v3.set.asn b/lib/megaco/src/binary/megaco_ber_bin_drv_media_gateway_control_v3.set.asn deleted file mode 100644 index 1d7950a283..0000000000 --- a/lib/megaco/src/binary/megaco_ber_bin_drv_media_gateway_control_v3.set.asn +++ /dev/null @@ -1 +0,0 @@ -MEDIA-GATEWAY-CONTROL-v3.asn diff --git a/lib/megaco/src/binary/megaco_ber_bin_encoder.erl b/lib/megaco/src/binary/megaco_ber_bin_encoder.erl deleted file mode 100644 index bf9926c7e5..0000000000 --- a/lib/megaco/src/binary/megaco_ber_bin_encoder.erl +++ /dev/null @@ -1,716 +0,0 @@ -%% -%% %CopyrightBegin% -%% -%% Copyright Ericsson AB 2000-2009. All Rights Reserved. -%% -%% The contents of this file are subject to the Erlang Public License, -%% Version 1.1, (the "License"); you may not use this file except in -%% compliance with the License. You should have received a copy of the -%% Erlang Public License along with this software. If not, it can be -%% retrieved online at http://www.erlang.org/. -%% -%% Software distributed under the License is distributed on an "AS IS" -%% basis, WITHOUT WARRANTY OF ANY KIND, either express or implied. See -%% the License for the specific language governing rights and limitations -%% under the License. -%% -%% %CopyrightEnd% -%% - -%% -%%---------------------------------------------------------------------- -%% Purpose : Handle ASN.1 BER encoding of Megaco/H.248 -%%---------------------------------------------------------------------- - --module(megaco_ber_bin_encoder). - --behaviour(megaco_encoder). - --export([encode_message/3, decode_message/3, - decode_mini_message/3, - - encode_transaction/3, - encode_action_requests/3, - encode_action_request/3, - encode_action_reply/3, - - version_of/2]). - -%% Backward compatible functions: --export([encode_message/2, decode_message/2]). - --include_lib("megaco/src/engine/megaco_message_internal.hrl"). - --define(V1_ASN1_MOD, megaco_ber_bin_media_gateway_control_v1). --define(V2_ASN1_MOD, megaco_ber_bin_media_gateway_control_v2). --define(V3_ASN1_MOD, megaco_ber_bin_media_gateway_control_v3). --define(PREV3A_ASN1_MOD, megaco_ber_bin_media_gateway_control_prev3a). --define(PREV3B_ASN1_MOD, megaco_ber_bin_media_gateway_control_prev3b). --define(PREV3C_ASN1_MOD, megaco_ber_bin_media_gateway_control_prev3c). --define(V1_ASN1_MOD_DRV, megaco_ber_bin_drv_media_gateway_control_v1). --define(V2_ASN1_MOD_DRV, megaco_ber_bin_drv_media_gateway_control_v2). --define(V3_ASN1_MOD_DRV, megaco_ber_bin_drv_media_gateway_control_v3). --define(PREV3A_ASN1_MOD_DRV, megaco_ber_bin_drv_media_gateway_control_prev3a). --define(PREV3B_ASN1_MOD_DRV, megaco_ber_bin_drv_media_gateway_control_prev3b). --define(PREV3C_ASN1_MOD_DRV, megaco_ber_bin_drv_media_gateway_control_prev3c). - --define(V1_TRANS_MOD, megaco_binary_transformer_v1). --define(V2_TRANS_MOD, megaco_binary_transformer_v2). --define(V3_TRANS_MOD, megaco_binary_transformer_v3). --define(PREV3A_TRANS_MOD, megaco_binary_transformer_prev3a). --define(PREV3B_TRANS_MOD, megaco_binary_transformer_prev3b). --define(PREV3C_TRANS_MOD, megaco_binary_transformer_prev3c). - --define(BIN_LIB, megaco_binary_encoder_lib). - - -%%---------------------------------------------------------------------- -%% Detect (check/get) message version -%% Return {ok, Version} | {error, Reason} -%%---------------------------------------------------------------------- - -version_of([{version3,v3},driver|EC], Binary) -> - Decoders = [?V1_ASN1_MOD_DRV, ?V2_ASN1_MOD_DRV, ?V3_ASN1_MOD_DRV], - ?BIN_LIB:version_of(EC, Binary, dynamic, Decoders); -version_of([{version3,prev3c},driver|EC], Binary) -> - Decoders = [?V1_ASN1_MOD_DRV, ?V2_ASN1_MOD_DRV, ?PREV3C_ASN1_MOD_DRV], - ?BIN_LIB:version_of(EC, Binary, dynamic, Decoders); -version_of([{version3,prev3b},driver|EC], Binary) -> - Decoders = [?V1_ASN1_MOD_DRV, ?V2_ASN1_MOD_DRV, ?PREV3B_ASN1_MOD_DRV], - ?BIN_LIB:version_of(EC, Binary, dynamic, Decoders); -version_of([{version3,prev3a},driver|EC], Binary) -> - Decoders = [?V1_ASN1_MOD_DRV, ?V2_ASN1_MOD_DRV, ?PREV3A_ASN1_MOD_DRV], - ?BIN_LIB:version_of(EC, Binary, dynamic, Decoders); -version_of([{version3,v3}|EC], Binary) -> - Decoders = [?V1_ASN1_MOD, ?V2_ASN1_MOD, ?V3_ASN1_MOD], - ?BIN_LIB:version_of(EC, Binary, dynamic, Decoders); -version_of([{version3,prev3c}|EC], Binary) -> - Decoders = [?V1_ASN1_MOD, ?V2_ASN1_MOD, ?PREV3C_ASN1_MOD], - ?BIN_LIB:version_of(EC, Binary, dynamic, Decoders); -version_of([{version3,prev3b}|EC], Binary) -> - Decoders = [?V1_ASN1_MOD, ?V2_ASN1_MOD, ?PREV3B_ASN1_MOD], - ?BIN_LIB:version_of(EC, Binary, dynamic, Decoders); -version_of([{version3,prev3a}|EC], Binary) -> - Decoders = [?V1_ASN1_MOD, ?V2_ASN1_MOD, ?PREV3A_ASN1_MOD], - ?BIN_LIB:version_of(EC, Binary, dynamic, Decoders); -version_of([driver|EC], Binary) -> - Decoders = [?V1_ASN1_MOD_DRV, ?V2_ASN1_MOD_DRV, ?V3_ASN1_MOD_DRV], - ?BIN_LIB:version_of(EC, Binary, dynamic, Decoders); - -%% All values we need to take (special) care of has been delt with, -%% so just pass the rest on -version_of(EC, Binary) -> - Decoders = [?V1_ASN1_MOD, ?V2_ASN1_MOD, ?V3_ASN1_MOD], - ?BIN_LIB:version_of(EC, Binary, dynamic, Decoders). - - -%%---------------------------------------------------------------------- -%% Convert a 'MegacoMessage' record into a binary -%% Return {ok, Binary} | {error, Reason} -%%---------------------------------------------------------------------- - - -encode_message(EC, - #'MegacoMessage'{mess = #'Message'{version = V}} = MegaMsg) -> - encode_message(EC, V, MegaMsg). - - -%% -- Version 1 -- - -encode_message([{version3, _},driver|EC], 1, MegaMsg) -> - AsnMod = ?V1_ASN1_MOD_DRV, - TransMod = ?V1_TRANS_MOD, - ?BIN_LIB:encode_message(EC, MegaMsg, AsnMod, TransMod, io_list); -encode_message([driver|EC], 1, MegaMsg) -> - AsnMod = ?V1_ASN1_MOD_DRV, - TransMod = ?V1_TRANS_MOD, - ?BIN_LIB:encode_message(EC, MegaMsg, AsnMod, TransMod, io_list); -encode_message([{version3,_}|EC], 1, MegaMsg) -> - AsnMod = ?V1_ASN1_MOD, - TransMod = ?V1_TRANS_MOD, - ?BIN_LIB:encode_message(EC, MegaMsg, AsnMod, TransMod, io_list); - -%% All values we need to take (special) care of has been delt with, -%% so just pass the rest on -encode_message(EC, 1, MegaMsg) -> - AsnMod = ?V1_ASN1_MOD, - TransMod = ?V1_TRANS_MOD, - ?BIN_LIB:encode_message(EC, MegaMsg, AsnMod, TransMod, io_list); - - -%% -- Version 2 -- - -encode_message([{version3,_},driver|EC], 2, MegaMsg) -> - AsnMod = ?V2_ASN1_MOD_DRV, - TransMod = ?V2_TRANS_MOD, - ?BIN_LIB:encode_message(EC, MegaMsg, AsnMod, TransMod, io_list); -encode_message([driver|EC], 2, MegaMsg) -> - AsnMod = ?V2_ASN1_MOD_DRV, - TransMod = ?V2_TRANS_MOD, - ?BIN_LIB:encode_message(EC, MegaMsg, AsnMod, TransMod, io_list); -encode_message([{version3,_}|EC], 2, MegaMsg) -> - AsnMod = ?V2_ASN1_MOD, - TransMod = ?V2_TRANS_MOD, - ?BIN_LIB:encode_message(EC, MegaMsg, AsnMod, TransMod, io_list); - -%% All values we need to take (special) care of has been delt with, -%% so just pass the rest on -encode_message(EC, 2, MegaMsg) -> - AsnMod = ?V2_ASN1_MOD, - TransMod = ?V2_TRANS_MOD, - ?BIN_LIB:encode_message(EC, MegaMsg, AsnMod, TransMod, io_list); - - -%% -- Version 3 -- - -encode_message([{version3,v3},driver|EC], 3, MegaMsg) -> - AsnMod = ?V3_ASN1_MOD_DRV, - TransMod = ?V3_TRANS_MOD, - ?BIN_LIB:encode_message(EC, MegaMsg, AsnMod, TransMod, io_list); -encode_message([{version3,prev3c},driver|EC], 3, MegaMsg) -> - AsnMod = ?PREV3C_ASN1_MOD_DRV, - TransMod = ?PREV3C_TRANS_MOD, - ?BIN_LIB:encode_message(EC, MegaMsg, AsnMod, TransMod, io_list); -encode_message([{version3,prev3b},driver|EC], 3, MegaMsg) -> - AsnMod = ?PREV3B_ASN1_MOD_DRV, - TransMod = ?PREV3B_TRANS_MOD, - ?BIN_LIB:encode_message(EC, MegaMsg, AsnMod, TransMod, io_list); -encode_message([{version3,prev3a},driver|EC], 3, MegaMsg) -> - AsnMod = ?PREV3A_ASN1_MOD_DRV, - TransMod = ?PREV3A_TRANS_MOD, - ?BIN_LIB:encode_message(EC, MegaMsg, AsnMod, TransMod, io_list); -encode_message([{version3,v3}|EC], 3, MegaMsg) -> - AsnMod = ?V3_ASN1_MOD, - TransMod = ?V3_TRANS_MOD, - ?BIN_LIB:encode_message(EC, MegaMsg, AsnMod, TransMod, io_list); -encode_message([{version3,prev3c}|EC], 3, MegaMsg) -> - AsnMod = ?PREV3C_ASN1_MOD, - TransMod = ?PREV3C_TRANS_MOD, - ?BIN_LIB:encode_message(EC, MegaMsg, AsnMod, TransMod, io_list); -encode_message([{version3,prev3b}|EC], 3, MegaMsg) -> - AsnMod = ?PREV3B_ASN1_MOD, - TransMod = ?PREV3B_TRANS_MOD, - ?BIN_LIB:encode_message(EC, MegaMsg, AsnMod, TransMod, io_list); -encode_message([{version3,prev3a}|EC], 3, MegaMsg) -> - AsnMod = ?PREV3A_ASN1_MOD, - TransMod = ?PREV3A_TRANS_MOD, - ?BIN_LIB:encode_message(EC, MegaMsg, AsnMod, TransMod, io_list); -encode_message([driver|EC], 3, MegaMsg) -> - AsnMod = ?V3_ASN1_MOD_DRV, - TransMod = ?V3_TRANS_MOD, - ?BIN_LIB:encode_message(EC, MegaMsg, AsnMod, TransMod, io_list); - -%% All values we need to take (special) care of has been delt with, -%% so just pass the rest on -encode_message(EC, 3, MegaMsg) -> - AsnMod = ?V3_ASN1_MOD, - TransMod = ?V3_TRANS_MOD, - ?BIN_LIB:encode_message(EC, MegaMsg, AsnMod, TransMod, io_list). - - -%%---------------------------------------------------------------------- -%% Convert a transaction (or transactions in the case of ack) record(s) -%% into a binary -%% Return {ok, Binary} | {error, Reason} -%%---------------------------------------------------------------------- - -%% encode_transaction([] = EC, 1, Trans) -> -%% AsnMod = ?V1_ASN1_MOD, -%% TransMod = ?V1_TRANS_MOD, -%% ?BIN_LIB:encode_transaction(EC, Trans, AsnMod, TransMod, -%% io_list); -%% encode_transaction([native] = EC, 1, Trans) -> -%% AsnMod = ?V1_ASN1_MOD, -%% TransMod = ?V1_TRANS_MOD, -%% ?BIN_LIB:encode_transaction(EC, Trans, AsnMod, TransMod, -%% io_list); -%% encode_transaction([driver|EC], 1, Trans) -> -%% AsnMod = ?V1_ASN1_MOD_DRV, -%% TransMod = ?V1_TRANS_MOD, -%% ?BIN_LIB:encode_transaction(EC, Trans, AsnMod, TransMod, -%% io_list); -%% encode_transaction(_EC, 1, _Trans) -> -%% AsnMod = ?V1_ASN1_MOD, -%% TransMod = ?V1_TRANS_MOD, -%% ?BIN_LIB:encode_transaction(EC, Trans, AsnMod, TransMod, -%% io_list); -%% encode_transaction([] = EC, 2, Trans) -> -%% AsnMod = ?V2_ASN1_MOD, -%% TransMod = ?V2_TRANS_MOD, -%% ?BIN_LIB:encode_transaction(EC, Trans, AsnMod, TransMod, -%% io_list); -%% encode_transaction([native] = EC, 2, Trans) -> -%% AsnMod = ?V2_ASN1_MOD, -%% TransMod = ?V2_TRANS_MOD, -%% ?BIN_LIB:encode_transaction(EC, Trans, AsnMod, TransMod, -%% io_list); -%% encode_transaction([driver|EC], 2, Trans) -> -%% AsnMod = ?V2_ASN1_MOD_DRV, -%% TransMod = ?V2_TRANS_MOD, -%% ?BIN_LIB:encode_transaction(EC, Trans, AsnMod, TransMod, -%% io_list); -%%encode_transaction(_EC, 2, _Trans) -> -%% AsnMod = ?V2_ASN1_MOD, -%% TransMod = ?V2_TRANS_MOD, -%% ?BIN_LIB:encode_transaction(EC, Trans, AsnMod, TransMod, -%% io_list). -%% encode_transaction([] = EC, 3, Trans) -> -%% AsnMod = ?V3_ASN1_MOD, -%% TransMod = ?V3_TRANS_MOD, -%% ?BIN_LIB:encode_transaction(EC, Trans, AsnMod, TransMod, -%% io_list); -%% encode_transaction([native] = EC, 3, Trans) -> -%% AsnMod = ?V3_ASN1_MOD, -%% TransMod = ?V3_TRANS_MOD, -%% ?BIN_LIB:encode_transaction(EC, Trans, AsnMod, TransMod, -%% io_list); -%% encode_transaction([driver|EC], 3, Trans) -> -%% AsnMod = ?V3_ASN1_MOD_DRV, -%% TransMod = ?V3_TRANS_MOD, -%% ?BIN_LIB:encode_transaction(EC, Trans, AsnMod, TransMod, -%% io_list); -%%encode_transaction(_EC, 3, _Trans) -> -%% AsnMod = ?V3_ASN1_MOD, -%% TransMod = ?V3_TRANS_MOD, -%% ?BIN_LIB:encode_transaction(EC, Trans, AsnMod, TransMod, -%% io_list). -encode_transaction(_EC, _V, _Trans) -> - {error, not_implemented}. - - -%%---------------------------------------------------------------------- -%% Convert a list of ActionRequest record's into a binary -%% Return {ok, DeepIoList} | {error, Reason} -%%---------------------------------------------------------------------- -%% encode_action_requests([] = EC, 1, ActReqs) when is_list(ActReqs) -> -%% AsnMod = ?V1_ASN1_MOD, -%% TransMod = ?V1_TRANS_MOD, -%% ?BIN_LIB:encode_action_requests(EC, ActReqs, -%% AsnMod, TransMod, -%% io_list); -%% encode_action_requests([native] = EC, 1, ActReqs) when is_list(ActReqs) -> -%% AsnMod = ?V1_ASN1_MOD, -%% TransMod = ?V1_TRANS_MOD, -%% ?BIN_LIB:encode_action_requests(EC, ActReqs, -%% AsnMod, TransMod, -%% io_list); -%% encode_action_requests([driver|EC], 1, ActReqs) when is_list(ActReqs) -> -%% AsnMod = ?V1_ASN1_MOD_DRV, -%% TransMod = ?V1_TRANS_MOD, -%% ?BIN_LIB:encode_action_requests(EC, ActReqs, -%% AsnMod, TransMod, -%% io_list); -%% encode_action_requests(_EC, 1, ActReqs) when is_list(ActReqs) -> -%% AsnMod = ?V1_ASN1_MOD, -%% TransMod = ?V1_TRANS_MOD, -%% ?BIN_LIB:encode_action_requests(EC, ActReqs, -%% AsnMod, TransMod, -%% io_list); -%% encode_action_requests([] = EC, 2, ActReqs) when is_list(ActReqs) -> -%% AsnMod = ?V2_ASN1_MOD, -%% TransMod = ?V2_TRANS_MOD, -%% ?BIN_LIB:encode_action_requests(EC, ActReqs, -%% AsnMod, TransMod, -%% io_list); -%% encode_action_requests([native] = EC, 2, ActReqs) when is_list(ActReqs) -> -%% AsnMod = ?V2_ASN1_MOD, -%% TransMod = ?V2_TRANS_MOD, -%% ?BIN_LIB:encode_action_requests(EC, ActReqs, -%% AsnMod, TransMod, -%% io_list); -%% encode_action_requests([driver|EC], 2, ActReqs) when is_list(ActReqs) -> -%% AsnMod = ?V2_ASN1_MOD_DRV, -%% TransMod = ?V2_TRANS_MOD, -%% ?BIN_LIB:encode_action_requests(EC, ActReqs, -%% AsnMod, TransMod, -%% io_list); -%% encode_action_requests(_EC, 2, ActReqs) when is_list(ActReqs) -> -%% AsnMod = ?V2_ASN1_MOD, -%% TransMod = ?V2_TRANS_MOD, -%% ?BIN_LIB:encode_action_requests(EC, ActReqs, -%% AsnMod, TransMod, -%% io_list); -%% encode_action_requests([] = EC, 3, ActReqs) when is_list(ActReqs) -> -%% AsnMod = ?V3_ASN1_MOD, -%% TransMod = ?V3_TRANS_MOD, -%% ?BIN_LIB:encode_action_requests(EC, ActReqs, -%% AsnMod, TransMod, -%% io_list); -%% encode_action_requests([native] = EC, 3, ActReqs) when is_list(ActReqs) -> -%% AsnMod = ?V3_ASN1_MOD, -%% TransMod = ?V3_TRANS_MOD, -%% ?BIN_LIB:encode_action_requests(EC, ActReqs, -%% AsnMod, TransMod, -%% io_list); -%% encode_action_requests([driver|EC], 3, ActReqs) when is_list(ActReqs) -> -%% AsnMod = ?V3_ASN1_MOD_DRV, -%% TransMod = ?V3_TRANS_MOD, -%% ?BIN_LIB:encode_action_requests(EC, ActReqs, -%% AsnMod, TransMod, -%% io_list); -%%encode_action_requests(_EC, 3, ActReqs) when is_list(ActReqs) -> -%% AsnMod = ?V3_ASN1_MOD, -%% TransMod = ?V3_TRANS_MOD, -%% ?BIN_LIB:encode_action_requests(EC, ActReqs, -%% AsnMod, TransMod, -%% io_list); -%% encode_action_requests(_EC, V, _ActReqs) -> -%% {error, {bad_version, V}}. -encode_action_requests(_EC, _V, _ActReqs) -> - {error, not_implemented}. - - -%%---------------------------------------------------------------------- -%% Convert a ActionRequest record into a binary -%% Return {ok, DeepIoList} | {error, Reason} -%%---------------------------------------------------------------------- -%% encode_action_request([] = EC, 1, ActReq) -> -%% AsnMod = ?V1_ASN1_MOD, -%% TransMod = ?V1_TRANS_MOD, -%% ?BIN_LIB:encode_action_request(EC, ActReq, -%% AsnMod, TransMod, -%% io_list); -%% encode_action_request([native] = EC, 1, ActReq) -> -%% AsnMod = ?V1_ASN1_MOD, -%% TransMod = ?V1_TRANS_MOD, -%% ?BIN_LIB:encode_action_request(EC, ActReq, -%% AsnMod, TransMod, -%% io_list); -%% encode_action_request([driver|EC], 1, ActReq) -> -%% AsnMod = ?V1_ASN1_MOD_DRV, -%% TransMod = ?V1_TRANS_MOD, -%% ?BIN_LIB:encode_action_request(EC, ActReq, -%% AsnMod, TransMod, -%% io_list); -%% encode_action_request(_EC, 1, ActReq) -> -%% AsnMod = ?V1_ASN1_MOD, -%% TransMod = ?V1_TRANS_MOD, -%% ?BIN_LIB:encode_action_request(EC, ActReq, -%% AsnMod, TransMod, -%% io_list); -%% encode_action_request([] = EC, 2, ActReq) -> -%% AsnMod = ?V2_ASN1_MOD, -%% TransMod = ?V2_TRANS_MOD, -%% ?BIN_LIB:encode_action_request(EC, ActReq, -%% AsnMod, TransMod, -%% io_list); -%% encode_action_request([native] = EC, 2, ActReq) -> -%% AsnMod = ?V2_ASN1_MOD, -%% TransMod = ?V2_TRANS_MOD, -%% ?BIN_LIB:encode_action_request(EC, ActReq, -%% AsnMod, TransMod, -%% io_list); -%% encode_action_request([driver|EC], 2, ActReq) -> -%% AsnMod = ?V2_ASN1_MOD_DRV, -%% TransMod = ?V2_TRANS_MOD, -%% ?BIN_LIB:encode_action_request(EC, ActReq, -%% AsnMod, TransMod, -%% io_list); -%% encode_action_request(_EC, 2, ActReq) -> -%% AsnMod = ?V2_ASN1_MOD, -%% TransMod = ?V2_TRANS_MOD, -%% ?BIN_LIB:encode_action_request(EC, ActReq, -%% AsnMod, TransMod, -%% io_list); -%% encode_action_request([] = EC, 3, ActReq) -> -%% AsnMod = ?V3_ASN1_MOD, -%% TransMod = ?V3_TRANS_MOD, -%% ?BIN_LIB:encode_action_request(EC, ActReq, -%% AsnMod, TransMod, -%% io_list); -%% encode_action_request([native] = EC, 3, ActReq) -> -%% AsnMod = ?V3_ASN1_MOD, -%% TransMod = ?V3_TRANS_MOD, -%% ?BIN_LIB:encode_action_request(EC, ActReq, -%% AsnMod, TransMod, -%% io_list); -%% encode_action_request([driver|EC], 3, ActReq) -> -%% AsnMod = ?V3_ASN1_MOD_DRV, -%% TransMod = ?V3_TRANS_MOD, -%% ?BIN_LIB:encode_action_request(EC, ActReq, -%% AsnMod, TransMod, -%% io_list); -%% encode_action_request(_EC, 3, ActReq) -> -%% AsnMod = ?V3_ASN1_MOD, -%% TransMod = ?V3_TRANS_MOD, -%% ?BIN_LIB:encode_action_request(EC, ActReq, -%% AsnMod, TransMod, -%% io_list); -encode_action_request(_EC, _V, _ActReq) -> - {error, not_implemented}. - - -%%---------------------------------------------------------------------- -%% Convert a action reply into a deep io list -%% Not yest supported by this binary codec! -%% Return {ok, DeepIoList} | {error, Reason} -%%---------------------------------------------------------------------- - -encode_action_reply(_EC, _V, _AcionReply) -> - {error, not_implemented}. - - -%%---------------------------------------------------------------------- -%% Convert a binary into a 'MegacoMessage' record -%% Return {ok, MegacoMessageRecord} | {error, Reason} -%%---------------------------------------------------------------------- - -%% Old decode function -decode_message(EC, Binary) -> - decode_message(EC, 1, Binary). - -%% -- Dynamic version detection -- - -%% Select from message -decode_message([{version3,v3},driver|EC], dynamic, Binary) -> - Mods = [{?V1_ASN1_MOD_DRV, ?V1_TRANS_MOD}, - {?V2_ASN1_MOD_DRV, ?V2_TRANS_MOD}, - {?V3_ASN1_MOD_DRV, ?V3_TRANS_MOD}], - ?BIN_LIB:decode_message_dynamic(EC, Binary, Mods, binary); -decode_message([{version3,prev3c},driver|EC], dynamic, Binary) -> - Mods = [{?V1_ASN1_MOD_DRV, ?V1_TRANS_MOD}, - {?V2_ASN1_MOD_DRV, ?V2_TRANS_MOD}, - {?PREV3C_ASN1_MOD_DRV, ?PREV3C_TRANS_MOD}], - ?BIN_LIB:decode_message_dynamic(EC, Binary, Mods, binary); -decode_message([{version3,prev3b},driver|EC], dynamic, Binary) -> - Mods = [{?V1_ASN1_MOD_DRV, ?V1_TRANS_MOD}, - {?V2_ASN1_MOD_DRV, ?V2_TRANS_MOD}, - {?PREV3B_ASN1_MOD_DRV, ?PREV3B_TRANS_MOD}], - ?BIN_LIB:decode_message_dynamic(EC, Binary, Mods, binary); -decode_message([{version3,prev3a},driver|EC], dynamic, Binary) -> - Mods = [{?V1_ASN1_MOD_DRV, ?V1_TRANS_MOD}, - {?V2_ASN1_MOD_DRV, ?V2_TRANS_MOD}, - {?PREV3A_ASN1_MOD_DRV, ?PREV3A_TRANS_MOD}], - ?BIN_LIB:decode_message_dynamic(EC, Binary, Mods, binary); -decode_message([{version3,v3}|EC], dynamic, Binary) -> - Mods = [{?V1_ASN1_MOD, ?V1_TRANS_MOD}, - {?V2_ASN1_MOD, ?V2_TRANS_MOD}, - {?V3_ASN1_MOD, ?V3_TRANS_MOD}], - ?BIN_LIB:decode_message_dynamic(EC, Binary, Mods, binary); -decode_message([{version3,prev3c}|EC], dynamic, Binary) -> - Mods = [{?V1_ASN1_MOD, ?V1_TRANS_MOD}, - {?V2_ASN1_MOD, ?V2_TRANS_MOD}, - {?PREV3C_ASN1_MOD, ?PREV3C_TRANS_MOD}], - ?BIN_LIB:decode_message_dynamic(EC, Binary, Mods, binary); -decode_message([{version3,prev3b}|EC], dynamic, Binary) -> - Mods = [{?V1_ASN1_MOD, ?V1_TRANS_MOD}, - {?V2_ASN1_MOD, ?V2_TRANS_MOD}, - {?PREV3B_ASN1_MOD, ?PREV3B_TRANS_MOD}], - ?BIN_LIB:decode_message_dynamic(EC, Binary, Mods, binary); -decode_message([{version3,prev3a}|EC], dynamic, Binary) -> - Mods = [{?V1_ASN1_MOD, ?V1_TRANS_MOD}, - {?V2_ASN1_MOD, ?V2_TRANS_MOD}, - {?PREV3A_ASN1_MOD, ?PREV3A_TRANS_MOD}], - ?BIN_LIB:decode_message_dynamic(EC, Binary, Mods, binary); -decode_message([driver|EC], dynamic, Binary) -> - Mods = [{?V1_ASN1_MOD_DRV, ?V1_TRANS_MOD}, - {?V2_ASN1_MOD_DRV, ?V2_TRANS_MOD}, - {?V3_ASN1_MOD_DRV, ?V3_TRANS_MOD}], - ?BIN_LIB:decode_message_dynamic(EC, Binary, Mods, binary); - -%% All values we need to take (special) care of has been delt with, -%% so just pass the rest on -decode_message(EC, dynamic, Binary) -> - Mods = [{?V1_ASN1_MOD, ?V1_TRANS_MOD}, - {?V2_ASN1_MOD, ?V2_TRANS_MOD}, - {?V3_ASN1_MOD, ?V3_TRANS_MOD}], - ?BIN_LIB:decode_message_dynamic(EC, Binary, Mods, binary); - - -%% -- Version 1 -- - -decode_message([{version3,_},driver|EC], 1, Binary) -> - AsnMod = ?V1_ASN1_MOD_DRV, - TransMod = ?V1_TRANS_MOD, - ?BIN_LIB:decode_message(EC, Binary, AsnMod, TransMod, binary); -decode_message([driver|EC], 1, Binary) -> - AsnMod = ?V1_ASN1_MOD_DRV, - TransMod = ?V1_TRANS_MOD, - ?BIN_LIB:decode_message(EC, Binary, AsnMod, TransMod, binary); -decode_message([{version3,_}|EC], 1, Binary) -> - AsnMod = ?V1_ASN1_MOD, - TransMod = ?V1_TRANS_MOD, - ?BIN_LIB:decode_message(EC, Binary, AsnMod, TransMod, binary); - -%% All values we need to take (special) care of has been delt with, -%% so just pass the rest on -decode_message(EC, 1, Binary) -> - AsnMod = ?V1_ASN1_MOD, - TransMod = ?V1_TRANS_MOD, - ?BIN_LIB:decode_message(EC, Binary, AsnMod, TransMod, binary); - - -%% -- Version 2 -- - -decode_message([{version3,_},driver|EC], 2, Binary) -> - AsnMod = ?V2_ASN1_MOD_DRV, - TransMod = ?V2_TRANS_MOD, - ?BIN_LIB:decode_message(EC, Binary, AsnMod, TransMod, binary); -decode_message([driver|EC], 2, Binary) -> - AsnMod = ?V2_ASN1_MOD_DRV, - TransMod = ?V2_TRANS_MOD, - ?BIN_LIB:decode_message(EC, Binary, AsnMod, TransMod, binary); -decode_message([{version3,_}|EC], 2, Binary) -> - AsnMod = ?V2_ASN1_MOD, - TransMod = ?V2_TRANS_MOD, - ?BIN_LIB:decode_message(EC, Binary, AsnMod, TransMod, binary); - -%% All values we need to take (special) care of has been delt with, -%% so just pass the rest on -decode_message(EC, 2, Binary) -> - AsnMod = ?V2_ASN1_MOD, - TransMod = ?V2_TRANS_MOD, - ?BIN_LIB:decode_message(EC, Binary, AsnMod, TransMod, binary); - - -%% -- Version 3 -- - -decode_message([{version3,v3},driver|EC], 3, Binary) -> - AsnMod = ?V3_ASN1_MOD_DRV, - TransMod = ?V3_TRANS_MOD, - ?BIN_LIB:decode_message(EC, Binary, AsnMod, TransMod, binary); -decode_message([{version3,prev3c},driver|EC], 3, Binary) -> - AsnMod = ?PREV3C_ASN1_MOD_DRV, - TransMod = ?PREV3C_TRANS_MOD, - ?BIN_LIB:decode_message(EC, Binary, AsnMod, TransMod, binary); -decode_message([{version3,prev3b},driver|EC], 3, Binary) -> - AsnMod = ?PREV3B_ASN1_MOD_DRV, - TransMod = ?PREV3B_TRANS_MOD, - ?BIN_LIB:decode_message(EC, Binary, AsnMod, TransMod, binary); -decode_message([{version3,prev3a},driver|EC], 3, Binary) -> - AsnMod = ?PREV3A_ASN1_MOD_DRV, - TransMod = ?PREV3A_TRANS_MOD, - ?BIN_LIB:decode_message(EC, Binary, AsnMod, TransMod, binary); -decode_message([{version3,v3}|EC], 3, Binary) -> - AsnMod = ?V3_ASN1_MOD, - TransMod = ?V3_TRANS_MOD, - ?BIN_LIB:decode_message(EC, Binary, AsnMod, TransMod, binary); -decode_message([{version3,prev3c}|EC], 3, Binary) -> - AsnMod = ?PREV3C_ASN1_MOD, - TransMod = ?PREV3C_TRANS_MOD, - ?BIN_LIB:decode_message(EC, Binary, AsnMod, TransMod, binary); -decode_message([{version3,prev3b}|EC], 3, Binary) -> - AsnMod = ?PREV3B_ASN1_MOD, - TransMod = ?PREV3B_TRANS_MOD, - ?BIN_LIB:decode_message(EC, Binary, AsnMod, TransMod, binary); -decode_message([{version3,prev3a}|EC], 3, Binary) -> - AsnMod = ?PREV3A_ASN1_MOD, - TransMod = ?PREV3A_TRANS_MOD, - ?BIN_LIB:decode_message(EC, Binary, AsnMod, TransMod, binary); -decode_message([driver|EC], 3, Binary) -> - AsnMod = ?V3_ASN1_MOD_DRV, - TransMod = ?V3_TRANS_MOD, - ?BIN_LIB:decode_message(EC, Binary, AsnMod, TransMod, binary); - -%% All values we need to take (special) care of has been delt with, -%% so just pass the rest on -decode_message(EC, 3, Binary) -> - AsnMod = ?V3_ASN1_MOD, - TransMod = ?V3_TRANS_MOD, - ?BIN_LIB:decode_message(EC, Binary, AsnMod, TransMod, binary). - - -decode_mini_message([{version3,v3},driver|EC], dynamic, Bin) -> - Mods = [?V1_ASN1_MOD_DRV, - ?V2_ASN1_MOD_DRV, - ?V3_ASN1_MOD_DRV], - ?BIN_LIB:decode_mini_message_dynamic(EC, Bin, Mods, binary); -decode_mini_message([{version3,prev3c},driver|EC], dynamic, Bin) -> - Mods = [?V1_ASN1_MOD_DRV, - ?V2_ASN1_MOD_DRV, - ?PREV3C_ASN1_MOD_DRV], - ?BIN_LIB:decode_mini_message_dynamic(EC, Bin, Mods, binary); -decode_mini_message([{version3,prev3b},driver|EC], dynamic, Bin) -> - Mods = [?V1_ASN1_MOD_DRV, - ?V2_ASN1_MOD_DRV, - ?PREV3B_ASN1_MOD_DRV], - ?BIN_LIB:decode_mini_message_dynamic(EC, Bin, Mods, binary); -decode_mini_message([{version3,prev3a},driver|EC], dynamic, Bin) -> - Mods = [?V1_ASN1_MOD_DRV, - ?V2_ASN1_MOD_DRV, - ?PREV3A_ASN1_MOD_DRV], - ?BIN_LIB:decode_mini_message_dynamic(EC, Bin, Mods, binary); -decode_mini_message([{version3,v3}|EC], dynamic, Bin) -> - Mods = [?V1_ASN1_MOD, - ?V2_ASN1_MOD, - ?V3_ASN1_MOD], - ?BIN_LIB:decode_mini_message_dynamic(EC, Bin, Mods, binary); -decode_mini_message([{version3,prev3c}|EC], dynamic, Bin) -> - Mods = [?V1_ASN1_MOD, - ?V2_ASN1_MOD, - ?PREV3C_ASN1_MOD], - ?BIN_LIB:decode_mini_message_dynamic(EC, Bin, Mods, binary); -decode_mini_message([{version3,prev3b}|EC], dynamic, Bin) -> - Mods = [?V1_ASN1_MOD, - ?V2_ASN1_MOD, - ?PREV3B_ASN1_MOD], - ?BIN_LIB:decode_mini_message_dynamic(EC, Bin, Mods, binary); -decode_mini_message([{version3,prev3a}|EC], dynamic, Bin) -> - Mods = [?V1_ASN1_MOD, - ?V2_ASN1_MOD, - ?PREV3A_ASN1_MOD], - ?BIN_LIB:decode_mini_message_dynamic(EC, Bin, Mods, binary); -decode_mini_message([driver|EC], dynamic, Bin) -> - Mods = [?V1_ASN1_MOD_DRV, - ?V2_ASN1_MOD_DRV, - ?V3_ASN1_MOD_DRV], - ?BIN_LIB:decode_mini_message_dynamic(EC, Bin, Mods, binary); -decode_mini_message(EC, dynamic, Bin) -> - Mods = [?V1_ASN1_MOD, - ?V2_ASN1_MOD, - ?V3_ASN1_MOD], - ?BIN_LIB:decode_mini_message_dynamic(EC, Bin, Mods, binary); -decode_mini_message([{version3,_},driver|EC], 1, Bin) -> - AsnMod = ?V1_ASN1_MOD_DRV, - ?BIN_LIB:decode_mini_message(EC, Bin, AsnMod, binary); -decode_mini_message([{version3,_}|EC], 1, Bin) -> - AsnMod = ?V1_ASN1_MOD, - ?BIN_LIB:decode_mini_message(EC, Bin, AsnMod, binary); -decode_mini_message([driver|EC], 1, Bin) -> - AsnMod = ?V1_ASN1_MOD_DRV, - ?BIN_LIB:decode_mini_message(EC, Bin, AsnMod, binary); -decode_mini_message(EC, 1, Bin) -> - AsnMod = ?V1_ASN1_MOD, - ?BIN_LIB:decode_mini_message(EC, Bin, AsnMod, binary); -decode_mini_message([{version3,_},driver|EC], 2, Bin) -> - AsnMod = ?V2_ASN1_MOD_DRV, - ?BIN_LIB:decode_mini_message(EC, Bin, AsnMod, binary); -decode_mini_message([{version3,_}|EC], 2, Bin) -> - AsnMod = ?V2_ASN1_MOD, - ?BIN_LIB:decode_mini_message(EC, Bin, AsnMod, binary); -decode_mini_message([driver|EC], 2, Bin) -> - AsnMod = ?V2_ASN1_MOD_DRV, - ?BIN_LIB:decode_mini_message(EC, Bin, AsnMod, binary); -decode_mini_message(EC, 2, Bin) -> - AsnMod = ?V2_ASN1_MOD, - ?BIN_LIB:decode_mini_message(EC, Bin, AsnMod, binary); -decode_mini_message([{version3,v3},driver|EC], 3, Bin) -> - AsnMod = ?V3_ASN1_MOD_DRV, - ?BIN_LIB:decode_mini_message(EC, Bin, AsnMod, binary); -decode_mini_message([{version3,prev3c},driver|EC], 3, Bin) -> - AsnMod = ?PREV3C_ASN1_MOD_DRV, - ?BIN_LIB:decode_mini_message(EC, Bin, AsnMod, binary); -decode_mini_message([{version3,prev3b},driver|EC], 3, Bin) -> - AsnMod = ?PREV3B_ASN1_MOD_DRV, - ?BIN_LIB:decode_mini_message(EC, Bin, AsnMod, binary); -decode_mini_message([{version3,prev3a},driver|EC], 3, Bin) -> - AsnMod = ?PREV3A_ASN1_MOD_DRV, - ?BIN_LIB:decode_mini_message(EC, Bin, AsnMod, binary); -decode_mini_message([{version3,v3}|EC], 3, Bin) -> - AsnMod = ?V3_ASN1_MOD, - ?BIN_LIB:decode_mini_message(EC, Bin, AsnMod, binary); -decode_mini_message([{version3,prev3c}|EC], 3, Bin) -> - AsnMod = ?PREV3C_ASN1_MOD, - ?BIN_LIB:decode_mini_message(EC, Bin, AsnMod, binary); -decode_mini_message([{version3,prev3b}|EC], 3, Bin) -> - AsnMod = ?PREV3B_ASN1_MOD, - ?BIN_LIB:decode_mini_message(EC, Bin, AsnMod, binary); -decode_mini_message([{version3,prev3a}|EC], 3, Bin) -> - AsnMod = ?PREV3A_ASN1_MOD, - ?BIN_LIB:decode_mini_message(EC, Bin, AsnMod, binary); -decode_mini_message([driver|EC], 3, Bin) -> - AsnMod = ?V3_ASN1_MOD_DRV, - ?BIN_LIB:decode_mini_message(EC, Bin, AsnMod, binary); -decode_mini_message(EC, 3, Bin) -> - AsnMod = ?V3_ASN1_MOD, - ?BIN_LIB:decode_mini_message(EC, Bin, AsnMod, binary). diff --git a/lib/megaco/src/binary/megaco_ber_bin_media_gateway_control_prev3a.set.asn b/lib/megaco/src/binary/megaco_ber_bin_media_gateway_control_prev3a.set.asn deleted file mode 100644 index b9ba7ffdb4..0000000000 --- a/lib/megaco/src/binary/megaco_ber_bin_media_gateway_control_prev3a.set.asn +++ /dev/null @@ -1 +0,0 @@ -MEDIA-GATEWAY-CONTROL-prev3a.asn diff --git a/lib/megaco/src/binary/megaco_ber_bin_media_gateway_control_prev3b.set.asn b/lib/megaco/src/binary/megaco_ber_bin_media_gateway_control_prev3b.set.asn deleted file mode 100644 index 0437bde310..0000000000 --- a/lib/megaco/src/binary/megaco_ber_bin_media_gateway_control_prev3b.set.asn +++ /dev/null @@ -1 +0,0 @@ -MEDIA-GATEWAY-CONTROL-prev3b.asn diff --git a/lib/megaco/src/binary/megaco_ber_bin_media_gateway_control_prev3c.asn1config b/lib/megaco/src/binary/megaco_ber_bin_media_gateway_control_prev3c.asn1config deleted file mode 100644 index c74422b9a2..0000000000 --- a/lib/megaco/src/binary/megaco_ber_bin_media_gateway_control_prev3c.asn1config +++ /dev/null @@ -1,43 +0,0 @@ -{exclusive_decode, - {'megaco_ber_bin_media_gateway_control_prev3c', - [ - {decode_message_trans_partial, - [ - 'MegacoMessage',[{mess,[{messageBody,[{transactions,parts}]}]}] - ] - }, - {decode_message_acts_partial, - ['Transaction', - [ - {transactionRequest, - [ - {actions,parts} - ] - }, - {transactionReply, - [ - {transactionResult, [{actionReplies,parts}]} - ] - } - ] - ] - }, - {decode_message_version, - ['MegacoMessage', - [ - {authHeader,undecoded}, - {mess,[{mId,undecoded},{messageBody,undecoded}]} - ] - ] - }, - {decode_message_mId, - ['MegacoMessage', - [ - {authHeader,undecoded}, - {mess,[{messageBody,undecoded}]} - ] - ] - } - ] - } -}. diff --git a/lib/megaco/src/binary/megaco_ber_bin_media_gateway_control_prev3c.set.asn b/lib/megaco/src/binary/megaco_ber_bin_media_gateway_control_prev3c.set.asn deleted file mode 100644 index e78055fbad..0000000000 --- a/lib/megaco/src/binary/megaco_ber_bin_media_gateway_control_prev3c.set.asn +++ /dev/null @@ -1 +0,0 @@ -MEDIA-GATEWAY-CONTROL-prev3c.asn diff --git a/lib/megaco/src/binary/megaco_ber_bin_media_gateway_control_v1.asn1config b/lib/megaco/src/binary/megaco_ber_bin_media_gateway_control_v1.asn1config deleted file mode 100644 index e815e90948..0000000000 --- a/lib/megaco/src/binary/megaco_ber_bin_media_gateway_control_v1.asn1config +++ /dev/null @@ -1,44 +0,0 @@ -{exclusive_decode, - {'megaco_ber_bin_media_gateway_control_v1', - [ - {decode_message_trans_partial, - [ - 'MegacoMessage',[{mess,[{messageBody,[{transactions,parts}]}]}] - ] - }, - {decode_message_acts_partial, - ['Transaction', - [ - {transactionRequest, - [ - {actions,parts} - ] - }, - {transactionReply, - [ - {transactionResult, [{actionReplies,parts}]} - ] - } - ] - ] - }, - {decode_message_version, - ['MegacoMessage', - [ - {authHeader,undecoded}, - {mess,[{mId,undecoded},{messageBody,undecoded}]} - ] - ] - }, - {decode_message_mId, - ['MegacoMessage', - [ - {authHeader,undecoded}, - {mess,[{messageBody,undecoded}]} - ] - ] - } - - ] - } -}. diff --git a/lib/megaco/src/binary/megaco_ber_bin_media_gateway_control_v1.set.asn b/lib/megaco/src/binary/megaco_ber_bin_media_gateway_control_v1.set.asn deleted file mode 100644 index 0f5a92dba1..0000000000 --- a/lib/megaco/src/binary/megaco_ber_bin_media_gateway_control_v1.set.asn +++ /dev/null @@ -1 +0,0 @@ -MEDIA-GATEWAY-CONTROL-v1.asn diff --git a/lib/megaco/src/binary/megaco_ber_bin_media_gateway_control_v2.set.asn b/lib/megaco/src/binary/megaco_ber_bin_media_gateway_control_v2.set.asn deleted file mode 100644 index 7fc82b127f..0000000000 --- a/lib/megaco/src/binary/megaco_ber_bin_media_gateway_control_v2.set.asn +++ /dev/null @@ -1 +0,0 @@ -MEDIA-GATEWAY-CONTROL-v2.asn diff --git a/lib/megaco/src/binary/megaco_ber_bin_media_gateway_control_v3.set.asn b/lib/megaco/src/binary/megaco_ber_bin_media_gateway_control_v3.set.asn deleted file mode 100644 index 1d7950a283..0000000000 --- a/lib/megaco/src/binary/megaco_ber_bin_media_gateway_control_v3.set.asn +++ /dev/null @@ -1 +0,0 @@ -MEDIA-GATEWAY-CONTROL-v3.asn diff --git a/lib/megaco/src/binary/megaco_ber_bin_media_gateway_control_v2.asn1config b/lib/megaco/src/binary/megaco_ber_media_gateway_control_prev3a.asn1config index cc072b30ee..da67561f1b 100644 --- a/lib/megaco/src/binary/megaco_ber_bin_media_gateway_control_v2.asn1config +++ b/lib/megaco/src/binary/megaco_ber_media_gateway_control_prev3a.asn1config @@ -1,5 +1,5 @@ {exclusive_decode, - {'megaco_ber_bin_media_gateway_control_v2', + {'megaco_ber_media_gateway_control_prev3a', [ {decode_message_trans_partial, [ @@ -14,7 +14,7 @@ {actions,parts} ] }, - {transactionReply, + {transactionReply, [ {transactionResult, [{actionReplies,parts}]} ] diff --git a/lib/megaco/src/binary/megaco_ber_bin_media_gateway_control_v3.asn1config b/lib/megaco/src/binary/megaco_ber_media_gateway_control_prev3b.asn1config index deeb2b2da9..2f25f03d97 100644 --- a/lib/megaco/src/binary/megaco_ber_bin_media_gateway_control_v3.asn1config +++ b/lib/megaco/src/binary/megaco_ber_media_gateway_control_prev3b.asn1config @@ -1,5 +1,5 @@ {exclusive_decode, - {'megaco_ber_bin_media_gateway_control_v3', + {'megaco_ber_media_gateway_control_prev3b', [ {decode_message_trans_partial, [ @@ -14,7 +14,7 @@ {actions,parts} ] }, - {transactionReply, + {transactionReply, [ {transactionResult, [{actionReplies,parts}]} ] diff --git a/lib/megaco/src/binary/megaco_ber_bin_media_gateway_control_prev3a.asn1config b/lib/megaco/src/binary/megaco_ber_media_gateway_control_prev3c.asn1config index 456ce750ad..23c343eed0 100644 --- a/lib/megaco/src/binary/megaco_ber_bin_media_gateway_control_prev3a.asn1config +++ b/lib/megaco/src/binary/megaco_ber_media_gateway_control_prev3c.asn1config @@ -1,5 +1,5 @@ {exclusive_decode, - {'megaco_ber_bin_media_gateway_control_prev3a', + {'megaco_ber_media_gateway_control_prev3c', [ {decode_message_trans_partial, [ @@ -14,7 +14,7 @@ {actions,parts} ] }, - {transactionReply, + {transactionReply, [ {transactionResult, [{actionReplies,parts}]} ] diff --git a/lib/megaco/src/binary/megaco_ber_bin_drv_media_gateway_control_v1.asn1config b/lib/megaco/src/binary/megaco_ber_media_gateway_control_v1.asn1config index ea10a7d527..951825c0aa 100644 --- a/lib/megaco/src/binary/megaco_ber_bin_drv_media_gateway_control_v1.asn1config +++ b/lib/megaco/src/binary/megaco_ber_media_gateway_control_v1.asn1config @@ -1,5 +1,5 @@ {exclusive_decode, - {'megaco_ber_bin_drv_media_gateway_control_v1', + {'megaco_ber_media_gateway_control_v1', [ {decode_message_trans_partial, [ @@ -14,7 +14,7 @@ {actions,parts} ] }, - {transactionReply, + {transactionReply, [ {transactionResult, [{actionReplies,parts}]} ] @@ -38,7 +38,7 @@ ] ] } + ] } }. - diff --git a/lib/megaco/src/binary/megaco_ber_bin_media_gateway_control_prev3b.asn1config b/lib/megaco/src/binary/megaco_ber_media_gateway_control_v3.asn1config index fa5cd80baf..e4b1f9ece9 100644 --- a/lib/megaco/src/binary/megaco_ber_bin_media_gateway_control_prev3b.asn1config +++ b/lib/megaco/src/binary/megaco_ber_media_gateway_control_v3.asn1config @@ -1,5 +1,5 @@ {exclusive_decode, - {'megaco_ber_bin_media_gateway_control_prev3b', + {'megaco_ber_media_gateway_control_v3', [ {decode_message_trans_partial, [ @@ -14,7 +14,7 @@ {actions,parts} ] }, - {transactionReply, + {transactionReply, [ {transactionResult, [{actionReplies,parts}]} ] diff --git a/lib/megaco/src/binary/megaco_binary_encoder.erl b/lib/megaco/src/binary/megaco_binary_encoder.erl index f825f91a45..51e167590d 100644 --- a/lib/megaco/src/binary/megaco_binary_encoder.erl +++ b/lib/megaco/src/binary/megaco_binary_encoder.erl @@ -241,55 +241,30 @@ encode_action_reply(_EC, _V, _AcionReply) -> %% Return {ok, Version} | {error, Reason} %%---------------------------------------------------------------------- -version_of([{version3,v3},driver|EC], Binary) -> - Decoders = [megaco_ber_bin_drv_media_gateway_control_v1, - megaco_ber_bin_drv_media_gateway_control_v2, - megaco_ber_bin_drv_media_gateway_control_v3], - ?BIN_LIB:version_of(EC, Binary, dynamic, Decoders); -version_of([{version3,prev3c},driver|EC], Binary) -> - Decoders = [megaco_ber_bin_drv_media_gateway_control_v1, - megaco_ber_bin_drv_media_gateway_control_v2, - megaco_ber_bin_drv_media_gateway_control_prev3c], - ?BIN_LIB:version_of(EC, Binary, dynamic, Decoders); -version_of([{version3,prev3b},driver|EC], Binary) -> - Decoders = [megaco_ber_bin_drv_media_gateway_control_v1, - megaco_ber_bin_drv_media_gateway_control_v2, - megaco_ber_bin_drv_media_gateway_control_prev3b], - ?BIN_LIB:version_of(EC, Binary, dynamic, Decoders); -version_of([{version3,prev3a},driver|EC], Binary) -> - Decoders = [megaco_ber_bin_drv_media_gateway_control_v1, - megaco_ber_bin_drv_media_gateway_control_v2, - megaco_ber_bin_drv_media_gateway_control_prev3a], - ?BIN_LIB:version_of(EC, Binary, dynamic, Decoders); -version_of([driver|EC], Binary) -> - Decoders = [megaco_ber_bin_drv_media_gateway_control_v1, - megaco_ber_bin_drv_media_gateway_control_v2, - megaco_ber_bin_drv_media_gateway_control_v3], - ?BIN_LIB:version_of(EC, Binary, dynamic, Decoders); version_of([{version3,v3}|EC], Binary) -> - Decoders = [megaco_ber_bin_media_gateway_control_v1, - megaco_ber_bin_media_gateway_control_v2, - megaco_ber_bin_media_gateway_control_v3], + Decoders = [megaco_ber_media_gateway_control_v1, + megaco_ber_media_gateway_control_v2, + megaco_ber_media_gateway_control_v3], ?BIN_LIB:version_of(EC, Binary, dynamic, Decoders); version_of([{version3,prev3c}|EC], Binary) -> - Decoders = [megaco_ber_bin_media_gateway_control_v1, - megaco_ber_bin_media_gateway_control_v2, - megaco_ber_bin_media_gateway_control_prev3c], + Decoders = [megaco_ber_media_gateway_control_v1, + megaco_ber_media_gateway_control_v2, + megaco_ber_media_gateway_control_prev3c], ?BIN_LIB:version_of(EC, Binary, dynamic, Decoders); version_of([{version3,prev3b}|EC], Binary) -> - Decoders = [megaco_ber_bin_media_gateway_control_v1, - megaco_ber_bin_media_gateway_control_v2, - megaco_ber_bin_media_gateway_control_prev3b], + Decoders = [megaco_ber_media_gateway_control_v1, + megaco_ber_media_gateway_control_v2, + megaco_ber_media_gateway_control_prev3b], ?BIN_LIB:version_of(EC, Binary, dynamic, Decoders); version_of([{version3,prev3a}|EC], Binary) -> - Decoders = [megaco_ber_bin_media_gateway_control_v1, - megaco_ber_bin_media_gateway_control_v2, - megaco_ber_bin_media_gateway_control_prev3a], + Decoders = [megaco_ber_media_gateway_control_v1, + megaco_ber_media_gateway_control_v2, + megaco_ber_media_gateway_control_prev3a], ?BIN_LIB:version_of(EC, Binary, dynamic, Decoders); version_of(EC, Binary) -> - Decoders = [megaco_ber_bin_media_gateway_control_v1, - megaco_ber_bin_media_gateway_control_v2, - megaco_ber_bin_media_gateway_control_v3], + Decoders = [megaco_ber_media_gateway_control_v1, + megaco_ber_media_gateway_control_v2, + megaco_ber_media_gateway_control_v3], ?BIN_LIB:version_of(EC, Binary, dynamic, Decoders). @@ -301,287 +276,153 @@ version_of(EC, Binary) -> decode_message(EC, Binary) -> decode_message(EC, 1, Binary). -decode_message([{version3,v3},driver|EC], dynamic, Binary) -> - Decoders = [{megaco_ber_bin_drv_media_gateway_control_v1, - megaco_binary_transformer_v1}, - {megaco_ber_bin_drv_media_gateway_control_v2, - megaco_binary_transformer_v2}, - {megaco_ber_bin_drv_media_gateway_control_v3, - megaco_binary_transformer_v3}], - ?BIN_LIB:decode_message_dynamic(EC, Binary, Decoders, binary); -decode_message([{version3,prev3c},driver|EC], dynamic, Binary) -> - Decoders = [{megaco_ber_bin_drv_media_gateway_control_v1, - megaco_binary_transformer_v1}, - {megaco_ber_bin_drv_media_gateway_control_v2, - megaco_binary_transformer_v2}, - {megaco_ber_bin_drv_media_gateway_control_prev3c, - megaco_binary_transformer_prev3c}], - ?BIN_LIB:decode_message_dynamic(EC, Binary, Decoders, binary); -decode_message([{version3,prev3b},driver|EC], dynamic, Binary) -> - Decoders = [{megaco_ber_bin_drv_media_gateway_control_v1, - megaco_binary_transformer_v1}, - {megaco_ber_bin_drv_media_gateway_control_v2, - megaco_binary_transformer_v2}, - {megaco_ber_bin_drv_media_gateway_control_prev3b, - megaco_binary_transformer_prev3b}], - ?BIN_LIB:decode_message_dynamic(EC, Binary, Decoders, binary); -decode_message([{version3,prev3a},driver|EC], dynamic, Binary) -> - Decoders = [{megaco_ber_bin_drv_media_gateway_control_v1, - megaco_binary_transformer_v1}, - {megaco_ber_bin_drv_media_gateway_control_v2, - megaco_binary_transformer_v2}, - {megaco_ber_bin_drv_media_gateway_control_prev3a, - megaco_binary_transformer_prev3a}], - ?BIN_LIB:decode_message_dynamic(EC, Binary, Decoders, binary); -decode_message([driver|EC], dynamic, Binary) -> - Decoders = [{megaco_ber_bin_drv_media_gateway_control_v1, - megaco_binary_transformer_v1}, - {megaco_ber_bin_drv_media_gateway_control_v2, - megaco_binary_transformer_v2}, - {megaco_ber_bin_drv_media_gateway_control_v3, - megaco_binary_transformer_v3}], - ?BIN_LIB:decode_message_dynamic(EC, Binary, Decoders, binary); decode_message([{version3,v3}|EC], dynamic, Binary) -> - Decoders = [{megaco_ber_bin_media_gateway_control_v1, + Decoders = [{megaco_ber_media_gateway_control_v1, megaco_binary_transformer_v1}, - {megaco_ber_bin_media_gateway_control_v2, + {megaco_ber_media_gateway_control_v2, megaco_binary_transformer_v2}, - {megaco_ber_bin_media_gateway_control_v3, + {megaco_ber_media_gateway_control_v3, megaco_binary_transformer_v3}], ?BIN_LIB:decode_message_dynamic(EC, Binary, Decoders, binary); decode_message([{version3,prev3c}|EC], dynamic, Binary) -> - Decoders = [{megaco_ber_bin_media_gateway_control_v1, + Decoders = [{megaco_ber_media_gateway_control_v1, megaco_binary_transformer_v1}, - {megaco_ber_bin_media_gateway_control_v2, + {megaco_ber_media_gateway_control_v2, megaco_binary_transformer_v2}, - {megaco_ber_bin_media_gateway_control_prev3c, + {megaco_ber_media_gateway_control_prev3c, megaco_binary_transformer_prev3c}], ?BIN_LIB:decode_message_dynamic(EC, Binary, Decoders, binary); decode_message([{version3,prev3b}|EC], dynamic, Binary) -> - Decoders = [{megaco_ber_bin_media_gateway_control_v1, + Decoders = [{megaco_ber_media_gateway_control_v1, megaco_binary_transformer_v1}, - {megaco_ber_bin_media_gateway_control_v2, + {megaco_ber_media_gateway_control_v2, megaco_binary_transformer_v2}, - {megaco_ber_bin_media_gateway_control_prev3b, + {megaco_ber_media_gateway_control_prev3b, megaco_binary_transformer_prev3b}], ?BIN_LIB:decode_message_dynamic(EC, Binary, Decoders, binary); decode_message([{version3,prev3a}|EC], dynamic, Binary) -> - Decoders = [{megaco_ber_bin_media_gateway_control_v1, + Decoders = [{megaco_ber_media_gateway_control_v1, megaco_binary_transformer_v1}, - {megaco_ber_bin_media_gateway_control_v2, + {megaco_ber_media_gateway_control_v2, megaco_binary_transformer_v2}, - {megaco_ber_bin_media_gateway_control_prev3a, + {megaco_ber_media_gateway_control_prev3a, megaco_binary_transformer_prev3a}], ?BIN_LIB:decode_message_dynamic(EC, Binary, Decoders, binary); decode_message(EC, dynamic, Binary) -> - Decoders = [{megaco_ber_bin_media_gateway_control_v1, + Decoders = [{megaco_ber_media_gateway_control_v1, megaco_binary_transformer_v1}, - {megaco_ber_bin_media_gateway_control_v2, + {megaco_ber_media_gateway_control_v2, megaco_binary_transformer_v2}, - {megaco_ber_bin_media_gateway_control_v3, + {megaco_ber_media_gateway_control_v3, megaco_binary_transformer_v3}], ?BIN_LIB:decode_message_dynamic(EC, Binary, Decoders, binary); %% -- Version 1 -- -decode_message([{version3,_},driver|EC], 1, Binary) -> - AsnMod = megaco_ber_bin_drv_media_gateway_control_v1, - TransMod = megaco_binary_transformer_v1, - ?BIN_LIB:decode_message(EC, Binary, AsnMod, TransMod, binary); - -decode_message([driver|EC], 1, Binary) -> - AsnMod = megaco_ber_bin_drv_media_gateway_control_v1, - TransMod = megaco_binary_transformer_v1, - ?BIN_LIB:decode_message(EC, Binary, AsnMod, TransMod, binary); - decode_message([{version3,_}|EC], 1, Binary) -> - AsnMod = megaco_ber_bin_media_gateway_control_v1, + AsnMod = megaco_ber_media_gateway_control_v1, TransMod = megaco_binary_transformer_v1, ?BIN_LIB:decode_message(EC, Binary, AsnMod, TransMod, binary); decode_message(EC, 1, Binary) -> - AsnMod = megaco_ber_bin_media_gateway_control_v1, + AsnMod = megaco_ber_media_gateway_control_v1, TransMod = megaco_binary_transformer_v1, ?BIN_LIB:decode_message(EC, Binary, AsnMod, TransMod, binary); %% -- Version 2 -- -decode_message([{version3,_},driver|EC], 2, Binary) -> - AsnMod = megaco_ber_bin_drv_media_gateway_control_v2, - TransMod = megaco_binary_transformer_v2, - ?BIN_LIB:decode_message(EC, Binary, AsnMod, TransMod, binary); - -decode_message([driver|EC], 2, Binary) -> - AsnMod = megaco_ber_bin_drv_media_gateway_control_v2, - TransMod = megaco_binary_transformer_v2, - ?BIN_LIB:decode_message(EC, Binary, AsnMod, TransMod, binary); - decode_message([{version3,_}|EC], 2, Binary) -> - AsnMod = megaco_ber_bin_media_gateway_control_v2, + AsnMod = megaco_ber_media_gateway_control_v2, TransMod = megaco_binary_transformer_v2, ?BIN_LIB:decode_message(EC, Binary, AsnMod, TransMod, binary); decode_message(EC, 2, Binary) -> - AsnMod = megaco_ber_bin_media_gateway_control_v2, + AsnMod = megaco_ber_media_gateway_control_v2, TransMod = megaco_binary_transformer_v2, ?BIN_LIB:decode_message(EC, Binary, AsnMod, TransMod, binary); %% -- Version 3 -- -decode_message([{version3,v3},driver|EC], 3, Binary) -> - AsnMod = megaco_ber_bin_drv_media_gateway_control_v3, - TransMod = megaco_binary_transformer_v3, - ?BIN_LIB:decode_message(EC, Binary, AsnMod, TransMod, binary); -decode_message([{version3,prev3c},driver|EC], 3, Binary) -> - AsnMod = megaco_ber_bin_drv_media_gateway_control_prev3c, - TransMod = megaco_binary_transformer_prev3c, - ?BIN_LIB:decode_message(EC, Binary, AsnMod, TransMod, binary); -decode_message([{version3,prev3b},driver|EC], 3, Binary) -> - AsnMod = megaco_ber_bin_drv_media_gateway_control_prev3b, - TransMod = megaco_binary_transformer_prev3b, - ?BIN_LIB:decode_message(EC, Binary, AsnMod, TransMod, binary); -decode_message([{version3,prev3a},driver|EC], 3, Binary) -> - AsnMod = megaco_ber_bin_drv_media_gateway_control_prev3a, - TransMod = megaco_binary_transformer_prev3a, - ?BIN_LIB:decode_message(EC, Binary, AsnMod, TransMod, binary); - -decode_message([driver|EC], 3, Binary) -> - AsnMod = megaco_ber_bin_drv_media_gateway_control_v3, - TransMod = megaco_binary_transformer_v3, - ?BIN_LIB:decode_message(EC, Binary, AsnMod, TransMod, binary); - decode_message([{version3,v3}|EC], 3, Binary) -> - AsnMod = megaco_ber_bin_media_gateway_control_v3, + AsnMod = megaco_ber_media_gateway_control_v3, TransMod = megaco_binary_transformer_v3, ?BIN_LIB:decode_message(EC, Binary, AsnMod, TransMod, binary); decode_message([{version3,prev3c}|EC], 3, Binary) -> - AsnMod = megaco_ber_bin_media_gateway_control_prev3c, + AsnMod = megaco_ber_media_gateway_control_prev3c, TransMod = megaco_binary_transformer_prev3c, ?BIN_LIB:decode_message(EC, Binary, AsnMod, TransMod, binary); decode_message([{version3,prev3b}|EC], 3, Binary) -> - AsnMod = megaco_ber_bin_media_gateway_control_prev3b, + AsnMod = megaco_ber_media_gateway_control_prev3b, TransMod = megaco_binary_transformer_prev3b, ?BIN_LIB:decode_message(EC, Binary, AsnMod, TransMod, binary); decode_message([{version3,prev3a}|EC], 3, Binary) -> - AsnMod = megaco_ber_bin_media_gateway_control_prev3a, + AsnMod = megaco_ber_media_gateway_control_prev3a, TransMod = megaco_binary_transformer_prev3a, ?BIN_LIB:decode_message(EC, Binary, AsnMod, TransMod, binary); decode_message(EC, 3, Binary) -> - AsnMod = megaco_ber_bin_media_gateway_control_v3, + AsnMod = megaco_ber_media_gateway_control_v3, TransMod = megaco_binary_transformer_v3, ?BIN_LIB:decode_message(EC, Binary, AsnMod, TransMod, binary). -decode_mini_message([{version3,v3},driver|EC], dynamic, Bin) -> - Mods = [megaco_ber_bin_drv_media_gateway_control_v1, - megaco_ber_bin_drv_media_gateway_control_v2, - megaco_ber_bin_drv_media_gateway_control_v3], - ?BIN_LIB:decode_mini_message_dynamic(EC, Bin, Mods, binary); -decode_mini_message([{version3,prev3c},driver|EC], dynamic, Bin) -> - Mods = [megaco_ber_bin_drv_media_gateway_control_v1, - megaco_ber_bin_drv_media_gateway_control_v2, - megaco_ber_bin_drv_media_gateway_control_prev3c], - ?BIN_LIB:decode_mini_message_dynamic(EC, Bin, Mods, binary); -decode_mini_message([{version3,prev3b},driver|EC], dynamic, Bin) -> - Mods = [megaco_ber_bin_drv_media_gateway_control_v1, - megaco_ber_bin_drv_media_gateway_control_v2, - megaco_ber_bin_drv_media_gateway_control_prev3b], - ?BIN_LIB:decode_mini_message_dynamic(EC, Bin, Mods, binary); -decode_mini_message([{version3,prev3a},driver|EC], dynamic, Bin) -> - Mods = [megaco_ber_bin_drv_media_gateway_control_v1, - megaco_ber_bin_drv_media_gateway_control_v2, - megaco_ber_bin_drv_media_gateway_control_prev3a], - ?BIN_LIB:decode_mini_message_dynamic(EC, Bin, Mods, binary); -decode_mini_message([driver|EC], dynamic, Bin) -> - Mods = [megaco_ber_bin_drv_media_gateway_control_v1, - megaco_ber_bin_drv_media_gateway_control_v2, - megaco_ber_bin_drv_media_gateway_control_v3], - ?BIN_LIB:decode_mini_message_dynamic(EC, Bin, Mods, binary); decode_mini_message([{version3,v3}|EC], dynamic, Bin) -> - Mods = [megaco_ber_bin_media_gateway_control_v1, - megaco_ber_bin_media_gateway_control_v2, - megaco_ber_bin_media_gateway_control_v3], + Mods = [megaco_ber_media_gateway_control_v1, + megaco_ber_media_gateway_control_v2, + megaco_ber_media_gateway_control_v3], ?BIN_LIB:decode_mini_message_dynamic(EC, Bin, Mods, binary); decode_mini_message([{version3,prev3c}|EC], dynamic, Bin) -> - Mods = [megaco_ber_bin_media_gateway_control_v1, - megaco_ber_bin_media_gateway_control_v2, - megaco_ber_bin_media_gateway_control_prev3c], + Mods = [megaco_ber_media_gateway_control_v1, + megaco_ber_media_gateway_control_v2, + megaco_ber_media_gateway_control_prev3c], ?BIN_LIB:decode_mini_message_dynamic(EC, Bin, Mods, binary); decode_mini_message([{version3,prev3b}|EC], dynamic, Bin) -> - Mods = [megaco_ber_bin_media_gateway_control_v1, - megaco_ber_bin_media_gateway_control_v2, - megaco_ber_bin_media_gateway_control_prev3b], + Mods = [megaco_ber_media_gateway_control_v1, + megaco_ber_media_gateway_control_v2, + megaco_ber_media_gateway_control_prev3b], ?BIN_LIB:decode_mini_message_dynamic(EC, Bin, Mods, binary); decode_mini_message([{version3,prev3a}|EC], dynamic, Bin) -> - Mods = [megaco_ber_bin_media_gateway_control_v1, - megaco_ber_bin_media_gateway_control_v2, - megaco_ber_bin_media_gateway_control_prev3a], + Mods = [megaco_ber_media_gateway_control_v1, + megaco_ber_media_gateway_control_v2, + megaco_ber_media_gateway_control_prev3a], ?BIN_LIB:decode_mini_message_dynamic(EC, Bin, Mods, binary); decode_mini_message(EC, dynamic, Bin) -> - Mods = [megaco_ber_bin_media_gateway_control_v1, - megaco_ber_bin_media_gateway_control_v2, - megaco_ber_bin_media_gateway_control_v3], + Mods = [megaco_ber_media_gateway_control_v1, + megaco_ber_media_gateway_control_v2, + megaco_ber_media_gateway_control_v3], ?BIN_LIB:decode_mini_message_dynamic(EC, Bin, Mods, binary); -decode_mini_message([{version3,_},driver|EC], 1, Bin) -> - AsnMod = megaco_ber_bin_drv_media_gateway_control_v1, - ?BIN_LIB:decode_mini_message(EC, Bin, AsnMod, binary); -decode_mini_message([driver|EC], 1, Bin) -> - AsnMod = megaco_ber_bin_drv_media_gateway_control_v1, - ?BIN_LIB:decode_mini_message(EC, Bin, AsnMod, binary); decode_mini_message([{version3,_}|EC], 1, Bin) -> - AsnMod = megaco_ber_bin_media_gateway_control_v1, + AsnMod = megaco_ber_media_gateway_control_v1, ?BIN_LIB:decode_mini_message(EC, Bin, AsnMod, binary); decode_mini_message(EC, 1, Bin) -> - AsnMod = megaco_ber_bin_media_gateway_control_v1, + AsnMod = megaco_ber_media_gateway_control_v1, ?BIN_LIB:decode_mini_message(EC, Bin, AsnMod, binary); -decode_mini_message([{version3,_},driver|EC], 2, Bin) -> - AsnMod = megaco_ber_bin_drv_media_gateway_control_v2, - ?BIN_LIB:decode_mini_message(EC, Bin, AsnMod, binary); -decode_mini_message([driver|EC], 2, Bin) -> - AsnMod = megaco_ber_bin_drv_media_gateway_control_v2, - ?BIN_LIB:decode_mini_message(EC, Bin, AsnMod, binary); decode_mini_message([{version3,_}|EC], 2, Bin) -> - AsnMod = megaco_ber_bin_media_gateway_control_v2, + AsnMod = megaco_ber_media_gateway_control_v2, ?BIN_LIB:decode_mini_message(EC, Bin, AsnMod, binary); decode_mini_message(EC, 2, Bin) -> - AsnMod = megaco_ber_bin_media_gateway_control_v2, + AsnMod = megaco_ber_media_gateway_control_v2, ?BIN_LIB:decode_mini_message(EC, Bin, AsnMod, binary); -decode_mini_message([{version3,v3},driver|EC], 3, Bin) -> - AsnMod = megaco_ber_bin_drv_media_gateway_control_v3, - ?BIN_LIB:decode_mini_message(EC, Bin, AsnMod, binary); -decode_mini_message([{version3,prev3c},driver|EC], 3, Bin) -> - AsnMod = megaco_ber_bin_drv_media_gateway_control_prev3c, - ?BIN_LIB:decode_mini_message(EC, Bin, AsnMod, binary); -decode_mini_message([{version3,prev3b},driver|EC], 3, Bin) -> - AsnMod = megaco_ber_bin_drv_media_gateway_control_prev3b, - ?BIN_LIB:decode_mini_message(EC, Bin, AsnMod, binary); -decode_mini_message([{version3,prev3a},driver|EC], 3, Bin) -> - AsnMod = megaco_ber_bin_drv_media_gateway_control_prev3a, - ?BIN_LIB:decode_mini_message(EC, Bin, AsnMod, binary); -decode_mini_message([driver|EC], 3, Bin) -> - AsnMod = megaco_ber_bin_drv_media_gateway_control_v3, - ?BIN_LIB:decode_mini_message(EC, Bin, AsnMod, binary); decode_mini_message([{version3,v3}|EC], 3, Bin) -> - AsnMod = megaco_ber_bin_media_gateway_control_v3, + AsnMod = megaco_ber_media_gateway_control_v3, ?BIN_LIB:decode_mini_message(EC, Bin, AsnMod, binary); decode_mini_message([{version3,prev3c}|EC], 3, Bin) -> - AsnMod = megaco_ber_bin_media_gateway_control_prev3c, + AsnMod = megaco_ber_media_gateway_control_prev3c, ?BIN_LIB:decode_mini_message(EC, Bin, AsnMod, binary); decode_mini_message([{version3,prev3b}|EC], 3, Bin) -> - AsnMod = megaco_ber_bin_media_gateway_control_prev3b, + AsnMod = megaco_ber_media_gateway_control_prev3b, ?BIN_LIB:decode_mini_message(EC, Bin, AsnMod, binary); decode_mini_message([{version3,prev3a}|EC], 3, Bin) -> - AsnMod = megaco_ber_bin_media_gateway_control_prev3a, + AsnMod = megaco_ber_media_gateway_control_prev3a, ?BIN_LIB:decode_mini_message(EC, Bin, AsnMod, binary); decode_mini_message(EC, 3, Bin) -> - AsnMod = megaco_ber_bin_media_gateway_control_v3, + AsnMod = megaco_ber_media_gateway_control_v3, ?BIN_LIB:decode_mini_message(EC, Bin, AsnMod, binary). diff --git a/lib/megaco/src/binary/megaco_binary_encoder_lib.erl b/lib/megaco/src/binary/megaco_binary_encoder_lib.erl index 967ee93935..262889db39 100644 --- a/lib/megaco/src/binary/megaco_binary_encoder_lib.erl +++ b/lib/megaco/src/binary/megaco_binary_encoder_lib.erl @@ -275,7 +275,7 @@ decode_message_dynamic(_EC, _BadBin, _Mods, _Type) -> {error, no_binary}. -decode_message(EC, Bin, AsnMod, TransMod, binary) -> +decode_message(EC, Bin, AsnMod, TransMod, _) -> case asn1rt:decode(AsnMod, 'MegacoMessage', Bin) of {ok, MegaMsg} -> case EC of @@ -286,19 +286,6 @@ decode_message(EC, Bin, AsnMod, TransMod, binary) -> end; {error, Reason} -> {error, Reason} - end; -decode_message(EC, Bin, AsnMod, TransMod, io_list) -> - ShallowIoList = erlang:binary_to_list(Bin), - case asn1rt:decode(AsnMod, 'MegacoMessage', ShallowIoList) of - {ok, MegaMsg} -> - case EC of - [native] -> - {ok, MegaMsg}; - _ -> - {ok, TransMod:tr_message(MegaMsg, decode, EC)} - end; - {error, Reason} -> - {error, Reason} end. diff --git a/lib/megaco/src/binary/megaco_binary_name_resolver_prev3a.erl b/lib/megaco/src/binary/megaco_binary_name_resolver_prev3a.erl index 72b3112053..24dc410dbe 100644 --- a/lib/megaco/src/binary/megaco_binary_name_resolver_prev3a.erl +++ b/lib/megaco/src/binary/megaco_binary_name_resolver_prev3a.erl @@ -1,7 +1,7 @@ %% %% %CopyrightBegin% %% -%% Copyright Ericsson AB 2004-2009. All Rights Reserved. +%% Copyright Ericsson AB 2004-2012. All Rights Reserved. %% %% The contents of this file are subject to the Erlang Public License, %% Version 1.1, (the "License"); you may not use this file except in @@ -204,7 +204,7 @@ decode_name(_Config, Scope, Item) -> %% to command requests from the MGC that audit %% ObservedEventsDescriptor, and found in the %% ObservedEventsDescriptor. See 12.2. If there are no parameters -%% for the ObservedEvents Descriptor, then �none� shall be specified. +%% for the ObservedEvents Descriptor, then 'none' shall be specified. %% %% %% 12.1.4 Signals @@ -225,7 +225,7 @@ decode_name(_Config, Scope, Item) -> %% %% BR (Brief) %% -%% NOTE -�SignalType may be defined such that it is dependent on +%% NOTE - SignalType may be defined such that it is dependent on %% the value of one or more parameters. The package MUST specify a %% default signal type. If the default type is TO, the package MUST %% specify a default duration which may be provisioned. A default diff --git a/lib/megaco/src/binary/megaco_binary_name_resolver_prev3b.erl b/lib/megaco/src/binary/megaco_binary_name_resolver_prev3b.erl index 12e673ac81..d89717c00a 100644 --- a/lib/megaco/src/binary/megaco_binary_name_resolver_prev3b.erl +++ b/lib/megaco/src/binary/megaco_binary_name_resolver_prev3b.erl @@ -1,7 +1,7 @@ %% %% %CopyrightBegin% %% -%% Copyright Ericsson AB 2005-2009. All Rights Reserved. +%% Copyright Ericsson AB 2005-2012. All Rights Reserved. %% %% The contents of this file are subject to the Erlang Public License, %% Version 1.1, (the "License"); you may not use this file except in @@ -204,7 +204,7 @@ decode_name(_Config, Scope, Item) -> %% to command requests from the MGC that audit %% ObservedEventsDescriptor, and found in the %% ObservedEventsDescriptor. See 12.2. If there are no parameters -%% for the ObservedEvents Descriptor, then �none� shall be specified. +%% for the ObservedEvents Descriptor, then 'none' shall be specified. %% %% %% 12.1.4 Signals @@ -225,7 +225,7 @@ decode_name(_Config, Scope, Item) -> %% %% BR (Brief) %% -%% NOTE -�SignalType may be defined such that it is dependent on +%% NOTE - SignalType may be defined such that it is dependent on %% the value of one or more parameters. The package MUST specify a %% default signal type. If the default type is TO, the package MUST %% specify a default duration which may be provisioned. A default diff --git a/lib/megaco/src/binary/megaco_binary_name_resolver_prev3c.erl b/lib/megaco/src/binary/megaco_binary_name_resolver_prev3c.erl index d08231caac..8b4192ad44 100644 --- a/lib/megaco/src/binary/megaco_binary_name_resolver_prev3c.erl +++ b/lib/megaco/src/binary/megaco_binary_name_resolver_prev3c.erl @@ -1,7 +1,7 @@ %% %% %CopyrightBegin% %% -%% Copyright Ericsson AB 2005-2009. All Rights Reserved. +%% Copyright Ericsson AB 2005-2012. All Rights Reserved. %% %% The contents of this file are subject to the Erlang Public License, %% Version 1.1, (the "License"); you may not use this file except in @@ -204,7 +204,7 @@ decode_name(_Config, Scope, Item) -> %% to command requests from the MGC that audit %% ObservedEventsDescriptor, and found in the %% ObservedEventsDescriptor. See 12.2. If there are no parameters -%% for the ObservedEvents Descriptor, then �none� shall be specified. +%% for the ObservedEvents Descriptor, then 'none' shall be specified. %% %% %% 12.1.4 Signals @@ -225,7 +225,7 @@ decode_name(_Config, Scope, Item) -> %% %% BR (Brief) %% -%% NOTE -�SignalType may be defined such that it is dependent on +%% NOTE - SignalType may be defined such that it is dependent on %% the value of one or more parameters. The package MUST specify a %% default signal type. If the default type is TO, the package MUST %% specify a default duration which may be provisioned. A default diff --git a/lib/megaco/src/binary/megaco_binary_name_resolver_v3.erl b/lib/megaco/src/binary/megaco_binary_name_resolver_v3.erl index c101aa15bc..20954d4c9d 100644 --- a/lib/megaco/src/binary/megaco_binary_name_resolver_v3.erl +++ b/lib/megaco/src/binary/megaco_binary_name_resolver_v3.erl @@ -1,7 +1,7 @@ %% %% %CopyrightBegin% %% -%% Copyright Ericsson AB 2005-2009. All Rights Reserved. +%% Copyright Ericsson AB 2005-2012. All Rights Reserved. %% %% The contents of this file are subject to the Erlang Public License, %% Version 1.1, (the "License"); you may not use this file except in @@ -204,7 +204,7 @@ decode_name(_Config, Scope, Item) -> %% to command requests from the MGC that audit %% ObservedEventsDescriptor, and found in the %% ObservedEventsDescriptor. See 12.2. If there are no parameters -%% for the ObservedEvents Descriptor, then �none� shall be specified. +%% for the ObservedEvents Descriptor, then 'none' shall be specified. %% %% %% 12.1.4 Signals @@ -225,7 +225,7 @@ decode_name(_Config, Scope, Item) -> %% %% BR (Brief) %% -%% NOTE -�SignalType may be defined such that it is dependent on +%% NOTE - SignalType may be defined such that it is dependent on %% the value of one or more parameters. The package MUST specify a %% default signal type. If the default type is TO, the package MUST %% specify a default duration which may be provisioned. A default diff --git a/lib/megaco/src/binary/megaco_per_bin_drv_media_gateway_control_prev3a.set.asn b/lib/megaco/src/binary/megaco_per_bin_drv_media_gateway_control_prev3a.set.asn deleted file mode 100644 index b9ba7ffdb4..0000000000 --- a/lib/megaco/src/binary/megaco_per_bin_drv_media_gateway_control_prev3a.set.asn +++ /dev/null @@ -1 +0,0 @@ -MEDIA-GATEWAY-CONTROL-prev3a.asn diff --git a/lib/megaco/src/binary/megaco_per_bin_drv_media_gateway_control_prev3b.set.asn b/lib/megaco/src/binary/megaco_per_bin_drv_media_gateway_control_prev3b.set.asn deleted file mode 100644 index 0437bde310..0000000000 --- a/lib/megaco/src/binary/megaco_per_bin_drv_media_gateway_control_prev3b.set.asn +++ /dev/null @@ -1 +0,0 @@ -MEDIA-GATEWAY-CONTROL-prev3b.asn diff --git a/lib/megaco/src/binary/megaco_per_bin_drv_media_gateway_control_prev3c.set.asn b/lib/megaco/src/binary/megaco_per_bin_drv_media_gateway_control_prev3c.set.asn deleted file mode 100644 index e78055fbad..0000000000 --- a/lib/megaco/src/binary/megaco_per_bin_drv_media_gateway_control_prev3c.set.asn +++ /dev/null @@ -1 +0,0 @@ -MEDIA-GATEWAY-CONTROL-prev3c.asn diff --git a/lib/megaco/src/binary/megaco_per_bin_drv_media_gateway_control_v1.set.asn b/lib/megaco/src/binary/megaco_per_bin_drv_media_gateway_control_v1.set.asn deleted file mode 100644 index 0f5a92dba1..0000000000 --- a/lib/megaco/src/binary/megaco_per_bin_drv_media_gateway_control_v1.set.asn +++ /dev/null @@ -1 +0,0 @@ -MEDIA-GATEWAY-CONTROL-v1.asn diff --git a/lib/megaco/src/binary/megaco_per_bin_drv_media_gateway_control_v2.set.asn b/lib/megaco/src/binary/megaco_per_bin_drv_media_gateway_control_v2.set.asn deleted file mode 100644 index 7fc82b127f..0000000000 --- a/lib/megaco/src/binary/megaco_per_bin_drv_media_gateway_control_v2.set.asn +++ /dev/null @@ -1 +0,0 @@ -MEDIA-GATEWAY-CONTROL-v2.asn diff --git a/lib/megaco/src/binary/megaco_per_bin_drv_media_gateway_control_v3.set.asn b/lib/megaco/src/binary/megaco_per_bin_drv_media_gateway_control_v3.set.asn deleted file mode 100644 index 1d7950a283..0000000000 --- a/lib/megaco/src/binary/megaco_per_bin_drv_media_gateway_control_v3.set.asn +++ /dev/null @@ -1 +0,0 @@ -MEDIA-GATEWAY-CONTROL-v3.asn diff --git a/lib/megaco/src/binary/megaco_per_bin_encoder.erl b/lib/megaco/src/binary/megaco_per_bin_encoder.erl deleted file mode 100644 index f7280f4e04..0000000000 --- a/lib/megaco/src/binary/megaco_per_bin_encoder.erl +++ /dev/null @@ -1,447 +0,0 @@ -%% -%% %CopyrightBegin% -%% -%% Copyright Ericsson AB 2001-2009. All Rights Reserved. -%% -%% The contents of this file are subject to the Erlang Public License, -%% Version 1.1, (the "License"); you may not use this file except in -%% compliance with the License. You should have received a copy of the -%% Erlang Public License along with this software. If not, it can be -%% retrieved online at http://www.erlang.org/. -%% -%% Software distributed under the License is distributed on an "AS IS" -%% basis, WITHOUT WARRANTY OF ANY KIND, either express or implied. See -%% the License for the specific language governing rights and limitations -%% under the License. -%% -%% %CopyrightEnd% -%% - -%% -%%---------------------------------------------------------------------- -%% Purpose : Handle ASN.1 PER encoding of Megaco/H.248 -%%---------------------------------------------------------------------- - --module(megaco_per_bin_encoder). - --behaviour(megaco_encoder). - --export([encode_message/3, decode_message/3, - decode_mini_message/3, - - encode_transaction/3, - encode_action_requests/3, - encode_action_request/3, - encode_action_reply/3, - - version_of/2]). - -%% Backward compatible functions: --export([encode_message/2, decode_message/2]). - --include_lib("megaco/src/engine/megaco_message_internal.hrl"). - --define(V1_ASN1_MOD, megaco_per_bin_media_gateway_control_v1). --define(V2_ASN1_MOD, megaco_per_bin_media_gateway_control_v2). --define(V3_ASN1_MOD, megaco_per_bin_media_gateway_control_v3). --define(PREV3A_ASN1_MOD, megaco_per_bin_media_gateway_control_prev3a). --define(PREV3B_ASN1_MOD, megaco_per_bin_media_gateway_control_prev3b). --define(PREV3C_ASN1_MOD, megaco_per_bin_media_gateway_control_prev3c). --define(V1_ASN1_MOD_DRV, megaco_per_bin_drv_media_gateway_control_v1). --define(V2_ASN1_MOD_DRV, megaco_per_bin_drv_media_gateway_control_v2). --define(V3_ASN1_MOD_DRV, megaco_per_bin_drv_media_gateway_control_v3). --define(PREV3A_ASN1_MOD_DRV, megaco_per_bin_drv_media_gateway_control_prev3a). --define(PREV3B_ASN1_MOD_DRV, megaco_per_bin_drv_media_gateway_control_prev3b). --define(PREV3C_ASN1_MOD_DRV, megaco_per_bin_drv_media_gateway_control_prev3c). - --define(V1_TRANS_MOD, megaco_binary_transformer_v1). --define(V2_TRANS_MOD, megaco_binary_transformer_v2). --define(V3_TRANS_MOD, megaco_binary_transformer_v3). --define(PREV3A_TRANS_MOD, megaco_binary_transformer_prev3a). --define(PREV3B_TRANS_MOD, megaco_binary_transformer_prev3b). --define(PREV3C_TRANS_MOD, megaco_binary_transformer_prev3c). - --define(BIN_LIB, megaco_binary_encoder_lib). - - -%%---------------------------------------------------------------------- -%% Detect (check/get) message version -%% Return {ok, Version} | {error, Reason} -%%---------------------------------------------------------------------- - -version_of([{version3,v3},driver|EC], Binary) -> - Decoders = [?V1_ASN1_MOD_DRV, ?V2_ASN1_MOD_DRV, ?V3_ASN1_MOD_DRV], - ?BIN_LIB:version_of(EC, Binary, 1, Decoders); -version_of([{version3,prev3c},driver|EC], Binary) -> - Decoders = [?V1_ASN1_MOD_DRV, ?V2_ASN1_MOD_DRV, ?PREV3C_ASN1_MOD_DRV], - ?BIN_LIB:version_of(EC, Binary, 1, Decoders); -version_of([{version3,prev3b},driver|EC], Binary) -> - Decoders = [?V1_ASN1_MOD_DRV, ?V2_ASN1_MOD_DRV, ?PREV3B_ASN1_MOD_DRV], - ?BIN_LIB:version_of(EC, Binary, 1, Decoders); -version_of([{version3,prev3a},driver|EC], Binary) -> - Decoders = [?V1_ASN1_MOD_DRV, ?V2_ASN1_MOD_DRV, ?PREV3A_ASN1_MOD_DRV], - ?BIN_LIB:version_of(EC, Binary, 1, Decoders); -version_of([driver|EC], Binary) -> - Decoders = [?V1_ASN1_MOD_DRV, ?V2_ASN1_MOD_DRV, ?V3_ASN1_MOD_DRV], - ?BIN_LIB:version_of(EC, Binary, 1, Decoders); -version_of([{version3,v3}|EC], Binary) -> - Decoders = [?V1_ASN1_MOD, ?V2_ASN1_MOD, ?V3_ASN1_MOD], - ?BIN_LIB:version_of(EC, Binary, 1, Decoders); -version_of([{version3,prev3c}|EC], Binary) -> - Decoders = [?V1_ASN1_MOD, ?V2_ASN1_MOD, ?PREV3C_ASN1_MOD], - ?BIN_LIB:version_of(EC, Binary, 1, Decoders); -version_of([{version3,prev3b}|EC], Binary) -> - Decoders = [?V1_ASN1_MOD, ?V2_ASN1_MOD, ?PREV3B_ASN1_MOD], - ?BIN_LIB:version_of(EC, Binary, 1, Decoders); -version_of([{version3,prev3a}|EC], Binary) -> - Decoders = [?V1_ASN1_MOD, ?V2_ASN1_MOD, ?PREV3A_ASN1_MOD], - ?BIN_LIB:version_of(EC, Binary, 1, Decoders); - -%% All values we need to take (special) care of has been delt with, -%% so just pass the rest on -version_of(EC, Binary) -> - Decoders = [?V1_ASN1_MOD, ?V2_ASN1_MOD, ?V3_ASN1_MOD], - ?BIN_LIB:version_of(EC, Binary, 1, Decoders). - - -%%---------------------------------------------------------------------- -%% Convert a 'MegacoMessage' record into a binary -%% Return {ok, Binary} | {error, Reason} -%%---------------------------------------------------------------------- - -encode_message(EC, - #'MegacoMessage'{mess = #'Message'{version = V}} = MegaMsg) -> - encode_message(EC, V, MegaMsg). - - -%% -- Version 1 -- - -encode_message([{version3, _},driver|EC], 1, MegaMsg) -> - AsnMod = ?V1_ASN1_MOD_DRV, - TransMod = ?V1_TRANS_MOD, - ?BIN_LIB:encode_message(EC, MegaMsg, AsnMod, TransMod, io_list); -encode_message([driver|EC], 1, MegaMsg) -> - AsnMod = ?V1_ASN1_MOD_DRV, - TransMod = ?V1_TRANS_MOD, - ?BIN_LIB:encode_message(EC, MegaMsg, AsnMod, TransMod, io_list); -encode_message([{version3,_}|EC], 1, MegaMsg) -> - AsnMod = ?V1_ASN1_MOD, - TransMod = ?V1_TRANS_MOD, - ?BIN_LIB:encode_message(EC, MegaMsg, AsnMod, TransMod, io_list); - -%% All values we need to take (special) care of has been delt with, -%% so just pass the rest on -encode_message(EC, 1, MegaMsg) -> - AsnMod = ?V1_ASN1_MOD, - TransMod = ?V1_TRANS_MOD, - ?BIN_LIB:encode_message(EC, MegaMsg, AsnMod, TransMod, io_list); - - -%% -- Version 2 -- - -encode_message([{version3,_},driver|EC], 2, MegaMsg) -> - AsnMod = ?V2_ASN1_MOD_DRV, - TransMod = ?V2_TRANS_MOD, - ?BIN_LIB:encode_message(EC, MegaMsg, AsnMod, TransMod, io_list); -encode_message([driver|EC], 2, MegaMsg) -> - AsnMod = ?V2_ASN1_MOD_DRV, - TransMod = ?V2_TRANS_MOD, - ?BIN_LIB:encode_message(EC, MegaMsg, AsnMod, TransMod, io_list); -encode_message([{version3,_}|EC], 2, MegaMsg) -> - AsnMod = ?V2_ASN1_MOD, - TransMod = ?V2_TRANS_MOD, - ?BIN_LIB:encode_message(EC, MegaMsg, AsnMod, TransMod, io_list); - -%% All values we need to take (special) care of has been delt with, -%% so just pass the rest on -encode_message(EC, 2, MegaMsg) -> - AsnMod = ?V2_ASN1_MOD, - TransMod = ?V2_TRANS_MOD, - ?BIN_LIB:encode_message(EC, MegaMsg, AsnMod, TransMod, io_list); - - -%% -- Version 3 -- - -encode_message([{version3,v3},driver|EC], 3, MegaMsg) -> - AsnMod = ?V3_ASN1_MOD_DRV, - TransMod = ?V3_TRANS_MOD, - ?BIN_LIB:encode_message(EC, MegaMsg, AsnMod, TransMod, io_list); -encode_message([{version3,prev3c},driver|EC], 3, MegaMsg) -> - AsnMod = ?PREV3C_ASN1_MOD_DRV, - TransMod = ?PREV3C_TRANS_MOD, - ?BIN_LIB:encode_message(EC, MegaMsg, AsnMod, TransMod, io_list); -encode_message([{version3,prev3b},driver|EC], 3, MegaMsg) -> - AsnMod = ?PREV3B_ASN1_MOD_DRV, - TransMod = ?PREV3B_TRANS_MOD, - ?BIN_LIB:encode_message(EC, MegaMsg, AsnMod, TransMod, io_list); -encode_message([{version3,prev3a},driver|EC], 3, MegaMsg) -> - AsnMod = ?PREV3A_ASN1_MOD_DRV, - TransMod = ?PREV3A_TRANS_MOD, - ?BIN_LIB:encode_message(EC, MegaMsg, AsnMod, TransMod, io_list); -encode_message([driver|EC], 3, MegaMsg) -> - AsnMod = ?V3_ASN1_MOD_DRV, - TransMod = ?V3_TRANS_MOD, - ?BIN_LIB:encode_message(EC, MegaMsg, AsnMod, TransMod, io_list); -encode_message([{version3,v3}|EC], 3, MegaMsg) -> - AsnMod = ?V3_ASN1_MOD, - TransMod = ?V3_TRANS_MOD, - ?BIN_LIB:encode_message(EC, MegaMsg, AsnMod, TransMod, io_list); -encode_message([{version3,prev3c}|EC], 3, MegaMsg) -> - AsnMod = ?PREV3C_ASN1_MOD, - TransMod = ?PREV3C_TRANS_MOD, - ?BIN_LIB:encode_message(EC, MegaMsg, AsnMod, TransMod, io_list); -encode_message([{version3,prev3b}|EC], 3, MegaMsg) -> - AsnMod = ?PREV3B_ASN1_MOD, - TransMod = ?PREV3B_TRANS_MOD, - ?BIN_LIB:encode_message(EC, MegaMsg, AsnMod, TransMod, io_list); -encode_message([{version3,prev3a}|EC], 3, MegaMsg) -> - AsnMod = ?PREV3A_ASN1_MOD, - TransMod = ?PREV3A_TRANS_MOD, - ?BIN_LIB:encode_message(EC, MegaMsg, AsnMod, TransMod, io_list); - -%% All values we need to take (special) care of has been delt with, -%% so just pass the rest on -encode_message(EC, 3, MegaMsg) -> - AsnMod = ?V3_ASN1_MOD, - TransMod = ?V3_TRANS_MOD, - ?BIN_LIB:encode_message(EC, MegaMsg, AsnMod, TransMod, io_list). - - -%%---------------------------------------------------------------------- -%% Convert a transaction (or transactions in the case of ack) record(s) -%% into a binary -%% Return {ok, Binary} | {error, Reason} -%%---------------------------------------------------------------------- - -%% encode_transaction([] = EC, 1, Trans) -> -%% AsnMod = ?V1_ASN1_MOD, -%% TransMod = ?V1_TRANS_MOD, -%% ?BIN_LIB:encode_transaction(EC, Trans, AsnMod, TransMod, -%% io_list); -%% encode_transaction([native] = EC, 1, Trans) -> -%% AsnMod = ?V1_ASN1_MOD, -%% TransMod = ?V1_TRANS_MOD, -%% ?BIN_LIB:encode_transaction(EC, Trans, AsnMod, TransMod, -%% io_list); -%% encode_transaction([driver|EC], 1, Trans) -> -%% AsnMod = ?V1_ASN1_MOD_DRV, -%% TransMod = ?V1_TRANS_MOD, -%% ?BIN_LIB:encode_transaction(EC, Trans, AsnMod, TransMod, -%% io_list); -encode_transaction(_EC, 1, _Trans) -> - %% AsnMod = ?V1_ASN1_MOD, - %% TransMod = ?V1_TRANS_MOD, - %% ?BIN_LIB:encode_transaction(EC, Trans, AsnMod, TransMod, - %% io_list); - {error, not_implemented}; - -%% encode_transaction([] = EC, 2, Trans) -> -%% AsnMod = ?V2_ASN1_MOD, -%% TransMod = ?V2_TRANS_MOD, -%% ?BIN_LIB:encode_transaction(EC, Trans, AsnMod, TransMod, -%% io_list); -%% encode_transaction([native] = EC, 2, Trans) -> -%% AsnMod = ?V2_ASN1_MOD, -%% TransMod = ?V2_TRANS_MOD, -%% ?BIN_LIB:encode_transaction(EC, Trans, AsnMod, TransMod, -%% io_list); -%% encode_transaction([driver|EC], 2, Trans) -> -%% AsnMod = ?V2_ASN1_MOD_DRV, -%% TransMod = ?V2_TRANS_MOD, -%% ?BIN_LIB:encode_transaction(EC, Trans, AsnMod, TransMod, -%% io_list); -encode_transaction(_EC, 2, _Trans) -> - %% AsnMod = ?V2_ASN1_MOD, - %% TransMod = ?V2_TRANS_MOD, - %% ?BIN_LIB:encode_transaction(EC, Trans, AsnMod, TransMod, - %% io_list). - {error, not_implemented}; - -%% encode_transaction([] = EC, 3, Trans) -> -%% AsnMod = ?V3_ASN1_MOD, -%% TransMod = ?V3_TRANS_MOD, -%% ?BIN_LIB:encode_transaction(EC, Trans, AsnMod, TransMod, -%% io_list); -%% encode_transaction([native] = EC, 3, Trans) -> -%% AsnMod = ?V3_ASN1_MOD, -%% TransMod = ?V3_TRANS_MOD, -%% ?BIN_LIB:encode_transaction(EC, Trans, AsnMod, TransMod, -%% io_list); -%% encode_transaction([driver|EC], 3, Trans) -> -%% AsnMod = ?V3_ASN1_MOD_DRV, -%% TransMod = ?V3_TRANS_MOD, -%% ?BIN_LIB:encode_transaction(EC, Trans, AsnMod, TransMod, -%% io_list); -encode_transaction(_EC, 3, _Trans) -> - %% AsnMod = ?V3_ASN1_MOD, - %% TransMod = ?V3_TRANS_MOD, - %% ?BIN_LIB:encode_transaction(EC, Trans, AsnMod, TransMod, %% io_list). - {error, not_implemented}. - - -%%---------------------------------------------------------------------- -%% Convert a list of ActionRequest record's into a binary -%% Return {ok, DeepIoList} | {error, Reason} -%%---------------------------------------------------------------------- -encode_action_requests(_EC, 1, ActReqs) when is_list(ActReqs) -> - %% ?BIN_LIB:encode_action_requests(EC, ActReqs, - %% ?V1_ASN1_MOD, - %% ?V1_TRANS_MOD, - %% io_list); - {error, not_implemented}; -encode_action_requests(_EC, 2, ActReqs) when is_list(ActReqs) -> - %% ?BIN_LIB:encode_action_requests(EC, ActReqs, - %% ?V1_ASN1_MOD, - %% ?V1_TRANS_MOD, - %% io_list). - {error, not_implemented}; -encode_action_requests(_EC, 3, ActReqs) when is_list(ActReqs) -> - %% ?BIN_LIB:encode_action_requests(EC, ActReqs, - %% ?V1_ASN1_MOD, - %% ?V1_TRANS_MOD, - %% io_list). - {error, not_implemented}. - - -%%---------------------------------------------------------------------- -%% Convert a ActionRequest record into a binary -%% Return {ok, DeepIoList} | {error, Reason} -%%---------------------------------------------------------------------- -encode_action_request(_EC, 1, _ActReq) -> - %% ?BIN_LIB:encode_action_request(EC, ActReq, - %% ?V1_ASN1_MOD, - %% ?V1_TRANS_MOD, - %% io_list); - {error, not_implemented}; -encode_action_request(_EC, 2, _ActReq) -> - %% ?BIN_LIB:encode_action_request(EC, ActReq, - %% ?V1_ASN1_MOD, - %% ?V1_TRANS_MOD, - %% io_list). - {error, not_implemented}; -encode_action_request(_EC, 3, _ActReq) -> - %% ?BIN_LIB:encode_action_request(EC, ActReq, - %% ?V1_ASN1_MOD, - %% ?V1_TRANS_MOD, - %% io_list). - {error, not_implemented}. - - -%%---------------------------------------------------------------------- -%% Convert a action reply into a deep io list -%% Not yest supported by this binary codec! -%% Return {ok, DeepIoList} | {error, Reason} -%%---------------------------------------------------------------------- - -encode_action_reply(_EC, _V, _AcionReply) -> - {error, not_implemented}. - - -%%---------------------------------------------------------------------- -%% Convert a binary into a 'MegacoMessage' record -%% Return {ok, MegacoMessageRecord} | {error, Reason} -%%---------------------------------------------------------------------- - -decode_message(EC, Binary) -> - decode_message(EC, 1, Binary). - -%% PER does not support partial decode, so this means V1 -decode_message(EC, dynamic, Binary) -> - decode_message(EC, 1, Binary); - - -%% -- Version 1 -- - -decode_message([{version3,_},driver|EC], 1, Binary) -> - AsnMod = ?V1_ASN1_MOD_DRV, - TransMod = ?V1_TRANS_MOD, - ?BIN_LIB:decode_message(EC, Binary, AsnMod, TransMod, binary); -decode_message([driver|EC], 1, Binary) -> - AsnMod = ?V1_ASN1_MOD_DRV, - TransMod = ?V1_TRANS_MOD, - ?BIN_LIB:decode_message(EC, Binary, AsnMod, TransMod, binary); -decode_message([{version3,_}|EC], 1, Binary) -> - AsnMod = ?V1_ASN1_MOD, - TransMod = ?V1_TRANS_MOD, - ?BIN_LIB:decode_message(EC, Binary, AsnMod, TransMod, binary); - -%% All values we need to take (special) care of has been delt with, -%% so just pass the rest on -decode_message(EC, 1, Binary) -> - AsnMod = ?V1_ASN1_MOD, - TransMod = ?V1_TRANS_MOD, - ?BIN_LIB:decode_message(EC, Binary, AsnMod, TransMod, binary); - - -%% -- Version 2 -- - -decode_message([{version3,_},driver|EC], 2, Binary) -> - AsnMod = ?V2_ASN1_MOD_DRV, - TransMod = ?V2_TRANS_MOD, - ?BIN_LIB:decode_message(EC, Binary, AsnMod, TransMod, binary); -decode_message([driver|EC], 2, Binary) -> - AsnMod = ?V2_ASN1_MOD_DRV, - TransMod = ?V2_TRANS_MOD, - ?BIN_LIB:decode_message(EC, Binary, AsnMod, TransMod, binary); -decode_message([{version3,_}|EC], 2, Binary) -> - AsnMod = ?V2_ASN1_MOD, - TransMod = ?V2_TRANS_MOD, - ?BIN_LIB:decode_message(EC, Binary, AsnMod, TransMod, binary); - -%% All values we need to take (special) care of has been delt with, -%% so just pass the rest on -decode_message(EC, 2, Binary) -> - AsnMod = ?V2_ASN1_MOD, - TransMod = ?V2_TRANS_MOD, - ?BIN_LIB:decode_message(EC, Binary, AsnMod, TransMod, binary); - - -%% -- Version 3 -- - -decode_message([{version3,v3},driver|EC], 3, Binary) -> - AsnMod = ?V3_ASN1_MOD_DRV, - TransMod = ?V3_TRANS_MOD, - ?BIN_LIB:decode_message(EC, Binary, AsnMod, TransMod, binary); -decode_message([{version3,prev3c},driver|EC], 3, Binary) -> - AsnMod = ?PREV3C_ASN1_MOD_DRV, - TransMod = ?PREV3C_TRANS_MOD, - ?BIN_LIB:decode_message(EC, Binary, AsnMod, TransMod, binary); -decode_message([{version3,prev3b},driver|EC], 3, Binary) -> - AsnMod = ?PREV3B_ASN1_MOD_DRV, - TransMod = ?PREV3B_TRANS_MOD, - ?BIN_LIB:decode_message(EC, Binary, AsnMod, TransMod, binary); -decode_message([{version3,prev3a},driver|EC], 3, Binary) -> - AsnMod = ?PREV3A_ASN1_MOD_DRV, - TransMod = ?PREV3A_TRANS_MOD, - ?BIN_LIB:decode_message(EC, Binary, AsnMod, TransMod, binary); -decode_message([driver|EC], 3, Binary) -> - AsnMod = ?V3_ASN1_MOD_DRV, - TransMod = ?V3_TRANS_MOD, - ?BIN_LIB:decode_message(EC, Binary, AsnMod, TransMod, binary); -decode_message([{version3,v3}|EC], 3, Binary) -> - AsnMod = ?V3_ASN1_MOD, - TransMod = ?V3_TRANS_MOD, - ?BIN_LIB:decode_message(EC, Binary, AsnMod, TransMod, binary); -decode_message([{version3,prev3c}|EC], 3, Binary) -> - AsnMod = ?PREV3C_ASN1_MOD, - TransMod = ?PREV3C_TRANS_MOD, - ?BIN_LIB:decode_message(EC, Binary, AsnMod, TransMod, binary); -decode_message([{version3,prev3b}|EC], 3, Binary) -> - AsnMod = ?PREV3B_ASN1_MOD, - TransMod = ?PREV3B_TRANS_MOD, - ?BIN_LIB:decode_message(EC, Binary, AsnMod, TransMod, binary); -decode_message([{version3,prev3a}|EC], 3, Binary) -> - AsnMod = ?PREV3A_ASN1_MOD, - TransMod = ?PREV3A_TRANS_MOD, - ?BIN_LIB:decode_message(EC, Binary, AsnMod, TransMod, binary); - -%% All values we need to take (special) care of has been delt with, -%% so just pass the rest on -decode_message(EC, 3, Binary) -> - AsnMod = ?V3_ASN1_MOD, - TransMod = ?V3_TRANS_MOD, - ?BIN_LIB:decode_message(EC, Binary, AsnMod, TransMod, binary). - - -decode_mini_message(_EC, _Vsn, _Bin) -> - {error, not_implemented}. diff --git a/lib/megaco/src/binary/megaco_per_bin_media_gateway_control_prev3a.set.asn b/lib/megaco/src/binary/megaco_per_bin_media_gateway_control_prev3a.set.asn deleted file mode 100644 index b9ba7ffdb4..0000000000 --- a/lib/megaco/src/binary/megaco_per_bin_media_gateway_control_prev3a.set.asn +++ /dev/null @@ -1 +0,0 @@ -MEDIA-GATEWAY-CONTROL-prev3a.asn diff --git a/lib/megaco/src/binary/megaco_per_bin_media_gateway_control_prev3b.set.asn b/lib/megaco/src/binary/megaco_per_bin_media_gateway_control_prev3b.set.asn deleted file mode 100644 index 0437bde310..0000000000 --- a/lib/megaco/src/binary/megaco_per_bin_media_gateway_control_prev3b.set.asn +++ /dev/null @@ -1 +0,0 @@ -MEDIA-GATEWAY-CONTROL-prev3b.asn diff --git a/lib/megaco/src/binary/megaco_per_bin_media_gateway_control_prev3c.set.asn b/lib/megaco/src/binary/megaco_per_bin_media_gateway_control_prev3c.set.asn deleted file mode 100644 index e78055fbad..0000000000 --- a/lib/megaco/src/binary/megaco_per_bin_media_gateway_control_prev3c.set.asn +++ /dev/null @@ -1 +0,0 @@ -MEDIA-GATEWAY-CONTROL-prev3c.asn diff --git a/lib/megaco/src/binary/megaco_per_bin_media_gateway_control_v1.set.asn b/lib/megaco/src/binary/megaco_per_bin_media_gateway_control_v1.set.asn deleted file mode 100644 index 0f5a92dba1..0000000000 --- a/lib/megaco/src/binary/megaco_per_bin_media_gateway_control_v1.set.asn +++ /dev/null @@ -1 +0,0 @@ -MEDIA-GATEWAY-CONTROL-v1.asn diff --git a/lib/megaco/src/binary/megaco_per_bin_media_gateway_control_v2.set.asn b/lib/megaco/src/binary/megaco_per_bin_media_gateway_control_v2.set.asn deleted file mode 100644 index 7fc82b127f..0000000000 --- a/lib/megaco/src/binary/megaco_per_bin_media_gateway_control_v2.set.asn +++ /dev/null @@ -1 +0,0 @@ -MEDIA-GATEWAY-CONTROL-v2.asn diff --git a/lib/megaco/src/binary/megaco_per_bin_media_gateway_control_v3.set.asn b/lib/megaco/src/binary/megaco_per_bin_media_gateway_control_v3.set.asn deleted file mode 100644 index 1d7950a283..0000000000 --- a/lib/megaco/src/binary/megaco_per_bin_media_gateway_control_v3.set.asn +++ /dev/null @@ -1 +0,0 @@ -MEDIA-GATEWAY-CONTROL-v3.asn diff --git a/lib/megaco/src/binary/modules.mk b/lib/megaco/src/binary/modules.mk index a86ce2aecc..bbaf087ceb 100644 --- a/lib/megaco/src/binary/modules.mk +++ b/lib/megaco/src/binary/modules.mk @@ -27,19 +27,6 @@ MODULES = \ megaco_ber_media_gateway_control_prev3b \ megaco_ber_media_gateway_control_prev3c \ megaco_ber_media_gateway_control_v3 \ - megaco_ber_bin_encoder \ - megaco_ber_bin_media_gateway_control_v1 \ - megaco_ber_bin_media_gateway_control_v2 \ - megaco_ber_bin_media_gateway_control_prev3a \ - megaco_ber_bin_media_gateway_control_prev3b \ - megaco_ber_bin_media_gateway_control_prev3c \ - megaco_ber_bin_media_gateway_control_v3 \ - megaco_ber_bin_drv_media_gateway_control_v1 \ - megaco_ber_bin_drv_media_gateway_control_v2 \ - megaco_ber_bin_drv_media_gateway_control_prev3a \ - megaco_ber_bin_drv_media_gateway_control_prev3b \ - megaco_ber_bin_drv_media_gateway_control_prev3c \ - megaco_ber_bin_drv_media_gateway_control_v3 \ megaco_per_encoder \ megaco_per_media_gateway_control_v1 \ megaco_per_media_gateway_control_v2 \ @@ -47,19 +34,6 @@ MODULES = \ megaco_per_media_gateway_control_prev3b \ megaco_per_media_gateway_control_prev3c \ megaco_per_media_gateway_control_v3 \ - megaco_per_bin_encoder \ - megaco_per_bin_media_gateway_control_v1 \ - megaco_per_bin_media_gateway_control_v2 \ - megaco_per_bin_media_gateway_control_prev3a \ - megaco_per_bin_media_gateway_control_prev3b \ - megaco_per_bin_media_gateway_control_prev3c \ - megaco_per_bin_media_gateway_control_v3 \ - megaco_per_bin_drv_media_gateway_control_v1 \ - megaco_per_bin_drv_media_gateway_control_v2 \ - megaco_per_bin_drv_media_gateway_control_prev3a \ - megaco_per_bin_drv_media_gateway_control_prev3b \ - megaco_per_bin_drv_media_gateway_control_prev3c \ - megaco_per_bin_drv_media_gateway_control_v3 \ megaco_binary_name_resolver_v1 \ megaco_binary_name_resolver_v2 \ megaco_binary_name_resolver_prev3a \ @@ -85,44 +59,20 @@ ASN1_PREV3C_SPEC = MEDIA-GATEWAY-CONTROL-prev3c ASN1_V3_SPEC = MEDIA-GATEWAY-CONTROL-v3 BER_ASN1_V1_SPEC = megaco_ber_media_gateway_control_v1 -BER_BIN_ASN1_V1_SPEC = megaco_ber_bin_media_gateway_control_v1 -BER_BIN_DRV_ASN1_V1_SPEC = megaco_ber_bin_drv_media_gateway_control_v1 PER_ASN1_V1_SPEC = megaco_per_media_gateway_control_v1 -PER_BIN_ASN1_V1_SPEC = megaco_per_bin_media_gateway_control_v1 -PER_BIN_DRV_ASN1_V1_SPEC = megaco_per_bin_drv_media_gateway_control_v1 BER_ASN1_V2_SPEC = megaco_ber_media_gateway_control_v2 -BER_BIN_ASN1_V2_SPEC = megaco_ber_bin_media_gateway_control_v2 -BER_BIN_DRV_ASN1_V2_SPEC = megaco_ber_bin_drv_media_gateway_control_v2 PER_ASN1_V2_SPEC = megaco_per_media_gateway_control_v2 -PER_BIN_ASN1_V2_SPEC = megaco_per_bin_media_gateway_control_v2 -PER_BIN_DRV_ASN1_V2_SPEC = megaco_per_bin_drv_media_gateway_control_v2 BER_ASN1_PREV3A_SPEC = megaco_ber_media_gateway_control_prev3a -BER_BIN_ASN1_PREV3A_SPEC = megaco_ber_bin_media_gateway_control_prev3a -BER_BIN_DRV_ASN1_PREV3A_SPEC = megaco_ber_bin_drv_media_gateway_control_prev3a PER_ASN1_PREV3A_SPEC = megaco_per_media_gateway_control_prev3a -PER_BIN_ASN1_PREV3A_SPEC = megaco_per_bin_media_gateway_control_prev3a -PER_BIN_DRV_ASN1_PREV3A_SPEC = megaco_per_bin_drv_media_gateway_control_prev3a BER_ASN1_PREV3B_SPEC = megaco_ber_media_gateway_control_prev3b -BER_BIN_ASN1_PREV3B_SPEC = megaco_ber_bin_media_gateway_control_prev3b -BER_BIN_DRV_ASN1_PREV3B_SPEC = megaco_ber_bin_drv_media_gateway_control_prev3b PER_ASN1_PREV3B_SPEC = megaco_per_media_gateway_control_prev3b -PER_BIN_ASN1_PREV3B_SPEC = megaco_per_bin_media_gateway_control_prev3b -PER_BIN_DRV_ASN1_PREV3B_SPEC = megaco_per_bin_drv_media_gateway_control_prev3b BER_ASN1_PREV3C_SPEC = megaco_ber_media_gateway_control_prev3c -BER_BIN_ASN1_PREV3C_SPEC = megaco_ber_bin_media_gateway_control_prev3c -BER_BIN_DRV_ASN1_PREV3C_SPEC = megaco_ber_bin_drv_media_gateway_control_prev3c PER_ASN1_PREV3C_SPEC = megaco_per_media_gateway_control_prev3c -PER_BIN_ASN1_PREV3C_SPEC = megaco_per_bin_media_gateway_control_prev3c -PER_BIN_DRV_ASN1_PREV3C_SPEC = megaco_per_bin_drv_media_gateway_control_prev3c BER_ASN1_V3_SPEC = megaco_ber_media_gateway_control_v3 -BER_BIN_ASN1_V3_SPEC = megaco_ber_bin_media_gateway_control_v3 -BER_BIN_DRV_ASN1_V3_SPEC = megaco_ber_bin_drv_media_gateway_control_v3 PER_ASN1_V3_SPEC = megaco_per_media_gateway_control_v3 -PER_BIN_ASN1_V3_SPEC = megaco_per_bin_media_gateway_control_v3 -PER_BIN_DRV_ASN1_V3_SPEC = megaco_per_bin_drv_media_gateway_control_v3 diff --git a/lib/megaco/test/megaco_actions_test.erl b/lib/megaco/test/megaco_actions_test.erl index 2efb6e834a..6d0e80281d 100644 --- a/lib/megaco/test/megaco_actions_test.erl +++ b/lib/megaco/test/megaco_actions_test.erl @@ -80,8 +80,7 @@ end_per_testcase(Case, Config) -> all() -> [pretty_text, flex_pretty_text, compact_text, - flex_compact_text, erl_dist, erl_dist_mc, ber_bin, - ber_bin_drv, ber_bin_native, ber_bin_drv_native]. + flex_compact_text, erl_dist, erl_dist_mc]. groups() -> []. @@ -170,39 +169,6 @@ erl_dist_mc(Config) when is_list(Config) -> req_and_rep(Config, Codec, Version, EncodingConfig). -ber_bin(suite) -> - []; -ber_bin(doc) -> - []; -ber_bin(Config) when is_list(Config) -> - ?SKIP(currently_not_supported_by_asn1). - - -ber_bin_drv(suite) -> - []; -ber_bin_drv(doc) -> - []; -ber_bin_drv(Config) when is_list(Config) -> - ?SKIP(currently_not_supported_by_asn1). - - -ber_bin_native(suite) -> - []; -ber_bin_native(doc) -> - []; -ber_bin_native(Config) when is_list(Config) -> - ?SKIP(currently_not_supported_by_asn1). - - -ber_bin_drv_native(suite) -> - []; -ber_bin_drv_native(doc) -> - []; -ber_bin_drv_native(Config) when is_list(Config) -> - ?SKIP(currently_not_supported_by_asn1). - - - %%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%% req_and_rep(Config, Codec, _Version, EC) when is_list(Config) -> diff --git a/lib/megaco/test/megaco_call_flow_test.erl b/lib/megaco/test/megaco_call_flow_test.erl index b9d64ca8b2..d5888018cd 100644 --- a/lib/megaco/test/megaco_call_flow_test.erl +++ b/lib/megaco/test/megaco_call_flow_test.erl @@ -25,7 +25,6 @@ %% megaco_call_flow_test:compact_text(). %% megaco_call_flow_test:bin(). %% megaco_call_flow_test:asn1_ber(). -%% megaco_call_flow_test:asn1_ber_bin(). %% megaco_call_flow_test:asn1_per(). %% megaco_call_flow_test:erl_dist(). %% megaco_call_flow_test:compressed_erl_dist(). @@ -62,7 +61,7 @@ all() -> groups() -> [{text, [], [pretty, compact]}, {flex, [], [pretty_flex, compact_flex]}, - {binary, [], [bin, ber, ber_bin, per]}]. + {binary, [], [bin, ber, per]}]. init_per_group(_GroupName, Config) -> Config. @@ -106,12 +105,6 @@ ber(Config) when is_list(Config) -> ?ACQUIRE_NODES(1, Config), asn1_ber(). -ber_bin(suite) -> - []; -ber_bin(Config) when is_list(Config) -> - ?ACQUIRE_NODES(1, Config), - asn1_ber_bin(). - per(suite) -> []; per(Config) when is_list(Config) -> @@ -1198,8 +1191,7 @@ encoders() -> {megaco_pretty_text_encoder, [], []}, {megaco_compact_text_encoder, [], []}, {megaco_binary_encoder, [], [native]}, - %% {megaco_ber_encoder, [], [native]}, - %% {megaco_ber_bin_encoder, [], [native]}, + {megaco_ber_encoder, [], [native]}, {megaco_per_encoder, [], [native]}, {megaco_erl_dist_encoder, [], []}, {megaco_erl_dist_encoder, [compressed], [compressed]} @@ -1214,7 +1206,6 @@ pretty_mod({Mod, Opt, _Opt2}) -> megaco_compact_text_encoder -> compact_text; megaco_binary_encoder -> asn1_ber; megaco_ber_encoder -> asn1_ber_old; - megaco_ber_bin_encoder -> asn1_ber_bin; megaco_per_encoder -> asn1_per; megaco_erl_dist_encoder when Opt == [] -> standard_erl; megaco_erl_dist_encoder when Opt == [compressed] -> compressed_erl; @@ -1263,13 +1254,6 @@ asn1_ber() -> All = [encode(Slogan, Msg, Encoder) || {Slogan, Msg} <- messages()], compute_res(All). -asn1_ber_bin() -> - Default = [], - Native = [native], - Encoder = {megaco_ber_bin_encoder, Default, Native}, - All = [encode(Slogan, Msg, Encoder) || {Slogan, Msg} <- messages()], - compute_res(All). - asn1_per() -> Default = [], Native = [native], @@ -1634,7 +1618,7 @@ gen_ber_header() -> %% Generate headerfile for asn.1 BER test in C %%---------------------------------------------------------------------- gen_ber_bin_header() -> - Encoder = {megaco_ber_bin_encoder, [], []}, + Encoder = {megaco_ber_encoder, [], []}, L = [{S, gen_byte_msg(Msg, Encoder)} || {S, Msg} <- messages()], gen_header_file_binary(L). diff --git a/lib/megaco/test/megaco_codec_prev3a_test.erl b/lib/megaco/test/megaco_codec_prev3a_test.erl index d50e72aef1..b2316eb509 100644 --- a/lib/megaco/test/megaco_codec_prev3a_test.erl +++ b/lib/megaco/test/megaco_codec_prev3a_test.erl @@ -1,7 +1,7 @@ %% %% %CopyrightBegin% %% -%% Copyright Ericsson AB 2005-2010. All Rights Reserved. +%% Copyright Ericsson AB 2005-2012. All Rights Reserved. %% %% The contents of this file are subject to the Erlang Public License, %% Version 1.1, (the "License"); you may not use this file except in @@ -63,12 +63,8 @@ ber_test_msgs/1, - ber_bin_test_msgs/1, - per_test_msgs/1, - per_bin_test_msgs/1, - erl_dist_m_test_msgs/1, tickets/0, @@ -280,17 +276,14 @@ groups() -> [{group, pretty}, {group, flex_pretty}, {group, compact}, {group, flex_compact}]}, {binary, [], - [{group, bin}, {group, ber}, {group, ber_bin}, - {group, per}, {group, per_bin}]}, + [{group, bin}, {group, ber}, {group, per}]}, {erl_dist, [], [{group, erl_dist_m}]}, {pretty, [], [pretty_test_msgs]}, {compact, [], [compact_test_msgs]}, {flex_pretty, [], flex_pretty_cases()}, {flex_compact, [], flex_compact_cases()}, {bin, [], [bin_test_msgs]}, {ber, [], [ber_test_msgs]}, - {ber_bin, [], [ber_bin_test_msgs]}, {per, [], [per_test_msgs]}, - {per_bin, [], [per_bin_test_msgs]}, {erl_dist_m, [], [erl_dist_m_test_msgs]}, {tickets, [], [{group, compact_tickets}, @@ -1106,17 +1099,6 @@ ber_test_msgs(Config) when is_list(Config) -> %%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%% -ber_bin_test_msgs(suite) -> - []; -ber_bin_test_msgs(Config) when is_list(Config) -> - ?ACQUIRE_NODES(1, Config), - Msgs = msgs1(binary) ++ msgs4(binary) ++ msgs5(binary) ++ msgs6(binary), - DynamicDecode = true, - test_msgs(megaco_ber_bin_encoder, DynamicDecode, ?EC, Msgs). - - -%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%% - per_test_msgs(suite) -> []; per_test_msgs(Config) when is_list(Config) -> @@ -1128,17 +1110,6 @@ per_test_msgs(Config) when is_list(Config) -> %%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%% -per_bin_test_msgs(suite) -> - []; -per_bin_test_msgs(Config) when is_list(Config) -> - ?ACQUIRE_NODES(1, Config), - Msgs = msgs1(binary) ++ msgs4(binary) ++ msgs5(binary) ++ msgs6(binary), - DynamicDecode = false, - test_msgs(megaco_per_bin_encoder, DynamicDecode, ?EC, Msgs). - - -%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%% - erl_dist_m_test_msgs(suite) -> []; erl_dist_m_test_msgs(Config) when is_list(Config) -> @@ -3133,7 +3104,7 @@ pretty_otp5068_msg1() -> 190, asn1_NOVALUE, {actionReplies, - [{'ActionReply', %% Comments: Detta upprepas m�nga g�nger + [{'ActionReply', %% Comments: This is repeated many times. 0, asn1_NOVALUE, asn1_NOVALUE, diff --git a/lib/megaco/test/megaco_codec_prev3b_test.erl b/lib/megaco/test/megaco_codec_prev3b_test.erl index eaab8f37c1..fa24f49372 100644 --- a/lib/megaco/test/megaco_codec_prev3b_test.erl +++ b/lib/megaco/test/megaco_codec_prev3b_test.erl @@ -1,7 +1,7 @@ %% %% %CopyrightBegin% %% -%% Copyright Ericsson AB 2005-2010. All Rights Reserved. +%% Copyright Ericsson AB 2005-2012. All Rights Reserved. %% %% The contents of this file are subject to the Erlang Public License, %% Version 1.1, (the "License"); you may not use this file except in @@ -63,12 +63,8 @@ ber_test_msgs/1, - ber_bin_test_msgs/1, - per_test_msgs/1, - per_bin_test_msgs/1, - erl_dist_m_test_msgs/1, tickets/0, @@ -296,17 +292,14 @@ groups() -> [{group, pretty}, {group, flex_pretty}, {group, compact}, {group, flex_compact}]}, {binary, [], - [{group, bin}, {group, ber}, {group, ber_bin}, - {group, per}, {group, per_bin}]}, + [{group, bin}, {group, ber}, {group, per}]}, {erl_dist, [], [{group, erl_dist_m}]}, {pretty, [], [pretty_test_msgs]}, {compact, [], [compact_test_msgs]}, {flex_pretty, [], flex_pretty_cases()}, {flex_compact, [], flex_compact_cases()}, {bin, [], [bin_test_msgs]}, {ber, [], [ber_test_msgs]}, - {ber_bin, [], [ber_bin_test_msgs]}, {per, [], [per_test_msgs]}, - {per_bin, [], [per_bin_test_msgs]}, {erl_dist_m, [], [erl_dist_m_test_msgs]}, {tickets, [], [{group, compact_tickets}, @@ -1171,16 +1164,6 @@ ber_test_msgs(Config) when is_list(Config) -> %%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%% -ber_bin_test_msgs(suite) -> - []; -ber_bin_test_msgs(Config) when is_list(Config) -> - ?ACQUIRE_NODES(1, Config), - Msgs = msgs1(binary) ++ msgs4(binary) ++ msgs5(binary) ++ msgs6(binary), - DynamicDecode = true, - test_msgs(megaco_ber_bin_encoder, DynamicDecode, ?EC, Msgs). - - -%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%% per_test_msgs(suite) -> []; @@ -1193,17 +1176,6 @@ per_test_msgs(Config) when is_list(Config) -> %%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%% -per_bin_test_msgs(suite) -> - []; -per_bin_test_msgs(Config) when is_list(Config) -> - ?ACQUIRE_NODES(1, Config), - Msgs = msgs1(binary) ++ msgs4(binary) ++ msgs5(binary) ++ msgs6(binary), - DynamicDecode = false, - test_msgs(megaco_per_bin_encoder, DynamicDecode, ?EC, Msgs). - - -%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%% - erl_dist_m_test_msgs(suite) -> []; erl_dist_m_test_msgs(Config) when is_list(Config) -> @@ -3251,7 +3223,7 @@ pretty_otp5068_msg1() -> 190, asn1_NOVALUE, {actionReplies, - [{'ActionReply', %% Comments: Detta upprepas m�nga g�nger + [{'ActionReply', %% Comments: This is repeated many times. 0, asn1_NOVALUE, asn1_NOVALUE, diff --git a/lib/megaco/test/megaco_codec_prev3c_test.erl b/lib/megaco/test/megaco_codec_prev3c_test.erl index 7f9c0fe4e7..7f6d098ed8 100644 --- a/lib/megaco/test/megaco_codec_prev3c_test.erl +++ b/lib/megaco/test/megaco_codec_prev3c_test.erl @@ -1,7 +1,7 @@ %% %% %CopyrightBegin% %% -%% Copyright Ericsson AB 2006-2010. All Rights Reserved. +%% Copyright Ericsson AB 2006-2012. All Rights Reserved. %% %% The contents of this file are subject to the Erlang Public License, %% Version 1.1, (the "License"); you may not use this file except in @@ -65,12 +65,8 @@ ber_test_msgs/1, - ber_bin_test_msgs/1, - per_test_msgs/1, - per_bin_test_msgs/1, - erl_dist_m_test_msgs/1, tickets/0, @@ -301,17 +297,14 @@ groups() -> [{group, pretty}, {group, flex_pretty}, {group, compact}, {group, flex_compact}]}, {binary, [], - [{group, bin}, {group, ber}, {group, ber_bin}, - {group, per}, {group, per_bin}]}, + [{group, bin}, {group, ber}, {group, per}]}, {erl_dist, [], [{group, erl_dist_m}]}, {pretty, [], [pretty_test_msgs]}, {compact, [], [compact_test_msgs]}, {flex_pretty, [], flex_pretty_cases()}, {flex_compact, [], flex_compact_cases()}, {bin, [], [bin_test_msgs]}, {ber, [], [ber_test_msgs]}, - {ber_bin, [], [ber_bin_test_msgs]}, {per, [], [per_test_msgs]}, - {per_bin, [], [per_bin_test_msgs]}, {erl_dist_m, [], [erl_dist_m_test_msgs]}, {tickets, [], [{group, compact_tickets}, @@ -823,21 +816,6 @@ ber_test_msgs(Config) when is_list(Config) -> %%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%% -ber_bin_test_msgs(suite) -> - []; -ber_bin_test_msgs(Config) when is_list(Config) -> - ?ACQUIRE_NODES(1, Config), - Msgs = - msgs1a(binary) ++ - msgs5(binary) ++ - msgs6(binary) ++ - msgs7(binary), - DynamicDecode = true, - test_msgs(megaco_ber_bin_encoder, DynamicDecode, ?EC, Msgs). - - -%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%% - per_test_msgs(suite) -> []; per_test_msgs(Config) when is_list(Config) -> @@ -853,21 +831,6 @@ per_test_msgs(Config) when is_list(Config) -> %%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%% -per_bin_test_msgs(suite) -> - []; -per_bin_test_msgs(Config) when is_list(Config) -> - ?ACQUIRE_NODES(1, Config), - Msgs = - msgs1a(binary) ++ - msgs5(binary) ++ - msgs6(binary) ++ - msgs7(binary), - DynamicDecode = false, - test_msgs(megaco_per_bin_encoder, DynamicDecode, ?EC, Msgs). - - -%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%% - erl_dist_m_test_msgs(suite) -> []; erl_dist_m_test_msgs(Config) when is_list(Config) -> @@ -2716,7 +2679,7 @@ pretty_otp5068_msg1() -> 190, asn1_NOVALUE, {actionReplies, - [{'ActionReply', %% Comments: Detta upprepas m�nga g�nger + [{'ActionReply', %% Comments: This is repeated many times. 0, asn1_NOVALUE, asn1_NOVALUE, diff --git a/lib/megaco/test/megaco_codec_v1_test.erl b/lib/megaco/test/megaco_codec_v1_test.erl index e9c19605dd..3be0da3ae4 100644 --- a/lib/megaco/test/megaco_codec_v1_test.erl +++ b/lib/megaco/test/megaco_codec_v1_test.erl @@ -1,7 +1,7 @@ %% %% %CopyrightBegin% %% -%% Copyright Ericsson AB 2003-2011. All Rights Reserved. +%% Copyright Ericsson AB 2003-2012. All Rights Reserved. %% %% The contents of this file are subject to the Erlang Public License, %% Version 1.1, (the "License"); you may not use this file except in @@ -66,12 +66,8 @@ ber_test_msgs/1, - ber_bin_test_msgs/1, - per_test_msgs/1, - per_bin_test_msgs/1, - erl_dist_m_test_msgs/1, tickets/0, @@ -476,9 +472,7 @@ groups() -> {group, flex_compact}]}, {binary, [], [{group, bin}, {group, ber}, - {group, ber_bin}, - {group, per}, - {group, per_bin}]}, + {group, per}]}, {erl_dist, [], [{group, erl_dist_m}]}, {pretty, [], [pretty_test_msgs]}, {compact, [], [compact_test_msgs]}, @@ -486,9 +480,7 @@ groups() -> {flex_compact, [], flex_compact_cases()}, {bin, [], [bin_test_msgs]}, {ber, [], [ber_test_msgs]}, - {ber_bin, [], [ber_bin_test_msgs]}, {per, [], [per_test_msgs]}, - {per_bin, [], [per_bin_test_msgs]}, {erl_dist_m, [], [erl_dist_m_test_msgs]}, {tickets, [], [{group, compact_tickets}, {group, pretty_tickets}, @@ -1266,17 +1258,6 @@ ber_test_msgs(Config) when is_list(Config) -> %%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%% -ber_bin_test_msgs(suite) -> - []; -ber_bin_test_msgs(Config) when is_list(Config) -> - ?ACQUIRE_NODES(1, Config), - Msgs = msgs1(), - DynamicDecode = true, - test_msgs(megaco_ber_bin_encoder, DynamicDecode, [], Msgs). - - -%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%% - per_test_msgs(suite) -> []; per_test_msgs(Config) when is_list(Config) -> @@ -1288,17 +1269,6 @@ per_test_msgs(Config) when is_list(Config) -> %%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%% -per_bin_test_msgs(suite) -> - []; -per_bin_test_msgs(Config) when is_list(Config) -> - ?ACQUIRE_NODES(1, Config), - Msgs = msgs1(), - DynamicDecode = false, - test_msgs(megaco_per_bin_encoder, DynamicDecode, [], Msgs). - - -%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%% - erl_dist_m_test_msgs(suite) -> []; erl_dist_m_test_msgs(Config) when is_list(Config) -> @@ -3328,7 +3298,7 @@ pretty_otp5068_msg1() -> 190, asn1_NOVALUE, {actionReplies, - [{'ActionReply', %% Comments: Detta upprepas m�nga g�nger + [{'ActionReply', %% Comments: This is repeated many times. 0, asn1_NOVALUE, asn1_NOVALUE, diff --git a/lib/megaco/test/megaco_codec_v2_test.erl b/lib/megaco/test/megaco_codec_v2_test.erl index a44f74166c..1f522504d6 100644 --- a/lib/megaco/test/megaco_codec_v2_test.erl +++ b/lib/megaco/test/megaco_codec_v2_test.erl @@ -1,7 +1,7 @@ %% %% %CopyrightBegin% %% -%% Copyright Ericsson AB 2003-2011. All Rights Reserved. +%% Copyright Ericsson AB 2003-2012. All Rights Reserved. %% %% The contents of this file are subject to the Erlang Public License, %% Version 1.1, (the "License"); you may not use this file except in @@ -64,12 +64,8 @@ ber_test_msgs/1, - ber_bin_test_msgs/1, - per_test_msgs/1, - per_bin_test_msgs/1, - erl_dist_m_test_msgs/1, tickets/0, @@ -447,17 +443,14 @@ groups() -> [{group, pretty}, {group, flex_pretty}, {group, compact}, {group, flex_compact}]}, {binary, [], - [{group, bin}, {group, ber}, {group, ber_bin}, - {group, per}, {group, per_bin}]}, + [{group, bin}, {group, ber}, {group, per}]}, {erl_dist, [], [{group, erl_dist_m}]}, {pretty, [], [pretty_test_msgs]}, {compact, [], [compact_test_msgs]}, {flex_pretty, [], flex_pretty_cases()}, {flex_compact, [], flex_compact_cases()}, {bin, [], [bin_test_msgs]}, {ber, [], [ber_test_msgs]}, - {ber_bin, [], [ber_bin_test_msgs]}, {per, [], [per_test_msgs]}, - {per_bin, [], [per_bin_test_msgs]}, {erl_dist_m, [], [erl_dist_m_test_msgs]}, {tickets, [], [{group, compact_tickets}, {group, pretty_tickets}, @@ -1285,17 +1278,6 @@ ber_test_msgs(Config) when is_list(Config) -> %%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%% -ber_bin_test_msgs(suite) -> - []; -ber_bin_test_msgs(Config) when is_list(Config) -> - ?ACQUIRE_NODES(1, Config), - Msgs = msgs1() ++ msgs4(), - DynamicDecode = true, - test_msgs(megaco_ber_bin_encoder, DynamicDecode, [], Msgs). - - -%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%% - per_test_msgs(suite) -> []; per_test_msgs(Config) when is_list(Config) -> @@ -1307,17 +1289,6 @@ per_test_msgs(Config) when is_list(Config) -> %%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%% -per_bin_test_msgs(suite) -> - []; -per_bin_test_msgs(Config) when is_list(Config) -> - ?ACQUIRE_NODES(1, Config), - Msgs = msgs1() ++ msgs4(), - DynamicDecode = false, - test_msgs(megaco_per_bin_encoder, DynamicDecode, [], Msgs). - - -%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%% - erl_dist_m_test_msgs(suite) -> []; erl_dist_m_test_msgs(Config) when is_list(Config) -> @@ -3751,7 +3722,7 @@ pretty_otp5068_msg1() -> 190, asn1_NOVALUE, {actionReplies, - [{'ActionReply', %% Comments: Detta upprepas m�nga g�nger + [{'ActionReply', %% Comments: This is repeated many times. 0, asn1_NOVALUE, asn1_NOVALUE, diff --git a/lib/megaco/test/megaco_codec_v3_test.erl b/lib/megaco/test/megaco_codec_v3_test.erl index 2c35ce13b3..9d564a0ae3 100644 --- a/lib/megaco/test/megaco_codec_v3_test.erl +++ b/lib/megaco/test/megaco_codec_v3_test.erl @@ -1,7 +1,7 @@ %% %% %CopyrightBegin% %% -%% Copyright Ericsson AB 2006-2010. All Rights Reserved. +%% Copyright Ericsson AB 2006-2012. All Rights Reserved. %% %% The contents of this file are subject to the Erlang Public License, %% Version 1.1, (the "License"); you may not use this file except in @@ -56,9 +56,7 @@ flex_compact_dm_timers8/1, bin_test_msgs/1, ber_test_msgs/1, - ber_bin_test_msgs/1, per_test_msgs/1, - per_bin_test_msgs/1, erl_dist_m_test_msgs/1, tickets/0, @@ -288,17 +286,14 @@ groups() -> [{group, pretty}, {group, flex_pretty}, {group, compact}, {group, flex_compact}]}, {binary, [], - [{group, bin}, {group, ber}, {group, ber_bin}, - {group, per}, {group, per_bin}]}, + [{group, bin}, {group, ber}, {group, per}]}, {erl_dist, [], [{group, erl_dist_m}]}, {pretty, [], [pretty_test_msgs]}, {compact, [], [compact_test_msgs]}, {flex_pretty, [], flex_pretty_cases()}, {flex_compact, [], flex_compact_cases()}, {bin, [], [bin_test_msgs]}, {ber, [], [ber_test_msgs]}, - {ber_bin, [], [ber_bin_test_msgs]}, {per, [], [per_test_msgs]}, - {per_bin, [], [per_bin_test_msgs]}, {erl_dist_m, [], [erl_dist_m_test_msgs]}, {tickets, [], [{group, compact_tickets}, @@ -823,22 +818,6 @@ ber_test_msgs(Config) when is_list(Config) -> %%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%% -ber_bin_test_msgs(suite) -> - []; -ber_bin_test_msgs(Config) when is_list(Config) -> - ?ACQUIRE_NODES(1, Config), - Msgs = - msgs1a(binary) ++ - msgs5(binary) ++ - msgs6(binary) ++ - msgs7(binary) ++ - msgs8(binary), - DynamicDecode = true, - test_msgs(megaco_ber_bin_encoder, DynamicDecode, ?EC, Msgs). - - -%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%% - per_test_msgs(suite) -> []; per_test_msgs(Config) when is_list(Config) -> @@ -855,22 +834,6 @@ per_test_msgs(Config) when is_list(Config) -> %%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%% -per_bin_test_msgs(suite) -> - []; -per_bin_test_msgs(Config) when is_list(Config) -> - ?ACQUIRE_NODES(1, Config), - Msgs = - msgs1a(binary) ++ - msgs5(binary) ++ - msgs6(binary) ++ - msgs7(binary) ++ - msgs8(binary), - DynamicDecode = false, - test_msgs(megaco_per_bin_encoder, DynamicDecode, ?EC, Msgs). - - -%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%% - erl_dist_m_test_msgs(suite) -> []; erl_dist_m_test_msgs(Config) when is_list(Config) -> @@ -2738,7 +2701,7 @@ pretty_otp5068_msg1() -> 190, asn1_NOVALUE, {actionReplies, - [{'ActionReply', %% Comments: Detta upprepas m�nga g�nger + [{'ActionReply', %% Comments: This is repeated many times. 0, asn1_NOVALUE, asn1_NOVALUE, diff --git a/lib/megaco/test/megaco_mess_test.erl b/lib/megaco/test/megaco_mess_test.erl index 663ac8c329..f8be96c254 100644 --- a/lib/megaco/test/megaco_mess_test.erl +++ b/lib/megaco/test/megaco_mess_test.erl @@ -1393,7 +1393,7 @@ rarpaop_mgc_event_sequence(text, tcp) -> rarpaop_mgc_event_sequence(binary, tcp) -> Port = 2945, TranspMod = megaco_tcp, - EncMod = megaco_ber_bin_encoder, + EncMod = megaco_ber_encoder, EncConf = [], rarpaop_mgc_event_sequence(Port, TranspMod, EncMod, EncConf). @@ -1680,7 +1680,7 @@ rarpaop_mg_event_sequence(text, tcp) -> rarpaop_mg_event_sequence(Port, EncMod, EncConf); rarpaop_mg_event_sequence(binary, tcp) -> Port = 2945, - EncMod = megaco_ber_bin_encoder, + EncMod = megaco_ber_encoder, EncConf = [], rarpaop_mg_event_sequence(Port, EncMod, EncConf). diff --git a/lib/megaco/test/megaco_mib_test.erl b/lib/megaco/test/megaco_mib_test.erl index 52d99d1442..ddc74ab741 100644 --- a/lib/megaco/test/megaco_mib_test.erl +++ b/lib/megaco/test/megaco_mib_test.erl @@ -647,13 +647,13 @@ mk_recv_info([{text,udp}|ET], Acc) -> {port, 2944}], mk_recv_info(ET, [RI|Acc]); mk_recv_info([{binary,tcp}|ET], Acc) -> - RI = [{encoding_module, megaco_ber_bin_encoder}, + RI = [{encoding_module, megaco_ber_encoder}, {encoding_config, []}, {transport_module, megaco_tcp}, {port, 2945}], mk_recv_info(ET, [RI|Acc]); mk_recv_info([{binary,udp}|ET], Acc) -> - RI = [{encoding_module, megaco_ber_bin_encoder}, + RI = [{encoding_module, megaco_ber_encoder}, {encoding_config, []}, {transport_module, megaco_udp}, {port, 2945}], @@ -1013,7 +1013,7 @@ start_mg(Node, Mid, Encoding, Transport, Verbosity) -> {encoding_config, []}, {port,2944}]; binary -> - [{encoding_module, megaco_ber_bin_encoder}, + [{encoding_module, megaco_ber_encoder}, {encoding_config, []}, {port,2945}] end, diff --git a/lib/megaco/test/megaco_test_mg.erl b/lib/megaco/test/megaco_test_mg.erl index ecb3cedc83..947f0eebbb 100644 --- a/lib/megaco/test/megaco_test_mg.erl +++ b/lib/megaco/test/megaco_test_mg.erl @@ -158,7 +158,7 @@ select_encoding(pretty_text) -> select_encoding(compact_text) -> {megaco_compact_text_encoder, 2944}; select_encoding(binary) -> - {megaco_ber_bin_encoder, 2945}; + {megaco_ber_encoder, 2945}; select_encoding(erl_dist) -> {megaco_erl_dist_encoder, 2946}; select_encoding(Encoding) -> diff --git a/lib/megaco/test/megaco_test_mgc.erl b/lib/megaco/test/megaco_test_mgc.erl index 13c1cebe56..a964983861 100644 --- a/lib/megaco/test/megaco_test_mgc.erl +++ b/lib/megaco/test/megaco_test_mgc.erl @@ -135,7 +135,7 @@ select_encoding(pretty_text) -> select_encoding(compact_text) -> {megaco_compact_text_encoder, 2944}; select_encoding(binary) -> - {megaco_ber_bin_encoder, 2945}; + {megaco_ber_encoder, 2945}; select_encoding(erl_dist) -> {megaco_erl_dist_encoder, 2946}; select_encoding(Encoding) -> diff --git a/lib/mnesia/src/mnesia_controller.erl b/lib/mnesia/src/mnesia_controller.erl index d488a33d67..ec67d9ec12 100644 --- a/lib/mnesia/src/mnesia_controller.erl +++ b/lib/mnesia/src/mnesia_controller.erl @@ -593,6 +593,12 @@ multicall(Nodes, Msg) -> {PatchedGood, Bad}. %% Make the replies look like rpc:multicalls.. %% rpc:multicall(Nodes, ?MODULE, call, [Msg]). +next_async_dump_log() -> + Interval = mnesia_monitor:get_env(dump_log_time_threshold), + Msg = {next_async_dump_log, time_threshold}, + Ref = erlang:send_after(Interval, self(), Msg), + Ref. + %%%---------------------------------------------------------------------- %%% Callback functions from gen_server %%%---------------------------------------------------------------------- @@ -614,9 +620,7 @@ init([Parent]) -> mnesia_lib:unset(original_nodes), mnesia_recover:connect_nodes(Diff), - Interval = mnesia_monitor:get_env(dump_log_time_threshold), - Msg = {async_dump_log, time_threshold}, - {ok, Ref} = timer:send_interval(Interval, Msg), + Ref = next_async_dump_log(), mnesia_dumper:start_regulator(), Empty = gb_trees:empty(), @@ -1121,6 +1125,11 @@ handle_sync_tabs([], _From) -> %% {stop, Reason, State} (terminate/2 is called) %%---------------------------------------------------------------------- +handle_info({next_async_dump_log, InitBy}, State) -> + async_dump_log(InitBy), + Ref = next_async_dump_log(), + noreply(State#state{dump_log_timer_ref=Ref}); + handle_info({async_dump_log, InitBy}, State) -> Worker = #dump_log{initiated_by = InitBy}, State2 = add_worker(Worker, State), diff --git a/lib/mnesia/src/mnesia_event.erl b/lib/mnesia/src/mnesia_event.erl index 8085155fd5..9fd0342d31 100644 --- a/lib/mnesia/src/mnesia_event.erl +++ b/lib/mnesia/src/mnesia_event.erl @@ -153,7 +153,7 @@ handle_system_event({mnesia_down, Node}, State) -> end; handle_system_event({mnesia_overload, Details}, State) -> - report_warning("Mnesia is overloaded: ~p~n", [Details]), + report_warning("Mnesia is overloaded: ~w~n", [Details]), {ok, State}; handle_system_event({mnesia_info, Format, Args}, State) -> diff --git a/lib/mnesia/src/mnesia_locker.erl b/lib/mnesia/src/mnesia_locker.erl index a22c95d454..14011003d3 100644 --- a/lib/mnesia/src/mnesia_locker.erl +++ b/lib/mnesia/src/mnesia_locker.erl @@ -1177,9 +1177,9 @@ system_code_change(State, _Module, _OldVsn, _Extra) -> %% AXD301 patch sort pids according to R9B sort order %%%%%%%%%%%%%%%%%%%%%%%%%%% -%% Om R9B == true, g�rs j�mf�relsen som i R9B plain. -%% Om R9B == false, g�rs j�mf�relsen som i alla andra releaser. -%% cmp_tid(T1, T2) returnerar -1 om T1 < T2, 0 om T1 = T2 och 1 om T1 > T2. +%% Om R9B == true, the comparison is done as in R9B plain. +%% Om R9B == false, the comparison is done as in any other release. +%% cmp_tid(T1, T2) returns -1 if T1 < T2, 0 if T1 = T2 and 1 if T1 > T2. -define(VERSION_MAGIC, 131). -define(ATOM_EXT, 100). diff --git a/lib/mnesia/src/mnesia_recover.erl b/lib/mnesia/src/mnesia_recover.erl index 4750291a10..b64f428f15 100644 --- a/lib/mnesia/src/mnesia_recover.erl +++ b/lib/mnesia/src/mnesia_recover.erl @@ -45,7 +45,8 @@ note_log_decision/2, outcome/2, start/0, - start_garb/0, + next_garb/0, + next_check_overload/0, still_pending/1, sync_trans_tid_serial/1, sync/0, @@ -91,10 +92,38 @@ start() -> init() -> call(init). -start_garb() -> +next_garb() -> Pid = whereis(mnesia_recover), - {ok, _} = timer:send_interval(timer:minutes(2), Pid, garb_decisions), - {ok, _} = timer:send_interval(timer:seconds(10), Pid, check_overload). + erlang:send_after(timer:minutes(2), Pid, garb_decisions). + +next_check_overload() -> + Pid = whereis(mnesia_recover), + erlang:send_after(timer:seconds(10), Pid, check_overload). + + +do_check_overload(S) -> + %% Time to check if mnesia_tm is overloaded + case whereis(mnesia_tm) of + Pid when is_pid(Pid) -> + Threshold = 100, + Prev = S#state.tm_queue_len, + {message_queue_len, Len} = + process_info(Pid, message_queue_len), + if + Len > Threshold, Prev > Threshold -> + What = {mnesia_tm, message_queue_len, [Prev, Len]}, + mnesia_lib:report_system_event({mnesia_overload, What}), + mnesia_lib:overload_set(mnesia_tm, true), + S#state{tm_queue_len = 0}; + Len > Threshold -> + S#state{tm_queue_len = Len}; + true -> + mnesia_lib:overload_set(mnesia_tm, false), + S#state{tm_queue_len = 0} + end; + undefined -> + S + end. allow_garb() -> cast(allow_garb). @@ -853,34 +882,13 @@ handle_info({connect_nodes, Ns, From}, State) -> handle_call({connect_nodes,Ns},From,State); handle_info(check_overload, S) -> - %% Time to check if mnesia_tm is overloaded - case whereis(mnesia_tm) of - Pid when is_pid(Pid) -> - - Threshold = 100, - Prev = S#state.tm_queue_len, - {message_queue_len, Len} = - process_info(Pid, message_queue_len), - if - Len > Threshold, Prev > Threshold -> - What = {mnesia_tm, message_queue_len, [Prev, Len]}, - mnesia_lib:report_system_event({mnesia_overload, What}), - mnesia_lib:overload_set(mnesia_tm, true), - {noreply, S#state{tm_queue_len = 0}}; - - Len > Threshold -> - {noreply, S#state{tm_queue_len = Len}}; - - true -> - mnesia_lib:overload_set(mnesia_tm, false), - {noreply, S#state{tm_queue_len = 0}} - end; - undefined -> - {noreply, S} - end; + State2 = do_check_overload(S), + next_check_overload(), + {noreply, State2}; handle_info(garb_decisions, State) -> do_garb_decisions(), + next_garb(), {noreply, State}; handle_info({force_decision, Tid}, State) -> diff --git a/lib/mnesia/src/mnesia_tm.erl b/lib/mnesia/src/mnesia_tm.erl index 0af7f55c06..b5b14ac05b 100644 --- a/lib/mnesia/src/mnesia_tm.erl +++ b/lib/mnesia/src/mnesia_tm.erl @@ -103,7 +103,8 @@ init(Parent) -> end, mnesia_schema:purge_tmp_files(), - mnesia_recover:start_garb(), + mnesia_recover:next_garb(), + mnesia_recover:next_check_overload(), ?eval_debug_fun({?MODULE, init}, [{nodes, AllOthers}]), diff --git a/lib/observer/doc/src/notes.xml b/lib/observer/doc/src/notes.xml index ea665eee88..4ec03782a7 100644 --- a/lib/observer/doc/src/notes.xml +++ b/lib/observer/doc/src/notes.xml @@ -4,7 +4,7 @@ <chapter> <header> <copyright> - <year>2004</year><year>2011</year> + <year>2004</year><year>2012</year> <holder>Ericsson AB. All Rights Reserved.</holder> </copyright> <legalnotice> diff --git a/lib/observer/test/crashdump_viewer_SUITE.erl b/lib/observer/test/crashdump_viewer_SUITE.erl index 6f882d0be9..4c04126d4f 100644 --- a/lib/observer/test/crashdump_viewer_SUITE.erl +++ b/lib/observer/test/crashdump_viewer_SUITE.erl @@ -34,6 +34,7 @@ -define(default_timeout, ?t:minutes(30)). -define(sl_alloc_vsns,[r9b]). +-define(failed_file,"failed-cases.txt"). init_per_testcase(_Case, Config) -> DataDir = ?config(data_dir,Config), @@ -42,9 +43,18 @@ init_per_testcase(_Case, Config) -> catch crashdump_viewer:stop(), Dog = ?t:timetrap(?default_timeout), [{watchdog, Dog}|Config]. -end_per_testcase(_Case, Config) -> +end_per_testcase(Case, Config) -> Dog=?config(watchdog, Config), ?t:timetrap_cancel(Dog), + case ?config(tc_status,Config) of + ok -> + ok; + _Fail -> + File = filename:join(?config(data_dir,Config),?failed_file), + {ok,Fd}=file:open(File,[append]), + file:write(Fd,io_lib:format("~w.~n",[Case])), + file:close(Fd) + end, ok. suite() -> [{ct_hooks,[ts_install_cth]}]. @@ -67,15 +77,26 @@ init_per_suite(doc) -> ["Create a lot of crashdumps which can be used in the testcases below"]; init_per_suite(Config) when is_list(Config) -> Dog = ?t:timetrap(?default_timeout), + delete_saved(Config), application:start(inets), % will be using the http client later httpc:set_options([{ipfamily,inet6fb4}]), DataDir = ?config(data_dir,Config), - Rels = [R || R <- [r13b,r14b], ?t:is_release_available(R)] ++ [current], + Rels = [R || R <- [r14b,r15b], ?t:is_release_available(R)] ++ [current], io:format("Creating crash dumps for the following releases: ~p", [Rels]), AllDumps = create_dumps(DataDir,Rels), ?t:timetrap_cancel(Dog), [{dumps,AllDumps}|Config]. +delete_saved(Config) -> + DataDir = ?config(data_dir,Config), + file:delete(filename:join(DataDir,?failed_file)), + SaveDir = filename:join(DataDir,"save"), + Dumps = filelib:wildcard(filename:join(SaveDir,"*")), + lists:foreach(fun(F) -> file:delete(F) end, Dumps), + file:del_dir(SaveDir), + ok. + + translate(suite) -> []; translate(doc) -> @@ -196,6 +217,23 @@ end_per_suite(doc) -> ["Remove generated crashdumps"]; end_per_suite(Config) when is_list(Config) -> Dumps = ?config(dumps,Config), + DataDir = ?config(data_dir,Config), + FailedFile = filename:join(DataDir,?failed_file), + case filelib:is_file(FailedFile) of + true -> + SaveDir = filename:join(DataDir,"save"), + file:make_dir(SaveDir), + file:copy(FailedFile,filename:join(SaveDir,?failed_file)), + lists:foreach( + fun(CD) -> + File = filename:basename(CD), + New = filename:join(SaveDir,File), + file:copy(CD,New) + end, Dumps); + false -> + ok + end, + file:delete(FailedFile), lists:foreach(fun(CD) -> ok = file:delete(CD) end,Dumps), lists:keydelete(dumps,1,Config). @@ -388,7 +426,7 @@ special(Port,File) -> %% I registered a process as aaaaaaaa in the full_dist dumps %% to make sure it will be the first in the list when sorted - %% on names. There are some special data here, s� I'll thoroughly + %% on names. There are some special data here, so I'll thoroughly %% read the process details for this process. Other processes %% are just briefly traversed. {Pid,Rest1} = get_first_process(AllProcs), @@ -568,11 +606,14 @@ expand_link(Html) -> port_details(Port) -> - Port1 = contents(Port,"port?port=Port<0.1>"), - "#Port<0.1>" = title(Port1), - Port0 = contents(Port,"port?port=Port<0.0>"), - "Could not find port: #Port<0.0>" = title(Port0). + Port1 = contents(Port,"port?port=Port<0.1>"), + case title(Port0) of + "#Port<0.0>" -> % R16 or later + "Could not find port: #Port<0.1>" = title(Port1); + "Could not find port: #Port<0.0>" -> % R15 or earlier + "#Port<0.1>" = title(Port1) + end. is_truncated(File) -> case filename:extension(filename:rootname(File)) of @@ -691,6 +732,12 @@ dump_with_strange_module_name(DataDir,Rel,DumpName) -> CD. dump(Node,DataDir,Rel,DumpName) -> + case Rel of + _ when Rel<r15b, Rel=/=current -> + rpc:call(Node,os,putenv,["ERL_CRASH_DUMP_SECONDS","600"]); + _ -> + ok + end, rpc:call(Node,erlang,halt,[DumpName]), Crashdump0 = filename:join(filename:dirname(code:which(?t)), "erl_crash_dump.n1"), @@ -752,6 +799,7 @@ rel_opt(Rel) -> r12b -> [{erl,[{release,"r12b_patched"}]}]; r13b -> [{erl,[{release,"r13b_patched"}]}]; r14b -> [{erl,[{release,"r14b_latest"}]}]; %naming convention changed + r15b -> [{erl,[{release,"r15b_latest"}]}]; current -> [] end. @@ -764,7 +812,8 @@ dump_prefix(Rel) -> r12b -> "r12b_dump."; r13b -> "r13b_dump."; r14b -> "r14b_dump."; - current -> "r15b_dump." + r15b -> "r15b_dump."; + current -> "r16b_dump." end. compat_rel(Rel) -> @@ -776,5 +825,6 @@ compat_rel(Rel) -> r12b -> "+R12 "; r13b -> "+R13 "; r14b -> "+R14 "; + r15b -> "+R15 "; current -> "" end. diff --git a/lib/odbc/aclocal.m4 b/lib/odbc/aclocal.m4 index b1cf1fe404..9578cd35c4 100644 --- a/lib/odbc/aclocal.m4 +++ b/lib/odbc/aclocal.m4 @@ -740,11 +740,16 @@ dnl Try to find POSIX threads dnl The usual pthread lib... AC_CHECK_LIB(pthread, pthread_create, THR_LIBS="-lpthread") -dnl FreeBSD has pthreads in special c library, c_r... +dnl Very old versions of FreeBSD have pthreads in special c library, c_r... if test "x$THR_LIBS" = "x"; then AC_CHECK_LIB(c_r, pthread_create, THR_LIBS="-lc_r") fi +dnl QNX has pthreads in standard C library + if test "x$THR_LIBS" = "x"; then + AC_CHECK_FUNC(pthread_create, THR_LIBS="none_needed") + fi + dnl On ofs1 the '-pthread' switch should be used if test "x$THR_LIBS" = "x"; then AC_MSG_CHECKING([if the '-pthread' switch can be used]) @@ -765,6 +770,9 @@ dnl On ofs1 the '-pthread' switch should be used if test "x$THR_LIBS" != "x"; then THR_DEFS="$THR_DEFS -D_THREAD_SAFE -D_REENTRANT -DPOSIX_THREADS" THR_LIB_NAME=pthread + if test "x$THR_LIBS" = "xnone_needed"; then + THR_LIBS= + fi case $host_os in solaris*) THR_DEFS="$THR_DEFS -D_POSIX_PTHREAD_SEMANTICS" ;; diff --git a/lib/odbc/c_src/odbcserver.c b/lib/odbc/c_src/odbcserver.c index fe81d1dd3a..4a7a5224e5 100644 --- a/lib/odbc/c_src/odbcserver.c +++ b/lib/odbc/c_src/odbcserver.c @@ -153,7 +153,7 @@ static db_result_msg db_describe_table(byte *sql, db_state *state); /* ------------- Encode/decode functions -------- ------------------------*/ static db_result_msg encode_empty_message(void); -static db_result_msg encode_error_message(char *reason); +static db_result_msg encode_error_message(char *reason, char *errCode, SQLINTEGER nativeError); static db_result_msg encode_atom_message(char *atom); static db_result_msg encode_result(db_state *state); static db_result_msg encode_result_set(SQLSMALLINT num_of_columns, @@ -230,7 +230,7 @@ static void init_param_statement(int cols, static void map_dec_num_2_c_column(col_type *type, int precision, int scale); -static db_result_msg map_sql_2_c_column(db_column* column); +static db_result_msg map_sql_2_c_column(db_column* column, db_state *state); static param_array * bind_parameter_arrays(byte *buffer, int *index, @@ -246,7 +246,7 @@ static db_result_msg retrive_scrollable_cursor_support_info(db_state static int num_out_params(int num_of_params, param_array* params); /* ------------- Error handling functions --------------------------------*/ -static diagnos get_diagnos(SQLSMALLINT handleType, SQLHANDLE handle); +static diagnos get_diagnos(SQLSMALLINT handleType, SQLHANDLE handle, Boolean extendedErrors); /* ------------- Boolean functions ---------------------------------------*/ @@ -349,7 +349,7 @@ DWORD WINAPI database_handler(const char *port) byte *request_buffer = NULL; db_state state = {NULL, NULL, NULL, NULL, 0, {NULL, 0, 0}, - FALSE, FALSE, FALSE, FALSE, FALSE, FALSE}; + FALSE, FALSE, FALSE, FALSE, FALSE, FALSE, FALSE}; byte request_id; #ifdef WIN32 SOCKET socket; @@ -438,14 +438,16 @@ static db_result_msg db_connect(byte *args, db_state *state) diagnos diagnos; byte *connStrIn; int erl_auto_commit_mode, erl_trace_driver, - use_srollable_cursors, tuple_row_state, binary_strings; + use_srollable_cursors, tuple_row_state, binary_strings, + extended_errors; erl_auto_commit_mode = args[0]; erl_trace_driver = args[1]; use_srollable_cursors = args[2]; tuple_row_state = args[3]; binary_strings = args[4]; - connStrIn = args + 5 * sizeof(byte); + extended_errors = args[5]; + connStrIn = args + 6 * sizeof(byte); if(tuple_row_state == ON) { tuple_row(state) = TRUE; @@ -465,6 +467,12 @@ static db_result_msg db_connect(byte *args, db_state *state) use_srollable_cursors(state) = FALSE; } + if(extended_errors == ON) { + extended_errors(state) = TRUE; + } else { + extended_errors(state) = FALSE; + } + init_driver(erl_auto_commit_mode, erl_trace_driver, state); connlen = (SQLSMALLINT)strlen((const char*)connStrIn); @@ -475,10 +483,10 @@ static db_result_msg db_connect(byte *args, db_state *state) &stringlength2ptr, SQL_DRIVER_NOPROMPT); if (!sql_success(result)) { - diagnos = get_diagnos(SQL_HANDLE_DBC, connection_handle(state)); + diagnos = get_diagnos(SQL_HANDLE_DBC, connection_handle(state), extended_errors(state)); strcat((char *)diagnos.error_msg, " Connection to database failed."); - msg = encode_error_message(diagnos.error_msg); + msg = encode_error_message(diagnos.error_msg, extended_error(state, diagnos.sqlState), diagnos.nativeError ); if(!sql_success(SQLFreeHandle(SQL_HANDLE_DBC, connection_handle(state)))) @@ -509,8 +517,8 @@ static db_result_msg db_close_connection(db_state *state) result = SQLDisconnect(connection_handle(state)); if (!sql_success(result)) { - diagnos = get_diagnos(SQL_HANDLE_DBC, connection_handle(state)); - return encode_error_message(diagnos.error_msg); + diagnos = get_diagnos(SQL_HANDLE_DBC, connection_handle(state), extended_errors(state)); + return encode_error_message(diagnos.error_msg, extended_error(state, diagnos.sqlState), diagnos.nativeError); } if(!sql_success(SQLFreeHandle(SQL_HANDLE_DBC, @@ -536,8 +544,8 @@ static db_result_msg db_end_tran(byte compleationtype, db_state *state) (SQLSMALLINT)compleationtype); if (!sql_success(result)) { - diagnos = get_diagnos(SQL_HANDLE_DBC, connection_handle(state)); - return encode_error_message(diagnos.error_msg); + diagnos = get_diagnos(SQL_HANDLE_DBC, connection_handle(state), extended_errors(state)); + return encode_error_message(diagnos.error_msg, extended_error(state, diagnos.sqlState), diagnos.nativeError); } else { return encode_atom_message("ok"); } @@ -572,7 +580,7 @@ static db_result_msg db_query(byte *sql, db_state *state) /* SQL_SUCCESS_WITH_INFO at this point may indicate an error in user input. */ if (result != SQL_SUCCESS && result != SQL_NO_DATA_FOUND) { - diagnos = get_diagnos(SQL_HANDLE_STMT, statement_handle(state)); + diagnos = get_diagnos(SQL_HANDLE_STMT, statement_handle(state), extended_errors(state)); if(strcmp((char *)diagnos.sqlState, INFO) == 0) { is_error[0] = 0; strncat((char *)is_error, (char *)diagnos.error_msg, @@ -583,12 +591,12 @@ static db_result_msg db_query(byte *sql, db_state *state) it as we want a nice and clean Erlang API */ if((strcmp((char *)is_error, "error") == 0)) { - msg = encode_error_message((char *)diagnos.error_msg); + msg = encode_error_message((char *)diagnos.error_msg, extended_error(state, diagnos.sqlState), diagnos.nativeError); clean_state(state); return msg; } } else { - msg = encode_error_message((char *)diagnos.error_msg); + msg = encode_error_message((char *)diagnos.error_msg, extended_error(state, diagnos.sqlState), diagnos.nativeError); clean_state(state); return msg; } @@ -661,9 +669,9 @@ static db_result_msg db_select_count(byte *sql, db_state *state) } if(!sql_success(SQLExecDirect(statement_handle(state), sql, SQL_NTS))) { - diagnos = get_diagnos(SQL_HANDLE_STMT, statement_handle(state)); + diagnos = get_diagnos(SQL_HANDLE_STMT, statement_handle(state), extended_errors(state)); clean_state(state); - return encode_error_message(diagnos.error_msg); + return encode_error_message(diagnos.error_msg, extended_error(state, diagnos.sqlState), diagnos.nativeError); } if(!sql_success(SQLNumResultCols(statement_handle(state), @@ -803,13 +811,13 @@ static db_result_msg db_param_query(byte *buffer, db_state *state) result = SQLExecDirect(statement_handle(state), sql, SQL_NTS); if (!sql_success(result) || result == SQL_NO_DATA) { - diagnos = get_diagnos(SQL_HANDLE_STMT, statement_handle(state)); + diagnos = get_diagnos(SQL_HANDLE_STMT, statement_handle(state), extended_errors(state)); } /* SQL_NO_DATA and SQLSTATE 00000 indicate success for updates/deletes that affect no rows */ if(!sql_success(result) && !(result == SQL_NO_DATA && !strcmp((char *)diagnos.sqlState, INFO))) { - msg = encode_error_message(diagnos.error_msg); + msg = encode_error_message(diagnos.error_msg, extended_error(state, diagnos.sqlState), diagnos.nativeError); } else { for (i = 0; i < param_status.params_processed; i++) { switch (param_status.param_status_array[i]) { @@ -823,8 +831,8 @@ static db_result_msg db_param_query(byte *buffer, db_state *state) break; default: diagnos = - get_diagnos(SQL_HANDLE_STMT, statement_handle(state)); - msg = encode_error_message(diagnos.error_msg); + get_diagnos(SQL_HANDLE_STMT, statement_handle(state), extended_errors(state)); + msg = encode_error_message(diagnos.error_msg, extended_error(state, diagnos.sqlState), diagnos.nativeError); i = param_status.params_processed; break; } @@ -893,16 +901,16 @@ static db_result_msg db_describe_table(byte *sql, db_state *state) DO_EXIT(EXIT_ALLOC); if (!sql_success(SQLPrepare(statement_handle(state), sql, SQL_NTS))){ - diagnos = get_diagnos(SQL_HANDLE_STMT, statement_handle(state)); - msg = encode_error_message(diagnos.error_msg); + diagnos = get_diagnos(SQL_HANDLE_STMT, statement_handle(state), extended_errors(state)); + msg = encode_error_message(diagnos.error_msg, extended_error(state, diagnos.sqlState), diagnos.nativeError); clean_state(state); return msg; } if(!sql_success(SQLNumResultCols(statement_handle(state), &num_of_columns))) { - diagnos = get_diagnos(SQL_HANDLE_STMT, statement_handle(state)); - msg = encode_error_message(diagnos.error_msg); + diagnos = get_diagnos(SQL_HANDLE_STMT, statement_handle(state), extended_errors(state)); + msg = encode_error_message(diagnos.error_msg, extended_error(state, diagnos.sqlState), diagnos.nativeError); clean_state(state); return msg; } @@ -951,7 +959,7 @@ static db_result_msg encode_empty_message(void) } /* Description: Encode an error-message to send back to erlang*/ -static db_result_msg encode_error_message(char *reason) +static db_result_msg encode_error_message(char *reason, char *errCode, SQLINTEGER nativeError ) { int index; db_result_msg msg; @@ -960,6 +968,12 @@ static db_result_msg encode_error_message(char *reason) ei_encode_version(NULL, &index); ei_encode_tuple_header(NULL, &index, 2); ei_encode_atom(NULL, &index, "error"); + if (errCode) + { + ei_encode_tuple_header(NULL, &index, 3); + ei_encode_string(NULL, &index, errCode); + ei_encode_long(NULL, &index, nativeError); + } ei_encode_string(NULL, &index, reason); msg.length = index; @@ -970,6 +984,12 @@ static db_result_msg encode_error_message(char *reason) ei_encode_version((char *)msg.buffer, &index); ei_encode_tuple_header((char *)msg.buffer, &index, 2); ei_encode_atom((char *)msg.buffer, &index, "error"); + if (errCode) + { + ei_encode_tuple_header((char *)msg.buffer, &index, 3); + ei_encode_string((char *)msg.buffer, &index, errCode); + ei_encode_long((char *)msg.buffer, &index, nativeError); + } ei_encode_string((char *)msg.buffer, &index, reason); return msg; @@ -1013,8 +1033,8 @@ static db_result_msg encode_result(db_state *state) if(!sql_success(SQLNumResultCols(statement_handle(state), &num_of_columns))) { - diagnos = get_diagnos(SQL_HANDLE_STMT, statement_handle(state)); - msg = encode_error_message(diagnos.error_msg); + diagnos = get_diagnos(SQL_HANDLE_STMT, statement_handle(state), extended_errors(state)); + msg = encode_error_message(diagnos.error_msg, extended_error(state, diagnos.sqlState), diagnos.nativeError); clean_state(state); return msg; } @@ -1030,8 +1050,8 @@ static db_result_msg encode_result(db_state *state) } if(!sql_success(SQLRowCount(statement_handle(state), &RowCountPtr))) { - diagnos = get_diagnos(SQL_HANDLE_STMT, statement_handle(state)); - msg = encode_error_message(diagnos.error_msg); + diagnos = get_diagnos(SQL_HANDLE_STMT, statement_handle(state), extended_errors(state)); + msg = encode_error_message(diagnos.error_msg, extended_error(state, diagnos.sqlState), diagnos.nativeError); clean_state(state); return msg; } @@ -1239,7 +1259,7 @@ static db_result_msg encode_column_name_list(SQLSMALLINT num_of_columns, (columns(state)[i]).type.sql = sql_type; (columns(state)[i]).type.col_size = size; - msg = map_sql_2_c_column(&columns(state)[i]); + msg = map_sql_2_c_column(&columns(state)[i], state); if (msg.length > 0) { return msg; /* An error has occurred */ } else { @@ -2346,7 +2366,7 @@ static void map_dec_num_2_c_column(col_type *type, int precision, int scale) /* Description: Transform SQL columntype to C columntype. Returns a dummy db_result_msg with length 0 on success and an errormessage otherwise.*/ -static db_result_msg map_sql_2_c_column(db_column* column) +static db_result_msg map_sql_2_c_column(db_column* column, db_state *state) { db_result_msg msg; @@ -2415,10 +2435,10 @@ static db_result_msg map_sql_2_c_column(db_column* column) column -> type.strlen_or_indptr = (SQLLEN)NULL; break; case SQL_UNKNOWN_TYPE: - msg = encode_error_message("Unknown column type"); + msg = encode_error_message("Unknown column type", extended_error(state, ""), 0); break; default: - msg = encode_error_message("Column type not supported"); + msg = encode_error_message("Column type not supported", extended_error(state, ""), 0); break; } return msg; @@ -2527,7 +2547,7 @@ static db_column retrive_binary_data(db_column column, int column_nr, while (result == SQL_SUCCESS_WITH_INFO) { - diagnos = get_diagnos(SQL_HANDLE_STMT, statement_handle(state)); + diagnos = get_diagnos(SQL_HANDLE_STMT, statement_handle(state), extended_errors(state)); if(strcmp((char *)diagnos.sqlState, TRUNCATED) == 0) { outputlen = column.type.len - 1; @@ -2614,10 +2634,10 @@ static db_result_msg more_result_sets(db_state *state) /* As we found an error we do not care about any potential more result sets */ exists_more_result_sets(state) = FALSE; - diagnos = get_diagnos(SQL_HANDLE_STMT, statement_handle(state)); + diagnos = get_diagnos(SQL_HANDLE_STMT, statement_handle(state), extended_errors(state)); strcat((char *)diagnos.error_msg, "Failed to create on of the result sets"); - msg = encode_error_message(diagnos.error_msg); + msg = encode_error_message(diagnos.error_msg, extended_error(state, diagnos.sqlState), diagnos.nativeError); return msg; } } @@ -2634,7 +2654,7 @@ static Boolean sql_success(SQLRETURN result) diagnostic records scaning for error messages and the sqlstate. If this function is called when no error has ocurred only the sqlState field may be referenced.*/ -static diagnos get_diagnos(SQLSMALLINT handleType, SQLHANDLE handle) +static diagnos get_diagnos(SQLSMALLINT handleType, SQLHANDLE handle, Boolean extendedErrors) { diagnos diagnos; SQLINTEGER nativeError; @@ -2658,17 +2678,16 @@ static diagnos get_diagnos(SQLSMALLINT handleType, SQLHANDLE handle) result = SQLGetDiagRec(handleType, handle, record_nr, current_sql_state, &nativeError, current_errmsg_pos, (SQLSMALLINT)errmsg_buffer_size, &errmsg_size); - if(result != SQL_SUCCESS && result != SQL_NO_DATA) { - - - break; - } else { + if(result == SQL_SUCCESS) { /* update the sqlstate in the diagnos record, because the SQLGetDiagRec call succeeded */ memcpy(diagnos.sqlState, current_sql_state, SQL_STATE_SIZE); + diagnos.nativeError = nativeError; errmsg_buffer_size = errmsg_buffer_size - errmsg_size; acc_errmsg_size = acc_errmsg_size + errmsg_size; current_errmsg_pos = current_errmsg_pos + errmsg_size; + } else { + break; } } @@ -2676,7 +2695,7 @@ static diagnos get_diagnos(SQLSMALLINT handleType, SQLHANDLE handle) strcat((char *)diagnos.error_msg, "No SQL-driver information available."); } - else { + else if (!extendedErrors){ strcat(strcat((char *)diagnos.error_msg, " SQLSTATE IS: "), (char *)diagnos.sqlState); } diff --git a/lib/odbc/c_src/odbcserver.h b/lib/odbc/c_src/odbcserver.h index a76cedf1af..71760189e7 100644 --- a/lib/odbc/c_src/odbcserver.h +++ b/lib/odbc/c_src/odbcserver.h @@ -145,6 +145,7 @@ typedef struct { typedef struct { SQLCHAR sqlState[SQL_STATE_SIZE]; + SQLINTEGER nativeError; byte error_msg[MAX_ERR_MSG]; } diagnos; @@ -179,6 +180,7 @@ typedef struct { Boolean exists_more_result_sets; Boolean param_query; Boolean out_params; + Boolean extended_errors; } db_state; typedef enum { @@ -198,3 +200,5 @@ typedef enum { #define exists_more_result_sets(db_state) (db_state -> exists_more_result_sets) #define param_query(db_state) (db_state -> param_query) #define out_params(db_state) (db_state -> out_params) +#define extended_errors(db_state) (db_state -> extended_errors) +#define extended_error(db_state, errorcode) ( extended_errors(state) ? errorcode : NULL ) diff --git a/lib/odbc/doc/src/notes.xml b/lib/odbc/doc/src/notes.xml index 5f6cf91961..7ba0307a45 100644 --- a/lib/odbc/doc/src/notes.xml +++ b/lib/odbc/doc/src/notes.xml @@ -4,7 +4,7 @@ <chapter> <header> <copyright> - <year>2004</year><year>2011</year> + <year>2004</year><year>2012</year> <holder>Ericsson AB. All Rights Reserved.</holder> </copyright> <legalnotice> diff --git a/lib/odbc/doc/src/odbc.xml b/lib/odbc/doc/src/odbc.xml index 8a58dc2848..a984bf4485 100644 --- a/lib/odbc/doc/src/odbc.xml +++ b/lib/odbc/doc/src/odbc.xml @@ -4,7 +4,7 @@ <erlref> <header> <copyright> - <year>1999</year><year>2011</year> + <year>1999</year><year>2013</year> <holder>Ericsson AB. All Rights Reserved.</holder> </copyright> <legalnotice> @@ -74,8 +74,12 @@ <code type="none"> milliseconds() = integer() >= 0 </code> <code type="none"> - common_reason() = connection_closed | term() - some kind of - explanation of what went wrong </code> + common_reason() = connection_closed | extended_error() | term() - some kind of + explanation of what went wrong </code> + <code type="none"> + extended_error() = {string(), integer(), Reason} - extended error type with ODBC + and native database error codes, as well as the base reason that would have been + returned had extended_errors not been enabled. </code> <code type="none"> string() = list of ASCII characters </code> <code type="none"> @@ -101,8 +105,14 @@ odbc_data_type() = sql_integer | sql_smallint | sql_tinyint | {sql_decimal, precision(), scale()} | {sql_numeric, precision(), scale()} | - {sql_char, size()} | {sql_wchar, size()} | {sql_varchar, size()} | {sql_wvarchar, size()}| {sql_float, precision()} | - {sql_wlongvarchar, size()} | {sql_float, precision()} | sql_real | sql_double | sql_bit | atom() + {sql_char, size()} | + {sql_wchar, size()} | + {sql_varchar, size()} | + {sql_wvarchar, size()}| + {sql_float, precision()} | + {sql_wlongvarchar, size()} | + {sql_float, precision()} | + sql_real | sql_double | sql_bit | atom() </code> <code type="none"> precision() = integer() </code> @@ -143,7 +153,7 @@ <d>All options has default values. </d> <v>option() = {auto_commit, on | off} | {timeout, milliseconds()} | {binary_strings, on | off} | {tuple_row, on | off} | {scrollable_cursors, on | off} | - {trace_driver, on | off} </v> + {trace_driver, on | off} | {extended_errors, on | off} </v> <v>Ref = connection_reference() - should be used to access the connection. </v> <v>Reason = port_program_executable_not_found | common_reason()</v> </type> @@ -196,6 +206,19 @@ <p>For more information about the <c>ConnectStr</c> see description of the function SQLDriverConnect in [1].</p> </note> + + <p>The <c>extended_errors</c> option enables extended ODBC error + information when an operation fails. Rather than returning <c>{error, Reason}</c>, + the failing function will reutrn <c>{error, {ODBCErrorCode, NativeErrorCode, Reason}}</c>. + Note that this information is probably of little use when writing database-independent code, + but can be of assistance in providing more sophisticated error handling when dealing with + a known underlying database. + <list type="bulleted"> + <item><c>ODBCErrorCode</c> is the ODBC error string returned by the ODBC driver.</item> + <item><c>NativeErrorCode</c> is the numberic error code returned by the underlying database. The possible values + and their meanings are dependent on the database being used.</item> + <item><c>Reason</c> is as per the <c>Reason</c> field when extended errors are not enabled.</item> + </list></p> </desc> </func> <func> @@ -203,7 +226,7 @@ <fsummary>Closes a connection to a database. </fsummary> <type> <v>Ref = connection_reference()</v> - <v>Reason = process_not_owner_of_odbc_connection</v> + <v>Reason = process_not_owner_of_odbc_connection | extended_error()</v> </type> <desc> <p>Closes a connection to a database. This will also diff --git a/lib/odbc/src/odbc.erl b/lib/odbc/src/odbc.erl index 16fdb4aabd..3eabec9ec3 100644 --- a/lib/odbc/src/odbc.erl +++ b/lib/odbc/src/odbc.erl @@ -810,10 +810,11 @@ connect(ConnectionReferense, ConnectionStr, Options) -> {C_TupleRow, _} = connection_config(tuple_row, Options), {BinaryStrings, _} = connection_config(binary_strings, Options), + {ExtendedErrors, _} = connection_config(extended_errors, Options), ODBCCmd = [?OPEN_CONNECTION, C_AutoCommitMode, C_TraceDriver, - C_SrollableCursors, C_TupleRow, BinaryStrings, ConnectionStr], + C_SrollableCursors, C_TupleRow, BinaryStrings, ExtendedErrors, ConnectionStr], %% Send request, to open a database connection, to the control process. case call(ConnectionReferense, @@ -860,6 +861,8 @@ connection_default(trace_driver) -> connection_default(scrollable_cursors) -> {?ON, on}; connection_default(binary_strings) -> + {?OFF, off}; +connection_default(extended_errors) -> {?OFF, off}. %%------------------------------------------------------------------------- diff --git a/lib/odbc/test/odbc_connect_SUITE.erl b/lib/odbc/test/odbc_connect_SUITE.erl index a076c4dfff..7d732f20f7 100644 --- a/lib/odbc/test/odbc_connect_SUITE.erl +++ b/lib/odbc/test/odbc_connect_SUITE.erl @@ -51,7 +51,7 @@ all() -> {group, client_dies}, connect_timeout, timeout, many_timeouts, timeout_reset, disconnect_on_timeout, connection_closed, disable_scrollable_cursors, - return_rows_as_lists, api_missuse]; + return_rows_as_lists, api_missuse, extended_errors]; Other -> {skip, Other} end. @@ -838,3 +838,33 @@ transaction_support_str(mysql) -> "ENGINE = InnoDB"; transaction_support_str(_) -> "". + + +%%------------------------------------------------------------------------- +extended_errors(doc)-> + ["Test the extended errors connection option: When off; the old behaviour of just an error " + "string is returned on error. When on, the error string is replaced by a 3 element tuple " + "that also exposes underlying ODBC provider error codes."]; +extended_errors(suite) -> []; +extended_errors(Config) when is_list(Config)-> + Table = ?config(tableName, Config), + {ok, Ref} = odbc:connect(?RDBMS:connection_string(), odbc_test_lib:platform_options()), + {updated, _} = odbc:sql_query(Ref, "create table " ++ Table ++" ( id integer, data varchar(10))"), + + % Error case WITHOUT extended errors on... + case odbc:sql_query(Ref, "create table " ++ Table ++" ( id integer, data varchar(10))") of + {error, ErrorString} when is_list(ErrorString) -> ok + end, + + % Now the test case with extended errors on - This should return a tuple, not a list/string now. + % The first element is a string that is the ODBC error string; the 2nd element is a native integer error + % code passed from the underlying provider driver. The last is the familiar old error string. + % We can't check the actual error code; as each different underlying provider will return + % a different value - So we just check the return types at least. + {ok, RefExtended} = odbc:connect(?RDBMS:connection_string(), odbc_test_lib:platform_options() ++ [{extended_errors, on}]), + case odbc:sql_query(RefExtended, "create table " ++ Table ++" ( id integer, data varchar(10))") of + {error, {ODBCCodeString, NativeCodeNum, ShortErrorString}} when is_list(ODBCCodeString), is_number(NativeCodeNum), is_list(ShortErrorString) -> ok + end, + + ok = odbc:disconnect(Ref), + ok = odbc:disconnect(RefExtended). diff --git a/lib/odbc/test/odbc_data_type_SUITE.erl b/lib/odbc/test/odbc_data_type_SUITE.erl index d61a91f973..2d33546622 100644 --- a/lib/odbc/test/odbc_data_type_SUITE.erl +++ b/lib/odbc/test/odbc_data_type_SUITE.erl @@ -1,7 +1,8 @@ +%% -*- coding: utf-8 -*- %% %% %CopyrightBegin% %% -%% Copyright Ericsson AB 2002-2011. All Rights Reserved. +%% Copyright Ericsson AB 2002-2012. All Rights Reserved. %% %% The contents of this file are subject to the Erlang Public License, %% Version 1.1, (the "License"); you may not use this file except in @@ -1469,7 +1470,7 @@ utf8(Config) when is_list(Config) -> odbc:sql_query(Ref, "CREATE TABLE " ++ Table ++ "(FIELD text)"), - Latin1Data = ["���������", + Latin1Data = ["ÖÄÅÄÖÅäöå", "testasdf", "Row 3", "Row 4", @@ -1564,7 +1565,7 @@ timestamp(Config) when is_list(Config) -> %%------------------------------------------------------------------------ w_char_support(Ref, Table, CharType, Size) -> - Latin1Data = ["���������", + Latin1Data = ["ÖÄÅÄÖÅäöå", "testasdf", "Row 3", "Row 4", diff --git a/lib/odbc/vsn.mk b/lib/odbc/vsn.mk index 3bb2fe5bce..585b92b2d2 100644 --- a/lib/odbc/vsn.mk +++ b/lib/odbc/vsn.mk @@ -1 +1 @@ -ODBC_VSN = 2.10.13 +ODBC_VSN = 2.10.14 diff --git a/lib/orber/doc/src/ch_idl_to_erlang_mapping.xml b/lib/orber/doc/src/ch_idl_to_erlang_mapping.xml index 964ae3e92d..1fd2f644cb 100644 --- a/lib/orber/doc/src/ch_idl_to_erlang_mapping.xml +++ b/lib/orber/doc/src/ch_idl_to_erlang_mapping.xml @@ -4,7 +4,7 @@ <chapter> <header> <copyright> - <year>1997</year><year>2010</year> + <year>1997</year><year>2013</year> <holder>Ericsson AB. All Rights Reserved.</holder> </copyright> <legalnotice> @@ -1142,7 +1142,8 @@ handle_info(_Info, State) -> module, and <c>lookup</c> in <c>DB_Administrator_impl.erl</c><em>and</em><c>DB_CommonUser_impl.erl</c>. But wait, is that really necessary? Actually, it is not. We simple use the IC compile option <em>impl</em>:</p> <pre> -$ <input>erlc +'{{impl, "DB::CommonUser"}, "DBUser_impl"}' +'{{impl, "DB::Administrator"}, "DBUser_impl"}' DB.idl</input> +$ <input>erlc +'{{impl, "DB::CommonUser"}, "DBUser_impl"}'\ + +'{{impl, "DB::Administrator"}, "DBUser_impl"}' DB.idl</input> $ <input>erlc *.erl</input> </pre> <p>Instead of creating, and not the least, maintaining two call-back modules, diff --git a/lib/orber/src/Makefile b/lib/orber/src/Makefile index 72610def2b..6eb659407d 100644 --- a/lib/orber/src/Makefile +++ b/lib/orber/src/Makefile @@ -200,8 +200,7 @@ ERL_COMPILE_FLAGS += $(ERL_IDL_FLAGS) \ +'{attribute,insert,app_vsn,"orber_$(ORBER_VSN)"}' \ -D'ORBVSN="$(ORBER_VSN)"' -ASN_FLAGS = -bber_bin +der +compact_bit_string +optimize \ - +nowarn_unused_record +ASN_FLAGS = -bber +der +compact_bit_string +nowarn_unused_record # ---------------------------------------------------- # Targets diff --git a/lib/orber/src/orber_iiop_net.erl b/lib/orber/src/orber_iiop_net.erl index 2eed0b538a..33e02e3c04 100644 --- a/lib/orber/src/orber_iiop_net.erl +++ b/lib/orber/src/orber_iiop_net.erl @@ -2,7 +2,7 @@ %% %% %CopyrightBegin% %% -%% Copyright Ericsson AB 1997-2011. All Rights Reserved. +%% Copyright Ericsson AB 1997-2012. All Rights Reserved. %% %% The contents of this file are subject to the Erlang Public License, %% Version 1.1, (the "License"); you may not use this file except in @@ -415,11 +415,10 @@ handle_info({'EXIT', Pid, _Reason}, State) when is_pid(Pid) -> ref = Ref, options = Options, proxy_options = POpts}] -> ets:delete(?CONNECTION_DB, Pid), unlink(Pid), - NewListen = new_listen_socket(Type, Listen, Port, Options), - {ok, NewPid} = orber_iiop_socketsup:start_accept(Type, NewListen, + {ok, NewPid} = orber_iiop_socketsup:start_accept(Type, Listen, Ref, POpts), link(NewPid), - ets:insert(?CONNECTION_DB, #listen{pid = NewPid, socket = NewListen, + ets:insert(?CONNECTION_DB, #listen{pid = NewPid, socket = Listen, port = Port, type = Type, ref = Ref, options = Options, proxy_options = POpts}), @@ -445,23 +444,6 @@ handle_info({'EXIT', Pid, _Reason}, State) when is_pid(Pid) -> handle_info(_, State) -> {noreply, State}. -new_listen_socket(normal, ListenFd, _Port, _Options) -> - ListenFd; -new_listen_socket(ssl, ListenFd, Port, Options) -> - Generation = orber_env:ssl_generation(), - if - Generation > 2 -> - ListenFd; - true -> - case is_process_alive(ssl:pid(ListenFd)) of - true -> - ListenFd; - _ -> - {ok, Listen, _NP} = orber_socket:listen(ssl, Port, Options, true), - Listen - end - end. - from_list(List) -> from_list(List, queue:new()). diff --git a/lib/os_mon/src/os_mon.erl b/lib/os_mon/src/os_mon.erl index df1eccb064..19acae9f0e 100644 --- a/lib/os_mon/src/os_mon.erl +++ b/lib/os_mon/src/os_mon.erl @@ -176,9 +176,7 @@ services({unix, sunos}) -> services({unix, _}) -> % Other unix. [cpu_sup, disksup, memsup]; services({win32, _}) -> - [disksup, memsup, os_sup, sysinfo]; -services(_) -> - []. + [disksup, memsup, os_sup, sysinfo]. server_name(cpu_sup) -> cpu_sup; server_name(disksup) -> disksup; diff --git a/lib/parsetools/include/yeccpre.hrl b/lib/parsetools/include/yeccpre.hrl index 3672394fc5..e4c3ba52be 100644 --- a/lib/parsetools/include/yeccpre.hrl +++ b/lib/parsetools/include/yeccpre.hrl @@ -1,7 +1,7 @@ %% %% %CopyrightBegin% %% -%% Copyright Ericsson AB 1996-2011. All Rights Reserved. +%% Copyright Ericsson AB 1996-2012. All Rights Reserved. %% %% The contents of this file are subject to the Erlang Public License, %% Version 1.1, (the "License"); you may not use this file except in @@ -36,7 +36,7 @@ parse_and_scan({M, F, A}) -> -spec format_error(any()) -> [char() | list()]. format_error(Message) -> - case io_lib:deep_char_list(Message) of + case io_lib:deep_unicode_char_list(Message) of true -> Message; _ -> @@ -164,7 +164,7 @@ yecctoken_location(Token) -> yecctoken2string({atom, _, A}) -> io_lib:write(A); yecctoken2string({integer,_,N}) -> io_lib:write(N); yecctoken2string({float,_,F}) -> io_lib:write(F); -yecctoken2string({char,_,C}) -> io_lib:write_char(C); +yecctoken2string({char,_,C}) -> io_lib:write_unicode_char(C); yecctoken2string({var,_,V}) -> io_lib:format("~s", [V]); yecctoken2string({string,_,S}) -> io_lib:write_unicode_string(S); yecctoken2string({reserved_symbol, _, A}) -> io_lib:write(A); diff --git a/lib/parsetools/src/esyntax.yrl b/lib/parsetools/src/esyntax.yrl deleted file mode 100644 index 1ecb54f0a7..0000000000 --- a/lib/parsetools/src/esyntax.yrl +++ /dev/null @@ -1,360 +0,0 @@ -%% -%% %CopyrightBegin% -%% -%% Copyright Ericsson AB 1996-2009. All Rights Reserved. -%% -%% The contents of this file are subject to the Erlang Public License, -%% Version 1.1, (the "License"); you may not use this file except in -%% compliance with the License. You should have received a copy of the -%% Erlang Public License along with this software. If not, it can be -%% retrieved online at http://www.erlang.org/. -%% -%% Software distributed under the License is distributed on an "AS IS" -%% basis, WITHOUT WARRANTY OF ANY KIND, either express or implied. See -%% the License for the specific language governing rights and limitations -%% under the License. -%% -%% %CopyrightEnd% -%% -Nonterminals -add_op attribute basic_type bif_test -case_expr clause_body -clause_guard clause_head comp_op cr_clause cr_clauses expr expr_tail -exprs farity farity_list form formal_parameter_list function -function_call function_clause guard guard_call guard_expr -guard_expr_list guard_exprs guard_expr_tail guard_expr_tuple -guard_parameter_list -guard_tests guard_test if_clause if_clauses if_expr list match_expr -mult_op parameter_list pattern patterns pattern_list pattern_tail pattern_tuple -prefix_op receive_expr send_expr tuple. - -Terminals -'!' '(' ')' '*' '+' ',' '-' '->' '/' '/=' ':' ';' '<' '=' '=/=' '=:=' -'=<' '==' '>' '>=' '[' ']' 'after' 'band' 'begin' 'bnot' -'bor' 'bsl' 'bsr' 'bxor' 'case' 'catch' 'div' 'end' 'if' 'of' -'receive' 'rem' 'when' '{' '|' '}' atom float integer string var. -% 'receive' 'rem' 'true' 'when' '{' '|' '}' atom float integer string var. - -Rootsymbol form. - -Endsymbol dot. - -Unary 0 'catch'. -Right 200 '='. -Right 200 '!'. -Left 300 add_op. -Left 400 mult_op. -Unary 500 prefix_op. - - -add_op -> '+' : '$1'. -add_op -> '-' : '$1'. -add_op -> 'bor' : '$1'. -add_op -> 'bxor' : '$1'. -add_op -> 'bsl' : '$1'. -add_op -> 'bsr' : '$1'. - -comp_op -> '==' : '$1'. -comp_op -> '/=' : '$1'. -comp_op -> '=<' : '$1'. -comp_op -> '<' : '$1'. -comp_op -> '>=' : '$1'. -comp_op -> '>' : '$1'. -comp_op -> '=:=' : '$1'. -comp_op -> '=/=' : '$1'. - -mult_op -> '*' : '$1'. -mult_op -> '/' : '$1'. -mult_op -> 'div' : '$1'. -mult_op -> 'rem' : '$1'. -mult_op -> 'band' : '$1'. - -prefix_op -> '+' : '$1'. -prefix_op -> '-' : '$1'. -prefix_op -> 'bnot' : '$1'. - - -basic_type -> atom : '$1'. -basic_type -> float : '$1'. -basic_type -> integer : '$1'. -basic_type -> string : '$1'. -basic_type -> var : '$1'. -% basic_type -> 'true' : {atom, element(2, '$1'), 'true'}. - - -pattern -> basic_type : '$1'. -pattern -> pattern_list : '$1'. -pattern -> pattern_tuple : '$1'. - -pattern_list -> '[' ']' : {nil, element(2, '$1')}. -pattern_list -> '[' pattern pattern_tail ']' : - {cons, element(2, '$1'), '$2', '$3'}. - -pattern_tail -> '|' pattern : '$2'. -pattern_tail -> ',' pattern pattern_tail : - {cons, element(2, '$2'), '$2', '$3'}. -pattern_tail -> '$empty' : {nil, 0}. - -pattern_tuple -> '{' '}' : {tuple, element(2, '$1'), []}. -pattern_tuple -> '{' patterns '}' : {tuple, element(2, '$1'), '$2'}. - -patterns -> pattern : ['$1']. -patterns -> pattern ',' patterns : ['$1' | '$3']. - - -expr -> basic_type : '$1'. -expr -> list : '$1'. -expr -> tuple : '$1'. -expr -> function_call : '$1'. - -expr -> expr add_op expr : - {Op, Pos} = '$2', - {arith, Pos, Op, '$1', '$3'}. -expr -> expr mult_op expr : - {Op, Pos} = '$2', - {arith, Pos, Op, '$1', '$3'}. -expr -> prefix_op expr: - case '$2' of - {float, Pos, N} -> - case '$1' of - {'-', _} -> - {float, Pos, -N}; - {'+', _} -> - {float, Pos, N}; - {Op, Pos1} -> - {arith, Pos1, Op, {float, Pos, N}} - end; - {integer, Pos, N} -> - case '$1' of - {'-', _} -> - {integer, Pos, -N}; - {'+', _} -> - {integer, Pos, N}; - {Op, Pos1} -> - {arith, Pos1, Op, {integer, Pos, N}} - end; - _ -> - {Op, Pos} = '$1', - {arith, Pos, Op, '$2'} - end. - -expr -> '(' expr ')' : '$2'. -expr -> 'begin' exprs 'end' : {block, element(2, '$1'), '$2'}. -expr -> 'catch' expr : {'catch', element(2, '$1'), '$2'}. - -expr -> case_expr : '$1'. -expr -> if_expr : '$1'. -expr -> receive_expr : '$1'. -expr -> match_expr : '$1'. -expr -> send_expr : '$1'. - - -list -> '[' ']' : {nil, element(2, '$1')}. -list -> '[' expr expr_tail ']' : {cons, element(2, '$1'), '$2', '$3'}. - -expr_tail -> '|' expr : '$2'. -expr_tail -> ',' expr expr_tail : {cons, element(2, '$2'), '$2', '$3'}. -expr_tail -> '$empty' : {nil, 0}. - -tuple -> '{' '}' : {tuple, element(2, '$1'), []}. -tuple -> '{' exprs '}' : {tuple, element(2, '$1'), '$2'}. - - -function_call -> atom '(' parameter_list ')' : - case erl_parse:erlang_bif(element(3, '$1'), length('$3')) of - true -> - {bif, element(2, '$1'), element(3, '$1'), '$3'}; - false -> - {call, element(2, '$1'), [], element(3, '$1'), '$3'} - end. -function_call -> atom ':' atom '(' parameter_list ')' : - {call, element(2, '$1'), element(3, '$1'), element(3, '$3'), '$5'}. - -parameter_list -> exprs : '$1'. -parameter_list -> '$empty' : []. - - -case_expr -> 'case' expr 'of' cr_clauses 'end' : - {'case', element(2, '$1'), '$2', '$4'}. - -cr_clause -> pattern clause_guard clause_body : - {clause, element(2, '$1'), ['$1'], '$2', '$3'}. - -cr_clauses -> cr_clause : ['$1']. -cr_clauses -> cr_clause ';' cr_clauses : ['$1' | '$3']. - -if_expr -> 'if' if_clauses 'end' : {'if', element(2, '$1'), '$2'}. - -if_clause -> guard clause_body : {clause, element(2, hd('$2')), '$1', '$2'}. - -if_clauses -> if_clause : ['$1']. -if_clauses -> if_clause ';' if_clauses : ['$1' | '$3']. - -receive_expr -> 'receive' 'after' expr clause_body 'end' : - {'receive', element(2, '$1'), [], '$3', '$4'}. -receive_expr -> 'receive' cr_clauses 'end' : - {'receive', element(2, '$1'), '$2'}. -receive_expr -> 'receive' cr_clauses 'after' expr clause_body 'end' : - {'receive', element(2, '$1'), '$2', '$4', '$5'}. - - -match_expr -> expr '=' expr : - case erl_parse:is_term('$1') of - true -> - {match, element(2, '$1'), '$1', '$3'}; - false -> - throw({error, {element(2, '$1'), yecc, "illegal lhs in match **"}}) - end. - -send_expr -> expr '!' expr : - Pos = element(2, '$1'), - {send, Pos, '$1', '$3'}. - - -exprs -> expr : ['$1']. -exprs -> expr ',' exprs : ['$1' | '$3']. - - -guard_expr -> basic_type : '$1'. -guard_expr -> guard_expr_list : '$1'. -guard_expr -> guard_expr_tuple : '$1'. -guard_expr -> guard_call : '$1'. -guard_expr -> '(' guard_expr ')' : '$2'. -guard_expr -> guard_expr add_op guard_expr : - {Op, Pos} = '$2', - {arith, Pos, Op, '$1', '$3'}. -guard_expr -> guard_expr mult_op guard_expr : - {Op, Pos} = '$2', - {arith, Pos, Op, '$1', '$3'}. -guard_expr -> prefix_op guard_expr: - case '$2' of - {float, Pos, N} -> - case '$1' of - {'-', _} -> - {float, Pos, -N}; - {'+', _} -> - {float, Pos, N}; - {Op, Pos1} -> - {arith, Pos1, Op, {float, Pos, N}} - end; - {integer, Pos, N} -> - case '$1' of - {'-', _} -> - {integer, Pos, -N}; - {'+', _} -> - {integer, Pos, N}; - {Op, Pos1} -> - {arith, Pos1, Op, {integer, Pos, N}} - end; - _ -> - {Op, Pos} = '$1', - {arith, Pos, Op, '$2'} - end. - -guard_expr_list -> '[' ']' : {nil, element(2, '$1')}. -guard_expr_list -> '[' guard_expr guard_expr_tail ']' : - {cons, element(2, '$1'), '$2', '$3'}. - -guard_expr_tail -> '|' guard_expr : '$2'. -guard_expr_tail -> ',' guard_expr guard_expr_tail : - {cons, element(2, '$2'), '$2', '$3'}. -guard_expr_tail -> '$empty' : {nil, 0}. - -guard_expr_tuple -> '{' '}' : {tuple, element(2, '$1'), []}. -guard_expr_tuple -> '{' guard_exprs '}' : {tuple, element(2, '$1'), '$2'}. - -guard_exprs -> guard_expr : ['$1']. -guard_exprs -> guard_expr ',' guard_exprs : ['$1' | '$3']. - - -guard_call -> atom '(' guard_parameter_list ')' : - case erl_parse:erlang_guard_bif(element(3, '$1'), length('$3')) of - true -> - {bif, element(2, '$1'), element(3, '$1'), '$3'}; - false -> - throw({error, {element(2, '$1'), yecc, "illegal test in guard **"}}) - end. - -guard_parameter_list -> guard_exprs : '$1'. -guard_parameter_list -> '$empty' : []. - - -bif_test -> atom '(' guard_parameter_list ')' : - case erl_parse:erlang_guard_test(element(3, '$1'), length('$3')) of - true -> - {test, element(2, '$1'), element(3, '$1'), '$3'}; - false -> - throw({error, {element(2, '$1'), yecc, "illegal test in guard **"}}) - end. - - -guard_test -> bif_test : '$1'. -guard_test -> guard_expr comp_op guard_expr : - {Op, Pos} = '$2', - {comp, Pos, Op, '$1', '$3'}. - -guard_tests -> guard_test : ['$1']. -guard_tests -> guard_test ',' guard_tests : ['$1' | '$3']. - -% guard -> 'true' : []. -guard -> atom : - case '$1' of - {atom, _, true} -> - []; - _ -> - throw({error, {element(2, '$1'), yecc, "illegal test in guard **"}}) - end. -guard -> guard_tests : '$1'. - - -function_clause -> clause_head clause_guard clause_body : - {Name, Line, Arity, Parameters} = '$1', - {function, Line, Name, Arity, - [{clause, element(2, hd('$3')), Parameters, '$2', '$3'}]}. - -clause_head -> atom '(' formal_parameter_list ')' : - {element(3, '$1'), element(2, '$1'), length('$3'), '$3'}. - -formal_parameter_list -> patterns : '$1'. -formal_parameter_list -> '$empty' : []. - -clause_guard -> 'when' guard : '$2'. -clause_guard -> '$empty' : []. - -clause_body -> '->' exprs: '$2'. - - -function -> function_clause : '$1'. -function -> function_clause ';' function : - case '$1' of - {function, Pos1, Name1, Arity1, [Clause]} -> - case '$3' of - {function, _, Name1, Arity2, Clauses} -> - if - Arity1 /= Arity2 -> - throw({error, {Pos1, yecc, - io_lib:format('arity conflict in definition of ~w', - [Name1])}}); - true -> - {function, Pos1, Name1, Arity1, [Clause | Clauses]} - end; - _ -> - throw({error, {Pos1, yecc, - io_lib:format('missing final dot in def of ~w/~w', - [Name1, Arity1])}}) - end - end. - - -attribute -> atom : element(3, '$1'). -attribute -> '[' farity_list ']' : '$2'. - -farity_list -> farity : ['$1']. -farity_list -> farity ',' farity_list : ['$1' | '$3']. - -farity -> atom '/' integer : {element(3, '$1'), element(3, '$3')}. - - -form -> '-' atom '(' attribute ')' : - {attribute, element(2, '$2'), element(3, '$2'), '$4'}. -form -> function : '$1'. diff --git a/lib/parsetools/src/leex.erl b/lib/parsetools/src/leex.erl index cdf20461d9..bbef4053b4 100644 --- a/lib/parsetools/src/leex.erl +++ b/lib/parsetools/src/leex.erl @@ -58,6 +58,7 @@ gfile=[], % Graph file module, % Module name opts=[], % Options + encoding=none, % Encoding of Xrl file % posix=false, % POSIX regular expressions errors=[], warnings=[] @@ -146,7 +147,9 @@ format_error({regexp,E})-> end, ["bad regexp `",Es,"'"]; format_error(ignored_characters) -> - "ignored characters". + "ignored characters"; +format_error(cannot_parse) -> + io_lib:fwrite("cannot parse; probably encoding mismatch", []). %%% %%% Local functions @@ -298,10 +301,10 @@ pack_warnings([]) -> report_errors(St) -> when_opt(fun () -> foreach(fun({File,{none,Mod,E}}) -> - io:fwrite("~s: ~s\n", + io:fwrite("~s: ~ts\n", [File,Mod:format_error(E)]); ({File,{Line,Mod,E}}) -> - io:fwrite("~s:~w: ~s\n", + io:fwrite("~s:~w: ~ts\n", [File,Line,Mod:format_error(E)]) end, sort(St#leex.errors)) end, report_errors, St#leex.opts). @@ -316,11 +319,11 @@ report_warnings(St) -> ShouldReport = member(report_warnings, St#leex.opts) orelse ReportWerror, when_bool(fun () -> foreach(fun({File,{none,Mod,W}}) -> - io:fwrite("~s: ~s~s\n", + io:fwrite("~s: ~s~ts\n", [File,Prefix, Mod:format_error(W)]); ({File,{Line,Mod,W}}) -> - io:fwrite("~s:~w: ~s~s\n", + io:fwrite("~s:~w: ~s~ts\n", [File,Line,Prefix, Mod:format_error(W)]) end, sort(St#leex.warnings)) @@ -396,17 +399,18 @@ verbose_print(St, Format, Args) -> parse_file(St0) -> case file:open(St0#leex.xfile, [read]) of {ok,Xfile} -> + St1 = St0#leex{encoding = epp:set_encoding(Xfile)}, try - verbose_print(St0, "Parsing file ~s, ", [St0#leex.xfile]), + verbose_print(St1, "Parsing file ~s, ", [St1#leex.xfile]), %% We KNOW that errors throw so we can ignore them here. - {ok,Line1,St1} = parse_head(Xfile, St0), - {ok,Line2,Macs,St2} = parse_defs(Xfile, Line1, St1), - {ok,Line3,REAs,Actions,St3} = - parse_rules(Xfile, Line2, Macs, St2), - {ok,Code,St4} = parse_code(Xfile, Line3, St3), - verbose_print(St1, "contained ~w rules.~n", [length(REAs)]), - {ok,REAs,Actions,Code,St4} - after file:close(Xfile) + {ok,Line1,St2} = parse_head(Xfile, St1), + {ok,Line2,Macs,St3} = parse_defs(Xfile, Line1, St2), + {ok,Line3,REAs,Actions,St4} = + parse_rules(Xfile, Line2, Macs, St3), + {ok,Code,St5} = parse_code(Xfile, Line3, St4), + verbose_print(St5, "contained ~w rules.~n", [length(REAs)]), + {ok,REAs,Actions,Code,St5} + after ok = file:close(Xfile) end; {error,Error} -> add_error({none,leex,{file_error,Error}}, St0) @@ -415,7 +419,7 @@ parse_file(St0) -> %% parse_head(File, State) -> {ok,NextLine,State}. %% Parse the head of the file. Skip all comments and blank lines. -parse_head(Ifile, St) -> {ok,nextline(Ifile, 0),St}. +parse_head(Ifile, St) -> {ok,nextline(Ifile, 0, St),St}. %% parse_defs(File, Line, State) -> {ok,NextLine,Macros,State}. %% Parse the macro definition section of a file. This must exist. @@ -423,7 +427,7 @@ parse_head(Ifile, St) -> {ok,nextline(Ifile, 0),St}. parse_defs(Ifile, {ok,?DEFS_HEAD ++ Rest,L}, St) -> St1 = warn_ignored_chars(L, Rest, St), - parse_defs(Ifile, nextline(Ifile, L), [], St1); + parse_defs(Ifile, nextline(Ifile, L, St), [], St1); parse_defs(_, {ok,_,L}, St) -> add_error({L,leex,missing_defs}, St); parse_defs(_, {eof,L}, St) -> @@ -435,7 +439,7 @@ parse_defs(Ifile, {ok,Chars,L}=Line, Ms, St) -> case re:run(Chars, MS, [{capture,all_but_first,list}]) of {match,[Name,Def]} -> %%io:fwrite("~p = ~p\n", [Name,Def]), - parse_defs(Ifile, nextline(Ifile, L), [{Name,Def}|Ms], St); + parse_defs(Ifile, nextline(Ifile, L, St), [{Name,Def}|Ms], St); _ -> {ok,Line,Ms,St} % Anything else end; parse_defs(_, Line, Ms, St) -> @@ -446,7 +450,7 @@ parse_defs(_, Line, Ms, St) -> parse_rules(Ifile, {ok,?RULE_HEAD ++ Rest,L}, Ms, St) -> St1 = warn_ignored_chars(L, Rest, St), - parse_rules(Ifile, nextline(Ifile, L), Ms, [], [], 0, St1); + parse_rules(Ifile, nextline(Ifile, L, St), Ms, [], [], 0, St1); parse_rules(_, {ok,_,L}, _, St) -> add_error({L,leex,missing_rules}, St); parse_rules(_, {eof,L}, _, St) -> @@ -464,7 +468,7 @@ parse_rules(Ifile, NextLine, Ms, REAs, As, N, St) -> case collect_rule(Ifile, Chars, L0) of {ok,Re,Atoks,L1} -> {ok,REA,A,St1} = parse_rule(Re, L0, Atoks, Ms, N, St), - parse_rules(Ifile, nextline(Ifile, L1), Ms, + parse_rules(Ifile, nextline(Ifile, L1, St), Ms, [REA|REAs], [A|As], N+1, St1); {error,E} -> add_error(E, St) end; @@ -497,8 +501,10 @@ collect_rule(Ifile, Chars, L0) -> {error,E,_} -> {error,E} end. +collect_action(_Ifile, {error, _}, L, _Cont0) -> + {error, {L, leex, cannot_parse}, ignored_end_line}; collect_action(Ifile, Chars, L0, Cont0) -> - case erl_scan:tokens(Cont0, Chars, L0) of + case erl_scan:tokens(Cont0, Chars, L0, [unicode]) of {done,{ok,Toks,_},_} -> {ok,Toks,L0}; {done,{eof,_},_} -> {eof,L0}; {done,{error,E,_},_} -> {error,E,L0}; @@ -560,29 +566,32 @@ parse_code(Ifile, {ok,?CODE_HEAD ++ Rest,CodeL}, St) -> St1 = warn_ignored_chars(CodeL, Rest, St), {ok, CodePos} = file:position(Ifile, cur), %% Just count the lines; copy the code from file to file later. - NCodeLines = count_lines(Ifile, 0), + EndCodeLine = count_lines(Ifile, CodeL, St), + NCodeLines = EndCodeLine - CodeL, {ok,{CodeL,CodePos,NCodeLines},St1}; parse_code(_, {ok,_,L}, St) -> add_error({L,leex,missing_code}, St); parse_code(_, {eof,L}, St) -> add_error({L,leex,missing_code}, St). -count_lines(File, N) -> +count_lines(File, N, St) -> case io:get_line(File, leex) of eof -> N; - _Line -> count_lines(File, N+1) + {error, _} -> add_error({N+1, leex, cannot_parse}, St); + _Line -> count_lines(File, N+1, St) end. -%% nextline(InputFile, PrevLineNo) -> {ok,Chars,LineNo} | {eof,LineNo}. +%% nextline(InputFile, PrevLineNo, State) -> {ok,Chars,LineNo} | {eof,LineNo}. %% Get the next line skipping comment lines and blank lines. -nextline(Ifile, L) -> +nextline(Ifile, L, St) -> case io:get_line(Ifile, leex) of eof -> {eof,L}; + {error, _} -> add_error({L+1, leex, cannot_parse}, St); Chars -> case substr(Chars, span(Chars, " \t\n")+1) of - [$%|_Rest] -> nextline(Ifile, L+1); - [] -> nextline(Ifile, L+1); + [$%|_Rest] -> nextline(Ifile, L+1, St); + [] -> nextline(Ifile, L+1, St); _Other -> {ok,Chars,L+1} end end. @@ -1289,19 +1298,21 @@ out_file(St0, DFA, DF, Actions, Code) -> try case file:open(St0#leex.efile, [write]) of {ok,Ofile} -> + set_encoding(St0, Ofile), try + output_encoding_comment(Ofile, St0), output_file_directive(Ofile, St0#leex.ifile, 0), out_file(Ifile, Ofile, St0, DFA, DF, Actions, Code, 1), verbose_print(St0, "ok~n", []), St0 - after file:close(Ofile) + after ok = file:close(Ofile) end; {error,Error} -> verbose_print(St0, "error~n", []), add_error({none,leex,{file_error,Error}}, St0) end - after file:close(Ifile) + after ok = file:close(Ifile) end; {{error,Error},Ifile} -> add_error(Ifile, {none,leex,{file_error,Error}}, St0) @@ -1310,7 +1321,9 @@ out_file(St0, DFA, DF, Actions, Code) -> open_inc_file(State) -> Ifile = State#leex.ifile, case file:open(Ifile, [read]) of - {ok,F} -> {ok,F}; + {ok,F} -> + _ = epp:set_encoding(F), + {ok,F}; Error -> {Error,Ifile} end. @@ -1328,6 +1341,7 @@ inc_file_name(Filename) -> out_file(Ifile, Ofile, St, DFA, DF, Actions, Code, L) -> case io:get_line(Ifile, leex) of eof -> output_file_directive(Ofile, St#leex.ifile, L); + {error, _} -> add_error(St#leex.ifile, {L, leex, cannot_parse}, St); Line -> case substr(Line, 1, 5) of "##mod" -> out_module(Ofile, St); @@ -1347,14 +1361,23 @@ out_erlang_code(File, St, Code, L) -> output_file_directive(File, St#leex.xfile, CodeL), {ok,Xfile} = file:open(St#leex.xfile, [read]), try + set_encoding(St, Xfile), {ok,_} = file:position(Xfile, CodePos), - {ok,_} = file:copy(Xfile, File) + ok = file_copy(Xfile, File) after - file:close(Xfile) + ok = file:close(Xfile) end, io:nl(File), output_file_directive(File, St#leex.ifile, L). +file_copy(From, To) -> + case io:get_line(From, leex) of + eof -> ok; + Line when is_list(Line) -> + io:fwrite(To, "~ts", [Line]), + file_copy(From, To) + end. + out_dfa(File, St, DFA, Code, DF, L) -> {_CodeL,_CodePos,NCodeLines} = Code, %% Three file attributes before this one... @@ -1569,7 +1592,7 @@ out_dfa_graph(St, DFA, DF) -> io:fwrite(Gfile, "}~n", []), verbose_print(St, "ok~n", []), St - after file:close(Gfile) + after ok = file:close(Gfile) end; {error,Error} -> verbose_print(St, "error~n", []), @@ -1610,6 +1633,16 @@ dfa_edgelabel(Cranges) -> (C) -> [quote(C)] end, Cranges) ++ "]". +set_encoding(#leex{encoding = none}, File) -> + ok = io:setopts(File, [{encoding, epp:default_encoding()}]); +set_encoding(#leex{encoding = E}, File) -> + ok = io:setopts(File, [{encoding, E}]). + +output_encoding_comment(_File, #leex{encoding = none}) -> + ok; +output_encoding_comment(File, #leex{encoding = Encoding}) -> + io:fwrite(File, <<"%% ~s\n">>, [epp:encoding_to_string(Encoding)]). + output_file_directive(File, Filename, Line) -> io:fwrite(File, <<"-file(~s, ~w).\n">>, [format_filename(Filename), Line]). diff --git a/lib/parsetools/src/yecc.erl b/lib/parsetools/src/yecc.erl index b0792a6ed8..dbb7d025ae 100644 --- a/lib/parsetools/src/yecc.erl +++ b/lib/parsetools/src/yecc.erl @@ -1,7 +1,7 @@ %% %% %CopyrightBegin% %% -%% Copyright Ericsson AB 1996-2011. All Rights Reserved. +%% Copyright Ericsson AB 1996-2012. All Rights Reserved. %% %% The contents of this file are subject to the Erlang Public License, %% Version 1.1, (the "License"); you may not use this file except in @@ -42,6 +42,7 @@ includefile, includefile_version, module, + encoding = none, options = [], verbose = false, file_attrs = true, @@ -224,7 +225,11 @@ format_error({unused_nonterminal, Nonterminal}) -> [format_symbol(Nonterminal)]); format_error({unused_terminal, Terminal}) -> io_lib:fwrite("terminal symbol ~s not used", - [format_symbol(Terminal)]). + [format_symbol(Terminal)]); +format_error({bad_symbol, String}) -> + io_lib:fwrite("bad symbol ~ts", [String]); +format_error(cannot_parse) -> + io_lib:fwrite("cannot parse; possibly encoding mismatch", []). file(File) -> file(File, [report_errors, report_warnings]). @@ -257,7 +262,7 @@ yecc(Infile, Outfile, Verbose) -> yecc(Infile, Outfile, Verbose, []). yecc(Infilex, Outfilex, Verbose, Includefilex) -> - statistics(runtime), + _ = statistics(runtime), case file(Infilex, [{parserfile, Outfilex}, {verbose, Verbose}, {report, true}, @@ -407,7 +412,9 @@ infile(Parent, Infilex, Options) -> St = case file:open(St0#yecc.infile, [read, read_ahead]) of {ok, Inport} -> try - outfile(St0#yecc{inport = Inport}) + Encoding = epp:set_encoding(Inport), + St1 = St0#yecc{inport = Inport, encoding = Encoding}, + outfile(St1) after ok = file:close(Inport) end; @@ -428,6 +435,8 @@ outfile(St0) -> case file:open(St0#yecc.outfile, [write, delayed_write]) of {ok, Outport} -> try + %% Set the same encoding as infile: + set_encoding(St0, Outport), generate(St0#yecc{outport = Outport, line = 1}) catch throw: St1 -> @@ -466,13 +475,14 @@ timeit(Name, Fun, St0) -> -define(PASS(P), {P, fun P/1}). generate(St0) -> + St1 = output_encoding_comment(St0), Passes = [?PASS(parse_grammar), ?PASS(check_grammar), ?PASS(states_and_goto_table), ?PASS(parse_actions), ?PASS(action_conflicts), ?PASS(write_file)], - F = case member(time, St0#yecc.options) of + F = case member(time, St1#yecc.options) of true -> io:fwrite(<<"Generating parser from grammar in ~s\n">>, - [format_filename(St0#yecc.infile)]), + [format_filename(St1#yecc.infile)]), fun timeit/3; false -> fun(_Name, Fn, St) -> Fn(St) end @@ -484,13 +494,13 @@ generate(St0) -> true -> throw(St2) end end, - foldl(Fun, St0, Passes). + foldl(Fun, St1, Passes). parse_grammar(St) -> parse_grammar(St#yecc.inport, 1, St). parse_grammar(Inport, Line, St) -> - {NextLine, Grammar} = read_grammar(Inport, Line), + {NextLine, Grammar} = read_grammar(Inport, St, Line), parse_grammar(Grammar, Inport, NextLine, St). parse_grammar(eof, _Inport, _NextLine, St) -> @@ -523,6 +533,8 @@ parse_grammar({rule, Rule, Tokens}, St0) -> St#yecc{rules_list = [RuleDef | St#yecc.rules_list]}; parse_grammar({prec, Prec}, St) -> St#yecc{prec = Prec ++ St#yecc.prec}; +parse_grammar({#symbol{}, [{string,Line,String}]}, St) -> + add_error(Line, {bad_symbol, String}, St); parse_grammar({#symbol{line = Line, name = Name}, Symbols}, St) -> CF = fun(I) -> case element(I, St) of @@ -543,12 +555,17 @@ parse_grammar({#symbol{line = Line, name = Name}, Symbols}, St) -> _ -> add_warning(Line, bad_declaration, St) end. -read_grammar(Inport, Line) -> +read_grammar(Inport, St, Line) -> case yeccscan:scan(Inport, '', Line) of {eof, NextLine} -> {NextLine, eof}; {error, {ErrorLine, Mod, What}, NextLine} -> {NextLine, {error, ErrorLine, {error, Mod, What}}}; + {error, terminated} -> + throw(St); + {error, _} -> + File = St#yecc.infile, + throw(add_error(File, none, cannot_parse, St)); {ok, Input, NextLine} -> {NextLine, case yeccparser:parse(Input) of {error, {ErrorLine, Mod, Message}} -> @@ -738,9 +755,9 @@ states_and_goto_table(St0) -> create_precedence_table(St). parse_actions(St) -> - erase(), % the pd is used when decoding lookahead sets + _ = erase(), % the pd is used when decoding lookahead sets ParseActions = compute_parse_actions(St#yecc.n_states, St, []), - erase(), + _ = erase(), St#yecc{parse_actions = ParseActions, state_tab = []}. action_conflicts(St0) -> @@ -841,10 +858,10 @@ report_errors(St) -> case member(report_errors, St#yecc.options) of true -> foreach(fun({File,{none,Mod,E}}) -> - io:fwrite(<<"~s: ~s\n">>, + io:fwrite(<<"~s: ~ts\n">>, [File,Mod:format_error(E)]); ({File,{Line,Mod,E}}) -> - io:fwrite(<<"~s:~w: ~s\n">>, + io:fwrite(<<"~s:~w: ~ts\n">>, [File,Line,Mod:format_error(E)]) end, sort(St#yecc.errors)); false -> @@ -861,11 +878,11 @@ report_warnings(St) -> case member(report_warnings, St#yecc.options) orelse ReportWerror of true -> foreach(fun({File,{none,Mod,W}}) -> - io:fwrite(<<"~s: ~s~s\n">>, + io:fwrite(<<"~s: ~s~ts\n">>, [File,Prefix, Mod:format_error(W)]); ({File,{Line,Mod,W}}) -> - io:fwrite(<<"~s:~w: ~s~s\n">>, + io:fwrite(<<"~s:~w: ~s~ts\n">>, [File,Line,Prefix, Mod:format_error(W)]) end, sort(St#yecc.warnings)); @@ -1024,7 +1041,7 @@ compute_states(St0) -> rp_info = RulePointerInfo, goto = GotoTab}, - erase(), + _ = erase(), EndsymCode = code_terminal(StC#yecc.endsymbol, StC#yecc.symbol_tab), {StateId, State0} = compute_state([{EndsymCode, 1}], Tables), @@ -1923,9 +1940,10 @@ output_prelude(Outport, Inport, St0) when St0#yecc.includefile =:= [] -> {St20, 0, no_erlang_code}; Next_line -> St_10 = output_file_directive(St20, Infile, Next_line-1), - Nmbr_of_lines = include1([], Inport, Outport), - {St_10, Nmbr_of_lines, - {last_erlang_code_line, Next_line+Nmbr_of_lines}} + Last_line = include1([], Inport, Outport, Infile, + Next_line, St_10), + Nmbr_of_lines = Last_line - Next_line, + {St_10, Nmbr_of_lines, {last_erlang_code_line, Last_line}} end, St30 = nl(St25), IncludeFile = @@ -1946,13 +1964,13 @@ output_prelude(Outport, Inport, St0) -> {St30, N_lines_1, no_erlang_code}; Next_line -> St = output_file_directive(St30, Infile, Next_line-1), - Nmbr_of_lines = include1([], Inport, Outport), - {St, Nmbr_of_lines + N_lines_1, - {last_erlang_code_line, Next_line+Nmbr_of_lines}} + Last_line = include1([], Inport, Outport, Infile, Next_line, St), + Nmbr_of_lines = Last_line - Next_line, + {St, Nmbr_of_lines + N_lines_1, {last_erlang_code_line, Last_line}} end. output_header(St0) -> - lists:foldl(fun(Str, St) -> fwrite(St, <<"~s\n">>, [Str]) + lists:foldl(fun(Str, St) -> fwrite(St, <<"~ts\n">>, [Str]) end, St0, St0#yecc.header). output_goto(St, [{_Nonterminal, []} | Go], StateInfo) -> @@ -2250,8 +2268,8 @@ output_inlined(St0, FunctionName, Reduce, Infile) -> [append(["[", tl(A), " | __Stack]"])]) end, St = St40#yecc{line = St40#yecc.line + NLines}, - fwrite(St, <<" [begin\n ~s\n end | ~s].\n\n">>, - [pp_tokens(Tokens, Line0), Stack]). + fwrite(St, <<" [begin\n ~ts\n end | ~s].\n\n">>, + [pp_tokens(Tokens, Line0, St#yecc.encoding), Stack]). inlined_function_name(State, "Cat") -> inlined_function_name(State, ""); @@ -2421,24 +2439,24 @@ include(St, File, Outport) -> {error, Reason} -> throw(add_error(File, none, {file_error, Reason}, St)); {ok, Inport} -> + _ = epp:set_encoding(Inport), Line = io:get_line(Inport, ''), - N_lines = include1(Line, Inport, Outport), - file:close(Inport), - N_lines + try include1(Line, Inport, Outport, File, 1, St) - 1 + after ok = file:close(Inport) + end end. -include1(Line, Inport, Outport) -> - include1(Line, Inport, Outport, 0). - -include1(eof, _, _, Nmbr_of_lines) -> - Nmbr_of_lines; -include1(Line, Inport, Outport, Nmbr_of_lines) -> +include1(eof, _, _, _File, L, _St) -> + L; +include1({error, _}=_Error, _Inport, _Outport, File, L, St) -> + throw(add_error(File, L, cannot_parse, St)); +include1(Line, Inport, Outport, File, L, St) -> Incr = case member($\n, Line) of true -> 1; false -> 0 end, io:put_chars(Outport, Line), - include1(io:get_line(Inport, ''), Inport, Outport, Nmbr_of_lines + Incr). + include1(io:get_line(Inport, ''), Inport, Outport, File, L + Incr, St). includefile_version([]) -> {1,4}; @@ -2465,18 +2483,22 @@ parse_file(Epp) -> end. %% Keeps the line breaks of the original code. -pp_tokens(Tokens, Line0) -> - concat(pp_tokens1(Tokens, Line0, [])). +pp_tokens(Tokens, Line0, Enc) -> + concat(pp_tokens1(Tokens, Line0, Enc, [])). -pp_tokens1([], _Line0, _T0) -> +pp_tokens1([], _Line0, _Enc, _T0) -> []; -pp_tokens1([T | Ts], Line0, T0) -> +pp_tokens1([T | Ts], Line0, Enc, T0) -> Line = element(2, T), - [pp_sep(Line, Line0, T0), pp_symbol(T) | pp_tokens1(Ts, Line, T)]. + [pp_sep(Line, Line0, T0), pp_symbol(T, Enc)|pp_tokens1(Ts, Line, Enc, T)]. -pp_symbol({var,_,Var}) -> Var; -pp_symbol({_,_,Symbol}) -> io_lib:fwrite(<<"~p">>, [Symbol]); -pp_symbol({Symbol, _}) -> Symbol. +pp_symbol({var,_,Var}, _Enc) -> Var; +pp_symbol({string,_,String}, latin1) -> + io_lib:write_unicode_string_as_latin1(String); +pp_symbol({string,_,String}, _Enc) -> io_lib:write_unicode_string(String); +pp_symbol({_,_,Symbol}, latin1) -> io_lib:fwrite(<<"~p">>, [Symbol]); +pp_symbol({_,_,Symbol}, _Enc) -> io_lib:fwrite(<<"~tp">>, [Symbol]); +pp_symbol({Symbol, _}, _Enc) -> Symbol. pp_sep(Line, Line0, T0) when Line > Line0 -> ["\n " | pp_sep(Line - 1, Line0, T0)]; @@ -2485,6 +2507,16 @@ pp_sep(_Line, _Line0, {'.',_}) -> pp_sep(_Line, _Line0, _T0) -> " ". +set_encoding(#yecc{encoding = none}, Port) -> + ok = io:setopts(Port, [{encoding, epp:default_encoding()}]); +set_encoding(#yecc{encoding = E}, Port) -> + ok = io:setopts(Port, [{encoding, E}]). + +output_encoding_comment(#yecc{encoding = none}=St) -> + St; +output_encoding_comment(#yecc{encoding = Encoding}=St) -> + fwrite(St, <<"%% ~s\n">>, [epp:encoding_to_string(Encoding)]). + output_file_directive(St, Filename, Line) when St#yecc.file_attrs -> fwrite(St, <<"-file(~s, ~w).\n">>, [format_filename(Filename), Line]); @@ -2529,7 +2561,7 @@ format_assoc(nonassoc) -> format_symbol(Symbol) -> String = concat([Symbol]), - case erl_scan:string(String) of + case erl_scan:string(String, 1, [unicode]) of {ok, [{atom, _, _}], _} -> io_lib:fwrite(<<"~w">>, [Symbol]); {ok, [{Word, _}], _} when Word =/= ':', Word =/= '->' -> diff --git a/lib/parsetools/src/yeccscan.erl b/lib/parsetools/src/yeccscan.erl index d7ec3ba8d3..9e0e85143a 100644 --- a/lib/parsetools/src/yeccscan.erl +++ b/lib/parsetools/src/yeccscan.erl @@ -1,7 +1,7 @@ %% %% %CopyrightBegin% %% -%% Copyright Ericsson AB 1996-2009. All Rights Reserved. +%% Copyright Ericsson AB 1996-2012. All Rights Reserved. %% %% The contents of this file are subject to the Erlang Public License, %% Version 1.1, (the "License"); you may not use this file except in @@ -24,7 +24,7 @@ scan(Inport) -> scan(Inport, '', 1). scan(Inport, Prompt, Line1) -> - case catch io:scan_erl_form(Inport, Prompt, Line1) of + case catch io:scan_erl_form(Inport, Prompt, Line1, [unicode]) of {eof, Line2} -> {eof, Line2}; {ok, Tokens, Line2} -> @@ -34,6 +34,8 @@ scan(Inport, Prompt, Line1) -> _ -> {ok, lex(Tokens), Line2} end; + {error, Reason} -> + {error, Reason}; {error, Descriptor, Line2} -> {error, Descriptor, Line2}; {'EXIT', Why} -> diff --git a/lib/parsetools/test/leex_SUITE.erl b/lib/parsetools/test/leex_SUITE.erl index 1e50aedf07..4a0f70ba71 100644 --- a/lib/parsetools/test/leex_SUITE.erl +++ b/lib/parsetools/test/leex_SUITE.erl @@ -1,7 +1,8 @@ +%% -*- coding: latin-1 -*- %% %% %CopyrightBegin% %% -%% Copyright Ericsson AB 2010-2011. All Rights Reserved. +%% Copyright Ericsson AB 2010-2013. All Rights Reserved. %% %% The contents of this file are subject to the Erlang Public License, %% Version 1.1, (the "License"); you may not use this file except in @@ -42,7 +43,9 @@ -export([ file/1, compile/1, syntax/1, - pt/1, man/1, ex/1, ex2/1, not_yet/1]). + pt/1, man/1, ex/1, ex2/1, not_yet/1, + + otp_10302/1]). % Default timetrap timeout (set in init_per_testcase). -define(default_timeout, ?t:minutes(1)). @@ -63,7 +66,8 @@ all() -> groups() -> [{checks, [], [file, compile, syntax]}, - {examples, [], [pt, man, ex, ex2, not_yet]}]. + {examples, [], [pt, man, ex, ex2, not_yet]}, + {tickets, [], [otp_10302]}]. init_per_suite(Config) -> Config. @@ -875,6 +879,111 @@ not_yet(Config) when is_list(Config) -> ok. +otp_10302(doc) -> + "OTP-10302. Unicode characters scanner/parser."; +otp_10302(suite) -> []; +otp_10302(Config) when is_list(Config) -> + Dir = ?privdir, + Filename = filename:join(Dir, "file.xrl"), + Ret = [return, {report, true}], + + ok = file:write_file(Filename,<< + "%% coding: UTF-8\n" + "�" + >>), + {error,[{_,[{2,leex,cannot_parse}]}],[]} = + leex:file(Filename, Ret), + + ok = file:write_file(Filename,<< + "%% coding: UTF-8\n" + "Definitions.\n" + "�" + >>), + {error,[{_,[{3,leex,cannot_parse}]}],[]} = leex:file(Filename, Ret), + + ok = file:write_file(Filename,<< + "%% coding: UTF-8\n" + "Definitions.\n" + "A = a\n" + "L = [{A}-{Z}]\n" + "Z = z\n" + "Rules.\n" + "{L}+ : {token,{list_to_atom(TokenChars),H�pp}}.\n" + >>), + {error,[{_,[{7,leex,cannot_parse}]}],[]} = leex:file(Filename, Ret), + + ok = file:write_file(Filename,<< + "%% coding: UTF-8\n" + "Definitions.\n" + "A = a\n" + "L = [{A}-{Z}]\n" + "Z = z\n" + "Rules.\n" + "{L}+ : {token,{list_to_atom(TokenChars)}}.\n" + "Erlang code.\n" + "-export([t/0]).\n" + "t() ->\n" + " H�pp\n" + >>), + {error,[{_,[{11,leex,cannot_parse}]}],[]} = leex:file(Filename, Ret), + + Mini = <<"Definitions.\n" + "D = [0-9]\n" + "Rules.\n" + "{L}+ : {token,{word,TokenLine,TokenChars}}.\n" + "Erlang code.\n">>, + LeexPre = filename:join(Dir, "leexinc.hrl"), + ?line ok = file:write_file(LeexPre, <<"%% coding: UTF-8\n �">>), + PreErrors = run_test(Config, Mini, LeexPre), + {error,[{IncludeFile,[{2,leex,cannot_parse}]}],[]} = PreErrors, + "leexinc.hrl" = filename:basename(IncludeFile), + + Ts = [{uni_1, + <<"%% coding: UTF-8\n" + "Definitions.\n" + "A = a\n" + "L = [{A}-{Z}]\n" + "Z = z\n" + "Rules.\n" + "{L}+ : {token,{list_to_atom(TokenChars),\n" + "begin Häpp = foo, Häpp end," + " 'Häpp',\"\\x{400}B\",\"örn_Ѐ\"}}.\n" + "Erlang code.\n" + "-export([t/0]).\n" + "t() ->\n" + " %% Häpp, 'Häpp',\"\\x{400}B\",\"örn_Ѐ\"\n" + " {ok, [R], 1} = string(\"tip\"),\n" + " {tip,foo,'Häpp',[1024,66],[246,114,110,95,1024]} = R,\n" + " Häpp = foo,\n" + " {tip, Häpp, 'Häpp',\"\\x{400}B\",\"örn_Ѐ\"} = R,\n" + " ok.\n">>, + default, + ok}, + {uni_2, + <<"%% coding: Latin-1\n" + "Definitions.\n" + "A = a\n" + "L = [{A}-{Z}]\n" + "Z = z\n" + "Rules.\n" + "{L}+ : {token,{list_to_atom(TokenChars),\n" + "begin H�pp = foo, H�pp end," + " 'H�pp',\"\\x{400}B\",\"örn_Ѐ\"}}.\n" + "Erlang code.\n" + "-export([t/0]).\n" + "t() ->\n" + " %% H�pp, 'H�pp',\"\\x{400}B\",\"örn_Ѐ\"\n" + " {ok, [R], 1} = string(\"tip\"),\n" + " {tip,foo,'H�pp',[1024,66],[195,182,114,110,95,208,128]} = R,\n" + " H�pp = foo,\n" + " {tip, H�pp, 'H�pp',\"\\x{400}B\",\"örn_Ѐ\"} = R,\n" + " ok.\n">>, + default, + ok}], + run(Config, Ts), + + ok. + unwritable(Fname) -> {ok, Info} = file:read_file_info(Fname), Mode = Info#file_info.mode - 8#00200, diff --git a/lib/parsetools/test/yecc_SUITE.erl b/lib/parsetools/test/yecc_SUITE.erl index 3d26adf1be..c306dbe833 100644 --- a/lib/parsetools/test/yecc_SUITE.erl +++ b/lib/parsetools/test/yecc_SUITE.erl @@ -1,7 +1,8 @@ +%% -*- coding: latin-1 -*- %% %% %CopyrightBegin% %% -%% Copyright Ericsson AB 2005-2011. All Rights Reserved. +%% Copyright Ericsson AB 2005-2012. All Rights Reserved. %% %% The contents of this file are subject to the Erlang Public License, %% Version 1.1, (the "License"); you may not use this file except in @@ -48,7 +49,7 @@ otp_5369/1, otp_6362/1, otp_7945/1, otp_8483/1, otp_8486/1, - otp_7292/1, otp_7969/1, otp_8919/1]). + otp_7292/1, otp_7969/1, otp_8919/1, otp_10302/1]). % Default timetrap timeout (set in init_per_testcase). -define(default_timeout, ?t:minutes(1)). @@ -75,7 +76,7 @@ groups() -> [empty, prec, yeccpre, lalr, old_yecc, other_examples]}, {bugs, [], [otp_5369, otp_6362, otp_7945, otp_8483, otp_8486]}, - {improvements, [], [otp_7292, otp_7969, otp_8919]}]. + {improvements, [], [otp_7292, otp_7969, otp_8919, otp_10302]}]. init_per_suite(Config) -> Config. @@ -1815,6 +1816,153 @@ otp_8919(Config) when is_list(Config) -> "syntax error before: \"hello\"" = lists:flatten(Mod:format_error(Mess)), ok. +otp_10302(doc) -> + "OTP-10302. Unicode characters scanner/parser."; +otp_10302(suite) -> []; +otp_10302(Config) when is_list(Config) -> + Dir = ?privdir, + Filename = filename:join(Dir, "OTP-10302.yrl"), + Ret = [return, {report, true}], + Mini1 = <<"%% coding: utf-8 + Nonterminals H�pp. + nt -> t.">>, + ok = file:write_file(Filename, Mini1), + %% This could (and should) be refined: + {error,[{Filename,[{2,Mod1,Err1}]}],[]} = + yecc:file(Filename, Ret), + "cannot translate from UTF-8" = Mod1:format_error(Err1), + + Mini2 = <<"%% coding: Utf-8 + Nonterminals Hopp. + Terminals t. + Rootsymbol Hopp. + + Hopp -> t. + + Erlang code. + + t() -> + H�pp.">>, + ok = file:write_file(Filename, Mini2), + {error,[{Filename,[{11,Mod2,Err2}]}],[]} = + yecc:file(Filename, Ret), + "cannot parse; possibly encoding mismatch" = Mod2:format_error(Err2), + + Mini3 = <<"%% coding: latin-1 + Nonterminals Hopp. + Terminals t. + Rootsymbol Hopp. + + Hopp -> t. + + Erlang code. + + t() -> + H�pp.">>, + ok = file:write_file(Filename, Mini3), + YeccPre = filename:join(Dir, "yeccpre.hrl"), + ok = file:write_file(YeccPre, [<<"%% coding: UTF-8\n �.\n">>]), + Inc = [{includefile,YeccPre}], + {error,[{_,[{2,yecc,cannot_parse}]}],[]} = + yecc:file(Filename, Inc ++ Ret), + + ok = file:write_file(Filename, + <<"%% coding: UTF-8 + Nonterminals Hopp. + Terminals t. + Rootsymbol \"örn_Ѐ\". + Hopp -> t : '$1'.">>), + {error,[{Filename,[{4,yecc,{bad_symbol,"�rn_"++[1024]}}]}],[]} = + yecc:file(Filename, Ret), + + ok = file:write_file(Filename, + <<"%% coding: UTF-8 + Nonterminals Hopp. + Terminals t. + Rootsymbol Hopp. + Endsymbol \"örn_Ѐ\". + Hopp -> t : '$1'.">>), + {error,[{Filename,[{5,yecc,{bad_symbol,"�rn_"++[1024]}}]}],[]} = + yecc:file(Filename, Ret), + + ok = file:write_file(Filename, + <<"%% coding: UTF-8 + Nonterminals Hopp. + Terminals t. + Rootsymbol Hopp. + Expect \"örn_Ѐ\". + Hopp -> t : '$1'.">>), + {error,[{Filename,[{5,yecc,{bad_symbol,"�rn_"++[1024]}}]}],[]} = + yecc:file(Filename, Ret), + + ok = file:write_file(Filename, + <<"%% coding: UTF-8 + Nonterminals Hopp. + Terminals t. + Rootsymbol Hopp. + States \"örn_Ѐ\". + Hopp -> t : '$1'.">>), + {error,[{Filename,[{5,yecc,{bad_symbol,"�rn_"++[1024]}}]}],[]} = + yecc:file(Filename, Ret), + + Ts = [{otp_10302_1,<<" + %% coding: UTF-8 + Header \"%% örn_Ѐ\" \"%% \\x{400}B\". + Nonterminals Häpp list. + Terminals element. + Rootsymbol Häpp. + + Häpp -> list : '$1'. + + list -> element : '$1'. + list -> list element : + begin + Häpp = foo, + {Häpp, 'Häpp',\"\\x{400}B\",\"örn_Ѐ\"} + end. + + Erlang code. + + -export([t/0]). + + t() -> + L = [{element, 1}, {element,2}], + {ok, R} = parse(L), + Häpp = foo, + {_,_,[1024,66],[246,114,110,95,1024]} = R, + {Häpp,'Häpp',\"\\x{400}B\",\"örn_Ѐ\"} = R, + ok. + ">>,default,ok}, + {otp_10302_2,<<" + %% coding: Latin-1 + Nonterminals H�pp list. + Terminals element. + Rootsymbol H�pp. + + H�pp -> list : '$1'. + + list -> element : '$1'. + list -> list element : + begin + H�pp = foo, + {H�pp, 'H�pp',\"\\x{400}B\",\"örn_Ѐ\"} + end. + + Erlang code. + + -export([t/0]). + + t() -> + L = [{element, 1}, {element,2}], + {ok, R} = parse(L), + H�pp = foo, + {_,_,[1024,66],[195,182,114,110,95,208,128]} = R, + {H�pp,'H�pp',\"\\x{400}B\",\"örn_Ѐ\"} = R, + ok. + ">>,default,ok}], + run(Config, Ts), + ok. + yeccpre_size() -> yeccpre_size(default_yeccpre()). diff --git a/lib/percept/doc/src/notes.xml b/lib/percept/doc/src/notes.xml index 1ac1a5ab62..dd70cfe994 100644 --- a/lib/percept/doc/src/notes.xml +++ b/lib/percept/doc/src/notes.xml @@ -32,6 +32,21 @@ </header> <p>This document describes the changes made to the Percept application.</p> +<section><title>Percept 0.8.7</title> + + <section><title>Fixed Bugs and Malfunctions</title> + <list> + <item> + <p> + Add missing modules in app-file</p> + <p> + Own Id: OTP-10439</p> + </item> + </list> + </section> + +</section> + <section><title>Percept 0.8.6.1</title> <section><title>Improvements and New Features</title> diff --git a/lib/percept/src/percept.app.src b/lib/percept/src/percept.app.src index 7b20093ece..cf4a9fc438 100644 --- a/lib/percept/src/percept.app.src +++ b/lib/percept/src/percept.app.src @@ -1,7 +1,7 @@ %% %% %CopyrightBegin% %% -%% Copyright Ericsson AB 2007-2009. All Rights Reserved. +%% Copyright Ericsson AB 2007-2012. All Rights Reserved. %% %% The contents of this file are subject to the Erlang Public License, %% Version 1.1, (the "License"); you may not use this file except in diff --git a/lib/percept/vsn.mk b/lib/percept/vsn.mk index 9267a41055..f868246f7b 100644 --- a/lib/percept/vsn.mk +++ b/lib/percept/vsn.mk @@ -1 +1 @@ -PERCEPT_VSN = 0.8.6.1 +PERCEPT_VSN = 0.8.7 diff --git a/lib/public_key/asn1/Makefile b/lib/public_key/asn1/Makefile index 957c332cad..763b788e53 100644 --- a/lib/public_key/asn1/Makefile +++ b/lib/public_key/asn1/Makefile @@ -66,7 +66,7 @@ EBIN = ../ebin EXTRA_ERLC_FLAGS = ERL_COMPILE_FLAGS += $(EXTRA_ERLC_FLAGS) -ASN_FLAGS = -bber_bin +der +compact_bit_string +optimize +noobj +asn1config +inline +nif +ASN_FLAGS = -bber +der +compact_bit_string +noobj +asn1config +inline # ---------------------------------------------------- # Targets diff --git a/lib/public_key/doc/src/cert_records.xml b/lib/public_key/doc/src/cert_records.xml index edef664245..94c7c46350 100644 --- a/lib/public_key/doc/src/cert_records.xml +++ b/lib/public_key/doc/src/cert_records.xml @@ -39,8 +39,8 @@ describe the data types and not to specify the meaning of each component for this we refer you to <url href="http://www.ietf.org/rfc/rfc5280.txt">RFC 5280</url>. Also - descirbed is <p>CertificationRequest</p> that is defined by <url - href=http://www.rsa.com/rsalabs/node.asp?id=2124">PKCS-10</url>. + descirbed is <p>CertificationRequest</p> that is defined by + <url href="http://www.rsa.com/rsalabs/node.asp?id=2124">PKCS-10</url>. </p> <p>Use the following include directive to get access to the diff --git a/lib/public_key/doc/src/introduction.xml b/lib/public_key/doc/src/introduction.xml index b1d1114a6c..e0dd5603f7 100644 --- a/lib/public_key/doc/src/introduction.xml +++ b/lib/public_key/doc/src/introduction.xml @@ -39,7 +39,8 @@ <p> This application provides an API to public key infrastructure from <url href="http://www.ietf.org/rfc/rfc5280.txt">RFC 5280</url> (X.509 certificates) and public key formats defined by - the <url href=http://www.rsa.com/rsalabs/node.asp?id=2124"> PKCS-standard</url></p> + the <url href="http://www.rsa.com/rsalabs/node.asp?id=2124"> + PKCS-standard</url></p> </section> <section> diff --git a/lib/public_key/doc/src/notes.xml b/lib/public_key/doc/src/notes.xml index d895042570..a5e8beedf0 100644 --- a/lib/public_key/doc/src/notes.xml +++ b/lib/public_key/doc/src/notes.xml @@ -34,6 +34,41 @@ <file>notes.xml</file> </header> +<section><title>Public_Key 0.17</title> + + <section><title>Fixed Bugs and Malfunctions</title> + <list> + <item> + <p> + ssh_decode now handles comments, at the end of the line, + containing withe spaces correctly</p> + <p> + Own Id: OTP-9361</p> + </item> + <item> + <p> + Add missing references to sha224 and sha384</p> + <p> + Own Id: OTP-9362 Aux Id: seq12116 </p> + </item> + </list> + </section> + + + <section><title>Improvements and New Features</title> + <list> + <item> + <p> + public_key now supports PKCS-10 and includes exprimental + support for PKCS-7</p> + <p> + Own Id: OTP-10509 Aux Id: kunagi-291 [202] </p> + </item> + </list> + </section> + +</section> + <section><title>Public_Key 0.16</title> <section><title>Improvements and New Features</title> diff --git a/lib/public_key/doc/src/public_key.xml b/lib/public_key/doc/src/public_key.xml index 2ec1fcff9d..b240d53571 100644 --- a/lib/public_key/doc/src/public_key.xml +++ b/lib/public_key/doc/src/public_key.xml @@ -5,7 +5,7 @@ <header> <copyright> <year>2008</year> - <year>2012</year> + <year>2013</year> <holder>Ericsson AB, All Rights Reserved</holder> </copyright> <legalnotice> @@ -60,8 +60,9 @@ <p><code>string = [bytes()]</code></p> - <p><code>pki_asn1_type() = 'Certificate' | 'RSAPrivateKey'| 'RSAPublicKey' - 'DSAPrivateKey' | 'DSAPublicKey' | 'DHParameter' | 'SubjectPublicKeyInfo'| 'PrivateKeyInfo' | 'CertificationRequest'</code></p> + <p><code>pki_asn1_type() = 'Certificate' | 'RSAPrivateKey'| 'RSAPublicKey' | + 'DSAPrivateKey' | 'DSAPublicKey' | 'DHParameter' | 'SubjectPublicKeyInfo' | + 'PrivateKeyInfo' | 'CertificationRequest'</code></p> <p><code>pem_entry () = {pki_asn1_type(), binary(), %% DER or encrypted DER not_encrypted | cipher_info()} </code></p> diff --git a/lib/public_key/include/public_key.hrl b/lib/public_key/include/public_key.hrl index 2dfdbbb8f3..90ca7256ea 100644 --- a/lib/public_key/include/public_key.hrl +++ b/lib/public_key/include/public_key.hrl @@ -1,7 +1,7 @@ %% %% %CopyrightBegin% %% -%% Copyright Ericsson AB 2008-2011. All Rights Reserved. +%% Copyright Ericsson AB 2008-2012. All Rights Reserved. %% %% The contents of this file are subject to the Erlang Public License, %% Version 1.1, (the "License"); you may not use this file except in diff --git a/lib/public_key/src/pubkey_cert_records.erl b/lib/public_key/src/pubkey_cert_records.erl index 33fe940ea2..98004c71a3 100644 --- a/lib/public_key/src/pubkey_cert_records.erl +++ b/lib/public_key/src/pubkey_cert_records.erl @@ -119,7 +119,7 @@ encode_supportedPublicKey(#'OTPSubjectPublicKeyInfo'{algorithm= PA = subjectPublicKey = SPK0}) -> Type = supportedPublicKeyAlgorithms(Algo), {ok, SPK} = 'OTP-PUB-KEY':encode(Type, SPK0), - #'OTPSubjectPublicKeyInfo'{subjectPublicKey = {0,list_to_binary(SPK)}, algorithm=PA}. + #'OTPSubjectPublicKeyInfo'{subjectPublicKey = {0,SPK}, algorithm=PA}. %%% Extensions @@ -161,7 +161,7 @@ decode_extensions(Exts) -> case extension_id(Id) of undefined -> Ext; Type -> - {ok, Value} = 'OTP-PUB-KEY':decode(Type, list_to_binary(Value0)), + {ok, Value} = 'OTP-PUB-KEY':decode(Type, iolist_to_binary(Value0)), Ext#'Extension'{extnValue=transform(Value,decode)} end end, Exts). @@ -176,7 +176,7 @@ encode_extensions(Exts) -> Type -> Value1 = transform(Value0,encode), {ok, Value} = 'OTP-PUB-KEY':encode(Type, Value1), - Ext#'Extension'{extnValue=list_to_binary(Value)} + Ext#'Extension'{extnValue=Value} end end, Exts). diff --git a/lib/public_key/test/erl_make_certs.erl b/lib/public_key/test/erl_make_certs.erl index 254aa6d2f9..d6bdd05d01 100644 --- a/lib/public_key/test/erl_make_certs.erl +++ b/lib/public_key/test/erl_make_certs.erl @@ -137,10 +137,10 @@ decode_key(PemBin, Pw) -> encode_key(Key = #'RSAPrivateKey'{}) -> {ok, Der} = 'OTP-PUB-KEY':encode('RSAPrivateKey', Key), - {'RSAPrivateKey', list_to_binary(Der), not_encrypted}; + {'RSAPrivateKey', Der, not_encrypted}; encode_key(Key = #'DSAPrivateKey'{}) -> {ok, Der} = 'OTP-PUB-KEY':encode('DSAPrivateKey', Key), - {'DSAPrivateKey', list_to_binary(Der), not_encrypted}. + {'DSAPrivateKey', Der, not_encrypted}. make_tbs(SubjectKey, Opts) -> Version = list_to_atom("v"++integer_to_list(proplists:get_value(version, Opts, 3))), diff --git a/lib/reltool/doc/src/reltool.xml b/lib/reltool/doc/src/reltool.xml index fbe29753be..8437b7a623 100644 --- a/lib/reltool/doc/src/reltool.xml +++ b/lib/reltool/doc/src/reltool.xml @@ -5,7 +5,7 @@ <header> <copyright> <year>2009</year> - <year>2012</year> + <year>2013</year> <holder>Ericsson AB, All Rights Reserved</holder> </copyright> <legalnotice> @@ -274,7 +274,8 @@ </taglist> <p>Example:</p> - <p><code>erl -sasl releases_dir \"mytarget/releases\" -boot mytarget/releases/1.0/myrel -boot_var RELTOOL_EXT_LIB mytarget/lib</code></p> + <p><code>erl -sasl releases_dir \"mytarget/releases\" -boot mytarget/releases/1.0/myrel\ + -boot_var RELTOOL_EXT_LIB mytarget/lib</code></p> </item> <tag><c>incl_sys_filters</c></tag> diff --git a/lib/reltool/src/reltool_target.erl b/lib/reltool/src/reltool_target.erl index 6cb7ba0163..1f4ce7226a 100644 --- a/lib/reltool/src/reltool_target.erl +++ b/lib/reltool/src/reltool_target.erl @@ -482,33 +482,12 @@ do_gen_script(#rel{name = RelName, vsn = RelVsn}, %%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%% -load_app_mods(#app{mods = Mods} = App, Mand, PathFlag, Variables) -> +load_app_mods(#app{mods = Mods0} = App, Mand, PathFlag, Variables) -> Path = cr_path(App, PathFlag, Variables), - PartNames = - lists:sort([{packages:split(M),M} || - #mod{name = M, is_included=true} <- Mods, - not lists:member(M, Mand)]), - SplitMods = - lists:foldl( - fun({Parts,M}, [{Last, Acc}|Rest]) -> - [_|Tail] = lists:reverse(Parts), - case lists:reverse(Tail) of - Subs when Subs == Last -> - [{Last,[M|Acc]}|Rest]; - Subs -> - [{Subs, [M]}|[{Last,Acc}|Rest]] - end - end, - [{[], - []}], - PartNames), - lists:foldl( - fun({Subs,Ms}, Cmds) -> - [{path, [filename:join([Path | Subs])]}, - {primLoad, lists:sort(Ms)} | Cmds] - end, - [], - SplitMods). + Mods = [M || #mod{name = M, is_included=true} <- Mods0, + not lists:member(M, Mand)], + [{path, [filename:join([Path])]}, + {primLoad, lists:sort(Mods)}]. %%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%% %% sort_used_and_incl_apps(Apps, OrderedApps) -> Apps diff --git a/lib/runtime_tools/doc/src/dbg.xml b/lib/runtime_tools/doc/src/dbg.xml index c7c5cd4ff0..d8c82b2459 100644 --- a/lib/runtime_tools/doc/src/dbg.xml +++ b/lib/runtime_tools/doc/src/dbg.xml @@ -4,7 +4,7 @@ <erlref> <header> <copyright> - <year>1996</year><year>2011</year> + <year>1996</year><year>2013</year> <holder>Ericsson AB. All Rights Reserved.</holder> </copyright> <legalnotice> @@ -101,7 +101,8 @@ allowed:</p> <pre> 4> <input>dbg:fun2ms(fun([M,N]) when N > X, is_atomm(M) -> return_trace() end).</input> -Error: fun containing local erlang function calls ('is_atomm' called in guard) cannot be translated into match_spec +Error: fun containing local erlang function calls ('is_atomm' called in guard)\ + cannot be translated into match_spec {error,transform_error} 5> <input>dbg:fun2ms(fun([M,N]) when N > X, is_atom(M) -> return_trace() end).</input> [{['$1','$2'],[{'>','$2',{const,3}},{is_atom,'$1'}],[{return_trace}]}]</pre> diff --git a/lib/runtime_tools/src/observer_backend.erl b/lib/runtime_tools/src/observer_backend.erl index 01e99f3f5e..9498412505 100644 --- a/lib/runtime_tools/src/observer_backend.erl +++ b/lib/runtime_tools/src/observer_backend.erl @@ -83,7 +83,7 @@ get_table2(Parent, Table, Type) -> ets -> ets:info(Table, size); mnesia -> mnesia:table_info(Table, size) end, - case Size > 0 of + case Size =/= undefined andalso Size > 0 of false -> Parent ! {self(), '$end_of_table'}, normal; diff --git a/lib/runtime_tools/src/runtime_tools_sup.erl b/lib/runtime_tools/src/runtime_tools_sup.erl index c8dab250dd..264e172a3c 100644 --- a/lib/runtime_tools/src/runtime_tools_sup.erl +++ b/lib/runtime_tools/src/runtime_tools_sup.erl @@ -1,3 +1,4 @@ +%% -*- coding: utf-8 -*- %% %% %CopyrightBegin% %% @@ -18,7 +19,7 @@ %% %% ------------------------------------------------------------------------------ %% File : runtime_tools_sup.erl -%% Author : Lennart �hman <[email protected]> +%% Author : Lennart Öhman <[email protected]> -module(runtime_tools_sup). -behaviour(supervisor). diff --git a/lib/sasl/src/overload.erl b/lib/sasl/src/overload.erl index 5a4782efff..97f7bebe00 100644 --- a/lib/sasl/src/overload.erl +++ b/lib/sasl/src/overload.erl @@ -1,7 +1,8 @@ +%% -*- coding: utf-8 -*- %% %% %CopyrightBegin% %% -%% Copyright Ericsson AB 1996-2010. All Rights Reserved. +%% Copyright Ericsson AB 1996-2012. All Rights Reserved. %% %% The contents of this file are subject to the Erlang Public License, %% Version 1.1, (the "License"); you may not use this file except in @@ -25,7 +26,7 @@ format_status/2]). %%%----------------------------------------------------------------- -%%% This is a rewrite of overload from BS.3, by Peter H�gfeldt. +%%% This is a rewrite of overload from BS.3, by Peter Högfeldt. %%% %%% DESCRIPTION %%% diff --git a/lib/sasl/src/systools_make.erl b/lib/sasl/src/systools_make.erl index e8b28998c1..9b2e2c809b 100644 --- a/lib/sasl/src/systools_make.erl +++ b/lib/sasl/src/systools_make.erl @@ -1067,11 +1067,10 @@ check_mod(Mod,App,Dir,Ext,IncPath) -> end. mod_to_filename(Dir, Mod, Ext) -> - Parts = packages:split(Mod), - filename:join([Dir | Parts]) ++ Ext. + filename:join(Dir, atom_to_list(Mod) ++ Ext). check_module(Mod, Dir, ObjModTime, IncPath) -> - {SrcDirs,_IncDirs}= smart_guess(Mod, Dir,IncPath), + {SrcDirs,_IncDirs}= smart_guess(Dir,IncPath), case locate_src(Mod,SrcDirs) of {ok,_FDir,_File,LastModTime} -> if @@ -1085,7 +1084,7 @@ check_module(Mod, Dir, ObjModTime, IncPath) -> end. locate_src(Mod,[Dir|Dirs]) -> - File = filename:join(Dir, mod_to_fname(Mod) ++ ".erl"), + File = mod_to_filename(Dir, Mod, ".erl"), case file:read_file_info(File) of {ok,FileInfo} -> LastModTime = FileInfo#file_info.mtime, @@ -1096,9 +1095,6 @@ locate_src(Mod,[Dir|Dirs]) -> locate_src(_,[]) -> false. -mod_to_fname(Mod) -> - hd(lists:reverse(packages:split(Mod))). - %%______________________________________________________________________ %% smart_guess(Mod, Dir,IncludePath) -> {[Dirs],[IncDirs]} @@ -1106,17 +1102,12 @@ mod_to_fname(Mod) -> %% src-dir should be one of .../src or .../src/e_src %% If dir does not contain .../ebin set dir to the same directory. -smart_guess(Mod, Dir,IncPath) -> +smart_guess(Dir,IncPath) -> case reverse(filename:split(Dir)) of ["ebin"|D] -> - Subdirs = case packages:split(Mod) of - [_] -> []; - [_|_] = Parts -> - lists:reverse(tl(lists:reverse(Parts))) - end, D1 = reverse(D), - Dirs = [filename:join(D1 ++ ["src" | Subdirs]), - filename:join(D1 ++ ["src", "e_src" | Subdirs])], + Dirs = [filename:join(D1 ++ ["src"]), + filename:join(D1 ++ ["src", "e_src"])], {Dirs,Dirs ++ IncPath}; _ -> {[Dir],[Dir] ++ IncPath} @@ -1423,23 +1414,8 @@ load_appl_mods([], _, _, _) -> [{progress, modules_loaded}]. load_commands(Mods, Path) -> - SplitMods = lists:foldl( - fun({Parts,M}, [{Last, Acc}|Rest]) -> - [_|Tail] = lists:reverse(Parts), - case lists:reverse(Tail) of - Subs when Subs == Last -> - [{Last,[M|Acc]}|Rest]; - Subs -> - [{Subs, [M]}|[{Last,Acc}|Rest]] - end - end, [{[],[]}], - lists:sort([{packages:split(M),M} || M <- Mods])), - lists:foldl( - fun({Subs,Ms}, Cmds) -> - [{path, [filename:join([Path | Subs])]}, - {primLoad,lists:sort(Ms)} | Cmds] - end, [], SplitMods). - + [{path, [filename:join([Path])]}, + {primLoad,lists:sort(Mods)}]. %%______________________________________________________________________ %% Pack an application to an application term. @@ -1485,7 +1461,8 @@ mandatory_modules() -> preloaded() -> %% Sorted - [erl_prim_loader,erlang,init,otp_ring0,prim_file,prim_inet, prim_zip,zlib]. + [erl_prim_loader,erlang,erts_internal,init,otp_ring0,prim_file,prim_inet, + prim_zip,zlib]. %%______________________________________________________________________ %% Kernel processes; processes that are specially treated by the init diff --git a/lib/snmp/doc/src/notes.xml b/lib/snmp/doc/src/notes.xml index 442837d57d..b6b8751f6c 100644 --- a/lib/snmp/doc/src/notes.xml +++ b/lib/snmp/doc/src/notes.xml @@ -126,6 +126,70 @@ <section> + <title>SNMP Development Toolkit 4.23</title> + <p>Version 4.23 supports code replacement in runtime from/to + version 4.22, + 4.21.7 4.21.6 4.21.5, 4.21.4, 4.21.3, 4.21.2, 4.21.1 and 4.21. </p> + + <section> + <title>Improvements and new features</title> + <p>-</p> + +<!-- + <list type="bulleted"> + <item> + <p>[agent] Documenting previously existing but undocumented function, + <seealso marker="snmp_generic#get_table_info">snmp_generic:get_table_info/2</seealso>. </p> + <p>Own Id: OTP-9942</p> + </item> + + </list> +--> + + </section> + + <section> + <title>Fixed Bugs and Malfunctions</title> + <p>-</p> + + <!-- + <list type="bulleted"> + <item> + <p>[agent] Simultaneous + <seealso marker="snmpa#backup">snmpa:backup/1,2</seealso> + calls can interfere. + The master agent did not check if a backup was already in + progress when a backup request was accepted. </p> + <p>Own Id: OTP-9884</p> + <p>Aux Id: Seq 11995</p> + </item> + + </list> + --> + + </section> + + <section> + <title>Incompatibilities</title> +<!-- + <p>-</p> +--> + + <list type="bulleted"> + <item> + <p>[manager] The old Addr-and-Port based API functions, previously + long deprecated and marked for deletion in R16B, has now been + removed. </p> + <p>Own Id: OTP-10027</p> + </item> + + </list> + </section> + + </section> <!-- 4.23 --> + + + <section> <title>SNMP Development Toolkit 4.22</title> <p>Version 4.22 supports code replacement in runtime from/to version 4.21.7 4.21.6 4.21.5, 4.21.4, 4.21.3, 4.21.2, 4.21.1 and 4.21. </p> diff --git a/lib/snmp/doc/src/snmpm.xml b/lib/snmp/doc/src/snmpm.xml index 8ab3be8e18..07fdd208ff 100644 --- a/lib/snmp/doc/src/snmpm.xml +++ b/lib/snmp/doc/src/snmpm.xml @@ -279,7 +279,6 @@ sec_level = noAuthNoPriv | authNoPriv | authPriv <fsummary>Register this agent</fsummary> <type> <v>UserId = term()</v> - <v>Addr = ip_address()</v> <v>TargetName = target_name()</v> <v>Config = [agent_config()]</v> <v>agent_config() = {Item, Val}</v> @@ -367,7 +366,7 @@ sec_level = noAuthNoPriv | authNoPriv | authPriv <p>Update agent config. The function <c>update_agent_info/3</c> should be used when several values needs to be updated atomically. </p> <p>See function - <seealso marker="#register_agent">register_agent</seealso>) + <seealso marker="#register_agent">register_agent</seealso> for more info about what kind of items are allowed. </p> <marker id="which_agents"></marker> diff --git a/lib/snmp/src/agent/snmp_view_based_acm_mib.erl b/lib/snmp/src/agent/snmp_view_based_acm_mib.erl index 436f15eb9c..ad9540e886 100644 --- a/lib/snmp/src/agent/snmp_view_based_acm_mib.erl +++ b/lib/snmp/src/agent/snmp_view_based_acm_mib.erl @@ -496,7 +496,7 @@ verify_vacmSecurityToGroupTable_col(_, Val) -> %% %%----------------------------------------------------------------- vacmAccessTable(print) -> - %% M�ste jag g�ra om alla entrien till {RowIdx, Row}? + %% Do I need to turn all entries into {RowIdx, Row}? TableInfo = get_table(vacmAccessTable), PrintRow = fun(Prefix, Row) -> diff --git a/lib/snmp/src/manager/snmpm.erl b/lib/snmp/src/manager/snmpm.erl index f590892c66..6ac0115dad 100644 --- a/lib/snmp/src/manager/snmpm.erl +++ b/lib/snmp/src/manager/snmpm.erl @@ -107,63 +107,9 @@ async_get_bulk/5, async_get_bulk/6, async_get_bulk/7, async_get_bulk/8 ]). -%% Backward compatibility exports (API version "1") --deprecated({agent_info, 3}). --deprecated({update_agent_info, 5}). --deprecated({g, 3}). --deprecated({g, 4}). --deprecated({g, 5}). --deprecated({g, 6}). --deprecated({g, 7}). --deprecated({ag, 3}). --deprecated({ag, 4}). --deprecated({ag, 5}). --deprecated({ag, 6}). --deprecated({ag, 7}). --deprecated({gn, 3}). --deprecated({gn, 4}). --deprecated({gn, 5}). --deprecated({gn, 6}). --deprecated({gn, 7}). --deprecated({agn, 3}). --deprecated({agn, 4}). --deprecated({agn, 5}). --deprecated({agn, 6}). --deprecated({agn, 7}). --deprecated({gb, 5}). --deprecated({gb, 6}). --deprecated({gb, 7}). --deprecated({gb, 8}). --deprecated({gb, 9}). --deprecated({agb, 5}). --deprecated({agb, 6}). --deprecated({agb, 7}). --deprecated({agb, 8}). --deprecated({agb, 9}). --deprecated({s, 3}). --deprecated({s, 4}). --deprecated({s, 5}). --deprecated({s, 6}). --deprecated({s, 7}). --deprecated({as, 3}). --deprecated({as, 4}). --deprecated({as, 5}). --deprecated({as, 6}). --deprecated({as, 7}). --export([ - agent_info/3, update_agent_info/5, - g/3, g/4, g/5, g/6, g/7, - ag/3, ag/4, ag/5, ag/6, ag/7, - gn/3, gn/4, gn/5, gn/6, gn/7, - agn/3, agn/4, agn/5, agn/6, agn/7, - gb/5, gb/6, gb/7, gb/8, gb/9, - agb/5, agb/6, agb/7, agb/8, agb/9, - s/3, s/4, s/5, s/6, s/7, - as/3, as/4, as/5, as/6, as/7 - ]). - %% Application internal export -export([start_link/3, snmpm_start_verify/2, snmpm_start_verify/3]). +-export([target_name/1, target_name/2]). -include_lib("snmp/src/misc/snmp_debug.hrl"). @@ -439,32 +385,16 @@ unregister_agent(UserId, Addr, Port) -> Error end. + agent_info(TargetName, Item) -> snmpm_config:agent_info(TargetName, Item). -%% Backward compatibility -agent_info(Addr, Port, Item) -> - case target_name(Addr, Port) of - {ok, TargetName} -> - agent_info(TargetName, Item); - Error -> - Error - end. - update_agent_info(UserId, TargetName, Info) when is_list(Info) -> snmpm_config:update_agent_info(UserId, TargetName, Info). update_agent_info(UserId, TargetName, Item, Val) -> update_agent_info(UserId, TargetName, [{Item, Val}]). -%% Backward compatibility functions -update_agent_info(UserId, Addr, Port, Item, Val) -> - case target_name(Addr, Port) of - {ok, TargetName} -> - update_agent_info(UserId, TargetName, Item, Val); - Error -> - Error - end. which_agents() -> snmpm_config:which_agents(). @@ -552,55 +482,6 @@ sync_get(UserId, TargetName, Context, Oids, Timeout, ExtraInfo) -> %% </BACKWARD-COMPAT> -%% <DEPRECATED> -g(UserId, Addr, Oids) -> - g(UserId, Addr, ?DEFAULT_AGENT_PORT, Oids). - -g(UserId, Addr, CtxName, Oids) when is_list(CtxName) andalso is_list(Oids) -> - g(UserId, Addr, ?DEFAULT_AGENT_PORT, CtxName, Oids); - -g(UserId, Addr, Port, Oids) when is_integer(Port) andalso is_list(Oids) -> - g(UserId, Addr, Port, ?DEFAULT_CONTEXT, Oids); - -g(UserId, Addr, Oids, Timeout) - when is_list(Oids) andalso is_integer(Timeout) -> - g(UserId, Addr, ?DEFAULT_AGENT_PORT, Oids, Timeout). - -g(UserId, Addr, Port, CtxName, Oids) - when is_integer(Port) andalso is_list(CtxName) andalso is_list(Oids) -> - case target_name(Addr, Port) of - {ok, TargetName} -> - sync_get(UserId, TargetName, CtxName, Oids); - Error -> - Error - end; - -g(UserId, Addr, Port, Oids, Timeout) - when is_integer(Port) andalso is_list(Oids) andalso is_integer(Timeout) -> - g(UserId, Addr, Port, ?DEFAULT_CONTEXT, Oids, Timeout); - -g(UserId, Addr, CtxName, Oids, Timeout) - when is_list(CtxName) andalso is_list(Oids) andalso is_integer(Timeout) -> - g(UserId, Addr, ?DEFAULT_AGENT_PORT, CtxName, Oids, Timeout). - -g(UserId, Addr, Port, CtxName, Oids, Timeout) -> - case target_name(Addr, Port) of - {ok, TargetName} -> - sync_get(UserId, TargetName, CtxName, Oids, Timeout); - Error -> - Error - end. - -g(UserId, Addr, Port, CtxName, Oids, Timeout, ExtraInfo) -> - case target_name(Addr, Port) of - {ok, TargetName} -> - sync_get(UserId, TargetName, CtxName, Oids, Timeout, ExtraInfo); - Error -> - Error - end. -%% </DEPRECATED> - - %% --- asynchroneous get-request --- %% @@ -637,55 +518,6 @@ async_get(UserId, TargetName, Context, Oids, Expire, ExtraInfo) -> %% </BACKWARD-COMPAT> -%% <DEPRECATED> -ag(UserId, Addr, Oids) -> - ag(UserId, Addr, ?DEFAULT_AGENT_PORT, Oids). - -ag(UserId, Addr, Port, Oids) when is_integer(Port) andalso is_list(Oids) -> - ag(UserId, Addr, Port, ?DEFAULT_CONTEXT, Oids); - -ag(UserId, Addr, CtxName, Oids) when is_list(CtxName) andalso is_list(Oids) -> - ag(UserId, Addr, ?DEFAULT_AGENT_PORT, CtxName, Oids); - -ag(UserId, Addr, Oids, Expire) when is_list(Oids) andalso is_integer(Expire) -> - ag(UserId, Addr, ?DEFAULT_AGENT_PORT, ?DEFAULT_CONTEXT, Oids, Expire). - -ag(UserId, Addr, Port, CtxName, Oids) - when is_integer(Port) andalso is_list(CtxName) andalso is_list(Oids) -> - case target_name(Addr, Port) of - {ok, TargetName} -> - async_get(UserId, TargetName, CtxName, Oids); - Error -> - Error - end; - -ag(UserId, Addr, Port, Oids, Expire) - when is_integer(Port) andalso is_list(Oids) andalso is_integer(Expire) -> - ag(UserId, Addr, Port, ?DEFAULT_CONTEXT, Oids, Expire); - -ag(UserId, Addr, CtxName, Oids, Expire) - when is_list(CtxName) andalso is_list(Oids) andalso is_integer(Expire) -> - ag(UserId, Addr, ?DEFAULT_AGENT_PORT, CtxName, Oids, Expire). - -ag(UserId, Addr, Port, CtxName, Oids, Expire) -> - case target_name(Addr, Port) of - {ok, TargetName} -> - async_get(UserId, TargetName, CtxName, Oids, Expire); - Error -> - Error - end. - -ag(UserId, Addr, Port, CtxName, Oids, Expire, ExtraInfo) -> - case target_name(Addr, Port) of - {ok, TargetName} -> - async_get(UserId, TargetName, CtxName, Oids, Expire, ExtraInfo); - Error -> - Error - end. -%% </DEPRECATED> - - - %% --- synchroneous get_next-request --- %% @@ -719,55 +551,6 @@ sync_get_next(UserId, TargetName, Context, Oids, Timeout, ExtraInfo) -> %% </BACKWARD-COMPAT> -%% <DEPRECATED> -gn(UserId, Addr, Oids) -> - gn(UserId, Addr, ?DEFAULT_AGENT_PORT, Oids). - -gn(UserId, Addr, CtxName, Oids) when is_list(CtxName) andalso is_list(Oids) -> - gn(UserId, Addr, ?DEFAULT_AGENT_PORT, CtxName, Oids); - -gn(UserId, Addr, Port, Oids) when is_integer(Port) andalso is_list(Oids) -> - gn(UserId, Addr, Port, ?DEFAULT_CONTEXT, Oids); - -gn(UserId, Addr, Oids, Timeout) - when is_list(Oids) andalso is_integer(Timeout) -> - gn(UserId, Addr, ?DEFAULT_AGENT_PORT, Oids, Timeout). - -gn(UserId, Addr, Port, CtxName, Oids) - when is_integer(Port) andalso is_list(CtxName) andalso is_list(Oids) -> - case target_name(Addr, Port) of - {ok, TargetName} -> - sync_get_next(UserId, TargetName, CtxName, Oids); - Error -> - Error - end; - -gn(UserId, Addr, Port, Oids, Timeout) - when is_integer(Port) andalso is_list(Oids) andalso is_integer(Timeout) -> - gn(UserId, Addr, Port, ?DEFAULT_CONTEXT, Oids, Timeout); -gn(UserId, Addr, CtxName, Oids, Timeout) - when is_list(CtxName) andalso is_list(Oids) andalso is_integer(Timeout) -> - gn(UserId, Addr, ?DEFAULT_AGENT_PORT, CtxName, Oids, Timeout). - -gn(UserId, Addr, Port, CtxName, Oids, Timeout) -> - case target_name(Addr, Port) of - {ok, TargetName} -> - sync_get_next(UserId, TargetName, CtxName, Oids, Timeout); - Error -> - Error - end. - -gn(UserId, Addr, Port, CtxName, Oids, Timeout, ExtraInfo) -> - case target_name(Addr, Port) of - {ok, TargetName} -> - sync_get_next(UserId, TargetName, CtxName, Oids, Timeout, ExtraInfo); - Error -> - Error - end. -%% </DEPRECATED> - - - %% --- asynchroneous get_next-request --- %% @@ -801,56 +584,6 @@ async_get_next(UserId, TargetName, Context, Oids, Expire, ExtraInfo) -> %% </BACKWARD-COMPAT> -%% <DEPRECATED> -agn(UserId, Addr, Oids) -> - agn(UserId, Addr, ?DEFAULT_AGENT_PORT, Oids). - -agn(UserId, Addr, CtxName, Oids) when is_list(CtxName) andalso is_list(Oids) -> - agn(UserId, Addr, ?DEFAULT_AGENT_PORT, CtxName, Oids); - -agn(UserId, Addr, Port, Oids) when is_integer(Port) andalso is_list(Oids) -> - agn(UserId, Addr, Port, ?DEFAULT_CONTEXT, Oids); - -agn(UserId, Addr, Oids, Expire) - when is_list(Oids) andalso is_integer(Expire) -> - agn(UserId, Addr, ?DEFAULT_AGENT_PORT, Oids, Expire). - -agn(UserId, Addr, Port, CtxName, Oids) - when is_integer(Port) andalso is_list(CtxName) andalso is_list(Oids) -> - case target_name(Addr, Port) of - {ok, TargetName} -> - async_get_next(UserId, TargetName, CtxName, Oids); - Error -> - Error - end; - -agn(UserId, Addr, Port, Oids, Expire) - when is_integer(Port) andalso is_list(Oids) andalso is_integer(Expire) -> - agn(UserId, Addr, Port, ?DEFAULT_CONTEXT, Oids, Expire); -agn(UserId, Addr, CtxName, Oids, Expire) - when is_list(CtxName) andalso is_list(CtxName) andalso is_integer(Expire) -> - agn(UserId, Addr, ?DEFAULT_AGENT_PORT, CtxName, Oids, Expire). - -agn(UserId, Addr, Port, CtxName, Oids, Expire) -> - case target_name(Addr, Port) of - {ok, TargetName} -> - async_get_next(UserId, TargetName, CtxName, Oids, Expire); - Error -> - Error - end. - -agn(UserId, Addr, Port, CtxName, Oids, Expire, ExtraInfo) -> - case target_name(Addr, Port) of - {ok, TargetName} -> - async_get_next(UserId, TargetName, CtxName, Oids, Expire, - ExtraInfo); - Error -> - Error - end. -%% </DEPRECATED> - - - %% --- synchroneous set-request --- %% @@ -884,64 +617,6 @@ sync_set(UserId, TargetName, Context, VarsAndVals, Timeout, ExtraInfo) -> %% </BACKWARD-COMPAT> -%% <DEPRECATED> -s(UserId, Addr, VarsAndVals) -> - s(UserId, Addr, ?DEFAULT_AGENT_PORT, VarsAndVals). - -s(UserId, Addr, Port, VarsAndVals) - when is_integer(Port) andalso is_list(VarsAndVals) -> - s(UserId, Addr, Port, ?DEFAULT_CONTEXT, VarsAndVals); - -s(UserId, Addr, CtxName, VarsAndVals) - when is_list(CtxName) andalso is_list(VarsAndVals) -> - s(UserId, Addr, ?DEFAULT_AGENT_PORT, CtxName, VarsAndVals); - -s(UserId, Addr, VarsAndVals, Timeout) - when is_list(VarsAndVals) andalso is_integer(Timeout) -> - s(UserId, Addr, ?DEFAULT_AGENT_PORT, VarsAndVals, Timeout). - -s(UserId, Addr, Port, CtxName, VarsAndVals) - when is_integer(Port) andalso - is_list(CtxName) andalso - is_list(VarsAndVals) -> - case target_name(Addr, Port) of - {ok, TargetName} -> - sync_set(UserId, TargetName, CtxName, VarsAndVals); - Error -> - Error - end; - -s(UserId, Addr, Port, VarsAndVals, Timeout) - when is_integer(Port) andalso - is_list(VarsAndVals) andalso - is_integer(Timeout) -> - s(UserId, Addr, Port, ?DEFAULT_CONTEXT, VarsAndVals, Timeout); - -s(UserId, Addr, CtxName, VarsAndVals, Timeout) - when is_list(CtxName) andalso - is_list(VarsAndVals) andalso - is_integer(Timeout) -> - s(UserId, Addr, ?DEFAULT_AGENT_PORT, CtxName, VarsAndVals, Timeout). - -s(UserId, Addr, Port, CtxName, VarsAndVals, Timeout) -> - case target_name(Addr, Port) of - {ok, TargetName} -> - sync_set(UserId, TargetName, CtxName, VarsAndVals, Timeout); - Error -> - Error - end. - -s(UserId, Addr, Port, CtxName, VarsAndVals, Timeout, ExtraInfo) -> - case target_name(Addr, Port) of - {ok, TargetName} -> - sync_set(UserId, TargetName, CtxName, VarsAndVals, Timeout, ExtraInfo); - Error -> - Error - end. -%% </DEPRECATED> - - - %% --- asynchroneous set-request --- %% @@ -975,63 +650,6 @@ async_set(UserId, TargetName, Context, VarsAndVals, Expire, ExtraInfo) -> %% </BACKWARD-COMPAT> -%% <DEPRECATED> -as(UserId, Addr, VarsAndVals) -> - as(UserId, Addr, ?DEFAULT_AGENT_PORT, VarsAndVals). - -as(UserId, Addr, Port, VarsAndVals) - when is_integer(Port) andalso is_list(VarsAndVals) -> - as(UserId, Addr, Port, ?DEFAULT_CONTEXT, VarsAndVals); - -as(UserId, Addr, CtxName, VarsAndVals) - when is_list(CtxName) andalso is_list(VarsAndVals) -> - as(UserId, Addr, ?DEFAULT_AGENT_PORT, CtxName, VarsAndVals); - -as(UserId, Addr, VarsAndVals, Expire) - when is_list(VarsAndVals) andalso is_integer(Expire) -> - as(UserId, Addr, ?DEFAULT_AGENT_PORT, VarsAndVals, Expire). - -as(UserId, Addr, Port, CtxName, VarsAndVals) - when is_integer(Port) andalso - is_list(CtxName) andalso - is_list(VarsAndVals) -> - case target_name(Addr, Port) of - {ok, TargetName} -> - async_set(UserId, TargetName, CtxName, VarsAndVals); - Error -> - Error - end; - -as(UserId, Addr, Port, VarsAndVals, Expire) - when is_integer(Port) andalso - is_list(VarsAndVals) andalso - is_integer(Expire) -> - as(UserId, Addr, Port, ?DEFAULT_CONTEXT, VarsAndVals, Expire); - -as(UserId, Addr, CtxName, VarsAndVals, Expire) - when is_list(CtxName) andalso - is_list(VarsAndVals) andalso - is_integer(Expire) -> - as(UserId, Addr, ?DEFAULT_AGENT_PORT, CtxName, VarsAndVals, Expire). - -as(UserId, Addr, Port, CtxName, VarsAndVals, Expire) -> - case target_name(Addr, Port) of - {ok, TargetName} -> - async_set(UserId, TargetName, CtxName, VarsAndVals, Expire); - Error -> - Error - end. - -as(UserId, Addr, Port, CtxName, VarsAndVals, Expire, ExtraInfo) -> - case target_name(Addr, Port) of - {ok, TargetName} -> - async_set(UserId, TargetName, CtxName, VarsAndVals, Expire, ExtraInfo); - Error -> - Error - end. -%% </DEPRECATED> - - %% --- synchroneous get-bulk --- %% @@ -1091,162 +709,6 @@ sync_get_bulk(UserId, TargetName, NonRep, MaxRep, Context, Oids, Timeout, %% </BACKWARD-COMPAT> -%% <DEPRECATED> -gb(UserId, Addr, NonRep, MaxRep, Oids) -> - %% p("gb -> entry with" - %% "~n UserId: ~p" - %% "~n Addr: ~p" - %% "~n NonRep: ~p" - %% "~n MaxRep: ~p" - %% "~n Oids: ~p", - %% [UserId, Addr, NonRep, MaxRep, Oids]), - gb(UserId, Addr, ?DEFAULT_AGENT_PORT, NonRep, MaxRep, Oids). - -gb(UserId, Addr, Port, NonRep, MaxRep, Oids) - when is_integer(Port) andalso - is_integer(NonRep) andalso - is_integer(MaxRep) andalso - is_list(Oids) -> - %% p("gb -> entry with" - %% "~n UserId: ~p" - %% "~n Addr: ~p" - %% "~n Port: ~p" - %% "~n NonRep: ~p" - %% "~n MaxRep: ~p" - %% "~n Oids: ~p", - %% [UserId, Addr, Port, NonRep, MaxRep, Oids]), - gb(UserId, Addr, Port, NonRep, MaxRep, ?DEFAULT_CONTEXT, Oids); - -gb(UserId, Addr, NonRep, MaxRep, CtxName, Oids) - when is_integer(NonRep) andalso - is_integer(MaxRep) andalso - is_list(CtxName) andalso - is_list(Oids) -> - %% p("gb -> entry with" - %% "~n UserId: ~p" - %% "~n Addr: ~p" - %% "~n NonRep: ~p" - %% "~n MaxRep: ~p" - %% "~n CtxName: ~p" - %% "~n Oids: ~p", - %% [UserId, Addr, NonRep, MaxRep, CtxName, Oids]), - gb(UserId, Addr, ?DEFAULT_AGENT_PORT, NonRep, MaxRep, CtxName, Oids); - -gb(UserId, Addr, NonRep, MaxRep, Oids, Timeout) - when is_integer(NonRep) andalso - is_integer(MaxRep) andalso - is_list(Oids) andalso - is_integer(Timeout) -> - %% p("gb -> entry with" - %% "~n UserId: ~p" - %% "~n Addr: ~p" - %% "~n NonRep: ~p" - %% "~n MaxRep: ~p" - %% "~n Oids: ~p" - %% "~n Timeout: ~p", - %% [UserId, Addr, NonRep, MaxRep, Oids, Timeout]), - gb(UserId, Addr, ?DEFAULT_AGENT_PORT, NonRep, MaxRep, Oids, Timeout). - -gb(UserId, Addr, Port, NonRep, MaxRep, CtxName, Oids) - when is_integer(Port) andalso - is_integer(NonRep) andalso - is_integer(MaxRep) andalso - is_list(CtxName) andalso - is_list(Oids) -> - %% p("gb -> entry with" - %% "~n UserId: ~p" - %% "~n Addr: ~p" - %% "~n Port: ~p" - %% "~n NonRep: ~p" - %% "~n MaxRep: ~p" - %% "~n CtxName: ~p" - %% "~n Oids: ~p", - %% [UserId, Addr, Port, NonRep, MaxRep, CtxName, Oids]), - case target_name(Addr, Port) of - {ok, TargetName} -> - %% p("gb -> TargetName: ~p", [TargetName]), - sync_get_bulk(UserId, TargetName, NonRep, MaxRep, CtxName, Oids); - Error -> - Error - end; - -gb(UserId, Addr, Port, NonRep, MaxRep, Oids, Timeout) - when is_integer(Port) andalso - is_integer(NonRep) andalso - is_integer(MaxRep) andalso - is_list(Oids) andalso - is_integer(Timeout) -> - %% p("gb -> entry with" - %% "~n UserId: ~p" - %% "~n Addr: ~p" - %% "~n Port: ~p" - %% "~n NonRep: ~p" - %% "~n MaxRep: ~p" - %% "~n Oids: ~p" - %% "~n Timeout: ~p", - %% [UserId, Addr, Port, NonRep, MaxRep, Oids, Timeout]), - gb(UserId, Addr, Port, NonRep, MaxRep, ?DEFAULT_CONTEXT, Oids, Timeout); - -gb(UserId, Addr, NonRep, MaxRep, CtxName, Oids, Timeout) - when is_integer(NonRep) andalso - is_integer(MaxRep) andalso - is_list(CtxName) andalso - is_list(Oids) andalso - is_integer(Timeout) -> - %% p("gb -> entry with" - %% "~n UserId: ~p" - %% "~n Addr: ~p" - %% "~n NonRep: ~p" - %% "~n MaxRep: ~p" - %% "~n CtxName: ~p" - %% "~n Oids: ~p" - %% "~n Timeout: ~p", - %% [UserId, Addr, NonRep, MaxRep, CtxName, Oids, Timeout]), - gb(UserId, Addr, ?DEFAULT_AGENT_PORT, NonRep, MaxRep, CtxName, Oids, - Timeout). - -gb(UserId, Addr, Port, NonRep, MaxRep, CtxName, Oids, Timeout) -> - %% p("gb -> entry with" - %% "~n UserId: ~p" - %% "~n Addr: ~p" - %% "~n Port: ~p" - %% "~n NonRep: ~p" - %% "~n MaxRep: ~p" - %% "~n CtxName: ~p" - %% "~n Oids: ~p" - %% "~n Timeout: ~p", - %% [UserId, Addr, Port, NonRep, MaxRep, CtxName, Oids, Timeout]), - case target_name(Addr, Port) of - {ok, TargetName} -> - sync_get_bulk(UserId, TargetName, - NonRep, MaxRep, CtxName, Oids, Timeout); - Error -> - Error - end. - -gb(UserId, Addr, Port, NonRep, MaxRep, CtxName, Oids, Timeout, ExtraInfo) -> - %% p("gb -> entry with" - %% "~n UserId: ~p" - %% "~n Addr: ~p" - %% "~n Port: ~p" - %% "~n NonRep: ~p" - %% "~n MaxRep: ~p" - %% "~n CtxName: ~p" - %% "~n Oids: ~p" - %% "~n Timeout: ~p" - %% "~n ExtraInfo: ~p", - %% [UserId, Addr, Port, NonRep, MaxRep, CtxName, Oids, Timeout, ExtraInfo]), - case target_name(Addr, Port) of - {ok, TargetName} -> - sync_get_bulk(UserId, TargetName, - NonRep, MaxRep, CtxName, Oids, Timeout, ExtraInfo); - Error -> - Error - end. -%% </DEPRECATED> - - - %% --- asynchroneous get-bulk --- %% @@ -1291,81 +753,6 @@ async_get_bulk(UserId, TargetName, NonRep, MaxRep, Context, Oids, Expire, %% </BACKWARD-COMPAT> -%% <DEPRECATED> -agb(UserId, Addr, NonRep, MaxRep, Oids) -> - agb(UserId, Addr, ?DEFAULT_AGENT_PORT, NonRep, MaxRep, Oids). - -agb(UserId, Addr, Port, NonRep, MaxRep, Oids) - when is_integer(Port) andalso - is_integer(NonRep) andalso - is_integer(MaxRep) andalso - is_list(Oids) -> - agb(UserId, Addr, Port, NonRep, MaxRep, ?DEFAULT_CONTEXT, Oids); - -agb(UserId, Addr, NonRep, MaxRep, CtxName, Oids) - when is_integer(NonRep) andalso - is_integer(MaxRep) andalso - is_list(CtxName) andalso - is_list(Oids) -> - agb(UserId, Addr, ?DEFAULT_AGENT_PORT, NonRep, MaxRep, CtxName, Oids); - -agb(UserId, Addr, NonRep, MaxRep, Oids, Expire) - when is_integer(NonRep) andalso - is_integer(MaxRep) andalso - is_list(Oids) andalso - is_integer(Expire) -> - agb(UserId, Addr, ?DEFAULT_AGENT_PORT, NonRep, MaxRep, Oids, Expire). - -agb(UserId, Addr, Port, NonRep, MaxRep, CtxName, Oids) - when is_integer(Port) andalso - is_integer(NonRep) andalso - is_integer(MaxRep), - is_list(CtxName) andalso - is_list(Oids) -> - case target_name(Addr, Port) of - {ok, TargetName} -> - async_get_bulk(UserId, TargetName, - NonRep, MaxRep, CtxName, Oids); - Error -> - Error - end; - -agb(UserId, Addr, Port, NonRep, MaxRep, Oids, Expire) - when is_integer(Port) andalso - is_integer(NonRep) andalso - is_integer(MaxRep) andalso - is_list(Oids) andalso - is_integer(Expire) -> - agb(UserId, Addr, Port, NonRep, MaxRep, ?DEFAULT_CONTEXT, Oids, Expire); - -agb(UserId, Addr, NonRep, MaxRep, CtxName, Oids, Expire) - when is_integer(NonRep) andalso - is_integer(MaxRep) andalso - is_list(CtxName) andalso - is_list(Oids) andalso - is_integer(Expire) -> - agb(UserId, Addr, ?DEFAULT_AGENT_PORT, NonRep, MaxRep, CtxName, Oids). - -agb(UserId, Addr, Port, NonRep, MaxRep, CtxName, Oids, Expire) -> - case target_name(Addr, Port) of - {ok, TargetName} -> - async_get_bulk(UserId, TargetName, - NonRep, MaxRep, CtxName, Oids, Expire); - Error -> - Error - end. - -agb(UserId, Addr, Port, NonRep, MaxRep, CtxName, Oids, Expire, ExtraInfo) -> - case target_name(Addr, Port) of - {ok, TargetName} -> - async_get_bulk(UserId, TargetName, - NonRep, MaxRep, CtxName, Oids, Expire, - ExtraInfo); - Error -> - Error - end. -%% </DEPRECATED> - cancel_async_request(UserId, ReqId) -> snmpm_server:cancel_async_request(UserId, ReqId). @@ -1457,7 +844,7 @@ sys_up_time() -> format_reason(Reason) -> format_reason("", Reason). -format_reason(Prefix, Reason) when is_integer(Prefix) and (Prefix >= 0) -> +format_reason(Prefix, Reason) when is_integer(Prefix) andalso (Prefix >= 0) -> format_reason(lists:duplicate(Prefix, $ ), Reason); format_reason(Prefix, Reason) when is_list(Prefix) -> case (catch do_format_reason(Prefix, Reason)) of @@ -1691,6 +1078,9 @@ format_vb_value(Prefix, _Type, Val) -> %% --- Internal utility functions --- %% +target_name(Addr) -> + target_name(Addr, ?DEFAULT_AGENT_PORT). + target_name(Addr, Port) -> snmpm_config:agent_info(Addr, Port, target_name). diff --git a/lib/snmp/src/manager/snmpm_conf.erl b/lib/snmp/src/manager/snmpm_conf.erl index e50508c489..03dbd028f7 100644 --- a/lib/snmp/src/manager/snmpm_conf.erl +++ b/lib/snmp/src/manager/snmpm_conf.erl @@ -1,7 +1,8 @@ +%% -*- coding: utf-8 -*- %% %% %CopyrightBegin% %% -%% Copyright Ericsson AB 2006-2010. All Rights Reserved. +%% Copyright Ericsson AB 2006-2012. All Rights Reserved. %% %% The contents of this file are subject to the Erlang Public License, %% Version 1.1, (the "License"); you may not use this file except in @@ -365,7 +366,7 @@ do_write_usm_conf(Fd, do_write_usm_conf(Fd, {EngineID, UserName, SecName, AuthP, AuthKey, PrivP, PrivKey}) -> - io:format(Fd, "{\"~s\", \"~s\", \"~s\", �~w, ~w, ~w, ~w}.~n", + io:format(Fd, "{\"~s\", \"~s\", \"~s\", í~w, ~w, ~w, ~w}.~n", [EngineID, UserName, SecName, AuthP, AuthKey, PrivP, PrivKey]); do_write_usm_conf(_Fd, Crap) -> error({bad_usm_conf, Crap}). diff --git a/lib/snmp/src/manager/snmpm_mpd.erl b/lib/snmp/src/manager/snmpm_mpd.erl index d7148bb4a4..953c94ab54 100644 --- a/lib/snmp/src/manager/snmpm_mpd.erl +++ b/lib/snmp/src/manager/snmpm_mpd.erl @@ -591,8 +591,8 @@ sec_engine_id(TargetName) -> %% BMK BMK BMK -%% Denna verkar v�ldigt lik generate_v1_v2c_response_msg! -%% Gemensam? Borde det finnas olikheter? +%% This one looks very similar to lik generate_v1_v2c_response_msg! +%% Common/shared? Should there be differences? %% generate_v1_v2c_msg(Vsn, Pdu, Community, Log) -> ?vdebug("generate_v1_v2c_msg -> encode pdu", []), diff --git a/lib/snmp/test/snmp_agent_test.erl b/lib/snmp/test/snmp_agent_test.erl index bfe14bc080..09e1eb25a9 100644 --- a/lib/snmp/test/snmp_agent_test.erl +++ b/lib/snmp/test/snmp_agent_test.erl @@ -1,3 +1,4 @@ +%% -*- coding: utf-8 -*- %% %% %CopyrightBegin% %% @@ -5752,14 +5753,14 @@ otp_4394_config(AgentConfDir, MgrDir, Ip0) -> ?line write_community_conf(AgentConfDir, [C1, C2]), ?line update_vacm(Vsn, AgentConfDir), Ta1 = {"shelob v1", - [134,138,177,177], 5000, 1500, 3, %% Anv�nd Ip och modda + [134,138,177,177], 5000, 1500, 3, %% Use Ip and modify "pc1", "target_v1", "", %% [255,255,255,255,0,0], [], 2048}, Ta2 = {"bifur v1", - [134,138,177,75], 5000, 1500, 3, %% Anv�nd Ip + [134,138,177,75], 5000, 1500, 3, %% Use Ip "pc2", "target_v1", "", %% [255,255,255,255,0,0], diff --git a/lib/snmp/test/snmp_log_test.erl b/lib/snmp/test/snmp_log_test.erl index aeac4d1f71..e9345b44cc 100644 --- a/lib/snmp/test/snmp_log_test.erl +++ b/lib/snmp/test/snmp_log_test.erl @@ -375,9 +375,9 @@ log_to_io1(Config) when is_list(Config) -> %%====================================================================== -%% Starta en logger-process som med ett visst intervall loggar -%% meddelanden. Starta en reader-process som vid ett viss tillf�lle -%% l�ser fr�n loggen. +%% Start a logger-process that logs messages with a certain interval. +%% Start a reader-process that reads messages from the log at a certain +%% point of time. log_to_io2(suite) -> []; log_to_io2(doc) -> "Log to io from a different process than which " @@ -578,9 +578,9 @@ log_to_txt(Name, SeqNoGen, Config) when is_list(Config) -> %%====================================================================== -%% Starta en logger-process som med ett visst intervall loggar -%% meddelanden. Starta en reader-process som vid ett viss tillf�lle -%% l�ser fr�n loggen. +%% Start a logger-process that logs messages with a certain interval. +%% Start a reader-process that reads messages from the log at a certain +%% point of time. %% %% Test: ts:run(snmp, snmp_log_test, log_to_txt2, [batch]). diff --git a/lib/snmp/test/snmp_manager_test.erl b/lib/snmp/test/snmp_manager_test.erl index c374a2f0a6..dedbae5ce4 100644 --- a/lib/snmp/test/snmp_manager_test.erl +++ b/lib/snmp/test/snmp_manager_test.erl @@ -174,7 +174,26 @@ end_per_suite(Config) when is_list(Config) -> init_per_testcase(Case, Config) when is_list(Config) -> io:format(user, "~n~n*** INIT ~w:~w ***~n~n", [?MODULE,Case]), - init_per_testcase2(Case, Config). + %% This version of the API, based on Addr and Port, has been deprecated + DeprecatedApiCases = + [ + simple_sync_get1, + simple_async_get1, + simple_sync_get_next1, + simple_async_get_next1, + simple_sync_set1, + simple_async_set1, + simple_sync_get_bulk1, + simple_async_get_bulk1, + misc_async1 + ], + case lists:member(Case, DeprecatedApiCases) of + true -> + %% ?SKIP(api_no_longer_supported); + {skip, api_no_longer_supported}; + false -> + init_per_testcase2(Case, Config) + end. init_per_testcase2(Case, Config) -> ?DBG("init_per_testcase2 -> ~p", [erlang:nodes()]), @@ -226,18 +245,6 @@ init_per_testcase2(Case, Config) -> Conf2. init_per_testcase3(Case, Config) -> - ApiCases01 = - [ - simple_sync_get1, - simple_async_get1, - simple_sync_get_next1, - simple_async_get_next1, - simple_sync_set1, - simple_async_set1, - simple_sync_get_bulk1, - simple_async_get_bulk1, - misc_async1 - ], ApiCases02 = [ simple_sync_get2, @@ -273,7 +280,6 @@ init_per_testcase3(Case, Config) -> inform_swarm, report ] ++ - ApiCases01 ++ ApiCases02 ++ ApiCases03, case lists:member(Case, Cases) of @@ -319,18 +325,6 @@ end_per_testcase(Case, Config) when is_list(Config) -> Conf2. end_per_testcase2(Case, Config) -> - ApiCases01 = - [ - simple_sync_get1, - simple_async_get1, - simple_sync_get_next1, - simple_async_get_next1, - simple_sync_set1, - simple_async_set1, - simple_sync_get_bulk1, - simple_async_get_bulk1, - misc_async1 - ], ApiCases02 = [ simple_sync_get2, @@ -366,7 +360,6 @@ end_per_testcase2(Case, Config) -> inform_swarm, report ] ++ - ApiCases01 ++ ApiCases02 ++ ApiCases03, case lists:member(Case, Cases) of @@ -1596,12 +1589,14 @@ register_agent3(Config) when is_list(Config) -> simple_sync_get1(doc) -> ["Simple sync get-request - Old style (Addr & Port)"]; simple_sync_get1(suite) -> []; simple_sync_get1(Config) when is_list(Config) -> + ?SKIP(api_no_longer_supported), + process_flag(trap_exit, true), put(tname, ssg1), p("starting with Config: ~p~n", [Config]), Node = ?config(manager_node, Config), - Addr = ?config(ip, Config), + Addr = ?config(manager_agent_target_name, Config), Port = ?AGENT_PORT, p("issue get-request without loading the mib"), @@ -1757,6 +1752,8 @@ simple_async_get1(doc) -> ["Simple (async) get-request - Old style (Addr & Port)"]; simple_async_get1(suite) -> []; simple_async_get1(Config) when is_list(Config) -> + ?SKIP(api_no_longer_supported), + process_flag(trap_exit, true), put(tname, sag1), p("starting with Config: ~p~n", [Config]), @@ -1972,6 +1969,8 @@ simple_sync_get_next1(doc) -> ["Simple (sync) get_next-request - " "Old style (Addr & Port)"]; simple_sync_get_next1(suite) -> []; simple_sync_get_next1(Config) when is_list(Config) -> + ?SKIP(api_no_longer_supported), + process_flag(trap_exit, true), put(tname, ssgn1), p("starting with Config: ~p~n", [Config]), @@ -2264,6 +2263,8 @@ simple_async_get_next1(doc) -> ["Simple (async) get_next-request - " "Old style (Addr & Port)"]; simple_async_get_next1(suite) -> []; simple_async_get_next1(Config) when is_list(Config) -> + ?SKIP(api_no_longer_supported), + process_flag(trap_exit, true), put(tname, ssgn1), p("starting with Config: ~p~n", [Config]), @@ -2516,6 +2517,8 @@ simple_sync_set1(doc) -> ["Simple (sync) set-request - " "Old style (Addr & Port)"]; simple_sync_set1(suite) -> []; simple_sync_set1(Config) when is_list(Config) -> + ?SKIP(api_no_longer_supported), + process_flag(trap_exit, true), put(tname, sss1), p("starting with Config: ~p~n", [Config]), @@ -2686,6 +2689,8 @@ simple_async_set1(doc) -> ["Simple (async) set-request - " "Old style (Addr & Port)"]; simple_async_set1(suite) -> []; simple_async_set1(Config) when is_list(Config) -> + ?SKIP(api_no_longer_supported), + process_flag(trap_exit, true), put(tname, sas1), p("starting with Config: ~p~n", [Config]), @@ -2896,6 +2901,8 @@ simple_sync_get_bulk1(doc) -> ["Simple (sync) get_bulk-request - " "Old style (Addr & Port)"]; simple_sync_get_bulk1(suite) -> []; simple_sync_get_bulk1(Config) when is_list(Config) -> + ?SKIP(api_no_longer_supported), + process_flag(trap_exit, true), put(tname, ssgb1), p("starting with Config: ~p~n", [Config]), @@ -3261,6 +3268,8 @@ simple_async_get_bulk1(doc) -> ["Simple (async) get_bulk-request - " "Old style (Addr & Port)"]; simple_async_get_bulk1(suite) -> []; simple_async_get_bulk1(Config) when is_list(Config) -> + ?SKIP(api_no_longer_supported), + process_flag(trap_exit, true), put(tname, sagb1), p("starting with Config: ~p~n", [Config]), @@ -3605,6 +3614,8 @@ misc_async1(doc) -> ["Misc (async) request(s) - " "Old style (Addr & Port)"]; misc_async1(suite) -> []; misc_async1(Config) when is_list(Config) -> + ?SKIP(api_no_longer_supported), + process_flag(trap_exit, true), put(tname, ms1), p("starting with Config: ~p~n", [Config]), @@ -5591,29 +5602,35 @@ fin_mgr_user(Conf) -> init_mgr_user_data1(Conf) -> Node = ?config(manager_node, Conf), - Addr = ?config(ip, Conf), - Port = ?AGENT_PORT, - ?line ok = mgr_user_register_agent(Node, Addr, Port), + TargetName = ?config(manager_agent_target_name, Conf), + Addr = ?config(ip, Conf), + Port = ?AGENT_PORT, + ?line ok = mgr_user_register_agent(Node, TargetName, + [{address, Addr}, + {port, Port}, + {engine_id, "agentEngine"}]), Agents = mgr_user_which_own_agents(Node), ?DBG("Own agents: ~p", [Agents]), - ?line {ok, DefAgentConf} = mgr_user_agent_info(Node, Addr, Port, all), + ?line {ok, DefAgentConf} = mgr_user_agent_info(Node, TargetName, all), ?DBG("Default agent config: ~n~p", [DefAgentConf]), - ?line ok = mgr_user_update_agent_info(Node, Addr, Port, + ?line ok = mgr_user_update_agent_info(Node, TargetName, community, "all-rights"), - ?line ok = mgr_user_update_agent_info(Node, Addr, Port, + ?line ok = mgr_user_update_agent_info(Node, TargetName, sec_name, "all-rights"), - ?line ok = mgr_user_update_agent_info(Node, Addr, Port, + ?line ok = mgr_user_update_agent_info(Node, TargetName, engine_id, "agentEngine"), - ?line ok = mgr_user_update_agent_info(Node, Addr, Port, + ?line ok = mgr_user_update_agent_info(Node, TargetName, max_message_size, 1024), - ?line {ok, AgentConf} = mgr_user_agent_info(Node, Addr, Port, all), + ?line {ok, AgentConf} = mgr_user_agent_info(Node, TargetName, all), ?DBG("Updated agent config: ~n~p", [AgentConf]), Conf. init_mgr_user_data2(Conf) -> + ?DBG("init_mgr_user_data2 -> entry with" + "~n Conf: ~p", [Conf]), Node = ?config(manager_node, Conf), TargetName = ?config(manager_agent_target_name, Conf), Addr = ?config(ip, Conf), @@ -5641,9 +5658,8 @@ init_mgr_user_data2(Conf) -> fin_mgr_user_data1(Conf) -> Node = ?config(manager_node, Conf), - Addr = ?config(ip, Conf), - Port = ?AGENT_PORT, - mgr_user_unregister_agent(Node, Addr, Port), + TargetName = ?config(manager_agent_target_name, Conf), + mgr_user_unregister_agent(Node, TargetName), mgr_user_which_own_agents(Node), Conf. @@ -5670,33 +5686,41 @@ mgr_user_stop(Node) -> %% mgr_user_register_agent(Node, Addr, ?AGENT_PORT, []). mgr_user_register_agent(Node, TargetName, Conf) when is_list(TargetName) andalso is_list(Conf) -> - rcall(Node, snmp_manager_user, register_agent, [TargetName, Conf]); -mgr_user_register_agent(Node, Addr, Port) -> - mgr_user_register_agent(Node, Addr, Port, []). -mgr_user_register_agent(Node, Addr, Port, Conf) -> - rcall(Node, snmp_manager_user, register_agent, [Addr, Port, Conf]). + rcall(Node, snmp_manager_user, register_agent, [TargetName, Conf]). +%% <REMOVED-IN-R16B> +%% mgr_user_register_agent(Node, Addr, Port) -> +%% mgr_user_register_agent(Node, Addr, Port, []). +%% mgr_user_register_agent(Node, Addr, Port, Conf) -> +%% rcall(Node, snmp_manager_user, register_agent, [Addr, Port, Conf]). +%% </REMOVED-IN-R16B> %% mgr_user_unregister_agent(Node) -> %% mgr_user_unregister_agent(Node, ?LOCALHOST(), ?AGENT_PORT). -mgr_user_unregister_agent(Node, Addr_or_TargetName) -> - rcall(Node, snmp_manager_user, unregister_agent, [Addr_or_TargetName]). -mgr_user_unregister_agent(Node, Addr, Port) -> - rcall(Node, snmp_manager_user, unregister_agent, [Addr, Port]). - -mgr_user_agent_info(Node, Addr_or_TargetName, Item) when is_atom(Item) -> - rcall(Node, snmp_manager_user, agent_info, [Addr_or_TargetName, Item]). -mgr_user_agent_info(Node, Addr, Port, Item) when is_atom(Item) -> - rcall(Node, snmp_manager_user, agent_info, [Addr, Port, Item]). +mgr_user_unregister_agent(Node, TargetName) when is_list(TargetName) -> + rcall(Node, snmp_manager_user, unregister_agent, [TargetName]). +%% <REMOVED-IN-R16B> +%% mgr_user_unregister_agent(Node, Addr, Port) -> +%% rcall(Node, snmp_manager_user, unregister_agent, [Addr, Port]). +%% </REMOVED-IN-R16B> + +mgr_user_agent_info(Node, TargetName, Item) + when is_list(TargetName) andalso is_atom(Item) -> + rcall(Node, snmp_manager_user, agent_info, [TargetName, Item]). +%% <REMOVED-IN-R16B> +%% mgr_user_agent_info(Node, Addr, Port, Item) when is_atom(Item) -> +%% rcall(Node, snmp_manager_user, agent_info, [Addr, Port, Item]). +%% </REMOVED-IN-R16B> %% mgr_user_update_agent_info(Node, Item, Val) when atom(Item) -> %% mgr_user_update_agent_info(Node, ?LOCALHOST(), ?AGENT_PORT, Item, Val). -mgr_user_update_agent_info(Node, Addr_or_TargetName, Item, Val) - when is_atom(Item) -> - rcall(Node, snmp_manager_user, update_agent_info, - [Addr_or_TargetName, Item, Val]). -mgr_user_update_agent_info(Node, Addr, Port, Item, Val) when is_atom(Item) -> - rcall(Node, snmp_manager_user, update_agent_info, - [Addr, Port, Item, Val]). +mgr_user_update_agent_info(Node, TargetName, Item, Val) + when is_list(TargetName) andalso is_atom(Item) -> + rcall(Node, snmp_manager_user, update_agent_info, [TargetName, Item, Val]). +%% <REMOVED-IN-R16B> +%% mgr_user_update_agent_info(Node, Addr, Port, Item, Val) when is_atom(Item) -> +%% rcall(Node, snmp_manager_user, update_agent_info, +%% [Addr, Port, Item, Val]). +%% </REMOVED-IN-R16B> %% mgr_user_which_all_agents(Node) -> %% rcall(Node, snmp_manager_user, which_all_agents, []). @@ -5709,89 +5733,112 @@ mgr_user_load_mib(Node, Mib) -> %% mgr_user_sync_get(Node, Oids) -> %% mgr_user_sync_get(Node, ?LOCALHOST(), ?AGENT_PORT, Oids). -mgr_user_sync_get(Node, Addr_or_TargetName, Oids) -> - rcall(Node, snmp_manager_user, sync_get, [Addr_or_TargetName, Oids]). +mgr_user_sync_get(Node, TargetName, Oids) when is_list(TargetName) -> + rcall(Node, snmp_manager_user, sync_get, [TargetName, Oids]). +%% <REMOVED-IN-R16B> mgr_user_sync_get(Node, Addr, Port, Oids) -> rcall(Node, snmp_manager_user, sync_get, [Addr, Port, Oids]). +%% </REMOVED-IN-R16B> -mgr_user_sync_get2(Node, TargetName, Oids, SendOpts) -> +mgr_user_sync_get2(Node, TargetName, Oids, SendOpts) when is_list(TargetName) -> rcall(Node, snmp_manager_user, sync_get2, [TargetName, Oids, SendOpts]). %% mgr_user_async_get(Node, Oids) -> %% mgr_user_async_get(Node, ?LOCALHOST(), ?AGENT_PORT, Oids). -mgr_user_async_get(Node, Addr_or_TargetName, Oids) -> - rcall(Node, snmp_manager_user, async_get, [Addr_or_TargetName, Oids]). +mgr_user_async_get(Node, TargetName, Oids) when is_list(TargetName) -> + rcall(Node, snmp_manager_user, async_get, [TargetName, Oids]). +%% <REMOVED-IN-R16B> mgr_user_async_get(Node, Addr, Port, Oids) -> rcall(Node, snmp_manager_user, async_get, [Addr, Port, Oids]). +%% </REMOVED-IN-R16B> -mgr_user_async_get2(Node, TargetName, Oids, SendOpts) -> +mgr_user_async_get2(Node, TargetName, Oids, SendOpts) + when is_list(TargetName) -> rcall(Node, snmp_manager_user, async_get2, [TargetName, Oids, SendOpts]). %% mgr_user_sync_get_next(Node, Oids) -> %% mgr_user_sync_get_next(Node, ?LOCALHOST(), ?AGENT_PORT, Oids). -mgr_user_sync_get_next(Node, Addr_or_TargetName, Oids) -> - rcall(Node, snmp_manager_user, sync_get_next, [Addr_or_TargetName, Oids]). +mgr_user_sync_get_next(Node, TargetName, Oids) when is_list(TargetName) -> + rcall(Node, snmp_manager_user, sync_get_next, [TargetName, Oids]). +%% <REMOVED-IN-R16B> mgr_user_sync_get_next(Node, Addr, Port, Oids) -> rcall(Node, snmp_manager_user, sync_get_next, [Addr, Port, Oids]). +%% </REMOVED-IN-R16B> -mgr_user_sync_get_next2(Node, TargetName, Oids, SendOpts) -> +mgr_user_sync_get_next2(Node, TargetName, Oids, SendOpts) + when is_list(TargetName) -> rcall(Node, snmp_manager_user, sync_get_next2, [TargetName, Oids, SendOpts]). %% mgr_user_async_get_next(Node, Oids) -> %% mgr_user_async_get_next(Node, ?LOCALHOST(), ?AGENT_PORT, Oids). -mgr_user_async_get_next(Node, Addr_or_TargetName, Oids) -> - rcall(Node, snmp_manager_user, async_get_next, [Addr_or_TargetName, Oids]). +mgr_user_async_get_next(Node, TargetName, Oids) when is_list(TargetName) -> + rcall(Node, snmp_manager_user, async_get_next, [TargetName, Oids]). +%% <REMOVED-IN-R16B> mgr_user_async_get_next(Node, Addr, Port, Oids) -> rcall(Node, snmp_manager_user, async_get_next, [Addr, Port, Oids]). +%% </REMOVED-IN-R16B> -mgr_user_async_get_next2(Node, TargetName, Oids, SendOpts) -> +mgr_user_async_get_next2(Node, TargetName, Oids, SendOpts) + when is_list(TargetName) -> rcall(Node, snmp_manager_user, async_get_next2, [TargetName, Oids, SendOpts]). %% mgr_user_sync_set(Node, VAV) -> %% mgr_user_sync_set(Node, ?LOCALHOST(), ?AGENT_PORT, VAV). -mgr_user_sync_set(Node, Addr_or_TargetName, VAV) -> - rcall(Node, snmp_manager_user, sync_set, [Addr_or_TargetName, VAV]). +mgr_user_sync_set(Node, TargetName, VAV) when is_list(TargetName) -> + rcall(Node, snmp_manager_user, sync_set, [TargetName, VAV]). +%% <REMOVED-IN-R16B> mgr_user_sync_set(Node, Addr, Port, VAV) -> rcall(Node, snmp_manager_user, sync_set, [Addr, Port, VAV]). +%% </REMOVED-IN-R16B> -mgr_user_sync_set2(Node, TargetName, VAV, SendOpts) -> +mgr_user_sync_set2(Node, TargetName, VAV, SendOpts) when is_list(TargetName) -> rcall(Node, snmp_manager_user, sync_set2, [TargetName, VAV, SendOpts]). %% mgr_user_async_set(Node, VAV) -> %% mgr_user_async_set(Node, ?LOCALHOST(), ?AGENT_PORT, VAV). -mgr_user_async_set(Node, Addr_or_TargetName, VAV) -> - rcall(Node, snmp_manager_user, async_set, [Addr_or_TargetName, VAV]). +mgr_user_async_set(Node, TargetName, VAV) when is_list(TargetName) -> + rcall(Node, snmp_manager_user, async_set, [TargetName, VAV]). +%% <REMOVED-IN-R16B> mgr_user_async_set(Node, Addr, Port, VAV) -> rcall(Node, snmp_manager_user, async_set, [Addr, Port, VAV]). +%% </REMOVED-IN-R16B> -mgr_user_async_set2(Node, TargetName, VAV, SendOpts) -> +mgr_user_async_set2(Node, TargetName, VAV, SendOpts) when is_list(TargetName) -> rcall(Node, snmp_manager_user, async_set2, [TargetName, VAV, SendOpts]). %% mgr_user_sync_get_bulk(Node, NonRep, MaxRep, Oids) -> %% mgr_user_sync_get_bulk(Node, ?LOCALHOST(), ?AGENT_PORT, %% NonRep, MaxRep, Oids). -mgr_user_sync_get_bulk(Node, Addr_or_TargetName, NonRep, MaxRep, Oids) -> +mgr_user_sync_get_bulk(Node, TargetName, NonRep, MaxRep, Oids) + when is_list(TargetName) -> rcall(Node, snmp_manager_user, sync_get_bulk, - [Addr_or_TargetName, NonRep, MaxRep, Oids]). + [TargetName, NonRep, MaxRep, Oids]). +%% <REMOVED-IN-R16B> mgr_user_sync_get_bulk(Node, Addr, Port, NonRep, MaxRep, Oids) -> rcall(Node, snmp_manager_user, sync_get_bulk, [Addr, Port, NonRep, MaxRep, Oids]). +%% </REMOVED-IN-R16B> -mgr_user_sync_get_bulk2(Node, TargetName, NonRep, MaxRep, Oids, SendOpts) -> +mgr_user_sync_get_bulk2(Node, TargetName, NonRep, MaxRep, Oids, SendOpts) + when is_list(TargetName) -> rcall(Node, snmp_manager_user, sync_get_bulk2, [TargetName, NonRep, MaxRep, Oids, SendOpts]). %% mgr_user_async_get_bulk(Node, NonRep, MaxRep, Oids) -> %% mgr_user_async_get_bulk(Node, ?LOCALHOST(), ?AGENT_PORT, %% NonRep, MaxRep, Oids). -mgr_user_async_get_bulk(Node, Addr_or_TargetName, NonRep, MaxRep, Oids) -> +mgr_user_async_get_bulk(Node, TargetName, NonRep, MaxRep, Oids) + when is_list(TargetName) -> rcall(Node, snmp_manager_user, async_get_bulk, - [Addr_or_TargetName, NonRep, MaxRep, Oids]). + [TargetName, NonRep, MaxRep, Oids]). +%% <REMOVED-IN-R16B> mgr_user_async_get_bulk(Node, Addr, Port, NonRep, MaxRep, Oids) -> rcall(Node, snmp_manager_user, async_get_bulk, [Addr, Port, NonRep, MaxRep, Oids]). +%% </REMOVED-IN-R16B> -mgr_user_async_get_bulk2(Node, TargetName, NonRep, MaxRep, Oids, SendOpts) -> +mgr_user_async_get_bulk2(Node, TargetName, NonRep, MaxRep, Oids, SendOpts) + when is_list(TargetName) -> rcall(Node, snmp_manager_user, async_get_bulk2, [TargetName, NonRep, MaxRep, Oids, SendOpts]). diff --git a/lib/snmp/test/snmp_manager_user.erl b/lib/snmp/test/snmp_manager_user.erl index 1b62b04960..4e789bbaec 100644 --- a/lib/snmp/test/snmp_manager_user.erl +++ b/lib/snmp/test/snmp_manager_user.erl @@ -1,7 +1,7 @@ %% %% %CopyrightBegin% %% -%% Copyright Ericsson AB 2005-2011. All Rights Reserved. +%% Copyright Ericsson AB 2005-2012. All Rights Reserved. %% %% The contents of this file are subject to the Erlang Public License, %% Version 1.1, (the "License"); you may not use this file except in @@ -44,20 +44,20 @@ info/0, system_info/0, simulate_crash/1, - register_agent/2, register_agent/3, - unregister_agent/1, unregister_agent/2, - agent_info/2, agent_info/3, - update_agent_info/3, update_agent_info/4, + register_agent/2, + unregister_agent/1, + agent_info/2, + update_agent_info/3, which_all_agents/0, which_own_agents/0, load_mib/1, unload_mib/1, - sync_get/1, sync_get/2, sync_get/3, sync_get2/3, - async_get/1, async_get/2, async_get/3, async_get2/3, - sync_get_next/1, sync_get_next/2, sync_get_next/3, sync_get_next2/3, - async_get_next/1, async_get_next/2, async_get_next/3, async_get_next2/3, - sync_set/1, sync_set/2, sync_set/3, sync_set2/3, - async_set/1, async_set/2, async_set/3, async_set2/3, - sync_get_bulk/3, sync_get_bulk/4, sync_get_bulk/5, sync_get_bulk2/5, - async_get_bulk/3, async_get_bulk/4, async_get_bulk/5, async_get_bulk2/5, + sync_get/1, sync_get/2, sync_get2/3, + async_get/1, async_get/2, async_get2/3, + sync_get_next/1, sync_get_next/2, sync_get_next2/3, + async_get_next/1, async_get_next/2, async_get_next2/3, + sync_set/1, sync_set/2, sync_set2/3, + async_set/1, async_set/2, async_set2/3, + sync_get_bulk/3, sync_get_bulk/4, sync_get_bulk2/5, + async_get_bulk/3, async_get_bulk/4, async_get_bulk2/5, name_to_oid/1, oid_to_name/1, purify_oid/1 ]). @@ -73,15 +73,11 @@ -export([ handle_error/3, - handle_agent/4, + handle_agent/5, handle_pdu/4, handle_trap/3, handle_inform/3, - handle_report/3, - handle_pdu/5, % For backwards compatibillity - handle_trap/4, % For backwards compatibillity - handle_inform/4, % For backwards compatibillity - handle_report/4 % For backwards compatibillity + handle_report/3 ]). @@ -126,27 +122,16 @@ simulate_crash(Reason) -> register_agent(TargetName, Config) when is_list(TargetName) andalso is_list(Config) -> - call({register_agent, TargetName, Config}); -register_agent(Addr, Port) -> - register_agent(Addr, Port, []). - -register_agent(Addr, Port, Conf) -> - call({register_agent, Addr, Port, Conf}). + call({register_agent, TargetName, Config}). unregister_agent(TargetName) -> call({unregister_agent, TargetName}). -unregister_agent(Addr, Port) -> - call({unregister_agent, Addr, Port}). agent_info(TargetName, Item) -> call({agent_info, TargetName, Item}). -agent_info(Addr, Port, Item) -> - call({agent_info, Addr, Port, Item}). update_agent_info(TargetName, Item, Val) -> call({update_agent_info, TargetName, Item, Val}). -update_agent_info(Addr, Port, Item, Val) -> - call({update_agent_info, Addr, Port, Item, Val}). which_all_agents() -> call(which_all_agents). @@ -165,11 +150,8 @@ unload_mib(Mib) -> sync_get(Oids) -> call({sync_get, Oids}). -sync_get(Addr_or_TargetName, Oids) -> - call({sync_get, Addr_or_TargetName, Oids}). - -sync_get(Addr, Port, Oids) -> - call({sync_get, Addr, Port, Oids}). +sync_get(TargetName, Oids) -> + call({sync_get, TargetName, Oids}). sync_get2(TargetName, Oids, SendOpts) -> call({sync_get2, TargetName, Oids, SendOpts}). @@ -180,11 +162,8 @@ sync_get2(TargetName, Oids, SendOpts) -> async_get(Oids) -> call({async_get, Oids}). -async_get(Addr_or_TargetName, Oids) -> - call({async_get, Addr_or_TargetName, Oids}). - -async_get(Addr, Port, Oids) -> - call({async_get, Addr, Port, Oids}). +async_get(TargetName, Oids) -> + call({async_get, TargetName, Oids}). async_get2(TargetName, Oids, SendOpts) -> call({async_get2, TargetName, Oids, SendOpts}). @@ -194,11 +173,8 @@ async_get2(TargetName, Oids, SendOpts) -> sync_get_next(Oids) -> call({sync_get_next, Oids}). -sync_get_next(Addr_or_TargetName, Oids) -> - call({sync_get_next, Addr_or_TargetName, Oids}). - -sync_get_next(Addr, Port, Oids) -> - call({sync_get_next, Addr, Port, Oids}). +sync_get_next(TargetName, Oids) -> + call({sync_get_next, TargetName, Oids}). sync_get_next2(TargetName, Oids, SendOpts) -> call({sync_get_next2, TargetName, Oids, SendOpts}). @@ -208,11 +184,8 @@ sync_get_next2(TargetName, Oids, SendOpts) -> async_get_next(Oids) -> call({async_get_next, Oids}). -async_get_next(Addr_or_TargetName, Oids) -> - call({async_get_next, Addr_or_TargetName, Oids}). - -async_get_next(Addr, Port, Oids) -> - call({async_get_next, Addr, Port, Oids}). +async_get_next(TargetName, Oids) -> + call({async_get_next, TargetName, Oids}). async_get_next2(TargetName, Oids, SendOpts) -> call({async_get_next2, TargetName, Oids, SendOpts}). @@ -222,11 +195,8 @@ async_get_next2(TargetName, Oids, SendOpts) -> sync_set(VAV) -> call({sync_set, VAV}). -sync_set(Addr_or_TargetName, VAV) -> - call({sync_set, Addr_or_TargetName, VAV}). - -sync_set(Addr, Port, VAV) -> - call({sync_set, Addr, Port, VAV}). +sync_set(TargetName, VAV) -> + call({sync_set, TargetName, VAV}). sync_set2(TargetName, VAV, SendOpts) -> call({sync_set2, TargetName, VAV, SendOpts}). @@ -236,11 +206,8 @@ sync_set2(TargetName, VAV, SendOpts) -> async_set(VAV) -> call({async_set, VAV}). -async_set(Addr_or_TargetName, VAV) -> - call({async_set, Addr_or_TargetName, VAV}). - -async_set(Addr, Port, VAV) -> - call({async_set, Addr, Port, VAV}). +async_set(TargetName, VAV) -> + call({async_set, TargetName, VAV}). async_set2(TargetName, VAV, SendOpts) -> call({async_set2, TargetName, VAV, SendOpts}). @@ -250,11 +217,8 @@ async_set2(TargetName, VAV, SendOpts) -> sync_get_bulk(NonRep, MaxRep, Oids) -> call({sync_get_bulk, NonRep, MaxRep, Oids}). -sync_get_bulk(Addr_or_TargetName, NonRep, MaxRep, Oids) -> - call({sync_get_bulk, Addr_or_TargetName, NonRep, MaxRep, Oids}). - -sync_get_bulk(Addr, Port, NonRep, MaxRep, Oids) -> - call({sync_get_bulk, Addr, Port, NonRep, MaxRep, Oids}). +sync_get_bulk(TargetName, NonRep, MaxRep, Oids) -> + call({sync_get_bulk, TargetName, NonRep, MaxRep, Oids}). sync_get_bulk2(TargetName, NonRep, MaxRep, Oids, SendOpts) -> call({sync_get_bulk2, TargetName, NonRep, MaxRep, Oids, SendOpts}). @@ -264,11 +228,8 @@ sync_get_bulk2(TargetName, NonRep, MaxRep, Oids, SendOpts) -> async_get_bulk(NonRep, MaxRep, Oids) -> call({async_get_bulk, NonRep, MaxRep, Oids}). -async_get_bulk(Addr_or_TargetName, NonRep, MaxRep, Oids) -> - call({async_get_bulk, Addr_or_TargetName, NonRep, MaxRep, Oids}). - -async_get_bulk(Addr, Port, NonRep, MaxRep, Oids) -> - call({async_get_bulk, Addr, Port, NonRep, MaxRep, Oids}). +async_get_bulk(TargetName, NonRep, MaxRep, Oids) -> + call({async_get_bulk, TargetName, NonRep, MaxRep, Oids}). async_get_bulk2(TargetName, NonRep, MaxRep, Oids, SendOpts) -> call({async_get_bulk2, TargetName, NonRep, MaxRep, Oids, SendOpts}). @@ -340,24 +301,12 @@ loop(#state{parent = Parent, id = Id} = S) -> reply(From, Res, Ref), loop(S); - {{register_agent, Addr, Port, Conf}, From, Ref} -> - d("loop -> received register_agent request"), - Res = snmpm:register_agent(Id, Addr, Port, Conf), - reply(From, Res, Ref), - loop(S); - {{unregister_agent, TargetName}, From, Ref} -> d("loop -> received unregister_agent request"), Res = snmpm:unregister_agent(Id, TargetName), reply(From, Res, Ref), loop(S); - {{unregister_agent, Addr, Port}, From, Ref} -> - d("loop -> received unregister_agent request"), - Res = snmpm:unregister_agent(Id, Addr, Port), - reply(From, Res, Ref), - loop(S); - {{agent_info, TargetName, Item}, From, Ref} -> d("loop -> received agent_info request with" "~n TargetName: ~p" @@ -368,15 +317,6 @@ loop(#state{parent = Parent, id = Id} = S) -> reply(From, Res, Ref), loop(S); - {{agent_info, Addr, Port, Item}, From, Ref} -> - d("loop -> received agent_info request with" - "~n Addr: ~p" - "~n Port: ~p" - "~n Item: ~p", [Addr, Port, Item]), - Res = snmpm:agent_info(Addr, Port, Item), - reply(From, Res, Ref), - loop(S); - {{update_agent_info, TargetName, Item, Val}, From, Ref} -> d("loop -> received update_agent_info request with" "~n TargetName: ~p" @@ -386,16 +326,6 @@ loop(#state{parent = Parent, id = Id} = S) -> reply(From, Res, Ref), loop(S); - {{update_agent_info, Addr, Port, Item, Val}, From, Ref} -> - d("loop -> received update_agent_info request with" - "~n Addr: ~p" - "~n Port: ~p" - "~n Item: ~p" - "~n Val: ~p", [Addr, Port, Item, Val]), - Res = snmpm:update_agent_info(Id, Addr, Port, Item, Val), - reply(From, Res, Ref), - loop(S); - {which_all_agents, From, Ref} -> d("loop -> received which_all_agents request"), Res = snmpm:which_agents(), @@ -452,23 +382,6 @@ loop(#state{parent = Parent, id = Id} = S) -> reply(From, Res, Ref), loop(S); - {{sync_get, Addr, Oids}, From, Ref} -> - d("loop -> received sync_get request with" - "~n Addr: ~p" - "~n Oids: ~p", [Addr, Oids]), - Res = snmpm:g(Id, Addr, Oids), - reply(From, Res, Ref), - loop(S); - - {{sync_get, Addr, Port, Oids}, From, Ref} -> - d("loop -> received sync_get request with" - "~n Addr: ~p" - "~n Port: ~p" - "~n Oids: ~p", [Addr, Port, Oids]), - Res = snmpm:g(Id, Addr, Port, Oids), - reply(From, Res, Ref), - loop(S); - %% %% -- (async) get-request -- @@ -500,18 +413,6 @@ loop(#state{parent = Parent, id = Id} = S) -> reply(From, Res, Ref), loop(S); - {{async_get, Addr, Oids}, From, Ref} -> - d("loop -> received async_get request"), - Res = snmpm:ag(Id, Addr, Oids), - reply(From, Res, Ref), - loop(S); - - {{async_get, Addr, Port, Oids}, From, Ref} -> - d("loop -> received async_get request"), - Res = snmpm:ag(Id, Addr, Port, Oids), - reply(From, Res, Ref), - loop(S); - %% %% -- (sync) get_next-request -- @@ -543,18 +444,6 @@ loop(#state{parent = Parent, id = Id} = S) -> reply(From, Res, Ref), loop(S); - {{sync_get_next, Addr, Oids}, From, Ref} -> - d("loop -> received sync_get_next request"), - Res = snmpm:gn(Id, Addr, Oids), - reply(From, Res, Ref), - loop(S); - - {{sync_get_next, Addr, Port, Oids}, From, Ref} -> - d("loop -> received sync_get_next request"), - Res = snmpm:gn(Id, Addr, Port, Oids), - reply(From, Res, Ref), - loop(S); - %% %% -- (async) get_next-request -- @@ -586,18 +475,6 @@ loop(#state{parent = Parent, id = Id} = S) -> reply(From, Res, Ref), loop(S); - {{async_get_next, Addr, Oids}, From, Ref} -> - d("loop -> received async_get_next request"), - Res = snmpm:agn(Id, Addr, Oids), - reply(From, Res, Ref), - loop(S); - - {{async_get_next, Addr, Port, Oids}, From, Ref} -> - d("loop -> received async_get_next request"), - Res = snmpm:agn(Id, Addr, Port, Oids), - reply(From, Res, Ref), - loop(S); - %% %% -- (sync) set-request -- @@ -626,18 +503,6 @@ loop(#state{parent = Parent, id = Id} = S) -> reply(From, Res, Ref), loop(S); - {{sync_set, Addr, VAV}, From, Ref} -> - d("loop -> received sync_set request"), - Res = snmpm:s(Id, Addr, VAV), - reply(From, Res, Ref), - loop(S); - - {{sync_set, Addr, Port, VAV}, From, Ref} -> - d("loop -> received sync_set request"), - Res = snmpm:s(Id, Addr, Port, VAV), - reply(From, Res, Ref), - loop(S); - %% %% -- (async) set-request -- @@ -666,18 +531,6 @@ loop(#state{parent = Parent, id = Id} = S) -> reply(From, Res, Ref), loop(S); - {{async_set, Addr, VAV}, From, Ref} -> - d("loop -> received async_set request"), - Res = snmpm:as(Id, Addr, VAV), - reply(From, Res, Ref), - loop(S); - - {{async_set, Addr, Port, VAV}, From, Ref} -> - d("loop -> received async_set request"), - Res = snmpm:as(Id, Addr, Port, VAV), - reply(From, Res, Ref), - loop(S); - %% %% -- (sync) get-bulk-request -- @@ -718,27 +571,6 @@ loop(#state{parent = Parent, id = Id} = S) -> reply(From, Res, Ref), loop(S); - {{sync_get_bulk, Addr, NonRep, MaxRep, Oids}, From, Ref} -> - d("loop -> received sync_get_bulk request with" - "~n Addr: ~p" - "~n NonRep: ~w" - "~n MaxRep: ~w" - "~n Oids: ~p", [Addr, NonRep, MaxRep, Oids]), - Res = snmpm:gb(Id, Addr, NonRep, MaxRep, Oids), - reply(From, Res, Ref), - loop(S); - - {{sync_get_bulk, Addr, Port, NonRep, MaxRep, Oids}, From, Ref} -> - d("loop -> received sync_get_bulk request with" - "~n Addr: ~p" - "~n Port: ~w" - "~n NonRep: ~w" - "~n MaxRep: ~w" - "~n Oids: ~p", [Addr, Port, NonRep, MaxRep, Oids]), - Res = snmpm:gb(Id, Addr, Port, NonRep, MaxRep, Oids), - reply(From, Res, Ref), - loop(S); - %% %% -- (async) get-bulk-request -- @@ -779,27 +611,6 @@ loop(#state{parent = Parent, id = Id} = S) -> reply(From, Res, Ref), loop(S); - {{async_get_bulk, Addr, NonRep, MaxRep, Oids}, From, Ref} -> - d("loop -> received async_get_bulk request with" - "~n Addr: ~p" - "~n NonRep: ~w" - "~n MaxRep: ~w" - "~n Oids: ~p", [Addr, NonRep, MaxRep, Oids]), - Res = snmpm:agb(Id, Addr, NonRep, MaxRep, Oids), - reply(From, Res, Ref), - loop(S); - - {{async_get_bulk, Addr, Port, NonRep, MaxRep, Oids}, From, Ref} -> - d("loop -> received async_get_bulk request with" - "~n Addr: ~p" - "~n Port: ~w" - "~n NonRep: ~w" - "~n MaxRep: ~w" - "~n Oids: ~p", [Addr, Port, NonRep, MaxRep, Oids]), - Res = snmpm:agb(Id, Addr, Port, NonRep, MaxRep, Oids), - reply(From, Res, Ref), - loop(S); - %% %% -- logical name translation -- @@ -846,12 +657,6 @@ loop(#state{parent = Parent, id = Id} = S) -> Parent ! {async_event, ReqId, {pdu, SnmpResponse}}, loop(S); - %% For backwards compatibillity - {handle_pdu, _Pid, _Addr, _Port, ReqId, SnmpResponse} -> - d("loop -> received pdu callback from manager for ~w", [ReqId]), - Parent ! {async_event, ReqId, {pdu, SnmpResponse}}, - loop(S); - {handle_trap, _Pid, TargetName, SnmpTrap} -> d("loop -> received trap callback from manager for " "~n ~p", @@ -860,15 +665,6 @@ loop(#state{parent = Parent, id = Id} = S) -> Parent ! {async_event, TargetName, {trap, SnmpTrap}}, loop(S); - %% For backwards compatibillity - {handle_trap, _Pid, Addr, Port, SnmpTrap} -> - d("loop -> received trap callback from manager for " - "~n ~p:~w", - "~n ~p", - [Addr, Port, SnmpTrap]), - Parent ! {async_event, {Addr, Port}, {trap, SnmpTrap}}, - loop(S); - {handle_inform, Pid, TargetName, SnmpInform} -> d("loop -> received inform callback from manager for " "~n ~p", @@ -877,15 +673,6 @@ loop(#state{parent = Parent, id = Id} = S) -> Parent ! {async_event, TargetName, {inform, Pid, SnmpInform}}, loop(S); - %% For backwards compatibillity - {handle_inform, Pid, Addr, Port, SnmpInform} -> - d("loop -> received inform callback from manager for " - "~n ~p:~w", - "~n ~p", - [Addr, Port, SnmpInform]), - Parent ! {async_event, {Addr, Port}, {inform, Pid, SnmpInform}}, - loop(S); - {handle_report, _Pid, TargetName, SnmpReport} -> d("loop -> received report callback from manager for " "~n ~p", @@ -894,15 +681,6 @@ loop(#state{parent = Parent, id = Id} = S) -> Parent ! {async_event, TargetName, {report, SnmpReport}}, loop(S); - %% For backwards compatibillity - {handle_report, _Pid, Addr, Port, SnmpReport} -> - d("loop -> received report callback from manager for " - "~n ~p:~w", - "~n ~p", - [Addr, Port, SnmpReport]), - Parent ! {async_event, {Addr, Port}, {report, SnmpReport}}, - loop(S); - {'EXIT', Parent, Reason} -> d("received exit signal from parent: ~n~p", [Reason]), info("received exit signal from parent: ~n~p", [Reason]), @@ -981,8 +759,8 @@ handle_error(ReqId, Reason, UserPid) -> ignore. -handle_agent(Addr, Port, SnmpInfo, UserPid) -> - UserPid ! {handle_agent, self(), Addr, Port, SnmpInfo}, +handle_agent(Addr, Port, SnmpInfo, UserPid, UserData) -> + UserPid ! {handle_agent, self(), Addr, Port, SnmpInfo, UserData}, ignore. @@ -990,22 +768,10 @@ handle_pdu(TargetName, ReqId, SnmpResponse, UserPid) -> UserPid ! {handle_pdu, self(), TargetName, ReqId, SnmpResponse}, ignore. -%% For backwards compatibillity -handle_pdu(Addr, Port, ReqId, SnmpResponse, UserPid) -> - UserPid ! {handle_pdu, self(), Addr, Port, ReqId, SnmpResponse}, - ignore. - - handle_trap(TargetName, SnmpTrap, UserPid) -> UserPid ! {handle_trap, self(), TargetName, SnmpTrap}, ok. -%% For backwards compatibillity -handle_trap(Addr, Port, SnmpTrap, UserPid) -> - UserPid ! {handle_trap, self(), Addr, Port, SnmpTrap}, - ok. - - handle_inform(TargetName, SnmpInform, UserPid) -> UserPid ! {handle_inform, self(), TargetName, SnmpInform}, receive @@ -1015,26 +781,10 @@ handle_inform(TargetName, SnmpInform, UserPid) -> ok end. -%% For backwards compatibillity -handle_inform(Addr, Port, SnmpInform, UserPid) -> - UserPid ! {handle_inform, self(), Addr, Port, SnmpInform}, - receive - {handle_inform_no_response, {Addr, Port}} -> - no_reply; - {handle_inform_response, {Addr, Port}} -> - ok - end. - - handle_report(TargetName, SnmpReport, UserPid) -> UserPid ! {handle_report, self(), TargetName, SnmpReport}, ok. -%% For backwards compatibillity -handle_report(Addr, Port, SnmpReport, UserPid) -> - UserPid ! {handle_report, self(), Addr, Port, SnmpReport}, - ok. - %%---------------------------------------------------------------------- %% Debug diff --git a/lib/snmp/test/snmp_manager_user_old.erl b/lib/snmp/test/snmp_manager_user_old.erl index 6280cef51f..9f951bf64d 100644 --- a/lib/snmp/test/snmp_manager_user_old.erl +++ b/lib/snmp/test/snmp_manager_user_old.erl @@ -107,10 +107,20 @@ unregister_agent(Addr, Port) -> snmpm:unregister_agent(?USER_ID, Addr, Port). agent_info(Addr, Port, Item) -> - snmpm:agent_info(?USER_ID, Addr, Port, Item). + case snmpm:target_name(Addr, Port) of + {ok, TargetName} -> + snmpm:agent_info(TargetName, Item); + Error -> + Error + end. update_agent_info(Addr, Port, Item, Val) -> - snmpm:update_agent_info(?USER_ID, Addr, Port, Item, Val). + case snmpm:target_name(Addr, Port) of + {ok, TargetName} -> + snmpm:update_agent_info(?USER_ID, TargetName, Item, Val); + Error -> + Error + end. which_agents() -> snmpm:which_agents(). @@ -128,73 +138,153 @@ unload_mib(Mib) -> %% -- sync_get(Addr, Oids) -> - snmpm:g(?USER_ID, Addr, Oids). + case snmpm:target_name(Addr) of + {ok, TargetName} -> + snmpm:sync_get2(?USER_ID, TargetName, Oids); + Error -> + Error + end. sync_get(Addr, Port, Oids) -> - snmpm:g(?USER_ID, Addr, Port, Oids). + case snmpm:target_name(Addr, Port) of + {ok, TargetName} -> + snmpm:sync_get2(?USER_ID, TargetName, Oids); + Error -> + Error + end. %% -- async_get(Addr, Oids) -> - snmpm:ag(?USER_ID, Addr, Oids). + case snmpm:target_name(Addr) of + {ok, TargetName} -> + snmpm:async_get2(?USER_ID, TargetName, Oids); + Error -> + Error + end. async_get(Addr, Port, Oids) -> - snmpm:ag(?USER_ID, Addr, Port, Oids). + case snmpm:target_name(Addr, Port) of + {ok, TargetName} -> + snmpm:async_get2(?USER_ID, TargetName, Oids); + Error -> + Error + end. %% -- sync_get_next(Addr, Oids) -> - snmpm:gn(?USER_ID, Addr, Oids). + case snmpm:target_name(Addr) of + {ok, TargetName} -> + snmpm:sync_get_next2(?USER_ID, TargetName, Oids); + Error -> + Error + end. sync_get_next(Addr, Port, Oids) -> - snmpm:gn(?USER_ID, Addr, Port, Oids). + case snmpm:target_name(Addr, Port) of + {ok, TargetName} -> + snmpm:sync_get_next2(?USER_ID, TargetName, Oids); + Error -> + Error + end. %% -- async_get_next(Addr, Oids) -> - snmpm:agn(?USER_ID, Addr, Oids). + case snmpm:target_name(Addr) of + {ok, TargetName} -> + snmpm:async_get_next2(?USER_ID, TargetName, Oids); + Error -> + Error + end. async_get_next(Addr, Port, Oids) -> - snmpm:agn(?USER_ID, Addr, Port, Oids). + case snmpm:target_name(Addr, Port) of + {ok, TargetName} -> + snmpm:async_get_next2(?USER_ID, TargetName, Oids); + Error -> + Error + end. %% -- sync_set(Addr, VAV) -> - snmpm:s(?USER_ID, Addr, VAV). + case snmpm:target_name(Addr) of + {ok, TargetName} -> + snmpm:sync_set2(?USER_ID, TargetName, VAV); + Error -> + Error + end. sync_set(Addr, Port, VAV) -> - snmpm:s(?USER_ID, Addr, Port, VAV). + case snmpm:target_name(Addr, Port) of + {ok, TargetName} -> + snmpm:sync_set2(?USER_ID, TargetName, VAV); + Error -> + Error + end. %% -- async_set(Addr, VAV) -> - snmpm:as(?USER_ID, Addr, VAV). + case snmpm:target_name(Addr) of + {ok, TargetName} -> + snmpm:async_set2(?USER_ID, TargetName, VAV); + Error -> + Error + end. async_set(Addr, Port, VAV) -> - snmpm:as(?USER_ID, Addr, Port, VAV). + case snmpm:target_name(Addr, Port) of + {ok, TargetName} -> + snmpm:async_set2(?USER_ID, TargetName, VAV); + Error -> + Error + end. %% -- sync_get_bulk(Addr, NonRep, MaxRep, Oids) -> - snmpm:gb(?USER_ID, Addr, NonRep, MaxRep, Oids). + case snmpm:target_name(Addr) of + {ok, TargetName} -> + snmpm:sync_get_bulk2(?USER_ID, TargetName, NonRep, MaxRep, Oids); + Error -> + Error + end. sync_get_bulk(Addr, Port, NonRep, MaxRep, Oids) -> - snmpm:gb(?USER_ID, Addr, Port, NonRep, MaxRep, Oids). + case snmpm:target_name(Addr, Port) of + {ok, TargetName} -> + snmpm:sync_get_bulk2(?USER_ID, TargetName, NonRep, MaxRep, Oids); + Error -> + Error + end. %% -- async_get_bulk(Addr, NonRep, MaxRep, Oids) -> - snmpm:agb(?USER_ID, Addr, NonRep, MaxRep, Oids). + case snmpm:target_name(Addr) of + {ok, TargetName} -> + snmpm:async_get_bulk2(?USER_ID, TargetName, NonRep, MaxRep, Oids); + Error -> + Error + end. async_get_bulk(Addr, Port, NonRep, MaxRep, Oids) -> - snmpm:agb(?USER_ID, Addr, Port, NonRep, MaxRep, Oids). + case snmpm:target_name(Addr, Port) of + {ok, TargetName} -> + snmpm:async_get_bulk2(?USER_ID, TargetName, NonRep, MaxRep, Oids); + Error -> + Error + end. %% -- diff --git a/lib/snmp/test/snmp_manager_user_test.erl b/lib/snmp/test/snmp_manager_user_test.erl index fefa1ad713..41d5c50b19 100644 --- a/lib/snmp/test/snmp_manager_user_test.erl +++ b/lib/snmp/test/snmp_manager_user_test.erl @@ -36,7 +36,7 @@ %% -compile(export_all). -export([ -all/0,groups/0,init_per_group/2,end_per_group/2, + all/0,groups/0,init_per_group/2,end_per_group/2, init_per_testcase/2, end_per_testcase/2, diff --git a/lib/snmp/test/snmp_test_lib.erl b/lib/snmp/test/snmp_test_lib.erl index e327456bc4..f0abae73e8 100644 --- a/lib/snmp/test/snmp_test_lib.erl +++ b/lib/snmp/test/snmp_test_lib.erl @@ -324,7 +324,7 @@ fail(Reason, Mod, Line) -> skip(Reason, Module, Line) -> String = lists:flatten(io_lib:format("Skipping ~p(~p): ~p~n", [Module, Line, Reason])), - exit({skipped, String}). + exit({skip, String}). %% ---------------------------------------------------------------- diff --git a/lib/snmp/test/snmp_test_manager.erl b/lib/snmp/test/snmp_test_manager.erl index 4cc6d36acc..0df2350d58 100644 --- a/lib/snmp/test/snmp_test_manager.erl +++ b/lib/snmp/test/snmp_test_manager.erl @@ -43,7 +43,7 @@ %% Manager callback API: -export([ handle_error/3, - handle_agent/4, + handle_agent/5, handle_pdu/4, handle_trap/3, handle_inform/3, @@ -239,7 +239,7 @@ handle_info({snmp_error, ReqId, Reason}, P ! {snmp_error, ReqId, Reason}, {noreply, State}; -handle_info({snmp_agent, Addr, Port, Info, Pid}, +handle_info({snmp_agent, Addr, Port, Info, Pid, _UserData}, #state{parent = P} = State) -> error_msg("detected new agent: " "~n Addr: ~w" @@ -326,8 +326,8 @@ handle_error(ReqId, Reason, Pid) -> ignore. -handle_agent(Addr, Port, SnmpInfo, Pid) -> - Pid ! {snmp_agent, Addr, Port, SnmpInfo, self()}, +handle_agent(Addr, Port, SnmpInfo, Pid, UserData) -> + Pid ! {snmp_agent, Addr, Port, SnmpInfo, self(), UserData}, receive {snmp_agent_reply, Reply, Pid} -> Reply diff --git a/lib/snmp/vsn.mk b/lib/snmp/vsn.mk index b90dbe4eef..8145e415f3 100644 --- a/lib/snmp/vsn.mk +++ b/lib/snmp/vsn.mk @@ -18,6 +18,6 @@ # %CopyrightEnd% APPLICATION = snmp -SNMP_VSN = 4.22.1 +SNMP_VSN = 4.23 PRE_VSN = APP_VSN = "$(APPLICATION)-$(SNMP_VSN)$(PRE_VSN)" diff --git a/lib/ssh/doc/html/SSH_protocols.png b/lib/ssh/doc/html/SSH_protocols.png Binary files differnew file mode 100644 index 0000000000..145c96c4cd --- /dev/null +++ b/lib/ssh/doc/html/SSH_protocols.png diff --git a/lib/ssh/doc/man6/.gitignore b/lib/ssh/doc/man6/.gitignore new file mode 100644 index 0000000000..e69de29bb2 --- /dev/null +++ b/lib/ssh/doc/man6/.gitignore diff --git a/lib/ssh/doc/src/Makefile b/lib/ssh/doc/src/Makefile index da99c4ea0f..0e79d9979f 100644 --- a/lib/ssh/doc/src/Makefile +++ b/lib/ssh/doc/src/Makefile @@ -37,21 +37,30 @@ RELSYSDIR = $(RELEASE_PATH)/lib/$(APPLICATION)-$(VSN) # Target Specs # ---------------------------------------------------- XML_APPLICATION_FILES = ref_man.xml -XML_REF3_FILES = \ - ssh.xml \ +XML_REF3_FILES = ssh.xml \ ssh_channel.xml \ - ssh_connection.xml\ + ssh_connection.xml \ + ssh_client_key_api.xml \ + ssh_server_key_api.xml \ ssh_sftp.xml \ ssh_sftpd.xml \ -XML_PART_FILES = part_notes.xml -XML_CHAPTER_FILES = notes.xml +XML_REF6_FILES = ssh_app.xml + +XML_PART_FILES = part_notes.xml \ + usersguide.xml +XML_CHAPTER_FILES = notes.xml \ + introduction.xml \ + ssh_protocol.xml \ + using_ssh.xml BOOK_FILES = book.xml -XML_FILES = $(BOOK_FILES) $(XML_APPLICATION_FILES) $(XML_REF3_FILES) \ +XML_FILES = $(BOOK_FILES) $(XML_APPLICATION_FILES) $(XML_REF3_FILES) $(XML_REF6_FILES)\ $(XML_PART_FILES) $(XML_CHAPTER_FILES) +IMAGE_FILES = SSH_protocols.png + # ---------------------------------------------------- HTML_FILES = $(XML_APPLICATION_FILES:%.xml=$(HTMLDIR)/%.html) \ @@ -62,10 +71,12 @@ EXTRA_FILES = \ $(DEFAULT_GIF_FILES) \ $(DEFAULT_HTML_FILES) \ $(XML_REF3_FILES:%.xml=$(HTMLDIR)/%.html) \ + $(XML_REF6_FILES:%.xml=$(HTMLDIR)/%.html) \ $(XML_CHAPTER_FILES:%.xml=$(HTMLDIR)/%.html) MAN3_FILES = $(XML_REF3_FILES:%.xml=$(MAN3DIR)/%.3) +MAN6_FILES = $(XML_REF6_FILES:%_app.xml=$(MAN6DIR)/%.6) HTML_REF_MAN_FILE = $(HTMLDIR)/index.html @@ -80,16 +91,19 @@ DVIPS_FLAGS += # ---------------------------------------------------- # Targets # ---------------------------------------------------- -$(HTMLDIR)/%.gif: %.gif +$(HTMLDIR)/%.png: %.png $(INSTALL_DATA) $< $@ docs: pdf html man $(TOP_PDF_FILE): $(XML_FILES) +images: $(IMAGE_FILES:%=$(HTMLDIR)/%) + pdf: $(TOP_PDF_FILE) -html: $(HTML_REF_MAN_FILE) +html: images $(HTML_REF_MAN_FILE) + clean clean_docs: rm -rf $(HTMLDIR)/* @@ -97,7 +111,7 @@ clean clean_docs: rm -f $(TOP_PDF_FILE) $(TOP_PDF_FILE:%.pdf=%.fo) rm -f errs core *~ -man: $(MAN3_FILES) +man: $(MAN3_FILES) $(MAN6_FILES) debug opt: @@ -117,5 +131,8 @@ release_docs_spec: docs $(INSTALL_DATA) $(INFO_FILE) "$(RELSYSDIR)" $(INSTALL_DIR) "$(RELEASE_PATH)/man/man3" $(INSTALL_DATA) $(MAN3DIR)/* "$(RELEASE_PATH)/man/man3" + $(INSTALL_DIR) "$(RELEASE_PATH)/man/man6" + $(INSTALL_DATA) $(MAN6_FILES) "$(RELEASE_PATH)/man/man6" + release_spec: diff --git a/lib/ssh/doc/src/SSH_protocols.png b/lib/ssh/doc/src/SSH_protocols.png Binary files differnew file mode 100644 index 0000000000..145c96c4cd --- /dev/null +++ b/lib/ssh/doc/src/SSH_protocols.png diff --git a/lib/ssh/doc/src/book.xml b/lib/ssh/doc/src/book.xml index fcec1d6f70..3c2375f96d 100644 --- a/lib/ssh/doc/src/book.xml +++ b/lib/ssh/doc/src/book.xml @@ -1,10 +1,10 @@ -<?xml version="1.0" encoding="latin1" ?> +<?xml version="1.0" encoding="iso-8859-1" ?> <!DOCTYPE book SYSTEM "book.dtd"> <book xmlns:xi="http://www.w3.org/2001/XInclude"> <header titlestyle="normal"> <copyright> - <year>2005</year><year>2010</year> + <year>2005</year><year>2012</year> <holder>Ericsson AB. All Rights Reserved.</holder> </copyright> <legalnotice> @@ -34,6 +34,9 @@ <preamble> <contents level="2"></contents> </preamble> + <parts lift="yes"> + <xi:include href="usersguide.xml"/> + </parts> <applications> <xi:include href="ref_man.xml"/> </applications> diff --git a/lib/ssh/doc/src/fascicules.xml b/lib/ssh/doc/src/fascicules.xml index 43090b4aed..069d9002e0 100644 --- a/lib/ssh/doc/src/fascicules.xml +++ b/lib/ssh/doc/src/fascicules.xml @@ -1,7 +1,10 @@ -<?xml version="1.0" encoding="latin1" ?> +<?xml version="1.0" encoding="iso-8859-1" ?> <!DOCTYPE fascicules SYSTEM "fascicules.dtd"> <fascicules> + <fascicule file="usersguide" href="usersguide_frame.html" entry="no"> + User's Guide + </fascicule> <fascicule file="ref_man" href="ref_man_frame.html" entry="yes"> Reference Manual </fascicule> diff --git a/lib/ssh/doc/src/introduction.xml b/lib/ssh/doc/src/introduction.xml new file mode 100644 index 0000000000..aac8de0f76 --- /dev/null +++ b/lib/ssh/doc/src/introduction.xml @@ -0,0 +1,54 @@ +<?xml version="1.0" encoding="iso-8859-1" ?> +<!DOCTYPE chapter SYSTEM "chapter.dtd"> + +<chapter> + <header> + <copyright> + <year>2012</year> + <holder>Ericsson AB, All Rights Reserved</holder> + </copyright> + <legalnotice> + The contents of this file are subject to the Erlang Public License, + Version 1.1, (the "License"); you may not use this file except in + compliance with the License. You should have received a copy of the + Erlang Public License along with this software. If not, it can be + retrieved online at http://www.erlang.org/. + + Software distributed under the License is distributed on an "AS IS" + basis, WITHOUT WARRANTY OF ANY KIND, either express or implied. See + the License for the specific language governing rights and limitations + under the License. + + The Initial Developer of the Original Code is Ericsson AB. + </legalnotice> + + <title>Introduction</title> + <prepared>OTP team</prepared> + <file>introduction.xml</file> + </header> + + <section> + <title>Purpose</title> + + <p>Secure Shell (SSH) is a protocol for secure remote login and + other secure network services over an insecure network. SSH + provides a single, full-duplex, byte-oriented connection between + client and server. The protocol also provides privacy, integrity, + server authentication and man-in-the-middle protection.</p> + + <p>The Erlang SSH application is an implementation of the SSH + protocol in Erlang which offers API functions to write customized + SSH clients and servers as well as making the Erlang shell + available via SSH. Also included in the SSH application are an + SFTP (SSH File Transfer Protocol) client <seealso + marker="ssh_sftp">ssh_sftp</seealso> and server <seealso + marker="ssh_sftp">ssh_sftpd</seealso>.</p> + </section> + + <section> + <title>Prerequisites</title> + <p>It is assumed that the reader is familiar with the concepts of <seealso marker="doc/design_principals:users_guide">OTP</seealso> + and has a basic understanding of <url href="http://en.wikipedia.org/wiki/Public-key_cryptography">public keys</url>.</p> + </section> + +</chapter> diff --git a/lib/ssh/doc/src/notes.xml b/lib/ssh/doc/src/notes.xml index 36010da07a..e58d4e2c36 100644 --- a/lib/ssh/doc/src/notes.xml +++ b/lib/ssh/doc/src/notes.xml @@ -1,4 +1,4 @@ -<?xml version="1.0" encoding="latin1" ?> +<?xml version="1.0" encoding="iso-8859-1" ?> <!DOCTYPE chapter SYSTEM "chapter.dtd"> <chapter> @@ -29,6 +29,50 @@ <file>notes.xml</file> </header> +<section><title>Ssh 2.1.2</title> + + <section><title>Fixed Bugs and Malfunctions</title> + <list> + <item> + <p> + SSH quiet mode</p> + <p> + A new option to ssh:connect/3,4, quiet_mode. If true, the + client will not print out anything on authorization.</p> + <p> + Own Id: OTP-10429 Aux Id: kunagi-273 [184] </p> + </item> + <item> + <p> + Restrict which key algorithms to use</p> + <p> + A new option to ssh:connect/3,4 is introduced, + public_key_algs, where you can restrict which key + algorithms to use and in which order to try them.</p> + <p> + Own Id: OTP-10498 Aux Id: kunagi-289 [200] </p> + </item> + <item> + <p> + Confidentiality of client password</p> + <p> + Unsets clients password after authentication.</p> + <p> + Own Id: OTP-10511 Aux Id: kunagi-292 [203] </p> + </item> + <item> + <p> + Fixed user interaction for SSH</p> + <p> + It's now available to accept hosts and input password</p> + <p> + Own Id: OTP-10513 Aux Id: kunagi-293 [204] </p> + </item> + </list> + </section> + +</section> + <section><title>Ssh 2.1.1</title> <section><title>Fixed Bugs and Malfunctions</title> @@ -736,7 +780,7 @@ <list> <item> <p> - Ssh timeouts will now behave as expected e.i. defaults to + Ssh timeouts will now behave as expected i.e. defaults to infinity only the user of the ssh application can know of a reasonable timeout value for their application.</p> <p> @@ -789,7 +833,7 @@ caused the ssh daemon to close the connections to all currently logged in users if one user logged out. Another problem related to the supervision tree caused the closing - down of clients to leak processes e.i. all processes was + down of clients to leak processes i.e. all processes was not shutdown correctly.</p> <p> Own Id: OTP-7676</p> @@ -881,7 +925,7 @@ slightly changes the options to the API function ssh:daemon/[1,2,3] deprecating all no longer documented options. Note that the new API enforces the "logical way" - of using the old API e.i. making the subsystem process + of using the old API i.e. making the subsystem process part of the ssh applications supervisor tree, so missuses of the old API are not compatible with the new API.</p> <p> diff --git a/lib/ssh/doc/src/ref_man.xml b/lib/ssh/doc/src/ref_man.xml index 9ab56b28ec..88203b5034 100644 --- a/lib/ssh/doc/src/ref_man.xml +++ b/lib/ssh/doc/src/ref_man.xml @@ -1,10 +1,10 @@ -<?xml version="1.0" encoding="latin1" ?> +<?xml version="1.0" encoding="iso-8859-1" ?> <!DOCTYPE application SYSTEM "application.dtd"> <application xmlns:xi="http://www.w3.org/2001/XInclude"> <header> <copyright> - <year>2004</year><year>2010</year> + <year>2004</year><year>2012</year> <holder>Ericsson AB. All Rights Reserved.</holder> </copyright> <legalnotice> @@ -22,19 +22,22 @@ </legalnotice> <title>SSH Reference Manual</title> - <prepared>Jakob Cederlund</prepared> - <docno></docno> + <prepared>OTP</prepared> <date>2007-10-06</date> <rev>%VSN%</rev> - <file>application.sgml</file> + <file>ref_man.xml</file> </header> <description> <p>The SSH application is an erlang implementation of the - secure shell protocol.</p> + secure shell protocol (SSH) as defined by RFC 4250 - 4254</p> + </description> + <xi:include href="ssh_app.xml"/> <xi:include href="ssh.xml"/> <xi:include href="ssh_channel.xml"/> <xi:include href="ssh_connection.xml"/> + <xi:include href="ssh_client_key_api.xml"/> + <xi:include href="ssh_server_key_api.xml"/> <xi:include href="ssh_sftp.xml"/> <xi:include href="ssh_sftpd.xml"/> </application> diff --git a/lib/ssh/doc/src/ssh.xml b/lib/ssh/doc/src/ssh.xml index aac4b462a2..a0afb5056e 100644 --- a/lib/ssh/doc/src/ssh.xml +++ b/lib/ssh/doc/src/ssh.xml @@ -22,13 +22,7 @@ </legalnotice> <title>ssh</title> - <prepared>Ingela Anderton Andin</prepared> - <responsible>Håkan Mattsson</responsible> - <docno></docno> - <approved>Håkan Mattsson</approved> - <checked></checked> <date>2007-10-06</date> - <rev>PA1</rev> </header> <module>ssh</module> <modulesummary>Main API of the SSH application</modulesummary> @@ -40,80 +34,85 @@ <title>SSH</title> <list type="bulleted"> - <item>ssh requires the crypto and public_key applications.</item> - <item>Supported SSH-version is 2.0 </item> - <item>Currently supports only a minimum of mac and encryption algorithms i.e. - hmac-sha1, and aes128-cb and 3des-cbc.</item> + <item>SSH requires the crypto and public_key applications.</item> + <item>Supported SSH version is 2.0 </item> + <item>Supported MAC algorithms: hmac-sha1</item> + <item>Supported encryption algorithms: aes128-cb and 3des-cbc</item> </list> </section> <section> - <title>COMMON DATA TYPES </title> + <title>DATA TYPES </title> <p>Type definitions that are used more than once in - this module:</p> + this module and/or abstractions to indicate the intended use of the data + type:</p> <p><c>boolean() = true | false </c></p> - <p><c>string() = list of ASCII characters</c></p> + <p><c>string() = [byte()]</c></p> <p><c>ssh_daemon_ref() - opaque to the user returned by ssh:daemon/[1,2,3]</c></p> <p><c>ssh_connection_ref() - opaque to the user returned by ssh:connect/3</c></p> <p><c>ip_address() - {N1,N2,N3,N4} % IPv4 | {K1,K2,K3,K4,K5,K6,K7,K8} % IPv6</c></p> - <p><c>subsystem_spec() = {subsystem_name(), {channel_callback(), channel_init_args()}} </c></p> + <p><c>subsystem_spec() = {subsystem_name(), + {channel_callback(), channel_init_args()}} </c></p> <p><c>subsystem_name() = string() </c></p> <p><c>channel_callback() = atom() - Name of the erlang module implementing the subsystem using the ssh_channel behavior see</c> <seealso marker="ssh_channel">ssh_channel(3)</seealso></p> <p><c>channel_init_args() = list()</c></p> </section> - + <funcs> <func> <name>close(ConnectionRef) -> ok </name> - <fsummary>Closes a ssh connection</fsummary> + <fsummary>Closes an SSH connection</fsummary> <type> <v>ConnectionRef = ssh_connection_ref()</v> </type> - <desc><p>Closes a ssh connection.</p> + <desc><p>Closes an SSH connection.</p> </desc> </func> <func> <name>connect(Host, Port, Options) -> </name> - <name>connect(Host, Port, Options, Timeout) -> {ok, ssh_connection_ref()} - | {error, Reason}</name> + <name>connect(Host, Port, Options, Timeout) -> {ok, + ssh_connection_ref()} | {error, Reason}</name> <fsummary>Connect to an ssh server.</fsummary> <type> <v>Host = string()</v> <v>Port = integer()</v> - <d>The default is <c><![CDATA[22]]></c>, the registered port for SSH.</d> + <d>The default is <c><![CDATA[22]]></c>, the assigned well known port + number for SSH.</d> <v>Options = [{Option, Value}]</v> <v>Timeout = infinity | integer(milliseconds)</v> </type> <desc> - <p>Connects to an SSH server. No channel is started this is done + <p>Connects to an SSH server. No channel is started. This is done by calling ssh_connect:session_channel/2.</p> <p>Options are:</p> <taglist> <tag><c><![CDATA[{user_dir, string()}]]></c></tag> <item> - <p>Sets the user directory e.i. the directory containing + <p>Sets the user directory i.e. the directory containing ssh configuration files for the user such as - <c><![CDATA[known_hosts]]></c>, <c><![CDATA[id_rsa, id_dsa]]></c> and - <c><![CDATA[authorized_key]]></c>. Defaults to the directory normally - referred to as <c><![CDATA[~/.ssh]]></c> </p> + <c><![CDATA[known_hosts]]></c>, <c><![CDATA[id_rsa, + id_dsa]]></c> and + <c><![CDATA[authorized_key]]></c>. Defaults to the + directory normally referred to as + <c><![CDATA[~/.ssh]]></c> </p> </item> <tag><c><![CDATA[{dsa_pass_phrase, string()}]]></c></tag> <item> - <p>If the user dsa key is protected by a pass phrase it can be + <p>If the user dsa key is protected by a passphrase it can be supplied with this option. </p> </item> <tag><c><![CDATA[{rsa_pass_phrase, string()}]]></c></tag> <item> - <p>If the user rsa key is protected by a pass phrase it can be + <p>If the user rsa key is protected by a passphrase it can be supplied with this option. </p> </item> @@ -135,25 +134,26 @@ password. Do note that it may not always be desirable to use those options from a security point of view.</p> </item> - <tag><c><![CDATA[{public_key_alg, ssh_rsa | ssh_dsa}]]></c></tag> + <tag><c><![CDATA[{public_key_alg, 'ssh-rsa' | 'ssh-dss'}]]></c></tag> <item> <p>Sets the preferred public key algorithm to use for user - authentication. If the the preferred algorithm fails of + authentication. If the the preferred algorithm fails for some reason, the other algorithm is tried. The default is to try <c><![CDATA[ssh_rsa]]></c> first.</p> </item> <tag><c><![CDATA[{pref_public_key_algs, list()}]]></c></tag> <item> - <p>List of public key algorithms to try to use, ssh_rsa and ssh_dsa available. - Will override <c><![CDATA[{public_key_alg, ssh_rsa | ssh_dsa}]]></c></p> + <p>List of public key algorithms to try to use, 'ssh-rsa' and 'ssh-dss' available. + Will override <c><![CDATA[{public_key_alg, 'ssh-rsa' | 'ssh-dss'}]]></c></p> </item> <tag><c><![CDATA[{connect_timeout, timeout()}]]></c></tag> <item> - <p>Sets a timeout on the transport layer connection. Defaults to infinity.</p> + <p>Sets a timeout on the transport layer + connection. Defaults to <c>infinity</c>.</p> </item> - <tag><c><![CDATA[{user, String}]]></c></tag> + <tag><c><![CDATA[{user, string()}]]></c></tag> <item> - <p>Provide a user name. If this option is not given, ssh + <p>Provides a user name. If this option is not given, ssh reads from the environment (<c><![CDATA[LOGNAME]]></c> or <c><![CDATA[USER]]></c> on unix, <c><![CDATA[USERNAME]]></c> on Windows).</p> @@ -165,23 +165,11 @@ password if the password authentication method is attempted.</p> </item> - <tag><c><![CDATA[{user_auth, Fun/3}]]></c></tag> - <item> - <p>Provide a fun for password authentication. The fun - will be called as <c><![CDATA[fun(User, Password, Opts)]]></c> and - should return <c><![CDATA[true]]></c> or <c><![CDATA[false]]></c>.</p> - </item> - <tag><c><![CDATA[{key_cb, atom() = KeyCallbackModule}]]></c></tag> + <tag><c><![CDATA[{key_cb, atom()}]]></c></tag> <item> - <p>Provide a special call-back module for key handling. - The call-back module should be modeled after the - <c><![CDATA[ssh_file]]></c> module. The functions that must - be exported are: - <c><![CDATA[private_host_rsa_key/2]]></c>, - <c><![CDATA[private_host_dsa_key/2]]></c>, - <c><![CDATA[lookup_host_key/3]]></c> and - <c><![CDATA[add_host_key/3]]></c>. This is considered - somewhat experimental and will be better documented later on.</p> + <p>Module implementing the behaviour <seealso marker="ssh_client_key_api">ssh_client_key_api</seealso>. + Can be used to customize the handling of public keys. + </p> </item> <tag><c><![CDATA[{quiet_mode, atom() = boolean()}]]></c></tag> <item> @@ -189,20 +177,24 @@ </item> <tag><c><![CDATA[{fd, file_descriptor()}]]></c></tag> <item> - <p>Allow an existing file-descriptor to be used + <p>Allow an existing file descriptor to be used (simply passed on to the transport protocol).</p></item> - <tag><c><![CDATA[{ip_v6_disabled, boolean()}]]></c></tag> + <tag><c><![CDATA[{ipv6_disabled, boolean()}]]></c></tag> <item> - <p>Determines if SSH shall use IPv6 or not.</p></item> - <tag><c><![CDATA[{idle_time, timeout()}]]></c></tag> + <p>Determines if SSH shall use IPv6 or not.</p> + </item> + <tag><c><![CDATA[{rekey_limit, integer()}]]></c></tag> <item> - <p>Sets a timeout on connection when no channels are active, default is infinity</p></item> + <p>Provide, in bytes, when rekeying should be initiated, + defaults to one time each GB and one time per hour.</p> + </item> </taglist> </desc> </func> <func> - <name>connection_info(ConnectionRef, [Option]) ->[{Option, Value}] </name> + <name>connection_info(ConnectionRef, [Option]) ->[{Option, + Value}] </name> <fsummary> Retrieves information about a connection. </fsummary> <type> <v>Option = client_version | server_version | peer</v> @@ -217,7 +209,8 @@ <func> <name>daemon(Port) -> </name> <name>daemon(Port, Options) -> </name> - <name>daemon(HostAddress, Port, Options) -> ssh_daemon_ref()</name> + <name>daemon(HostAddress, Port, Options) -> {ok, + ssh_daemon_ref()} | {error, atom()}</name> <fsummary>Starts a server listening for SSH connections on the given port.</fsummary> <type> @@ -228,30 +221,32 @@ <v>Value = term()</v> </type> <desc> - <p>Starts a server listening for SSH connections on the given port.</p> - - <p>Options are:</p> + <p>Starts a server listening for SSH connections on the given + port.</p> + <p>Options are:</p> <taglist> <tag><c><![CDATA[{subsystems, [subsystem_spec()]]]></c></tag> <item> - Provides specifications for handling of subsystems. The - "sftp" subsystem-spec can be retrieved by calling - ssh_sftpd:subsystem_spec/1. If the subsystems option in not present - the value of <c>[ssh_sftpd:subsystem_spec([])]</c> will be used. - It is of course possible to set the option to the empty list - if you do not want the daemon to run any subsystems at all. + Provides specifications for handling of subsystems. The + "sftp" subsystem spec can be retrieved by calling + ssh_sftpd:subsystem_spec/1. If the subsystems option in + not present the value of + <c>[ssh_sftpd:subsystem_spec([])]</c> will be used. It is + of course possible to set the option to the empty list if + you do not want the daemon to run any subsystems at all. </item> - <tag><c><![CDATA[{shell, {Module, Function, Args} | fun(string() = User) - > pid() | - fun(string() = User, ip_address() = PeerAddr) -> pid()}]]></c></tag> + <tag><c><![CDATA[{shell, {Module, Function, Args} | + fun(string() = User) - > pid() | fun(string() = User, + ip_address() = PeerAddr) -> pid()}]]></c></tag> <item> - Defines the read-eval-print loop used when a shell is requested - by the client. Example use the - erlang shell: <c><![CDATA[{shell, start, []}]]></c> which is - the default behavior. + Defines the read-eval-print loop used when a shell is + requested by the client. Default is to use the erlang shell: + <c><![CDATA[{shell, start, []}]]></c> </item> - <tag><c><![CDATA[{ssh_cli,{channel_callback(), channel_init_args()}}]]></c></tag> + <tag><c><![CDATA[{ssh_cli,{channel_callback(), + channel_init_args()}}]]></c></tag> <item> - Provide your own cli implementation, e.i. a channel callback + Provides your own cli implementation, i.e. a channel callback module that implements a shell and command execution. Note that you may customize the shell read-eval-print loop using the option <c>shell</c> which is much less work than implementing @@ -259,27 +254,30 @@ </item> <tag><c><![CDATA[{user_dir, String}]]></c></tag> <item> - <p>Sets the user directory e.i. the directory containing + <p>Sets the user directory i.e. the directory containing ssh configuration files for the user such as - <c><![CDATA[known_hosts]]></c>, <c><![CDATA[id_rsa, id_dsa]]></c> and - <c><![CDATA[authorized_key]]></c>. Defaults to the directory normally - referred to as <c><![CDATA[~/.ssh]]></c> </p> + <c><![CDATA[known_hosts]]></c>, <c><![CDATA[id_rsa, + id_dsa]]></c> and + <c><![CDATA[authorized_key]]></c>. Defaults to the + directory normally referred to as + <c><![CDATA[~/.ssh]]></c> </p> </item> <tag><c><![CDATA[{system_dir, string()}]]></c></tag> <item> - <p>Sets the system directory, containing the host files - that identifies the host for ssh. The default is - <c><![CDATA[/etc/ssh]]></c>, note that SSH normally - requires the host files there to be readable only by - root.</p> + <p>Sets the system directory, containing the host key files + that identifies the host keys for ssh. The default is + <c><![CDATA[/etc/ssh]]></c>, note that for security reasons + this directory is normally only accessible by the root user.</p> </item> <tag><c><![CDATA[{auth_methods, string()}]]></c></tag> <item> - <p>Comma separated string that determines which authentication methodes that the server - should support and in what order they will be tried. Defaults to + <p>Comma separated string that determines which + authentication methodes that the server should support and + in what order they will be tried. Defaults to <c><![CDATA["publickey,keyboard-interactive,password"]]></c></p> </item> - <tag><c><![CDATA[{user_passwords, [{string() = User, string() = Password}]}]]></c></tag> + <tag><c><![CDATA[{user_passwords, [{string() = User, + string() = Password}]}]]></c></tag> <item> <p>Provide passwords for password authentication.They will be used when someone tries to connect to the server and @@ -293,13 +291,19 @@ user. From a security perspective this option makes the server very vulnerable.</p> </item> - <tag><c><![CDATA[{pwdfun, fun/2}]]></c></tag> + <tag><c><![CDATA[{pwdfun, fun(User::string(), password::string() -> boolean()}]]></c></tag> <item> <p>Provide a function for password validation. This is called with user and password as strings, and should return <c><![CDATA[true]]></c> if the password is valid and <c><![CDATA[false]]></c> otherwise.</p> </item> + <tag><c><![CDATA[{key_cb, atom()}]]></c></tag> + <item> + <p>Module implementing the behaviour <seealso marker="ssh_server_key_api">ssh_server_key_api</seealso>. + Can be used to customize the handling of public keys. + </p> + </item> <tag><c><![CDATA[{fd, file_descriptor()}]]></c></tag> <item> <p>Allow an existing file-descriptor to be used @@ -308,6 +312,18 @@ <item> <p>Determines if SSH shall use IPv6 or not (only used when HostAddress is set to any).</p></item> + <tag><c><![CDATA[{failfun, fun()}]]></c></tag> + <item> + <p>Provide a fun() to implement your own logging when a user fails to authenticate.</p> + </item> + <tag><c><![CDATA[{connectfun, fun()}]]></c></tag> + <item> + <p>Provide a fun() to implement your own logging when a user authenticates to the server.</p> + </item> + <tag><c><![CDATA[{disconnectfun, fun()}]]></c></tag> + <item> + <p>Provide a fun() to implement your own logging when a user disconnects from the server.</p> + </item> </taglist> </desc> </func> @@ -323,9 +339,9 @@ <v> Options - see ssh:connect/3</v> </type> <desc> - <p>Starts an interactive shell to an SSH server on the + <p>Starts an interactive shell via an SSH server on the given <c>Host</c>. The function waits for user input, - and will not return until the remote shell is ended (e.g. on + and will not return until the remote shell is ended (i.e. exit from the shell). </p> </desc> @@ -334,25 +350,24 @@ <func> <name>start() -> </name> <name>start(Type) -> ok | {error, Reason}</name> - <fsummary>Starts the Ssh application. </fsummary> + <fsummary>Starts the SSH application. </fsummary> <type> <v>Type = permanent | transient | temporary</v> <v>Reason = term() </v> </type> <desc> - <p>Starts the Ssh application. Default type - is temporary. See also - <seealso marker="kernel:application">application(3)</seealso> - Requires that the crypto application has been started. + <p>Utility function that starts crypto, public_key and the SSH + application. Defult type is temporary. + See also <seealso marker="kernel:application">application(3)</seealso> </p> </desc> </func> <func> <name>stop() -> ok </name> - <fsummary>Stops the Ssh application.</fsummary> + <fsummary>Stops the SSH application.</fsummary> <desc> - <p>Stops the Ssh application. See also + <p>Stops the SSH application. See also <seealso marker="kernel:application">application(3)</seealso></p> </desc> </func> diff --git a/lib/ssh/doc/src/ssh_app.xml b/lib/ssh/doc/src/ssh_app.xml new file mode 100644 index 0000000000..c01f44936a --- /dev/null +++ b/lib/ssh/doc/src/ssh_app.xml @@ -0,0 +1,100 @@ +<?xml version="1.0" encoding="iso-8859-1" ?> +<!DOCTYPE appref SYSTEM "appref.dtd"> + +<appref> + <header> + <copyright> + <year>2012</year> + <holder>Ericsson AB. All Rights Reserved.</holder> + </copyright> + <legalnotice> + The contents of this file are subject to the Erlang Public License, + Version 1.1, (the "License"); you may not use this file except in + compliance with the License. You should have received a copy of the + Erlang Public License along with this software. If not, it can be + retrieved online at http://www.erlang.org/. + + Software distributed under the License is distributed on an "AS IS" + basis, WITHOUT WARRANTY OF ANY KIND, either express or implied. See + the License for the specific language governing rights and limitations + under the License. + + </legalnotice> + + <title>SSH</title> + <file>ssh_app.xml</file> + </header> + <app>SSH</app> + <appsummary>The ssh application implements the SSH (Secure Shell) protocol and + provides an SFTP (SSH File Transfer Protocol) client and server. </appsummary> + + <section> + <title>DEPENDENCIES</title> + <p>The ssh application uses the Erlang applications public_key and + crypto to handle public keys and encryption, hence these + applications needs to be loaded for the ssh application to work. In + an embedded environment that means they need to be started with + application:start/[1,2] before the ssh application is started. + </p> + </section> + + <section> + <title>CONFIGURATION</title> + + <p>The ssh application does not currently have an application + specific configuration file as described in application(3), + however it will by default use the following configuration files + from openssh: known_hosts, authorized_keys, authorized_keys2, + id_dsa and id_rsa, ssh_host_dsa_key and ssh_host_rsa_key. By + default Erlang SSH will look for id_dsa, id_rsa, known_hosts + and authorized_keys in ~/.ssh, and the host key files in /etc/ssh + . These locations may be changed by the options user_dir and + system_dir. Public key handling may also be customized by + providing a callback module implementing the behaviors + <seealso marker="ssh_client_key_api">ssh_client_key_api</seealso> and + <seealso marker="ssh_server_key_api">ssh_server_key_api</seealso>. + </p> + + <section> + <title>PUBLIC KEYS</title> + <p> + id_dsa and id_rsa are the users private key files, note that + the public key is part of the private key so the ssh + application will not use the id_<*>.pub files. These are + for the users convenience when he/she needs to convey their + public key. + </p> + </section> + + <section> + <title>KNOW HOSTS</title> + <p>The known_hosts file contains a list of approved servers and + their public keys. Once a server is listed, it can be verified + without user interaction. + </p> + </section> + + <section> + <title>AUTHORIZED KEYS</title> + <p>The authorized key file keeps track of the user's authorized + public keys. The most common use of this file is to let users + log in without entering their password which is supported by the + Erlang SSH daemon. + </p> + </section> + + <section> + <title>HOST KEYS</title> + <p>Currently rsa and dsa host keys are supported and are + expected to be found in files named ssh_host_rsa_key and + ssh_host_dsa_key. + </p> + </section> + </section> + + <section> + <title>SEE ALSO</title> + <p>application(3)</p> + </section> + +</appref> diff --git a/lib/ssh/doc/src/ssh_channel.xml b/lib/ssh/doc/src/ssh_channel.xml index c2b7aa94a5..f0083ae8d1 100644 --- a/lib/ssh/doc/src/ssh_channel.xml +++ b/lib/ssh/doc/src/ssh_channel.xml @@ -1,11 +1,11 @@ -<?xml version="1.0" encoding="latin1" ?> +<?xml version="1.0" encoding="iso-8859-1" ?> <!DOCTYPE erlref SYSTEM "erlref.dtd"> <erlref> <header> <copyright> <year>2009</year> - <year>2009</year> + <year>2012</year> <holder>Ericsson AB, All Rights Reserved</holder> </copyright> <legalnotice> @@ -22,31 +22,35 @@ The Initial Developer of the Original Code is Ericsson AB. </legalnotice> - <title>ssh_channel</title> - <prepared>Ingela Anderton Andin</prepared> - <responsible></responsible> - <docno></docno> - <approved></approved> - <checked></checked> - <date></date> - <rev></rev> </header> <module>ssh_channel</module> - <modulesummary>Generic Ssh Channel Behavior + <modulesummary>-behaviour(ssh_channel). </modulesummary> <description> - <p>Ssh services are implemented as channels that are multiplexed - over an ssh connection and communicates via the ssh connection - protocol. This module provides a callback API that takes care of - generic channel aspects such as flow control and close messages - and lets the callback functions take care of the service specific - parts. + <p>SSH services (clients and servers) are implemented as channels + that are multiplexed over an SSH connection and communicates via + the <url href="http://www.ietf.org/rfc/rfc4254.txt"> SSH + Connection Protocol </url>. This module provides a callback API + that takes care of generic channel aspects such as flow control + and close messages and lets the callback functions take care of + the service (application) specific parts. This behavior also ensures + that the channel process honors the principal of an OTP-process so + that it can be part of a supervisor tree. This is a requirement of + channel processes implementing a subsystem that will be added to + the SSH applications supervisor tree. </p> + + <note> When implementing a SSH subsystem use the + <c>-behaviour(ssh_subsystem).</c> instead of <c>-behaviour(ssh_channel).</c> + as the only relevant callback functions for subsystems are + init/1, handle_ssh_msg/2, handle_msg/2 and terminate/2, so the ssh_subsystem + behaviour is limited version of the ssh_channel behaviour. + </note> </description> <section> - <title>COMMON DATA TYPES </title> + <title>DATA TYPES </title> <p>Type definitions that are used more than once in this module and/or abstractions to indicate the intended use of the data @@ -56,10 +60,10 @@ <p><c>string() = list of ASCII characters</c></p> <p><c>timeout() = infinity | integer() - in milliseconds.</c></p> <p><c>ssh_connection_ref() - opaque to the user returned by - ssh:connect/3 or sent to a ssh channel process</c></p> + ssh:connect/3 or sent to an SSH channel process</c></p> <p><c>ssh_channel_id() = integer() </c></p> <p><c>ssh_data_type_code() = 1 ("stderr") | 0 ("normal") are - currently valid values see RFC 4254 section 5.2.</c></p> + currently valid values see <url href="http://www.ietf.org/rfc/rfc4254.txt">RFC 4254 </url> section 5.2.</c></p> </section> <funcs> @@ -74,13 +78,15 @@ <v>Timeout = timeout() </v> <v>Reply = term() </v> <v>Reason = closed | timeout </v> + </type> <desc> <p>Makes a synchronous call to the channel process by sending a message and waiting until a reply arrives or a timeout - occurs. The channel will call - <c>CallbackModule:handle_call/3</c> to handle the message. - If the channel process does not exist <c>{error, closed}</c> is returned. + occurs. The channel will call <seealso mark = + "#Module:handle_call-3">Module:handle_call/3</seealso> + to handle the message. If the channel process does not exist + <c>{error, closed}</c> is returned. </p> </desc> </func> @@ -98,26 +104,27 @@ <p>Sends an asynchronous message to the channel process and returns ok immediately, ignoring if the destination node or channel process does not exist. The channel will call - <c>CallbackModule:handle_cast/2</c> to handle the message. + <seealso mark = "#Module:handle_cast-3">Module:handle_cast/2</seealso> + to handle the message. </p> </desc> </func> <func> <name>enter_loop(State) -> _ </name> - <fsummary> Makes an existing process into a ssh_channel process. </fsummary> + <fsummary> Makes an existing process an ssh_channel process. </fsummary> <type> - <v> State = term() - as returned by ssh_channel:init/1</v> + <v> State = term() - as returned by <seealso mark = "#init-1">ssh_channel:init/1</seealso></v> </type> <desc> - <p> Makes an existing process into a <c>ssh_channel</c> + <p> Makes an existing process an <c>ssh_channel</c> process. Does not return, instead the calling process will - enter the <c>ssh_channel</c> process receive loop and become a + enter the <c>ssh_channel</c> process receive loop and become an <c>ssh_channel process.</c> The process must have been started using one of the start functions in proc_lib, see <seealso marker="stdlib:proc_lib">proc_lib(3)</seealso>. The user is responsible for any initialization of the process - and needs to call ssh_channel:init/1. + and needs to call <seealso mark = "#init-1">ssh_channel:init/1</seealso> </p> </desc> </func> @@ -126,7 +133,10 @@ <name>init(Options) -> {ok, State} | {ok, State, Timeout} | {stop, Reason} </name> <fsummary> Initiates a ssh_channel process.</fsummary> <type> - <v> Options = [{Option, Value}]</v> + <v>Options = [{Option, Value}]</v> + <v>State = term()</v> + <v>Timeout = timeout() </v> + <v>Reason = term() </v> </type> <desc> <p> @@ -134,24 +144,27 @@ </p> <taglist> <tag><c><![CDATA[{channel_cb, atom()}]]></c></tag> - <item>The module that implements the channel behavior.</item> + <item>The module that implements the channel behaviour.</item> <tag><c><![CDATA[{init_args(), list()}]]></c></tag> - <item> The list of arguments to the callback modules + <item> The list of arguments to the callback module's init function.</item> <tag><c><![CDATA[{cm, connection_ref()}]]></c></tag> - <item> Reference to the ssh connection.</item> + <item> Reference to the ssh connection as returned by <seealso + marker="ssh#connect-3">ssh:connect/3</seealso></item> <tag><c><![CDATA[{channel_id, channel_id()}]]></c></tag> <item> Id of the ssh channel.</item> </taglist> - <note><p>This function is normally not called by the user, it is - only needed if for some reason the channel process needs - to be started with help of <c>proc_lib</c> instead calling - <c>ssh_channel:start/4</c> or <c>ssh_channel:start_link/4</c> </p> + <note><p>This function is normally not called by the + user. The user only needs to call if for some reason the + channel process needs to be started with help of + <c>proc_lib</c> instead of calling + <c>ssh_channel:start/4</c> or + <c>ssh_channel:start_link/4</c> </p> </note> </desc> </func> @@ -167,12 +180,12 @@ <p>This function can be used by a channel to explicitly send a reply to a client that called <c>call/[2,3]</c> when the reply cannot be defined in the return value of - <c>CallbackModule:handle_call/3</c>.</p> + <seealso marker ="#Module:handle_call-3">Module:handle_call/3</seealso>.</p> <p><c>Client</c> must be the <c>From</c> argument provided to the callback function <c>handle_call/3</c>. <c>Reply</c> is an arbitrary term, - which will be given back to the client as the return value of - <c>ssh_channel:call/[2,3].</c></p> + which will be given back to the client as the return value of + <seealso marker="#call-2">ssh_channel:call/[2,3].</seealso>></p> </desc> </func> @@ -180,11 +193,12 @@ <name>start(SshConnection, ChannelId, ChannelCb, CbInitArgs) -> </name> <name>start_link(SshConnection, ChannelId, ChannelCb, CbInitArgs) -> {ok, ChannelRef} | {error, Reason}</name> - <fsummary> Starts a processes that handles a ssh channel. </fsummary> + <fsummary> Starts a processes that handles a SSH channel. </fsummary> <type> <v>SshConnection = ssh_connection_ref()</v> <v>ChannelId = ssh_channel_id() </v> - <d> As returned by ssh_connection:session_channel/[2,4]</d> + <d> As returned by cannot be defined in the return value of + <seealso marker ="ssh_connection#session_channel/2">ssh_connection:session_channel/[2,4]</seealso></d> <v>ChannelCb = atom()</v> <d> The name of the module implementing the service specific parts of the channel.</d> @@ -193,10 +207,10 @@ <v>ChannelRef = pid()</v> </type> <desc> - <p>Starts a processes that handles a ssh channel. Will be - called internally by the ssh daemon or explicitly by the ssh - client implementations. A channel process traps exit signals - by default. + <p>Starts a processes that handles an SSH channel. It will be + called internally by the SSH daemon or explicitly by the SSH + client implementations. The behavior will set the + <c>trap_exit</c> flag to true. </p> </desc> </func> @@ -204,72 +218,61 @@ </funcs> <section> - <title>CALLBACK FUNCTIONS</title> - - <p>The functions init/1, terminate/2, handle_ssh_msg/2 and - handle_msg/2 are the functions that are required to provide the - implementation for a server side channel, such as a ssh subsystem - channel that can be plugged into the erlang ssh daemon see - <seealso marker="ssh">ssh:daemon/[2, 3]</seealso>. The - handle_call/3, handle_cast/2 code_change/3 and enter_loop/1 - functions are only relevant when implementing a client side - channel.</p> - </section> - - <section> <marker id="cb_timeouts"></marker> <title> CALLBACK TIMEOUTS</title> - <p> If an integer timeout value is provided in a return value of - one of the callback functions, a timeout will occur unless a - message is received within <c>Timeout</c> milliseconds. A timeout - is represented by the atom <c>timeout</c> which should be handled - by the <seealso marker="#handle_msg">handle_msg/2</seealso> - callback function. The atom infinity can be used to wait - indefinitely, this is the default value. </p> + + <p>The timeout values that may be returned by the callback functions + has the same semantics as in a <seealso marker="stdlib#gen_server">gen_server</seealso> + If the timeout occurs <seealso marker="#handle_msg">handle_msg/2</seealso> + will be called as <c>handle_msg(timeout, State). </c></p> </section> <funcs> <func> - <name>CallbackModule:code_change(OldVsn, State, Extra) -> {ok, + <name>Module:code_change(OldVsn, State, Extra) -> {ok, NewState}</name> <fsummary> Converts process state when code is changed.</fsummary> <type> - <v> Converts process state when code is changed.</v> + <v>OldVsn = term()</v> + <d>In the case of an upgrade, <c>OldVsn</c> is <c>Vsn</c>, and + in the case of a downgrade, <c>OldVsn</c> is + <c>{down,Vsn}</c>. <c>Vsn</c> is defined by the <c>vsn</c> + attribute(s) of the old version of the callback module + <c>Module</c>. If no such attribute is defined, the version is + the checksum of the BEAM file.</d> + <v>State = term()</v> + <d>The internal state of the channel.</d> + <v>Extra = term()</v> + <d>Passed as-is from the <c>{advanced,Extra}</c> + part of the update instruction.</d> </type> <desc> - <p>This function is called by a client side channel when it - should update its internal state during a release - upgrade/downgrade, i.e. when the instruction - <c>{update,Module,Change,...}</c> where - <c>Change={advanced,Extra}</c> is given in the <c>appup</c> - file. See <seealso - marker="doc/design_principles:release_handling#instr">OTP - Design Principles</seealso> for more information. Any new - connection will benefit from a server side upgrade but - already started connections on the server side will not be - affected. - </p> + <p> Converts process state when code is changed.</p> + + <p>This function is called by a client side channel when it + should update its internal state during a release + upgrade/downgrade, i.e. when the instruction + <c>{update,Module,Change,...}</c> where + <c>Change={advanced,Extra}</c> is given in the <c>appup</c> + file. See <seealso marker="doc/design_principles:release_handling#instr">OTP + Design Principles</seealso> for more information. + </p> - <note><p>If there are long lived ssh connections and more - than one upgrade in a short time this may cause the old - connections to fail as only two versions of the code may - be loaded simultaneously.</p></note> + <note><p>Soft upgrade according to the OTP release concept + is not straight forward for the server side, as subsystem + channel processes are spawned by the SSH application and + hence added to its supervisor tree. It could be possible to + upgrade the subsystem channels, when upgrading the user + application, if the callback functions can handle two + versions of the state, but this function can not be used in + the normal way.</p> + </note> - <p>In the case of an upgrade, <c>OldVsn</c> is <c>Vsn</c>, and - in the case of a downgrade, <c>OldVsn</c> is - <c>{down,Vsn}</c>. <c>Vsn</c> is defined by the <c>vsn</c> - attribute(s) of the old version of the callback module - <c>Module</c>. If no such attribute is defined, the version - is the checksum of the BEAM file.</p> - <p><c>State</c> is the internal state of the channel.</p> - <p><c>Extra</c> is passed as-is from the <c>{advanced,Extra}</c> - part of the update instruction.</p> - <p>The function should return the updated internal state.</p> </desc> </func> <func> - <name>CallbackModule:init(Args) -> {ok, State} | {ok, State, Timeout} | + <name>Module:init(Args) -> {ok, State} | {ok, State, timeout()} | {stop, Reason}</name> <fsummary> Makes necessary initializations and returns the initial channel state if the initializations succeed.</fsummary> @@ -277,7 +280,6 @@ <v> Args = term() </v> <d> Last argument to ssh_channel:start_link/4.</d> <v> State = term() </v> - <v>Timeout = timeout() </v> <v> Reason = term() </v> </type> <desc> @@ -290,7 +292,7 @@ </func> <func> - <name>CallbackModule:handle_call(Msg, From, State) -> Result</name> + <name>Module:handle_call(Msg, From, State) -> Result</name> <fsummary> Handles messages sent by calling <c>ssh_channel:call/[2,3]</c></fsummary> <type> @@ -298,17 +300,16 @@ <v>From = opaque to the user should be used as argument to ssh_channel:reply/2</v> <v>State = term()</v> - <v>Result = {reply, Reply, NewState} | {reply, Reply, NewState, Timeout} - | {noreply, NewState} | {noreply , NewState, Timeout} + <v>Result = {reply, Reply, NewState} | {reply, Reply, NewState, timeout()} + | {noreply, NewState} | {noreply , NewState, timeout()} | {stop, Reason, Reply, NewState} | {stop, Reason, NewState} </v> <v>Reply = term() - will be the return value of ssh_channel:call/[2,3]</v> - <v>Timeout = timeout() </v> - <v>NewState = term() - a possible updated version of State</v> + <v>NewState = term()</v> <v>Reason = term()</v> </type> <desc> <p>Handles messages sent by calling - <c>ssh_channel:call/[2,3]</c> + <seealso marker="#call-2">ssh_channel:call/[2,3]</seealso> </p> <p>For more detailed information on timeouts see the section <seealso marker="#cb_timeouts">CALLBACK TIMEOUTS</seealso>. </p> @@ -316,16 +317,15 @@ </func> <func> - <name>CallbackModule:handle_cast(Msg, State) -> Result</name> + <name>Module:handle_cast(Msg, State) -> Result</name> <fsummary> Handles messages sent by calling <c>ssh_channel:cact/2</c></fsummary> <type> <v>Msg = term()</v> <v>State = term()</v> - <v>Result = {noreply, NewState} | {noreply, NewState, Timeout} + <v>Result = {noreply, NewState} | {noreply, NewState, timeout()} | {stop, Reason, NewState}</v> - <v>NewState = term() - a possible updated version of State</v> - <v>Timeout = timeout() </v> + <v>NewState = term() </v> <v>Reason = term()</v> </type> <desc> @@ -339,7 +339,7 @@ </func> <func> - <name>CallbackModule:handle_msg(Msg, State) -> {ok, State} | + <name>Module:handle_msg(Msg, State) -> {ok, State} | {stop, ChannelId, State}</name> <fsummary> Handle other messages than ssh connection protocol, @@ -359,136 +359,37 @@ <taglist> <tag><c><![CDATA[{ssh_channel_up, ssh_channel_id(), ssh_connection_ref()}]]></c></tag> - <item>This is the first messages that will be received - by the channel, it is sent just before - the ssh_channel:init/1 function returns successfully. - This is especially useful if the server wants - to send a message to the client without first receiving - a message from the client. If the message is not useful - for your particular problem just ignore it by immediately - returning {ok, State}. + <item>This is the first messages that will be received by + the channel, it is sent just before the <seealso + marker="#init-2">ssh_channel:init/1</seealso> function + returns successfully. This is especially useful if the + server wants to send a message to the client without first + receiving a message from it. If the message is not + useful for your particular scenario just ignore it by + immediately returning {ok, State}. </item> </taglist> </desc> </func> <func> - <name>CallbackModule:handle_ssh_msg(Msg, State) -> {ok, State} | {stop, + <name>Module:handle_ssh_msg(Msg, State) -> {ok, State} | {stop, ssh_channel_id(), State}</name> <fsummary> Handles ssh connection protocol messages. </fsummary> <type> - <v>Msg = {ssh_cm, ssh_connection_ref(), SshMsg}</v> - <v> SshMsg = tuple() - see message list below</v> + <v>Msg = <seealso marker="ssh_connection"> ssh_connection:event() </seealso> </v> <v>State = term()</v> </type> <desc> <p> Handles ssh connection protocol messages that may need service specific attention. </p> - - <p> All channels should handle the following messages. For - channels implementing subsystems the handle_ssh_msg-callback - will not be called for any other messages. </p> - - <taglist> - <tag><c><![CDATA[{ssh_cm, ssh_connection_ref(), {data, ssh_channel_id(), - ssh_data_type_code(), binary() = Data}}]]></c></tag> - <item> Data has arrived on the channel. When the callback - for this message returns the channel behavior will adjust - the ssh flow control window.</item> - - <tag><c><![CDATA[{ssh_cm, ssh_connection_ref(), {eof, - ssh_channel_id()}}]]></c></tag> - <item>Indicteas that the other side will not send any more - data.</item> - - <tag><c><![CDATA[{ssh_cm, ssh_connection_ref(), {signal, - ssh_channel_id(), ssh_signal()}} ]]></c></tag> - <item>A signal can be delivered to the remote - process/service using the following message. Some systems - may not implement signals, in which case they should ignore - this message.</item> - - <tag><c><![CDATA[{ssh_cm, ssh_connection_ref(), - {exit_signal, ssh_channel_id(), string() = exit_signal, - string() = ErrorMsg, string() = - LanguageString}}]]></c></tag> - <item>A remote execution may terminate violently due to a - signal then this message may be received. For details on valid string - values see RFC 4254 section 6.10</item> - - <tag><c><![CDATA[{ssh_cm, ssh_connection_ref(), {exit_status, - ssh_channel_id(), integer() = ExitStatus}}]]></c></tag> - <item> When the command running at the other end terminates, - the following message can be sent to return the exit status - of the command. A zero 'exit_status' usually means that the - command terminated successfully.</item> - </taglist> - - <p> Channels implementing a shell and command execution on the server side - should also handle the following messages. </p> - - <taglist> - <tag><c><![CDATA[{ssh_cm, ssh_connection_ref(), {env, ssh_channel_id(), - boolean() = WantReply, string() = Var, string() = Value}}]]></c></tag> - <item> Environment variables may be passed to the - shell/command to be started later. Note that before the - callback returns it should call the function - ssh_connection:reply_request/4 with the boolean value of <c> - WantReply</c> as the second argument. - </item> - - <tag><c><![CDATA[{ssh_cm, ConnectionRef, {exec, ssh_channel_id(), - boolean() = WantReply, string() = Cmd}}]]></c></tag> - <item> This message will request that the server start the - execution of the given command. Note that before the - callback returns it should call the function - ssh_connection:reply_request/4 with the boolean value of <c> - WantReply</c> as the second argument.</item> - - <tag><c><![CDATA[{ssh_cm, ssh_connection_ref(), {pty, ssh_channel_id(), - boolean() = WantReply, {string() = Terminal, integer() = CharWidth, - integer() = RowHeight, integer() = PixelWidth, integer() = PixelHight, - [{atom() | integer() = Opcode, - integer() = Value}] = TerminalModes}}}]]></c></tag> - <item>A pseudo-terminal has been requested for the - session. Terminal is the value of the TERM environment - variable value (e.g., vt100). Zero dimension parameters must - be ignored. The character/row dimensions override the pixel - dimensions (when nonzero). Pixel dimensions refer to the - drawable area of the window. The <c>Opcode</c> in the - <c>TerminalModes</c> list is the mnemonic name, represented - as an lowercase erlang atom, defined in RFC 4254 section 8, - or the opcode if the mnemonic name is not listed in the - RFC. Example <c>OP code: 53, mnemonic name ECHO erlang atom: - echo</c>. Note that before the callback returns it should - call the function ssh_connection:reply_request/4 with the - boolean value of <c> WantReply</c> as the second - argument.</item> - - <tag><c><![CDATA[{ssh_cm, ConnectionRef, {shell, boolean() = - WantReply}}]]></c></tag> - <item> This message will request that the user's default - shell be started at the other end. Note that before the - callback returns it should call the function - ssh_connection:reply_request/4 with the value of <c> - WantReply</c> as the second argument. - </item> - - <tag><c><![CDATA[ {ssh_cm, ssh_connection_ref(), {window_change, - ssh_channel_id(), integer() = CharWidth, integer() = RowHeight, - integer() = PixWidth, integer() = PixHeight}}]]></c></tag> - <item> When the window (terminal) size changes on the client - side, it MAY send a message to the other side to inform it - of the new dimensions.</item> - </taglist> <p> The following message is completely taken care of by the ssh channel behavior</p> <taglist> - <tag><c><![CDATA[{ssh_cm, ssh_connection_ref(), {closed, - ssh_channel_id()}}]]></c></tag> + <tag><c><![CDATA[{closed, ssh_channel_id()}]]></c></tag> <item> The channel behavior will send a close message to the other side if such a message has not already been sent and then terminate the channel with reason normal.</item> @@ -497,7 +398,7 @@ </func> <func> - <name>CallbackModule:terminate(Reason, State) -> _</name> + <name>Module:terminate(Reason, State) -> _</name> <fsummary> </fsummary> <type> <v>Reason = term()</v> @@ -505,12 +406,12 @@ </type> <desc> <p>This function is called by a channel process when it is - about to terminate. Before this function is called ssh_connection:close/2 - will be called if it has not been called earlier. - This function should be the opposite of <c>CallbackModule:init/1</c> - and do any necessary cleaning up. When it returns, the - channel process terminates with reason <c>Reason</c>. The return value is - ignored. + about to terminate. Before this function is called <seealso + marker="ssh_connection#close-2"> ssh_connection:close/2 + </seealso> will be called if it has not been called earlier. + This function should do any necessary cleaning + up. When it returns, the channel process terminates with + reason <c>Reason</c>. The return value is ignored. </p> </desc> </func> diff --git a/lib/ssh/doc/src/ssh_client_key_api.xml b/lib/ssh/doc/src/ssh_client_key_api.xml new file mode 100644 index 0000000000..abc1070e78 --- /dev/null +++ b/lib/ssh/doc/src/ssh_client_key_api.xml @@ -0,0 +1,124 @@ +<?xml version="1.0" encoding="iso-8859-1" ?> +<!DOCTYPE erlref SYSTEM "erlref.dtd"> + +<erlref> + <header> + <copyright> + <year>2012</year> + <holder>Ericsson AB, All Rights Reserved</holder> + </copyright> + <legalnotice> + The contents of this file are subject to the Erlang Public License, + Version 1.1, (the "License"); you may not use this file except in + compliance with the License. You should have received a copy of the + Erlang Public License along with this software. If not, it can be + retrieved online at http://www.erlang.org/. + + Software distributed under the License is distributed on an "AS IS" + basis, WITHOUT WARRANTY OF ANY KIND, either express or implied. See + the License for the specific language governing rights and limitations + under the License. + + The Initial Developer of the Original Code is Ericsson AB. + </legalnotice> + <title>ssh_client_key_api</title> + </header> + <module>ssh_client_key_api</module> + <modulesummary> + -behaviour(ssh_client_key_api). + </modulesummary> + <description> + <p> Behavior describing the API for an SSH client's public key handling. + By implementing the callbacks defined. + in this behavior it is possible to customize the SSH client's public key + handling. By default the SSH application implements this behavior + with help of the standard openssh files, see <seealso marker="SSH_app"> ssh(6)</seealso>. </p> + </description> + + <section> + <title>DATA TYPES </title> + + <p>Type definitions that are used more than once in this module + and/or abstractions to indicate the intended use of the data + type:</p> + + <p> boolean() = true | false</p> + <p> string() = [byte()] </p> + <p> public_key() = #'RSAPublicKey'{}| {integer(), #'Dss-Parms'{}}| term()</p> + <p> private_key() = #'RSAPublicKey'{}| {integer(), #'Dss-Parms'{}}| term()</p> + <p> public_key_algorithm() = 'ssh-rsa'| 'ssh-dss' | atom()</p> + + </section> + + <funcs> + <func> + <name>Module:add_host_key(HostNames, Key, ConnectOptions) -> ok | {error, Reason}</name> + <fsummary>Adds a host key to the set of trusted host keys</fsummary> + <type> + <v>HostNames = string()</v> + <d>Description of the host that owns the <c>PublicKey</c></d> + + <v>Key = public_key() </v> + <d> Normally an RSA or DSA public key but handling of other public keys can be added</d> + + <v>ConnectOptions = proplists:proplist() </v> + <d>Options provided to <seealso marker="ssh#daemon">ssh:connect/[3,4]</seealso></d> + <v>Reason = term() </v> + </type> + <desc> + <p> Adds a host key to the set of trusted host keys</p> + </desc> + </func> + + <func> + <name>Module:is_host_key(Key, Host, Algorithm, ConnectOptions) -> Result</name> + <fsummary>Checks if a host key is trusted</fsummary> + <type> + <v>Key = public_key() </v> + <d> Normally an RSA or DSA public key but handling of other public keys can be added</d> + + <v>Host = string()</v> + <d>Description of the host</d> + + <v>Algorithm = public_key_algorithm()</v> + <d> Host key algorithm. Should support 'ssh-rsa'| 'ssh-dss' but additional algorithms + can be handled.</d> + + <v> ConnectOptions = proplists:proplist() </v> + <d>Options provided to <seealso marker="ssh#daemon">ssh:connect/[3,4]</seealso></d> + + <v> Result = boolean()</v> + </type> + <desc> + <p>Checks if a host key is trusted</p> + </desc> + </func> + + <func> + <name>Module:user_key(Algorithm, ConnectOptions) -> + {ok, PrivateKey} | {error, Reason}</name> + <fsummary>Fetches the users "public key" matching the <c>Algorithm</c>.</fsummary> + <type> + <v>Algorithm = public_key_algorithm()</v> + <d> Host key algorithm. Should support 'ssh-rsa'| 'ssh-dss' but additional algorithms + can be handled.</d> + + <v> ConnectOptions = proplists:proplist() </v> + <d>Options provided to <seealso marker="ssh#daemon">ssh:connect/[3,4]</seealso></d> + + <v> PrivateKey = private_key()</v> + <d> The private key of the user matching the <c>Algorithm</c></d> + + <v>Reason = term() </v> + </type> + + <desc> + <p>Fetches the users "public key" matching the <c>Algorithm</c>. + <note>The private key contains the public key</note> + </p> + </desc> + </func> + + </funcs> + +</erlref> diff --git a/lib/ssh/doc/src/ssh_connection.xml b/lib/ssh/doc/src/ssh_connection.xml index a9ae13d556..c66622307f 100644 --- a/lib/ssh/doc/src/ssh_connection.xml +++ b/lib/ssh/doc/src/ssh_connection.xml @@ -1,11 +1,11 @@ -<?xml version="1.0" encoding="latin1" ?> +<?xml version="1.0" encoding="iso-8859-1" ?> <!DOCTYPE erlref SYSTEM "erlref.dtd"> <erlref> <header> <copyright> <year>2008</year> - <year>2011</year> + <year>2012</year> <holder>Ericsson AB, All Rights Reserved</holder> </copyright> <legalnotice> @@ -24,69 +24,178 @@ </legalnotice> <title>ssh_connection</title> - <prepared>Ingela Anderton Andin</prepared> - <responsible></responsible> - <docno></docno> - <approved></approved> - <checked></checked> <date></date> - <rev></rev> </header> <module>ssh_connection</module> - <modulesummary>This module provides an API to the ssh connection protocol. + <modulesummary>This module provides API functions to send <url href="http://www.ietf.org/rfc/rfc4254.txt"> SSH Connection Protocol </url> + events to the other side of an SSH channel. </modulesummary> + <description> - <p>This module provides an API to the ssh connection protocol. - Not all features of the connection protocol are officially supported yet. - Only the ones supported are documented here.</p> + <p>The SSH Connection Protocol is used by clients and servers + (i.e. SSH channels) to communicate over the SSH connection. The + API functions in this module sends SSH Connection Protocol events + that are received as messages by the remote channel. + In the case that the receiving channel is an Erlang process the + message will be on the following format + <c><![CDATA[{ssh_cm, ssh_connection_ref(), ssh_event_msg()}]]></c>. If the <seealso + marker="ssh_channel">ssh_channel</seealso> behavior is used to + implement the channel process these will be handled by + <seealso + marker="ssh_channel#CallbackModule:handled_ssh_msg-2">handle_ssh_msg/2 </seealso>.</p> </description> - <section> - <title>COMMON DATA TYPES </title> + <section> + <title>DATA TYPES </title> + <p>Type definitions that are used more than once in this module and/or abstractions to indicate the intended use of the data type:</p> - + <p><c>boolean() = true | false </c></p> <p><c>string() = list of ASCII characters</c></p> <p><c>timeout() = infinity | integer() - in milliseconds.</c></p> <p><c>ssh_connection_ref() - opaque to the user returned by - ssh:connect/3 or sent to a ssh channel processes</c></p> + ssh:connect/3 or sent to an SSH channel processes</c></p> <p><c>ssh_channel_id() = integer() </c></p> <p><c>ssh_data_type_code() = 1 ("stderr") | 0 ("normal") are - currently valid values see RFC 4254 section 5.2.</c></p> + currently valid values see</c> <url href="http://www.ietf.org/rfc/rfc4254.txt">RFC 4254 </url> section 5.2.</p> <p><c>ssh_request_status() = success | failure</c></p> - </section> + <p><c>event() = {ssh_cm, ssh_connection_ref(), ssh_event_msg()} </c></p> + <p><c>ssh_event_msg() = data_events() | status_events() | terminal_events() </c></p> + + <taglist> + <tag><b>data_events()</b></tag> + <item> + <taglist> + <tag><c><![CDATA[{data, ssh_channel_id(), ssh_data_type_code(), binary() = Data}]]></c></tag> + <item> Data has arrived on the channel. This event is sent as + result of calling <seealso marker="#send-3"> ssh_connection:send/[3,4,5] </seealso></item> + + <tag><c><![CDATA[{eof, ssh_channel_id()}]]></c></tag> + <item>Indicates that the other side will not send any more + data. This event is sent as result of calling <seealso + marker="#send_eof-2"> ssh_connection:send_eof/2</seealso> + </item> + </taglist> + </item> + + <tag><b>status_events()</b></tag> + <item> + + <taglist> + <tag><c><![CDATA[{signal, ssh_channel_id(), ssh_signal()}]]></c></tag> + <item>A signal can be delivered to the remote process/service + using the following message. Some systems will not support + signals, in which case they should ignore this message. There is + currently no funtion to generate this event as the signals + refered to are on OS-level and not something generated by an + Erlang program.</item> + + <tag><c><![CDATA[{exit_signal, ssh_channel_id(), string() = ExitSignal, string() = ErrorMsg, + string() = LanguageString}]]></c></tag> + + <item>A remote execution may terminate violently due to a signal + then this message may be received. For details on valid string + values see <url href="http://www.ietf.org/rfc/rfc4254.txt">RFC 4254</url> section 6.10. Special case of the signals + mentioned above.</item> + + <tag><c><![CDATA[{exit_status, ssh_channel_id(), integer() = ExitStatus}]]></c></tag> + <item> When the command running at the other end terminates, the + following message can be sent to return the exit status of the + command. A zero 'exit_status' usually means that the command + terminated successfully. This event is sent as result of calling + <seealso marker="#exit_status"> + ssh_connection:exit_status/3</seealso></item> + + <tag><c><![CDATA[{closed, ssh_channel_id()}]]></c></tag> + <item> This event is sent as result of calling + <seealso marker="#close">ssh_connection:close/2</seealso> Both the handling of this + event and sending of it will be taken care of by the + <seealso marker="ssh_channel">ssh_channel</seealso> behavior.</item> + + </taglist> + </item> - <section> - <title>MESSAGES SENT TO CHANNEL PROCESSES</title> + <tag><b>terminal_events()</b></tag> + + <item> + <p> Channels implementing a shell and command execution on the + server side should handle the following messages that may be sent by client channel processes. </p> + + <p><note>Events that includes a <c> WantReply</c> expects the event handling + process to call <seealso marker="#reply_request">ssh_connection:reply_request/4</seealso> + with the boolean value of <c> WantReply</c> as the second + argument. </note> </p> + + <taglist> + <tag><c><![CDATA[{env, ssh_channel_id(), boolean() = WantReply, + string() = Var, string() = Value}]]></c></tag> + <item> Environment variables may be passed to the shell/command + to be started later. This event is sent as result of calling <seealso + marker="#setenv"> ssh_connection:setenv/5</seealso> + </item> + + <tag><c><![CDATA[{pty, ssh_channel_id(), + boolean() = WantReply, {string() = Terminal, integer() = CharWidth, + integer() = RowHeight, integer() = PixelWidth, integer() = PixelHight, + [{atom() | integer() = Opcode, + integer() = Value}] = TerminalModes}}]]></c></tag> + <item>A pseudo-terminal has been requested for the + session. Terminal is the value of the TERM environment + variable value (e.g., vt100). Zero dimension parameters must + be ignored. The character/row dimensions override the pixel + dimensions (when nonzero). Pixel dimensions refer to the + drawable area of the window. The <c>Opcode</c> in the + <c>TerminalModes</c> list is the mnemonic name, represented + as an lowercase erlang atom, defined in + <url href="http://www.ietf.org/rfc/rfc4254.txt">RFC 4254 </url> section 8, + or the opcode if the mnemonic name is not listed in the + RFC. Example <c>OP code: 53, mnemonic name ECHO erlang atom: + echo</c>. There is currently no API function to generate this + event.</item> + + <tag><c><![CDATA[{shell, boolean() = WantReply}]]></c></tag> + <item> This message will request that the user's default shell + be started at the other end. This event is sent as result of calling <seealso + marker="#shell"> ssh_connection:shell/2</seealso> + </item> + + <tag><c><![CDATA[{window_change, ssh_channel_id(), integer() = CharWidth, + integer() = RowHeight, integer() = PixWidth, integer() = PixHeight}]]></c></tag> + <item> When the window (terminal) size changes on the client + side, it MAY send a message to the server side to inform it of + the new dimensions. There is currently no API function to generate this + event.</item> - <p>As a result of the ssh connection protocol messages on the form - <c><![CDATA[{ssh_cm, ssh_connection_ref(), term()}]]></c> - will be sent to a channel process. The term will contain - information regarding the ssh connection protocol event, - for details see the ssh channel behavior callback <seealso - marker="ssh_channel">handle_ssh_msg/2 </seealso> </p> - </section> - - <funcs> + <tag><c><![CDATA[{exec, ssh_channel_id(), + boolean() = WantReply, string() = Cmd}]]></c></tag> + <item> This message will request that the server starts + execution of the given command. This event is sent as result of calling <seealso + marker="#exec">ssh_connection:exec/4 </seealso> + </item> + </taglist> + </item> + </taglist> + </section> + + <funcs> <func> <name>adjust_window(ConnectionRef, ChannelId, NumOfBytes) -> ok</name> - <fsummary>Adjusts the ssh flowcontrol window. </fsummary> + <fsummary>Adjusts the SSH flowcontrol window. </fsummary> <type> <v> ConnectionRef = ssh_connection_ref() </v> <v> ChannelId = ssh_channel_id() </v> <v> NumOfBytes = integer()</v> </type> <desc> - <p>Adjusts the ssh flowcontrol window. </p> + <p>Adjusts the SSH flowcontrol window. This shall be done by both client and server side channel processes.</p> - <note><p>This will be taken care of by the ssh_channel - behavior when the callback <seealso marker="ssh_channel"> - handle_ssh_msg/2 </seealso> has returned after processing a - {ssh_cm, ssh_connection_ref(), {data, ssh_channel_id(), - ssh_data_type_code(), binary()}} - message, and should normally not be called explicitly.</p></note> + <note><p>Channels implemented with the <seealso marker="ssh_channel"> ssh_channel + behavior</seealso> will normaly not need to call this function as flow control + will be handled by the behavior. The behavior will adjust the window every time + the callback <seealso marker="ssh_channel#handled_ssh_msg-2"> + handle_ssh_msg/2 </seealso> has returned after processing channel data</p> </note> </desc> </func> @@ -98,20 +207,19 @@ <v> ChannelId = ssh_channel_id()</v> </type> <desc> - <p>Sends a close message on the channel <c>ChannelId</c> + <p>A server or client channel process can choose to close their session by sending a close event. </p> - <note><p>This function will be called by the ssh channel + <note><p>This function will be called by the ssh_channel behavior when the channel is terminated see <seealso - marker="ssh_channel"> ssh_channel(3) </seealso> and should - normally not be called explicitly.</p></note> + marker="ssh_channel"> ssh_channel(3) </seealso> so channels implemented with the + behavior should not call this function explicitly.</p></note> </desc> </func> <func> <name>exec(ConnectionRef, ChannelId, Command, TimeOut) -> ssh_request_status() </name> - <fsummary>Will request that the server start the - execution of the given command. </fsummary> + <fsummary>Request that the server start the execution of the given command. </fsummary> <type> <v> ConnectionRef = ssh_connection_ref() </v> <v> ChannelId = ssh_channel_id()</v> @@ -119,44 +227,39 @@ <v>Timeout = timeout() </v> </type> <desc> - <p>Will request that the server start the execution of the - given command, the result will be received as:</p> + <p>Should be called by a client channel process to request that the server starts execution of the + given command, the result will be several messages according to the following pattern. Note + that the last message will be a channel close message, as the exec request is a one time + execution that closes the channel when it is done.</p> <taglist> - <tag><c> N X {ssh_cm, - ssh_connection_ref(), {data, ssh_channel_id(), ssh_data_type_code(), - binary() = Data}} </c></tag> + <tag><c> N x {ssh_cm, ssh_connection_ref(), + {data, ssh_channel_id(), ssh_data_type_code(), binary() = Data}} </c></tag> <item>The result of executing the command may be only one line or thousands of lines depending on the command.</item> - <tag><c> 1 X {ssh_cm, ssh_connection_ref(), {eof, ssh_channel_id()}}</c></tag> + <tag><c>0 or 1 x {ssh_cm, ssh_connection_ref(), {eof, ssh_channel_id()}}</c></tag> <item>Indicates that no more data will be sent.</item> - <tag><c>0 or 1 X {ssh_cm, + <tag><c>0 or 1 x {ssh_cm, ssh_connection_ref(), {exit_signal, ssh_channel_id(), string() = ExitSignal, string() = ErrorMsg, string() = LanguageString}}</c></tag> <item>Not all systems send signals. For details on valid string values see RFC 4254 section 6.10 </item> - <tag><c>0 or 1 X {ssh_cm, ssh_connection_ref(), {exit_status, + <tag><c>0 or 1 x {ssh_cm, ssh_connection_ref(), {exit_status, ssh_channel_id(), integer() = ExitStatus}}</c></tag> <item>It is recommended by the <c>ssh connection protocol</c> that this message shall be sent, but that may not always be the case.</item> - <tag><c> 1 X {ssh_cm, ssh_connection_ref(), + <tag><c> 1 x {ssh_cm, ssh_connection_ref(), {closed, ssh_channel_id()}}</c></tag> <item>Indicates that the ssh channel started for the execution of the command has now been shutdown.</item> </taglist> - - <p> These message should be handled by the - client. The <seealso marker="ssh_channel">ssh channel - behavior</seealso> can be used when writing a client. - </p> </desc> </func> - <func> <name>exit_status(ConnectionRef, ChannelId, Status) -> ok</name> <fsummary>Sends the exit status of a command to the client.</fsummary> @@ -166,12 +269,12 @@ <v> Status = integer()</v> </type> <desc> - <p>Sends the exit status of a command to the client.</p> + <p>Should be called by a server channel process to sends the exit status of a command to the client.</p> </desc> - </func> + </func> <func> - <name>reply_request(ConnectionRef, WantReply, Status, CannelId) -> ok</name> + <name>reply_request(ConnectionRef, WantReply, Status, ChannelId) -> ok</name> <fsummary>Send status replies to requests that want such replies. </fsummary> <type> <v> ConnectionRef = ssh_connection_ref() </v> @@ -183,10 +286,9 @@ <p>Sends status replies to requests where the requester has stated that they want a status report e.i .<c> WantReply = true</c>, if <c> WantReply</c> is false calling this function will be a - "noop". Should be called after handling an ssh connection + "noop". Should be called while handling an ssh connection protocol message containing a <c>WantReply</c> boolean - value. See the ssh_channel behavior callback <seealso - marker="ssh_channel"> handle_ssh_msg/2 </seealso> + value. </p> </desc> </func> @@ -206,7 +308,7 @@ <v> Timeout = timeout()</v> </type> <desc> - <p>Sends channel data. + <p>Should be called by client- and server channel processes to send data to each other. </p> </desc> </func> @@ -228,9 +330,7 @@ <name>session_channel(ConnectionRef, Timeout) -> </name> <name>session_channel(ConnectionRef, InitialWindowSize, MaxPacketSize, Timeout) -> {ok, ssh_channel_id()} | {error, Reason}</name> - <fsummary>Opens a channel for a ssh session. A session is a - remote execution of a program. The program may be a shell, an - application, a system command, or some built-in subsystem. </fsummary> + <fsummary>Opens a channel for a ssh session. </fsummary> <type> <v> ConnectionRef = ssh_connection_ref()</v> <v> InitialWindowSize = integer() </v> @@ -239,9 +339,8 @@ <v> Reason = term() </v> </type> <desc> - <p>Opens a channel for a ssh session. A session is a - remote execution of a program. The program may be a shell, an - application, a system command, or some built-in subsystem. + <p>Opens a channel for an SSH session. The channel id returned from this function + is the id used as input to the other funtions in this module. </p> </desc> </func> @@ -258,8 +357,8 @@ <v> Timeout = timeout()</v> </type> <desc> - <p> Environment variables may be passed to the shell/command to be - started later. + <p> Environment variables may be passed before starting the + shell/command. Should be called by a client channel processes. </p> </desc> </func> @@ -267,17 +366,16 @@ <func> <name>shell(ConnectionRef, ChannelId) -> ssh_request_status() </name> - <fsummary> Will request that the user's default shell (typically - defined in /etc/passwd in UNIX systems) be started at the other + <fsummary> Requests that the user's default shell (typically + defined in /etc/passwd in UNIX systems) shall be executed at the server end. </fsummary> <type> <v> ConnectionRef = ssh_connection_ref() </v> <v> ChannelId = ssh_channel_id()</v> </type> <desc> - <p> Will request that the user's default shell (typically - defined in /etc/passwd in UNIX systems) be started at the - other end. + <p> Should be called by a client channel process to request that the user's default shell (typically + defined in /etc/passwd in UNIX systems) shall be executed at the server end. </p> </desc> </func> @@ -292,7 +390,7 @@ <v> Timeout = timeout()</v> </type> <desc> - <p> Sends a request to execute a predefined subsystem. + <p> Should be called by a client channel process for requesting to execute a predefined subsystem on the server. </p> </desc> </func> diff --git a/lib/ssh/doc/src/ssh_protocol.xml b/lib/ssh/doc/src/ssh_protocol.xml new file mode 100644 index 0000000000..6a253c43eb --- /dev/null +++ b/lib/ssh/doc/src/ssh_protocol.xml @@ -0,0 +1,150 @@ +<?xml version="1.0" encoding="iso-8859-1" ?> +<!DOCTYPE chapter SYSTEM "chapter.dtd"> +<!-- %EricssonCopyright% --> +<chapter> + <header> + <copyright> + <year>2012</year> + <holder>Ericsson AB. All Rights Reserved.</holder> + </copyright> + <legalnotice> + The program may be used and/or copied only with the written permission from + Ericsson AB, or in accordance with the terms and conditions stipulated in + the agreement/contract under which the program has been supplied. + </legalnotice> + <title>Secure Shell (SSH)</title> + <prepared>OTP</prepared> + <date></date> + <rev>%VSN%</rev> + <file>ssh_protocol.xml</file> + </header> + + <section> + <title>SSH Protocol Overview</title> + + <p> Conceptually the SSH protocol can be partitioned into four + layers:</p> + + <image file="SSH_protocols.png"> + <icaption>SSH Protocol Architecture</icaption> + </image> + + <section> + <title>Transport Protocol</title> + + <p> The SSH Transport Protocol is a secure, low level transport. + It provides strong encryption, cryptographic host + authentication and integrity protection. Currently, only a + minimum of MAC- (message authentication code, a short piece of + information used to authenticate a message) and encryption + algorithms are supported see <seealso marker="ssh">ssh(3)</seealso> + </p> + </section> + + <section> + <title>Authentication Protocol</title> + + <p>The SSH authentication protocol is a general-purpose user + authentication protocol run over the SSH transport + protocol. Erlang SSH supports user authentication using public + key technology (RSA and DSA, X509-certificates are currently not + supported). It is also possible to use a so called keyboard + interactive authentication. This method is suitable for + interactive authentication methods that do not need any special + software support on the client side. Instead, all authentication + data should be entered via the keyboard. It is also possible + to use a pure password based authentication scheme, note that in + this case the the plain text password will be encrypted before sent + over the network. There are several configuration options for + authentication handling available in + <seealso marker="ssh#connect-3">ssh:connect/[3,4]</seealso> + and <seealso marker="ssh#daemon-2">ssh:daemon/[2,3]</seealso> + It is also possible to customize the public key handling + by implementing the behaviours <seealso + marker="ssh_client_key_api">ssh_client_key_api</seealso> and + <seealso + marker="ssh_server_key_api">ssh_server_key_api</seealso> + </p> + </section> + + <section> + <title>Connection Protocol</title> + + <p>The SSH Connection Protocol provides application-support + services over the transport pipe, such as channel multiplexing, + flow control, remote program execution, signal propagation, + connection forwarding, etc. Functions for handling the SSH + Connection Protocol can be found in the module <seealso + marker="ssh_connection">ssh_connection</seealso>. + </p> + </section> + + <section> + <title>Channels</title> + + <p>All terminal sessions, forwarded connections etc., are + channels. Multiple channels are multiplexed into a single + connection, and all channels are flow-controlled. Typically an + SSH client will open a channel, send data/commands, receive + data/"control information" and when it is done close the + channel. The <seealso + marker="ssh_channel">ssh_channel</seealso> behaviour makes it easy to + write your own SSH client/server processes that use flow + control. It handles generic parts of SSH channel management and + lets you focus on the application logic. + </p> + + <p>Channels comes in three flavors</p> + + <list type="bulleted"> + <item><em>Subsystem</em> - named services that can be run as + part of an SSH server such as SFTP <seealso + marker="ssh_sftpd">ssh_sftpd</seealso>, that is built in to the + SSH daemon (server) by default but may be disabled. The Erlang SSH + daemon may be configured to run any Erlang + implemented SSH subsystem. + </item> + <item><em>Shell</em> - interactive shell. By default the + Erlang daemon will run the Erlang shell. It is + possible to customize the shell by providing your own + read-eval-print loop. It is also possible, but much more work, + to provide your own CLI (Command Line Interface) implementation. + </item> + <item><em>Exec</em> - one-time remote execution (like + SCP). See <seealso + marker="ssh_connection#exec-4">ssh_connection:exec/4</seealso></item> + </list> + </section> + + <p>Channels are flow controlled. No data may be sent to a channel + peer until a message is received to indicate that window space is + available. The 'initial window size' specifies how many bytes of + channel data that can be sent to the channel peer without adjusting the + window. + </p> + + <p> + For more detailed information about the SSH protocol, see the + following RFCs: + </p> + + <list type="bulleted"> + <item><url href="http://www.ietf.org/rfc/rfc4250.txt">RFC 4250</url> - + Protocol Assigned Numbers.</item> + <item><url href="http://www.ietf.org/rfc/rfc4251.txt">RFC 4251</url> - + Protocol Architecture.</item> + <item><url href="http://www.ietf.org/rfc/rfc4252.txt">RFC 4252</url> - + Authentication Protocol.</item> + <item><url href="http://www.ietf.org/rfc/rfc4253.txt">RFC 4253</url> - + Transport Layer Protocol.</item> + <item><url href="http://www.ietf.org/rfc/rfc4254.txt">RFC 4254</url> - + Connection Protocol.</item> + <item><url href="http://www.ietf.org/rfc/rfc4255.txt">RFC 4255</url> - + Key Fingerprints.</item> + <item><url href="http://www.ietf.org/rfc/rfc4344.txt">RFC 4344</url> - + Transport Layer Encryption Modes.</item> + <item><url href="http://www.ietf.org/rfc/rfc4716.txt">RFC 4716</url> - + Public Key File Format.</item> + </list> + </section> +</chapter> diff --git a/lib/ssh/doc/src/ssh_server_key_api.xml b/lib/ssh/doc/src/ssh_server_key_api.xml new file mode 100644 index 0000000000..78ff105387 --- /dev/null +++ b/lib/ssh/doc/src/ssh_server_key_api.xml @@ -0,0 +1,90 @@ +<?xml version="1.0" encoding="iso-8859-1" ?> +<!DOCTYPE erlref SYSTEM "erlref.dtd"> + +<erlref> + <header> + <copyright> + <year>2012</year> + <holder>Ericsson AB, All Rights Reserved</holder> + </copyright> + <legalnotice> + The contents of this file are subject to the Erlang Public License, + Version 1.1, (the "License"); you may not use this file except in + compliance with the License. You should have received a copy of the + Erlang Public License along with this software. If not, it can be + retrieved online at http://www.erlang.org/. + + Software distributed under the License is distributed on an "AS IS" + basis, WITHOUT WARRANTY OF ANY KIND, either express or implied. See + the License for the specific language governing rights and limitations + under the License. + + The Initial Developer of the Original Code is Ericsson AB. + </legalnotice> + <title>ssh_server_key_api</title> + </header> + <module>ssh_server_key_api</module> + <modulesummary> + -behaviour(ssh_server_key_api). + </modulesummary> + <description> + <p> Behaviour describing the API for an SSH server's public key handling.By implementing the callbacks defined + in this behavior it is possible to customize the SSH server's public key + handling. By default the SSH application implements this behavior + with help of the standard openssh files, see <seealso marker="SSH_app"> ssh(6)</seealso>.</p> + </description> + + <section> + <title>DATA TYPES </title> + + <p>Type definitions that are used more than once in this module + and/or abstractions to indicate the intended use of the data + type:</p> + + <p> boolean() = true | false</p> + <p> string() = [byte()]</p> + <p> public_key() = #'RSAPublicKey'{}| {integer(), #'Dss-Parms'{}}| term()</p> + <p> private_key() = #'RSAPublicKey'{}| {integer(), #'Dss-Parms'{}}| term()</p> + <p> public_key_algorithm() = 'ssh-rsa'| 'ssh-dss' | atom()</p> + </section> + + <funcs> + <func> + <name>Module:host_key(Algorithm, DaemonOptions) -> + {ok, Key} | {error, Reason}</name> + <fsummary>Fetches the hosts private key </fsummary> + <type> + <v>Algorithm = public_key_algorithm()</v> + <d> Host key algorithm. Should support 'ssh-rsa'| 'ssh-dss' but additional algorithms + can be handled.</d> + <v> DaemonOptions = proplists:proplist() </v> + <d>Options provided to <seealso marker="ssh#daemon">ssh:daemon/[2,3]</seealso></d> + <v> Key = private_key()</v> + <d> The private key of the host matching the <c>Algorithm</c></d> + <v>Reason = term() </v> + </type> + <desc> + <p>Fetches the hosts private key</p> + </desc> + </func> + + <func> + <name>Module:is_auth_key(Key, User, DaemonOptions) -> Result</name> + <fsummary> Checks if the user key is authorized</fsummary> + <type> + <v> Key = public_key() </v> + <d> Normally an RSA or DSA public key but handling of other public keys can be added</d> + <v> User = string()</v> + <d> The user owning the public key</d> + <v> DaemonOptions = proplists:proplist() </v> + <d> Options provided to <seealso marker="ssh#daemon">ssh:daemon/[2,3]</seealso></d> + <v> Result = boolean()</v> + </type> + <desc> + <p> Checks if the user key is authorized </p> + </desc> + </func> + + </funcs> + +</erlref> diff --git a/lib/ssh/doc/src/ssh_sftp.xml b/lib/ssh/doc/src/ssh_sftp.xml index c1f75461b1..0d61e57edb 100644 --- a/lib/ssh/doc/src/ssh_sftp.xml +++ b/lib/ssh/doc/src/ssh_sftp.xml @@ -1,10 +1,10 @@ -<?xml version="1.0" encoding="latin1" ?> +<?xml version="1.0" encoding="iso-8859-1" ?> <!DOCTYPE erlref SYSTEM "erlref.dtd"> <erlref> <header> <copyright> - <year>2005</year><year>2010</year> + <year>2005</year><year>2012</year> <holder>Ericsson AB. All Rights Reserved.</holder> </copyright> <legalnotice> @@ -22,25 +22,20 @@ </legalnotice> <title>ssh_sftp</title> - <prepared>Jakob Cederlund</prepared> - <responsible></responsible> - <docno>1</docno> - <approved></approved> - <checked></checked> + <prepared>OTP</prepared> <date>2005-09-22</date> - <rev>PA1</rev> <file>ssh_sftp.sgml</file> </header> <module>ssh_sftp</module> <modulesummary>SFTP client.</modulesummary> <description> <p>This module implements an SFTP (SSH FTP) client. SFTP is a - secure, encrypted file transfer service available for - SSH.</p> + secure, encrypted file transfer service available for + SSH.</p> </description> <section> - <title>COMMON DATA TYPES </title> + <title>DATA TYPES </title> <p>Type definitions that are used more than once in this module and/or abstractions to indicate the intended use of the data type: </p> @@ -51,8 +46,8 @@ <section> <title>TIMEOUTS </title> - <p>If the request functions for the sftp channel return {error, timeout} - it does not mean that the request did not reach the server and was + <p>If the request functions for the SFTP channel return {error, timeout} + it does not guarantee that the request did not reach the server and was not performed, it only means that we did not receive an answer from the server within the time that was expected.</p> </section> @@ -64,7 +59,7 @@ <name>start_channel(Host, Options) -></name> <name>start_channel(Host, Port, Options) -> {ok, Pid} | {ok, Pid, ConnectionRef} | {error, Reason}</name> - <fsummary>Starts a sftp client</fsummary> + <fsummary>Starts a SFTP client</fsummary> <type> <v>Host = string()</v> <v>ConnectionRef = ssh_connection_ref()</v> @@ -73,11 +68,11 @@ <v>Reason = term()</v> </type> <desc> - <p>If not provided, setups a ssh connection in this case a - connection reference will be returned too. A ssh channel - process is started to handle the communication with the SFTP - server, the returned pid for this process should be used as - input to all other API functions in this module.</p> + <p>If no connection reference is provided, a connection is set + up and the new connection is returned. An SSH channel process + is started to handle the communication with the SFTP server. + The returned pid for this process should be used as input to + all other API functions in this module.</p> <p>Options are:</p> <taglist> @@ -95,13 +90,13 @@ <func> <name>stop_channel(ChannelPid) -> ok</name> - <fsummary>Stops the sftp client channel.</fsummary> + <fsummary>Stops the SFTP client channel.</fsummary> <type> <v>ChannelPid = pid()</v> </type> <desc> - <p>Stops a sftp channel. If the ssh connection should be closed - call <seealso marker="ssh">ssh:close/1</seealso>.</p> + <p>Stops an SFTP channel. Does not close the SSH connetion. + Use <seealso marker="ssh">ssh:close/1</seealso> to close it.</p> </desc> </func> @@ -133,8 +128,9 @@ <v>Reason = term()</v> </type> <desc> - <p>Writes a file to the server, like <c><![CDATA[file:write_file/2]]></c>. - The file is created if it's not there.</p> + <p>Writes a file to the server, like + <c><![CDATA[file:write_file/2]]></c>. The file is created if + it does not exist or is owerwritten if it does.</p> </desc> </func> <func> @@ -169,7 +165,7 @@ </type> <desc> <p>Opens a file on the server, and returns a handle that - is used for reading or writing.</p> + can be used for reading or writing.</p> </desc> </func> <func> @@ -184,7 +180,7 @@ </type> <desc> <p>Opens a handle to a directory on the server, the handle - is used for reading directory contents.</p> + can be used for reading directory contents.</p> </desc> </func> <func> @@ -218,7 +214,7 @@ </type> <desc> <p>Reads <c><![CDATA[Len]]></c> bytes from the file referenced by - <c><![CDATA[Handle]]></c>. Returns <c><![CDATA[{ok, Data}]]></c>, or <c><![CDATA[eof]]></c>, or + <c><![CDATA[Handle]]></c>. Returns <c><![CDATA[{ok, Data}]]></c>, <c><![CDATA[eof]]></c>, or <c><![CDATA[{error, Reason}]]></c>. If the file is opened with <c><![CDATA[binary]]></c>, <c><![CDATA[Data]]></c> is a binary, otherwise it is a string.</p> <p>If the file is read past eof, only the remaining bytes @@ -267,9 +263,9 @@ <v>Reason = term()</v> </type> <desc> - <p>Write <c><![CDATA[data]]></c> to the file referenced by <c><![CDATA[Handle]]></c>. + <p>Writes<c><![CDATA[data]]></c> to the file referenced by <c><![CDATA[Handle]]></c>. The file should be opened with <c><![CDATA[write]]></c> or <c><![CDATA[append]]></c> - flag. Returns <c><![CDATA[ok]]></c> if successful and <c><![CDATA[{error, Reason}]]></c> + flag. Returns <c><![CDATA[ok]]></c> if successful or S<c><![CDATA[{error, Reason}]]></c> otherwise.</p> <p>Typical error reasons are:</p> <taglist> @@ -317,14 +313,14 @@ <v>ChannelPid = pid()</v> <v>Handle = term()</v> <v>Location = Offset | {bof, Offset} | {cur, Offset} | {eof, Offset} | bof | cur | eof</v> - <v>Offset = int()</v> + <v>Offset = integer()</v> <v>Timeout = timeout()</v> <v>NewPosition = integer()</v> <v>Reason = term()</v> </type> <desc> <p>Sets the file position of the file referenced by <c><![CDATA[Handle]]></c>. - Returns <c><![CDATA[{ok, NewPosition]]></c> (as an absolute offset) if + Returns <c><![CDATA[{ok, NewPosition}]]></c> (as an absolute offset) if successful, otherwise <c><![CDATA[{error, Reason}]]></c>. <c><![CDATA[Location]]></c> is one of the following:</p> <taglist> @@ -413,7 +409,7 @@ <v>Reason = term()</v> </type> <desc> - <p>Read the link target from the symbolic link specified + <p>Reads the link target from the symbolic link specified by <c><![CDATA[name]]></c>, like <c><![CDATA[file:read_link/1]]></c>.</p> </desc> </func> @@ -490,8 +486,9 @@ <v>Reason = term()</v> </type> <desc> - <p>Deletes a directory specified by <c><![CDATA[Name]]></c>. The directory - should be empty.</p> + <p>Deletes a directory specified by <c><![CDATA[Name]]></c>. + Note that the directory must be empty before it can be successfully deleted + </p> </desc> </func> diff --git a/lib/ssh/doc/src/ssh_sftpd.xml b/lib/ssh/doc/src/ssh_sftpd.xml index b3d64e72b4..3666bc7692 100644 --- a/lib/ssh/doc/src/ssh_sftpd.xml +++ b/lib/ssh/doc/src/ssh_sftpd.xml @@ -1,10 +1,10 @@ -<?xml version="1.0" encoding="latin1" ?> +<?xml version="1.0" encoding="iso-8859-1" ?> <!DOCTYPE erlref SYSTEM "erlref.dtd"> <erlref> <header> <copyright> - <year>2005</year><year>2010</year> + <year>2005</year><year>2012</year> <holder>Ericsson AB. All Rights Reserved.</holder> </copyright> <legalnotice> @@ -22,23 +22,17 @@ </legalnotice> <title>ssh_sftpd</title> - <prepared>Ingela Anderton Andin</prepared> - <responsible></responsible> - <docno>1</docno> - <approved></approved> - <checked></checked> <date>2005-09-22</date> - <rev>PA1</rev> <file>ssh_sftpd.sgml</file> </header> <module>ssh_sftpd</module> - <modulesummary>Specifies a channel process to handle a sftp subsystem.</modulesummary> + <modulesummary>Specifies the channel process to handle an sftp subsystem.</modulesummary> <description> <p>Specifies a channel process to handle a sftp subsystem.</p> </description> <section> - <title>COMMON DATA TYPES </title> + <title>DATA TYPES </title> <p><c>subsystem_spec() = {subsystem_name(), {channel_callback(), channel_init_args()}} </c></p> <p><c>subsystem_name() = "sftp"</c></p> <p><c>channel_callback() = atom()</c> - Name of the erlang module implementing the @@ -65,27 +59,32 @@ </item> <tag><c><![CDATA[{file_handler, CallbackModule}]]></c></tag> <item> - <p>Determines which module to call for communicating with - the file server. Default value is <c>ssh_sftpd_file</c> that uses the - file and filelib API:s to access the standard OTP file - server. This option may be used to plug in the use of - other file servers.</p> - </item> - <tag><c><![CDATA[{max_files, Integer}]]></c></tag> - <item> - <p>The default value is <c>0</c>, which means that there is no upper limit. - If supplied, the number of filenames returned to the sftp client per <c>READDIR</c> - request, is limited to at most the given value.</p> - </item> + <p>Determines which module to call for accessing + the file server. The default value is <c>ssh_sftpd_file</c> that uses the + <seealso marker="kernel#file">file</seealso> and <seealso marker="kernel#filelib">filelib</seealso> API:s to access the standard OTP file + server. This option may be used to plug in + other file servers.</p> + </item> + <tag><c><![CDATA[{max_files, Integer}]]></c></tag> + <item> + <p>The default value is <c>0</c>, which means that there is no upper limit. + If supplied, the number of filenames returned to the sftp client per <c>READDIR</c> + request is limited to at most the given value.</p> + </item> <tag><c><![CDATA[{root, String}]]></c></tag> <item> <p>Sets the sftp root directory. The user will then not be - able to see any files above this root. If for instance - the root is set to <c>/tmp</c> the user will see this - directory as <c>/</c> and if the user does cd <c>/etc</c> - the user will end up in <c>/tmp/etc</c>. + able to see any files above this root. If for instance + the root is set to <c>/tmp</c> the user will see this + directory as <c>/</c> and if the user does cd <c>/etc</c> + the user will end up in <c>/tmp/etc</c>. </p> </item> + <tag><c><![CDATA[{sftpd_vsn, integer()}]]></c></tag> + <item> + <p>Sets the sftp version to use, defaults to 5. Version 6 is under + development and limited.</p> + </item> </taglist> </desc> </func> diff --git a/lib/ssh/doc/src/user_guide.gif b/lib/ssh/doc/src/user_guide.gif Binary files differdeleted file mode 100644 index e6275a803d..0000000000 --- a/lib/ssh/doc/src/user_guide.gif +++ /dev/null diff --git a/lib/ssh/doc/src/usersguide.xml b/lib/ssh/doc/src/usersguide.xml new file mode 100644 index 0000000000..c818003090 --- /dev/null +++ b/lib/ssh/doc/src/usersguide.xml @@ -0,0 +1,37 @@ +<?xml version="1.0" encoding="iso-8859-1" ?> +<!DOCTYPE part SYSTEM "part.dtd"> + +<part xmlns:xi="http://www.w3.org/2001/XInclude"> + <header> + <copyright> + <year>2012</year> + <holder>Ericsson AB. All Rights Reserved.</holder> + </copyright> + <legalnotice> + The contents of this file are subject to the Erlang Public License, + Version 1.1, (the "License"); you may not use this file except in + compliance with the License. You should have received a copy of the + Erlang Public License along with this software. If not, it can be + retrieved online at http://www.erlang.org/. + + Software distributed under the License is distributed on an "AS IS" + basis, WITHOUT WARRANTY OF ANY KIND, either express or implied. See + the License for the specific language governing rights and limitations + under the License. + + </legalnotice> + + <title>SSH User's Guide</title> + <prepared>OTP Team</prepared> + <date>2012-10-11</date> + <file>usersguide.xml</file> + </header> + <description> + <p>The <em>SSH</em> application implements the SSH (Secure Shell) protocol and + provides an SFTP (Secret File Transfer Protocol) client and server. + </p> + </description> + <xi:include href="introduction.xml"/> + <xi:include href="ssh_protocol.xml"/> + <xi:include href="using_ssh.xml"/> +</part> diff --git a/lib/ssh/doc/src/using_ssh.xml b/lib/ssh/doc/src/using_ssh.xml new file mode 100644 index 0000000000..87b811d591 --- /dev/null +++ b/lib/ssh/doc/src/using_ssh.xml @@ -0,0 +1,298 @@ +<?xml version="1.0" encoding="iso-8859-1" ?> +<!DOCTYPE chapter SYSTEM "chapter.dtd"> + +<chapter> + <header> + <copyright> + <year>2012</year> + <year>2013</year> + <holder>Ericsson AB. All Rights Reserved.</holder> + </copyright> + <legalnotice> + The contents of this file are subject to the Erlang Public License, + Version 1.1, (the "License"); you may not use this file except in + compliance with the License. You should have received a copy of the + Erlang Public License along with this software. If not, it can be + retrieved online at http://www.erlang.org/. + + Software distributed under the License is distributed on an "AS IS" + basis, WITHOUT WARRANTY OF ANY KIND, either express or implied. See + the License for the specific language governing rights and limitations + under the License. + + </legalnotice> + + <title>Getting started</title> + <file>using_ssh.xml</file> + </header> + + <section> + <title> General information</title> + <p>The examples in the following sections use the utility function + <seealso marker="ssh#start"> ssh:start/0 </seealso> that starts + all needed applications (crypto, public_key and ssh). All examples + are run in an Erlang shell, or in a bash shell using openssh to + illustrate how the erlang ssh application can be used. The + exampels are run as the user otptest on a local network where the + user is authorized to login in over ssh to the host "tarlop". If + nothing else is stated it is persumed that the otptest user has an + entry in tarlop's authorized_keys file (may log in via ssh without + entering a password). Also tarlop is a known host in the user + otptest's known_hosts file so that host verification can be done + without user interaction. + </p> + </section> + + <section> + <title>Using the Erlang SSH Terminal Client</title> + + <p>The user otptest, that has bash as default shell, uses the + ssh:shell/1 client to connect to the openssh daemon running on a + host called tarlop. Note that currently this client is very simple + and you should not be expected to be as fancy as the openssh + client.</p> + + <code type="erl" > + 1> ssh:start(). + ok + 2> {ok, S} = ssh:shell("tarlop"). + >pwd + /home/otptest + >exit + logout + 3> + </code> + </section> + + <section> + <title>Running an Erlang SSH Daemon </title> + + <p> The option system_dir must be a directory containing a host + key file and it defaults to /etc/ssh. For details see section + Configuration Files in <seealso + marker="ssh_app">ssh(6)</seealso>. + </p> + + <note><p>Normally the /etc/ssh directory is only readable by root. </p> + </note> + + <p> The option user_dir defaults to the users ~/.ssh directory</p> + + <p>In the following example we generate new keys and host keys as + to be able to run the example without having root privilages</p> + + <code> + $bash> ssh-keygen -t rsa -f /tmp/ssh_daemon/ssh_host_rsa_key + [...] + $bash> ssh-keygen -t rsa -f /tmp/otptest_user/.ssh/id_rsa + [...] + </code> + + <p>Create the file /tmp/otptest_user/.ssh/authrized_keys and add the content + of /tmp/otptest_user/.ssh/id_rsa.pub Now we can do</p> + + <code type="erl"> + 1> ssh:start(). + ok + 2> {ok, Sshd} = ssh:daemon(8989, [{system_dir, "/tmp/ssh_daemon"}, + {user_dir, "/tmp/otptest_user/.ssh"}]). + {ok,<0.54.0>} + 3> + </code> + + <p>Use the openssh client from a shell to connect to the Erlang ssh daemon.</p> + + <code> + $bash> ssh tarlop -p 8989 -i /tmp/otptest_user/.ssh/id_rsa\ + -o UserKnownHostsFile=/tmp/otptest_user/.ssh/known_hosts + The authenticity of host 'tarlop' can't be established. + RSA key fingerprint is 14:81:80:50:b1:1f:57:dd:93:a8:2d:2f:dd:90:ae:a8. + Are you sure you want to continue connecting (yes/no)? yes + Warning: Permanently added 'tarlop' (RSA) to the list of known hosts. + Eshell V5.10 (abort with ^G) + 1> + </code> + + <p>There are two ways of shutting down an SSH daemon</p> + + <p>1: Stops the listener, but leaves existing connections started by the listener up and running.</p> + + <code type="erl"> + 3> ssh:stop_listener(Sshd). + ok + 4> + </code> + + <p>2: Stops the listener and all connections started by the listener.</p> + + <code type="erl"> + 3> ssh:stop_daemon(Sshd) + ok + 4> + </code> + + </section> + + <section> + <title>One Time Execution</title> + + <p>In the following example the Erlang shell is the client process + that receives the channel replies. <note> If you run this example + in your environment you may get fewer or more messages back as + this depends on the OS and shell on the machine running the ssh + daemon. See also <seealso marker="ssh_connection#exec-4">ssh_connection:exec/4</seealso> + </note> + </p> + + <code type="erl" > + 1> ssh:start(). + ok + 2> {ok, ConnectionRef} = ssh:connect("tarlop", 22, []). + {ok,<0.57.0>} + 3>{ok, ChannelId} = ssh_connection:session_channel(ConnectionRef, infinity). + {ok,0} + 4> success = ssh_connection:exec(ConnectionRef, ChannelId, "pwd", infinity). + 5> flush(). + Shell got {ssh_cm,<0.57.0>,{data,0,0,<<"/home/otptest\n">>}} + Shell got {ssh_cm,<0.57.0>,{eof,0}} + Shell got {ssh_cm,<0.57.0>,{exit_status,0,0}} + Shell got {ssh_cm,<0.57.0>,{closed,0}} + ok + 6> + </code> + + <p>Note only the channel is closed the connection is still up and can handle other channels</p> + + <code type="erl" > + 6> {ok, NewChannelId} = ssh_connection:session_channel(ConnectionRef, infinity). + {ok,1} + ... + </code> + </section> + + <section> + <title>SFTP (SSH File Transport Protocol) server</title> + + <code type="erl" > + 1> ssh:start(). + ok + 2> ssh:daemon(8989, [{system_dir, "/tmp/ssh_daemon"}, + {user_dir, "/tmp/otptest_user/.ssh"}, + {subsystems, [ssh_sftpd:subsystem_spec([{cwd, "/tmp/sftp/example"}])]}]). + {ok,<0.54.0>} + 3> + </code> + + <p> Run the openssh sftp client</p> + + <code type="erl"> + $bash> sftp -oPort=8989 -o IdentityFile=/tmp/otptest_user/.ssh/id_rsa\ + -o UserKnownHostsFile=/tmp/otptest_user/.ssh/known_hosts tarlop + Connecting to tarlop... + sftp> pwd + Remote working directory: /tmp/sftp/example + sftp> + </code> + </section> + + <section> + <title>SFTP (SSH File Transport Protocol) client</title> + + <code type="erl" > + 1> ssh:start(). + ok + 2> {ok, ChannelPid, Connection} = ssh_sftp:start_channel("tarlop", []). + {ok,<0.57.0>,<0.51.0>} + 3> ssh_sftp:read_file(ChannelPid, "/home/otptest/test.txt"). + {ok,<<"This is a test file\n">>} + </code> + </section> + + <section> + <title>Creating a subsystem</title> + + <p>A very small SSH subsystem that echos N bytes could be implemented like this. + See also <seealso marker="ssh_channel"> ssh_channel(3)</seealso> </p> + + <code type="erl" > +-module(ssh_echo_server). +-behaviour(ssh_subsystem). +-record(state, { + n, + id, + cm + }). +-export([init/1, handle_msg/2, handle_ssh_msg/2, terminate/2]). + +init([N]) -> + {ok, #state{n = N}}. + +handle_msg({ssh_channel_up, ChannelId, ConnectionManager}, State) -> + {ok, State#state{id = ChannelId, + cm = ConnectionManager}}. + +handle_ssh_msg({ssh_cm, CM, {data, ChannelId, 0, Data}}, #state{n = N} = State) -> + M = N - size(Data), + case M > 0 of + true -> + ssh_connection:send(CM, ChannelId, Data), + {ok, State#state{n = M}}; + false -> + <<SendData:N/binary, _/binary>> = Data, + ssh_connection:send(CM, ChannelId, SendData), + ssh_connection:send_eof(CM, ChannelId), + {stop, ChannelId, State} + end; +handle_ssh_msg({ssh_cm, _ConnectionManager, + {data, _ChannelId, 1, Data}}, State) -> + error_logger:format(standard_error, " ~p~n", [binary_to_list(Data)]), + {ok, State}; + +handle_ssh_msg({ssh_cm, _ConnectionManager, {eof, _ChannelId}}, State) -> + {ok, State}; + +handle_ssh_msg({ssh_cm, _, {signal, _, _}}, State) -> + %% Ignore signals according to RFC 4254 section 6.9. + {ok, State}; + +handle_ssh_msg({ssh_cm, _, {exit_signal, ChannelId, _, _Error, _}}, + State) -> + {stop, ChannelId, State}; + +handle_ssh_msg({ssh_cm, _, {exit_status, ChannelId, _Status}}, State) -> + {stop, ChannelId, State}. + +terminate(_Reason, _State) -> + ok. + </code> + + <p>And run like this on the host tarlop with the keys generated in section 3.3</p> + + <code type="erl" > + 1> ssh:start(). + ok + 2> ssh:daemon(8989, [{system_dir, "/tmp/ssh_daemon"}, + {user_dir, "/tmp/otptest_user/.ssh"} + {subsystems, [{"echo_n", {ssh_echo_server, [10]}}]}]). + {ok,<0.54.0>} + 3> + </code> + + <code type="erl" > + 1> ssh:start(). + ok + 2>{ok, ConnectionRef} = ssh:connect("tarlop", 8989, [{user_dir, "/tmp/otptest_user/.ssh"}]). + {ok,<0.57.0>} + 3>{ok, ChannelId} = ssh_connection:session_channel(ConnectionRef, infinity). + 4> success = ssh_connection:subsystem(ConnectionRef, ChannelId, "echo_n", infinity). + 5> ok = ssh_connection:send(ConnectionRef, ChannelId, "0123456789", infinity). + 6> flush(). + {ssh_msg, <0.57.0>, {data, 0, 1, "0123456789"}} + {ssh_msg, <0.57.0>, {eof, 0}} + {ssh_msg, <0.57.0>, {closed, 0}} + 7> {error, closed} = ssh_connection:send(ConnectionRef, ChannelId, "10", infinity). + </code> + +</section> + +</chapter> diff --git a/lib/ssh/src/Makefile b/lib/ssh/src/Makefile index b8eecd3fa2..323f0af191 100644 --- a/lib/ssh/src/Makefile +++ b/lib/ssh/src/Makefile @@ -41,7 +41,9 @@ RELSYSDIR = $(RELEASE_PATH)/lib/ssh-$(VSN) BEHAVIOUR_MODULES= \ ssh_sftpd_file_api \ ssh_channel \ - ssh_key_api + ssh_subsystem \ + ssh_client_key_api \ + ssh_server_key_api MODULES= \ ssh \ diff --git a/lib/ssh/src/ssh.app.src b/lib/ssh/src/ssh.app.src index 316c09eb06..a0ba7cf7d9 100644 --- a/lib/ssh/src/ssh.app.src +++ b/lib/ssh/src/ssh.app.src @@ -10,6 +10,7 @@ ssh_auth, ssh_bits, ssh_cli, + ssh_client_key_api, ssh_channel, ssh_channel_sup, ssh_connection, @@ -21,13 +22,14 @@ sshd_sup, ssh_file, ssh_io, - ssh_key_api, ssh_math, ssh_no_io, + ssh_server_key_api, ssh_sftp, ssh_sftpd, ssh_sftpd_file, ssh_sftpd_file_api, + ssh_subsystem, ssh_subsystem_sup, ssh_sup, ssh_system_sup, @@ -35,7 +37,7 @@ ssh_userreg, ssh_xfer]}, {registered, []}, - {applications, [kernel, stdlib, crypto]}, + {applications, [kernel, stdlib, crypto, public_key]}, {env, []}, {mod, {ssh_app, []}}]}. diff --git a/lib/ssh/src/ssh.appup.src b/lib/ssh/src/ssh.appup.src index 6ba32e018f..826a11f1f4 100644 --- a/lib/ssh/src/ssh.appup.src +++ b/lib/ssh/src/ssh.appup.src @@ -19,29 +19,15 @@ {"%VSN%", [ - {<<"2.1.1">>, [{restart_application, ssh}]}, - {<<"2.1">>, [{load_module, ssh_sftpd_file_api, soft_purge, soft_purge, []}, - {load_module, ssh_connection, soft_purge, soft_purge, []}, - {load_module, ssh_connection_manager, soft_purge, soft_purge, []}, - {load_module, ssh_auth, soft_purge, soft_purge, []}, - {load_module, ssh_connection_handler, soft_purge, soft_purge, []}, - {load_module, ssh_channel, soft_purge, soft_purge, []}, - {load_module, ssh_file, soft_purge, soft_purge, []}]}, - {load_module, ssh, soft_purge, soft_purge, []}]}, + {<<"2.1.1">>, [{restart_application, ssh}]}, + {<<"2.1">>, [{restart_application, ssh}]}, {<<"2.0\\.*">>, [{restart_application, ssh}]}, {<<"1\\.*">>, [{restart_application, ssh}]} ], [ {<<"2.1.1">>, [{restart_application, ssh}]}, - {<<"2.1">>,[{load_module, ssh_sftpd_file_api, soft_purge, soft_purge, []}, - {load_module, ssh_connection, soft_purge, soft_purge, []}, - {load_module, ssh_connection_manager, soft_purge, soft_purge, []}, - {load_module, ssh_auth, soft_purge, soft_purge, []}, - {load_module, ssh_connection_handler, soft_purge, soft_purge, []}, - {load_module, ssh_channel, soft_purge, soft_purge, []}, - {load_module, ssh_file, soft_purge, soft_purge, []}]}, - {load_module, ssh, soft_purge, soft_purge, []}]}, + {<<"2.1">>,[{restart_application, ssh}]}, {<<"2.0\\.*">>, [{restart_application, ssh}]}, {<<"1\\.*">>, [{restart_application, ssh}]} ] -}.
\ No newline at end of file +}. diff --git a/lib/ssh/src/ssh.erl b/lib/ssh/src/ssh.erl index a569298056..3ef26b1678 100644 --- a/lib/ssh/src/ssh.erl +++ b/lib/ssh/src/ssh.erl @@ -41,19 +41,23 @@ %% %% Type = permanent | transient | temporary %% -%% Description: Starts the inets application. Default type +%% Description: Starts the ssh application. Default type %% is temporary. see application(3) %%-------------------------------------------------------------------- start() -> + application:start(crypto), + application:start(public_key), application:start(ssh). start(Type) -> + application:start(crypto, Type), + application:start(public_key, Type), application:start(ssh, Type). %%-------------------------------------------------------------------- %% Function: stop() -> ok %% -%% Description: Stops the inets application. +%% Description: Stops the ssh application. %%-------------------------------------------------------------------- stop() -> application:stop(ssh). @@ -76,7 +80,7 @@ connect(Host, Port, Options, Timeout) -> {error, _Reason} = Error -> Error; {SocketOptions, SshOptions} -> - DisableIpv6 = proplists:get_value(ip_v6_disabled, SshOptions, false), + DisableIpv6 = proplists:get_value(ipv6_disabled, SshOptions, false), Inet = inetopt(DisableIpv6), do_connect(Host, Port, [Inet | SocketOptions], [{user_pid, self()}, {host, Host} | fix_idle_time(SshOptions)], Timeout, DisableIpv6) @@ -169,7 +173,7 @@ daemon(HostAddr, Port, Options0) -> _ -> Options0 end, - DisableIpv6 = proplists:get_value(ip_v6_disabled, Options0, false), + DisableIpv6 = proplists:get_value(ipv6_disabled, Options0, false), {Host, Inet, Options} = case HostAddr of any -> {ok, Host0} = inet:gethostname(), @@ -264,7 +268,7 @@ start_daemon(Host, Port, Options, Inet) -> do_start_daemon(Host, Port, Options, SocketOptions) -> case ssh_system_sup:system_supervisor(Host, Port) of undefined -> - %% TODO: It would proably make more sense to call the + %% It would proably make more sense to call the %% address option host but that is a too big change at the %% monent. The name is a legacy name! try sshd_sup:start_child([{address, Host}, @@ -325,8 +329,6 @@ handle_option([{user_passwords, _} = Opt | Rest], SocketOptions, SshOptions) -> handle_option(Rest, SocketOptions, [handle_ssh_option(Opt) | SshOptions]); handle_option([{pwdfun, _} = Opt | Rest], SocketOptions, SshOptions) -> handle_option(Rest, SocketOptions, [handle_ssh_option(Opt) | SshOptions]); -handle_option([{user_auth, _} = Opt | Rest],SocketOptions, SshOptions ) -> - handle_option(Rest, SocketOptions, [handle_ssh_option(Opt) | SshOptions]); handle_option([{key_cb, _} = Opt | Rest], SocketOptions, SshOptions) -> handle_option(Rest, SocketOptions, [handle_ssh_option(Opt) | SshOptions]); handle_option([{role, _} = Opt | Rest], SocketOptions, SshOptions) -> @@ -344,7 +346,10 @@ handle_option([{disconnectfun, _} = Opt | Rest], SocketOptions, SshOptions) -> handle_option(Rest, SocketOptions, [handle_ssh_option(Opt) | SshOptions]); handle_option([{failfun, _} = Opt | Rest], SocketOptions, SshOptions) -> handle_option(Rest, SocketOptions, [handle_ssh_option(Opt) | SshOptions]); -handle_option([{ip_v6_disabled, _} = Opt | Rest], SocketOptions, SshOptions) -> +%%Backwards compatibility should not be underscore between ip and v6 in API +handle_option([{ip_v6_disabled, Value} | Rest], SocketOptions, SshOptions) -> + handle_option(Rest, SocketOptions, [handle_ssh_option({ipv6_disabled, Value}) | SshOptions]); +handle_option([{ipv6_disabled, _} = Opt | Rest], SocketOptions, SshOptions) -> handle_option(Rest, SocketOptions, [handle_ssh_option(Opt) | SshOptions]); handle_option([{transport, _} = Opt | Rest], SocketOptions, SshOptions) -> handle_option(Rest, SocketOptions, [handle_ssh_option(Opt) | SshOptions]); @@ -364,6 +369,8 @@ handle_option([{quiet_mode, _} = Opt|Rest], SocketOptions, SshOptions) -> handle_option(Rest, SocketOptions, [handle_ssh_option(Opt) | SshOptions]); handle_option([{idle_time, _} = Opt | Rest], SocketOptions, SshOptions) -> handle_option(Rest, SocketOptions, [handle_ssh_option(Opt) | SshOptions]); +handle_option([{rekey_limit, _} = Opt|Rest], SocketOptions, SshOptions) -> + handle_option(Rest, SocketOptions, [handle_ssh_option(Opt) | SshOptions]); handle_option([Opt | Rest], SocketOptions, SshOptions) -> handle_option(Rest, [handle_inet_option(Opt) | SocketOptions], SshOptions). @@ -377,10 +384,14 @@ handle_ssh_option({silently_accept_hosts, Value} = Opt) when Value == true; Valu Opt; handle_ssh_option({user_interaction, Value} = Opt) when Value == true; Value == false -> Opt; -handle_ssh_option({public_key_alg, Value} = Opt) when Value == ssh_rsa; Value == ssh_dsa -> +handle_ssh_option({public_key_alg, ssh_dsa}) -> + {public_key_alg, 'ssh-dss'}; +handle_ssh_option({public_key_alg, ssh_rsa}) -> + {public_key_alg, 'ssh-rsa'}; +handle_ssh_option({public_key_alg, Value} = Opt) when Value == 'ssh-rsa'; Value == 'ssh-dss' -> Opt; handle_ssh_option({pref_public_key_algs, Value} = Opt) when is_list(Value), length(Value) >= 1 -> - case check_pref_algs(Value) of + case handle_pref_algs(Value, []) of true -> Opt; _ -> @@ -400,8 +411,6 @@ handle_ssh_option({user_passwords, Value} = Opt) when is_list(Value)-> Opt; handle_ssh_option({pwdfun, Value} = Opt) when is_function(Value) -> Opt; -handle_ssh_option({user_auth, Value} = Opt) when is_function(Value) -> - Opt; handle_ssh_option({key_cb, Value} = Opt) when is_atom(Value) -> Opt; handle_ssh_option({compression, Value} = Opt) when is_atom(Value) -> @@ -420,7 +429,8 @@ handle_ssh_option({disconnectfun , Value} = Opt) when is_function(Value) -> Opt; handle_ssh_option({failfun, Value} = Opt) when is_function(Value) -> Opt; -handle_ssh_option({ip_v6_disabled, Value} = Opt) when Value == true; + +handle_ssh_option({ipv6_disabled, Value} = Opt) when Value == true; Value == false -> Opt; handle_ssh_option({transport, {Protocol, Cb, ClosTag}} = Opt) when is_atom(Protocol), @@ -441,6 +451,8 @@ handle_ssh_option({quiet_mode, Value} = Opt) when Value == true; Opt; handle_ssh_option({idle_time, Value} = Opt) when is_integer(Value), Value > 0 -> Opt; +handle_ssh_option({rekey_limit, Value} = Opt) when is_integer(Value) -> + Opt; handle_ssh_option(Opt) -> throw({error, {eoptions, Opt}}). @@ -449,7 +461,7 @@ handle_inet_option({active, _} = Opt) -> "and activ is handled internaly user is not allowd" "to specify this option"}}); handle_inet_option({inet, _} = Opt) -> - throw({error, {{eoptions, Opt},"Is set internaly use ip_v6_disabled to" + throw({error, {{eoptions, Opt},"Is set internaly use ipv6_disabled to" " enforce iv4 in the server, client will fallback to ipv4 if" " it can not use ipv6"}}); handle_inet_option({reuseaddr, _} = Opt) -> @@ -459,14 +471,18 @@ handle_inet_option({reuseaddr, _} = Opt) -> handle_inet_option(Opt) -> Opt. %% Check preferred algs -check_pref_algs([]) -> - true; -check_pref_algs([H|T]) -> +handle_pref_algs([], Acc) -> + {true, lists:reverse(Acc)}; +handle_pref_algs([H|T], Acc) -> case H of ssh_dsa -> - check_pref_algs(T); + handle_pref_algs(T, ['ssh-dss'| Acc]); ssh_rsa -> - check_pref_algs(T); + handle_pref_algs(T, ['ssh-rsa'| Acc]); + 'ssh-dss' -> + handle_pref_algs(T, ['ssh-dss'| Acc]); + 'ssh-rsa' -> + handle_pref_algs(T, ['ssh-rsa'| Acc]); _ -> false end. @@ -474,7 +490,7 @@ check_pref_algs([H|T]) -> inetopt(true) -> inet; inetopt(false) -> - case gen_tcp:listen(0, [inet6, {ip, loopback}]) of + case gen_tcp:listen(0, [inet6]) of {ok, Dummyport} -> gen_tcp:close(Dummyport), inet6; diff --git a/lib/ssh/src/ssh_auth.erl b/lib/ssh/src/ssh_auth.erl index c436793dc4..cb0c7751f0 100644 --- a/lib/ssh/src/ssh_auth.erl +++ b/lib/ssh/src/ssh_auth.erl @@ -48,17 +48,18 @@ publickey_msg([Alg, #ssh{user = User, case KeyCb:user_key(Alg, Opts) of {ok, Key} -> + StrAlgo = algorithm_string(Alg), PubKeyBlob = encode_public_key(Key), SigData = build_sig_data(SessionId, - User, Service, PubKeyBlob, Alg), + User, Service, PubKeyBlob, StrAlgo), Sig = ssh_transport:sign(SigData, Hash, Key), - SigBlob = list_to_binary([?string(Alg), ?binary(Sig)]), + SigBlob = list_to_binary([?string(StrAlgo), ?binary(Sig)]), ssh_transport:ssh_packet( #ssh_msg_userauth_request{user = User, service = Service, method = "publickey", data = [?TRUE, - ?string(Alg), + ?string(StrAlgo), ?binary(PubKeyBlob), ?binary(SigBlob)]}, Ssh); @@ -120,8 +121,7 @@ init_userauth_request_msg(#ssh{opts = Opts} = Ssh) -> data = <<>>}, case proplists:get_value(pref_public_key_algs, Opts, false) of false -> - FirstAlg = algorithm(proplists:get_value(public_key_alg, Opts, - ?PREFERRED_PK_ALG)), + FirstAlg = proplists:get_value(public_key_alg, Opts, ?PREFERRED_PK_ALG), SecondAlg = other_alg(FirstAlg), AllowUserInt = proplists:get_value(user_interaction, Opts, true), Prefs = method_preference(FirstAlg, SecondAlg, AllowUserInt), @@ -130,7 +130,7 @@ init_userauth_request_msg(#ssh{opts = Opts} = Ssh) -> userauth_methods = none, service = "ssh-connection"}); Algs -> - FirstAlg = algorithm(lists:nth(1, Algs)), + FirstAlg = lists:nth(1, Algs), case length(Algs) =:= 2 of true -> SecondAlg = other_alg(FirstAlg), @@ -358,7 +358,7 @@ verify_sig(SessionId, User, Service, Alg, KeyBlob, SigWLen, Opts) -> {ok, Key} = decode_public_key_v2(KeyBlob, Alg), KeyCb = proplists:get_value(key_cb, Opts, ssh_file), - case KeyCb:is_auth_key(Key, User, Alg, Opts) of + case KeyCb:is_auth_key(Key, User, Opts) of true -> PlainText = build_sig_data(SessionId, User, Service, KeyBlob, Alg), @@ -381,9 +381,9 @@ build_sig_data(SessionId, User, Service, KeyBlob, Alg) -> ?binary(KeyBlob)], list_to_binary(Sig). -algorithm(ssh_rsa) -> +algorithm_string('ssh-rsa') -> "ssh-rsa"; -algorithm(ssh_dsa) -> +algorithm_string('ssh-dss') -> "ssh-dss". decode_keyboard_interactive_prompts(NumPrompts, Data) -> @@ -457,10 +457,10 @@ userauth_pk_messages() -> binary]} % key blob ]. -other_alg("ssh-rsa") -> - "ssh-dss"; -other_alg("ssh-dss") -> - "ssh-rsa". +other_alg('ssh-rsa') -> + 'ssh-dss'; +other_alg('ssh-dss') -> + 'ssh-rsa'. decode_public_key_v2(K_S, "ssh-rsa") -> case ssh_bits:decode(K_S,[string,mpint,mpint]) of ["ssh-rsa", E, N] -> diff --git a/lib/ssh/src/ssh_auth.hrl b/lib/ssh/src/ssh_auth.hrl index e74ee10041..6cd8e6bf14 100644 --- a/lib/ssh/src/ssh_auth.hrl +++ b/lib/ssh/src/ssh_auth.hrl @@ -1,7 +1,7 @@ %% %% %CopyrightBegin% %% -%% Copyright Ericsson AB 2008-2010. All Rights Reserved. +%% Copyright Ericsson AB 2008-2012. All Rights Reserved. %% %% The contents of this file are subject to the Erlang Public License, %% Version 1.1, (the "License"); you may not use this file except in @@ -23,7 +23,7 @@ -define(SUPPORTED_AUTH_METHODS, "publickey,keyboard-interactive,password"). --define(PREFERRED_PK_ALG, ssh_rsa). +-define(PREFERRED_PK_ALG, 'ssh-rsa'). -define(SSH_MSG_USERAUTH_REQUEST, 50). -define(SSH_MSG_USERAUTH_FAILURE, 51). diff --git a/lib/ssh/src/ssh_channel.erl b/lib/ssh/src/ssh_channel.erl index 1938858420..4e8f8538c2 100644 --- a/lib/ssh/src/ssh_channel.erl +++ b/lib/ssh/src/ssh_channel.erl @@ -23,14 +23,35 @@ -include("ssh_connect.hrl"). -%%% Optional callbacks handle_call/3, handle_cast/2, handle_msg/2, -%%% code_change/3 -%% Should be further specified later --callback init(Options::list()) -> - {ok, State::term()} | {ok, State::term(), Timeout::timeout()} | - {stop, Reason ::term()}. - --callback terminate(term(), term()) -> term(). +-callback init(Args :: term()) -> + {ok, State :: term()} | {ok, State :: term(), timeout() | hibernate} | + {stop, Reason :: term()} | ignore. +-callback handle_call(Request :: term(), From :: {pid(), Tag :: term()}, + State :: term()) -> + {reply, Reply :: term(), NewState :: term()} | + {reply, Reply :: term(), NewState :: term(), timeout() | hibernate} | + {noreply, NewState :: term()} | + {noreply, NewState :: term(), timeout() | hibernate} | + {stop, Reason :: term(), Reply :: term(), NewState :: term()} | + {stop, Reason :: term(), NewState :: term()}. +-callback handle_cast(Request :: term(), State :: term()) -> + {noreply, NewState :: term()} | + {noreply, NewState :: term(), timeout() | hibernate} | + {stop, Reason :: term(), NewState :: term()}. + +-callback terminate(Reason :: (normal | shutdown | {shutdown, term()} | + term()), + State :: term()) -> + term(). +-callback code_change(OldVsn :: (term() | {down, term()}), State :: term(), + Extra :: term()) -> + {ok, NewState :: term()} | {error, Reason :: term()}. + +-callback handle_msg(Msg ::term(), State :: term()) -> + {noreply, NewState :: term()} | + {noreply, NewState :: term(), timeout() | hibernate} | + {stop, Reason :: term(), NewState :: term()}. + -callback handle_ssh_msg({ssh_cm, ConnectionRef::term(), SshMsg::term()}, State::term()) -> {ok, State::term()} | diff --git a/lib/ssh/src/ssh_client_key.erl b/lib/ssh/src/ssh_client_key.erl new file mode 100644 index 0000000000..2c48884dc2 --- /dev/null +++ b/lib/ssh/src/ssh_client_key.erl @@ -0,0 +1,34 @@ +%% +%% %CopyrightBegin% +%% +%% Copyright Ericsson AB 2011-2012. All Rights Reserved. +%% +%% The contents of this file are subject to the Erlang Public License, +%% Version 1.1, (the "License"); you may not use this file except in +%% compliance with the License. You should have received a copy of the +%% Erlang Public License along with this software. If not, it can be +%% retrieved online at http://www.erlang.org/. +%% +%% Software distributed under the License is distributed on an "AS IS" +%% basis, WITHOUT WARRANTY OF ANY KIND, either express or implied. See +%% the License for the specific language governing rights and limitations +%% under the License. +%% +%% %CopyrightEnd% +%% + +-module(ssh_client_key). + +-include_lib("public_key/include/public_key.hrl"). +-include("ssh.hrl"). + +-callback is_host_key(Key :: public_key(), Host :: string(), + Algorithm :: 'ssh-rsa'| 'ssh-dsa'| atom(), Options :: proplists:proplist()) -> + boolean(). + +-callback user_key(Algorithm :: 'ssh-rsa'| 'ssh-dsa'| atom(), Options :: list()) -> + {ok, PrivateKey :: term()} | {error, string()}. + + +-callback add_host_key(Host :: string(), PublicKey :: term(), Options :: list()) -> + ok | {error, Error::term()}. diff --git a/lib/ssh/src/ssh_client_key_api.erl b/lib/ssh/src/ssh_client_key_api.erl new file mode 100644 index 0000000000..eed0b85f47 --- /dev/null +++ b/lib/ssh/src/ssh_client_key_api.erl @@ -0,0 +1,35 @@ +%% +%% %CopyrightBegin% +%% +%% Copyright Ericsson AB 2011-2012. All Rights Reserved. +%% +%% The contents of this file are subject to the Erlang Public License, +%% Version 1.1, (the "License"); you may not use this file except in +%% compliance with the License. You should have received a copy of the +%% Erlang Public License along with this software. If not, it can be +%% retrieved online at http://www.erlang.org/. +%% +%% Software distributed under the License is distributed on an "AS IS" +%% basis, WITHOUT WARRANTY OF ANY KIND, either express or implied. See +%% the License for the specific language governing rights and limitations +%% under the License. +%% +%% %CopyrightEnd% +%% + +-module(ssh_client_key_api). + +-include_lib("public_key/include/public_key.hrl"). +-include("ssh.hrl"). + +-callback is_host_key(PublicKey :: #'RSAPublicKey'{}| {integer(), #'Dss-Parms'{}}| term() , Host :: string(), + Algorithm :: 'ssh-rsa'| 'ssh-dss'| atom(), ConnectOptions :: proplists:proplist()) -> + boolean(). + +-callback user_key(Algorithm :: 'ssh-rsa'| 'ssh-dss'| atom(), ConnectOptions :: proplists:proplists()) -> + {ok, PrivateKey :: #'RSAPrivateKey'{}| #'DSAPrivateKey'{} | term()} | {error, string()}. + + +-callback add_host_key(Host :: string(), PublicKey :: #'RSAPublicKey'{}| {integer(), #'Dss-Parms'{}}| term(), + Options :: list()) -> + ok | {error, Error::term()}. diff --git a/lib/ssh/src/ssh_connection_handler.erl b/lib/ssh/src/ssh_connection_handler.erl index d8950a7b67..88b45111ff 100644 --- a/lib/ssh/src/ssh_connection_handler.erl +++ b/lib/ssh/src/ssh_connection_handler.erl @@ -35,7 +35,8 @@ -export([start_link/4, send/2, renegotiate/1, send_event/2, connection_info/3, - peer_address/1]). + peer_address/1, + renegotiate_data/1]). %% gen_fsm callbacks -export([hello/2, kexinit/2, key_exchange/2, new_keys/2, @@ -85,6 +86,8 @@ send(ConnectionHandler, Data) -> renegotiate(ConnectionHandler) -> send_all_state_event(ConnectionHandler, renegotiate). +renegotiate_data(ConnectionHandler) -> + send_all_state_event(ConnectionHandler, data_size). connection_info(ConnectionHandler, From, Options) -> send_all_state_event(ConnectionHandler, {info, From, Options}). @@ -500,7 +503,22 @@ handle_event(renegotiate, StateName, State) -> handle_event({info, From, Options}, StateName, #state{ssh_params = Ssh} = State) -> spawn(?MODULE, ssh_info_handler, [Options, Ssh, From]), {next_state, StateName, State}; - +handle_event(data_size, connected, #state{ssh_params = Ssh0} = State) -> + Sent = inet:getstat(State#state.socket, [send_oct]), + MaxSent = proplists:get_value(rekey_limit, State#state.opts, 1024000000), + case Sent >= MaxSent of + true -> + {KeyInitMsg, SshPacket, Ssh} = ssh_transport:key_exchange_init_msg(Ssh0), + send_msg(SshPacket, State), + {next_state, connected, + next_packet(State#state{ssh_params = Ssh, + key_exchange_init_msg = KeyInitMsg, + renegotiate = true})}; + _ -> + {next_state, connected, next_packet(State)} + end; +handle_event(data_size, StateName, State) -> + {next_state, StateName, State}; handle_event({unknown, Data}, StateName, State) -> Msg = #ssh_msg_unimplemented{sequence = Data}, send_msg(Msg, State), @@ -749,9 +767,9 @@ extract_algs([], NewList) -> lists:reverse(NewList); extract_algs([H|T], NewList) -> case H of - ssh_dsa -> + 'ssh-dss' -> extract_algs(T, ["ssh-dss"|NewList]); - ssh_rsa -> + 'ssh-rsa' -> extract_algs(T, ["ssh-rsa"|NewList]) end. available_host_key(KeyCb, "ssh-dss"= Alg, Opts) -> diff --git a/lib/ssh/src/ssh_connection_manager.erl b/lib/ssh/src/ssh_connection_manager.erl index 0c1eee5186..94a9ed505f 100644 --- a/lib/ssh/src/ssh_connection_manager.erl +++ b/lib/ssh/src/ssh_connection_manager.erl @@ -125,7 +125,8 @@ info(ConnectionManager, ChannelProcess) -> %% or amount of data sent counter! renegotiate(ConnectionManager) -> cast(ConnectionManager, renegotiate). - +renegotiate_data(ConnectionManager) -> + cast(ConnectionManager, renegotiate_data). connection_info(ConnectionManager, Options) -> call(ConnectionManager, {connection_info, Options}). @@ -481,7 +482,9 @@ handle_cast({global_request, _, _, _, _} = Request, State0) -> handle_cast(renegotiate, #state{connection = Pid} = State) -> ssh_connection_handler:renegotiate(Pid), {noreply, State}; - +handle_cast(renegotiate_data, #state{connection = Pid} = State) -> + ssh_connection_handler:renegotiate_data(Pid), + {noreply, State}; handle_cast({adjust_window, ChannelId, Bytes}, #state{connection = Pid, connection_state = #connection{channel_cache = Cache}} = State) -> @@ -520,6 +523,8 @@ handle_info({start_connection, server, Exec = proplists:get_value(exec, Options), CliSpec = proplists:get_value(ssh_cli, Options, {ssh_cli, [Shell]}), ssh_connection_handler:send_event(Connection, socket_control), + erlang:send_after(3600000, self(), rekey), + erlang:send_after(60000, self(), rekey_data), {noreply, State#state{connection = Connection, connection_state = CState#connection{address = Address, @@ -536,6 +541,8 @@ handle_info({start_connection, client, case (catch ssh_transport:connect(Parent, Address, Port, SocketOpts, Options)) of {ok, Connection} -> + erlang:send_after(60000, self(), rekey_data), + erlang:send_after(3600000, self(), rekey), {noreply, State#state{connection = Connection}}; Reason -> Pid ! {self(), not_connected, Reason}, @@ -568,8 +575,15 @@ handle_info({'DOWN', _Ref, process, ChannelPid, _Reason}, State) -> %%% So that terminate will be run when supervisor is shutdown handle_info({'EXIT', _Sup, Reason}, State) -> - {stop, Reason, State}. - + {stop, Reason, State}; +handle_info(rekey, State) -> + renegotiate(self()), + erlang:send_after(3600000, self(), rekey), + {noreply, State}; +handle_info(rekey_data, State) -> + renegotiate_data(self()), + erlang:send_after(60000, self(), rekey_data), + {noreply, State}. handle_password(Opts) -> handle_rsa_password(handle_dsa_password(handle_normal_password(Opts))). handle_normal_password(Opts) -> diff --git a/lib/ssh/src/ssh_file.erl b/lib/ssh/src/ssh_file.erl index a6b82a7a13..f115a32710 100644 --- a/lib/ssh/src/ssh_file.erl +++ b/lib/ssh/src/ssh_file.erl @@ -23,7 +23,8 @@ -module(ssh_file). --behaviour(ssh_key_api). +-behaviour(ssh_server_key_api). +-behaviour(ssh_client_key_api). -include_lib("public_key/include/public_key.hrl"). -include_lib("kernel/include/file.hrl"). @@ -34,7 +35,7 @@ user_key/2, is_host_key/4, add_host_key/3, - is_auth_key/4]). + is_auth_key/3]). -define(PERM_700, 8#700). @@ -53,8 +54,8 @@ host_key(Algorithm, Opts) -> decode(File, Password). -is_auth_key(Key, User, Alg, Opts) -> - case lookup_user_key(Key, User, Alg, Opts) of +is_auth_key(Key, User,Opts) -> + case lookup_user_key(Key, User, Opts) of {ok, Key} -> true; _ -> @@ -138,13 +139,13 @@ add_host_key(Host, Key, Opts) -> Error end. -lookup_user_key(Key, User, Alg, Opts) -> +lookup_user_key(Key, User, Opts) -> SshDir = ssh_dir({remoteuser,User}, Opts), - case lookup_user_key_f(Key, User, SshDir, Alg, "authorized_keys", Opts) of + case lookup_user_key_f(Key, User, SshDir, "authorized_keys", Opts) of {ok, Key} -> {ok, Key}; _ -> - lookup_user_key_f(Key, User, SshDir, Alg, "authorized_keys2", Opts) + lookup_user_key_f(Key, User, SshDir, "authorized_keys2", Opts) end. @@ -213,9 +214,9 @@ do_lookup_host_key(Host, Alg, Opts) -> Error -> Error end. -identity_key_filename("ssh-dss") -> +identity_key_filename('ssh-dss') -> "id_dsa"; -identity_key_filename("ssh-rsa") -> +identity_key_filename('ssh-rsa') -> "id_rsa". identity_pass_phrase("ssh-dss") -> @@ -261,9 +262,9 @@ host_name(Atom) when is_atom(Atom) -> host_name(List) -> List. -key_match(#'RSAPublicKey'{}, "ssh-rsa") -> +key_match(#'RSAPublicKey'{}, 'ssh-rsa') -> true; -key_match({_, #'Dss-Parms'{}}, "ssh-dss") -> +key_match({_, #'Dss-Parms'{}}, 'ssh-dss') -> true; key_match(_, _) -> false. @@ -272,11 +273,11 @@ add_key_fd(Fd, Host,Key) -> SshBin = public_key:ssh_encode([{Key, [{hostnames, [Host]}]}], known_hosts), file:write(Fd, SshBin). -lookup_user_key_f(_, _User, [], _Alg, _F, _Opts) -> +lookup_user_key_f(_, _User, [], _F, _Opts) -> {error, nouserdir}; -lookup_user_key_f(_, _User, nouserdir, _Alg, _F, _Opts) -> +lookup_user_key_f(_, _User, nouserdir, _F, _Opts) -> {error, nouserdir}; -lookup_user_key_f(Key, _User, Dir, _Alg, F, _Opts) -> +lookup_user_key_f(Key, _User, Dir, F, _Opts) -> FileName = filename:join(Dir, F), case file:open(FileName, [read, binary]) of {ok, Fd} -> diff --git a/lib/ssh/src/ssh_io.erl b/lib/ssh/src/ssh_io.erl index 17a7cebb4a..01fc713569 100644 --- a/lib/ssh/src/ssh_io.erl +++ b/lib/ssh/src/ssh_io.erl @@ -1,7 +1,7 @@ %% %% %CopyrightBegin% %% -%% Copyright Ericsson AB 2005-2011. All Rights Reserved. +%% Copyright Ericsson AB 2005-2012. All Rights Reserved. %% %% The contents of this file are subject to the Erlang Public License, %% Version 1.1, (the "License"); you may not use this file except in diff --git a/lib/ssh/src/ssh_key_api.erl b/lib/ssh/src/ssh_key_api.erl deleted file mode 100644 index 8085c12e21..0000000000 --- a/lib/ssh/src/ssh_key_api.erl +++ /dev/null @@ -1,45 +0,0 @@ -%% -%% %CopyrightBegin% -%% -%% Copyright Ericsson AB 2011-2012. All Rights Reserved. -%% -%% The contents of this file are subject to the Erlang Public License, -%% Version 1.1, (the "License"); you may not use this file except in -%% compliance with the License. You should have received a copy of the -%% Erlang Public License along with this software. If not, it can be -%% retrieved online at http://www.erlang.org/. -%% -%% Software distributed under the License is distributed on an "AS IS" -%% basis, WITHOUT WARRANTY OF ANY KIND, either express or implied. See -%% the License for the specific language governing rights and limitations -%% under the License. -%% -%% %CopyrightEnd% -%% - --module(ssh_key_api). - --include_lib("public_key/include/public_key.hrl"). --include("ssh.hrl"). - --type ssh_algorithm() :: string(). --type file_error() :: file:posix() | badarg | system_limit | terminated. - --callback host_key(Algorithm :: ssh_algorithm(), Options :: list()) -> - {ok, [{public_key(), Attributes::list()}]} | public_key() - | {error, string()}. - --callback user_key(Algorithm :: ssh_algorithm(), Options :: list()) -> - {ok, [{public_key(), Attributes::list()}]} | public_key() - | {error, string()}. - --callback is_host_key(Key :: public_key(), PeerName :: string(), - Algorithm :: ssh_algorithm(), Options :: list()) -> - boolean(). - --callback add_host_key(Host :: string(), Key :: public_key(), Options :: list()) -> - ok | {error, file_error()}. - --callback is_auth_key(Key :: public_key(), User :: string(), - Algorithm :: ssh_algorithm(), Options :: list()) -> - boolean(). diff --git a/lib/ssh/src/ssh_server_key.erl b/lib/ssh/src/ssh_server_key.erl new file mode 100644 index 0000000000..8140114990 --- /dev/null +++ b/lib/ssh/src/ssh_server_key.erl @@ -0,0 +1,33 @@ +%% +%% %CopyrightBegin% +%% +%% Copyright Ericsson AB 2011-2012. All Rights Reserved. +%% +%% The contents of this file are subject to the Erlang Public License, +%% Version 1.1, (the "License"); you may not use this file except in +%% compliance with the License. You should have received a copy of the +%% Erlang Public License along with this software. If not, it can be +%% retrieved online at http://www.erlang.org/. +%% +%% Software distributed under the License is distributed on an "AS IS" +%% basis, WITHOUT WARRANTY OF ANY KIND, either express or implied. See +%% the License for the specific language governing rights and limitations +%% under the License. +%% +%% %CopyrightEnd% +%% + +-module(ssh_server_key). + +-include_lib("public_key/include/public_key.hrl"). +-include("ssh.hrl"). + +-type ssh_algorithm() :: string(). + +-callback host_key(Algorithm :: ssh_algorithm(), Options :: list()) -> + {ok, [{public_key(), Attributes::list()}]} | public_key() + | {error, string()}. + +-callback is_auth_key(Key :: public_key(), User :: string(), + Algorithm :: ssh_algorithm(), Options :: list()) -> + boolean(). diff --git a/lib/ssh/src/ssh_server_key_api.erl b/lib/ssh/src/ssh_server_key_api.erl new file mode 100644 index 0000000000..4fd660ecb5 --- /dev/null +++ b/lib/ssh/src/ssh_server_key_api.erl @@ -0,0 +1,30 @@ +%% +%% %CopyrightBegin% +%% +%% Copyright Ericsson AB 2011-2012. All Rights Reserved. +%% +%% The contents of this file are subject to the Erlang Public License, +%% Version 1.1, (the "License"); you may not use this file except in +%% compliance with the License. You should have received a copy of the +%% Erlang Public License along with this software. If not, it can be +%% retrieved online at http://www.erlang.org/. +%% +%% Software distributed under the License is distributed on an "AS IS" +%% basis, WITHOUT WARRANTY OF ANY KIND, either express or implied. See +%% the License for the specific language governing rights and limitations +%% under the License. +%% +%% %CopyrightEnd% +%% + +-module(ssh_server_key_api). + +-include_lib("public_key/include/public_key.hrl"). +-include("ssh.hrl"). + +-callback host_key(Algorithm :: 'ssh-rsa'| 'ssh-dss'| atom(), DaemonOptions :: proplists:proplist()) -> + {ok, PrivateKey :: #'RSAPrivateKey'{}| #'DSAPrivateKey'{} | term()} | {error, string()}. + +-callback is_auth_key(PublicKey :: #'RSAPublicKey'{}| {integer(), #'Dss-Parms'{}}| term(), + User :: string(), DaemonOptions :: proplists:proplist()) -> + boolean(). diff --git a/lib/ssh/src/ssh_sftpd.erl b/lib/ssh/src/ssh_sftpd.erl index ec7b76b0b3..c7e8373840 100644 --- a/lib/ssh/src/ssh_sftpd.erl +++ b/lib/ssh/src/ssh_sftpd.erl @@ -24,7 +24,7 @@ -module(ssh_sftpd). %%-behaviour(gen_server). --behaviour(ssh_channel). +-behaviour(ssh_subsystem). -include_lib("kernel/include/file.hrl"). @@ -36,7 +36,7 @@ -export([subsystem_spec/1, listen/1, listen/2, listen/3, stop/1]). --export([init/1, handle_ssh_msg/2, handle_msg/2, terminate/2, code_change/3]). +-export([init/1, handle_ssh_msg/2, handle_msg/2, terminate/2]). -record(state, { xf, % [{channel,ssh_xfer states}...] @@ -119,23 +119,13 @@ init(Options) -> {Root0, State0} end, MaxLength = proplists:get_value(max_files, Options, 0), - - Vsn = proplists:get_value(vsn, Options, 5), - + Vsn = proplists:get_value(sftpd_vsn, Options, 5), {ok, State#state{cwd = CWD, root = Root, max_files = MaxLength, handles = [], pending = <<>>, xf = #ssh_xfer{vsn = Vsn, ext = []}}}. %%-------------------------------------------------------------------- -%% Function: code_change(OldVsn, State, Extra) -> {ok, NewState} -%% Description: -%%-------------------------------------------------------------------- -code_change(_OldVsn, State, _Extra) -> - {ok, State}. - - -%%-------------------------------------------------------------------- %% Function: handle_ssh_msg(Args) -> {ok, State} | {stop, ChannelId, State} %% %% Description: Handles channel messages @@ -369,17 +359,21 @@ handle_op(?SSH_FXP_FSETSTAT, ReqId, <<?UINT32(HLen), BinHandle:HLen/binary, State0 end; handle_op(?SSH_FXP_REMOVE, ReqId, <<?UINT32(PLen), BPath:PLen/binary>>, - State0 = #state{file_handler = FileMod, file_state = FS0}) -> + State0 = #state{file_handler = FileMod, file_state = FS0, xf = #ssh_xfer{vsn = Vsn}}) -> Path = relate_file_name(BPath, State0), - %% case FileMod:is_dir(Path) of %% This version 6 we still have ver 5 - %% true -> - %% ssh_xfer:xf_send_status(State#state.xf, ReqId, - %% ?SSH_FX_FILE_IS_A_DIRECTORY); - %% false -> - {Status, FS1} = FileMod:delete(Path, FS0), - State1 = State0#state{file_state = FS1}, - send_status(Status, ReqId, State1); - %%end; + {IsDir, _FS1} = FileMod:is_dir(Path, FS0), + case IsDir of %% This version 6 we still have ver 5 + true when Vsn > 5 -> + ssh_xfer:xf_send_status(State0#state.xf, ReqId, + ?SSH_FX_FILE_IS_A_DIRECTORY, "File is a directory"); + true -> + ssh_xfer:xf_send_status(State0#state.xf, ReqId, + ?SSH_FX_FAILURE, "File is a directory"); + false -> + {Status, FS1} = FileMod:delete(Path, FS0), + State1 = State0#state{file_state = FS1}, + send_status(Status, ReqId, State1) + end; handle_op(?SSH_FXP_RMDIR, ReqId, <<?UINT32(PLen), BPath:PLen/binary>>, State0 = #state{file_handler = FileMod, file_state = FS0}) -> Path = relate_file_name(BPath, State0), @@ -637,31 +631,34 @@ open(Vsn, ReqId, Data, State) when Vsn >= 4 -> do_open(ReqId, State, Path, Flags). do_open(ReqId, State0, Path, Flags) -> - #state{file_handler = FileMod, file_state = FS0, root = Root} = State0, + #state{file_handler = FileMod, file_state = FS0, root = Root, xf = #ssh_xfer{vsn = Vsn}} = State0, XF = State0#state.xf, F = [binary | Flags], - %% case FileMod:is_dir(Path) of %% This is version 6 we still have 5 - %% true -> - %% ssh_xfer:xf_send_status(State#state.xf, ReqId, - %% ?SSH_FX_FILE_IS_A_DIRECTORY); - %% false -> - - AbsPath = case Root of - "" -> - Path; - _ -> - relate_file_name(Path, State0) - end, - - {Res, FS1} = FileMod:open(AbsPath, F, FS0), - State1 = State0#state{file_state = FS1}, - case Res of - {ok, IoDevice} -> - add_handle(State1, XF, ReqId, file, {Path,IoDevice}); - {error, Error} -> - ssh_xfer:xf_send_status(State1#state.xf, ReqId, - ssh_xfer:encode_erlang_status(Error)), - State1 + {IsDir, _FS1} = FileMod:is_dir(Path, FS0), + case IsDir of + true when Vsn > 5 -> + ssh_xfer:xf_send_status(State0#state.xf, ReqId, + ?SSH_FX_FILE_IS_A_DIRECTORY, "File is a directory"); + true -> + ssh_xfer:xf_send_status(State0#state.xf, ReqId, + ?SSH_FX_FAILURE, "File is a directory"); + false -> + AbsPath = case Root of + "" -> + Path; + _ -> + relate_file_name(Path, State0) + end, + {Res, FS1} = FileMod:open(AbsPath, F, FS0), + State1 = State0#state{file_state = FS1}, + case Res of + {ok, IoDevice} -> + add_handle(State1, XF, ReqId, file, {Path,IoDevice}); + {error, Error} -> + ssh_xfer:xf_send_status(State1#state.xf, ReqId, + ssh_xfer:encode_erlang_status(Error)), + State1 + end end. %% resolve all symlinks in a path diff --git a/lib/ssh/src/ssh_subsystem.erl b/lib/ssh/src/ssh_subsystem.erl new file mode 100644 index 0000000000..5a9fa32668 --- /dev/null +++ b/lib/ssh/src/ssh_subsystem.erl @@ -0,0 +1,47 @@ +-module(ssh_subsystem). + +%% API to special server side channel that can be pluged into the erlang ssh daemeon +-callback init(Args :: term()) -> + {ok, State :: term()} | {ok, State :: term(), timeout() | hibernate} | + {stop, Reason :: term()} | ignore. + +-callback terminate(Reason :: (normal | shutdown | {shutdown, term()} | + term()), + State :: term()) -> + term(). + +-callback handle_msg(Msg ::term(), State :: term()) -> + {noreply, NewState :: term()} | + {noreply, NewState :: term(), timeout() | hibernate} | + {stop, Reason :: term(), NewState :: term()}. + +-callback handle_ssh_msg({ssh_cm, ConnectionRef::term(), SshMsg::term()}, + State::term()) -> {ok, State::term()} | + {stop, ChannelId::integer(), + State::term()}. + +%%% API +-export([start/4, start/5, start_link/4, start_link/5, enter_loop/1]). + +%% gen_server callbacks +-export([init/1, terminate/2]). + +start(ConnectionManager, ChannelId, CallBack, CbInitArgs) -> + ssh_channel:start(ConnectionManager, ChannelId, CallBack, CbInitArgs, undefined). + +start(ConnectionManager, ChannelId, CallBack, CbInitArgs, Exec) -> + ssh_channel:start(ConnectionManager, ChannelId, CallBack, CbInitArgs, Exec). + +start_link(ConnectionManager, ChannelId, CallBack, CbInitArgs) -> + ssh_channel:start_link(ConnectionManager, ChannelId, CallBack, CbInitArgs, undefined). + +start_link(ConnectionManager, ChannelId, CallBack, CbInitArgs, Exec) -> + ssh_channel:start_link(ConnectionManager, ChannelId, CallBack, CbInitArgs, Exec). + +enter_loop(State) -> + ssh_channel:enter_loop(State). + +init(Args) -> + ssh_channel:init(Args). +terminate(Reason, State) -> + ssh_channel:terminate(Reason, State). diff --git a/lib/ssh/src/ssh_transport.erl b/lib/ssh/src/ssh_transport.erl index 7f6e7d9946..1abb69921d 100644 --- a/lib/ssh/src/ssh_transport.erl +++ b/lib/ssh/src/ssh_transport.erl @@ -449,7 +449,7 @@ verify_host_key_rsa(SSH, K_S, H, H_SIG) -> false -> {error, bad_signature}; true -> - known_host_key(SSH, Public, "ssh-rsa") + known_host_key(SSH, Public, 'ssh-rsa') end; _ -> {error, bad_format} @@ -464,7 +464,7 @@ verify_host_key_dss(SSH, K_S, H, H_SIG) -> false -> {error, bad_signature}; true -> - known_host_key(SSH, Public, "ssh-dss") + known_host_key(SSH, Public, 'ssh-dss') end; _ -> {error, bad_host_key_format} diff --git a/lib/ssh/src/ssh_xfer.erl b/lib/ssh/src/ssh_xfer.erl index d5b6dd03d1..4dfd9ed8b0 100644 --- a/lib/ssh/src/ssh_xfer.erl +++ b/lib/ssh/src/ssh_xfer.erl @@ -383,6 +383,8 @@ decode_status(Status) -> ?SSH_FX_UNKNOWN_PRINCIPLE -> unknown_principle; ?SSH_FX_LOCK_CONFlICT -> lock_conflict; ?SSH_FX_NOT_A_DIRECTORY -> not_a_directory; + ?SSH_FX_FILE_IS_A_DIRECTORY -> file_is_a_directory; + ?SSH_FX_CANNOT_DELETE -> cannot_delete; _ -> {error,Status} end. @@ -392,6 +394,9 @@ encode_erlang_status(Status) -> eof -> ?SSH_FX_EOF; enoent -> ?SSH_FX_NO_SUCH_FILE; eacces -> ?SSH_FX_PERMISSION_DENIED; + eisdir -> ?SSH_FX_FILE_IS_A_DIRECTORY; + eperm -> ?SSH_FX_CANNOT_DELETE; + eexist -> ?SSH_FX_FILE_ALREADY_EXISTS; _ -> ?SSH_FX_FAILURE end. diff --git a/lib/ssh/src/ssh_xfer.hrl b/lib/ssh/src/ssh_xfer.hrl index c13950eb6e..0d85cf2094 100644 --- a/lib/ssh/src/ssh_xfer.hrl +++ b/lib/ssh/src/ssh_xfer.hrl @@ -58,7 +58,6 @@ %%% # SSH_FX_xxx %%% Description: Response packet types for file transfer protocol. %%%---------------------------------------------------------------------- - -define(SSH_FX_OK, 0). -define(SSH_FX_EOF, 1). -define(SSH_FX_NO_SUCH_FILE, 2). @@ -79,7 +78,18 @@ -define(SSH_FX_LOCK_CONFlICT, 17). -define(SSH_FX_DIR_NOT_EMPTY, 18). -define(SSH_FX_NOT_A_DIRECTORY, 19). +-define(SSH_FX_INVALID_FILENAME, 20). +-define(SSH_FX_LINK_LOOP, 21). +-define(SSH_FX_CANNOT_DELETE, 22). +-define(SSH_FX_INVALID_PARAMETER, 23). -define(SSH_FX_FILE_IS_A_DIRECTORY, 24). +-define(SSH_FX_BYTE_RANGE_LOCK_CONFLICT,25). +-define(SSH_FX_BYTE_RANGE_LOCK_REFUSED, 26). +-define(SSH_FX_DELETE_PENDING, 27). +-define(SSH_FX_FILE_CORRUPT, 28). +-define(SSH_FX_OWNER_INVALID, 29). +-define(SSH_FX_GROUP_INVALID, 30). +-define(SSH_FX_NO_MATCHING_BYTE_RANGE_LOCK,31). %%%---------------------------------------------------------------------- %%% # SSH_FILEXFER_xxx diff --git a/lib/ssh/test/ssh_basic_SUITE.erl b/lib/ssh/test/ssh_basic_SUITE.erl index 5fec7f0cd7..efcb11f88f 100644 --- a/lib/ssh/test/ssh_basic_SUITE.erl +++ b/lib/ssh/test/ssh_basic_SUITE.erl @@ -42,15 +42,14 @@ all() -> {group, dsa_pass_key}, {group, rsa_pass_key}, {group, internal_error}, - {group, idle_time}, daemon_already_started, server_password_option, server_userpassword_option, close]. groups() -> - [{dsa_key, [], [send, exec, exec_compressed, shell, known_hosts, idle_time]}, - {rsa_key, [], [send, exec, exec_compressed, shell, known_hosts, idle_time]}, + [{dsa_key, [], [send, exec, exec_compressed, shell, known_hosts, idle_time, rekey]}, + {rsa_key, [], [send, exec, exec_compressed, shell, known_hosts, idle_time, rekey]}, {dsa_pass_key, [], [pass_phrase]}, {rsa_pass_key, [], [pass_phrase]}, {internal_error, [], [internal_error]} @@ -247,7 +246,8 @@ idle_time(Config) -> ConnectionRef = ssh_test_lib:connect(Host, Port, [{silently_accept_hosts, true}, {user_dir, UserDir}, - {user_interaction, false}]), + {user_interaction, false}, + {idle_time, 2000}]), {ok, Id} = ssh_connection:session_channel(ConnectionRef, 1000), ssh_connection:close(ConnectionRef, Id), receive @@ -256,6 +256,28 @@ idle_time(Config) -> end, ssh:stop_daemon(Pid). %%-------------------------------------------------------------------- +rekey(doc) -> + ["Idle timeout test"]; +rekey(Config) -> + SystemDir = filename:join(?config(priv_dir, Config), system), + UserDir = ?config(priv_dir, Config), + + {Pid, Host, Port} = ssh_test_lib:daemon([{system_dir, SystemDir}, + {user_dir, UserDir}, + {failfun, fun ssh_test_lib:failfun/2}, + {rekey_limit, 0}]), + ConnectionRef = + ssh_test_lib:connect(Host, Port, [{silently_accept_hosts, true}, + {user_dir, UserDir}, + {user_interaction, false}, + {rekey_limit, 0}]), + receive + after 15000 -> + %%By this time rekeying would have been done + ssh:close(ConnectionRef), + ssh:stop_daemon(Pid) + end. +%%-------------------------------------------------------------------- shell(doc) -> ["Test that ssh:shell/2 works"]; shell(Config) when is_list(Config) -> diff --git a/lib/ssh/test/ssh_echo_server.erl b/lib/ssh/test/ssh_echo_server.erl index 739aabe6fb..007b00c373 100644 --- a/lib/ssh/test/ssh_echo_server.erl +++ b/lib/ssh/test/ssh_echo_server.erl @@ -21,7 +21,7 @@ %%% Description: Example ssh server -module(ssh_echo_server). --behaviour(ssh_channel). +-behaviour(ssh_subsytem). -record(state, { n, id, @@ -50,7 +50,7 @@ handle_ssh_msg({ssh_cm, CM, {data, ChannelId, 0, Data}}, #state{n = N} = State) end; handle_ssh_msg({ssh_cm, _ConnectionManager, {data, _ChannelId, 1, Data}}, State) -> - error_logger:format("ssh: STDERR: ~s\n", [binary_to_list(Data)]), + error_logger:format(standard_error, " ~p~n", [binary_to_list(Data)]), {ok, State}; handle_ssh_msg({ssh_cm, _ConnectionManager, {eof, _ChannelId}}, State) -> diff --git a/lib/ssh/test/ssh_sftpd_SUITE.erl b/lib/ssh/test/ssh_sftpd_SUITE.erl index b995eb9f0e..5aa46872ee 100644 --- a/lib/ssh/test/ssh_sftpd_SUITE.erl +++ b/lib/ssh/test/ssh_sftpd_SUITE.erl @@ -58,7 +58,8 @@ all() -> links, ver3_rename, relpath, - sshd_read_file]. + sshd_read_file, + ver6_basic]. groups() -> []. @@ -106,12 +107,18 @@ init_per_testcase(TestCase, Config) -> SystemDir = filename:join(?config(priv_dir, Config), system), Port = ssh_test_lib:inet_port(node()), - - {ok, Sftpd} = - ssh_sftpd:listen(Port, [{system_dir, SystemDir}, - {user_dir, PrivDir}, - {user_passwords,[{?USER, ?PASSWD}]}, - {pwdfun, fun(_,_) -> true end}]), + Options = [{system_dir, SystemDir}, + {user_dir, PrivDir}, + {user_passwords,[{?USER, ?PASSWD}]}, + {pwdfun, fun(_,_) -> true end}], + {ok, Sftpd} = case TestCase of + ver6_basic -> + SubSystems = [ssh_sftpd:subsystem_spec([{sftpd_vsn, 6}])], + ssh:daemon(Port, [{subsystems, SubSystems}|Options]); + _ -> + SubSystems = [ssh_sftpd:subsystem_spec([])], + ssh:daemon(Port, [{subsystems, SubSystems}|Options]) + end, Cm = ssh_test_lib:connect(Port, [{user_dir, ClientUserDir}, @@ -341,7 +348,7 @@ mk_rm_dir(Config) when is_list(Config) -> _/binary>>, _} = mkdir(DirName, Cm, Channel, ReqId), NewReqId = 1, - {ok, <<?SSH_FXP_STATUS, ?UINT32(NewReqId), ?UINT32(?SSH_FX_FAILURE), + {ok, <<?SSH_FXP_STATUS, ?UINT32(NewReqId), ?UINT32(?SSH_FX_FILE_ALREADY_EXISTS), _/binary>>, _} = mkdir(DirName, Cm, Channel, NewReqId), NewReqId1 = 2, @@ -591,7 +598,18 @@ sshd_read_file(Config) when is_list(Config) -> read_file(Handle, 100, 0, Cm, Channel, NewReqId), {ok, Data} = file:read_file(FileName). - +ver6_basic(doc) -> + ["Test SFTP Version 6"]; +ver6_basic(Config) when is_list(Config) -> + PrivDir = ?config(priv_dir, Config), + %FileName = filename:join(PrivDir, "test.txt"), + {Cm, Channel} = ?config(sftp, Config), + ReqId = 0, + {ok, <<?SSH_FXP_STATUS, ?UINT32(ReqId), % Ver 6 we have 5 + ?UINT32(?SSH_FX_FILE_IS_A_DIRECTORY), _/binary>>, _} = + open_file(PrivDir, Cm, Channel, ReqId, + ?ACE4_READ_DATA bor ?ACE4_READ_ATTRIBUTES, + ?SSH_FXF_OPEN_EXISTING). %%-------------------------------------------------------------------- %% Internal functions ------------------------------------------------ %%-------------------------------------------------------------------- diff --git a/lib/ssh/test/ssh_sftpd_erlclient_SUITE.erl b/lib/ssh/test/ssh_sftpd_erlclient_SUITE.erl index 7fc2312661..8f722941d4 100644 --- a/lib/ssh/test/ssh_sftpd_erlclient_SUITE.erl +++ b/lib/ssh/test/ssh_sftpd_erlclient_SUITE.erl @@ -42,7 +42,8 @@ all() -> quit, file_cb, root_dir, - list_dir_limited]. + list_dir_limited, + ver6_basic]. groups() -> []. @@ -112,7 +113,12 @@ init_per_testcase(TestCase, Config) -> [{system_dir, SystemDir}, {user_dir, PrivDir}, {subsystems, [Spec]}]; - + "ver6_basic" -> + Spec = + ssh_sftpd:subsystem_spec([{sftpd_vsn, 6}]), + [{system_dir, SystemDir}, + {user_dir, PrivDir}, + {subsystems, [Spec]}]; _ -> [{user_dir, PrivDir}, {system_dir, SystemDir}] @@ -232,7 +238,7 @@ file_cb(Config) when is_list(Config) -> NewDir = filename:join(PrivDir, "testdir"), ok = ssh_sftp:make_dir(Sftp, NewDir), alt_file_handler_check(alt_make_dir), - + ok = ssh_sftp:del_dir(Sftp, NewDir), alt_file_handler_check(alt_read_link_info), alt_file_handler_check(alt_write_file_info), @@ -260,6 +266,15 @@ list_dir_limited(Config) when is_list(Config) -> ssh_sftp:list_dir(Sftp, "."), ct:pal("Listing: ~p~n", [Listing]). +ver6_basic(doc) -> + ["Test some version 6 features"]; +ver6_basic(Config) when is_list(Config) -> + PrivDir = ?config(priv_dir, Config), + NewDir = filename:join(PrivDir, "testdir2"), + {Sftp, _} = ?config(sftp, Config), + ok = ssh_sftp:make_dir(Sftp, NewDir), + %%Test file_is_a_directory + {error, file_is_a_directory} = ssh_sftp:delete(Sftp, NewDir). %%-------------------------------------------------------------------- %% Internal functions ------------------------------------------------ %%-------------------------------------------------------------------- diff --git a/lib/ssh/test/ssh_sftpd_erlclient_SUITE_data/ssh_sftpd_file_alt.erl b/lib/ssh/test/ssh_sftpd_erlclient_SUITE_data/ssh_sftpd_file_alt.erl index 9e119c4929..9f8a7c496c 100644 --- a/lib/ssh/test/ssh_sftpd_erlclient_SUITE_data/ssh_sftpd_file_alt.erl +++ b/lib/ssh/test/ssh_sftpd_erlclient_SUITE_data/ssh_sftpd_file_alt.erl @@ -48,7 +48,7 @@ get_cwd(State) -> {file:get_cwd(), State}. is_dir(AbsPath, State) -> - sftpd_file_alt_tester ! alt_is_dir, + %sftpd_file_alt_tester ! alt_is_dir, {filelib:is_dir(AbsPath), State}. list_dir(AbsPath, State) -> diff --git a/lib/ssl/doc/src/notes.xml b/lib/ssl/doc/src/notes.xml index 6c01954010..49bbd5d27d 100644 --- a/lib/ssl/doc/src/notes.xml +++ b/lib/ssl/doc/src/notes.xml @@ -30,7 +30,54 @@ </header> <p>This document describes the changes made to the SSL application.</p> - <section><title>SSL 5.1</title> + <section><title>SSL 5.1.2</title> + + <section><title>Fixed Bugs and Malfunctions</title> + <list> + <item> + <p> + ssl:ssl_accept/2 timeout is no longer ignored</p> + <p> + Own Id: OTP-10600</p> + </item> + </list> + </section> + +</section> + +<section><title>SSL 5.1.1</title> + + <section><title>Fixed Bugs and Malfunctions</title> + <list> + <item> + <p> + ssl:recv/3 could "loose" data when the timeout occurs. If + the timout in ssl:connect or ssl:ssl_accept expired the + ssl connection process was not terminated as it should, + this due to gen_fsm:send_all_state_event timout is a + client side time out. These timouts are now handled by + the gen_fsm-procss instead.</p> + <p> + Own Id: OTP-10569</p> + </item> + </list> + </section> + + + <section><title>Improvements and New Features</title> + <list> + <item> + <p> + Better termination handling that avoids hanging.</p> + <p> + Own Id: OTP-10574</p> + </item> + </list> + </section> + +</section> + +<section><title>SSL 5.1</title> <section><title>Fixed Bugs and Malfunctions</title> <list> diff --git a/lib/ssl/doc/src/ssl.xml b/lib/ssl/doc/src/ssl.xml index f0eac76264..e45a4c774f 100644 --- a/lib/ssl/doc/src/ssl.xml +++ b/lib/ssl/doc/src/ssl.xml @@ -36,8 +36,8 @@ <list type="bulleted"> <item>ssl requires the crypto and public_key applications.</item> - <item>Supported SSL/TLS-versions are SSL-3.0 and TLS-1.0, experimental - support for TLS-1.1 and TLS-1.2 is also available (no support for elliptic curve cipher suites yet).</item> + <item>Supported SSL/TLS-versions are SSL-3.0, TLS-1.0, + TLS-1.1 and TLS-1.2 (no support for elliptic curve cipher suites yet).</item> <item>For security reasons sslv2 is not supported.</item> <item>Ephemeral Diffie-Hellman cipher suites are supported but not Diffie Hellman Certificates cipher suites.</item> diff --git a/lib/ssl/doc/src/ssl_app.xml b/lib/ssl/doc/src/ssl_app.xml index 2ba6f48611..178bbcaebb 100644 --- a/lib/ssl/doc/src/ssl_app.xml +++ b/lib/ssl/doc/src/ssl_app.xml @@ -1,4 +1,4 @@ -<?xml version="1.0" encoding="latin1" ?> +<?xml version="1.0" encoding="iso-8859-1" ?> <!DOCTYPE appref SYSTEM "appref.dtd"> <appref> @@ -29,7 +29,17 @@ sockets.</appsummary> <section> - <title>Environment</title> + <title>DEPENDENCIES</title> + <p>The ssl application uses the Erlang applications public_key and + crypto to handle public keys and encryption, hence these + applications needs to be loaded for the ssl application to work. In + an embedded environment that means they need to be started with + application:start/[1,2] before the ssl application is started. + </p> + </section> + + <section> + <title>ENVIRONMENT</title> <p>The following application environment configuration parameters are defined for the SSL application. Refer to application(3) for more information about configuration parameters. diff --git a/lib/ssl/doc/src/ssl_protocol.xml b/lib/ssl/doc/src/ssl_protocol.xml index 17268a634d..f540dc999b 100644 --- a/lib/ssl/doc/src/ssl_protocol.xml +++ b/lib/ssl/doc/src/ssl_protocol.xml @@ -4,7 +4,7 @@ <chapter> <header> <copyright> - <year>2003</year><year>2011</year> + <year>2003</year><year>2012</year> <holder>Ericsson AB. All Rights Reserved.</holder> </copyright> <legalnotice> @@ -25,9 +25,8 @@ <file>ssl_protocol.xml</file> </header> - <p>The erlang SSL application currently supports SSL 3.0 and TLS 1.0 - RFC 2246, and will in the future also support later versions of TLS. - SSL 2.0 is not supported. + <p>The erlang SSL application currently implements the protocol SSL/TLS + for currently supported versions see <seealso marker="ssl">ssl(3)</seealso> </p> <p>By default erlang SSL is run over the TCP/IP protocol even diff --git a/lib/ssl/src/Makefile b/lib/ssl/src/Makefile index c5c5bf593a..6be8a1456e 100644 --- a/lib/ssl/src/Makefile +++ b/lib/ssl/src/Makefile @@ -130,3 +130,23 @@ release_spec: opt release_docs_spec: +# ---------------------------------------------------- +# Dependencies +# ---------------------------------------------------- +$(EBIN)/inet_tls_dist.$(EMULATOR): ../../kernel/include/net_address.hrl ../../kernel/include/dist.hrl ../../kernel/include/dist_util.hrl +$(EBIN)/ssl.$(EMULATOR): ssl_internal.hrl ssl_record.hrl ssl_cipher.hrl ssl_handshake.hrl ../../public_key/include/public_key.hrl +$(EBIN)/ssl_alert.$(EMULATOR): ssl_alert.hrl ssl_record.hrl +$(EBIN)/ssl_certificate.$(EMULATOR): ssl_internal.hrl ssl_alert.hrl ssl_handshake.hrl ../../public_key/include/public_key.hrl +$(EBIN)/ssl_certificate_db.$(EMULATOR): ssl_internal.hrl ../../public_key/include/public_key.hrl ../../kernel/include/file.hrl +$(EBIN)/ssl_cipher.$(EMULATOR): ssl_internal.hrl ssl_record.hrl ssl_cipher.hrl ssl_handshake.hrl ssl_alert.hrl ../../public_key/include/public_key.hrl +$(EBIN)/ssl_connection.$(EMULATOR): ssl_internal.hrl ssl_record.hrl ssl_cipher.hrl ssl_handshake.hrl ssl_alert.hrl ../../public_key/include/public_key.hrl +$(EBIN)/ssl_handshake.$(EMULATOR): ssl_internal.hrl ssl_record.hrl ssl_cipher.hrl ssl_handshake.hrl ssl_alert.hrl ../../public_key/include/public_key.hrl +$(EBIN)/ssl_manager.$(EMULATOR): ssl_internal.hrl ssl_handshake.hrl ../../kernel/include/file.hrl +$(EBIN)/ssl_record.$(EMULATOR): ssl_internal.hrl ssl_record.hrl ssl_cipher.hrl ssl_handshake.hrl ssl_alert.hrl +$(EBIN)/ssl_session.$(EMULATOR): ssl_internal.hrl ssl_handshake.hrl +$(EBIN)/ssl_session_cache.$(EMULATOR): ssl_internal.hrl ssl_handshake.hrl +$(EBIN)/ssl_session_cache_api.$(EMULATOR): ssl_internal.hrl ssl_handshake.hrl +$(EBIN)/ssl_ssl3.$(EMULATOR): ssl_internal.hrl ssl_record.hrl ssl_cipher.hrl +$(EBIN)/ssl_tls1.$(EMULATOR): ssl_internal.hrl ssl_record.hrl ssl_cipher.hrl + + diff --git a/lib/ssl/src/ssl.appup.src b/lib/ssl/src/ssl.appup.src index c118c129e8..9b1227fa7f 100644 --- a/lib/ssl/src/ssl.appup.src +++ b/lib/ssl/src/ssl.appup.src @@ -1,6 +1,8 @@ %% -*- erlang -*- {"%VSN%", [ + {"5.1.1", [{restart_application, ssl}] + }, {"5.1", [ {load_module, ssl_connection, soft_purge, soft_purge, []} ] @@ -10,6 +12,8 @@ {<<"3\\.*">>, [{restart_application, ssl}]} ], [ + {"5.1.1", [{restart_application, ssl}] + }, {"5.1", [ {load_module, ssl_connection, soft_purge, soft_purge, []} ] diff --git a/lib/ssl/src/ssl.erl b/lib/ssl/src/ssl.erl index 771bfa5739..09f2819ca8 100644 --- a/lib/ssl/src/ssl.erl +++ b/lib/ssl/src/ssl.erl @@ -28,14 +28,11 @@ cipher_suites/0, cipher_suites/1, suite_definition/1, close/1, shutdown/2, connect/3, connect/2, connect/4, connection_info/1, - controlling_process/2, listen/2, pid/1, peername/1, peercert/1, + controlling_process/2, listen/2, peername/1, peercert/1, recv/2, recv/3, send/2, getopts/2, setopts/2, sockname/1, versions/0, session_info/1, format_error/1, renegotiate/1, prf/5, clear_pem_cache/0, random_bytes/1, negotiated_next_protocol/1]). - --deprecated({pid, 1, next_major_release}). - -include("ssl_internal.hrl"). -include("ssl_record.hrl"). -include("ssl_cipher.hrl"). @@ -47,7 +44,7 @@ -export_type([connect_option/0, listen_option/0, ssl_option/0, transport_option/0, erl_cipher_suite/0, %% From ssl_cipher.hrl tls_atom_version/0, %% From ssl_internal.hrl - prf_random/0]). + prf_random/0, sslsocket/0]). -record(config, {ssl, %% SSL parameters inet_user, %% User set inet options @@ -55,6 +52,8 @@ inet_ssl, %% inet options for internal ssl socket cb %% Callback info }). + +-type sslsocket() :: #sslsocket{}. -type connect_option() :: socket_connect_option() | ssl_option() | transport_option(). -type socket_connect_option() :: gen_tcp:connect_option(). -type listen_option() :: socket_listen_option() | ssl_option() | transport_option(). @@ -954,12 +953,3 @@ make_next_protocol_selector({server, AllProtocols, DefaultProtocol}) -> PreferredProtocol -> PreferredProtocol end end. - -%% Only used to remove exit messages from old ssl -%% First is a nonsense clause to provide some -%% backward compatibility for orber that uses this -%% function in a none recommended way, but will -%% work correctly if a valid pid is returned. -%% Deprcated to be removed in r16 -pid(#sslsocket{})-> - whereis(ssl_connection_sup). diff --git a/lib/ssl/src/ssl_connection.erl b/lib/ssl/src/ssl_connection.erl index 87cf49d07d..cde13069b5 100644 --- a/lib/ssl/src/ssl_connection.erl +++ b/lib/ssl/src/ssl_connection.erl @@ -89,6 +89,7 @@ log_alert, % boolean() renegotiation, % {boolean(), From | internal | peer} start_or_recv_from, % "gen_fsm From" + timer, % start_or_recv_timer send_queue, % queue() terminated = false, % allow_renegotiate = true, @@ -775,8 +776,9 @@ handle_sync_event({application_data, Data}, From, StateName, get_timeout(State)}; handle_sync_event({start, Timeout}, StartFrom, hello, State) -> - start_or_recv_cancel_timer(Timeout, StartFrom), - hello(start, State#state{start_or_recv_from = StartFrom}); + Timer = start_or_recv_cancel_timer(Timeout, StartFrom), + hello(start, State#state{start_or_recv_from = StartFrom, + timer = Timer}); %% The two clauses below could happen if a server upgrades a socket in %% active mode. Note that in this case we are lucky that @@ -792,8 +794,9 @@ handle_sync_event({start,_}, _From, error, {Error, State = #state{}}) -> {stop, {shutdown, Error}, {error, Error}, State}; handle_sync_event({start, Timeout}, StartFrom, StateName, State) -> - start_or_recv_cancel_timer(Timeout, StartFrom), - {next_state, StateName, State#state{start_or_recv_from = StartFrom}, get_timeout(State)}; + Timer = start_or_recv_cancel_timer(Timeout, StartFrom), + {next_state, StateName, State#state{start_or_recv_from = StartFrom, + timer = Timer}, get_timeout(State)}; handle_sync_event(close, _, StateName, State) -> %% Run terminate before returning @@ -825,14 +828,16 @@ handle_sync_event({shutdown, How0}, _, StateName, end; handle_sync_event({recv, N, Timeout}, RecvFrom, connection = StateName, State0) -> - start_or_recv_cancel_timer(Timeout, RecvFrom), - passive_receive(State0#state{bytes_to_read = N, start_or_recv_from = RecvFrom}, StateName); + Timer = start_or_recv_cancel_timer(Timeout, RecvFrom), + passive_receive(State0#state{bytes_to_read = N, + start_or_recv_from = RecvFrom, timer = Timer}, StateName); %% Doing renegotiate wait with handling request until renegotiate is %% finished. Will be handled by next_state_is_connection/2. handle_sync_event({recv, N, Timeout}, RecvFrom, StateName, State) -> - start_or_recv_cancel_timer(Timeout, RecvFrom), - {next_state, StateName, State#state{bytes_to_read = N, start_or_recv_from = RecvFrom}, + Timer = start_or_recv_cancel_timer(Timeout, RecvFrom), + {next_state, StateName, State#state{bytes_to_read = N, start_or_recv_from = RecvFrom, + timer = Timer}, get_timeout(State)}; handle_sync_event({new_user, User}, _From, StateName, @@ -996,13 +1001,20 @@ handle_info({'DOWN', MonitorRef, _, _, _}, _, handle_info(allow_renegotiate, StateName, State) -> {next_state, StateName, State#state{allow_renegotiate = true}, get_timeout(State)}; - -handle_info({cancel_start_or_recv, RecvFrom}, connection = StateName, #state{start_or_recv_from = RecvFrom} = State) -> + +handle_info({cancel_start_or_recv, StartFrom}, StateName, + #state{renegotiation = {false, first}} = State) when StateName =/= connection -> + gen_fsm:reply(StartFrom, {error, timeout}), + {stop, {shutdown, user_timeout}, State#state{timer = undefined}}; + +handle_info({cancel_start_or_recv, RecvFrom}, StateName, #state{start_or_recv_from = RecvFrom} = State) -> gen_fsm:reply(RecvFrom, {error, timeout}), - {next_state, StateName, State#state{start_or_recv_from = undefined}, get_timeout(State)}; + {next_state, StateName, State#state{start_or_recv_from = undefined, + bytes_to_read = undefined, + timer = undefined}, get_timeout(State)}; handle_info({cancel_start_or_recv, _RecvFrom}, StateName, State) -> - {next_state, StateName, State, get_timeout(State)}; + {next_state, StateName, State#state{timer = undefined}, get_timeout(State)}; handle_info(Msg, StateName, State) -> Report = io_lib:format("SSL: Got unexpected info: ~p ~n", [Msg]), @@ -1770,6 +1782,7 @@ read_application_data(Data, #state{user_application = {_Mon, Pid}, socket_options = SOpts, bytes_to_read = BytesToRead, start_or_recv_from = RecvFrom, + timer = Timer, user_data_buffer = Buffer0} = State0) -> Buffer1 = if Buffer0 =:= <<>> -> Data; @@ -1779,9 +1792,11 @@ read_application_data(Data, #state{user_application = {_Mon, Pid}, case get_data(SOpts, BytesToRead, Buffer1) of {ok, ClientData, Buffer} -> % Send data SocketOpt = deliver_app_data(Socket, SOpts, ClientData, Pid, RecvFrom), + cancel_timer(Timer), State = State0#state{user_data_buffer = Buffer, start_or_recv_from = undefined, - bytes_to_read = 0, + timer = undefined, + bytes_to_read = undefined, socket_options = SocketOpt }, if @@ -1794,6 +1809,8 @@ read_application_data(Data, #state{user_application = {_Mon, Pid}, end; {more, Buffer} -> % no reply, we need more data next_record(State0#state{user_data_buffer = Buffer}); + {passive, Buffer} -> + next_record_if_active(State0#state{user_data_buffer = Buffer}); {error,_Reason} -> %% Invalid packet in packet mode deliver_packet_error(Socket, SOpts, Buffer1, Pid, RecvFrom), {stop, normal, State0} @@ -1835,6 +1852,9 @@ is_time_to_renegotiate(_,_) -> %% Picks ClientData get_data(_, _, <<>>) -> {more, <<>>}; +%% Recv timed out save buffer data until next recv +get_data(#socket_options{active=false}, undefined, Buffer) -> + {passive, Buffer}; get_data(#socket_options{active=Active, packet=Raw}, BytesToRead, Buffer) when Raw =:= raw; Raw =:= 0 -> %% Raw Mode if @@ -2140,7 +2160,6 @@ initial_state(Role, Host, Port, Socket, {SSLOptions, SocketOptions}, User, tls_record_buffer = <<>>, tls_cipher_texts = [], user_application = {Monitor, User}, - bytes_to_read = 0, user_data_buffer = <<>>, log_alert = true, session_cache_cb = SessionCacheCb, @@ -2363,9 +2382,11 @@ ack_connection(#state{renegotiation = {true, From}} = State) -> gen_fsm:reply(From, ok), State#state{renegotiation = undefined}; ack_connection(#state{renegotiation = {false, first}, - start_or_recv_from = StartFrom} = State) when StartFrom =/= undefined -> + start_or_recv_from = StartFrom, + timer = Timer} = State) when StartFrom =/= undefined -> gen_fsm:reply(StartFrom, connected), - State#state{renegotiation = undefined, start_or_recv_from = undefined}; + cancel_timer(Timer), + State#state{renegotiation = undefined, start_or_recv_from = undefined, timer = undefined}; ack_connection(State) -> State. @@ -2503,10 +2524,15 @@ default_hashsign(_Version, KeyExchange) {null, anon}. start_or_recv_cancel_timer(infinity, _RecvFrom) -> - ok; + undefined; start_or_recv_cancel_timer(Timeout, RecvFrom) -> erlang:send_after(Timeout, self(), {cancel_start_or_recv, RecvFrom}). +cancel_timer(undefined) -> + ok; +cancel_timer(Timer) -> + erlang:cancel_timer(Timer). + handle_unrecv_data(StateName, #state{socket = Socket, transport_cb = Transport} = State) -> inet:setopts(Socket, [{active, false}]), case Transport:recv(Socket, 0, 0) of diff --git a/lib/ssl/src/ssl_handshake.erl b/lib/ssl/src/ssl_handshake.erl index fa1784714f..db21dac942 100644 --- a/lib/ssl/src/ssl_handshake.erl +++ b/lib/ssl/src/ssl_handshake.erl @@ -1401,6 +1401,7 @@ default_hash_signs() -> [?TLSEXT_SIGALG(sha512), ?TLSEXT_SIGALG(sha384), ?TLSEXT_SIGALG(sha256), + ?TLSEXT_SIGALG(sha224), ?TLSEXT_SIGALG(sha), ?TLSEXT_SIGALG_DSA(sha), ?TLSEXT_SIGALG_RSA(md5)]}. diff --git a/lib/ssl/src/ssl_internal.hrl b/lib/ssl/src/ssl_internal.hrl index a5db2dcee7..ed0dc34adf 100644 --- a/lib/ssl/src/ssl_internal.hrl +++ b/lib/ssl/src/ssl_internal.hrl @@ -69,8 +69,8 @@ -define(TRUE, 0). -define(FALSE, 1). --define(DEFAULT_SUPPORTED_VERSIONS, [tlsv1, sslv3]). %% Add 'tlsv1.1' in R16 -define(ALL_SUPPORTED_VERSIONS, ['tlsv1.2', 'tlsv1.1', tlsv1, sslv3]). +-define(MIN_SUPPORTED_VERSIONS, ['tlsv1.1', tlsv1, sslv3]). -record(ssl_options, { versions, % 'tlsv1.2' | 'tlsv1.1' | tlsv1 | sslv3 diff --git a/lib/ssl/src/ssl_manager.erl b/lib/ssl/src/ssl_manager.erl index 0cf4f2ce33..13689ce7d8 100644 --- a/lib/ssl/src/ssl_manager.erl +++ b/lib/ssl/src/ssl_manager.erl @@ -24,8 +24,6 @@ -module(ssl_manager). -behaviour(gen_server). --include("ssl_internal.hrl"). - %% Internal application API -export([start_link/1, start_link_dist/1, connection_init/2, cache_pem_file/2, diff --git a/lib/ssl/src/ssl_record.erl b/lib/ssl/src/ssl_record.erl index 8e93ce4634..173b9611c6 100644 --- a/lib/ssl/src/ssl_record.erl +++ b/lib/ssl/src/ssl_record.erl @@ -463,10 +463,9 @@ supported_protocol_versions() -> supported_protocol_versions([]) -> Vsns = case sufficient_tlsv1_2_crypto_support() of true -> - %%?ALL_SUPPORTED_VERSIONS; %% Add TlS-1.2 as default in R16 - ?DEFAULT_SUPPORTED_VERSIONS; + ?ALL_SUPPORTED_VERSIONS; false -> - ?DEFAULT_SUPPORTED_VERSIONS + ?MIN_SUPPORTED_VERSIONS end, application:set_env(ssl, protocol_version, Vsns), Vsns; diff --git a/lib/ssl/src/ssl_session.erl b/lib/ssl/src/ssl_session.erl index 2ad422fc03..a24b2d9444 100644 --- a/lib/ssl/src/ssl_session.erl +++ b/lib/ssl/src/ssl_session.erl @@ -72,15 +72,12 @@ valid_session(#session{time_stamp = TimeStamp}, LifeTime) -> server_id(Port, <<>>, _SslOpts, _Cert, _, _) -> {ssl_manager:new_session_id(Port), undefined}; -server_id(Port, SuggestedId, - #ssl_options{reuse_sessions = ReuseEnabled, - reuse_session = ReuseFun}, - Cert, Cache, CacheCb) -> +server_id(Port, SuggestedId, Options, Cert, Cache, CacheCb) -> LifeTime = case application:get_env(ssl, session_lifetime) of {ok, Time} when is_integer(Time) -> Time; _ -> ?'24H_in_sec' end, - case is_resumable(SuggestedId, Port, ReuseEnabled,ReuseFun, + case is_resumable(SuggestedId, Port, Options, Cache, CacheCb, LifeTime, Cert) of {true, Resumed} -> @@ -112,9 +109,9 @@ select_session(Sessions, #ssl_options{ciphers = Ciphers}, OwnCert) -> [[Id, _]|_] -> Id end. -is_resumable(_, _, false, _, _, _, _, _) -> +is_resumable(_, _, #ssl_options{reuse_sessions = false}, _, _, _, _) -> {false, undefined}; -is_resumable(SuggestedSessionId, Port, true, ReuseFun, Cache, +is_resumable(SuggestedSessionId, Port, #ssl_options{reuse_session = ReuseFun} = Options, Cache, CacheCb, SecondLifeTime, OwnCert) -> case CacheCb:lookup(Cache, {Port, SuggestedSessionId}) of #session{cipher_suite = CipherSuite, @@ -125,6 +122,7 @@ is_resumable(SuggestedSessionId, Port, true, ReuseFun, Cache, case resumable(IsResumable) andalso (OwnCert == SessionOwnCert) andalso valid_session(Session, SecondLifeTime) + andalso reusable_options(Options, Session) andalso ReuseFun(SuggestedSessionId, PeerCert, Compression, CipherSuite) of @@ -139,3 +137,9 @@ resumable(new) -> false; resumable(IsResumable) -> IsResumable. + +reusable_options(#ssl_options{fail_if_no_peer_cert = true, + verify = verify_peer}, Session) -> + (Session#session.peer_certificate =/= undefined); +reusable_options(_,_) -> + true. diff --git a/lib/ssl/test/erl_make_certs.erl b/lib/ssl/test/erl_make_certs.erl index 254aa6d2f9..d6bdd05d01 100644 --- a/lib/ssl/test/erl_make_certs.erl +++ b/lib/ssl/test/erl_make_certs.erl @@ -137,10 +137,10 @@ decode_key(PemBin, Pw) -> encode_key(Key = #'RSAPrivateKey'{}) -> {ok, Der} = 'OTP-PUB-KEY':encode('RSAPrivateKey', Key), - {'RSAPrivateKey', list_to_binary(Der), not_encrypted}; + {'RSAPrivateKey', Der, not_encrypted}; encode_key(Key = #'DSAPrivateKey'{}) -> {ok, Der} = 'OTP-PUB-KEY':encode('DSAPrivateKey', Key), - {'DSAPrivateKey', list_to_binary(Der), not_encrypted}. + {'DSAPrivateKey', Der, not_encrypted}. make_tbs(SubjectKey, Opts) -> Version = list_to_atom("v"++integer_to_list(proplists:get_value(version, Opts, 3))), diff --git a/lib/ssl/test/ssl_basic_SUITE.erl b/lib/ssl/test/ssl_basic_SUITE.erl index a202aca943..faed91e559 100644 --- a/lib/ssl/test/ssl_basic_SUITE.erl +++ b/lib/ssl/test/ssl_basic_SUITE.erl @@ -248,6 +248,7 @@ api_tests() -> [connection_info, peername, peercert, + peercert_with_client_cert, sockname, versions, controlling_process, @@ -258,7 +259,9 @@ api_tests() -> shutdown_both, shutdown_error, hibernate, - listen_socket + listen_socket, + ssl_accept_timeout, + ssl_recv_timeout ]. certificate_verify_tests() -> @@ -274,6 +277,7 @@ certificate_verify_tests() -> server_verify_client_once_passive, server_verify_client_once_active, server_verify_client_once_active_once, + new_server_wants_peer_cert, client_verify_none_passive, client_verify_none_active, client_verify_none_active_once, @@ -788,6 +792,43 @@ peercert(Config) when is_list(Config) -> peercert_result(Socket) -> ssl:peercert(Socket). +%%-------------------------------------------------------------------- + +peercert_with_client_cert(doc) -> + [""]; +peercert_with_client_cert(suite) -> + []; +peercert_with_client_cert(Config) when is_list(Config) -> + ClientOpts = ?config(client_dsa_opts, Config), + ServerOpts = ?config(server_dsa_verify_opts, Config), + {ClientNode, ServerNode, Hostname} = ssl_test_lib:run_where(Config), + + Server = ssl_test_lib:start_server([{node, ClientNode}, {port, 0}, + {from, self()}, + {mfa, {?MODULE, peercert_result, []}}, + {options, ServerOpts}]), + Port = ssl_test_lib:inet_port(Server), + Client = ssl_test_lib:start_client([{node, ServerNode}, {port, Port}, + {host, Hostname}, + {from, self()}, + {mfa, {?MODULE, peercert_result, []}}, + {options, ClientOpts}]), + + ServerCertFile = proplists:get_value(certfile, ServerOpts), + [{'Certificate', ServerBinCert, _}]= ssl_test_lib:pem_to_der(ServerCertFile), + ClientCertFile = proplists:get_value(certfile, ClientOpts), + [{'Certificate', ClientBinCert, _}]= ssl_test_lib:pem_to_der(ClientCertFile), + + ServerMsg = {ok, ClientBinCert}, + ClientMsg = {ok, ServerBinCert}, + + test_server:format("Testcase ~p, Client ~p Server ~p ~n", + [self(), Client, Server]), + + ssl_test_lib:check_result(Server, ServerMsg, Client, ClientMsg), + + ssl_test_lib:close(Server), + ssl_test_lib:close(Client). %%-------------------------------------------------------------------- sockname(doc) -> @@ -3610,9 +3651,14 @@ no_reuses_session_server_restart_new_cert(Config) when is_list(Config) -> %% Make sure session is registered test_server:sleep(?SLEEP), + Monitor = erlang:monitor(process, Server), ssl_test_lib:close(Server), ssl_test_lib:close(Client0), - + receive + {'DOWN', Monitor, _, _, _} -> + ok + end, + Server1 = ssl_test_lib:start_server([{node, ServerNode}, {port, Port}, {from, self()}, @@ -3719,10 +3765,14 @@ reuseaddr(Config) when is_list(Config) -> {from, self()}, {mfa, {ssl_test_lib, no_result, []}}, {options, [{active, false} | ClientOpts]}]), - test_server:sleep(?SLEEP), + Monitor = erlang:monitor(process, Server), ssl_test_lib:close(Server), ssl_test_lib:close(Client), - + receive + {'DOWN', Monitor, _, _, _} -> + ok + end, + Server1 = ssl_test_lib:start_server([{node, ServerNode}, {port, Port}, {from, self()}, @@ -3805,6 +3855,61 @@ listen_socket(Config) -> {error, enotconn} = ssl:shutdown(ListenSocket, read_write), ok = ssl:close(ListenSocket). +%%-------------------------------------------------------------------- +ssl_accept_timeout(doc) -> + ["Test ssl:ssl_accept timeout"]; +ssl_accept_timeout(suite) -> + []; +ssl_accept_timeout(Config) -> + process_flag(trap_exit, true), + ServerOpts = ?config(server_opts, Config), + {_, ServerNode, Hostname} = ssl_test_lib:run_where(Config), + Server = ssl_test_lib:start_server([{node, ServerNode}, {port, 0}, + {from, self()}, + {timeout, 5000}, + {mfa, {ssl_test_lib, + no_result_msg, []}}, + {options, ServerOpts}]), + Port = ssl_test_lib:inet_port(Server), + {ok, CSocket} = gen_tcp:connect(Hostname, Port, [binary, {active, true}]), + + receive + {tcp_closed, CSocket} -> + ssl_test_lib:check_result(Server, {error, timeout}), + receive + {'EXIT', Server, _} -> + [] = supervisor:which_children(ssl_connection_sup) + end + end. + +%%-------------------------------------------------------------------- +ssl_recv_timeout(doc) -> + ["Test ssl:ssl_accept timeout"]; +ssl_recv_timeout(suite) -> + []; +ssl_recv_timeout(Config) -> + ServerOpts = ?config(server_opts, Config), + ClientOpts = ?config(client_opts, Config), + + {ClientNode, ServerNode, Hostname} = ssl_test_lib:run_where(Config), + + Server = + ssl_test_lib:start_server([{node, ServerNode}, {port, 0}, + {from, self()}, + {mfa, {?MODULE, send_recv_result_timeout_server, []}}, + {options, [{active, false} | ServerOpts]}]), + Port = ssl_test_lib:inet_port(Server), + + Client = ssl_test_lib:start_client([{node, ClientNode}, {port, Port}, + {host, Hostname}, + {from, self()}, + {mfa, {?MODULE, + send_recv_result_timeout_client, []}}, + {options, [{active, false} | ClientOpts]}]), + + ssl_test_lib:check_result(Client, ok, Server, ok), + ssl_test_lib:close(Server), + ssl_test_lib:close(Client). %%-------------------------------------------------------------------- @@ -4041,6 +4146,67 @@ client_server_opts({KeyAlgo,_,_}, Config) when KeyAlgo == dss orelse KeyAlgo == {?config(client_dsa_opts, Config), ?config(server_dsa_opts, Config)}. + +%%-------------------------------------------------------------------- + +new_server_wants_peer_cert(doc) -> + ["Test that server configured to do client certification does" + " not reuse session without a client certificate."]; +new_server_wants_peer_cert(suite) -> + []; +new_server_wants_peer_cert(Config) when is_list(Config) -> + ServerOpts = ?config(server_opts, Config), + VServerOpts = [{verify, verify_peer}, {fail_if_no_peer_cert, true} + | ?config(server_verification_opts, Config)], + ClientOpts = ?config(client_verification_opts, Config), + + {ClientNode, ServerNode, Hostname} = ssl_test_lib:run_where(Config), + + Server = + ssl_test_lib:start_server([{node, ServerNode}, {port, 0}, + {from, self()}, + {mfa, {?MODULE, peercert_result, []}}, + {options, [ServerOpts]}]), + Port = ssl_test_lib:inet_port(Server), + Client = + ssl_test_lib:start_client([{node, ClientNode}, {port, Port}, + {host, Hostname}, + {from, self()}, + {mfa, {ssl_test_lib, no_result, []}}, + {options, ClientOpts}]), + + Monitor = erlang:monitor(process, Server), + ssl_test_lib:close(Server), + ssl_test_lib:close(Client), + receive + {'DOWN', Monitor, _, _, _} -> + ok + end, + + Server1 = ssl_test_lib:start_server([{node, ServerNode}, {port, Port}, + {from, self()}, + {mfa, {?MODULE, peercert_result, []}}, + {options, VServerOpts}]), + Client1 = + ssl_test_lib:start_client([{node, ClientNode}, {port, Port}, + {host, Hostname}, + {from, self()}, + {mfa, {ssl_test_lib, no_result, []}}, + {options, [ClientOpts]}]), + + CertFile = proplists:get_value(certfile, ClientOpts), + [{'Certificate', BinCert, _}]= ssl_test_lib:pem_to_der(CertFile), + + ServerMsg = {error, no_peercert}, + Sever1Msg = {ok, BinCert}, + + ssl_test_lib:check_result(Server, ServerMsg, Server1, Sever1Msg), + + ssl_test_lib:close(Server1), + ssl_test_lib:close(Client), + ssl_test_lib:close(Client1). + + %%-------------------------------------------------------------------- %%% Internal functions %%-------------------------------------------------------------------- @@ -4049,6 +4215,23 @@ send_recv_result(Socket) -> {ok,"Hello world"} = ssl:recv(Socket, 11), ok. +send_recv_result_timeout_client(Socket) -> + {error, timeout} = ssl:recv(Socket, 11, 500), + ssl:send(Socket, "Hello world"), + receive + Msg -> + io:format("Msg ~p~n",[Msg]) + after 500 -> + ok + end, + {ok, "Hello world"} = ssl:recv(Socket, 11, 500), + ok. +send_recv_result_timeout_server(Socket) -> + ssl:send(Socket, "Hello"), + {ok, "Hello world"} = ssl:recv(Socket, 11), + ssl:send(Socket, " world"), + ok. + recv_close(Socket) -> {error, closed} = ssl:recv(Socket, 11), receive diff --git a/lib/ssl/test/ssl_test_lib.erl b/lib/ssl/test/ssl_test_lib.erl index 63731ee25c..f1f5b9ae0a 100644 --- a/lib/ssl/test/ssl_test_lib.erl +++ b/lib/ssl/test/ssl_test_lib.erl @@ -72,7 +72,13 @@ run_server(Opts) -> run_server(ListenSocket, Opts). run_server(ListenSocket, Opts) -> - AcceptSocket = connect(ListenSocket, Opts), + do_run_server(ListenSocket, connect(ListenSocket, Opts), Opts). + +do_run_server(_, {error, timeout} = Result, Opts) -> + Pid = proplists:get_value(from, Opts), + Pid ! {self(), Result}; + +do_run_server(ListenSocket, AcceptSocket, Opts) -> Node = proplists:get_value(node, Opts), Pid = proplists:get_value(from, Opts), {Module, Function, Args} = proplists:get_value(mfa, Opts), @@ -102,7 +108,8 @@ run_server(ListenSocket, Opts) -> connect(ListenSocket, Opts) -> Node = proplists:get_value(node, Opts), ReconnectTimes = proplists:get_value(reconnect_times, Opts, 0), - AcceptSocket = connect(ListenSocket, Node, 1 + ReconnectTimes, dummy), + Timeout = proplists:get_value(timeout, Opts, infinity), + AcceptSocket = connect(ListenSocket, Node, 1 + ReconnectTimes, dummy, Timeout), case ReconnectTimes of 0 -> AcceptSocket; @@ -111,15 +118,21 @@ connect(ListenSocket, Opts) -> AcceptSocket end. -connect(_, _, 0, AcceptSocket) -> +connect(_, _, 0, AcceptSocket, _) -> AcceptSocket; -connect(ListenSocket, Node, N, _) -> +connect(ListenSocket, Node, N, _, Timeout) -> test_server:format("ssl:transport_accept(~p)~n", [ListenSocket]), {ok, AcceptSocket} = rpc:call(Node, ssl, transport_accept, [ListenSocket]), - test_server:format("ssl:ssl_accept(~p)~n", [AcceptSocket]), - ok = rpc:call(Node, ssl, ssl_accept, [AcceptSocket]), - connect(ListenSocket, Node, N-1, AcceptSocket). + test_server:format("ssl:ssl_accept(~p, ~p)~n", [AcceptSocket, Timeout]), + + case rpc:call(Node, ssl, ssl_accept, [AcceptSocket, Timeout]) of + ok -> + connect(ListenSocket, Node, N-1, AcceptSocket, Timeout); + Result -> + Result + end. + remove_close_msg(0) -> ok; diff --git a/lib/ssl/test/ssl_to_openssl_SUITE.erl b/lib/ssl/test/ssl_to_openssl_SUITE.erl index 98ef050b14..107220c335 100644 --- a/lib/ssl/test/ssl_to_openssl_SUITE.erl +++ b/lib/ssl/test/ssl_to_openssl_SUITE.erl @@ -106,8 +106,8 @@ init_per_testcase(TestCase, Config0) -> special_init(TestCase, Config) when TestCase == erlang_client_openssl_server_renegotiate; - TestCase == erlang_client_openssl_server_no_wrap_sequence_number; - TestCase == erlang_server_openssl_client_no_wrap_sequence_number + TestCase == erlang_client_openssl_server_nowrap_seqnum; + TestCase == erlang_server_openssl_client_nowrap_seqnum -> check_sane_openssl_renegotaite(Config); @@ -1080,14 +1080,13 @@ ssl2_erlang_server_openssl_client(Config) when is_list(Config) -> OpenSslPort = open_port({spawn, Cmd}, [stderr_to_stdout]), port_command(OpenSslPort, Data), - + receive + {'EXIT', OpenSslPort, _} -> + ok + + end, ssl_test_lib:check_result(Server, {error,"protocol version"}), - - %% Clean close down! Server needs to be closed first !! - ssl_test_lib:close(Server), - close_port(OpenSslPort), - process_flag(trap_exit, false), - ok. + process_flag(trap_exit, false). %%-------------------------------------------------------------------- erlang_client_openssl_server_npn(doc) -> diff --git a/lib/ssl/vsn.mk b/lib/ssl/vsn.mk index bc8b8fd039..adfb29e639 100644 --- a/lib/ssl/vsn.mk +++ b/lib/ssl/vsn.mk @@ -1 +1 @@ -SSL_VSN = 5.1.1 +SSL_VSN = 5.1.2 diff --git a/lib/stdlib/doc/src/Makefile b/lib/stdlib/doc/src/Makefile index 50f6427eaa..6f1e61e70c 100644 --- a/lib/stdlib/doc/src/Makefile +++ b/lib/stdlib/doc/src/Makefile @@ -101,7 +101,6 @@ XML_REF6_FILES = stdlib_app.xml XML_PART_FILES = part.xml part_notes.xml part_notes_history.xml XML_CHAPTER_FILES = io_protocol.xml unicode_usage.xml notes.xml notes_history.xml -GIF_FILES = ushell1.gif ushell2.gif ushell3.gif BOOK_FILES = book.xml @@ -112,8 +111,7 @@ XML_FILES = \ # ---------------------------------------------------- HTML_FILES = $(XML_APPLICATION_FILES:%.xml=$(HTMLDIR)/%.html) \ - $(XML_PART_FILES:%.xml=$(HTMLDIR)/%.html) \ - $(GIF_FILES:%.gif=$(HTMLDIR)/%.gif) + $(XML_PART_FILES:%.xml=$(HTMLDIR)/%.html) INFO_FILE = ../../info @@ -138,21 +136,16 @@ SPECS_FLAGS = -I../../include -I../../../kernel/include # ---------------------------------------------------- # Targets # ---------------------------------------------------- -$(HTMLDIR)/%.gif: %.gif - $(INSTALL_DATA) $< $@ - docs: man pdf html $(TOP_PDF_FILE): $(XML_FILES) pdf: $(TOP_PDF_FILE) -html: gifs $(HTML_REF_MAN_FILE) +html: $(HTML_REF_MAN_FILE) man: $(MAN3_FILES) $(MAN6_FILES) -gifs: $(GIF_FILES:%=$(HTMLDIR)/%) - debug opt: clean clean_docs: diff --git a/lib/stdlib/doc/src/epp.xml b/lib/stdlib/doc/src/epp.xml index 386ed89fe1..3e8aba2e5f 100644 --- a/lib/stdlib/doc/src/epp.xml +++ b/lib/stdlib/doc/src/epp.xml @@ -37,6 +37,18 @@ <p>The Erlang code preprocessor includes functions which are used by <c>compile</c> to preprocess macros and include files before the actual parsing takes place.</p> + <p>The Erlang source file <marker + id="encoding"><em>encoding</em></marker> is selected by a + comment in one of the first two lines of the source file. The + first string that matches the regular expression + <c>coding\s*[:=]\s*([-a-zA-Z0-9])+</c> selects the encoding. If + the matching string is not a valid encoding it is ignored. The + valid encodings are <c>Latin-1</c> and <c>UTF-8</c> where the + case of the characters can be chosen freely. Examples:</p> + <pre> +%% coding: utf-8 +%% For this file we have chosen encoding = Latin-1 +%% -*- coding: latin-1 -*-</pre> </description> <datatypes> <datatype> @@ -46,6 +58,9 @@ <name name="epp_handle"></name> <desc><p>Handle to the epp server.</p></desc> </datatype> + <datatype> + <name name="source_encoding"></name> + </datatype> </datatypes> <funcs> <func> @@ -83,6 +98,50 @@ </desc> </func> <func> + <name name="default_encoding" arity="0"/> + <fsummary>Return the default encoding of Erlang source files</fsummary> + <desc> + <p>Returns the default encoding of Erlang source files.</p> + </desc> + </func> + <func> + <name name="encoding_to_string" arity="1"/> + <fsummary>Return a string representation of an encoding</fsummary> + <desc> + <p>Returns a string representation of an encoding. The string + is recognized by <c>read_encoding/1,2</c> and + <c>set_encoding/1</c> as a valid encoding.</p> + </desc> + </func> + <func> + <name name="read_encoding" arity="1"/> + <name name="read_encoding" arity="2"/> + <fsummary>Read the encoding from a file</fsummary> + <desc> + <p>Read the <seealso marker="#encoding">encoding</seealso> from + a file. Returns the read encoding, or <c>none</c> if no + valid encoding was found.</p> + <p>The option <c>in_comment_only</c> is <c>true</c> by + default, which is correct for Erlang source files. If set to + <c>false</c> the encoding string does not necessarily have to + occur in a comment.</p> + </desc> + </func> + <func> + <name name="set_encoding" arity="1"/> + <fsummary>Read and set the encoding of an IO device</fsummary> + <desc> + <p>Reads the <seealso marker="#encoding">encoding</seealso> from + an IO device and sets the encoding of the device + accordingly. The position of the IO device referenced by + <c><anno>File</anno></c> is not affected. If no valid + encoding can be read from the IO device the encoding of the + IO device is set to the default encoding.</p> + <p>Returns the read encoding, or <c>none</c> if no valid + encoding was found.</p> + </desc> + </func> + <func> <name name="format_error" arity="1"/> <fsummary>Format an error descriptor</fsummary> <desc> diff --git a/lib/stdlib/doc/src/erl_pp.xml b/lib/stdlib/doc/src/erl_pp.xml index 57b5828bcd..9ae4f3d91f 100644 --- a/lib/stdlib/doc/src/erl_pp.xml +++ b/lib/stdlib/doc/src/erl_pp.xml @@ -5,7 +5,7 @@ <header> <copyright> <year>1996</year> - <year>2011</year> + <year>2012</year> <holder>Ericsson AB, All Rights Reserved</holder> </copyright> <legalnotice> @@ -63,6 +63,12 @@ breaks and only a space is used as a separator.</p> </desc> </datatype> + <datatype> + <name name="option"/> + </datatype> + <datatype> + <name name="options"/> + </datatype> </datatypes> <funcs> <func> diff --git a/lib/stdlib/doc/src/io.xml b/lib/stdlib/doc/src/io.xml index e6d262466c..22cd45a482 100644 --- a/lib/stdlib/doc/src/io.xml +++ b/lib/stdlib/doc/src/io.xml @@ -4,7 +4,7 @@ <erlref> <header> <copyright> - <year>1996</year><year>2011</year> + <year>1996</year><year>2012</year> <holder>Ericsson AB. All Rights Reserved.</holder> </copyright> <legalnotice> @@ -28,9 +28,9 @@ <rev></rev> </header> <module>io</module> - <modulesummary>Standard IO Server Interface Functions</modulesummary> + <modulesummary>Standard I/O Server Interface Functions</modulesummary> <description> - <p>This module provides an interface to standard Erlang IO servers. + <p>This module provides an interface to standard Erlang I/O servers. The output functions all return <c>ok</c> if they are successful, or exit if they are not.</p> <p>In the following description, all functions have an optional @@ -38,17 +38,16 @@ process which handles the IO protocols. Normally, it is the <c>IoDevice</c> returned by <seealso marker="kernel:file#open/2">file:open/2</seealso>.</p> - <p>For a description of the IO protocols refer to the STDLIB Users Guide.</p> + <p>For a description of the IO protocols refer to the <seealso marker="io_protocol">STDLIB User's Guide</seealso>.</p> <warning> <p>As of R13A, data supplied to the <seealso marker="#put_chars/2">put_chars</seealso> function should be in the <seealso marker="unicode#type-chardata"><c>unicode:chardata()</c></seealso> format. This means that programs supplying binaries to this function need to convert them to UTF-8 - before trying to output the data on an - <c>io_device()</c>.</p> + before trying to output the data on an IO device.</p> - <p>If an io_device() is set in binary mode, the functions <seealso + <p>If an IO device is set in binary mode, the functions <seealso marker="#get_chars/3">get_chars</seealso> and <seealso marker="#get_line/2">get_line</seealso> may return binaries instead of lists. The binaries will, as of R13A, be encoded in @@ -68,9 +67,9 @@ <datatype> <name name="device"/> <desc> - <p>Either <c>standard_io</c>, <c>standard_error</c>, a + <p>An IO device. Either <c>standard_io</c>, <c>standard_error</c>, a registered name, or a pid handling IO protocols (returned from - <seealso marker="kernel:file#open/2">file:open/2</seealso>).</p> + <seealso marker="kernel:file#open/2">file:open/2</seealso>).</p> </desc> </datatype> <datatype> @@ -89,17 +88,14 @@ <name name="format"/> </datatype> <datatype> - <name name="line"/> + <name name="location"/> </datatype> <datatype> <name name="prompt"/> </datatype> <datatype> - <name name="request_error"/> - </datatype> - <datatype> - <name name="error_description"/> - <desc><p>Whatever the I/O-server sends.</p></desc> + <name name="server_no_data"/> + <desc><p>What the I/O-server sends when there is no data.</p></desc> </datatype> </datatypes> @@ -107,11 +103,11 @@ <func> <name name="columns" arity="0"/> <name name="columns" arity="1"/> - <fsummary>Get the number of columns of a device</fsummary> + <fsummary>Get the number of columns of an IO device</fsummary> <desc> <p>Retrieves the number of columns of the <c><anno>IoDevice</anno></c> (i.e. the width of a terminal). The function - only succeeds for terminal devices, for all other devices + only succeeds for terminal devices, for all other IO devices the function returns <c>{error, enotsup}</c></p> </desc> </func> @@ -120,7 +116,7 @@ <name name="put_chars" arity="2"/> <fsummary>Write a list of characters</fsummary> <desc> - <p>Writes the characters of <c><anno>CharData</anno></c> to the io_server() + <p>Writes the characters of <c><anno>CharData</anno></c> to the I/O server (<c><anno>IoDevice</anno></c>).</p> </desc> </func> @@ -135,6 +131,7 @@ <func> <name name="get_chars" arity="2"/> <name name="get_chars" arity="3"/> + <type name="server_no_data"/> <fsummary>Read a specified number of characters</fsummary> <desc> <p>Reads <c><anno>Count</anno></c> characters from standard input @@ -143,19 +140,19 @@ <taglist> <tag><c><anno>Data</anno></c></tag> <item> - <p>The input characters. If the device supports Unicode, + <p>The input characters. If the IO device supports Unicode, the data may represent codepoints larger than 255 (the - latin1 range). If the io_server() is set to deliver + latin1 range). If the I/O server is set to deliver binaries, they will be encoded in UTF-8 (regardless of if - the device actually supports Unicode or not).</p> + the IO device actually supports Unicode or not).</p> </item> <tag><c>eof</c></tag> <item> <p>End of file was encountered.</p> </item> - <tag><c>{error,<anno>Reason</anno>}</c></tag> + <tag><c>{error, <anno>ErrorDescription</anno>}</c></tag> <item> - <p>Other (rare) error condition, for instance <c>{error,estale}</c> + <p>Other (rare) error condition, for instance <c>{error, estale}</c> if reading from an NFS file system.</p> </item> </taglist> @@ -164,6 +161,7 @@ <func> <name name="get_line" arity="1"/> <name name="get_line" arity="2"/> + <type name="server_no_data"/> <fsummary>Read a line</fsummary> <desc> <p>Reads a line from the standard input (<c><anno>IoDevice</anno></c>), @@ -172,19 +170,19 @@ <tag><c><anno>Data</anno></c></tag> <item> <p>The characters in the line terminated by a LF (or end of - file). If the device supports Unicode, + file). If the IO device supports Unicode, the data may represent codepoints larger than 255 (the - latin1 range). If the io_server() is set to deliver + latin1 range). If the I/O server is set to deliver binaries, they will be encoded in UTF-8 (regardless of if - the device actually supports Unicode or not).</p> + the IO device actually supports Unicode or not).</p> </item> <tag><c>eof</c></tag> <item> <p>End of file was encountered.</p> </item> - <tag><c>{error,<anno>Reason</anno>}</c></tag> + <tag><c>{error, <anno>ErrorDescription</anno>}</c></tag> <item> - <p>Other (rare) error condition, for instance <c>{error,estale}</c> + <p>Other (rare) error condition, for instance <c>{error, estale}</c> if reading from an NFS file system.</p> </item> </taglist> @@ -195,7 +193,7 @@ <name name="getopts" arity="1"/> <fsummary>Get the supported options and values from an I/O-server</fsummary> <desc> - <p>This function requests all available options and their current values for a specific io_device(). Example:</p> + <p>This function requests all available options and their current values for a specific IO device. Example:</p> <pre> 1> <input>{ok,F} = file:open("/dev/null",[read]).</input> {ok,<0.42.0>} @@ -217,30 +215,30 @@ <name name="setopts" arity="2"/> <fsummary>Set options</fsummary> <desc> - <p>Set options for the io_device() (<c><anno>IoDevice</anno></c>).</p> + <p>Set options for the standard IO device (<c><anno>IoDevice</anno></c>).</p> <p>Possible options and values vary depending on the actual - io_device(). For a list of supported options and their current values - on a specific device, use the <seealso + IO device. For a list of supported options and their current values + on a specific IO device, use the <seealso marker="#getopts/1">getopts/1</seealso> function.</p> - <p>The options and values supported by the current OTP io_devices are:</p> + <p>The options and values supported by the current OTP IO devices are:</p> <taglist> <tag><c>binary, list or {binary, boolean()}</c></tag> <item> - <p>If set in binary mode (binary or {binary,true}), the io_server() sends binary data (encoded in UTF-8) as answers to the get_line, get_chars and, if possible, get_until requests (see the I/O protocol description in STDLIB User's Guide for details). The immediate effect is that <c>get_chars/2,3</c> and <c>get_line/1,2</c> return UTF-8 binaries instead of lists of chars for the affected device.</p> - <p>By default, all io_devices in OTP are set in list mode, but the io functions can handle any of these modes and so should other, user written, modules behaving as clients to I/O-servers.</p> - <p>This option is supported by the standard shell (group.erl), the 'oldshell' (user.erl) and the file I/O servers.</p> + <p>If set in binary mode (<c>binary</c> or <c>{binary, true}</c>), the I/O server sends binary data (encoded in UTF-8) as answers to the <c>get_line</c>, <c>get_chars</c> and, if possible, <c>get_until</c> requests (see the I/O protocol description in <seealso marker="io_protocol">STDLIB User's Guide</seealso> for details). The immediate effect is that <c>get_chars/2,3</c> and <c>get_line/1,2</c> return UTF-8 binaries instead of lists of chars for the affected IO device.</p> + <p>By default, all IO devices in OTP are set in list mode, but the I/O functions can handle any of these modes and so should other, user written, modules behaving as clients to I/O-servers.</p> + <p>This option is supported by the standard shell (<c>group.erl</c>), the 'oldshell' (<c>user.erl</c>) and the file I/O servers.</p> </item> <tag><c>{echo, boolean()}</c></tag> <item> - <p>Denotes if the terminal should echo input. Only supported for the standard shell I/O-server (group.erl)</p> + <p>Denotes if the terminal should echo input. Only supported for the standard shell I/O-server (<c>group.erl</c>)</p> </item> <tag><c>{expand_fun, expand_fun()}</c></tag> <item> <p>Provide a function for tab-completion (expansion) - like the erlang shell. This function is called - when the user presses the Tab key. The expansion is + like the Erlang shell. This function is called + when the user presses the TAB key. The expansion is active when calling line-reading functions such as <c>get_line/1,2</c>.</p> <p>The function is called with the current line, upto @@ -253,25 +251,25 @@ will be printed and the current input line will be written once again.</p> <p>Trivial example (beep on anything except empty line, which - is expanded to "quit"):</p> + is expanded to <c>"quit"</c>):</p> <code type="none"> fun("") -> {yes, "quit", []}; (_) -> {no, "", ["quit"]} end</code> - <p>This option is supported by the standard shell only (group.erl).</p> + <p>This option is supported by the standard shell only (<c>group.erl</c>).</p> </item> <tag><c>{encoding, latin1 | unicode}</c></tag> <item> - <p>Specifies how characters are input or output from or to the actual device, implying that i.e. a terminal is set to handle Unicode input and output or a file is set to handle UTF-8 data encoding.</p> - <p>The option <em>does not</em> affect how data is returned from the io-functions or how it is sent in the I/O-protocol, it only affects how the io_device() is to handle Unicode characters towards the "physical" device.</p> - <p>The standard shell will be set for either unicode or latin1 encoding when the system is started. The actual encoding is set with the help of the "LANG" or "LC_CTYPE" environment variables on Unix-like system or by other means on other systems. The bottom line is that the user can input Unicode characters and the device will be in {encoding, unicode} mode if the device supports it. The mode can be changed, if the assumption of the runtime system is wrong, by setting this option.</p> - <p>The io_device() used when Erlang is started with the "-oldshell" or "-noshell" flags is by default set to latin1 encoding, meaning that any characters beyond codepoint 255 will be escaped and that input is expected to be plain 8-bit ISO-latin-1. If the encoding is changed to Unicode, input and output from the standard file descriptors will be in UTF-8 (regardless of operating system).</p> - <p>Files can also be set in {encoding, unicode}, meaning that data is written and read as UTF-8. More encodings are possible for files, see below.</p> - <p>{encoding, unicode | latin1} is supported by both the standard shell (group.erl including werl on windows), the 'oldshell' (user.erl) and the file I/O servers.</p> + <p>Specifies how characters are input or output from or to the actual IO device, implying that i.e. a terminal is set to handle Unicode input and output or a file is set to handle UTF-8 data encoding.</p> + <p>The option <em>does not</em> affect how data is returned from the I/O functions or how it is sent in the I/O-protocol, it only affects how the IO device is to handle Unicode characters towards the "physical" device.</p> + <p>The standard shell will be set for either Unicode or latin1 encoding when the system is started. The actual encoding is set with the help of the <c>LANG</c> or <c>LC_CTYPE</c> environment variables on Unix-like system or by other means on other systems. The bottom line is that the user can input Unicode characters and the IO device will be in <c>{encoding, unicode}</c> mode if the IO device supports it. The mode can be changed, if the assumption of the runtime system is wrong, by setting this option.</p> + <p>The IO device used when Erlang is started with the "-oldshell" or "-noshell" flags is by default set to latin1 encoding, meaning that any characters beyond codepoint 255 will be escaped and that input is expected to be plain 8-bit ISO-latin-1. If the encoding is changed to Unicode, input and output from the standard file descriptors will be in UTF-8 (regardless of operating system).</p> + <p>Files can also be set in <c>{encoding, unicode}</c>, meaning that data is written and read as UTF-8. More encodings are possible for files, see below.</p> + <p><c>{encoding, unicode | latin1}</c> is supported by both the standard shell (<c>group.erl</c> including <c>werl</c> on Windows®), the 'oldshell' (<c>user.erl</c>) and the file I/O servers.</p> </item> <tag><c>{encoding, utf8 | utf16 | utf32 | {utf16,big} | {utf16,little} | {utf32,big} | {utf32,little}}</c></tag> <item> <p>For disk files, the encoding can be set to various UTF variants. This will have the effect that data is expected to be read as the specified encoding from the file and the data will be written in the specified encoding to the disk file.</p> - <p>{encoding, utf8} will have the same effect as {encoding,unicode} on files.</p> + <p><c>{encoding, utf8}</c> will have the same effect as <c>{encoding, unicode}</c> on files.</p> <p>The extended encodings are only supported on disk files (opened by the <seealso marker="kernel:file#open/2">file:open/2</seealso> function)</p> </item> </taglist> @@ -289,6 +287,7 @@ <func> <name name="read" arity="1"/> <name name="read" arity="2"/> + <type name="server_no_data"/> <fsummary>Read a term</fsummary> <desc> <p>Reads a term <c><anno>Term</anno></c> from the standard input @@ -312,21 +311,25 @@ </func> <func> <name name="read" arity="3"/> + <name name="read" arity="4"/> + <type name="server_no_data"/> <fsummary>Read a term</fsummary> <desc> <p>Reads a term <c><anno>Term</anno></c> from <c><anno>IoDevice</anno></c>, prompting it - with <c><anno>Prompt</anno></c>. Reading starts at line number - <c><anno>StartLine</anno></c>. It returns:</p> + with <c><anno>Prompt</anno></c>. Reading starts at location + <c><anno>StartLocation</anno></c>. The argument + <c><anno>Options</anno></c> is passed on as the <c>Options</c> + argument of the <c>erl_scan:tokens/4</c> function. It returns:</p> <taglist> - <tag><c>{ok, Term, <anno>EndLine</anno>}</c></tag> + <tag><c>{ok, Term, <anno>EndLocation</anno>}</c></tag> <item> <p>The parsing was successful.</p> </item> - <tag><c>{eof, <anno>EndLine</anno>}</c></tag> + <tag><c>{eof, <anno>EndLocation</anno>}</c></tag> <item> <p>End of file was encountered.</p> </item> - <tag><c>{error, <anno>ErrorInfo</anno>, <anno>ErrorLine</anno>}</c></tag> + <tag><c>{error, <anno>ErrorInfo</anno>, <anno>ErrorLocation</anno>}</c></tag> <item> <p>The parsing failed.</p> </item> @@ -377,7 +380,7 @@ ok</pre> applicable, it is used for both the field width and precision. The default padding character is <c>' '</c> (space).</p> <p><c>Mod</c> is the control sequence modifier. It is either a - single character (currently only 't', for unicode translation, + single character (currently only <c>t</c>, for Unicode translation, is supported) that changes the interpretation of Data.</p> <p>The following control sequences are available:</p> @@ -397,9 +400,9 @@ ok</pre> 2> <input>io:fwrite("|~10.5c|~-10.5c|~5c|~n", [$a, $b, $c]).</input> | aaaaa|bbbbb |ccccc| ok</pre> - <p>If the Unicode translation modifier ('t') is in effect, + <p>If the Unicode translation modifier (<c>t</c>) is in effect, the integer argument can be any number representing a - valid unicode codepoint, otherwise it should be an integer + valid Unicode codepoint, otherwise it should be an integer less than or equal to 255, otherwise it is masked with 16#FF:</p> <pre> 1> <input>io:fwrite("~tc~n",[1024]).</input> @@ -439,7 +442,7 @@ ok</pre> <item> <p>Prints the argument with the <c>string</c> syntax. The argument is, if no Unicode translation modifier is present, an - iolist(), a binary, or an atom. If the Unicode translation modifier ('t') is in effect, the argument is unicode:chardata(), meaning that binaries are in UTF-8. The characters + iolist(), a binary, or an atom. If the Unicode translation modifier (<c>t</c>) is in effect, the argument is unicode:chardata(), meaning that binaries are in UTF-8. The characters are printed without quotes. The string is first truncated by the given precision and then padded and justified to the given field width. The default precision is the field width.</p> @@ -476,8 +479,10 @@ ok <p>Writes the data with standard syntax in the same way as <c>~w</c>, but breaks terms whose printed representation is longer than one line into many lines and indents each - line sensibly. It also tries to detect lists of printable - characters and to output these as strings. For example:</p> + line sensibly. It also tries to detect lists of + printable characters and to output these as strings. The + Unicode translation modifier is used for determining + what characters are printable. For example:</p> <pre> 5> <input>T = [{attributes,[[{id,age,1.50000},{mode,explicit},</input> <input>{typename,"INTEGER"}], [{id,cho},{mode,explicit},{typename,'Cho'}]]},</input> @@ -516,6 +521,19 @@ Here T = [{attributes,[[{id,age,1.5}, {tag,{'PRIVATE',3}}, {mode,implicit}] ok</pre> + <p>Binaries that look like UTF-8 encoded strings will be + output with the string syntax if the Unicode translation + modifier is given:</p> + <pre> +9> <input>io:fwrite("~p~n",[[1024]]).</input> +[1024] +10> <input>io:fwrite("~tp~n",[[1024]]).</input> +"\x{400}" +11> <input>io:fwrite("~tp~n", [<<128,128>>]).</input> +<<128,128>> +12> <input>io:fwrite("~tp~n", [<<208,128>>]).</input> +<<"\x{400}"/utf8>> +ok</pre> </item> <tag><c>W</c></tag> <item> @@ -583,7 +601,7 @@ ok</pre> <tag><c>#</c></tag> <item> <p>Like <c>B</c>, but prints the number with an Erlang style - '#'-separated base prefix.</p> + <c>#</c>-separated base prefix.</p> <pre> 16> <input>io:fwrite("~.10#~n", [31]).</input> 10#31 @@ -633,13 +651,14 @@ ok {shell,eval_loop,3}]} in function io:o_request/2</pre> <p>In this example, an attempt was made to output the single - character '65' with the aid of the string formatting directive + character 65 with the aid of the string formatting directive "~s".</p> </desc> </func> <func> <name name="fread" arity="2"/> <name name="fread" arity="3"/> + <type name="server_no_data"/> <fsummary>Read formatted input</fsummary> <desc> <p>Reads characters from the standard input (<c><anno>IoDevice</anno></c>), @@ -664,7 +683,7 @@ ok return suppression character. It provides a method to specify a field which is to be omitted. <c>F</c> is the <c>field width</c> of the input field, <c>M</c> is an optional - translation modifier (of which 't' is the only currently + translation modifier (of which <c>t</c> is the only currently supported, meaning Unicode translation) and <c>C</c> determines the type of control sequence.</p> @@ -690,8 +709,8 @@ ok <tag><c>-</c></tag> <item> <p>An optional sign character is expected. A sign - character '-' gives the return value <c>-1</c>. Sign - character '+' or none gives <c>1</c>. The field width + character <c>-</c> gives the return value <c>-1</c>. Sign + character <c>+</c> or none gives <c>1</c>. The field width parameter is ignored. Leading white-space characters are not skipped.</p> </item> @@ -713,7 +732,7 @@ ok characters are stripped. An Erlang string (list of characters) is returned.</p> - <p>If Unicode translation is in effect (~ts), + <p>If Unicode translation is in effect (<c>~ts</c>), characters larger than 255 are accepted, otherwise not. With the translation modifier, the list returned may as a consequence also contain @@ -769,10 +788,15 @@ Prompt> <input><Character beyond latin1 range not printable in this medium> <item> <p>End of file was encountered.</p> </item> - <tag><c>{error, <anno>What</anno>}</c></tag> + <tag><c>{error, <anno>FreadError</anno>}</c></tag> + <item> + <p>The reading failed and <c>FreadError</c> gives a + hint about the error.</p> + </item> + <tag><c>{error, <anno>ErrorDescription</anno>}</c></tag> <item> <p>The read operation failed and the parameter - <c><anno>What</anno></c> gives a hint about the error.</p> + <c><anno>ErrorDescription</anno></c> gives a hint about the error.</p> </item> </taglist> </item> @@ -793,11 +817,11 @@ enter><input>:</input> <input>alan</input> <input>:</input> <input>joe</in <func> <name name="rows" arity="0"/> <name name="rows" arity="1"/> - <fsummary>Get the number of rows of a device</fsummary> + <fsummary>Get the number of rows of an IO device</fsummary> <desc> <p>Retrieves the number of rows of the <c><anno>IoDevice</anno></c> (i.e. the height of a terminal). The function - only succeeds for terminal devices, for all other devices + only succeeds for terminal devices, for all other IO devices the function returns <c>{error, enotsup}</c></p> </desc> </func> @@ -805,23 +829,28 @@ enter><input>:</input> <input>alan</input> <input>:</input> <input>joe</in <name name="scan_erl_exprs" arity="1"/> <name name="scan_erl_exprs" arity="2"/> <name name="scan_erl_exprs" arity="3"/> + <name name="scan_erl_exprs" arity="4"/> + <type name="server_no_data"/> <fsummary>Read and tokenize Erlang expressions</fsummary> <desc> <p>Reads data from the standard input (<c>IoDevice</c>), - prompting it with <c>Prompt</c>. Reading starts at line number - <c>StartLine</c> (1). The data is tokenized as if it were a - sequence of Erlang expressions until a final <c>'.'</c> is + prompting it with <c>Prompt</c>. Reading starts at location + <c>StartLocation</c> (<c>1</c>). The argument <c><anno>Options</anno></c> + is passed on as the <c>Options</c> argument of the + <c>erl_scan:tokens/4</c> function. The data is tokenized as if + it were a + sequence of Erlang expressions until a final dot (<c>.</c>) is reached. This token is also returned. It returns:</p> <taglist> - <tag><c>{ok, Tokens, EndLine}</c></tag> + <tag><c>{ok, Tokens, EndLocation}</c></tag> <item> <p>The tokenization succeeded.</p> </item> - <tag><c>{eof, EndLine}</c></tag> + <tag><c>{eof, EndLocation}</c></tag> <item> <p>End of file was encountered.</p> </item> - <tag><c>{error, ErrorInfo, ErrorLine}</c></tag> + <tag><c>{error, ErrorInfo, ErrorLocation}</c></tag> <item> <p>An error occurred.</p> </item> @@ -840,13 +869,18 @@ enter><input>1.0er.</input> <name name="scan_erl_form" arity="1"/> <name name="scan_erl_form" arity="2"/> <name name="scan_erl_form" arity="3"/> + <name name="scan_erl_form" arity="4"/> + <type name="server_no_data"/> <fsummary>Read and tokenize an Erlang form</fsummary> <desc> <p>Reads data from the standard input (<c><anno>IoDevice</anno></c>), - prompting it with <c><anno>Prompt</anno></c>. Starts reading at line number - <c><anno>StartLine</anno></c> (1). The data is tokenized as if it were an + prompting it with <c><anno>Prompt</anno></c>. Starts reading + at location <c><anno>StartLocation</anno></c> (<c>1</c>). The + argument <c><anno>Options</anno></c> is passed on as the + <c>Options</c> argument of the <c>erl_scan:tokens/4</c> + function. The data is tokenized as if it were an Erlang form - one of the valid Erlang expressions in an - Erlang source file - until a final <c>'.'</c> is reached. + Erlang source file - until a final dot (<c>.</c>) is reached. This last token is also returned. The return values are the same as for <c>scan_erl_exprs/1,2,3</c> above.</p> </desc> @@ -855,24 +889,30 @@ enter><input>1.0er.</input> <name name="parse_erl_exprs" arity="1"/> <name name="parse_erl_exprs" arity="2"/> <name name="parse_erl_exprs" arity="3"/> + <name name="parse_erl_exprs" arity="4"/> <type name="parse_ret"/> + <type name="server_no_data"/> <fsummary>Read, tokenize and parse Erlang expressions</fsummary> <desc> - <p>Reads data from the standard input (<c><anno>IoDevice</anno></c>), - prompting it with <c><anno>Prompt</anno></c>. Starts reading at line number - <c><anno>StartLine</anno></c> (1). The data is tokenized and parsed as if - it were a sequence of Erlang expressions until a final '.' is - reached. It returns:</p> + <p>Reads data from the standard input + (<c><anno>IoDevice</anno></c>), prompting it with + <c><anno>Prompt</anno></c>. Starts reading at location + <c><anno>StartLocation</anno></c> (<c>1</c>). The argument + <c><anno>Options</anno></c> is passed on as the + <c>Options</c> argument of the <c>erl_scan:tokens/4</c> + function. The data is tokenized and parsed as if it were a + sequence of Erlang expressions until a final dot (<c>.</c>) is reached. + It returns:</p> <taglist> - <tag><c>{ok, ExprList, EndLine}</c></tag> + <tag><c>{ok, ExprList, EndLocation}</c></tag> <item> <p>The parsing was successful.</p> </item> - <tag><c>{eof, EndLine}</c></tag> + <tag><c>{eof, EndLocation}</c></tag> <item> <p>End of file was encountered.</p> </item> - <tag><c>{error, ErrorInfo, ErrorLine}</c></tag> + <tag><c>{error, ErrorInfo, ErrorLocation}</c></tag> <item> <p>An error occurred.</p> </item> @@ -891,25 +931,30 @@ enter><input>abc("hey".</input> <name name="parse_erl_form" arity="1"/> <name name="parse_erl_form" arity="2"/> <name name="parse_erl_form" arity="3"/> + <name name="parse_erl_form" arity="4"/> <type name="parse_form_ret"/> + <type name="server_no_data"/> <fsummary>Read, tokenize and parse an Erlang form</fsummary> <desc> <p>Reads data from the standard input (<c><anno>IoDevice</anno></c>), - prompting it with <c><anno>Prompt</anno></c>. Starts reading at line number - <c><anno>StartLine</anno></c> (1). The data is tokenized and parsed as if + prompting it with <c><anno>Prompt</anno></c>. Starts reading at + location <c><anno>StartLocation</anno></c> (<c>1</c>). The argument + <c><anno>Options</anno></c> is passed on as the + <c>Options</c> argument of the <c>erl_scan:tokens/4</c> + function. The data is tokenized and parsed as if it were an Erlang form - one of the valid Erlang expressions - in an Erlang source file - until a final '.' is reached. It + in an Erlang source file - until a final dot (<c>.</c>) is reached. It returns:</p> <taglist> - <tag><c>{ok, AbsForm, EndLine}</c></tag> + <tag><c>{ok, AbsForm, EndLocation}</c></tag> <item> <p>The parsing was successful.</p> </item> - <tag><c>{eof, EndLine}</c></tag> + <tag><c>{eof, EndLocation}</c></tag> <item> <p>End of file was encountered.</p> </item> - <tag><c>{error, ErrorInfo, ErrorLine}</c></tag> + <tag><c>{error, ErrorInfo, ErrorLocation}</c></tag> <item> <p>An error occurred.</p> </item> @@ -940,7 +985,7 @@ enter><input>bar.</input> </section> <section> <title>Standard Error</title> - <p>In certain situations, especially when the standard output is redirected, access to an io_server() specific for error messages might be convenient. The io_device 'standard_error' can be used to direct output to whatever the current operating system considers a suitable device for error output. Example on a Unix-like operating system:</p> + <p>In certain situations, especially when the standard output is redirected, access to an I/O-server specific for error messages might be convenient. The IO device <c>standard_error</c> can be used to direct output to whatever the current operating system considers a suitable IO device for error output. Example on a Unix-like operating system:</p> <pre> $ <input>erl -noshell -noinput -eval 'io:format(standard_error,"Error: ~s~n",["error 11"]),'\</input> <input>'init:stop().' > /dev/null</input> @@ -956,7 +1001,7 @@ Error: error 11</pre> <c>ErrorInfo</c> structure which is returned from all IO modules. It has the format:</p> <code type="none"> -{ErrorLine, Module, ErrorDescriptor}</code> +{ErrorLocation, Module, ErrorDescriptor}</code> <p>A string which describes the error is obtained with the following call:</p> <code type="none"> diff --git a/lib/stdlib/doc/src/io_lib.xml b/lib/stdlib/doc/src/io_lib.xml index 506c1792f1..617a6b74fc 100644 --- a/lib/stdlib/doc/src/io_lib.xml +++ b/lib/stdlib/doc/src/io_lib.xml @@ -4,7 +4,7 @@ <erlref> <header> <copyright> - <year>1996</year><year>2011</year> + <year>1996</year><year>2012</year> <holder>Ericsson AB. All Rights Reserved.</holder> </copyright> <legalnotice> @@ -43,6 +43,12 @@ <name name="chars"/> </datatype> <datatype> + <name name="unicode_chars"/> + </datatype> + <datatype> + <name name="unicode_string"/> + </datatype> + <datatype> <name name="continuation"/> <desc><p>A continuation as returned by <seealso marker="#fread/3"><c>fread/3</c></seealso>.</p> </desc> @@ -50,6 +56,9 @@ <datatype> <name name="depth"/> </datatype> + <datatype> + <name name="fread_error"/> + </datatype> </datatypes> <funcs> <func> @@ -209,6 +218,23 @@ </desc> </func> <func> + <name name="write_unicode_string" arity="1"/> + <fsummary>Write a Unicode string</fsummary> + <desc> + <p>Returns the list of characters needed to print + <c><anno>UnicodeString</anno></c> as a string.</p> + </desc> + </func> + <func> + <name name="write_unicode_string_as_latin1" arity="1"/> + <fsummary>Write a Unicode string</fsummary> + <desc> + <p>Returns the list of characters needed to print + <c><anno>UnicodeString</anno></c> as a string. Non-Latin-1 + characters are escaped.</p> + </desc> + </func> + <func> <name name="write_char" arity="1"/> <fsummary>Write a character</fsummary> <desc> @@ -217,6 +243,23 @@ </desc> </func> <func> + <name name="write_unicode_char" arity="1"/> + <fsummary>Write a Unicode character</fsummary> + <desc> + <p>Returns the list of characters needed to print a character + constant in the Unicode character set.</p> + </desc> + </func> + <func> + <name name="write_unicode_char_as_latin1" arity="1"/> + <fsummary>Write a Unicode character</fsummary> + <desc> + <p>Returns the list of characters needed to print a character + constant in the Unicode character set. Non-Latin-1 characters + are escaped.</p> + </desc> + </func> + <func> <name name="indentation" arity="2"/> <fsummary>Indentation after printing string</fsummary> <desc> @@ -233,6 +276,14 @@ </desc> </func> <func> + <name name="unicode_char_list" arity="1"/> + <fsummary>Test for a list of Unicode characters</fsummary> + <desc> + <p>Returns <c>true</c> if <c><anno>Term</anno></c> is a flat list of + characters in the Unicode range, otherwise it returns <c>false</c>.</p> + </desc> + </func> + <func> <name name="deep_char_list" arity="1"/> <fsummary>Test for a deep list of characters</fsummary> <desc> @@ -241,6 +292,14 @@ </desc> </func> <func> + <name name="deep_unicode_char_list" arity="1"/> + <fsummary>Test for a deep list of Unicode characters</fsummary> + <desc> + <p>Returns <c>true</c> if <c><anno>Term</anno></c> is a, possibly deep, list + of characters in the Unicode range, otherwise it returns <c>false</c>.</p> + </desc> + </func> + <func> <name name="printable_list" arity="1"/> <fsummary>Test for a list of printable ISO-latin-1 characters</fsummary> <desc> @@ -248,6 +307,14 @@ printable ISO-latin-1 characters, otherwise it returns <c>false</c>.</p> </desc> </func> + <func> + <name name="printable_unicode_list" arity="1"/> + <fsummary>Test for a list of printable Unicode characters</fsummary> + <desc> + <p>Returns <c>true</c> if <c><anno>Term</anno></c> is a flat list of + printable Unicode characters, otherwise it returns <c>false</c>.</p> + </desc> + </func> </funcs> </erlref> diff --git a/lib/stdlib/doc/src/io_protocol.xml b/lib/stdlib/doc/src/io_protocol.xml index 0ff3d5c1ee..d36bf2042f 100644 --- a/lib/stdlib/doc/src/io_protocol.xml +++ b/lib/stdlib/doc/src/io_protocol.xml @@ -5,7 +5,7 @@ <header> <copyright> <year>1999</year> - <year>2011</year> + <year>2012</year> <holder>Ericsson AB. All Rights Reserved.</holder> </copyright> <legalnotice> @@ -35,10 +35,10 @@ <p>The I/O-protocol in Erlang specifies a way for a client to communicate -with an io_server and vice versa. The io_server is a process handling the -requests and that performs the requested task on i.e. a device. The +with an I/O server and vice versa. The I/O server is a process that handles +the requests and performs the requested task on e.g. an IO device. The client is any Erlang process wishing to read or write data from/to the -device.</p> +IO device.</p> <p>The common I/O-protocol has been present in OTP since the beginning, but has been fairly undocumented and has also somewhat @@ -53,85 +53,85 @@ implement than the original. It can certainly be argumented that the current protocol is too complex, but this text describes how it looks today, not how it should have looked.</p> -<p>The basic ideas from the original protocol still hold. The io_server +<p>The basic ideas from the original protocol still hold. The I/O server and client communicate with one single, rather simplistic protocol and -no server state is ever present in the client. Any io_server can be +no server state is ever present in the client. Any I/O server can be used together with any client code and client code need not be aware -of the actual device the io_server communicates with.</p> +of the actual IO device the I/O server communicates with.</p> <section> -<title>Protocol basics</title> +<title>Protocol Basics</title> -<p>As described in Robert's paper, servers and clients communicate using -io_request/io_reply tuples as follows:</p> +<p>As described in Robert's paper, I/O servers and clients communicate using +<c>io_request</c>/<c>io_reply</c> tuples as follows:</p> <p><em>{io_request, From, ReplyAs, Request}</em><br/> <em>{io_reply, ReplyAs, Reply}</em></p> -<p>The client sends an io_request to the io_server and the server -eventually sends a corresponding reply.</p> +<p>The client sends an <c>io_request</c> tuple to the I/O server and +the server eventually sends a corresponding <c>io_reply</c> tuple.</p> <list type="bulleted"> -<item>From is the pid() of the client, the process which the io_server -sends the reply to.</item> - -<item>ReplyAs can be any datum and is simply returned in the corresponding -io_reply. The io-module in the Erlang standard library simply uses the pid() -of the io_server as the ReplyAs datum, but a more complicated client -could have several outstanding io-requests to the same server and -would then use i.e. a reference() or something else to differentiate among -the incoming io_reply's. The ReplyAs element should be considered -opaque by the io_server. Note that the pid() of the server is not -explicitly present in the io_reply. The reply can be sent from any -process, not necessarily the actual io_server. The ReplyAs element is -the only thing that connects one io_request with an io_reply.</item> - -<item>Request and Reply are described below.</item> +<item><c>From</c> is the <c>pid()</c> of the client, the process which +the I/O server sends the IO reply to.</item> + +<item><c>ReplyAs</c> can be any datum and is returned in the corresponding +<c>io_reply</c>. The <seealso marker="stdlib:io">io</seealso> module simply uses the pid() +of the I/O server as the <c>ReplyAs</c> datum, but a more complicated client +could have several outstanding I/O requests to the same I/O server and +would then use i.e. a <c>reference()</c> or something else to differentiate among +the incoming IO replies. The <c>ReplyAs</c> element should be considered +opaque by the I/O server. Note that the <c>pid()</c> of the I/O server is not +explicitly present in the <c>io_reply</c> tuple. The reply can be sent from any +process, not necessarily the actual I/O server. The <c>ReplyAs</c> element is +the only thing that connects one I/O request with an I/O-reply.</item> + +<item><c>Request</c> and <c>Reply</c> are described below.</item> </list> -<p>When an io_server receives an io_request, it acts upon the actual -Request part and eventually sends an io_reply with the corresponding -Reply part.</p> +<p>When an I/O server receives an <c>io_request</c> tuple, it acts upon the actual +<c>Request</c> part and eventually sends an <c>io_reply</c> tuple with the corresponding +<c>Reply</c> part.</p> </section> <section> -<title>Output requests</title> +<title>Output Requests</title> -<p>To output characters on a device, the following Requests exist:</p> +<p>To output characters on an IO device, the following <c>Request</c>s exist:</p> <p> <em>{put_chars, Encoding, Characters}</em><br/> <em>{put_chars, Encoding, Module, Function, Args}</em> </p> <list type="bulleted"> -<item>Encoding is either 'unicode' or 'latin1', meaning that the +<item><c>Encoding</c> is either <c>unicode</c> or <c>latin1</c>, meaning that the characters are (in case of binaries) encoded as either UTF-8 or - iso-latin-1 (pure bytes). A well behaved io_server should also - return error if list elements contain integers > 255 when the - Encoding is set to latin1. Note that this does not in any way tell - how characters should be put on the actual device or how the - io_server should handle them. Different io_servers may handle the - characters however they want, this simply tells the io_server which - format the data is expected to have. In the Module/Function/argument - case, the Encoding tells which format the designated function - produces. Note that byte-oriented data is simplest sent using latin1 - Encoding</item> - -<item>Characters are the data to be put on the device. If Encoding is - latin1, this is an iolist(). If Encoding is unicode, this is an - Erlang standard mixed unicode list (one integer in a list per + ISO-latin-1 (pure bytes). A well behaved I/O server should also + return error if list elements contain integers > 255 when + <c>Encoding</c> is set to <c>latin1</c>. Note that this does not in any way tell + how characters should be put on the actual IO device or how the + I/O server should handle them. Different I/O servers may handle the + characters however they want, this simply tells the I/O server which + format the data is expected to have. In the <c>Module</c>/<c>Function</c>/<c>Args</c> + case, <c>Encoding</c> tells which format the designated function + produces. Note that byte-oriented data is simplest sent using the ISO-latin-1 + encoding.</item> + +<item>Characters are the data to be put on the IO device. If <c>Encoding</c> is + <c>latin1</c>, this is an <c>iolist()</c>. If <c>Encoding</c> is <c>unicode</c>, this is an + Erlang standard mixed Unicode list (one integer in a list per character, characters in binaries represented as UTF-8).</item> -<item>Module, Function, Args denotes a function which will be called to - produce the data (like io_lib:format). Args is a list of arguments +<item><c>Module</c>, <c>Function</c>, and <c>Args</c> denote a function which will be called to + produce the data (like <c>io_lib:format/2</c>). <c>Args</c> is a list of arguments to the function. The function should produce data in the given - Encoding. The io_server should call the function as apply(Mod, Func, - Args) and will put the returned data on the device as if it was sent - in a {put_chars, Encoding, Characters} request. If the function + <c>Encoding</c>. The I/O server should call the function as + <c>apply(Mod, Func, Args)</c> and will put the returned data on the IO device as if it was sent + in a <c>{put_chars, Encoding, Characters}</c> request. If the function returns anything else than a binary or list or throws an exception, an error should be sent back to the client.</item> </list> -<p>The server replies to the client with an io_reply where the Reply +<p>The I/O server replies to the client with an <c>io_reply</c> tuple where the <c>Reply</c> element is one of:</p> <p> <em>ok</em><br/> @@ -139,49 +139,50 @@ element is one of:</p> </p> <list type="bulleted"> -<item>Error describes the error to the client, which may do whatever it - wants with it. The Erlang io-module typically returns it as is.</item> +<item><c>Error</c> describes the error to the client, which may do whatever + it wants with it. The Erlang <seealso marker="stdlib:io">io</seealso> + module typically returns it as is.</item> </list> -<p>For backward compatibility the following Requests should also be -handled by an io_server (these messages should not be present after +<p>For backward compatibility the following <c>Request</c>s should also be +handled by an I/O server (these requests should not be present after R15B of OTP):</p> <p> <em>{put_chars, Characters}</em><br/> <em>{put_chars, Module, Function, Args}</em> </p> -<p>These should behave as {put_chars, latin1, Characters} and {put_chars, -latin1, Module, Function, Args} respectively. </p> +<p>These should behave as <c>{put_chars, latin1, Characters}</c> and +<c>{put_chars, latin1, Module, Function, Args}</c> respectively. </p> </section> <section> <title>Input Requests</title> -<p>To read characters from a device, the following Requests exist:</p> +<p>To read characters from an IO device, the following <c>Request</c>s exist:</p> <p><em>{get_until, Encoding, Prompt, Module, Function, ExtraArgs}</em></p> <list type="bulleted"> -<item>Encoding denotes how data is to be sent back to the client and +<item><c>Encoding</c> denotes how data is to be sent back to the client and what data is sent to the function denoted by - Module/Function/ExtraArgs. If the function supplied returns data as a + <c>Module</c>/<c>Function</c>/<c>ExtraArgs</c>. If the function supplied returns data as a list, the data is converted to this encoding. If however the function supplied returns data in some other format, no conversion - can be done and it's up to the client supplied function to return - data in a proper way. If Encoding is latin1, lists of integers + can be done and it is up to the client supplied function to return + data in a proper way. If <c>Encoding</c> is <c>latin1</c>, lists of integers 0..255 or binaries containing plain bytes are sent back to the - client when possible, if Encoding is unicode, lists with integers in - the whole unicode range or binaries encoded in UTF-8 are sent to the + client when possible; if <c>Encoding</c> is <c>unicode</c>, lists with integers in + the whole Unicode range or binaries encoded in UTF-8 are sent to the client. The user supplied function will always see lists of integers, never - binaries, but the list may contain numbers > 255 if the Encoding is - 'unicode'.</item> + binaries, but the list may contain numbers > 255 if the <c>Encoding</c> is + <c>unicode</c>.</item> -<item>Prompt is a list of characters (not mixed, no binaries) or an atom() - to be output as a prompt for input on the device. The Prompt is - often ignored by the io_server and a Prompt set to '' should always - be ignored (and result in nothing being written to the device).</item> +<item><c>Prompt</c> is a list of characters (not mixed, no binaries) or an atom + to be output as a prompt for input on the IO device. <c>Prompt</c> is + often ignored by the I/O server and if set to <c>''</c> it should always + be ignored (and result in nothing being written to the IO device).</item> -<item><p>Module, Function, ExtraArgs denotes a function and arguments to +<item><p><c>Module</c>, <c>Function</c>, and <c>ExtraArgs</c> denote a function and arguments to determine when enough data is written. The function should take two additional arguments, the last state, and a list of characters. The function should return one of:</p> @@ -189,23 +190,23 @@ latin1, Module, Function, Args} respectively. </p> <em>{done, Result, RestChars}</em><br/> <em>{more, Continuation}</em> </p> - <p>The Result can be any Erlang term, but if it is a list(), the - io_server may convert it to a binary() of appropriate format before - returning it to the client, if the server is set in binary mode (see + <p>The <c>Result</c> can be any Erlang term, but if it is a <c>list()</c>, the + I/O server may convert it to a <c>binary()</c> of appropriate format before + returning it to the client, if the I/O server is set in binary mode (see below).</p> - <p>The function will be called with the data the io_server finds on - its device, returning {done, Result, RestChars} when enough data is - read (in which case Result is sent to the client and RestChars are - kept in the io_server as a buffer for subsequent input) or {more, - Continuation}, indicating that more characters are needed to - complete the request. The Continuation will be sent as the state in + <p>The function will be called with the data the I/O server finds on + its IO device, returning <c>{done, Result, RestChars}</c> when enough data is + read (in which case <c>Result</c> is sent to the client and <c>RestChars</c> is + kept in the I/O server as a buffer for subsequent input) or + <c>{more, Continuation}</c>, indicating that more characters are needed to + complete the request. The <c>Continuation</c> will be sent as the state in subsequent calls to the function when more characters are available. When no more characters are available, the function - shall return {done,eof,Rest}. + shall return <c>{done, eof, Rest}</c>. The initial state is the empty list and the data when an - end of file is reached on the device is the atom 'eof'. An emulation - of the get_line request could be (inefficiently) implemented using + end of file is reached on the IO device is the atom <c>eof</c>. An emulation + of the <c>get_line</c> request could be (inefficiently) implemented using the following functions:</p> <code> -module(demo). @@ -214,7 +215,9 @@ latin1, Module, Function, Args} respectively. </p> until_newline(_ThisFar,eof,_MyStopCharacter) -> {done,eof,[]}; until_newline(ThisFar,CharList,MyStopCharacter) -> - case lists:splitwith(fun(X) -> X =/= MyStopCharacter end, CharList) of + case + lists:splitwith(fun(X) -> X =/= MyStopCharacter end, CharList) + of {L,[]} -> {more,ThisFar++L}; {L2,[MyStopCharacter|Rest]} -> @@ -222,45 +225,47 @@ until_newline(ThisFar,CharList,MyStopCharacter) -> end. get_line(IoServer) -> - IoServer ! {io_request, self(), IoServer, {get_until, unicode, '', - ?MODULE, until_newline, [$\n]}}, + IoServer ! {io_request, + self(), + IoServer, + {get_until, unicode, '', ?MODULE, until_newline, [$\n]}}, receive {io_reply, IoServer, Data} -> Data end. </code> - <p>Note especially that the last element in the Request tuple ([$\n]) + <p>Note especially that the last element in the <c>Request</c> tuple (<c>[$\n]</c>) is appended to the argument list when the function is called. The function should be called like - apply(Module, Function, [ State, Data | ExtraArgs ]) by the io_server</p> + <c>apply(Module, Function, [ State, Data | ExtraArgs ])</c> by the I/O server</p> </item> </list> -<p>A defined number of characters is requested using this Request:</p> +<p>A fixed number of characters is requested using this <c>Request</c>:</p> <p> <em>{get_chars, Encoding, Prompt, N}</em> </p> <list type="bulleted"> -<item>Encoding and Prompt as for get_until.</item> +<item><c>Encoding</c> and <c>Prompt</c> as for <c>get_until</c>.</item> -<item>N is the number of characters to be read from the device.</item> +<item><c>N</c> is the number of characters to be read from the IO device.</item> </list> -<p>A single line (like in the example above) is requested with this Request:</p> +<p>A single line (like in the example above) is requested with this <c>Request</c>:</p> <p> <em>{get_line, Encoding, Prompt}</em> </p> <list type="bulleted"> -<item>Encoding and prompt as above.</item> +<item><c>Encoding</c> and <c>Prompt</c> as above.</item> </list> -<p>Obviously, get_chars and get_line could be implemented with the -get_until request (and indeed was originally), but demands for +<p>Obviously, the <c>get_chars</c> and <c>get_line</c> could be implemented with the +<c>get_until</c> request (and indeed they were originally), but demands for efficiency has made these additions necessary.</p> -<p>The server replies to the client with an io_reply where the Reply +<p>The I/O server replies to the client with an <c>io_reply</c> tuple where the <c>Reply</c> element is one of:</p> <p> <em>Data</em><br/> @@ -269,16 +274,17 @@ element is one of:</p> </p> <list type="bulleted"> -<item>Data is the characters read, in either list or binary form - (depending on the io_server mode, see below).</item> -<item>Error describes the error to the client, which may do whatever it - wants with it. The Erlang io-module typically returns it as is.</item> -<item>eof is returned when input end is reached and no more data is +<item><c>Data</c> is the characters read, in either list or binary form + (depending on the I/O server mode, see below).</item> +<item><c>Error</c> describes the error to the client, which may do whatever it + wants with it. The Erlang <seealso marker="stdlib:io">io</seealso> + module typically returns it as is.</item> +<item><c>eof</c> is returned when input end is reached and no more data is available to the client process.</item> </list> -<p>For backward compatibility the following Requests should also be -handled by an io_server (these messages should not be present after +<p>For backward compatibility the following <c>Request</c>s should also be +handled by an I/O server (these reqeusts should not be present after R15B of OTP):</p> <p> @@ -287,30 +293,30 @@ R15B of OTP):</p> <em>{get_line, Prompt}</em><br/> </p> -<p>These should behave as {get_until, latin1, Prompt, Module, Function, -ExtraArgs}, {get_chars, latin1, Prompt, N} and {get_line, latin1, -Prompt} respectively.</p> +<p>These should behave as <c>{get_until, latin1, Prompt, Module, Function, +ExtraArgs}</c>, <c>{get_chars, latin1, Prompt, N}</c> and <c>{get_line, latin1, +Prompt}</c> respectively.</p> </section> <section> -<title>I/O-server modes</title> - -<p>Demands for efficiency when reading data from an io_server has not -only lead to the addition of the get_line and get_chars requests, but -has also added the concept of io_server options. No options are -mandatory to implement, but all io_servers in the Erlang standard -libraries honor the 'binary' option, which allows the Data in the -io_reply to be binary instead of in list form <em>when possible</em>. +<title>I/O-server Modes</title> + +<p>Demands for efficiency when reading data from an I/O server has not +only lead to the addition of the <c>get_line</c> and <c>get_chars</c> requests, but +has also added the concept of I/O server options. No options are +mandatory to implement, but all I/O servers in the Erlang standard +libraries honor the <c>binary</c> option, which allows the <c>Data</c> element of the +<c>io_reply</c> tuple to be a binary instead of a list <em>when possible</em>. If the data is sent as a binary, Unicode data will be sent in the -standard Erlang unicode -format, i.e. UTF-8 (note that the function in get_until still gets -list data regardless of the io_server mode).</p> +standard Erlang Unicode +format, i.e. UTF-8 (note that the function of the <c>get_until</c> request still gets +list data regardless of the I/O server mode).</p> -<p>Note that i.e. the <c>get_until</c> request allows for a function with the data specified as always being a list. Also the return value data from such a function can be of any type (as is indeed the case when an io:fread request is sent to an io_server). The client has to be prepared for data received as answers to those requests to be in a variety of forms, but the server should convert the results to binaries whenever possible (i.e. when the function supplied to get_until actually returns a list). The example shown later in this text does just that.</p> +<p>Note that i.e. the <c>get_until</c> request allows for a function with the data specified as always being a list. Also the return value data from such a function can be of any type (as is indeed the case when an <c>io:fread</c> request is sent to an I/O server). The client has to be prepared for data received as answers to those requests to be in a variety of forms, but the I/O server should convert the results to binaries whenever possible (i.e. when the function supplied to <c>get_until</c> actually returns a list). The example shown later in this text does just that.</p> <p>An I/O-server in binary mode will affect the data sent to the client, so that it has to be able to handle binary data. For convenience, it -is possible to set and retrieve the modes of an io_server using the -following I/O-requests:</p> +is possible to set and retrieve the modes of an I/O server using the +following I/O requests:</p> <p> <em>{setopts, Opts}</em> @@ -318,72 +324,72 @@ following I/O-requests:</p> <list type="bulleted"> -<item>Opts is a list of options in the format recognized by proplists (and - of course by the io_server itself).</item> +<item><c>Opts</c> is a list of options in the format recognized by <seealso marker="stdlib:proplists">proplists</seealso> (and + of course by the I/O server itself).</item> </list> -<p>As an example, the io_server for the interactive shell (in group.erl) +<p>As an example, the I/O server for the interactive shell (in <c>group.erl</c>) understands the following options:</p> <p> -<em>{binary, bool()} (or 'binary'/'list')</em><br/> -<em>{echo, bool()}</em><br/> +<em>{binary, boolean()}</em> (or <em>binary</em>/<em>list</em>)<br/> +<em>{echo, boolean()}</em><br/> <em>{expand_fun, fun()}</em><br/> -<em>{encoding, 'unicode'/'latin1'} (or 'unicode'/'latin1')</em> +<em>{encoding, unicode/latin1}</em> (or <em>unicode</em>/<em>latin1</em>) </p> -<p>- of which the 'binary' and 'encoding' options are common for all -io_servers in OTP, while 'echo' and 'expand' is valid only for this -io_server. It's worth noting that the 'unicode' option notifies how -characters are actually put on the physical device, i.e. if the -terminal per se is unicode aware, it does not affect how characters +<p>- of which the <c>binary</c> and <c>encoding</c> options are common for all +I/O servers in OTP, while <c>echo</c> and <c>expand</c> are valid only for this +I/O server. It is worth noting that the <c>unicode</c> option notifies how +characters are actually put on the physical IO device, i.e. if the +terminal per se is Unicode aware, it does not affect how characters are sent in the I/O-protocol, where each request contains encoding information for the provided or returned data.</p> -<p>The server should send one of the following as Reply:</p> +<p>The I/O server should send one of the following as <c>Reply</c>:</p> <p> <em>ok</em><br/> <em>{error, Error}</em> </p> -<p>An error (preferably enotsup) is to be expected if the option is -not supported by the io_server (like if an 'echo' option is sent in a -setopt Request to a plain file).</p> +<p>An error (preferably <c>enotsup</c>) is to be expected if the option is +not supported by the I/O server (like if an <c>echo</c> option is sent in a +<c>setopts</c> request to a plain file).</p> -<p>To retrieve options, this message is used:</p> +<p>To retrieve options, this request is used:</p> <p> <em>getopts</em> </p> -<p>The 'getopts' message requests a complete list of all options -supported by the io_server as well as their current values.</p> +<p>The <c>getopts</c> request asks for a complete list of all options +supported by the I/O server as well as their current values.</p> -<p>The server replies:</p> +<p>The I/O server replies:</p> <p> <em>OptList</em><br/> -<em>{error,Error}</em> +<em>{error, Error}</em> </p> <list type="bulleted"> -<item>OptList is a list of tuples {Option, Value} where Option is always +<item><c>OptList</c> is a list of tuples <c>{Option, Value}</c> where <c>Option</c> is always an atom.</item> </list> </section> <section> -<title>Multiple I/O requests</title> +<title>Multiple I/O Requests</title> -<p>The Request element can in itself contain several Requests by using +<p>The <c>Request</c> element can in itself contain several <c>Request</c>s by using the following format:</p> <p> <em>{requests, Requests}</em> </p> <list type="bulleted"> -<item>Requests is a list of valid Request tuples for the protocol, they +<item><c>Requests</c> is a list of valid <c>io_request</c> tuples for the protocol, they shall be executed in the order in which they appear in the list and the execution should continue until one of the requests result in an error or the list is consumed. The result of the last request is sent back to the client.</item> </list> -<p>The server can for a list of requests send any of the valid results in +<p>The I/O server can for a list of requests send any of the valid results in the reply:</p> <p> @@ -395,7 +401,7 @@ the reply:</p> <p>- depending on the actual requests in the list.</p> </section> <section> -<title>Optional I/O-requests</title> +<title>Optional I/O Requests</title> <p>The following I/O request is optional to implement and a client should be prepared for an error return:</p> @@ -403,47 +409,47 @@ should be prepared for an error return:</p> <em>{get_geometry, Geometry}</em> </p> <list type="bulleted"> -<item>Geometry is either the atom 'rows' or the atom 'columns'.</item> +<item><c>Geometry</c> is either the atom <c>rows</c> or the atom <c>columns</c>.</item> </list> -<p>The server should send the Reply as:</p> +<p>The I/O server should send the <c>Reply</c> as:</p> <p> <em>{ok, N}</em><br/> <em>{error, Error}</em> </p> <list type="bulleted"> -<item>N is the number of character rows or columns the device has, if - applicable to the device the io_server handles, otherwise {error, - enotsup} is a good answer.</item> +<item><c>N</c> is the number of character rows or columns the IO device has, if + applicable to the IO device the I/O server handles, otherwise <c>{error, + enotsup}</c> is a good answer.</item> </list> </section> <section> -<title>Unimplemented request types:</title> +<title>Unimplemented Request Types</title> -<p>If an io_server encounters a request it does not recognize (i.e. the -io_request tuple is in the expected format, but the actual Request is -unknown), the server should send a valid reply with the error tuple:</p> +<p>If an I/O server encounters a request it does not recognize (i.e. the +<c>io_request</c> tuple is in the expected format, but the actual <c>Request</c> is +unknown), the I/O server should send a valid reply with the error tuple:</p> <p> <em>{error, request}</em> </p> -<p>This makes it possible to extend the protocol with optional messages +<p>This makes it possible to extend the protocol with optional requests and for the clients to be somewhat backwards compatible.</p> </section> <section> -<title>An annotated and working example io_server:</title> +<title>An Annotated and Working Example I/O Server</title> -<p>An io_server is any process capable of handling the protocol. There is -no generic io_server behavior, but could well be. The framework is +<p>An I/O server is any process capable of handling the I/O protocol. There is +no generic I/O server behavior, but could well be. The framework is simple enough, a process handling incoming requests, usually both -io_requests and other device-specific requests (for i.e. positioning , +I/O-requests and other IO device-specific requests (for i.e. positioning, closing etc.).</p> -<p>Our example io_server stores characters in an ets table, making up a +<p>Our example I/O server stores characters in an ETS table, making up a fairly crude ram-file (it is probably not useful, but working).</p> <p>The module begins with the usual directives, a function to start the -server and a main loop handling the requests:</p> +I/O server and a main loop handling the requests:</p> <code> -module(ets_io_server). @@ -486,18 +492,19 @@ loop(State) -> </code> <p>The main loop receives messages from the client (which might be using -the io-module to send requests). For each request the function -request/2 is called and a reply is eventually sent using the reply/3 +the <seealso marker="stdlib:io">io</seealso> module to send requests). +For each request the function +<c>request/2</c> is called and a reply is eventually sent using the <c>reply/3</c> function.</p> -<p>The "private" message {From, rewind} results in the +<p>The "private" message <c>{From, rewind}</c> results in the current position in the pseudo-file to be reset to 0 (the beginning of -the "file"). This is a typical example of device-specific +the "file"). This is a typical example of IO device-specific messages not being part of the I/O-protocol. It is usually a bad idea -to embed such private messages in io_request tuples, as that might be +to embed such private messages in <c>io_request</c> tuples, as that might be confusing to the reader.</p> -<p>Let's look at the reply function first...</p> +<p>Let us look at the reply function first...</p> <code> @@ -506,8 +513,8 @@ reply(From, ReplyAs, Reply) -> </code> -<p>Simple enough, it sends the io_reply tuple back to the client, -providing the ReplyAs element received in the request along with the +<p>Simple enough, it sends the <c>io_reply</c> tuple back to the client, +providing the <c>ReplyAs</c> element received in the request along with the result of the request, as described above.</p> <p>Now look at the different requests we need to handle. First the @@ -525,18 +532,18 @@ request({put_chars, Encoding, Module, Function, Args}, State) -> end; </code> -<p>The Encoding tells us how the characters in the message are +<p>The <c>Encoding</c> tells us how the characters in the request are represented. We want to store the characters as lists in the -ets-table, so we convert them to lists using the -unicode:characters_to_list/2 function. The conversion function -conveniently accepts the encoding types unicode or latin1, so we can -use the Encoding parameter directly.</p> +ETS table, so we convert them to lists using the +<seealso marker="stdlib:unicode#characters_to_list/2"><c>unicode:characters_to_list/2</c></seealso> function. The conversion function +conveniently accepts the encoding types <c>unicode</c> or <c>latin1</c>, so we can +use <c>Encoding</c> directly.</p> -<p>When Module, Function and Arguments are provided, we simply apply it +<p>When <c>Module</c>, <c>Function</c> and <c>Arguments</c> are provided, we simply apply it and do the same thing with the result as if the data was provided directly.</p> -<p>Let's handle the requests for retrieving data too:</p> +<p>Let us handle the requests for retrieving data too:</p> <code> request({get_until, Encoding, _Prompt, M, F, As}, State) -> @@ -550,11 +557,11 @@ request({get_line, Encoding, _Prompt}, State) -> </code> <p>Here we have cheated a little by more or less only implementing -get_until and using internal helpers to implement get_chars and -get_line. In production code, this might be to inefficient, but that +<c>get_until</c> and using internal helpers to implement <c>get_chars</c> and +<c>get_line</c>. In production code, this might be too inefficient, but that of course depends on the frequency of the different requests. Before -we start actually implementing the functions put_chars/2 and -get_until/5, lets look into the few remaining requests:</p> +we start actually implementing the functions <c>put_chars/2</c> and +<c>get_until/5</c>, let us look into the few remaining requests:</p> <code> request({get_geometry,_}, State) -> @@ -567,18 +574,18 @@ request({requests, Reqs}, State) -> multi_request(Reqs, {ok, ok, State}); </code> -<p>The get_geometry request has no meaning for this io_server, so the -reply will be {error, enotsup}. The only option we handle is the -binary/list option, which is done in separate functions.</p> +<p>The <c>get_geometry</c> request has no meaning for this I/O server, so the +reply will be <c>{error, enotsup}</c>. The only option we handle is the +<c>binary</c>/<c>list</c> option, which is done in separate functions.</p> -<p>The multi-request tag (requests) is handled in a separate loop +<p>The multi-request tag (<c>requests</c>) is handled in a separate loop function applying the requests in the list one after another, returning the last result.</p> -<p>What's left is to handle backward compatibility and the file-module +<p>What is left is to handle backward compatibility and the <seealso marker="kernel:file">file</seealso> module (which uses the old requests until backward compatibility with pre-R13 -nodes is no longer needed). Note that the io_server will not work with -a simple file:write if these are not added:</p> +nodes is no longer needed). Note that the I/O server will not work with +a simple <c>file:write/2</c> if these are not added:</p> <code> request({put_chars,Chars}, State) -> @@ -593,7 +600,7 @@ request({get_until, Prompt,M,F,As}, State) -> request({get_until,latin1,Prompt,M,F,As}, State); </code> -<p>Ok, what's left now is to return {error, request} if the request is +<p>OK, what is left now is to return <c>{error, request}</c> if the request is not recognized:</p> <code> @@ -601,7 +608,7 @@ request(_Other, State) -> {error, {error, request}, State}. </code> -<p>Let's move further and actually handle the different requests, first +<p>Let us move further and actually handle the different requests, first the fairly generic multi-request type:</p> <code> @@ -615,10 +622,10 @@ multi_request([], Result) -> <p>We loop through the requests one at the time, stopping when we either encounter an error or the list is exhausted. The last return value is -sent back to the client (it's first returned to the main loop and then -sent back by the function io_reply).</p> +sent back to the client (it is first returned to the main loop and then +sent back by the function <c>io_reply</c>).</p> -<p>The getopt and setopt requests are also simple to handle, we just +<p>The <c>getopts</c> and <c>setopts</c> requests are also simple to handle, we just change or read our state record:</p> <code> @@ -656,24 +663,24 @@ getopts(#state{mode=M} = S) -> end}],S}. </code> -<p>As a convention, all io_servers handle both {setopts, [binary]}, -{setopts, [list]} and {setopts,[{binary, bool()}]}, hence the trick -with proplists:substitute_negations/2 and proplists:unfold/1. If -invalid options are sent to us, we send {error,enotsup} back to the +<p>As a convention, all I/O servers handle both <c>{setopts, [binary]}</c>, +<c>{setopts, [list]}</c> and <c>{setopts,[{binary, boolean()}]}</c>, hence the trick +with <c>proplists:substitute_negations/2</c> and <c>proplists:unfold/1</c>. If +invalid options are sent to us, we send <c>{error, enotsup}</c> back to the client.</p> -<p>The getopts request should return a list of {Option, Value} tuples, +<p>The <c>getopts</c> request should return a list of <c>{Option, Value}</c> tuples, which has the twofold function of providing both the current values -and the available options of this io_server. We have only one option, +and the available options of this I/O server. We have only one option, and hence return that.</p> -<p>So far our io_server has been fairly generic (except for the rewind -request handled in the main loop and the creation of an ets table). -Most io_servers contain code similar to what's above.</p> +<p>So far our I/O server has been fairly generic (except for the <c>rewind</c> +request handled in the main loop and the creation of an ETS table). +Most I/O servers contain code similar to the one above.</p> <p>To make the example runnable, we now start implementing the actual -reading and writing of the data to/from the ets-table. First the -put_chars function:</p> +reading and writing of the data to/from the ETS table. First the +<c>put_chars/3</c> function:</p> <code> put_chars(Chars, #state{table = T, position = P} = State) -> @@ -686,10 +693,10 @@ put_chars(Chars, #state{table = T, position = P} = State) -> <p>We already have the data as (Unicode) lists and therefore just split the list in runs of a predefined size and put each run in the table at the current position (and forward). The functions -split_data/3 and apply_update/2 are implemented below.</p> +<c>split_data/3</c> and <c>apply_update/2</c> are implemented below.</p> -<p>Now we want to read data from the table. The get_until function reads -data and applies the function until it says it's done. The result is +<p>Now we want to read data from the table. The <c>get_until/5</c> function reads +data and applies the function until it says it is done. The result is sent back to the client:</p> <code> @@ -700,11 +707,12 @@ get_until(Encoding, Mod, Func, As, if M =:= binary -> {ok, - unicode:characters_to_binary(Data,unicode,Encoding), + unicode:characters_to_binary(Data, unicode, Encoding), State#state{position = NewP}}; true -> case check(Encoding, - unicode:characters_to_list(Data, unicode)) of + unicode:characters_to_list(Data, unicode)) + of {error, _} = E -> {error, E, State}; List -> @@ -730,24 +738,24 @@ get_loop(M,F,A,T,P,C) -> end. </code> -<p>Here we also handle the mode (binary or list) that can be set by -the setopts request. By default, all OTP io_servers send data back to -the client as lists, but switching mode to binary might increase -efficiency if the server handles it in an appropriate way. The -implementation of get_until is hard to get efficient as the supplied -function is defined to take lists as arguments, but get_chars and -get_line can be optimized for binary mode. This example does not +<p>Here we also handle the mode (<c>binary</c> or <c>list</c>) that can be set by +the <c>setopts</c> request. By default, all OTP I/O servers send data back to +the client as lists, but switching mode to <c>binary</c> might increase +efficiency if the I/O server handles it in an appropriate way. The +implementation of <c>get_until</c> is hard to get efficient as the supplied +function is defined to take lists as arguments, but <c>get_chars</c> and +<c>get_line</c> can be optimized for binary mode. This example does not optimize anything however. It is important though that the returned data is of the right type depending on the options set, so we convert the lists to binaries in the correct encoding <em>if possible</em> -before returning. The function supplied in the get_until request may, +before returning. The function supplied in the <c>get_until</c> request tuple may, as its final result return anything, so only functions actually returning lists can get them converted to binaries. If the request -contained the encoding tag unicode, the lists can contain all unicode +contained the encoding tag <c>unicode</c>, the lists can contain all Unicode codepoints and the binaries should be in UTF-8, if the encoding tag -was latin1, the client should only get characters in the range -0..255. The function check/2 takes care of not returning arbitrary -unicode codepoints in lists if the encoding was given as latin1. If +was <c>latin1</c>, the client should only get characters in the range +0..255. The function <c>check/2</c> takes care of not returning arbitrary +Unicode codepoints in lists if the encoding was given as <c>latin1</c>. If the function did not return a list, the check cannot be performed and the result will be that of the supplied function untouched.</p> @@ -768,13 +776,13 @@ check(latin1, List) -> end. </code> -<p>The function check takes care of providing an error tuple if unicode +<p>The function check takes care of providing an error tuple if Unicode codepoints above 255 is to be returned if the client requested latin1.</p> -<p>The two functions until_newline/3 and until_enough/3 are helpers used -together with the get_until function to implement get_chars and -get_line (inefficiently):</p> +<p>The two functions <c>until_newline/3</c> and <c>until_enough/3</c> are helpers used +together with the <c>get_until/5</c> function to implement <c>get_chars</c> and +<c>get_line</c> (inefficiently):</p> <code> until_newline([],eof,_MyStopCharacter) -> @@ -782,7 +790,9 @@ until_newline([],eof,_MyStopCharacter) -> until_newline(ThisFar,eof,_MyStopCharacter) -> {done,ThisFar,[]}; until_newline(ThisFar,CharList,MyStopCharacter) -> - case lists:splitwith(fun(X) -> X =/= MyStopCharacter end, CharList) of + case + lists:splitwith(fun(X) -> X =/= MyStopCharacter end, CharList) + of {L,[]} -> {more,ThisFar++L}; {L2,[MyStopCharacter|Rest]} -> @@ -802,10 +812,10 @@ until_enough(ThisFar,CharList,_N) -> </code> <p>As can be seen, the functions above are just the type of functions -that should be provided in get_until requests.</p> +that should be provided in <c>get_until</c> requests.</p> <p>Now we only need to read and write the table in an appropriate way to -complete the server:</p> +complete the I/O server:</p> <code> get(P,Tab) -> @@ -847,13 +857,13 @@ apply_update(Table, {Row, Col, List}) -> end. </code> -<p>The table is read or written in chunks of ?CHARS_PER_REC, overwriting +<p>The table is read or written in chunks of <c>?CHARS_PER_REC</c>, overwriting when necessary. The implementation is obviously not efficient, it is just working.</p> <p>This concludes the example. It is fully runnable and you can read or -write to the io_server by using i.e. the io_module or even the file -module. It's as simple as that to implement a fully fledged io_server +write to the I/O server by using i.e. the <seealso marker="stdlib:io">io</seealso> module or even the <seealso marker="kernel:file">file</seealso> +module. It is as simple as that to implement a fully fledged I/O server in Erlang.</p> </section> </chapter> diff --git a/lib/stdlib/doc/src/notes.xml b/lib/stdlib/doc/src/notes.xml index b4d9f9a444..2a308cbe09 100644 --- a/lib/stdlib/doc/src/notes.xml +++ b/lib/stdlib/doc/src/notes.xml @@ -30,6 +30,21 @@ </header> <p>This document describes the changes made to the STDLIB application.</p> +<section><title>STDLIB 1.18.3</title> + + <section><title>Fixed Bugs and Malfunctions</title> + <list> + <item> + <p> + Minor test updates</p> + <p> + Own Id: OTP-10591</p> + </item> + </list> + </section> + +</section> + <section><title>STDLIB 1.18.2</title> <section><title>Fixed Bugs and Malfunctions</title> diff --git a/lib/stdlib/doc/src/proplists.xml b/lib/stdlib/doc/src/proplists.xml index 225c5e97eb..8d64319344 100644 --- a/lib/stdlib/doc/src/proplists.xml +++ b/lib/stdlib/doc/src/proplists.xml @@ -70,7 +70,7 @@ <fsummary></fsummary> <desc> <p>Minimizes the representation of all entries in the list. This is - equivalent to <c><![CDATA[[property(P) || P <- List]]]></c>.</p> + equivalent to <c><![CDATA[[property(P) || P <- ListIn]]]></c>.</p> <p>See also: <c>property/1</c>, <c>unfold/1</c>.</p> </desc> </func> @@ -88,11 +88,11 @@ <desc> <p>Expands particular properties to corresponding sets of properties (or other terms). For each pair <c>{<anno>Property</anno>, <anno>Expansion</anno>}</c> in <c><anno>Expansions</anno></c>, if <c>E</c> is - the first entry in <c><anno>List</anno></c> with the same key as + the first entry in <c><anno>ListIn</anno></c> with the same key as <c><anno>Property</anno></c>, and <c>E</c> and <c><anno>Property</anno></c> have equivalent normal forms, then <c>E</c> is replaced with the terms in <c><anno>Expansion</anno></c>, and any following entries with - the same key are deleted from <c><anno>List</anno></c>.</p> + the same key are deleted from <c><anno>ListIn</anno></c>.</p> <p>For example, the following expressions all return <c>[fie, bar, baz, fum]</c>:</p> <code type="none"> expand([{foo, [bar, baz]}], @@ -198,7 +198,7 @@ <name name="normalize" arity="2"/> <fsummary></fsummary> <desc> - <p>Passes <c><anno>List</anno></c> through a sequence of + <p>Passes <c><anno>ListIn</anno></c> through a sequence of substitution/expansion stages. For an <c>aliases</c> operation, the function <c>substitute_aliases/2</c> is applied using the given list of aliases; for a <c>negations</c> operation, @@ -221,9 +221,9 @@ <fsummary></fsummary> <desc> <p>Creates a normal form (minimal) representation of a property. If - <c><anno>Property</anno></c> is <c>{Key, true}</c> where <c>Key</c> is - an atom, this returns <c>Key</c>, otherwise the whole term - <c><anno>Property</anno></c> is returned.</p> + <c><anno>PropertyIn</anno></c> is <c>{Key, true}</c> where + <c>Key</c> is an atom, this returns <c>Key</c>, otherwise + the whole term <c><anno>PropertyIn</anno></c> is returned.</p> <p>See also: <c>property/2</c>.</p> </desc> </func> @@ -260,7 +260,7 @@ <fsummary></fsummary> <desc> <p>Substitutes keys of properties. For each entry in - <c><anno>List</anno></c>, if it is associated with some key <c>K1</c> + <c><anno>ListIn</anno></c>, if it is associated with some key <c>K1</c> such that <c>{K1, K2}</c> occurs in <c><anno>Aliases</anno></c>, the key of the entry is changed to <c>K2</c>. If the same <c>K1</c> occurs more than once in <c><anno>Aliases</anno></c>, only @@ -278,13 +278,13 @@ <desc> <p>Substitutes keys of boolean-valued properties and simultaneously negates their values. For each entry in - <c><anno>List</anno></c>, if it is associated with some key <c>K1</c> + <c><anno>ListIn</anno></c>, if it is associated with some key <c>K1</c> such that <c>{K1, K2}</c> occurs in <c><anno>Negations</anno></c>, then if the entry was <c>{K1, true}</c> it will be replaced with <c>{K2, false}</c>, otherwise it will be replaced with <c>{K2, true}</c>, thus changing the name of the option and simultaneously negating the value given by - <c>get_bool(List)</c>. If the same <c>K1</c> occurs more + <c>get_bool(ListIn)</c>. If the same <c>K1</c> occurs more than once in <c><anno>Negations</anno></c>, only the first occurrence is used.</p> <p>Example: <c>substitute_negations([{no_foo, foo}], L)</c> @@ -300,7 +300,7 @@ <name name="unfold" arity="1"/> <fsummary></fsummary> <desc> - <p>Unfolds all occurrences of atoms in <c><anno>List</anno></c> to tuples + <p>Unfolds all occurrences of atoms in <c><anno>ListIn</anno></c> to tuples <c>{Atom, true}</c>.</p> </desc> </func> diff --git a/lib/stdlib/doc/src/re.xml b/lib/stdlib/doc/src/re.xml index 2211bfb925..71a6e34513 100644 --- a/lib/stdlib/doc/src/re.xml +++ b/lib/stdlib/doc/src/re.xml @@ -96,12 +96,12 @@ subjects during the program's lifetime. Compiling once and executing many times is far more efficient than compiling each time one wants to match.</p> - <p>When the unicode option is given, the regular expression should be given as a valid unicode <c>charlist()</c>, otherwise as any valid <c>iodata()</c>.</p> + <p>When the unicode option is given, the regular expression should be given as a valid Unicode <c>charlist()</c>, otherwise as any valid <c>iodata()</c>.</p> <p><marker id="compile_options"/>The options have the following meanings:</p> <taglist> <tag><c>unicode</c></tag> - <item>The regular expression is given as a unicode <c>charlist()</c> and the resulting regular expression code is to be run against a valid unicode <c>charlist()</c> subject.</item> + <item>The regular expression is given as a Unicode <c>charlist()</c> and the resulting regular expression code is to be run against a valid Unicode <c>charlist()</c> subject.</item> <tag><c>anchored</c></tag> <item>The pattern is forced to be "anchored", that is, it is constrained to match only at the first matching point in the string that is being searched (the "subject string"). This effect can also be achieved by appropriate constructs in the pattern itself.</item> <tag><c>caseless</c></tag> @@ -478,7 +478,7 @@ This option makes it possible to include comments inside complicated patterns. N <p>Replaces the matched part of the <c><anno>Subject</anno></c> string with the contents of <c><anno>Replacement</anno></c>.</p> <p>The permissible options are the same as for <c>re:run/3</c>, except that the <c>capture</c> option is not allowed. Instead a <c>{return, <anno>ReturnType</anno>}</c> is present. The default return type is <c>iodata</c>, constructed in a - way to minimize copying. The <c>iodata</c> result can be used directly in many i/o-operations. If a flat <c>list()</c> is + way to minimize copying. The <c>iodata</c> result can be used directly in many I/O-operations. If a flat <c>list()</c> is desired, specify <c>{return, list}</c> and if a binary is preferred, specify <c>{return, binary}</c>.</p> <p>As in the <c>re:run/3</c> function, an <c>mp()</c> compiled diff --git a/lib/stdlib/doc/src/unicode.xml b/lib/stdlib/doc/src/unicode.xml index 1f6cbaccd7..d235f3e180 100644 --- a/lib/stdlib/doc/src/unicode.xml +++ b/lib/stdlib/doc/src/unicode.xml @@ -32,9 +32,9 @@ <module>unicode</module> <modulesummary>Functions for converting Unicode characters</modulesummary> <description> - <p>This module contains functions for converting between different character representations. Basically it converts between iso-latin-1 characters and Unicode ditto, but it can also convert between different Unicode encodings (like UTF-8, UTF-16 and UTF-32).</p> + <p>This module contains functions for converting between different character representations. Basically it converts between ISO-latin-1 characters and Unicode ditto, but it can also convert between different Unicode encodings (like UTF-8, UTF-16 and UTF-32).</p> <p>The default Unicode encoding in Erlang is in binaries UTF-8, which is also the format in which built in functions and libraries in OTP expect to find binary Unicode data. In lists, Unicode data is encoded as integers, each integer representing one character and encoded simply as the Unicode codepoint for the character.</p> - <p>Other Unicode encodings than integers representing codepoints or UTF-8 in binaries are referred to as "external encodings". The iso-latin-1 encoding is in binaries and lists referred to as latin1-encoding.</p> + <p>Other Unicode encodings than integers representing codepoints or UTF-8 in binaries are referred to as "external encodings". The ISO-latin-1 encoding is in binaries and lists referred to as latin1-encoding.</p> <p>It is recommended to only use external encodings for communication with external entities where this is required. When working inside the Erlang/OTP environment, it is recommended to keep binaries in UTF-8 when representing Unicode characters. Latin1 encoding is supported both for backward compatibility and for communication with external entities not supporting Unicode character sets.</p> </description> @@ -48,13 +48,13 @@ <datatype> <name name="unicode_binary"/> <desc> - <p>A binary() with characters encoded in the UTF-8 coding standard.</p> + <p>A <c>binary()</c> with characters encoded in the UTF-8 coding standard.</p> </desc> </datatype> <datatype> <name name="unicode_char"/> <desc> - <p>An integer() representing a valid unicode codepoint.</p> + <p>An <c>integer()</c> representing a valid Unicode codepoint.</p> </desc> </datatype> <datatype> @@ -63,7 +63,7 @@ <datatype> <name name="charlist"/> <desc> - <p>A unicode_binary is allowed as the tail of the list.</p> + <p>A <c>unicode_binary()</c> is allowed as the tail of the list.</p> </desc> </datatype> <datatype> @@ -85,7 +85,7 @@ </datatype> <datatype> <name name="latin1_binary"/> - <desc><p>A <c>binary()</c> with characters coded in iso-latin-1.</p> + <desc><p>A <c>binary()</c> with characters coded in ISO-latin-1.</p> </desc> </datatype> <datatype> @@ -110,7 +110,9 @@ <name name="bom_to_encoding" arity="1"/> <fsummary>Identify UTF byte order marks in a binary.</fsummary> <type name="endian"/> - <type_desc variable="Bin">A binary() of byte_size 4 or more.</type_desc> + <type_desc variable="Bin"> + A <c>binary()</c> such that <c>byte_size(<anno>Bin</anno>) >= 4</c>. + </type_desc> <desc> <p>Check for a UTF byte order mark (BOM) in the beginning of a @@ -126,7 +128,7 @@ <name name="characters_to_list" arity="1"/> <fsummary>Convert a collection of characters to list of Unicode characters</fsummary> <desc> - <p>Same as characters_to_list(<anno>Data</anno>,unicode).</p> + <p>Same as <c>characters_to_list(<anno>Data</anno>, unicode)</c>.</p> </desc> </func> <func> @@ -134,8 +136,8 @@ <fsummary>Convert a collection of characters to list of Unicode characters</fsummary> <desc> - <p>This function converts a possibly deep list of integers and - binaries into a list of integers representing unicode + <p>Converts a possibly deep list of integers and + binaries into a list of integers representing Unicode characters. The binaries in the input may have characters encoded as latin1 (0 - 255, one character per byte), in which case the <c><anno>InEncoding</anno></c> parameter should be given as @@ -148,18 +150,18 @@ <p>If <c><anno>InEncoding</anno></c> is <c>latin1</c>, the <c><anno>Data</anno></c> parameter corresponds to the <c>iodata()</c> type, but for <c>unicode</c>, the <c><anno>Data</anno></c> parameter can contain integers greater than 255 - (unicode characters beyond the iso-latin-1 range), which would + (Unicode characters beyond the ISO-latin-1 range), which would make it invalid as <c>iodata()</c>.</p> <p>The purpose of the function is mainly to be able to convert - combinations of unicode characters into a pure unicode + combinations of Unicode characters into a pure Unicode string in list representation for further processing. For writing the data to an external entity, the reverse function <seealso - marker="#characters_to_binary/3">characters_to_binary/3</seealso> + marker="#characters_to_binary/3"><c>characters_to_binary/3</c></seealso> comes in handy.</p> - <p>The option <c>unicode</c> is an alias for <c>utf8</c>, as this is the + <p>The option <c>unicode</c> is an alias for <c>utf8</c>, as this is the preferred encoding for Unicode characters in binaries. <c>utf16</c> is an alias for <c>{utf16,big}</c> and <c>utf32</c> is an alias for <c>{utf32,big}</c>. The <c>big</c> @@ -167,7 +169,7 @@ encoding.</p> <p>If for some reason, the data cannot be converted, either - because of illegal unicode/latin1 characters in the list, or + because of illegal Unicode/latin1 characters in the list, or because of invalid UTF encoding in any binaries, an error tuple is returned. The error tuple contains the tag <c>error</c>, a list representing the characters that could be @@ -176,7 +178,7 @@ last part is mostly for debugging as it still constitutes a possibly deep and/or mixed list, not necessarily of the same depth as the original data. The error occurs when traversing the - list and whatever's left to decode is simply returned as is.</p> + list and whatever is left to decode is simply returned as is.</p> <p>However, if the input <c><anno>Data</anno></c> is a pure binary, the third part of the error tuple is guaranteed to be a binary as @@ -191,7 +193,7 @@ of a Unicode type, an error occurs whenever an integer <list type="bulleted"> <item>greater than <c>16#10FFFF</c> - (the maximum unicode character),</item> + (the maximum Unicode character),</item> <item>in the range <c>16#D800</c> to <c>16#DFFF</c> (invalid range reserved for UTF-16 surrogate pairs)</item> </list> @@ -205,14 +207,14 @@ (like the upper bits of the bytes being wrong), the bytes are decoded to a too large number, the bytes are decoded to a code-point in the - invalid unicode - range or encoding is "overlong", meaning that a + invalid Unicode + range, or encoding is "overlong", meaning that a number should have been encoded in fewer bytes. The case of a truncated UTF is handled specially, see the paragraph about incomplete binaries below. If <c><anno>InEncoding</anno></c> is <c>latin1</c>, binaries are always valid as long as they contain whole bytes, - as each byte falls into the valid iso-latin-1 range.</item> + as each byte falls into the valid ISO-latin-1 range.</item> </list> @@ -250,7 +252,7 @@ ever be decoded.</p> <p>If any parameters are of the wrong type, the list structure - is invalid (a number as tail) or the binaries does not contain + is invalid (a number as tail) or the binaries do not contain whole bytes (bit-strings), a <c>badarg</c> exception is thrown.</p> @@ -258,28 +260,27 @@ </func> <func> <name name="characters_to_binary" arity="1"/> - <fsummary>Convert a collection of characters to an UTF-8 binary</fsummary> + <fsummary>Convert a collection of characters to a UTF-8 binary</fsummary> <desc> - <p>Same as characters_to_binary(Data, unicode, unicode).</p> + <p>Same as <c>characters_to_binary(<anno>Data</anno>, unicode, unicode)</c>.</p> </desc> </func> <func> <name name="characters_to_binary" arity="2"/> - <fsummary>Convert a collection of characters to an UTF-8 binary</fsummary> + <fsummary>Convert a collection of characters to a UTF-8 binary</fsummary> <desc> - <p>Same as characters_to_binary(<anno>Data</anno>, <anno>InEncoding</anno>, unicode).</p> + <p>Same as <c>characters_to_binary(<anno>Data</anno>, <anno>InEncoding</anno>, unicode)</c>.</p> </desc> </func> <func> <name name="characters_to_binary" arity="3"/> - <fsummary>Convert a collection of characters to an UTF-8 binary</fsummary> + <fsummary>Convert a collection of characters to a UTF-8 binary</fsummary> <desc> - <p>This function behaves as <seealso - marker="#characters_to_list/2"> - characters_to_list/2</seealso>, but produces an binary - instead of a unicode list. The + <p>Behaves as <seealso marker="#characters_to_list/2"> + <c>characters_to_list/2</c></seealso>, but produces an binary + instead of a Unicode list. The <c><anno>InEncoding</anno></c> defines how input is to be interpreted if binaries are present in the <c>Data</c>, while <c><anno>OutEncoding</anno></c> defines in what format output is to be @@ -294,7 +295,7 @@ <p>Errors and exceptions occur as in <seealso marker="#characters_to_list/2"> - characters_to_list/2</seealso>, but the second element + <c>characters_to_list/2</c></seealso>, but the second element in the <c>error</c> or <c>incomplete</c> tuple will be a <c>binary()</c> and not a <c>list()</c>.</p> @@ -304,16 +305,18 @@ <func> <name name="encoding_to_bom" arity="1"/> <fsummary>Create a binary UTF byte order mark from encoding.</fsummary> - <type_desc variable="Bin">A binary() of byte_size 4 or more.</type_desc> + <type_desc variable="Bin"> + A <c>binary()</c> such that <c>byte_size(<anno>Bin</anno>) >= 4</c>. + </type_desc> <desc> - <p>Create an UTF byte order mark (BOM) as a binary from the + <p>Create a UTF byte order mark (BOM) as a binary from the supplied <c><anno>InEncoding</anno></c>. The BOM is, if supported at all, expected to be placed first in UTF encoded files or messages.</p> <p>The function returns <c><<>></c> for the - <c>latin1</c> encoding, there is no BOM for ISO-latin-1.</p> + <c>latin1</c> encoding as there is no BOM for ISO-latin-1.</p> <p>It can be noted that the BOM for UTF-8 is seldom used, and it is really not a <em>byte order</em> mark. There are obviously no diff --git a/lib/stdlib/doc/src/unicode_usage.xml b/lib/stdlib/doc/src/unicode_usage.xml index bbcd49a934..320b5b2e84 100644 --- a/lib/stdlib/doc/src/unicode_usage.xml +++ b/lib/stdlib/doc/src/unicode_usage.xml @@ -1,4 +1,4 @@ -<?xml version="1.0" encoding="latin1" ?> +<?xml version="1.0" encoding="utf8" ?> <!DOCTYPE chapter SYSTEM "chapter.dtd"> <chapter> @@ -33,14 +33,14 @@ <file>unicode_usage.xml</file> </header> <p>Implementing support for Unicode character sets is an ongoing process. The Erlang Enhancement Proposal (EEP) 10 outlines the basics of Unicode support and also specifies a default encoding in binaries that all Unicode-aware modules should handle in the future.</p> -<p>The functionality described in EEP10 is implemented in Erlang/OTP as of R13A, but that's by no means the end of it. More functionality will be needed in the future and more OTP-libraries might need updating to cope with Unicode data. One example of future development is obvious when reading this manual, our documentation format is limited to the ISO-latin-1 character range, why no Unicode characters beyond that range will occur in this document.</p> +<p>The functionality described in EEP10 is implemented in Erlang/OTP as of R13A, but that is by no means the end of it. More functionality will be needed in the future and more OTP-libraries might need updating to cope with Unicode data.</p> <p>This guide outlines the current Unicode support and gives a couple of recipes for working with Unicode data.</p> <section> -<title>What Unicode is</title> +<title>What Unicode Is</title> <p>Unicode is a standard defining codepoints (numbers) for all known, living or dead, scripts. In principle, every known symbol used in any language has a Unicode codepoint.</p> <p>Unicode codepoints are defined and published by the <em>Unicode Consortium</em>, which is a non profit organization.</p> <p>Support for Unicode is increasing throughout the world of computing, as the benefits of one common character set are overwhelming when programs are used in a global environment.</p> -<p>Along with the base of the standard, the codepoints for all the scripts, there are a couple of encoding standards available. Different operating systems and tools support different encodings. For example Linux and MacOS X has chosen the UTF-8 encoding, which is backwards compatible with 7-bit ASCII and therefore affects programs written in plain English the least. Windows® on the other hand supports a limited version of UTF-16, namely all the code planes where the characters can be stored in one single 16-bit entity, which includes most living languages.</p> +<p>Along with the base of the standard, the codepoints for all the scripts, there are a couple of encoding standards available. Different operating systems and tools support different encodings. For example Linux and MacOSX has chosen the UTF-8 encoding, which is backwards compatible with 7-bit ASCII and therefore affects programs written in plain English the least. Windows® on the other hand supports a limited version of UTF-16, namely all the code planes where the characters can be stored in one single 16-bit entity, which includes most living languages.</p> <p>The most widely spread encodings are:</p> <taglist> <tag>UTF-8</tag> @@ -56,38 +56,47 @@ <p>Additionally, the codepoint 16#FEFF is used for byte order marks (BOM's) and use of that character is not encouraged in other contexts than that. It actually is valid though, as the character "ZWNBS" (Zero Width Non Breaking Space). BOM's are used to identify encodings and byte order for programs where such parameters are not known in advance. Byte order marks are more seldom used than one could expect, but their use is becoming more widely spread as they provide the means for programs to make educated guesses about the Unicode format of a certain file.</p> </section> <section> -<title>Standard Unicode representation in Erlang</title> +<title>Standard Unicode Representation in Erlang</title> <p>In Erlang, strings are actually lists of integers. A string is defined to be encoded in the ISO-latin-1 (ISO8859-1) character set, which is, codepoint by codepoint, a sub-range of the Unicode character set.</p> <p>The standard list encoding for strings is therefore easily extendible to cope with the whole Unicode range: A Unicode string in Erlang is simply a list containing integers, each integer being a valid Unicode codepoint and representing one character in the Unicode character set.</p> -<p>Regular Erlang strings in ISO-latin-1 are a subset of their Unicode strings.</p> +<p>Regular Erlang strings in ISO-latin-1 are a subset of their Unicode +strings.</p> -<p>Binaries on the other hand are more troublesome. For performance reasons, programs often store textual data in binaries instead of lists, mainly because they are more compact (one byte per character instead of two words per character, as is the case with lists). Using erlang:list_to_binary/1, an regular Erlang string can be converted into a binary, effectively using the ISO-latin-1 encoding in the binary - one byte per character. This is very convenient for those regular Erlang strings, but cannot be done for Unicode lists.</p> +<p>Binaries on the other hand are more troublesome. For performance reasons, programs often store textual data in binaries instead of lists, mainly because they are more compact (one byte per character instead of two words per character, as is the case with lists). Using <c>erlang:list_to_binary/1</c>, a regular Erlang string can be converted into a binary, effectively using the ISO-latin-1 encoding in the binary - one byte per character. This is very convenient for those regular Erlang strings, but cannot be done for Unicode lists.</p> <p>As the UTF-8 encoding is widely spread and provides the most compact storage, it is selected as the standard encoding of Unicode characters in binaries for Erlang.</p> <p>The standard binary encoding is used whenever a library function in Erlang should cope with Unicode data in binaries, but is of course not enforced when communicating externally. Functions and bit-syntax exist to encode and decode both UTF-8, UTF-16 and UTF-32 in binaries. Library functions dealing with binaries and Unicode in general, however, only deal with the default encoding.</p> -<p>Character data may be combined from several sources, sometimes available in a mix of strings and binaries. Erlang has for long had the concept of iodata or iolists, where binaries and lists can be combined to represent a sequence of bytes. In the same way, the Unicode aware modules often allow for combinations of binaries and lists where the binaries have characters encoded in UTF-8 and the lists contain such binaries or numbers representing Unicode codepoints:</p> +<p>Character data may be combined from several sources, sometimes available in a mix of strings and binaries. Erlang has for long had the concept of <c>iodata</c> or <c>iolists</c>, where binaries and lists can be combined to represent a sequence of bytes. In the same way, the Unicode aware modules often allow for combinations of binaries and lists where the binaries have characters encoded in UTF-8 and the lists contain such binaries or numbers representing Unicode codepoints:</p> <code type="none"> unicode_binary() = binary() with characters encoded in UTF-8 coding standard -unicode_char() = integer() representing valid unicode codepoint +unicode_char() = integer() >= 0 representing valid Unicode codepoint chardata() = charlist() | unicode_binary() charlist() = [unicode_char() | unicode_binary() | charlist()] a unicode_binary is allowed as the tail of the list</code> -<p>The module <c>unicode</c> in stdlib even supports similar mixes with binaries containing other encodings than UTF-8, but that is a special case to allow for conversions to and from external data:</p> +<p>The module <c>unicode</c> in STDLIB even supports similar mixes with binaries containing other encodings than UTF-8, but that is a special case to allow for conversions to and from external data:</p> <code type="none"> -external_unicode_binary() = binary() with characters coded in a user specified Unicode - encoding other than UTF-8 (UTF-16 or UTF-32) +external_unicode_binary() = binary() with characters coded in + a user specified Unicode encoding other than UTF-8 (UTF-16 or UTF-32) external_chardata() = external_charlist() | external_unicode_binary() -external_charlist() = [unicode_char() | external_unicode_binary() | external_charlist()] - an external_unicode_binary is allowed as the tail of the list</code> +external_charlist() = [unicode_char() | + external_unicode_binary() | + external_charlist()] + an external_unicode_binary() is allowed as the tail of the list</code> </section> <section> -<title>Basic language support for Unicode</title> -<p>First of all, Erlang is still defined to be written in the ISO-latin-1 character set. Functions have to be named in that character set, atoms are restricted to ISO-latin-1 and regular strings are still lists of characters 0..255 in the ISO-latin-1 encoding. This has not (yet) changed, but the language has been slightly extended to cope with Unicode characters and encodings.</p> - +<title>Basic Language Support for Unicode</title> +<p><marker id="unicode_in_erlang"/>As of Erlang/OTP R16 Erlang can be +written in ISO-latin-1 or Unicode (UTF-8). The details on how to state +the encoding of an Erlang source file can be found in <seealso +marker="stdlib:epp#encoding">epp(3)</seealso>. Strings and comments +can be written using Unicode, but functions still have to be named in +ISO-latin-1 and atoms are restricted to ISO-latin-1. Erlang/OTP R18 is +expected to handle functions named in Unicode as well as Unicode +atoms.</p> <section> <title>Bit-syntax</title> <p>The bit-syntax contains types for coping with binary data in the three main encodings. The types are named <c>utf8</c>, <c>utf16</c> and <c>utf32</c> respectively. The <c>utf16</c> and <c>utf32</c> types can be in a big- or little-endian variant:</p> @@ -101,74 +110,79 @@ Bin3 = <<$H/utf32-little, $e/utf32-little, $l/utf32-little, $l/utf32-littl Bin4 = <<"Hello"/utf16>>,</code> </section> <section> -<title>String- and character-literals</title> -<warning> -<p>The literal syntax described here may be subject to change in R13B, it has not yet passed the usual process for language changes approval.</p> -</warning> -<p>It is convenient to be able to write a list of Unicode characters in the string syntax. However, the language specifies strings as being in the ISO-latin-1 character set which the compiler tool chain as well as many other tools expect.</p> -<p>Also the source code is (for now) still expected to be written using the ISO-latin-1 character set, why Unicode characters beyond that range cannot be entered in string literals.</p> -<p>To make it easier to enter Unicode characters in the shell, it allows strings with Unicode characters on input, immediately converting them to regular lists of integers. They will, by the evaluator etc be viewed as if they were input using the regular list syntax, which is - in the end - how the language actually treats them. They will in the same way not be output as strings by i.e <c>io:write/2</c> or <c>io:format/3</c> unless the format string supplied to <c>io:format</c> uses the Unicode translation modifier (which we will talk about later).</p> -<p>For source code, there is an extension to the \OOO (backslash followed by three octal numbers) and \xHH (backslash followed by 'x', followed by two hexadecimal characters) syntax, namely \x{H ...} (a backslash followed by an 'x', followed by left curly bracket, any number of hexadecimal digits and a terminating right curly bracket). This allows for entering characters of any codepoint literally in a string. The string is immediately converted into a list by the scanner however, which is obvious when calling it directly:</p> -<pre> -1> <input>erl_scan:string("\"X\".").</input> -{ok,[{string,1,"X"},{dot,1}],1} -2> <input>erl_scan:string("\"\x{400}\".").</input> -{ok,[{'[',1},{integer,1,1024},{']',1},{dot,1}],1}</pre> -<p>Character literals, or rather integers representing Unicode codepoints can be expressed in a similar way using $\x{H ...}:</p> -<pre> -4> <input>$\x{400}.</input> -1024</pre> -<p>This also is a translation by the scanner:</p> -<pre> -5> <input>erl_scan:string("$Y.").</input> -{ok,[{char,1,89},{dot,1}],1} -6> <input>erl_scan:string("$\x{400}.").</input> -{ok,[{integer,1,1024},{dot,1}],1}</pre> -<p>In the shell, if using a Unicode input device, '$' can be followed directly by a Unicode character producing an integer. In the following example, let's imagine the character 'c' is actually a Cyrillic 's' (looking fairly similar):</p> +<title>String- and Character-literals</title> +<p>For source code, there is an extension to the <c>\</c>OOO (backslash +followed by three octal numbers) and <c>\x</c>HH (backslash followed by <c>x</c>, +followed by two hexadecimal characters) syntax, namely <c>\x{</c>H ...<c>}</c> (a +backslash followed by an <c>x</c>, followed by left curly bracket, any +number of hexadecimal digits and a terminating right curly bracket). +This allows for entering characters of any codepoint literally in a +string even when the encoding is ISO-latin-1.</p> +</section> +<p>In the shell, if using a Unicode input device, <c>$</c> can be followed directly by a Unicode character producing an integer. In the following example the codepoint of a Cyrillic <c>s</c> is output:</p> <pre> -7> <input>$c.</input> +7> <input>$с.</input> 1089</pre> </section> -<p>The literal syntax allowing Unicode characters is to be viewed as "syntactic sugar", but is, as such, fairly useful.</p> -</section> <section> -<title>The interactive shell</title> +<title>The Interactive Shell</title> <p>The interactive Erlang shell, when started towards a terminal or started using the <c>werl</c> command on windows, can support Unicode input and output.</p> -<p>On Windows®, proper operation requires that a suitable font is installed and selected for the Erlang application to use. If no suitable font is available on your system, try installing the DejaVu fonts (dejavu-fonts.org), which are freely available and then select that font in the Erlang shell application.</p> -<p>On Unix®-like operating systems, the terminal should be able to handle UTF-8 on input and output (modern versions of XTerm, KDE konsole and the Gnome terminal do for example) and your locale settings have to be proper. As an example, my LANG environment variable is set as this:</p> +<p>On Windows®, proper operation requires that a suitable font is installed and selected for the Erlang application to use. If no suitable font is available on your system, try installing the DejaVu fonts (<c>dejavu-fonts.org</c>), which are freely available and then select that font in the Erlang shell application.</p> +<p>On Unix®-like operating systems, the terminal should be able to handle UTF-8 on input and output (modern versions of XTerm, KDE konsole and the Gnome terminal do for example) and your locale settings have to be proper. As an example, my <c>LANG</c> environment variable is set as this:</p> <pre> $ <input>echo $LANG</input> en_US.UTF-8</pre> -<p>Actually, most systems handle the LC_CTYPE variable before LANG, so if that is set, it has to be set to UTF-8:</p> +<p>Actually, most systems handle the <c>LC_CTYPE</c> variable before <c>LANG</c>, so if that is set, it has to be set to <c>UTF-8</c>:</p> <pre> $ echo <input>$LC_CTYPE</input> en_US.UTF-8</pre> -<p>The LANG or LC_CTYPE setting should be consistent with what the terminal is capable of, there is no portable way for Erlang to ask the actual terminal about its UTF-8 capacity, we have to rely on the language and character type settings.</p> +<p>The <c>LANG</c> or <c>LC_CTYPE</c> setting should be consistent with what the terminal is capable of, there is no portable way for Erlang to ask the actual terminal about its UTF-8 capacity, we have to rely on the language and character type settings.</p> <p>To investigate what Erlang thinks about the terminal, the <c>io:getopts()</c> call can be used when the shell is started:</p> <pre> $ <input>LC_CTYPE=en_US.ISO-8859-1 erl</input> -Erlang R13A (erts-5.7) [source] [64-bit] [smp:4:4] [rq:4] [async-threads:0] [kernel-poll:false] +Erlang R16B (erts-5.10) [source] [async-threads:0] [hipe] [kernel-poll:false] -Eshell V5.7 (abort with ^G) -1> <input>lists:keyfind(encoding,1,io:getopts()).</input> +Eshell V5.10 (abort with ^G) +1> <input>lists:keyfind(encoding, 1, io:getopts()).</input> {encoding,latin1} 2> <input>q().</input> ok $ <input>LC_CTYPE=en_US.UTF-8 erl</input> -Erlang R13A (erts-5.7) [source] [64-bit] [smp:4:4] [rq:4] [async-threads:0] [kernel-poll:false] +Erlang R16B (erts-5.10) [source] [async-threads:0] [hipe] [kernel-poll:false] -Eshell V5.7 (abort with ^G) -1> <input>lists:keyfind(encoding,1,io:getopts()).</input> +Eshell V5.10 (abort with ^G) +1> <input>lists:keyfind(encoding, 1, io:getopts()).</input> {encoding,unicode} 2></pre> <p>When (finally?) everything is in order with the locale settings, fonts and the terminal emulator, you probably also have discovered a way to input characters in the script you desire. For testing, the simplest way is to add some keyboard mappings for other languages, usually done with some applet in your desktop environment. In my KDE environment, I start the KDE Control Center (Personal Settings), select "Regional and Accessibility" and then "Keyboard Layout". On Windows XP®, I start Control Panel->Regional and Language Options, select the Language tab and click the Details... button in the square named "Text services and input Languages". Your environment probably provides similar means of changing the keyboard layout. Make sure you have a way to easily switch back and forth between keyboards if you are not used to this, entering commands using a Cyrillic character set is, as an example, not easily done in the Erlang shell.</p> <p>Now you are set up for some Unicode input and output. The simplest thing to do is of course to enter a string in the shell:</p> -<image file="ushell1.gif"><icaption>Cyrillic characters in an Erlang shell</icaption></image> +<pre> +$ <input>erl</input> +Erlang R16B (erts-5.10) [source] [async-threads:0] [hipe] [kernel-poll:false] + +Eshell V5.10 (abort with ^G) +1> <input>lists:keyfind(encoding, 1, io:getopts()).</input> +{encoding,unicode} +2> <input>"уницоде"</input> +"уницоде" +3> <input>io:format("~ts~n", [v(2)]).</input> +уницоде +ok +4> </pre> <p>While strings can be input as Unicode characters, the language elements are still limited to the ISO-latin-1 character set. Only character constants and strings are allowed to be beyond that range:</p> -<image file="ushell2.gif"><icaption>Unicode characters in allowed and disallowed context</icaption></image> +<pre> +$ <input>erl</input> +Erlang R16B (erts-5.10) [source] [async-threads:0] [hipe] [kernel-poll:false] + +Eshell V5.10 (abort with ^G) +1> <input>$ξ</input> +958 +2> <input>уницоде.</input> +* 1: illegal character +2> </pre> </section> <section> -<title>Unicode file names</title> +<title>Unicode File Names</title> <p>Most modern operating systems support Unicode file names in some way or another. There are several different ways to do this and Erlang by default treats the different approaches differently:</p> <taglist> <tag>Mandatory Unicode file naming</tag> @@ -183,7 +197,7 @@ Eshell V5.7 (abort with ^G) <p>A raw file name is not a list, but a binary. Many non core applications still do not handle file names given as binaries, why such raw names are avoided by default. This means that systems having implemented Unicode file naming through transparent file systems and an UTF-8 convention, do not by default have Unicode file naming turned on. Explicitly turning Unicode file name handling on for these types of systems is considered experimental.</p> </item> </taglist> -<p>The Unicode file naming support was introduced with OTP release R14B01. A VM operating in Unicode file mode can work with files having names in any language or character set (as long as it's supported by the underlying OS and file system). The Unicode character list is used to denote file or directory names and if the file system content is listed, you will also be able to get Unicode lists as return value. The support lies in the kernel and stdlib modules, why most applications (that does not explicitly require the file names to be in the ISO-latin-1 range) will benefit from the Unicode support without change.</p> +<p>The Unicode file naming support was introduced with OTP release R14B01. A VM operating in Unicode file mode can work with files having names in any language or character set (as long as it is supported by the underlying OS and file system). The Unicode character list is used to denote file or directory names and if the file system content is listed, you will also be able to get Unicode lists as return value. The support lies in the Kernel and STDLIB modules, why most applications (that does not explicitly require the file names to be in the ISO-latin-1 range) will benefit from the Unicode support without change.</p> <p>On Operating systems with mandatory Unicode file names, this means that you more easily conform to the file names of other (non Erlang) applications, and you can also process file names that, at least on Windows, were completely inaccessible (due to having names that could not be represented in ISO-latin-1). Also you will avoid creating incomprehensible file names on MacOSX as the vfs layer of the OS will accept all your file names as UTF-8 and will not rewrite them.</p> @@ -193,31 +207,32 @@ Eshell V5.7 (abort with ^G) <p>It is worth noting that the file <c>encoding</c> options given when opening a file has nothing to do with the file <em>name</em> encoding convention. You can very well open files containing UTF-8 but having file names in ISO-latin-1 or vice versa.</p> -<note>Erlang drivers and NIF shared objects still can not be named with names containing codepoints beyond 127. This is a known limitation to be removed in a future release. Erlang modules however can, but it is definitely not a good idea and is still considered experimental.</note> +<note><p>Erlang drivers and NIF shared objects still can not be named with names containing codepoints beyond 127. This is a known limitation to be removed in a future release. Erlang modules however can, but it is definitely not a good idea and is still considered experimental.</p></note> <section> -<title>Notes about raw file names and automatic file name conversion</title> -<p>Raw file names is introduced together with Unicode file name support in erts-5.8.2 (OTP R14B01). The reason "raw file names" is introduced in the system is to be able to consistently represent file names given in different encodings on the same system. Having the VM automatically translate a file name that is not in UTF-8 to a list of Unicode characters might seem practical, but this would open up for both duplicate file names and other inconsistent behavior. Consider a directory containing a file named "bj�rn" in ISO-latin-1, while the Erlang VM is operating in Unicode file name mode (and therefore expecting UTF-8 file naming). The ISO-latin-1 name is not valid UTF-8 and one could be tempted to think that automatic conversion in for example <c>file:list_dir/1</c> is a good idea. But what would happen if we later tried to open the file and have the name as a Unicode list (magically converted from the ISO-latin-1 file name)? The VM will convert the file name given to UTF-8, as this is the encoding expected. Effectively this means trying to open the file named <<"bj�rn"/utf8>>. This file does not exist, and even if it existed it would not be the same file as the one that was listed. We could even create two files named "bj�rn", one named in the UTF-8 encoding and one not. If <c>file:list_dir/1</c> would automatically convert the ISO-latin-1 file name to a list, we would get two identical file names as the result. To avoid this, we need to differentiate between file names being properly encoded according to the Unicode file naming convention (i.e. UTF-8) and file names being invalid under the encoding. This is done by representing invalid encoding as "raw" file names, i.e. as binaries.</p> -<p>The core system of Erlang (kernel and stdlib) accepts raw file names except for loadable drivers and executables invoked using <c>open_port({spawn, ...} ...)</c>. <c>open_port({spawn_executable, ...} ...)</c> however does accept them. As mentioned earlier, the arguments given in the option list to <c>open_port({spawn_executable, ...} ...)</c> undergo the same conversion as the file names, meaning that the executable will be provided with arguments in UTF-8 as well. This translation is avoided consistently with how the file names are treated, by giving the argument as a binary.</p> +<title>Notes About Raw File Names and Automatic File Name Conversion</title> + +<p>Raw file names is introduced together with Unicode file name support in erts-5.8.2 (OTP R14B01). The reason "raw file names" is introduced in the system is to be able to consistently represent file names given in different encodings on the same system. Having the VM automatically translate a file name that is not in UTF-8 to a list of Unicode characters might seem practical, but this would open up for both duplicate file names and other inconsistent behavior. Consider a directory containing a file named "björn" in ISO-latin-1, while the Erlang VM is operating in Unicode file name mode (and therefore expecting UTF-8 file naming). The ISO-latin-1 name is not valid UTF-8 and one could be tempted to think that automatic conversion in for example <c>file:list_dir/1</c> is a good idea. But what would happen if we later tried to open the file and have the name as a Unicode list (magically converted from the ISO-latin-1 file name)? The VM will convert the file name given to UTF-8, as this is the encoding expected. Effectively this means trying to open the file named <<"björn"/utf8>>. This file does not exist, and even if it existed it would not be the same file as the one that was listed. We could even create two files named "björn", one named in the UTF-8 encoding and one not. If <c>file:list_dir/1</c> would automatically convert the ISO-latin-1 file name to a list, we would get two identical file names as the result. To avoid this, we need to differentiate between file names being properly encoded according to the Unicode file naming convention (i.e. UTF-8) and file names being invalid under the encoding. This is done by representing invalid encoding as "raw" file names, i.e. as binaries.</p> +<p>The core system of Erlang (Kernel and STDLIB) accepts raw file names except for loadable drivers and executables invoked using <c>open_port({spawn, ...} ...)</c>. <c>open_port({spawn_executable, ...} ...)</c> however does accept them. As mentioned earlier, the arguments given in the option list to <c>open_port({spawn_executable, ...} ...)</c> undergo the same conversion as the file names, meaning that the executable will be provided with arguments in UTF-8 as well. This translation is avoided consistently with how the file names are treated, by giving the argument as a binary.</p> <p>To force Unicode file name translation mode on systems where this is not the default is considered experimental in OTP R14B01 due to the raw file names possibly being a new experience to the programmer and that the non core applications of OTP are not tested for compliance with raw file names yet. Unicode file name translation is expected to be default in future releases.</p> -<p>If working with raw file names, one can still conform to the encoding convention of the Erlang VM by using the <c>file:native_name_encoding/0</c> function, which returns either the atom <c>latin1</c> or the atom <c>utf8</c> depending on the file name translation mode. On Linux, a VM started without explicitly stating the file name translation mode will default to <c>latin1</c> as the native file name encoding, why file names on the disk encoded as UTF-8 will be returned as a list of the names interpreted as ISO-latin-1. The "UTF-8 list" is not a practical type for displaying or operating on in Erlang, but it is backward compatible and usable in all functions requiring a file name. On Windows and MacOSX, the default behavior is that of file name translation, why the <c>file:native_name_encoding/0</c> by default returns <c>utf8</c> on those systems (the fact that Windows actually does not use UTF-8 on the file system level can safely be ignored by the Erlang programmer). The default behavior can be changed using the <c>+fnu</c> or <c>+fnl</c> options to the VM, see the <c>erl</c> command manual page.</p> +<p>If working with raw file names, one can still conform to the encoding convention of the Erlang VM by using the <c>file:native_name_encoding/0</c> function, which returns either the atom <c>latin1</c> or the atom <c>utf8</c> depending on the file name translation mode. On Linux, a VM started without explicitly stating the file name translation mode will default to <c>latin1</c> as the native file name encoding, why file names on the disk encoded as UTF-8 will be returned as a list of the names interpreted as ISO-latin-1. The "UTF-8 list" is not a practical type for displaying or operating on in Erlang, but it is backward compatible and usable in all functions requiring a file name. On Windows and MacOSX, the default behavior is that of file name translation, why the <c>file:native_name_encoding/0</c> by default returns <c>utf8</c> on those systems (the fact that Windows actually does not use UTF-8 on the file system level can safely be ignored by the Erlang programmer). The default behavior can be changed using the <c>+fnu</c> or <c>+fnl</c> options to the VM, see the <seealso marker="erts:erl"><c>erl(1)</c></seealso> command manual page.</p> <p>Even if you are operating without Unicode file naming translation automatically done by the VM, you can access and create files with names in UTF-8 encoding by using raw file names encoded as UTF-8. Enforcing the UTF-8 encoding regardless of the mode the Erlang VM is started in might, in some circumstances be a good idea, as the convention of using UTF-8 file names is spreading.</p> </section> <section> -<title>Notes about MacOSX</title> +<title>Notes About MacOSX</title> <p>MacOSXs vfs layer enforces UTF-8 file names in a quite aggressive way. Older versions did this by simply refusing to create non UTF-8 conforming file names, while newer versions replace offending bytes with the sequence "%HH", where HH is the original character in hexadecimal notation. As Unicode translation is enabled by default on MacOSX, the only way to come up against this is to either start the VM with the <c>+fnl</c> flag or to use a raw file name in <c>latin1</c> encoding. In that case, the file can not be opened with the same name as the one used to create this. The problem is by design in newer versions of MacOSX.</p> -<p>MacOSX also reorganizes the names of files so that the representation of accents etc is denormalized, i.e. the character <c>�</c> is represented as the codepoints [111,776], where 111 is the character <c>o</c> and 776 is a special accent character. This type of denormalized Unicode is otherwise very seldom used and Erlang normalizes those file names on retrieval, so that denormalized file names is not passed up to the Erlang application. In Erlang the file name "bj�rn" is retrieved as [98,106,246,114,110], not as [98,106,117,776,114,110], even though the file system might think differently.</p> +<p>MacOSX also reorganizes the names of files so that the representation of accents etc is denormalized, i.e. the character <c>ö</c> is represented as the codepoints [111,776], where 111 is the character <c>o</c> and 776 is a special accent character. This type of denormalized Unicode is otherwise very seldom used and Erlang normalizes those file names on retrieval, so that denormalized file names is not passed up to the Erlang application. In Erlang the file name "björn" is retrieved as [98,106,246,114,110], not as [98,106,117,776,114,110], even though the file system might think differently.</p> </section> </section> <section> -<title>Unicode in environment variables and parameters</title> +<title>Unicode in Environment Variables and Parameters</title> <p>Environment variables and their interpretation is handled much in the same way as file names. If Unicode file names are enabled, environment variables as well as parameters to the Erlang VM are expected to be in Unicode.</p> <p>If Unicode file names are enabled, the calls to <seealso marker="kernel:os#getenv/0"><c>os:getenv/0</c></seealso>, <seealso marker="kernel:os#getenv/1"><c>os:getenv/1</c></seealso> and <seealso marker="kernel:os#putenv/2"><c>os:putenv/2</c></seealso> will handle Unicode strings. On Unix-like platforms, the built-in functions will translate environment variables in UTF-8 to/from Unicode strings, possibly with codepoints > 255. On Windows the Unicode versions of the environment system API will be used, also allowing for codepoints > 255.</p> <p>On Unix-like operating systems, parameters are expected to be UTF-8 without translation if Unicode file names are enabled.</p> </section> <section> -<title>Unicode-aware modules</title> -<p>Most of the modules in Erlang/OTP are of course Unicode-unaware in the sense that they have no notion of Unicode and really shouldn't have. Typically they handle non-textual or byte-oriented data (like <c>gen_tcp</c> etc).</p> +<title>Unicode-aware Modules</title> +<p>Most of the modules in Erlang/OTP are of course Unicode-unaware in the sense that they have no notion of Unicode and really should not have. Typically they handle non-textual or byte-oriented data (like <c>gen_tcp</c> etc).</p> <p>Modules that actually handle textual data (like <c>io_lib</c>, <c>string</c> etc) are sometimes subject to conversion or extension to be able to handle Unicode characters.</p> <p>Fortunately, most textual data has been stored in lists and range checking has been sparse, why modules like <c>string</c> works well for Unicode lists with little need for conversion or extension.</p> <p>Some modules are however changed to be explicitly Unicode-aware. These modules include:</p> @@ -230,7 +245,7 @@ Eshell V5.7 (abort with ^G) <item> <p>The <seealso marker="stdlib:io">io</seealso> module has been extended along with the actual I/O-protocol to handle Unicode data. This means that several functions require binaries to be in UTF-8 and there are modifiers to formatting control sequences to allow for outputting of Unicode strings.</p> </item> -<tag><c>file</c>, <c>group</c> and <c>user</c></tag> +<tag><c>file</c>, <c>group</c>, <c>user</c></tag> <item> <p>I/O-servers throughout the system are able both to handle Unicode data and has options for converting data upon actual output or input to/from the device. As shown earlier, the <seealso marker="stdlib:shell">shell</seealso> has support for Unicode terminals and the <seealso marker="kernel:file">file</seealso> module allows for translation to and from various Unicode formats on disk.</p> <p>The actual reading and writing of files with Unicode data is however not best done with the <c>file</c> module as its interface is byte oriented. A file opened with a Unicode encoding (like UTF-8), is then best read or written using the <seealso marker="stdlib:io">io</seealso> module.</p> @@ -247,10 +262,10 @@ Eshell V5.7 (abort with ^G) <p>The module <seealso marker="stdlib:string">string</seealso> works perfect for Unicode strings as well as for ISO-latin-1 strings with the exception of the language-dependent <seealso marker="stdlib:string#to_upper/1">to_upper</seealso> and <seealso marker="stdlib:string#to_lower/1">to_lower</seealso> functions, which are only correct for the ISO-latin-1 character set. Actually they can never function correctly for Unicode characters in their current form, there are language and locale issues as well as multi-character mappings to consider when conversion text between cases. Converting case in an international environment is a big subject not yet addressed in OTP.</p> </section> <section> -<title>Unicode recipes</title> +<title>Unicode Recipes</title> <p>When starting with Unicode, one often stumbles over some common issues. I try to outline some methods of dealing with Unicode data in this section.</p> <section> -<title>Byte order marks</title> +<title>Byte Order Marks</title> <p>A common method of identifying encoding in text-files is to put a byte order mark (BOM) first in the file. The BOM is the codepoint 16#FEFF encoded in the same way as the rest of the file. If such a file is to be read, the first few bytes (depending on encoding) is not part of the actual text. This code outlines how to open a file which is believed to have a BOM and set the files encoding and position for further sequential reading (preferably using the <seealso marker="stdlib:io">io</seealso> module). Note that error handling is omitted from the code:</p> <code> open_bom_file_for_reading(File) -> @@ -261,7 +276,7 @@ open_bom_file_for_reading(File) -> io:setopts(F,[{encoding,Type}]), {ok,F}. </code> -<p>The <c>unicode:bom_to_encoding/1</c> function identifies the encoding from a binary of at least four bytes. It returns, along with an term suitable for setting the encoding of the file, the actual length of the BOM, so that the file position can be set accordingly. Note that <c>file:position</c> always works on byte-offsets, so that the actual byte-length of the BOM is needed.</p> +<p>The <c>unicode:bom_to_encoding/1</c> function identifies the encoding from a binary of at least four bytes. It returns, along with an term suitable for setting the encoding of the file, the actual length of the BOM, so that the file position can be set accordingly. Note that <c>file:position/2</c> always works on byte-offsets, so that the actual byte-length of the BOM is needed.</p> <p>To open a file for writing and putting the BOM first is even simpler:</p> <code> open_bom_file_for_writing(File,Encoding) -> @@ -273,24 +288,33 @@ open_bom_file_for_writing(File,Encoding) -> <p>In both cases the file is then best processed using the <c>io</c> module, as the functions in <c>io</c> can handle codepoints beyond the ISO-latin-1 range.</p> </section> <section> -<title>Formatted input and output</title> -<p>When reading and writing to Unicode-aware entities, like the User or a file opened for Unicode translation, you will probably want to format text strings using the functions in <seealso marker="stdlib:io">io</seealso> or <seealso marker="stdlib:io_lib">io_lib</seealso>. For backward compatibility reasons, these functions don't accept just any list as a string, but require e special "translation modifier" when working with Unicode texts. The modifier is "t". When applied to the "s" control character in a formatting string, it accepts all Unicode codepoints and expect binaries to be in UTF-8:</p> +<title>Formatted Input and Output</title> +<p>When reading and writing to Unicode-aware entities, like the User or a file opened for Unicode translation, you will probably want to format text strings using the functions in <seealso marker="stdlib:io">io</seealso> or <seealso marker="stdlib:io_lib">io_lib</seealso>. For backward compatibility reasons, these functions do not accept just any list as a string, but require a special <em>translation modifier</em> when working with Unicode texts. The modifier is <c>t</c>. When applied to the <c>s</c> control character in a formatting string, it accepts all Unicode codepoints and expect binaries to be in UTF-8:</p> <pre> -1> <input>io:format("~ts~n",[<<"���"/utf8>>]).</input> -��� -ok -2> <input>io:format("~s~n",[<<"���"/utf8>>]).</input> +1> <input>io:format("~ts~n",[<<"åäö"/utf8>>]).</input> åäö +ok +2> <input>io:format("~s~n",[<<"åäö"/utf8>>]).</input> +åäö +ok</pre> +<p>Obviously the second <c>io:format/2</c> gives undesired output because the UTF-8 binary is not in latin1. Because ISO-latin-1 is still the defined character set of Erlang, the non prefixed <c>s</c> control character expects ISO-latin-1 in binaries as well as lists.</p> +<p>As long as the data is always lists, the <c>t</c> modifier can be used for any string, but when binary data is involved, care must be taken to make the right choice of formatting characters.</p> +<p>The function <c>format/2</c> in <c>io_lib</c> behaves similarly. This function is defined to return a deep list of characters and the output could easily be converted to binary data for outputting on a device of any kind by a simple <c>erlang:list_to_binary/1</c>. When the translation modifier is used, the list can however contain characters that cannot be stored in one byte. The call to <c>erlang:list_to_binary/1</c> will in that case fail. However, if the I/O server you want to communicate with is Unicode-aware, the list returned can still be used directly:</p> +<pre> +$ <input>erl</input> +Erlang R16B (erts-5.10) [source] [async-threads:0] [hipe] [kernel-poll:false] + +Eshell V5.10 (abort with ^G) +1> <input>io_lib:format("~ts~n", ["θνιψοδε"]).</input> +["θνιψοδε","\n"] +2> <input>io:put_chars(io_lib:format("~ts~n", ["θνιψοδε"])).</input> +θνιψοδε ok</pre> -<p>Obviously the second <c>io:format</c> gives undesired output because the UTF-8 binary is not in latin1. Because ISO-latin-1 is still the defined character set of Erlang, the non prefixed "s" control character expects ISO-latin-1 in binaries as well as lists.</p> -<p>As long as the data is always lists, the "t" modifier can be used for any string, but when binary data is involved, care must be taken to make the tight choice of formatting characters.</p> -<p>The function <c>format</c> in <c>io_lib</c> behaves similarly. This function is defined to return a deep list of characters and the output could easily be converted to binary data for outputting on a device of any kind by a simple <c>erlang:list_to_binary</c>. When the translation modifier is used, the list can however contain characters that cannot be stored in one byte. The call to <c>erlang:list_to_binary</c> will in that case fail. However, if the io_server you want to communicate with is Unicode-aware, the list returned can still be used directly:</p> -<image file="ushell3.gif"><icaption>io_lib:format with Unicode translation</icaption></image> -<p>The Unicode string is returned as a Unicode list, why the return value of <c>io_lib:format</c> no longer qualifies as a regular Erlang string (the function <seealso marker="stdlib:io_lib#deep_char_list/1">io_lib:deep_char_list</seealso> will, as an example, return <c>false</c>). The Unicode list is however valid input to the <seealso marker="stdlib:io#put_chars/2">io:put_chars</seealso> function, so data can be output on any Unicode capable device anyway. If the device is a terminal, characters will be output in the \x{H ...} format if encoding is <c>latin1</c> otherwise in UTF-8 (for the non-interactive terminal - "oldshell" or "noshell") or whatever is suitable to show the character properly (for an interactive terminal - the regular shell). The bottom line is that you can always send Unicode data to the <c>standard_io</c> device. Files will however only accept Unicode codepoints beyond ISO-latin-1 if <c>encoding</c> is set to something else than <c>latin1</c>.</p> +<p>The Unicode string is returned as a Unicode list, which is recognized as such since the Erlang shell uses the Unicode encoding. The Unicode list is valid input to the <seealso marker="stdlib:io#put_chars/2">io:put_chars/2</seealso> function, so data can be output on any Unicode capable device. If the device is a terminal, characters will be output in the <c>\x{</c>H ...<c>}</c> format if encoding is <c>latin1</c> otherwise in UTF-8 (for the non-interactive terminal - "oldshell" or "noshell") or whatever is suitable to show the character properly (for an interactive terminal - the regular shell). The bottom line is that you can always send Unicode data to the <c>standard_io</c> device. Files will however only accept Unicode codepoints beyond ISO-latin-1 if <c>encoding</c> is set to something else than <c>latin1</c>.</p> </section> <section> -<title>Heuristic identification of UTF-8</title> -<p>While it's strongly encouraged that the actual encoding of characters in binary data is known prior to processing, that is not always possible. On a typical Linux® system, there is a mix of UTF-8 and ISO-latin-1 text files and there are seldom any BOM's in the files to identify them.</p> +<title>Heuristic Identification of UTF-8</title> +<p>While it iss strongly encouraged that the actual encoding of characters in binary data is known prior to processing, that is not always possible. On a typical Linux® system, there is a mix of UTF-8 and ISO-latin-1 text files and there are seldom any BOM's in the files to identify them.</p> <p>UTF-8 is designed in such a way that ISO-latin-1 characters with numbers beyond the 7-bit ASCII range are seldom considered valid when decoded as UTF-8. Therefore one can usually use heuristics to determine if a file is in UTF-8 or if it is encoded in ISO-latin-1 (one byte per character) encoding. The <c>unicode</c> module can be used to determine if data can be interpreted as UTF-8:</p> <code> heuristic_encoding_bin(Bin) when is_binary(Bin) -> diff --git a/lib/stdlib/doc/src/ushell1.gif b/lib/stdlib/doc/src/ushell1.gif Binary files differdeleted file mode 100644 index 7c46464fd2..0000000000 --- a/lib/stdlib/doc/src/ushell1.gif +++ /dev/null diff --git a/lib/stdlib/doc/src/ushell1.ps b/lib/stdlib/doc/src/ushell1.ps deleted file mode 100644 index 95bfebf194..0000000000 --- a/lib/stdlib/doc/src/ushell1.ps +++ /dev/null @@ -1,1196 +0,0 @@ -%!PS-Adobe-3.0 -%%Creator: GIMP PostScript file plugin V 1.17 by Peter Kirchgessner -%%Title: ushell1.ps -%%CreationDate: Mon Mar 16 09:53:27 2009 -%%DocumentData: Clean7Bit -%%LanguageLevel: 2 -%%Pages: 1 -%%BoundingBox: 14 14 468 142 -%%EndComments -%%BeginProlog -% Use own dictionary to avoid conflicts -10 dict begin -%%EndProlog -%%Page: 1 1 -% Translate for offset -14.173228346456694 14.173228346456694 translate -% Translate to begin of first scanline -0 127.55905511811024 translate -453.54330708661422 -127.55905511811024 scale -% Image geometry -640 180 8 -% Transformation matrix -[ 640 0 0 180 0 0 ] -% Strings to hold RGB-samples per scanline -/rstr 640 string def -/gstr 640 string def -/bstr 640 string def -{currentfile /ASCII85Decode filter /RunLengthDecode filter rstr readstring pop} -{currentfile /ASCII85Decode filter /RunLengthDecode filter gstr readstring pop} -{currentfile /ASCII85Decode filter /RunLengthDecode filter bstr readstring pop} -true 3 -%%BeginData: 67571 ASCII Bytes -colorimage -JP1PeJP1PeJP1L~> -JP1PeJP1PeJP1L~> -JP1PeJP1PeJP1L~> -!!e'9JNA?CJNABDJ,~> -!!e'9JNA?CJNABDJ,~> -!!e'9JNA?CJNABDJ,~> -!$-XjJcC<$JcC?%J,~> -!$-XjJcC<$JcC?%J,~> -!$-XjJcC<$JcC?%J,~> -!$-XjJcC<$JcC?%J,~> -!$-XjJcC<$JcC?%J,~> -!$-XjJcC<$JcC?%J,~> -!$2%<!Lr-2rr]6=Dt\b^JcC<$JcD\KJ,~> -!$2%<!Lr-2rr]6=Dt\b^JcC<$JcD\KJ,~> -!$2%<!Lr-2rr]6=Dt\b^JcC<$JcD\KJ,~> -#9EjEY>Q6)=6BMbc`J;XJcC<$JcC<$W;hA~> -#9EjEY>Q6)=6BMbc`J;XJcC<$JcC<$W;hA~> -#9EjEY>Q6)=6BMbc`J;XJcC<$JcC<$W;hA~> -#T`rUH-/@H`Dbmu!A3bks+13$s+13Ks*t~> -#T`rUH-/@H`Dbmu!A3bks+13$s+13Ks*t~> -#T`rUH-/@H`Dbmu!A3bks+13$s+13Ks*t~> -#T`rEG5Y4]rp0@Z!W@cP*=4P!s-oDP9H?1krVlmMmt(Lis+13$s/>sJ~> -#T`rEG5Y4]rp0@Z!W@cP*=4P!s-oDP9H?1krVlmMmt(Lis+13$s/>sJ~> -#T`rEG5Y4]rp0@Z!W@cP*=4P!s-oDP9H?1krVlmMmt(Lis+13$s/>sJ~> -#9EjER>jY@r:g3lQA5D=$0F2Hs)Ak%s4op<rrF_?JcC<$JcC<$W;hA~> -#9EjER>jY@r:g3lQA5D=$0F2Hs)Ak%s4op<rrF_?JcC<$JcC<$W;hA~> -#9EjER>jY@r:g3lQA5D=$0F2Hs)Ak%s4op<rrF_?JcC<$JcC<$W;hA~> -!$2(="-F9Q\G-"(Q=^'2$*?/ds)G.?s5e.srrF_?JcC<$JcC<$W;hA~> -!$2(="-F9Q\G-"(Q=^'2$*?/ds)G.?s5e.srrF_?JcC<$JcC<$W;hA~> -!$2(="-F9Q\G-"(Q=^'2$*?/ds)G.?s5e.srrF_?JcC<$JcC<$W;hA~> -!$2%<"&I0WZ2">"Q8&8B"FpIOENK!9!A3bks+13$s+13Ks*t~> -!$2%<"&I0WZ2">"Q8&8B"FpIOENK!9!A3bks+13$s+13Ks*t~> -!$2%<"&I0WZ2">"Q8&8B"FpIOENK!9!A3bks+13$s+13Ks*t~> -#T`rq^Ah!RpEo\8!LAK:rrI,@q>UIImt(Lis+13$s/>sJ~> -#T`rq^Ah!RpEo\8!LAK:rrI,@q>UIImt(Lis+13$s/>sJ~> -#T`rq^Ah!RpEo\8!LAK:rrI,@q>UIImt(Lis+13$s/>sJ~> -#T`s"JSH?.CSgh+!RuStC&iMXrrI,@qu6esC(k*/JcC<$JcC<$WW.J~> -#T`s"JSH?.CSgh+!RuStC&iMXrrI,@qu6esC(k*/JcC<$JcC<$WW.J~> -#T`s"JSH?.CSgh+!RuStC&iMXrrI,@qu6esC(k*/JcC<$JcC<$WW.J~> -#9EjEgp/o#[J'V#fD`&U])M^1eE6Z.!7LkP!5F*bJcC<$JcD_LJ,~> -#9EjEgp/o#[J'V#fD`&U])M^1eE6Z.!7LkP!5F*bJcC<$JcD_LJ,~> -#9EjEgp/o#[J'V#fD`&U])M^1eE6Z.!7LkP!5F*bJcC<$JcD_LJ,~> -!$2%<!T*O$s+13$s+13+s*t~> -!$2%<!T*O$s+13$s+13+s*t~> -!$2%<!T*O$s+13$s+13+s*t~> -!$-XjJcC<$JcC?%J,~> -!$-XjJcC<$JcC?%J,~> -!$-XjJcC<$JcC?%J,~> -!$.X1!VbZgrrMcToD\g]rn%5Fci4"4rn%5Kp&>-Vf\#!)rr_95f\lE%"5DJ\kK!T&r;P=KmIgJZ -p&<SDo\]ZKoD[ABp@\Fcm/GW;rJ1CTgXt0Ap&>$hrn%5=kPp&~> -!$.X1!VbZgrrMcToD\g]rn%5Fci4"4rn%5Kp&>-Vf\#!)rr_95f\lE%"5DJ\kK!T&r;P=KmIgJZ -p&<SDo\]ZKoD[ABp@\Fcm/GW;rJ1CTgXt0Ap&>$hrn%5=kPp&~> -!$.X1!VbZgrrMcToD\g]rn%5Fci4"4rn%5Kp&>-Vf\#!)rr_95f\lE%"5DJ\kK!T&r;P=KmIgJZ -p&<SDo\]ZKoD[ABp@\Fcm/GW;rJ1CTgXt0Ap&>$hrn%5=kPp&~> -!Zh=)r4;sJp\t?M]=[dnrrC^M])^+RrrBt8rrCOG])^O`rrN,;oD\mfUOUkp!5JHD!:K[_!<2l- -"nM]nb];P3rrf;C6q#_Srrf>i6k8JSrrZO_6rs9d"4Ffn`;BQ8r5SL2!SZc5rrL6To`##66p3I# -rrWs'7#p%%s5ESL#i+SBs6i?D-]@Zt"M5R?7*O]k#gN&Bs2.5m(WQ.J"Fqsi7/+e&!SceorrM<; -mf3=!!7LiG"&[email protected])jpjdIXT/=<lMlA~> -!Zh=)r4;sJp\t?M]=[dnrrC^M])^+RrrBt8rrCOG])^O`rrN,;oD\mfUOUkp!5JHD!:K[_!<2l- -"nM]nb];P3rrf;C6q#_Srrf>i6k8JSrrZO_6rs9d"4Ffn`;BQ8r5SL2!SZc5rrL6To`##66p3I# -rrWs'7#p%%s5ESL#i+SBs6i?D-]@Zt"M5R?7*O]k#gN&Bs2.5m(WQ.J"Fqsi7/+e&!SceorrM<; -mf3=!!7LiG"&[email protected])jpjdIXT/=<lMlA~> -!Zh=)r4;sJp\t?M]=[dnrrC^M])^+RrrBt8rrCOG])^O`rrN,;oD\mfUOUkp!5JHD!:K[_!<2l- -"nM]nb];P3rrf;C6q#_Srrf>i6k8JSrrZO_6rs9d"4Ffn`;BQ8r5SL2!SZc5rrL6To`##66p3I# -rrWs'7#p%%s5ESL#i+SBs6i?D-]@Zt"M5R?7*O]k#gN&Bs2.5m(WQ.J"Fqsi7/+e&!SceorrM<; -mf3=!!7LiG"&[email protected])jpjdIXT/=<lMlA~> -"!.E>/H&uWGk_4?XC2t&hZ!Um6iLKpEPqeU!Nm==rrAJc@0&Z7rr3%sBt![6!U2E,rrMo!m/I&" -r*]TWq>UHor*]UHci3qFSUgA4!NC.^rrFq?p&>%u_#438a`V$#Fo21>SLO=5!Hk=9rrIX^qu6]" -qYL3mju2l3!$0bm!oI+@o`#!G9q_Fa!Ki<5rrK`?iVrtr.,4\"!ER55rrIY@bPqUQnGW@fGGa6s -!Mju/rrf?1@<TfkrrKf@p&>'T2Z3RT*U<ZT~> -"!.E>/H&uWGk_4?XC2t&hZ!Um6iLKpEPqeU!Nm==rrAJc@0&Z7rr3%sBt![6!U2E,rrMo!m/I&" -r*]TWq>UHor*]UHci3qFSUgA4!NC.^rrFq?p&>%u_#438a`V$#Fo21>SLO=5!Hk=9rrIX^qu6]" -qYL3mju2l3!$0bm!oI+@o`#!G9q_Fa!Ki<5rrK`?iVrtr.,4\"!ER55rrIY@bPqUQnGW@fGGa6s -!Mju/rrf?1@<TfkrrKf@p&>'T2Z3RT*U<ZT~> -"!.E>/H&uWGk_4?XC2t&hZ!Um6iLKpEPqeU!Nm==rrAJc@0&Z7rr3%sBt![6!U2E,rrMo!m/I&" -r*]TWq>UHor*]UHci3qFSUgA4!NC.^rrFq?p&>%u_#438a`V$#Fo21>SLO=5!Hk=9rrIX^qu6]" -qYL3mju2l3!$0bm!oI+@o`#!G9q_Fa!Ki<5rrK`?iVrtr.,4\"!ER55rrIY@bPqUQnGW@fGGa6s -!Mju/rrf?1@<TfkrrKf@p&>'T2Z3RT*U<ZT~> -"!.E>Fn#D352P/r!Go">rs$)Ds8SmD(]"(;jFF`>s."Z-J$8PF!JQp-rrML@m/I&+mf*?sJ*6h2 -!mQ8Ap&>&lF34F_1[4T4!F<M>rrU8keG9+GR!:(SrrI8?nc&V>o(r@eju2l3!$0en"6=u%O7iMT -gjhP\r;Qe[MtR)N]4'_!"297-g\h'P<lXh4!J$`arrGO?rVlnBVV_=RUj2D/#+pAEs3lJ>rVlo3 [email protected]#ju3/;!$1A)J,~> -"!.E>Fn#D352P/r!Go">rs$)Ds8SmD(]"(;jFF`>s."Z-J$8PF!JQp-rrML@m/I&+mf*?sJ*6h2 -!mQ8Ap&>&lF34F_1[4T4!F<M>rrU8keG9+GR!:(SrrI8?nc&V>o(r@eju2l3!$0en"6=u%O7iMT -gjhP\r;Qe[MtR)N]4'_!"297-g\h'P<lXh4!J$`arrGO?rVlnBVV_=RUj2D/#+pAEs3lJ>rVlo3 [email protected]#ju3/;!$1A)J,~> -"!.E>Fn#D352P/r!Go">rs$)Ds8SmD(]"(;jFF`>s."Z-J$8PF!JQp-rrML@m/I&+mf*?sJ*6h2 -!mQ8Ap&>&lF34F_1[4T4!F<M>rrU8keG9+GR!:(SrrI8?nc&V>o(r@eju2l3!$0en"6=u%O7iMT -gjhP\r;Qe[MtR)N]4'_!"297-g\h'P<lXh4!J$`arrGO?rVlnBVV_=RUj2D/#+pAEs3lJ>rVlo3 [email protected]#ju3/;!$1A)J,~> -"!.E>Fo21B`NfH(A8hDG!C#B=rrB%tA-dJ\s0]Y*A7U/grro!)A9-Unq#:A2Yl=Y,]4(`5N:.e` -rsS:7]`8"tO7rV9LTgFO!$1t:!-S9O&?Pp3K(J/9A?l17I9dLsA@MR<!+u4@!5/(+!?L;$A,sX@ -rrHo@r;QeISb<!`WH8";!8%0["6KR_df!\YlMob-rr3C]n,M+2s.s7:o`+s/rF?$+s8Th3A,uW, -rrFq?p&>%u_#=96O,`u8"-Sf5.JNiGFf1%+A;L6b!gR;1rVm%pA.`0ZP5P=\ju2l3!$1t:!.4]U% -&a.*Lh(\`A?>h2KC/YK!3uM&!omgerVm#hE4h0krVlrPA?Pn2"QaaBIA$N<!Ki<5rrK`?rVm2rJ -cEJqAD7(`h>Updc%Y]errTT=df'1K_1:6*g\h'P<lXh4!J$a=rrBe3A-;i2s1/,>"DIi8`ioCA# -^67J_6fbDA97bM!36$/!-n8<'sV9g<Fg^]s*(>sA7UJqs/40BEb(..rrA/[A,qbkrr@uVA,qqpr -s4?iA;[1-s8RZLA,r>%rrVe&PlC[`45p2=!K3->rrKf@p&>'T2Z3RT*W?!?Us]5;rF?8Ss8R9Bc -X^S/s8RT~> -"!.E>Fo21B`NfH(A8hDG!C#B=rrB%tA-dJ\s0]Y*A7U/grro!)A9-Unq#:A2Yl=Y,]4(`5N:.e` -rsS:7]`8"tO7rV9LTgFO!$1t:!-S9O&?Pp3K(J/9A?l17I9dLsA@MR<!+u4@!5/(+!?L;$A,sX@ -rrHo@r;QeISb<!`WH8";!8%0["6KR_df!\YlMob-rr3C]n,M+2s.s7:o`+s/rF?$+s8Th3A,uW, -rrFq?p&>%u_#=96O,`u8"-Sf5.JNiGFf1%+A;L6b!gR;1rVm%pA.`0ZP5P=\ju2l3!$1t:!.4]U% -&a.*Lh(\`A?>h2KC/YK!3uM&!omgerVm#hE4h0krVlrPA?Pn2"QaaBIA$N<!Ki<5rrK`?rVm2rJ -cEJqAD7(`h>Updc%Y]errTT=df'1K_1:6*g\h'P<lXh4!J$a=rrBe3A-;i2s1/,>"DIi8`ioCA# -^67J_6fbDA97bM!36$/!-n8<'sV9g<Fg^]s*(>sA7UJqs/40BEb(..rrA/[A,qbkrr@uVA,qqpr -s4?iA;[1-s8RZLA,r>%rrVe&PlC[`45p2=!K3->rrKf@p&>'T2Z3RT*W?!?Us]5;rF?8Ss8R9Bc -X^S/s8RT~> -"!.E>Fo21B`NfH(A8hDG!C#B=rrB%tA-dJ\s0]Y*A7U/grro!)A9-Unq#:A2Yl=Y,]4(`5N:.e` -rsS:7]`8"tO7rV9LTgFO!$1t:!-S9O&?Pp3K(J/9A?l17I9dLsA@MR<!+u4@!5/(+!?L;$A,sX@ -rrHo@r;QeISb<!`WH8";!8%0["6KR_df!\YlMob-rr3C]n,M+2s.s7:o`+s/rF?$+s8Th3A,uW, -rrFq?p&>%u_#=96O,`u8"-Sf5.JNiGFf1%+A;L6b!gR;1rVm%pA.`0ZP5P=\ju2l3!$1t:!.4]U% -&a.*Lh(\`A?>h2KC/YK!3uM&!omgerVm#hE4h0krVlrPA?Pn2"QaaBIA$N<!Ki<5rrK`?rVm2rJ -cEJqAD7(`h>Updc%Y]errTT=df'1K_1:6*g\h'P<lXh4!J$a=rrBe3A-;i2s1/,>"DIi8`ioCA# -^67J_6fbDA97bM!36$/!-n8<'sV9g<Fg^]s*(>sA7UJqs/40BEb(..rrA/[A,qbkrr@uVA,qqpr -s4?iA;[1-s8RZLA,r>%rrVe&PlC[`45p2=!K3->rrKf@p&>'T2Z3RT*W?!?Us]5;rF?8Ss8R9Bc -X^S/s8RT~> -"!.E>+oY"=nGiOLJ4Q':Y\F(9!C#B=rrC[M[h5T^s+Mqh\$r2\s.PF]\#s&_q#:A21B'fKA^g\. -li/"%rrth%K`D'Nb5VDAU3cP5!$2";!HG1>[hOJ#s2@f;\$pmts21i\\$t2ns'E/5[fL`lq#:@. ->Q2)4ER=CY!i_%@r;QeISb<!`WH8%<!Tdkj[f]k-he;tt"'>BB;Z?\)39C2.O@Y5;[oi[TF8`QM ->5uWerO2c3r;QdRl1P&W?,6F="cnXr84[+erreOOo(+=)rsNs&T!u2#L]@D"[ho#C"k:"&Y.)XQ -rrM7?o`"n3r;QeBVuF.6Lo^P:UT.b3KX(P47fKjtJ@GOX!mgp]rVm"#a8_-[rVlr7!.+YE"nc'C -s*^R<rrJ7?p&>')@K$34qP-T)\$ol2lsTh%!c/D1rr3%Q!4Dh+"j;#Cs$>E;rrH0?p&>&CSc&Kf -iVg4f>lVWdrjN$<@K3$Jrr3BIs8SH4NjlL"D?$enrjMp-F85bQgUAo,\&JCIG;#Mr\"8?Sen[gZ -\!r?VOdQ-W"IG;Os2k8L%$R(YKV,`_S1]'dI\ZnO!K<iRrrVKd:]C@p45p2=!K3->rrKf@p&>'T -2Z3R\*WQ/Q8;ZX3=oGf:EmOcj88>WCDUnc;~> -"!.E>+oY"=nGiOLJ4Q':Y\F(9!C#B=rrC[M[h5T^s+Mqh\$r2\s.PF]\#s&_q#:A21B'fKA^g\. -li/"%rrth%K`D'Nb5VDAU3cP5!$2";!HG1>[hOJ#s2@f;\$pmts21i\\$t2ns'E/5[fL`lq#:@. ->Q2)4ER=CY!i_%@r;QeISb<!`WH8%<!Tdkj[f]k-he;tt"'>BB;Z?\)39C2.O@Y5;[oi[TF8`QM ->5uWerO2c3r;QdRl1P&W?,6F="cnXr84[+erreOOo(+=)rsNs&T!u2#L]@D"[ho#C"k:"&Y.)XQ -rrM7?o`"n3r;QeBVuF.6Lo^P:UT.b3KX(P47fKjtJ@GOX!mgp]rVm"#a8_-[rVlr7!.+YE"nc'C -s*^R<rrJ7?p&>')@K$34qP-T)\$ol2lsTh%!c/D1rr3%Q!4Dh+"j;#Cs$>E;rrH0?p&>&CSc&Kf -iVg4f>lVWdrjN$<@K3$Jrr3BIs8SH4NjlL"D?$enrjMp-F85bQgUAo,\&JCIG;#Mr\"8?Sen[gZ -\!r?VOdQ-W"IG;Os2k8L%$R(YKV,`_S1]'dI\ZnO!K<iRrrVKd:]C@p45p2=!K3->rrKf@p&>'T -2Z3R\*WQ/Q8;ZX3=oGf:EmOcj88>WCDUnc;~> -"!.E>+oY"=nGiOLJ4Q':Y\F(9!C#B=rrC[M[h5T^s+Mqh\$r2\s.PF]\#s&_q#:A21B'fKA^g\. -li/"%rrth%K`D'Nb5VDAU3cP5!$2";!HG1>[hOJ#s2@f;\$pmts21i\\$t2ns'E/5[fL`lq#:@. ->Q2)4ER=CY!i_%@r;QeISb<!`WH8%<!Tdkj[f]k-he;tt"'>BB;Z?\)39C2.O@Y5;[oi[TF8`QM ->5uWerO2c3r;QdRl1P&W?,6F="cnXr84[+erreOOo(+=)rsNs&T!u2#L]@D"[ho#C"k:"&Y.)XQ -rrM7?o`"n3r;QeBVuF.6Lo^P:UT.b3KX(P47fKjtJ@GOX!mgp]rVm"#a8_-[rVlr7!.+YE"nc'C -s*^R<rrJ7?p&>')@K$34qP-T)\$ol2lsTh%!c/D1rr3%Q!4Dh+"j;#Cs$>E;rrH0?p&>&CSc&Kf -iVg4f>lVWdrjN$<@K3$Jrr3BIs8SH4NjlL"D?$enrjMp-F85bQgUAo,\&JCIG;#Mr\"8?Sen[gZ -\!r?VOdQ-W"IG;Os2k8L%$R(YKV,`_S1]'dI\ZnO!K<iRrrVKd:]C@p45p2=!K3->rrKf@p&>'T -2Z3R\*WQ/Q8;ZX3=oGf:EmOcj88>WCDUnc;~> -"!.E>@K+aVr;Q`rJHPNDmOnJ<!C#B9rt+!Ns+Ppms8UV?s*^O>s62?6rrm5&e/6]nr;Qa;qYpcM -e=;Er;p,+>!MXo6rr=);rrG.?rr3F`PlLb'g].;(SH&WV0`:qO+T;<>"6]4S55Zo5_Z/6Crr3"* -^@hL,Lm7f:!Ip[5rrK*?rVloQ62gfch#5\orVlsEq8uV7rsUmKs8Sm*mJm2,s1)Y<rr^sSZ#'C= -!$2";!Aj!5rrHE@rVmFi(nB+*^0^i/Nq35A.KBF4r$r%hs)j7ms8U&>rVlj<qu6[Ho(r@eju2l3 -!$2";!CPT?rs\;\s#T/u49(2%s"_RmrrI\?rr3&oeE6c1"\.)Cs,E*<rrViBl2L\d6/2G>IA$N< -!Ki<5rrK`?r;R&L5j&+H4l>?\rr3&Z/Ed$4$gQ75s8UhXjo>?Hg\h'P<lXh4!J$a8rrX;AWH8(= -"TI-TTmQe=$m#BJQu;Bms6)??OGs2="8^pTS,=c:AcD]17+hJ<&U0-*s8U)>s8Prqqu=?:s(8b> -rrKN?qu7)+E;rqZs8VbKGlLIarr3#elMgebr7'^)rrG4?rr3"WP5YC]]jUO5!Tl<<rs0Y*.qmH" -s8O,<rsE/Hs6;)ns8Q3>s*t~> -"!.E>@K+aVr;Q`rJHPNDmOnJ<!C#B9rt+!Ns+Ppms8UV?s*^O>s62?6rrm5&e/6]nr;Qa;qYpcM -e=;Er;p,+>!MXo6rr=);rrG.?rr3F`PlLb'g].;(SH&WV0`:qO+T;<>"6]4S55Zo5_Z/6Crr3"* -^@hL,Lm7f:!Ip[5rrK*?rVloQ62gfch#5\orVlsEq8uV7rsUmKs8Sm*mJm2,s1)Y<rr^sSZ#'C= -!$2";!Aj!5rrHE@rVmFi(nB+*^0^i/Nq35A.KBF4r$r%hs)j7ms8U&>rVlj<qu6[Ho(r@eju2l3 -!$2";!CPT?rs\;\s#T/u49(2%s"_RmrrI\?rr3&oeE6c1"\.)Cs,E*<rrViBl2L\d6/2G>IA$N< -!Ki<5rrK`?r;R&L5j&+H4l>?\rr3&Z/Ed$4$gQ75s8UhXjo>?Hg\h'P<lXh4!J$a8rrX;AWH8(= -"TI-TTmQe=$m#BJQu;Bms6)??OGs2="8^pTS,=c:AcD]17+hJ<&U0-*s8U)>s8Prqqu=?:s(8b> -rrKN?qu7)+E;rqZs8VbKGlLIarr3#elMgebr7'^)rrG4?rr3"WP5YC]]jUO5!Tl<<rs0Y*.qmH" -s8O,<rsE/Hs6;)ns8Q3>s*t~> -"!.E>@K+aVr;Q`rJHPNDmOnJ<!C#B9rt+!Ns+Ppms8UV?s*^O>s62?6rrm5&e/6]nr;Qa;qYpcM -e=;Er;p,+>!MXo6rr=);rrG.?rr3F`PlLb'g].;(SH&WV0`:qO+T;<>"6]4S55Zo5_Z/6Crr3"* -^@hL,Lm7f:!Ip[5rrK*?rVloQ62gfch#5\orVlsEq8uV7rsUmKs8Sm*mJm2,s1)Y<rr^sSZ#'C= -!$2";!Aj!5rrHE@rVmFi(nB+*^0^i/Nq35A.KBF4r$r%hs)j7ms8U&>rVlj<qu6[Ho(r@eju2l3 -!$2";!CPT?rs\;\s#T/u49(2%s"_RmrrI\?rr3&oeE6c1"\.)Cs,E*<rrViBl2L\d6/2G>IA$N< -!Ki<5rrK`?r;R&L5j&+H4l>?\rr3&Z/Ed$4$gQ75s8UhXjo>?Hg\h'P<lXh4!J$a8rrX;AWH8(= -"TI-TTmQe=$m#BJQu;Bms6)??OGs2="8^pTS,=c:AcD]17+hJ<&U0-*s8U)>s8Prqqu=?:s(8b> -rrKN?qu7)+E;rqZs8VbKGlLIarr3#elMgebr7'^)rrG4?rr3"WP5YC]]jUO5!Tl<<rs0Y*.qmH" -s8O,<rsE/Hs6;)ns8Q3>s*t~> -"!.E>Fo)+AJXc]>rq-0h!C#B>rrMn@rZ)+Z5Q?G(rr38S8H4C6nF56up\tD5YlF#Po_e^h*VfX; -WcJ,<(]GEU&pj9O!$2";!BD(t*<[.Hs8O,=rrr2os8VI?r;Qfh+9!8_PktFNr:'daqu6\'^@qR. -Wd+@:rrIV?p&>&lF8c+>rO`"K"3gbn9)\bl,PfJ[rr3-]jo>@VGlI^FoDc@2qu6]%%/h1H.fB;I -1[4T4!F<M>rt(6GqZ$TP56$WZ*??+'>Q=KrnH8IaFf56=!QA.=rr=):rrFV?qYpTY2Y@"L*W5p< -ofW3o%!VLH3o]*[s,*$?1@+r>!J-a8rr=\N*WHZN[eTk&+9!8^%MHbZrrJ7?p&>')@Jp-+cqXN> -"82WS3;rjX2<Xf8!S.q`*<HH`q>L<o<lXh4!J$a>rrMt`r>btZs8%lW*<[SWs.Of=rraABs-SK= -rrhOCs,`3:rrDlmnGr7]rrGO?rVlnBVZ-T(``E->;9](?qtC&%(&f3V)0#WK;Z7[>'QF(PaSu2B -Uj2q47/e2-Dts,-!B]9>rrJ%@[email protected]#ju3/;!ua,Xm/?qa')`gR'Ysb61B.:TpAFr@~> -"!.E>Fo)+AJXc]>rq-0h!C#B>rrMn@rZ)+Z5Q?G(rr38S8H4C6nF56up\tD5YlF#Po_e^h*VfX; -WcJ,<(]GEU&pj9O!$2";!BD(t*<[.Hs8O,=rrr2os8VI?r;Qfh+9!8_PktFNr:'daqu6\'^@qR. -Wd+@:rrIV?p&>&lF8c+>rO`"K"3gbn9)\bl,PfJ[rr3-]jo>@VGlI^FoDc@2qu6]%%/h1H.fB;I -1[4T4!F<M>rt(6GqZ$TP56$WZ*??+'>Q=KrnH8IaFf56=!QA.=rr=):rrFV?qYpTY2Y@"L*W5p< -ofW3o%!VLH3o]*[s,*$?1@+r>!J-a8rr=\N*WHZN[eTk&+9!8^%MHbZrrJ7?p&>')@Jp-+cqXN> -"82WS3;rjX2<Xf8!S.q`*<HH`q>L<o<lXh4!J$a>rrMt`r>btZs8%lW*<[SWs.Of=rraABs-SK= -rrhOCs,`3:rrDlmnGr7]rrGO?rVlnBVZ-T(``E->;9](?qtC&%(&f3V)0#WK;Z7[>'QF(PaSu2B -Uj2q47/e2-Dts,-!B]9>rrJ%@[email protected]#ju3/;!ua,Xm/?qa')`gR'Ysb61B.:TpAFr@~> -"!.E>Fo)+AJXc]>rq-0h!C#B>rrMn@rZ)+Z5Q?G(rr38S8H4C6nF56up\tD5YlF#Po_e^h*VfX; -WcJ,<(]GEU&pj9O!$2";!BD(t*<[.Hs8O,=rrr2os8VI?r;Qfh+9!8_PktFNr:'daqu6\'^@qR. -Wd+@:rrIV?p&>&lF8c+>rO`"K"3gbn9)\bl,PfJ[rr3-]jo>@VGlI^FoDc@2qu6]%%/h1H.fB;I -1[4T4!F<M>rt(6GqZ$TP56$WZ*??+'>Q=KrnH8IaFf56=!QA.=rr=):rrFV?qYpTY2Y@"L*W5p< -ofW3o%!VLH3o]*[s,*$?1@+r>!J-a8rr=\N*WHZN[eTk&+9!8^%MHbZrrJ7?p&>')@Jp-+cqXN> -"82WS3;rjX2<Xf8!S.q`*<HH`q>L<o<lXh4!J$a>rrMt`r>btZs8%lW*<[SWs.Of=rraABs-SK= -rrhOCs,`3:rrDlmnGr7]rrGO?rVlnBVZ-T(``E->;9](?qtC&%(&f3V)0#WK;Z7[>'QF(PaSu2B -Uj2q47/e2-Dts,-!B]9>rrJ%@[email protected]#ju3/;!ua,Xm/?qa')`gR'Ysb61B.:TpAFr@~> -"!.E>Fo)+=JXcK8!C#B>s8S,ZrrrD25Q?G(rr35R8H7pi/1a!Yrs!;Ds8UtYjSf)Y*W?!=j7N?N -"KHMB;p,+>!MXo6rrGC?rVlmYj8/cU*W#d@m4eS?kjSQ)rVln:Xn_nrf_tgN?G?F=!qFb*rVlrl -R#h.E!q(Q@p&>&lF8c+>q;2)M""Woj9)\bl,PfJ[rr3-]jo>@VGl7RB\RYU<"5*XYD"mr11[4T4 -!F<M>rrJ1?rr3#U55bE]o-sG6#'Ggrs8U&>rVlj<qu6dKo)J:BrVlo\2Y@"L*W5p<htd9O%$HMJ -3o]*[s,*$?1;s1l!J-a>rrVrDjneuXNK=&<!qat*qYpSET`"fjOc/o4!P;e<rrLJ@r;QfZ3<&pZ -i@O0krrVK7o(r@e6/2>;!ER55rrIY@rVlo(C]=>:fH("]kPkJiq'?!6HiO-#)uor*K`:uSkV`C% -N;ihXqVLrG#=R5EpYc'qVZ-T!``E->;9\t<!G8h<rt/POs8VeRE;rqZs8V_IGlQ^rrr3"hJc>ZN -r6sX(rrucDs8VSDV>^Dp]jUO5!Tl<<rs0Y+/83N"s8O,9rrMC?qu;0~> -"!.E>Fo)+=JXcK8!C#B>s8S,ZrrrD25Q?G(rr35R8H7pi/1a!Yrs!;Ds8UtYjSf)Y*W?!=j7N?N -"KHMB;p,+>!MXo6rrGC?rVlmYj8/cU*W#d@m4eS?kjSQ)rVln:Xn_nrf_tgN?G?F=!qFb*rVlrl -R#h.E!q(Q@p&>&lF8c+>q;2)M""Woj9)\bl,PfJ[rr3-]jo>@VGl7RB\RYU<"5*XYD"mr11[4T4 -!F<M>rrJ1?rr3#U55bE]o-sG6#'Ggrs8U&>rVlj<qu6dKo)J:BrVlo\2Y@"L*W5p<htd9O%$HMJ -3o]*[s,*$?1;s1l!J-a>rrVrDjneuXNK=&<!qat*qYpSET`"fjOc/o4!P;e<rrLJ@r;QfZ3<&pZ -i@O0krrVK7o(r@e6/2>;!ER55rrIY@rVlo(C]=>:fH("]kPkJiq'?!6HiO-#)uor*K`:uSkV`C% -N;ihXqVLrG#=R5EpYc'qVZ-T!``E->;9\t<!G8h<rt/POs8VeRE;rqZs8V_IGlQ^rrr3"hJc>ZN -r6sX(rrucDs8VSDV>^Dp]jUO5!Tl<<rs0Y+/83N"s8O,9rrMC?qu;0~> -"!.E>Fo)+=JXcK8!C#B>s8S,ZrrrD25Q?G(rr35R8H7pi/1a!Yrs!;Ds8UtYjSf)Y*W?!=j7N?N -"KHMB;p,+>!MXo6rrGC?rVlmYj8/cU*W#d@m4eS?kjSQ)rVln:Xn_nrf_tgN?G?F=!qFb*rVlrl -R#h.E!q(Q@p&>&lF8c+>q;2)M""Woj9)\bl,PfJ[rr3-]jo>@VGl7RB\RYU<"5*XYD"mr11[4T4 -!F<M>rrJ1?rr3#U55bE]o-sG6#'Ggrs8U&>rVlj<qu6dKo)J:BrVlo\2Y@"L*W5p<htd9O%$HMJ -3o]*[s,*$?1;s1l!J-a>rrVrDjneuXNK=&<!qat*qYpSET`"fjOc/o4!P;e<rrLJ@r;QfZ3<&pZ -i@O0krrVK7o(r@e6/2>;!ER55rrIY@rVlo(C]=>:fH("]kPkJiq'?!6HiO-#)uor*K`:uSkV`C% -N;ihXqVLrG#=R5EpYc'qVZ-T!``E->;9\t<!G8h<rt/POs8VeRE;rqZs8V_IGlQ^rrr3"hJc>ZN -r6sX(rrucDs8VSDV>^Dp]jUO5!Tl<<rs0Y+/83N"s8O,9rrMC?qu;0~> -"!.E><;j6._>jOdS,<3sfsWK(^&S,8L:4Ot5<o1%Qi@!feO]_7NW-?d`V9B5Cp<p=%(fsJe$c\" -[^O]cMOa[S"Ho5R;p,+>!MXo6rrM@?rVln=WrBF,f`(mN*W#dAnm/]4I)#\h[JmT8GfBIX!*]?0 -!HHNdrrT>'N;`bW_HQg9!M+c5rrK*?rVlo[Ac9%>>P6lerO)f1qq5fb[KU*@s8Sm>r;QfBF8`NL ->Q;`frO2V'!Aj!5rrHE@rVlo%L&SL]W-/%<!V7c7rsa*)S[PttL]@D![M?6mrVm"5Z*os^rVlo\ -2Y@"L*W5p<Gc(JK%#:qa3o]FGs,*$?1.V>P!J%]ZrrUOIC&7i1NK=&<!l+e^qYpSET`"fjOc/o4 -!P;e<rrLJ@r;Qfa@K*\:C_,_.rrRiR[JKn(6/2>;!ER55rrIY@rVlo>GQ,#R<ZM.VHN(>]ZXWsI -I^Z[i'EA*"K`:uSkV`CEL&SL]WH@k6#C<5TTn35fVZ-T!``E->;9\t<!Ki`J[M6pbs,3AT[\#9n -s+R&Q[[]!qs*pdB[K2>`rr3&c!)NRn"CAOFOdu@L!PMn6rrM7?r;R$Cs8Tc(M<Y%DrO)jhs8V@> -qu;0~> -"!.E><;j6._>jOdS,<3sfsWK(^&S,8L:4Ot5<o1%Qi@!feO]_7NW-?d`V9B5Cp<p=%(fsJe$c\" -[^O]cMOa[S"Ho5R;p,+>!MXo6rrM@?rVln=WrBF,f`(mN*W#dAnm/]4I)#\h[JmT8GfBIX!*]?0 -!HHNdrrT>'N;`bW_HQg9!M+c5rrK*?rVlo[Ac9%>>P6lerO)f1qq5fb[KU*@s8Sm>r;QfBF8`NL ->Q;`frO2V'!Aj!5rrHE@rVlo%L&SL]W-/%<!V7c7rsa*)S[PttL]@D![M?6mrVm"5Z*os^rVlo\ -2Y@"L*W5p<Gc(JK%#:qa3o]FGs,*$?1.V>P!J%]ZrrUOIC&7i1NK=&<!l+e^qYpSET`"fjOc/o4 -!P;e<rrLJ@r;Qfa@K*\:C_,_.rrRiR[JKn(6/2>;!ER55rrIY@rVlo>GQ,#R<ZM.VHN(>]ZXWsI -I^Z[i'EA*"K`:uSkV`CEL&SL]WH@k6#C<5TTn35fVZ-T!``E->;9\t<!Ki`J[M6pbs,3AT[\#9n -s+R&Q[[]!qs*pdB[K2>`rr3&c!)NRn"CAOFOdu@L!PMn6rrM7?r;R$Cs8Tc(M<Y%DrO)jhs8V@> -qu;0~> -"!.E><;j6._>jOdS,<3sfsWK(^&S,8L:4Ot5<o1%Qi@!feO]_7NW-?d`V9B5Cp<p=%(fsJe$c\" -[^O]cMOa[S"Ho5R;p,+>!MXo6rrM@?rVln=WrBF,f`(mN*W#dAnm/]4I)#\h[JmT8GfBIX!*]?0 -!HHNdrrT>'N;`bW_HQg9!M+c5rrK*?rVlo[Ac9%>>P6lerO)f1qq5fb[KU*@s8Sm>r;QfBF8`NL ->Q;`frO2V'!Aj!5rrHE@rVlo%L&SL]W-/%<!V7c7rsa*)S[PttL]@D![M?6mrVm"5Z*os^rVlo\ -2Y@"L*W5p<Gc(JK%#:qa3o]FGs,*$?1.V>P!J%]ZrrUOIC&7i1NK=&<!l+e^qYpSET`"fjOc/o4 -!P;e<rrLJ@r;Qfa@K*\:C_,_.rrRiR[JKn(6/2>;!ER55rrIY@rVlo>GQ,#R<ZM.VHN(>]ZXWsI -I^Z[i'EA*"K`:uSkV`CEL&SL]WH@k6#C<5TTn35fVZ-T!``E->;9\t<!Ki`J[M6pbs,3AT[\#9n -s+R&Q[[]!qs*pdB[K2>`rr3&c!)NRn"CAOFOdu@L!PMn6rrM7?r;R$Cs8Tc(M<Y%DrO)jhs8V@> -qu;0~> -!Zh<`r+6(Zs8TIDqu6Z!ral.Qrr38'AnGcCN;p?%rr3,`OoNUMral1M\bQ1*VpPGC"4j.FUASU* -V>pRRral;$s8SGCrr3#>YP.ttno2/<rr@6AAcSt4rr@9=rr_7mB".d>!,):C!5/(+!;?A'!65!; -!mYDhrVloQSGW<fj+Xf2rs,MlR@3@?s4I9^"6KR_e,<k\lMpn0ral>jGQ7]bY5A5!_u40Lq>^K/ -rFQ<6s8UK7R31\drrZPJRA9c]!36$1!."[email protected];Qh8As<5o"1">P -2Y@"O(n$f.rr2tGral;2s8RrDrr3,2^AftMral/8rVlrZAu5A(!P?#CrrUkcYPS8)Z,ZhDp6h=M -MtR)U]/uFKj8]/>Pl(I\hYq*gc8FearrTT?e,'(MNTpKChjKlh`qB?:J6nY3qu?]2ral<"GQ7]S -ral.Frr383AnL-Fs8TpCrr3,kL&_1Rral.Uo`#,DAqU-`Xi^SB"l5XIs-DU?rrA2\AcS"nrt(-$ -AqnR0s8RjdArFa5s8R]MAcSS(rrVe(Q2L[^AcS:urr]!`EFAJ>#Nd.sRF;-8GQ%ODV:,D=rFQ2O -s8VYCqu;0~> -!Zh<`r+6(Zs8TIDqu6Z!ral.Qrr38'AnGcCN;p?%rr3,`OoNUMral1M\bQ1*VpPGC"4j.FUASU* -V>pRRral;$s8SGCrr3#>YP.ttno2/<rr@6AAcSt4rr@9=rr_7mB".d>!,):C!5/(+!;?A'!65!; -!mYDhrVloQSGW<fj+Xf2rs,MlR@3@?s4I9^"6KR_e,<k\lMpn0ral>jGQ7]bY5A5!_u40Lq>^K/ -rFQ<6s8UK7R31\drrZPJRA9c]!36$1!."[email protected];Qh8As<5o"1">P -2Y@"O(n$f.rr2tGral;2s8RrDrr3,2^AftMral/8rVlrZAu5A(!P?#CrrUkcYPS8)Z,ZhDp6h=M -MtR)U]/uFKj8]/>Pl(I\hYq*gc8FearrTT?e,'(MNTpKChjKlh`qB?:J6nY3qu?]2ral<"GQ7]S -ral.Frr383AnL-Fs8TpCrr3,kL&_1Rral.Uo`#,DAqU-`Xi^SB"l5XIs-DU?rrA2\AcS"nrt(-$ -AqnR0s8RjdArFa5s8R]MAcSS(rrVe(Q2L[^AcS:urr]!`EFAJ>#Nd.sRF;-8GQ%ODV:,D=rFQ2O -s8VYCqu;0~> -!Zh<`r+6(Zs8TIDqu6Z!ral.Qrr38'AnGcCN;p?%rr3,`OoNUMral1M\bQ1*VpPGC"4j.FUASU* -V>pRRral;$s8SGCrr3#>YP.ttno2/<rr@6AAcSt4rr@9=rr_7mB".d>!,):C!5/(+!;?A'!65!; -!mYDhrVloQSGW<fj+Xf2rs,MlR@3@?s4I9^"6KR_e,<k\lMpn0ral>jGQ7]bY5A5!_u40Lq>^K/ -rFQ<6s8UK7R31\drrZPJRA9c]!36$1!."[email protected];Qh8As<5o"1">P -2Y@"O(n$f.rr2tGral;2s8RrDrr3,2^AftMral/8rVlrZAu5A(!P?#CrrUkcYPS8)Z,ZhDp6h=M -MtR)U]/uFKj8]/>Pl(I\hYq*gc8FearrTT?e,'(MNTpKChjKlh`qB?:J6nY3qu?]2ral<"GQ7]S -ral.Frr383AnL-Fs8TpCrr3,kL&_1Rral.Uo`#,DAqU-`Xi^SB"l5XIs-DU?rrA2\AcS"nrt(-$ -AqnR0s8RjdArFa5s8R]MAcSS(rrVe(Q2L[^AcS:urr]!`EFAJ>#Nd.sRF;-8GQ%ODV:,D=rFQ2O -s8VYCqu;0~> -!$0_l!I^U>rrL>?c2Rh,WkJE5h6Z_Q!7h($!9VW-!65"j!;,sa"0dE1O3[b-Tn@ugo`##OK7gQ" -rrFn@g]%9Grdt3jp&>$Krdt4,o)A_JkOAKOfD^C&j7WEP_>]&eqXFLccb08W!$.[2"-%qcZ1\+s -oDX@BaQNSR~> -!$0_l!I^U>rrL>?c2Rh,WkJE5h6Z_Q!7h($!9VW-!65"j!;,sa"0dE1O3[b-Tn@ugo`##OK7gQ" -rrFn@g]%9Grdt3jp&>$Krdt4,o)A_JkOAKOfD^C&j7WEP_>]&eqXFLccb08W!$.[2"-%qcZ1\+s -oDX@BaQNSR~> -!$0_l!I^U>rrL>?c2Rh,WkJE5h6Z_Q!7h($!9VW-!65"j!;,sa"0dE1O3[b-Tn@ugo`##OK7gQ" -rrFn@g]%9Grdt3jp&>$Krdt4,o)A_JkOAKOfD^C&j7WEP_>]&eqXFLccb08W!$.[2"-%qcZ1\+s -oDX@BaQNSR~> -!$0_l!R>og?NFuWJcC<$dJj5&laHfo2<W*]!Si8*?N?dNs6ou<~> -!$0_l!R>og?NFuWJcC<$dJj5&laHfo2<W*]!Si8*?N?dNs6ou<~> -!$0_l!R>og?NFuWJcC<$dJj5&laHfo2<W*]!Si8*?N?dNs6ou<~> -!$0\k!7h(]!6TlmJcF*s!Qk2HrrL:<aSu7trk&7.JcG3=J,~> -!$0\k!7h(]!6TlmJcF*s!Qk2HrrL:<aSu7trk&7.JcG3=J,~> -!$0\k!7h(]!6TlmJcF*s!Qk2HrrL:<aSu7trk&7.JcG3=J,~> -!$..#"#_JQH%H!Hs+13$s7lVE~> -!$..#"#_JQH%H!Hs+13$s7lVE~> -!$..#"#_JQH%H!Hs+13$s7lVE~> -!$1P.",Is-hrXk>C)m`\rrQ[N'D)5,m2m?Wo_8@e73")Lrr`#hZsnUdJcC<$JcGNFJ,~> -!$1P.",Is-hrXk>C)m`\rrQ[N'D)5,m2m?Wo_8@e73")Lrr`#hZsnUdJcC<$JcGNFJ,~> -!$1P.",Is-hrXk>C)m`\rrQ[N'D)5,m2m?Wo_8@e73")Lrr`#hZsnUdJcC<$JcGNFJ,~> -!$1J,!C#B#rr=)9rr=)2rraJBs/L,5rrMX?lMgms@Y+Q1s+13$s7lVE~> -!$1J,!C#B#rr=)9rr=)2rraJBs/L,5rrMX?lMgms@Y+Q1s+13$s7lVE~> -!$1J,!C#B#rr=)9rr=)2rraJBs/L,5rrMX?lMgms@Y+Q1s+13$s7lVE~> -#9Ej*e^C_-[/U(*g&A5V[f$.+52Q#5"kqkTZ*D%BrrC@DYlMW<rr=)9rr=)9rrKOBr;R!Er;X^+ -s8Tq7YlN)JrrMX?r;Qc0rilIPrr2u0rilIRr;Qf0@Y+Q1s+13$s7lVE~> -#9Ej*e^C_-[/U(*g&A5V[f$.+52Q#5"kqkTZ*D%BrrC@DYlMW<rr=)9rr=)9rrKOBr;R!Er;X^+ -s8Tq7YlN)JrrMX?r;Qc0rilIPrr2u0rilIRr;Qf0@Y+Q1s+13$s7lVE~> -#9Ej*e^C_-[/U(*g&A5V[f$.+52Q#5"kqkTZ*D%BrrC@DYlMW<rr=)9rr=)9rrKOBr;R!Er;X^+ -s8Tq7YlN)JrrMX?r;Qc0rilIPrr2u0rilIRr;Qf0@Y+Q1s+13$s7lVE~> -$Q]8F7!rccCo%*_JGs<bDQ*O6!C#B6rs=B]GACu7ZN&$mrbDOU[f-4+*W#d9*W#d:$nqPY!?h=< -rr@rUCB8b%rr3#h/,fJKY]9YX"F\VrXDe)R!IiJqrrK`@JcC<$JcC<$q#>j~> -$Q]8F7!rccCo%*_JGs<bDQ*O6!C#B6rs=B]GACu7ZN&$mrbDOU[f-4+*W#d9*W#d:$nqPY!?h=< -rr@rUCB8b%rr3#h/,fJKY]9YX"F\VrXDe)R!IiJqrrK`@JcC<$JcC<$q#>j~> -$Q]8F7!rccCo%*_JGs<bDQ*O6!C#B6rs=B]GACu7ZN&$mrbDOU[f-4+*W#d9*W#d:$nqPY!?h=< -rr@rUCB8b%rr3#h/,fJKY]9YX"F\VrXDe)R!IiJqrrK`@JcC<$JcC<$q#>j~> -$Q]8F50X',pEop4IfB?JmOnJ<!C#B>rrBt7G70i=Ki$S)s4'[?I@pN=!R+C=rr=)9rr=)9rrJ+N -rr3,"G7Sk^q>UJiHN*pFnLOS<!CGQ?rrgQfs#K-=rrIq?rVlo1@Y+Q1s+13$s7lVE~> -$Q]8F50X',pEop4IfB?JmOnJ<!C#B>rrBt7G70i=Ki$S)s4'[?I@pN=!R+C=rr=)9rr=)9rrJ+N -rr3,"G7Sk^q>UJiHN*pFnLOS<!CGQ?rrgQfs#K-=rrIq?rVlo1@Y+Q1s+13$s7lVE~> -$Q]8F50X',pEop4IfB?JmOnJ<!C#B>rrBt7G70i=Ki$S)s4'[?I@pN=!R+C=rr=)9rr=)9rrJ+N -rr3,"G7Sk^q>UJiHN*pFnLOS<!CGQ?rrgQfs#K-=rrIq?rVlo1@Y+Q1s+13$s7lVE~> -"!.E>FoMCDpEop4/cJoS<%e.L!C#B>rrC[KV$"@0KpVf="P$'CI@pN=!R+C=rr=)9rr=)4rrg<; -'r/;;rr@KH=ogX0rr3#h/,fJK]Oh(G"JPkq3DocZ!AKc:rrK`@JcC<$JcC<$q#>j~> -"!.E>FoMCDpEop4/cJoS<%e.L!C#B>rrC[KV$"@0KpVf="P$'CI@pN=!R+C=rr=)9rr=)4rrg<; -'r/;;rr@KH=ogX0rr3#h/,fJK]Oh(G"JPkq3DocZ!AKc:rrK`@JcC<$JcC<$q#>j~> -"!.E>FoMCDpEop4/cJoS<%e.L!C#B>rrC[KV$"@0KpVf="P$'CI@pN=!R+C=rr=)9rr=)4rrg<; -'r/;;rr@KH=ogX0rr3#h/,fJK]Oh(G"JPkq3DocZ!AKc:rrK`@JcC<$JcC<$q#>j~> -"!.E>FoMCDpEop4?i@e@c2IYC52Q#5!JQm>rrgkCs*^O=rrL>?rVlj<qYpO9oD\h6r;HWrI&?nZ -!IK.lrrMX?r;Qc>rkS_oVuJcYrP8KqrVlo1@Y+Q1s+13$s7lVE~> -"!.E>FoMCDpEop4?i@e@c2IYC52Q#5!JQm>rrgkCs*^O=rrL>?rVlj<qYpO9oD\h6r;HWrI&?nZ -!IK.lrrMX?r;Qc>rkS_oVuJcYrP8KqrVlo1@Y+Q1s+13$s7lVE~> -"!.E>FoMCDpEop4?i@e@c2IYC52Q#5!JQm>rrgkCs*^O=rrL>?rVlj<qYpO9oD\h6r;HWrI&?nZ -!IK.lrrMX?r;Qc>rkS_oVuJcYrP8KqrVlo1@Y+Q1s+13$s7lVE~> -"!.E>FoMCDpEop4Ie`pD52Q#5$&'&,s8UV?s*^O=rrL>?rVlj<qYpO9qYpRH9)S\i+T23<##i\E -s2Pk#rr3#h/,fJK>2T>Z"HeWB3TKo7!P;fls+13$s+14Fs*t~> -"!.E>FoMCDpEop4Ie`pD52Q#5$&'&,s8UV?s*^O=rrL>?rVlj<qYpO9qYpRH9)S\i+T23<##i\E -s2Pk#rr3#h/,fJK>2T>Z"HeWB3TKo7!P;fls+13$s+14Fs*t~> -"!.E>FoMCDpEop4Ie`pD52Q#5$&'&,s8UV?s*^O=rrL>?rVlj<qYpO9qYpRH9)S\i+T23<##i\E -s2Pk#rr3#h/,fJK>2T>Z"HeWB3TKo7!P;fls+13$s+14Fs*t~> -"!.EGKDtlRpH/ESNrC%!/H5YPL`IBR1\^nUKp>sb*CB](rG_`V#YFsos(WPm*Duh9"CiGj*Ei=? -!@1&1rrG%UrVmK-9-#$QWJCNR73*9eGQ7^@4oQH)IJs3D2?"TrLAq2TkiSgQJcC<$JcC<$q#>j~> -"!.EGKDtlRpH/ESNrC%!/H5YPL`IBR1\^nUKp>sb*CB](rG_`V#YFsos(WPm*Duh9"CiGj*Ei=? -!@1&1rrG%UrVmK-9-#$QWJCNR73*9eGQ7^@4oQH)IJs3D2?"TrLAq2TkiSgQJcC<$JcC<$q#>j~> -"!.EGKDtlRpH/ESNrC%!/H5YPL`IBR1\^nUKp>sb*CB](rG_`V#YFsos(WPm*Duh9"CiGj*Ei=? -!@1&1rrG%UrVmK-9-#$QWJCNR73*9eGQ7^@4oQH)IJs3D2?"TrLAq2TkiSgQJcC<$JcC<$q#>j~> -!$1"t!JQlGrrY\J2M6S\JcC<$JcGNFJ,~> -!$1"t!JQlGrrY\J2M6S\JcC<$JcGNFJ,~> -!$1"t!JQlGrrY\J2M6S\JcC<$JcGNFJ,~> -!$1"t!JQkks+13$s+13Hs*t~> -!$1"t!JQkks+13$s+13Hs*t~> -!$1"t!JQkks+13$s+13Hs*t~> -!$1"t!P?=%s+13$s+13Hs*t~> -!$1"t!P?=%s+13$s+13Hs*t~> -!$1"t!P?=%s+13$s+13Hs*t~> -!$-XjJcC<$JcC?%J,~> -!$-XjJcC<$JcC?%J,~> -!$-XjJcC<$JcC?%J,~> -!$-XjJcC<$JcC?%J,~> -!$-XjJcC<$JcC?%J,~> -!$-XjJcC<$JcC?%J,~> -!$-XjJcC<$JcC?%J,~> -!$-XjJcC<$JcC?%J,~> -!$-XjJcC<$JcC?%J,~> -!$-XjJcC<$JcC?%J,~> -!$-XjJcC<$JcC?%J,~> -!$-XjJcC<$JcC?%J,~> -!$-XjJcC<$JcC?%J,~> -!$-XjJcC<$JcC?%J,~> -!$-XjJcC<$JcC?%J,~> -!$-XjJcC<$JcC?%J,~> -!$-XjJcC<$JcC?%J,~> -!$-XjJcC<$JcC?%J,~> -!$-XjJcC<$JcC?%J,~> -!$-XjJcC<$JcC?%J,~> -!$-XjJcC<$JcC?%J,~> -!$-XjJcC<$JcC?%J,~> -!$-XjJcC<$JcC?%J,~> -!$-XjJcC<$JcC?%J,~> -!$-XjJcC<$JcC?%J,~> -!$-XjJcC<$JcC?%J,~> -!$-XjJcC<$JcC?%J,~> -!$-XjJcC<$JcC?%J,~> -!$-XjJcC<$JcC?%J,~> -!$-XjJcC<$JcC?%J,~> -!$-XjJcC<$JcC?%J,~> -!$-XjJcC<$JcC?%J,~> -!$-XjJcC<$JcC?%J,~> -!$-XjJcC<$JcC?%J,~> -!$-XjJcC<$JcC?%J,~> -!$-XjJcC<$JcC?%J,~> -!$-XjJcC<$JcC?%J,~> -!$-XjJcC<$JcC?%J,~> -!$-XjJcC<$JcC?%J,~> -!$.@)!V"X)rrM*>JcC<$JcFU,J,~> -!$.@)!V"X)rrM*>JcC<$JcFU,J,~> -!$.@)!V"X)rrM*>JcC<$JcFU,J,~> -!Zh<ur1X1qq#:BJe+3M@aJ,F)rVlu=U8%SZrrL7$rr3)_`W*sUU&`:0rrBk4U&`O-rrVhUQM1=Z -Z2;uL!p66)pAY0d\+'Cuce\R"!9*mN!;6?k!jdU@JcC<$JcFX-J,~> -!Zh<ur1X1qq#:BJe+3M@aJ,F)rVlu=U8%SZrrL7$rr3)_`W*sUU&`:0rrBk4U&`O-rrVhUQM1=Z -Z2;uL!p66)pAY0d\+'Cuce\R"!9*mN!;6?k!jdU@JcC<$JcFX-J,~> -!Zh<ur1X1qq#:BJe+3M@aJ,F)rVlu=U8%SZrrL7$rr3)_`W*sUU&`:0rrBk4U&`O-rrVhUQM1=Z -Z2;uL!p66)pAY0d\+'Cuce\R"!9*mN!;6?k!jdU@JcC<$JcFX-J,~> -"!.E>3;n.'NqiVTQA4u1"0d(en,<7gXEkNRpAY/0Yl=Y*]4(_L=T*OGZ2">!V>Y]@9mZ7-!T6*5 -rrFP?lMgnMW9aHbgNpR2rrLi`rVloY3V!+Sc@:E's8V55rH\HtrVln5Z@W%,s+14-s*t~> -"!.E>3;n.'NqiVTQA4u1"0d(en,<7gXEkNRpAY/0Yl=Y*]4(_L=T*OGZ2">!V>Y]@9mZ7-!T6*5 -rrFP?lMgnMW9aHbgNpR2rrLi`rVloY3V!+Sc@:E's8V55rH\HtrVln5Z@W%,s+14-s*t~> -"!.E>3;n.'NqiVTQA4u1"0d(en,<7gXEkNRpAY/0Yl=Y*]4(_L=T*OGZ2">!V>Y]@9mZ7-!T6*5 -rrFP?lMgnMW9aHbgNpR2rrLi`rVloY3V!+Sc@:E's8V55rH\HtrVln5Z@W%,s+14-s*t~> -"!.E>Fn>V6QA4o/!A3d;rrFG?pAY/0Yl=Y*]4(_L\aTP"fm1^.rrW1;YP%nr..lg)!RjX#rrKB? -rVloY3V*1UdsK?QFR&nK;Z6UqU](2o_KOsjs+13$s5<p-~> -"!.E>Fn>V6QA4o/!A3d;rrFG?pAY/0Yl=Y*]4(_L\aTP"fm1^.rrW1;YP%nr..lg)!RjX#rrKB? -rVloY3V*1UdsK?QFR&nK;Z6UqU](2o_KOsjs+13$s5<p-~> -"!.E>Fn>V6QA4o/!A3d;rrFG?pAY/0Yl=Y*]4(_L\aTP"fm1^.rrW1;YP%nr..lg)!RjX#rrKB? -rVloY3V*1UdsK?QFR&nK;Z6UqU](2o_KOsjs+13$s5<p-~> -"!.E>Fo)+<VuB<p=9&;dL5\bu?2jj(S,Q%\A,Q?-/arT:!@@L6rsdIoh#HGJd/RUdBM2!LoD\j+ -?1.^nrZD%;!*T:o"Ju.u.-LS&!3?,!!)<Gc&@)98@/nYJ9=Os$<^%9l9>1-#!WHO+rs")2s5QaF -V>gK%nk1e^92!h8j#P!U9*!]@rs@Uudf9?f>5QH?p\t7gd"24Js+14.s*t~> -"!.E>Fo)+<VuB<p=9&;dL5\bu?2jj(S,Q%\A,Q?-/arT:!@@L6rsdIoh#HGJd/RUdBM2!LoD\j+ -?1.^nrZD%;!*T:o"Ju.u.-LS&!3?,!!)<Gc&@)98@/nYJ9=Os$<^%9l9>1-#!WHO+rs")2s5QaF -V>gK%nk1e^92!h8j#P!U9*!]@rs@Uudf9?f>5QH?p\t7gd"24Js+14.s*t~> -"!.E>Fo)+<VuB<p=9&;dL5\bu?2jj(S,Q%\A,Q?-/arT:!@@L6rsdIoh#HGJd/RUdBM2!LoD\j+ -?1.^nrZD%;!*T:o"Ju.u.-LS&!3?,!!)<Gc&@)98@/nYJ9=Os$<^%9l9>1-#!WHO+rs")2s5QaF -V>gK%nk1e^92!h8j#P!U9*!]@rs@Uudf9?f>5QH?p\t7gd"24Js+14.s*t~> -"!.E>(&ffgmf3<kIf@_'``2u((m`Rs^g$i5M>km']41a=!A3d;rrFG?p&>IscMuQcs8QRQ\'`Tr -L%YHIj&b4-rrN*@qu6ZGrm:k!Y5]n0rm:jp[JreDrm;:"]`6A3=4,E7_uJ2f4OMREq#:Bn+8u3D -:!`k7d=?cHrsJ\OO]TrXo`*qYAcC'X:[RuX!VbILrr^mPb#8!6!DUpls+13$s5<p-~> -"!.E>(&ffgmf3<kIf@_'``2u((m`Rs^g$i5M>km']41a=!A3d;rrFG?p&>IscMuQcs8QRQ\'`Tr -L%YHIj&b4-rrN*@qu6ZGrm:k!Y5]n0rm:jp[JreDrm;:"]`6A3=4,E7_uJ2f4OMREq#:Bn+8u3D -:!`k7d=?cHrsJ\OO]TrXo`*qYAcC'X:[RuX!VbILrr^mPb#8!6!DUpls+13$s5<p-~> -"!.E>(&ffgmf3<kIf@_'``2u((m`Rs^g$i5M>km']41a=!A3d;rrFG?p&>IscMuQcs8QRQ\'`Tr -L%YHIj&b4-rrN*@qu6ZGrm:k!Y5]n0rm:jp[JreDrm;:"]`6A3=4,E7_uJ2f4OMREq#:Bn+8u3D -:!`k7d=?cHrsJ\OO]TrXo`*qYAcC'X:[RuX!VbILrr^mPb#8!6!DUpls+13$s5<p-~> -"!.E>D>ro*rr<"nIfAsJoC;jHIJNpCju<=#M#R#Idm*g2!A3d;rrFG?p&>J%a8XI[s8V`Yr;Zf' -C%_K,e4B!,!WF2<rrD`koE9K1s!Zt-rrdSCruh:>rs`nKs7mi/s8QrHs8UP>p&>9q+9/Bms%Ui= -rrJ[@r;Qf&C]487j#-H-rrFt?nc&g9;ZHc1*>SMP!DUpls+13$s5<p-~> -"!.E>D>ro*rr<"nIfAsJoC;jHIJNpCju<=#M#R#Idm*g2!A3d;rrFG?p&>J%a8XI[s8V`Yr;Zf' -C%_K,e4B!,!WF2<rrD`koE9K1s!Zt-rrdSCruh:>rs`nKs7mi/s8QrHs8UP>p&>9q+9/Bms%Ui= -rrJ[@r;Qf&C]487j#-H-rrFt?nc&g9;ZHc1*>SMP!DUpls+13$s5<p-~> -"!.E>D>ro*rr<"nIfAsJoC;jHIJNpCju<=#M#R#Idm*g2!A3d;rrFG?p&>J%a8XI[s8V`Yr;Zf' -C%_K,e4B!,!WF2<rrD`koE9K1s!Zt-rrdSCruh:>rs`nKs7mi/s8QrHs8UP>p&>9q+9/Bms%Ui= -rrJ[@r;Qf&C]487j#-H-rrFt?nc&g9;ZHc1*>SMP!DUpls+13$s5<p-~> -"!.E>Fo)+<Q2W071uA7uLAq2Uju<=#(B#W]?2ad(/arT:!@@L4rrOq7-MdZBZYB.5!W"&-rrN*@ -r;QfS2?#!,'V,1Oo`"jnGbtE_rVlg"Dls'8,PqE@dn064#Q5bEV0Dr6ci3qFSUgY<!O6G=rrM.? -rVlmTkjeZRb#83<!$2";!DUpls+13$s5<p-~> -"!.E>Fo)+<Q2W071uA7uLAq2Uju<=#(B#W]?2ad(/arT:!@@L4rrOq7-MdZBZYB.5!W"&-rrN*@ -r;QfS2?#!,'V,1Oo`"jnGbtE_rVlg"Dls'8,PqE@dn064#Q5bEV0Dr6ci3qFSUgY<!O6G=rrM.? -rVlmTkjeZRb#83<!$2";!DUpls+13$s5<p-~> -"!.E>Fo)+<Q2W071uA7uLAq2Uju<=#(B#W]?2ad(/arT:!@@L4rrOq7-MdZBZYB.5!W"&-rrN*@ -r;QfS2?#!,'V,1Oo`"jnGbtE_rVlg"Dls'8,PqE@dn064#Q5bEV0Dr6ci3qFSUgY<!O6G=rrM.? -rVlmTkjeZRb#83<!$2";!DUpls+13$s5<p-~> -"!.E>Fo21>jkTk8"R[oBQA5D="QhZCNfNo7!A3d;rrFG?o`"tIi[4[)!S-Q9rrKH?rVlo1ao)/> -^0^1+!rc-@rVm0Ym/R+I?(CpC\,QC1GbtE_rVlg"Dls'8,PqEDdn0T>e*Zu2#Q5b5OaQ._ci3qF -SUgY<#I/(Es2t)r3W8sY2!FK0!Qn==rr=)<rrUech1>TWs+14.s*t~> -"!.E>Fo21>jkTk8"R[oBQA5D="QhZCNfNo7!A3d;rrFG?o`"tIi[4[)!S-Q9rrKH?rVlo1ao)/> -^0^1+!rc-@rVm0Ym/R+I?(CpC\,QC1GbtE_rVlg"Dls'8,PqEDdn0T>e*Zu2#Q5b5OaQ._ci3qF -SUgY<#I/(Es2t)r3W8sY2!FK0!Qn==rr=)<rrUech1>TWs+14.s*t~> -"!.E>Fo21>jkTk8"R[oBQA5D="QhZCNfNo7!A3d;rrFG?o`"tIi[4[)!S-Q9rrKH?rVlo1ao)/> -^0^1+!rc-@rVm0Ym/R+I?(CpC\,QC1GbtE_rVlg"Dls'8,PqEDdn0T>e*Zu2#Q5b5OaQ._ci3qF -SUgY<#I/(Es2t)r3W8sY2!FK0!Qn==rr=)<rrUech1>TWs+14.s*t~> -"!.E>8H#(]XT-4grga1[IfG^grr3,`2ugF@rga%hrr3,?SK*iqrr3,<SJRZup&>)DQ?rQ1!M>AN -SH4YDrVlm#2uN[U*U<Y*i&pu<$%SG7SVAkhs!V@USHOD^s)sq3SHO>bs7mo9rrqJ)SXk#Wq>Us( -Boe[jKQQ2Qn=<r_S`TkN#L@afST*rU3W8sY2!FK0!T%ttSH*F'rrHl?JcC<$JcFX-J,~> -"!.E>8H#(]XT-4grga1[IfG^grr3,`2ugF@rga%hrr3,?SK*iqrr3,<SJRZup&>)DQ?rQ1!M>AN -SH4YDrVlm#2uN[U*U<Y*i&pu<$%SG7SVAkhs!V@USHOD^s)sq3SHO>bs7mo9rrqJ)SXk#Wq>Us( -Boe[jKQQ2Qn=<r_S`TkN#L@afST*rU3W8sY2!FK0!T%ttSH*F'rrHl?JcC<$JcFX-J,~> -"!.E>8H#(]XT-4grga1[IfG^grr3,`2ugF@rga%hrr3,?SK*iqrr3,<SJRZup&>)DQ?rQ1!M>AN -SH4YDrVlm#2uN[U*U<Y*i&pu<$%SG7SVAkhs!V@USHOD^s)sq3SHO>bs7mo9rrqJ)SXk#Wq>Us( -Boe[jKQQ2Qn=<r_S`TkN#L@afST*rU3W8sY2!FK0!T%ttSH*F'rrHl?JcC<$JcFX-J,~> -!Zh<ir-ng3s8Tn6IftQ,s2r4Xrrhn]s8TS-IfPT0rrBD)IfP`4rrB8%IfPl.rrA)WrrAemIfQ>C -rrIY>r;QbWlMgqTJ$&\L#`4%\YeSH_POSR$!5ng9!.b&u"NUQBr/pgT"586Sd.dPGn"'LY[+PEY -lhu;5h#76Waa\g!s7)TWrrJP[nG`L>rI4h<rr3&;J(]DQJcC<$huA3~> -!Zh<ir-ng3s8Tn6IftQ,s2r4Xrrhn]s8TS-IfPT0rrBD)IfP`4rrB8%IfPl.rrA)WrrAemIfQ>C -rrIY>r;QbWlMgqTJ$&\L#`4%\YeSH_POSR$!5ng9!.b&u"NUQBr/pgT"586Sd.dPGn"'LY[+PEY -lhu;5h#76Waa\g!s7)TWrrJP[nG`L>rI4h<rr3&;J(]DQJcC<$huA3~> -!Zh<ir-ng3s8Tn6IftQ,s2r4Xrrhn]s8TS-IfPT0rrBD)IfP`4rrB8%IfPl.rrA)WrrAemIfQ>C -rrIY>r;QbWlMgqTJ$&\L#`4%\YeSH_POSR$!5ng9!.b&u"NUQBr/pgT"586Sd.dPGn"'LY[+PEY -lhu;5h#76Waa\g!s7)TWrrJP[nG`L>rI4h<rr3&;J(]DQJcC<$huA3~> -!$.@)!U.7_rrLKtJcC<$JcFU,J,~> -!$.@)!U.7_rrLKtJcC<$JcFU,J,~> -!$.@)!U.7_rrLKtJcC<$JcFU,J,~> -!$-XjJcC<$JcC?%J,~> -!$-XjJcC<$JcC?%J,~> -!$-XjJcC<$JcC?%J,~> -!$-XjJcC<$JcC?%J,~> -!$-XjJcC<$JcC?%J,~> -!$-XjJcC<$JcC?%J,~> -!$2%<!VH<grrMlioD\pdn*g5QrrVf\rndYUo_[nQ"9.cXpAP!ln+6&K!VZ?grrW(rKA$8/oDJXg -qX=1=rrMiio`#!en,'Hk!THTHrrG4Kqu6[UnU^^ks2P(h~> -!$2%<!VH<grrMlioD\pdn*g5QrrVf\rndYUo_[nQ"9.cXpAP!ln+6&K!VZ?grrW(rKA$8/oDJXg -qX=1=rrMiio`#!en,'Hk!THTHrrG4Kqu6[UnU^^ks2P(h~> -!$2%<!VH<grrMlioD\pdn*g5QrrVf\rndYUo_[nQ"9.cXpAP!ln+6&K!VZ?grrW(rKA$8/oDJXg -qX=1=rrMiio`#!en,'Hk!THTHrrG4Kqu6[UnU^^ks2P(h~> -!$2(=!p)_mrVlqdOn//E",\Z;k5>5\CG#,LrrLpOmJd2k_<Lt+`@WZ`lMpn`/RS#I!I1I?rrL/? -f`).EfDklc/Y1u&!pa0mo`"uX/]ZZH!TE_;rrLpOo)A\1qYpO9qu6]o,(]cFs2P(h~> -!$2(=!p)_mrVlqdOn//E",\Z;k5>5\CG#,LrrLpOmJd2k_<Lt+`@WZ`lMpn`/RS#I!I1I?rrL/? -f`).EfDklc/Y1u&!pa0mo`"uX/]ZZH!TE_;rrLpOo)A\1qYpO9qu6]o,(]cFs2P(h~> -!$2(=!p)_mrVlqdOn//E",\Z;k5>5\CG#,LrrLpOmJd2k_<Lt+`@WZ`lMpn`/RS#I!I1I?rrL/? -f`).EfDklc/Y1u&!pa0mo`"uX/]ZZH!TE_;rrLpOo)A\1qYpO9qu6]o,(]cFs2P(h~> -"s*aDh09a\r;Qh\Qh0hK!B0*,rrG1?mJd2k_<Lt&*WQ/%MsLBIGbtH?pRR)lrrGR?i;WoBL*<S@ -rrH*?li-uIiUd'M_-?d9!Ed>=rrW+#a+=8As2Y.i~> -"s*aDh09a\r;Qh\Qh0hK!B0*,rrG1?mJd2k_<Lt&*WQ/%MsLBIGbtH?pRR)lrrGR?i;WoBL*<S@ -rrH*?li-uIiUd'M_-?d9!Ed>=rrW+#a+=8As2Y.i~> -"s*aDh09a\r;Qh\Qh0hK!B0*,rrG1?mJd2k_<Lt&*WQ/%MsLBIGbtH?pRR)lrrGR?i;WoBL*<S@ -rrH*?li-uIiUd'M_-?d9!Ed>=rrW+#a+=8As2Y.i~> -"s*a!J\Fq)qu6_[OS\VL!B0*=rr^-*R/$X[!5JN##d+.,a-\'+R'?Si!4`#q!42V'!M-:jrrH?? -rr3,W`rH(Arg3u*s8TOps8V)prVm0Es8TdDs8Sj]Yl4S&XRlFY!6bBA$(RBM_g&$Xs5Z0;rrE#r -Qiu"?s.o&]Qil%As7ZDY"5NqVo)4pXi;`iGrg3rB7G%S*R$c5!rrVhrh#<ZClMpnGrg3cHWqlJj -"R:fBEiSg4"4rp-jSf)YeGYd4rrT]tpAY'lc2O(5o;_ijao7Y5UAt8AQnilQV>gJp_Z#o6WW3"A -fX$s4XT/=@Qm7?QYl=Y&\GhiqZi0n).e<H9!Tl<;rrLq?JcC<$a8^Y~> -"s*a!J\Fq)qu6_[OS\VL!B0*=rr^-*R/$X[!5JN##d+.,a-\'+R'?Si!4`#q!42V'!M-:jrrH?? -rr3,W`rH(Arg3u*s8TOps8V)prVm0Es8TdDs8Sj]Yl4S&XRlFY!6bBA$(RBM_g&$Xs5Z0;rrE#r -Qiu"?s.o&]Qil%As7ZDY"5NqVo)4pXi;`iGrg3rB7G%S*R$c5!rrVhrh#<ZClMpnGrg3cHWqlJj -"R:fBEiSg4"4rp-jSf)YeGYd4rrT]tpAY'lc2O(5o;_ijao7Y5UAt8AQnilQV>gJp_Z#o6WW3"A -fX$s4XT/=@Qm7?QYl=Y&\GhiqZi0n).e<H9!Tl<;rrLq?JcC<$a8^Y~> -"s*a!J\Fq)qu6_[OS\VL!B0*=rr^-*R/$X[!5JN##d+.,a-\'+R'?Si!4`#q!42V'!M-:jrrH?? -rr3,W`rH(Arg3u*s8TOps8V)prVm0Es8TdDs8Sj]Yl4S&XRlFY!6bBA$(RBM_g&$Xs5Z0;rrE#r -Qiu"?s.o&]Qil%As7ZDY"5NqVo)4pXi;`iGrg3rB7G%S*R$c5!rrVhrh#<ZClMpnGrg3cHWqlJj -"R:fBEiSg4"4rp-jSf)YeGYd4rrT]tpAY'lc2O(5o;_ijao7Y5UAt8AQnilQV>gJp_Z#o6WW3"A -fX$s4XT/=@Qm7?QYl=Y&\GhiqZi0n).e<H9!Tl<;rrLq?JcC<$a8^Y~> -"s*`o]`1dPqYpS`K_59F2Wju<"1Ek"n,E=f]Rg'8%"b>U]S%>KKVAGGZ[r+/!J7HPrrEN]rr3:/ -_>j(QI_5WWK)UE/J\(kRdf9?VB)V`0*W,j<O`W_arrFipre(B(bl<e(KEcrfV>pSV3;idVL])l/ -J(ai;IK"m%Ibk$RrIb9%ir4?(KEH\cqL&9q#C[iJs6,,1M#RDUiu<IHKEHYom!\kd!e,arq>UTZ -XT*=@p&>-?KJL1=rrL`$rIb0.rVm'u!9sO`c@>hH"FQ[<aajAD%"joHab23XKTuN:_1;N<'S<"X -IS>?HKS$$"]n@GLKV8AFZ[r+/!J7HQrrFV?qYpTY2Z*LTh*6JjJcEdjJ,~> -"s*`o]`1dPqYpS`K_59F2Wju<"1Ek"n,E=f]Rg'8%"b>U]S%>KKVAGGZ[r+/!J7HPrrEN]rr3:/ -_>j(QI_5WWK)UE/J\(kRdf9?VB)V`0*W,j<O`W_arrFipre(B(bl<e(KEcrfV>pSV3;idVL])l/ -J(ai;IK"m%Ibk$RrIb9%ir4?(KEH\cqL&9q#C[iJs6,,1M#RDUiu<IHKEHYom!\kd!e,arq>UTZ -XT*=@p&>-?KJL1=rrL`$rIb0.rVm'u!9sO`c@>hH"FQ[<aajAD%"joHab23XKTuN:_1;N<'S<"X -IS>?HKS$$"]n@GLKV8AFZ[r+/!J7HQrrFV?qYpTY2Z*LTh*6JjJcEdjJ,~> -"s*`o]`1dPqYpS`K_59F2Wju<"1Ek"n,E=f]Rg'8%"b>U]S%>KKVAGGZ[r+/!J7HPrrEN]rr3:/ -_>j(QI_5WWK)UE/J\(kRdf9?VB)V`0*W,j<O`W_arrFipre(B(bl<e(KEcrfV>pSV3;idVL])l/ -J(ai;IK"m%Ibk$RrIb9%ir4?(KEH\cqL&9q#C[iJs6,,1M#RDUiu<IHKEHYom!\kd!e,arq>UTZ -XT*=@p&>-?KJL1=rrL`$rIb0.rVm'u!9sO`c@>hH"FQ[<aajAD%"joHab23XKTuN:_1;N<'S<"X -IS>?HKS$$"]n@GLKV8AFZ[r+/!J7HQrrFV?qYpTY2Z*LTh*6JjJcEdjJ,~> -!$2%<!Dgu9rr=)3rrG%?qu6[Kn,E=fI@pN=!URQ"rrG1?rVln7Yl=Y'k/I<!!N!+$rs;oGF(M&\ -s8Q$?rr3M'HN-Ucs8TQ?s8Rm]$XStnrrM%?rVlmQR/[*pJ=QWfo`+sD9;V[gj>d);!$2%<"D>1C -(pX)?"(&_A*W?!?_!:k?rVlsoch&XbrrU.ifDbdNP_f>=!okQBrr3'WkhAE&rrG^@p&>&5Wq65k -@CuO=!O6J>rr=)<rs$32p](8dIK'6L*WQ/+L&V)Qk;N>>!E@/=rrIk?rr3;U8,n$Kk5YJ+:]C@p -45p/<!H#%>rrM9#rr3!Ko(r@eju3,:!Sotks+13js*t~> -!$2%<!Dgu9rr=)3rrG%?qu6[Kn,E=fI@pN=!URQ"rrG1?rVln7Yl=Y'k/I<!!N!+$rs;oGF(M&\ -s8Q$?rr3M'HN-Ucs8TQ?s8Rm]$XStnrrM%?rVlmQR/[*pJ=QWfo`+sD9;V[gj>d);!$2%<"D>1C -(pX)?"(&_A*W?!?_!:k?rVlsoch&XbrrU.ifDbdNP_f>=!okQBrr3'WkhAE&rrG^@p&>&5Wq65k -@CuO=!O6J>rr=)<rs$32p](8dIK'6L*WQ/+L&V)Qk;N>>!E@/=rrIk?rr3;U8,n$Kk5YJ+:]C@p -45p/<!H#%>rrM9#rr3!Ko(r@eju3,:!Sotks+13js*t~> -!$2%<!Dgu9rr=)3rrG%?qu6[Kn,E=fI@pN=!URQ"rrG1?rVln7Yl=Y'k/I<!!N!+$rs;oGF(M&\ -s8Q$?rr3M'HN-Ucs8TQ?s8Rm]$XStnrrM%?rVlmQR/[*pJ=QWfo`+sD9;V[gj>d);!$2%<"D>1C -(pX)?"(&_A*W?!?_!:k?rVlsoch&XbrrU.ifDbdNP_f>=!okQBrr3'WkhAE&rrG^@p&>&5Wq65k -@CuO=!O6J>rr=)<rs$32p](8dIK'6L*WQ/+L&V)Qk;N>>!E@/=rrIk?rr3;U8,n$Kk5YJ+:]C@p -45p/<!H#%>rrM9#rr3!Ko(r@eju3,:!Sotks+13js*t~> -!$2%<!Dgu:rrHN?o`"qMk55/Z/ar]=!T%ep5lbQrrrG1?rVloOB`:9tFSGe;>;EA"rVlms2#]cO -.=2"ddf9?VB)hnV^)"H2r;QfU4T59\1$no>"G!$B..mN="aHmDs5Z0;rr<r85m&%6ruM(<"(&_A -*W,j;r?)"<"&7,6./j/H7G%P=!L/E>rrM.?rVlsVkhAE&rrG^@p&>&5Wq65k@CuO=!O6J>rr=)7 -rrJd@rr3*As8SE0r]pQH2ZE^W<65%<!JZp>rrgnCs*gR=rrLA?rr3!\iVicWg1^IO!-A,=!@m[: -rrM7?qu6]Q5Crics2Y.i~> -!$2%<!Dgu:rrHN?o`"qMk55/Z/ar]=!T%ep5lbQrrrG1?rVloOB`:9tFSGe;>;EA"rVlms2#]cO -.=2"ddf9?VB)hnV^)"H2r;QfU4T59\1$no>"G!$B..mN="aHmDs5Z0;rr<r85m&%6ruM(<"(&_A -*W,j;r?)"<"&7,6./j/H7G%P=!L/E>rrM.?rVlsVkhAE&rrG^@p&>&5Wq65k@CuO=!O6J>rr=)7 -rrJd@rr3*As8SE0r]pQH2ZE^W<65%<!JZp>rrgnCs*gR=rrLA?rr3!\iVicWg1^IO!-A,=!@m[: -rrM7?qu6]Q5Crics2Y.i~> -!$2%<!Dgu:rrHN?o`"qMk55/Z/ar]=!T%ep5lbQrrrG1?rVloOB`:9tFSGe;>;EA"rVlms2#]cO -.=2"ddf9?VB)hnV^)"H2r;QfU4T59\1$no>"G!$B..mN="aHmDs5Z0;rr<r85m&%6ruM(<"(&_A -*W,j;r?)"<"&7,6./j/H7G%P=!L/E>rrM.?rVlsVkhAE&rrG^@p&>&5Wq65k@CuO=!O6J>rr=)7 -rrJd@rr3*As8SE0r]pQH2ZE^W<65%<!JZp>rrgnCs*gR=rrLA?rr3!\iVicWg1^IO!-A,=!@m[: -rrM7?qu6]Q5Crics2Y.i~> -!$2%<!Dgu;rrQWBrUg*j2Wjo:!A3d=rrD9^gB"`srr3![ir&fVk5O*9Wd+=="'_Wb48o0[;m$#Q -#OMI_df9?VB)V`0*W#d:i&pu<!AWs?rrdkBs![O=rrmYDs8V.>r;Qa:r7_;GruM(<"(&_A*W,j; -r?)"<"&7,6./j/H7G%P=!L/E>rrM.?rVlsVkk_C[,QXh=p&>&5Wq65k@CuO=!O6J>rr=)7rrMYA -rZhWqs8SF+rS%>3rr3!uaSl,>L6hi="P-*CI\-Q=!R4F>rrG4?r;[email protected]<H9!Tl<; -rrLq?JcC<$a8^Y~> -!$2%<!Dgu;rrQWBrUg*j2Wjo:!A3d=rrD9^gB"`srr3![ir&fVk5O*9Wd+=="'_Wb48o0[;m$#Q -#OMI_df9?VB)V`0*W#d:i&pu<!AWs?rrdkBs![O=rrmYDs8V.>r;Qa:r7_;GruM(<"(&_A*W,j; -r?)"<"&7,6./j/H7G%P=!L/E>rrM.?rVlsVkk_C[,QXh=p&>&5Wq65k@CuO=!O6J>rr=)7rrMYA -rZhWqs8SF+rS%>3rr3!uaSl,>L6hi="P-*CI\-Q=!R4F>rrG4?r;[email protected]<H9!Tl<; -rrLq?JcC<$a8^Y~> -!$2%<!Dgu;rrQWBrUg*j2Wjo:!A3d=rrD9^gB"`srr3![ir&fVk5O*9Wd+=="'_Wb48o0[;m$#Q -#OMI_df9?VB)V`0*W#d:i&pu<!AWs?rrdkBs![O=rrmYDs8V.>r;Qa:r7_;GruM(<"(&_A*W,j; -r?)"<"&7,6./j/H7G%P=!L/E>rrM.?rVlsVkk_C[,QXh=p&>&5Wq65k@CuO=!O6J>rr=)7rrMYA -rZhWqs8SF+rS%>3rr3!uaSl,>L6hi="P-*CI\-Q=!R4F>rrG4?r;[email protected]<H9!Tl<; -rrLq?JcC<$a8^Y~> -%NYTElKUsRlL"WOoPM5>p&>0dl7pcSrr38pl71QUs8S%KrosO!?iL'19%*"9C]Ag_rosNhDZ0S9 -'/fdh$!Y7@i>l@us'`V?l3Qq[s&m;*`t`4brr=):rs7u:2sU&8s"Wm>rsjRLs#T-(lBDdTs8W)9 ->5eI$0`C8=pAb.5rVlt+^Afhal2tM?r@e-7"'3nD2u`(NC^AYmqs&uclMCP@3W8s[2!Egmro*tA -qu-O&IgEmjs82*HBq4JRrVlq:#grYO#l!UeX6T]a[r;61!&4BO#$)#ps8T-Jro+(Hs8SaFrosLM -rVm+4]^!gBs,`ECl3YL3s*bTflKZ/krr33rdH\>Ys*h!Il2e2-rr3#p,l7NA,Ph9;!rFA@r;Qii -#YfmMJcF'rJ,~> -%NYTElKUsRlL"WOoPM5>p&>0dl7pcSrr38pl71QUs8S%KrosO!?iL'19%*"9C]Ag_rosNhDZ0S9 -'/fdh$!Y7@i>l@us'`V?l3Qq[s&m;*`t`4brr=):rs7u:2sU&8s"Wm>rsjRLs#T-(lBDdTs8W)9 ->5eI$0`C8=pAb.5rVlt+^Afhal2tM?r@e-7"'3nD2u`(NC^AYmqs&uclMCP@3W8s[2!Egmro*tA -qu-O&IgEmjs82*HBq4JRrVlq:#grYO#l!UeX6T]a[r;61!&4BO#$)#ps8T-Jro+(Hs8SaFrosLM -rVm+4]^!gBs,`ECl3YL3s*bTflKZ/krr33rdH\>Ys*h!Il2e2-rr3#p,l7NA,Ph9;!rFA@r;Qii -#YfmMJcF'rJ,~> -%NYTElKUsRlL"WOoPM5>p&>0dl7pcSrr38pl71QUs8S%KrosO!?iL'19%*"9C]Ag_rosNhDZ0S9 -'/fdh$!Y7@i>l@us'`V?l3Qq[s&m;*`t`4brr=):rs7u:2sU&8s"Wm>rsjRLs#T-(lBDdTs8W)9 ->5eI$0`C8=pAb.5rVlt+^Afhal2tM?r@e-7"'3nD2u`(NC^AYmqs&uclMCP@3W8s[2!Egmro*tA -qu-O&IgEmjs82*HBq4JRrVlq:#grYO#l!UeX6T]a[r;61!&4BO#$)#ps8T-Jro+(Hs8SaFrosLM -rVm+4]^!gBs,`ECl3YL3s*bTflKZ/krr33rdH\>Ys*h!Il2e2-rr3#p,l7NA,Ph9;!rFA@r;Qii -#YfmMJcF'rJ,~> -!Zh<Nr%\CPs8SOkoD\fMr\=IJrr2tQr\=IQrr2tJr\=IXr;Qgi11L7_!-.un!,;B2!BaK`rrIJk -rr3,5NrT,br\=J%rr3+%10(eqrVljiqu6Xfr\=V=s8Q6jrr3:mYlF_b1,=WJ\,QC/dUqP=rrDoo -1'=ZYs%<7i"+JGnmJSdB])VfmrA"Jks8V!U1'FgqhuD@-1'>l&l!XJi"'k6gV>W.MZMa_%"dK_8 -s8UXI1&s31rrRrdd/O%FYPg3Yp](8mrA"BPrVm(C1@>,Am",-k"?b98s-3L<!(Qnd!a[WVrr2tR -r\=aXs8RP>BJM>Hr;Qgj11C1^!-8&o!,2<1!CkZ<rrLS?qu6]C:&FqloeL<6s+13rs*t~> -!Zh<Nr%\CPs8SOkoD\fMr\=IJrr2tQr\=IQrr2tJr\=IXr;Qgi11L7_!-.un!,;B2!BaK`rrIJk -rr3,5NrT,br\=J%rr3+%10(eqrVljiqu6Xfr\=V=s8Q6jrr3:mYlF_b1,=WJ\,QC/dUqP=rrDoo -1'=ZYs%<7i"+JGnmJSdB])VfmrA"Jks8V!U1'FgqhuD@-1'>l&l!XJi"'k6gV>W.MZMa_%"dK_8 -s8UXI1&s31rrRrdd/O%FYPg3Yp](8mrA"BPrVm(C1@>,Am",-k"?b98s-3L<!(Qnd!a[WVrr2tR -r\=aXs8RP>BJM>Hr;Qgj11C1^!-8&o!,2<1!CkZ<rrLS?qu6]C:&FqloeL<6s+13rs*t~> -!Zh<Nr%\CPs8SOkoD\fMr\=IJrr2tQr\=IQrr2tJr\=IXr;Qgi11L7_!-.un!,;B2!BaK`rrIJk -rr3,5NrT,br\=J%rr3+%10(eqrVljiqu6Xfr\=V=s8Q6jrr3:mYlF_b1,=WJ\,QC/dUqP=rrDoo -1'=ZYs%<7i"+JGnmJSdB])VfmrA"Jks8V!U1'FgqhuD@-1'>l&l!XJi"'k6gV>W.MZMa_%"dK_8 -s8UXI1&s31rrRrdd/O%FYPg3Yp](8mrA"BPrVm(C1@>,Am",-k"?b98s-3L<!(Qnd!a[WVrr2tR -r\=aXs8RP>BJM>Hr;Qgj11C1^!-8&o!,2<1!CkZ<rrLS?qu6]C:&FqloeL<6s+13rs*t~> -!$/$<"ip05s0lUqrrL'0a8Z1p62gfa/+NT<!F<J3rrGj@kPkRUIK'6Imk*c'!IgX(rrMb1r;Qe0 -r;6Ko@tFZ2s2G"g~> -!$/$<"ip05s0lUqrrL'0a8Z1p62gfa/+NT<!F<J3rrGj@kPkRUIK'6Imk*c'!IgX(rrMb1r;Qe0 -r;6Ko@tFZ2s2G"g~> -!$/$<"ip05s0lUqrrL'0a8Z1p62gfa/+NT<!F<J3rrGj@kPkRUIK'6Imk*c'!IgX(rrMb1r;Qe0 -r;6Ko@tFZ2s2G"g~> -!$/$<"db13PDH-0rrMD.rK@8'p\t0oa*QJ3rrTu\jQHODaFF2Q!Ki-#rrIS?JcC<$SH"*~> -!$/$<"db13PDH-0rrMD.rK@8'p\t0oa*QJ3rrTu\jQHODaFF2Q!Ki-#rrIS?JcC<$SH"*~> -!$/$<"db13PDH-0rrMD.rK@8'p\t0oa*QJ3rrTu\jQHODaFF2Q!Ki-#rrIS?JcC<$SH"*~> -!$/!;".k@+Wh04jiVeT5li$ha_US2W!P7(JrrC%;M#`Y#rrL$cJcC<$SH"*~> -!$/!;".k@+Wh04jiVeT5li$ha_US2W!P7(JrrC%;M#`Y#rrL$cJcC<$SH"*~> -!$/!;".k@+Wh04jiVeT5li$ha_US2W!P7(JrrC%;M#`Y#rrL$cJcC<$SH"*~> -!$2";!d\!iJc>iPBN]t=s+13$s,R,0~> -!$2";!d\!iJc>iPBN]t=s+13$s,R,0~> -!$2";!d\!iJc>iPBN]t=s+13$s,R,0~> -!$2%<"*pi_bhN-ufQmJr!GFXFrrGGskl1[N](l:-aMX^)JcC<$JcCf2J,~> -!$2%<"*pi_bhN-ufQmJr!GFXFrrGGskl1[N](l:-aMX^)JcC<$JcCf2J,~> -!$2%<"*pi_bhN-ufQmJr!GFXFrrGGskl1[N](l:-aMX^)JcC<$JcCf2J,~> -!$2%<!DgtnrrL>?rr3#^i7%],i9'8'!I1I7rrKi?JcC<$JcCf2J,~> -!$2%<!DgtnrrL>?rr3#^i7%],i9'8'!I1I7rrKi?JcC<$JcCf2J,~> -!$2%<!DgtnrrL>?rr3#^i7%],i9'8'!I1I7rrKi?JcC<$JcCf2J,~> -!$2%<!b/kArr2uhroFFJs8V`^mcEQnrr2ufroF.Drr2ueroF.Err3>ojlP[L;#gR`jlQI@"n;9O -jlPk.rrqcNjluO/q#:fjq>^Kimf3=RqW?o$n,E=gkiM+-rrD9^jT+iMrrD6]jT+lNrs.]Jjm[Mk -s8W&Z!;-9j!rK6?JcC<$JcCf2J,~> -!$2%<!b/kArr2uhroFFJs8V`^mcEQnrr2ufroF.Drr2ueroF.Err3>ojlP[L;#gR`jlQI@"n;9O -jlPk.rrqcNjluO/q#:fjq>^Kimf3=RqW?o$n,E=gkiM+-rrD9^jT+iMrrD6]jT+lNrs.]Jjm[Mk -s8W&Z!;-9j!rK6?JcC<$JcCf2J,~> -!$2%<!b/kArr2uhroFFJs8V`^mcEQnrr2ufroF.Drr2ueroF.Err3>ojlP[L;#gR`jlQI@"n;9O -jlPk.rrqcNjluO/q#:fjq>^Kimf3=RqW?o$n,E=gkiM+-rrD9^jT+iMrrD6]jT+lNrs.]Jjm[Mk -s8W&Z!;-9j!rK6?JcC<$JcCf2J,~> -!$2(=#N*U%n,NFQJc7S:6.5e!BkoXd7b%J#GQ'N(9%Et'ErJ!-:Y5X,D)XC@TMY[gHoD</rsik& -;`?XAh>c;<2`FX*MYR2b;p,.?^Kpm>8u4d32f[pYs&:a0rVloJ8,bFMHKbCW6N/nPJ*R'\4Z><@ -?D[\J3W:rBM=(?Cj'9[bJcC<$JcCi3J,~> -!$2(=#N*U%n,NFQJc7S:6.5e!BkoXd7b%J#GQ'N(9%Et'ErJ!-:Y5X,D)XC@TMY[gHoD</rsik& -;`?XAh>c;<2`FX*MYR2b;p,.?^Kpm>8u4d32f[pYs&:a0rVloJ8,bFMHKbCW6N/nPJ*R'\4Z><@ -?D[\J3W:rBM=(?Cj'9[bJcC<$JcCi3J,~> -!$2(=#N*U%n,NFQJc7S:6.5e!BkoXd7b%J#GQ'N(9%Et'ErJ!-:Y5X,D)XC@TMY[gHoD</rsik& -;`?XAh>c;<2`FX*MYR2b;p,.?^Kpm>8u4d32f[pYs&:a0rVloJ8,bFMHKbCW6N/nPJ*R'\4Z><@ -?D[\J3W:rBM=(?Cj'9[bJcC<$JcCi3J,~> -!$2(=!c7q^rr3"kIfB?UmOnO*/AhGeju<=#NrK%]k^NPcQi@!keO]^gTE"r]6W!a]!$2%<%;\)^ -s8T]>s'rV>s3OI6rshuMs8Ti>s8P\[p&G&[KDtlOkqi;<!BK3>rrfR8s"Wm>rsXFJs![O>s0sGQ -ruh:>rrI&?rr3&6!.0:sJcC<$OT0h~> -!$2(=!c7q^rr3"kIfB?UmOnO*/AhGeju<=#NrK%]k^NPcQi@!keO]^gTE"r]6W!a]!$2%<%;\)^ -s8T]>s'rV>s3OI6rshuMs8Ti>s8P\[p&G&[KDtlOkqi;<!BK3>rrfR8s"Wm>rsXFJs![O>s0sGQ -ruh:>rrI&?rr3&6!.0:sJcC<$OT0h~> -!$2(=!c7q^rr3"kIfB?UmOnO*/AhGeju<=#NrK%]k^NPcQi@!keO]^gTE"r]6W!a]!$2%<%;\)^ -s8T]>s'rV>s3OI6rshuMs8Ti>s8P\[p&G&[KDtlOkqi;<!BK3>rrfR8s"Wm>rsXFJs![O>s0sGQ -ruh:>rrI&?rr3&6!.0:sJcC<$OT0h~> -!$2(=#Hi'&pAb/b7fJDNKeECkLAq2Uju<=#Nr/hWKpVf="P$'CI@pN=!R+C=rr=)<rrHr?rr389 -@K0iJs8UG>p\tOub5_LV?2spsdf07IR=kM=!U2E=rrG.?qu6[Om/I"fJ=QWfo`"jnGbtE_NW+qC -5-=kbmtYnkJcC<$JcCi3J,~> -!$2(=#Hi'&pAb/b7fJDNKeECkLAq2Uju<=#Nr/hWKpVf="P$'CI@pN=!R+C=rr=)<rrHr?rr389 -@K0iJs8UG>p\tOub5_LV?2spsdf07IR=kM=!U2E=rrG.?qu6[Om/I"fJ=QWfo`"jnGbtE_NW+qC -5-=kbmtYnkJcC<$JcCi3J,~> -!$2(=#Hi'&pAb/b7fJDNKeECkLAq2Uju<=#Nr/hWKpVf="P$'CI@pN=!R+C=rr=)<rrHr?rr389 -@K0iJs8UG>p\tOub5_LV?2spsdf07IR=kM=!U2E=rrG.?qu6[Om/I"fJ=QWfo`"jnGbtE_NW+qC -5-=kbmtYnkJcC<$JcCi3J,~> -!$2%<#+^SDs8S]\rJguSs-AE=rrhICs,N-:rrIh?rr3,O8H4+1rr3#C;#UCo*W?!=Cp<p=#eOOF -\p8:8G-L`@#uf"Hs1_k>s%:`=rrJO?rr3#`1]@=S3TL#:!AWs?rrdkBs![O=rrdSCrud="NWn2; -s6k`>JcC<$JcCf2J,~> -!$2%<#+^SDs8S]\rJguSs-AE=rrhICs,N-:rrIh?rr3,O8H4+1rr3#C;#UCo*W?!=Cp<p=#eOOF -\p8:8G-L`@#uf"Hs1_k>s%:`=rrJO?rr3#`1]@=S3TL#:!AWs?rrdkBs![O=rrdSCrud="NWn2; -s6k`>JcC<$JcCf2J,~> -!$2%<#+^SDs8S]\rJguSs-AE=rrhICs,N-:rrIh?rr3,O8H4+1rr3#C;#UCo*W?!=Cp<p=#eOOF -\p8:8G-L`@#uf"Hs1_k>s%:`=rrJO?rr3#`1]@=S3TL#:!AWs?rrdkBs![O=rrdSCrud="NWn2; -s6k`>JcC<$JcCf2J,~> -!$2%<!Dgu>rrJa@qu6\_LAq2Uju<=#NrK%]k^NPcQi@!keO]^gTE"r]6W!a]!$2%<!Go">rs-/E -s-JqMX1J3.!EOLFrs;WHs8Ti>s8P^>rr3"eKDtlOkqi;<!BK3>rrfR8s"Wm>rsXFJs![O>s0sGQ -ruh:9rrKi?JcC<$JcCf2J,~> -!$2%<!Dgu>rrJa@qu6\_LAq2Uju<=#NrK%]k^NPcQi@!keO]^gTE"r]6W!a]!$2%<!Go">rs-/E -s-JqMX1J3.!EOLFrs;WHs8Ti>s8P^>rr3"eKDtlOkqi;<!BK3>rrfR8s"Wm>rsXFJs![O>s0sGQ -ruh:9rrKi?JcC<$JcCf2J,~> -!$2%<!Dgu>rrJa@qu6\_LAq2Uju<=#NrK%]k^NPcQi@!keO]^gTE"r]6W!a]!$2%<!Go">rs-/E -s-JqMX1J3.!EOLFrs;WHs8Ti>s8P^>rr3"eKDtlOkqi;<!BK3>rrfR8s"Wm>rsXFJs![O>s0sGQ -ruh:9rrKi?JcC<$JcCf2J,~> -!$2%<!Dgu>rrMD9r\jsKs8SC>rr3,`2ui#ar\jsPec4`Or\kNefDjlJ2`F*s;#gQC2[23Qs8Qo> -rr3,[email protected]\jgur;Qd"2u`g`g/U'j>EbER9%*_=%%2bJs&:`]2i[k;e49Ks"FB;3dR*pl%"IRB -coj<HF'b^CcT1t`!3#kr!PVlks+13$s,[21~> -!$2%<!Dgu>rrMD9r\jsKs8SC>rr3,`2ui#ar\jsPec4`Or\kNefDjlJ2`F*s;#gQC2[23Qs8Qo> -rr3,[email protected]\jgur;Qd"2u`g`g/U'j>EbER9%*_=%%2bJs&:`]2i[k;e49Ks"FB;3dR*pl%"IRB -coj<HF'b^CcT1t`!3#kr!PVlks+13$s,[21~> -!$2%<!Dgu>rrMD9r\jsKs8SC>rr3,`2ui#ar\jsPec4`Or\kNefDjlJ2`F*s;#gQC2[23Qs8Qo> -rr3,[email protected]\jgur;Qd"2u`g`g/U'j>EbER9%*_=%%2bJs&:`]2i[k;e49Ks"FB;3dR*pl%"IRB -coj<HF'b^CcT1t`!3#kr!PVlks+13$s,[21~> -!$2%<#44f!56(ZRroF:Fs8V`^rr3-!lMpnRroF.Drr2ueroF.Err38mjlP\%mJm4SroF:Ks8VT_ -rr3,rn,IL3roF0ZH2[aDc8Y_%#jUO5lh0fJm/$_]"SD9bs60ID!:^!f!9jFD!:g'g!9a@C!:p-h -#Nk.0nF?5Ps8D$`o`+qH*Dc*Ss+13$s,[21~> -!$2%<#44f!56(ZRroF:Fs8V`^rr3-!lMpnRroF.Drr2ueroF.Err38mjlP\%mJm4SroF:Ks8VT_ -rr3,rn,IL3roF0ZH2[aDc8Y_%#jUO5lh0fJm/$_]"SD9bs60ID!:^!f!9jFD!:g'g!9a@C!:p-h -#Nk.0nF?5Ps8D$`o`+qH*Dc*Ss+13$s,[21~> -!$2%<#44f!56(ZRroF:Fs8V`^rr3-!lMpnRroF.Drr2ueroF.Err38mjlP\%mJm4SroF:Ks8VT_ -rr3,rn,IL3roF0ZH2[aDc8Y_%#jUO5lh0fJm/$_]"SD9bs60ID!:^!f!9jFD!:g'g!9a@C!:p-h -#Nk.0nF?5Ps8D$`o`+qH*Dc*Ss+13$s,[21~> -!$/lT!I(UDhuT^&rVloB:kAXts+13$s3Udr~> -!$/lT!I(UDhuT^&rVloB:kAXts+13$s3Udr~> -!$/lT!I(UDhuT^&rVloB:kAXts+13$s3Udr~> -!$/iS!-8'$!-\;?!)nIKJcC<$JcF'rJ,~> -!$/iS!-8'$!-\;?!)nIKJcC<$JcF'rJ,~> -!$/iS!-8'$!-\;?!)nIKJcC<$JcF'rJ,~> -!$-XjJcC<$JcC?%J,~> -!$-XjJcC<$JcC?%J,~> -!$-XjJcC<$JcC?%J,~> -"!.FAao7+sNW/qY`R48V"0^kcRDo1.QN,gOJcC<$JcC<$iW"E~> -"!.FAao7+sNW/qY`R48V"0^kcRDo1.QN,gOJcC<$JcC<$iW"E~> -"!.FAao7+sNW/qY`R48V"0^kcRDo1.QN,gOJcC<$JcC<$iW"E~> -"!.EcIf>N8P(N`6b'V_2rrZ3AqBEoY"!.ENAqBu5s+13$s5F!.~> -"!.EcIf>N8P(N`6b'V_2rrZ3AqBEoY"!.ENAqBu5s+13$s5F!.~> -"!.EcIf>N8P(N`6b'V_2rrZ3AqBEoY"!.ENAqBu5s+13$s5F!.~> -"!.E>FoMCApEon>!mbW@o`#"land4#rrX;A[q:3kJcC<$JcF^/J,~> -"!.E>FoMCApEon>!mbW@o`#"land4#rrX;A[q:3kJcC<$JcF^/J,~> -"!.E>FoMCApEon>!mbW@o`#"land4#rrX;A[q:3kJcC<$JcF^/J,~> -"!.F0jSo/[8C[V<!o$B@p&>9Qo`!>>s.-4irrh.os-Btirrgnos,aejrs6qrs,"Pks57Vjrr?j6 -1&u7krrYqM13WZs!+#RZ!.FhG"4R;?iIV#[s+13$s5F!.~> -"!.F0jSo/[8C[V<!o$B@p&>9Qo`!>>s.-4irrh.os-Btirrgnos,aejrs6qrs,"Pks57Vjrr?j6 -1&u7krrYqM13WZs!+#RZ!.FhG"4R;?iIV#[s+13$s5F!.~> -"!.F0jSo/[8C[V<!o$B@p&>9Qo`!>>s.-4irrh.os-Btirrgnos,aejrs6qrs,"Pks57Vjrr?j6 -1&u7krrYqM13WZs!+#RZ!.FhG"4R;?iIV#[s+13$s5F!.~> -!$2%<!r>IAqu6]a0CSoAKpVf="P$'CI@pN='[0DRFK#6>hA08oCp<s>fgPl?E3K9+!NL>Brs'\0 -[oNJ.@D2[*!M"jps+13$s+14(s*t~> -!$2%<!r>IAqu6]a0CSoAKpVf="P$'CI@pN='[0DRFK#6>hA08oCp<s>fgPl?E3K9+!NL>Brs'\0 -[oNJ.@D2[*!M"jps+13$s+14(s*t~> -!$2%<!r>IAqu6]a0CSoAKpVf="P$'CI@pN='[0DRFK#6>hA08oCp<s>fgPl?E3K9+!NL>Brs'\0 -[oNJ.@D2[*!M"jps+13$s+14(s*t~> -!$2(=!pX(@qYpTT4R`:NKpVf="P$'CI>RsX'X(@5FK#5r5/^V(Cp<s>fgPl?A%DU=!OHM>rs'\E -aAr9?;m$&R!K28Ts+13$s+14(s*t~> -!$2(=!pX(@qYpTT4R`:NKpVf="P$'CI>RsX'X(@5FK#5r5/^V(Cp<s>fgPl?A%DU=!OHM>rs'\E -aAr9?;m$&R!K28Ts+13$s+14(s*t~> -!$2(=!pX(@qYpTT4R`:NKpVf="P$'CI>RsX'X(@5FK#5r5/^V(Cp<s>fgPl?A%DU=!OHM>rs'\E -aAr9?;m$&R!K28Ts+13$s+14(s*t~> -"WdXCiHE09rrUPJp@&"^KpVf="P$'CI4>.]'JrYdFK!D[s28(?Cp<s>fgPl?A%DU=%'s[Js34F> -aAr9?;`+G=!1\W?JcC<$JcFI(J,~> -"WdXCiHE09rrUPJp@&"^KpVf="P$'CI4>.]'JrYdFK!D[s28(?Cp<s>fgPl?A%DU=%'s[Js34F> -aAr9?;`+G=!1\W?JcC<$JcFI(J,~> -"WdXCiHE09rrUPJp@&"^KpVf="P$'CI4>.]'JrYdFK!D[s28(?Cp<s>fgPl?A%DU=%'s[Js34F> -aAr9?;`+G=!1\W?JcC<$JcFI(J,~> -"<IO"K"LmZ!lf6Amf*TIQiI*^L.M>mTDnj)bY\=]C2NA<`**+VYlFaV70!9s\c2U;ZYBI>52ZC] -<WE(tb45K5pS#?Qs+13$s+146s*t~> -"<IO"K"LmZ!lf6Amf*TIQiI*^L.M>mTDnj)bY\=]C2NA<`**+VYlFaV70!9s\c2U;ZYBI>52ZC] -<WE(tb45K5pS#?Qs+13$s+146s*t~> -"<IO"K"LmZ!lf6Amf*TIQiI*^L.M>mTDnj)bY\=]C2NA<`**+VYlFaV70!9s\c2U;ZYBI>52ZC] -<WE(tb45K5pS#?Qs+13$s+146s*t~> -"!.E>4oL$;QN.!%J(j;i$,D(EKS*o.s*^O=rtN[Rs)_,ss8U#?s(knmKQ&0Bs0%"QKFEF;s0*Ve -KP)jJs/(DIKE/:MrrVEb;1\aus+13$s60K5~> -"!.E>4oL$;QN.!%J(j;i$,D(EKS*o.s*^O=rtN[Rs)_,ss8U#?s(knmKQ&0Bs0%"QKFEF;s0*Ve -KP)jJs/(DIKE/:MrrVEb;1\aus+13$s60K5~> -"!.E>4oL$;QN.!%J(j;i$,D(EKS*o.s*^O=rtN[Rs)_,ss8U#?s(knmKQ&0Bs0%"QKFEF;s0*Ve -KP)jJs/(DIKE/:MrrVEb;1\aus+13$s60K5~> -!Zh<rr0RVbs8U<pli.7-R$aGp8H6ibrr3,d\GsV)rr3)`]`5qKQiqA:s8TJ*Qisnus''RDQiq#A -s8T8$QiOknrrVn]\Ujd3s+13$s60K5~> -!Zh<rr0RVbs8U<pli.7-R$aGp8H6ibrr3,d\GsV)rr3)`]`5qKQiqA:s8TJ*Qisnus''RDQiq#A -s8T8$QiOknrrVn]\Ujd3s+13$s60K5~> -!Zh<rr0RVbs8U<pli.7-R$aGp8H6ibrr3,d\GsV)rr3)`]`5qKQiqA:s8TJ*Qisnus''RDQiq#A -s8T8$QiOknrrVn]\Ujd3s+13$s60K5~> -!$1"t!NEL+rrLY@li.!t@JKj'cgC`3!Tqsas+13$s+13us*t~> -!$1"t!NEL+rrLY@li.!t@JKj'cgC`3!Tqsas+13$s+13us*t~> -!$1"t!NEL+rrLY@li.!t@JKj'cgC`3!Tqsas+13$s+13us*t~> -!$1"t!U4S&/HJH"li."Wj+75]s+13$s2+ed~> -!$1"t!U4S&/HJH"li."Wj+75]s+13$s2+ed~> -!$1"t!U4S&/HJH"li."Wj+75]s+13$s2+ed~> -!$2(="8MHXoB-&Qp&=C[nU^_$rr`#do(.G4JcDABJ,~> -!$2(="8MHXoB-&Qp&=C[nU^_$rr`#do(.G4JcDABJ,~> -!$2(="8MHXoB-&Qp&=C[nU^_$rr`#do(.G4JcDABJ,~> -!$2(=")T&-1&V%PiV<?N"6A%arr2otkl0-5g\h'Ph>HmE!8@>M!oDM`r;QiRf%p?*!9F(/!9j.V -!WCjOrrUg'j8ArWec=e0rr`5Nf&cQ(!VYROrr`)Jf'<87!<)lI!:]mc!U0(HrrMTRr;Qlif%0?h -rr_fBf(/e>!:^!=!;Q6e!U0(Ors%i@f(]4Eo_d8?g&D!Ol2K9+rrM*Rr;Qu_f%0j#s6]a8!oX+E -r72)2s8RBo.2IL7JcDABJ,~> -!$2(=")T&-1&V%PiV<?N"6A%arr2otkl0-5g\h'Ph>HmE!8@>M!oDM`r;QiRf%p?*!9F(/!9j.V -!WCjOrrUg'j8ArWec=e0rr`5Nf&cQ(!VYROrr`)Jf'<87!<)lI!:]mc!U0(HrrMTRr;Qlif%0?h -rr_fBf(/e>!:^!=!;Q6e!U0(Ors%i@f(]4Eo_d8?g&D!Ol2K9+rrM*Rr;Qu_f%0j#s6]a8!oX+E -r72)2s8RBo.2IL7JcDABJ,~> -!$2(=")T&-1&V%PiV<?N"6A%arr2otkl0-5g\h'Ph>HmE!8@>M!oDM`r;QiRf%p?*!9F(/!9j.V -!WCjOrrUg'j8ArWec=e0rr`5Nf&cQ(!VYROrr`)Jf'<87!<)lI!:]mc!U0(HrrMTRr;Qlif%0?h -rr_fBf(/e>!:^!=!;Q6e!U0(Ors%i@f(]4Eo_d8?g&D!Ol2K9+rrM*Rr;Qu_f%0j#s6]a8!oX+E -r72)2s8RBo.2IL7JcDABJ,~> -!$2(=!H#(<rrU\+ec#LRg2BGM^An5kH2^86<m(FA!mMnWo`"s6(&\(7c<Nh@bPqMIbZRD>cN!oK -2>ouERJ-X]ZRbtR"N*i$CY/Rc"MdJrDV>$h"M.&lEnpBg!r.jWrVm0%WCB@+j8].frC-gJmJ[%d -i)?WKrrVP,JGoK[ns2a7ORE/Kmug.0Pk"eRm>h08!9!SN!oR"WrVm,bK18>*qZ"e;7KaM,s5A>( -7KEG]rrU_-eG]CPg2KMN^&S,8rC-lP56%S[7KGS@r;Qa;JcC<$TDsE~> -!$2(=!H#(<rrU\+ec#LRg2BGM^An5kH2^86<m(FA!mMnWo`"s6(&\(7c<Nh@bPqMIbZRD>cN!oK -2>ouERJ-X]ZRbtR"N*i$CY/Rc"MdJrDV>$h"M.&lEnpBg!r.jWrVm0%WCB@+j8].frC-gJmJ[%d -i)?WKrrVP,JGoK[ns2a7ORE/Kmug.0Pk"eRm>h08!9!SN!oR"WrVm,bK18>*qZ"e;7KaM,s5A>( -7KEG]rrU_-eG]CPg2KMN^&S,8rC-lP56%S[7KGS@r;Qa;JcC<$TDsE~> -!$2(=!H#(<rrU\+ec#LRg2BGM^An5kH2^86<m(FA!mMnWo`"s6(&\(7c<Nh@bPqMIbZRD>cN!oK -2>ouERJ-X]ZRbtR"N*i$CY/Rc"MdJrDV>$h"M.&lEnpBg!r.jWrVm0%WCB@+j8].frC-gJmJ[%d -i)?WKrrVP,JGoK[ns2a7ORE/Kmug.0Pk"eRm>h08!9!SN!oR"WrVm,bK18>*qZ"e;7KaM,s5A>( -7KEG]rrU_-eG]CPg2KMN^&S,8rC-lP56%S[7KGS@r;Qa;JcC<$TDsE~> -!$2(=!H#(=rr^Pl*7b&g$.aRGs8Ql>s,N-=rs.@Es8U#Z'(>Mn!krU^rVmJLFlEAaEl.jnF6<M^ -FN+8a_=IU-X^NaYrt`"Xqu=rhdf6Ufs8U/hec2gjs8Tudg%YLJQVCKYrs5BFs8TH[ir:%trrHH? -rr3)XK.X(srr__K4b3P3')7TQs8SOJoCS*2s8S@Gp@4-5oD\pMK.O&%rs%YBXT/<NSGW<iX)\0u -KX^RQ"4)95eGfIPf7(a>s(nt<rrW1aM>[ATqNHs;rr='js+13Bs*t~> -!$2(=!H#(=rr^Pl*7b&g$.aRGs8Ql>s,N-=rs.@Es8U#Z'(>Mn!krU^rVmJLFlEAaEl.jnF6<M^ -FN+8a_=IU-X^NaYrt`"Xqu=rhdf6Ufs8U/hec2gjs8Tudg%YLJQVCKYrs5BFs8TH[ir:%trrHH? -rr3)XK.X(srr__K4b3P3')7TQs8SOJoCS*2s8S@Gp@4-5oD\pMK.O&%rs%YBXT/<NSGW<iX)\0u -KX^RQ"4)95eGfIPf7(a>s(nt<rrW1aM>[ATqNHs;rr='js+13Bs*t~> -!$2(=!H#(=rr^Pl*7b&g$.aRGs8Ql>s,N-=rs.@Es8U#Z'(>Mn!krU^rVmJLFlEAaEl.jnF6<M^ -FN+8a_=IU-X^NaYrt`"Xqu=rhdf6Ufs8U/hec2gjs8Tudg%YLJQVCKYrs5BFs8TH[ir:%trrHH? -rr3)XK.X(srr__K4b3P3')7TQs8SOJoCS*2s8S@Gp@4-5oD\pMK.O&%rs%YBXT/<NSGW<iX)\0u -KX^RQ"4)95eGfIPf7(a>s(nt<rrW1aM>[ATqNHs;rr='js+13Bs*t~> -!$2(=!H#(>rrgYtbt$=`rrJF?rr3,`2uenurr35Z5QATNg+Dderr]f^iZ8!t!Go">rt)eNs0@.> -s6"m>s'*=#It+Hlrr]'Il7;i5!CGQ?rsY$Js.+]?s3cD>s"Wm>rrIY?q#:GWNQ5&trr=)<rrZoB -ruM(<#[Y7Fs5ea>F/es7"T!VCSUg_>!UMN=rsLsInURY>s/:J?gHkH2"QY'BEiT-=!Q&(>rr=): -rra_Bs06D3rrgYtbt-@`rrJI?rr3#^2Z3RUq]Yk:!V7c;rr='js+13Bs*t~> -!$2(=!H#(>rrgYtbt$=`rrJF?rr3,`2uenurr35Z5QATNg+Dderr]f^iZ8!t!Go">rt)eNs0@.> -s6"m>s'*=#It+Hlrr]'Il7;i5!CGQ?rsY$Js.+]?s3cD>s"Wm>rrIY?q#:GWNQ5&trr=)<rrZoB -ruM(<#[Y7Fs5ea>F/es7"T!VCSUg_>!UMN=rsLsInURY>s/:J?gHkH2"QY'BEiT-=!Q&(>rr=): -rra_Bs06D3rrgYtbt-@`rrJI?rr3#^2Z3RUq]Yk:!V7c;rr='js+13Bs*t~> -!$2(=!H#(>rrgYtbt$=`rrJF?rr3,`2uenurr35Z5QATNg+Dderr]f^iZ8!t!Go">rt)eNs0@.> -s6"m>s'*=#It+Hlrr]'Il7;i5!CGQ?rsY$Js.+]?s3cD>s"Wm>rrIY?q#:GWNQ5&trr=)<rrZoB -ruM(<#[Y7Fs5ea>F/es7"T!VCSUg_>!UMN=rsLsInURY>s/:J?gHkH2"QY'BEiT-=!Q&(>rr=): -rra_Bs06D3rrgYtbt-@`rrJI?rr3#^2Z3RUq]Yk:!V7c;rr='js+13Bs*t~> -!$2(=!H#(>rrh'#s$tW=rrJF?rr3Mk2uenus8W!s5QB*^s"3^5rr^F#ruM(<!Go">rt2kOs8TTE -@Vr^Vs'&*]SXkV9q#:H,mdVh+rrGI@rr3ChMuWgT@UaQ)s8Oh?rr3"LSG<*dWW'q<rVlj<rVlt4 -[f7BIrs>hR^Am_@]`3&Pq#:KoXT+iArr3#c0`D"[4Q-8(M.0qOli5^*`Vs<HjS8`Xlb3==WrE#! -_H[!=!$2";#)nfDs06C@raGm,q#:KNc2U>arr3"cL&V)Qk;N8<!HY:<rrI&?qu6X:JcC<$TDsE~> -!$2(=!H#(>rrh'#s$tW=rrJF?rr3Mk2uenus8W!s5QB*^s"3^5rr^F#ruM(<!Go">rt2kOs8TTE -@Vr^Vs'&*]SXkV9q#:H,mdVh+rrGI@rr3ChMuWgT@UaQ)s8Oh?rr3"LSG<*dWW'q<rVlj<rVlt4 -[f7BIrs>hR^Am_@]`3&Pq#:KoXT+iArr3#c0`D"[4Q-8(M.0qOli5^*`Vs<HjS8`Xlb3==WrE#! -_H[!=!$2";#)nfDs06C@raGm,q#:KNc2U>arr3"cL&V)Qk;N8<!HY:<rrI&?qu6X:JcC<$TDsE~> -!$2(=!H#(>rrh'#s$tW=rrJF?rr3Mk2uenus8W!s5QB*^s"3^5rr^F#ruM(<!Go">rt2kOs8TTE -@Vr^Vs'&*]SXkV9q#:H,mdVh+rrGI@rr3ChMuWgT@UaQ)s8Oh?rr3"LSG<*dWW'q<rVlj<rVlt4 -[f7BIrs>hR^Am_@]`3&Pq#:KoXT+iArr3#c0`D"[4Q-8(M.0qOli5^*`Vs<HjS8`Xlb3==WrE#! -_H[!=!$2";#)nfDs06C@raGm,q#:KNc2U>arr3"cL&V)Qk;N8<!HY:<rrI&?qu6X:JcC<$TDsE~> -!$2(=!H#(;rrGX?rr3"bLAq2Zju<=uGst.lTg/MU!A3d3rr=)<rrHr?rr3SB@K6@PVn/[Ps8UQZ -pAb/mEq]M6nLOS<!CGQ?rsY$Js8Q96\riK^s"Wm>rrIY?pAY06<W2pt*W?!?B=@iLr^@-E7DAe$ -aT)9]WV-8lSUg_>!UMN=rsV$Js42%u\n]pugCeK)\cGt,pAY/6WrE#!_H[!=!$2";#"7,Ds03St -rji)5p&>%`eGfIKQ\GG=!TuB=rr=)9rr=)9rr='js+13Bs*t~> -!$2(=!H#(;rrGX?rr3"bLAq2Zju<=uGst.lTg/MU!A3d3rr=)<rrHr?rr3SB@K6@PVn/[Ps8UQZ -pAb/mEq]M6nLOS<!CGQ?rsY$Js8Q96\riK^s"Wm>rrIY?pAY06<W2pt*W?!?B=@iLr^@-E7DAe$ -aT)9]WV-8lSUg_>!UMN=rsV$Js42%u\n]pugCeK)\cGt,pAY/6WrE#!_H[!=!$2";#"7,Ds03St -rji)5p&>%`eGfIKQ\GG=!TuB=rr=)9rr=)9rr='js+13Bs*t~> -!$2(=!H#(;rrGX?rr3"bLAq2Zju<=uGst.lTg/MU!A3d3rr=)<rrHr?rr3SB@K6@PVn/[Ps8UQZ -pAb/mEq]M6nLOS<!CGQ?rsY$Js8Q96\riK^s"Wm>rrIY?pAY06<W2pt*W?!?B=@iLr^@-E7DAe$ -aT)9]WV-8lSUg_>!UMN=rsV$Js42%u\n]pugCeK)\cGt,pAY/6WrE#!_H[!=!$2";#"7,Ds03St -rji)5p&>%`eGfIKQ\GG=!TuB=rr=)9rr=)9rr='js+13Bs*t~> -!$2(=!H#(;rrGX?rr3"bLAq2Zju<>?l.=P>h*:l<!A3d3rr=)<rrHr?rr3;:@K1_Qq>^5bJc#HJ -WcIh5!V%]=rrGI@rr3ChMuQnts8VVDV>i::rr3"LSG)s`a&W*<!$2%<")#(BrVkCOls@F+$N;FT -&f]0jrrJ[@rr3#c0`D"[4Q,F!jT#8@9^LLmk5PA]/+N?5!HP4>rrKu@rVlj<rVm+KW;6JnZ!6So -rr=)4rrG[?rr3"cL&V)Qk;N;=!E[;<rrGp?qYpO9JcC<$TDsE~> -!$2(=!H#(;rrGX?rr3"bLAq2Zju<>?l.=P>h*:l<!A3d3rr=)<rrHr?rr3;:@K1_Qq>^5bJc#HJ -WcIh5!V%]=rrGI@rr3ChMuQnts8VVDV>i::rr3"LSG)s`a&W*<!$2%<")#(BrVkCOls@F+$N;FT -&f]0jrrJ[@rr3#c0`D"[4Q,F!jT#8@9^LLmk5PA]/+N?5!HP4>rrKu@rVlj<rVm+KW;6JnZ!6So -rr=)4rrG[?rr3"cL&V)Qk;N;=!E[;<rrGp?qYpO9JcC<$TDsE~> -!$2(=!H#(;rrGX?rr3"bLAq2Zju<>?l.=P>h*:l<!A3d3rr=)<rrHr?rr3;:@K1_Qq>^5bJc#HJ -WcIh5!V%]=rrGI@rr3ChMuQnts8VVDV>i::rr3"LSG)s`a&W*<!$2%<")#(BrVkCOls@F+$N;FT -&f]0jrrJ[@rr3#c0`D"[4Q,F!jT#8@9^LLmk5PA]/+N?5!HP4>rrKu@rVlj<rVm+KW;6JnZ!6So -rr=)4rrG[?rr3"cL&V)Qk;N;=!E[;<rrGp?qYpO9JcC<$TDsE~> -!$2(=!H#(;rrGX?rr31nL&(cKfL5W:!ROO=rrF_?qu6]dn,*+b*W?!MG,kK?r3L5>DR'->r2b#? -o_\Xf!NL2=rrMNXr;Qff/,fJ`:X9"?q2^a>8(IY>q1kI>5McA>pjf.=rrW,cp&+gja&W*<"t]9D -s7^9?r;QlmA&J?FoE&p)r;Q]tq<dtTrrJ[@rr3St47iLPoL@j%6hC?Xo02Ho62gfa/+NW=!qu$Y -r;Qe<WrE#&`aJN?s7%Z>rrVn+_#F?7Z#'C=!$2%<!qYgXr;QdgeGfIPSqQq>s4Kd=rrVdXkPY>] -o/c@:rr='js+13Bs*t~> -!$2(=!H#(;rrGX?rr31nL&(cKfL5W:!ROO=rrF_?qu6]dn,*+b*W?!MG,kK?r3L5>DR'->r2b#? -o_\Xf!NL2=rrMNXr;Qff/,fJ`:X9"?q2^a>8(IY>q1kI>5McA>pjf.=rrW,cp&+gja&W*<"t]9D -s7^9?r;QlmA&J?FoE&p)r;Q]tq<dtTrrJ[@rr3St47iLPoL@j%6hC?Xo02Ho62gfa/+NW=!qu$Y -r;Qe<WrE#&`aJN?s7%Z>rrVn+_#F?7Z#'C=!$2%<!qYgXr;QdgeGfIPSqQq>s4Kd=rrVdXkPY>] -o/c@:rr='js+13Bs*t~> -!$2(=!H#(;rrGX?rr31nL&(cKfL5W:!ROO=rrF_?qu6]dn,*+b*W?!MG,kK?r3L5>DR'->r2b#? -o_\Xf!NL2=rrMNXr;Qff/,fJ`:X9"?q2^a>8(IY>q1kI>5McA>pjf.=rrW,cp&+gja&W*<"t]9D -s7^9?r;QlmA&J?FoE&p)r;Q]tq<dtTrrJ[@rr3St47iLPoL@j%6hC?Xo02Ho62gfa/+NW=!qu$Y -r;Qe<WrE#&`aJN?s7%Z>rrVn+_#F?7Z#'C=!$2%<!qYgXr;QdgeGfIPSqQq>s4Kd=rrVdXkPY>] -o/c@:rr='js+13Bs*t~> -!$2(=!H#(>rseACd5nGXg&M)7QI4)Lrr3GhdF$63s8V?2d3Z]XhYmHT,o6L[%Hc'c(t$ais8R#B -dC<ffrs3;IdBd]is*:UCd/f_qrr3&r!'pP`%GoLO,LOp&s8Pd:d@H*mrrbg=d?p!orrbd<d?9go -rrULHCB"5Ef$U6^dF%I`qF/fZJueqOr;P(EIBrbM!HY7=ruS!:PlLd^dF!JXdF%ahf3m"RB?pPW -d:L_Q@b(M<=T-VJ9(W&^'70_os8Vc>d9V[XdJs6SHdU58nG`FgJ"HW=!P`^Sd/ZZ7rt8-.l2UeO -dEt%YdFnR"P*XMC7fE>f<64t:!DLl9rr='js+13Bs*t~> -!$2(=!H#(>rseACd5nGXg&M)7QI4)Lrr3GhdF$63s8V?2d3Z]XhYmHT,o6L[%Hc'c(t$ais8R#B -dC<ffrs3;IdBd]is*:UCd/f_qrr3&r!'pP`%GoLO,LOp&s8Pd:d@H*mrrbg=d?p!orrbd<d?9go -rrULHCB"5Ef$U6^dF%I`qF/fZJueqOr;P(EIBrbM!HY7=ruS!:PlLd^dF!JXdF%ahf3m"RB?pPW -d:L_Q@b(M<=T-VJ9(W&^'70_os8Vc>d9V[XdJs6SHdU58nG`FgJ"HW=!P`^Sd/ZZ7rt8-.l2UeO -dEt%YdFnR"P*XMC7fE>f<64t:!DLl9rr='js+13Bs*t~> -!$2(=!H#(>rseACd5nGXg&M)7QI4)Lrr3GhdF$63s8V?2d3Z]XhYmHT,o6L[%Hc'c(t$ais8R#B -dC<ffrs3;IdBd]is*:UCd/f_qrr3&r!'pP`%GoLO,LOp&s8Pd:d@H*mrrbg=d?p!orrbd<d?9go -rrULHCB"5Ef$U6^dF%I`qF/fZJueqOr;P(EIBrbM!HY7=ruS!:PlLd^dF!JXdF%ahf3m"RB?pPW -d:L_Q@b(M<=T-VJ9(W&^'70_os8Vc>d9V[XdJs6SHdU58nG`FgJ"HW=!P`^Sd/ZZ7rt8-.l2UeO -dEt%YdFnR"P*XMC7fE>f<64t:!DLl9rr='js+13Bs*t~> -!$2(=#&Sap[f?B>r(@$+rr3(Z92#3^rr\``92GQe!3,lh!.OnH!G*,IrrAVf9)skKrrR(2B)MZ1 -BM31?rr@9B9)sbGrrW'`5lUc`JG`%?W;ceu:eQK?rr`6n98`]J"8GM^OSo+[h)/sHrr?O+9*WB8 -s5lsIUAk/mp&7SaW;?MpSAY=*"i)ONs8Vlk9*XGVs2[i+_#=98_FmPArr2uGrC[.KrVm%q*5DOR -gAFSD:&b.pX%Q+BrVlnp]);R.Zi*Msrr)j#KFd>Is1J8&!,MT6"-#rHr;HWrLY2M(!J0,*rr^WT -Zl=SrJcDABJ,~> -!$2(=#&Sap[f?B>r(@$+rr3(Z92#3^rr\``92GQe!3,lh!.OnH!G*,IrrAVf9)skKrrR(2B)MZ1 -BM31?rr@9B9)sbGrrW'`5lUc`JG`%?W;ceu:eQK?rr`6n98`]J"8GM^OSo+[h)/sHrr?O+9*WB8 -s5lsIUAk/mp&7SaW;?MpSAY=*"i)ONs8Vlk9*XGVs2[i+_#=98_FmPArr2uGrC[.KrVm%q*5DOR -gAFSD:&b.pX%Q+BrVlnp]);R.Zi*Msrr)j#KFd>Is1J8&!,MT6"-#rHr;HWrLY2M(!J0,*rr^WT -Zl=SrJcDABJ,~> -!$2(=#&Sap[f?B>r(@$+rr3(Z92#3^rr\``92GQe!3,lh!.OnH!G*,IrrAVf9)skKrrR(2B)MZ1 -BM31?rr@9B9)sbGrrW'`5lUc`JG`%?W;ceu:eQK?rr`6n98`]J"8GM^OSo+[h)/sHrr?O+9*WB8 -s5lsIUAk/mp&7SaW;?MpSAY=*"i)ONs8Vlk9*XGVs2[i+_#=98_FmPArr2uGrC[.KrVm%q*5DOR -gAFSD:&b.pX%Q+BrVlnp]);R.Zi*Msrr)j#KFd>Is1J8&!,MT6"-#rHr;HWrLY2M(!J0,*rr^WT -Zl=SrJcDABJ,~> -!$2(="/omHD;5'j*S^Spgd'Wo!O?IprrIq?g]%:[^YAbh1[3cr"/0CAFFjICs.B=A~> -!$2(="/omHD;5'j*S^Spgd'Wo!O?IprrIq?g]%:[^YAbh1[3cr"/0CAFFjICs.B=A~> -!$2(="/omHD;5'j*S^Spgd'Wo!O?IprrIq?g]%:[^YAbh1[3cr"/0CAFFjICs.B=A~> -!$0Vi!KE/qrrW+FUtktOk'abprrUPsbhW4"\q/eprrT!Kon!-os+14Fs*t~> -!$0Vi!KE/qrrW+FUtktOk'abprrUPsbhW4"\q/eprrT!Kon!-os+14Fs*t~> -!$0Vi!KE/qrrW+FUtktOk'abprrUPsbhW4"\q/eprrT!Kon!-os+14Fs*t~> -!$0Vi!O2LWrrN)%g]%<B^>&YghTjmU!RKTVrrKn$JcC<$JcGKEJ,~> -!$0Vi!O2LWrrN)%g]%<B^>&YghTjmU!RKTVrrKn$JcC<$JcGKEJ,~> -!$0Vi!O2LWrrN)%g]%<B^>&YghTjmU!RKTVrrKn$JcC<$JcGKEJ,~> -!$.X1!SDh6rrC7A5QK%#rrJ>#p&>&uU\t,l_>Z\"ec#LKV6GONJcC<$hZ&*~> -!$.X1!SDh6rrC7A5QK%#rrJ>#p&>&uU\t,l_>Z\"ec#LKV6GONJcC<$hZ&*~> -!$.X1!SDh6rrC7A5QK%#rrJ>#p&>&uU\t,l_>Z\"ec#LKV6GONJcC<$hZ&*~> -!Zh<Ir$)>4s8S:^o)Ac,,O4Up!^%].ec,^;8E]sO"TCUbZAJS\"Z/[Js,sDOrs&<E>Q=_N_"[j3 -dV8`9[J'V'\lIP@p@\FeS;d+>rrC+<,66HsrrV_<Y5SA'o\A4na8Q#>mVojjs+13$s53j,~> -!Zh<Ir$)>4s8S:^o)Ac,,O4Up!^%].ec,^;8E]sO"TCUbZAJS\"Z/[Js,sDOrs&<E>Q=_N_"[j3 -dV8`9[J'V'\lIP@p@\FeS;d+>rrC+<,66HsrrV_<Y5SA'o\A4na8Q#>mVojjs+13$s53j,~> -!Zh<Ir$)>4s8S:^o)Ac,,O4Up!^%].ec,^;8E]sO"TCUbZAJS\"Z/[Js,sDOrs&<E>Q=_N_"[j3 -dV8`9[J'V'\lIP@p@\FeS;d+>rrC+<,66HsrrV_<Y5SA'o\A4na8Q#>mVojjs+13$s53j,~> -!Zh=>rV6]S3WK-WF0PQ6rrW)nrp9Xf5Ml4O?/GS],PqE@f1,]<&cNmPVK`(?2WiUik^]N0s6V]9 -rrs_D_.`%J]_DF/b>J:\WV6>m\mk:2!r%S@rr3#S7/co^0_,/B!f!3?qu6[sa8Gr=GH(Ijs+13$ -s5<p-~> -!Zh=>rV6]S3WK-WF0PQ6rrW)nrp9Xf5Ml4O?/GS],PqE@f1,]<&cNmPVK`(?2WiUik^]N0s6V]9 -rrs_D_.`%J]_DF/b>J:\WV6>m\mk:2!r%S@rr3#S7/co^0_,/B!f!3?qu6[sa8Gr=GH(Ijs+13$ -s5<p-~> -!Zh=>rV6]S3WK-WF0PQ6rrW)nrp9Xf5Ml4O?/GS],PqE@f1,]<&cNmPVK`(?2WiUik^]N0s6V]9 -rrs_D_.`%J]_DF/b>J:\WV6>m\mk:2!r%S@rr3#S7/co^0_,/B!f!3?qu6[sa8Gr=GH(Ijs+13$ -s5<p-~> -!$2";!S0d=rrR#CrV$6nilfOArVllVrlY;kr;QfHd/<nK3o^1dHN4$"rlYl+s8V$Is3AgBs8Us! -bPB?;rr2uLrlYT(s8Ufr(X("[rVlj<qu7B09`MVcs8Q9ls2[$bs8UT^-HjTqrr2uBrlY`7s8Pal -s0OVbs8UL?rlY<.rr3,OFoRN7pAY0)@f?<-qpkZF"k`YNs/C)<rrM+mrr3&R:%\Da!F<J;rrH-? -qu6[kd"24Js+14.s*t~> -!$2";!S0d=rrR#CrV$6nilfOArVllVrlY;kr;QfHd/<nK3o^1dHN4$"rlYl+s8V$Is3AgBs8Us! -bPB?;rr2uLrlYT(s8Ufr(X("[rVlj<qu7B09`MVcs8Q9ls2[$bs8UT^-HjTqrr2uBrlY`7s8Pal -s0OVbs8UL?rlY<.rr3,OFoRN7pAY0)@f?<-qpkZF"k`YNs/C)<rrM+mrr3&R:%\Da!F<J;rrH-? -qu6[kd"24Js+14.s*t~> -!$2";!S0d=rrR#CrV$6nilfOArVllVrlY;kr;QfHd/<nK3o^1dHN4$"rlYl+s8V$Is3AgBs8Us! -bPB?;rr2uLrlYT(s8Ufr(X("[rVlj<qu7B09`MVcs8Q9ls2[$bs8UT^-HjTqrr2uBrlY`7s8Pal -s0OVbs8UL?rlY<.rr3,OFoRN7pAY0)@f?<-qpkZF"k`YNs/C)<rrM+mrr3&R:%\Da!F<J;rrH-? -qu6[kd"24Js+14.s*t~> -!$2%<!luR\qu6\6YP.tuSPTp\rr3#PGQ(D>@)`*E!@8NZrrG1?rVloGD>m?GC!H[BQ*41aD:&;e -4]%qNE7=jeFoG2CF4L;C:^KrpQMpg`*UNe4CeG:8:p'ct\P`?&!J]_2rrF"/r_NWSlgOiS\mkX< -!UMN=rrkOCs8T'>q>UNBCZ53k!F<J;rrH-?qu6[kd"24Js+14.s*t~> -!$2%<!luR\qu6\6YP.tuSPTp\rr3#PGQ(D>@)`*E!@8NZrrG1?rVloGD>m?GC!H[BQ*41aD:&;e -4]%qNE7=jeFoG2CF4L;C:^KrpQMpg`*UNe4CeG:8:p'ct\P`?&!J]_2rrF"/r_NWSlgOiS\mkX< -!UMN=rrkOCs8T'>q>UNBCZ53k!F<J;rrH-?qu6[kd"24Js+14.s*t~> -!$2%<!luR\qu6\6YP.tuSPTp\rr3#PGQ(D>@)`*E!@8NZrrG1?rVloGD>m?GC!H[BQ*41aD:&;e -4]%qNE7=jeFoG2CF4L;C:^KrpQMpg`*UNe4CeG:8:p'ct\P`?&!J]_2rrF"/r_NWSlgOiS\mkX< -!UMN=rrkOCs8T'>q>UNBCZ53k!F<J;rrH-?qu6[kd"24Js+14.s*t~> -!$2%<".O.FUAOrj*VB@42Wk#=!JQm>rrLY@rVlnMHN*pI`0)9pTDnikCp<p=&\DKOs$g8gs0HG> ->J^=[s/L,:rrJj?rr2s=qYpO9li."Q0`:qO+T;<>!Nj`DrrEjhrr3"*^@2(&\mkX<$hSPIs8TTP -mJm3cF8Gn<b'_b:rrHE?qu6[sa8>l;:!eIkJcC<$i;\<~> -!$2%<".O.FUAOrj*VB@42Wk#=!JQm>rrLY@rVlnMHN*pI`0)9pTDnikCp<p=&\DKOs$g8gs0HG> ->J^=[s/L,:rrJj?rr2s=qYpO9li."Q0`:qO+T;<>!Nj`DrrEjhrr3"*^@2(&\mkX<$hSPIs8TTP -mJm3cF8Gn<b'_b:rrHE?qu6[sa8>l;:!eIkJcC<$i;\<~> -!$2%<".O.FUAOrj*VB@42Wk#=!JQm>rrLY@rVlnMHN*pI`0)9pTDnikCp<p=&\DKOs$g8gs0HG> ->J^=[s/L,:rrJj?rr2s=qYpO9li."Q0`:qO+T;<>!Nj`DrrEjhrr3"*^@2(&\mkX<$hSPIs8TTP -mJm3cF8Gn<b'_b:rrHE?qu6[sa8>l;:!eIkJcC<$i;\<~> -!$1t:!V[r=rrJ%?o`"qMk5PA]KpVf=!S'a9rrg)o++'CSrrHr?rr3\E@K6?sf)Pd*])Q!NruM-> -ErZ0&rcJ66HN*pE*W#d9*UE_+m4eJ<!1Ee.!5nR2!$2%<!FEM/rrK]?r;QrH:B1>t_uBZ:WH7t: -!mH/@qYpS%^\e$3<QG":!DUpls+13$s5<p-~> -!$1t:!V[r=rrJ%?o`"qMk5PA]KpVf=!S'a9rrg)o++'CSrrHr?rr3\E@K6?sf)Pd*])Q!NruM-> -ErZ0&rcJ66HN*pE*W#d9*UE_+m4eJ<!1Ee.!5nR2!$2%<!FEM/rrK]?r;QrH:B1>t_uBZ:WH7t: -!mH/@qYpS%^\e$3<QG":!DUpls+13$s5<p-~> -!$1t:!V[r=rrJ%?o`"qMk5PA]KpVf=!S'a9rrg)o++'CSrrHr?rr3\E@K6?sf)Pd*])Q!NruM-> -ErZ0&rcJ66HN*pE*W#d9*UE_+m4eJ<!1Ee.!5nR2!$2%<!FEM/rrK]?r;QrH:B1>t_uBZ:WH7t: -!mH/@qYpS%^\e$3<QG":!DUpls+13$s5<p-~> -!$1t:!V[r>rrRqDqt0mh2Wk#=!JQm>[email protected]/lXCp<p="hS4Cs$kT;rs;oGruM->ErV/d -rhfd3HN*pE*W#d9*UE_+m4eG;!3#mp!IDl_rr=)<rrHH?n,EF"@f660mZ3s:MsC<A!NC/<rrU\m -deWnD?,-::!EI2;rrGj@JcC<$JcF[.J,~> -!$1t:!V[r>rrRqDqt0mh2Wk#=!JQm>[email protected]/lXCp<p="hS4Cs$kT;rs;oGruM->ErV/d -rhfd3HN*pE*W#d9*UE_+m4eG;!3#mp!IDl_rr=)<rrHH?n,EF"@f660mZ3s:MsC<A!NC/<rrU\m -deWnD?,-::!EI2;rrGj@JcC<$JcF[.J,~> -!$1t:!V[r>rrRqDqt0mh2Wk#=!JQm>[email protected]/lXCp<p="hS4Cs$kT;rs;oGruM->ErV/d -rhfd3HN*pE*W#d9*UE_+m4eG;!3#mp!IDl_rr=)<rrHH?n,EF"@f660mZ3s:MsC<A!NC/<rrU\m -deWnD?,-::!EI2;rrGj@JcC<$JcF[.J,~> -"!.ESQ2^dapEon>!f3H@oD\hLk5PA]KpVf=!S'a>rrH1pr;QdYir/lXCp<p="hS4Cs$kT;rsi8L -ruM->ErT(\s8U@jHN*pI*WQ/ET`4rl\mt+,#jj>Gs-L=s;uQ^q!H5+8rr=)<rrHH?pAY3*40AJ] -!P2b;rr]NAGGY9<!pbT@rr3&[?J>5T!h#5@qu6[sa8Gr=N19J;rrRp:iIV#[s+146s*t~> -"!.ESQ2^dapEon>!f3H@oD\hLk5PA]KpVf=!S'a>rrH1pr;QdYir/lXCp<p="hS4Cs$kT;rsi8L -ruM->ErT(\s8U@jHN*pI*WQ/ET`4rl\mt+,#jj>Gs-L=s;uQ^q!H5+8rr=)<rrHH?pAY3*40AJ] -!P2b;rr]NAGGY9<!pbT@rr3&[?J>5T!h#5@qu6[sa8Gr=N19J;rrRp:iIV#[s+146s*t~> -"!.ESQ2^dapEon>!f3H@oD\hLk5PA]KpVf=!S'a>rrH1pr;QdYir/lXCp<p="hS4Cs$kT;rsi8L -ruM->ErT(\s8U@jHN*pI*WQ/ET`4rl\mt+,#jj>Gs-L=s;uQ^q!H5+8rr=)<rrHH?pAY3*40AJ] -!P2b;rr]NAGGY9<!pbT@rr3&[?J>5T!h#5@qu6[sa8Gr=N19J;rrRp:iIV#[s+146s*t~> -"!.F7L]/>!-0>1,OHfM3rs4/s"rf])s7("+*W\IhrVlmE(B"153o^,<!Ua.j*X,I.s8PR>r;R72 -_>iTrWcJ.*7ii^IO*^g*"P=b:D".H(!U2E,rs8S9*Zg(.s5bC&*W_5mq#:=7rVln)^@qR.V#ZbY -rrK]?qu6clO,!<)rrJ(@rr3#R#lG_DoD\ajH_UB:!EI2<rrHl?qu6_+!8)l&JcC<$kl6/~> -"!.F7L]/>!-0>1,OHfM3rs4/s"rf])s7("+*W\IhrVlmE(B"153o^,<!Ua.j*X,I.s8PR>r;R72 -_>iTrWcJ.*7ii^IO*^g*"P=b:D".H(!U2E,rs8S9*Zg(.s5bC&*W_5mq#:=7rVln)^@qR.V#ZbY -rrK]?qu6clO,!<)rrJ(@rr3#R#lG_DoD\ajH_UB:!EI2<rrHl?qu6_+!8)l&JcC<$kl6/~> -"!.F7L]/>!-0>1,OHfM3rs4/s"rf])s7("+*W\IhrVlmE(B"153o^,<!Ua.j*X,I.s8PR>r;R72 -_>iTrWcJ.*7ii^IO*^g*"P=b:D".H(!U2E,rs8S9*Zg(.s5bC&*W_5mq#:=7rVln)^@qR.V#ZbY -rrK]?qu6clO,!<)rrJ(@rr3#R#lG_DoD\ajH_UB:!EI2<rrHl?qu6_+!8)l&JcC<$kl6/~> -!Zh=@r;Q]q!<2Qhr;Q]qr;QWos8Mrr!<2lqr;Q]q!<2lq!<2or!ri6"rVcitrr)lrrqucsct2PC -rr2osr;QHj!<2rs!<2Wj!Jcp<rrg.X*?E#qs8W)qrrW1K\,QF)rr2ouUltZU"L]<k*6nH^!L9>S -s8W(Ls+13$s6'E4~> -!Zh=@r;Q]q!<2Qhr;Q]qr;QWos8Mrr!<2lqr;Q]q!<2lq!<2or!ri6"rVcitrr)lrrqucsct2PC -rr2osr;QHj!<2rs!<2Wj!Jcp<rrg.X*?E#qs8W)qrrW1K\,QF)rr2ouUltZU"L]<k*6nH^!L9>S -s8W(Ls+13$s6'E4~> -!Zh=@r;Q]q!<2Qhr;Q]qr;QWos8Mrr!<2lqr;Q]q!<2lq!<2or!ri6"rVcitrr)lrrqucsct2PC -rr2osr;QHj!<2rs!<2Wj!Jcp<rrg.X*?E#qs8W)qrrW1K\,QF)rr2ouUltZU"L]<k*6nH^!L9>S -s8W(Ls+13$s6'E4~> -!$-XjhZ!ZCKWKb)JcC<$W;hA~> -!$-XjhZ!ZCKWKb)JcC<$W;hA~> -!$-XjhZ!ZCKWKb)JcC<$W;hA~> -!$-XjhZ!VoV1JYts+13Js*t~> -!$-XjhZ!VoV1JYts+13Js*t~> -!$-XjhZ!VoV1JYts+13Js*t~> -!$-XjJcC<$JcC?%J,~> -!$-XjJcC<$JcC?%J,~> -!$-XjJcC<$JcC?%J,~> -!$-XjJcC<$JcC?%J,~> -!$-XjJcC<$JcC?%J,~> -!$-XjJcC<$JcC?%J,~> -!$-XjJcC<$JcC?%J,~> -!$-XjJcC<$JcC?%J,~> -!$-XjJcC<$JcC?%J,~> -!$-XjJcC<$JcC?%J,~> -!$-XjJcC<$JcC?%J,~> -!$-XjJcC<$JcC?%J,~> -"!.EGKDtlRpH/DRMZ3VYn3d5LP5bIij\#Q@RfEEb3<0$WGlA]a6N$i_#?;`a8,rVfC&T+R;>l(G -JcC<$JcE=]J,~> -"!.EGKDtlRpH/DRMZ3VYn3d5LP5bIij\#Q@RfEEb3<0$WGlA]a6N$i_#?;`a8,rVfC&T+R;>l(G -JcC<$JcE=]J,~> -"!.EGKDtlRpH/DRMZ3VYn3d5LP5bIij\#Q@RfEEb3<0$WGlA]a6N$i_#?;`a8,rVfC&T+R;>l(G -JcC<$JcE=]J,~> -"!.E>FoMCDpEop4IfB?]mOnO*LB%;6+?0=\NrT.U,6.[sQi@!ceO]]>#=@//1]RJsW;cet`*%Zk -JcC<$JcE=]J,~> -"!.E>FoMCDpEop4IfB?]mOnO*LB%;6+?0=\NrT.U,6.[sQi@!ceO]]>#=@//1]RJsW;cet`*%Zk -JcC<$JcE=]J,~> -"!.E>FoMCDpEop4IfB?]mOnO*LB%;6+?0=\NrT.U,6.[sQi@!ceO]]>#=@//1]RJsW;cet`*%Zk -JcC<$JcE=]J,~> -"!.E>FoMCDpEop4?iIkU[4_KFLB#Xsj#@!uNrT.U,6.[sQi@!ceO]]>#=@//1]RJsJ,[7bPZ`S; -JcC<$JcE=]J,~> -"!.E>FoMCDpEop4?iIkU[4_KFLB#Xsj#@!uNrT.U,6.[sQi@!ceO]]>#=@//1]RJsJ,[7bPZ`S; -JcC<$JcE=]J,~> -"!.E>FoMCDpEop4?iIkU[4_KFLB#Xsj#@!uNrT.U,6.[sQi@!ceO]]>#=@//1]RJsJ,[7bPZ`S; -JcC<$JcE=]J,~> -"!.E>FoMCDpEop4/cJof<%e3:L;e#pju<=#NrT.U,6.[sQi@!neO]`?`bsu(1]RJs4T/F`P_&jc -s+13$s185\~> -"!.E>FoMCDpEop4/cJof<%e3:L;e#pju<=#NrT.U,6.[sQi@!neO]`?`bsu(1]RJs4T/F`P_&jc -s+13$s185\~> -"!.E>FoMCDpEop4/cJof<%e3:L;e#pju<=#NrT.U,6.[sQi@!neO]`?`bsu(1]RJs4T/F`P_&jc -s+13$s185\~> -"!.E>FoMCDN@+[tIfB?]mOnO*7t'7oju<=#NrT.U,6.[sQi@!neO]`?@(lU)1]RJsW.Fu"s+13$ -s0_lW~> -"!.E>FoMCDN@+[tIfB?]mOnO*7t'7oju<=#NrT.U,6.[sQi@!neO]`?@(lU)1]RJsW.Fu"s+13$ -s0_lW~> -"!.E>FoMCDN@+[tIfB?]mOnO*7t'7oju<=#NrT.U,6.[sQi@!neO]`?@(lU)1]RJsW.Fu"s+13$ -s0_lW~> -"!.EqJc9EfN$eRsIfB?]mOnO*:>,[Iju<=#4DS_H%ZgY6I/[miEjG_O-r4Vf'q,*@H2_RZNIh+\ -s+13$s1//[~> -"!.EqJc9EfN$eRsIfB?]mOnO*:>,[Iju<=#4DS_H%ZgY6I/[miEjG_O-r4Vf'q,*@H2_RZNIh+\ -s+13$s1//[~> -"!.EqJc9EfN$eRsIfB?]mOnO*:>,[Iju<=#4DS_H%ZgY6I/[miEjG_O-r4Vf'q,*@H2_RZNIh+\ -s+13$s1//[~> -"!.FAg]"G\n0\1pfDbdQqS3'fgA_*SpVQscrNQKo5QCc2rilU?s8RLcrilTd;#gR>rilI@JcC<$ -JcC<$\c70~> -"!.FAg]"G\n0\1pfDbdQqS3'fgA_*SpVQscrNQKo5QCc2rilU?s8RLcrilTd;#gR>rilI@JcC<$ -JcC<$\c70~> -"!.FAg]"G\n0\1pfDbdQqS3'fgA_*SpVQscrNQKo5QCc2rilU?s8RLcrilTd;#gR>rilI@JcC<$ -JcC<$\c70~> -"!.EOOT,7\pEo5+!Sp!8rrM9Lrr3#lh1>TWs+13$s0DZT~> -"!.EOOT,7\pEo5+!Sp!8rrM9Lrr3#lh1>TWs+13$s0DZT~> -"!.EOOT,7\pEo5+!Sp!8rrM9Lrr3#lh1>TWs+13$s0DZT~> -"!.F:N;agu,jt!u!W;G<s+13$s+13Es*t~> -"!.F:N;agu,jt!u!W;G<s+13$s+13Es*t~> -"!.F:N;agu,jt!u!W;G<s+13$s+13Es*t~> -!$-XjJcC<$JcC?%J,~> -!$-XjJcC<$JcC?%J,~> -!$-XjJcC<$JcC?%J,~> -!$1k7!Tr3hs+13$s+130s*t~> -!$1k7!Tr3hs+13$s+130s*t~> -!$1k7!Tr3hs+13$s+130s*t~> -!$1k7!M=gls+13$s+130s*t~> -!$1k7!M=gls+13$s+130s*t~> -!$1k7!M=gls+13$s+130s*t~> -!$1k7!M=gls+13$s+130s*t~> -!$1k7!M=gls+13$s+130s*t~> -!$1k7!M=gls+13$s+130s*t~> -"!.FA\c-1DC]FF,IfB?Jon%bpJcC<$JcCo5J,~> -"!.FA\c-1DC]FF,IfB?Jon%bpJcC<$JcCo5J,~> -"!.FA\c-1DC]FF,IfB?Jon%bpJcC<$JcCo5J,~> -"!.EWHiCG[ZXj*.IfKF\:r@kaJcC<$JcCo5J,~> -"!.EWHiCG[ZXj*.IfKF\:r@kaJcC<$JcCo5J,~> -"!.EWHiCG[ZXj*.IfKF\:r@kaJcC<$JcCo5J,~> -"!.E>FoMCHpEop4IX`oYeq*jPs+13$s,m>3~> -"!.E>FoMCHpEop4IX`oYeq*jPs+13$s,m>3~> -"!.E>FoMCHpEop4IX`oYeq*jPs+13$s,m>3~> -"!.E>FoMCGpEop4%[H&MJcC<$JcC<$OT0h~> -"!.E>FoMCGpEop4%[H&MJcC<$JcC<$OT0h~> -"!.E>FoMCGpEop4%[H&MJcC<$JcC<$OT0h~> -"!.E>FoMCHpEop4IXWfXf7EsQs+13$s,m>3~> -"!.E>FoMCHpEop4IXWfXf7EsQs+13$s,m>3~> -"!.E>FoMCHpEop4IXWfXf7EsQs+13$s,m>3~> -"!.EWHiCJ\Zt'-.IfKF[:W.haJcC<$JcCo5J,~> -"!.EWHiCJ\Zt'-.IfKF[:W.haJcC<$JcCo5J,~> -"!.EWHiCJ\Zt'-.IfKF[:W.haJcC<$JcCo5J,~> -"!.FA\Gg"ACB+=QZ2Xb(omhVnJcC<$JcCo5J,~> -"!.FA\Gg"ACB+=QZ2Xb(omhVnJcC<$JcCo5J,~> -"!.FA\Gg"ACB+=QZ2Xb(omhVnJcC<$JcCo5J,~> -!$-XjJcC<$JcC?%J,~> -!$-XjJcC<$JcC?%J,~> -!$-XjJcC<$JcC?%J,~> -!$-XjJcC<$JcC?%J,~> -!$-XjJcC<$JcC?%J,~> -!$-XjJcC<$JcC?%J,~> -!$1>(!5\M"!/Q4+JcC<$JcDMFJ,~> -!$1>(!5\M"!/Q4+JcC<$JcDMFJ,~> -!$1>(!5\M"!/Q4+JcC<$JcDMFJ,~> -!$1>(!NTW+fE%aPJcC<$JcC<$U]5i~> -!$1>(!NTW+fE%aPJcC<$JcC<$U]5i~> -!$1>(!NTW+fE%aPJcC<$JcC<$U]5i~> -!$2";!UqE7rrS:OqY'piX)\(;!Phrks+13$s+13Fs*t~> -!$2";!UqE7rrS:OqY'piX)\(;!Phrks+13$s+13Fs*t~> -!$2";!UqE7rrS:OqY'piX)\(;!Phrks+13$s+13Fs*t~> -!$2%<!o?]]rVlqMU\FN_!NU5<rrKo?JcC<$JcC<$U]5i~> -!$2%<!o?]]rVlqMU\FN_!NU5<rrKo?JcC<$JcC<$U]5i~> -!$2%<!o?]]rVlqMU\FN_!NU5<rrKo?JcC<$JcC<$U]5i~> -!$2(="4M)A*W5p=I\Q`7rrK0?r;Qf5>_2p+s+13$s.fUE~> -!$2(="4M)A*W5p=I\Q`7rrK0?r;Qf5>_2p+s+13$s.fUE~> -!$2(="4M)A*W5p=I\Q`7rrK0?r;Qf5>_2p+s+13$s.fUE~> -#9EjEc[=+>*W,j;JsuK7!NU5<rrKo?JcC<$JcC<$U]5i~> -#9EjEc[=+>*W,j;JsuK7!NU5<rrKo?JcC<$JcC<$U]5i~> -#9EjEc[=+>*W,j;JsuK7!NU5<rrKo?JcC<$JcC<$U]5i~> -#9EilJ'@rm*W#d9*VfX8X)\(;!Phrks+13$s+13Fs*t~> -#9EilJ'@rm*W#d9*VfX8X)\(;!Phrks+13$s+13Fs*t~> -#9EilJ'@rm*W#d9*VfX8X)\(;!Phrks+13$s+13Fs*t~> -"!.E>.fNZR$Zu=H!Ht@8rrK0?r;Qf5>_2p+s+13$s.fUE~> -"!.E>.fNZR$Zu=H!Ht@8rrK0?r;Qf5>_2p+s+13$s.fUE~> -"!.E>.fNZR$Zu=H!Ht@8rrK0?r;Qf5>_2p+s+13$s.fUE~> -!Zh=*rP/FLao25@FK>?7rrK0?r;Qf5>_2p+s+13$s.fUE~> -!Zh=*rP/FLao25@FK>?7rrK0?r;Qf5>_2p+s+13$s.fUE~> -!Zh=*rP/FLao25@FK>?7rrK0?r;Qf5>_2p+s+13$s.fUE~> -!$1t:!$2%<!dUd@p\t8pEW#h;^g)HjJcC<$JcDMFJ,~> -!$1t:!$2%<!dUd@p\t8pEW#h;^g)HjJcC<$JcDMFJ,~> -!$1t:!$2%<!dUd@p\t8pEW#h;^g)HjJcC<$JcDMFJ,~> -!$1t:!&FQR!g'2VpAY/oEW#h;^g)HjJcC<$JcDMFJ,~> -!$1t:!&FQR!g'2VpAY/oEW#h;^g)HjJcC<$JcDMFJ,~> -!$1t:!&FQR!g'2VpAY/oEW#h;^g)HjJcC<$JcDMFJ,~> -!$1>(!NU5<rrKo?JcC<$JcC<$U]5i~> -!$1>(!NU5<rrKo?JcC<$JcC<$U]5i~> -!$1>(!NU5<rrKo?JcC<$JcC<$U]5i~> -!$1>(!NSrmXT=#YJcC<$JcC<$U]5i~> -!$1>(!NSrmXT=#YJcC<$JcC<$U]5i~> -!$1>(!NSrmXT=#YJcC<$JcC<$U]5i~> -!$1>(!71L[!2G,FJcC<$JcDMFJ,~> -!$1>(!71L[!2G,FJcC<$JcDMFJ,~> -!$1>(!71L[!2G,FJcC<$JcDMFJ,~> -!$-XjJcC<$JcC?%J,~> -!$-XjJcC<$JcC?%J,~> -!$-XjJcC<$JcC?%J,~> -!$-XjJcC<$JcC?%J,~> -!$-XjJcC<$JcC?%J,~> -!$-XjJcC<$JcC?%J,~> -!$-XjJcC<$JcC?%J,~> -!$-XjJcC<$JcC?%J,~> -!$-XjJcC<$JcC?%J,~> -!$-XjJcC<$JcC?%J,~> -!$-XjJcC<$JcC?%J,~> -!$-XjJcC<$JcC?%J,~> -%%EndData -showpage -%%Trailer -end -%%EOF diff --git a/lib/stdlib/doc/src/ushell2.gif b/lib/stdlib/doc/src/ushell2.gif Binary files differdeleted file mode 100644 index 273cf2078a..0000000000 --- a/lib/stdlib/doc/src/ushell2.gif +++ /dev/null diff --git a/lib/stdlib/doc/src/ushell2.ps b/lib/stdlib/doc/src/ushell2.ps deleted file mode 100644 index e6db3c2be2..0000000000 --- a/lib/stdlib/doc/src/ushell2.ps +++ /dev/null @@ -1,404 +0,0 @@ -%!PS-Adobe-3.0 -%%Creator: GIMP PostScript file plugin V 1.17 by Peter Kirchgessner -%%Title: ushell2.ps -%%CreationDate: Mon Mar 16 09:52:14 2009 -%%DocumentData: Clean7Bit -%%LanguageLevel: 2 -%%Pages: 1 -%%BoundingBox: 14 14 468 74 -%%EndComments -%%BeginProlog -% Use own dictionary to avoid conflicts -10 dict begin -%%EndProlog -%%Page: 1 1 -% Translate for offset -14.173228346456694 14.173228346456694 translate -% Translate to begin of first scanline -0 59.527559055118118 translate -453.54330708661422 -59.527559055118118 scale -% Image geometry -640 84 8 -% Transformation matrix -[ 640 0 0 84 0 0 ] -% Strings to hold RGB-samples per scanline -/rstr 640 string def -/gstr 640 string def -/bstr 640 string def -{currentfile /ASCII85Decode filter /RunLengthDecode filter rstr readstring pop} -{currentfile /ASCII85Decode filter /RunLengthDecode filter gstr readstring pop} -{currentfile /ASCII85Decode filter /RunLengthDecode filter bstr readstring pop} -true 3 -%%BeginData: 18704 ASCII Bytes -colorimage -J`MCCJ`MCCJ`M=~> -J`MCCJ`MCCJ`M=~> -J`MCCJ`MCCJ`M=~> -!)nG7JO+iQJO+lRJ,~> -!)nG7JO+iQJO+lRJ,~> -!)nG7JO+iQJO+lRJ,~> -!D=/Y=b0_,=b0_.=b$~> -!D=/Y=b0_,=b0_.=b$~> -!D=/Y=b0_,=b0_.=b$~> -!D>M*s+13$s+13&s*t~> -!D>M*s+13$s+13&s*t~> -!D>M*s+13$s+13&s*t~> -!D>M>rrKOJQ2^i3JcC<$JcFU,J,~> -!D>M>rrKOJQ2^i3JcC<$JcFU,J,~> -!D>M>rrKOJQ2^i3JcC<$JcFU,J,~> -"%t`UXo(ooJ,'$ES](+!!dROKr;Qh<:2'>#!I!f.rr]@2s).a0!581-!+,Ru!6ss3!kNp@p\t9X -H-uWnOA=s3rrK.0nG`Jtr;QbGr_<HIrVlo/@tFZ2s+14-s*t~> -"%t`UXo(ooJ,'$ES](+!!dROKr;Qh<:2'>#!I!f.rr]@2s).a0!581-!+,Ru!6ss3!kNp@p\t9X -H-uWnOA=s3rrK.0nG`Jtr;QbGr_<HIrVlo/@tFZ2s+14-s*t~> -"%t`UXo(ooJ,'$ES](+!!dROKr;Qh<:2'>#!I!f.rr]@2s).a0!581-!+,Ru!6ss3!kNp@p\t9X -H-uWnOA=s3rrK.0nG`Jtr;QbGr_<HIrVlo/@tFZ2s+14-s*t~> -"A:iVM0T!F!8RAL!HP42rrUp#*W5p=fZ5E4rrG1@rr3+VQ2`JQrQbK2q#:?CrQbLB\*j7sS:L>5 -!T-**rrJOjli.%FchRG<!HlihrrIe?nc&Yn]MJP,!J%!FcN0Vnrr3#u++aHCs+14-s*t~> -"A:iVM0T!F!8RAL!HP42rrUp#*W5p=fZ5E4rrG1@rr3+VQ2`JQrQbK2q#:?CrQbLB\*j7sS:L>5 -!T-**rrJOjli.%FchRG<!HlihrrIe?nc&Yn]MJP,!J%!FcN0Vnrr3#u++aHCs+14-s*t~> -"A:iVM0T!F!8RAL!HP42rrUp#*W5p=fZ5E4rrG1@rr3+VQ2`JQrQbK2q#:?CrQbLB\*j7sS:L>5 -!T-**rrJOjli.%FchRG<!HlihrrIe?nc&Yn]MJP,!J%!FcN0Vnrr3#u++aHCs+14-s*t~> -"A:iVM3Rl:!;QQc#kn;uEiSm+o_8@b!;HKb!;HEk!$;%;!V[r6rs;$Hs8VtORf>#.rq$3eoD\lN -QMU(M!l&CArVllrrU^6ks5-$;rq$<ks8W#qoFLs$rV6Ego^r._r:GB$o^qnX!W2]krs&2ss8)B[ -rVlg+qXdY#o_/=bKUDN+o_/%Z$2J#IrQSC<s'rY>rrMrfrVloH97d+os+14.s*t~> -"A:iVM3Rl:!;QQc#kn;uEiSm+o_8@b!;HKb!;HEk!$;%;!V[r6rs;$Hs8VtORf>#.rq$3eoD\lN -QMU(M!l&CArVllrrU^6ks5-$;rq$<ks8W#qoFLs$rV6Ego^r._r:GB$o^qnX!W2]krs&2ss8)B[ -rVlg+qXdY#o_/=bKUDN+o_/%Z$2J#IrQSC<s'rY>rrMrfrVloH97d+os+14.s*t~> -"A:iVM3Rl:!;QQc#kn;uEiSm+o_8@b!;HKb!;HEk!$;%;!V[r6rs;$Hs8VtORf>#.rq$3eoD\lN -QMU(M!l&CArVllrrU^6ks5-$;rq$<ks8W#qoFLs$rV6Ego^r._r:GB$o^qnX!W2]krs&2ss8)B[ -rVlg+qXdY#o_/=bKUDN+o_/%Z$2J#IrQSC<s'rY>rrMrfrVloH97d+os+14.s*t~> -#>7/YM1+"-g&D!PlY?DI$r#a^Efa$E-rf=.k?@j5!F+acrr=,;rrMj@o`#1<s8Sp?s8O^er[7f! -nFce_n:I\.rrK]@r;QcHr?qfQqqj5tr[7lYrV35f-kO2nd:1^L-n*1:a!_W=-n+farrK!?rr35q -.KBF2-jn]-rsk_N%3R&fs8Rb&=stdhj7rWTKtI?a"NGcfA%MI8!T6+ls+13$s5<p-~> -#>7/YM1+"-g&D!PlY?DI$r#a^Efa$E-rf=.k?@j5!F+acrr=,;rrMj@o`#1<s8Sp?s8O^er[7f! -nFce_n:I\.rrK]@r;QcHr?qfQqqj5tr[7lYrV35f-kO2nd:1^L-n*1:a!_W=-n+farrK!?rr35q -.KBF2-jn]-rsk_N%3R&fs8Rb&=stdhj7rWTKtI?a"NGcfA%MI8!T6+ls+13$s5<p-~> -#>7/YM1+"-g&D!PlY?DI$r#a^Efa$E-rf=.k?@j5!F+acrr=,;rrMj@o`#1<s8Sp?s8O^er[7f! -nFce_n:I\.rrK]@r;QcHr?qfQqqj5tr[7lYrV35f-kO2nd:1^L-n*1:a!_W=-n+farrK!?rr35q -.KBF2-jn]-rsk_N%3R&fs8Rb&=stdhj7rWTKtI?a"NGcfA%MI8!T6+ls+13$s5<p-~> -#>7/YM'th_:B(7oI@pN=%,$43E\H&Es1hq?B=@g>!OQP=rr=,;rrMj@o`#1<s8Sp?s8Qb.rr3"H -U%SEdTmZ8-!P2e8rrb=Bh[t\DrrXkBdn0N<#U90GWZSDDruV1>!J6g6rs#?Ds3C3-.K08I6JDA; -!EI5>rs"-+b5_Li9CVrcA%M^:\[g>jrrM%@JcC<$JcF[.J,~> -#>7/YM'th_:B(7oI@pN=%,$43E\H&Es1hq?B=@g>!OQP=rr=,;rrMj@o`#1<s8Sp?s8Qb.rr3"H -U%SEdTmZ8-!P2e8rrb=Bh[t\DrrXkBdn0N<#U90GWZSDDruV1>!J6g6rs#?Ds3C3-.K08I6JDA; -!EI5>rs"-+b5_Li9CVrcA%M^:\[g>jrrM%@JcC<$JcF[.J,~> -#>7/YM'th_:B(7oI@pN=%,$43E\H&Es1hq?B=@g>!OQP=rr=,;rrMj@o`#1<s8Sp?s8Qb.rr3"H -U%SEdTmZ8-!P2e8rrb=Bh[t\DrrXkBdn0N<#U90GWZSDDruV1>!J6g6rs#?Ds3C3-.K08I6JDA; -!EI5>rs"-+b5_Li9CVrcA%M^:\[g>jrrM%@JcC<$JcF[.J,~> -"A:iVM3Ro;!O.@YS-6(rs)P.=rrg,Cs(?9LS,mM@rVlj=qu6]k-M7<@Ff=JZXSVqtI%g96!p59A -mJd3uA,Q?,li!=U,37WFrVlsOnC'u-rraPCs/L,=rr@3@rrI_@p&>5nG5k:_oI9b=!CGN<rrH-@ -rVlnOR/[*ddS'*0#%P7Eo4(4*rVloV4b<Was+14.s*t~> -"A:iVM3Ro;!O.@YS-6(rs)P.=rrg,Cs(?9LS,mM@rVlj=qu6]k-M7<@Ff=JZXSVqtI%g96!p59A -mJd3uA,Q?,li!=U,37WFrVlsOnC'u-rraPCs/L,=rr@3@rrI_@p&>5nG5k:_oI9b=!CGN<rrH-@ -rVlnOR/[*ddS'*0#%P7Eo4(4*rVloV4b<Was+14.s*t~> -"A:iVM3Ro;!O.@YS-6(rs)P.=rrg,Cs(?9LS,mM@rVlj=qu6]k-M7<@Ff=JZXSVqtI%g96!p59A -mJd3uA,Q?,li!=U,37WFrVlsOnC'u-rraPCs/L,=rr@3@rrI_@p&>5nG5k:_oI9b=!CGN<rrH-@ -rVlnOR/[*ddS'*0#%P7Eo4(4*rVloV4b<Was+14.s*t~> -"A:iVM3Rl:!3lHR"G!EMEiT-="Me=CB3bCd!4;_)!$;%;!V[r5rr`6B_HQd8!IUU7rrKi?m/I*t -A,ZE.oRHgj"!$CZ55kK`/b%4\rVm!Equ<[:qYpSJS+ZdcVKVu6ch&[brrGI?r;QdtaSl,>KUDc= -!RaX1rrHW@rr3#'D#OA7i&uYkJcC<$i;\<~> -"A:iVM3Rl:!3lHR"G!EMEiT-="Me=CB3bCd!4;_)!$;%;!V[r5rr`6B_HQd8!IUU7rrKi?m/I*t -A,ZE.oRHgj"!$CZ55kK`/b%4\rVm!Equ<[:qYpSJS+ZdcVKVu6ch&[brrGI?r;QdtaSl,>KUDc= -!RaX1rrHW@rr3#'D#OA7i&uYkJcC<$i;\<~> -"A:iVM3Rl:!3lHR"G!EMEiT-="Me=CB3bCd!4;_)!$;%;!V[r5rr`6B_HQd8!IUU7rrKi?m/I*t -A,ZE.oRHgj"!$CZ55kK`/b%4\rVm!Equ<[:qYpSJS+ZdcVKVu6ch&[brrGI?r;QdtaSl,>KUDc= -!RaX1rrHW@rr3#'D#OA7i&uYkJcC<$i;\<~> -"A:iVM3Ro;!O'H@rrgMCs)P.=rrg,Cs(Ae8rr=,;rrMj@o`#$m+2V4\rrITArr3"HU&P&nhI3>\ -rrVG(_X.:(jbC#[email protected]($+6gRf<<h/b%4\rVm!Equ<[:qYp\MS,`M0q#:PqG,("4kUHK1 -!CGN<rs)QFs6>F?KUDc=!RaX1rrHW@rr3#'D#XG9rb/]js+13$s5<p-~> -"A:iVM3Ro;!O'H@rrgMCs)P.=rrg,Cs(Ae8rr=,;rrMj@o`#$m+2V4\rrITArr3"HU&P&nhI3>\ -rrVG(_X.:(jbC#[email protected]($+6gRf<<h/b%4\rVm!Equ<[:qYp\MS,`M0q#:PqG,("4kUHK1 -!CGN<rs)QFs6>F?KUDc=!RaX1rrHW@rr3#'D#XG9rb/]js+13$s5<p-~> -"A:iVM3Ro;!O'H@rrgMCs)P.=rrg,Cs(Ae8rr=,;rrMj@o`#$m+2V4\rrITArr3"HU&P&nhI3>\ -rrVG(_X.:(jbC#[email protected]($+6gRf<<h/b%4\rVm!Equ<[:qYp\MS,`M0q#:PqG,("4kUHK1 -!CGN<rs)QFs6>F?KUDc=!RaX1rrHW@rr3#'D#XG9rb/]js+13$s5<p-~> -"A:iVM'r6h"*ae$E;i3->g`RlWrE#$^g6ulB)Y-tHN*pIEC1"!K)YcQBL)tmMtR)OX^KZXrrL26 -r^-^;k5PA^aT->ZrrLn@lMgmTJH#QLq5"!W"d:Fsh_(&j6Na7`oU5YF"K_V&WcIt9"Q"/d6bE-n% -HrJ%lcE#Es8S]Y&NNhhrs-tX6W;80KUDc=!RaX1rrL\]r^-^"f`(mOrZ?^kJcC<$huA3~> -"A:iVM'r6h"*ae$E;i3->g`RlWrE#$^g6ulB)Y-tHN*pIEC1"!K)YcQBL)tmMtR)OX^KZXrrL26 -r^-^;k5PA^aT->ZrrLn@lMgmTJH#QLq5"!W"d:Fsh_(&j6Na7`oU5YF"K_V&WcIt9"Q"/d6bE-n% -HrJ%lcE#Es8S]Y&NNhhrs-tX6W;80KUDc=!RaX1rrL\]r^-^"f`(mOrZ?^kJcC<$huA3~> -"A:iVM'r6h"*ae$E;i3->g`RlWrE#$^g6ulB)Y-tHN*pIEC1"!K)YcQBL)tmMtR)OX^KZXrrL26 -r^-^;k5PA^aT->ZrrLn@lMgmTJH#QLq5"!W"d:Fsh_(&j6Na7`oU5YF"K_V&WcIt9"Q"/d6bE-n% -HrJ%lcE#Es8S]Y&NNhhrs-tX6W;80KUDc=!RaX1rrL\]r^-^"f`(mOrZ?^kJcC<$huA3~> -"%t`Umek`?jT#8Drn.G5s8VBUrr3,pkPtSCrn.;4rr2uYrn.;6rr2uWrn.;8p&>'hir&fVh#>t, -mJ[%dpY"j1rrMuVlMgqTJ#rYL!;QQH"nL[MqW%/Gf`V$Ls7H9C"T&/uoBQ/O"6eFkrV-<pmdL2U -hZ!NTm/GZ<h>I9Win<2gs6]=TrrMoUn,ECKrn.;5rVlm_hLY]Xs+14-s*t~> -"%t`Umek`?jT#8Drn.G5s8VBUrr3,pkPtSCrn.;4rr2uYrn.;6rr2uWrn.;8p&>'hir&fVh#>t, -mJ[%dpY"j1rrMuVlMgqTJ#rYL!;QQH"nL[MqW%/Gf`V$Ls7H9C"T&/uoBQ/O"6eFkrV-<pmdL2U -hZ!NTm/GZ<h>I9Win<2gs6]=TrrMoUn,ECKrn.;5rVlm_hLY]Xs+14-s*t~> -"%t`Umek`?jT#8Drn.G5s8VBUrr3,pkPtSCrn.;4rr2uYrn.;6rr2uWrn.;8p&>'hir&fVh#>t, -mJ[%dpY"j1rrMuVlMgqTJ#rYL!;QQH"nL[MqW%/Gf`V$Ls7H9C"T&/uoBQ/O"6eFkrV-<pmdL2U -hZ!NTm/GZ<h>I9Win<2gs6]=TrrMoUn,ECKrn.;5rVlm_hLY]Xs+14-s*t~> -!D>M>rrN#pQ2^jZJcC<$JcFU,J,~> -!D>M>rrN#pQ2^jZJcC<$JcFU,J,~> -!D>M>rrN#pQ2^jZJcC<$JcFU,J,~> -!D>M*s+13$s+13&s*t~> -!D>M*s+13$s+13&s*t~> -!D>M*s+13$s+13&s*t~> -!D>M*s+13$s+13&s*t~> -!D>M*s+13$s+13&s*t~> -!D>M*s+13$s+13&s*t~> -!D>NQrrK"krVlo0ipm$KT)JZh\GVHh_h%i=s+13$s/Z0M~> -!D>NQrrK"krVlo0ipm$KT)JZh\GVHh_h%i=s+13$s/Z0M~> -!D>NQrrK"krVlo0ipm$KT)JZh\GVHh_h%i=s+13$s/Z0M~> -!D>NRrrSc-mJ[%d[sIB4rsbK>%<J'>s0[duMN!M0JcC<$JcC<$X8d\~> -!D>NRrrSc-mJ[%d[sIB4rsbK>%<J'>s0[duMN!M0JcC<$JcC<$X8d\~> -!D>NRrrSc-mJ[%d[sIB4rsbK>%<J'>s0[duMN!M0JcC<$JcC<$X8d\~> -#>7/Ys.,5omJQtc^LI'5rsP9XW>)=oS,`M<TDjEAJcC<$JcD\KJ,~> -#>7/Ys.,5omJQtc^LI'5rsP9XW>)=oS,`M<TDjEAJcC<$JcD\KJ,~> -#>7/Ys.,5omJQtc^LI'5rsP9XW>)=oS,`M<TDjEAJcC<$JcD\KJ,~> -#>7/YUPnOomJHnacVF36$>6gI*rkZbs8P:?JcC<$JcC<$VuM8~> -#>7/YUPnOomJHnacVF36$>6gI*rkZbs8P:?JcC<$JcC<$VuM8~> -#>7/YUPnOomJHnacVF36$>6gI*rkZbs8P:?JcC<$JcC<$VuM8~> -#>7/YoCW&:mJ?h`dn0<6"RfFC*W>s:"6h]b:P&Oss+13$s/H$K~> -#>7/YoCW&:mJ?h`dn0<6"RfFC*W>s:"6h]b:P&Oss+13$s/H$K~> -#>7/YoCW&:mJ?h`dn0<6"RfFC*W>s:"6h]b:P&Oss+13$s/H$K~> -!D>NQrrFh@qYpTB:A+Vm7Kc'As8V@[j+75]s+13$s/5mI~> -!D>NQrrFh@qYpTB:A+Vm7Kc'As8V@[j+75]s+13$s/5mI~> -!D>NQrrFh@qYpTB:A+Vm7Kc'As8V@[j+75]s+13$s/5mI~> -!D>NQrrFh@qu6]>;Y0nm*o1A]s'3Bks+13$s+13Is*t~> -!D>NQrrFh@qu6]>;Y0nm*o1A]s'3Bks+13$s+13Is*t~> -!D>NQrrFh@qu6]>;Y0nm*o1A]s'3Bks+13$s+13Is*t~> -!D>NQrrFh@r;Qi1B);6$$-D</*riT\s'3D9rrW+o]Rg*6s+13$s0;TS~> -!D>NQrrFh@r;Qi1B);6$$-D</*riT\s'3D9rrW+o]Rg*6s+13$s0;TS~> -!D>NQrrFh@r;Qi1B);6$$-D</*riT\s'3D9rrW+o]Rg*6s+13$s0;TS~> -%SJn`a+f<dI>4^#s0R1?p&>?%JqahkK!>9SKDpT*YPnJ&pAdU4s+13$s+13Ts*t~> -%SJn`a+f<dI>4^#s0R1?p&>?%JqahkK!>9SKDpT*YPnJ&pAdU4s+13$s+13Ts*t~> -%SJn`a+f<dI>4^#s0R1?p&>?%JqahkK!>9SKDpT*YPnJ&pAdU4s+13$s+13Ts*t~> -"%t`UcMWt2ZiC'>jR`BS[$D;i[Jp1+Yl9phLTURU!rS@iJcC<$JcC<$Z2]=~> -"%t`UcMWt2ZiC'>jR`BS[$D;i[Jp1+Yl9phLTURU!rS@iJcC<$JcC<$Z2]=~> -"%t`UcMWt2ZiC'>jR`BS[$D;i[Jp1+Yl9phLTURU!rS@iJcC<$JcC<$Z2]=~> -!D>N9rrBb-rrK*@JcC<$JcC<$X8d\~> -!D>N9rrBb-rrK*@JcC<$JcC<$X8d\~> -!D>N9rrBb-rrK*@JcC<$JcC<$X8d\~> -!D>N1rr_L<A`eRDJcC<$JcDeNJ,~> -!D>N1rr_L<A`eRDJcC<$JcDeNJ,~> -!D>N1rr_L<A`eRDJcC<$JcDeNJ,~> -!D>N1rrW/foR[$ns+13$s/Q*L~> -!D>N1rrW/foR[$ns+13$s/Q*L~> -!D>N1rrW/foR[$ns+13$s/Q*L~> -"A:iVs5s=1"5EkUl29$2ir8rZh:1/0s+13$s+13<s*t~> -"A:iVs5s=1"5EkUl29$2ir8rZh:1/0s+13$s+13<s*t~> -"A:iVs5s=1"5EkUl29$2ir8rZh:1/0s+13$s+13<s*t~> -"A:iVh08ih"Bs"KI4bCg#D<'VcX9:FaasJCs+13$s-it<~> -"A:iVh08ih"Bs"KI4bCg#D<'VcX9:FaasJCs+13$s-it<~> -"A:iVh08ih"Bs"KI4bCg#D<'VcX9:FaasJCs+13$s-it<~> -"A:iVM3S#>"P-'BI@pE:#01rEs6Fa?JcC<$JcC<$S,\!~> -"A:iVM3S#>"P-'BI@pE:#01rEs6Fa?JcC<$JcC<$S,\!~> -"A:iVM3S#>"P-'BI@pE:#01rEs6Fa?JcC<$JcC<$S,\!~> -"A:iVM3S#>&_9GOI@m*XH^=^3]7/f?mX;?ks+13$s+13>s*t~> -"A:iVM3S#>&_9GOI@m*XH^=^3]7/f?mX;?ks+13$s+13>s*t~> -"A:iVM3S#>&_9GOI@m*XH^=^3]7/f?mX;?ks+13$s+13>s*t~> -(eZsjM3S&?peOBZI63OaTp(Z>s1iGWB=NNmJcC<$JcD2=J,~> -(eZsjM3S&?peOBZI63OaTp(Z>s1iGWB=NNmJcC<$JcD2=J,~> -(eZsjM3S&?peOBZI63OaTp(Z>s1iGWB=NNmJcC<$JcD2=J,~> -(eZsjiH7CN7\E6NhS8LYs3"=?s+$lZY&JslJcC<$JcD2=J,~> -(eZsjiH7CN7\E6NhS8LYs3"=?s+$lZY&JslJcC<$JcD2=J,~> -(eZsjiH7CN7\E6NhS8LYs3"=?s+$lZY&JslJcC<$JcD2=J,~> -#tmA[s6JIqhUY0nrs?kHs+R*>s8&m>JcC<$JcC<$S,\!~> -#tmA[s6JIqhUY0nrs?kHs+R*>s8&m>JcC<$JcC<$S,\!~> -#tmA[s6JIqhUY0nrs?kHs+R*>s8&m>JcC<$JcC<$S,\!~> -!D>NPrrrDA=TA!drr3;J;ZD6@qZ$K"Ck;V;s+13$s-s%=~> -!D>NPrrrDA=TA!drr3;J;ZD6@qZ$K"Ck;V;s+13$s-s%=~> -!D>NPrrrDA=TA!drr3;J;ZD6@qZ$K"Ck;V;s+13$s-s%=~> -$qi\^s5;#@_cHg;R?IQk#ci.Ks*1Qc[V16lJcC<$JcD2=J,~> -$qi\^s5;#@_cHg;R?IQk#ci.Ks*1Qc[V16lJcC<$JcD2=J,~> -$qi\^s5;#@_cHg;R?IQk#ci.Ks*1Qc[V16lJcC<$JcD2=J,~> -#>7/Ys-lqc>Q+R%Pl=S]G5_FBGZ/@.s+13$s+13<s*t~> -#>7/Ys-lqc>Q+R%Pl=S]G5_FBGZ/@.s+13$s+13<s*t~> -#>7/Ys-lqc>Q+R%Pl=S]G5_FBGZ/@.s+13$s+13<s*t~> -!D>M*s+13$s+13&s*t~> -!D>M*s+13$s+13&s*t~> -!D>M*s+13$s+13&s*t~> -!D>M*s+13$s+13&s*t~> -!D>M*s+13$s+13&s*t~> -!D>M*s+13$s+13&s*t~> -!D>M*s+13$s+13&s*t~> -!D>M*s+13$s+13&s*t~> -!D>M*s+13$s+13&s*t~> -!D>M*s+13$s+13&s*t~> -!D>M*s+13$s+13&s*t~> -!D>M*s+13$s+13&s*t~> -"A:iVrHeGa#!r.ds)PpSp&>3n>ok%TqqqDSq^h^f!K[21s+13$s+13ns*t~> -"A:iVrHeGa#!r.ds)PpSp&>3n>ok%TqqqDSq^h^f!K[21s+13$s+13ns*t~> -"A:iVrHeGa#!r.ds)PpSp&>3n>ok%TqqqDSq^h^f!K[21s+13$s+13ns*t~> -"A:iVM3S#>!S0a>rrQHBrq??qB=@j?dS&Kt!DCl?rrJ._JcC<$JcC<$bQ!(~> -"A:iVM3S#>!S0a>rrQHBrq??qB=@j?dS&Kt!DCl?rrJ._JcC<$JcC<$bQ!(~> -"A:iVM3S#>!S0a>rrQHBrq??qB=@j?dS&Kt!DCl?rrJ._JcC<$JcC<$bQ!(~> -#tmA[Vm$.#r2Oo<rrHB@pAYG3[f?BU9E5%Cn,E=fl.l:<!64s:$./AAbQ$Y^s8U+<^B!*hrr<u: -^B!3krrBk6^B!;Fs+13$s+13us*t~> -#tmA[Vm$.#r2Oo<rrHB@pAYG3[f?BU9E5%Cn,E=fl.l:<!64s:$./AAbQ$Y^s8U+<^B!*hrr<u: -^B!3krrBk6^B!;Fs+13$s+13us*t~> -#tmA[Vm$.#r2Oo<rrHB@pAYG3[f?BU9E5%Cn,E=fl.l:<!64s:$./AAbQ$Y^s8U+<^B!*hrr<u: -^B!3krrBk6^B!;Fs+13$s+13us*t~> -!D>NPrrJs@qu6\1[.jS,B60c?KhMIG>f$F>!NC2?rr=,<rs;-HlS8F'MZ:+rra#_Uf`/6dra#_X -h#FNara#VXi.:oZs+13$s3q!u~> -!D>NPrrJs@qu6\1[.jS,B60c?KhMIG>f$F>!NC2?rr=,<rs;-HlS8F'MZ:+rra#_Uf`/6dra#_X -h#FNara#VXi.:oZs+13$s3q!u~> -!D>NPrrJs@qu6\1[.jS,B60c?KhMIG>f$F>!NC2?rr=,<rs;-HlS8F'MZ:+rra#_Uf`/6dra#_X -h#FNara#VXi.:oZs+13$s3q!u~> -!D>NQrrJU@qYpT[2#%"[B39M-AkW1(X+Kg?jFOf>rr=,<rs;-HlS8F'MZ5`trr3+VQ2`K)rr3+K -U&QA5rr3#-e:IXNs+13$s3q!u~> -!D>NQrrJU@qYpT[2#%"[B39M-AkW1(X+Kg?jFOf>rr=,<rs;-HlS8F'MZ5`trr3+VQ2`K)rr3+K -U&QA5rr3#-e:IXNs+13$s3q!u~> -!D>NQrrJU@qYpT[2#%"[B39M-AkW1(X+Kg?jFOf>rr=,<rs;-HlS8F'MZ5`trr3+VQ2`K)rr3+K -U&QA5rr3#-e:IXNs+13$s3q!u~> -!D>NRrrSFGq"k!kDm&j7"_Y:Ds3aR>rrbOCs2S1=rr=,<rs;-HlS8F'MZ5`trr3+VQ2`K)rr34N -U&V$'AnJ&os+13$s+13ts*t~> -!D>NRrrSFGq"k!kDm&j7"_Y:Ds3aR>rrbOCs2S1=rr=,<rs;-HlS8F'MZ5`trr3+VQ2`K)rr34N -U&V$'AnJ&os+13$s+13ts*t~> -!D>NRrrSFGq"k!kDm&j7"_Y:Ds3aR>rrbOCs2S1=rr=,<rs;-HlS8F'MZ5`trr3+VQ2`K)rr34N -U&V$'AnJ&os+13$s+13ts*t~> -#"q&Xs-8l>qYpS+](Z.-B=@j?dS'T>"NXX/@F+oP!$;(<#t<M41&mGps#T3>rre+Bs"<a=rs*qF -s'i@E\(?32JcC<$JcF-tJ,~> -#"q&Xs-8l>qYpS+](Z.-B=@j?dS'T>"NXX/@F+oP!$;(<#t<M41&mGps#T3>rre+Bs"<a=rs*qF -s'i@E\(?32JcC<$JcF-tJ,~> -#"q&Xs-8l>qYpS+](Z.-B=@j?dS'T>"NXX/@F+oP!$;(<#t<M41&mGps#T3>rre+Bs"<a=rs*qF -s'i@E\(?32JcC<$JcF-tJ,~> -"\UrWSW34:rrQZBrq??qB=@j?dS'Q=![-G]r;R9Ks2TNes$bT+1&mGps#T3>rre+Bs"<a=rrd_C -s!.@=rrItfrr3&A/!>J`JcC<$JcFF'J,~> -"\UrWSW34:rrQZBrq??qB=@j?dS'Q=![-G]r;R9Ks2TNes$bT+1&mGps#T3>rre+Bs"<a=rrd_C -s!.@=rrItfrr3&A/!>J`JcC<$JcFF'J,~> -"\UrWSW34:rrQZBrq??qB=@j?dS'Q=![-G]r;R9Ks2TNes$bT+1&mGps#T3>rre+Bs"<a=rrd_C -s!.@=rrItfrr3&A/!>J`JcC<$JcFF'J,~> -"A:iVO;Rp;"^M*-EkD;Crrr.#*Zd]BrVlrj2YI"J&*t?<p](9d4rX\IGkqC42?"X"Jbf?</H-[n -MYdAE,5rVaPl(I[!l>M%JcC<$JcC<$g&HR~> -"A:iVO;Rp;"^M*-EkD;Crrr.#*Zd]BrVlrj2YI"J&*t?<p](9d4rX\IGkqC42?"X"Jbf?</H-[n -MYdAE,5rVaPl(I[!l>M%JcC<$JcC<$g&HR~> -"A:iVO;Rp;"^M*-EkD;Crrr.#*Zd]BrVlrj2YI"J&*t?<p](9d4rX\IGkqC42?"X"Jbf?</H-[n -MYdAE,5rVaPl(I[!l>M%JcC<$JcC<$g&HR~> -!D>N"rrMF?JcC<$JcC<$\Gq'~> -!D>N"rrMF?JcC<$JcC<$\Gq'~> -!D>N"rrMF?JcC<$JcC<$\Gq'~> -!D>N"rrMF?JcC<$JcC<$\Gq'~> -!D>N"rrMF?JcC<$JcC<$\Gq'~> -!D>N"rrMF?JcC<$JcC<$\Gq'~> -!D>N"rrM_GJcC<$JcC<$\Gq'~> -!D>N"rrM_GJcC<$JcC<$\Gq'~> -!D>N"rrM_GJcC<$JcC<$\Gq'~> -!D>M*s+13$s+13&s*t~> -!D>M*s+13$s+13&s*t~> -!D>M*s+13$s+13&s*t~> -!D>NQrrIH;oD\f.li.!"FSu.?F^'4jrrR%AK'!"7oNPOlmf*=XI"D<Ks+13$s7QDB~> -!D>NQrrIH;oD\f.li.!"FSu.?F^'4jrrR%AK'!"7oNPOlmf*=XI"D<Ks+13$s7QDB~> -!D>NQrrIH;oD\f.li.!"FSu.?F^'4jrrR%AK'!"7oNPOlmf*=XI"D<Ks+13$s7QDB~> -#tmA[V69hPmGN'ZrrHa>li."*b5D8@bH`l#rrU9A4lu\Gqn\)'mf*=Q1YMd/Qb7nLJcC<$TDsE~> -#tmA[V69hPmGN'ZrrHa>li."*b5D8@bH`l#rrU9A4lu\Gqn\)'mf*=Q1YMd/Qb7nLJcC<$TDsE~> -#tmA[V69hPmGN'ZrrHa>li."*b5D8@bH`l#rrU9A4lu\Gqn\)'mf*=Q1YMd/Qb7nLJcC<$TDsE~> -#tmA[n8Punj$2c%rrQrD)WUhulS8;:!T6-$rrKH@mf*=Q1YMd/G,BijJcC<$TDsE~> -#tmA[n8Punj$2c%rrQrD)WUhulS8;:!T6-$rrKH@mf*=Q1YMd/G,BijJcC<$TDsE~> -#tmA[n8Punj$2c%rrQrD)WUhulS8;:!T6-$rrKH@mf*=Q1YMd/G,BijJcC<$TDsE~> -#YR8Zs6a"o,j+k$"+miX*r>m;&MNhW!_OI]qu6]_1&LtOi'%&=!U2-6*XdYos6(XJ*d)V6s8)[8 -!L<EYrrKH@pAY0m`Vgh\YP[T:VZ%V=\GZBJr>l?_rRb34I3'HPs8TJ)*WsmkpmV.&$f1io*Y&i) -*rl96M#JG%-KtK-K&fcM0C",gJcC<$ZN#F~> -#YR8Zs6a"o,j+k$"+miX*r>m;&MNhW!_OI]qu6]_1&LtOi'%&=!U2-6*XdYos6(XJ*d)V6s8)[8 -!L<EYrrKH@pAY0m`Vgh\YP[T:VZ%V=\GZBJr>l?_rRb34I3'HPs8TJ)*WsmkpmV.&$f1io*Y&i) -*rl96M#JG%-KtK-K&fcM0C",gJcC<$ZN#F~> -#YR8Zs6a"o,j+k$"+miX*r>m;&MNhW!_OI]qu6]_1&LtOi'%&=!U2-6*XdYos6(XJ*d)V6s8)[8 -!L<EYrrKH@pAY0m`Vgh\YP[T:VZ%V=\GZBJr>l?_rRb34I3'HPs8TJ)*WsmkpmV.&$f1io*Y&i) -*rl96M#JG%-KtK-K&fcM0C",gJcC<$ZN#F~> -#tmA[j`JBojA4A"rr=,:rrGJko`"sd-N!fDlS8;:!T6->rrFb?rr37OU&QA5s8S+>qYpS-\,QC. -ZYK46!VIi=rrd2tl3hh8rrG+?qu6sSnGfp5T`>$9qtpBq*rio]rVljnrr3"BVZ$MqVKVt=$1o\H -s*Z-8s6):js+13$s0DZT~> -#tmA[j`JBojA4A"rr=,:rrGJko`"sd-N!fDlS8;:!T6->rrFb?rr37OU&QA5s8S+>qYpS-\,QC. -ZYK46!VIi=rrd2tl3hh8rrG+?qu6sSnGfp5T`>$9qtpBq*rio]rVljnrr3"BVZ$MqVKVt=$1o\H -s*Z-8s6):js+13$s0DZT~> -#tmA[j`JBojA4A"rr=,:rrGJko`"sd-N!fDlS8;:!T6->rrFb?rr37OU&QA5s8S+>qYpS-\,QC. -ZYK46!VIi=rrd2tl3hh8rrG+?qu6sSnGfp5T`>$9qtpBq*rio]rVljnrr3"BVZ$MqVKVt=$1o\H -s*Z-8s6):js+13$s0DZT~> -#tmA[Z+0ffmGruorr=,+rrMj@qu6]_1&LtOi'%&=!A9uCW=)Uts!.@>s,N->s8Dnq!CO?qrrKH@ -pAY0d-i<oEl8/D="$#BAlM[[b*q93<BE%o5D#F=nrM]l7s1Mh9rrI;?rVlnq:B%4!U+--BI@pQ> -n9BNaJcC<$ZN#F~> -#tmA[Z+0ffmGruorr=,+rrMj@qu6]_1&LtOi'%&=!A9uCW=)Uts!.@>s,N->s8Dnq!CO?qrrKH@ -pAY0d-i<oEl8/D="$#BAlM[[b*q93<BE%o5D#F=nrM]l7s1Mh9rrI;?rVlnq:B%4!U+--BI@pQ> -n9BNaJcC<$ZN#F~> -#tmA[Z+0ffmGruorr=,+rrMj@qu6]_1&LtOi'%&=!A9uCW=)Uts!.@>s,N->s8Dnq!CO?qrrKH@ -pAY0d-i<oEl8/D="$#BAlM[[b*q93<BE%o5D#F=nrM]l7s1Mh9rrI;?rVlnq:B%4!U+--BI@pQ> -n9BNaJcC<$ZN#F~> -!D>NQrrJ/QoD\e3li."[-N!fDlS8;:!T6->rrF`grGr=hs+cMkN.JhFs-EV)!A:k\rrKH@pAY0d --i<oEl8/D="$#B2LAc/r'_).2BDhc1k(N\S!tkRH@/9g'G,G6<!N%dREs.C#s*^Mjs+13$s/uBP~> -!D>NQrrJ/QoD\e3li."[-N!fDlS8;:!T6->rrF`grGr=hs+cMkN.JhFs-EV)!A:k\rrKH@pAY0d --i<oEl8/D="$#B2LAc/r'_).2BDhc1k(N\S!tkRH@/9g'G,G6<!N%dREs.C#s*^Mjs+13$s/uBP~> -!D>NQrrJ/QoD\e3li."[-N!fDlS8;:!T6->rrF`grGr=hs+cMkN.JhFs-EV)!A:k\rrKH@pAY0d --i<oEl8/D="$#B2LAc/r'_).2BDhc1k(N\S!tkRH@/9g'G,G6<!N%dREs.C#s*^Mjs+13$s/uBP~> -!D>NArr=,:rrI;!o`"sd-N!fDlS8;:!T6->rrFb?qu6eNOHG[Brr2s>rr3&1/@YWY!OHP7rrMd? -rVltal0:)'rrY7Ah`h&>"dftms0cS<rrL2@rr3+[&c]OPrVlkIrr34HVZ6[Gs/'u9rrIP?JcC<$ -JcDnQJ,~> -!D>NArr=,:rrI;!o`"sd-N!fDlS8;:!T6->rrFb?qu6eNOHG[Brr2s>rr3&1/@YWY!OHP7rrMd? -rVltal0:)'rrY7Ah`h&>"dftms0cS<rrL2@rr3+[&c]OPrVlkIrr34HVZ6[Gs/'u9rrIP?JcC<$ -JcDnQJ,~> -!D>NArr=,:rrI;!o`"sd-N!fDlS8;:!T6->rrFb?qu6eNOHG[Brr2s>rr3&1/@YWY!OHP7rrMd? -rVltal0:)'rrY7Ah`h&>"dftms0cS<rrL2@rr3+[&c]OPrVlkIrr34HVZ6[Gs/'u9rrIP?JcC<$ -JcDnQJ,~> -!D>NDrs#)m;$g)sOT#1[%P@AR"ER<H;3V"Z"Dg[A;4IRb#\6F=;54*j]iY21"KMM%\l8T*"1%t, -[/No/IRUaGo2.Lk;8N&-!W8Vh;$3-Ul8/D="$#B7R/TqdP99;o[V,O<!Uc$J;$<@(s6>O@;#mj" -rs.%n;,Ok'j+I>.!*K7#!I^Sks+13$s/uBP~> -!D>NDrs#)m;$g)sOT#1[%P@AR"ER<H;3V"Z"Dg[A;4IRb#\6F=;54*j]iY21"KMM%\l8T*"1%t, -[/No/IRUaGo2.Lk;8N&-!W8Vh;$3-Ul8/D="$#B7R/TqdP99;o[V,O<!Uc$J;$<@(s6>O@;#mj" -rs.%n;,Ok'j+I>.!*K7#!I^Sks+13$s/uBP~> -!D>NDrs#)m;$g)sOT#1[%P@AR"ER<H;3V"Z"Dg[A;4IRb#\6F=;54*j]iY21"KMM%\l8T*"1%t, -[/No/IRUaGo2.Lk;8N&-!W8Vh;$3-Ul8/D="$#B7R/TqdP99;o[V,O<!Uc$J;$<@(s6>O@;#mj" -rs.%n;,Ok'j+I>.!*K7#!I^Sks+13$s/uBP~> -!D>NDrrD*WbQ-Q!rrLHrp&>$CrlbB"rr2uJrlbB%rr2uHrlbB'rr2uErlbN-s8Pm:rlbIg^&S*2 -bQR%cnc/OcbQ.&)rrDflbQID8qpt`G"5!DLoDZr;nC@I:nDX9E!:Kj1"SL4Cs6T^.!<)lr!oD/F -rr2u]rlbAfrr3#\m",1fs+13Qs*t~> -!D>NDrrD*WbQ-Q!rrLHrp&>$CrlbB"rr2uJrlbB%rr2uHrlbB'rr2uErlbN-s8Pm:rlbIg^&S*2 -bQR%cnc/OcbQ.&)rrDflbQID8qpt`G"5!DLoDZr;nC@I:nDX9E!:Kj1"SL4Cs6T^.!<)lr!oD/F -rr2u]rlbAfrr3#\m",1fs+13Qs*t~> -!D>NDrrD*WbQ-Q!rrLHrp&>$CrlbB"rr2uJrlbB%rr2uHrlbB'rr2uErlbN-s8Pm:rlbIg^&S*2 -bQR%cnc/OcbQ.&)rrDflbQID8qpt`G"5!DLoDZr;nC@I:nDX9E!:Kj1"SL4Cs6T^.!<)lr!oD/F -rr2u]rlbAfrr3#\m",1fs+13Qs*t~> -!D>MarrFV?rq?G6YCZ_)s+13$s3q!u~> -!D>MarrFV?rq?G6YCZ_)s+13$s3q!u~> -!D>MarrFV?rq?G6YCZ_)s+13$s3q!u~> -!D>M`rr=PJ-30Zhs+13$s+13us*t~> -!D>M`rr=PJ-30Zhs+13$s+13us*t~> -!D>M`rr=PJ-30Zhs+13$s+13us*t~> -!D>M*s+13$s+13&s*t~> -!D>M*s+13$s+13&s*t~> -!D>M*s+13$s+13&s*t~> -"A:iVs2Y,1!3Z>$!Q4&Ns+13$s+132s*t~> -"A:iVs2Y,1!3Z>$!Q4&Ns+13$s+132s*t~> -"A:iVs2Y,1!3Z>$!Q4&Ns+13$s+132s*t~> -"A:iV`..c8#(NHKs0$t?JcC<$JcC<$OT0h~> -"A:iV`..c8#(NHKs0$t?JcC<$JcC<$OT0h~> -"A:iV`..c8#(NHKs0$t?JcC<$JcC<$OT0h~> -"A:iVM3S#>!S0a>rrTTDqgncus+13$s,m>3~> -"A:iVM3S#>!S0a>rrTTDqgncus+13$s,m>3~> -"A:iVM3S#>!S0a>rrTTDqgncus+13$s,m>3~> -#tmA[hr=\9pK5Z<rrL#?JcC<$JcC<$OoKq~> -#tmA[hr=\9pK5Z<rrL#?JcC<$JcC<$OoKq~> -#tmA[hr=\9pK5Z<rrL#?JcC<$JcC<$OoKq~> -!D>NPrrG@?qu6]<<.Y(#s+13$s-!D4~> -!D>NPrrG@?qu6]<<.Y(#s+13$s-!D4~> -!D>NPrrG@?qu6]<<.Y(#s+13$s-!D4~> -!D>NRrrVaZjneuXg-^GkJcC<$JcCo5J,~> -!D>NRrrVaZjneuXg-^GkJcC<$JcCo5J,~> -!D>NRrrVaZjneuXg-^GkJcC<$JcCo5J,~> -#"q&Xs5RM>qYpTI7tL\ks+13$s,m>3~> -#"q&Xs5RM>qYpTI7tL\ks+13$s,m>3~> -#"q&Xs5RM>qYpTI7tL\ks+13$s,m>3~> -"\UrWgN^j:rrU/EqLSZts+13$s,m>3~> -"\UrWgN^j:rrU/EqLSZts+13$s,m>3~> -"\UrWgN^j:rrU/EqLSZts+13$s,m>3~> -"A:iVM,sS>"gS+-]QWRks+13$s+133s*t~> -"A:iVM,sS>"gS+-]QWRks+13$s+133s*t~> -"A:iVM,sS>"gS+-]QWRks+13$s+133s*t~> -"%t`UaS^ktWrN+,i.:oZs+13$s,[21~> -"%t`UaS^ktWrN+,i.:oZs+13$s,[21~> -"%t`UaS^ktWrN+,i.:oZs+13$s,[21~> -!D>M*s+13$s+13&s*t~> -!D>M*s+13$s+13&s*t~> -!D>M*s+13$s+13&s*t~> -!D>M*s+13$s+13&s*t~> -!D>M*s+13$s+13&s*t~> -!D>M*s+13$s+13&s*t~> -!D>M*s+13$s+13&s*t~> -!D>M*s+13$s+13&s*t~> -!D>M*s+13$s+13&s*t~> -%%EndData -showpage -%%Trailer -end -%%EOF diff --git a/lib/stdlib/doc/src/ushell3.gif b/lib/stdlib/doc/src/ushell3.gif Binary files differdeleted file mode 100644 index 2141268b91..0000000000 --- a/lib/stdlib/doc/src/ushell3.gif +++ /dev/null diff --git a/lib/stdlib/doc/src/ushell3.ps b/lib/stdlib/doc/src/ushell3.ps deleted file mode 100644 index dd64eeab5c..0000000000 --- a/lib/stdlib/doc/src/ushell3.ps +++ /dev/null @@ -1,662 +0,0 @@ -%!PS-Adobe-3.0 -%%Creator: GIMP PostScript file plugin V 1.17 by Peter Kirchgessner -%%Title: ushell3.ps -%%CreationDate: Mon Mar 16 12:15:17 2009 -%%DocumentData: Clean7Bit -%%LanguageLevel: 2 -%%Pages: 1 -%%BoundingBox: 14 14 468 78 -%%EndComments -%%BeginProlog -% Use own dictionary to avoid conflicts -10 dict begin -%%EndProlog -%%Page: 1 1 -% Translate for offset -14.173228346456694 14.173228346456694 translate -% Translate to begin of first scanline -0 63.070866141732289 translate -453.54330708661422 -63.070866141732289 scale -% Image geometry -640 89 8 -% Transformation matrix -[ 640 0 0 89 0 0 ] -% Strings to hold RGB-samples per scanline -/rstr 640 string def -/gstr 640 string def -/bstr 640 string def -{currentfile /ASCII85Decode filter /RunLengthDecode filter rstr readstring pop} -{currentfile /ASCII85Decode filter /RunLengthDecode filter gstr readstring pop} -{currentfile /ASCII85Decode filter /RunLengthDecode filter bstr readstring pop} -true 3 -%%BeginData: 37475 ASCII Bytes -colorimage -!1\W%J`VIEJ`VLFJ,~> -!1\W%J`VIEJ`VLFJ,~> -!1\W%J`VIEJ`VLFJ,~> -!T[+/6@hIS6@hIU6@]~> -!T[+/6@hIS6@hIU6@]~> -!T[+/6@hIS6@hIU6@]~> -!ouWfJQ.2"JQ.2"KN*I~> -!ouWfJQ.2"JQ.2"KN*I~> -!ouWfJQ.2"JQ.2"KN*I~> -!ouXLJcC<$JcC<$K`?Q~> -!ouXLJcC<$JcC<$K`?Q~> -!ouXLJcC<$JcC<$K`?Q~> -!ouXLQ2^m!`0L?'S`PG&JcC<$i;\<~> -!ouXLQ2^m!`0L?'S`PG&JcC<$i;\<~> -!ouXLQ2^m!`0L?'S`PG&JcC<$i;\<~> -"QVjNs.9-i!1*E[!L)+*rrR=EF8Z%>Dc_2VrrI?7rr3)%^&N-N<rg/5rr?^0<rgP6rrU,eg\:^K -o6/O]!ioDNpAY05U[\9aJa!(4!2BHo!,VW6!lAR@JcC<$JcF^/J,~> -"QVjNs.9-i!1*E[!L)+*rrR=EF8Z%>Dc_2VrrI?7rr3)%^&N-N<rg/5rr?^0<rgP6rrU,eg\:^K -o6/O]!ioDNpAY05U[\9aJa!(4!2BHo!,VW6!lAR@JcC<$JcF^/J,~> -"QVjNs.9-i!1*E[!L)+*rrR=EF8Z%>Dc_2VrrI?7rr3)%^&N-N<rg/5rr?^0<rgP6rrU,eg\:^K -o6/O]!ioDNpAY05U[\9aJa!(4!2BHo!,VW6!lAR@JcC<$JcF^/J,~> -"lqsOs)!Pi_uSNdrrHB@o)Ad;\gmXX!mS).p&>%Ili-neJXl`h]D]YJlM1AZb5K6Z:po`k!N()7 -rrML@l2LdgT^2UZil(<`rrJFsrVlntF7K88K;)-<s8T'RrkniDCAn/50C=>jJcC<$iW"E~> -"lqsOs)!Pi_uSNdrrHB@o)Ad;\gmXX!mS).p&>%Ili-neJXl`h]D]YJlM1AZb5K6Z:po`k!N()7 -rrML@l2LdgT^2UZil(<`rrJFsrVlntF7K88K;)-<s8T'RrkniDCAn/50C=>jJcC<$iW"E~> -"lqsOs)!Pi_uSNdrrHB@o)Ad;\gmXX!mS).p&>%Ili-neJXl`h]D]YJlM1AZb5K6Z:po`k!N()7 -rrML@l2LdgT^2UZil(<`rrJFsrVlntF7K88K;)-<s8T'RrkniDCAn/50C=>jJcC<$iW"E~> -"lqsOs)#"3rrHB@nc&X\.f95Hkqi#4!AWp>rrdnCs!dR.rrSCIp@&"_b$si5rrML@l2LdPK]E(5 -GGY9<!NC/2rs4ONpZ4C7s,`6>rrN#\rVlo\2M(mZs+14/s*t~> -"lqsOs)#"3rrHB@nc&X\.f95Hkqi#4!AWp>rrdnCs!dR.rrSCIp@&"_b$si5rrML@l2LdPK]E(5 -GGY9<!NC/2rs4ONpZ4C7s,`6>rrN#\rVlo\2M(mZs+14/s*t~> -"lqsOs)#"3rrHB@nc&X\.f95Hkqi#4!AWp>rrdnCs!dR.rrSCIp@&"_b$si5rrML@l2LdPK]E(5 -GGY9<!NC/2rs4ONpZ4C7s,`6>rrN#\rVlo\2M(mZs+14/s*t~> -#in9Rs)"@hjludE!Us(f*"5s=s'2Za*#r;Ts6`DV*!'XArr3#i.f95Hkqi#4$2ktGs.=c>s!c4l -*!(ffp&>*eKVIo7!Q/(<rrCjQ*!EJBm4518*!<VHr42bG&A7u":Z[6L*5hd.*"`Z#*8gPk!Q/1@ -rrsVGs/-hF]Dhg?\f;.l*$"qWWG$>m*$OSArs>H^rr<#^9E1*5p\t46JcC<$JcF^/J,~> -#in9Rs)"@hjludE!Us(f*"5s=s'2Za*#r;Ts6`DV*!'XArr3#i.f95Hkqi#4$2ktGs.=c>s!c4l -*!(ffp&>*eKVIo7!Q/(<rrCjQ*!EJBm4518*!<VHr42bG&A7u":Z[6L*5hd.*"`Z#*8gPk!Q/1@ -rrsVGs/-hF]Dhg?\f;.l*$"qWWG$>m*$OSArs>H^rr<#^9E1*5p\t46JcC<$JcF^/J,~> -#in9Rs)"@hjludE!Us(f*"5s=s'2Za*#r;Ts6`DV*!'XArr3#i.f95Hkqi#4$2ktGs.=c>s!c4l -*!(ffp&>*eKVIo7!Q/(<rrCjQ*!EJBm4518*!<VHr42bG&A7u":Z[6L*5hd.*"`Z#*8gPk!Q/1@ -rrsVGs/-hF]Dhg?\f;.l*$"qWWG$>m*$OSArs>H^rr<#^9E1*5p\t46JcC<$JcF^/J,~> -#in9Rs(s&?2H0VT!G&_>rsZf$s'-u6s8T->s&7&=rrJm@rr3#i.f95Hkqhu3#T`sFSpp_><^Zld -!I1F5rrJj?mJd4)=nhq!7G$o6ErQ(@4l><[rVm<akl8@2QiI(:nGiNVK_>?L_HQutn,FF,rrHT? -r;QeAV>^DuW>MT6s7dl/rs"REs8UEef)5OJ*J+6As+14/s*t~> -#in9Rs(s&?2H0VT!G&_>rsZf$s'-u6s8T->s&7&=rrJm@rr3#i.f95Hkqhu3#T`sFSpp_><^Zld -!I1F5rrJj?mJd4)=nhq!7G$o6ErQ(@4l><[rVm<akl8@2QiI(:nGiNVK_>?L_HQutn,FF,rrHT? -r;QeAV>^DuW>MT6s7dl/rs"REs8UEef)5OJ*J+6As+14/s*t~> -#in9Rs(s&?2H0VT!G&_>rsZf$s'-u6s8T->s&7&=rrJm@rr3#i.f95Hkqhu3#T`sFSpp_><^Zld -!I1F5rrJj?mJd4)=nhq!7G$o6ErQ(@4l><[rVm<akl8@2QiI(:nGiNVK_>?L_HQutn,FF,rrHT? -r;QeAV>^DuW>MT6s7dl/rs"REs8UEef)5OJ*J+6As+14/s*t~> -"lqsOs)#";rrJFMriH=Cs8Q??rr3,%EW8soriH3>HN*pFngaP:!U2E4rrciCl&)D8rrIA?pAY3[ -N2>qA!Q/(<rrDWgXTL6.m4eM="$PQ&3;rj[2<b(S?N0s.E:s82R=t85#/XRDCU3s\rVln-]);R/ -GGY9<!NC/>rrMm?nG`]SNW9#h7m6eM!$-XjJcC<$iW"E~> -"lqsOs)#";rrJFMriH=Cs8Q??rr3,%EW8soriH3>HN*pFngaP:!U2E4rrciCl&)D8rrIA?pAY3[ -N2>qA!Q/(<rrDWgXTL6.m4eM="$PQ&3;rj[2<b(S?N0s.E:s82R=t85#/XRDCU3s\rVln-]);R/ -GGY9<!NC/>rrMm?nG`]SNW9#h7m6eM!$-XjJcC<$iW"E~> -"lqsOs)#";rrJFMriH=Cs8Q??rr3,%EW8soriH3>HN*pFngaP:!U2E4rrciCl&)D8rrIA?pAY3[ -N2>qA!Q/(<rrDWgXTL6.m4eM="$PQ&3;rj[2<b(S?N0s.E:s82R=t85#/XRDCU3s\rVln-]);R/ -GGY9<!NC/>rrMm?nG`]SNW9#h7m6eM!$-XjJcC<$iW"E~> -"lqsOs)#":rrAAaD?P3us'3D>rrfBBs&2tsD?.$BrrM[?qu6]]1\C\Lp*RC[q>UJ?V"Xfh^Kp4+ -!Q/(=rrN"UrGD]YfBk9jrrYFAj>d,<"?#EC^0^[9!L\W6rs$>Ds(eq?*W?!=@_2L;!I(C=rrK*? -rr3#o,k1g7OH'8>!T-'<rr='js+13$s5F!.~> -"lqsOs)#":rrAAaD?P3us'3D>rrfBBs&2tsD?.$BrrM[?qu6]]1\C\Lp*RC[q>UJ?V"Xfh^Kp4+ -!Q/(=rrN"UrGD]YfBk9jrrYFAj>d,<"?#EC^0^[9!L\W6rs$>Ds(eq?*W?!=@_2L;!I(C=rrK*? -rr3#o,k1g7OH'8>!T-'<rr='js+13$s5F!.~> -"lqsOs)#":rrAAaD?P3us'3D>rrfBBs&2tsD?.$BrrM[?qu6]]1\C\Lp*RC[q>UJ?V"Xfh^Kp4+ -!Q/(=rrN"UrGD]YfBk9jrrYFAj>d,<"?#EC^0^[9!L\W6rs$>Ds(eq?*W?!=@_2L;!I(C=rrK*? -rr3#o,k1g7OH'8>!T-'<rr='js+13$s5F!.~> -"lqsOs)#";rrK1Rrr3,.C&_GSrr3,%EW8tZq>UKd.f95Hkqhu3"84(R@K-9-Mu!AP!I1F>rrV/% -Zi0n*m!?)+rrV55[Jp1,odBb="fictm1u>nrrYFAj>d,<"?#EC^0^[9"dt&Ds,$[Lrs$>D^O^an -*W?!=@_2L;#C!$Es+UKPF8l1?pa#A/!K`<?rrM"?rVlnZNIh+\s+14/s*t~> -"lqsOs)#";rrK1Rrr3,.C&_GSrr3,%EW8tZq>UKd.f95Hkqhu3"84(R@K-9-Mu!AP!I1F>rrV/% -Zi0n*m!?)+rrV55[Jp1,odBb="fictm1u>nrrYFAj>d,<"?#EC^0^[9"dt&Ds,$[Lrs$>D^O^an -*W?!=@_2L;#C!$Es+UKPF8l1?pa#A/!K`<?rrM"?rVlnZNIh+\s+14/s*t~> -"lqsOs)#";rrK1Rrr3,.C&_GSrr3,%EW8tZq>UKd.f95Hkqhu3"84(R@K-9-Mu!AP!I1F>rrV/% -Zi0n*m!?)+rrV55[Jp1,odBb="fictm1u>nrrYFAj>d,<"?#EC^0^[9"dt&Ds,$[Lrs$>D^O^an -*W?!=@_2L;#C!$Es+UKPF8l1?pa#A/!K`<?rrM"?rVlnZNIh+\s+14/s*t~> -"lqsOs(spt=Tb&ka`%/]"E;N`>f$F>"KQPB_eK*Q!1!Q`"F!iV=e#Ej"E@<P=ePKg"8nU*aSu2? -ZsEZ6!J95.rrU7AErH"=gd(0)!N()?rrN%dr`KD\/_BA3O8`8aPP"R6rE08hp]%s6qYp`RIT]gC -rV?I'lAL)E_.]YEs1_\\6!=!^rs-kl=]qs.WH8(=!Vdr0rrLc"r`K83]D_a10C=>jJcC<$iW"E~> -"lqsOs(spt=Tb&ka`%/]"E;N`>f$F>"KQPB_eK*Q!1!Q`"F!iV=e#Ej"E@<P=ePKg"8nU*aSu2? -ZsEZ6!J95.rrU7AErH"=gd(0)!N()?rrN%dr`KD\/_BA3O8`8aPP"R6rE08hp]%s6qYp`RIT]gC -rV?I'lAL)E_.]YEs1_\\6!=!^rs-kl=]qs.WH8(=!Vdr0rrLc"r`K83]D_a10C=>jJcC<$iW"E~> -"lqsOs(spt=Tb&ka`%/]"E;N`>f$F>"KQPB_eK*Q!1!Q`"F!iV=e#Ej"E@<P=ePKg"8nU*aSu2? -ZsEZ6!J95.rrU7AErH"=gd(0)!N()?rrN%dr`KD\/_BA3O8`8aPP"R6rE08hp]%s6qYp`RIT]gC -rV?I'lAL)E_.]YEs1_\\6!=!^rs-kl=]qs.WH8(=!Vdr0rrLc"r`K83]D_a10C=>jJcC<$iW"E~> -"QVjNs4mOh"53_Se,I2eeGoR$nG`FjleVU@ci1c]f`(mNc2PQ[gA_*PbPo?Yh"C[Jp<rm=!6+rS -!93tW!qO4arVlomdH^`5l@c>>rrDcl_?K,Np&!#$rk\d,s8VZg_?BH0s7"\:rr_/q_Y3a(#jL4G -s3:H@s6'?t!<)lr#1p`/s8VB?rr3#tb4#?1h>Y7kb5M>AGH(Ijs+13$s5F!.~> -"QVjNs4mOh"53_Se,I2eeGoR$nG`FjleVU@ci1c]f`(mNc2PQ[gA_*PbPo?Yh"C[Jp<rm=!6+rS -!93tW!qO4arVlomdH^`5l@c>>rrDcl_?K,Np&!#$rk\d,s8VZg_?BH0s7"\:rr_/q_Y3a(#jL4G -s3:H@s6'?t!<)lr#1p`/s8VB?rr3#tb4#?1h>Y7kb5M>AGH(Ijs+13$s5F!.~> -"QVjNs4mOh"53_Se,I2eeGoR$nG`FjleVU@ci1c]f`(mNc2PQ[gA_*PbPo?Yh"C[Jp<rm=!6+rS -!93tW!qO4arVlomdH^`5l@c>>rrDcl_?K,Np&!#$rk\d,s8VZg_?BH0s7"\:rr_/q_Y3a(#jL4G -s3:H@s6'?t!<)lr#1p`/s8VB?rr3#tb4#?1h>Y7kb5M>AGH(Ijs+13$s5F!.~> -!ouXLQ2^mSnWj+TkkTf0JcC<$i;\<~> -!ouXLQ2^mSnWj+TkkTf0JcC<$i;\<~> -!ouXLQ2^mSnWj+TkkTf0JcC<$i;\<~> -!ouXLJcC<$JcC<$K`?Q~> -!ouXLJcC<$JcC<$K`?Q~> -!ouXLJcC<$JcC<$K`?Q~> -!ouXLJcG]K!TNR]rr_0&bgEm!"4YQ=h#.0PdXhFLs/5mI~> -!ouXLJcG]K!TNR]rr_0&bgEm!"4YQ=h#.0PdXhFLs/5mI~> -!ouXLJcG]K!TNR]rr_0&bgEm!"4YQ=h#.0PdXhFLs/5mI~> -!ouXLrVll.r;Qf;o()e]\@V&,!kGPPr;QirYf6S@!PSC#rr_SkYi""2!o&"@rVm>df)Ne\s8V;e -g&M)arUBgji3;2?s1.k'"4H?3\b5t)>rthhrVluBmf'fsrr^ZQYd<0c!6"l@!6Y!7"1eHrci3qH -Bhpuir;Qe$_L_`<s/>sJ~> -!ouXLrVll.r;Qf;o()e]\@V&,!kGPPr;QirYf6S@!PSC#rr_SkYi""2!o&"@rVm>df)Ne\s8V;e -g&M)arUBgji3;2?s1.k'"4H?3\b5t)>rthhrVluBmf'fsrr^ZQYd<0c!6"l@!6Y!7"1eHrci3qH -Bhpuir;Qe$_L_`<s/>sJ~> -!ouXLrVll.r;Qf;o()e]\@V&,!kGPPr;QirYf6S@!PSC#rr_SkYi""2!o&"@rVm>df)Ne\s8V;e -g&M)arUBgji3;2?s1.k'"4H?3\b5t)>rthhrVluBmf'fsrr^ZQYd<0c!6"l@!6Y!7"1eHrci3qH -Bhpuir;Qe$_L_`<s/>sJ~> -!ouXLrr3"u(&\(5P`>P2rrICpm/I-7?T\2c!r?\qrVlmIoCDnbn;;!sKAbYe!T1]hrrJ(?r;RA& -HiH[ds6YVXJCjfHq>^KAch7;Bie2*#ir:%mrr[/AruLe4!F<J;rsMiIr#l%>c$X;BD5H.i!P<LR -CB8[tpAY48s0uV;rrLA?r;QfW3e@<^s/>sJ~> -!ouXLrr3"u(&\(5P`>P2rrICpm/I-7?T\2c!r?\qrVlmIoCDnbn;;!sKAbYe!T1]hrrJ(?r;RA& -HiH[ds6YVXJCjfHq>^KAch7;Bie2*#ir:%mrr[/AruLe4!F<J;rsMiIr#l%>c$X;BD5H.i!P<LR -CB8[tpAY48s0uV;rrLA?r;QfW3e@<^s/>sJ~> -!ouXLrr3"u(&\(5P`>P2rrICpm/I-7?T\2c!r?\qrVlmIoCDnbn;;!sKAbYe!T1]hrrJ(?r;RA& -HiH[ds6YVXJCjfHq>^KAch7;Bie2*#ir:%mrr[/AruLe4!F<J;rsMiIr#l%>c$X;BD5H.i!P<LR -CB8[tpAY48s0uV;rrLA?r;QfW3e@<^s/>sJ~> -#NS0Qs8SgH'Dqe1RtBDp!T?-5rrFS?o)AmK5l^js[GUubZ>0::!n)SAr;RA&HiH[ds3jR>_1$W_ -r;ZeYN:m2T\mk]II!C_Grr[/AruLe4!F<J;rsMiIr#l%>LR%o>oI8\t#.&^Es8U`apAY48s0uV; -rrLA?r;QigKVj>#JcD_LJ,~> -#NS0Qs8SgH'Dqe1RtBDp!T?-5rrFS?o)AmK5l^js[GUubZ>0::!n)SAr;RA&HiH[ds3jR>_1$W_ -r;ZeYN:m2T\mk]II!C_Grr[/AruLe4!F<J;rsMiIr#l%>LR%o>oI8\t#.&^Es8U`apAY48s0uV; -rrLA?r;QigKVj>#JcD_LJ,~> -#NS0Qs8SgH'Dqe1RtBDp!T?-5rrFS?o)AmK5l^js[GUubZ>0::!n)SAr;RA&HiH[ds3jR>_1$W_ -r;ZeYN:m2T\mk]II!C_Grr[/AruLe4!F<J;rsMiIr#l%>LR%o>oI8\t#.&^Es8U`apAY48s0uV; -rrLA?r;QigKVj>#JcD_LJ,~> -#NS0Qs.PG>*W#d:UNuP4!e+Bor;QbBr`fGnp&>'O48f*[Bj?GlrrFS4r`fH/rVlrS>c%E!#M_TE -s2a:$pAJ50_Z0XSs3"YPaT)6E>a;X#c2[h+rEKWWs79J\-<sg.rr3#!F8PtRh9c29n,Mjis8U'5 -o)IQO1br<@mf3=+rEKd&s5qB%s1G-(s3748r`fI&rr3)Dir?1SrrHE?qu72Dli+*fs+cm>s7@c? -s/j-:rrM&<rr3"Nm/?qjU<NXUs4*P;s,-e\!/pjV"@#[e>`o$c!.XuQ!1*T`"0V[da8Gr<btn6: -!QA,ks+13Ls*t~> -#NS0Qs.PG>*W#d:UNuP4!e+Bor;QbBr`fGnp&>'O48f*[Bj?GlrrFS4r`fH/rVlrS>c%E!#M_TE -s2a:$pAJ50_Z0XSs3"YPaT)6E>a;X#c2[h+rEKWWs79J\-<sg.rr3#!F8PtRh9c29n,Mjis8U'5 -o)IQO1br<@mf3=+rEKd&s5qB%s1G-(s3748r`fI&rr3)Dir?1SrrHE?qu72Dli+*fs+cm>s7@c? -s/j-:rrM&<rr3"Nm/?qjU<NXUs4*P;s,-e\!/pjV"@#[e>`o$c!.XuQ!1*T`"0V[da8Gr<btn6: -!QA,ks+13Ls*t~> -#NS0Qs.PG>*W#d:UNuP4!e+Bor;QbBr`fGnp&>'O48f*[Bj?GlrrFS4r`fH/rVlrS>c%E!#M_TE -s2a:$pAJ50_Z0XSs3"YPaT)6E>a;X#c2[h+rEKWWs79J\-<sg.rr3#!F8PtRh9c29n,Mjis8U'5 -o)IQO1br<@mf3=+rEKd&s5qB%s1G-(s3748r`fI&rr3)Dir?1SrrHE?qu72Dli+*fs+cm>s7@c? -s/j-:rrM&<rr3"Nm/?qjU<NXUs4*P;s,-e\!/pjV"@#[e>`o$c!.XuQ!1*T`"0V[da8Gr<btn6: -!QA,ks+13Ls*t~> -#NS0Qs4Zf=*W#d;ql"c5rrUHX,5qNBGcC\W!KEfHrrM(?r;Qi:U."t[!@^M;^B&_orr3&@!-8&< -!Sfs<rr>sq^CtP2s0b1+^V=R6rYN>5HbX4Is7Q?*#\)ch^P5S"^[V7&!NL5.rsA82Ch^Z!q#AcZ -rP&>0q#:E#)he4*!*/Ie!F<J5rs=H,5X7I]s8RS?rr3#E:B(7o4Q68=$#ZsH*WNf[s,<HH^BC!c -s+R-F^BBjes*pjD^B'Ldp&>';:](.m`E.WjJcD_LJ,~> -#NS0Qs4Zf=*W#d;ql"c5rrUHX,5qNBGcC\W!KEfHrrM(?r;Qi:U."t[!@^M;^B&_orr3&@!-8&< -!Sfs<rr>sq^CtP2s0b1+^V=R6rYN>5HbX4Is7Q?*#\)ch^P5S"^[V7&!NL5.rsA82Ch^Z!q#AcZ -rP&>0q#:E#)he4*!*/Ie!F<J5rs=H,5X7I]s8RS?rr3#E:B(7o4Q68=$#ZsH*WNf[s,<HH^BC!c -s+R-F^BBjes*pjD^B'Ldp&>';:](.m`E.WjJcD_LJ,~> -#NS0Qs4Zf=*W#d;ql"c5rrUHX,5qNBGcC\W!KEfHrrM(?r;Qi:U."t[!@^M;^B&_orr3&@!-8&< -!Sfs<rr>sq^CtP2s0b1+^V=R6rYN>5HbX4Is7Q?*#\)ch^P5S"^[V7&!NL5.rsA82Ch^Z!q#AcZ -rP&>0q#:E#)he4*!*/Ie!F<J5rs=H,5X7I]s8RS?rr3#E:B(7o4Q68=$#ZsH*WNf[s,<HH^BC!c -s+R-F^BBjes*pjD^B'Ldp&>';:](.m`E.WjJcD_LJ,~> -!ouXLrVlj<q>UKC9D/;cqBGn<!DCl?rrJR?pAY0P48]$YfgPf=!@cOtrrIA?rr3&og["k="?7J0 -,`Vg'!$2%<&RW6Okq)#ts&R/=+9/3^s%^l:rrkjCs8TB>qu6\sF6ii,OGs/<!Q&%=rrCsOrrTH1 -mf*4d*U`q.?,-(4%YoShgX7PNs6XR>s6`D>rr3!]iVic_DQj'\s1Me>A[h[="LVqC?,6I>"KZSB -<65(=!V5UMrrLA?qu6]9=+UC&s/H$K~> -!ouXLrVlj<q>UKC9D/;cqBGn<!DCl?rrJR?pAY0P48]$YfgPf=!@cOtrrIA?rr3&og["k="?7J0 -,`Vg'!$2%<&RW6Okq)#ts&R/=+9/3^s%^l:rrkjCs8TB>qu6\sF6ii,OGs/<!Q&%=rrCsOrrTH1 -mf*4d*U`q.?,-(4%YoShgX7PNs6XR>s6`D>rr3!]iVic_DQj'\s1Me>A[h[="LVqC?,6I>"KZSB -<65(=!V5UMrrLA?qu6]9=+UC&s/H$K~> -!ouXLrVlj<q>UKC9D/;cqBGn<!DCl?rrJR?pAY0P48]$YfgPf=!@cOtrrIA?rr3&og["k="?7J0 -,`Vg'!$2%<&RW6Okq)#ts&R/=+9/3^s%^l:rrkjCs8TB>qu6\sF6ii,OGs/<!Q&%=rrCsOrrTH1 -mf*4d*U`q.?,-(4%YoShgX7PNs6XR>s6`D>rr3!]iVic_DQj'\s1Me>A[h[="LVqC?,6I>"KZSB -<65(=!V5UMrrLA?qu6]9=+UC&s/H$K~> -!ouXLrVlj<qYpWR7/HTV!W"#=rrGd@rr3"fK(f3FiB-r:!SKm>rrFS?rr3"DV"jrmpsqY'qu6Tq -*W?!@?bQL(1]IC^qYgC8s/(#?:<rj^r>YtHfDklVCA\#3WcRM+!K`9=rrN(ur>Ygmq#:B"C]485 -*U`q.?,-(4"c&BCs7@c>rrd5Er7aR:rrG7@rVm1AYQ#XC]jLeN\,QC1[:oSG_#F?=XDn4+7iWLI -oD\j9:](.m`E.WjJcD_LJ,~> -!ouXLrVlj<qYpWR7/HTV!W"#=rrGd@rr3"fK(f3FiB-r:!SKm>rrFS?rr3"DV"jrmpsqY'qu6Tq -*W?!@?bQL(1]IC^qYgC8s/(#?:<rj^r>YtHfDklVCA\#3WcRM+!K`9=rrN(ur>Ygmq#:B"C]485 -*U`q.?,-(4"c&BCs7@c>rrd5Er7aR:rrG7@rVm1AYQ#XC]jLeN\,QC1[:oSG_#F?=XDn4+7iWLI -oD\j9:](.m`E.WjJcD_LJ,~> -!ouXLrVlj<qYpWR7/HTV!W"#=rrGd@rr3"fK(f3FiB-r:!SKm>rrFS?rr3"DV"jrmpsqY'qu6Tq -*W?!@?bQL(1]IC^qYgC8s/(#?:<rj^r>YtHfDklVCA\#3WcRM+!K`9=rrN(ur>Ygmq#:B"C]485 -*U`q.?,-(4"c&BCs7@c>rrd5Er7aR:rrG7@rVm1AYQ#XC]jLeN\,QC1[:oSG_#F?=XDn4+7iWLI -oD\j9:](.m`E.WjJcD_LJ,~> -!ouXLrVlj<qu6]J7e?W\qBGn<!DCl?rrJR?pAY0P48]$YfgPf=!@c+hrrIA?rr3&lcfG')!Sfs< -rr=)<rrc]Bs62?;rsAZHs/(#?:<rM[rr3Mf37n31Z>0F>gA1dK\p3N,rs+XEs8UUKoAKTI!$1k7 -!O6G=rr=)4rrV!%r;HWr?,-(4"c&BCs7@c=rrPU@*W5pJ4Q6.is8Qu?ruM-Q?iO]Hrr3,/B`DAR -rr3,&E;rnYrr3#ejn8WSbtn9;!rAp@r;Qincf'HTJcE"TJ,~> -!ouXLrVlj<qu6]J7e?W\qBGn<!DCl?rrJR?pAY0P48]$YfgPf=!@c+hrrIA?rr3&lcfG')!Sfs< -rr=)<rrc]Bs62?;rsAZHs/(#?:<rM[rr3Mf37n31Z>0F>gA1dK\p3N,rs+XEs8UUKoAKTI!$1k7 -!O6G=rr=)4rrV!%r;HWr?,-(4"c&BCs7@c=rrPU@*W5pJ4Q6.is8Qu?ruM-Q?iO]Hrr3,/B`DAR -rr3,&E;rnYrr3#ejn8WSbtn9;!rAp@r;Qincf'HTJcE"TJ,~> -!ouXLrVlj<qu6]J7e?W\qBGn<!DCl?rrJR?pAY0P48]$YfgPf=!@c+hrrIA?rr3&lcfG')!Sfs< -rr=)<rrc]Bs62?;rsAZHs/(#?:<rM[rr3Mf37n31Z>0F>gA1dK\p3N,rs+XEs8UUKoAKTI!$1k7 -!O6G=rr=)4rrV!%r;HWr?,-(4"c&BCs7@c=rrPU@*W5pJ4Q6.is8Qu?ruM-Q?iO]Hrr3,/B`DAR -rr3,&E;rnYrr3#ejn8WSbtn9;!rAp@r;Qincf'HTJcE"TJ,~> -&*-#Ys3n9Z'XG$js8U/Gp@S@j_ljo_cN!oqT`24tMQ$:L"M:6SZI]$T#e?<UZIo3W.8^#L!I_`^ -rrU7AErH"=gd(f;!+#Q/"EMoikqi8;$2u%H_4Ua&cM1ZDZ3k[gfDklmGI"Mcl2L\`N0*E+#I/ug -Z<@4XE;d'B>PS4!Z"s=<!$1b4!_EC^rVln(^\.U1X,+'eY%dk7!dpbLr;R:NVPeS:s-]%U'XF*e -s-&cJZN$0ks,EHGZ36;Zs+m3EZ2ouZp&>';:]14ni]?u:!nmW^JcC<$Z2]=~> -&*-#Ys3n9Z'XG$js8U/Gp@S@j_ljo_cN!oqT`24tMQ$:L"M:6SZI]$T#e?<UZIo3W.8^#L!I_`^ -rrU7AErH"=gd(f;!+#Q/"EMoikqi8;$2u%H_4Ua&cM1ZDZ3k[gfDklmGI"Mcl2L\`N0*E+#I/ug -Z<@4XE;d'B>PS4!Z"s=<!$1b4!_EC^rVln(^\.U1X,+'eY%dk7!dpbLr;R:NVPeS:s-]%U'XF*e -s-&cJZN$0ks,EHGZ36;Zs+m3EZ2ouZp&>';:]14ni]?u:!nmW^JcC<$Z2]=~> -&*-#Ys3n9Z'XG$js8U/Gp@S@j_ljo_cN!oqT`24tMQ$:L"M:6SZI]$T#e?<UZIo3W.8^#L!I_`^ -rrU7AErH"=gd(f;!+#Q/"EMoikqi8;$2u%H_4Ua&cM1ZDZ3k[gfDklmGI"Mcl2L\`N0*E+#I/ug -Z<@4XE;d'B>PS4!Z"s=<!$1b4!_EC^rVln(^\.U1X,+'eY%dk7!dpbLr;R:NVPeS:s-]%U'XF*e -s-&cJZN$0ks,EHGZ36;Zs+m3EZ2ouZp&>';:]14ni]?u:!nmW^JcC<$Z2]=~> -"QVjNs/Gp1"JPkqP32B9!/UVg!2'8j!/(8b!i5k1qj%?1s8RBDB`P=8rr@0?B`tdBs+0V;B`P[A -rrV%kXSr/"l]1oC!;HG+"NUQBnq[/C!WIHErr\VIs60Gr!l^>crr3)9Bkc?#rrTfah<b.GXD)D; -s8U7?B`Rf#rrLNGrVlkDp&>)W'(Pr#"'oo4T_ABfVeKj`qu6\Hq>:0mH?oJh"INm3Boi8n!0dCr -!0mK_!07%m!1Eid!/^\h!1roa"/_B0:]14nOc0,:!pOEmJcC<$Z2]=~> -"QVjNs/Gp1"JPkqP32B9!/UVg!2'8j!/(8b!i5k1qj%?1s8RBDB`P=8rr@0?B`tdBs+0V;B`P[A -rrV%kXSr/"l]1oC!;HG+"NUQBnq[/C!WIHErr\VIs60Gr!l^>crr3)9Bkc?#rrTfah<b.GXD)D; -s8U7?B`Rf#rrLNGrVlkDp&>)W'(Pr#"'oo4T_ABfVeKj`qu6\Hq>:0mH?oJh"INm3Boi8n!0dCr -!0mK_!07%m!1Eid!/^\h!1roa"/_B0:]14nOc0,:!pOEmJcC<$Z2]=~> -"QVjNs/Gp1"JPkqP32B9!/UVg!2'8j!/(8b!i5k1qj%?1s8RBDB`P=8rr@0?B`tdBs+0V;B`P[A -rrV%kXSr/"l]1oC!;HG+"NUQBnq[/C!WIHErr\VIs60Gr!l^>crr3)9Bkc?#rrTfah<b.GXD)D; -s8U7?B`Rf#rrLNGrVlkDp&>)W'(Pr#"'oo4T_ABfVeKj`qu6\Hq>:0mH?oJh"INm3Boi8n!0dCr -!0mK_!07%m!1Eid!/^\h!1roa"/_B0:]14nOc0,:!pOEmJcC<$Z2]=~> -!ouXLg&D&Xqh5$jXoAF5fYd^i/+NT<"0mQ6OPKj9*SgYsQ\C-or;Qb\JcC<$VuM8~> -!ouXLg&D&Xqh5$jXoAF5fYd^i/+NT<"0mQ6OPKj9*SgYsQ\C-or;Qb\JcC<$VuM8~> -!ouXLg&D&Xqh5$jXoAF5fYd^i/+NT<"0mQ6OPKj9*SgYsQ\C-or;Qb\JcC<$VuM8~> -!ouXLJcEIa!fWE@eGfLhJcC<$JcGNFJ,~> -!ouXLJcEIa!fWE@eGfLhJcC<$JcGNFJ,~> -!ouXLJcEIa!fWE@eGfLhJcC<$JcGNFJ,~> -!ouXLJcEIa!RUJfrrBugs+13$s7lVE~> -!ouXLJcEIa!RUJfrrBugs+13$s7lVE~> -!ouXLJcEIa!RUJfrrBugs+13$s7lVE~> -!ouXLrr3't-73*urrYk?->%i/VuHj>-71q,rrYb<->A&2JcDGDJ,~> -!ouXLrr3't-73*urrYk?->%i/VuHj>-71q,rrYb<->A&2JcDGDJ,~> -!ouXLrr3't-73*urrYk?->%i/VuHj>-71q,rrYb<->A&2JcDGDJ,~> -!ouXLrr3'UiUusDrs(%<p%\ReoM>H+"C_!'=r@2X",R!J62hi)Dt`u+!V%uF*!C]as#M+t*!3#; -5lDZ''<(^+!:'M&".8rb)Z1QV`W%.\*!$Vlq>UKpc2AUb[/Bt%Y5TCHaoDD<V]6\VqtU0lr3?2? -"47(m&H*RN*rl,4r>Ygpq#:BkQMhd'mem(cnKn27"SQdS*96en!V^p>*!%tfr;Qr]1&q:ID#PCZ -7.^HYq"=;#o`#1Qo$gS;s)5IHo)Ae=s1<%Frr_uf`D;'bJcDGDJ,~> -!ouXLrr3'UiUusDrs(%<p%\ReoM>H+"C_!'=r@2X",R!J62hi)Dt`u+!V%uF*!C]as#M+t*!3#; -5lDZ''<(^+!:'M&".8rb)Z1QV`W%.\*!$Vlq>UKpc2AUb[/Bt%Y5TCHaoDD<V]6\VqtU0lr3?2? -"47(m&H*RN*rl,4r>Ygpq#:BkQMhd'mem(cnKn27"SQdS*96en!V^p>*!%tfr;Qr]1&q:ID#PCZ -7.^HYq"=;#o`#1Qo$gS;s)5IHo)Ae=s1<%Frr_uf`D;'bJcDGDJ,~> -!ouXLrr3'UiUusDrs(%<p%\ReoM>H+"C_!'=r@2X",R!J62hi)Dt`u+!V%uF*!C]as#M+t*!3#; -5lDZ''<(^+!:'M&".8rb)Z1QV`W%.\*!$Vlq>UKpc2AUb[/Bt%Y5TCHaoDD<V]6\VqtU0lr3?2? -"47(m&H*RN*rl,4r>Ygpq#:BkQMhd'mem(cnKn27"SQdS*96en!V^p>*!%tfr;Qr]1&q:ID#PCZ -7.^HYq"=;#o`#1Qo$gS;s)5IHo)Ae=s1<%Frr_uf`D;'bJcDGDJ,~> -!ouXLrr3!SlMLS^.eEW=!F3J?rrfBBs&7&:rrGd@rr3"fK)#?H3o^/="Gr?B1$eT6!pk9@q#:=7 -rVlt4[Jq9@rrV=od/!\BodB_<"@^r?ZYfX?#PtQEs8R)Bqt^6mdn0N<"!m]c;>^@o_HQp<!$1k7 -!O6G=rr=)<rsRa\$ig7pJueqO6eV87!K`<?rrM"?rVm)U2_"e,J"HZ>!R=I<rr=)3rrXeBdRsN= -!C,E2rrX;A\7GO;!R4Dks+13Ds*t~> -!ouXLrr3!SlMLS^.eEW=!F3J?rrfBBs&7&:rrGd@rr3"fK)#?H3o^/="Gr?B1$eT6!pk9@q#:=7 -rVlt4[Jq9@rrV=od/!\BodB_<"@^r?ZYfX?#PtQEs8R)Bqt^6mdn0N<"!m]c;>^@o_HQp<!$1k7 -!O6G=rr=)<rsRa\$ig7pJueqO6eV87!K`<?rrM"?rVm)U2_"e,J"HZ>!R=I<rr=)3rrXeBdRsN= -!C,E2rrX;A\7GO;!R4Dks+13Ds*t~> -!ouXLrr3!SlMLS^.eEW=!F3J?rrfBBs&7&:rrGd@rr3"fK)#?H3o^/="Gr?B1$eT6!pk9@q#:=7 -rVlt4[Jq9@rrV=od/!\BodB_<"@^r?ZYfX?#PtQEs8R)Bqt^6mdn0N<"!m]c;>^@o_HQp<!$1k7 -!O6G=rr=)<rsRa\$ig7pJueqO6eV87!K`<?rrM"?rVm)U2_"e,J"HZ>!R=I<rr=)3rrXeBdRsN= -!C,E2rrX;A\7GO;!R4Dks+13Ds*t~> -!ouXLrr3!SlMLS^.eEW=!F3J?rt2;Os&7%P[C+B]s*rr+s6>N>q#:@Uir8r\M3Irokl.sigACmN -QA>57!$2%<"D>.B*V99<!9="W!oJ@Aq#:Bf-iO&I7G$o\qu6of@a>#Rc!]u8rrLS?rVm!Gq9)Xm -rj;k&s2%t<rr=)7rrKB?rVlj<rr3DWCt]p0s1Eg>s8TE?q#:AVNW/qYh`^u=#KpH:.KBEtT)S`j -c;+<;!$1_3").AbJ,]HLGGkH=rrp[9]sY8prVlt,s36](rrLA?JcC<$U&TW~> -!ouXLrr3!SlMLS^.eEW=!F3J?rt2;Os&7%P[C+B]s*rr+s6>N>q#:@Uir8r\M3Irokl.sigACmN -QA>57!$2%<"D>.B*V99<!9="W!oJ@Aq#:Bf-iO&I7G$o\qu6of@a>#Rc!]u8rrLS?rVm!Gq9)Xm -rj;k&s2%t<rr=)7rrKB?rVlj<rr3DWCt]p0s1Eg>s8TE?q#:AVNW/qYh`^u=#KpH:.KBEtT)S`j -c;+<;!$1_3").AbJ,]HLGGkH=rrp[9]sY8prVlt,s36](rrLA?JcC<$U&TW~> -!ouXLrr3!SlMLS^.eEW=!F3J?rt2;Os&7%P[C+B]s*rr+s6>N>q#:@Uir8r\M3Irokl.sigACmN -QA>57!$2%<"D>.B*V99<!9="W!oJ@Aq#:Bf-iO&I7G$o\qu6of@a>#Rc!]u8rrLS?rVm!Gq9)Xm -rj;k&s2%t<rr=)7rrKB?rVlj<rr3DWCt]p0s1Eg>s8TE?q#:AVNW/qYh`^u=#KpH:.KBEtT)S`j -c;+<;!$1_3").AbJ,]HLGGkH=rrp[9]sY8prVlt,s36](rrLA?JcC<$U&TW~> -!ouXLrr3!SlMLS^.eEW=%U?jMs6409s&42YAnH;1rVlreMkg%H#s.)Gs4gM9s"TH2Ac[D5rVlrX -KX^^U!$2(="jT<>rtpjsAc[YJrr3)X=&duMrrMd?rr3/<.bF&Ai;T,MkPtS<GF=kBht[3Pdn0Q= -"g\s>b;"YKAcn:t_HQs=!N[(3rrKB?rr38#*WQ/eI*Va4rr3,/Kpe?Lp\u"jNW9%XL->S:_fb#3 -.KBEtT)\ibHqsV>!$1G+!ILR>rruGICM%1'an>Z7btiojJcDGDJ,~> -!ouXLrr3!SlMLS^.eEW=%U?jMs6409s&42YAnH;1rVlreMkg%H#s.)Gs4gM9s"TH2Ac[D5rVlrX -KX^^U!$2(="jT<>rtpjsAc[YJrr3)X=&duMrrMd?rr3/<.bF&Ai;T,MkPtS<GF=kBht[3Pdn0Q= -"g\s>b;"YKAcn:t_HQs=!N[(3rrKB?rr38#*WQ/eI*Va4rr3,/Kpe?Lp\u"jNW9%XL->S:_fb#3 -.KBEtT)\ibHqsV>!$1G+!ILR>rruGICM%1'an>Z7btiojJcDGDJ,~> -!ouXLrr3!SlMLS^.eEW=%U?jMs6409s&42YAnH;1rVlreMkg%H#s.)Gs4gM9s"TH2Ac[D5rVlrX -KX^^U!$2(="jT<>rtpjsAc[YJrr3)X=&duMrrMd?rr3/<.bF&Ai;T,MkPtS<GF=kBht[3Pdn0Q= -"g\s>b;"YKAcn:t_HQs=!N[(3rrKB?rr38#*WQ/eI*Va4rr3,/Kpe?Lp\u"jNW9%XL->S:_fb#3 -.KBEtT)\ibHqsV>!$1G+!ILR>rruGICM%1'an>Z7btiojJcDGDJ,~> -!ouXLrr3!SlMLS^.eEW=%^1>0I=>7;s/UbCs8Sj?rr3&pI\lc<$(cToI=F_Ms-e`BrrI\@rVlo) -C%hQ,NrFG1Il4Y6MO4>B!FNP>rrgr*Jq)eUrrMtIrd=s#,M2<!If=p)Idd<rItE9%jS8`Ul@Jq_ -"b@<^jat#ArrX;AiILoU!J(s[rrLf'rd>''*<4K;aoDA]rr3,:J:`B,p\t98J,Xj)E]sH>JD'tp -.KBF`J,XisBhnU+!$1D*!$2%<##o4*s8Sm?p&>';:P&Oss.TIC~> -!ouXLrr3!SlMLS^.eEW=%^1>0I=>7;s/UbCs8Sj?rr3&pI\lc<$(cToI=F_Ms-e`BrrI\@rVlo) -C%hQ,NrFG1Il4Y6MO4>B!FNP>rrgr*Jq)eUrrMtIrd=s#,M2<!If=p)Idd<rItE9%jS8`Ul@Jq_ -"b@<^jat#ArrX;AiILoU!J(s[rrLf'rd>''*<4K;aoDA]rr3,:J:`B,p\t98J,Xj)E]sH>JD'tp -.KBF`J,XisBhnU+!$1D*!$2%<##o4*s8Sm?p&>';:P&Oss.TIC~> -!ouXLrr3!SlMLS^.eEW=%^1>0I=>7;s/UbCs8Sj?rr3&pI\lc<$(cToI=F_Ms-e`BrrI\@rVlo) -C%hQ,NrFG1Il4Y6MO4>B!FNP>rrgr*Jq)eUrrMtIrd=s#,M2<!If=p)Idd<rItE9%jS8`Ul@Jq_ -"b@<^jat#ArrX;AiILoU!J(s[rrLf'rd>''*<4K;aoDA]rr3,:J:`B,p\t98J,Xj)E]sH>JD'tp -.KBF`J,XisBhnU+!$1D*!$2%<##o4*s8Sm?p&>';:P&Oss.TIC~> -!ouXLrr3!SlMLS^.eET<"h("oZ`A*CrrJm@rr3":Y4V_tWLf]sM3Ii:!J-d?rrV2$`qKE4rViAi -c"FH]rrHK?qu6[t`q]Q6nG]!_fgXN[*kVFO#sI26DSQ5O]l*B8rrD*YSc\%"q>:0o*WQ/qrgj/X -*V]R6f)D6Do,[k1/H0&a$[;WhJ(Xf$TR?M6(!<,>TAMg3LH/dO>n;nos21G"VTqs7rr=)*rrJR? -rr3!uaSu2?Uj;Y5!R4Dks+13Ds*t~> -!ouXLrr3!SlMLS^.eET<"h("oZ`A*CrrJm@rr3":Y4V_tWLf]sM3Ii:!J-d?rrV2$`qKE4rViAi -c"FH]rrHK?qu6[t`q]Q6nG]!_fgXN[*kVFO#sI26DSQ5O]l*B8rrD*YSc\%"q>:0o*WQ/qrgj/X -*V]R6f)D6Do,[k1/H0&a$[;WhJ(Xf$TR?M6(!<,>TAMg3LH/dO>n;nos21G"VTqs7rr=)*rrJR? -rr3!uaSu2?Uj;Y5!R4Dks+13Ds*t~> -!ouXLrr3!SlMLS^.eET<"h("oZ`A*CrrJm@rr3":Y4V_tWLf]sM3Ii:!J-d?rrV2$`qKE4rViAi -c"FH]rrHK?qu6[t`q]Q6nG]!_fgXN[*kVFO#sI26DSQ5O]l*B8rrD*YSc\%"q>:0o*WQ/qrgj/X -*V]R6f)D6Do,[k1/H0&a$[;WhJ(Xf$TR?M6(!<,>TAMg3LH/dO>n;nos21G"VTqs7rr=)*rrJR? -rr3!uaSu2?Uj;Y5!R4Dks+13Ds*t~> -!ouXLrr3!SlMLS^.eEH8!NU5;rro0Ds8Q]>n,EEFPPb@\JXl`>!ROO0rrHc?qu6\(^&.g1<lXb2 -"%Ci/0`D"R4l><[rVlmUkk"fT,l.?;!$1q9!$1Y1!Zh=)rON+H`;cKXrr3#g/G&lDh`_"srji0: -*mOT^!R=I<rr=)*rs&7lJcGaLaSu2?Uj;Y5!R4Dks+13Ds*t~> -!ouXLrr3!SlMLS^.eEH8!NU5;rro0Ds8Q]>n,EEFPPb@\JXl`>!ROO0rrHc?qu6\(^&.g1<lXb2 -"%Ci/0`D"R4l><[rVlmUkk"fT,l.?;!$1q9!$1Y1!Zh=)rON+H`;cKXrr3#g/G&lDh`_"srji0: -*mOT^!R=I<rr=)*rs&7lJcGaLaSu2?Uj;Y5!R4Dks+13Ds*t~> -!ouXLrr3!SlMLS^.eEH8!NU5;rro0Ds8Q]>n,EEFPPb@\JXl`>!ROO0rrHc?qu6\(^&.g1<lXb2 -"%Ci/0`D"R4l><[rVlmUkk"fT,l.?;!$1q9!$1Y1!Zh=)rON+H`;cKXrr3#g/G&lDh`_"srji0: -*mOT^!R=I<rr=)*rs&7lJcGaLaSu2?Uj;Y5!R4Dks+13Ds*t~> -!ouXLrr3!SlMLS^.eEK9"jpfCs)6?brro0Ds(Jn>qYpWa.;\k\"grXDs&/:ars"%Es8VR[iqr`W -cmaeIrre^NoD_,TrrZWAs$?V`!ER5>rrTPVV>C2pI&Hf-0`D"X4l?.@E<#rUZMXY$!gkFQqYp\* -^AcSurVlj<qu6[obPhGBHR[hJrrGL?qu6sCs8VS?WW2u_h>R?T?RbjKrrW-`i;ETUoI9\;!q&\@ -r;Qa;qu6[Y8Flcaa]&6><65(=!Mk#6rrLA?JcC<$U&TW~> -!ouXLrr3!SlMLS^.eEK9"jpfCs)6?brro0Ds(Jn>qYpWa.;\k\"grXDs&/:ars"%Es8VR[iqr`W -cmaeIrre^NoD_,TrrZWAs$?V`!ER5>rrTPVV>C2pI&Hf-0`D"X4l?.@E<#rUZMXY$!gkFQqYp\* -^AcSurVlj<qu6[obPhGBHR[hJrrGL?qu6sCs8VS?WW2u_h>R?T?RbjKrrW-`i;ETUoI9\;!q&\@ -r;Qa;qu6[Y8Flcaa]&6><65(=!Mk#6rrLA?JcC<$U&TW~> -!ouXLrr3!SlMLS^.eEK9"jpfCs)6?brro0Ds(Jn>qYpWa.;\k\"grXDs&/:ars"%Es8VR[iqr`W -cmaeIrre^NoD_,TrrZWAs$?V`!ER5>rrTPVV>C2pI&Hf-0`D"X4l?.@E<#rUZMXY$!gkFQqYp\* -^AcSurVlj<qu6[obPhGBHR[hJrrGL?qu6sCs8VS?WW2u_h>R?T?RbjKrrW-`i;ETUoI9\;!q&\@ -r;Qa;qu6[Y8Flcaa]&6><65(=!Mk#6rrLA?JcC<$U&TW~> -!ouXLrr3'Uh!jt7rrXb3nFZ_[#ZMc`9(r;bp.>6'"D[`3<Z:oV!0I3[!p^7krVm-\*#rScs8VaD -rYu)rqZ$T`48SsYb6i:hrrDop*!Cojs8V9\*!;u6s5<hq!NQ(srrT5,PlC[_h#88rTD\`gb5E:c -^]"31WuN+^r;HWsM[$WhrrBe4*!EqQs89%u*!>!rs0)DA!QtE?rrQs=fDbdMT`,o2f_YUJ/,oPO -q/RGpkPY>]9EP%hrrdon*$4bLrrMaLrr3+B*#pR'rr3)nnF.IqrrF>jo)AmkXapFYbPqMBWIagD -"7krt:P&Oss.TIC~> -!ouXLrr3'Uh!jt7rrXb3nFZ_[#ZMc`9(r;bp.>6'"D[`3<Z:oV!0I3[!p^7krVm-\*#rScs8VaD -rYu)rqZ$T`48SsYb6i:hrrDop*!Cojs8V9\*!;u6s5<hq!NQ(srrT5,PlC[_h#88rTD\`gb5E:c -^]"31WuN+^r;HWsM[$WhrrBe4*!EqQs89%u*!>!rs0)DA!QtE?rrQs=fDbdMT`,o2f_YUJ/,oPO -q/RGpkPY>]9EP%hrrdon*$4bLrrMaLrr3+B*#pR'rr3)nnF.IqrrF>jo)AmkXapFYbPqMBWIagD -"7krt:P&Oss.TIC~> -!ouXLrr3'Uh!jt7rrXb3nFZ_[#ZMc`9(r;bp.>6'"D[`3<Z:oV!0I3[!p^7krVm-\*#rScs8VaD -rYu)rqZ$T`48SsYb6i:hrrDop*!Cojs8V9\*!;u6s5<hq!NQ(srrT5,PlC[_h#88rTD\`gb5E:c -^]"31WuN+^r;HWsM[$WhrrBe4*!EqQs89%u*!>!rs0)DA!QtE?rrQs=fDbdMT`,o2f_YUJ/,oPO -q/RGpkPY>]9EP%hrrdon*$4bLrrMaLrr3+B*#pR'rr3)nnF.IqrrF>jo)AmkXapFYbPqMBWIagD -"7krt:P&Oss.TIC~> -!ouXLrr3(#.k>-.rrZ"H.r+/F!R+C#rrK-?j8T.9PiMcEB=?k#!CbW#rrFA?k5PO+.k=!arrMm@ -h>[RM.k>Ifs+13Ds*t~> -!ouXLrr3(#.k>-.rrZ"H.r+/F!R+C#rrK-?j8T.9PiMcEB=?k#!CbW#rrFA?k5PO+.k=!arrMm@ -h>[RM.k>Ifs+13Ds*t~> -!ouXLrr3(#.k>-.rrZ"H.r+/F!R+C#rrK-?j8T.9PiMcEB=?k#!CbW#rrFA?k5PO+.k=!arrMm@ -h>[RM.k>Ifs+13Ds*t~> -!ouXLf)GgITn2;)!q2;?jSo;DK<jS<!o8"AjSo;1B&htP!m4UAh#@Dd@Y+Q1s+14Gs*t~> -!ouXLf)GgITn2;)!q2;?jSo;DK<jS<!o8"AjSo;1B&htP!m4UAh#@Dd@Y+Q1s+14Gs*t~> -!ouXLf)GgITn2;)!q2;?jSo;DK<jS<!o8"AjSo;1B&htP!m4UAh#@Dd@Y+Q1s+14Gs*t~> -!ouXLf)GdAK&ck3h3[1%!QhG'rrKLBj8T.Pe`6Z1MsB's!-NkmJcC<$q#>j~> -!ouXLf)GdAK&ck3h3[1%!QhG'rrKLBj8T.Pe`6Z1MsB's!-NkmJcC<$q#>j~> -!ouXLf)GdAK&ck3h3[1%!QhG'rrKLBj8T.Pe`6Z1MsB's!-NkmJcC<$q#>j~> -!ouXLL&V1cdZa\2NkPGmp\oXB_RKOFg\u[&hYmHTaMn&V!Q<*2s-Nb9~> -!ouXLL&V1cdZa\2NkPGmp\oXB_RKOFg\u[&hYmHTaMn&V!Q<*2s-Nb9~> -!ouXLL&V1cdZa\2NkPGmp\oXB_RKOFg\u[&hYmHTaMn&V!Q<*2s-Nb9~> -"lqsOs8S>_@fVJerrJ2Co)A`3GfB[_Fl<9$^4G)<rrUAR]C#V#c!t8,r;Qi,A'k5T!S*h3rrSHq -B[m%_G,G3;",$1/N;ih\O(7qad\H:2"b:Cgs2))8rr[EDb,b79"R[8"SC%98#OmHI]"7qAr;L1+ -Xks'Xj8EHfg%PFR`if?(j8]/>S=DXRrVlrKIEMKf!muA@JcD,;J,~> -"lqsOs8S>_@fVJerrJ2Co)A`3GfB[_Fl<9$^4G)<rrUAR]C#V#c!t8,r;Qi,A'k5T!S*h3rrSHq -B[m%_G,G3;",$1/N;ih\O(7qad\H:2"b:Cgs2))8rr[EDb,b79"R[8"SC%98#OmHI]"7qAr;L1+ -Xks'Xj8EHfg%PFR`if?(j8]/>S=DXRrVlrKIEMKf!muA@JcD,;J,~> -"lqsOs8S>_@fVJerrJ2Co)A`3GfB[_Fl<9$^4G)<rrUAR]C#V#c!t8,r;Qi,A'k5T!S*h3rrSHq -B[m%_G,G3;",$1/N;ih\O(7qad\H:2"b:Cgs2))8rr[EDb,b79"R[8"SC%98#OmHI]"7qAr;L1+ -Xks'Xj8EHfg%PFR`if?(j8]/>S=DXRrVlrKIEMKf!muA@JcD,;J,~> -"lqsOs-ArL\-AC]s8U8Gp@J:b^VS_*!V0skrr=(trrJC?r;Qi]\DZcS"6d6nWVlbthRMhLrrK]? -nc&`YPdn,0hZ!V$j8/cV*rc*;$o@/%6N@(IR(-<![:oR>!FYftrs!qIZ=X'*JbK*G*WN'\pAY0] -0)PYRju37"NrT,`rjVu\[,Crbo3D7(!Ft<errf'Bs%:`;rrHK?r;Qe>W;HSqCpAQkR/_[~> -"lqsOs-ArL\-AC]s8U8Gp@J:b^VS_*!V0skrr=(trrJC?r;Qi]\DZcS"6d6nWVlbthRMhLrrK]? -nc&`YPdn,0hZ!V$j8/cV*rc*;$o@/%6N@(IR(-<![:oR>!FYftrs!qIZ=X'*JbK*G*WN'\pAY0] -0)PYRju37"NrT,`rjVu\[,Crbo3D7(!Ft<errf'Bs%:`;rrHK?r;Qe>W;HSqCpAQkR/_[~> -"lqsOs-ArL\-AC]s8U8Gp@J:b^VS_*!V0skrr=(trrJC?r;Qi]\DZcS"6d6nWVlbthRMhLrrK]? -nc&`YPdn,0hZ!V$j8/cV*rc*;$o@/%6N@(IR(-<![:oR>!FYftrs!qIZ=X'*JbK*G*WN'\pAY0] -0)PYRju37"NrT,`rjVu\[,Crbo3D7(!Ft<errf'Bs%:`;rrHK?r;Qe>W;HSqCpAQkR/_[~> -"lqsOs)#">rrKc?rr3#R6HoH1m4[i+!$1%u!l&a@j8T.$WV6>m\mk40"E(CBpa"_r!CYT;rrHE@ -qu7'\l.SK(s(/\>b:?iZrr3!CqXsjm9[NaJ.rFSFrrX;AW-.h6!U_T;rs.[Es,N->ruM+=!JZs! -rrW,+CB"56hYHpG"JU5B9%*Y;!FNP<rrV@pci!eEk$Q\js-`n;~> -"lqsOs)#">rrKc?rr3#R6HoH1m4[i+!$1%u!l&a@j8T.$WV6>m\mk40"E(CBpa"_r!CYT;rrHE@ -qu7'\l.SK(s(/\>b:?iZrr3!CqXsjm9[NaJ.rFSFrrX;AW-.h6!U_T;rs.[Es,N->ruM+=!JZs! -rrW,+CB"56hYHpG"JU5B9%*Y;!FNP<rrV@pci!eEk$Q\js-`n;~> -"lqsOs)#">rrKc?rr3#R6HoH1m4[i+!$1%u!l&a@j8T.$WV6>m\mk40"E(CBpa"_r!CYT;rrHE@ -qu7'\l.SK(s(/\>b:?iZrr3!CqXsjm9[NaJ.rFSFrrX;AW-.h6!U_T;rs.[Es,N->ruM+=!JZs! -rrW,+CB"56hYHpG"JU5B9%*Y;!FNP<rrV@pci!eEk$Q\js-`n;~> -$KOKTs7Z<hs3js?rVlrY6h^6R!_aRerVlol6N.r*EV]V6!p^A$rr3!gf_uj%J,91/m/R*]LB%9= -)?h*#PkY1Xn,44/S,<4+cMedcU\t/Xr>Z3Fr;JVFV]6\]rVt:D*!NkNs8TB>r;QlD*#=n?rrN)2 -r>Yj_rq??mF/f-<"0W[Z[f6:.\m:*j*!#='rrQ^Air/l^DQa$>o)JQDrYuV$o`(\EnkpG]p&C<n -*5aNjp&G&<rYu2-pA^*l#8eFOrVlm\iVWWkoD\^\s8Vfis8MN[s8R!W"<8Cns8VdbrYuMaq#C!a -s8DH\s8PmA,9.\Iq>UBrnc/I[pAY0]0)PYRrU^'aq#C@7rpg2AQiI(9rVm%8_#OG]>5\C+/,u]" -s'!eLdet-g\,H<I&cNaP^A\&Lr>Z*^rr<#op]'m`r;Qe)^&.g1=2t.;!D^pks-`n;~> -$KOKTs7Z<hs3js?rVlrY6h^6R!_aRerVlol6N.r*EV]V6!p^A$rr3!gf_uj%J,91/m/R*]LB%9= -)?h*#PkY1Xn,44/S,<4+cMedcU\t/Xr>Z3Fr;JVFV]6\]rVt:D*!NkNs8TB>r;QlD*#=n?rrN)2 -r>Yj_rq??mF/f-<"0W[Z[f6:.\m:*j*!#='rrQ^Air/l^DQa$>o)JQDrYuV$o`(\EnkpG]p&C<n -*5aNjp&G&<rYu2-pA^*l#8eFOrVlm\iVWWkoD\^\s8Vfis8MN[s8R!W"<8Cns8VdbrYuMaq#C!a -s8DH\s8PmA,9.\Iq>UBrnc/I[pAY0]0)PYRrU^'aq#C@7rpg2AQiI(9rVm%8_#OG]>5\C+/,u]" -s'!eLdet-g\,H<I&cNaP^A\&Lr>Z*^rr<#op]'m`r;Qe)^&.g1=2t.;!D^pks-`n;~> -$KOKTs7Z<hs3js?rVlrY6h^6R!_aRerVlol6N.r*EV]V6!p^A$rr3!gf_uj%J,91/m/R*]LB%9= -)?h*#PkY1Xn,44/S,<4+cMedcU\t/Xr>Z3Fr;JVFV]6\]rVt:D*!NkNs8TB>r;QlD*#=n?rrN)2 -r>Yj_rq??mF/f-<"0W[Z[f6:.\m:*j*!#='rrQ^Air/l^DQa$>o)JQDrYuV$o`(\EnkpG]p&C<n -*5aNjp&G&<rYu2-pA^*l#8eFOrVlm\iVWWkoD\^\s8Vfis8MN[s8R!W"<8Cns8VdbrYuMaq#C!a -s8DH\s8PmA,9.\Iq>UBrnc/I[pAY0]0)PYRrU^'aq#C@7rpg2AQiI(9rVm%8_#OG]>5\C+/,u]" -s'!eLdet-g\,H<I&cNaP^A\&Lr>Z*^rr<#op]'m`r;Qe)^&.g1=2t.;!D^pks-`n;~> -!ouXLrVloJ8c&Gfd7a04!W"#=rrGd@rr3"fK)YcOm3ulNrrG/,rr37\PQ*B's8S[>rr3#d0D,8E -*W?!@JAM6u:&b.n?bQ@:$Wb:Ii!OT+s%^l4-iO&KA)71bZ>079!K<->rrLn@rVlmLnFlk_F/f': -!Gf"?rrTbCPlC[_*W?!JF=H>Os7+ZGP1KO1s.Fc=rt4`Os8REaaoCN]s,`6?39B$\qu6]J7fNDg -7+hD:!Bf?,rrF>?rVlmtaSu2?ZB"_[#"'X1s8S:?n,EFV0(f/D)ZD/q'pnt#5McA>l>"B>s34C< -rs0\GVfi#8c1WL_rrYdBmP"P="$YT'2uWaW<pTGY!FNP;rrH3@qu6[lc@Q"`s*t~> -!ouXLrVloJ8c&Gfd7a04!W"#=rrGd@rr3"fK)YcOm3ulNrrG/,rr37\PQ*B's8S[>rr3#d0D,8E -*W?!@JAM6u:&b.n?bQ@:$Wb:Ii!OT+s%^l4-iO&KA)71bZ>079!K<->rrLn@rVlmLnFlk_F/f': -!Gf"?rrTbCPlC[_*W?!JF=H>Os7+ZGP1KO1s.Fc=rt4`Os8REaaoCN]s,`6?39B$\qu6]J7fNDg -7+hD:!Bf?,rrF>?rVlmtaSu2?ZB"_[#"'X1s8S:?n,EFV0(f/D)ZD/q'pnt#5McA>l>"B>s34C< -rs0\GVfi#8c1WL_rrYdBmP"P="$YT'2uWaW<pTGY!FNP;rrH3@qu6[lc@Q"`s*t~> -!ouXLrVloJ8c&Gfd7a04!W"#=rrGd@rr3"fK)YcOm3ulNrrG/,rr37\PQ*B's8S[>rr3#d0D,8E -*W?!@JAM6u:&b.n?bQ@:$Wb:Ii!OT+s%^l4-iO&KA)71bZ>079!K<->rrLn@rVlmLnFlk_F/f': -!Gf"?rrTbCPlC[_*W?!JF=H>Os7+ZGP1KO1s.Fc=rt4`Os8REaaoCN]s,`6?39B$\qu6]J7fNDg -7+hD:!Bf?,rrF>?rVlmtaSu2?ZB"_[#"'X1s8S:?n,EFV0(f/D)ZD/q'pnt#5McA>l>"B>s34C< -rs0\GVfi#8c1WL_rrYdBmP"P="$YT'2uWaW<pTGY!FNP;rrH3@qu6[lc@Q"`s*t~> -!ouXLrr3#?<Vl^sqPAT4rrMs?rVlmkdf07IRY(>7!BT6>rs4IFs"Wj>s.4]=rrML?o`"n3qYpO9 -rVlt,^&Rp,\cb7;s56$=rr[`)pfIF*!:9^b!O?J:rrJ(?rr3#S62gfa/F`B5!HY7;rrHo@rr3#1 -@f?<,*VfX@^dT:t7IU9VTR?b='_![Rs*LL?s6a\&OH'9"jki6$s4@7]!M)^ZrrGO?qu6[ZiTpLE -,5V9<!J7$E\cC4RrrGL?rr3"_MXUQGmOn/3!$2(=$&4fHs7.Z>s*^O=rrLA@r;R$Ds/1#>:X/S[ -rVlsif'Y3irrYIAlsB\&!9s+T!FNP;rrH3@qu6[lc@Q"`s*t~> -!ouXLrr3#?<Vl^sqPAT4rrMs?rVlmkdf07IRY(>7!BT6>rs4IFs"Wj>s.4]=rrML?o`"n3qYpO9 -rVlt,^&Rp,\cb7;s56$=rr[`)pfIF*!:9^b!O?J:rrJ(?rr3#S62gfa/F`B5!HY7;rrHo@rr3#1 -@f?<,*VfX@^dT:t7IU9VTR?b='_![Rs*LL?s6a\&OH'9"jki6$s4@7]!M)^ZrrGO?qu6[ZiTpLE -,5V9<!J7$E\cC4RrrGL?rr3"_MXUQGmOn/3!$2(=$&4fHs7.Z>s*^O=rrLA@r;R$Ds/1#>:X/S[ -rVlsif'Y3irrYIAlsB\&!9s+T!FNP;rrH3@qu6[lc@Q"`s*t~> -!ouXLrr3#?<Vl^sqPAT4rrMs?rVlmkdf07IRY(>7!BT6>rs4IFs"Wj>s.4]=rrML?o`"n3qYpO9 -rVlt,^&Rp,\cb7;s56$=rr[`)pfIF*!:9^b!O?J:rrJ(?rr3#S62gfa/F`B5!HY7;rrHo@rr3#1 -@f?<,*VfX@^dT:t7IU9VTR?b='_![Rs*LL?s6a\&OH'9"jki6$s4@7]!M)^ZrrGO?qu6[ZiTpLE -,5V9<!J7$E\cC4RrrGL?rr3"_MXUQGmOn/3!$2(=$&4fHs7.Z>s*^O=rrLA@r;R$Ds/1#>:X/S[ -rVlsif'Y3irrYIAlsB\&!9s+T!FNP;rrH3@qu6[lc@Q"`s*t~> -#NS0Qs8T`Ko(r@eT76G4!W"#=rrGd@rr3"fK)#?H3o^/=#`4cF1$el>Spp\=!UVQ4rr=)9rr=)< -rrZWAs-rsq"<kebi&po:!9a;h"c`$Os0?D9rrJ(?rr3#S62gfa/F`B5!HY7;rrHo@rr3#1@f?<, -*VTL6DQ`s<!MFi>rrqmCs8RG?r;R(bNW2!"h`_"eHN%=R;+CQb!CYT;rrG7@li-u0qY^?nKDo9[ -GKfj^!CPQ>rrJ=@n,EFV0(f/D*WH'FL6qr?pM7=mJ&M?d!R4I=rs0\GVfi#8c1WL_rrYdBmP"P= -"$YT7OoAbhbk(i8?bQ@:!E[;<rrGm?JcD/<J,~> -#NS0Qs8T`Ko(r@eT76G4!W"#=rrGd@rr3"fK)#?H3o^/=#`4cF1$el>Spp\=!UVQ4rr=)9rr=)< -rrZWAs-rsq"<kebi&po:!9a;h"c`$Os0?D9rrJ(?rr3#S62gfa/F`B5!HY7;rrHo@rr3#1@f?<, -*VTL6DQ`s<!MFi>rrqmCs8RG?r;R(bNW2!"h`_"eHN%=R;+CQb!CYT;rrG7@li-u0qY^?nKDo9[ -GKfj^!CPQ>rrJ=@n,EFV0(f/D*WH'FL6qr?pM7=mJ&M?d!R4I=rs0\GVfi#8c1WL_rrYdBmP"P= -"$YT7OoAbhbk(i8?bQ@:!E[;<rrGm?JcD/<J,~> -#NS0Qs8T`Ko(r@eT76G4!W"#=rrGd@rr3"fK)#?H3o^/=#`4cF1$el>Spp\=!UVQ4rr=)9rr=)< -rrZWAs-rsq"<kebi&po:!9a;h"c`$Os0?D9rrJ(?rr3#S62gfa/F`B5!HY7;rrHo@rr3#1@f?<, -*VTL6DQ`s<!MFi>rrqmCs8RG?r;R(bNW2!"h`_"eHN%=R;+CQb!CYT;rrG7@li-u0qY^?nKDo9[ -GKfj^!CPQ>rrJ=@n,EFV0(f/D*WH'FL6qr?pM7=mJ&M?d!R4I=rs0\GVfi#8c1WL_rrYdBmP"P= -"$YT7OoAbhbk(i8?bQ@:!E[;<rrGm?JcD/<J,~> -#38'Ps0mL>qYpS^L@bEGqBGn<!DCl?rrJR?rr3&lJuSbL!BP?%rs4IFs"Wj>s.4]=rrqdCs8Td\ -q#:=7rVlu-g]&;mrrZWAruM+="hQ>*s56$;rrMt]rVm'jfDkm'J&M6a!K<->rrLn@rVlmLnFlk_ -F/f':!Gf"?rrTbe_>aH7*W?!>Y(H0ArrI#?rVlnkI/a-Ln1=V>H_UH<%Zl\M39B$\s+cm>s7pM$ -rr3*hf`1-5rr3%hJc>$9#9s$Ef"(g]jSo/[Uj;b8!CPQ>rrJ=@pAY3]K!G:S!U_T4rr=)=rrIk@ -rr3)N8",&.rt!@Ns1?e\ruV3<FoP7^p*Tb<"%Ur10E(nQ52PB[rVlngoC`+a?bQC;!kWsAr;Qi- -J*-\/!jdLEJcDGDJ,~> -#38'Ps0mL>qYpS^L@bEGqBGn<!DCl?rrJR?rr3&lJuSbL!BP?%rs4IFs"Wj>s.4]=rrqdCs8Td\ -q#:=7rVlu-g]&;mrrZWAruM+="hQ>*s56$;rrMt]rVm'jfDkm'J&M6a!K<->rrLn@rVlmLnFlk_ -F/f':!Gf"?rrTbe_>aH7*W?!>Y(H0ArrI#?rVlnkI/a-Ln1=V>H_UH<%Zl\M39B$\s+cm>s7pM$ -rr3*hf`1-5rr3%hJc>$9#9s$Ef"(g]jSo/[Uj;b8!CPQ>rrJ=@pAY3]K!G:S!U_T4rr=)=rrIk@ -rr3)N8",&.rt!@Ns1?e\ruV3<FoP7^p*Tb<"%Ur10E(nQ52PB[rVlngoC`+a?bQC;!kWsAr;Qi- -J*-\/!jdLEJcDGDJ,~> -#38'Ps0mL>qYpS^L@bEGqBGn<!DCl?rrJR?rr3&lJuSbL!BP?%rs4IFs"Wj>s.4]=rrqdCs8Td\ -q#:=7rVlu-g]&;mrrZWAruM+="hQ>*s56$;rrMt]rVm'jfDkm'J&M6a!K<->rrLn@rVlmLnFlk_ -F/f':!Gf"?rrTbe_>aH7*W?!>Y(H0ArrI#?rVlnkI/a-Ln1=V>H_UH<%Zl\M39B$\s+cm>s7pM$ -rr3*hf`1-5rr3%hJc>$9#9s$Ef"(g]jSo/[Uj;b8!CPQ>rrJ=@pAY3]K!G:S!U_T4rr=)=rrIk@ -rr3)N8",&.rt!@Ns1?e\ruV3<FoP7^p*Tb<"%Ur10E(nQ52PB[rVlngoC`+a?bQC;!kWsAr;Qi- -J*-\/!jdLEJcDGDJ,~> -"lqsOs(t7(Ad+k-s,iH?o`#2OA.E4Bs8TQHral1Wc2R_Ekl=QZrrG0:ralakeGlXcAnH==_uKc3 -OCi*Ug\CdKUA\[+K]2qOrVlt,^&OGuAd*cC`rFsZr;QfpOo8kmMspZF!L8H=rs7Fm3'([/s7)'H -Ac\"hq#:rOAhHJ`n,NF0Ah-A`o`)KaJ,X$[TDeck=odLZrrI#?rVloOJGs-aBrh:6H_UH<%Zl\M -YOp^8s3H%(AnZ`brr3,.DJ!jtrVlm>rp9XiVe9Uc`rEYjral1UanYl:6eVJ=!L&E7rrUmS@/^*+ -mOn/3!2BI)!IiSurr_\IHHlEg&*o'aK&$D+P@d08Ar5jNOo8koMsgA%rFQ.ko(A%AAc[qbp&>&" -^&7m2FK#*:!Go%<rrQ[1eq*jps*t~> -"lqsOs(t7(Ad+k-s,iH?o`#2OA.E4Bs8TQHral1Wc2R_Ekl=QZrrG0:ralakeGlXcAnH==_uKc3 -OCi*Ug\CdKUA\[+K]2qOrVlt,^&OGuAd*cC`rFsZr;QfpOo8kmMspZF!L8H=rs7Fm3'([/s7)'H -Ac\"hq#:rOAhHJ`n,NF0Ah-A`o`)KaJ,X$[TDeck=odLZrrI#?rVloOJGs-aBrh:6H_UH<%Zl\M -YOp^8s3H%(AnZ`brr3,.DJ!jtrVlm>rp9XiVe9Uc`rEYjral1UanYl:6eVJ=!L&E7rrUmS@/^*+ -mOn/3!2BI)!IiSurr_\IHHlEg&*o'aK&$D+P@d08Ar5jNOo8koMsgA%rFQ.ko(A%AAc[qbp&>&" -^&7m2FK#*:!Go%<rrQ[1eq*jps*t~> -"lqsOs(t7(Ad+k-s,iH?o`#2OA.E4Bs8TQHral1Wc2R_Ekl=QZrrG0:ralakeGlXcAnH==_uKc3 -OCi*Ug\CdKUA\[+K]2qOrVlt,^&OGuAd*cC`rFsZr;QfpOo8kmMspZF!L8H=rs7Fm3'([/s7)'H -Ac\"hq#:rOAhHJ`n,NF0Ah-A`o`)KaJ,X$[TDeck=odLZrrI#?rVloOJGs-aBrh:6H_UH<%Zl\M -YOp^8s3H%(AnZ`brr3,.DJ!jtrVlm>rp9XiVe9Uc`rEYjral1UanYl:6eVJ=!L&E7rrUmS@/^*+ -mOn/3!2BI)!IiSurr_\IHHlEg&*o'aK&$D+P@d08Ar5jNOo8koMsgA%rFQ.ko(A%AAc[qbp&>&" -^&7m2FK#*:!Go%<rrQ[1eq*jps*t~> -"QVjNs4.%T"O[8Lb4G6)!6+rF!7:`F!5e`C!7LiG!r97Jrr3![ir6=cfDbdR]=#&og>`,3"nB". -h#DQp8cls2rVa,+iW&qlrVluIm/Qn\[0>C3mf3"#qu6ZgrNuXkrVlrWI)#[\!:Tlo"7Z?jm/=<m -o(DiO!rDr[rNub%s8V3Z[06@+ldFMd[/g.'rrUNSpAP!le`Zl1!8IL\!4i+/!SQQ3rrM$6rr3;u -a8c1h[C*O9ao25@^pV)XrrKi?li.$p[E\^N!6+rF!7:K?!Qk!5rrM'6pAY3dL1'u["RZ^k8u_Rb -!<2u*!8.5L!UA,1rrVDlh>[E[p9f'B[HRYjo_l0"li7"TrNuaps8VQd[/f[irrgP<8fYPCrrTHY -k5>5\X,-!:rrUWVo7?q8s*t~> -"QVjNs4.%T"O[8Lb4G6)!6+rF!7:`F!5e`C!7LiG!r97Jrr3![ir6=cfDbdR]=#&og>`,3"nB". -h#DQp8cls2rVa,+iW&qlrVluIm/Qn\[0>C3mf3"#qu6ZgrNuXkrVlrWI)#[\!:Tlo"7Z?jm/=<m -o(DiO!rDr[rNub%s8V3Z[06@+ldFMd[/g.'rrUNSpAP!le`Zl1!8IL\!4i+/!SQQ3rrM$6rr3;u -a8c1h[C*O9ao25@^pV)XrrKi?li.$p[E\^N!6+rF!7:K?!Qk!5rrM'6pAY3dL1'u["RZ^k8u_Rb -!<2u*!8.5L!UA,1rrVDlh>[E[p9f'B[HRYjo_l0"li7"TrNuaps8VQd[/f[irrgP<8fYPCrrTHY -k5>5\X,-!:rrUWVo7?q8s*t~> -"QVjNs4.%T"O[8Lb4G6)!6+rF!7:`F!5e`C!7LiG!r97Jrr3![ir6=cfDbdR]=#&og>`,3"nB". -h#DQp8cls2rVa,+iW&qlrVluIm/Qn\[0>C3mf3"#qu6ZgrNuXkrVlrWI)#[\!:Tlo"7Z?jm/=<m -o(DiO!rDr[rNub%s8V3Z[06@+ldFMd[/g.'rrUNSpAP!le`Zl1!8IL\!4i+/!SQQ3rrM$6rr3;u -a8c1h[C*O9ao25@^pV)XrrKi?li.$p[E\^N!6+rF!7:K?!Qk!5rrM'6pAY3dL1'u["RZ^k8u_Rb -!<2u*!8.5L!UA,1rrVDlh>[E[p9f'B[HRYjo_l0"li7"TrNuaps8VQd[/f[irrgP<8fYPCrrTHY -k5>5\X,-!:rrUWVo7?q8s*t~> -!ouXLd/O,-ip?[Fh##J!nC@O>kO7p?!;u]@!<0,#!7o'f!PDh=rrDurd/`FerrK$?h>[KGrm:`; -rVloalMLS^lKj*%QiDR~> -!ouXLd/O,-ip?[Fh##J!nC@O>kO7p?!;u]@!<0,#!7o'f!PDh=rrDurd/`FerrK$?h>[KGrm:`; -rVloalMLS^lKj*%QiDR~> -!ouXLd/O,-ip?[Fh##J!nC@O>kO7p?!;u]@!<0,#!7o'f!PDh=rrDurd/`FerrK$?h>[KGrm:`; -rVloalMLS^lKj*%QiDR~> -!ouXLd/O,5jFR>grrV%Uo[NmAWdB<sJcF[.J,~> -!ouXLd/O,5jFR>grrV%Uo[NmAWdB<sJcF[.J,~> -!ouXLd/O,5jFR>grrV%Uo[NmAWdB<sJcF[.J,~> -!ouXLJcC<$JcC<$K`?Q~> -!ouXLJcC<$JcC<$K`?Q~> -!ouXLJcC<$JcC<$K`?Q~> -#in9Rs8UK[]uSt/!65#O!82r'JcC<$JcE(VJ,~> -#in9Rs8UK[]uSt/!65#O!82r'JcC<$JcE(VJ,~> -#in9Rs8UK[]uSt/!65#O!82r'JcC<$JcE(VJ,~> -$04BSs2f=g?>-n,rrKZDra#VPe:IXNs+13$s0VfV~> -$04BSs2f=g?>-n,rrKZDra#VPe:IXNs+13$s0VfV~> -$04BSs2f=g?>-n,rrKZDra#VPe:IXNs+13$s0VfV~> -$04BSs)#"?s4Kcsrs#$Hqu?]9_L_`<s+13$s0VfV~> -$04BSs)#"?s4Kcsrs#$Hqu?]9_L_`<s+13$s0VfV~> -$04BSs)#"?s4Kcsrs#$Hqu?]9_L_`<s+13$s0VfV~> -%HKfWs)#"?s4Kd>s.n3ErrLQFrr3"HrVca&Rc"$ks2_GEs+C;b!2TVo"=7VnBrV+3!."QX!3UnQ -JcC<$JcE@^J,~> -%HKfWs)#"?s4Kd>s.n3ErrLQFrr3"HrVca&Rc"$ks2_GEs+C;b!2TVo"=7VnBrV+3!."QX!3UnQ -JcC<$JcE@^J,~> -%HKfWs)#"?s4Kd>s.n3ErrLQFrr3"HrVca&Rc"$ks2_GEs+C;b!2TVo"=7VnBrV+3!."QX!3UnQ -JcC<$JcE@^J,~> -%HKfWs(sVe9L2&Gs(&Y=rrKK@rr3!Bqu-O$;p"k[s.ao?JY<"K"H'/XH_gYI"Ga/[Ff55F!J@_0 -s+13$s+13_s*t~> -%HKfWs(sVe9L2&Gs(&Y=rrKK@rr3!Bqu-O$;p"k[s.ao?JY<"K"H'/XH_gYI"Ga/[Ff55F!J@_0 -s+13$s+13_s*t~> -%HKfWs(sVe9L2&Gs(&Y=rrKK@rr3!Bqu-O$;p"k[s.ao?JY<"K"H'/XH_gYI"Ga/[Ff55F!J@_0 -s+13$s+13_s*t~> -'')>\s)!eBc`fe:s5I;>s4Ui>rr3!Bqu-O$;p"k[s.ao?9@Eh>"IarB6JDG="Hn]C3o^/=!U8m# -s+13$s+13_s*t~> -'')>\s)!eBc`fe:s5I;>s4Ui>rr3!Bqu-O$;p"k[s.ao?9@Eh>"IarB6JDG="Hn]C3o^/=!U8m# -s+13$s+13_s*t~> -'')>\s)!eBc`fe:s5I;>s4Ui>rr3!Bqu-O$;p"k[s.ao?9@Eh>"IarB6JDG="Hn]C3o^/=!U8m# -s+13$s+13_s*t~> -$04BSs)#"?s4Kd=rrc$Brle7<rrF;?rVm1&and4]UO)r5df07LRY(Q+gA_*WP)KA)1)q9LJcC<$ -JcC<$])R9~> -$04BSs)#"?s4Kd=rrc$Brle7<rrF;?rVm1&and4]UO)r5df07LRY(Q+gA_*WP)KA)1)q9LJcC<$ -JcC<$])R9~> -$04BSs)#"?s4Kd=rrc$Brle7<rrF;?rVm1&and4]UO)r5df07LRY(Q+gA_*WP)KA)1)q9LJcC<$ -JcC<$])R9~> -$04BSs)#"?s4Kd<rrO\00)Y_[+oD#ss8Q$>qBGs7HN-Xdrr3+iK)\0krr3+aMuPitrr3#fn:CUj -s+13$s1JA^~> -$04BSs)#"?s4Kd<rrO\00)Y_[+oD#ss8Q$>qBGs7HN-Xdrr3+iK)\0krr3+aMuPitrr3#fn:CUj -s+13$s1JA^~> -$04BSs)#"?s4Kd<rrO\00)Y_[+oD#ss8Q$>qBGs7HN-Xdrr3+iK)\0krr3+aMuPitrr3#fn:CUj -s+13$s1JA^~> -$04BSs,i\`^T;JSrrQQ4@f66:=hUV\s8RPD]G\JHS,[`4rkASqU&T,7rkASmW;gY<rkAJeY(?V( -s+13$s1JA^~> -$04BSs,i\`^T;JSrrQQ4@f66:=hUV\s8RPD]G\JHS,[`4rkASqU&T,7rkASmW;gY<rkAJeY(?V( -s+13$s1JA^~> -$04BSs,i\`^T;JSrrQQ4@f66:=hUV\s8RPD]G\JHS,[`4rkASqU&T,7rkASmW;gY<rkAJeY(?V( -s+13$s1JA^~> -#in9Rs8S-b>]flC!,q`6!FC9SrrddS$s].urr@?D>QC;nrr@0?>QCMtrr?s9>QC[Qs+13$s+13^ -s*t~> -#in9Rs8S-b>]flC!,q`6!FC9SrrddS$s].urr@?D>QC;nrr@0?>QCMtrr?s9>QC[Qs+13$s+13^ -s*t~> -#in9Rs8S-b>]flC!,q`6!FC9SrrddS$s].urr@?D>QC;nrr@0?>QCMtrr?s9>QC[Qs+13$s+13^ -s*t~> -!ouXLk5PJY,(]cFs+13$s.TIC~> -!ouXLk5PJY,(]cFs+13$s.TIC~> -!ouXLk5PJY,(]cFs+13$s.TIC~> -!ouXLk5PJY,(]cFs+13$s.TIC~> -!ouXLk5PJY,(]cFs+13$s.TIC~> -!ouXLk5PJY,(]cFs+13$s.TIC~> -!ouXLk5PJ]\q0m4s+13$s.TIC~> -!ouXLk5PJ]\q0m4s+13$s.TIC~> -!ouXLk5PJ]\q0m4s+13$s.TIC~> -!ouXLJcC<$JcC<$K`?Q~> -!ouXLJcC<$JcC<$K`?Q~> -!ouXLJcC<$JcC<$K`?Q~> -!ouXLq#:A8^4H<8s+13$s,[21~> -!ouXLq#:A8^4H<8s+13$s,[21~> -!ouXLq#:A8^4H<8s+13$s,[21~> -!ouXLq#:A*\:O[2s+13$s,[21~> -!ouXLq#:A*\:O[2s+13$s,[21~> -!ouXLq#:A*\:O[2s+13$s,[21~> -"lqsOs8UXK_?A<es(&Y=rrMNAJcC<$JcC<$PlH7~> -"lqsOs8UXK_?A<es(&Y=rrMNAJcC<$JcC<$PlH7~> -"lqsOs8UXK_?A<es(&Y=rrMNAJcC<$JcC<$PlH7~> -"lqsOs3,K^=UArcs(&Y>q504DJcC<$JcC<$PlH7~> -"lqsOs3,K^=UArcs(&Y>q504DJcC<$JcC<$PlH7~> -"lqsOs3,K^=UArcs(&Y>q504DJcC<$JcC<$PlH7~> -"lqsOs)#">rs68Fs(&X1Lio;?s+13$s+136s*t~> -"lqsOs)#">rs68Fs(&X1Lio;?s+13$s+136s*t~> -"lqsOs)#">rs68Fs(&X1Lio;?s+13$s+136s*t~> -"lqsOs)#">rs$,Ds("oaR=YBhs+13$s,m>3~> -"lqsOs)#">rs$,Ds("oaR=YBhs+13$s,m>3~> -"lqsOs)#">rs$,Ds("oaR=YBhs+13$s,m>3~> -"lqsOs)#">rs$,Ds($Od?%N$,s+13$s,m>3~> -"lqsOs)#">rs$,Ds($Od?%N$,s+13$s,m>3~> -"lqsOs)#">rs$,Ds($Od?%N$,s+13$s,m>3~> -"lqsOs)#">rs68Fs(&Xf^c$1`s+13$s+136s*t~> -"lqsOs)#">rs68Fs(&Xf^c$1`s+13$s+136s*t~> -"lqsOs)#">rs68Fs(&Xf^c$1`s+13$s+136s*t~> -"lqsOs6s7l*!oL2s(8hArT1&)JcC<$JcC<$PlH7~> -"lqsOs6s7l*!oL2s(8hArT1&)JcC<$JcC<$PlH7~> -"lqsOs6s7l*!oL2s(8hArT1&)JcC<$JcC<$PlH7~> -!ouXLJcC<$JcC<$K`?Q~> -!ouXLJcC<$JcC<$K`?Q~> -!ouXLJcC<$JcC<$K`?Q~> -!ouXLJcC<$JcC<$K`?Q~> -!ouXLJcC<$JcC<$K`?Q~> -!ouXLJcC<$JcC<$K`?Q~> -!ouXLJcC<$JcC<$K`?Q~> -!ouXLJcC<$JcC<$K`?Q~> -!ouXLJcC<$JcC<$K`?Q~> -!ouXLJcC<$JcC<$K`?Q~> -!ouXLJcC<$JcC<$K`?Q~> -!ouXLJcC<$JcC<$K`?Q~> -"QVjNs/l3<"K)5!R-9,$JcC<$JcCi3J,~> -"QVjNs/l3<"K)5!R-9,$JcC<$JcCi3J,~> -"QVjNs/l3<"K)5!R-9,$JcC<$JcCi3J,~> -"QVjNs3LYE#(Bt[s2&7>JcC<$JcC<$OoKq~> -"QVjNs3LYE#(Bt[s2&7>JcC<$JcC<$OoKq~> -"QVjNs3LYE#(Bt[s2&7>JcC<$JcC<$OoKq~> -!ouXLr;QiX?JPP[!ROMks+13$s+134s*t~> -!ouXLr;QiX?JPP[!ROMks+13$s+134s*t~> -!ouXLr;QiX?JPP[!ROMks+13$s+134s*t~> -!ouXLrVlrn3U6PB!S9bks+13$s+135s*t~> -!ouXLrVlrn3U6PB!S9bks+13$s+135s*t~> -!ouXLrVlrn3U6PB!S9bks+13$s+135s*t~> -%%EndData -showpage -%%Trailer -end -%%EOF diff --git a/lib/stdlib/doc/src/zip.xml b/lib/stdlib/doc/src/zip.xml index cf0d581352..61f49f5940 100644 --- a/lib/stdlib/doc/src/zip.xml +++ b/lib/stdlib/doc/src/zip.xml @@ -217,7 +217,7 @@ <tag><c>{uncompress, <anno>What</anno>}</c></tag> <item> <p>Controls what types of files will be uncompressed. It is by - default set to <c>[".Z",".zip",".zoo",".arc",".lzh",".arj"]</c>. + default set to <c>[".Z", ".zip", ".zoo", ".arc", ".lzh", ".arj"]</c>. The following values of <c>What</c> are allowed:</p> <taglist> <tag><c>all</c></tag> @@ -355,7 +355,7 @@ {ok,{"dummy.zip", <<80,75,3,4,20,0,0,0,0,0,74,152,97,60,171,39,212,26,3,0, 0,0,3,0,0,...>>}} -> <input>catch zip:foldl(fun("foo", _, B, _) -> throw(B()); (_, _, _, Acc) -> Acc end, [], {Name, Bin}). </input> +> <input>catch zip:foldl(fun("foo", _, B, _) -> throw(B()); (_,_,_,Acc) -> Acc end, [], {Name, Bin}). </input> <<"FOO">> </pre> </desc> diff --git a/lib/stdlib/src/Makefile b/lib/stdlib/src/Makefile index 14304824d3..575a5cbe4a 100644 --- a/lib/stdlib/src/Makefile +++ b/lib/stdlib/src/Makefile @@ -171,6 +171,7 @@ primary_bootstrap_compiler: \ $(BOOTSTRAP_COMPILER)/ebin/erl_scan.beam \ $(BOOTSTRAP_COMPILER)/ebin/erl_parse.beam \ $(BOOTSTRAP_COMPILER)/ebin/erl_lint.beam \ + $(BOOTSTRAP_COMPILER)/ebin/io.beam \ $(BOOTSTRAP_COMPILER)/ebin/otp_internal.beam $(BOOTSTRAP_COMPILER)/ebin/erl_parse.beam: erl_parse.yrl diff --git a/lib/stdlib/src/c.erl b/lib/stdlib/src/c.erl index a920921a5e..4c1c0f904b 100644 --- a/lib/stdlib/src/c.erl +++ b/lib/stdlib/src/c.erl @@ -116,7 +116,7 @@ machine_load(Mod, File, Opts) -> File2 = filename:join(Dir, filename:basename(File, ".erl")), case compile:output_generated(Opts) of true -> - Base = packages:last(Mod), + Base = atom_to_list(Mod), case filename:basename(File, ".erl") of Base -> code:purge(Mod), diff --git a/lib/stdlib/src/dict.erl b/lib/stdlib/src/dict.erl index 2e9eba4bfa..4f8d45dc8d 100644 --- a/lib/stdlib/src/dict.erl +++ b/lib/stdlib/src/dict.erl @@ -1,7 +1,8 @@ +%% -*- coding: utf-8 -*- %% %% %CopyrightBegin% %% -%% Copyright Ericsson AB 2000-2011. All Rights Reserved. +%% Copyright Ericsson AB 2000-2012. All Rights Reserved. %% %% The contents of this file are subject to the Erlang Public License, %% Version 1.1, (the "License"); you may not use this file except in @@ -17,7 +18,7 @@ %% %CopyrightEnd% %% -%% We use the dynamic hashing techniques by Per-�ke Larsson as +%% We use the dynamic hashing techniques by Per-Åke Larsson as %% described in "The Design and Implementation of Dynamic Hashing for %% Sets and Tables in Icon" by Griswold and Townsend. Much of the %% terminology comes from that paper as well. diff --git a/lib/stdlib/src/edlin.erl b/lib/stdlib/src/edlin.erl index 026bd9038f..1164ee49eb 100644 --- a/lib/stdlib/src/edlin.erl +++ b/lib/stdlib/src/edlin.erl @@ -1,7 +1,8 @@ +%% -*- coding: utf-8 -*- %% %% %CopyrightBegin% %% -%% Copyright Ericsson AB 1996-2010. All Rights Reserved. +%% Copyright Ericsson AB 1996-2012. All Rights Reserved. %% %% The contents of this file are subject to the Erlang Public License, %% Version 1.1, (the "License"); you may not use this file except in @@ -317,9 +318,9 @@ over_non_word([], Stack, N) -> {[],Stack,N}. word_char(C) when C >= $A, C =< $Z -> true; -word_char(C) when C >= $�, C =< $�, C =/= $� -> true; +word_char(C) when C >= $À, C =< $Þ, C =/= $× -> true; word_char(C) when C >= $a, C =< $z -> true; -word_char(C) when C >= $�, C =< $�, C =/= $� -> true; +word_char(C) when C >= $ß, C =< $ÿ, C =/= $÷ -> true; word_char(C) when C >= $0, C =< $9 -> true; word_char(C) when C =:= $_ -> true; word_char(C) when C =:= $. -> true; % accept dot-separated names diff --git a/lib/stdlib/src/epp.erl b/lib/stdlib/src/epp.erl index 2c8d84a9e1..a0f7660ecf 100644 --- a/lib/stdlib/src/epp.erl +++ b/lib/stdlib/src/epp.erl @@ -23,13 +23,18 @@ -export([open/2,open/3,open/5,close/1,format_error/1]). -export([scan_erl_form/1,parse_erl_form/1,macro_defs/1]). -export([parse_file/1, parse_file/3]). +-export([default_encoding/0, encoding_to_string/1, + read_encoding/1, read_encoding/2, set_encoding/1]). -export([interpret_file_attribute/1]). -export([normalize_typed_record_fields/1,restore_typed_record_fields/1]). %%------------------------------------------------------------------------ +-export_type([source_encoding/0]). + -type macros() :: [{atom(), term()}]. -type epp_handle() :: pid(). +-type source_encoding() :: latin1 | utf8. %% Epp state record. -record(epp, {file, %Current file @@ -213,6 +218,173 @@ parse_file(Epp) -> [{eof,Location}] end. +-define(DEFAULT_ENCODING, latin1). + +-spec default_encoding() -> source_encoding(). + +default_encoding() -> + ?DEFAULT_ENCODING. + +-spec encoding_to_string(Encoding) -> string() when + Encoding :: source_encoding(). + +encoding_to_string(latin1) -> "coding: latin-1"; +encoding_to_string(utf8) -> "coding: utf-8". + +-spec read_encoding(FileName) -> source_encoding() | none when + FileName :: file:name(). + +read_encoding(Name) -> + read_encoding(Name, []). + +-spec read_encoding(FileName, Options) -> source_encoding() | none when + FileName :: file:name(), + Options :: [Option], + Option :: {in_comment_only, boolean()}. + +read_encoding(Name, Options) -> + InComment = proplists:get_value(in_comment_only, Options, true), + case file:open(Name, [read]) of + {ok,File} -> + try read_encoding_from_file(File, InComment) + after ok = file:close(File) + end; + _Error -> + none + end. + +-spec set_encoding(File) -> source_encoding() | none when + File :: io:device(). % pid(); raw files don't work + +set_encoding(File) -> + Encoding = read_encoding_from_file(File, true), + Enc = case Encoding of + none -> default_encoding(); + Encoding -> Encoding + end, + ok = io:setopts(File, [{encoding, Enc}]), + Encoding. + +-spec read_encoding_from_file(File, InComment) -> source_encoding() | none when + File :: io:device(), + InComment :: boolean(). + +-define(ENC_CHUNK, 32). +-define(N_ENC_CHUNK, 16). % a total of 512 bytes + +read_encoding_from_file(File, InComment) -> + {ok, Pos0} = file:position(File, cur), + Opts = io:getopts(File), + Encoding0 = lists:keyfind(encoding, 1, Opts), + Binary0 = lists:keyfind(binary, 1, Opts), + ok = io:setopts(File, [binary, {encoding, latin1}]), + try + {B, Fun} = (reader(File, 0))(), + com_nl(B, Fun, 0, InComment) + catch + throw:no -> + none + after + {ok, Pos0} = file:position(File, Pos0), + ok = io:setopts(File, [Binary0, Encoding0]) + end. + +reader(Fd, N) -> + fun() when N =:= ?N_ENC_CHUNK -> + throw(no); + () -> + case file:read(Fd, ?ENC_CHUNK) of + eof -> + {<<>>, reader(Fd, N+1)}; + {ok, Bin} -> + {Bin, reader(Fd, N+1)}; + {error, _} -> + throw(no) % ignore errors + end + end. + +com_nl(_, _, 2, _) -> + throw(no); +com_nl(B, Fun, N, false=Com) -> + com_c(B, Fun, N, Com); +com_nl(B, Fun, N, true=Com) -> + com(B, Fun, N, Com). + +com(<<"\n",B/binary>>, Fun, N, Com) -> + com_nl(B, Fun, N+1, Com); +com(<<"%", B/binary>>, Fun, N, Com) -> + com_c(B, Fun, N, Com); +com(<<_:1/unit:8,B/binary>>, Fun, N, Com) -> + com(B, Fun, N, Com); +com(<<>>, Fun, N, Com) -> + {B, Fun1} = Fun(), + com(B, Fun1, N, Com). + +com_c(<<"c",B/binary>>, Fun, N, Com) -> + com_oding(B, Fun, N, Com); +com_c(<<"\n",B/binary>>, Fun, N, Com) -> + com_nl(B, Fun, N+1, Com); +com_c(<<_:1/unit:8,B/binary>>, Fun, N, Com) -> + com_c(B, Fun, N, Com); +com_c(<<>>, Fun, N, Com) -> + {B, Fun1} = Fun(), + com_c(B, Fun1, N, Com). + +com_oding(<<"oding",B/binary>>, Fun, N, Com) -> + com_sep(B, Fun, N, Com); +com_oding(B, Fun, N, Com) when byte_size(B) >= length("oding") -> + com_c(B, Fun, N, Com); +com_oding(B, Fun, N, Com) -> + {B1, Fun1} = Fun(), + com_oding(list_to_binary([B, B1]), Fun1, N, Com). + +com_sep(<<":",B/binary>>, Fun, N, Com) -> + com_space(B, Fun, N, Com); +com_sep(<<"=",B/binary>>, Fun, N, Com) -> + com_space(B, Fun, N, Com); +com_sep(<<"\s",B/binary>>, Fun, N, Com) -> + com_sep(B, Fun, N, Com); +com_sep(<<>>, Fun, N, Com) -> + {B, Fun1} = Fun(), + com_sep(B, Fun1, N, Com); +com_sep(B, Fun, N, Com) -> + com_c(B, Fun, N, Com). + +com_space(<<"\s",B/binary>>, Fun, N, Com) -> + com_space(B, Fun, N, Com); +com_space(<<>>, Fun, N, Com) -> + {B, Fun1} = Fun(), + com_space(B, Fun1, N, Com); +com_space(B, Fun, N, _Com) -> + com_enc(B, Fun, N, [], []). + +com_enc(<<C:1/unit:8,B/binary>>, Fun, N, L, Ps) when C >= $a, C =< $z; + C >= $A, C =< $Z; + C >= $0, C =< $9 -> + com_enc(B, Fun, N, [C | L], Ps); +com_enc(<<>>, Fun, N, L, Ps) -> + case Fun() of + {<<>>, _} -> + com_enc_end([L | Ps]); + {B, Fun1} -> + com_enc(B, Fun1, N, L, Ps) + end; +com_enc(<<"-",B/binary>>, Fun, N, L, Ps) -> + com_enc(B, Fun, N, [], [L | Ps]); +com_enc(_B, _Fun, _N, L, Ps) -> + com_enc_end([L | Ps]). + +com_enc_end(Ps0) -> + Ps = lists:reverse([lists:reverse(string:to_lower(P)) || P <- Ps0]), + com_encoding(Ps). + +com_encoding(["latin","1"|_]) -> + latin1; +com_encoding(["utf","8"|_]) -> + utf8; +com_encoding(_) -> + throw(no). % Don't try any further + normalize_typed_record_fields([]) -> {typed, []}; normalize_typed_record_fields(Fields) -> @@ -266,14 +438,17 @@ init_server(Pid, Name, File, AtLocation, Path, Pdm, Pre) -> Ms0 = predef_macros(Name), case user_predef(Pdm, Ms0) of {ok,Ms1} -> - epp_reply(Pid, {ok,self()}), - %% ensure directory of current source file is first in path + _ = set_encoding(File), + epp_reply(Pid, {ok,self()}), + %% ensure directory of current source file is + %% first in path Path1 = [filename:dirname(Name) | Path], - St = #epp{file=File, location=AtLocation, delta=0, name=Name, - name2=Name, path=Path1, macs=Ms1, pre_opened = Pre}, - From = wait_request(St), - enter_file_reply(From, Name, AtLocation, AtLocation), - wait_req_scan(St); + St = #epp{file=File, location=AtLocation, delta=0, + name=Name, name2=Name, path=Path1, macs=Ms1, + pre_opened = Pre}, + From = wait_request(St), + enter_file_reply(From, Name, AtLocation, AtLocation), + wait_req_scan(St); {error,E} -> epp_reply(Pid, {error,E}) end. @@ -385,19 +560,20 @@ enter_file(NewName, Inc, From, St) -> %% enter_file2(File, FullName, From, EppState, AtLocation) -> EppState. %% Set epp to use this file and "enter" it. -enter_file2(NewF, Pname, From, St, AtLocation) -> +enter_file2(NewF, Pname, From, St0, AtLocation) -> Loc = start_loc(AtLocation), enter_file_reply(From, Pname, Loc, AtLocation), - Ms = dict:store({atom,'FILE'}, {none,[{string,Loc,Pname}]}, St#epp.macs), + Ms = dict:store({atom,'FILE'}, {none,[{string,Loc,Pname}]}, St0#epp.macs), %% update the head of the include path to be the directory of the new %% source file, so that an included file can always include other files %% relative to its current location (this is also how C does it); note %% that the directory of the parent source file (the previous head of %% the path) must be dropped, otherwise the path used within the current %% file will depend on the order of file inclusions in the parent files - Path = [filename:dirname(Pname) | tl(St#epp.path)], + Path = [filename:dirname(Pname) | tl(St0#epp.path)], + _ = set_encoding(NewF), #epp{file=NewF,location=Loc,name=Pname,delta=0, - sstk=[St|St#epp.sstk],path=Path,macs=Ms}. + sstk=[St0|St0#epp.sstk],path=Path,macs=Ms}. enter_file_reply(From, Name, Location, AtLocation) -> Attr = loc_attr(AtLocation), @@ -456,7 +632,7 @@ leave_file(From, St) -> %% scan_toks(Tokens, From, EppState) scan_toks(From, St) -> - case io:scan_erl_form(St#epp.file, '', St#epp.location) of + case io:scan_erl_form(St#epp.file, '', St#epp.location, [unicode]) of {ok,Toks,Cl} -> scan_toks(Toks, From, St#epp{location=Cl}); {error,E,Cl} -> @@ -830,7 +1006,7 @@ new_location(Ln, {Le,_}, {Lf,_}) -> %% nested conditionals and repeated 'else's. skip_toks(From, St, [I|Sis]) -> - case io:scan_erl_form(St#epp.file, '', St#epp.location) of + case io:scan_erl_form(St#epp.file, '', St#epp.location, [unicode]) of {ok,[{'-',_Lh},{atom,_Li,ifdef}|_Toks],Cl} -> skip_toks(From, St#epp{location=Cl}, [ifdef,I|Sis]); {ok,[{'-',_Lh},{atom,_Li,ifndef}|_Toks],Cl} -> @@ -1094,6 +1270,7 @@ expand_arg([], Ts, L, Rest, Bs) -> %%% tokenized would yield the token list Ts. %% erl_scan:token_info(T, text) is not backward compatible with this. +%% Note that escaped characters will be replaced by themselves. token_src({dot, _}) -> "."; token_src({X, _}) when is_atom(X) -> @@ -1101,16 +1278,16 @@ token_src({X, _}) when is_atom(X) -> token_src({var, _, X}) -> atom_to_list(X); token_src({char,_,C}) -> - io_lib:write_char(C); + io_lib:write_unicode_char(C); token_src({string, _, X}) -> - lists:flatten(io_lib:format("~p", [X])); + io_lib:write_unicode_string(X); token_src({_, _, X}) -> - lists:flatten(io_lib:format("~w", [X])). + io_lib:format("~w", [X]). stringify1([]) -> []; stringify1([T | Tokens]) -> - [io_lib:format(" ~s", [token_src(T)]) | stringify1(Tokens)]. + [io_lib:format(" ~ts", [token_src(T)]) | stringify1(Tokens)]. stringify(Ts, L) -> [$\s | S] = lists:flatten(stringify1(Ts)), diff --git a/lib/stdlib/src/erl_eval.erl b/lib/stdlib/src/erl_eval.erl index 95ba6b1096..8471ae6b64 100644 --- a/lib/stdlib/src/erl_eval.erl +++ b/lib/stdlib/src/erl_eval.erl @@ -227,13 +227,6 @@ expr({bc,_,E,Qs}, Bs, Lf, Ef, RBs) -> expr({tuple,_,Es}, Bs0, Lf, Ef, RBs) -> {Vs,Bs} = expr_list(Es, Bs0, Lf, Ef), ret_expr(list_to_tuple(Vs), Bs, RBs); -expr({record_field,_,_,_}=Mod, Bs, _Lf, _Ef, RBs) -> - case expand_module_name(Mod, Bs) of - {atom,_,A} -> - ret_expr(A, Bs, RBs); %% This is the "x.y" syntax - _ -> - erlang:raise(error, {badexpr, '.'}, stacktrace()) - end; expr({record_field,_,_,Name,_}, _Bs, _Lf, _Ef, _RBs) -> erlang:raise(error, {undef_record,Name}, stacktrace()); expr({record_index,_,Name,_}, _Bs, _Lf, _Ef, _RBs) -> @@ -332,8 +325,7 @@ expr({call,L1,{remote,L2,{record_field,_,{atom,_,''},{atom,_,qlc}=Mod}, Bs, Lf, Ef, RBs) when length(As0) =< 1 -> expr({call,L1,{remote,L2,Mod,Func},As}, Bs, Lf, Ef, RBs); expr({call,_,{remote,_,Mod,Func},As0}, Bs0, Lf, Ef, RBs) -> - Mod1 = expand_module_name(Mod, Bs0), - {value,M,Bs1} = expr(Mod1, Bs0, Lf, Ef, none), + {value,M,Bs1} = expr(Mod, Bs0, Lf, Ef, none), {value,F,Bs2} = expr(Func, Bs0, Lf, Ef, none), {As,Bs3} = expr_list(As0, merge_bindings(Bs1, Bs2), Lf, Ef), %% M could be a parameterized module (not an atom). @@ -1210,41 +1202,6 @@ ret_expr(_Old, New) -> line(Expr) -> element(2, Expr). -%% In syntax trees, module/package names are atoms or lists of atoms. - -expand_module_name({atom,L,A} = M, Bs) -> - case binding({module,A}, Bs) of - {value, A1} -> - {atom,L,A1}; - unbound -> - case packages:is_segmented(A) of - true -> - M; - false -> -%%% P = case binding({module,'$package'}, Bs) of -%%% {value, P1} -> P1; -%%% unbound -> "" -%%% end, -%%% A1 = list_to_atom(packages:concat(P, A)), -%%% {atom,L,list_to_atom(A1)} - {atom,L,A} - end - end; -expand_module_name(M, _) -> - case erl_parse:package_segments(M) of - error -> - M; - M1 -> - L = element(2,M), - Mod = packages:concat(M1), - case packages:is_valid(Mod) of - true -> - {atom,L,list_to_atom(Mod)}; - false -> - erlang:raise(error, {bad_module_name, Mod}, stacktrace()) - end - end. - %% {?MODULE,expr,3} is still the stacktrace, despite the %% fact that expr() now takes two, three or four arguments... stacktrace() -> [{?MODULE,expr,3}]. diff --git a/lib/stdlib/src/erl_expand_records.erl b/lib/stdlib/src/erl_expand_records.erl index 9759a8f001..d05f630d8e 100644 --- a/lib/stdlib/src/erl_expand_records.erl +++ b/lib/stdlib/src/erl_expand_records.erl @@ -135,8 +135,6 @@ pattern({tuple,Line,Ps}, St0) -> %%pattern({struct,Line,Tag,Ps}, St0) -> %% {TPs,TPsvs,St1} = pattern_list(Ps, St0), %% {{struct,Line,Tag,TPs},TPsvs,St1}; -pattern({record_field,_,_,_}=M, St) -> - {M,St}; % must be a package name pattern({record_index,Line,Name,Field}, St) -> {index_expr(Line, Field, Name, record_fields(Name, St)),St}; pattern({record,Line,Name,Pfs}, St0) -> @@ -306,8 +304,6 @@ expr({tuple,Line,Es0}, St0) -> %%expr({struct,Line,Tag,Es0}, Vs, St0) -> %% {Es1,Esvs,Esus,St1} = expr_list(Es0, Vs, St0), %% {{struct,Line,Tag,Es1},Esvs,Esus,St1}; -expr({record_field,_,_,_}=M, St) -> - {M,St}; % must be a package name expr({record_index,Line,Name,F}, St) -> I = index_expr(Line, F, Name, record_fields(Name, St)), expr(I, St); @@ -375,15 +371,9 @@ expr({call,Line,{atom,_La,N}=Atom,As0}, St0) -> end end end; -expr({call,Line,{record_field,_,_,_}=M,As0}, St0) -> - {As,St1} = expr_list(As0, St0), - {{call,Line,M,As},St1}; expr({call,Line,{remote,Lr,M,F},As0}, St0) -> {[M1,F1 | As1],St1} = expr_list([M,F | As0], St0), {{call,Line,{remote,Lr,M1,F1},As1},St1}; -expr({call,Line,{tuple,Lt,[{atom,_,_}=M,{atom,_,_}=F]},As0}, St0) -> - {As,St1} = expr_list(As0, St0), - {{call,Line,{tuple,Lt,[M,F]},As},St1}; expr({call,Line,F,As0}, St0) -> {[Fun1 | As1],St1} = expr_list([F | As0], St0), {{call,Line,Fun1,As1},St1}; diff --git a/lib/stdlib/src/erl_lint.erl b/lib/stdlib/src/erl_lint.erl index 1e5f962375..d24e2fff44 100644 --- a/lib/stdlib/src/erl_lint.erl +++ b/lib/stdlib/src/erl_lint.erl @@ -94,12 +94,10 @@ value_option(Flag, Default, On, OnVal, Off, OffVal, Opts) -> %% the other function collections contain {Function, Arity}. -record(lint, {state=start :: 'start' | 'attribute' | 'function', module=[], %Module - package="", %Module package extends=[], %Extends behaviour=[], %Behaviour exports=gb_sets:empty() :: gb_set(), %Exports imports=[], %Imports - mod_imports=dict:new() :: dict(), %Module Imports compile=[], %Compile flags records=dict:new() :: dict(), %Record definitions locals=gb_sets:empty() :: gb_set(), %All defined functions (prescanned) @@ -539,7 +537,6 @@ start(File, Opts) -> end, #lint{state = start, exports = gb_sets:from_list([{module_info,0},{module_info,1}]), - mod_imports = dict:from_list([{erlang,erlang}]), compile = Opts, %% Internal pseudo-functions must appear as defined/reached. defined = gb_sets:from_list(pseudolocals()), @@ -681,8 +678,8 @@ form(Form, #lint{state=State}=St) -> %% start_state(Form, State) -> State' -start_state({attribute,L,module,{M,Ps}}, St) -> - St1 = set_module(M, L, St), +start_state({attribute,_,module,{M,Ps}}, St0) -> + St1 = St0#lint{module=M}, Arity = length(Ps), Ps1 = if is_atom(St1#lint.extends) -> ['BASE', 'THIS' | Ps]; @@ -693,23 +690,13 @@ start_state({attribute,L,module,{M,Ps}}, St) -> St2 = add_instance(Arity, St1), St3 = ensure_new(Arity, St2), St3#lint{state=attribute, extends=[], global_vt=Vt}; -start_state({attribute,L,module,M}, St) -> - St1 = set_module(M, L, St), +start_state({attribute,_,module,M}, St0) -> + St1 = St0#lint{module=M}, St1#lint{state=attribute, extends=[]}; start_state(Form, St) -> St1 = add_error(element(2, Form), undefined_module, St), attribute_state(Form, St1#lint{state=attribute, extends=[]}). -set_module(M, L, St) -> - M1 = package_to_string(M), - case packages:is_valid(M1) of - true -> - St#lint{module=list_to_atom(M1), - package=packages:strip_last(M1)}; - false -> - add_error(L, {bad_module_name, M1}, St) - end. - ensure_new(Arity, St) -> case St#lint.new of true -> @@ -1007,9 +994,9 @@ check_imports(Forms, St0) -> true -> Usage = St0#lint.usage, Unused = ordsets:subtract(St0#lint.imports, Usage#usage.imported), - Imports = [{{FA,list_to_atom(package_to_string(Mod))},L} - || {attribute,L,import,{Mod,Fs}} <- Forms, - FA <- lists:usort(Fs)], + Imports = [{{FA,Mod},L} || + {attribute,L,import,{Mod,Fs}} <- Forms, + FA <- lists:usort(Fs)], Bad = [{FM,L} || FM <- Unused, {FM2,L} <- Imports, FM =:= FM2], func_line_warning(unused_import, Bad, St0) end. @@ -1222,73 +1209,46 @@ export_type(Line, ETs, #lint{usage = Usage, exp_types = ETs0} = St0) -> -spec import(line(), import(), lint_state()) -> lint_state(). import(Line, {Mod,Fs}, St) -> - Mod1 = package_to_string(Mod), - case packages:is_valid(Mod1) of - true -> - Mfs = ordsets:from_list(Fs), - case check_imports(Line, Mfs, St#lint.imports) of - [] -> - St#lint{imports=add_imports(list_to_atom(Mod1), Mfs, - St#lint.imports)}; - Efs -> - {Err, St1} = - foldl(fun ({bif,{F,A},_}, {Err,St0}) -> - %% BifClash - import directive - Warn = is_warn_enabled(bif_clash, St0) - and (not bif_clash_specifically_disabled(St0,{F,A})), - AutoImpSup = is_autoimport_suppressed(St0#lint.no_auto,{F,A}), - OldBif = erl_internal:old_bif(F,A), - {Err,if - Warn and (not AutoImpSup) and OldBif -> - add_error - (Line, - {redefine_old_bif_import, {F,A}}, - St0); - Warn and (not AutoImpSup) -> - add_warning - (Line, - {redefine_bif_import, {F,A}}, - St0); - true -> - St0 - end}; - (Ef, {_Err,St0}) -> - {true,add_error(Line, - {redefine_import,Ef}, - St0)} - end, - {false,St}, Efs), - if - not Err -> - St1#lint{imports= - add_imports(list_to_atom(Mod1), Mfs, + Mfs = ordsets:from_list(Fs), + case check_imports(Line, Mfs, St#lint.imports) of + [] -> + St#lint{imports=add_imports(Mod, Mfs, + St#lint.imports)}; + Efs -> + {Err, St1} = + foldl(fun ({bif,{F,A},_}, {Err,St0}) -> + %% BifClash - import directive + Warn = is_warn_enabled(bif_clash, St0) andalso + (not bif_clash_specifically_disabled(St0,{F,A})), + AutoImpSup = is_autoimport_suppressed(St0#lint.no_auto,{F,A}), + OldBif = erl_internal:old_bif(F,A), + {Err,if + Warn and (not AutoImpSup) and OldBif -> + add_error + (Line, + {redefine_old_bif_import, {F,A}}, + St0); + Warn and (not AutoImpSup) -> + add_warning + (Line, + {redefine_bif_import, {F,A}}, + St0); + true -> + St0 + end}; + (Ef, {_Err,St0}) -> + {true,add_error(Line, + {redefine_import,Ef}, + St0)} + end, + {false,St}, Efs), + if + not Err -> + St1#lint{imports=add_imports(Mod, Mfs, St#lint.imports)}; - true -> - St1 - end - end; - false -> - add_error(Line, {bad_module_name, Mod1}, St) - end; -import(Line, Mod, St) -> - Mod1 = package_to_string(Mod), - case packages:is_valid(Mod1) of - true -> - Key = list_to_atom(packages:last(Mod1)), - Imps = St#lint.mod_imports, -%%% case dict:is_key(Key, Imps) of -%%% true -> -%%% M = packages:last(Mod1), -%%% P = packages:strip_last(Mod1), -%%% add_error(Line, {redefine_mod_import, M, P}, St); -%%% false -> -%%% St#lint{mod_imports = -%%% dict:store(Key, list_to_atom(Mod1), Imps)} -%%% end; - St#lint{mod_imports = dict:store(Key, list_to_atom(Mod1), - Imps)}; - false -> - add_error(Line, {bad_module_name, Mod1}, St) + true -> + St1 + end end. check_imports(_Line, Fs, Is) -> @@ -1463,13 +1423,6 @@ pattern({record_index,Line,Name,Field}, _Vt, _Old, _Bvt, St) -> pattern_field(Field, Name, Dfs, St1) end), {Vt1,[],St1}; -pattern({record_field,Line,_,_}=M, _Vt, _Old, _Bvt, St0) -> - case expand_package(M, St0) of - {error, St1} -> - {[],[],add_error(Line, illegal_expr, St1)}; - {_, St1} -> - {[],[],St1} - end; pattern({record,Line,Name,Pfs}, Vt, Old, Bvt, St) -> case dict:find(Name, St#lint.records) of {ok,{_Line,Fields}} -> @@ -1851,13 +1804,6 @@ gexpr({tuple,_Line,Es}, Vt, St) -> gexpr({record_index,Line,Name,Field}, _Vt, St) -> check_record(Line, Name, St, fun (Dfs, St1) -> record_field(Field, Name, Dfs, St1) end ); -gexpr({record_field,Line,_,_}=M, _Vt, St0) -> - case expand_package(M, St0) of - {error, St1} -> - {[],add_error(Line, illegal_expr, St1)}; - {_, St1} -> - {[], St1} - end; gexpr({record_field,Line,Rec,Name,Field}, Vt, St0) -> {Rvt,St1} = gexpr(Rec, Vt, St0), {Fvt,St2} = check_record(Line, Name, St1, @@ -1996,8 +1942,6 @@ is_gexpr({tuple,_L,Es}, RDs) -> is_gexpr_list(Es, RDs); %% is_gexpr_list(Es, RDs); is_gexpr({record_index,_L,_Name,Field}, RDs) -> is_gexpr(Field, RDs); -is_gexpr({record_field,_L,_,_}=M, _RDs) -> - erl_parse:package_segments(M) =/= error; is_gexpr({record_field,_L,Rec,_Name,Field}, RDs) -> is_gexpr_list([Rec,Field], RDs); is_gexpr({record,L,Name,Inits}, RDs) -> @@ -2086,13 +2030,6 @@ expr({record,Line,Name,Inits}, Vt, St) -> fun (Dfs, St1) -> init_fields(Inits, Line, Name, Dfs, Vt, St1) end); -expr({record_field,Line,_,_}=M, _Vt, St0) -> - case expand_package(M, St0) of - {error, St1} -> - {[],add_error(Line, illegal_expr, St1)}; - {_, St1} -> - {[], St1} - end; expr({record_field,Line,Rec,Name,Field}, Vt, St0) -> {Rvt,St1} = record_expr(Line, Rec, Vt, St0), {Fvt,St2} = check_record(Line, Name, St1, @@ -2163,20 +2100,14 @@ expr({call,Line,{remote,_Lr,{atom,_Lm,erlang},{atom,Lf,is_record}},[E,A]}, expr({call,Line,{atom,Lf,is_record},[E,A]}, Vt, St0); expr({call,L,{tuple,Lt,[{atom,Lm,erlang},{atom,Lf,is_record}]},As}, Vt, St) -> expr({call,L,{remote,Lt,{atom,Lm,erlang},{atom,Lf,is_record}},As}, Vt, St); +expr({call,Line,{remote,_Lr,{atom,_Lm,M},{atom,Lf,F}},As}, Vt, St0) -> + St1 = keyword_warning(Lf, F, St0), + St2 = check_remote_function(Line, M, F, As, St1), + expr_list(As, Vt, St2); expr({call,Line,{remote,_Lr,M,F},As}, Vt, St0) -> - case expand_package(M, St0) of - {error, _} -> - expr_list([M,F|As], Vt, St0); - {{atom,_La,M1}, St1} -> - case F of - {atom,Lf,F1} -> - St2 = keyword_warning(Lf, F1, St1), - St3 = check_remote_function(Line, M1, F1, As, St2), - expr_list(As, Vt, St3); - _ -> - expr_list([F|As], Vt, St1) - end - end; + St1 = keyword_warning(Line, M, St0), + St2 = keyword_warning(Line, F, St1), + expr_list([M,F|As], Vt, St2); expr({call,Line,{atom,La,F},As}, Vt, St0) -> St1 = keyword_warning(La, F, St0), {Asvt,St2} = expr_list(As, Vt, St1), @@ -2232,13 +2163,6 @@ expr({call,Line,{atom,La,F},As}, Vt, St0) -> end end} end; -expr({call,Line,{record_field,_,_,_}=F,As}, Vt, St0) -> - case expand_package(F, St0) of - {error, _} -> - expr_list([F|As], Vt, St0); - {A, St1} -> - expr({call,Line,A,As}, Vt, St1) - end; expr({call,Line,F,As}, Vt, St0) -> St = warn_invalid_call(Line,F,St0), expr_list([F|As], Vt, St); %They see the same variables @@ -3618,6 +3542,10 @@ extract_sequence(4, [$t, $c | Fmt], Need) -> extract_sequence(5, [$c|Fmt], Need); extract_sequence(4, [$t, $s | Fmt], Need) -> extract_sequence(5, [$s|Fmt], Need); +extract_sequence(4, [$t, $p | Fmt], Need) -> + extract_sequence(5, [$p|Fmt], Need); +extract_sequence(4, [$t, $P | Fmt], Need) -> + extract_sequence(5, [$P|Fmt], Need); extract_sequence(4, [$t, C | _Fmt], _Need) -> {error,"invalid control ~t" ++ [C]}; extract_sequence(4, Fmt, Need) -> @@ -3654,49 +3582,6 @@ control_type($n, Need) -> Need; control_type($i, Need) -> [term|Need]; control_type(_C, _Need) -> error. -%% In syntax trees, module/package names are atoms or lists of atoms. - -package_to_string(A) when is_atom(A) -> atom_to_list(A); -package_to_string(L) when is_list(L) -> packages:concat(L). - -expand_package({atom,L,A} = M, St0) -> - St1 = keyword_warning(L, A, St0), - case dict:find(A, St1#lint.mod_imports) of - {ok, A1} -> - {{atom,L,A1}, St1}; - error -> - Name = atom_to_list(A), - case packages:is_valid(Name) of - true -> - case packages:is_segmented(Name) of - true -> - {M, St1}; - false -> - M1 = packages:concat(St1#lint.package, - Name), - {{atom,L,list_to_atom(M1)}, St1} - end; - false -> - St2 = add_error(L, {bad_module_name, Name}, St1), - {error, St2} - end - end; -expand_package(M, St0) -> - L = element(2, M), - case erl_parse:package_segments(M) of - error -> - {error, St0}; - M1 -> - Name = package_to_string(M1), - case packages:is_valid(Name) of - true -> - {{atom,L,list_to_atom(Name)}, St0}; - false -> - St1 = add_error(L, {bad_module_name, Name}, St0), - {error, St1} - end - end. - %% Prebuild set of local functions (to override auto-import) local_functions(Forms) -> diff --git a/lib/stdlib/src/erl_parse.yrl b/lib/stdlib/src/erl_parse.yrl index 928c10f7f2..002abc11e8 100644 --- a/lib/stdlib/src/erl_parse.yrl +++ b/lib/stdlib/src/erl_parse.yrl @@ -2,7 +2,7 @@ %% %% %CopyrightBegin% %% -%% Copyright Ericsson AB 1996-2011. All Rights Reserved. +%% Copyright Ericsson AB 1996-2012. All Rights Reserved. %% %% The contents of this file are subject to the Erlang Public License, %% Version 1.1, (the "License"); you may not use this file except in @@ -26,7 +26,7 @@ attribute attr_val function function_clauses function_clause clause_args clause_guard clause_body expr expr_100 expr_150 expr_160 expr_200 expr_300 expr_400 expr_500 -expr_600 expr_700 expr_800 expr_900 +expr_600 expr_700 expr_800 expr_max list tail list_comprehension lc_expr lc_exprs @@ -253,15 +253,9 @@ expr_700 -> function_call : '$1'. expr_700 -> record_expr : '$1'. expr_700 -> expr_800 : '$1'. -expr_800 -> expr_900 ':' expr_max : +expr_800 -> expr_max ':' expr_max : {remote,?line('$2'),'$1','$3'}. -expr_800 -> expr_900 : '$1'. - -expr_900 -> '.' atom : - {record_field,?line('$1'),{atom,?line('$1'),''},'$2'}. -expr_900 -> expr_900 '.' atom : - {record_field,?line('$2'),'$1','$3'}. -expr_900 -> expr_max : '$1'. +expr_800 -> expr_max : '$1'. expr_max -> var : '$1'. expr_max -> atomic : '$1'. @@ -510,7 +504,7 @@ Erlang code. -export([parse_form/1,parse_exprs/1,parse_term/1]). -export([normalise/1,abstract/1,tokens/1,tokens/2]). --export([abstract/2, package_segments/1]). +-export([abstract/2]). -export([inop_prec/1,preop_prec/1,func_prec/0,max_prec/0]). -export([set_line/2,get_attribute/2,get_attributes/1]). @@ -679,20 +673,6 @@ build_attribute({atom,La,module}, Val) -> {attribute,La,module,Module}; [{atom,_Lm,Module},ExpList] -> {attribute,La,module,{Module,var_list(ExpList)}}; - [Name] -> - case package_segments(Name) of - error -> - error_bad_decl(La, module); - Module -> - {attribute,La,module,Module} - end; - [Name,ExpList] -> - case package_segments(Name) of - error -> - error_bad_decl(La, module); - Module -> - {attribute,La,module,{Module,var_list(ExpList)}} - end; _Other -> error_bad_decl(La, module) end; @@ -704,22 +684,8 @@ build_attribute({atom,La,export}, Val) -> end; build_attribute({atom,La,import}, Val) -> case Val of - [Name] -> - case package_segments(Name) of - error -> - error_bad_decl(La, import); - Module -> - {attribute,La,import,Module} - end; [{atom,_Lm,Mod},ImpList] -> {attribute,La,import,{Mod,farity_list(ImpList)}}; - [Name, ImpList] -> - case package_segments(Name) of - error -> - error_bad_decl(La, import); - Module -> - {attribute,La,import,{Module,farity_list(ImpList)}} - end; _Other -> error_bad_decl(La, import) end; build_attribute({atom,La,record}, Val) -> @@ -820,18 +786,6 @@ term(Expr) -> catch _:_R -> ret_err(?line(Expr), "bad attribute") end. -package_segments(Name) -> - package_segments(Name, [], []). - -package_segments({record_field, _, F1, F2}, Fs, As) -> - package_segments(F1, [F2 | Fs], As); -package_segments({atom, _, A}, [F | Fs], As) -> - package_segments(F, Fs, [A | As]); -package_segments({atom, _, A}, [], As) -> - lists:reverse([A | As]); -package_segments(_, _, _) -> - error. - %% build_function([Clause]) -> {function,Line,Name,Arity,[Clause]} build_function(Cs) -> @@ -900,12 +854,6 @@ normalise({cons,_,Head,Tail}) -> [normalise(Head)|normalise(Tail)]; normalise({tuple,_,Args}) -> list_to_tuple(normalise_list(Args)); -%% Atom dot-notation, as in 'foo.bar.baz' -normalise({record_field,_,_,_}=A) -> - case package_segments(A) of - error -> erlang:error({badarg, A}); - As -> list_to_atom(packages:concat(As)) - end; %% Special case for unary +/-. normalise({op,_,'+',{char,_,I}}) -> I; normalise({op,_,'+',{integer,_,I}}) -> I; @@ -923,73 +871,77 @@ normalise_list([]) -> -spec abstract(Data) -> AbsTerm when Data :: term(), AbsTerm :: abstract_expr(). -abstract(T) when is_integer(T) -> {integer,0,T}; -abstract(T) when is_float(T) -> {float,0,T}; -abstract(T) when is_atom(T) -> {atom,0,T}; -abstract([]) -> {nil,0}; -abstract(B) when is_bitstring(B) -> - {bin, 0, [abstract_byte(Byte, 0) || Byte <- bitstring_to_list(B)]}; -abstract([C|T]) when is_integer(C), 0 =< C, C < 256 -> - abstract_string(T, [C]); -abstract([H|T]) -> - {cons,0,abstract(H),abstract(T)}; -abstract(Tuple) when is_tuple(Tuple) -> - {tuple,0,abstract_list(tuple_to_list(Tuple))}. - -abstract_string([C|T], String) when is_integer(C), 0 =< C, C < 256 -> - abstract_string(T, [C|String]); -abstract_string([], String) -> - {string, 0, lists:reverse(String)}; -abstract_string(T, String) -> - not_string(String, abstract(T)). - -not_string([C|T], Result) -> - not_string(T, {cons, 0, {integer, 0, C}, Result}); -not_string([], Result) -> +abstract(T) -> + abstract(T, 0, epp:default_encoding()). + +%%% abstract/2 takes line and encoding options +-spec abstract(Data, Options) -> AbsTerm when + Data :: term(), + Options :: Line | [Option], + Option :: {line, Line} | {encoding, Encoding}, + Encoding :: latin1 | unicode | utf8, + Line :: erl_scan:line(), + AbsTerm :: abstract_expr(). + +abstract(T, Line) when is_integer(Line) -> + abstract(T, Line, epp:default_encoding()); +abstract(T, Options) when is_list(Options) -> + Line = proplists:get_value(line, Options, 0), + Encoding = proplists:get_value(encoding, Options,epp:default_encoding()), + abstract(T, Line, Encoding). + +-define(UNICODE(C), + (C >= 0 andalso C < 16#D800 orelse + C > 16#DFFF andalso C < 16#FFFE orelse + C > 16#FFFF andalso C =< 16#10FFFF)). + +abstract(T, L, _E) when is_integer(T) -> {integer,L,T}; +abstract(T, L, _E) when is_float(T) -> {float,L,T}; +abstract(T, L, _E) when is_atom(T) -> {atom,L,T}; +abstract([], L, _E) -> {nil,L}; +abstract(B, L, _E) when is_bitstring(B) -> + {bin, L, [abstract_byte(Byte, L) || Byte <- bitstring_to_list(B)]}; +abstract([C|T], L, unicode=E) when ?UNICODE(C) -> + abstract_unicode_string(T, [C], L, E); +abstract([C|T], L, utf8=E) when ?UNICODE(C) -> + abstract_unicode_string(T, [C], L, E); +abstract([C|T], L, latin1=E) when is_integer(C), 0 =< C, C < 256 -> + abstract_string(T, [C], L, E); +abstract([H|T], L, E) -> + {cons,L,abstract(H, L, E),abstract(T, L, E)}; +abstract(Tuple, L, E) when is_tuple(Tuple) -> + {tuple,L,abstract_list(tuple_to_list(Tuple), L, E)}. + +abstract_string([C|T], String, L, E) when is_integer(C), 0 =< C, C < 256 -> + abstract_string(T, [C|String], L, E); +abstract_string([], String, L, _E) -> + {string, L, lists:reverse(String)}; +abstract_string(T, String, L, E) -> + not_string(String, abstract(T, L, E), L, E). + +abstract_unicode_string([C|T], String, L, E) when ?UNICODE(C) -> + abstract_unicode_string(T, [C|String], L, E); +abstract_unicode_string([], String, L, _E) -> + {string, L, lists:reverse(String)}; +abstract_unicode_string(T, String, L, E) -> + not_string(String, abstract(T, L, E), L, E). + +not_string([C|T], Result, L, E) -> + not_string(T, {cons, L, {integer, L, C}, Result}, L, E); +not_string([], Result, _L, _E) -> Result. -abstract_list([H|T]) -> - [abstract(H)|abstract_list(T)]; -abstract_list([]) -> +abstract_list([H|T], L, E) -> + [abstract(H, L, E)|abstract_list(T, L, E)]; +abstract_list([], _L, _E) -> []. -abstract_byte(Byte, Line) when is_integer(Byte) -> - {bin_element, Line, {integer, Line, Byte}, default, default}; -abstract_byte(Bits, Line) -> +abstract_byte(Byte, L) when is_integer(Byte) -> + {bin_element, L, {integer, L, Byte}, default, default}; +abstract_byte(Bits, L) -> Sz = bit_size(Bits), <<Val:Sz>> = Bits, - {bin_element, Line, {integer, Line, Val}, {integer, Line, Sz}, default}. - -%%% abstract/2 keeps the line number -abstract(T, Line) when is_integer(T) -> {integer,Line,T}; -abstract(T, Line) when is_float(T) -> {float,Line,T}; -abstract(T, Line) when is_atom(T) -> {atom,Line,T}; -abstract([], Line) -> {nil,Line}; -abstract(B, Line) when is_bitstring(B) -> - {bin, Line, [abstract_byte(Byte, Line) || Byte <- bitstring_to_list(B)]}; -abstract([C|T], Line) when is_integer(C), 0 =< C, C < 256 -> - abstract_string(T, [C], Line); -abstract([H|T], Line) -> - {cons,Line,abstract(H, Line),abstract(T, Line)}; -abstract(Tuple, Line) when is_tuple(Tuple) -> - {tuple,Line,abstract_list(tuple_to_list(Tuple), Line)}. - -abstract_string([C|T], String, Line) when is_integer(C), 0 =< C, C < 256 -> - abstract_string(T, [C|String], Line); -abstract_string([], String, Line) -> - {string, Line, lists:reverse(String)}; -abstract_string(T, String, Line) -> - not_string(String, abstract(T, Line), Line). - -not_string([C|T], Result, Line) -> - not_string(T, {cons, Line, {integer, Line, C}, Result}, Line); -not_string([], Result, _Line) -> - Result. - -abstract_list([H|T], Line) -> - [abstract(H, Line)|abstract_list(T, Line)]; -abstract_list([], _Line) -> - []. + {bin_element, L, {integer, L, Val}, {integer, L, Sz}, default}. %% Generate a list of tokens representing the abstract term. @@ -1079,9 +1031,9 @@ preop_prec('#') -> {700,800}. func_prec() -> {800,700}. --spec max_prec() -> 1000. +-spec max_prec() -> 900. -max_prec() -> 1000. +max_prec() -> 900. %%% [Experimental]. The parser just copies the attributes of the %%% scanner tokens to the abstract format. This design decision has diff --git a/lib/stdlib/src/erl_pp.erl b/lib/stdlib/src/erl_pp.erl index 6b5aa951cf..0383ce6839 100644 --- a/lib/stdlib/src/erl_pp.erl +++ b/lib/stdlib/src/erl_pp.erl @@ -1,7 +1,7 @@ %% %% %CopyrightBegin% %% -%% Copyright Ericsson AB 1996-2011. All Rights Reserved. +%% Copyright Ericsson AB 1996-2012. All Rights Reserved. %% %% The contents of this file are subject to the Erlang Public License, %% Version 1.1, (the "License"); you may not use this file except in @@ -26,7 +26,7 @@ guard/1,guard/2,exprs/1,exprs/2,exprs/3,expr/1,expr/2,expr/3,expr/4]). -import(lists, [append/1,foldr/3,mapfoldl/3,reverse/1,reverse/2]). --import(io_lib, [write/1,format/2,write_char/1,write_string/1]). +-import(io_lib, [write/1,format/2]). -import(erl_parse, [inop_prec/1,preop_prec/1,func_prec/0,max_prec/0]). -define(MAXLINE, 72). @@ -36,7 +36,15 @@ CurrentIndentation :: integer(), CurrentPrecedence :: non_neg_integer(), HookFunction :: hook_function()) -> - io_lib:chars())). + io_lib:chars())). + +-type(option() :: {hook, hook_function()} + | {encoding, latin1 | unicode | utf8}). +-type(options() :: hook_function() | [option()]). + +-record(pp, {string_fun, char_fun}). + +-record(options, {hook, encoding, opts}). %%% %%% Exported functions @@ -48,12 +56,12 @@ form(Thing) -> form(Thing, none). --spec(form(Form, HookFunction) -> io_lib:chars() when +-spec(form(Form, Options) -> io_lib:chars() when Form :: erl_parse:abstract_form(), - HookFunction :: hook_function()). + Options :: options()). -form(Thing, Hook) -> - frmt(lform(Thing, Hook)). +form(Thing, Options) -> + frmt(lform(Thing, options(Options)), state(Options)). -spec(attribute(Attribute) -> io_lib:chars() when Attribute :: erl_parse:abstract_form()). @@ -61,12 +69,12 @@ form(Thing, Hook) -> attribute(Thing) -> attribute(Thing, none). --spec(attribute(Attribute, HookFunction) -> io_lib:chars() when +-spec(attribute(Attribute, Options) -> io_lib:chars() when Attribute :: erl_parse:abstract_form(), - HookFunction :: hook_function()). + Options :: options()). -attribute(Thing, Hook) -> - frmt(lattribute(Thing, Hook)). +attribute(Thing, Options) -> + frmt(lattribute(Thing, options(Options)), state(Options)). -spec(function(Function) -> io_lib:chars() when Function :: erl_parse:abstract_form()). @@ -74,18 +82,18 @@ attribute(Thing, Hook) -> function(F) -> function(F, none). --spec(function(Function, HookFunction) -> io_lib:chars() when +-spec(function(Function, Options) -> io_lib:chars() when Function :: erl_parse:abstract_form(), - HookFunction :: hook_function()). + Options :: options()). -function(F, Hook) -> - frmt(lfunction(F, Hook)). +function(F, Options) -> + frmt(lfunction(F, options(Options)), state(Options)). rule(R) -> rule(R, none). -rule(R, Hook) -> - frmt(lrule(R, Hook)). +rule(R, Options) -> + frmt(lrule(R, options(Options)), state(Options)). -spec(guard(Guard) -> io_lib:chars() when Guard :: [erl_parse:abstract_expr()]). @@ -93,12 +101,12 @@ rule(R, Hook) -> guard(Gs) -> guard(Gs, none). --spec(guard(Guard, HookFunction) -> io_lib:chars() when +-spec(guard(Guard, Options) -> io_lib:chars() when Guard :: [erl_parse:abstract_expr()], - HookFunction :: hook_function()). + Options :: options()). -guard(Gs, Hook) -> - frmt(lguard(Gs, Hook)). +guard(Gs, Options) -> + frmt(lguard(Gs, options(Options)), state(Options)). -spec(exprs(Expressions) -> io_lib:chars() when Expressions :: [erl_parse:abstract_expr()]). @@ -106,99 +114,129 @@ guard(Gs, Hook) -> exprs(Es) -> exprs(Es, 0, none). --spec(exprs(Expressions, HookFunction) -> io_lib:chars() when +-spec(exprs(Expressions, Options) -> io_lib:chars() when Expressions :: [erl_parse:abstract_expr()], - HookFunction :: hook_function()). + Options :: options()). -exprs(Es, Hook) -> - exprs(Es, 0, Hook). +exprs(Es, Options) -> + exprs(Es, 0, Options). --spec(exprs(Expressions, Indent, HookFunction) -> io_lib:chars() when +-spec(exprs(Expressions, Indent, Options) -> io_lib:chars() when Expressions :: [erl_parse:abstract_expr()], Indent :: integer(), - HookFunction :: hook_function()). + Options :: options()). -exprs(Es, I, Hook) -> - frmt({seq,[],[],[$,],lexprs(Es, Hook)}, I). +exprs(Es, I, Options) -> + frmt({seq,[],[],[$,],lexprs(Es, options(Options))}, I, state(Options)). -spec(expr(Expression) -> io_lib:chars() when Expression :: erl_parse:abstract_expr()). expr(E) -> - frmt(lexpr(E, 0, none)). + frmt(lexpr(E, 0, options(none)), state(none)). --spec(expr(Expression, HookFunction) -> io_lib:chars() when +-spec(expr(Expression, Options) -> io_lib:chars() when Expression :: erl_parse:abstract_expr(), - HookFunction :: hook_function()). + Options :: options()). -expr(E, Hook) -> - frmt(lexpr(E, 0, Hook)). +expr(E, Options) -> + frmt(lexpr(E, 0, options(Options)), state(Options)). --spec(expr(Expression, Indent, HookFunction) -> io_lib:chars() when +-spec(expr(Expression, Indent, Options) -> io_lib:chars() when Expression :: erl_parse:abstract_expr(), Indent :: integer(), - HookFunction :: hook_function()). + Options :: options()). -expr(E, I, Hook) -> - frmt(lexpr(E, 0, Hook), I). +expr(E, I, Options) -> + frmt(lexpr(E, 0, options(Options)), I, state(Options)). --spec(expr(Expression, Indent, Precedence, HookFunction) -> io_lib:chars() when +-spec(expr(Expression, Indent, Precedence, Options) -> io_lib:chars() when Expression :: erl_parse:abstract_expr(), Indent :: integer(), Precedence :: non_neg_integer(), - HookFunction :: hook_function()). + Options :: options()). -expr(E, I, P, Hook) -> - frmt(lexpr(E, P, Hook), I). +expr(E, I, P, Options) -> + frmt(lexpr(E, P, options(Options)), I, state(Options)). %%% %%% Local functions %%% -lform({attribute,Line,Name,Arg}, Hook) -> - lattribute({attribute,Line,Name,Arg}, Hook); -lform({function,Line,Name,Arity,Clauses}, Hook) -> - lfunction({function,Line,Name,Arity,Clauses}, Hook); -lform({rule,Line,Name,Arity,Clauses}, Hook) -> - lrule({rule,Line,Name,Arity,Clauses}, Hook); +options(Options) when is_list(Options) -> + Hook = proplists:get_value(hook, Options, none), + Encoding = encoding(Options), + #options{hook = Hook, encoding = Encoding, opts = Options}; +options(Hook) -> + #options{hook = Hook, encoding = encoding([]), opts = Hook}. + +state(Options) when is_list(Options) -> + case encoding(Options) of + latin1 -> state(); + unicode -> unicode_state() + end; +state(_Hook) -> + state(). + +state() -> + #pp{string_fun = fun io_lib:write_unicode_string_as_latin1/1, + char_fun = fun io_lib:write_unicode_char_as_latin1/1}. + +unicode_state() -> + #pp{string_fun = fun io_lib:write_unicode_string/1, + char_fun = fun io_lib:write_unicode_char/1}. + +encoding(Options) -> + case proplists:get_value(encoding, Options, epp:default_encoding()) of + latin1 -> latin1; + utf8 -> unicode; + unicode -> unicode + end. + +lform({attribute,Line,Name,Arg}, Opts) -> + lattribute({attribute,Line,Name,Arg}, Opts); +lform({function,Line,Name,Arity,Clauses}, Opts) -> + lfunction({function,Line,Name,Arity,Clauses}, Opts); +lform({rule,Line,Name,Arity,Clauses}, Opts) -> + lrule({rule,Line,Name,Arity,Clauses}, Opts); %% These are specials to make it easier for the compiler. -lform({error,E}, _Hook) -> +lform({error,E}, _Opts) -> leaf(format("~p\n", [{error,E}])); -lform({warning,W}, _Hook) -> +lform({warning,W}, _Opts) -> leaf(format("~p\n", [{warning,W}])); -lform({eof,_Line}, _Hook) -> +lform({eof,_Line}, _Opts) -> $\n. -lattribute({attribute,_Line,type,Type}, Hook) -> - [typeattr(type, Type, Hook),leaf(".\n")]; -lattribute({attribute,_Line,opaque,Type}, Hook) -> - [typeattr(opaque, Type, Hook),leaf(".\n")]; -lattribute({attribute,_Line,spec,Arg}, _Hook) -> +lattribute({attribute,_Line,type,Type}, Opts) -> + [typeattr(type, Type, Opts),leaf(".\n")]; +lattribute({attribute,_Line,opaque,Type}, Opts) -> + [typeattr(opaque, Type, Opts),leaf(".\n")]; +lattribute({attribute,_Line,spec,Arg}, _Opts) -> [specattr(Arg),leaf(".\n")]; -lattribute({attribute,_Line,Name,Arg}, Hook) -> - [lattribute(Name, Arg, Hook),leaf(".\n")]. +lattribute({attribute,_Line,Name,Arg}, Opts) -> + [lattribute(Name, Arg, Opts),leaf(".\n")]. -lattribute(module, {M,Vs}, _Hook) -> +lattribute(module, {M,Vs}, _Opts) -> attr("module",[{var,0,pname(M)}, foldr(fun(V, C) -> {cons,0,{var,0,V},C} end, {nil,0}, Vs)]); -lattribute(module, M, _Hook) -> +lattribute(module, M, _Opts) -> attr("module", [{var,0,pname(M)}]); -lattribute(export, Falist, _Hook) -> +lattribute(export, Falist, _Opts) -> call({var,0,"-export"}, [falist(Falist)], 0, none); -lattribute(import, Name, _Hook) when is_list(Name) -> +lattribute(import, Name, _Opts) when is_list(Name) -> attr("import", [{var,0,pname(Name)}]); -lattribute(import, {From,Falist}, _Hook) -> +lattribute(import, {From,Falist}, _Opts) -> attr("import",[{var,0,pname(From)},falist(Falist)]); -lattribute(file, {Name,Line}, _Hook) -> +lattribute(file, {Name,Line}, _Opts) -> attr("file", [{var,0,format("~p", [Name])},{integer,0,Line}]); -lattribute(record, {Name,Is}, Hook) -> +lattribute(record, {Name,Is}, Opts) -> Nl = leaf(format("-record(~w,", [Name])), - [{first,Nl,record_fields(Is, Hook)},$)]; -lattribute(Name, Arg, _Hook) -> - attr(write(Name), [erl_parse:abstract(Arg)]). + [{first,Nl,record_fields(Is, Opts)},$)]; +lattribute(Name, Arg, #options{encoding = Encoding}) -> + attr(write(Name), [erl_parse:abstract(Arg, [{encoding,Encoding}])]). -typeattr(Tag, {TypeName,Type,Args}, _Hook) -> +typeattr(Tag, {TypeName,Type,Args}, _Opts) -> {first,leaf("-"++atom_to_list(Tag)++" "), typed(call({atom,0,TypeName}, Args, 0, none), Type)}. @@ -293,7 +331,7 @@ guard_type(Before, Gs) -> Gl = {list,[{step,'when',expr_list(Gs, [$,], fun constraint/2, none)}]}, {list,[{step,Before,Gl}]}. -constraint({type,_Line,constraint,[Tag,As]}, _Hook) -> +constraint({type,_Line,constraint,[Tag,As]}, _Opts) -> simple_type(Tag, As). fun_type(Before, {type,_,'fun',[FType,Ret]}) -> @@ -333,231 +371,232 @@ falist([]) -> falist([{Name,Arity}|Falist]) -> {cons,0,{var,0,format("~w/~w", [Name,Arity])},falist(Falist)}. -lfunction({function,_Line,Name,_Arity,Cs}, Hook) -> - Cll = nl_clauses(fun (C, H) -> func_clause(Name, C, H) end, $;, Hook, Cs), +lfunction({function,_Line,Name,_Arity,Cs}, Opts) -> + Cll = nl_clauses(fun (C, H) -> func_clause(Name, C, H) end, $;, Opts, Cs), [Cll,leaf(".\n")]. -func_clause(Name, {clause,Line,Head,Guard,Body}, Hook) -> - Hl = call({atom,Line,Name}, Head, 0, Hook), - Gl = guard_when(Hl, Guard, Hook), - Bl = body(Body, Hook), +func_clause(Name, {clause,Line,Head,Guard,Body}, Opts) -> + Hl = call({atom,Line,Name}, Head, 0, Opts), + Gl = guard_when(Hl, Guard, Opts), + Bl = body(Body, Opts), {step,Gl,Bl}. -lrule({rule,_Line,Name,_Arity,Cs}, Hook) -> - Cll = nl_clauses(fun (C, H) -> rule_clause(Name, C, H) end, $;, Hook, Cs), +lrule({rule,_Line,Name,_Arity,Cs}, Opts) -> + Cll = nl_clauses(fun (C, H) -> rule_clause(Name, C, H) end, $;, Opts, Cs), [Cll,leaf(".\n")]. -rule_clause(Name, {clause,Line,Head,Guard,Body}, Hook) -> - Hl = call({atom,Line,Name}, Head, 0, Hook), - Gl = guard_when(Hl, Guard, Hook, leaf(" :-")), - Bl = rule_body(Body, Hook), +rule_clause(Name, {clause,Line,Head,Guard,Body}, Opts) -> + Hl = call({atom,Line,Name}, Head, 0, Opts), + Gl = guard_when(Hl, Guard, Opts, leaf(" :-")), + Bl = rule_body(Body, Opts), {step,Gl,Bl}. -rule_body(Es, Hook) -> - lc_quals(Es, Hook). +rule_body(Es, Opts) -> + lc_quals(Es, Opts). -guard_when(Before, Guard, Hook) -> - guard_when(Before, Guard, Hook, ' ->'). +guard_when(Before, Guard, Opts) -> + guard_when(Before, Guard, Opts, ' ->'). -guard_when(Before, Guard, Hook, After) -> - Gl = lguard(Guard, Hook), +guard_when(Before, Guard, Opts, After) -> + Gl = lguard(Guard, Opts), [{list,[{step,Before,Gl}]},After]. -lguard([E|Es], Hook) when is_list(E) -> - {list,[{step,'when',expr_list([E|Es], [$;], fun guard0/2, Hook)}]}; -lguard([E|Es], Hook) -> % before R6 - lguard([[E|Es]], Hook); +lguard([E|Es], Opts) when is_list(E) -> + {list,[{step,'when',expr_list([E|Es], [$;], fun guard0/2, Opts)}]}; +lguard([E|Es], Opts) -> % before R6 + lguard([[E|Es]], Opts); lguard([], _) -> []. -guard0(Es, Hook) -> - expr_list(Es, [$,], fun lexpr/2, Hook). +guard0(Es, Opts) -> + expr_list(Es, [$,], fun lexpr/2, Opts). -%% body(Before, Es, Hook) -> [Char]. +%% body(Before, Es, Opts) -> [Char]. -body([E], Hook) -> - lexpr(E, Hook); -body(Es, Hook) -> - {prefer_nl,[$,],lexprs(Es, Hook)}. +body([E], Opts) -> + lexpr(E, Opts); +body(Es, Opts) -> + {prefer_nl,[$,],lexprs(Es, Opts)}. -lexpr(E, Hook) -> - lexpr(E, 0, Hook). +lexpr(E, Opts) -> + lexpr(E, 0, Opts). lexpr({var,_,V}, _, _) when is_integer(V) -> %Special hack for Robert leaf(format("_~w", [V])); lexpr({var,_,V}, _, _) -> leaf(format("~s", [V])); -lexpr({char,_,C}, _, _) -> leaf(write_char(C)); +lexpr({char,_,C}, _, _) -> {char,C}; lexpr({integer,_,N}, _, _) -> leaf(write(N)); lexpr({float,_,F}, _, _) -> leaf(write(F)); lexpr({atom,_,A}, _, _) -> leaf(write(A)); lexpr({string,_,S}, _, _) -> {string,S}; lexpr({nil,_}, _, _) -> '[]'; -lexpr({cons,_,H,T}, _, Hook) -> - list(T, [H], Hook); -lexpr({lc,_,E,Qs}, _Prec, Hook) -> - Lcl = {list,[{step,[lexpr(E, Hook),leaf(" ||")],lc_quals(Qs, Hook)}]}, +lexpr({cons,_,H,T}, _, Opts) -> + list(T, [H], Opts); +lexpr({lc,_,E,Qs}, _Prec, Opts) -> + Lcl = {list,[{step,[lexpr(E, Opts),leaf(" ||")],lc_quals(Qs, Opts)}]}, {list,[{seq,$[,[],[[]],[{force_nl,leaf(" "),[Lcl]}]},$]]}; %% {list,[{step,$[,Lcl},$]]}; -lexpr({bc,_,E,Qs}, _Prec, Hook) -> - Lcl = {list,[{step,[lexpr(E, Hook),leaf(" ||")],lc_quals(Qs, Hook)}]}, +lexpr({bc,_,E,Qs}, _Prec, Opts) -> + Lcl = {list,[{step,[lexpr(E, Opts),leaf(" ||")],lc_quals(Qs, Opts)}]}, {list,[{seq,'<<',[],[[]],[{force_nl,leaf(" "),[Lcl]}]},'>>']}; %% {list,[{step,'<<',Lcl},'>>']}; -lexpr({tuple,_,Elts}, _, Hook) -> - tuple(Elts, Hook); -%%lexpr({struct,_,Tag,Elts}, _, Hook) -> -%% {first,format("~w", [Tag]),tuple(Elts, Hook)}; -lexpr({record_index, _, Name, F}, Prec, Hook) -> +lexpr({tuple,_,Elts}, _, Opts) -> + tuple(Elts, Opts); +%%lexpr({struct,_,Tag,Elts}, _, Opts) -> +%% {first,format("~w", [Tag]),tuple(Elts, Opts)}; +lexpr({record_index, _, Name, F}, Prec, Opts) -> {P,R} = preop_prec('#'), Nl = record_name(Name), - El = [Nl,$.,lexpr(F, R, Hook)], + El = [Nl,$.,lexpr(F, R, Opts)], maybe_paren(P, Prec, El); -lexpr({record, _, Name, Fs}, Prec, Hook) -> +lexpr({record, _, Name, Fs}, Prec, Opts) -> {P,_R} = preop_prec('#'), Nl = record_name(Name), - El = {first,Nl,record_fields(Fs, Hook)}, + El = {first,Nl,record_fields(Fs, Opts)}, maybe_paren(P, Prec, El); -lexpr({record_field, _, Rec, Name, F}, Prec, Hook) -> +lexpr({record_field, _, Rec, Name, F}, Prec, Opts) -> {L,P,R} = inop_prec('#'), - Rl = lexpr(Rec, L, Hook), + Rl = lexpr(Rec, L, Opts), Nl = leaf(format("#~w.", [Name])), - El = [Rl,Nl,lexpr(F, R, Hook)], + El = [Rl,Nl,lexpr(F, R, Opts)], maybe_paren(P, Prec, El); -lexpr({record, _, Rec, Name, Fs}, Prec, Hook) -> +lexpr({record, _, Rec, Name, Fs}, Prec, Opts) -> {L,P,_R} = inop_prec('#'), - Rl = lexpr(Rec, L, Hook), + Rl = lexpr(Rec, L, Opts), Nl = record_name(Name), - El = {first,[Rl,Nl],record_fields(Fs, Hook)}, + El = {first,[Rl,Nl],record_fields(Fs, Opts)}, maybe_paren(P, Prec, El); -lexpr({record_field, _, {atom,_,''}, F}, Prec, Hook) -> +lexpr({record_field, _, {atom,_,''}, F}, Prec, Opts) -> {_L,P,R} = inop_prec('.'), - El = [$.,lexpr(F, R, Hook)], + El = [$.,lexpr(F, R, Opts)], maybe_paren(P, Prec, El); -lexpr({record_field, _, Rec, F}, Prec, Hook) -> +lexpr({record_field, _, Rec, F}, Prec, Opts) -> {L,P,R} = inop_prec('.'), - El = [lexpr(Rec, L, Hook),$.,lexpr(F, R, Hook)], + El = [lexpr(Rec, L, Opts),$.,lexpr(F, R, Opts)], maybe_paren(P, Prec, El); -lexpr({block,_,Es}, _, Hook) -> - {list,[{step,'begin',body(Es, Hook)},'end']}; -lexpr({'if',_,Cs}, _, Hook) -> - {list,[{step,'if',if_clauses(Cs, Hook)},'end']}; -lexpr({'case',_,Expr,Cs}, _, Hook) -> - {list,[{step,{list,[{step,'case',lexpr(Expr, Hook)},'of']}, - cr_clauses(Cs, Hook)}, +lexpr({block,_,Es}, _, Opts) -> + {list,[{step,'begin',body(Es, Opts)},'end']}; +lexpr({'if',_,Cs}, _, Opts) -> + {list,[{step,'if',if_clauses(Cs, Opts)},'end']}; +lexpr({'case',_,Expr,Cs}, _, Opts) -> + {list,[{step,{list,[{step,'case',lexpr(Expr, Opts)},'of']}, + cr_clauses(Cs, Opts)}, 'end']}; -lexpr({'cond',_,Cs}, _, Hook) -> - {list,[{step,leaf("cond"),cond_clauses(Cs, Hook)},'end']}; -lexpr({'receive',_,Cs}, _, Hook) -> - {list,[{step,'receive',cr_clauses(Cs, Hook)},'end']}; -lexpr({'receive',_,Cs,To,ToOpt}, _, Hook) -> - Al = {list,[{step,[lexpr(To, Hook),' ->'],body(ToOpt, Hook)}]}, - {list,[{step,'receive',cr_clauses(Cs, Hook)}, +lexpr({'cond',_,Cs}, _, Opts) -> + {list,[{step,leaf("cond"),cond_clauses(Cs, Opts)},'end']}; +lexpr({'receive',_,Cs}, _, Opts) -> + {list,[{step,'receive',cr_clauses(Cs, Opts)},'end']}; +lexpr({'receive',_,Cs,To,ToOpt}, _, Opts) -> + Al = {list,[{step,[lexpr(To, Opts),' ->'],body(ToOpt, Opts)}]}, + {list,[{step,'receive',cr_clauses(Cs, Opts)}, {step,'after',Al}, 'end']}; -lexpr({'fun',_,{function,F,A}}, _Prec, _Hook) -> +lexpr({'fun',_,{function,F,A}}, _Prec, _Opts) -> leaf(format("fun ~w/~w", [F,A])); -lexpr({'fun',_,{function,F,A},Extra}, _Prec, _Hook) -> +lexpr({'fun',_,{function,F,A},Extra}, _Prec, _Opts) -> {force_nl,fun_info(Extra),leaf(format("fun ~w/~w", [F,A]))}; -lexpr({'fun',_,{function,M,F,A}}, _Prec, _Hook) +lexpr({'fun',_,{function,M,F,A}}, _Prec, _Opts) when is_atom(M), is_atom(F), is_integer(A) -> %% For backward compatibility with pre-R15 abstract format. leaf(format("fun ~w:~w/~w", [M,F,A])); -lexpr({'fun',_,{function,M,F,A}}, _Prec, Hook) -> +lexpr({'fun',_,{function,M,F,A}}, _Prec, Opts) -> %% New format in R15. - NameItem = lexpr(M, Hook), - CallItem = lexpr(F, Hook), - ArityItem = lexpr(A, Hook), + NameItem = lexpr(M, Opts), + CallItem = lexpr(F, Opts), + ArityItem = lexpr(A, Opts), ["fun ",NameItem,$:,CallItem,$/,ArityItem]; -lexpr({'fun',_,{clauses,Cs}}, _Prec, Hook) -> - {list,[{first,'fun',fun_clauses(Cs, Hook)},'end']}; -lexpr({'fun',_,{clauses,Cs},Extra}, _Prec, Hook) -> +lexpr({'fun',_,{clauses,Cs}}, _Prec, Opts) -> + {list,[{first,'fun',fun_clauses(Cs, Opts)},'end']}; +lexpr({'fun',_,{clauses,Cs},Extra}, _Prec, Opts) -> {force_nl,fun_info(Extra), - {list,[{first,'fun',fun_clauses(Cs, Hook)},'end']}}; -lexpr({'query',_,Lc}, _Prec, Hook) -> - {list,[{step,leaf("query"),lexpr(Lc, 0, Hook)},'end']}; -lexpr({call,_,{remote,_,{atom,_,M},{atom,_,F}=N}=Name,Args}, Prec, Hook) -> + {list,[{first,'fun',fun_clauses(Cs, Opts)},'end']}}; +lexpr({'query',_,Lc}, _Prec, Opts) -> + {list,[{step,leaf("query"),lexpr(Lc, 0, Opts)},'end']}; +lexpr({call,_,{remote,_,{atom,_,M},{atom,_,F}=N}=Name,Args}, Prec, Opts) -> case erl_internal:bif(M, F, length(Args)) of true -> - call(N, Args, Prec, Hook); + call(N, Args, Prec, Opts); false -> - call(Name, Args, Prec, Hook) + call(Name, Args, Prec, Opts) end; -lexpr({call,_,Name,Args}, Prec, Hook) -> - call(Name, Args, Prec, Hook); -lexpr({'try',_,Es,Scs,Ccs,As}, _, Hook) -> +lexpr({call,_,Name,Args}, Prec, Opts) -> + call(Name, Args, Prec, Opts); +lexpr({'try',_,Es,Scs,Ccs,As}, _, Opts) -> {list,[if Scs =:= [] -> - {step,'try',body(Es, Hook)}; + {step,'try',body(Es, Opts)}; true -> - {step,{list,[{step,'try',body(Es, Hook)},'of']}, - cr_clauses(Scs, Hook)} + {step,{list,[{step,'try',body(Es, Opts)},'of']}, + cr_clauses(Scs, Opts)} end, if Ccs =:= [] -> []; true -> - {step,'catch',try_clauses(Ccs, Hook)} + {step,'catch',try_clauses(Ccs, Opts)} end, if As =:= [] -> []; true -> - {step,'after',body(As, Hook)} + {step,'after',body(As, Opts)} end, 'end']}; -lexpr({'catch',_,Expr}, Prec, Hook) -> +lexpr({'catch',_,Expr}, Prec, Opts) -> {P,R} = preop_prec('catch'), - El = {list,[{step,'catch',lexpr(Expr, R, Hook)}]}, + El = {list,[{step,'catch',lexpr(Expr, R, Opts)}]}, maybe_paren(P, Prec, El); -lexpr({match,_,Lhs,Rhs}, Prec, Hook) -> +lexpr({match,_,Lhs,Rhs}, Prec, Opts) -> {L,P,R} = inop_prec('='), - Pl = lexpr(Lhs, L, Hook), - Rl = lexpr(Rhs, R, Hook), + Pl = lexpr(Lhs, L, Opts), + Rl = lexpr(Rhs, R, Opts), El = {list,[{cstep,[Pl,' ='],Rl}]}, maybe_paren(P, Prec, El); -lexpr({op,_,Op,Arg}, Prec, Hook) -> +lexpr({op,_,Op,Arg}, Prec, Opts) -> {P,R} = preop_prec(Op), Ol = leaf(format("~s ", [Op])), - El = [Ol,lexpr(Arg, R, Hook)], + El = [Ol,lexpr(Arg, R, Opts)], maybe_paren(P, Prec, El); -lexpr({op,_,Op,Larg,Rarg}, Prec, Hook) when Op =:= 'orelse'; +lexpr({op,_,Op,Larg,Rarg}, Prec, Opts) when Op =:= 'orelse'; Op =:= 'andalso' -> %% Breaks lines since R12B. {L,P,R} = inop_prec(Op), - Ll = lexpr(Larg, L, Hook), + Ll = lexpr(Larg, L, Opts), Ol = leaf(format("~s", [Op])), - Lr = lexpr(Rarg, R, Hook), + Lr = lexpr(Rarg, R, Opts), El = {prefer_nl,[[]],[Ll,Ol,Lr]}, maybe_paren(P, Prec, El); -lexpr({op,_,Op,Larg,Rarg}, Prec, Hook) -> +lexpr({op,_,Op,Larg,Rarg}, Prec, Opts) -> {L,P,R} = inop_prec(Op), - Ll = lexpr(Larg, L, Hook), + Ll = lexpr(Larg, L, Opts), Ol = leaf(format("~s", [Op])), - Lr = lexpr(Rarg, R, Hook), + Lr = lexpr(Rarg, R, Opts), El = {list,[Ll,Ol,Lr]}, maybe_paren(P, Prec, El); %% Special expressions which are not really legal everywhere. -lexpr({remote,_,M,F}, Prec, Hook) -> +lexpr({remote,_,M,F}, Prec, Opts) -> {L,P,R} = inop_prec(':'), - NameItem = lexpr(M, L, Hook), - CallItem = lexpr(F, R, Hook), + NameItem = lexpr(M, L, Opts), + CallItem = lexpr(F, R, Opts), maybe_paren(P, Prec, [NameItem,$:,CallItem]); %% BIT SYNTAX: -lexpr({bin,_,Fs}, _, Hook) -> - bit_grp(Fs, Hook); +lexpr({bin,_,Fs}, _, Opts) -> + bit_grp(Fs, Opts); %% Special case for straight values. lexpr({value,_,Val}, _,_) -> leaf(write(Val)); %% Now do the hook. -lexpr(Other, _Precedence, none) -> +lexpr(Other, _Precedence, #options{hook = none}) -> leaf(format("INVALID-FORM:~w:",[Other])); -lexpr(HookExpr, Precedence, {Mod,Func,Eas}) when Mod =/= 'fun' -> +lexpr(HookExpr, Precedence, #options{hook = {Mod,Func,Eas}}) + when Mod =/= 'fun' -> {ehook,HookExpr,Precedence,{Mod,Func,Eas}}; -lexpr(HookExpr, Precedence, Func) -> - {hook,HookExpr,Precedence,Func}. +lexpr(HookExpr, Precedence, #options{hook = Func, opts = Options}) -> + {hook,HookExpr,Precedence,Func,Options}. -call(Name, Args, Prec, Hook) -> +call(Name, Args, Prec, Opts) -> {F,P} = func_prec(), - Item = {first,lexpr(Name, F, Hook),args(Args, Hook)}, + Item = {first,lexpr(Name, F, Opts),args(Args, Opts)}, maybe_paren(P, Prec, Item). fun_info(Extra) -> @@ -565,32 +604,18 @@ fun_info(Extra) -> %% BITS: -bit_grp(Fs, Hook) -> - append([['<<'], - [try - true = Fs =/= [], - S = bin_string(Fs), - true = io_lib:printable_list(S), - {string,S} - catch _:_ -> - bit_elems(Fs, Hook) - end], - ['>>']]). - -bin_string([]) -> - []; -bin_string([{bin_element,_,{char,_,C},_,_}|Bin]) -> - [C | bin_string(Bin)]. +bit_grp(Fs, Opts) -> + append([['<<'], [bit_elems(Fs, Opts)], ['>>']]). -bit_elems(Es, Hook) -> - expr_list(Es, $,, fun bit_elem/2, Hook). +bit_elems(Es, Opts) -> + expr_list(Es, $,, fun bit_elem/2, Opts). -bit_elem({bin_element,_,Expr,Sz,Types}, Hook) -> +bit_elem({bin_element,_,Expr,Sz,Types}, Opts) -> P = max_prec(), - VChars = lexpr(Expr, P, Hook), + VChars = lexpr(Expr, P, Opts), SChars = if Sz =/= default -> - [VChars,$:,lexpr(Sz, P, Hook)]; + [VChars,$:,lexpr(Sz, P, Opts)]; true -> VChars end, @@ -618,157 +643,157 @@ bit_elem_type(T) -> record_name(Name) -> leaf(format("#~w", [Name])). -record_fields(Fs, Hook) -> - tuple(Fs, fun record_field/2, Hook). +record_fields(Fs, Opts) -> + tuple(Fs, fun record_field/2, Opts). -record_field({record_field,_,F,Val}, Hook) -> +record_field({record_field,_,F,Val}, Opts) -> {L,_P,R} = inop_prec('='), - Fl = lexpr(F, L, Hook), - Vl = lexpr(Val, R, Hook), + Fl = lexpr(F, L, Opts), + Vl = lexpr(Val, R, Opts), {list,[{cstep,[Fl,' ='],Vl}]}; -record_field({typed_record_field,{record_field,_,F,Val},Type}, Hook) -> +record_field({typed_record_field,{record_field,_,F,Val},Type}, Opts) -> {L,_P,R} = inop_prec('='), - Fl = lexpr(F, L, Hook), - Vl = typed(lexpr(Val, R, Hook), Type), + Fl = lexpr(F, L, Opts), + Vl = typed(lexpr(Val, R, Opts), Type), {list,[{cstep,[Fl,' ='],Vl}]}; -record_field({typed_record_field,Field,Type}, Hook) -> - typed(record_field(Field, Hook), Type); -record_field({record_field,_,F}, Hook) -> - lexpr(F, 0, Hook). - -list({cons,_,H,T}, Es, Hook) -> - list(T, [H|Es], Hook); -list({nil,_}, Es, Hook) -> - proper_list(reverse(Es), Hook); -list(Other, Es, Hook) -> - improper_list(reverse(Es, [Other]), Hook). - -%% if_clauses(Clauses, Hook) -> [Char]. +record_field({typed_record_field,Field,Type}, Opts) -> + typed(record_field(Field, Opts), Type); +record_field({record_field,_,F}, Opts) -> + lexpr(F, 0, Opts). + +list({cons,_,H,T}, Es, Opts) -> + list(T, [H|Es], Opts); +list({nil,_}, Es, Opts) -> + proper_list(reverse(Es), Opts); +list(Other, Es, Opts) -> + improper_list(reverse(Es, [Other]), Opts). + +%% if_clauses(Clauses, Opts) -> [Char]. %% Print 'if' clauses. -if_clauses(Cs, Hook) -> - clauses(fun if_clause/2, Hook, Cs). +if_clauses(Cs, Opts) -> + clauses(fun if_clause/2, Opts, Cs). -if_clause({clause,_,[],G,B}, Hook) -> - Gl = [guard_no_when(G, Hook),' ->'], - {step,Gl,body(B, Hook)}. +if_clause({clause,_,[],G,B}, Opts) -> + Gl = [guard_no_when(G, Opts),' ->'], + {step,Gl,body(B, Opts)}. -guard_no_when([E|Es], Hook) when is_list(E) -> - expr_list([E|Es], $;, fun guard0/2, Hook); -guard_no_when([E|Es], Hook) -> % before R6 - guard_no_when([[E|Es]], Hook); +guard_no_when([E|Es], Opts) when is_list(E) -> + expr_list([E|Es], $;, fun guard0/2, Opts); +guard_no_when([E|Es], Opts) -> % before R6 + guard_no_when([[E|Es]], Opts); guard_no_when([], _) -> % cannot happen leaf("true"). -%% cr_clauses(Clauses, Hook) -> [Char]. +%% cr_clauses(Clauses, Opts) -> [Char]. %% Print 'case'/'receive' clauses. -cr_clauses(Cs, Hook) -> - clauses(fun cr_clause/2, Hook, Cs). +cr_clauses(Cs, Opts) -> + clauses(fun cr_clause/2, Opts, Cs). -cr_clause({clause,_,[T],G,B}, Hook) -> - El = lexpr(T, 0, Hook), - Gl = guard_when(El, G, Hook), - Bl = body(B, Hook), +cr_clause({clause,_,[T],G,B}, Opts) -> + El = lexpr(T, 0, Opts), + Gl = guard_when(El, G, Opts), + Bl = body(B, Opts), {step,Gl,Bl}. -%% try_clauses(Clauses, Hook) -> [Char]. +%% try_clauses(Clauses, Opts) -> [Char]. %% Print 'try' clauses. -try_clauses(Cs, Hook) -> - clauses(fun try_clause/2, Hook, Cs). +try_clauses(Cs, Opts) -> + clauses(fun try_clause/2, Opts, Cs). -try_clause({clause,_,[{tuple,_,[{atom,_,throw},V,S]}],G,B}, Hook) -> - El = lexpr(V, 0, Hook), - Sl = stack_backtrace(S, [El], Hook), - Gl = guard_when(Sl, G, Hook), - Bl = body(B, Hook), +try_clause({clause,_,[{tuple,_,[{atom,_,throw},V,S]}],G,B}, Opts) -> + El = lexpr(V, 0, Opts), + Sl = stack_backtrace(S, [El], Opts), + Gl = guard_when(Sl, G, Opts), + Bl = body(B, Opts), {step,Gl,Bl}; -try_clause({clause,_,[{tuple,_,[C,V,S]}],G,B}, Hook) -> - Cs = lexpr(C, 0, Hook), - El = lexpr(V, 0, Hook), +try_clause({clause,_,[{tuple,_,[C,V,S]}],G,B}, Opts) -> + Cs = lexpr(C, 0, Opts), + El = lexpr(V, 0, Opts), CsEl = [Cs,$:,El], - Sl = stack_backtrace(S, CsEl, Hook), - Gl = guard_when(Sl, G, Hook), - Bl = body(B, Hook), + Sl = stack_backtrace(S, CsEl, Opts), + Gl = guard_when(Sl, G, Opts), + Bl = body(B, Opts), {step,Gl,Bl}. -stack_backtrace({var,_,'_'}, El, _Hook) -> +stack_backtrace({var,_,'_'}, El, _Opts) -> El; -stack_backtrace(S, El, Hook) -> - El++[$:,lexpr(S, 0, Hook)]. +stack_backtrace(S, El, Opts) -> + El++[$:,lexpr(S, 0, Opts)]. -%% fun_clauses(Clauses, Hook) -> [Char]. +%% fun_clauses(Clauses, Opts) -> [Char]. %% Print 'fun' clauses. -fun_clauses(Cs, Hook) -> - nl_clauses(fun fun_clause/2, [$;], Hook, Cs). +fun_clauses(Cs, Opts) -> + nl_clauses(fun fun_clause/2, [$;], Opts, Cs). -fun_clause({clause,_,A,G,B}, Hook) -> - El = args(A, Hook), - Gl = guard_when(El, G, Hook), - Bl = body(B, Hook), +fun_clause({clause,_,A,G,B}, Opts) -> + El = args(A, Opts), + Gl = guard_when(El, G, Opts), + Bl = body(B, Opts), {step,Gl,Bl}. -%% cond_clauses(Clauses, Hook) -> [Char]. +%% cond_clauses(Clauses, Opts) -> [Char]. %% Print 'cond' clauses. -cond_clauses(Cs, Hook) -> - clauses(fun cond_clause/2, Hook, Cs). +cond_clauses(Cs, Opts) -> + clauses(fun cond_clause/2, Opts, Cs). -cond_clause({clause,_,[],[[E]],B}, Hook) -> - {step,[lexpr(E, Hook),' ->'],body(B, Hook)}. +cond_clause({clause,_,[],[[E]],B}, Opts) -> + {step,[lexpr(E, Opts),' ->'],body(B, Opts)}. -%% nl_clauses(Type, Hook, Clauses) -> [Char]. +%% nl_clauses(Type, Opts, Clauses) -> [Char]. %% Generic clause printing function (always breaks lines). -nl_clauses(Type, Sep, Hook, Cs) -> - {prefer_nl,Sep,lexprs(Cs, Type, Hook)}. +nl_clauses(Type, Sep, Opts, Cs) -> + {prefer_nl,Sep,lexprs(Cs, Type, Opts)}. -%% clauses(Type, Hook, Clauses) -> [Char]. +%% clauses(Type, Opts, Clauses) -> [Char]. %% Generic clause printing function (breaks lines since R12B). -clauses(Type, Hook, Cs) -> - {prefer_nl,[$;],lexprs(Cs, Type, Hook)}. +clauses(Type, Opts, Cs) -> + {prefer_nl,[$;],lexprs(Cs, Type, Opts)}. -%% lc_quals(Qualifiers, After, Hook) +%% lc_quals(Qualifiers, After, Opts) %% List comprehension qualifiers (breaks lines since R12B). -lc_quals(Qs, Hook) -> - {prefer_nl,[$,],lexprs(Qs, fun lc_qual/2, Hook)}. +lc_quals(Qs, Opts) -> + {prefer_nl,[$,],lexprs(Qs, fun lc_qual/2, Opts)}. -lc_qual({b_generate,_,Pat,E}, Hook) -> - Pl = lexpr(Pat, 0, Hook), - {list,[{step,[Pl,leaf(" <=")],lexpr(E, 0, Hook)}]}; -lc_qual({generate,_,Pat,E}, Hook) -> - Pl = lexpr(Pat, 0, Hook), - {list,[{step,[Pl,leaf(" <-")],lexpr(E, 0, Hook)}]}; -lc_qual(Q, Hook) -> - lexpr(Q, 0, Hook). +lc_qual({b_generate,_,Pat,E}, Opts) -> + Pl = lexpr(Pat, 0, Opts), + {list,[{step,[Pl,leaf(" <=")],lexpr(E, 0, Opts)}]}; +lc_qual({generate,_,Pat,E}, Opts) -> + Pl = lexpr(Pat, 0, Opts), + {list,[{step,[Pl,leaf(" <-")],lexpr(E, 0, Opts)}]}; +lc_qual(Q, Opts) -> + lexpr(Q, 0, Opts). -proper_list(Es, Hook) -> - {seq,$[,$],$,,lexprs(Es, Hook)}. +proper_list(Es, Opts) -> + {seq,$[,$],$,,lexprs(Es, Opts)}. -improper_list(Es, Hook) -> - {seq,$[,$],{$,,$|},lexprs(Es, Hook)}. +improper_list(Es, Opts) -> + {seq,$[,$],{$,,$|},lexprs(Es, Opts)}. -tuple(L, Hook) -> - tuple(L, fun lexpr/2, Hook). +tuple(L, Opts) -> + tuple(L, fun lexpr/2, Opts). -tuple(Es, F, Hook) -> - {seq,${,$},$,,lexprs(Es, F, Hook)}. +tuple(Es, F, Opts) -> + {seq,${,$},$,,lexprs(Es, F, Opts)}. -args(As, Hook) -> - {seq,$(,$),[$,],lexprs(As, Hook)}. +args(As, Opts) -> + {seq,$(,$),[$,],lexprs(As, Opts)}. -expr_list(Es, Sep, F, Hook) -> - {seq,[],[],Sep,lexprs(Es, F, Hook)}. +expr_list(Es, Sep, F, Opts) -> + {seq,[],[],Sep,lexprs(Es, F, Opts)}. -lexprs(Es, Hook) -> - lexprs(Es, fun lexpr/2, Hook). +lexprs(Es, Opts) -> + lexprs(Es, fun lexpr/2, Opts). -lexprs(Es, F, Hook) -> - [F(E, Hook) || E <- Es]. +lexprs(Es, F, Opts) -> + [F(E, Opts) || E <- Es]. maybe_paren(P, Prec, Expr) when P < Prec -> [$(,Expr,$)]; @@ -781,13 +806,13 @@ leaf(S) -> %%% Do the formatting. Currently nothing fancy. Could probably have %%% done it in one single pass. -frmt(Item) -> - frmt(Item, 0). +frmt(Item, PP) -> + frmt(Item, 0, PP). -frmt(Item, I) -> +frmt(Item, I, PP) -> ST = spacetab(), WT = wordtable(), - {Chars,_Length} = f(Item, I, ST, WT), + {Chars,_Length} = f(Item, I, ST, WT, PP), [Chars]. %%% What the tags mean: @@ -803,6 +828,7 @@ frmt(Item, I) -> %%% - {force_nl,ExtraInfo,I}: fun-info (a comment) forces linebreak before I. %%% - {prefer_nl,Sep,IPs}: forces linebreak between Is unlesss negative %%% indentation. +%%% - {char,C}: a character %%% - {string,S}: a string. %%% - {hook,...}, {ehook,...}: hook expressions. %%% @@ -812,22 +838,22 @@ frmt(Item, I) -> %%% cstep works similarly, but no linebreak if the width of I1 is less %%% than the indentation (this is for "A = <expression over several lines>). -f([]=Nil, _I0, _ST, _WT) -> +f([]=Nil, _I0, _ST, _WT, _PP) -> {Nil,0}; -f(C, _I0, _ST, _WT) when is_integer(C) -> +f(C, _I0, _ST, _WT, _PP) when is_integer(C) -> {C,1}; -f({leaf,Length,Chars}, _I0, _ST, _WT) -> +f({leaf,Length,Chars}, _I0, _ST, _WT, _PP) -> {Chars,Length}; -f([Item|Items], I0, ST, WT) -> - consecutive(Items, f(Item, I0, ST, WT), I0, ST, WT); -f({list,Items}, I0, ST, WT) -> - f({seq,[],[],[[]],Items}, I0, ST, WT); -f({first,E,Item}, I0, ST, WT) -> - f({seq,E,[],[[]],[Item]}, I0, ST, WT); -f({seq,Before,After,Sep,LItems}, I0, ST, WT) -> - BCharsSize = f(Before, I0, ST, WT), +f([Item|Items], I0, ST, WT, PP) -> + consecutive(Items, f(Item, I0, ST, WT, PP), I0, ST, WT, PP); +f({list,Items}, I0, ST, WT, PP) -> + f({seq,[],[],[[]],Items}, I0, ST, WT, PP); +f({first,E,Item}, I0, ST, WT, PP) -> + f({seq,E,[],[[]],[Item]}, I0, ST, WT, PP); +f({seq,Before,After,Sep,LItems}, I0, ST, WT, PP) -> + BCharsSize = f(Before, I0, ST, WT, PP), I = indent(BCharsSize, I0), - CharsSizeL = fl(LItems, Sep, I, After, ST, WT), + CharsSizeL = fl(LItems, Sep, I, After, ST, WT, PP), {CharsL,SizeL} = unz(CharsSizeL), {BCharsL,BSizeL} = unz1([BCharsSize]), Sizes = BSizeL ++ SizeL, @@ -848,15 +874,15 @@ f({seq,Before,After,Sep,LItems}, I0, ST, WT) -> {BCharsL++insert_newlines(CharsSizeL, I, ST), nsz(lists:last(Sizes), I0)} end; -f({force_nl,_ExtraInfoItem,Item}, I, ST, WT) when I < 0 -> +f({force_nl,_ExtraInfoItem,Item}, I, ST, WT, PP) when I < 0 -> %% Extra info is a comment; cannot have that on the same line - f(Item, I, ST, WT); -f({force_nl,ExtraInfoItem,Item}, I, ST, WT) -> - f({prefer_nl,[],[ExtraInfoItem,Item]}, I, ST, WT); -f({prefer_nl,Sep,LItems}, I, ST, WT) when I < 0 -> - f({seq,[],[],Sep,LItems}, I, ST, WT); -f({prefer_nl,Sep,LItems}, I0, ST, WT) -> - CharsSize2L = fl(LItems, Sep, I0, [], ST, WT), + f(Item, I, ST, WT, PP); +f({force_nl,ExtraInfoItem,Item}, I, ST, WT, PP) -> + f({prefer_nl,[],[ExtraInfoItem,Item]}, I, ST, WT, PP); +f({prefer_nl,Sep,LItems}, I, ST, WT, PP) when I < 0 -> + f({seq,[],[],Sep,LItems}, I, ST, WT, PP); +f({prefer_nl,Sep,LItems}, I0, ST, WT, PP) -> + CharsSize2L = fl(LItems, Sep, I0, [], ST, WT, PP), {_CharsL,Sizes} = unz(CharsSize2L), if Sizes =:= [] -> @@ -864,37 +890,40 @@ f({prefer_nl,Sep,LItems}, I0, ST, WT) -> true -> {insert_newlines(CharsSize2L, I0, ST),nsz(lists:last(Sizes), I0)} end; -f({string,S}, I, ST, WT) -> - f(write_a_string(S, I), I, ST, WT); -f({hook,HookExpr,Precedence,Func}, I, _ST, _WT) -> - Chars = Func(HookExpr, I, Precedence, Func), +f({char,C}, I, ST, WT, PP) -> + f(write_a_char(C, PP), I, ST, WT, PP); +f({string,S}, I, ST, WT, PP) -> + f(write_a_string(S, I, PP), I, ST, WT, PP); +f({hook,HookExpr,Precedence,Func,Options}, I, _ST, _WT, _PP) -> + Chars = Func(HookExpr, I, Precedence, Options), {Chars,indentation(Chars, I)}; -f({ehook,HookExpr,Precedence,{Mod,Func,Eas}=ModFuncEas}, I, _ST, _WT) -> +f({ehook,HookExpr,Precedence,{Mod,Func,Eas}=ModFuncEas}, I, _ST, _WT, _PP) -> Chars = apply(Mod, Func, [HookExpr,I,Precedence,ModFuncEas|Eas]), {Chars,indentation(Chars, I)}; -f(WordName, _I, _ST, WT) -> % when is_atom(WordName) +f(WordName, _I, _ST, WT, _PP) -> % when is_atom(WordName) word(WordName, WT). -define(IND, 4). %% fl(ListItems, I0, ST, WT) -> [[CharsSize1,CharsSize2]] %% ListItems = [{Item,Items}|Item] -fl([], _Sep, I0, After, ST, WT) -> - [[f(After, I0, ST, WT),{[],0}]]; -fl(CItems, Sep0, I0, After, ST, WT) -> +fl([], _Sep, I0, After, ST, WT, PP) -> + [[f(After, I0, ST, WT, PP),{[],0}]]; +fl(CItems, Sep0, I0, After, ST, WT, PP) -> F = fun({step,Item1,Item2}, S) -> - [f(Item1, I0, ST, WT),f([Item2,S], incr(I0, ?IND), ST, WT)]; + [f(Item1, I0, ST, WT, PP), + f([Item2,S], incr(I0, ?IND), ST, WT, PP)]; ({cstep,Item1,Item2}, S) -> - {_,Sz1} = CharSize1 = f(Item1, I0, ST, WT), + {_,Sz1} = CharSize1 = f(Item1, I0, ST, WT, PP), if is_integer(Sz1), Sz1 < ?IND -> Item2p = [leaf("\s"),Item2,S], - [consecutive(Item2p, CharSize1, I0, ST, WT),{[],0}]; + [consecutive(Item2p, CharSize1, I0, ST, WT, PP),{[],0}]; true -> - [CharSize1,f([Item2,S], incr(I0, ?IND), ST, WT)] + [CharSize1,f([Item2,S], incr(I0, ?IND), ST, WT, PP)] end; (Item, S) -> - [f([Item,S], I0, ST, WT),{[],0}] + [f([Item,S], I0, ST, WT, PP),{[],0}] end, {Sep,LastSep} = case Sep0 of {_,_} -> Sep0; _ -> {Sep0,Sep0} end, fl1(CItems, F, Sep, LastSep, After). @@ -906,10 +935,10 @@ fl1([CItem1,CItem2], F, _Sep, LastSep, After) -> fl1([CItem|CItems], F, Sep, LastSep, After) -> [F(CItem, Sep)|fl1(CItems, F, Sep, LastSep, After)]. -consecutive(Items, CharSize1, I0, ST, WT) -> +consecutive(Items, CharSize1, I0, ST, WT, PP) -> {CharsSizes,_Length} = mapfoldl(fun(Item, Len) -> - CharsSize = f(Item, Len, ST, WT), + CharsSize = f(Item, Len, ST, WT, PP), {CharsSize,indent(CharsSize, Len)} end, indent(CharSize1, I0), Items), {CharsL,SizeL} = unz1([CharSize1|CharsSizes]), @@ -999,26 +1028,40 @@ has_nl([C|Cs]) -> has_nl([]) -> false. +write_a_char(C, PP) -> + flat_leaf(write_char(C, PP)). + -define(MIN_SUBSTRING, 5). -write_a_string(S, I) when I < 0; S =:= [] -> - leaf(write_string(S)); -write_a_string(S, I) -> +write_a_string(S, I, PP) when I < 0; S =:= [] -> + flat_leaf(write_string(S, PP)); +write_a_string(S, I, PP) -> Len = erlang:max(?MAXLINE-I, ?MIN_SUBSTRING), - {list,write_a_string(S, Len, Len)}. + {list,write_a_string(S, Len, Len, PP)}. -write_a_string([], _N, _Len) -> +write_a_string([], _N, _Len, _PP) -> []; -write_a_string(S, N, Len) -> +write_a_string(S, N, Len, PP) -> SS = string:sub_string(S, 1, N), - Sl = write_string(SS), - case (iolist_size(Sl) > Len) and (N > ?MIN_SUBSTRING) of + Sl = write_string(SS, PP), + case (length(Sl) > Len) and (N > ?MIN_SUBSTRING) of true -> - write_a_string(S, N-1, Len); + write_a_string(S, N-1, Len, PP); false -> - [leaf(Sl)|write_a_string(lists:nthtail(length(SS), S), Len, Len)] + [flat_leaf(Sl) | + write_a_string(lists:nthtail(length(SS), S), Len, Len, PP)] end. +flat_leaf(S) -> + L = lists:flatten(S), + {leaf,length(L),L}. + +write_string(S, PP) -> + lists:flatten((PP#pp.string_fun)(S)). + +write_char(C, PP) -> + lists:flatten((PP#pp.char_fun)(C)). + %% %% Utilities %% diff --git a/lib/stdlib/src/erl_scan.erl b/lib/stdlib/src/erl_scan.erl index 0c8735bb6d..e5bb287c45 100644 --- a/lib/stdlib/src/erl_scan.erl +++ b/lib/stdlib/src/erl_scan.erl @@ -1,3 +1,4 @@ +%% -*- coding: utf-8 -*- %% %% %CopyrightBegin% %% @@ -32,19 +33,19 @@ %% 173 - 176 { - ~ punctuation %% 177 DEL control %% 200 - 237 control -%% 240 - 277 NBSP - � punctuation -%% 300 - 326 � - � uppercase -%% 327 � punctuation -%% 330 - 336 � - � uppercase -%% 337 - 366 � - � lowercase -%% 367 � punctuation -%% 370 - 377 � - � lowercase +%% 240 - 277 NBSP - ¿ punctuation +%% 300 - 326 À - Ö uppercase +%% 327 × punctuation +%% 330 - 336 Ø - Þ uppercase +%% 337 - 366 ß - ö lowercase +%% 367 ÷ punctuation +%% 370 - 377 ø - ÿ lowercase %% %% Many punctuation characters have special meaning: %% $\s, $_, $", $$, $%, $', $. %% DEL is a punctuation. %% -%% Must watch using � \327, very close to x \170. +%% Must watch using × \327, very close to x \170. -module(erl_scan). @@ -55,7 +56,15 @@ token_info/1,token_info/2, attributes_info/1,attributes_info/2,set_attribute/3]). --export_type([error_info/0, line/0, return_cont/0, tokens_result/0]). +%%% Private +-export([continuation_location/1]). + +-export_type([error_info/0, + line/0, + location/0, + options/0, + return_cont/0, + tokens_result/0]). %%% %%% Defines and type definitions @@ -74,7 +83,8 @@ -type location() :: line() | {line(),column()}. -type resword_fun() :: fun((atom()) -> boolean()). -type option() :: 'return' | 'return_white_spaces' | 'return_comments' - | 'text' | {'reserved_word_fun', resword_fun()}. + | 'text' | {'reserved_word_fun', resword_fun()} + | 'unicode'. -type options() :: option() | [option()]. -type symbol() :: atom() | float() | integer() | string(). -type info_line() :: integer() | term(). @@ -95,7 +105,8 @@ {resword_fun = fun reserved_word/1 :: resword_fun(), ws = false :: boolean(), comment = false :: boolean(), - text = false :: boolean()}). + text = false :: boolean(), + unicode = false :: boolean()}). %%---------------------------------------------------------------------------- @@ -183,6 +194,11 @@ tokens({erl_scan_continuation,Cs,Col,Toks,Line,St,Any,Fun}, CharSpec, _Loc, _Opts) -> tokens1(Cs++CharSpec, St, Line, Col, Toks, Fun, Any). +continuation_location({erl_scan_continuation,_,no_col,_,Line,_,_,_}) -> + Line; +continuation_location({erl_scan_continuation,_,Col,_,Line,_,_,_}) -> + {Line,Col}. + -type attribute_item() :: 'column' | 'length' | 'line' | 'location' | 'text'. -type info_location() :: location() | term(). @@ -201,15 +217,14 @@ token_info(Token) -> Items = [category,column,length,line,symbol,text], % undefined order token_info(Token, Items). --spec token_info(Token, TokenItem) -> TokenInfo | 'undefined' when - Token :: token(), - TokenItem :: token_item(), - TokenInfo :: TokenInfoTuple :: token_info(); - (Token, TokenItems) -> [TokenInfo] when - Token :: token(), - TokenItems :: [TokenItem], - TokenItem :: token_item(), - TokenInfo :: [TokenInfoTuple :: token_info()]. +-spec token_info(Token, TokenItem) -> TokenInfoTuple | 'undefined' when + Token :: token(), + TokenItem :: token_item(), + TokenInfoTuple :: token_info(); + (Token, TokenItems) -> TokenInfo when + Token :: token(), + TokenItems :: [TokenItem :: token_item()], + TokenInfo :: [TokenInfoTuple :: token_info()]. token_info(_Token, []) -> []; token_info(Token, [Item|Items]) when is_atom(Item) -> @@ -239,16 +254,15 @@ attributes_info(Attributes) -> Items = [column,length,line,text], % undefined order attributes_info(Attributes, Items). --spec attributes_info(Attributes, AttributeItem) -> - AttributeInfo | 'undefined' when - Attributes :: attributes(), - AttributeItem :: attribute_item(), - AttributeInfo :: AttributeInfoTuple :: attribute_info(); - (Attributes, AttributeItems) -> [AttributeInfo] when - Attributes :: attributes(), - AttributeItems :: [AttributeItem], - AttributeItem :: attribute_item(), - AttributeInfo :: [AttributeInfoTuple :: attribute_info()]. +-spec attributes_info + (Attributes, AttributeItem) -> AttributeInfoTuple | 'undefined' when + Attributes :: attributes(), + AttributeItem :: attribute_item(), + AttributeInfoTuple :: attribute_info(); + (Attributes, AttributeItems) -> AttributeInfo when + Attributes :: attributes(), + AttributeItems :: [AttributeItem :: attribute_item()], + AttributeInfo :: [AttributeInfoTuple :: attribute_info()]. attributes_info(_Attrs, []) -> []; attributes_info(Attrs, [A|As]) when is_atom(A) -> @@ -324,13 +338,20 @@ string_thing(_) -> "string". (C >= $\000 andalso C =< $\s orelse C >= $\200 andalso C =< $\240)). -define(DIGIT(C), C >= $0, C =< $9). -define(CHAR(C), is_integer(C), C >= 0). - -%% A workaround: Unicode strings are not returned as strings, but as -%% lists of integers. For instance, "b\x{aaa}c" => [98,2730,99]. This -%% is to protect the system from character codes greater than 255. To -%% be removed. Search for UNI to find workaround code. +-define(UNICODE(C), + (C >= 0 andalso C < 16#D800 orelse + C > 16#DFFF andalso C < 16#FFFE orelse + C > 16#FFFF andalso C =< 16#10FFFF)). + +%% When the option 'unicode' is false: return Unicode strings as lists +%% of integers and Unicode characters as integers. For instance, +%% erl_scan:string("\"b\x{aaa}c\".") is equivalent to +%% erl_scan:string("[98,2730,99]."). This is to protect the caller +%% from character codes greater than 255. Search for UNI to find code +%% implementing this "feature". The 'unicode' option is undocumented +%% and will probably be removed later. -define(NO_UNICODE, 0). --define(UNI255(C), (C) =< 16#ff). +-define(UNI255(C), (C =< 16#ff)). options(Opts0) when is_list(Opts0) -> Opts = lists:foldr(fun expand_opt/2, [], Opts0), @@ -344,10 +365,12 @@ options(Opts0) when is_list(Opts0) -> Comment = proplists:get_bool(return_comments, Opts), WS = proplists:get_bool(return_white_spaces, Opts), Txt = proplists:get_bool(text, Opts), + Unicode = proplists:get_bool(unicode, Opts), #erl_scan{resword_fun = RW_fun, comment = Comment, ws = WS, - text = Txt}; + text = Txt, + unicode = Unicode}; options(Opt) -> options([Opt]). @@ -513,9 +536,9 @@ scan1([$$|Cs], St, Line, Col, Toks) -> scan_char(Cs, St, Line, Col, Toks); scan1([$\r|Cs], St, Line, Col, Toks) when St#erl_scan.ws -> white_space_end(Cs, St, Line, Col, Toks, 1, "\r"); -scan1([C|Cs], St, Line, Col, Toks) when C >= $�, C =< $�, C =/= $� -> +scan1([C|Cs], St, Line, Col, Toks) when C >= $ß, C =< $ÿ, C =/= $÷ -> scan_atom(Cs, St, Line, Col, Toks, [C]); -scan1([C|Cs], St, Line, Col, Toks) when C >= $�, C =< $�, C /= $� -> +scan1([C|Cs], St, Line, Col, Toks) when C >= $À, C =< $Þ, C /= $× -> scan_variable(Cs, St, Line, Col, Toks, [C]); scan1([$\t|Cs], St, Line, Col, Toks) when St#erl_scan.ws -> scan_tabs(Cs, St, Line, Col, Toks, 1); @@ -628,15 +651,12 @@ scan1([$~|Cs], St, Line, Col, Toks) -> scan1([$&|Cs], St, Line, Col, Toks) -> tok2(Cs, St, Line, Col, Toks, "&", '&', 1); %% End of optimization. -scan1([C|Cs], St, Line, Col, Toks) when ?CHAR(C) -> +scan1([C|Cs], St, Line, Col, Toks) when ?CHAR(C), ?UNI255(C) -> Str = [C], - case catch list_to_atom(Str) of - Sym when is_atom(Sym) -> - tok2(Cs, St, Line, Col, Toks, Str, Sym, 1); - _ -> - Ncol = incr_column(Col, 1), - scan_error({illegal,character}, Line, Col, Line, Ncol, Cs) - end; + tok2(Cs, St, Line, Col, Toks, Str, list_to_atom(Str), 1); +scan1([C|Cs], _St, Line, Col, _Toks) when ?CHAR(C) -> + Ncol = incr_column(Col, 1), + scan_error({illegal,character}, Line, Col, Line, Ncol, Cs); scan1([]=Cs, _St, Line, Col, Toks) -> {more,{Cs,Col,Toks,Line,[],fun scan/6}}; scan1(eof=Cs, _St, Line, Col, Toks) -> @@ -685,9 +705,9 @@ scan_name([C|Cs], Ncs) when ?DIGIT(C) -> scan_name(Cs, [C|Ncs]); scan_name([$@=C|Cs], Ncs) -> scan_name(Cs, [C|Ncs]); -scan_name([C|Cs], Ncs) when C >= $�, C =< $�, C =/= $� -> +scan_name([C|Cs], Ncs) when C >= $ß, C =< $ÿ, C =/= $÷ -> scan_name(Cs, [C|Ncs]); -scan_name([C|Cs], Ncs) when C >= $�, C =< $�, C =/= $� -> +scan_name([C|Cs], Ncs) when C >= $À, C =< $Þ, C =/= $× -> scan_name(Cs, [C|Ncs]); scan_name([], Ncs) -> {more,Ncs}; @@ -834,32 +854,44 @@ scan_char([$\\|Cs]=Cs0, St, Line, Col, Toks) -> {eof,Ncol} -> scan_error(char, Line, Col, Line, Ncol, eof); {nl,Val,Str,Ncs,Ncol} -> - Attrs = attributes(Line, Col, St, "$\\"++Str), + Attrs = attributes(Line, Col, St, "$\\"++Str), %" Ntoks = [{char,Attrs,Val}|Toks], scan1(Ncs, St, Line+1, Ncol, Ntoks); {unicode,Val,Str,Ncs,Ncol} -> - Attrs = attributes(Line, Col, St, "$\\"++Str), - Ntoks = [{integer,Attrs,Val}|Toks], % UNI + Attrs = attributes(Line, Col, St, "$\\"++Str), %" + Tag = char_tag(Val, St), % UNI + Ntoks = [{Tag,Attrs,Val}|Toks], scan1(Ncs, St, Line, Ncol, Ntoks); {Val,Str,Ncs,Ncol} -> - Attrs = attributes(Line, Col, St, "$\\"++Str), + Attrs = attributes(Line, Col, St, "$\\"++Str), %" Ntoks = [{char,Attrs,Val}|Toks], scan1(Ncs, St, Line, Ncol, Ntoks) end; scan_char([$\n=C|Cs], St, Line, Col, Toks) -> Attrs = attributes(Line, Col, St, [$$,C]), scan1(Cs, St, Line+1, new_column(Col, 1), [{char,Attrs,C}|Toks]); -scan_char([C|Cs], St, Line, Col, Toks) when ?CHAR(C) -> - Tag = if ?UNI255(C) -> char; true -> integer end, % UNI +scan_char([C|Cs], St, Line, Col, Toks) when ?UNICODE(C) -> + Tag = char_tag(C, St), % UNI Attrs = attributes(Line, Col, St, [$$,C]), scan1(Cs, St, Line, incr_column(Col, 2), [{Tag,Attrs,C}|Toks]); +scan_char([C|_Cs], _St, Line, Col, _Toks) when ?CHAR(C) -> + scan_error({illegal,character}, Line, Col, Line, incr_column(Col, 1), eof); scan_char([], _St, Line, Col, Toks) -> {more,{[$$],Col,Toks,Line,[],fun scan/6}}; scan_char(eof, _St, Line, Col, _Toks) -> scan_error(char, Line, Col, Line, incr_column(Col, 1), eof). +-compile({inline,[char_tag/2]}). + +char_tag(C, _St) when ?UNI255(C) -> + char; +char_tag(_C, #erl_scan{unicode = true}) -> + char; +char_tag(_C, _St) -> + integer. + scan_string(Cs, St, Line, Col, Toks, {Wcs,Str,Line0,Col0,Uni0}) -> - case scan_string0(Cs, St, Line, Col, $\", Str, Wcs, Uni0) of + case scan_string0(Cs, St, Line, Col, $\", true, Str, Wcs, Uni0) of %" {more,Ncs,Nline,Ncol,Nstr,Nwcs,Uni} -> State = {Nwcs,Nstr,Line0,Col0,Uni}, {more,{Ncs,Ncol,Toks,Nline,State,fun scan_string/6}}; @@ -867,8 +899,9 @@ scan_string(Cs, St, Line, Col, Toks, {Wcs,Str,Line0,Col0,Uni0}) -> scan_error(Error, Nline, Ncol, Nline, EndCol, Ncs); {error,Nline,Ncol,Nwcs,Ncs} -> Estr = string:substr(Nwcs, 1, 16), % Expanded escape chars. - scan_error({string,$\",Estr}, Line0, Col0, Nline, Ncol, Ncs); - {Ncs,Nline,Ncol,Nstr,Nwcs,?NO_UNICODE} -> + scan_error({string,$\",Estr}, Line0, Col0, Nline, Ncol, Ncs); %" + {Ncs,Nline,Ncol,Nstr,Nwcs,Uni} when Uni =:= ?NO_UNICODE; + St#erl_scan.unicode -> Attrs = attributes(Line0, Col0, St, Nstr), scan1(Ncs, St, Nline, Ncol, [{string,Attrs,Nwcs}|Toks]); {Ncs,Nline,Ncol,Nstr,_Nwcs,_Uni} -> @@ -920,7 +953,8 @@ unicode_tokens(Line, Col, Str, Val, St, Toks, Cs, Cline, Ccol) -> [{',',attributes(Cline, Ccol, St, "")} || Cs =/= "\""] ++ [Token|Toks]. scan_qatom(Cs, St, Line, Col, Toks, {Wcs,Str,Line0,Col0,Uni0}) -> - case scan_string0(Cs, St, Line, Col, $\', Str, Wcs, Uni0) of + AllowUni = St#erl_scan.unicode, + case scan_string0(Cs, St, Line, Col, $\', AllowUni, Str, Wcs, Uni0) of %' {more,Ncs,Nline,Ncol,Nstr,Nwcs,Uni} -> State = {Nwcs,Nstr,Line0,Col0,Uni}, {more,{Ncs,Ncol,Toks,Nline,State,fun scan_qatom/6}}; @@ -928,8 +962,9 @@ scan_qatom(Cs, St, Line, Col, Toks, {Wcs,Str,Line0,Col0,Uni0}) -> scan_error(Error, Nline, Ncol, Nline, EndCol, Ncs); {error,Nline,Ncol,Nwcs,Ncs} -> Estr = string:substr(Nwcs, 1, 16), % Expanded escape chars. - scan_error({string,$\',Estr}, Line0, Col0, Nline, Ncol, Ncs); - {Ncs,Nline,Ncol,Nstr,Nwcs,?NO_UNICODE} -> + scan_error({string,$\',Estr}, Line0, Col0, Nline, Ncol, Ncs); %' + {Ncs,Nline,Ncol,Nstr,Nwcs,Uni} -> + true = Uni =:= ?NO_UNICODE orelse AllowUni, case catch list_to_atom(Nwcs) of A when is_atom(A) -> Attrs = attributes(Line0, Col0, St, Nstr), @@ -939,38 +974,40 @@ scan_qatom(Cs, St, Line, Col, Toks, {Wcs,Str,Line0,Col0,Uni0}) -> end end. -scan_string0(Cs, #erl_scan{text=false}, Line, no_col=Col, Q, [], Wcs, Uni) -> - scan_string_no_col(Cs, Line, Col, Q, Wcs, Uni); -scan_string0(Cs, #erl_scan{text=true}, Line, no_col=Col, Q, Str, Wcs, Uni) -> - scan_string1(Cs, Line, Col, Q, Str, Wcs, Uni); -scan_string0(Cs, _St, Line, Col, Q, [], Wcs, Uni) -> - scan_string_col(Cs, Line, Col, Q, Wcs, Uni); -scan_string0(Cs, _St, Line, Col, Q, Str, Wcs, Uni) -> - scan_string1(Cs, Line, Col, Q, Str, Wcs, Uni). +scan_string0(Cs, #erl_scan{text=false}, Line, no_col=Col, Q, U, [], Wcs, Uni) -> + scan_string_no_col(Cs, Line, Col, Q, U, Wcs, Uni); +scan_string0(Cs, #erl_scan{text=true}, Line, no_col=Col, Q, U, Str, Wcs, Uni) -> + scan_string1(Cs, Line, Col, Q, U, Str, Wcs, Uni); +scan_string0(Cs, _St, Line, Col, Q, U, [], Wcs, Uni) -> + scan_string_col(Cs, Line, Col, Q, U, Wcs, Uni); +scan_string0(Cs, _St, Line, Col, Q, U, Str, Wcs, Uni) -> + scan_string1(Cs, Line, Col, Q, U, Str, Wcs, Uni). %% Optimization. Col =:= no_col. -scan_string_no_col([Q|Cs], Line, Col, Q, Wcs, Uni) -> +scan_string_no_col([Q|Cs], Line, Col, Q, _U, Wcs, Uni) -> {Cs,Line,Col,_DontCare=[],lists:reverse(Wcs),Uni}; -scan_string_no_col([$\n=C|Cs], Line, Col, Q, Wcs, Uni) -> - scan_string_no_col(Cs, Line+1, Col, Q, [C|Wcs], Uni); -scan_string_no_col([C|Cs], Line, Col, Q, Wcs, Uni) when C =/= $\\, - ?CHAR(C), ?UNI255(C) -> - scan_string_no_col(Cs, Line, Col, Q, [C|Wcs], Uni); -scan_string_no_col(Cs, Line, Col, Q, Wcs, Uni) -> - scan_string1(Cs, Line, Col, Q, Wcs, Wcs, Uni). +scan_string_no_col([$\n=C|Cs], Line, Col, Q, U, Wcs, Uni) -> + scan_string_no_col(Cs, Line+1, Col, Q, U, [C|Wcs], Uni); +scan_string_no_col([C|Cs], Line, Col, Q, U, Wcs, Uni) when C =/= $\\, + ?CHAR(C), + ?UNI255(C) -> + scan_string_no_col(Cs, Line, Col, Q, U, [C|Wcs], Uni); +scan_string_no_col(Cs, Line, Col, Q, U, Wcs, Uni) -> + scan_string1(Cs, Line, Col, Q, U, Wcs, Wcs, Uni). %% Optimization. Col =/= no_col. -scan_string_col([Q|Cs], Line, Col, Q, Wcs0, Uni) -> +scan_string_col([Q|Cs], Line, Col, Q, _U, Wcs0, Uni) -> Wcs = lists:reverse(Wcs0), Str = [Q|Wcs++[Q]], {Cs,Line,Col+1,Str,Wcs,Uni}; -scan_string_col([$\n=C|Cs], Line, _xCol, Q, Wcs, Uni) -> - scan_string_col(Cs, Line+1, 1, Q, [C|Wcs], Uni); -scan_string_col([C|Cs], Line, Col, Q, Wcs, Uni) when C =/= $\\, - ?CHAR(C), ?UNI255(C) -> - scan_string_col(Cs, Line, Col+1, Q, [C|Wcs], Uni); -scan_string_col(Cs, Line, Col, Q, Wcs, Uni) -> - scan_string1(Cs, Line, Col, Q, Wcs, Wcs, Uni). +scan_string_col([$\n=C|Cs], Line, _xCol, Q, U, Wcs, Uni) -> + scan_string_col(Cs, Line+1, 1, Q, U, [C|Wcs], Uni); +scan_string_col([C|Cs], Line, Col, Q, U, Wcs, Uni) when C =/= $\\, + ?CHAR(C), + ?UNI255(C) -> + scan_string_col(Cs, Line, Col+1, Q, U, [C|Wcs], Uni); +scan_string_col(Cs, Line, Col, Q, U, Wcs, Uni) -> + scan_string1(Cs, Line, Col, Q, U, Wcs, Wcs, Uni). %% UNI_STR is to be replaced by STR when the Unicode-string-to-list %% workaround is eventually removed. @@ -981,14 +1018,14 @@ scan_string_col(Cs, Line, Col, Q, Wcs, Uni) -> %% but then the end location of the error tuple would not correspond %% to the start location of the returned Rest string. (Maybe the end %% location could be modified, but that too is ugly.) -scan_string1([Q|Cs], Line, Col, Q, Str0, Wcs0, Uni) -> +scan_string1([Q|Cs], Line, Col, Q, _U, Str0, Wcs0, Uni) -> Wcs = lists:reverse(Wcs0), Str = ?UNI_STR(Col, [Q|lists:reverse(Str0, [Q])]), {Cs,Line,incr_column(Col, 1),Str,Wcs,Uni}; -scan_string1([$\n=C|Cs], Line, Col, Q, Str, Wcs, Uni) -> +scan_string1([$\n=C|Cs], Line, Col, Q, U, Str, Wcs, Uni) -> Ncol = new_column(Col, 1), - scan_string1(Cs, Line+1, Ncol, Q, ?UNI_STR(Col, [C|Str]), [C|Wcs], Uni); -scan_string1([$\\|Cs]=Cs0, Line, Col, Q, Str, Wcs, Uni) -> + scan_string1(Cs, Line+1, Ncol, Q, U, ?UNI_STR(Col, [C|Str]), [C|Wcs], Uni); +scan_string1([$\\|Cs]=Cs0, Line, Col, Q, U, Str, Wcs, Uni) -> case scan_escape(Cs, Col) of more -> {more,Cs0,Line,Col,Str,Wcs,Uni}; @@ -999,31 +1036,33 @@ scan_string1([$\\|Cs]=Cs0, Line, Col, Q, Str, Wcs, Uni) -> {nl,Val,ValStr,Ncs,Ncol} -> Nstr = ?UNI_STR(Ncol, lists:reverse(ValStr, [$\\|Str])), Nwcs = [Val|Wcs], - scan_string1(Ncs, Line+1, Ncol, Q, Nstr, Nwcs, Uni); - {unicode,_Val,_ValStr,Ncs,Ncol} when Q =:= $' -> %' Emacs + scan_string1(Ncs, Line+1, Ncol, Q, U, Nstr, Nwcs, Uni); + {unicode,_Val,_ValStr,Ncs,Ncol} when not U -> %' Emacs {char_error,Ncs,{illegal,character},Line,Col,incr_column(Ncol, 1)}; {unicode,Val,ValStr,Ncs,Ncol} -> % UNI. Uni is set to Val. Nstr = ?UNI_STR(Ncol, lists:reverse(ValStr, [$\\|Str])), Nwcs = [Val|Wcs], % not used - scan_string1(Ncs, Line, incr_column(Ncol, 1), Q, Nstr, Nwcs, Val); + scan_string1(Ncs, Line, incr_column(Ncol, 1), Q, U, Nstr, Nwcs, Val); {Val,ValStr,Ncs,Ncol} -> Nstr = ?UNI_STR(Ncol, lists:reverse(ValStr, [$\\|Str])), Nwcs = [Val|Wcs], - scan_string1(Ncs, Line, incr_column(Ncol, 1), Q, Nstr, Nwcs, Uni) + scan_string1(Ncs, Line, incr_column(Ncol, 1), Q, U, Nstr, Nwcs, Uni) end; -scan_string1([C|Cs], Line, no_col=Col, Q, Str, Wcs, Uni) when ?CHAR(C), - ?UNI255(C) -> - %% scan_string1(Cs, Line, Col, Q, Str, [C|Wcs], Uni); - scan_string1(Cs, Line, Col, Q, [C|Str], [C|Wcs], Uni); % UNI -scan_string1([C|Cs], Line, Col, Q, Str, Wcs, Uni) when ?CHAR(C), ?UNI255(C) -> - scan_string1(Cs, Line, Col+1, Q, [C|Str], [C|Wcs], Uni); -scan_string1([C|Cs], Line, Col, $', _Str, _Wcs, _Uni) when ?CHAR(C) -> %' UNI +scan_string1([C|Cs], Line, no_col=Col, Q, U, Str, Wcs, Uni) when ?CHAR(C), + ?UNI255(C) -> + %% scan_string1(Cs, Line, Col, Q, U, Str, [C|Wcs], Uni); + scan_string1(Cs, Line, Col, Q, U, [C|Str], [C|Wcs], Uni); % UNI +scan_string1([C|Cs], Line, Col, Q, U, Str, Wcs, Uni) when ?CHAR(C), ?UNI255(C) -> + scan_string1(Cs, Line, Col+1, Q, U, [C|Str], [C|Wcs], Uni); +scan_string1([C|Cs], Line, Col, _Q, false, _Str, _Wcs, _Uni) when ?CHAR(C) -> %' UNI + {char_error,Cs,{illegal,character},Line,Col,incr_column(Col, 1)}; +scan_string1([C|Cs], Line, Col, Q, U, Str, Wcs, _Uni) when ?UNICODE(C) -> + scan_string1(Cs, Line, incr_column(Col, 1), Q, U, [C|Str], [C|Wcs], C); +scan_string1([C|Cs], Line, Col, _Q, _U, _Str, _Wcs, _Uni) when ?CHAR(C) -> % UNI {char_error,Cs,{illegal,character},Line,Col,incr_column(Col, 1)}; -scan_string1([C|Cs], Line, Col, Q, Str, Wcs, _Uni) when ?CHAR(C) -> % UNI - scan_string1(Cs, Line, incr_column(Col, 1), Q, [C|Str], [C|Wcs], C); -scan_string1([]=Cs, Line, Col, _Q, Str, Wcs, Uni) -> +scan_string1([]=Cs, Line, Col, _Q, _U, Str, Wcs, Uni) -> {more,Cs,Line,Col,Str,Wcs,Uni}; -scan_string1(eof, Line, Col, _Q, _Str, Wcs, _Uni) -> +scan_string1(eof, Line, Col, _Q, _U, _Str, Wcs, _Uni) -> {error,Line,Col,lists:reverse(Wcs),eof}. -define(OCT(C), C >= $0, C =< $7). @@ -1074,8 +1113,10 @@ scan_escape([$\n=C|Cs], Col) -> scan_escape([C0|Cs], Col) when ?CHAR(C0), ?UNI255(C0) -> C = escape_char(C0), {C,?UNI_STR(Col, [C0]),Cs,incr_column(Col, 1)}; -scan_escape([C|Cs], Col) when ?CHAR(C) -> % UNI +scan_escape([C|Cs], Col) when ?UNICODE(C) -> {unicode,C,?UNI_STR(Col, [C]),Cs,incr_column(Col, 1)}; +scan_escape([C|Cs], Col) when ?CHAR(C) -> % UNI + {error,Cs,{illegal,character},incr_column(Col, 1)}; scan_escape([], _Col) -> more; scan_escape(eof, Col) -> @@ -1093,7 +1134,7 @@ scan_esc_end([$}|Cs], Col, Wcs0, B, Str0) -> case catch erlang:list_to_integer(Wcs, B) of Val when Val =< 16#FF -> {Val,?UNI_STR(Col, Str0++Wcs++[$}]),Cs,incr_column(Col, 1)}; - Val when Val =< 16#10FFFF -> + Val when ?UNICODE(Val) -> {unicode,Val,?UNI_STR(Col, Str0++Wcs++[$}]),Cs,incr_column(Col,1)}; _ -> {error,Cs,{illegal,character},incr_column(Col, 1)} @@ -1199,18 +1240,36 @@ float_end(Cs, St, Line, Col, Toks, Ncs0) -> scan_error({illegal,float}, Line, Col, Line, Ncol, Cs) end. -skip_comment([C|Cs], St, Line, Col, Toks, N) when C =/= $\n, ?CHAR(C) -> - skip_comment(Cs, St, Line, Col, Toks, N+1); -skip_comment([]=Cs, _St, Line, Col, Toks, N) -> - {more,{Cs,Col,Toks,Line,N,fun skip_comment/6}}; skip_comment(Cs, St, Line, Col, Toks, N) -> + skip_comment(Cs, St, Line, Col, Toks, N, St#erl_scan.unicode). + +skip_comment([C|Cs], St, Line, Col, Toks, N, U) when C =/= $\n, ?CHAR(C) -> + case ?UNI255(C) orelse U andalso ?UNICODE(C) of + true -> + skip_comment(Cs, St, Line, Col, Toks, N+1, U); + false -> + Ncol = incr_column(Col, N+1), + scan_error({illegal,character}, Line, Col, Line, Ncol, Cs) + end; +skip_comment([]=Cs, _St, Line, Col, Toks, N, _U) -> + {more,{Cs,Col,Toks,Line,N,fun skip_comment/6}}; +skip_comment(Cs, St, Line, Col, Toks, N, _U) -> scan1(Cs, St, Line, incr_column(Col, N), Toks). -scan_comment([C|Cs], St, Line, Col, Toks, Ncs) when C =/= $\n, ?CHAR(C) -> - scan_comment(Cs, St, Line, Col, Toks, [C|Ncs]); -scan_comment([]=Cs, _St, Line, Col, Toks, Ncs) -> +scan_comment(Cs, St, Line, Col, Toks, Ncs) -> + scan_comment(Cs, St, Line, Col, Toks, Ncs, St#erl_scan.unicode). + +scan_comment([C|Cs], St, Line, Col, Toks, Ncs, U) when C =/= $\n, ?CHAR(C) -> + case ?UNI255(C) orelse U andalso ?UNICODE(C) of + true -> + scan_comment(Cs, St, Line, Col, Toks, [C|Ncs], U); + false -> + Ncol = incr_column(Col, length(Ncs)+1), + scan_error({illegal,character}, Line, Col, Line, Ncol, Cs) + end; +scan_comment([]=Cs, _St, Line, Col, Toks, Ncs, _U) -> {more,{Cs,Col,Toks,Line,Ncs,fun scan_comment/6}}; -scan_comment(Cs, St, Line, Col, Toks, Ncs0) -> +scan_comment(Cs, St, Line, Col, Toks, Ncs0, _U) -> Ncs = lists:reverse(Ncs0), tok3(Cs, St, Line, Col, Toks, comment, Ncs, Ncs). diff --git a/lib/stdlib/src/escript.erl b/lib/stdlib/src/escript.erl index 498d850df3..99a9d138ac 100644 --- a/lib/stdlib/src/escript.erl +++ b/lib/stdlib/src/escript.erl @@ -710,7 +710,7 @@ epp_parse_file2(Epp, S, Forms, Parsed) -> epp_parse_file(Epp, S, [Form | Forms]) end; {error,{Ln,Mod,Args}} = Form -> - io:format("~s:~w: ~s\n", + io:format("~s:~w: ~ts\n", [S#state.file,Ln,Mod:format_error(Args)]), epp_parse_file(Epp, S#state{n_errors = S#state.n_errors + 1}, [Form | Forms]); {eof, _LastLine} = Eof -> @@ -780,10 +780,10 @@ report_errors(Errors) -> Errors). list_errors(F, [{Line,Mod,E}|Es]) -> - io:fwrite("~s:~w: ~s\n", [F,Line,Mod:format_error(E)]), + io:fwrite("~s:~w: ~ts\n", [F,Line,Mod:format_error(E)]), list_errors(F, Es); list_errors(F, [{Mod,E}|Es]) -> - io:fwrite("~s: ~s\n", [F,Mod:format_error(E)]), + io:fwrite("~s: ~ts\n", [F,Mod:format_error(E)]), list_errors(F, Es); list_errors(_F, []) -> ok. @@ -795,10 +795,10 @@ report_warnings(Ws0) -> lists:foreach(fun({_,Str}) -> io:put_chars(Str) end, Ws). format_message(F, [{Line,Mod,E}|Es]) -> - M = {{F,Line},io_lib:format("~s:~w: Warning: ~s\n", [F,Line,Mod:format_error(E)])}, + M = {{F,Line},io_lib:format("~s:~w: Warning: ~ts\n", [F,Line,Mod:format_error(E)])}, [M|format_message(F, Es)]; format_message(F, [{Mod,E}|Es]) -> - M = {none,io_lib:format("~s: Warning: ~s\n", [F,Mod:format_error(E)])}, + M = {none,io_lib:format("~s: Warning: ~ts\n", [F,Mod:format_error(E)])}, [M|format_message(F, Es)]; format_message(_, []) -> []. @@ -851,12 +851,27 @@ eval_exprs([E|Es], Bs0, Lf, Ef, RBs) -> eval_exprs(Es, Bs, Lf, Ef, RBs). format_exception(Class, Reason) -> + Enc = encoding(), + P = case Enc of + latin1 -> "P"; + _ -> "tP" + end, PF = fun(Term, I) -> - io_lib:format("~." ++ integer_to_list(I) ++ "P", [Term, 50]) + io_lib:format("~." ++ integer_to_list(I) ++ P, [Term, 50]) end, StackTrace = erlang:get_stacktrace(), StackFun = fun(M, _F, _A) -> (M =:= erl_eval) or (M =:= ?MODULE) end, - lib:format_exception(1, Class, Reason, StackTrace, StackFun, PF). + lib:format_exception(1, Class, Reason, StackTrace, StackFun, PF, Enc). + +encoding() -> + [{encoding, Encoding}] = enc(), + Encoding. + +enc() -> + case lists:keyfind(encoding, 1, io:getopts()) of + false -> [{encoding,latin1}]; % should never happen + Enc -> [Enc] + end. fatal(Str) -> throw(Str). diff --git a/lib/stdlib/src/filename.erl b/lib/stdlib/src/filename.erl index 59d6de5d10..0c50eb34e6 100644 --- a/lib/stdlib/src/filename.erl +++ b/lib/stdlib/src/filename.erl @@ -878,7 +878,7 @@ filter_options(_Base, [], Result) -> %% Gets the source file given path of object code and module name. get_source_file(Obj, Mod, Rules) -> - source_by_rules(dirname(Obj), packages:last(Mod), Rules). + source_by_rules(dirname(Obj), atom_to_list(Mod), Rules). source_by_rules(Dir, Base, [{From, To}|Rest]) -> case try_rule(Dir, Base, From, To) of diff --git a/lib/stdlib/src/gb_sets.erl b/lib/stdlib/src/gb_sets.erl index 391f1cff64..ba35a7170a 100644 --- a/lib/stdlib/src/gb_sets.erl +++ b/lib/stdlib/src/gb_sets.erl @@ -1,3 +1,4 @@ +%% -*- coding: utf-8 -*- %% %% %CopyrightBegin% %% @@ -165,7 +166,7 @@ -export([new/0, is_element/2, add_element/2, del_element/2, subtract/2]). -%% GB-trees adapted from Sven-Olof Nystr�m's implementation for +%% GB-trees adapted from Sven-Olof Nyström's implementation for %% representation of sets. %% %% Data structures: diff --git a/lib/stdlib/src/gb_trees.erl b/lib/stdlib/src/gb_trees.erl index 258713c90f..de0c239e26 100644 --- a/lib/stdlib/src/gb_trees.erl +++ b/lib/stdlib/src/gb_trees.erl @@ -1,3 +1,4 @@ +%% -*- coding: utf-8 -*- %% %% %CopyrightBegin% %% @@ -19,7 +20,7 @@ %% ===================================================================== %% General Balanced Trees - highly efficient dictionaries. %% -%% Copyright (C) 1999-2001 Sven-Olof Nystr�m, Richard Carlsson +%% Copyright (C) 1999-2001 Sven-Olof Nyström, Richard Carlsson %% %% An efficient implementation of Prof. Arne Andersson's General %% Balanced Trees. These have no storage overhead compared to plain diff --git a/lib/stdlib/src/io.erl b/lib/stdlib/src/io.erl index 9f65bbfa3a..ecf2aeb375 100644 --- a/lib/stdlib/src/io.erl +++ b/lib/stdlib/src/io.erl @@ -1,7 +1,7 @@ %% %% %CopyrightBegin% %% -%% Copyright Ericsson AB 1996-2011. All Rights Reserved. +%% Copyright Ericsson AB 1996-2012. All Rights Reserved. %% %% The contents of this file are subject to the Erlang Public License, %% Version 1.1, (the "License"); you may not use this file except in @@ -22,28 +22,28 @@ get_chars/2,get_chars/3,get_line/1,get_line/2, get_password/0, get_password/1, setopts/1, setopts/2, getopts/0, getopts/1]). --export([write/1,write/2,read/1,read/2,read/3]). +-export([write/1,write/2,read/1,read/2,read/3,read/4]). -export([columns/0,columns/1,rows/0,rows/1]). -export([fwrite/1,fwrite/2,fwrite/3,fread/2,fread/3, format/1,format/2,format/3]). --export([scan_erl_exprs/1,scan_erl_exprs/2,scan_erl_exprs/3, - scan_erl_form/1,scan_erl_form/2,scan_erl_form/3, +-export([scan_erl_exprs/1,scan_erl_exprs/2,scan_erl_exprs/3,scan_erl_exprs/4, + scan_erl_form/1,scan_erl_form/2,scan_erl_form/3,scan_erl_form/4, parse_erl_exprs/1,parse_erl_exprs/2,parse_erl_exprs/3, - parse_erl_form/1,parse_erl_form/2,parse_erl_form/3]). + parse_erl_exprs/4,parse_erl_form/1,parse_erl_form/2, + parse_erl_form/3,parse_erl_form/4]). -export([request/1,request/2,requests/1,requests/2]). --export_type([device/0, format/0]). +-export_type([device/0, format/0, server_no_data/0]). %%------------------------------------------------------------------------- -type device() :: atom() | pid(). -type prompt() :: atom() | string(). --type error_description() :: term(). % Whatever the io-server sends. --type request_error() :: {'error',error_description()}. +%% ErrorDescription is whatever the I/O-server sends. +-type server_no_data() :: {'error', ErrorDescription :: term()} | 'eof'. -%% XXX: Some uses of line() in this file may need to read erl_scan:location() --type line() :: pos_integer(). +-type location() :: erl_scan:location(). %%------------------------------------------------------------------------- @@ -73,9 +73,9 @@ o_request(Io, Request, Func) -> put_chars(Chars) -> put_chars(default_output(), Chars). --spec put_chars(IoDevice, IoData) -> 'ok' when +-spec put_chars(IoDevice, CharData) -> 'ok' when IoDevice :: device(), - IoData :: unicode:chardata(). + CharData :: unicode:chardata(). put_chars(Io, Chars) -> o_request(Io, {put_chars,unicode,Chars}, put_chars). @@ -124,7 +124,7 @@ rows(Io) -> {error,enotsup} end. --spec get_chars(Prompt, Count) -> Data | 'eof' when +-spec get_chars(Prompt, Count) -> Data | server_no_data() when Prompt :: prompt(), Count :: non_neg_integer(), Data :: [unicode:unicode_char()] | unicode:unicode_binary(). @@ -132,25 +132,23 @@ rows(Io) -> get_chars(Prompt, N) -> get_chars(default_input(), Prompt, N). --spec get_chars(IoDevice, Prompt, Count) -> Data | 'eof' | {error, Reason} when +-spec get_chars(IoDevice, Prompt, Count) -> Data | server_no_data() when IoDevice :: device(), Prompt :: prompt(), Count :: non_neg_integer(), - Reason :: term(), Data :: [unicode:unicode_char()] | unicode:unicode_binary(). get_chars(Io, Prompt, N) when is_integer(N), N >= 0 -> request(Io, {get_chars,unicode,Prompt,N}). --spec get_line(Prompt) -> Data | 'eof' | {'error', Reason} when +-spec get_line(Prompt) -> Data | server_no_data() when Prompt :: prompt(), - Reason :: term(), Data :: [unicode:unicode_char()] | unicode:unicode_binary(). get_line(Prompt) -> get_line(default_input(), Prompt). --spec get_line(IoDevice, Prompt) -> Data | 'eof' | {'error', term()} when +-spec get_line(IoDevice, Prompt) -> Data | server_no_data() when IoDevice :: device(), Prompt :: prompt(), Data :: [unicode:unicode_char()] | unicode:unicode_binary(). @@ -219,8 +217,9 @@ write(Io, Term) -> -spec read(Prompt) -> Result when Prompt :: prompt(), Result :: {'ok', Term :: term()} - | 'eof' - | {'error', ErrorInfo :: erl_scan:error_info()}. + | server_no_data() + | {'error', ErrorInfo}, + ErrorInfo :: erl_scan:error_info() | erl_parse:error_info(). % Read does not use get_until as erl_scan does not work with unicode % XXX:PaN fixme? @@ -231,8 +230,9 @@ read(Prompt) -> IoDevice :: device(), Prompt :: prompt(), Result :: {'ok', Term :: term()} - | 'eof' - | {'error', ErrorInfo :: erl_scan:error_info()}. + | server_no_data() + | {'error', ErrorInfo}, + ErrorInfo :: erl_scan:error_info() | erl_parse:error_info(). read(Io, Prompt) -> case request(Io, {get_until,unicode,Prompt,erl_scan,tokens,[1]}) of @@ -248,24 +248,41 @@ read(Io, Prompt) -> Other end. --spec read(IoDevice, Prompt, StartLine) -> Result when +-spec read(IoDevice, Prompt, StartLocation) -> Result when + IoDevice :: device(), + Prompt :: prompt(), + StartLocation :: location(), + Result :: {'ok', Term :: term(), EndLocation :: location()} + | {'eof', EndLocation :: location()} + | server_no_data() + | {'error', ErrorInfo, ErrorLocation :: location()}, + ErrorInfo :: erl_scan:error_info() | erl_parse:error_info(). + +read(Io, Prompt, Pos0) -> + read(Io, Prompt, Pos0, []). + +-spec read(IoDevice, Prompt, StartLocation, Options) -> Result when IoDevice :: device(), Prompt :: prompt(), - StartLine :: line(), - Result :: {'ok', Term :: term(), EndLine :: line()} - | {'eof', EndLine :: line()} - | {'error', ErrorInfo :: erl_scan:error_info(), ErrorLine :: line()}. - -read(Io, Prompt, StartLine) when is_integer(StartLine) -> - case request(Io, {get_until,unicode,Prompt,erl_scan,tokens,[StartLine]}) of - {ok,Toks,EndLine} -> + StartLocation :: location(), + Options :: erl_scan:options(), + Result :: {'ok', Term :: term(), EndLocation :: location()} + | {'eof', EndLocation :: location()} + | server_no_data() + | {'error', ErrorInfo, ErrorLocation :: location()}, + ErrorInfo :: erl_scan:error_info() | erl_parse:error_info(). + +read(Io, Prompt, Pos0, Options) -> + Args = [Pos0,Options], + case request(Io, {get_until,unicode,Prompt,erl_scan,tokens,Args}) of + {ok,Toks,EndLocation} -> case erl_parse:parse_term(Toks) of - {ok,Term} -> {ok,Term,EndLine}; - {error,ErrorInfo} -> {error,ErrorInfo,EndLine} + {ok,Term} -> {ok,Term,EndLocation}; + {error,ErrorInfo} -> {error,ErrorInfo,EndLocation} end; - {error,_E,_EndLine} = Error -> + {error,_E,_EndLocation} = Error -> Error; - {eof,_EndLine} = Eof -> + {eof,_EndLocation} = Eof -> Eof; Other -> Other @@ -313,7 +330,9 @@ fread(Prompt, Format) -> IoDevice :: device(), Prompt :: prompt(), Format :: format(), - Result :: {'ok', Terms :: [term()]} | 'eof' | {'error', What :: term()}. + Result :: {'ok', Terms :: [term()]} + | {'error', FreadError :: io_lib:fread_error()} + | server_no_data(). fread(Io, Prompt, Format) -> case request(Io, {fread,Prompt,Format}) of @@ -348,7 +367,7 @@ format(Io, Format, Args) -> -spec scan_erl_exprs(Prompt) -> Result when Prompt :: prompt(), - Result :: erl_scan:tokens_result() | request_error(). + Result :: erl_scan:tokens_result() | server_no_data(). scan_erl_exprs(Prompt) -> scan_erl_exprs(default_input(), Prompt, 1). @@ -356,23 +375,33 @@ scan_erl_exprs(Prompt) -> -spec scan_erl_exprs(Device, Prompt) -> Result when Device :: device(), Prompt :: prompt(), - Result :: erl_scan:tokens_result() | request_error(). + Result :: erl_scan:tokens_result() | server_no_data(). scan_erl_exprs(Io, Prompt) -> scan_erl_exprs(Io, Prompt, 1). --spec scan_erl_exprs(Device, Prompt, StartLine) -> Result when +-spec scan_erl_exprs(Device, Prompt, StartLocation) -> Result when Device :: device(), Prompt :: prompt(), - StartLine :: line(), - Result :: erl_scan:tokens_result() | request_error(). + StartLocation :: location(), + Result :: erl_scan:tokens_result() | server_no_data(). scan_erl_exprs(Io, Prompt, Pos0) -> - request(Io, {get_until,unicode,Prompt,erl_scan,tokens,[Pos0]}). + scan_erl_exprs(Io, Prompt, Pos0, []). + +-spec scan_erl_exprs(Device, Prompt, StartLocation, Options) -> Result when + Device :: device(), + Prompt :: prompt(), + StartLocation :: location(), + Options :: erl_scan:options(), + Result :: erl_scan:tokens_result() | server_no_data(). + +scan_erl_exprs(Io, Prompt, Pos0, Options) -> + request(Io, {get_until,unicode,Prompt,erl_scan,tokens,[Pos0,Options]}). -spec scan_erl_form(Prompt) -> Result when Prompt :: prompt(), - Result :: erl_scan:tokens_result() | request_error(). + Result :: erl_scan:tokens_result() | server_no_data(). scan_erl_form(Prompt) -> scan_erl_form(default_input(), Prompt, 1). @@ -380,26 +409,41 @@ scan_erl_form(Prompt) -> -spec scan_erl_form(IoDevice, Prompt) -> Result when IoDevice :: device(), Prompt :: prompt(), - Result :: erl_scan:tokens_result() | request_error(). + Result :: erl_scan:tokens_result() | server_no_data(). scan_erl_form(Io, Prompt) -> scan_erl_form(Io, Prompt, 1). --spec scan_erl_form(IoDevice, Prompt, StartLine) -> Result when +-spec scan_erl_form(IoDevice, Prompt, StartLocation) -> Result when IoDevice :: device(), Prompt :: prompt(), - StartLine :: line(), - Result :: erl_scan:tokens_result() | request_error(). + StartLocation :: location(), + Result :: erl_scan:tokens_result() | server_no_data(). scan_erl_form(Io, Prompt, Pos0) -> - request(Io, {get_until,unicode,Prompt,erl_scan,tokens,[Pos0]}). + scan_erl_form(Io, Prompt, Pos0, []). + +-spec scan_erl_form(IoDevice, Prompt, StartLocation, Options) -> Result when + IoDevice :: device(), + Prompt :: prompt(), + StartLocation :: location(), + Options :: erl_scan:options(), + Result :: erl_scan:tokens_result() | server_no_data(). + +scan_erl_form(Io, Prompt, Pos0, Options) -> + request(Io, {get_until,unicode,Prompt,erl_scan,tokens,[Pos0,Options]}). %% Parsing Erlang code. --type parse_ret() :: {'ok', ExprList :: erl_parse:abstract_expr(), EndLine :: line()} - | {'eof', EndLine :: line()} - | {'error', ErrorInfo :: erl_scan:error_info(), ErrorLine :: line()} - | request_error(). +-type parse_ret() :: {'ok', + ExprList :: erl_parse:abstract_expr(), + EndLocation :: location()} + | {'eof', EndLocation :: location()} + | {'error', + ErrorInfo :: erl_scan:error_info() + | erl_parse:error_info(), + ErrorLocation :: location()} + | server_no_data(). -spec parse_erl_exprs(Prompt) -> Result when Prompt :: prompt(), @@ -416,14 +460,24 @@ parse_erl_exprs(Prompt) -> parse_erl_exprs(Io, Prompt) -> parse_erl_exprs(Io, Prompt, 1). --spec parse_erl_exprs(IoDevice, Prompt, StartLine) -> Result when +-spec parse_erl_exprs(IoDevice, Prompt, StartLocation) -> Result when IoDevice :: device(), Prompt :: prompt(), - StartLine :: line(), + StartLocation :: location(), Result :: parse_ret(). parse_erl_exprs(Io, Prompt, Pos0) -> - case request(Io, {get_until,unicode,Prompt,erl_scan,tokens,[Pos0]}) of + parse_erl_exprs(Io, Prompt, Pos0, []). + +-spec parse_erl_exprs(IoDevice, Prompt, StartLocation, Options) -> Result when + IoDevice :: device(), + Prompt :: prompt(), + StartLocation :: location(), + Options :: erl_scan:options(), + Result :: parse_ret(). + +parse_erl_exprs(Io, Prompt, Pos0, Options) -> + case request(Io, {get_until,unicode,Prompt,erl_scan,tokens,[Pos0,Options]}) of {ok,Toks,EndPos} -> case erl_parse:parse_exprs(Toks) of {ok,Exprs} -> {ok,Exprs,EndPos}; @@ -433,10 +487,15 @@ parse_erl_exprs(Io, Prompt, Pos0) -> Other end. --type parse_form_ret() :: {'ok', AbsForm :: erl_parse:abstract_form(), EndLine :: line()} - | {'eof', EndLine :: line()} - | {'error', ErrorInfo :: erl_scan:error_info(), ErrorLine :: line()} - | request_error(). +-type parse_form_ret() :: {'ok', + AbsForm :: erl_parse:abstract_form(), + EndLocation :: location()} + | {'eof', EndLocation :: location()} + | {'error', + ErrorInfo :: erl_scan:error_info() + | erl_parse:error_info(), + ErrorLocation :: location()} + | server_no_data(). -spec parse_erl_form(Prompt) -> Result when Prompt :: prompt(), @@ -453,14 +512,25 @@ parse_erl_form(Prompt) -> parse_erl_form(Io, Prompt) -> parse_erl_form(Io, Prompt, 1). --spec parse_erl_form(IoDevice, Prompt, StartLine) -> Result when +-spec parse_erl_form(IoDevice, Prompt, StartLocation) -> Result when IoDevice :: device(), Prompt :: prompt(), - StartLine :: line(), + StartLocation :: location(), Result :: parse_form_ret(). parse_erl_form(Io, Prompt, Pos0) -> - case request(Io, {get_until,unicode,Prompt,erl_scan,tokens,[Pos0]}) of + parse_erl_form(Io, Prompt, Pos0, []). + +-spec parse_erl_form(IoDevice, Prompt, StartLocation, Options) -> Result when + IoDevice :: device(), + Prompt :: prompt(), + StartLocation :: location(), + Options :: erl_scan:options(), + Result :: parse_form_ret(). + +parse_erl_form(Io, Prompt, Pos0, Options) -> + Args = [Pos0, Options], + case request(Io, {get_until,unicode,Prompt,erl_scan,tokens,Args}) of {ok,Toks,EndPos} -> case erl_parse:parse_form(Toks) of {ok,Exprs} -> {ok,Exprs,EndPos}; diff --git a/lib/stdlib/src/io_lib.erl b/lib/stdlib/src/io_lib.erl index 513d904c39..5ad505f683 100644 --- a/lib/stdlib/src/io_lib.erl +++ b/lib/stdlib/src/io_lib.erl @@ -1,3 +1,4 @@ +%% -*- coding: utf-8 -*- %% %% %CopyrightBegin% %% @@ -46,25 +47,28 @@ %% 173 - 176 { - ~ punctuation %% 177 DEL control %% 200 - 237 control -%% 240 - 277 NBSP - � punctuation -%% 300 - 326 � - � uppercase -%% 327 � punctuation -%% 330 - 336 � - � uppercase -%% 337 - 366 � - � lowercase -%% 367 � punctuation -%% 370 - 377 � - � lowercase +%% 240 - 277 NBSP - ¿ punctuation +%% 300 - 326 À - Ö uppercase +%% 327 × punctuation +%% 330 - 336 Ø - Þ uppercase +%% 337 - 366 ß - ö lowercase +%% 367 ÷ punctuation +%% 370 - 377 ø - ÿ lowercase %% %% Many punctuation characters region have special meaning. Must -%% watch using � \327, very close to x \170 +%% watch using × \327, very close to x \170 -module(io_lib). -export([fwrite/2,fread/2,fread/3,format/2]). -export([print/1,print/4,indentation/2]). --export([write/1,write/2,write/3,nl/0,format_prompt/1]). +-export([write/1,write/2,write/3,nl/0,format_prompt/1,format_prompt/2]). -export([write_atom/1,write_string/1,write_string/2,write_unicode_string/1, - write_unicode_string/2, write_char/1, write_unicode_char/1]). + write_unicode_string/2, write_char/1, write_unicode_char/1]). + +-export([write_unicode_string_as_latin1/1, write_unicode_string_as_latin1/2, + write_unicode_char_as_latin1/1]). -export([quote_atom/2, char_list/1, unicode_char_list/1, deep_char_list/1, deep_unicode_char_list/1, @@ -75,11 +79,14 @@ collect_line/2, collect_line/3, collect_line/4, get_until/3, get_until/4]). --export_type([chars/0, continuation/0]). +-export_type([chars/0, unicode_chars/0, unicode_string/0, continuation/0, + fread_error/0]). %%---------------------------------------------------------------------- -type chars() :: [char() | chars()]. +-type unicode_chars() :: [unicode:unicode_char() | unicode_chars()]. +-type unicode_string() :: [unicode:unicode_char()]. -type depth() :: -1 | non_neg_integer(). -opaque continuation() :: {Format :: string(), @@ -87,6 +94,16 @@ Nchars :: non_neg_integer(), Results :: [term()]}. +-type fread_error() :: 'atom' + | 'based' + | 'character' + | 'float' + | 'format' + | 'input' + | 'integer' + | 'string' + | 'unsigned'. + %%---------------------------------------------------------------------- %% Interface calls to sub-modules. @@ -107,7 +124,7 @@ fwrite(Format, Args) -> | {'more', RestFormat :: string(), Nchars :: non_neg_integer(), InputStack :: chars()} - | {'error', What :: term()}. + | {'error', What :: fread_error()}. fread(Chars, Format) -> io_lib_fread:fread(Chars, Format). @@ -120,7 +137,7 @@ fread(Chars, Format) -> | {'done', Result, LeftOverChars :: string()}, Result :: {'ok', InputList :: [term()]} | 'eof' - | {'error', What :: term()}. + | {'error', What :: fread_error()}. fread(Cont, Chars, Format) -> io_lib_fread:fread(Cont, Chars, Format). @@ -162,27 +179,34 @@ indentation(Chars, Current) -> %% Format an IO-request prompt (handles formatting errors safely). -%% Atoms, binaries, and iolists can be used as-is, and will be -%% printed without any additional quotes. -%% Note that the output is a deep string, and not an iolist (i.e., -%% it may be deep, but never contains binaries, due to the "~s"). +%% Atoms, binaries, and iolists (or unicode:charlist()) can be used +%% as-is, and will be printed without any additional quotes. -spec format_prompt(term()) -> chars(). -format_prompt({format,Format,Args}) -> - format_prompt(Format,Args); -format_prompt(Prompt) - when is_list(Prompt); is_atom(Prompt); is_binary(Prompt) -> - format_prompt("~ts", [Prompt]); format_prompt(Prompt) -> - format_prompt("~tp", [Prompt]). + format_prompt(Prompt, latin1). + +-spec format_prompt(term(), atom()) -> chars(). + +format_prompt({format,Format,Args}, _Encoding) -> + do_format_prompt(Format, Args); +format_prompt(Prompt, Encoding) + when is_list(Prompt); is_atom(Prompt); is_binary(Prompt) -> + do_format_prompt(add_modifier(Encoding, "s"), [Prompt]); +format_prompt(Prompt, Encoding) -> + do_format_prompt(add_modifier(Encoding, "p"), [Prompt]). -format_prompt(Format, Args) -> +do_format_prompt(Format, Args) -> case catch io_lib:format(Format, Args) of {'EXIT',_} -> "???"; List -> List end. +add_modifier(latin1, C) -> + "~"++C; +add_modifier(_, C) -> + "~t"++C. %% write(Term) %% write(Term, Depth) @@ -294,7 +318,7 @@ quote_atom(Atom, Cs0) -> case Cs0 of [C|Cs] when C >= $a, C =< $z -> not name_chars(Cs); - [C|Cs] when C >= $�, C =< $�, C =/= $� -> + [C|Cs] when C >= $ß, C =< $ÿ, C =/= $÷ -> not name_chars(Cs); _ -> true end @@ -308,9 +332,9 @@ name_chars([C|Cs]) -> name_chars([]) -> true. name_char(C) when C >= $a, C =< $z -> true; -name_char(C) when C >= $�, C =< $�, C =/= $� -> true; +name_char(C) when C >= $ß, C =< $ÿ, C =/= $÷ -> true; name_char(C) when C >= $A, C =< $Z -> true; -name_char(C) when C >= $�, C =< $�, C =/= $� -> true; +name_char(C) when C >= $À, C =< $Þ, C =/= $× -> true; name_char(C) when C >= $0, C =< $9 -> true; name_char($_) -> true; name_char($@) -> true; @@ -330,11 +354,32 @@ write_string(S) -> write_string(S, Q) -> [Q|write_string1(latin1, S, Q)]. +%%% There are two functions to write Unicode strings: +%%% - they both escape control characters < 160; +%%% - write_unicode_string() never escapes characters >= 160; +%%% - write_unicode_string_as_latin1() also escapes characters >= 255. + +-spec write_unicode_string(UnicodeString) -> unicode_string() when + UnicodeString :: unicode_string(). + write_unicode_string(S) -> write_unicode_string(S, $"). %" +-spec write_unicode_string(unicode_string(), char()) -> unicode_string(). + write_unicode_string(S, Q) -> - [Q|write_string1(unicode, S, Q)]. + [Q|write_string1(unicode_as_unicode, S, Q)]. + +-spec write_unicode_string_as_latin1(UnicodeString) -> string() when + UnicodeString :: unicode_string(). + +write_unicode_string_as_latin1(S) -> + write_unicode_string_as_latin1(S, $"). %" + +-spec write_unicode_string_as_latin1(unicode_string(), char()) -> string(). + +write_unicode_string_as_latin1(S, Q) -> + [Q|write_string1(unicode_as_latin1, S, Q)]. write_string1(_,[], Q) -> [Q]; @@ -347,7 +392,11 @@ string_char(_,C, _, Tail) when C >= $\s, C =< $~ -> [C|Tail]; string_char(latin1,C, _, Tail) when C >= $\240, C =< $\377 -> [C|Tail]; -string_char(unicode,C, _, Tail) when C >= $\240 -> +string_char(unicode_as_unicode,C, _, Tail) when C >= $\240 -> + [C|Tail]; +string_char(unicode_as_latin1,C, _, Tail) when C >= $\240, C =< $\377 -> + [C|Tail]; +string_char(unicode_as_latin1,C, _, Tail) when C >= $\377 -> "\\x{"++erlang:integer_to_list(C, 16)++"}"++Tail; string_char(_,$\n, _, Tail) -> [$\\,$n|Tail]; %\n = LF string_char(_,$\r, _, Tail) -> [$\\,$r|Tail]; %\r = CR @@ -374,10 +423,22 @@ write_char($\s) -> "$\\s"; %Must special case this. write_char(C) when is_integer(C), C >= $\000, C =< $\377 -> [$$|string_char(latin1,C, -1, [])]. -write_unicode_char(Ch) when Ch =< 255 -> - write_char(Ch); -write_unicode_char(Uni) -> - [$$|string_char(unicode,Uni, -1, [])]. +%%% There are two functions to write a Unicode character: +%%% - they both escape control characters < 160; +%%% - write_unicode_char() never escapes characters >= 160; +%%% - write_unicode_char_as_latin1() also escapes characters >= 255. + +-spec write_unicode_char(UnicodeChar) -> unicode_string() when + UnicodeChar :: unicode:unicode_char(). + +write_unicode_char(Uni) when is_integer(Uni), Uni >= $\000 -> + [$$|string_char(unicode_as_unicode,Uni, -1, [])]. + +-spec write_unicode_char_as_latin1(UnicodeChar) -> string() when + UnicodeChar :: unicode:unicode_char(). + +write_unicode_char_as_latin1(Uni) when is_integer(Uni), Uni >= $\000 -> + [$$|string_char(unicode_as_latin1,Uni, -1, [])]. %% char_list(CharList) %% deep_char_list(CharList) @@ -392,7 +453,8 @@ char_list([C|Cs]) when is_integer(C), C >= $\000, C =< $\377 -> char_list([]) -> true; char_list(_) -> false. %Everything else is false --spec unicode_char_list(term()) -> boolean(). +-spec unicode_char_list(Term) -> boolean() when + Term :: term(). unicode_char_list([C|Cs]) when is_integer(C), C >= 0, C < 16#D800; is_integer(C), C > 16#DFFF, C < 16#FFFE; @@ -417,7 +479,8 @@ deep_char_list([], []) -> true; deep_char_list(_, _More) -> %Everything else is false false. --spec deep_unicode_char_list(term()) -> boolean(). +-spec deep_unicode_char_list(Term) -> boolean() when + Term :: term(). deep_unicode_char_list(Cs) -> deep_unicode_char_list(Cs, []). @@ -462,7 +525,8 @@ printable_list(_) -> false. %Everything else is false %% Everything that is not a control character and not invalid unicode %% will be considered printable. --spec printable_unicode_list(term()) -> boolean(). +-spec printable_unicode_list(Term) -> boolean() when + Term :: term(). printable_unicode_list([C|Cs]) when is_integer(C), C >= $\040, C =< $\176 -> printable_unicode_list(Cs); diff --git a/lib/stdlib/src/io_lib_format.erl b/lib/stdlib/src/io_lib_format.erl index 49a00a4ec7..5680f83ab6 100644 --- a/lib/stdlib/src/io_lib_format.erl +++ b/lib/stdlib/src/io_lib_format.erl @@ -1,7 +1,7 @@ %% %% %CopyrightBegin% %% -%% Copyright Ericsson AB 1996-2011. All Rights Reserved. +%% Copyright Ericsson AB 1996-2012. All Rights Reserved. %% %% The contents of this file are subject to the Erlang Public License, %% Version 1.1, (the "License"); you may not use this file except in @@ -22,7 +22,7 @@ -export([fwrite/2,fwrite_g/1,indentation/2]). -%% fwrite(Format, ArgList) -> [Char]. +%% fwrite(Format, ArgList) -> [unicode:unicode:char()]. %% Format the arguments in ArgList after string Format. Just generate %% an error if there is an error in the arguments. %% @@ -133,7 +133,7 @@ pcount([{$P,_As,_F,_Ad,_P,_Pad,_Enc}|Cs], Acc) -> pcount(Cs, Acc+1); pcount([_|Cs], Acc) -> pcount(Cs, Acc); pcount([], Acc) -> Acc. -%% build([Control], Pc, Indentation) -> [Char]. +%% build([Control], Pc, Indentation) -> [unicode:unicode_char()]. %% Interpret the control structures. Count the number of print %% remaining and only calculate indentation when necessary. Must also %% be smart when calculating indentation for characters in format. @@ -154,7 +154,7 @@ decr_pc($p, Pc) -> Pc - 1; decr_pc($P, Pc) -> Pc - 1; decr_pc(_, Pc) -> Pc. -%% indentation([Char], Indentation) -> Indentation. +%% indentation([unicode:unicode_char()], Indentation) -> Indentation. %% Calculate the indentation of the end of a string given its start %% indentation. We assume tabs at 8 cols. @@ -167,19 +167,19 @@ indentation([C|Cs], I) -> indentation([], I) -> I. %% control(FormatChar, [Argument], FieldWidth, Adjust, Precision, PadChar, -%% Indentation) -> -%% [Char] +%% Encoding, Indentation) -> +%% [unicode:unicode_char()] %% This is the main dispatch function for the various formatting commands. %% Field widths and precisions have already been calculated. control($w, [A], F, Adj, P, Pad, _Enc,_I) -> term(io_lib:write(A, -1), F, Adj, P, Pad); -control($p, [A], F, Adj, P, Pad, _Enc, I) -> - print(A, -1, F, Adj, P, Pad, I); +control($p, [A], F, Adj, P, Pad, Enc, I) -> + print(A, -1, F, Adj, P, Pad, Enc, I); control($W, [A,Depth], F, Adj, P, Pad, _Enc, _I) when is_integer(Depth) -> term(io_lib:write(A, Depth), F, Adj, P, Pad); -control($P, [A,Depth], F, Adj, P, Pad, _Enc, I) when is_integer(Depth) -> - print(A, Depth, F, Adj, P, Pad, I); +control($P, [A,Depth], F, Adj, P, Pad, Enc, I) when is_integer(Depth) -> + print(A, Depth, F, Adj, P, Pad, Enc, I); control($s, [A], F, Adj, P, Pad, _Enc, _I) when is_atom(A) -> string(atom_to_list(A), F, Adj, P, Pad); control($s, [L0], F, Adj, P, Pad, latin1, _I) -> @@ -187,6 +187,7 @@ control($s, [L0], F, Adj, P, Pad, latin1, _I) -> string(L, F, Adj, P, Pad); control($s, [L0], F, Adj, P, Pad, unicode, _I) -> L = unicode:characters_to_list(L0), + true = is_list(L), uniconv(string(L, F, Adj, P, Pad)); control($e, [A], F, Adj, P, Pad, _Enc, _I) when is_float(A) -> fwrite_e(A, F, Adj, P, Pad); @@ -256,13 +257,17 @@ term(T, F, Adj, P0, Pad) -> adjust(T, chars(Pad, F-L), Adj) end. -%% print(Term, Depth, Field, Adjust, Precision, PadChar, Indentation) +%% print(Term, Depth, Field, Adjust, Precision, PadChar, Encoding, +%% Indentation) %% Print a term. -print(T, D, none, Adj, P, Pad, I) -> print(T, D, 80, Adj, P, Pad, I); -print(T, D, F, Adj, none, Pad, I) -> print(T, D, F, Adj, I+1, Pad, I); -print(T, D, F, right, P, _Pad, _I) -> - io_lib_pretty:print(T, P, F, D). +print(T, D, none, Adj, P, Pad, E, I) -> print(T, D, 80, Adj, P, Pad, E, I); +print(T, D, F, Adj, none, Pad, E, I) -> print(T, D, F, Adj, I+1, Pad, E, I); +print(T, D, F, right, P, _Pad, latin1, _I) -> + io_lib_pretty:print(T, P, F, D); +print(T, D, F, right, P, _Pad, Enc, _I) -> + Options = [{column, P}, {line_length, F}, {depth, D}, {encoding, Enc}], + io_lib_pretty:print(T, Options). %% fwrite_e(Float, Field, Adjust, Precision, PadChar) @@ -608,7 +613,7 @@ prefixed_integer(Int, F, Adj, Base, Pad, Prefix, Lowercase) term([Prefix|S], F, Adj, none, Pad) end. -%% char(Char, Field, Adjust, Precision, PadChar) -> [Char]. +%% char(Char, Field, Adjust, Precision, PadChar) -> [unicode:unicode_char()]. char(C, none, _Adj, none, _Pad) -> [C]; char(C, F, _Adj, none, _Pad) -> chars(C, F); diff --git a/lib/stdlib/src/io_lib_fread.erl b/lib/stdlib/src/io_lib_fread.erl index ded1346097..84d4b8bba0 100644 --- a/lib/stdlib/src/io_lib_fread.erl +++ b/lib/stdlib/src/io_lib_fread.erl @@ -43,7 +43,7 @@ | {'done', Result, LeftOverChars :: string()}, Result :: {'ok', InputList :: io_lib:chars()} | 'eof' - | {'error', What :: term()}. + | {'error', What :: io_lib:fread_error()}. fread([], Chars, Format) -> %%io:format("FREAD: ~w `~s'~n", [Format,Chars]), diff --git a/lib/stdlib/src/io_lib_pretty.erl b/lib/stdlib/src/io_lib_pretty.erl index 169410796b..99ad281a9b 100644 --- a/lib/stdlib/src/io_lib_pretty.erl +++ b/lib/stdlib/src/io_lib_pretty.erl @@ -1,7 +1,7 @@ %% %% %CopyrightBegin% %% -%% Copyright Ericsson AB 1996-2009. All Rights Reserved. +%% Copyright Ericsson AB 1996-2012. All Rights Reserved. %% %% The contents of this file are subject to the Erlang Public License, %% Version 1.1, (the "License"); you may not use this file except in @@ -33,43 +33,76 @@ %% print(Term, Column, LineLength, Depth) -> [Chars] %% Depth = -1 gives unlimited print depth. Use io_lib:write for atomic terms. +-spec print(term()) -> io_lib:chars(). + print(Term) -> print(Term, 1, 80, -1). %% print(Term, RecDefFun) -> [Chars] %% print(Term, Depth, RecDefFun) -> [Chars] %% RecDefFun = fun(Tag, NoFields) -> [FieldTag] | no -%% Used by the shell for printing records. +%% Used by the shell for printing records and for Unicode. + +-type rec_print_fun() :: fun((Tag :: atom(), NFields :: non_neg_integer()) -> + no | [FieldName :: atom()]). +-type column() :: integer(). +-type line_length() :: pos_integer(). +-type depth() :: integer(). +-type max_chars() :: integer(). + +-type chars() :: io_lib:chars(). +-type unicode_chars() :: io_lib:unicode_chars(). +-type option() :: {column, column()} + | {line_length, line_length()} + | {depth, depth()} + | {max_chars, max_chars()} + | {record_print_fun, rec_print_fun()} + | {encoding, latin1 | utf8 | unicode}. +-type options() :: [option()]. + +-spec print(term(), rec_print_fun()) -> chars() | unicode_chars(); + (term(), options()) -> chars() | unicode_chars(). + +print(Term, Options) when is_list(Options) -> + Col = proplists:get_value(column, Options, 1), + Ll = proplists:get_value(line_length, Options, 80), + D = proplists:get_value(depth, Options, -1), + M = proplists:get_value(max_chars, Options, -1), + RecDefFun = proplists:get_value(record_print_fun, Options, no_fun), + Encoding = proplists:get_value(encoding, Options, epp:default_encoding()), + print(Term, Col, Ll, D, M, RecDefFun, Encoding); print(Term, RecDefFun) -> print(Term, -1, RecDefFun). +-spec print(term(), depth(), rec_print_fun()) -> chars() | unicode_chars(). + print(Term, Depth, RecDefFun) -> print(Term, 1, 80, Depth, RecDefFun). +-spec print(term(), column(), line_length(), depth()) -> + chars() | unicode_chars(). + print(Term, Col, Ll, D) -> - print(Term, Col, Ll, D, _M=-1, no_fun). + print(Term, Col, Ll, D, _M=-1, no_fun, latin1). +-spec print(term(), column(), line_length(), depth(), rec_print_fun()) -> + chars() | unicode_chars(). print(Term, Col, Ll, D, RecDefFun) -> print(Term, Col, Ll, D, _M=-1, RecDefFun). -print(_, _, _, 0, _M, _RF) -> "..."; -print(Term, Col, Ll, D, M, RecDefFun) when Col =< 0 -> - print(Term, 1, Ll, D, M, RecDefFun); -print(Term, Col, Ll, D, M0, RecDefFun) when is_tuple(Term); - is_list(Term) -> - If = {_S, Len} = print_length(Term, D, RecDefFun), - M = max_cs(M0, Len), - if - Len < Ll - Col, Len =< M -> - write(If); - true -> - TInd = while_fail([-1, 4], - fun(I) -> cind(If, Col, Ll, M, I, 0, 0) end, - 1), - pp(If, Col, Ll, M, TInd, indent(Col), 0, 0) - end; -print(<<_/bitstring>>=Term, Col, Ll, D, M0, RecDefFun) -> - If = {_S, Len} = print_length(Term, D, RecDefFun), +-spec print(term(), column(), line_length(), depth(), max_chars(), + rec_print_fun()) -> chars() | unicode_chars(). + +print(Term, Col, Ll, D, M, RecDefFun) -> + print(Term, Col, Ll, D, M, RecDefFun, latin1). + +print(_, _, _, 0, _M, _RF, _Enc) -> "..."; +print(Term, Col, Ll, D, M, RecDefFun, Enc) when Col =< 0 -> + print(Term, 1, Ll, D, M, RecDefFun, Enc); +print(Term, Col, Ll, D, M0, RecDefFun, Enc) when is_tuple(Term); + is_list(Term); + is_bitstring(Term) -> + If = {_S, Len} = print_length(Term, D, RecDefFun, Enc), M = max_cs(M0, Len), if Len < Ll - Col, Len =< M -> @@ -80,7 +113,7 @@ print(<<_/bitstring>>=Term, Col, Ll, D, M0, RecDefFun) -> 1), pp(If, Col, Ll, M, TInd, indent(Col), 0, 0) end; -print(Term, _Col, _Ll, _D, _M, _RF) -> +print(Term, _Col, _Ll, _D, _M, _RF, _Enc) -> io_lib:write(Term). %%% @@ -294,50 +327,56 @@ write_tail(E, S) -> %% counted but need to be added later. %% D =/= 0 -print_length([], _D, _RF) -> +print_length([], _D, _RF, _Enc) -> {"[]", 2}; -print_length({}, _D, _RF) -> +print_length({}, _D, _RF, _Enc) -> {"{}", 2}; -print_length(List, D, RF) when is_list(List) -> - case printable_list(List, D) of +print_length(List, D, RF, Enc) when is_list(List) -> + case printable_list(List, D, Enc) of true -> - S = io_lib:write_string(List, $"), %" + S = write_string(List, Enc), {S, length(S)}; %% Truncated lists could break some existing code. % {true, Prefix} -> - % S = io_lib:write_string(Prefix, $"), %" + % S = write_string(Prefix, Enc), % {[S | "..."], 3 + length(S)}; false -> - print_length_list(List, D, RF) + print_length_list(List, D, RF, Enc) end; -print_length(Fun, _D, _RF) when is_function(Fun) -> +print_length(Fun, _D, _RF, _Enc) when is_function(Fun) -> S = io_lib:write(Fun), {S, iolist_size(S)}; -print_length(R, D, RF) when is_atom(element(1, R)), - is_function(RF) -> +print_length(R, D, RF, Enc) when is_atom(element(1, R)), + is_function(RF) -> case RF(element(1, R), tuple_size(R) - 1) of no -> - print_length_tuple(R, D, RF); + print_length_tuple(R, D, RF, Enc); RDefs -> - print_length_record(R, D, RF, RDefs) + print_length_record(R, D, RF, RDefs, Enc) end; -print_length(Tuple, D, RF) when is_tuple(Tuple) -> - print_length_tuple(Tuple, D, RF); -print_length(<<>>, _D, _RF) -> +print_length(Tuple, D, RF, Enc) when is_tuple(Tuple) -> + print_length_tuple(Tuple, D, RF, Enc); +print_length(<<>>, _D, _RF, _Enc) -> {"<<>>", 4}; -print_length(<<_/bitstring>>, 1, _RF) -> +print_length(<<_/bitstring>>, 1, _RF, _Enc) -> {"<<...>>", 7}; -print_length(<<_/bitstring>>=Bin, D, _RF) -> +print_length(<<_/bitstring>>=Bin, D, _RF, Enc) -> case bit_size(Bin) rem 8 of 0 -> D1 = D - 1, - case printable_bin(Bin, D1) of - List when is_list(List) -> - S = io_lib:write_string(List, $"), + case printable_bin(Bin, D1, Enc) of + {true, List} when is_list(List) -> + S = io_lib:write_string(List, $"), %" {[$<,$<,S,$>,$>], 4 + length(S)}; - {true, Prefix} -> - S = io_lib:write_string(Prefix, $"), - {[$<,$<, S | "...>>"], 4 + length(S)}; + {false, List} when is_list(List) -> + S = io_lib:write_unicode_string(List, $"), %" + {[$<,$<,S,"/utf8>>"], 9 + length(S)}; + {true, true, Prefix} -> + S = io_lib:write_string(Prefix, $"), %" + {[$<,$<, S | "...>>"], 7 + length(S)}; + {false, true, Prefix} -> + S = io_lib:write_unicode_string(Prefix, $"), %" + {[$<,$<, S | "/utf8...>>"], 12 + length(S)}; false -> S = io_lib:write(Bin, D), {{bin,S}, iolist_size(S)} @@ -346,51 +385,51 @@ print_length(<<_/bitstring>>=Bin, D, _RF) -> S = io_lib:write(Bin, D), {{bin,S}, iolist_size(S)} end; -print_length(Term, _D, _RF) -> +print_length(Term, _D, _RF, _Enc) -> S = io_lib:write(Term), {S, iolist_size(S)}. -print_length_tuple(_Tuple, 1, _RF) -> +print_length_tuple(_Tuple, 1, _RF, _Enc) -> {"{...}", 5}; -print_length_tuple(Tuple, D, RF) -> - L = print_length_list1(tuple_to_list(Tuple), D, RF), +print_length_tuple(Tuple, D, RF, Enc) -> + L = print_length_list1(tuple_to_list(Tuple), D, RF, Enc), IsTagged = is_atom(element(1, Tuple)) and (tuple_size(Tuple) > 1), {{tuple,IsTagged,L}, list_length(L, 2)}. -print_length_record(_Tuple, 1, _RF, _RDefs) -> +print_length_record(_Tuple, 1, _RF, _RDefs, _Enc) -> {"{...}", 5}; -print_length_record(Tuple, D, RF, RDefs) -> +print_length_record(Tuple, D, RF, RDefs, Enc) -> Name = [$# | io_lib:write_atom(element(1, Tuple))], NameL = length(Name), - L = print_length_fields(RDefs, D - 1, tl(tuple_to_list(Tuple)), RF), + L = print_length_fields(RDefs, D - 1, tl(tuple_to_list(Tuple)), RF, Enc), {{record, [{Name,NameL} | L]}, list_length(L, NameL + 2)}. -print_length_fields([], _D, [], _RF) -> +print_length_fields([], _D, [], _RF, _Enc) -> []; -print_length_fields(_, 1, _, _RF) -> +print_length_fields(_, 1, _, _RF, _Enc) -> {dots, 3}; -print_length_fields([Def | Defs], D, [E | Es], RF) -> - [print_length_field(Def, D - 1, E, RF) | - print_length_fields(Defs, D - 1, Es, RF)]. +print_length_fields([Def | Defs], D, [E | Es], RF, Enc) -> + [print_length_field(Def, D - 1, E, RF, Enc) | + print_length_fields(Defs, D - 1, Es, RF, Enc)]. -print_length_field(Def, D, E, RF) -> +print_length_field(Def, D, E, RF, Enc) -> Name = io_lib:write_atom(Def), - {S, L} = print_length(E, D, RF), + {S, L} = print_length(E, D, RF, Enc), NameL = length(Name) + 3, {{field, Name, NameL, {S, L}}, NameL + L}. -print_length_list(List, D, RF) -> - L = print_length_list1(List, D, RF), +print_length_list(List, D, RF, Enc) -> + L = print_length_list1(List, D, RF, Enc), {{list, L}, list_length(L, 2)}. -print_length_list1([], _D, _RF) -> +print_length_list1([], _D, _RF, _Enc) -> []; -print_length_list1(_, 1, _RF) -> +print_length_list1(_, 1, _RF, _Enc) -> {dots, 3}; -print_length_list1([E | Es], D, RF) -> - [print_length(E, D - 1, RF) | print_length_list1(Es, D - 1, RF)]; -print_length_list1(E, D, RF) -> - print_length(E, D - 1, RF). +print_length_list1([E | Es], D, RF, Enc) -> + [print_length(E, D - 1, RF, Enc) | print_length_list1(Es, D - 1, RF, Enc)]; +print_length_list1(E, D, RF, Enc) -> + print_length(E, D - 1, RF, Enc). list_length([], Acc) -> Acc; @@ -409,16 +448,16 @@ list_length_tail({_, Len}, Acc) -> %% ?CHARS printable characters has depth 1. -define(CHARS, 4). -printable_list(L, D) when D < 0 -> - io_lib:printable_list(L); -printable_list(_L, 1) -> +printable_list(_L, 1, _Enc) -> false; -printable_list(L, _D) -> - io_lib:printable_list(L). +printable_list(L, _D, latin1) -> + io_lib:printable_list(L); +printable_list(L, _D, _Uni) -> + io_lib:printable_unicode_list(L). %% Truncated lists could break some existing code. -% printable_list(L, D) -> +% printable_list(L, D, Enc) when D >= 0 -> % Len = ?CHARS * (D - 1), -% case printable_list1(L, Len) of +% case printable_list1(L, Len, Enc) of % all -> % true; % N when is_integer(N), Len - N >= D - 1 -> @@ -428,32 +467,41 @@ printable_list(L, _D) -> % false % end. -printable_bin(Bin, D) when D >= 0, ?CHARS * D =< byte_size(Bin) -> - printable_bin(Bin, erlang:min(?CHARS * D, byte_size(Bin)), D); -printable_bin(Bin, D) -> - printable_bin(Bin, byte_size(Bin), D). +printable_bin(Bin, D, Enc) when D >= 0, ?CHARS * D =< byte_size(Bin) -> + printable_bin(Bin, erlang:min(?CHARS * D, byte_size(Bin)), D, Enc); +printable_bin(Bin, D, Enc) -> + printable_bin(Bin, byte_size(Bin), D, Enc). -printable_bin(Bin, Len, D) -> +printable_bin(Bin, Len, D, latin1) -> N = erlang:min(20, Len), L = binary_to_list(Bin, 1, N), case printable_list1(L, N) of all when N =:= byte_size(Bin) -> - L; - all when N =:= Len -> % N < byte_size(Bin) {true, L}; + all when N =:= Len -> % N < byte_size(Bin) + {true, true, L}; all -> case printable_bin1(Bin, 1 + N, Len - N) of 0 when byte_size(Bin) =:= Len -> - binary_to_list(Bin); + {true, binary_to_list(Bin)}; NC when D > 0, Len - NC >= D -> - {true, binary_to_list(Bin, 1, Len - NC)}; + {true, true, binary_to_list(Bin, 1, Len - NC)}; NC when is_integer(NC) -> false end; NC when is_integer(NC), D > 0, N - NC >= D -> - {true, binary_to_list(Bin, 1, N - NC)}; + {true, true, binary_to_list(Bin, 1, N - NC)}; NC when is_integer(NC) -> false + end; +printable_bin(Bin, Len, D, _Uni) -> + case printable_unicode(Bin, Len, []) of + {_, <<>>, L} -> + {byte_size(Bin) =:= length(L), L}; + {NC, Bin1, L} when D > 0, Len - NC >= D -> + {byte_size(Bin)-byte_size(Bin1) =:= length(L), true, L}; + {_NC, _Bin, _L} -> + false end. printable_bin1(_Bin, _Start, 0) -> @@ -484,6 +532,16 @@ printable_list1([$\e | Cs], N) -> printable_list1(Cs, N - 1); printable_list1([], _) -> all; printable_list1(_, N) -> N. +printable_unicode(<<C/utf8, R/binary>>, I, L) when I > 0 -> + printable_unicode(R, I - 1, [C | L]); +printable_unicode(Bin, I, L) -> + {I, Bin, lists:reverse(L)}. + +write_string(S, latin1) -> + io_lib:write_string(S, $"); %" +write_string(S, _Uni) -> + io_lib:write_unicode_string(S, $"). %" + %% Throw 'no_good' if the indentation exceeds half the line length %% unless there is room for M characters on the line. diff --git a/lib/stdlib/src/lib.erl b/lib/stdlib/src/lib.erl index cf4b87d7eb..b2ce2a5a8f 100644 --- a/lib/stdlib/src/lib.erl +++ b/lib/stdlib/src/lib.erl @@ -21,8 +21,9 @@ -export([flush_receive/0, error_message/2, progname/0, nonl/1, send/2, sendw/2, eval_str/1]). --export([format_exception/6, format_stacktrace/4, - format_call/4, format_fun/1]). +-export([format_exception/6, format_exception/7, + format_stacktrace/4, format_stacktrace/5, + format_call/4, format_call/5, format_fun/1]). -spec flush_receive() -> 'ok'. @@ -128,32 +129,49 @@ all_white(_) -> false. %% as indentation whenever newline has been inserted); %% Class, Reason and StackTrace are the exception; %% FormatFun = fun(Term, I) -> iolist() formats terms; -%% StackFun = fun(Mod, Fun, Arity) -> bool() is used for trimming the +%% StackFun = fun(Mod, Fun, Arity) -> boolean() is used for trimming the %% end of the stack (typically calls to erl_eval are skipped). -format_exception(I, Class, Reason, StackTrace, StackFun, FormatFun) +format_exception(I, Class, Reason, StackTrace, StackFun, FormatFun) -> + format_exception(I, Class, Reason, StackTrace, StackFun, FormatFun, + latin1). + +%% -> iolist() | unicode:charlist() (no \n at end) +%% FormatFun = fun(Term, I) -> iolist() | unicode:charlist(). +format_exception(I, Class, Reason, StackTrace, StackFun, FormatFun, Encoding) when is_integer(I), I >= 1, is_function(StackFun, 3), is_function(FormatFun, 2) -> S = n_spaces(I-1), {Term,Trace1,Trace} = analyze_exception(Class, Reason, StackTrace), - Expl0 = explain_reason(Term, Class, Trace1, FormatFun, S), - Expl = io_lib:fwrite(<<"~s~s">>, [exited(Class), Expl0]), - case format_stacktrace1(S, Trace, FormatFun, StackFun) of + Expl0 = explain_reason(Term, Class, Trace1, FormatFun, S, Encoding), + FormatString = case Encoding of + latin1 -> "~s~s"; + _ -> "~s~ts" + end, + Expl = io_lib:fwrite(FormatString, [exited(Class), Expl0]), + case format_stacktrace1(S, Trace, FormatFun, StackFun, Encoding) of [] -> Expl; Stack -> [Expl, $\n, Stack] end. %% -> iolist() (no \n at end) -format_stacktrace(I, StackTrace, StackFun, FormatFun) +format_stacktrace(I, StackTrace, StackFun, FormatFun) -> + format_stacktrace(I, StackTrace, StackFun, FormatFun, latin1). + +%% -> iolist() | unicode:charlist() (no \n at end) +format_stacktrace(I, StackTrace, StackFun, FormatFun, Encoding) when is_integer(I), I >= 1, is_function(StackFun, 3), is_function(FormatFun, 2) -> S = n_spaces(I-1), - format_stacktrace1(S, StackTrace, FormatFun, StackFun). + format_stacktrace1(S, StackTrace, FormatFun, StackFun, Encoding). %% -> iolist() (no \n at end) -format_call(I, ForMForFun, As, FormatFun) when is_integer(I), I >= 1, - is_list(As), - is_function(FormatFun, 2) -> - format_call("", n_spaces(I-1), ForMForFun, As, FormatFun). +format_call(I, ForMForFun, As, FormatFun) -> + format_call(I, ForMForFun, As, FormatFun, latin1). + +%% -> iolist() | unicode:charlist() (no \n at end) +format_call(I, ForMForFun, As, FormatFun, Enc) + when is_integer(I), I >= 1, is_list(As), is_function(FormatFun, 2) -> + format_call("", n_spaces(I-1), ForMForFun, As, FormatFun, Enc). %% -> iolist() (no \n at end) format_fun(Fun) when is_function(Fun) -> @@ -204,79 +222,80 @@ is_stacktrace(_) -> false. %% ERTS exit codes (some of them are also returned by erl_eval): -explain_reason(badarg, error, [], _PF, _S) -> +explain_reason(badarg, error, [], _PF, _S, _Enc) -> <<"bad argument">>; -explain_reason({badarg,V}, error=Cl, [], PF, S) -> % orelse, andalso +explain_reason({badarg,V}, error=Cl, [], PF, S, _Enc) -> % orelse, andalso format_value(V, <<"bad argument: ">>, Cl, PF, S); -explain_reason(badarith, error, [], _PF, _S) -> +explain_reason(badarith, error, [], _PF, _S, _Enc) -> <<"an error occurred when evaluating an arithmetic expression">>; -explain_reason({badarity,{Fun,As}}, error, [], _PF, _S) +explain_reason({badarity,{Fun,As}}, error, [], _PF, _S, _Enc) when is_function(Fun) -> %% Only the arity is displayed, not the arguments As. io_lib:fwrite(<<"~s called with ~s">>, [format_fun(Fun), argss(length(As))]); -explain_reason({badfun,Term}, error=Cl, [], PF, S) -> +explain_reason({badfun,Term}, error=Cl, [], PF, S, _Enc) -> format_value(Term, <<"bad function ">>, Cl, PF, S); -explain_reason({badmatch,Term}, error=Cl, [], PF, S) -> - format_value(Term, <<"no match of right hand side value ">>, Cl, PF, S); -explain_reason({case_clause,V}, error=Cl, [], PF, S) -> +explain_reason({badmatch,Term}, error=Cl, [], PF, S, _Enc) -> + Str = <<"no match of right hand side value ">>, + format_value(Term, Str, Cl, PF, S); +explain_reason({case_clause,V}, error=Cl, [], PF, S, _Enc) -> %% "there is no case clause with a true guard sequence and a %% pattern matching..." format_value(V, <<"no case clause matching ">>, Cl, PF, S); -explain_reason(function_clause, error, [{F,A}], _PF, _S) -> +explain_reason(function_clause, error, [{F,A}], _PF, _S, _Enc) -> %% Shell commands FAs = io_lib:fwrite(<<"~w/~w">>, [F, A]), [<<"no function clause matching call to ">> | FAs]; -explain_reason(function_clause, error=Cl, [{M,F,As,Loc}], PF, S) -> +explain_reason(function_clause, error=Cl, [{M,F,As,Loc}], PF, S, Enc) -> Str = <<"no function clause matching ">>, - [format_errstr_call(Str, Cl, {M,F}, As, PF, S),$\s|location(Loc)]; -explain_reason(if_clause, error, [], _PF, _S) -> + [format_errstr_call(Str, Cl, {M,F}, As, PF, S, Enc),$\s|location(Loc)]; +explain_reason(if_clause, error, [], _PF, _S, _Enc) -> <<"no true branch found when evaluating an if expression">>; -explain_reason(noproc, error, [], _PF, _S) -> +explain_reason(noproc, error, [], _PF, _S, _Enc) -> <<"no such process or port">>; -explain_reason(notalive, error, [], _PF, _S) -> +explain_reason(notalive, error, [], _PF, _S, _Enc) -> <<"the node cannot be part of a distributed system">>; -explain_reason(system_limit, error, [], _PF, _S) -> +explain_reason(system_limit, error, [], _PF, _S, _Enc) -> <<"a system limit has been reached">>; -explain_reason(timeout_value, error, [], _PF, _S) -> +explain_reason(timeout_value, error, [], _PF, _S, _Enc) -> <<"bad receive timeout value">>; -explain_reason({try_clause,V}, error=Cl, [], PF, S) -> +explain_reason({try_clause,V}, error=Cl, [], PF, S, _Enc) -> %% "there is no try clause with a true guard sequence and a %% pattern matching..." format_value(V, <<"no try clause matching ">>, Cl, PF, S); -explain_reason(undef, error, [{M,F,A,_}], _PF, _S) -> +explain_reason(undef, error, [{M,F,A,_}], _PF, _S, _Enc) -> %% Only the arity is displayed, not the arguments, if there are any. io_lib:fwrite(<<"undefined function ~s">>, [mfa_to_string(M, F, n_args(A))]); -explain_reason({shell_undef,F,A,_}, error, [], _PF, _S) -> +explain_reason({shell_undef,F,A,_}, error, [], _PF, _S, _Enc) -> %% Give nicer reports for undefined shell functions %% (but not when the user actively calls shell_default:F(...)). io_lib:fwrite(<<"undefined shell command ~s/~w">>, [F, n_args(A)]); %% Exit codes returned by erl_eval only: -explain_reason({argument_limit,_Fun}, error, [], _PF, _S) -> +explain_reason({argument_limit,_Fun}, error, [], _PF, _S, _Enc) -> io_lib:fwrite(<<"limit of number of arguments to interpreted function" " exceeded">>, []); -explain_reason({bad_filter,V}, error=Cl, [], PF, S) -> +explain_reason({bad_filter,V}, error=Cl, [], PF, S, _Enc) -> format_value(V, <<"bad filter ">>, Cl, PF, S); -explain_reason({bad_generator,V}, error=Cl, [], PF, S) -> +explain_reason({bad_generator,V}, error=Cl, [], PF, S, _Enc) -> format_value(V, <<"bad generator ">>, Cl, PF, S); -explain_reason({unbound,V}, error, [], _PF, _S) -> +explain_reason({unbound,V}, error, [], _PF, _S, _Enc) -> io_lib:fwrite(<<"variable ~w is unbound">>, [V]); %% Exit codes local to the shell module (restricted shell): -explain_reason({restricted_shell_bad_return, V}, exit=Cl, [], PF, S) -> +explain_reason({restricted_shell_bad_return, V}, exit=Cl, [], PF, S, _Enc) -> Str = <<"restricted shell module returned bad value ">>, format_value(V, Str, Cl, PF, S); explain_reason({restricted_shell_disallowed,{ForMF,As}}, - exit=Cl, [], PF, S) -> + exit=Cl, [], PF, S, Enc) -> %% ForMF can be a fun, but not a shell fun. Str = <<"restricted shell does not allow ">>, - format_errstr_call(Str, Cl, ForMF, As, PF, S); -explain_reason(restricted_shell_started, exit, [], _PF, _S) -> + format_errstr_call(Str, Cl, ForMF, As, PF, S, Enc); +explain_reason(restricted_shell_started, exit, [], _PF, _S, _Enc) -> <<"restricted shell starts now">>; -explain_reason(restricted_shell_stopped, exit, [], _PF, _S) -> +explain_reason(restricted_shell_stopped, exit, [], _PF, _S, _Enc) -> <<"restricted shell stopped">>; %% Other exit code: -explain_reason(Reason, Class, [], PF, S) -> +explain_reason(Reason, Class, [], PF, S, _Enc) -> PF(Reason, (iolist_size(S)+1) + exited_size(Class)). n_args(A) when is_integer(A) -> @@ -293,28 +312,28 @@ argss(2) -> argss(I) -> io_lib:fwrite(<<"~w arguments">>, [I]). -format_stacktrace1(S0, Stack0, PF, SF) -> +format_stacktrace1(S0, Stack0, PF, SF, Enc) -> Stack1 = lists:dropwhile(fun({M,F,A,_}) -> SF(M, F, A) end, lists:reverse(Stack0)), S = [" " | S0], Stack = lists:reverse(Stack1), - format_stacktrace2(S, Stack, 1, PF). + format_stacktrace2(S, Stack, 1, PF, Enc). -format_stacktrace2(S, [{M,F,A,L}|Fs], N, PF) when is_integer(A) -> +format_stacktrace2(S, [{M,F,A,L}|Fs], N, PF, Enc) when is_integer(A) -> [io_lib:fwrite(<<"~s~s ~s ~s">>, [sep(N, S), origin(N, M, F, A), mfa_to_string(M, F, A), location(L)]) - | format_stacktrace2(S, Fs, N + 1, PF)]; -format_stacktrace2(S, [{M,F,As,_}|Fs], N, PF) when is_list(As) -> + | format_stacktrace2(S, Fs, N + 1, PF, Enc)]; +format_stacktrace2(S, [{M,F,As,_}|Fs], N, PF, Enc) when is_list(As) -> A = length(As), CalledAs = [S,<<" called as ">>], - C = format_call("", CalledAs, {M,F}, As, PF), - [io_lib:fwrite(<<"~s~s ~s\n~s~s">>, + C = format_call("", CalledAs, {M,F}, As, PF, Enc), + [io_lib:fwrite(<<"~s~s ~s\n~s~ts">>, [sep(N, S), origin(N, M, F, A), mfa_to_string(M, F, A), CalledAs, C]) - | format_stacktrace2(S, Fs, N + 1, PF)]; -format_stacktrace2(_S, [], _N, _PF) -> + | format_stacktrace2(S, Fs, N + 1, PF, Enc)]; +format_stacktrace2(_S, [], _N, _PF, _Enc) -> "". location(L) -> @@ -338,22 +357,22 @@ origin(1, M, F, A) -> origin(_N, _M, _F, _A) -> <<"in call from">>. -format_errstr_call(ErrStr, Class, ForMForFun, As, PF, Pre0) -> +format_errstr_call(ErrStr, Class, ForMForFun, As, PF, Pre0, Enc) -> Pre1 = [Pre0 | n_spaces(exited_size(Class))], - format_call(ErrStr, Pre1, ForMForFun, As, PF). + format_call(ErrStr, Pre1, ForMForFun, As, PF, Enc). -format_call(ErrStr, Pre1, ForMForFun, As, PF) -> +format_call(ErrStr, Pre1, ForMForFun, As, PF, Enc) -> Arity = length(As), [ErrStr | case is_op(ForMForFun, Arity) of {yes,Op} -> - format_op(ErrStr, Pre1, Op, As, PF); + format_op(ErrStr, Pre1, Op, As, PF, Enc); no -> MFs = mf_to_string(ForMForFun, Arity), I1 = iolist_size([Pre1,ErrStr|MFs]), - S1 = pp_arguments(PF, As, I1), - S2 = pp_arguments(PF, As, iolist_size([Pre1|MFs])), - Long = count_nl(pp_arguments(PF, [a2345,b2345], I1)) > 0, + S1 = pp_arguments(PF, As, I1, Enc), + S2 = pp_arguments(PF, As, iolist_size([Pre1|MFs]), Enc), + Long = count_nl(pp_arguments(PF, [a2345,b2345], I1, Enc)) > 0, case Long or (count_nl(S2) < count_nl(S1)) of true -> [$\n, Pre1, MFs, S2]; @@ -362,11 +381,11 @@ format_call(ErrStr, Pre1, ForMForFun, As, PF) -> end end]. -format_op(ErrStr, Pre, Op, [A1], PF) -> +format_op(ErrStr, Pre, Op, [A1], PF, _Enc) -> OpS = io_lib:fwrite(<<"~s ">>, [Op]), I1 = iolist_size([ErrStr,Pre,OpS]), [OpS | PF(A1, I1+1)]; -format_op(ErrStr, Pre, Op, [A1, A2], PF) -> +format_op(ErrStr, Pre, Op, [A1, A2], PF, Enc) -> I1 = iolist_size([ErrStr,Pre]), S1 = PF(A1, I1+1), S2 = PF(A2, I1+1), @@ -377,33 +396,40 @@ format_op(ErrStr, Pre, Op, [A1, A2], PF) -> [S1,Pre1,OpS,Pre1|S2]; false -> OpS2 = io_lib:fwrite(<<" ~s ">>, [Op]), - S2_2 = PF(A2, iolist_size([ErrStr,Pre,S1|OpS2])+1), + Size1 = iolist_size([ErrStr,Pre|OpS2]), + {Size2,S1_2} = size(Enc, S1), + S2_2 = PF(A2, Size1+Size2+1), case count_nl(S2) < count_nl(S2_2) of true -> - [S1,Pre1,OpS,Pre1|S2]; + [S1_2,Pre1,OpS,Pre1|S2]; false -> - [S1,OpS2|S2_2] + [S1_2,OpS2|S2_2] end end. -pp_arguments(PF, As, I) -> - case {As, io_lib:printable_list(As)} of +pp_arguments(PF, As, I, Enc) -> + case {As, printable_list(Enc, As)} of {[Int | T], true} -> L = integer_to_list(Int), Ll = length(L), A = list_to_atom(lists:duplicate(Ll, $a)), - S0 = binary_to_list(iolist_to_binary(PF([A | T], I+1))), - brackets_to_parens([$[,L,string:sub_string(S0, 2+Ll)]); + S0 = unicode:characters_to_list(PF([A | T], I+1), Enc), + brackets_to_parens([$[,L,string:sub_string(S0, 2+Ll)], Enc); _ -> - brackets_to_parens(PF(As, I+1)) + brackets_to_parens(PF(As, I+1), Enc) end. -brackets_to_parens(S) -> - B = iolist_to_binary(S), +brackets_to_parens(S, Enc) -> + B = unicode:characters_to_binary(S, Enc), Sz = byte_size(B) - 2, <<$[,R:Sz/binary,$]>> = B, [$(,R,$)]. +printable_list(latin1, As) -> + io_lib:printable_list(As); +printable_list(_, As) -> + io_lib:printable_unicode_list(As). + mfa_to_string(M, F, A) -> io_lib:fwrite(<<"~s/~w">>, [mf_to_string({M, F}, A), A]). @@ -472,3 +498,10 @@ exited(exit) -> <<"exception exit: ">>; exited(throw) -> <<"exception throw: ">>. + +size(latin1, S) -> + {iolist_size(S),S}; +size(_, S0) -> + S = unicode:characters_to_list(S0, unicode), + true = is_list(S), + {length(S),S}. diff --git a/lib/stdlib/src/otp_internal.erl b/lib/stdlib/src/otp_internal.erl index cddf345c76..400380a36d 100644 --- a/lib/stdlib/src/otp_internal.erl +++ b/lib/stdlib/src/otp_internal.erl @@ -78,89 +78,89 @@ obsolete_1(snmp, N, A) -> end; obsolete_1(snmpm, agent_info, 3) -> - {deprecated, {snmpm, agent_info, 2}, "R16B"}; + {removed, {snmpm, agent_info, 2}, "R16B"}; obsolete_1(snmpm, update_agent_info, 5) -> - {deprecated, {snmpm, update_agent_info, 4}, "R16B"}; + {removed, {snmpm, update_agent_info, 4}, "R16B"}; obsolete_1(snmpm, g, 3) -> - {deprecated, {snmpm, sync_get, 3}, "R16B"}; + {removed, {snmpm, sync_get, 3}, "R16B"}; obsolete_1(snmpm, g, 4) -> - {deprecated, {snmpm, sync_get, [3,4]}, "R16B"}; + {removed, {snmpm, sync_get, [3,4]}, "R16B"}; obsolete_1(snmpm, g, 5) -> - {deprecated, {snmpm, sync_get, [4,5]}, "R16B"}; + {removed, {snmpm, sync_get, [4,5]}, "R16B"}; obsolete_1(snmpm, g, 6) -> - {deprecated, {snmpm, sync_get, [5,6]}, "R16B"}; + {removed, {snmpm, sync_get, [5,6]}, "R16B"}; obsolete_1(snmpm, g, 7) -> - {deprecated, {snmpm, sync_get, 6}, "R16B"}; + {removed, {snmpm, sync_get, 6}, "R16B"}; obsolete_1(snmpm, ag, 3) -> - {deprecated, {snmpm, async_get, 3}, "R16B"}; + {removed, {snmpm, async_get, 3}, "R16B"}; obsolete_1(snmpm, ag, 4) -> - {deprecated, {snmpm, async_get, [3,4]}, "R16B"}; + {removed, {snmpm, async_get, [3,4]}, "R16B"}; obsolete_1(snmpm, ag, 5) -> - {deprecated, {snmpm, async_get, [4,5]}, "R16B"}; + {removed, {snmpm, async_get, [4,5]}, "R16B"}; obsolete_1(snmpm, ag, 6) -> - {deprecated, {snmpm, async_get, [5,6]}, "R16B"}; + {removed, {snmpm, async_get, [5,6]}, "R16B"}; obsolete_1(snmpm, ag, 7) -> - {deprecated, {snmpm, async_get, 6}, "R16B"}; + {removed, {snmpm, async_get, 6}, "R16B"}; obsolete_1(snmpm, gn, 3) -> - {deprecated, {snmpm, sync_get_next, 3}, "R16B"}; + {removed, {snmpm, sync_get_next, 3}, "R16B"}; obsolete_1(snmpm, gn, 4) -> - {deprecated, {snmpm, sync_get_next, [3,4]}, "R16B"}; + {removed, {snmpm, sync_get_next, [3,4]}, "R16B"}; obsolete_1(snmpm, gn, 5) -> - {deprecated, {snmpm, sync_get_next, [4,5]}, "R16B"}; + {removed, {snmpm, sync_get_next, [4,5]}, "R16B"}; obsolete_1(snmpm, gn, 6) -> - {deprecated, {snmpm, sync_get_next, [5,6]}, "R16B"}; + {removed, {snmpm, sync_get_next, [5,6]}, "R16B"}; obsolete_1(snmpm, gn, 7) -> - {deprecated, {snmpm, sync_get_next, 6}, "R16B"}; + {removed, {snmpm, sync_get_next, 6}, "R16B"}; obsolete_1(snmpm, agn, 3) -> - {deprecated, {snmpm, async_get_next, 3}, "R16B"}; + {removed, {snmpm, async_get_next, 3}, "R16B"}; obsolete_1(snmpm, agn, 4) -> - {deprecated, {snmpm, async_get_next, [3,4]}, "R16B"}; + {removed, {snmpm, async_get_next, [3,4]}, "R16B"}; obsolete_1(snmpm, agn, 5) -> - {deprecated, {snmpm, async_get_next, [4,5]}, "R16B"}; + {removed, {snmpm, async_get_next, [4,5]}, "R16B"}; obsolete_1(snmpm, agn, 6) -> - {deprecated, {snmpm, async_get_next, [5,6]}, "R16B"}; + {removed, {snmpm, async_get_next, [5,6]}, "R16B"}; obsolete_1(snmpm, agn, 7) -> - {deprecated, {snmpm, async_get_next, 6}, "R16B"}; + {removed, {snmpm, async_get_next, 6}, "R16B"}; obsolete_1(snmpm, s, 3) -> - {deprecated, {snmpm, sync_set, 3}, "R16B"}; + {removed, {snmpm, sync_set, 3}, "R16B"}; obsolete_1(snmpm, s, 4) -> - {deprecated, {snmpm, sync_set, [3,4]}, "R16B"}; + {removed, {snmpm, sync_set, [3,4]}, "R16B"}; obsolete_1(snmpm, s, 5) -> - {deprecated, {snmpm, sync_set, [4,5]}, "R16B"}; + {removed, {snmpm, sync_set, [4,5]}, "R16B"}; obsolete_1(snmpm, s, 6) -> - {deprecated, {snmpm, sync_set, [5,6]}, "R16B"}; + {removed, {snmpm, sync_set, [5,6]}, "R16B"}; obsolete_1(snmpm, s, 7) -> - {deprecated, {snmpm, sync_set, 6}, "R16B"}; + {removed, {snmpm, sync_set, 6}, "R16B"}; obsolete_1(snmpm, as, 3) -> - {deprecated, {snmpm, async_set, 3}, "R16B"}; + {removed, {snmpm, async_set, 3}, "R16B"}; obsolete_1(snmpm, as, 4) -> - {deprecated, {snmpm, async_set, [3,4]}, "R16B"}; + {removed, {snmpm, async_set, [3,4]}, "R16B"}; obsolete_1(snmpm, as, 5) -> - {deprecated, {snmpm, async_set, [4,5]}, "R16B"}; + {removed, {snmpm, async_set, [4,5]}, "R16B"}; obsolete_1(snmpm, as, 6) -> - {deprecated, {snmpm, async_set, [5,6]}, "R16B"}; + {removed, {snmpm, async_set, [5,6]}, "R16B"}; obsolete_1(snmpm, as, 7) -> - {deprecated, {snmpm, async_set, 6}, "R16B"}; + {removed, {snmpm, async_set, 6}, "R16B"}; obsolete_1(snmpm, gb, 5) -> - {deprecated, {snmpm, sync_get_bulk, 5}, "R16B"}; + {removed, {snmpm, sync_get_bulk, 5}, "R16B"}; obsolete_1(snmpm, gb, 6) -> - {deprecated, {snmpm, sync_get_bulk, [5,6]}, "R16B"}; + {removed, {snmpm, sync_get_bulk, [5,6]}, "R16B"}; obsolete_1(snmpm, gb, 7) -> - {deprecated, {snmpm, sync_get_bulk, [6,7]}, "R16B"}; + {removed, {snmpm, sync_get_bulk, [6,7]}, "R16B"}; obsolete_1(snmpm, gb, 8) -> - {deprecated, {snmpm, sync_get_bulk, [7,8]}, "R16B"}; + {removed, {snmpm, sync_get_bulk, [7,8]}, "R16B"}; obsolete_1(snmpm, gb, 9) -> - {deprecated, {snmpm, sync_get_bulk, 8}, "R16B"}; + {removed, {snmpm, sync_get_bulk, 8}, "R16B"}; obsolete_1(snmpm, agb, 5) -> - {deprecated, {snmpm, async_get_bulk, 5}, "R16B"}; + {removed, {snmpm, async_get_bulk, 5}, "R16B"}; obsolete_1(snmpm, agb, 6) -> - {deprecated, {snmpm, async_get_bulk, [5,6]}, "R16B"}; + {removed, {snmpm, async_get_bulk, [5,6]}, "R16B"}; obsolete_1(snmpm, agb, 7) -> - {deprecated, {snmpm, async_get_bulk, [6,7]}, "R16B"}; + {removed, {snmpm, async_get_bulk, [6,7]}, "R16B"}; obsolete_1(snmpm, agb, 8) -> - {deprecated, {snmpm, async_get_bulk, [7,8]}, "R16B"}; + {removed, {snmpm, async_get_bulk, [7,8]}, "R16B"}; obsolete_1(snmpm, agb, 9) -> - {deprecated, {snmpm, async_get_bulk, 8}, "R16B"}; + {removed, {snmpm, async_get_bulk, 8}, "R16B"}; %% *** MEGACO *** @@ -347,7 +347,7 @@ obsolete_1(docb_xml_check, _, _) -> obsolete_1(asn1rt, F, _) when F == load_driver; F == unload_driver -> {deprecated,"deprecated (will be removed in R16A); has no effect as drivers are no longer used."}; obsolete_1(ssl, pid, 1) -> - {deprecated,"deprecated (will be removed in R17); is no longer needed"}; + {removed,"was removed in R16; is no longer needed"}; obsolete_1(inviso, _, _) -> {removed,"the inviso application was removed in R16"}; diff --git a/lib/stdlib/src/proplists.erl b/lib/stdlib/src/proplists.erl index e3eda5d932..204f8e128c 100644 --- a/lib/stdlib/src/proplists.erl +++ b/lib/stdlib/src/proplists.erl @@ -51,20 +51,21 @@ -export_type([property/0, proplist/0]). --type property() :: atom() | tuple(). +-type property() :: atom() | tuple(). -type proplist() :: [property()]. %% --------------------------------------------------------------------- %% @doc Creates a normal form (minimal) representation of a property. If -%% <code>P</code> is <code>{Key, true}</code> where <code>Key</code> is -%% an atom, this returns <code>Key</code>, otherwise the whole term -%% <code>P</code> is returned. +%% <code>PropertyIn</code> is <code>{Key, true}</code> where +%% <code>Key</code> is an atom, this returns <code>Key</code>, otherwise +%% the whole term <code>PropertyIn</code> is returned. %% %% @see property/2 --spec property(Property) -> Property when - Property :: property(). +-spec property(PropertyIn) -> PropertyOut when + PropertyIn :: property(), + PropertyOut :: property(). property({Key, true}) when is_atom(Key) -> Key; @@ -92,13 +93,14 @@ property(Key, Value) -> %% --------------------------------------------------------------------- -%% @doc Unfolds all occurences of atoms in <code>List</code> to tuples +%% @doc Unfolds all occurences of atoms in <code>ListIn</code> to tuples %% <code>{Atom, true}</code>. %% %% @see compact/1 --spec unfold(List) -> List when - List :: [term()]. +-spec unfold(ListIn) -> ListOut when + ListIn :: [term()], + ListOut :: [term()]. unfold([P | Ps]) -> if is_atom(P) -> @@ -110,16 +112,17 @@ unfold([]) -> []. %% @doc Minimizes the representation of all entries in the list. This is -%% equivalent to <code>[property(P) || P <- List]</code>. +%% equivalent to <code>[property(P) || P <- ListIn]</code>. %% %% @see unfold/1 %% @see property/1 --spec compact(List) -> List when - List :: [property()]. +-spec compact(ListIn) -> ListOut when + ListIn :: [property()], + ListOut :: [property()]. -compact(List) -> - [property(P) || P <- List]. +compact(ListIn) -> + [property(P) || P <- ListIn]. %% --------------------------------------------------------------------- @@ -199,7 +202,7 @@ is_defined(_Key, []) -> -spec get_value(Key, List) -> term() when Key :: term(), - List :: List::[term()]. + List :: [term()]. get_value(Key, List) -> get_value(Key, List, undefined). @@ -272,9 +275,10 @@ get_all_values(_Key, []) -> %% %% @see get_all_values/2 --spec append_values(Key, List) -> List when +-spec append_values(Key, ListIn) -> ListOut when Key :: term(), - List :: [term()]. + ListIn :: [term()], + ListOut :: [term()]. append_values(Key, [P | Ps]) -> if is_atom(P), P =:= Key -> @@ -357,7 +361,7 @@ get_keys([], Keys) -> -spec delete(Key, List) -> List when Key :: term(), - List::[term()]. + List :: [term()]. delete(Key, [P | Ps]) -> if is_atom(P), P =:= Key -> @@ -374,7 +378,7 @@ delete(_, []) -> %% --------------------------------------------------------------------- %% @doc Substitutes keys of properties. For each entry in -%% <code>List</code>, if it is associated with some key <code>K1</code> +%% <code>ListIn</code>, if it is associated with some key <code>K1</code> %% such that <code>{K1, K2}</code> occurs in <code>Aliases</code>, the %% key of the entry is changed to <code>Key2</code>. If the same %% <code>K1</code> occurs more than once in <code>Aliases</code>, only @@ -388,10 +392,11 @@ delete(_, []) -> %% @see substitute_negations/2 %% @see normalize/2 --spec substitute_aliases(Aliases, List) -> List when +-spec substitute_aliases(Aliases, ListIn) -> ListOut when Aliases :: [{Key, Key}], Key :: term(), - List::[term()]. + ListIn :: [term()], + ListOut :: [term()]. substitute_aliases(As, Props) -> [substitute_aliases_1(As, P) || P <- Props]. @@ -411,13 +416,13 @@ substitute_aliases_1([], P) -> %% --------------------------------------------------------------------- %% @doc Substitutes keys of boolean-valued properties and simultaneously -%% negates their values. For each entry in <code>List</code>, if it is +%% negates their values. For each entry in <code>ListIn</code>, if it is %% associated with some key <code>K1</code> such that <code>{K1, %% K2}</code> occurs in <code>Negations</code>, then if the entry was %% <code>{K1, true}</code> it will be replaced with <code>{K2, %% false}</code>, otherwise it will be replaced with <code>{K2, %% true}</code>, thus changing the name of the option and simultaneously -%% negating the value given by <code>get_bool(List)</code>. If the same +%% negating the value given by <code>get_bool(ListIn)</code>. If the same %% <code>K1</code> occurs more than once in <code>Negations</code>, only %% the first occurrence is used. %% @@ -431,10 +436,11 @@ substitute_aliases_1([], P) -> %% @see substitute_aliases/2 %% @see normalize/2 --spec substitute_negations(Negations, List) -> List when +-spec substitute_negations(Negations, ListIn) -> ListOut when Negations :: [{Key, Key}], Key :: term(), - List :: [term()]. + ListIn :: [term()], + ListOut :: [term()]. substitute_negations(As, Props) -> [substitute_negations_1(As, P) || P <- Props]. @@ -466,11 +472,11 @@ substitute_negations_1([], P) -> %% @doc Expands particular properties to corresponding sets of %% properties (or other terms). For each pair <code>{Property, %% Expansion}</code> in <code>Expansions</code>, if <code>E</code> is -%% the first entry in <code>List</code> with the same key as +%% the first entry in <code>ListIn</code> with the same key as %% <code>Property</code>, and <code>E</code> and <code>Property</code> %% have equivalent normal forms, then <code>E</code> is replaced with %% the terms in <code>Expansion</code>, and any following entries with -%% the same key are deleted from <code>List</code>. +%% the same key are deleted from <code>ListIn</code>. %% %% <p>For example, the following expressions all return <code>[fie, bar, %% baz, fum]</code>: @@ -497,9 +503,10 @@ substitute_negations_1([], P) -> %% %% @see normalize/2 --spec expand(Expansions, List) -> List when +-spec expand(Expansions, ListIn) -> ListOut when Expansions :: [{Property :: property(), Expansion :: [term()]}], - List :: [term()]. + ListIn :: [term()], + ListOut :: [term()]. expand(Es, Ps) when is_list(Ps) -> Es1 = [{property(P), V} || {P, V} <- Es], @@ -599,15 +606,16 @@ flatten([]) -> %% @see expand/2 %% @see compact/1 --spec normalize(List, Stages) -> List when - List :: [term()], +-spec normalize(ListIn, Stages) -> ListOut when + ListIn :: [term()], Stages :: [Operation], Operation :: {'aliases', Aliases} | {'negations', Negations} | {'expand', Expansions}, Aliases :: [{Key, Key}], Negations :: [{Key, Key}], - Expansions :: [{Property :: property(), Expansion :: [term()]}]. + Expansions :: [{Property :: property(), Expansion :: [term()]}], + ListOut :: [term()]. normalize(L, [{aliases, As} | Xs]) -> normalize(substitute_aliases(As, L), Xs); diff --git a/lib/stdlib/src/qlc_pt.erl b/lib/stdlib/src/qlc_pt.erl index ad25fd559c..d441f38e44 100644 --- a/lib/stdlib/src/qlc_pt.erl +++ b/lib/stdlib/src/qlc_pt.erl @@ -31,9 +31,6 @@ %% Also in qlc.erl. -define(QLC_Q(L1, L2, L3, L4, LC, Os), {call,L1,{remote,L2,{atom,L3,?APIMOD},{atom,L4,?Q}},[LC | Os]}). --define(QLC_QQ(L1, L2, L3, L4, L5, L6, LC, Os), % packages... - {call,L1,{remote,L2,{record_field,L3,{atom,L4,''}, - {atom,L5,?APIMOD}},{atom,L6,?Q}},[LC | Os]}). -define(IMP_Q(L1, L2, LC, Os), {call,L,{atom,L2,?Q},[LC | Os]}). %% Also in qlc.erl. @@ -2475,13 +2472,6 @@ qlcmf(?QLC_Q(L1, L2, L3, L4, LC0, Os0), F, Imp, A0, No0) when length(Os0) < 2 -> NL = make_lcid(L1, No), {T, A} = F(NL, LC, A2), {?QLC_Q(L1, L2, L3, L4, T, Os), A, No + 1}; -qlcmf(?QLC_QQ(L1, L2, L3, L4, L5, L6, LC0, Os0), - F, Imp, A0, No0) when length(Os0) < 2 -> - {Os, A1, No1} = qlcmf(Os0, F, Imp, A0, No0), - {LC, A2, No} = qlcmf(LC0, F, Imp, A1, No1), % nested... - NL = make_lcid(L1, No), - {T, A} = F(NL, LC, A2), - {?QLC_QQ(L1, L2, L3, L4, L5, L6, T, Os), A, No + 1}; qlcmf(?IMP_Q(L1, L2, LC0, Os0), F, Imp=true, A0, No0) when length(Os0) < 2 -> {Os, A1, No1} = qlcmf(Os0, F, Imp, A0, No0), {LC, A2, No} = qlcmf(LC0, F, Imp, A1, No1), % nested... diff --git a/lib/stdlib/src/sets.erl b/lib/stdlib/src/sets.erl index 3fd6c81e5f..e6f05b71d4 100644 --- a/lib/stdlib/src/sets.erl +++ b/lib/stdlib/src/sets.erl @@ -1,7 +1,8 @@ +%% -*- coding: utf-8 -*- %% %% %CopyrightBegin% %% -%% Copyright Ericsson AB 2000-2011. All Rights Reserved. +%% Copyright Ericsson AB 2000-2012. All Rights Reserved. %% %% The contents of this file are subject to the Erlang Public License, %% Version 1.1, (the "License"); you may not use this file except in @@ -17,7 +18,7 @@ %% %CopyrightEnd% %% -%% We use the dynamic hashing techniques by Per-�ke Larsson as +%% We use the dynamic hashing techniques by Per-Åke Larsson as %% described in "The Design and Implementation of Dynamic Hashing for %% Sets and Tables in Icon" by Griswold and Townsend. Much of the %% terminology comes from that paper as well. diff --git a/lib/stdlib/src/shell.erl b/lib/stdlib/src/shell.erl index dc450f0ee6..5c929d2f51 100644 --- a/lib/stdlib/src/shell.erl +++ b/lib/stdlib/src/shell.erl @@ -1,7 +1,7 @@ %% %% %CopyrightBegin% %% -%% Copyright Ericsson AB 1996-2011. All Rights Reserved. +%% Copyright Ericsson AB 1996-2013. All Rights Reserved. %% %% The contents of this file are subject to the Erlang Public License, %% Version 1.1, (the "License"); you may not use this file except in @@ -128,7 +128,7 @@ start_restricted(RShMod) when is_atom(RShMod) -> error_logger:error_report( lists:flatten( io_lib:fwrite( - <<"Restricted shell module ~w not found: ~p\n">>, + "Restricted shell module ~w not found: ~"++cs_p() ++"\n", [RShMod,What]))), Error end. @@ -139,16 +139,6 @@ stop_restricted() -> application:unset_env(stdlib, restricted_shell), exit(restricted_shell_stopped). -default_packages() -> - []. -%%% ['erl','erl.lang']. - -default_modules() -> - []. -%%% [{pdict, 'erl.lang.proc.pdict'}, -%%% {keylist, 'erl.lang.list.keylist'}, -%%% {debug, 'erl.system.debug'}]. - -spec server(boolean(), boolean()) -> 'terminated'. server(NoCtrlG, StartSync) -> @@ -183,16 +173,7 @@ server(StartSync) -> end end, %% Our spawner has fixed the process groups. - Bs0 = erl_eval:new_bindings(), - Bs = lists:foldl(fun ({K, V}, D) -> - erl_eval:add_binding({module,K}, V, D) - end, - lists:foldl(fun (P, D) -> - import_all(P, D) - end, - Bs0, default_packages()), - default_modules()), - %% io:fwrite("Imported modules: ~p.\n", [erl_eval:bindings(Bs)]), + Bs = erl_eval:new_bindings(), %% Use an Ets table for record definitions. It takes too long to %% send a huge term to and from the evaluator. Ets makes it @@ -230,9 +211,10 @@ server(StartSync) -> ok; {RShMod2,What2} -> io:fwrite( - <<"Warning! Restricted shell module ~w not found: ~p.\n" - "Only the commands q() and init:stop() will be allowed!\n">>, - [RShMod2,What2]), + ("Warning! Restricted shell module ~w not found: ~" + ++cs_p()++".\n" + "Only the commands q() and init:stop() will be allowed!\n"), + [RShMod2,What2]), application:set_env(stdlib, restricted_shell, ?MODULE) end, @@ -244,7 +226,7 @@ server_loop(N0, Eval_0, Bs00, RT, Ds00, History0, Results0) -> {Eval_1,Bs0,Ds0,Prompt} = prompt(N, Eval_0, Bs00, RT, Ds00), {Res,Eval0} = get_command(Prompt, Eval_1, Bs0, RT, Ds0), case Res of - {ok,Es0,_EndLine} -> + {ok,Es0} -> case expand_hist(Es0, N) of {ok,Es} -> {V,Eval,Bs,Ds} = shell_cmd(Es, Eval0, Bs0, RT, Ds0, cmd), @@ -263,11 +245,11 @@ server_loop(N0, Eval_0, Bs00, RT, Ds00, History0, Results0) -> end, server_loop(N, Eval, Bs, RT, Ds, History, Results); {error,E} -> - fwrite_severity(benign, <<"~s">>, [E]), + fwrite_severity(benign, <<"~ts">>, [E]), server_loop(N0, Eval0, Bs0, RT, Ds0, History0, Results0) end; - {error,{Line,Mod,What},_EndLine} -> - fwrite_severity(benign, <<"~w: ~s">>, + {error,{Line,Mod,What}} -> + fwrite_severity(benign, <<"~w: ~ts">>, [Line, Mod:format_error(What)]), server_loop(N0, Eval0, Bs0, RT, Ds0, History0, Results0); {error,terminated} -> %Io process terminated @@ -277,20 +259,35 @@ server_loop(N0, Eval_0, Bs00, RT, Ds00, History0, Results0) -> exit(Eval0, kill), {_,Eval,_,_} = shell_rep(Eval0, Bs0, RT, Ds0), server_loop(N0, Eval, Bs0, RT, Ds0, History0, Results0); - {error,tokens} -> %Most probably unicode > 255 + {error,tokens} -> %Most probably character > 255 fwrite_severity(benign, <<"~w: Invalid tokens.">>, [N]), server_loop(N0, Eval0, Bs0, RT, Ds0, History0, Results0); - {eof,_EndLine} -> - fwrite_severity(fatal, <<"Terminating erlang (~w)">>, [node()]), - halt(); eof -> fwrite_severity(fatal, <<"Terminating erlang (~w)">>, [node()]), halt() end. get_command(Prompt, Eval, Bs, RT, Ds) -> - Parse = fun() -> exit(io:parse_erl_exprs(Prompt)) end, + Parse = + fun() -> + exit( + case + io:scan_erl_exprs(group_leader(), Prompt, 1, [unicode]) + of + {ok,Toks,_EndPos} -> + erl_parse:parse_exprs(Toks); + {eof,_EndPos} -> + eof; + {error,ErrorInfo,_EndPos} -> + %% Skip the rest of the line: + _ = io:get_line(''), + {error,ErrorInfo}; + Else -> + Else + end + ) + end, Pid = spawn_link(Parse), get_command1(Pid, Eval, Bs, RT, Ds). @@ -337,7 +334,7 @@ get_prompt_func() -> end. bad_prompt_func(M) -> - fwrite_severity(benign, <<"Bad prompt function: ~p">>, [M]). + fwrite_severity(benign, "Bad prompt function: ~"++cs_p(), [M]). default_prompt(N) -> %% Don't bother flattening the list irrespective of what the @@ -453,7 +450,8 @@ expand_bin_elements([{bin_element,L,E,Sz,Ts}|Fs], C) -> no_command(N) -> throw({error, - io_lib:fwrite(<<"~s: command not found">>, [erl_pp:expr(N)])}). + io_lib:fwrite(<<"~ts: command not found">>, + [erl_pp:expr(N, enc())])}). %% add_cmd(Number, Expressions, Value) %% get_cmd(Number, CurrentCommand) @@ -518,7 +516,7 @@ shell_rep(Ev, Bs0, RT, Ds0) -> {shell_rep,Ev,{value,V,Bs,Ds}} -> {V,Ev,Bs,Ds}; {shell_rep,Ev,{command_error,{Line,M,Error}}} -> - fwrite_severity(benign, <<"~w: ~s">>, + fwrite_severity(benign, <<"~w: ~ts">>, [Line, M:format_error(Error)]), {{'EXIT',Error},Ev,Bs0,Ds0}; {shell_req,Ev,get_cmd} -> @@ -570,9 +568,10 @@ report_exception(Class, Severity, {Reason,Stacktrace}, RT) -> I = iolist_size(Tag) + 1, PF = fun(Term, I1) -> pp(Term, I1, RT) end, SF = fun(M, _F, _A) -> (M =:= erl_eval) or (M =:= ?MODULE) end, - io:requests([{put_chars, Tag}, - {put_chars, - lib:format_exception(I, Class, Reason, Stacktrace, SF, PF)}, + Enc = encoding(), + Str = lib:format_exception(I, Class, Reason, Stacktrace, SF, PF, Enc), + io:requests([{put_chars, latin1, Tag}, + {put_chars, unicode, Str}, nl]). start_eval(Bs, RT, Ds) -> @@ -671,7 +670,8 @@ exprs([E0|Es], Bs1, RT, Lf, Ef, Bs0, W) -> if Es =:= [] -> VS = pp(V0, 1, RT), - [io:requests([{put_chars, VS}, nl]) || W =:= cmd], + [io:requests([{put_chars, unicode, VS}, nl]) || + W =:= cmd], %% Don't send the result back if it will be %% discarded anyway. V = if @@ -753,7 +753,7 @@ used_records(E) -> {expr, E}. fwrite_severity(Severity, S, As) -> - io:fwrite(<<"~s\n">>, [format_severity(Severity, S, As)]). + io:fwrite(<<"~ts\n">>, [format_severity(Severity, S, As)]). format_severity(Severity, S, As) -> add_severity(Severity, io_lib:fwrite(S, As)). @@ -958,13 +958,13 @@ local_func(rd, [{atom,_,RecName},RecDef0], Bs, _Shell, RT, _Lf, _Ef) -> RecDef = expand_value(RecDef0), RDs = lists:flatten(erl_pp:expr(RecDef)), Attr = lists:concat(["-record('", RecName, "',", RDs, ")."]), - {ok, Tokens, _} = erl_scan:string(Attr), + {ok, Tokens, _} = erl_scan:string(Attr, 1, [unicode]), case erl_parse:parse_form(Tokens) of {ok,AttrForm} -> [RN] = add_records([AttrForm], Bs, RT), {value,RN,Bs}; {error,{_Line,M,ErrDesc}} -> - ErrStr = io_lib:fwrite(<<"~s">>, [M:format_error(ErrDesc)]), + ErrStr = io_lib:fwrite(<<"~ts">>, [M:format_error(ErrDesc)]), exit(lists:flatten(ErrStr)) end; local_func(rd, [_,_], _Bs, _Shell, _RT, _Lf, _Ef) -> @@ -988,11 +988,13 @@ local_func(rl, [A], Bs0, _Shell, RT, Lf, Ef) -> {value,list_records(record_defs(RT, listify(Recs))),Bs}; local_func(rp, [A], Bs0, _Shell, RT, Lf, Ef) -> {[V],Bs} = expr_list([A], Bs0, Lf, Ef), - W = columns(), - io:requests([{put_chars, - io_lib_pretty:print(V, 1, W, -1, ?CHAR_MAX, - record_print_fun(RT))}, - nl]), + Cs = io_lib_pretty:print(V, ([{column, 1}, + {line_length, columns()}, + {depth, -1}, + {max_chars, ?CHAR_MAX}, + {record_print_fun, record_print_fun(RT)}] + ++ enc())), + io:requests([{put_chars, unicode, Cs}, nl]), {value,ok,Bs}; local_func(rr, [A], Bs0, _Shell, RT, Lf, Ef) -> {[File],Bs} = expr_list([A], Bs0, Lf, Ef), @@ -1012,38 +1014,6 @@ local_func(which, [{atom,_,M}], Bs, _Shell, _RT, _Lf, _Ef) -> end; local_func(which, [_Other], _Bs, _Shell, _RT, _Lf, _Ef) -> erlang:raise(error, function_clause, [{shell,which,1}]); -local_func(import, [M], Bs, _Shell, _RT, _Lf, _Ef) -> - case erl_parse:package_segments(M) of - error -> erlang:raise(error, function_clause, [{shell,import,1}]); - M1 -> - Mod = packages:concat(M1), - case packages:is_valid(Mod) of - true -> - Key = list_to_atom(packages:last(Mod)), - Mod1 = list_to_atom(Mod), - {value,ok,erl_eval:add_binding({module,Key}, Mod1, Bs)}; - false -> - exit({{bad_module_name, Mod}, [{shell,import,1}]}) - end - end; -local_func(import_all, [P], Bs0, _Shell, _RT, _Lf, _Ef) -> - case erl_parse:package_segments(P) of - error -> erlang:raise(error, function_clause, [{shell,import_all,1}]); - P1 -> - Name = packages:concat(P1), - case packages:is_valid(Name) of - true -> - Bs1 = import_all(Name, Bs0), - {value,ok,Bs1}; - false -> - exit({{bad_package_name, Name}, - [{shell,import_all,1}]}) - end - end; -local_func(use, [M], Bs, Shell, RT, Lf, Ef) -> - local_func(import, [M], Bs, Shell, RT, Lf, Ef); -local_func(use_all, [M], Bs, Shell, RT, Lf, Ef) -> - local_func(import_all, [M], Bs, Shell, RT, Lf, Ef); local_func(history, [{integer,_,N}], Bs, _Shell, _RT, _Lf, _Ef) -> {value,history(N),Bs}; local_func(history, [_Other], _Bs, _Shell, _RT, _Lf, _Ef) -> @@ -1166,7 +1136,7 @@ add_records(RAs, Bs0, RT) -> case check_command([], Bs1) of {error,{_Line,M,ErrDesc}} -> %% A source file that has not been compiled. - ErrStr = io_lib:fwrite(<<"~s">>, [M:format_error(ErrDesc)]), + ErrStr = io_lib:fwrite(<<"~ts">>, [M:format_error(ErrDesc)]), exit(lists:flatten(ErrStr)); ok -> true = ets:insert(RT, Recs), @@ -1323,15 +1293,6 @@ record_attrs(Forms) -> %%% End of reading record information from file(s) -import_all(P, Bs0) -> - Ms = packages:find_modules(P), - lists:foldl(fun (M, Bs) -> - Key = list_to_atom(M), - M1 = list_to_atom(packages:concat(P, M)), - erl_eval:add_binding({module,Key}, M1, Bs) - end, - Bs0, Ms). - shell_req(Shell, Req) -> Shell ! {shell_req,self(),Req}, receive @@ -1343,25 +1304,25 @@ list_commands([{{N,command},Es0}, {{N,result}, V} |Ds], RT) -> VS = pp(V, 4, RT), Ns = io_lib:fwrite(<<"~w: ">>, [N]), I = iolist_size(Ns), - io:requests([{put_chars, Ns}, - {format,<<"~s\n">>,[erl_pp:exprs(Es, I, none)]}, + io:requests([{put_chars, latin1, Ns}, + {format,<<"~ts\n">>,[erl_pp:exprs(Es, I, enc())]}, {format,<<"-> ">>,[]}, - {put_chars, VS}, + {put_chars, unicode, VS}, nl]), list_commands(Ds, RT); list_commands([{{N,command},Es0} |Ds], RT) -> Es = prep_list_commands(Es0), Ns = io_lib:fwrite(<<"~w: ">>, [N]), I = iolist_size(Ns), - io:requests([{put_chars, Ns}, - {format,<<"~s\n">>,[erl_pp:exprs(Es, I, none)]}]), + io:requests([{put_chars, latin1, Ns}, + {format,<<"~ts\n">>,[erl_pp:exprs(Es, I, enc())]}]), list_commands(Ds, RT); list_commands([_D|Ds], RT) -> list_commands(Ds, RT); list_commands([], _RT) -> ok. list_bindings([{{module,M},Val}|Bs], RT) -> - io:fwrite(<<"~p is ~p\n">>, [M,Val]), + io:fwrite(<<"~w is ~w\n">>, [M,Val]), list_bindings(Bs, RT); list_bindings([{Name,Val}|Bs], RT) -> case erl_eval:fun_data(Val) of @@ -1369,13 +1330,13 @@ list_bindings([{Name,Val}|Bs], RT) -> FCs = expand_value(FCs0), % looks nicer F = {'fun',0,{clauses,FCs}}, M = {match,0,{var,0,Name},F}, - io:fwrite(<<"~s\n">>, [erl_pp:expr(M)]); + io:fwrite(<<"~ts\n">>, [erl_pp:expr(M, enc())]); false -> Namel = io_lib:fwrite(<<"~s = ">>, [Name]), Nl = iolist_size(Namel)+1, ValS = pp(Val, Nl, RT), - io:requests([{put_chars, Namel}, - {put_chars, ValS}, + io:requests([{put_chars, latin1, Namel}, + {put_chars, unicode, ValS}, nl]) end, list_bindings(Bs, RT); @@ -1384,7 +1345,7 @@ list_bindings([], _RT) -> list_records(Records) -> lists:foreach(fun({_Name,Attr}) -> - io:fwrite(<<"~s">>, [erl_pp:attribute(Attr)]) + io:fwrite(<<"~ts">>, [erl_pp:attribute(Attr, enc())]) end, Records). record_defs(RT, Names) -> @@ -1427,8 +1388,20 @@ get_history_and_results() -> {History, erlang:min(Results, History)}. pp(V, I, RT) -> - io_lib_pretty:print(V, I, columns(), ?LINEMAX, ?CHAR_MAX, - record_print_fun(RT)). + pp(V, I, RT, enc()). + +pp(V, I, RT, Enc) -> + io_lib_pretty:print(V, ([{column, I}, {line_length, columns()}, + {depth, ?LINEMAX}, {max_chars, ?CHAR_MAX}, + {record_print_fun, record_print_fun(RT)}] + ++ Enc)). + +%% Control sequence 'p' possibly with Unicode translation modifier +cs_p() -> + case encoding() of + latin1 -> "p"; + unicode -> "tp" + end. columns() -> case io:columns() of @@ -1436,9 +1409,20 @@ columns() -> _ -> 80 end. +encoding() -> + [{encoding, Encoding}] = enc(), + Encoding. + +enc() -> + case lists:keyfind(encoding, 1, io:getopts()) of + false -> [{encoding,latin1}]; % should never happen + Enc -> [Enc] + end. + garb(Shell) -> erlang:garbage_collect(Shell), catch erlang:garbage_collect(whereis(user)), + catch erlang:garbage_collect(whereis(group)), catch erlang:garbage_collect(group_leader()), erlang:garbage_collect(). @@ -1458,7 +1442,8 @@ check_env(V) -> ok; {ok, Val} -> Txt = io_lib:fwrite( - <<"Invalid value of STDLIB configuration parameter ~p: ~p\n">>, + ("Invalid value of STDLIB configuration parameter ~w: ~" + ++cs_p()++"\n"), [V, Val]), error_logger:info_report(lists:flatten(Txt)) end. diff --git a/lib/stdlib/src/string.erl b/lib/stdlib/src/string.erl index fc029a582f..03f0a19f14 100644 --- a/lib/stdlib/src/string.erl +++ b/lib/stdlib/src/string.erl @@ -1,3 +1,4 @@ +%% -*- coding: utf-8 -*- %% %% %CopyrightBegin% %% @@ -257,7 +258,7 @@ chars(C, N, Tail) when N > 0 -> chars(C, 0, Tail) when is_integer(C) -> Tail. -%% Torbj�rn's bit. +%% Torbjörn's bit. %%% COPIES %%% diff --git a/lib/stdlib/src/sys.erl b/lib/stdlib/src/sys.erl index 4dd70ad425..2d6287814e 100644 --- a/lib/stdlib/src/sys.erl +++ b/lib/stdlib/src/sys.erl @@ -39,7 +39,18 @@ | {'in', Msg :: _, From :: _} | {'out', Msg :: _, To :: _} | term(). --opaque dbg_opt() :: list(). +-opaque dbg_opt() :: {'trace', 'true'} + | {'log', + {N :: non_neg_integer(), + [{Event :: system_event(), + FuncState :: _, + FormFunc :: dbg_fun()}]}} + | {'statistics', {file:date_time(), + {'reductions', non_neg_integer()}, + MessagesIn :: non_neg_integer(), + MessagesOut :: non_neg_integer()}} + | {'log_to_file', file:io_device()} + | {Func :: dbg_fun(), FuncState :: term()}. -type dbg_fun() :: fun((FuncState :: _, Event :: system_event(), ProcState :: _) -> 'done' | (NewFuncState :: _)). @@ -47,24 +58,22 @@ %%----------------------------------------------------------------- %% System messages %%----------------------------------------------------------------- --spec suspend(Name) -> Void when - Name :: name(), - Void :: term(). +-spec suspend(Name) -> 'ok' when + Name :: name(). suspend(Name) -> send_system_msg(Name, suspend). --spec suspend(Name, Timeout) -> Void when + +-spec suspend(Name, Timeout) -> 'ok' when Name :: name(), - Timeout :: timeout(), - Void :: term(). + Timeout :: timeout(). suspend(Name, Timeout) -> send_system_msg(Name, suspend, Timeout). --spec resume(Name) -> Void when - Name :: name(), - Void :: term(). +-spec resume(Name) -> 'ok' when + Name :: name(). resume(Name) -> send_system_msg(Name, resume). --spec resume(Name, Timeout) -> Void when + +-spec resume(Name, Timeout) -> 'ok' when Name :: name(), - Timeout :: timeout(), - Void :: term(). + Timeout :: timeout(). resume(Name, Timeout) -> send_system_msg(Name, resume, Timeout). -spec get_status(Name) -> Status when @@ -73,9 +82,10 @@ resume(Name, Timeout) -> send_system_msg(Name, resume, Timeout). SItem :: (PDict :: [{Key :: term(), Value :: term()}]) | (SysState :: 'running' | 'suspended') | (Parent :: pid()) - | (Dbg :: dbg_opt()) + | (Dbg :: [dbg_opt()]) | (Misc :: term()). get_status(Name) -> send_system_msg(Name, get_status). + -spec get_status(Name, Timeout) -> Status when Name :: name(), Timeout :: timeout(), @@ -83,7 +93,7 @@ get_status(Name) -> send_system_msg(Name, get_status). SItem :: (PDict :: [{Key :: term(), Value :: term()}]) | (SysState :: 'running' | 'suspended') | (Parent :: pid()) - | (Dbg :: dbg_opt()) + | (Dbg :: [dbg_opt()]) | (Misc :: term()). get_status(Name, Timeout) -> send_system_msg(Name, get_status, Timeout). @@ -95,6 +105,7 @@ get_status(Name, Timeout) -> send_system_msg(Name, get_status, Timeout). Reason :: term(). change_code(Name, Mod, Vsn, Extra) -> send_system_msg(Name, {change_code, Mod, Vsn, Extra}). + -spec change_code(Name, Module, OldVsn, Extra, Timeout) -> 'ok' | {error, Reason} when Name :: name(), @@ -191,35 +202,33 @@ no_debug(Name) -> send_system_msg(Name, {debug, no_debug}). Timeout :: timeout(). no_debug(Name, Timeout) -> send_system_msg(Name, {debug, no_debug}, Timeout). --spec install(Name, FuncSpec) -> Void when +-spec install(Name, FuncSpec) -> 'ok' when Name :: name(), FuncSpec :: {Func, FuncState}, Func :: dbg_fun(), - FuncState :: term(), - Void :: term(). + FuncState :: term(). install(Name, {Func, FuncState}) -> send_system_msg(Name, {debug, {install, {Func, FuncState}}}). --spec install(Name, FuncSpec, Timeout) -> Void when + +-spec install(Name, FuncSpec, Timeout) -> 'ok' when Name :: name(), FuncSpec :: {Func, FuncState}, Func :: dbg_fun(), FuncState :: term(), - Timeout :: timeout(), - Void :: term(). + Timeout :: timeout(). install(Name, {Func, FuncState}, Timeout) -> send_system_msg(Name, {debug, {install, {Func, FuncState}}}, Timeout). --spec remove(Name, Func) -> Void when +-spec remove(Name, Func) -> 'ok' when Name :: name(), - Func :: dbg_fun(), - Void :: term(). + Func :: dbg_fun(). remove(Name, Func) -> send_system_msg(Name, {debug, {remove, Func}}). --spec remove(Name, Func, Timeout) -> Void when + +-spec remove(Name, Func, Timeout) -> 'ok' when Name :: name(), Func :: dbg_fun(), - Timeout :: timeout(), - Void :: term(). + Timeout :: timeout(). remove(Name, Func, Timeout) -> send_system_msg(Name, {debug, {remove, Func}}, Timeout). @@ -245,18 +254,13 @@ mfa(Name, {change_code, Mod, Vsn, Extra}) -> {sys, change_code, [Name, Mod, Vsn, Extra]}; mfa(Name, Atom) -> {sys, Atom, [Name]}. + mfa(Name, Req, Timeout) -> {M, F, A} = mfa(Name, Req), {M, F, A ++ [Timeout]}. %%----------------------------------------------------------------- %% Func: handle_system_msg/6 -%% Args: Msg ::= term() -%% From ::= {pid(),Ref} but don't count on that -%% Parent ::= pid() -%% Module ::= atom() -%% Debug ::= [debug_opts()] -%% Misc ::= term() %% Purpose: Used by a process module that wishes to take care of %% system messages. The process receives a {system, From, %% Msg} message, and passes the Msg to this function. @@ -268,14 +272,14 @@ mfa(Name, Req, Timeout) -> %% The Module must export system_continue/3, system_terminate/4 %% and format_status/2 for status information. %%----------------------------------------------------------------- --spec handle_system_msg(Msg, From, Parent, Module, Debug, Misc) -> Void when +-spec handle_system_msg(Msg, From, Parent, Module, Debug, Misc) -> + no_return() when Msg :: term(), From :: {pid(), Tag :: _}, Parent :: pid(), Module :: module(), Debug :: [dbg_opt()], - Misc :: term(), - Void :: term(). + Misc :: term(). handle_system_msg(Msg, From, Parent, Module, Debug, Misc) -> handle_system_msg(running, Msg, From, Parent, Module, Debug, Misc, false). @@ -294,10 +298,6 @@ handle_system_msg(SysState, Msg, From, Parent, Mod, Debug, Misc, Hib) -> %%----------------------------------------------------------------- %% Func: handle_debug/4 -%% Args: Debug ::= [debug_opts()] -%% Func ::= {M,F} | fun() arity 3 -%% State ::= term() -%% Event ::= {in, Msg} | {in, Msg, From} | {out, Msg, To} | term() %% Purpose: Called by a process that wishes to debug an event. %% Func is a formatting function, called as Func(Device, Event). %% Returns: [debug_opts()] @@ -453,6 +453,7 @@ print_event(Dev, {Event, State, FormFunc}) -> FormFunc(Dev, Event, State). init_stat() -> {erlang:localtime(), process_info(self(), reductions), 0, 0}. + get_stat({Time, {reductions, Reds}, In, Out}) -> {reductions, Reds2} = process_info(self(), reductions), [{start_time, Time}, {current_time, erlang:localtime()}, @@ -492,9 +493,8 @@ get_debug2(Item, Debug, Default) -> _ -> Default end. --spec print_log(Debug) -> Void when - Debug :: [dbg_opt()], - Void :: term(). +-spec print_log(Debug) -> 'ok' when + Debug :: [dbg_opt()]. print_log(Debug) -> {_N, Logs} = get_debug(log, Debug, {0, []}), lists:foreach(fun print_event/1, @@ -511,8 +511,6 @@ close_log_file(Debug) -> %%----------------------------------------------------------------- %% Func: debug_options/1 -%% Args: [trace|log|{log,N}|statistics|{log_to_file, FileName}| -%% {install, {Func, FuncState}}] %% Purpose: Initiate a debug structure. Called by a process that %% wishes to initiate the debug structure without the %% system messages. @@ -521,7 +519,11 @@ close_log_file(Debug) -> -spec debug_options(Options) -> [dbg_opt()] when Options :: [Opt], - Opt :: 'trace' | 'log' | 'statistics' | {'log_to_file', FileName} + Opt :: 'trace' + | 'log' + | {'log', pos_integer()} + | 'statistics' + | {'log_to_file', FileName} | {'install', FuncSpec}, FileName :: file:name(), FuncSpec :: {Func, FuncState}, @@ -529,6 +531,7 @@ close_log_file(Debug) -> FuncState :: term(). debug_options(Options) -> debug_options(Options, []). + debug_options([trace | T], Debug) -> debug_options(T, install_debug(trace, true, Debug)); debug_options([log | T], Debug) -> diff --git a/lib/stdlib/test/base64_SUITE.erl b/lib/stdlib/test/base64_SUITE.erl index 7b8650f224..b28df94221 100644 --- a/lib/stdlib/test/base64_SUITE.erl +++ b/lib/stdlib/test/base64_SUITE.erl @@ -1,3 +1,4 @@ +%% -*- coding: utf-8 -*- %% %% %CopyrightBegin% %% @@ -179,7 +180,7 @@ mime_decode(Config) when is_list(Config) -> <<"o">> = base64:mime_decode(<<"b=w=====">>), %% Test misc white space and illegals with embedded padding <<"one">> = base64:mime_decode(<<" b~2=\r\n5()l===">>), - <<"on">> = base64:mime_decode(<<"\tb =2\"�4=�= ==">>), + <<"on">> = base64:mime_decode(<<"\tb =2\"¤4=¤= ==">>), <<"o">> = base64:mime_decode(<<"\nb=w=====">>), %% Two pads <<"Aladdin:open sesame">> = @@ -188,7 +189,7 @@ mime_decode(Config) when is_list(Config) -> <<"Hello World!!">> = base64:mime_decode(<<"SGVsb)(G8gV29ybGQ=h IQ= =">>), %% No pad <<"Aladdin:open sesam">> = - base64:mime_decode("QWxhZGRpbjpvcG�\")(VuIHNlc2Ft"), + base64:mime_decode("QWxhZGRpbjpvcG¤\")(VuIHNlc2Ft"), %% Encoded base 64 strings may be divided by non base 64 chars. %% In this cases whitespaces. <<"0123456789!@#0^&*();:<>,. []{}">> = @@ -222,7 +223,7 @@ mime_decode_to_string(Config) when is_list(Config) -> "o" = base64:mime_decode_to_string(<<"b=w=====">>), %% Test misc white space and illegals with embedded padding "one" = base64:mime_decode_to_string(<<" b~2=\r\n5()l===">>), - "on" = base64:mime_decode_to_string(<<"\tb =2\"�4=�= ==">>), + "on" = base64:mime_decode_to_string(<<"\tb =2\"¤4=¤= ==">>), "o" = base64:mime_decode_to_string(<<"\nb=w=====">>), %% Two pads "Aladdin:open sesame" = @@ -231,7 +232,7 @@ mime_decode_to_string(Config) when is_list(Config) -> "Hello World!!" = base64:mime_decode_to_string(<<"SGVsb)(G8gV29ybGQ=h IQ= =">>), %% No pad "Aladdin:open sesam" = - base64:mime_decode_to_string("QWxhZGRpbjpvcG�\")(VuIHNlc2Ft"), + base64:mime_decode_to_string("QWxhZGRpbjpvcG¤\")(VuIHNlc2Ft"), %% Encoded base 64 strings may be divided by non base 64 chars. %% In this cases whitespaces. "0123456789!@#0^&*();:<>,. []{}" = diff --git a/lib/stdlib/test/binary_module_SUITE.erl b/lib/stdlib/test/binary_module_SUITE.erl index bac59a3107..9b6f628aa9 100644 --- a/lib/stdlib/test/binary_module_SUITE.erl +++ b/lib/stdlib/test/binary_module_SUITE.erl @@ -1,7 +1,8 @@ +%% -*- coding: utf-8 -*- %% %% %CopyrightBegin% %% -%% Copyright Ericsson AB 1997-2011. All Rights Reserved. +%% Copyright Ericsson AB 1997-2012. All Rights Reserved. %% %% The contents of this file are subject to the Erlang Public License, %% Version 1.1, (the "License"); you may not use this file except in @@ -1331,9 +1332,9 @@ one_random_number(N) -> one_random(N) -> M = ((N - 1) rem 68) + 1, element(M,{$a,$b,$c,$d,$e,$f,$g,$h,$i,$j,$k,$l,$m,$n,$o,$p,$q,$r,$s,$t, - $u,$v,$w,$x,$y,$z,$�,$�,$�,$A,$B,$C,$D,$E,$F,$G,$H, - $I,$J,$K,$L,$M,$N,$O,$P,$Q,$R,$S,$T,$U,$V,$W,$X,$Y,$Z,$�, - $�,$�,$0,$1,$2,$3,$4,$5,$6,$7,$8,$9}). + $u,$v,$w,$x,$y,$z,$å,$ä,$ö,$A,$B,$C,$D,$E,$F,$G,$H, + $I,$J,$K,$L,$M,$N,$O,$P,$Q,$R,$S,$T,$U,$V,$W,$X,$Y,$Z,$Å, + $Ä,$Ö,$0,$1,$2,$3,$4,$5,$6,$7,$8,$9}). random_number({Min,Max}) -> % Min and Max are *length* of number in % decimal positions diff --git a/lib/stdlib/test/digraph_SUITE.erl b/lib/stdlib/test/digraph_SUITE.erl index 1d1326d60e..ed01b32a59 100644 --- a/lib/stdlib/test/digraph_SUITE.erl +++ b/lib/stdlib/test/digraph_SUITE.erl @@ -1,7 +1,7 @@ %% %% %CopyrightBegin% %% -%% Copyright Ericsson AB 1996-2011. All Rights Reserved. +%% Copyright Ericsson AB 1996-2012. All Rights Reserved. %% %% The contents of this file are subject to the Erlang Public License, %% Version 1.1, (the "License"); you may not use this file except in @@ -400,7 +400,7 @@ sane1(G) -> lists:foreach( fun(V) -> InEs = digraph:in_edges(G, V), - %% Nu har man *alla* inkanter f�r V + %% *All* in-edoges of V lists:foreach( fun(E) -> case digraph:edge(G, E) of diff --git a/lib/stdlib/test/epp_SUITE.erl b/lib/stdlib/test/epp_SUITE.erl index 77c615d6d9..606bbbcbb2 100644 --- a/lib/stdlib/test/epp_SUITE.erl +++ b/lib/stdlib/test/epp_SUITE.erl @@ -25,7 +25,7 @@ variable_1/1, otp_4870/1, otp_4871/1, otp_5362/1, pmod/1, not_circular/1, skip_header/1, otp_6277/1, otp_7702/1, otp_8130/1, overload_mac/1, otp_8388/1, otp_8470/1, otp_8503/1, - otp_8562/1, otp_8665/1, otp_8911/1]). + otp_8562/1, otp_8665/1, otp_8911/1, otp_10302/1]). -export([epp_parse_erl_form/2]). @@ -67,7 +67,7 @@ all() -> {group, variable}, otp_4870, otp_4871, otp_5362, pmod, not_circular, skip_header, otp_6277, otp_7702, otp_8130, overload_mac, otp_8388, otp_8470, otp_8503, otp_8562, - otp_8665, otp_8911]. + otp_8665, otp_8911, otp_10302]. groups() -> [{upcase_mac, [], [upcase_mac_1, upcase_mac_2]}, @@ -582,12 +582,13 @@ otp_8130(suite) -> otp_8130(Config) when is_list(Config) -> true = os:putenv("epp_inc1", "stdlib"), Ts = [{otp_8130_1, - %% The scanner handles UNICODE in a special way. Hopefully - %% temporarily. <<"-define(M(A), ??A). " "t() -> " - " \"{ 34 , [ $1 , 2730 ] , \\\"34\\\" , X . a , 2730 }\" = " - " ?M({34,\"1\\x{aaa}\",\"34\",X.a,$\\x{aaa}}), ok. ">>, + " L = \"{ 34 , \\\"1\\\\x{AAA}\\\" , \\\"34\\\" , X . a , $\\\\x{AAA} }\", " + " R = ?M({34,\"1\\x{aaa}\",\"34\",X.a,$\\x{aaa}})," + " Lt = erl_scan:string(L, 1, [unicode])," + " Rt = erl_scan:string(R, 1, [unicode])," + " Lt = Rt, ok. ">>, ok}, {otp_8130_2, @@ -1284,6 +1285,75 @@ otp_8665(Config) when is_list(Config) -> ?line [] = compile(Config, Cs), ok. +otp_10302(doc) -> + "OTP-10302. Unicode characters scanner/parser."; +otp_10302(suite) -> + []; +otp_10302(Config) when is_list(Config) -> + %% Two messages (one too many). Keeps otp_4871 happy. + Cs = [{otp_8562, + <<"%% coding: utf-8\n \n \x{E4}">>, + {errors,[{3,epp,cannot_parse}, + {3,file_io_server,invalid_unicode}],[]}} + ], + [] = compile(Config, Cs), + Dir = ?config(priv_dir, Config), + File = filename:join(Dir, "otp_10302.erl"), + utf8 = encoding("coding: utf-8", File), + utf8 = encoding("coding: UTF-8", File), + latin1 = encoding("coding: Latin-1", File), + latin1 = encoding("coding: latin-1", File), + none = encoding_com("coding: utf-8", File), + none = encoding_com("\n\n%% coding: utf-8", File), + none = encoding_nocom("\n\n coding: utf-8", File), + utf8 = encoding_com("\n%% coding: utf-8", File), + utf8 = encoding_nocom("\n coding: utf-8", File), + none = encoding("coding: \nutf-8", File), + latin1 = encoding("Encoding : latin-1", File), + utf8 = encoding("ccoding: UTF-8", File), + utf8 = encoding("coding= utf-8", File), + utf8 = encoding_com(" %% coding= utf-8", File), + utf8 = encoding("coding = utf-8", File), + none = encoding("coding: utf-16 coding: utf-8", File), %first is bad + none = encoding("Coding: utf-8", File), %capital c + utf8 = encoding("-*- coding: utf-8 -*-", File), + utf8 = encoding("-*-coding= utf-8-*-", File), + utf8 = encoding("codingcoding= utf-8", File), + ok = prefix("coding: utf-8", File, utf8), + + "coding: latin-1" = epp:encoding_to_string(latin1), + "coding: utf-8" = epp:encoding_to_string(utf8), + true = lists:member(epp:default_encoding(), [latin1, utf8]), + + ok. + +prefix(S, File, Enc) -> + prefix(0, S, File, Enc). + +prefix(100, _S, _File, _) -> + ok; +prefix(N, S, File, Enc) -> + Enc = encoding(lists:duplicate(N, $\s) ++ S, File), + prefix(N+1, S, File, Enc). + +encoding(Enc, File) -> + E = encoding_com("%% " ++ Enc, File), + none = encoding_com(Enc, File), + E = encoding_nocom(Enc, File). + +encoding_com(Enc, File) -> + ok = file:write_file(File, Enc), + {ok, Fd} = file:open(File, [read]), + E = epp:set_encoding(Fd), + ok = file:close(Fd), + E = epp:read_encoding(File). + +encoding_nocom(Enc, File) -> + ok = file:write_file(File, Enc), + {ok, Fd} = file:open(File, [read]), + ok = file:close(Fd), + epp:read_encoding(File, [{in_comment_only, false}]). + check(Config, Tests) -> eval_tests(Config, fun check_test/2, Tests). diff --git a/lib/stdlib/test/erl_expand_records_SUITE.erl b/lib/stdlib/test/erl_expand_records_SUITE.erl index e248934e10..e51c05a22c 100644 --- a/lib/stdlib/test/erl_expand_records_SUITE.erl +++ b/lib/stdlib/test/erl_expand_records_SUITE.erl @@ -107,8 +107,7 @@ attributes(doc) -> attributes(suite) -> []; attributes(Config) when is_list(Config) -> Ts = [ - <<"-import(erl_expand_records_SUITE). - -import(lists, [append/2, reverse/1]). + <<"-import(lists, [append/2, reverse/1]). -record(r, {a,b}). @@ -158,12 +157,12 @@ expr(Config) when is_list(Config) -> 2 = fun(X) -> X end(One + One), 3 = fun exprec_test:f/1(3), 4 = exprec_test:f(4), - 5 = ''.f(5), + 5 = f(5), L = receive {a,message,L0} -> L0 end, - case catch a.b.c:foo(bar) of + case catch a:foo(bar) of {'EXIT', _} -> ok end, _ = receive %Suppress warning. @@ -263,8 +262,7 @@ pattern(doc) -> pattern(suite) -> []; pattern(Config) when is_list(Config) -> Ts = [ - <<"-import(erl_expand_records_SUITE). - -import(lists, [append/2, reverse/1]). + <<"-import(lists, [append/2, reverse/1]). -record(r, {a,b}). @@ -292,10 +290,10 @@ pattern(Config) when is_list(Config) -> 21 = t(#r{a = #r{}}), 22 = t(2), 23 = t(#r{a = #r{}, b = b}), - 24 = t(a.b.c), + 24 = t(abc), ok. - t(a.b.c) -> + t(abc) -> 24; t($a) -> 2; diff --git a/lib/stdlib/test/erl_lint_SUITE.erl b/lib/stdlib/test/erl_lint_SUITE.erl index 90a37f6441..774229fca9 100644 --- a/lib/stdlib/test/erl_lint_SUITE.erl +++ b/lib/stdlib/test/erl_lint_SUITE.erl @@ -1732,7 +1732,7 @@ otp_5362(Config) when is_list(Config) -> {otp_5362_2, <<"-export([inline/0]). - -import(lists.foo, [a/1,b/1]). % b/1 is not used + -import(lists, [a/1,b/1]). % b/1 is not used -compile([{inline,{inl,7}}]). % undefined -compile([{inline,[{inl,17}]}]). % undefined @@ -1764,7 +1764,7 @@ otp_5362(Config) when is_list(Config) -> {6,erl_lint,{bad_inline,{inl,17}}}, {11,erl_lint,{undefined_function,{fipp,0}}}, {22,erl_lint,{bad_nowarn_unused_function,{and_not_used,2}}}], - [{3,erl_lint,{unused_import,{{b,1},'lists.foo'}}}, + [{3,erl_lint,{unused_import,{{b,1},lists}}}, {9,erl_lint,{unused_function,{foop,0}}}, {19,erl_lint,{unused_function,{not_used,0}}}, {23,erl_lint,{unused_function,{and_not_used,1}}}]}}, diff --git a/lib/stdlib/test/erl_pp_SUITE.erl b/lib/stdlib/test/erl_pp_SUITE.erl index 64853ca078..db416b03b0 100644 --- a/lib/stdlib/test/erl_pp_SUITE.erl +++ b/lib/stdlib/test/erl_pp_SUITE.erl @@ -1,7 +1,7 @@ %% %% %CopyrightBegin% %% -%% Copyright Ericsson AB 2006-2011. All Rights Reserved. +%% Copyright Ericsson AB 2006-2012. All Rights Reserved. %% %% The contents of this file are subject to the Erlang Public License, %% Version 1.1, (the "License"); you may not use this file except in @@ -40,7 +40,7 @@ init_per_testcase/2, end_per_testcase/2]). -export([ func/1, call/1, recs/1, try_catch/1, if_then/1, - receive_after/1, bits/1, head_tail/1, package/1, + receive_after/1, bits/1, head_tail/1, cond1/1, block/1, case1/1, ops/1, messages/1, old_mnemosyne_syntax/1, import_export/1, misc_attrs/1, @@ -48,7 +48,8 @@ neg_indent/1, otp_6321/1, otp_6911/1, otp_6914/1, otp_8150/1, otp_8238/1, - otp_8473/1, otp_8522/1, otp_8567/1, otp_8664/1, otp_9147/1]). + otp_8473/1, otp_8522/1, otp_8567/1, otp_8664/1, otp_9147/1, + otp_10302/1]). %% Internal export. -export([ehook/6]). @@ -74,12 +75,12 @@ all() -> groups() -> [{expr, [], [func, call, recs, try_catch, if_then, receive_after, - bits, head_tail, package, cond1, block, case1, ops, + bits, head_tail, cond1, block, case1, ops, messages, old_mnemosyne_syntax]}, {attributes, [], [misc_attrs, import_export]}, {tickets, [], [otp_6321, otp_6911, otp_6914, otp_8150, otp_8238, - otp_8473, otp_8522, otp_8567, otp_8664, otp_9147]}]. + otp_8473, otp_8522, otp_8567, otp_8664, otp_9147, otp_10302]}]. init_per_suite(Config) -> Config. @@ -438,9 +439,6 @@ bits(Config) when is_list(Config) -> ?line ok = pp_expr(<<"<<{a,b}/binary>>">>), ?line ok = pp_expr(<<"<<{foo:bar(),b}/binary>>">>), ?line ok = pp_expr(<<"<<(foo:bar()):(foo:bar())/binary>>">>), - ?line ok = pp_expr(<<"<<(foo.bar)/binary>>">>), - ?line ok = pp_expr(<<"<<(foo.bar):all/binary>>">>), - ?line ok = pp_expr(<<"<<(foo.bar):(foo.bar)/binary>>">>), ok. head_tail(suite) -> @@ -462,17 +460,6 @@ head_tail(Config) when is_list(Config) -> ?line compile(Config, Ts), ok. -package(suite) -> - []; -package(Config) when is_list(Config) -> - Ts = [{package_1, - <<"t() -> a.b:foo().">>}, - {package_2, - <<"t() -> .lists:sort([]).">>} - ], - ?line compile(Config, Ts), - ok. - cond1(suite) -> []; cond1(Config) when is_list(Config) -> @@ -614,13 +601,11 @@ misc_attrs(suite) -> []; misc_attrs(Config) when is_list(Config) -> ?line ok = pp_forms(<<"-module(m). ">>), - ?line ok = pp_forms(<<"-module(m.p, [A,B]). ">>), ?line ok = pp_forms(<<"-module(m, [Aafjlksfjdlsjflsdfjlsdjflkdsfjlk," "Blsjfdlslfjsdf]). ">>), ?line ok = pp_forms(<<"-export([]). ">>), ?line ok = pp_forms(<<"-export([foo/2, bar/0]). ">>), ?line ok = pp_forms(<<"-export([bar/0]). ">>), - ?line ok = pp_forms(<<"-import(.lists). ">>), ?line ok = pp_forms(<<"-import(lists, []). ">>), ?line ok = pp_forms(<<"-import(lists, [map/2]). ">>), ?line ok = pp_forms(<<"-import(lists, [map/2, foreach/2]). ">>), @@ -634,8 +619,12 @@ misc_attrs(Config) when is_list(Config) -> hook(suite) -> []; hook(Config) when is_list(Config) -> + F = fun(H) -> H end, + do_hook(F). + +do_hook(HookFun) -> Lc = parse_expr(binary_to_list(<<"[X || X <- [1,2,3]].">>)), - H = fun hook/4, + H = HookFun(fun hook/4), Expr = {call,0,{atom,0,fff},[{foo,Lc},{foo,Lc},{foo,Lc}]}, EChars = lists:flatten(erl_pp:expr(Expr, 0, H)), Call = {call,0,{atom,0,foo},[Lc]}, @@ -692,7 +681,7 @@ hook(Config) when is_list(Config) -> GChars2 = erl_pp:guard(G2), ?line true = GChars =:= lists:flatten(GChars2), - EH = {?MODULE, ehook, [foo,bar]}, + EH = HookFun({?MODULE, ehook, [foo,bar]}), XEChars = erl_pp:expr(Expr, -1, EH), ?line true = remove_indentation(EChars) =:= lists:flatten(XEChars), XEChars2 = erl_pp:expr(Expr, EH), @@ -1068,6 +1057,43 @@ otp_9147(Config) when is_list(Config) -> string:tokens(binary_to_list(Bin), "\n")), ok. +otp_10302(doc) -> + "OTP-10302. Unicode characters scanner/parser."; +otp_10302(suite) -> []; +otp_10302(Config) when is_list(Config) -> + Ts = [{uni_1, + <<"t() -> <<(<<\"abc\\x{aaa}\">>):3/binary>>.">>} + ], + compile(Config, Ts), + ok = pp_expr(<<"$\\x{aaa}">>), + ok = pp_expr(<<"\"1\\x{aaa}\"">>), + ok = pp_expr(<<"<<<<\"hej\">>/binary>>">>), + ok = pp_expr(<<"<< <<\"1\\x{aaa}\">>/binary>>">>), + + U = [{encoding,unicode}], + + do_hook(fun(H) -> [{hook,H}] end), + do_hook(fun(H) -> [{hook,H}]++U end), + + ok = pp_expr(<<"$\\x{aaa}">>, [{hook,fun hook/4}]), + + Opts = [{hook, fun unicode_hook/4},{encoding,unicode}], + Lc = parse_expr("[X || X <- [\"\x{400}\",\"\xFF\"]]."), + Expr = {call,0,{atom,0,fff},[{foo,{foo,Lc}},{foo,{foo,Lc}}]}, + EChars = lists:flatten(erl_pp:expr(Expr, 0, Opts)), + Call = {call,0,{atom,0,foo},[{call,0,{atom,0,foo},[Lc]}]}, + Expr2 = {call,0,{atom,0,fff},[Call,Call]}, + EChars2 = erl_pp:exprs([Expr2], U), + EChars = lists:flatten(EChars2), + [$\x{400},$\x{400}] = [C || C <- EChars, C > 255], + + ok = pp_forms(<<"function() -> {\"\x{400}\",$\x{400}}. "/utf8>>, U), + ok = pp_forms("function() -> {\"\x{400}\",$\x{400}}. ", []), + ok. + +unicode_hook({foo,E}, I, P, H) -> + erl_pp:expr({call,0,{atom,0,foo},[E]}, I, P, H). + %%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%% compile(Config, Tests) -> @@ -1137,9 +1163,11 @@ flat_expr(Expr) -> pp_forms(Bin) -> pp_forms(Bin, none). -pp_forms(Bin, Hook) -> - PP1 = (catch parse_and_pp_forms(binary_to_list(Bin), Hook)), - PP2 = (catch parse_and_pp_forms(PP1, Hook)), +pp_forms(Bin, Options) when is_binary(Bin) -> + pp_forms(to_list(Bin, Options), Options); +pp_forms(List, Options) when is_list(List) -> + PP1 = (catch parse_and_pp_forms(List, Options)), + PP2 = (catch parse_and_pp_forms(PP1, Options)), case PP1 =:= PP2 of % same line numbers true -> test_max_line(PP1); @@ -1147,8 +1175,8 @@ pp_forms(Bin, Hook) -> not_ok end. -parse_and_pp_forms(String, Hook) -> - lists:append(lists:map(fun(AF) -> erl_pp:form(AF, Hook) +parse_and_pp_forms(String, Options) -> + lists:append(lists:map(fun(AF) -> erl_pp:form(AF, Options) end, parse_forms(String))). parse_forms(Chars) -> @@ -1158,7 +1186,7 @@ parse_forms(Chars) -> parse_forms2([], _Cont, _Line, Forms) -> lists:reverse(Forms); parse_forms2(String, Cont0, Line, Forms) -> - case erl_scan:tokens(Cont0, String, Line) of + case erl_scan:tokens(Cont0, String, Line, [unicode]) of {done, {ok, Tokens, EndLine}, Chars} -> {ok, Form} = erl_parse:parse_form(Tokens), parse_forms2(Chars, [], EndLine, [Form | Forms]); @@ -1174,10 +1202,12 @@ pp_expr(Bin) -> pp_expr(Bin, none). %% Final dot is added. -pp_expr(Bin, Hook) -> - PP1 = (catch parse_and_pp_expr(binary_to_list(Bin), 0, Hook)), - PPneg = (catch parse_and_pp_expr(binary_to_list(Bin), -1, Hook)), - PP2 = (catch parse_and_pp_expr(PPneg, 0, Hook)), +pp_expr(Bin, Options) when is_binary(Bin) -> + pp_expr(to_list(Bin, Options), Options); +pp_expr(List, Options) when is_list(List) -> + PP1 = (catch parse_and_pp_expr(List, 0, Options)), + PPneg = (catch parse_and_pp_expr(List, -1, Options)), + PP2 = (catch parse_and_pp_expr(PPneg, 0, Options)), if PP1 =:= PP2 -> % same line numbers case @@ -1192,15 +1222,24 @@ pp_expr(Bin, Hook) -> not_ok end. -parse_and_pp_expr(String, Indent, Hook) -> +parse_and_pp_expr(String, Indent, Options) -> StringDot = lists:flatten(String) ++ ".", - erl_pp:expr(parse_expr(StringDot), Indent, Hook). + erl_pp:expr(parse_expr(StringDot), Indent, Options). parse_expr(Chars) -> - {ok, Tokens, _} = erl_scan:string(Chars), + {ok, Tokens, _} = erl_scan:string(Chars, 1, [unicode]), {ok, [Expr]} = erl_parse:parse_exprs(Tokens), Expr. +to_list(Bin, Options) when is_list(Options) -> + case proplists:get_value(encoding, Options) of + unicode -> unicode:characters_to_list(Bin); + encoding -> binary_to_list(Bin); + undefined -> binary_to_list(Bin) + end; +to_list(Bin, _Hook) -> + binary_to_list(Bin). + test_new_line(String) -> case string:chr(String, $\n) of 0 -> ok; diff --git a/lib/stdlib/test/erl_scan_SUITE.erl b/lib/stdlib/test/erl_scan_SUITE.erl index 4298b2c701..34e1b99abe 100644 --- a/lib/stdlib/test/erl_scan_SUITE.erl +++ b/lib/stdlib/test/erl_scan_SUITE.erl @@ -1,7 +1,8 @@ +%% -*- coding: utf-8 -*- %% %% %CopyrightBegin% %% -%% Copyright Ericsson AB 1998-2011. All Rights Reserved. +%% Copyright Ericsson AB 1998-2012. All Rights Reserved. %% %% The contents of this file are subject to the Erlang Public License, %% Version 1.1, (the "License"); you may not use this file except in @@ -20,7 +21,7 @@ -export([all/0, suite/0,groups/0,init_per_suite/1, end_per_suite/1, init_per_group/2,end_per_group/2]). --export([ error_1/1, error_2/1, iso88591/1, otp_7810/1]). +-export([ error_1/1, error_2/1, iso88591/1, otp_7810/1, otp_10302/1]). -import(lists, [nth/2,flatten/1]). -import(io_lib, [print/1]). @@ -59,7 +60,7 @@ end_per_testcase(_Case, Config) -> suite() -> [{ct_hooks,[ts_install_cth]}]. all() -> - [{group, error}, iso88591, otp_7810]. + [{group, error}, iso88591, otp_7810, otp_10302]. groups() -> [{error, [], [error_1, error_2]}]. @@ -131,10 +132,10 @@ iso88591(Config) when is_list(Config) -> ?line ok = case catch begin %% Some atom and variable names - V1s = [$�,$�,$�,$�], - V2s = [$N,$�,$r], - A1s = [$h,$�,$r], - A2s = [$�,$r,$e], + V1s = [$Á,$á,$é,$ë], + V2s = [$N,$ä,$r], + A1s = [$h,$ä,$r], + A2s = [$ö,$r,$e], %% Test parsing atom and variable characters. {ok,Ts1,_} = erl_scan:string(V1s ++ " " ++ V2s ++ "\327" ++ @@ -214,8 +215,8 @@ atoms() -> ?line test_string("'a b'", [{atom,1,'a b'}]), ?line test_string("a", [{atom,1,a}]), ?line test_string("a@2", [{atom,1,a@2}]), - ?line test_string([39,65,200,39], [{atom,1,'A�'}]), - ?line test_string("�rlig �sten", [{atom,1,�rlig},{atom,1,�sten}]), + ?line test_string([39,65,200,39], [{atom,1,'AÈ'}]), + ?line test_string("ärlig östen", [{atom,1,ärlig},{atom,1,östen}]), ?line {ok,[{atom,_,'$a'}],{1,6}} = erl_scan:string("'$\\a'", {1,1}), ?line test("'$\\a'"), @@ -289,7 +290,7 @@ errors() -> ?line {error,{1,erl_scan,{string,$","str"}},1} = %" erl_scan:string("\"str"), %" ?line {error,{1,erl_scan,char},1} = erl_scan:string("$"), - ?line test_string([34,65,200,34], [{string,1,"A�"}]), + ?line test_string([34,65,200,34], [{string,1,"AÈ"}]), ?line test_string("\\", [{'\\',1}]), ?line {'EXIT',_} = (catch {foo, erl_scan:string('$\\a', {1,1})}), % type error @@ -354,7 +355,7 @@ dots() -> {".\n", {ok,[{dot,1}],2}}, {".%", {ok,[{dot,1}],1}}, {".\210",{ok,[{dot,1}],1}}, - {".% �h",{ok,[{dot,1}],1}}, + {".% öh",{ok,[{dot,1}],1}}, {".%\n", {ok,[{dot,1}],2}}, {".$", {error,{1,erl_scan,char},1}}, {".$\\", {error,{1,erl_scan,char},1}}, @@ -369,7 +370,7 @@ dots() -> ?line [{column,1},{length,1},{line,1},{text,"."}] = erl_scan:token_info(T2, [column, length, line, text]), ?line {ok,[{dot,_}=T3],{1,6}} = - erl_scan:string(".% �h", {1,1}, text), + erl_scan:string(".% öh", {1,1}, text), ?line [{column,1},{length,1},{line,1},{text,"."}] = erl_scan:token_info(T3, [column, length, line, text]), ?line {error,{{1,2},erl_scan,char},{1,3}} = @@ -472,11 +473,11 @@ chars() -> variables() -> - ?line test_string(" \237_Aou�eiy��", [{var,1,'_Aou�eiy��'}]), + ?line test_string(" \237_Aouåeiyäö", [{var,1,'_Aouåeiyäö'}]), ?line test_string("A_b_c@", [{var,1,'A_b_c@'}]), ?line test_string("V@2", [{var,1,'V@2'}]), - ?line test_string("ABD�", [{var,1,'ABD�'}]), - ?line test_string("�rlig �sten", [{var,1,'�rlig'},{var,1,'�sten'}]), + ?line test_string("ABDÀ", [{var,1,'ABDÀ'}]), + ?line test_string("Ärlig Östen", [{var,1,'Ärlig'},{var,1,'Östen'}]), ok. eof() -> @@ -823,7 +824,7 @@ unicode() -> ?line {ok,[{char,1,1}],1} = erl_scan:string([$$,$\\,$^,1089]), ?line {error,{1,erl_scan,Error},1} = erl_scan:string("\"qa\x{aaa}"), - ?line "unterminated string starting with \"qa\\x{AAA}\"" = + ?line "unterminated string starting with \"qa"++[2730]++"\"" = erl_scan:format_error(Error), ?line {error,{{1,1},erl_scan,_},{1,11}} = erl_scan:string("\"qa\\x{aaa}",{1,1}), @@ -887,9 +888,10 @@ unicode() -> {char,_,$d},{']',_}],{1,8}} = erl_scan:string(Str1, {1,1}), ?line test(Str1), Comment = "%% "++[1089], - ?line {ok,[{comment,1,[$%,$%,$\s,1089]}],1} = + %% Returned a comment In R15B03: + {error,{1,erl_scan,{illegal,character}},1} = erl_scan:string(Comment, 1, return), - ?line {ok,[{comment,_,[$%,$%,$\s,1089]}],{1,5}} = + {error,{{1,1},erl_scan,{illegal,character}},{1,5}} = erl_scan:string(Comment, {1,1}, return), ok. @@ -958,6 +960,182 @@ more_chars() -> erl_scan:string("$\\xg", {1,1}), ok. +otp_10302(doc) -> + "OTP-10302. Unicode characters scanner/parser."; +otp_10302(suite) -> + []; +otp_10302(Config) when is_list(Config) -> + %% From unicode(): + {error,{1,erl_scan,{illegal,atom}},1} = + erl_scan:string("'a"++[1089]++"b'", 1, unicode), + {error,{{1,1},erl_scan,{illegal,atom}},{1,12}} = + erl_scan:string("'qa\\x{aaa}'",{1,1},unicode), + + {ok,[{char,1,1089}],1} = erl_scan:string([$$,1089], 1, unicode), + {ok,[{char,1,1089}],1} = erl_scan:string([$$,$\\,1089],1,unicode), + + Qs = "$\\x{aaa}", + {ok,[{char,1,2730}],1} = erl_scan:string(Qs,1,unicode), + {ok,[Q2],{1,9}} = erl_scan:string(Qs,{1,1},[unicode,text]), + [{category,char},{column,1},{length,8}, + {line,1},{symbol,16#aaa},{text,Qs}] = + erl_scan:token_info(Q2), + + Tags = [category, column, length, line, symbol, text], + + U1 = "\"\\x{aaa}\"", + {ok,[T1],{1,10}} = erl_scan:string(U1, {1,1}, [unicode,text]), + [{category,string},{column,1},{length,9},{line,1}, + {symbol,[16#aaa]},{text,U1}] = erl_scan:token_info(T1, Tags), + + U2 = "\"\\x41\\x{fff}\\x42\"", + {ok,[{string,1,[65,4095,66]}],1} = erl_scan:string(U2, 1, unicode), + + U3 = "\"a\n\\x{fff}\n\"", + {ok,[{string,1,[97,10,4095,10]}],3} = erl_scan:string(U3, 1,unicode), + + U4 = "\"\\^\n\\x{aaa}\\^\n\"", + {ok,[{string,1,[10,2730,10]}],3} = erl_scan:string(U4, 1,[unicode]), + + Str1 = "\"ab" ++ [1089] ++ "cd\"", + {ok,[{string,1,[97,98,1089,99,100]}],1} = + erl_scan:string(Str1,1,unicode), + {ok,[{string,{1,1},[97,98,1089,99,100]}],{1,8}} = + erl_scan:string(Str1, {1,1},unicode), + + OK1 = 16#D800-1, + OK2 = 16#DFFF+1, + OK3 = 16#FFFE-1, + OK4 = 16#FFFF+1, + OKL = [OK1,OK2,OK3,OK4], + + Illegal1 = 16#D800, + Illegal2 = 16#DFFF, + Illegal3 = 16#FFFE, + Illegal4 = 16#FFFF, + IllegalL = [Illegal1,Illegal2,Illegal3,Illegal4], + + [{ok,[{comment,1,[$%,$%,$\s,OK]}],1} = + erl_scan:string("%% "++[OK], 1, [unicode,return]) || + OK <- OKL], + {ok,[{comment,_,[$%,$%,$\s,OK1]}],{1,5}} = + erl_scan:string("%% "++[OK1], {1,1}, [unicode,return]), + [{error,{1,erl_scan,{illegal,character}},1} = + erl_scan:string("%% "++[Illegal], 1, [unicode,return]) || + Illegal <- IllegalL], + {error,{{1,1},erl_scan,{illegal,character}},{1,5}} = + erl_scan:string("%% "++[Illegal1], {1,1}, [unicode,return]), + + [{ok,[],1} = erl_scan:string("%% "++[OK], 1, [unicode]) || + OK <- OKL], + {ok,[],{1,5}} = erl_scan:string("%% "++[OK1], {1,1}, [unicode]), + [{error,{1,erl_scan,{illegal,character}},1} = + erl_scan:string("%% "++[Illegal], 1, [unicode]) || + Illegal <- IllegalL], + {error,{{1,1},erl_scan,{illegal,character}},{1,5}} = + erl_scan:string("%% "++[Illegal1], {1,1}, [unicode]), + + [{ok,[{string,{1,1},[OK]}],{1,4}} = + erl_scan:string("\""++[OK]++"\"",{1,1},unicode) || + OK <- OKL], + [{error,{{1,2},erl_scan,{illegal,character}},{1,3}} = + erl_scan:string("\""++[OK]++"\"",{1,1},unicode) || + OK <- IllegalL], + + [{error,{{1,1},erl_scan,{illegal,character}},{1,2}} = + erl_scan:string([Illegal],{1,1},unicode) || + Illegal <- IllegalL], + + {ok,[{char,{1,1},OK1}],{1,3}} = + erl_scan:string([$$,OK1],{1,1},unicode), + {error,{{1,1},erl_scan,{illegal,character}},{1,2}} = + erl_scan:string([$$,Illegal1],{1,1},unicode), + + {ok,[{char,{1,1},OK1}],{1,4}} = + erl_scan:string([$$,$\\,OK1],{1,1},unicode), + {error,{{1,1},erl_scan,{illegal,character}},{1,4}} = + erl_scan:string([$$,$\\,Illegal1],{1,1},unicode), + + {ok,[{string,{1,1},[55295]}],{1,5}} = + erl_scan:string("\"\\"++[OK1]++"\"",{1,1},unicode), + {error,{{1,2},erl_scan,{illegal,character}},{1,4}} = + erl_scan:string("\"\\"++[Illegal1]++"\"",{1,1},unicode), + + {ok,[{char,{1,1},OK1}],{1,10}} = + erl_scan:string("$\\x{D7FF}",{1,1},unicode), + {error,{{1,1},erl_scan,{illegal,character}},{1,10}} = + erl_scan:string("$\\x{D800}",{1,1},unicode), + + %% Not erl_scan, but erl_parse. + {integer,0,1} = erl_parse:abstract(1), + Float = 3.14, {float,0,Float} = erl_parse:abstract(Float), + {nil,0} = erl_parse:abstract([]), + {bin,0, + [{bin_element,0,{integer,0,1},default,default}, + {bin_element,0,{integer,0,2},default,default}]} = + erl_parse:abstract(<<1,2>>), + {cons,0,{tuple,0,[{atom,0,a}]},{atom,0,b}} = + erl_parse:abstract([{a} | b]), + {string,0,"str"} = erl_parse:abstract("str"), + {cons,0, + {integer,0,$a}, + {cons,0,{integer,0,1024},{string,0,"c"}}} = + erl_parse:abstract("a"++[1024]++"c"), + + Line = 17, + {integer,Line,1} = erl_parse:abstract(1, Line), + Float = 3.14, {float,Line,Float} = erl_parse:abstract(Float, Line), + {nil,Line} = erl_parse:abstract([], Line), + {bin,Line, + [{bin_element,Line,{integer,Line,1},default,default}, + {bin_element,Line,{integer,Line,2},default,default}]} = + erl_parse:abstract(<<1,2>>, Line), + {cons,Line,{tuple,Line,[{atom,Line,a}]},{atom,Line,b}} = + erl_parse:abstract([{a} | b], Line), + {string,Line,"str"} = erl_parse:abstract("str", Line), + {cons,Line, + {integer,Line,$a}, + {cons,Line,{integer,Line,1024},{string,Line,"c"}}} = + erl_parse:abstract("a"++[1024]++"c", Line), + + Opts1 = [{line,17}], + {integer,Line,1} = erl_parse:abstract(1, Opts1), + Float = 3.14, {float,Line,Float} = erl_parse:abstract(Float, Opts1), + {nil,Line} = erl_parse:abstract([], Opts1), + {bin,Line, + [{bin_element,Line,{integer,Line,1},default,default}, + {bin_element,Line,{integer,Line,2},default,default}]} = + erl_parse:abstract(<<1,2>>, Opts1), + {cons,Line,{tuple,Line,[{atom,Line,a}]},{atom,Line,b}} = + erl_parse:abstract([{a} | b], Opts1), + {string,Line,"str"} = erl_parse:abstract("str", Opts1), + {cons,Line, + {integer,Line,$a}, + {cons,Line,{integer,Line,1024},{string,Line,"c"}}} = + erl_parse:abstract("a"++[1024]++"c", Opts1), + + [begin + {integer,Line,1} = erl_parse:abstract(1, Opts2), + Float = 3.14, {float,Line,Float} = erl_parse:abstract(Float, Opts2), + {nil,Line} = erl_parse:abstract([], Opts2), + {bin,Line, + [{bin_element,Line,{integer,Line,1},default,default}, + {bin_element,Line,{integer,Line,2},default,default}]} = + erl_parse:abstract(<<1,2>>, Opts2), + {cons,Line,{tuple,Line,[{atom,Line,a}]},{atom,Line,b}} = + erl_parse:abstract([{a} | b], Opts2), + {string,Line,"str"} = erl_parse:abstract("str", Opts2), + {string,Line,[97,1024,99]} = + erl_parse:abstract("a"++[1024]++"c", Opts2) + end || Opts2 <- [[{encoding,unicode},{line,Line}], + [{encoding,utf8},{line,Line}]]], + + {cons,0, + {integer,0,97}, + {cons,0,{integer,0,1024},{string,0,"c"}}} = + erl_parse:abstract("a"++[1024]++"c", [{encoding,latin1}]), + ok. + test_string(String, Expected) -> {ok, Expected, _End} = erl_scan:string(String), test(String). diff --git a/lib/stdlib/test/escript_SUITE.erl b/lib/stdlib/test/escript_SUITE.erl index 5b592c65cc..7634c21a17 100644 --- a/lib/stdlib/test/escript_SUITE.erl +++ b/lib/stdlib/test/escript_SUITE.erl @@ -34,7 +34,8 @@ create_and_extract/1, foldl/1, overflow/1, - verify_sections/3 + verify_sections/3, + unicode/1 ]). -include_lib("test_server/include/test_server.hrl"). @@ -46,7 +47,7 @@ all() -> [basic, errors, strange_name, emulator_flags, module_script, beam_script, archive_script, epp, create_and_extract, foldl, overflow, - archive_script_file_access]. + archive_script_file_access, unicode]. groups() -> []. @@ -810,6 +811,8 @@ normalize_sections(Sections) -> end. %%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%% + + foldl(Config) when is_list(Config) -> {NewFile, _FileInfo, _EmuArg, _Source, @@ -887,6 +890,20 @@ emulate_escript_foldl(Fun, Acc, File) -> {error, Reason} end. +unicode(Config) when is_list(Config) -> + Data = ?config(data_dir, Config), + Dir = filename:absname(Data), %Get rid of trailing slash. + run(Dir, "unicode1", + [<<"escript: exception error: an error occurred when evaluating" + " an arithmetic expression\n in operator '/'/2\n " + "called as <<170>> / <<170>>\nExitCode:127">>]), + run(Dir, "unicode2", + [<<"escript: exception error: an error occurred when evaluating" + " an arithmetic expression\n in operator '/'/2\n " + "called as <<\"\xaa\">> / <<\"\xaa\">>\nExitCode:127">>]), + run(Dir, "unicode3", [<<"ExitCode:0">>]), + ok. + %%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%% overflow(Config) when is_list(Config) -> diff --git a/lib/stdlib/test/escript_SUITE_data/unicode1 b/lib/stdlib/test/escript_SUITE_data/unicode1 new file mode 100755 index 0000000000..a77574625e --- /dev/null +++ b/lib/stdlib/test/escript_SUITE_data/unicode1 @@ -0,0 +1,14 @@ +#!/usr/bin/env escript +%% -*- erlang -*- + +-export([main/1]). + +main(_) -> + ok = io:setopts([{encoding,unicode}]), + _D = erlang:system_flag(backtrace_depth, 0), + A = <<"\x{aa}">>, + S = lists:flatten(io_lib:format("~p/~p.", [A, A])), + {ok, Ts, _} = erl_scan:string(S, 1, [unicode]), + {ok, Es} = erl_parse:parse_exprs(Ts), + B = erl_eval:new_bindings(), + erl_eval:exprs(Es, B). diff --git a/lib/stdlib/test/escript_SUITE_data/unicode2 b/lib/stdlib/test/escript_SUITE_data/unicode2 new file mode 100755 index 0000000000..495188f6f0 --- /dev/null +++ b/lib/stdlib/test/escript_SUITE_data/unicode2 @@ -0,0 +1,14 @@ +#!/usr/bin/env escript +%% -*- erlang -*- + +-export([main/1]). + +main(_) -> + ok = io:setopts([{encoding,latin1}]), + _D = erlang:system_flag(backtrace_depth, 0), + A = <<"\x{aa}">>, + S = lists:flatten(io_lib:format("~p/~p.", [A, A])), + {ok, Ts, _} = erl_scan:string(S, 1, [unicode]), + {ok, Es} = erl_parse:parse_exprs(Ts), + B = erl_eval:new_bindings(), + erl_eval:exprs(Es, B). diff --git a/lib/stdlib/test/escript_SUITE_data/unicode3 b/lib/stdlib/test/escript_SUITE_data/unicode3 new file mode 100755 index 0000000000..944487dcae --- /dev/null +++ b/lib/stdlib/test/escript_SUITE_data/unicode3 @@ -0,0 +1,13 @@ +#!/usr/bin/env escript +%% -*- erlang; coding: utf-8 -*- + +-export([main/1]). + +main(_) -> + ok = io:setopts([{encoding,unicode}]), + Bin1 = <<"örn_Ѐ שלום-שלום+של 日本語">>, + + L = [246,114,110,95,1024,32,1513,1500,1493,1501,45,1513,1500,1493, + 1501,43,1513,1500,32,26085,26412,35486], + L = unicode:characters_to_list(Bin1, utf8), + ok. diff --git a/lib/stdlib/test/ets_SUITE.erl b/lib/stdlib/test/ets_SUITE.erl index d5c1c5276e..dc17e5d33c 100644 --- a/lib/stdlib/test/ets_SUITE.erl +++ b/lib/stdlib/test/ets_SUITE.erl @@ -2170,20 +2170,29 @@ heir_do(Opts) -> ?line undefined = ets:info(foo), %% When heir dies and pid reused before founder dies - NextPidIx = erts_debug:get_internal_state(next_pid), - {Founder4,MrefF4} = my_spawn_monitor(fun()->heir_founder(Master,"The dying heir",Opts)end), - {Heir4,MrefH4} = my_spawn_monitor(fun()->heir_heir(Founder4)end), - Founder4 ! {go, Heir4}, - ?line {'DOWN', MrefH4, process, Heir4, normal} = receive_any(), - erts_debug:set_internal_state(next_pid, NextPidIx), - {Heir4,MrefH4_B} = spawn_monitor_with_pid(Heir4, - fun()-> ?line die_please = receive_any() end), - Founder4 ! die_please, - ?line {'DOWN', MrefF4, process, Founder4, normal} = receive_any(), - Heir4 ! die_please, - ?line {'DOWN', MrefH4_B, process, Heir4, normal} = receive_any(), - ?line undefined = ets:info(foo), - + repeat_while(fun() -> + NextPidIx = erts_debug:get_internal_state(next_pid), + {Founder4,MrefF4} = my_spawn_monitor(fun()->heir_founder(Master,"The dying heir",Opts)end), + {Heir4,MrefH4} = my_spawn_monitor(fun()->heir_heir(Founder4)end), + Founder4 ! {go, Heir4}, + ?line {'DOWN', MrefH4, process, Heir4, normal} = receive_any(), + erts_debug:set_internal_state(next_pid, NextPidIx), + DoppelGanger = spawn_monitor_with_pid(Heir4, + fun()-> ?line die_please = receive_any() end), + Founder4 ! die_please, + ?line {'DOWN', MrefF4, process, Founder4, normal} = receive_any(), + case DoppelGanger of + {Heir4,MrefH4_B} -> + Heir4 ! die_please, + ?line {'DOWN', MrefH4_B, process, Heir4, normal} = receive_any(), + ?line undefined = ets:info(foo), + false; + failed -> + io:format("Failed to spawn process with pid ~p\n", [Heir4]), + true % try again + end + end), + ?line verify_etsmem(EtsMem). heir_founder(Master, HeirData, Opts) -> @@ -5787,25 +5796,20 @@ receive_any_spinning(Loops, N, Tries) when N>0 -> spawn_monitor_with_pid(Pid, Fun) when is_pid(Pid) -> - spawn_monitor_with_pid(Pid, Fun, 1, 10). + spawn_monitor_with_pid(Pid, Fun, 10). -spawn_monitor_with_pid(Pid, Fun, N, M) when N > M*10 -> - spawn_monitor_with_pid(Pid, Fun, N, M*10); -spawn_monitor_with_pid(Pid, Fun, N, M) -> - ?line false = is_process_alive(Pid), +spawn_monitor_with_pid(_, _, 0) -> + failed; +spawn_monitor_with_pid(Pid, Fun, N) -> case my_spawn(fun()-> case self() of Pid -> Fun(); _ -> die end end) of - Pid -> + Pid -> {Pid, erlang:monitor(process, Pid)}; Other -> - case N rem M of - 0 -> io:format("Failed ~p times to get pid ~p (current = ~p)\n",[N,Pid,Other]); - _ -> ok - end, - spawn_monitor_with_pid(Pid,Fun,N+1,M) + spawn_monitor_with_pid(Pid,Fun,N-1) end. diff --git a/lib/stdlib/test/io_SUITE.erl b/lib/stdlib/test/io_SUITE.erl index 74fcdcc7d2..521d7255ea 100644 --- a/lib/stdlib/test/io_SUITE.erl +++ b/lib/stdlib/test/io_SUITE.erl @@ -1,7 +1,8 @@ +%% -*- coding: utf-8 -*- %% %% %CopyrightBegin% %% -%% Copyright Ericsson AB 1999-2011. All Rights Reserved. +%% Copyright Ericsson AB 1999-2013. All Rights Reserved. %% %% The contents of this file are subject to the Erlang Public License, %% Version 1.1, (the "License"); you may not use this file except in @@ -28,7 +29,7 @@ manpage/1, otp_6708/1, otp_7084/1, otp_7421/1, io_lib_collect_line_3_wb/1, cr_whitespace_in_string/1, io_fread_newlines/1, otp_8989/1, io_lib_fread_literal/1, - io_lib_print_binary_depth_one/1]). + io_lib_print_binary_depth_one/1, otp_10302/1]). %-define(debug, true). @@ -64,7 +65,7 @@ all() -> manpage, otp_6708, otp_7084, otp_7421, io_lib_collect_line_3_wb, cr_whitespace_in_string, io_fread_newlines, otp_8989, io_lib_fread_literal, - io_lib_print_binary_depth_one]. + io_lib_print_binary_depth_one, otp_10302]. groups() -> []. @@ -894,7 +895,7 @@ otp_6354(Config) when is_list(Config) -> ?line "\"\\b\\t\\n\\v\\f\\r\\e\250\"" = p([8,9,10,11,12,13,27,168], 1, 40, -1), % ?line "\"\\b\\t\\n\"\n \"\\v\\f\\r\"\n \"\\e\250\"" = - ?line "\"\\b\\t\\n\\v\\f\\r\\e�\"" = + ?line "\"\\b\\t\\n\\v\\f\\r\\e¨\"" = p([8,9,10,11,12,13,27,168], 1, 10, -1), ?line "\"\\b\\t\\n\\v\\f\\r\\e\250\"" = p([8,9,10,11,12,13,27,168], 1, 40, 100), @@ -2034,3 +2035,44 @@ io_lib_print_binary_depth_one(Suite) when is_list(Suite) -> ?line "<<...>>" = fmt("~W", [<<1:7>>, 1]), ?line "<<...>>" = fmt("~P", [<<1:7>>, 1]), ok. + +otp_10302(doc) -> + "OTP-10302. Unicode"; +otp_10302(Suite) when is_list(Suite) -> + "\"\x{400}\"" = pretty("\x{400}", -1), + "<<\"\x{400}\"/utf8>>" = pretty(<<"\x{400}"/utf8>>, -1), + + "<<\"\x{400}foo\"/utf8>>" = pretty(<<"\x{400}foo"/utf8>>, 2), + "<<\"äppl\"/utf8>>" = pretty(<<"äppl"/utf8>>, 2), + "<<\"äppl\"/utf8...>>" = pretty(<<"äpple"/utf8>>, 2), + "<<\"apel\">>" = pretty(<<"apel">>, 2), + "<<\"apel\"...>>" = pretty(<<"apelsin">>, 2), + "<<228,112,112,108>>" = fmt("~tp", [<<"äppl">>]), + "<<228,...>>" = fmt("~tP", [<<"äppl">>, 2]), + + Chars = lists:seq(0, 512), % just a few... + [] = [C || C <- Chars, S <- io_lib:write_unicode_char_as_latin1(C), + not is_latin1(S)], + L1 = [S || C <- Chars, S <- io_lib:write_unicode_char(C), + not is_latin1(S)], + L1 = lists:seq(256, 512), + + [] = [C || C <- Chars, S <- io_lib:write_unicode_string_as_latin1([C]), + not is_latin1(S)], + L2 = [S || C <- Chars, S <- io_lib:write_unicode_string([C]), + not is_latin1(S)], + L2 = lists:seq(256, 512), + + ok. + +pretty(Term, Depth) when is_integer(Depth) -> + Opts = [{column, 1}, {line_length, 20}, + {depth, Depth}, {max_chars, 60}, + {encoding, unicode}], + pretty(Term, Opts); +pretty(Term, Opts) when is_list(Opts) -> + R = io_lib_pretty:print(Term, Opts), + lists:flatten(io_lib:format("~ts", [R])). + +is_latin1(S) -> + S >= 0 andalso S =< 255. diff --git a/lib/stdlib/test/io_proto_SUITE.erl b/lib/stdlib/test/io_proto_SUITE.erl index 17e69f7c1c..299daf0e42 100644 --- a/lib/stdlib/test/io_proto_SUITE.erl +++ b/lib/stdlib/test/io_proto_SUITE.erl @@ -736,7 +736,7 @@ binary_options(Config) when is_list(Config) -> {getline_re, ".*<<\"hej\\\\n\">>"}, {putline, "io:get_line('')."}, {putline, binary_to_list(<<"\345\344\366"/utf8>>)}, - {getline_re, ".*<<\""++binary_to_list(unicode:characters_to_binary(<<"\345\344\366"/utf8>>,latin1,utf8))++"\\\\n\">>"} + {getline_re, ".*<<\""++binary_to_list(<<"\345\344\366"/utf8>>)++"\\\\n\"/utf8>>"} ],[],[],"-oldshell"), ok. diff --git a/lib/stdlib/test/qlc_SUITE.erl b/lib/stdlib/test/qlc_SUITE.erl index e3090e4a47..cac8309bd9 100644 --- a/lib/stdlib/test/qlc_SUITE.erl +++ b/lib/stdlib/test/qlc_SUITE.erl @@ -6062,21 +6062,6 @@ otp_6673(Config) when is_list(Config) -> ], ?line run(Config, Ts_RT), - %% Ulf Wiger provided a patch that makes QLC work with packages: - Dir = filename:join(?privdir, "p"), - ?line ok = filelib:ensure_dir(filename:join(Dir, ".")), - File = filename:join(Dir, "p.erl"), - ?line ok = file:write_file(File, - <<"-module(p.p).\n" - "-export([q/0]).\n" - "-include_lib(\"stdlib/include/qlc.hrl\").\n" - "q() ->\n" - " .qlc:q([X || X <- [1,2]]).">>), - ?line {ok, 'p.p'} = compile:file(File, [{outdir,Dir}]), - ?line code:purge('p.p'), - ?line {module, 'p.p'} = code:load_abs(filename:rootname(File), 'p.p'), - ?line [1,2] = qlc:e(p.p:q()), - ok. otp_6964(doc) -> diff --git a/lib/stdlib/test/re_SUITE.erl b/lib/stdlib/test/re_SUITE.erl index 8ee0a13f4c..500f5fadb9 100644 --- a/lib/stdlib/test/re_SUITE.erl +++ b/lib/stdlib/test/re_SUITE.erl @@ -1,3 +1,4 @@ +%% -*- coding: utf-8 -*- %% %% %CopyrightBegin% %% @@ -291,10 +292,10 @@ global_capture(Config) when is_list(Config) -> ?line match = re:run("ABCabcdABCabcdA","a(?<FOO>bcd)",[global,{capture,none,index}]), ?line match = re:run("ABCabcdABCabcdA","a(?<FOO>bcd)",[global,{capture,none,binary}]), ?line match = re:run("ABCabcdABCabcdA","a(?<FOO>bcd)",[global,{capture,none,list}]), - ?line {match,[[<<195,133,98,99,100>>,<<"bcd">>],[<<"abcd">>,<<"bcd">>]]} = re:run("ABC�bcdABCabcdA",".(?<FOO>bcd)",[global,{capture,all,binary},unicode]), - ?line {match,[["�bcd","bcd"],["abcd","bcd"]]} = re:run(<<"ABC",8#303,8#205,"bcdABCabcdA">>,".(?<FOO>bcd)",[global,{capture,all,list},unicode]), - ?line {match,[["�bcd","bcd"],["abcd","bcd"]]} = re:run("ABC�bcdABCabcdA",".(?<FOO>bcd)",[global,{capture,all,list},unicode]), - ?line {match,[[{3,5},{5,3}],[{11,4},{12,3}]]} = re:run("ABC�bcdABCabcdA",".(?<FOO>bcd)",[global,{capture,all,index},unicode]), + ?line {match,[[<<195,133,98,99,100>>,<<"bcd">>],[<<"abcd">>,<<"bcd">>]]} = re:run("ABCÅbcdABCabcdA",".(?<FOO>bcd)",[global,{capture,all,binary},unicode]), + ?line {match,[["Åbcd","bcd"],["abcd","bcd"]]} = re:run(<<"ABC",8#303,8#205,"bcdABCabcdA">>,".(?<FOO>bcd)",[global,{capture,all,list},unicode]), + ?line {match,[["Åbcd","bcd"],["abcd","bcd"]]} = re:run("ABCÅbcdABCabcdA",".(?<FOO>bcd)",[global,{capture,all,list},unicode]), + ?line {match,[[{3,5},{5,3}],[{11,4},{12,3}]]} = re:run("ABCÅbcdABCabcdA",".(?<FOO>bcd)",[global,{capture,all,index},unicode]), ?t:timetrap_cancel(Dog), ok. @@ -314,17 +315,17 @@ replace_return(Config) when is_list(Config) -> Dog = ?t:timetrap(?t:minutes(3)), ?line {'EXIT',{badarg,_}} = (catch re:replace("na","(a","")), ?line <<"nasse">> = re:replace(<<"nisse">>,"i","a",[{return,binary}]), - ?line <<"ABC�XABCXA">> = re:replace("ABC\305abcdABCabcdA","a(?<FOO>bcd)","X",[global,{return,binary}]), + ?line <<"ABCÅXABCXA">> = re:replace("ABC\305abcdABCabcdA","a(?<FOO>bcd)","X",[global,{return,binary}]), - ?line [<<"ABC�">>, + ?line [<<"ABCÅ">>, <<"X">>, <<"ABC">>, <<"X">> | <<"A">> ] = - re:replace("ABC�abcdABCabcdA","a(?<FOO>bcd)","X",[global,{return,iodata}]), - ?line "ABC�XABCXA" = re:replace("ABC�abcdABCabcdA","a(?<FOO>bcd)","X",[global,{return,list},unicode]), - ?line <<65,66,67,195,133,88,65,66,67,88,65>> = re:replace("ABC�abcdABCabcdA","a(?<FOO>bcd)","X",[global,{return,binary},unicode]), - ?line <<65,66,67,195,133,88,65,66,67,97,98,99,100,65>> = re:replace("ABC�abcdABCabcdA","a(?<FOO>bcd)","X",[{return,binary},unicode]), + re:replace("ABCÅabcdABCabcdA","a(?<FOO>bcd)","X",[global,{return,iodata}]), + ?line "ABCÅXABCXA" = re:replace("ABCÅabcdABCabcdA","a(?<FOO>bcd)","X",[global,{return,list},unicode]), + ?line <<65,66,67,195,133,88,65,66,67,88,65>> = re:replace("ABCÅabcdABCabcdA","a(?<FOO>bcd)","X",[global,{return,binary},unicode]), + ?line <<65,66,67,195,133,88,65,66,67,97,98,99,100,65>> = re:replace("ABCÅabcdABCabcdA","a(?<FOO>bcd)","X",[{return,binary},unicode]), ?line <<"iXk">> = re:replace("abcdefghijk","(.)(.)(.)(.)(.)(.)(.)(.)(.)(.)","\\9X",[{return,binary}]), ?line <<"jXk">> = re:replace("abcdefghijk","(.)(.)(.)(.)(.)(.)(.)(.)(.)(.)","\\10X",[{return,binary}]), ?line <<"Xk">> = re:replace("abcdefghijk","(.)(.)(.)(.)(.)(.)(.)(.)(.)(.)","\\11X",[{return,binary}]), diff --git a/lib/stdlib/test/re_testoutput1_replacement_test.erl b/lib/stdlib/test/re_testoutput1_replacement_test.erl index 69cb140e0d..8f8d8762ad 100644 --- a/lib/stdlib/test/re_testoutput1_replacement_test.erl +++ b/lib/stdlib/test/re_testoutput1_replacement_test.erl @@ -1,7 +1,8 @@ +%% -*- coding: utf-8 -*- %% %% %CopyrightBegin% %% -%% Copyright Ericsson AB 2008-2010. All Rights Reserved. +%% Copyright Ericsson AB 2008-2012. All Rights Reserved. %% %% The contents of this file are subject to the Erlang Public License, %% Version 1.1, (the "License"); you may not use this file except in @@ -274,10 +275,10 @@ run() -> ?line <<"dthing">> = iolist_to_binary(re:replace("dthing","^[^]cde]","y\\1I&MoqRPG&GQa\\1l",[global])), ?line <<"ething">> = iolist_to_binary(re:replace("ething","^[^]cde]","AsxwUn\\1GqkWNdgRJk",[])), ?line <<"ething">> = iolist_to_binary(re:replace("ething","^[^]cde]","AsxwUn\\1GqkWNdgRJk",[global])), -?line <<"RornKmOnaFr�tWgtW">> = iolist_to_binary(re:replace("�","^\\�","R\\1o\\1r\\1nKmOnaFr&tWgtW",[])), -?line <<"RornKmOnaFr�tWgtW">> = iolist_to_binary(re:replace("�","^\\�","R\\1o\\1r\\1nKmOnaFr&tWgtW",[global])), -?line <<"ufbmbfOYuK�wf�E�dx">> = iolist_to_binary(re:replace("�","^�","ufbmbfOYuK&wf&E&\\1dx",[])), -?line <<"ufbmbfOYuK�wf�E�dx">> = iolist_to_binary(re:replace("�","^�","ufbmbfOYuK&wf&E&\\1dx",[global])), +?line <<"RornKmOnaFrtWgtW">> = iolist_to_binary(re:replace("","^\\","R\\1o\\1r\\1nKmOnaFr&tWgtW",[])), +?line <<"RornKmOnaFrtWgtW">> = iolist_to_binary(re:replace("","^\\","R\\1o\\1r\\1nKmOnaFr&tWgtW",[global])), +?line <<"ufbmbfOYuKÿwfÿEÿdx">> = iolist_to_binary(re:replace("ÿ","^ÿ","ufbmbfOYuK&wf&E&\\1dx",[])), +?line <<"ufbmbfOYuKÿwfÿEÿdx">> = iolist_to_binary(re:replace("ÿ","^ÿ","ufbmbfOYuK&wf&E&\\1dx",[global])), ?line <<"oAdJme0jw">> = iolist_to_binary(re:replace("0","^[0-9]+$","oAdJme\\1&jw",[])), ?line <<"oAdJme0jw">> = iolist_to_binary(re:replace("0","^[0-9]+$","oAdJme\\1&jw",[global])), ?line <<"1aoKN">> = iolist_to_binary(re:replace("1","^[0-9]+$","&aoKN",[])), @@ -14972,10 +14973,10 @@ def">> = iolist_to_binary(re:replace("abc def","abc$","M",[global])), ?line <<"abcWCabcSYXGPjRugTabcVGabcSX">> = iolist_to_binary(re:replace("abcS","(abc)\\123","\\1WC&YXGPjRugT\\1VG&X",[])), ?line <<"abcWCabcSYXGPjRugTabcVGabcSX">> = iolist_to_binary(re:replace("abcS","(abc)\\123","\\1WC&YXGPjRugT\\1VG&X",[global])), -?line <<"fabc�Uabc�UmiqabceCsabcabc�">> = iolist_to_binary(re:replace("abc�","(abc)\\223","f&U&Umiq\\1eCs\\1&",[])), -?line <<"fabc�Uabc�UmiqabceCsabcabc�">> = iolist_to_binary(re:replace("abc�","(abc)\\223","f&U&Umiq\\1eCs\\1&",[global])), -?line <<"JRFabcxnbabc�Vkabc�fWigQMuaY">> = iolist_to_binary(re:replace("abc�","(abc)\\323","JRF\\1xnb&Vk&fWigQMuaY",[])), -?line <<"JRFabcxnbabc�Vkabc�fWigQMuaY">> = iolist_to_binary(re:replace("abc�","(abc)\\323","JRF\\1xnb&Vk&fWigQMuaY",[global])), +?line <<"fabcUabcUmiqabceCsabcabc">> = iolist_to_binary(re:replace("abc","(abc)\\223","f&U&Umiq\\1eCs\\1&",[])), +?line <<"fabcUabcUmiqabceCsabcabc">> = iolist_to_binary(re:replace("abc","(abc)\\223","f&U&Umiq\\1eCs\\1&",[global])), +?line <<"JRFabcxnbabcÓVkabcÓfWigQMuaY">> = iolist_to_binary(re:replace("abcÓ","(abc)\\323","JRF\\1xnb&Vk&fWigQMuaY",[])), +?line <<"JRFabcxnbabcÓVkabcÓfWigQMuaY">> = iolist_to_binary(re:replace("abcÓ","(abc)\\323","JRF\\1xnb&Vk&fWigQMuaY",[global])), ?line <<"vgabc@QQ">> = iolist_to_binary(re:replace("abc@","(abc)\\100","vg&QQ",[])), ?line <<"vgabc@QQ">> = iolist_to_binary(re:replace("abc@","(abc)\\100","vg&QQ",[global])), ?line <<"abc@OkvNytabc@abcabc@a">> = iolist_to_binary(re:replace("abc@","(abc)\\100","&OkvNyt&\\1&a",[])), @@ -18343,24 +18344,24 @@ xb","(?!^)x","\\1tysI\\1v\\1BVwx\\1FOWG\\1&C",[multiline, ?line <<"cY">> = iolist_to_binary(re:replace("M","\\M","cY\\1",[global])), ?line <<"aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa">> = iolist_to_binary(re:replace("aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa","(a+)*b","yWOTIFhIX\\1H",[])), ?line <<"aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa">> = iolist_to_binary(re:replace("aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa","(a+)*b","yWOTIFhIX\\1H",[global])), -?line <<"NREGularsRREGularWEYrVRr">> = iolist_to_binary(re:replace("REGular","(?i)reg(?:ul(?:[a�]|ae)r|ex)","N&\\1sR&WEYrVRr",[])), -?line <<"NREGularsRREGularWEYrVRr">> = iolist_to_binary(re:replace("REGular","(?i)reg(?:ul(?:[a�]|ae)r|ex)","N&\\1sR&WEYrVRr",[global])), -?line <<"G">> = iolist_to_binary(re:replace("regulaer","(?i)reg(?:ul(?:[a�]|ae)r|ex)","G",[])), -?line <<"G">> = iolist_to_binary(re:replace("regulaer","(?i)reg(?:ul(?:[a�]|ae)r|ex)","G",[global])), -?line <<"PSsXtwlmy">> = iolist_to_binary(re:replace("Regex","(?i)reg(?:ul(?:[a�]|ae)r|ex)","PSsXtwlmy",[])), -?line <<"PSsXtwlmy">> = iolist_to_binary(re:replace("Regex","(?i)reg(?:ul(?:[a�]|ae)r|ex)","PSsXtwlmy",[global])), -?line <<"regul�rmiYTi">> = iolist_to_binary(re:replace("regul�r","(?i)reg(?:ul(?:[a�]|ae)r|ex)","&miYTi\\1\\1",[])), -?line <<"regul�rmiYTi">> = iolist_to_binary(re:replace("regul�r","(?i)reg(?:ul(?:[a�]|ae)r|ex)","&miYTi\\1\\1",[global])), -?line <<"W�����rxh�����yUoaLOIegmSA">> = iolist_to_binary(re:replace("�����","����[�-��-�]+","W&rxh&yUoaL\\1OIegmS\\1A",[])), -?line <<"W�����rxh�����yUoaLOIegmSA">> = iolist_to_binary(re:replace("�����","����[�-��-�]+","W&rxh&yUoaL\\1OIegmS\\1A",[global])), -?line <<"F�����gnWPyHeh�����tXTQ">> = iolist_to_binary(re:replace("�����","����[�-��-�]+","F&gnWPyHe\\1h&tXTQ",[])), -?line <<"F�����gnWPyHeh�����tXTQ">> = iolist_to_binary(re:replace("�����","����[�-��-�]+","F&gnWPyHe\\1h&tXTQ",[global])), -?line <<"sHerHnAhAdx">> = iolist_to_binary(re:replace("�����","����[�-��-�]+","sHer\\1HnA\\1h\\1Adx",[])), -?line <<"sHerHnAhAdx">> = iolist_to_binary(re:replace("�����","����[�-��-�]+","sHer\\1HnA\\1h\\1Adx",[global])), -?line <<"trobAQoU�����n">> = iolist_to_binary(re:replace("�����","����[�-��-�]+","tr\\1obAQoU&n",[])), -?line <<"trobAQoU�����n">> = iolist_to_binary(re:replace("�����","����[�-��-�]+","tr\\1obAQoU&n",[global])), -?line <<"�XAZSd">> = iolist_to_binary(re:replace("�XAZXB","(?<=Z)X.","Sd",[])), -?line <<"�XAZSd">> = iolist_to_binary(re:replace("�XAZXB","(?<=Z)X.","Sd",[global])), +?line <<"NREGularsRREGularWEYrVRr">> = iolist_to_binary(re:replace("REGular","(?i)reg(?:ul(?:[aä]|ae)r|ex)","N&\\1sR&WEYrVRr",[])), +?line <<"NREGularsRREGularWEYrVRr">> = iolist_to_binary(re:replace("REGular","(?i)reg(?:ul(?:[aä]|ae)r|ex)","N&\\1sR&WEYrVRr",[global])), +?line <<"G">> = iolist_to_binary(re:replace("regulaer","(?i)reg(?:ul(?:[aä]|ae)r|ex)","G",[])), +?line <<"G">> = iolist_to_binary(re:replace("regulaer","(?i)reg(?:ul(?:[aä]|ae)r|ex)","G",[global])), +?line <<"PSsXtwlmy">> = iolist_to_binary(re:replace("Regex","(?i)reg(?:ul(?:[aä]|ae)r|ex)","PSsXtwlmy",[])), +?line <<"PSsXtwlmy">> = iolist_to_binary(re:replace("Regex","(?i)reg(?:ul(?:[aä]|ae)r|ex)","PSsXtwlmy",[global])), +?line <<"regulärmiYTi">> = iolist_to_binary(re:replace("regulär","(?i)reg(?:ul(?:[aä]|ae)r|ex)","&miYTi\\1\\1",[])), +?line <<"regulärmiYTi">> = iolist_to_binary(re:replace("regulär","(?i)reg(?:ul(?:[aä]|ae)r|ex)","&miYTi\\1\\1",[global])), +?line <<"WÅæåäàrxhÅæåäàyUoaLOIegmSA">> = iolist_to_binary(re:replace("Åæåäà","Åæåä[à-ÿÀ-ß]+","W&rxh&yUoaL\\1OIegmS\\1A",[])), +?line <<"WÅæåäàrxhÅæåäàyUoaLOIegmSA">> = iolist_to_binary(re:replace("Åæåäà","Åæåä[à-ÿÀ-ß]+","W&rxh&yUoaL\\1OIegmS\\1A",[global])), +?line <<"FÅæåäÿgnWPyHehÅæåäÿtXTQ">> = iolist_to_binary(re:replace("Åæåäÿ","Åæåä[à-ÿÀ-ß]+","F&gnWPyHe\\1h&tXTQ",[])), +?line <<"FÅæåäÿgnWPyHehÅæåäÿtXTQ">> = iolist_to_binary(re:replace("Åæåäÿ","Åæåä[à-ÿÀ-ß]+","F&gnWPyHe\\1h&tXTQ",[global])), +?line <<"sHerHnAhAdx">> = iolist_to_binary(re:replace("ÅæåäÀ","Åæåä[à-ÿÀ-ß]+","sHer\\1HnA\\1h\\1Adx",[])), +?line <<"sHerHnAhAdx">> = iolist_to_binary(re:replace("ÅæåäÀ","Åæåä[à-ÿÀ-ß]+","sHer\\1HnA\\1h\\1Adx",[global])), +?line <<"trobAQoUÅæåäßn">> = iolist_to_binary(re:replace("Åæåäß","Åæåä[à-ÿÀ-ß]+","tr\\1obAQoU&n",[])), +?line <<"trobAQoUÅæåäßn">> = iolist_to_binary(re:replace("Åæåäß","Åæåä[à-ÿÀ-ß]+","tr\\1obAQoU&n",[global])), +?line <<"XAZSd">> = iolist_to_binary(re:replace("XAZXB","(?<=Z)X.","Sd",[])), +?line <<"XAZSd">> = iolist_to_binary(re:replace("XAZXB","(?<=Z)X.","Sd",[global])), ?line <<"A">> = iolist_to_binary(re:replace("ab cd defg","ab cd (?x) de fg","\\1A\\1",[])), ?line <<"A">> = iolist_to_binary(re:replace("ab cd defg","ab cd (?x) de fg","\\1A\\1",[global])), ?line <<"fab cddefgLdtKCtPab cddefgxvVUHDah">> = iolist_to_binary(re:replace("ab cddefg","ab cd(?x) de fg","f&LdtKC\\1\\1tP&xvVUHDah",[])), diff --git a/lib/stdlib/test/re_testoutput1_split_test.erl b/lib/stdlib/test/re_testoutput1_split_test.erl index e86a04b008..4fc85b95c0 100644 --- a/lib/stdlib/test/re_testoutput1_split_test.erl +++ b/lib/stdlib/test/re_testoutput1_split_test.erl @@ -1,7 +1,8 @@ +%% -*- coding: utf-8 -*- %% %% %CopyrightBegin% %% -%% Copyright Ericsson AB 2008-2010. All Rights Reserved. +%% Copyright Ericsson AB 2008-2012. All Rights Reserved. %% %% The contents of this file are subject to the Erlang Public License, %% Version 1.1, (the "License"); you may not use this file except in @@ -524,14 +525,14 @@ run() -> ?line <<"ething">> = iolist_to_binary(join(re:split("ething","^[^]cde]",[{parts, 2}]))), ?line <<"ething">> = iolist_to_binary(join(re:split("ething","^[^]cde]",[]))), -?line <<"">> = iolist_to_binary(join(re:split("�","^\\�",[trim]))), -?line <<":">> = iolist_to_binary(join(re:split("�","^\\�",[{parts, +?line <<"">> = iolist_to_binary(join(re:split("","^\\",[trim]))), +?line <<":">> = iolist_to_binary(join(re:split("","^\\",[{parts, 2}]))), -?line <<":">> = iolist_to_binary(join(re:split("�","^\\�",[]))), -?line <<"">> = iolist_to_binary(join(re:split("�","^�",[trim]))), -?line <<":">> = iolist_to_binary(join(re:split("�","^�",[{parts, +?line <<":">> = iolist_to_binary(join(re:split("","^\\",[]))), +?line <<"">> = iolist_to_binary(join(re:split("ÿ","^ÿ",[trim]))), +?line <<":">> = iolist_to_binary(join(re:split("ÿ","^ÿ",[{parts, 2}]))), -?line <<":">> = iolist_to_binary(join(re:split("�","^�",[]))), +?line <<":">> = iolist_to_binary(join(re:split("ÿ","^ÿ",[]))), ?line <<"">> = iolist_to_binary(join(re:split("0","^[0-9]+$",[trim]))), ?line <<":">> = iolist_to_binary(join(re:split("0","^[0-9]+$",[{parts, 2}]))), @@ -22879,14 +22880,14 @@ def","abc$",[]))), ?line <<":abc:">> = iolist_to_binary(join(re:split("abcS","(abc)\\123",[{parts, 2}]))), ?line <<":abc:">> = iolist_to_binary(join(re:split("abcS","(abc)\\123",[]))), -?line <<":abc">> = iolist_to_binary(join(re:split("abc�","(abc)\\223",[trim]))), -?line <<":abc:">> = iolist_to_binary(join(re:split("abc�","(abc)\\223",[{parts, +?line <<":abc">> = iolist_to_binary(join(re:split("abc","(abc)\\223",[trim]))), +?line <<":abc:">> = iolist_to_binary(join(re:split("abc","(abc)\\223",[{parts, 2}]))), -?line <<":abc:">> = iolist_to_binary(join(re:split("abc�","(abc)\\223",[]))), -?line <<":abc">> = iolist_to_binary(join(re:split("abc�","(abc)\\323",[trim]))), -?line <<":abc:">> = iolist_to_binary(join(re:split("abc�","(abc)\\323",[{parts, +?line <<":abc:">> = iolist_to_binary(join(re:split("abc","(abc)\\223",[]))), +?line <<":abc">> = iolist_to_binary(join(re:split("abcÓ","(abc)\\323",[trim]))), +?line <<":abc:">> = iolist_to_binary(join(re:split("abcÓ","(abc)\\323",[{parts, 2}]))), -?line <<":abc:">> = iolist_to_binary(join(re:split("abc�","(abc)\\323",[]))), +?line <<":abc:">> = iolist_to_binary(join(re:split("abcÓ","(abc)\\323",[]))), ?line <<":abc">> = iolist_to_binary(join(re:split("abc@","(abc)\\100",[trim]))), ?line <<":abc:">> = iolist_to_binary(join(re:split("abc@","(abc)\\100",[{parts, 2}]))), @@ -28929,42 +28930,42 @@ xb","(?!^)x",[multiline]))), ?line <<"aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa">> = iolist_to_binary(join(re:split("aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa","(a+)*b",[{parts, 2}]))), ?line <<"aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa">> = iolist_to_binary(join(re:split("aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa","(a+)*b",[]))), -?line <<"">> = iolist_to_binary(join(re:split("REGular","(?i)reg(?:ul(?:[a�]|ae)r|ex)",[trim]))), -?line <<":">> = iolist_to_binary(join(re:split("REGular","(?i)reg(?:ul(?:[a�]|ae)r|ex)",[{parts, +?line <<"">> = iolist_to_binary(join(re:split("REGular","(?i)reg(?:ul(?:[aä]|ae)r|ex)",[trim]))), +?line <<":">> = iolist_to_binary(join(re:split("REGular","(?i)reg(?:ul(?:[aä]|ae)r|ex)",[{parts, 2}]))), -?line <<":">> = iolist_to_binary(join(re:split("REGular","(?i)reg(?:ul(?:[a�]|ae)r|ex)",[]))), -?line <<"">> = iolist_to_binary(join(re:split("regulaer","(?i)reg(?:ul(?:[a�]|ae)r|ex)",[trim]))), -?line <<":">> = iolist_to_binary(join(re:split("regulaer","(?i)reg(?:ul(?:[a�]|ae)r|ex)",[{parts, +?line <<":">> = iolist_to_binary(join(re:split("REGular","(?i)reg(?:ul(?:[aä]|ae)r|ex)",[]))), +?line <<"">> = iolist_to_binary(join(re:split("regulaer","(?i)reg(?:ul(?:[aä]|ae)r|ex)",[trim]))), +?line <<":">> = iolist_to_binary(join(re:split("regulaer","(?i)reg(?:ul(?:[aä]|ae)r|ex)",[{parts, 2}]))), -?line <<":">> = iolist_to_binary(join(re:split("regulaer","(?i)reg(?:ul(?:[a�]|ae)r|ex)",[]))), -?line <<"">> = iolist_to_binary(join(re:split("Regex","(?i)reg(?:ul(?:[a�]|ae)r|ex)",[trim]))), -?line <<":">> = iolist_to_binary(join(re:split("Regex","(?i)reg(?:ul(?:[a�]|ae)r|ex)",[{parts, +?line <<":">> = iolist_to_binary(join(re:split("regulaer","(?i)reg(?:ul(?:[aä]|ae)r|ex)",[]))), +?line <<"">> = iolist_to_binary(join(re:split("Regex","(?i)reg(?:ul(?:[aä]|ae)r|ex)",[trim]))), +?line <<":">> = iolist_to_binary(join(re:split("Regex","(?i)reg(?:ul(?:[aä]|ae)r|ex)",[{parts, 2}]))), -?line <<":">> = iolist_to_binary(join(re:split("Regex","(?i)reg(?:ul(?:[a�]|ae)r|ex)",[]))), -?line <<"">> = iolist_to_binary(join(re:split("regul�r","(?i)reg(?:ul(?:[a�]|ae)r|ex)",[trim]))), -?line <<":">> = iolist_to_binary(join(re:split("regul�r","(?i)reg(?:ul(?:[a�]|ae)r|ex)",[{parts, +?line <<":">> = iolist_to_binary(join(re:split("Regex","(?i)reg(?:ul(?:[aä]|ae)r|ex)",[]))), +?line <<"">> = iolist_to_binary(join(re:split("regulär","(?i)reg(?:ul(?:[aä]|ae)r|ex)",[trim]))), +?line <<":">> = iolist_to_binary(join(re:split("regulär","(?i)reg(?:ul(?:[aä]|ae)r|ex)",[{parts, 2}]))), -?line <<":">> = iolist_to_binary(join(re:split("regul�r","(?i)reg(?:ul(?:[a�]|ae)r|ex)",[]))), -?line <<"">> = iolist_to_binary(join(re:split("�����","����[�-��-�]+",[trim]))), -?line <<":">> = iolist_to_binary(join(re:split("�����","����[�-��-�]+",[{parts, +?line <<":">> = iolist_to_binary(join(re:split("regulär","(?i)reg(?:ul(?:[aä]|ae)r|ex)",[]))), +?line <<"">> = iolist_to_binary(join(re:split("Åæåäà","Åæåä[à-ÿÀ-ß]+",[trim]))), +?line <<":">> = iolist_to_binary(join(re:split("Åæåäà","Åæåä[à-ÿÀ-ß]+",[{parts, 2}]))), -?line <<":">> = iolist_to_binary(join(re:split("�����","����[�-��-�]+",[]))), -?line <<"">> = iolist_to_binary(join(re:split("�����","����[�-��-�]+",[trim]))), -?line <<":">> = iolist_to_binary(join(re:split("�����","����[�-��-�]+",[{parts, +?line <<":">> = iolist_to_binary(join(re:split("Åæåäà","Åæåä[à-ÿÀ-ß]+",[]))), +?line <<"">> = iolist_to_binary(join(re:split("Åæåäÿ","Åæåä[à-ÿÀ-ß]+",[trim]))), +?line <<":">> = iolist_to_binary(join(re:split("Åæåäÿ","Åæåä[à-ÿÀ-ß]+",[{parts, 2}]))), -?line <<":">> = iolist_to_binary(join(re:split("�����","����[�-��-�]+",[]))), -?line <<"">> = iolist_to_binary(join(re:split("�����","����[�-��-�]+",[trim]))), -?line <<":">> = iolist_to_binary(join(re:split("�����","����[�-��-�]+",[{parts, +?line <<":">> = iolist_to_binary(join(re:split("Åæåäÿ","Åæåä[à-ÿÀ-ß]+",[]))), +?line <<"">> = iolist_to_binary(join(re:split("ÅæåäÀ","Åæåä[à-ÿÀ-ß]+",[trim]))), +?line <<":">> = iolist_to_binary(join(re:split("ÅæåäÀ","Åæåä[à-ÿÀ-ß]+",[{parts, 2}]))), -?line <<":">> = iolist_to_binary(join(re:split("�����","����[�-��-�]+",[]))), -?line <<"">> = iolist_to_binary(join(re:split("�����","����[�-��-�]+",[trim]))), -?line <<":">> = iolist_to_binary(join(re:split("�����","����[�-��-�]+",[{parts, +?line <<":">> = iolist_to_binary(join(re:split("ÅæåäÀ","Åæåä[à-ÿÀ-ß]+",[]))), +?line <<"">> = iolist_to_binary(join(re:split("Åæåäß","Åæåä[à-ÿÀ-ß]+",[trim]))), +?line <<":">> = iolist_to_binary(join(re:split("Åæåäß","Åæåä[à-ÿÀ-ß]+",[{parts, 2}]))), -?line <<":">> = iolist_to_binary(join(re:split("�����","����[�-��-�]+",[]))), -?line <<"�XAZ">> = iolist_to_binary(join(re:split("�XAZXB","(?<=Z)X.",[trim]))), -?line <<"�XAZ:">> = iolist_to_binary(join(re:split("�XAZXB","(?<=Z)X.",[{parts, +?line <<":">> = iolist_to_binary(join(re:split("Åæåäß","Åæåä[à-ÿÀ-ß]+",[]))), +?line <<"XAZ">> = iolist_to_binary(join(re:split("XAZXB","(?<=Z)X.",[trim]))), +?line <<"XAZ:">> = iolist_to_binary(join(re:split("XAZXB","(?<=Z)X.",[{parts, 2}]))), -?line <<"�XAZ:">> = iolist_to_binary(join(re:split("�XAZXB","(?<=Z)X.",[]))), +?line <<"XAZ:">> = iolist_to_binary(join(re:split("XAZXB","(?<=Z)X.",[]))), ?line <<"">> = iolist_to_binary(join(re:split("ab cd defg","ab cd (?x) de fg",[trim]))), ?line <<":">> = iolist_to_binary(join(re:split("ab cd defg","ab cd (?x) de fg",[{parts, 2}]))), diff --git a/lib/stdlib/test/shell_SUITE.erl b/lib/stdlib/test/shell_SUITE.erl index d49416c150..a32f846bd2 100644 --- a/lib/stdlib/test/shell_SUITE.erl +++ b/lib/stdlib/test/shell_SUITE.erl @@ -29,7 +29,7 @@ progex_bit_syntax/1, progex_records/1, progex_lc/1, progex_funs/1, otp_5990/1, otp_6166/1, otp_6554/1, otp_6785/1, - otp_7184/1, otp_7232/1, otp_8393/1]). + otp_7184/1, otp_7232/1, otp_8393/1, otp_10302/1]). -export([ start_restricted_from_shell/1, start_restricted_on_command_line/1,restricted_local/1]). @@ -93,7 +93,7 @@ groups() -> progex_funs]}, {tickets, [], [otp_5990, otp_6166, otp_6554, otp_6785, otp_7184, - otp_7232, otp_8393]}]. + otp_7232, otp_8393, otp_10302]}]. init_per_suite(Config) -> Config. @@ -108,7 +108,7 @@ end_per_group(_GroupName, Config) -> Config. --record(state, {bin, reply, leader}). +-record(state, {bin, reply, leader, unic = latin1}). start_restricted_from_shell(doc) -> @@ -374,15 +374,18 @@ records(Config) when is_list(Config) -> MS = ?MODULE_STRING, RR1 = "rr(" ++ MS ++ "). #state{}.", ?line "[state]\n" - "#state{bin = undefined,reply = undefined,leader = undefined}.\n" = + "#state{bin = undefined,reply = undefined,leader = undefined,\n" + " unic = latin1}.\n" = t(RR1), RR2 = "rr(" ++ MS ++ ",[state]). #state{}.", ?line "[state]\n" - "#state{bin = undefined,reply = undefined,leader = undefined}.\n" = + "#state{bin = undefined,reply = undefined,leader = undefined,\n" + " unic = latin1}.\n" = t(RR2), RR3 = "rr(" ++ MS ++ ",'_'). #state{}.", ?line "[state]\n" - "#state{bin = undefined,reply = undefined,leader = undefined}.\n" = + "#state{bin = undefined,reply = undefined,leader = undefined,\n" + " unic = latin1}.\n" = t(RR3), RR4 = "rr(" ++ MS ++ ", '_', {d,test1}).", ?line [[state]] = scan(RR4), @@ -2748,6 +2751,143 @@ prompt_err(B) -> S = string:strip(S2, both, $"), string:strip(S, right, $.). +otp_10302(doc) -> + "OTP-10302. Unicode."; +otp_10302(suite) -> []; +otp_10302(Config) when is_list(Config) -> + Test1 = + <<"begin + io:setopts([{encoding,utf8}]), + [1024] = \"\\x{400}\", + rd(rec, {a = \"\\x{400}\"}), + ok = rl(rec) + end.">>, + "-record(rec,{a = \"\x{400}\"}).\nok.\n" = t(Test1), + + Test3 = + <<"io:setopts([{encoding,utf8}]). + rd(rec, {a = \"\\x{400}\"}). + ok = rp(#rec{}).">>, + "ok.\nrec\n#rec{a = \"\x{400}\"}.\nok.\n" = t(Test3), + + Test4 = + <<"io:setopts([{encoding,utf8}]). + A = [1024] = \"\\x{400}\". + b(). + h().">>, + + "ok.\n\"\x{400}\"\nA = \"\x{400}\".\nok.\n" + "1: io:setopts([{encoding,utf8}])\n-> ok.\n" + "2: A = [1024] = \"\x{400}\"\n-> \"\x{400}\"\n" + "3: b()\n-> ok.\nok.\n" = t(Test4), + + Test5 = + <<"begin + io:setopts([{encoding,utf8}]), + results(0), + A = [1024] = \"\\x{400}\", + b(), + h() + end.">>, + "A = \"\x{400}\".\nok.\n" = t(Test5), + + %% One $" is "lost": + true = + "\x{400}\": command not found" =:= + prompt_err({<<"io:setopts([{encoding,utf8}]). v(\"\x{400}\")."/utf8>>, + unicode}), + + "ok.\ndefault\n* Bad prompt function: \"\x{400}\".\n" = + t({<<"io:setopts([{encoding,utf8}]). " + "shell:prompt_func(\"\x{400}\")."/utf8>>, + unicode}), + _ = shell:prompt_func(default), + + %% Test lib:format_exception() (cf. OTP-6554) + Test6 = + <<"begin + A = <<\"\\xaa\">>, + S = lists:flatten(io_lib:format(\"~p/~p.\", [A, A])), + {ok, Ts, _} = erl_scan:string(S, 1, [unicode]), + {ok, Es} = erl_parse:parse_exprs(Ts), + B = erl_eval:new_bindings(), + erl_eval:exprs(Es, B) + end.">>, + + "** exception error: an error occurred when evaluating" + " an arithmetic expression\n in operator '/'/2\n" + " called as <<\"\xaa\">> / <<\"\xaa\">>.\n" = t(Test6), + Test7 = + <<"io:setopts([{encoding,utf8}]). + A = <<\"\\xaa\">>, + S = lists:flatten(io_lib:format(\"~p/~p.\", [A, A])), + {ok, Ts, _} = erl_scan:string(S, 1, [unicode]), + {ok, Es} = erl_parse:parse_exprs(Ts), + B = erl_eval:new_bindings(), + erl_eval:exprs(Es, B).">>, + + "ok.\n** exception error: an error occurred when evaluating" + " an arithmetic expression\n in operator '/'/2\n" + " called as <<170>> / <<170>>.\n" = t(Test7), + Test8 = + <<"begin + A = [1089], + S = lists:flatten(io_lib:format(\"~tp/~tp.\", [A, A])), + {ok, Ts, _} = erl_scan:string(S, 1, [unicode]), + {ok, Es} = erl_parse:parse_exprs(Ts), + B = erl_eval:new_bindings(), + erl_eval:exprs(Es, B) + end.">>, + "** exception error: an error occurred when evaluating" + " an arithmetic expression\n in operator '/'/2\n" + " called as [1089] / [1089].\n" = t(Test8), + Test9 = + <<"io:setopts([{encoding,utf8}]). + A = [1089], + S = lists:flatten(io_lib:format(\"~tp/~tp.\", [A, A])), + {ok, Ts, _} = erl_scan:string(S, 1, [unicode]), + {ok, Es} = erl_parse:parse_exprs(Ts), + B = erl_eval:new_bindings(), + erl_eval:exprs(Es, B).">>, + + "ok.\n** exception error: an error occurred when evaluating" + " an arithmetic expression\n in operator '/'/2\n" + " called as \"\x{441}\" / \"\x{441}\".\n" = t(Test9), + Test10 = + <<"A = {\"1\\xaa\", + $\\xaa, + << <<\"hi\">>/binary >>, + <<\"1\xaa\">>}, + fun(a) -> true end(A).">>, + "** exception error: no function clause matching \n" + " erl_eval:'-inside-an-interpreted-fun-'" + "({\"1\xc2\xaa\",170,<<\"hi\">>,\n " + " <<\"1\xc2\xaa\">>}) .\n" = t(Test10), + Test11 = + <<"io:setopts([{encoding,utf8}]). + A = {\"1\\xaa\", + $\\xaa, + << <<\"hi\">>/binary >>, + <<\"1\xaa\">>}, + fun(a) -> true end(A).">>, + + "ok.\n** exception error: no function clause matching \n" + " erl_eval:'-inside-an-interpreted-fun-'" + "({\"1\xaa\",170,<<\"hi\">>,\n " + " <<\"1\xaa\"/utf8>>}) .\n" = t(Test11), + Test12 = <<"fun(a, b) -> false end(65, [1089]).">>, + "** exception error: no function clause matching \n" + " erl_eval:'-inside-an-interpreted-fun-'(65,[1089])" + " .\n" = t(Test12), + Test13 = + <<"io:setopts([{encoding,utf8}]). + fun(a, b) -> false end(65, [1089]).">>, + "ok.\n** exception error: no function clause matching \n" + " erl_eval:'-inside-an-interpreted-fun-'(65,\"\x{441}\")" + " .\n" = t(Test13), + + ok. + scan(B) -> F = fun(Ts) -> case erl_parse:parse_term(Ts) of @@ -2761,7 +2901,7 @@ scan(B) -> scan(t(B), F). scan(S0, F) -> - case erl_scan:tokens([], S0, 1) of + case erl_scan:tokens([], S0, 1, [unicode]) of {done,{ok,Ts,_},S} -> [F(Ts) | scan(S, F)]; _Else -> @@ -2769,29 +2909,36 @@ scan(S0, F) -> end. t({Node,Bin}) when is_atom(Node),is_binary(Bin) -> - t0(Bin, fun() -> start_new_shell(Node) end); + t0({Bin,latin1}, fun() -> start_new_shell(Node) end); t(Bin) when is_binary(Bin) -> - t0(Bin, fun() -> start_new_shell() end); + t0({Bin,latin1}, fun() -> start_new_shell() end); +t({Bin,Enc}) when is_binary(Bin), is_atom(Enc) -> + t0({Bin,Enc}, fun() -> start_new_shell() end); t(L) -> t(list_to_binary(L)). -t0(Bin, F) -> +t0({Bin,Enc}, F) -> %% Spawn a process so that io_request messages do not interfer. P = self(), - C = spawn(fun() -> t1(P, Bin, F) end), + C = spawn(fun() -> t1(P, {Bin, Enc}, F) end), receive {C, R} -> R end. -t1(Parent, Bin, F) -> - %% io:format("*** Testing ~s~n", [binary_to_list(Bin)]), - S = #state{bin = Bin, reply = [], leader = group_leader()}, +t1(Parent, {Bin,Enc}, F) -> + io:format("*** Testing ~s~n", [binary_to_list(Bin)]), + S = #state{bin = Bin, unic = Enc, reply = [], leader = group_leader()}, group_leader(self(), self()), _Shell = F(), try server_loop(S) catch exit:R -> Parent ! {self(), R}; - throw:{?MODULE,LoopReply} -> + throw:{?MODULE,LoopReply,latin1} -> L0 = binary_to_list(list_to_binary(LoopReply)), [$\n | L1] = lists:dropwhile(fun(X) -> X =/= $\n end, L0), + Parent ! {self(), dotify(L1)}; + throw:{?MODULE,LoopReply,_Uni} -> + Tmp = unicode:characters_to_binary(LoopReply), + L0 = unicode:characters_to_list(Tmp), + [$\n | L1] = lists:dropwhile(fun(X) -> X =/= $\n end, L0), Parent ! {self(), dotify(L1)} after group_leader(S#state.leader, self()) end. @@ -2835,7 +2982,7 @@ do_io_request(Req, From, S, ReplyAs) -> case io_requests([Req], [], S) of {_Status,{eof,_},S1} -> io_reply(From, ReplyAs, {error,terminated}), - throw({?MODULE,S1#state.reply}); + throw({?MODULE,S1#state.reply,S1#state.unic}); {_Status,Reply,S1} -> io_reply(From, ReplyAs, Reply), S1 @@ -2858,13 +3005,34 @@ io_requests([], [Rs|Cont], S) -> io_requests([], [], S) -> {ok,ok,S}. +io_request({setopts, Opts}, S) -> + #state{unic = OldEnc, bin = Bin} = S, + NewEnc = case proplists:get_value(encoding, Opts) of + undefined -> OldEnc; + utf8 -> unicode; + New -> New + end, + NewBin = case {OldEnc, NewEnc} of + {E, E} -> Bin; + {latin1, _} -> + unicode:characters_to_binary(Bin, latin1, unicode); + {_, latin1} -> + unicode:characters_to_binary(Bin, unicode, latin1); + {_, _} -> Bin + end, + {ok, ok, S#state{unic = NewEnc, bin = NewBin}}; +io_request(getopts, S) -> + {ok,[{encoding,S#state.unic}],S}; io_request({get_geometry,columns}, S) -> {ok,80,S}; io_request({get_geometry,rows}, S) -> {ok,24,S}; io_request({put_chars,Chars}, S) -> {ok,ok,S#state{reply = [S#state.reply | Chars]}}; -io_request({put_chars,_,Chars}, S) -> +io_request({put_chars,latin1,Chars}, S) -> + {ok,ok,S#state{reply = [S#state.reply | Chars]}}; +io_request({put_chars,unicode,Chars0}, S) -> + Chars = unicode:characters_to_list(Chars0), {ok,ok,S#state{reply = [S#state.reply | Chars]}}; io_request({put_chars,Mod,Func,Args}, S) -> case catch apply(Mod, Func, Args) of @@ -2890,9 +3058,12 @@ get_until_loop(M, F, As, S, {more,Cont}, Enc) -> 0 -> get_until_loop(M, F, As, S, catch apply(M, F, [Cont,eof|As]), Enc); + _ when S#state.unic =:= latin1 -> + get_until_loop(M, F, As, S#state{bin = <<>>}, + catch apply(M, F, [Cont,binary_to_list(Bin)|As]), Enc); _ -> get_until_loop(M, F, As, S#state{bin = <<>>}, - catch apply(M, F, [Cont,binary_to_list(Bin)|As]), Enc) + catch apply(M, F, [Cont,unicode:characters_to_list(Bin)|As]), Enc) end; get_until_loop(_M, _F, _As, S, {done,Res,Buf}, Enc) -> {ok,Res,S#state{bin = buf2bin(Buf, Enc)}}; @@ -2903,6 +3074,8 @@ buf2bin(eof,_) -> <<>>; buf2bin(Buf,latin1) -> list_to_binary(Buf); +buf2bin(Buf,utf8) -> + unicode:characters_to_binary(Buf,unicode,unicode); buf2bin(Buf,unicode) -> unicode:characters_to_binary(Buf,unicode,unicode). diff --git a/lib/stdlib/test/string_SUITE.erl b/lib/stdlib/test/string_SUITE.erl index 6969c095a0..96e653985f 100644 --- a/lib/stdlib/test/string_SUITE.erl +++ b/lib/stdlib/test/string_SUITE.erl @@ -1,7 +1,8 @@ +%% -*- coding: utf-8 -*- %% %% %CopyrightBegin% %% -%% Copyright Ericsson AB 2004-2011. All Rights Reserved. +%% Copyright Ericsson AB 2004-2012. All Rights Reserved. %% %% The contents of this file are subject to the Erlang Public License, %% Version 1.1, (the "License"); you may not use this file except in @@ -473,8 +474,8 @@ to_upper_to_lower(suite) -> to_upper_to_lower(doc) -> []; to_upper_to_lower(Config) when is_list(Config) -> - ?line "1234ABCDEF���=" = string:to_upper("1234abcdef���="), - ?line "����������abc()" = string:to_lower("����������abc()"), + ?line "1234ABCDEFÅÄÖ=" = string:to_upper("1234abcdefåäö="), + ?line "éèíúùòóåäöabc()" = string:to_lower("ÉÈÍÚÙÒÓÅÄÖabc()"), ?line All = lists:seq(0, 255), ?line UC = string:to_upper(All), diff --git a/lib/syntax_tools/src/epp_dodger.erl b/lib/syntax_tools/src/epp_dodger.erl index b3ced34c14..70395848a1 100644 --- a/lib/syntax_tools/src/epp_dodger.erl +++ b/lib/syntax_tools/src/epp_dodger.erl @@ -186,6 +186,7 @@ quick_parse_file(File, Options) -> parse_file(File, Parser, Options) -> case file:open(File, [read]) of {ok, Dev} -> + _ = epp:set_encoding(Dev), try Parser(Dev, 1, Options) after ok = file:close(Dev) end; @@ -400,7 +401,7 @@ quick_parse_form(Dev, L0, Options) -> parse_form(Dev, L0, Parser, Options) -> NoFail = proplists:get_bool(no_fail, Options), Opt = #opt{clever = proplists:get_bool(clever, Options)}, - case io:scan_erl_form(Dev, "", L0) of + case io:scan_erl_form(Dev, "", L0, [unicode]) of {ok, Ts, L1} -> case catch {ok, Parser(Ts, Opt)} of {'EXIT', Term} -> @@ -419,6 +420,7 @@ parse_form(Dev, L0, Parser, Options) -> {ok, F, L1} end; {error, _IoErr, _L1} = Err -> Err; + {error, _Reason} -> {eof, L0}; % This is probably encoding problem {eof, _L1} = Eof -> Eof end. diff --git a/lib/syntax_tools/src/erl_comment_scan.erl b/lib/syntax_tools/src/erl_comment_scan.erl index b833e1c069..a70e7ba413 100644 --- a/lib/syntax_tools/src/erl_comment_scan.erl +++ b/lib/syntax_tools/src/erl_comment_scan.erl @@ -72,7 +72,17 @@ file(Name) -> {ok, V} -> case V of {ok, B} -> - string(binary_to_list(B)); + Enc = case epp:read_encoding(Name) of + none -> epp:default_encoding(); + Enc0 -> Enc0 + end, + case catch unicode:characters_to_list(B, Enc) of + String when is_list(String) -> + string(String); + R -> + error_read_file(Name1), + exit(R) + end; {error, E} -> error_read_file(Name1), exit({read, E}) diff --git a/lib/syntax_tools/src/erl_prettypr.erl b/lib/syntax_tools/src/erl_prettypr.erl index f4bbf975c3..1a55a5e71c 100644 --- a/lib/syntax_tools/src/erl_prettypr.erl +++ b/lib/syntax_tools/src/erl_prettypr.erl @@ -60,7 +60,9 @@ hook = ?NOHOOK :: hook(), paper = ?PAPER :: integer(), ribbon = ?RIBBON :: integer(), - user = ?NOUSER :: term()}). + user = ?NOUSER :: term(), + encoding = epp:default_encoding() :: epp:source_encoding()}). + -type context() :: #ctxt{}. %% ===================================================================== @@ -231,6 +233,8 @@ format(Node) -> %% <dt>{user, term()}</dt> %% <dd>User-specific data for use in hook functions. The default %% value is `undefined'.</dd> +%% <dt>{encoding, epp:source_encoding()}</dt> +%% <dd>Specifies the encoding of the generated file.</dd> %% </dl> %% %% A hook function (cf. the {@link hook()} type) is passed the current @@ -342,7 +346,9 @@ layout(Node, Options) -> #ctxt{hook = proplists:get_value(hook, Options, ?NOHOOK), paper = proplists:get_value(paper, Options, ?PAPER), ribbon = proplists:get_value(ribbon, Options, ?RIBBON), - user = proplists:get_value(user, Options)}). + user = proplists:get_value(user, Options), + encoding = proplists:get_value(encoding, Options, + epp:default_encoding())}). lay(Node, Ctxt) -> case erl_syntax:get_ann(Node) of @@ -445,10 +451,10 @@ lay_2(Node, Ctxt) -> text(tidy_float(erl_syntax:float_literal(Node))); char -> - text(erl_syntax:char_literal(Node)); + text(erl_syntax:char_literal(Node, Ctxt#ctxt.encoding)); string -> - lay_string(erl_syntax:string_literal(Node), Ctxt); + lay_string(erl_syntax:string_literal(Node, Ctxt#ctxt.encoding), Ctxt); nil -> text("[]"); @@ -639,10 +645,6 @@ lay_2(Node, Ctxt) -> set_prec(Ctxt, PrecR)), beside(D1, beside(text(":"), D2)); - qualified_name -> - Ss = erl_syntax:qualified_name_segments(Node), - lay_qualified_name(Ss, Ctxt); - %% %% The rest is in alphabetical order %% @@ -966,26 +968,6 @@ maybe_parentheses(D, Prec, Ctxt) -> D end. -lay_qualified_name([S | Ss1] = Ss, Ctxt) -> - case erl_syntax:type(S) of - atom -> - case erl_syntax:atom_value(S) of - '' -> - beside(text("."), - lay_qualified_name_1(Ss1, Ctxt)); - _ -> - lay_qualified_name_1(Ss, Ctxt) - end; - _ -> - lay_qualified_name_1(Ss, Ctxt) - end. - -lay_qualified_name_1([S], Ctxt) -> - lay(S, Ctxt); -lay_qualified_name_1([S | Ss], Ctxt) -> - beside(lay(S, Ctxt), beside(text("."), - lay_qualified_name_1(Ss, Ctxt))). - lay_string(S, Ctxt) -> %% S includes leading/trailing double-quote characters. The segment %% width is 2/3 of the ribbon width - this seems to work well. diff --git a/lib/syntax_tools/src/erl_syntax.erl b/lib/syntax_tools/src/erl_syntax.erl index 151f04b03b..f7420030c3 100644 --- a/lib/syntax_tools/src/erl_syntax.erl +++ b/lib/syntax_tools/src/erl_syntax.erl @@ -161,6 +161,7 @@ is_char/2, char_value/1, char_literal/1, + char_literal/2, clause/2, clause/3, clause_body/1, @@ -234,8 +235,6 @@ prefix_expr/2, prefix_expr_argument/1, prefix_expr_operator/1, - qualified_name/1, - qualified_name_segments/1, query_expr/1, query_expr_body/1, receive_expr/1, @@ -271,6 +270,7 @@ is_string/2, string_value/1, string_literal/1, + string_literal/2, text/1, text_string/1, try_expr/2, @@ -449,7 +449,6 @@ %% <td>parentheses</td> %% <td>prefix_expr</td> %% </tr><tr> -%% <td>qualified_name</td> %% <td>query_expr</td> %% <td>receive_expr</td> %% <td>record_access</td> @@ -514,7 +513,6 @@ %% @see operator/1 %% @see parentheses/1 %% @see prefix_expr/2 -%% @see qualified_name/1 %% @see query_expr/1 %% @see receive_expr/3 %% @see record_access/3 @@ -584,11 +582,7 @@ type(Node) -> {record, _, _, _, _} -> record_expr; {record, _, _, _} -> record_expr; {record_field, _, _, _, _} -> record_access; - {record_field, _, _, _} -> - case is_qualified_name(Node) of - true -> qualified_name; - false -> record_access - end; + {record_field, _, _, _} -> record_access; {record_index, _, _, _} -> record_index_expr; {remote, _, _, _} -> module_qualifier; {rule, _, _, _, _} -> rule; @@ -1628,6 +1622,7 @@ float_literal(Node) -> %% %% @see char_value/1 %% @see char_literal/1 +%% @see char_literal/2 %% @see is_char/2 %% type(Node) = char @@ -1687,13 +1682,34 @@ char_value(Node) -> %% ===================================================================== %% @doc Returns the literal string represented by a `char' %% node. This includes the leading "`$'" character. +%% Characters beyond 255 will be escaped. %% %% @see char/1 -spec char_literal(syntaxTree()) -> nonempty_string(). char_literal(Node) -> - io_lib:write_char(char_value(Node)). + char_literal(Node, latin1). + + +%% ===================================================================== +%% @doc Returns the literal string represented by a `char' +%% node. This includes the leading "`$'" character. +%% Depending on the encoding a character beyond 255 will be escaped +%% ('latin1') or copied as is ('utf8'). +%% +%% @see char/1 + +-type encoding() :: 'utf8' | 'unicode' | 'latin1'. + +-spec char_literal(syntaxTree(), encoding()) -> nonempty_string(). + +char_literal(Node, unicode) -> + io_lib:write_unicode_char(char_value(Node)); +char_literal(Node, utf8) -> + io_lib:write_unicode_char(char_value(Node)); +char_literal(Node, latin1) -> + io_lib:write_unicode_char_as_latin1(char_value(Node)). %% ===================================================================== @@ -1708,6 +1724,7 @@ char_literal(Node) -> %% %% @see string_value/1 %% @see string_literal/1 +%% @see string_literal/2 %% @see is_string/2 %% @see char/1 @@ -1768,13 +1785,32 @@ string_value(Node) -> %% ===================================================================== %% @doc Returns the literal string represented by a `string' %% node. This includes surrounding double-quote characters. +%% Characters beyond 255 will be escaped. %% %% @see string/1 -spec string_literal(syntaxTree()) -> nonempty_string(). string_literal(Node) -> - io_lib:write_string(string_value(Node)). + string_literal(Node, latin1). + + +%% ===================================================================== +%% @doc Returns the literal string represented by a `string' +%% node. This includes surrounding double-quote characters. +%% Depending on the encoding characters beyond 255 will be escaped +%% ('latin1') or copied as is ('utf8'). +%% +%% @see string/1 + +-spec string_literal(syntaxTree(), encoding()) -> nonempty_string(). + +string_literal(Node, utf8) -> + io_lib:write_unicode_string(string_value(Node)); +string_literal(Node, unicode) -> + io_lib:write_unicode_string(string_value(Node)); +string_literal(Node, latin1) -> + io_lib:write_unicode_string_as_latin1(string_value(Node)). %% ===================================================================== @@ -3003,9 +3039,6 @@ revert_module_name(A) -> case type(A) of atom -> {ok, concrete(A)}; - qualified_name -> - Ss = qualified_name_segments(A), - {ok, [concrete(S) || S <- Ss]}; _ -> error end. @@ -3051,11 +3084,7 @@ attribute_arguments(Node) -> M0 -> {M0, none} end, - M2 = if is_list(M1) -> - qualified_name([atom(A) || A <- M1]); - true -> - atom(M1) - end, + M2 = atom(M1), M = set_pos(M2, Pos), if Vs == none -> [M]; true -> [M, set_pos(list(Vs), Pos)] @@ -3065,20 +3094,11 @@ attribute_arguments(Node) -> list(unfold_function_names(Data, Pos)), Pos)]; import -> - case Data of - {Module, Imports} -> - [if is_list(Module) -> - qualified_name([atom(A) - || A <- Module]); - true -> - set_pos(atom(Module), Pos) - end, - set_pos( - list(unfold_function_names(Imports, Pos)), - Pos)]; - _ -> - [qualified_name([atom(A) || A <- Data])] - end; + {Module, Imports} = Data, + [set_pos(atom(Module), Pos), + set_pos( + list(unfold_function_names(Imports, Pos)), + Pos)]; file -> {File, Line} = Data, [set_pos(string(File), Pos), @@ -3210,53 +3230,6 @@ module_qualifier_body(Node) -> %% ===================================================================== -%% @doc Creates an abstract qualified name. The result represents -%% "<code><em>S1</em>.<em>S2</em>. ... .<em>Sn</em></code>", if -%% `Segments' is `[S1, S2, ..., Sn]'. -%% -%% @see qualified_name_segments/1 - -%% type(Node) = qualified_name -%% data(Node) = [syntaxTree()] -%% -%% `erl_parse' representation: -%% -%% {record_field, Pos, Node, Node} -%% -%% Node = {atom, Pos, Value} | {record_field, Pos, Node, Node} -%% -%% Note that if not all leaf subnodes are (abstract) atoms, then Node -%% represents a Mnemosyne query record field access ('record_access'); -%% see type/1 for details. - --spec qualified_name([syntaxTree()]) -> syntaxTree(). - -qualified_name(Segments) -> - tree(qualified_name, Segments). - -revert_qualified_name(Node) -> - Pos = get_pos(Node), - fold_qualified_name(qualified_name_segments(Node), Pos). - - -%% ===================================================================== -%% @doc Returns the list of name segments of a -%% `qualified_name' node. -%% -%% @see qualified_name/1 - --spec qualified_name_segments(syntaxTree()) -> [syntaxTree()]. - -qualified_name_segments(Node) -> - case unwrap(Node) of - {record_field, _, _, _} = Node1 -> - unfold_qualified_name(Node1); - Node1 -> - data(Node1) - end. - - -%% ===================================================================== %% @doc Creates an abstract function definition. If `Clauses' %% is `[C1, ..., Cn]', the result represents %% "<code><em>Name</em> <em>C1</em>; ...; <em>Name</em> @@ -6068,8 +6041,6 @@ revert_root(Node) -> revert_parentheses(Node); prefix_expr -> revert_prefix_expr(Node); - qualified_name -> - revert_qualified_name(Node); query_expr -> revert_query_expr(Node); receive_expr -> @@ -6312,8 +6283,6 @@ subtrees(T) -> prefix_expr -> [[prefix_expr_operator(T)], [prefix_expr_argument(T)]]; - qualified_name -> - [qualified_name_segments(T)]; query_expr -> [[query_expr_body(T)]]; receive_expr -> @@ -6444,7 +6413,6 @@ make_tree(match_expr, [[P], [E]]) -> match_expr(P, E); make_tree(module_qualifier, [[M], [N]]) -> module_qualifier(M, N); make_tree(parentheses, [[E]]) -> parentheses(E); make_tree(prefix_expr, [[F], [A]]) -> prefix_expr(F, A); -make_tree(qualified_name, [S]) -> qualified_name(S); make_tree(query_expr, [[B]]) -> query_expr(B); make_tree(receive_expr, [C]) -> receive_expr(C); make_tree(receive_expr, [C, [E], A]) -> receive_expr(C, E, A); @@ -6788,32 +6756,6 @@ fold_variable_names(Vs) -> unfold_variable_names(Vs, Pos) -> [set_pos(variable(V), Pos) || V <- Vs]. -%% Support functions for qualified names ("foo.bar.baz", -%% "erl.lang.lists", etc.). The representation overlaps with the weird -%% "Mnesia query record access" operators. The '.' operator is left -%% associative, so folding should nest on the left. - -is_qualified_name({record_field, _, L, R}) -> - is_qualified_name(L) andalso is_qualified_name(R); -is_qualified_name({atom, _, _}) -> true; -is_qualified_name(_) -> false. - -unfold_qualified_name(Node) -> - lists:reverse(unfold_qualified_name(Node, [])). - -unfold_qualified_name({record_field, _, L, R}, Ss) -> - unfold_qualified_name(R, unfold_qualified_name(L, Ss)); -unfold_qualified_name(S, Ss) -> [S | Ss]. - -fold_qualified_name([S | Ss], Pos) -> - fold_qualified_name(Ss, Pos, {atom, Pos, atom_value(S)}). - -fold_qualified_name([S | Ss], Pos, Ack) -> - fold_qualified_name(Ss, Pos, {record_field, Pos, Ack, - {atom, Pos, atom_value(S)}}); -fold_qualified_name([], _Pos, Ack) -> - Ack. - %% Support functions for transforming lists of record field definitions. %% %% There is no unique representation for field definitions in the diff --git a/lib/syntax_tools/src/erl_syntax_lib.erl b/lib/syntax_tools/src/erl_syntax_lib.erl index 36cd35f15d..2c94ac776d 100644 --- a/lib/syntax_tools/src/erl_syntax_lib.erl +++ b/lib/syntax_tools/src/erl_syntax_lib.erl @@ -2223,11 +2223,6 @@ module_name_to_atom(M) -> case erl_syntax:type(M) of atom -> erl_syntax:atom_value(M); - qualified_name -> - list_to_atom(packages:concat( - [erl_syntax:atom_value(A) - || A <- erl_syntax:qualified_name_segments(M)]) - ); _ -> throw(syntax_error) end. diff --git a/lib/syntax_tools/src/erl_tidy.erl b/lib/syntax_tools/src/erl_tidy.erl index 59cf6c0a92..e9a88caff3 100644 --- a/lib/syntax_tools/src/erl_tidy.erl +++ b/lib/syntax_tools/src/erl_tidy.erl @@ -375,6 +375,8 @@ write_module(Tree, Name, Opts) -> end, filename(filename:join(Dir, Name1)) end, + Encoding = [{encoding,Enc} || Enc <- [epp:read_encoding(Name)], + Enc =/= none], case proplists:get_bool(backups, Opts) of true -> backup_file(File, Opts); @@ -382,9 +384,9 @@ write_module(Tree, Name, Opts) -> ok end, Printer = proplists:get_value(printer, Opts), - FD = open_output_file(File), + FD = open_output_file(File, Encoding), verbose("writing to file `~s'.", [File], Opts), - V = (catch {ok, output(FD, Printer, Tree, Opts)}), + V = (catch {ok, output(FD, Printer, Tree, Opts++Encoding)}), ok = file:close(FD), case V of {ok, _} -> @@ -432,8 +434,9 @@ file_type(Name, Links) -> throw(R) end. -open_output_file(FName) -> - case catch file:open(FName, [write]) of +open_output_file(FName, Options) -> +io:format("Options ~p~n", [Options]), + case catch file:open(FName, [write]++Options) of {ok, FD} -> FD; {error, R} -> diff --git a/lib/syntax_tools/src/igor.erl b/lib/syntax_tools/src/igor.erl index 37e561cbbe..8abc3f41cb 100644 --- a/lib/syntax_tools/src/igor.erl +++ b/lib/syntax_tools/src/igor.erl @@ -341,10 +341,12 @@ merge(Name, Files) -> merge(Name, Files, Opts) -> Opts1 = Opts ++ ?DEFAULT_MERGE_OPTS, - {Tree, Stubs} = merge_files(Name, Files, Opts1), + {Sources, Enc} = merge_files1(Files, Opts1), + {Tree, Stubs} = merge_sources(Name, Sources, Opts1), Dir = proplists:get_value(dir, Opts1, ""), Filename = proplists:get_value(outfile, Opts1, Name), - File = write_module(Tree, Filename, Dir, Opts1), + Encoding = [{encoding, Enc} || Enc =/= none], + File = write_module(Tree, Filename, Dir, Encoding ++ Opts1), [File | maybe_create_stubs(Stubs, Opts1)]. @@ -459,16 +461,21 @@ merge_files(Name, Files, Options) -> -spec merge_files(atom(), erl_syntax:forms(), [file:filename()], [option()]) -> {erl_syntax:syntaxTree(), [stubDescriptor()]}. -merge_files(_, _Trees, [], _) -> +merge_files(Name, Trees, Files, Opts) -> + {Sources, _Encoding} = merge_files1(Files, Opts), + merge_sources(Name, Trees ++ Sources, Opts). + +merge_files1([], _) -> report_error("no files to merge."), exit(badarg); -merge_files(Name, Trees, Files, Opts) -> +merge_files1(Files, Opts) -> Opts1 = Opts ++ [{includes, ?DEFAULT_INCLUDES}, {macros, ?DEFAULT_MACROS}, {preprocess, false}, comments], - Sources = [read_module(F, Opts1) || F <- Files], - merge_sources(Name, Trees ++ Sources, Opts1). + SourceEncodings = [read_module(F, Opts1) || F <- Files], + {Sources, [Encoding | _]} = lists:unzip(SourceEncodings), + {Sources, Encoding}. %% ===================================================================== @@ -2512,7 +2519,11 @@ rename(Files, Renamings, Opts) -> lists:flatmap(fun (F) -> rename_file(F, Dict, Opts1) end, Files). rename_file(File, Dict, Opts) -> - S = read_module(File, Opts), + {S, Enc} = read_module(File, Opts), + %% Try to avoid *two* coding: comments: + Encoding = [{encoding, Enc} || + Enc =/= none, + not proplists:get_bool(comments, Opts)], M = get_module_info(S), Name = M#module.name, Name1 = case dict:find(Name, Dict) of @@ -2526,10 +2537,10 @@ rename_file(File, Dict, Opts) -> Opts1 = [no_headers, {export, [Name]}, {static, [Name]}, - {redirect, dict:to_list(Dict1)}] ++ Opts, + {redirect, dict:to_list(Dict1)}] ++ Encoding ++ Opts, {Tree, Stubs} = merge_sources(Name1, [S], Opts1), Dir = filename:dirname(filename(File)), - File1 = write_module(Tree, Name1, Dir, Opts), + File1 = write_module(Tree, Name1, Dir, Opts++Encoding), %% We create the stub file in the same directory as the source file %% and the target file. @@ -2648,7 +2659,7 @@ error_text(D, Name) -> {L, M, E} when is_integer(L), is_atom(M) -> case catch M:format_error(E) of S when is_list(S) -> - io_lib:fwrite("`~w', line ~w: ~s.", + io_lib:fwrite("`~w', line ~w: ~ts.", [Name, L, S]); _ -> error_text_1(D, Name) @@ -2706,7 +2717,17 @@ open_output_file(FName) -> exit(R) end. -%% read_module(Name, Options) -> syntaxTree() +output_encoding(FD, Opts) -> + case proplists:get_value(encoding, Opts) of + undefined -> + ok = io:setopts(FD, [{encoding, epp:default_encoding()}]); + Encoding -> + ok = io:setopts(FD, [{encoding, Encoding}]), + EncS = epp:encoding_to_string(Encoding), + ok = io:fwrite(FD, <<"%% ~s\n">>, [EncS]) + end. + +%% read_module(Name, Options) -> {syntaxTree(), epp:source_encoding()} %% %% This also tries to locate the real source file, if "Name" does not %% point directly to a particular file. @@ -2729,20 +2750,21 @@ read_module(Name, Options) -> read_module_1(Name, Options) -> verbose("reading module `~s'.", [filename(Name)], Options), - Forms = read_module_2(Name, Options), + {Forms, Enc} = read_module_2(Name, Options), case proplists:get_bool(comments, Options) of false -> - Forms; + {Forms, Enc}; true -> Comments = erl_comment_scan:file(Name), - erl_recomment:recomment_forms(Forms, Comments) + {erl_recomment:recomment_forms(Forms, Comments), Enc} end. read_module_2(Name, Options) -> case read_module_3(Name, Options) of {ok, Forms} -> check_forms(Forms, Name), - Forms; + Enc = epp:read_encoding(Name), + {Forms, Enc}; {error, _} = Error -> error_read_file(Name), exit(Error) @@ -2772,7 +2794,7 @@ check_forms([F | Fs], File) -> _ -> "unknown error" end, - report_error("in file `~s' at line ~w:\n ~s", + report_error("in file `~s' at line ~w:\n ~ts", [filename(File), erl_syntax:get_pos(F), S]), exit(error); _ -> @@ -2847,6 +2869,7 @@ write_module(Tree, Name, Dir, Opts) -> end, Printer = proplists:get_value(printer, Opts), FD = open_output_file(File), + ok = output_encoding(FD, Opts), verbose("writing to file `~s'.", [File], Opts), V = (catch {ok, output(FD, Printer, Tree, Opts)}), ok = file:close(FD), diff --git a/lib/test_server/doc/src/notes.xml b/lib/test_server/doc/src/notes.xml index 3b109193a0..6a9add044a 100644 --- a/lib/test_server/doc/src/notes.xml +++ b/lib/test_server/doc/src/notes.xml @@ -32,6 +32,47 @@ <file>notes.xml</file> </header> +<section><title>Test_Server 3.5.3</title> + + <section><title>Improvements and New Features</title> + <list> + <item> + <p> + test_server_h will now recognize info_reports written by + ct connection handlers (according to the description in + cth_conn_log) and ignore them as they will be completely + handled by by ct_conn_log_h.</p> + <p> + Earlier test_server_h would print a tag (testcase name) + before forwarding the report to error_logger_tty_h. This + would cause lots of tags in the log with no info report + following (since error_logger_tty_h did not handle them).</p> + <p> + Own Id: OTP-10571</p> + </item> + </list> + </section> + + + <section><title>Known Bugs and Problems</title> + <list> + <item> + <p> + Restore Config data if lost when test case fails.</p> + <p> + Own Id: OTP-10070 Aux Id: kunagi-175 [86] </p> + </item> + <item> + <p> + IO server error in test_server.</p> + <p> + Own Id: OTP-10125 Aux Id: OTP-10101, kunagi-177 [88] </p> + </item> + </list> + </section> + +</section> + <section><title>Test_Server 3.5.2</title> <section><title>Fixed Bugs and Malfunctions</title> diff --git a/lib/test_server/src/erl2html2.erl b/lib/test_server/src/erl2html2.erl index 9c459c05d4..1729257809 100644 --- a/lib/test_server/src/erl2html2.erl +++ b/lib/test_server/src/erl2html2.erl @@ -34,11 +34,17 @@ convert(File, Dest) -> %% %% FIXME: The colours should *really* be set with %% stylesheets... + Encoding = encoding(File), Header = ["<!DOCTYPE HTML PUBLIC " "\"-//W3C//DTD HTML 3.2 Final//EN\">\n" "<!-- autogenerated by '"++atom_to_list(?MODULE)++"'. -->\n" "<html>\n" - "<head><title>", File, "</title></head>\n\n" + "<head>\n" + "<meta http-equiv=\"Content-Type\" content=\"text/html;" + "charset=", + Encoding,"\"/>\n" + "<title>", File, "</title>\n" + "</head>\n\n" "<body bgcolor=\"white\" text=\"black\"" " link=\"blue\" vlink=\"purple\" alink=\"red\">\n"], convert(File, Dest, Header). @@ -55,12 +61,12 @@ convert(File, Dest, Header) -> case file:open(Dest,[write,raw]) of {ok,DFd} -> file:write(DFd,[Header,"<pre>\n"]), - Lines = build_html(SFd,DFd,Functions), + _Lines = build_html(SFd,DFd,Functions), file:write(DFd,["</pre>\n",footer(), "</body>\n</html>\n"]), %% {_, Time2} = statistics(runtime), %% io:format("Converted ~p lines in ~.2f Seconds.~n", - %% [Lines, Time2/1000]), + %% [_Lines, Time2/1000]), file:close(SFd), file:close(DFd), ok; @@ -180,3 +186,20 @@ possibly_enhance(Str,false) -> %%% End of the file footer() -> "". + +%%%----------------------------------------------------------------- +%%% Read encoding from source file +encoding(File) -> + Encoding = + case epp:read_encoding(File) of + none -> + epp:default_encoding(); + E -> + E + end, + html_encoding(Encoding). + +html_encoding(latin1) -> + "iso-8859-1"; +html_encoding(utf8) -> + "utf-8". diff --git a/lib/test_server/src/test_server.erl b/lib/test_server/src/test_server.erl index 72bc7bb0d7..37cd8fac99 100644 --- a/lib/test_server/src/test_server.erl +++ b/lib/test_server/src/test_server.erl @@ -60,8 +60,6 @@ -include("test_server_internal.hrl"). -include_lib("kernel/include/file.hrl"). --define(pl2a(M), test_server_sup:package_atom(M)). - init_target_info() -> [$.|Emu] = code:objfile_extension(), {_, OTPRel} = init:script_id(), @@ -320,14 +318,21 @@ pmap(Fun,List) -> do_cover_for_node(Node,CoverFunc) -> + do_cover_for_node(Node,CoverFunc,true). +do_cover_for_node(Node,CoverFunc,StickUnstick) -> %% In case a slave node is starting another slave node! I.e. this %% function is executed on a slave node - then the cover function %% must be executed on the master node. This is for instance the %% case in test_server's own tests. MainCoverNode = cover:get_main_node(), - Sticky = unstick_all_sticky(MainCoverNode,Node), + Sticky = + if StickUnstick -> unstick_all_sticky(MainCoverNode,Node); + true -> ok + end, rpc:call(MainCoverNode,cover,CoverFunc,[Node]), - stick_all_sticky(Node,Sticky). + if StickUnstick -> stick_all_sticky(Node,Sticky); + true -> ok + end. unstick_all_sticky(Node) -> unstick_all_sticky(node(),Node). @@ -914,7 +919,7 @@ run_test_case_eval(Mod, Func, Args0, Name, Ref, RunInit, put(test_server_logopts, LogOpts), Where = [{Mod,Func}], put(test_server_loc, Where), - FWInitResult = test_server_sup:framework_call(init_tc,[?pl2a(Mod),Func,Args0], + FWInitResult = test_server_sup:framework_call(init_tc,[Mod,Func,Args0], {ok,Args0}), set_tc_state(running), {{Time,Value},Loc,Opts} = @@ -1048,7 +1053,7 @@ do_end_tc_call(Mod, Func, Res, Return) -> Ref = make_ref(), if FwMod == "ct_framework" ; FwMod == "undefined"; FwMod == false -> case test_server_sup:framework_call( - end_tc, [?pl2a(Mod),Func,Res, Return], ok) of + end_tc, [Mod,Func,Res, Return], ok) of {fail,FWReason} -> {failed,FWReason}; ok -> @@ -1063,7 +1068,7 @@ do_end_tc_call(Mod, Func, Res, Return) -> end; true -> case test_server_sup:framework_call(FwMod, end_tc, - [?pl2a(Mod),Func,Res], Ref) of + [Mod,Func,Res], Ref) of {fail,FWReason} -> {failed,FWReason}; _Else -> @@ -1297,11 +1302,11 @@ get_loc(Pid) -> fw_error_notify(Mod, Func, Args, Error) -> test_server_sup:framework_call(error_notification, - [?pl2a(Mod),Func,[Args], + [Mod,Func,[Args], {Error,unknown}]). fw_error_notify(Mod, Func, Args, Error, Loc) -> test_server_sup:framework_call(error_notification, - [?pl2a(Mod),Func,[Args], + [Mod,Func,[Args], {Error,Loc}]). %%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%% @@ -2189,12 +2194,9 @@ start_node(Name, Type, Options) -> %% Cannot run cover on shielded node or on a node started %% by a shielded node. - Cover = case is_cover() of + Cover = case is_cover(Node) of true -> - not is_shielded(Name) - andalso same_version(Node) - andalso proplists:get_value(start_cover,Options, - true); + proplists:get_value(start_cover,Options,true); false -> false end, @@ -2233,15 +2235,8 @@ wait_for_node(Slave) -> Result = receive {sync_result,R} -> R end, case Result of ok -> - Cover = case is_cover() of - true -> - not is_shielded(Slave) andalso same_version(Slave); - false -> - false - end, - net_adm:ping(Slave), - case Cover of + case is_cover(Slave) of true -> do_cover_for_node(Slave,start); _ -> @@ -2259,12 +2254,9 @@ wait_for_node(Slave) -> %% Kills a (remote) node. %% Also inform test_server_ctrl so it can clean up! stop_node(Slave) -> - Nocover = is_shielded(Slave) orelse not same_version(Slave), - case is_cover() of - true when not Nocover -> - do_cover_for_node(Slave,flush); - _ -> - ok + Cover = is_cover(Slave), + if Cover -> do_cover_for_node(Slave,flush,false); + true -> ok end, group_leader() ! {sync_apply,self(),{test_server_ctrl,stop_node,[Slave]}}, Result = receive {sync_result,R} -> R end, @@ -2276,10 +2268,15 @@ stop_node(Slave) -> {nodedown, Slave} -> format(minor, "Stopped slave node: ~p", [Slave]), format(major, "=node_stop ~p", [Slave]), + if Cover -> do_cover_for_node(Slave,stop,false); + true -> ok + end, true after 30000 -> format("=== WARNING: Node ~p does not seem to terminate.", [Slave]), + erlang:monitor_node(Slave, false), + receive {nodedown, Slave} -> ok after 0 -> ok end, false end; {error, _Reason} -> @@ -2291,9 +2288,27 @@ stop_node(Slave) -> [Slave]), case net_adm:ping(Slave)of pong -> + erlang:monitor_node(Slave, true), slave:stop(Slave), - true; + receive + {nodedown, Slave} -> + format(minor, "Stopped slave node: ~p", [Slave]), + format(major, "=node_stop ~p", [Slave]), + if Cover -> do_cover_for_node(Slave,stop,false); + true -> ok + end, + true + after 30000 -> + format("=== WARNING: Node ~p does not seem to terminate.", + [Slave]), + erlang:monitor_node(Slave, false), + receive {nodedown, Slave} -> ok after 0 -> ok end, + false + end; pang -> + if Cover -> do_cover_for_node(Slave,stop,false); + true -> ok + end, false end end. @@ -2380,6 +2395,14 @@ same_version(Name) -> OtherVersion = rpc:call(Name, erlang, system_info, [version]), ThisVersion =:= OtherVersion. +is_cover(Name) -> + case is_cover() of + true -> + not is_shielded(Name) andalso same_version(Name); + false -> + false + end. + %%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%% %% temp_name(Stem) -> string() %% Stem = string() diff --git a/lib/test_server/src/test_server_ctrl.erl b/lib/test_server/src/test_server_ctrl.erl index 3508623dd9..c5c57426b4 100644 --- a/lib/test_server/src/test_server_ctrl.erl +++ b/lib/test_server/src/test_server_ctrl.erl @@ -99,7 +99,6 @@ -define(html_ext, ".html"). -define(now, erlang:now()). --define(pl2a(M), test_server_sup:package_atom(M)). -define(void_fun, fun() -> ok end). -define(mod_result(X), if X == skip -> skipped; X == auto_skip -> skipped; @@ -2280,7 +2279,7 @@ run_test_cases_loop([{auto_skip_case,{Type,Ref,Case,Comment},SkipMode}|Cases], set_io_buffering(undefined), {Mod,Func} = skip_case(auto, Ref, 0, Case, Comment, false, SkipMode), test_server_sup:framework_call(report, [tc_auto_skip, - {?pl2a(Mod),Func,Comment}]), + {Mod,Func,Comment}]), run_test_cases_loop(Cases, Config, TimetrapData, ParentMode, delete_status(Ref, Status)); _ -> @@ -2289,7 +2288,7 @@ run_test_cases_loop([{auto_skip_case,{Type,Ref,Case,Comment},SkipMode}|Cases], wait_for_cases(Ref), {Mod,Func} = skip_case(auto, Ref, 0, Case, Comment, true, SkipMode), test_server_sup:framework_call(report, [tc_auto_skip, - {?pl2a(Mod),Func,Comment}]), + {Mod,Func,Comment}]), case CurrIOHandler of {Ref,_} -> %% current_io_handler was set by start conf of this @@ -2306,7 +2305,7 @@ run_test_cases_loop([{auto_skip_case,{Type,Ref,Case,Comment},SkipMode}|Cases], %% this is a skipped end conf for a non-parallel group that's not %% nested under a parallel group {Mod,Func} = skip_case(auto, Ref, 0, Case, Comment, false, SkipMode), - test_server_sup:framework_call(report, [tc_auto_skip,{?pl2a(Mod),Func,Comment}]), + test_server_sup:framework_call(report, [tc_auto_skip,{Mod,Func,Comment}]), %% Check if this group is auto skipped because of error in the init conf. %% If so, check if the parent group is a sequence, and if it is, skip @@ -2337,7 +2336,7 @@ run_test_cases_loop([{auto_skip_case,{Type,Ref,Case,Comment},SkipMode}|Cases], %% this is a skipped end conf for a non-parallel group nested under %% a parallel group (io buffering is active) {Mod,Func} = skip_case(auto, Ref, 0, Case, Comment, true, SkipMode), - test_server_sup:framework_call(report, [tc_auto_skip,{?pl2a(Mod),Func,Comment}]), + test_server_sup:framework_call(report, [tc_auto_skip,{Mod,Func,Comment}]), case CurrIOHandler of {Ref,_} -> %% current_io_handler was set by start conf of this @@ -2353,7 +2352,7 @@ run_test_cases_loop([{auto_skip_case,{Type,Ref,Case,Comment},SkipMode}|Cases], %% this is a skipped start conf for a group which is not nested %% under a parallel group {Mod,Func} = skip_case(auto, Ref, 0, Case, Comment, false, SkipMode), - test_server_sup:framework_call(report, [tc_auto_skip,{?pl2a(Mod),Func,Comment}]), + test_server_sup:framework_call(report, [tc_auto_skip,{Mod,Func,Comment}]), run_test_cases_loop(Cases, Config, TimetrapData, [conf(Ref,[])|Mode], Status); {_,Ref0} when is_reference(Ref0) -> %% this is a skipped start conf for a group nested under a parallel group @@ -2364,7 +2363,7 @@ run_test_cases_loop([{auto_skip_case,{Type,Ref,Case,Comment},SkipMode}|Cases], ok end, {Mod,Func} = skip_case(auto, Ref, 0, Case, Comment, true, SkipMode), - test_server_sup:framework_call(report, [tc_auto_skip,{?pl2a(Mod),Func,Comment}]), + test_server_sup:framework_call(report, [tc_auto_skip,{Mod,Func,Comment}]), run_test_cases_loop(Cases, Config, TimetrapData, [conf(Ref,[])|Mode], Status) end; @@ -2372,7 +2371,7 @@ run_test_cases_loop([{auto_skip_case,{Case,Comment},SkipMode}|Cases], Config, TimetrapData, Mode, Status) -> {Mod,Func} = skip_case(auto, undefined, get(test_server_case_num)+1, Case, Comment, is_io_buffered(), SkipMode), - test_server_sup:framework_call(report, [tc_auto_skip,{?pl2a(Mod),Func,Comment}]), + test_server_sup:framework_call(report, [tc_auto_skip,{Mod,Func,Comment}]), run_test_cases_loop(Cases, Config, TimetrapData, Mode, update_status(skipped, Mod, Func, Status)); @@ -2388,7 +2387,7 @@ run_test_cases_loop([{skip_case,{conf,Ref,Case,Comment}}|Cases0], %% skipped start conf {skip_cases_upto(Ref, Cases0, Comment, conf, Mode),Config} end, - test_server_sup:framework_call(report, [tc_user_skip,{?pl2a(Mod),Func,Comment}]), + test_server_sup:framework_call(report, [tc_user_skip,{Mod,Func,Comment}]), run_test_cases_loop(Cases, Config1, TimetrapData, Mode, update_status(skipped, Mod, Func, Status)); @@ -2396,7 +2395,7 @@ run_test_cases_loop([{skip_case,{Case,Comment}}|Cases], Config, TimetrapData, Mode, Status) -> {Mod,Func} = skip_case(user, undefined, get(test_server_case_num)+1, Case, Comment, is_io_buffered()), - test_server_sup:framework_call(report, [tc_user_skip,{?pl2a(Mod),Func,Comment}]), + test_server_sup:framework_call(report, [tc_user_skip,{Mod,Func,Comment}]), run_test_cases_loop(Cases, Config, TimetrapData, Mode, update_status(skipped, Mod, Func, Status)); @@ -3574,7 +3573,7 @@ run_test_case1(Ref, Num, Mod, Func, Args, RunInit, end, test_server_sup:framework_call(report, - [tc_start,{{?pl2a(Mod),Func},MinorName}]), + [tc_start,{{Mod,Func},MinorName}]), print_props((RunInit==skip_init), get_props(Mode)), GroupName = case get_name(Mode) of @@ -3759,7 +3758,7 @@ progress(skip, CaseNum, Mod, Func, Loc, Reason, Time, print(major, "=result skipped", []), print(1, "*** SKIPPED *** ~s", [get_info_str(Func, CaseNum, get(test_server_cases))]), - test_server_sup:framework_call(report, [tc_done,{?pl2a(Mod),Func, + test_server_sup:framework_call(report, [tc_done,{Mod,Func, {skipped,Reason1}}]), ReasonStr = reason_to_string(Reason1), ReasonStr1 = lists:flatten([string:strip(S,left) || @@ -3790,7 +3789,7 @@ progress(failed, CaseNum, Mod, Func, Loc, timetrap_timeout, T, print(1, "*** FAILED *** ~s", [get_info_str(Func, CaseNum, get(test_server_cases))]), test_server_sup:framework_call(report, - [tc_done,{?pl2a(Mod),Func, + [tc_done,{Mod,Func, {failed,timetrap_timeout}}]), FormatLastLoc = test_server_sup:format_loc(get_last_loc(Loc)), ErrorReason = io_lib:format("{timetrap_timeout,~s}", [FormatLastLoc]), @@ -3816,7 +3815,7 @@ progress(failed, CaseNum, Mod, Func, Loc, {testcase_aborted,Reason}, _T, print(1, "*** FAILED *** ~s", [get_info_str(Func, CaseNum, get(test_server_cases))]), test_server_sup:framework_call(report, - [tc_done,{?pl2a(Mod),Func, + [tc_done,{Mod,Func, {failed,testcase_aborted}}]), FormatLastLoc = test_server_sup:format_loc(get_last_loc(Loc)), ErrorReason = io_lib:format("{testcase_aborted,~s}", [FormatLastLoc]), @@ -3841,7 +3840,7 @@ progress(failed, CaseNum, Mod, Func, unknown, Reason, Time, print(major, "=result failed: ~p, ~p", [Reason,unknown]), print(1, "*** FAILED *** ~s", [get_info_str(Func, CaseNum, get(test_server_cases))]), - test_server_sup:framework_call(report, [tc_done,{?pl2a(Mod),Func, + test_server_sup:framework_call(report, [tc_done,{Mod,Func, {failed,Reason}}]), TimeStr = io_lib:format(if is_float(Time) -> "~.3fs"; true -> "~w" @@ -3877,7 +3876,7 @@ progress(failed, CaseNum, Mod, Func, Loc, Reason, Time, print(major, "=result failed: ~p, ~p", [Reason,Loc]), print(1, "*** FAILED *** ~s", [get_info_str(Func, CaseNum, get(test_server_cases))]), - test_server_sup:framework_call(report, [tc_done,{?pl2a(Mod),Func, + test_server_sup:framework_call(report, [tc_done,{Mod,Func, {failed,Reason}}]), TimeStr = io_lib:format(if is_float(Time) -> "~.3fs"; true -> "~w" @@ -3902,7 +3901,7 @@ progress(failed, CaseNum, Mod, Func, Loc, Reason, Time, progress(ok, _CaseNum, Mod, Func, _Loc, RetVal, Time, Comment0, {St0,St1}) -> print(minor, "successfully completed test case", []), - test_server_sup:framework_call(report, [tc_done,{?pl2a(Mod),Func,ok}]), + test_server_sup:framework_call(report, [tc_done,{Mod,Func,ok}]), Comment = case RetVal of {comment,RetComment} -> @@ -4549,7 +4548,7 @@ collect_case_invoke(Mod, Case, MFA, St) -> end; _ -> Suite = test_server_sup:framework_call(get_suite, - [?pl2a(Mod),Case], + [Mod,Case], []), collect_subcases(Mod, Case, MFA, St, Suite) end. diff --git a/lib/test_server/src/test_server_io.erl b/lib/test_server/src/test_server_io.erl index e960b3087a..777b377201 100644 --- a/lib/test_server/src/test_server_io.erl +++ b/lib/test_server/src/test_server_io.erl @@ -307,8 +307,10 @@ do_print_buffered(Q0, St) -> gc(#st{gls=Gls0}) -> InUse0 = [begin - {group_leader,GL} = process_info(P, group_leader), - GL + case process_info(P, group_leader) of + {group_leader,GL} -> GL; + undefined -> undefined + end end || P <- processes()], InUse = ordsets:from_list(InUse0), Gls = gb_sets:to_list(Gls0), diff --git a/lib/test_server/src/test_server_sup.erl b/lib/test_server/src/test_server_sup.erl index c7553cccb5..a6d426887e 100644 --- a/lib/test_server/src/test_server_sup.erl +++ b/lib/test_server/src/test_server_sup.erl @@ -28,7 +28,7 @@ get_username/0, get_os_family/0, hostatom/0, hostatom/1, hoststr/0, hoststr/1, framework_call/2,framework_call/3,framework_call/4, - format_loc/1, package_str/1, package_atom/1, + format_loc/1, call_trace/1]). -include("test_server_internal.hrl"). -define(crash_dump_tar,"crash_dumps.tar.gz"). @@ -553,7 +553,7 @@ format_loc([{Mod,Func,Line}|Rest]) -> format_loc([{Mod,LineOrFunc}]) -> format_loc({Mod,LineOrFunc}); format_loc({Mod,Func}) when is_atom(Func) -> - io_lib:format("{~s,~w}",[package_str(Mod),Func]); + io_lib:format("{~w,~w}",[Mod,Func]); format_loc(Loc) -> io_lib:format("~p",[Loc]). @@ -562,15 +562,15 @@ format_loc1([{Mod,Func,Line}]) -> format_loc1([{Mod,Func,Line}|Rest]) -> [" ",format_loc1({Mod,Func,Line}),",\n"|format_loc1(Rest)]; format_loc1({Mod,Func,Line}) -> - ModStr = package_str(Mod), + ModStr = atom_to_list(Mod), case {lists:member(no_src, get(test_server_logopts)), lists:reverse(ModStr)} of {false,[$E,$T,$I,$U,$S,$_|_]} -> - io_lib:format("{~s,~w,<a href=\"~s~s#~w\">~w</a>}", - [ModStr,Func,downcase(ModStr),?src_listing_ext, + io_lib:format("{~w,~w,<a href=\"~s~s#~w\">~w</a>}", + [Mod,Func,downcase(ModStr),?src_listing_ext, Line,Line]); _ -> - io_lib:format("{~s,~w,~w}",[ModStr,Func,Line]) + io_lib:format("{~w,~w,~w}",[Mod,Func,Line]) end. downcase(S) -> downcase(S, []). @@ -581,22 +581,6 @@ downcase([C|Rest], Result) -> downcase([], Result) -> lists:reverse(Result). -package_str(Mod) when is_atom(Mod) -> - atom_to_list(Mod); -package_str(Mod) when is_list(Mod), is_atom(hd(Mod)) -> - %% convert [s1,s2] -> "s1.s2" - [_|M] = lists:flatten(["."++atom_to_list(S) || S <- Mod]), - M; -package_str(Mod) when is_list(Mod) -> - Mod. - -package_atom(Mod) when is_atom(Mod) -> - Mod; -package_atom(Mod) when is_list(Mod), is_atom(hd(Mod)) -> - list_to_atom(package_str(Mod)); -package_atom(Mod) when is_list(Mod) -> - list_to_atom(Mod). - %%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%% %% call_trace(TraceSpecFile) -> ok %% diff --git a/lib/test_server/src/ts.erl b/lib/test_server/src/ts.erl index bb6fb991e0..cfd7161dbd 100644 --- a/lib/test_server/src/ts.erl +++ b/lib/test_server/src/ts.erl @@ -259,21 +259,43 @@ run(List, Opts) when is_list(List), is_list(Opts) -> run(Testspec, Config) when is_atom(Testspec), is_list(Config) -> Options=check_test_get_opts(Testspec, Config), File=atom_to_list(Testspec), - Spec = case code:lib_dir(Testspec) of - _ when Testspec == emulator; - Testspec == system; - Testspec == epmd -> - File++".spec"; - {error, bad_name} -> - create_skip_spec(Testspec, tests(Testspec)); - Path -> - case file:read_file_info(filename:join(Path,"ebin")) of - {ok,_} -> - File++".spec"; - _ -> - create_skip_spec(Testspec, tests(Testspec)) - end - end, + WhatToDo = + case Testspec of + %% Known to exist but fails generic tests below + emulator -> test; + system -> test; + erl_interface -> test; + epmd -> test; + _ -> + case code:lib_dir(Testspec) of + {error,bad_name} -> + %% Application does not exist + skip; + Path -> + case file:read_file_info(filename:join(Path,"ebin")) of + {ok,#file_info{type=directory}} -> + %% Erlang application is built + test; + _ -> + case filelib:wildcard( + filename:join([Path,"priv","*.jar"])) of + [] -> + %% The application is not built + skip; + [_|_] -> + %% Java application is built + test + end + end + end + end, + Spec = + case WhatToDo of + skip -> + create_skip_spec(Testspec, tests(Testspec)); + test -> + File++".spec" + end, run_test(File, [{spec,[Spec]}], Options); %% Runs one module in a spec (interactive) run(Testspec, Mod) when is_atom(Testspec), is_atom(Mod) -> diff --git a/lib/test_server/test/test_server_test_lib.erl b/lib/test_server/test/test_server_test_lib.erl index 4e89abf308..d466aa0110 100644 --- a/lib/test_server/test/test_server_test_lib.erl +++ b/lib/test_server/test/test_server_test_lib.erl @@ -83,14 +83,21 @@ start_slave(Config,_Level) -> post_end_per_testcase(_TC, Config, Return, State) -> Node = proplists:get_value(node, Config), - case test_server:is_cover() of - true -> - cover:flush(Node); - false -> - ok + Cover = test_server:is_cover(), + if Cover-> cover:flush(Node); + true -> ok end, + erlang:monitor_node(Node, true), slave:stop(Node), - + receive + {nodedown, Node} -> + if Cover -> cover:stop(Node); + true -> ok + end + after 5000 -> + erlang:monitor_node(Node, false), + receive {nodedown, Node} -> ok after 0 -> ok end %flush + end, {Return, State}. %% Parse an .suite log file diff --git a/lib/test_server/vsn.mk b/lib/test_server/vsn.mk index aecf595f3f..b956ebb2b3 100644 --- a/lib/test_server/vsn.mk +++ b/lib/test_server/vsn.mk @@ -1 +1 @@ -TEST_SERVER_VSN = 3.5.2 +TEST_SERVER_VSN = 3.5.3 diff --git a/lib/tools/doc/src/xref.xml b/lib/tools/doc/src/xref.xml index 17de66bb22..9706ae6746 100644 --- a/lib/tools/doc/src/xref.xml +++ b/lib/tools/doc/src/xref.xml @@ -1453,7 +1453,7 @@ Evaluates a predefined analysis. Note however that the code path will be traversed once for each used <seealso marker="#library_module">library module</seealso> while setting up module data. On the other hand, if there are only a few modules that are - used by not analyzed, using <c>code_path</c> may be faster + used but not analyzed, using <c>code_path</c> may be faster than setting the library path to <c>code:get_path()</c>. </p> <p>If the library path is set to <c>code_path</c>, the set of diff --git a/lib/tools/emacs/erlang.el b/lib/tools/emacs/erlang.el index e2bcd37def..2967acf310 100644 --- a/lib/tools/emacs/erlang.el +++ b/lib/tools/emacs/erlang.el @@ -997,6 +997,22 @@ behaviour.") 1 'font-lock-function-name-face t)) "Font lock keyword highlighting a function header.") +(defface erlang-font-lock-exported-function-name-face + '((default (:inherit font-lock-function-name-face))) + "Face used for highlighting exported functions.") + +(defvar erlang-font-lock-exported-function-name-face + 'erlang-font-lock-exported-function-name-face) + +(defvar erlang-inhibit-exported-function-name-face nil + "Inhibit separate face for exported functions") + +(defvar erlang-font-lock-keywords-exported-function-header + (list + (list #'erlang-match-next-exported-function + 1 'erlang-font-lock-exported-function-name-face t)) + "Font lock keyword highlighting an exported function header.") + (defvar erlang-font-lock-keywords-int-bifs (list (list (concat erlang-int-bif-regexp "\\s-*(") @@ -1132,7 +1148,8 @@ There exists three levels of Font Lock keywords for Erlang: `erlang-font-lock-keywords-1' - Function headers and reserved keywords. `erlang-font-lock-keywords-2' - Bifs, guards and `single quotes'. `erlang-font-lock-keywords-3' - Variables, macros and records. - `erlang-font-lock-keywords-4' - Function names, Funs, LCs (not Atoms) + `erlang-font-lock-keywords-4' - Exported functions, Function names, + Funs, LCs (not Atoms). To use a specific level, please set the variable `font-lock-maximum-decoration' to the appropriate level. Note that the @@ -1174,6 +1191,7 @@ Example: (defvar erlang-font-lock-keywords-4 (append erlang-font-lock-keywords-3 + erlang-font-lock-keywords-exported-function-header erlang-font-lock-keywords-int-function-calls erlang-font-lock-keywords-ext-function-calls erlang-font-lock-keywords-fun-n @@ -3662,6 +3680,38 @@ In the future the list may contain more elements." str) (store-match-data md)))) +(defun erlang-match-next-exported-function (max) + "Returns non-nil if there is an exported function in the current +buffer between point and MAX." + (block nil + (while (and (not erlang-inhibit-exported-function-name-face) + (erlang-match-next-function max)) + (when (erlang-last-match-exported-p) + (return (match-data)))))) + +(defun erlang-match-next-function (max) + "Searches forward in current buffer for the next erlang function, +bounded by position MAX." + (re-search-forward erlang-defun-prompt-regexp max 'move-point)) + +(defun erlang-last-match-exported-p () + "Returns non-nil if match-data describes the name and arity of an +exported function." + (save-excursion + (save-match-data + (goto-char (match-beginning 1)) + (erlang-function-exported-p + (erlang-remove-quotes (erlang-get-function-name)) + (erlang-get-function-arity))))) + +(defun erlang-function-exported-p (name arity) + "Returns non-nil if function of name and arity is exported in current buffer." + (save-excursion + (let* ((old-match-data (match-data)) + (exports (erlang-get-export))) + (store-match-data old-match-data) + (member (cons name arity) exports)))) + ;;; Check module name diff --git a/lib/tools/src/cover.erl b/lib/tools/src/cover.erl index 10f14b0a49..ab29d156aa 100644 --- a/lib/tools/src/cover.erl +++ b/lib/tools/src/cover.erl @@ -705,7 +705,9 @@ main_process_loop(State) -> remote_collect('_',Nodes,true), reply(From, ok), Nodes1 = State#main_state.nodes--Nodes, - main_process_loop(State#main_state{nodes=Nodes1}); + LostNodes1 = State#main_state.lost_nodes--Nodes, + main_process_loop(State#main_state{nodes=Nodes1, + lost_nodes=LostNodes1}); {From, {flush,Nodes}} -> remote_collect('_',Nodes,false), @@ -792,8 +794,15 @@ main_process_loop(State) -> {'DOWN', _MRef, process, {?SERVER,Node}, _Info} -> %% A remote cover_server is down, mark as lost - Nodes = State#main_state.nodes--[Node], - Lost = [Node|State#main_state.lost_nodes], + {Nodes,Lost} = + case lists:member(Node,State#main_state.nodes) of + true -> + N = State#main_state.nodes--[Node], + L = [Node|State#main_state.lost_nodes], + {N,L}; + false -> % node stopped + {State#main_state.nodes,State#main_state.lost_nodes} + end, main_process_loop(State#main_state{nodes=Nodes,lost_nodes=Lost}); {nodeup,Node} -> @@ -2087,30 +2096,40 @@ do_analyse_to_file(Module, OutFile, ErlFile, HTML) -> case file:open(OutFile, [write]) of {ok, OutFd} -> if HTML -> - io:format(OutFd, - "<html>\n" - "<head><title>~s</title></head>" - "<body bgcolor=white text=black>\n" - "<pre>\n", - [OutFile]); + Encoding = encoding(ErlFile), + Header = + ["<!DOCTYPE HTML PUBLIC " + "\"-//W3C//DTD HTML 3.2 Final//EN\">\n" + "<html>\n" + "<head>\n" + "<meta http-equiv=\"Content-Type\"" + " content=\"text/html; charset=", + Encoding,"\"/>\n" + "<title>",OutFile,"</title>\n" + "</head>" + "<body style='background-color: white;" + " color: black'>\n" + "<pre>\n"], + file:write(OutFd,Header); true -> ok end, %% Write some initial information to the output file {{Y,Mo,D},{H,Mi,S}} = calendar:local_time(), - io:format(OutFd, "File generated from ~s by COVER " - "~p-~s-~s at ~s:~s:~s~n", - [ErlFile, - Y, - string:right(integer_to_list(Mo), 2, $0), - string:right(integer_to_list(D), 2, $0), - string:right(integer_to_list(H), 2, $0), - string:right(integer_to_list(Mi), 2, $0), - string:right(integer_to_list(S), 2, $0)]), - io:format(OutFd, "~n" - "**************************************" - "**************************************" - "~n~n", []), + Timestamp = + io_lib:format("~p-~s-~s at ~s:~s:~s", + [Y, + string:right(integer_to_list(Mo), 2, $0), + string:right(integer_to_list(D), 2, $0), + string:right(integer_to_list(H), 2, $0), + string:right(integer_to_list(Mi), 2, $0), + string:right(integer_to_list(S), 2, $0)]), + file:write(OutFd, + ["File generated from ",ErlFile," by COVER ", + Timestamp,"\n\n" + "**************************************" + "**************************************" + "\n\n"]), print_lines(Module, InFd, OutFd, 1, HTML), @@ -2405,3 +2424,20 @@ pmap(Fun, [], [], Limit, Cnt, Acc) -> {'DOWN', _Ref, process, X, _} when is_pid(X) -> pmap(Fun, [], [], Limit, Cnt - 1, Acc) end. + +%%%----------------------------------------------------------------- +%%% Read encoding from source file +encoding(File) -> + Encoding = + case epp:read_encoding(File) of + none -> + epp:default_encoding(); + E -> + E + end, + html_encoding(Encoding). + +html_encoding(latin1) -> + "iso-8859-1"; +html_encoding(utf8) -> + "utf-8". diff --git a/lib/tools/src/lcnt.erl b/lib/tools/src/lcnt.erl index 989a661b75..70d62307c8 100644 --- a/lib/tools/src/lcnt.erl +++ b/lib/tools/src/lcnt.erl @@ -1,7 +1,8 @@ +%% -*- coding: utf-8 -*- %% %% %CopyrightBegin% %% -%% Copyright Ericsson AB 2010. All Rights Reserved. +%% Copyright Ericsson AB 2010-2012. All Rights Reserved. %% %% The contents of this file are subject to the Erlang Public License, %% Version 1.1, (the "License"); you may not use this file except in diff --git a/lib/tools/src/tools.app.src b/lib/tools/src/tools.app.src index 94998fb763..553c5eb96b 100644 --- a/lib/tools/src/tools.app.src +++ b/lib/tools/src/tools.app.src @@ -1,7 +1,7 @@ %% %% %CopyrightBegin% %% -%% Copyright Ericsson AB 1996-2009. All Rights Reserved. +%% Copyright Ericsson AB 1996-2012. All Rights Reserved. %% %% The contents of this file are subject to the Erlang Public License, %% Version 1.1, (the "License"); you may not use this file except in diff --git a/lib/tools/test/cover_SUITE.erl b/lib/tools/test/cover_SUITE.erl index 3bf1b44af8..57260a3869 100644 --- a/lib/tools/test/cover_SUITE.erl +++ b/lib/tools/test/cover_SUITE.erl @@ -23,7 +23,9 @@ init_per_group/2,end_per_group/2]). -export([start/1, compile/1, analyse/1, misc/1, stop/1, - distribution/1, reconnect/1, die_and_reconnect/1, export_import/1, + distribution/1, reconnect/1, die_and_reconnect/1, + dont_reconnect_after_stop/1, stop_node_after_disconnect/1, + export_import/1, otp_5031/1, eif/1, otp_5305/1, otp_5418/1, otp_6115/1, otp_7095/1, otp_8188/1, otp_8270/1, otp_8273/1, otp_8340/1]). @@ -47,6 +49,7 @@ all() -> undefined -> [start, compile, analyse, misc, stop, distribution, reconnect, die_and_reconnect, + dont_reconnect_after_stop, stop_node_after_disconnect, export_import, otp_5031, eif, otp_5305, otp_5418, otp_6115, otp_7095, otp_8188, otp_8270, otp_8273, otp_8340]; @@ -86,8 +89,11 @@ init_per_testcase(TC, Config) when TC =:= misc; init_per_testcase(_TestCase, Config) -> Config. -end_per_testcase(_TestCase, _Config) -> - %cover:stop(), +end_per_testcase(TestCase, _Config) -> + case lists:member(TestCase,[start,compile,analyse,misc]) of + true -> ok; + false -> cover:stop() + end, ok. start(suite) -> []; @@ -447,8 +453,8 @@ reconnect(Config) -> %% Disconnect and check that node is removed from main cover node net_kernel:disconnect(N1), + timer:sleep(500), % allow some to detect disconnect and for f:f2() call [] = cover:which_nodes(), - timer:sleep(500), % allow some time for the f:f2() call %% Do some add one module (b) and remove one module (a) code:purge(a), @@ -520,6 +526,94 @@ die_and_reconnect(Config) -> ?t:stop_node(N1), ok. +%% Test that a stopped node is not marked as lost, i.e. that it is not +%% reconnected if it is restarted (OTP-10638) +dont_reconnect_after_stop(Config) -> + DataDir = ?config(data_dir, Config), + ok = file:set_cwd(DataDir), + + {ok,f} = compile:file(f), + + NodeName = cover_SUITE_dont_reconnect_after_stop, + {ok,N1} = ?t:start_node(NodeName,peer, + [{args," -pa " ++ DataDir},{start_cover,false}]), + {ok,f} = cover:compile(f), + {ok,[N1]} = cover:start(nodes()), + + %% A call to check later + rpc:call(N1,f,f1,[]), + + %% Stop cover on the node, then terminate the node + cover:stop(N1), + rpc:call(N1,erlang,halt,[]), + [] = cover:which_nodes(), + + check_f_calls(1,0), + + %% Restart the node and check that cover does not reconnect + {ok,N1} = ?t:start_node(NodeName,peer, + [{args," -pa " ++ DataDir},{start_cover,false}]), + timer:sleep(300), + [] = cover:which_nodes(), + Beam = rpc:call(N1,code,which,[f]), + false = (Beam==cover_compiled), + + %% One more call... + rpc:call(N1,f,f1,[]), + cover:flush(N1), + + %% Ensure that the last call is not counted + check_f_calls(1,0), + + cover:stop(), + ?t:stop_node(N1), + ok. + +%% Test that a node which is stopped while it is marked as lost is not +%% reconnected if it is restarted (OTP-10638) +stop_node_after_disconnect(Config) -> + DataDir = ?config(data_dir, Config), + ok = file:set_cwd(DataDir), + + {ok,f} = compile:file(f), + + NodeName = cover_SUITE_stop_node_after_disconnect, + {ok,N1} = ?t:start_node(NodeName,peer, + [{args," -pa " ++ DataDir},{start_cover,false}]), + {ok,f} = cover:compile(f), + {ok,[N1]} = cover:start(nodes()), + + %% A call to check later + rpc:call(N1,f,f1,[]), + + %% Flush the node, then terminate the node to make it marked as lost + cover:flush(N1), + rpc:call(N1,erlang,halt,[]), + + check_f_calls(1,0), + + %% Stop cover on node + cover:stop(N1), + + %% Restart the node and check that cover does not reconnect + {ok,N1} = ?t:start_node(NodeName,peer, + [{args," -pa " ++ DataDir},{start_cover,false}]), + timer:sleep(300), + [] = cover:which_nodes(), + Beam = rpc:call(N1,code,which,[f]), + false = (Beam==cover_compiled), + + %% One more call... + rpc:call(N1,f,f1,[]), + cover:flush(N1), + + %% Ensure that the last call is not counted + check_f_calls(1,0), + + cover:stop(), + ?t:stop_node(N1), + ok. + export_import(suite) -> []; export_import(Config) when is_list(Config) -> ?line DataDir = ?config(data_dir, Config), @@ -908,7 +1002,7 @@ otp_8270(Config) when is_list(Config) -> ok. otp_8273(doc) -> - ["OTP-8270. Bug."]; + ["OTP-8273. Bug."]; otp_8273(suite) -> []; otp_8273(Config) when is_list(Config) -> Test = <<"-module(t). diff --git a/lib/tools/test/xref_SUITE.erl b/lib/tools/test/xref_SUITE.erl index fd3e111d8d..cf49526156 100644 --- a/lib/tools/test/xref_SUITE.erl +++ b/lib/tools/test/xref_SUITE.erl @@ -1,3 +1,4 @@ +%% -*- coding: utf-8 -*- %% %% %CopyrightBegin% %% @@ -2521,7 +2522,7 @@ otp_10192(doc) -> otp_10192(Conf) when is_list(Conf) -> PrivDir = ?privdir, {ok, _Pid} = xref:start(s), - Dir = filename:join(PrivDir, "�"), + Dir = filename:join(PrivDir, "ä"), ok = file:make_dir(Dir), {ok, []} = xref:add_directory(s, Dir), xref:stop(s), diff --git a/lib/wx/aclocal.m4 b/lib/wx/aclocal.m4 index b1cf1fe404..9578cd35c4 100644 --- a/lib/wx/aclocal.m4 +++ b/lib/wx/aclocal.m4 @@ -740,11 +740,16 @@ dnl Try to find POSIX threads dnl The usual pthread lib... AC_CHECK_LIB(pthread, pthread_create, THR_LIBS="-lpthread") -dnl FreeBSD has pthreads in special c library, c_r... +dnl Very old versions of FreeBSD have pthreads in special c library, c_r... if test "x$THR_LIBS" = "x"; then AC_CHECK_LIB(c_r, pthread_create, THR_LIBS="-lc_r") fi +dnl QNX has pthreads in standard C library + if test "x$THR_LIBS" = "x"; then + AC_CHECK_FUNC(pthread_create, THR_LIBS="none_needed") + fi + dnl On ofs1 the '-pthread' switch should be used if test "x$THR_LIBS" = "x"; then AC_MSG_CHECKING([if the '-pthread' switch can be used]) @@ -765,6 +770,9 @@ dnl On ofs1 the '-pthread' switch should be used if test "x$THR_LIBS" != "x"; then THR_DEFS="$THR_DEFS -D_THREAD_SAFE -D_REENTRANT -DPOSIX_THREADS" THR_LIB_NAME=pthread + if test "x$THR_LIBS" = "xnone_needed"; then + THR_LIBS= + fi case $host_os in solaris*) THR_DEFS="$THR_DEFS -D_POSIX_PTHREAD_SEMANTICS" ;; diff --git a/lib/wx/src/wx.erl b/lib/wx/src/wx.erl index 7d62305048..2a4b18d101 100644 --- a/lib/wx/src/wx.erl +++ b/lib/wx/src/wx.erl @@ -102,12 +102,18 @@ new() -> %% @doc Starts a wx server. %% Option may be {debug, Level}, see debug/1. --spec new([Option]) -> wx_object() when Option :: {debug, list() | atom()}. +%% Or {silent_start, Bool}, which causes error messages at startup to +%% be suppressed. The latter can be used as a silent test of whether +%% wx is properly installed or not. +-spec new([Option]) -> wx_object() when Option :: {debug, list() | atom()} | + {silent_start, boolean()}. new(Options) when is_list(Options) -> - #wx_env{port=Port} = wxe_server:start(), - put(opengl_port, Port), Debug = proplists:get_value(debug, Options, 0), - debug(Debug), + SilentStart = proplists:get_value(silent_start, Options, false), + Level = calc_level(Debug), + #wx_env{port=Port} = wxe_server:start(SilentStart andalso Level =:= 0), + put(opengl_port, Port), + set_debug(Level), null(). %% @doc Stops a wx server. @@ -282,13 +288,16 @@ release_memory(Bin) when is_binary(Bin) -> -spec debug(Level | [Level]) -> ok when Level :: none | verbose | trace | driver | integer(). -debug(none) -> debug(0); -debug(verbose) -> debug(1); -debug(trace) -> debug(2); -debug(driver) -> debug(16); -debug([]) -> debug(0); +debug(Debug) -> + Level = calc_level(Debug), + set_debug(Level). -debug(List) when is_list(List) -> +calc_level(none) -> calc_level(0); +calc_level(verbose) -> calc_level(1); +calc_level(trace) -> calc_level(2); +calc_level(driver) -> calc_level(16); +calc_level([]) -> calc_level(0); +calc_level(List) when is_list(List) -> {Drv,Erl} = lists:foldl(fun(verbose, {Drv,_Erl}) -> {Drv,1}; @@ -297,8 +306,11 @@ debug(List) when is_list(List) -> (driver, {_Drv,Erl}) -> {16, Erl} end, {0,0}, List), - debug(Drv + Erl); -debug(Level) when is_integer(Level) -> + Drv + Erl; +calc_level(Level) when is_integer(Level) -> + Level. + +set_debug(Level) when is_integer(Level) -> case get(?WXE_IDENTIFIER) of undefined -> erlang:error({wxe,unknown_port}); #wx_env{debug=Old} when Old =:= Level -> ok; diff --git a/lib/wx/src/wxe_master.erl b/lib/wx/src/wxe_master.erl index ac6e4a56e6..b98a7c793e 100644 --- a/lib/wx/src/wxe_master.erl +++ b/lib/wx/src/wxe_master.erl @@ -28,7 +28,7 @@ -behaviour(gen_server). %% API --export([start/0, init_port/0, init_opengl/0]). +-export([start/1, init_port/1, init_opengl/0]). %% gen_server callbacks -export([init/1, handle_call/3, handle_cast/2, handle_info/2, @@ -47,20 +47,20 @@ %% API %%==================================================================== %%-------------------------------------------------------------------- -%% Function: start_link() -> {ok,Pid} | ignore | {error,Error} +%% Function: start(SilentStart) -> {ok,Pid} | ignore | {error,Error} %% Description: Starts the server %%-------------------------------------------------------------------- -start() -> - gen_server:start({local, ?MODULE}, ?MODULE, [], []). +start(SilentStart) -> + gen_server:start({local, ?MODULE}, ?MODULE, [SilentStart], []). %%-------------------------------------------------------------------- -%% Function: init_port() -> {UserPort,CallBackPort} | error(Error) +%% Function: init_port(SilentStart) -> {UserPort,CallBackPort} | error(Error) %% Description: Creates the port %%-------------------------------------------------------------------- -init_port() -> +init_port(SilentStart) -> case whereis(?MODULE) of undefined -> - case start() of + case start(SilentStart) of {ok,Pid} -> Pid; {error,{already_started,Pid}} -> Pid; {error, {Reason,Stack}} -> @@ -93,14 +93,17 @@ init_opengl() -> %% {stop, Reason} %% Description: Initiates the server %%-------------------------------------------------------------------- -init([]) -> +init([SilentStart]) -> DriverName = ?DRIVER, - PrivDir = wxe_util:priv_dir(?DRIVER), + PrivDir = wxe_util:priv_dir(?DRIVER, SilentStart), erlang:group_leader(whereis(init), self()), case catch erlang:system_info(smp_support) of true -> ok; _ -> - error_logger:format("WX ERROR: SMP emulator required (start with erl -smp)", []), + wxe_util:opt_error_log(SilentStart, + "WX ERROR: SMP emulator required" + " (start with erl -smp)", + []), erlang:error(not_smp) end, @@ -114,7 +117,9 @@ init([]) -> case erl_ddll:load_driver(PrivDir,DriverName) of ok -> ok; {error, What} -> - error_logger:format("WX Failed loading ~p@~p ~n", [DriverName,PrivDir]), + wxe_util:opt_error_log(SilentStart, + "WX Failed loading ~p@~p ~n", + [DriverName,PrivDir]), Str = erl_ddll:format_error(What), erlang:error({load_driver,Str}) end, @@ -210,4 +215,3 @@ debug_ping(Port) -> _R = (catch erlang:port_call(Port, 0, [])), %% io:format("Erlang ping ~p ~n", [_R]), debug_ping(Port). - diff --git a/lib/wx/src/wxe_server.erl b/lib/wx/src/wxe_server.erl index 6e982c97f6..689fe16a70 100644 --- a/lib/wx/src/wxe_server.erl +++ b/lib/wx/src/wxe_server.erl @@ -29,7 +29,7 @@ -behaviour(gen_server). %% API --export([start/0, stop/0, register_me/1, set_debug/2, invoke_callback/1]). +-export([start/1, stop/0, register_me/1, set_debug/2, invoke_callback/1]). %% gen_server callbacks -export([init/1, handle_call/3, handle_cast/2, handle_info/2, @@ -49,13 +49,13 @@ %% API %%==================================================================== %%-------------------------------------------------------------------- -%% Function: start() -> #wx_env{} +%% Function: start(SilentStart) -> #wx_env{} %% Description: Starts the server %%-------------------------------------------------------------------- -start() -> +start(SilentStart) -> case get(?WXE_IDENTIFIER) of undefined -> - case gen_server:start(?MODULE, [], []) of + case gen_server:start(?MODULE, [SilentStart], []) of {ok, Pid} -> {ok, Port} = gen_server:call(Pid, get_port, infinity), wx:set_env(Env = #wx_env{port=Port,sv=Pid}), @@ -69,7 +69,7 @@ start() -> Env; false -> %% Ok we got an old wx env, someone forgot erase(?WXE_IDENTIFIER), %% to call wx:destroy() - start() + start(SilentStart) end end. @@ -88,8 +88,8 @@ set_debug(Pid, Level) -> %% gen_server callbacks %%==================================================================== -init([]) -> - {Port,CBPort} = wxe_master:init_port(), +init([SilentStart]) -> + {Port,CBPort} = wxe_master:init_port(SilentStart), put(?WXE_IDENTIFIER, #wx_env{port=Port,sv=self()}), {ok,#state{port=Port, cb_port=CBPort, users=gb_trees:empty(), cb=gb_trees:empty(), cb_cnt=1}}. diff --git a/lib/wx/src/wxe_util.erl b/lib/wx/src/wxe_util.erl index 02bca62486..3022b7f3f2 100644 --- a/lib/wx/src/wxe_util.erl +++ b/lib/wx/src/wxe_util.erl @@ -32,7 +32,7 @@ get_const/1,colour_bin/1,datetime_bin/1, to_bool/1,from_bool/1]). --export([wxgl_dl/0, priv_dir/1]). +-export([wxgl_dl/0, priv_dir/2, opt_error_log/3]). -include("wxe.hrl"). @@ -205,7 +205,7 @@ check_previous() -> wxgl_dl() -> DynLib0 = "erl_gl", - PrivDir = priv_dir(DynLib0), + PrivDir = priv_dir(DynLib0, false), DynLib = case os:type() of {win32,_} -> DynLib0 ++ ".dll"; @@ -214,7 +214,7 @@ wxgl_dl() -> end, filename:join(PrivDir, DynLib). -priv_dir(Driver0) -> +priv_dir(Driver0, Silent) -> {file, Path} = code:is_loaded(?MODULE), Priv = case filelib:is_regular(Path) of true -> @@ -229,13 +229,13 @@ priv_dir(Driver0) -> _ -> Driver0 ++ ".so" end, - case file:read_file_info(filename:join(Priv, Driver)) of {ok, _} -> Priv; {error, _} -> - error_logger:format("ERROR: Could not find \'~s\' in: ~s~n", - [Driver, Priv]), + opt_error_log(Silent, + "ERROR: Could not find \'~s\' in: ~s~n", + [Driver, Priv]), erlang:error({load_driver, "No driver found"}) end. @@ -244,3 +244,7 @@ strip(Src, Src) -> strip([H|R], Src) -> [H| strip(R, Src)]. +opt_error_log(false, Format, Args) -> + error_logger:format(Format, Args); +opt_error_log(true, _Format, _Args) -> + ok. diff --git a/lib/wx/test/wx_basic_SUITE.erl b/lib/wx/test/wx_basic_SUITE.erl index 46c72bb453..e240bbcfe8 100644 --- a/lib/wx/test/wx_basic_SUITE.erl +++ b/lib/wx/test/wx_basic_SUITE.erl @@ -1,7 +1,8 @@ +%% -*- coding: utf-8 -*- %% %% %CopyrightBegin% %% -%% Copyright Ericsson AB 2008-2011. All Rights Reserved. +%% Copyright Ericsson AB 2008-2012. All Rights Reserved. %% %% The contents of this file are subject to the Erlang Public License, %% Version 1.1, (the "License"); you may not use this file except in @@ -47,7 +48,7 @@ end_per_testcase(Func,Config) -> suite() -> [{ct_hooks,[ts_install_cth]}]. all() -> - [create_window, several_apps, wx_api, wx_misc, + [silent_start, create_window, several_apps, wx_api, wx_misc, data_types, wx_object]. groups() -> @@ -62,6 +63,25 @@ end_per_group(_GroupName, Config) -> %% The test cases +%% test silent start of wx +silent_start(TestInfo) when is_atom(TestInfo) -> wx_test_lib:tc_info(TestInfo); +silent_start(_Config) -> + ?mr(wx_ref, wx:new([])), + wx:destroy(), + + ?mr(wx_ref, wx:new([{silent_start, true}])), + wx:destroy(), + + ?mr(wx_ref, wx:new([{silent_start, true}, {debug, verbose}])), + wx:destroy(), + + ?mr(wx_ref, wx:new([{silent_start, false}])), + wx:destroy(), + + ?mr('EXIT', catch wx:new([{silent_start, foo}])), + + ok. + %% create and test creating a window create_window(TestInfo) when is_atom(TestInfo) -> wx_test_lib:tc_info(TestInfo); create_window(Config) -> @@ -109,7 +129,7 @@ wx_api(Config) -> ?m(true, wx:is_null(Wx)), Null = ?mr(wx_ref, wx:null()), ?m(true, wx:is_null(Null)), - Frame = ?mt(wxFrame, wxFrame:new(Wx, 1, "WX API: " ++ unicode:characters_to_list("������"))), + Frame = ?mt(wxFrame, wxFrame:new(Wx, 1, "WX API: " ++ unicode:characters_to_list("åäöÅÄÖ"))), ?m(false, wx:is_null(Frame)), ?m(wxFrame, wx:getObjectType(Frame)), Env = ?mr(wx_env, wx:get_env()), diff --git a/lib/xmerl/doc/src/motorcycles_dtd.txt b/lib/xmerl/doc/src/motorcycles_dtd.txt index bab0d563f0..62ad4ac5fe 100644 --- a/lib/xmerl/doc/src/motorcycles_dtd.txt +++ b/lib/xmerl/doc/src/motorcycles_dtd.txt @@ -15,4 +15,5 @@ <!ELEMENT date (#PCDATA)> <!ATTLIST bike year NMTOKEN #REQUIRED color NMTOKENS #REQUIRED - condition (useless | bad | serviceable | moderate | good | excellent | new | outstanding) "excellent" > + condition (useless | bad | serviceable | moderate | good | + excellent | new | outstanding) "excellent" > diff --git a/lib/xmerl/src/xmerl_scan.erl b/lib/xmerl/src/xmerl_scan.erl index 05431a5fd2..883153628c 100644 --- a/lib/xmerl/src/xmerl_scan.erl +++ b/lib/xmerl/src/xmerl_scan.erl @@ -1,7 +1,8 @@ +%% -*- coding: utf-8 -*- %% %% %CopyrightBegin% %% -%% Copyright Ericsson AB 2003-2011. All Rights Reserved. +%% Copyright Ericsson AB 2003-2012. All Rights Reserved. %% %% The contents of this file are subject to the Erlang Public License, %% Version 1.1, (the "License"); you may not use this file except in @@ -461,7 +462,7 @@ hook(X, State) -> event(_X, S) -> S. -%% The acc/3 function can return either {Acc�, S'} or {Acc', Pos', S'}, +%% The acc/3 function can return either {Acc´, S'} or {Acc', Pos', S'}, %% where Pos' can be derived from X#xmlElement.pos, X#xmlText.pos, or %% X#xmlAttribute.pos (whichever is the current object type.) %% The acc/3 function is not allowed to redefine the type of object diff --git a/lib/xmerl/src/xmerl_xsd.erl b/lib/xmerl/src/xmerl_xsd.erl index 78d84d23a4..ffe227c671 100644 --- a/lib/xmerl/src/xmerl_xsd.erl +++ b/lib/xmerl/src/xmerl_xsd.erl @@ -1,3 +1,4 @@ +%% -*- coding: utf-8 -*- %% %% %CopyrightBegin% %% @@ -849,7 +850,7 @@ element_content({import,S=#xsd_state{schema_name=ThisSchema, SchemaLocation = get_attribute_value(schemaLocation,I,absent), %% If SchemaLocation is absent, the identification of that schema %% is leaved to the instance, application or user, via the - %% mechanisms described ��4.3 in XML Schema Part 1. + %% mechanisms described §4.3 in XML Schema Part 1. S2 = process_external_schema_once(SchemaLocation,Namespace,S), {{import,[]},S2#xsd_state{schema_name=ThisSchema, diff --git a/lib/xmerl/test/xmerl_SUITE.erl b/lib/xmerl/test/xmerl_SUITE.erl index 55b6d1844c..e21355f877 100644 --- a/lib/xmerl/test/xmerl_SUITE.erl +++ b/lib/xmerl/test/xmerl_SUITE.erl @@ -1,7 +1,8 @@ +%% -*- coding: utf-8 -*- %% %% %CopyrightBegin% %% -%% Copyright Ericsson AB 2008-2011. All Rights Reserved. +%% Copyright Ericsson AB 2008-2012. All Rights Reserved. %% %% The contents of this file are subject to the Erlang Public License, %% Version 1.1, (the "License"); you may not use this file except in @@ -531,8 +532,8 @@ ticket_7430(Config) -> {xmlElement,a,a,[], {xmlNamespace,[],[]}, [],1,[], - [{xmlText,[{a,1}],1,[],"�",text}, - {xmlText,[{a,1}],2,[],"\n�",text}], + [{xmlText,[{a,1}],1,[],"é",text}, + {xmlText,[{a,1}],2,[],"\né",text}], [],_,undeclared} -> ok; _ -> diff --git a/make/otp.mk.in b/make/otp.mk.in index f2c2cee767..90f448d4a0 100644 --- a/make/otp.mk.in +++ b/make/otp.mk.in @@ -252,6 +252,7 @@ XSLTPROC = @XSLTPROC@ FOP = @FOP@ DOCGEN=$(ERL_TOP)/lib/erl_docgen +FOP_CONFIG = $(DOCGEN)/priv/fop.xconf ifneq (,$(findstring $(origin SPECS_ESRC),$(DUBIOUS_ORIGINS))) SPECS_ESRC = ../../src @@ -308,5 +309,5 @@ $(MAN9DIR)/%.9: %.xml escript $(DOCGEN)/priv/bin/codeline_preprocessing.escript $< $@ .fo.pdf: - $(FOP) -fo $< -pdf $@ + $(FOP) -c $(FOP_CONFIG) -fo $< -pdf $@ diff --git a/system/doc/efficiency_guide/advanced.xml b/system/doc/efficiency_guide/advanced.xml index 821175bb09..ac35a37bc4 100644 --- a/system/doc/efficiency_guide/advanced.xml +++ b/system/doc/efficiency_guide/advanced.xml @@ -123,12 +123,11 @@ On 64-bit architectures: 4 words for a reference from the current local node, an <tag><em>Processes</em></tag> <item> <p>The maximum number of simultaneously alive Erlang processes is - by default 32768. This limit can be raised up to at most 268435456 - processes at startup (see documentation of the system flag - <seealso marker="erts:erl#max_processes">+P</seealso> in the - <seealso marker="erts:erl">erl(1)</seealso> documentation). - The maximum limit of 268435456 processes will at least on a 32-bit - architecture be impossible to reach due to memory shortage.</p> + by default 32768. This limit can be configured at startup, + for more information see the + <seealso marker="erts:erl#max_processes"><c>+P</c></seealso> + command line flag of + <seealso marker="erts:erl"><c>erl(1)</c></seealso>.</p> </item> <tag><em>Distributed nodes</em></tag> <item> @@ -184,13 +183,12 @@ On 64-bit architectures: 4 words for a reference from the current local node, an <tag><em>Open ports</em></tag> <item> <marker id="ports"></marker> - <p>The maximum number of simultaneously open Erlang ports is - by default 1024. This limit can be raised up to at most 268435456 - at startup (see environment variable - <seealso marker="erts:erlang#ERL_MAX_PORTS">ERL_MAX_PORTS</seealso> - in <seealso marker="erts:erlang">erlang(3)</seealso>) - The maximum limit of 268435456 open ports will at least on a 32-bit - architecture be impossible to reach due to memory shortage.</p> + <p>The maximum number of simultaneously oper Erlang ports is + often by default 16384. This limit can be configured at startup, + for more information see the + <seealso marker="erts:erl#max_ports"><c>+Q</c></seealso> + command line flag of + <seealso marker="erts:erl"><c>erl(1)</c></seealso>.</p> </item> <tag><em>Open files, and sockets</em></tag> <item> <marker id="files_sockets"></marker> diff --git a/system/doc/efficiency_guide/drivers.xml b/system/doc/efficiency_guide/drivers.xml index fec68ca059..b10595ea4d 100644 --- a/system/doc/efficiency_guide/drivers.xml +++ b/system/doc/efficiency_guide/drivers.xml @@ -105,9 +105,9 @@ client_port() -> <p>If you know that the binaries you return are always small, you should use driver API calls that do not require a pre-allocated binary, for instance - <seealso marker="erts:erl_driver#int driver_output-3">driver_output()</seealso> + <seealso marker="erts:erl_driver#driver_output">driver_output()</seealso> or - <seealso marker="erts:erl_driver#int driver_output_term-3">driver_output_term()</seealso> + <seealso marker="erts:erl_driver#erl_drv_output_term">erl_drv_output_term()</seealso> using the <c>ERL_DRV_BUF2BINARY</c> format, to allow the run-time to construct a heap binary.</p> @@ -120,7 +120,7 @@ client_port() -> the driver to an Erlang process, the driver must first allocate the binary and then send it to an Erlang process in some way.</p> - <p>Use <seealso marker="erts:erl_driver#ErlDrvBinary* driver_alloc_binary-1">driver_alloc_binary()</seealso> to allocate a binary.</p> + <p>Use <seealso marker="erts:erl_driver#driver_alloc_binary">driver_alloc_binary()</seealso> to allocate a binary.</p> <p>There are several ways to send a binary created with <c>driver_alloc_binary()</c>.</p> @@ -128,17 +128,17 @@ client_port() -> <list type="bulleted"> <item><p>From the <c>control</c> callback, a binary can be returned provided that - <seealso marker="erts:erl_driver#void set_port_control_flags-2">set_port_control()</seealso> + <seealso marker="erts:erl_driver#set_port_control_flags">set_port_control_flags()</seealso> has been called with the flag value <c>PORT_CONTROL_FLAG_BINARY</c>.</p> </item> <item><p>A single binary can be sent with - <seealso marker="erts:erl_driver#int driver_output_binary-6">driver_output_binary()</seealso>.</p></item> + <seealso marker="erts:erl_driver#driver_output_binary">driver_output_binary()</seealso>.</p></item> <item><p>Using - <seealso marker="erts:erl_driver#int driver_output_term-3">driver_output_term()</seealso> + <seealso marker="erts:erl_driver#erl_drv_output_term">erl_drv_output_term()</seealso> or - <seealso marker="erts:erl_driver#int driver_send_term-4">driver_send_term()</seealso>, + <seealso marker="erts:erl_driver#erl_drv_send_term">erl_drv_send_term()</seealso>, a binary can be included in an Erlang term.</p> </item> </list> diff --git a/system/doc/reference_manual/introduction.xml b/system/doc/reference_manual/introduction.xml index 3dac5cfe13..7737c34469 100644 --- a/system/doc/reference_manual/introduction.xml +++ b/system/doc/reference_manual/introduction.xml @@ -4,7 +4,7 @@ <chapter> <header> <copyright> - <year>2003</year><year>2009</year> + <year>2003</year><year>2012</year> <holder>Ericsson AB. All Rights Reserved.</holder> </copyright> <legalnotice> @@ -150,6 +150,17 @@ </row> <tcaption>Character Classes.</tcaption> </table> + <p>In Erlang/OTP R16 the syntax of Erlang tokens was extended to + handle Unicode. To begin with the support is limited to strings, + but Erlang/OTP R18 is expected to handle Unicode atoms as well. + More about the usage of Unicode in Erlang source files can be + found in <seealso + marker="stdlib:unicode_usage#unicode_in_erlang">STDLIB's User'S + Guide</seealso>. The default encoding for Erlang source files + is still Latin-1, but in Erlang/OTP R17 the default encoding + will be UTF-8. The details on how to state the encoding of an + Erlang source file can be found in <seealso + marker="stdlib:epp#encoding">epp(3)</seealso>.</p> </section> </chapter> diff --git a/system/doc/reference_manual/ports.xml b/system/doc/reference_manual/ports.xml index 4847dd67cd..c4e4ef1d35 100644 --- a/system/doc/reference_manual/ports.xml +++ b/system/doc/reference_manual/ports.xml @@ -87,8 +87,14 @@ of bytes, the option <c>binary</c> must be included.</p> <p>The port owner <c>Pid</c> can communicate with the port <c>Port</c> by sending and receiving messages. (In fact, any - process can send the messages to the port, but the messages from - the port always go to the port owner).</p> + process can send the messages to the port, but the port owner must + be identified in the message).</p> + <p>As of OTP-R16 messages sent to ports are delivered truly + asynchronously. The underlying implementation previously + delivered messages to ports synchronously. Message passing has + however always been documented as an asynchronous operation, so + this should not be an issue for an Erlang program communicating + with ports, unless false assumptions about ports has been made.</p> <p>Below, <c>Data</c> must be an I/O list. An I/O list is a binary or a (possibly deep) list of binaries or integers in the range 0..255.</p> @@ -127,8 +133,7 @@ <tcaption>Messages Received From a Port.</tcaption> </table> <p>Instead of sending and receiving messages, there are also a - number of BIFs that can be used. These can be called by any - process, not only the port owner.</p> + number of BIFs that can be used.</p> <table> <row> <cell align="left" valign="middle"><c>port_command(Port,Data)</c></cell> |
