diff options
106 files changed, 2233 insertions, 1817 deletions
diff --git a/.gitignore b/.gitignore index b391ecbe57..d40f49b56f 100644 --- a/.gitignore +++ b/.gitignore @@ -98,6 +98,8 @@ lib/wx/c_src/win32/ lib/wx/priv/win32/ lib/wx/win32/ make/win32/ +make/otp_built +make/otp_doc_built # OSE *.d diff --git a/HOWTO/INSTALL-CROSS.md b/HOWTO/INSTALL-CROSS.md index 10f463c06d..af4b38f292 100644 --- a/HOWTO/INSTALL-CROSS.md +++ b/HOWTO/INSTALL-CROSS.md @@ -563,12 +563,6 @@ under the License. %CopyrightEnd% -Modifying This Document ------------------------ - -Before modifying this document you need to have a look at the -[$ERL_TOP/HOWTO/MARKDOWN.md][] document. - [$ERL_TOP/HOWTO/INSTALL.md]: INSTALL.md @@ -576,6 +570,4 @@ Before modifying this document you need to have a look at the [How to Build the Documentation]: INSTALL.md#The-ErlangOTP-Documentation_How-to-Build-the-Documentation [cross configuration variables]: #Currently-Used-Configuration-Variables [DESTDIR]: http://www.gnu.org/prep/standards/html_node/DESTDIR.html - [$ERL_TOP/HOWTO/MARKDOWN.md]: MARKDOWN.md - [?TOC]: true diff --git a/HOWTO/INSTALL-WIN32.md b/HOWTO/INSTALL-WIN32.md index 94d3688f23..0387572dd3 100644 --- a/HOWTO/INSTALL-WIN32.md +++ b/HOWTO/INSTALL-WIN32.md @@ -1018,7 +1018,7 @@ Copyright and License %CopyrightBegin% -Copyright Ericsson AB 2003-2012. All Rights Reserved. +Copyright Ericsson AB 2003-2014. All Rights Reserved. The contents of this file are subject to the Erlang Public License, Version 1.1, (the "License"); you may not use this file except in @@ -1033,15 +1033,7 @@ under the License. %CopyrightEnd% -Modifying This Document ------------------------ - -Before modifying this document you need to have a look at the -[$ERL_TOP/HOWTO/MARKDOWN.md][] document. - - [1]: http://www.erlang.org/faq.html "mailing lists" - [$ERL_TOP/HOWTO/MARKDOWN.md]: MARKDOWN.md [?TOC]: true diff --git a/HOWTO/INSTALL.md b/HOWTO/INSTALL.md index 368947b36c..f1bb8ec853 100644 --- a/HOWTO/INSTALL.md +++ b/HOWTO/INSTALL.md @@ -4,174 +4,109 @@ Building and Installing Erlang/OTP Introduction ------------ -This document describes how to build and install Erlang/OTP-%OTP-REL%. You -are advised to read the whole document before attempting to build and install -Erlang/OTP. You can find more information about Open Source Erlang/OTP at: +This document describes how to build and install Erlang/OTP-%OTP-REL%. +Erlang/OTP should be possible to build from source on any Unix/Linux system, +including OS X. You are advised to read the whole document +before attempting to build and install Erlang/OTP. - <http://www.erlang.org/> +The source code can be downloaded from the official site of Erlang/OTP or GitHub. +* <http://www.erlang.org> +* <https://github.com/erlang/otp> -The source code for Erlang/OTP can also be found in a Git repository: - - <http://github.com/erlang/otp> - -Erlang/OTP should be possible to build from source on any Unix system, -including Mac OS X. This document describes how to native compile Erlang/OTP -on Unix. For detailed instructions on how to - -* cross compile Erlang/OTP, see the [$ERL_TOP/HOWTO/INSTALL-CROSS.md][] - document. - -* build Erlang/OTP on Windows, see the [$ERL_TOP/HOWTO/INSTALL-WIN32.md][] - document. +Required Utilities +------------------ - Binary releases for Windows can be found at - <http://www.erlang.org/download.html>. +These are the tools you need in order to unpack and build Erlang/OTP. -Before reading the above mentioned documents you are in any case advised to -read this document first, since it covers building Erlang/OTP in general as -well as other important information. +> *WARNING*: Please have a look at the [Known platform issues][] chapter +> before you start. -Daily Build and Test --------------------- -At Ericsson we have a "Daily Build and Test" that runs on: - -* Solaris 8, 9 - * Sparc32 - * Sparc64 -* Solaris 10 - * Sparc32 - * Sparc64 - * x86 -* SuSE Linux/GNU 9.4, 10.1 - * x86 -* SuSE Linux/GNU 10.0, 10.1, 11.0 - * x86 - * x86\_64 -* openSuSE 11.4 (Celadon) - * x86\_64 (valgrind) -* Fedora 7 - * PowerPC -* Fedora 14 - * x86\_64 -* Gentoo Linux/GNU 1.12.11.1 - * x86 -* Ubuntu Linux/GNU 7.04, 10.04, 10.10, 11.0 - * x86\_64 -* MontaVista Linux/GNU 4.0.1 - * PowerPC -* FreeBSD 8.2 - * x86 -* OpenBSD 5.0 - * x86\_64 -* Mac OS X 10.5.8 (Leopard), 10.7.3 (Lion), 10.9 (Mavericks) - * x86 -* Windows XP SP3, 2003, Vista, 7 - * x86 -* Windows 7 - * x86\_64 +### Unpacking ### -We also have the following "Daily Cross Builds": +* GNU unzip, or a modern uncompress. +* A TAR program that understands the GNU TAR format for long filenames. -* SuSE Linux/GNU 10.1 x86 -> SuSE Linux/GNU 10.1 x86\_64 -* SuSE Linux/GNU 10.1 x86\_64 -> Linux/GNU TILEPro64 +### Building ### -and the following "Daily Cross Build Tests": +* GNU `make` +* Compiler -- GNU C Compiler, `gcc` or the C compiler frontend for LLVM, `clang`. +* Perl 5 +* GNU `m4` -- If HiPE (native code) support is enabled. HiPE can be + disabled using `--disable-hipe` +* `ncurses`, `termcap`, or `termlib` -- The development headers and + libraries are needed, often known as `ncurses-devel`. Use + `--without-termcap` to build without any of these libraries. Note that + in this case only the old shell (without any line editing) can be used. +* `sed` -- Stream Editor for basic text transformation. -* SuSE Linux/GNU 10.1 x86\_64 +#### Building in Git #### -Versions Known NOT to Work --------------------------- +* GNU `autoconf` of at least version 2.59. Note that `autoconf` is not + needed when building an unmodified version of the released source. -* Suse linux 9.1 is shipped with a patched GCC version 3.3.3, having the - rpm named `gcc-3.3.3-41`. That version has a serious optimization bug - that makes it unusable for building the Erlang emulator. Please - upgrade GCC to a newer version before building on Suse 9.1. Suse Linux - Enterprise edition 9 (SLES9) has `gcc-3.3.3-43` and is not affected. +#### Building on OS X #### -* `gcc-4.3.0` has a serious optimizer bug. It produces an Erlang emulator - that will crash immediately. The bug is supposed to be fixed in - `gcc-4.3.1`. +* Xcode -- Download and install via the Mac App Store. + Read about [Building on a Mac][] before proceeding. -* FreeBSD had a bug which caused `kqueue`/`poll`/`select` to fail to detect - that a `writev()` on a pipe has been made. This bug should have been fixed - in FreeBSD 6.3 and FreeBSD 7.0. NetBSD and DragonFlyBSD probably have or - have had the same bug. More information can be found at: +### Installing ### - * <http://www.freebsd.org/cgi/cvsweb.cgi/src/sys/kern/sys_pipe.c> - * <http://lists.freebsd.org/pipermail/freebsd-arch/2007-September/006790.html> +* An `install` program that can take multiple file names. -* `getcwd()` on Solaris 9 can cause an emulator crash. If you have - async-threads enabled you can increase the stack size of the - async-threads as a temporary workaround. See the `+a` command-line - argument in the documentation of `erl(1)`. Without async-threads the - emulator is not as vulnerable to this bug, but if you hit it without - async-threads the only workaround available is to enable async-threads - and increase the stack size of the async-threads. Sun has however - released patches that fixes the issue: - > Problem Description: 6448300 large mnttab can cause stack overrun - > during Solaris 9 getcwd +Optional Utilities +------------------ - More information can be found at: +Some applications are automatically skipped if the dependencies aren't met. +Here is a list of utilities needed for those applications. You will +also find the utilities needed for building the documentation. - * <http://sunsolve.sun.com/search/document.do?assetkey=1-21-112874-40-1&searchclause=6448300> - * <http://sunsolve.sun.com/search/document.do?assetkey=1-21-114432-29-1&searchclause=6448300> +### Building ### -Required Utilities ------------------- +* OpenSSL -- The opensource toolkit for Secure Socket Layer + and Transport Layer Security. + Required for building the application `crypto`. + Further, `ssl` and `ssh` require a working crypto application and + will also be skipped if OpenSSL is missing. The `public_key` + application will available without `crypto`, but the functionality + will be very limited. + + The development package of OpenSSL including the header files are needed as well + as the binary command program `openssl`. At least version 0.9.8 of OpenSSL is required. + Read more and download from <http://www.openssl.org>. +* Oracle Java SE JDK -- The Java Development Kit (Standard Edition). + Required for building the application `jinterface` and parts of `ic` and `orber`. + At least version 1.5.0 of the JDK is required. + + Download from <http://www.oracle.com/technetwork/java/javase/downloads>. + We have also tested with IBM's JDK 1.5.0. +* X Windows -- Development headers and libraries are needed + to build the Erlang/OTP application `gs` on Unix/Linux. +* `flex` -- Headers and libraries are needed to build the flex + scanner for the `megaco` application on Unix/Linux. +* wxWidgets -- Toolkit for GUI applications. + Required for building the `wx` application. At least + version 3.0 of wxWidgets is required. -These are the tools you will need in order to unpack and build Erlang/OTP. + Download from <http://sourceforge.net/projects/wxwindows/files/3.0.0/> + or get it from GitHub: <https://github.com/wxWidgets/wxWidgets> -### Unpacking ### + Further instructions on wxWidgets, read [Building with wxErlang][]. -* GNU unzip, or a modern uncompress. -* A TAR program that understands the GNU TAR format for long filenames - (such as GNU TAR). -### Building ### -* GNU `make` -* `gcc` -- GNU C compiler -* Perl 5 -* GNU `m4` -- If HiPE (native code) support is enabled. HiPE can be - disabled using `--disable-hipe` -* `ncurses`, `termcap`, or `termlib` -- The development headers and - libraries are needed, often known as `ncurses-devel`. Use - `--without-termcap` to build without any of these libraries. Note that - in this case only the old shell (without any line editing) can be used. -* OpenSSL -- Optional, but needed for building the Erlang/OTP applications - `ssl` and `crypto`. You need the "development package" of OpenSSL, i.e. - including the header files. For building the application `ssl` the OpenSSL - binary command program `openssl` is also needed. At least version 0.9.8 - of OpenSSL is required. Can be downloaded from <http://www.openssl.org>. -* Sun Java jdk-1.5.0 or higher -- Optional but needed for building the - Erlang/OTP application `jinterface` and parts of `ic` and `orber`. Can - be downloaded from <http://java.sun.com>. We have also tested IBM's - JDK 1.5.0. -* X Windows -- Optional, but development headers and libraries are needed - to build the Erlang/OTP application `gs` on Unix/Linux. -* `sed` -- There seem to be some problems with some of the `sed` version on - Solaris. Make sure `/bin/sed` or `/usr/bin/sed` is used on the Solaris - platform. -* `flex` -- Optional, headers and libraries are needed to build the `flex` - scanner for the `megaco` application on Unix/Linux. +### Building Documentation ### -#### Building Documentation #### +* `xsltproc` -- A command line XSLT processor. -* `xsltproc` -- XSLT processor. A tool for applying XSLT stylesheets - to XML documents. Can be downloaded from + A tool for applying XSLT stylesheets + to XML documents. Download xsltproc from <http://xmlsoft.org/XSLT/xsltproc2.html>. + * `fop` -- Apache FOP print formatter (requires Java). Can be downloaded from <http://xmlgraphics.apache.org/fop>. -#### Building in Git #### -* GNU `autoconf` of at least version 2.59. Note that `autoconf` is not - needed when building an unmodified version of the released source. - -### Installing ### - -* An `install` program that can take multiple file names. How to Build and Install Erlang/OTP ----------------------------------- @@ -186,65 +121,205 @@ section below before proceeding. ### Unpacking ### -Step 1: Start by unpacking the Erlang/OTP distribution file with your GNU +Start by unpacking the Erlang/OTP distribution file with your GNU compatible TAR program. - $ gunzip -c otp_src_%OTP-VSN%.tar.gz | tar xf - + $ tar -zxf otp_src_%OTP-VSN%.tar.gz # Assuming bash/sh -alternatively: - - $ zcat otp_src_%OTP-VSN%.tar.gz | tar xf - - - -Step 2: Now cd into the base directory (`$ERL_TOP`). +Now change directory into the base directory and set the `$ERL_TOP` variable. $ cd otp_src_%OTP-VSN% + $ export ERL_TOP=`pwd` # Assuming bash/sh ### Configuring ### -Step 3: On some platforms Perl may behave strangely if certain locales are -set, so optionally you may need to set the LANG variable: - - # Bourne shell - $ LANG=C; export LANG +Run the following commands to configure the build: -or + $ ./configure [ options ] - # C-Shell - $ setenv LANG C +> *NOTE*: If you are building Erlang/OTP from git you will need to run `./otp_build autoconf` to generate +> the configure scripts. -Step 4: Run the following commands to configure the build: +By default, Erlang/OTP release will be installed in `/usr/local/{bin,lib/erlang}`. +If you for instance don't have the permission to install in the standard location, + you can install Erlang/OTP somewhere else. For example, to install in +`/opt/erlang/%OTP-VSN%/{bin,lib/erlang}`, use the `--prefix=/opt/erlang/%OTP-VSN%` option. - $ ./configure [ options ] +On some platforms Perl may behave strangely if certain locales are +set. If you get errors when building, try setting the LANG variable: -If you are building it from git you will need to run `./otp_build autoconf` to generate configure file. -By default, Erlang/OTP will be installed in `/usr/local/{bin,lib/erlang}`. -To instead install in `<BaseDir>/{bin,lib/erlang}`, use the -`--prefix=<BaseDir>` option. + $ export LANG=C # Assuming bash/sh -If you upgraded the source with some patch you may need to clean up -from previous builds before the new build. Before doing a `make clean`, -be sure to read the [Pre-built Source Release][] section below. ### Building ### -Step 5: Build the Erlang/OTP package. +Build the Erlang/OTP release. $ make + +### Testing ### + +Before installation you should test whether your build is working properly +by running our smoke test. The smoke test is a subset of the complete Erlang/OTP test suites. +First you will need to build and release the test suites. + + $ make release_tests + +This creates an additional folder in `$ERL_TOP/release` called `tests`. +Now, it's time to start the smoke test. + + $ cd release/tests/test_server + $ $ERL_TOP/bin/erl -s ts install -s ts smoke_test batch -s init stop + +To verify that everything is ok you should open `$ERL_TOP/release/tests/test_server/index.html` +in your web browser and make sure that there are zero failed test cases. + +> *NOTE*: On builds without `crypto`, `ssl` and `ssh` there is a failed test case +> for undefined functions. Verify that the failed test case log only shows calls +> to skipped applications. + ### Installing ### -Step 6: Install then Erlang/OTP package +You are now ready to install the Erlang/OTP release! +The following command will install the release on your system. $ make install -### A Closer Look at the individual Steps ### -Let us go through them in some detail. +### Running ### + +You should now have a working release of Erlang/OTP! +Jump to [System Principles][] for instructions on running Erlang/OTP. + + +### How to Build the Documentation ### + +Make sure you're in the top directory in the source tree. + + $ cd $ERL_TOP + +If you have just built Erlang/OTP in the current source tree, you have +already ran `configure` and do not need to do this again; otherwise, run +`configure`. + + $ ./configure [Configure Args] + +When building the documentation you need a full Erlang/OTP-%OTP-VSN% system in +the `$PATH`. + + $ export PATH=$ERL_TOP/bin:$PATH # Assuming bash/sh + +Build the documentation. -#### Configuring #### + $ make docs + +#### Build Issues #### + +We have sometimes experienced problems with Oracle's `java` running out of +memory when running `fop`. Increasing the amount of memory available +as follows has in our case solved the problem. + + $ export FOP_OPTS="-Xmx<Installed amount of RAM in MB>m" + +More information can be found at +* <http://xmlgraphics.apache.org/fop/0.95/running.html#memory>. + + +### How to Install the Documentation ### + +The documentation can be installed either using the `install-docs` target, +or using the `release_docs` target. + +* If you have installed Erlang/OTP using the `install` target, install + the documentation using the `install-docs` target. Install locations + determined by `configure` will be used. `$DESTDIR` can be used the + same way as when doing `make install`. + + $ make install-docs + +* If you have installed Erlang/OTP using the `release` target, install + the documentation using the `release_docs` target. You typically want + to use the same `RELEASE_ROOT` as when invoking `make release`. + + $ make release_docs RELEASE_ROOT=<release dir> + + +### Accessing the Documentation ### + +After installation you can access the documentation by + +* Reading man pages. Make sure that `erl` is referring to the + installed version. For example `/usr/local/bin/erl`. + Try viewing at the man page for Mnesia + + $ erl -man mnesia + +* Browsing the html pages by loading the page `/usr/local/lib/erlang/doc/erlang/index.html` + or `<BaseDir>/lib/erlang/doc/erlang/index.html` if the prefix option has been used. + + +### How to Install the Pre-formatted Documentation ### + +Pre-formatted [html documentation][] and [man pages][] can be downloaded from +* <http://www.erlang.org/download.html>. + +Extract the html archive in the installation directory. + + $ cd <ReleaseDir> + $ tar -zxf otp_html_%OTP-VSN%.tar.gz + +For `erl -man <page>` to work the Unix manual pages have to be +installed in the same way, i.e. + + $ cd <ReleaseDir> + $ tar -zxf otp_man_%OTP-VSN%.tar.gz + +Where `<ReleaseDir>` is + +* `<PrefixDir>/lib/erlang` if you have installed Erlang/OTP using + `make install`. +* `$DESTDIR<PrefixDir>/lib/erlang` if you have installed Erlang/OTP + using `make install DESTDIR=<TmpInstallDir>`. +* `RELEASE_ROOT` if you have installed using + `make release RELEASE_ROOT=<ReleaseDir>`. + + +Advanced configuration and build of Erlang/OTP +---------------------------------------------- + +If you want to tailor your Erlang/OTP build and installation, please read +on for detailed information about the individual steps. + +### make and $ERL\_TOP ### + +All the makefiles in the entire directory tree use the environment +variable `ERL_TOP` to find the absolute path of the installation. The +`configure` script will figure this out and set it in the top level +Makefile (which, when building, it will pass on). However, when +developing it is sometimes convenient to be able to run make in a +subdirectory. To do this you must set the `ERL_TOP` variable +before you run make. -Step 4 runs a configuration script created by the GNU autoconf utility, which +For example, assume your GNU make program is called `make` and you +want to rebuild the application `STDLIB`, then you could do: + + $ cd lib/stdlib; env ERL_TOP=<Dir> make + +where `<Dir>` would be what you find `ERL_TOP` is set to in the top level +Makefile. + +### otp\_build vs configure/make ### + +Building Erlang/OTP can be done either by using the `$ERL_TOP/otp_build` +script, or by invoking `$ERL_TOP/configure` and `make` directly. Building using +`otp_build` is easier since it involves fewer steps, but the `otp_build` build +procedure is not as flexible as the `configure`/`make` build procedure. The binary +releases for Windows that we deliver are built using `otp_build`. + +### Configuring ### + +The configure script is created by the GNU autoconf utility, which checks for system specific features and then creates a number of makefiles. The configure script allows you to customize a number of parameters; @@ -260,10 +335,10 @@ use the `--prefix` argument like this: `./configure --prefix=<Dir>`. Some of the available `configure` options are: * `--prefix=PATH` - Specify installation prefix. -* `--{enable,disable}-threads` - Thread support (enabled by default if - possible) + +* `--{enable,disable}-threads` - Thread support. This is enabled by default if possible. * `--{enable,disable}-smp-support` - SMP support (enabled by default if - possible) + a usable POSIX thread library or native Windows threads is found) * `--{enable,disable}-kernel-poll` - Kernel poll support (enabled by default if possible) * `--{enable,disable}-hipe` - HiPE support (enabled by default on supported @@ -321,9 +396,9 @@ Some of the available `configure` options are: to configure. * `--without-$app` - By default all applications in Erlang/OTP will be included in a release. If this is not wanted it is possible to specify that Erlang/OTP - should be compiled without that applications, i.e. `--without-wx`. There is - no automatic dependency handling inbetween applications. So if you disable - an application that another depends on, you also have to disable the + should be compiled without one or more applications, i.e. `--without-wx`. There is + no automatic dependency handling between applications. If you disable + an application that another application depends on, you also have to disable the dependant application. * `--enable-dirty-schedulers` - Enable the **experimental** dirty schedulers functionality. Note that the dirty schedulers functionality is experimental, @@ -334,26 +409,133 @@ Some of the available `configure` options are: If you or your system has special requirements please read the `Makefile` for additional configuration information. -#### Building #### -Step 5 builds the Erlang/OTP system. On a fast computer, this will take about -5 minutes. After completion of this step, you should have a working -Erlang/OTP system which you can try by typing `bin/erl`. This should start -up Erlang/OTP and give you a prompt: +### Building ### + +Building Erlang/OTP on a relatively fast computer takes approximately +5 minutes. To speed it up, you can utilize parallel make with the `-j<num_jobs>` option. + + $ export MAKEFLAGS=-j8 # Assuming bash/sh + $ make + +If you've upgraded the source with a patch you may need to clean up from previous +builds before the new build. +Make sure to read the [Pre-built Source Release][] section below before doing a `make clean`. + +#### Within Git #### + +When building in a Git working directory you also have to have a GNU `autoconf` +of at least version 2.59 on your system, because you need to generate the +`configure` scripts before you can start building. + +The `configure` scripts are generated by invoking `./otp_build autoconf` in +the `$ERL_TOP` directory. The `configure` scripts also have to be regenerated +when a `configure.in` or `aclocal.m4` file has been modified. Note that when +checking out a branch a `configure.in` or `aclocal.m4` file may change +content, and you may therefore have to regenerate the `configure` scripts +when checking out a branch. Regenerated `configure` scripts imply that you +have to run `configure` and build again. + +> *NOTE*: Running `./otp_build autoconf` is **not** needed when building +> an unmodified version of the released source. + +Other useful information can be found at our GitHub wiki: +* <http://wiki.github.com/erlang/otp> + +#### OS X (Darwin) #### + +Make sure that the command `hostname` returns a valid fully qualified host +name (this is configured in `/etc/hostconfig`). Otherwise you might experience +problems when running distributed systems. + +If you develop linked-in drivers (shared library) you need to link using +`gcc` and the flags `-bundle -flat_namespace -undefined suppress`. You also +include `-fno-common` in `CFLAGS` when compiling. Use `.so` as the library +suffix. + +If you have Xcode 4.3, or later, you will also need to download +"Command Line Tools" via the Downloads preference pane in Xcode. + +#### Building with wxErlang #### + +Be aware that the wxWidgets-3.0 is a new release of wxWidgets, it is not as matured +as the old releases and the OS X port still lags behind the other ports. + +Configure and build wxWidgets (on Mavericks - 10.9): + + $ ./configure --with-cocoa --prefix=/usr/local + or without support for old versions and with static libs + $ ./configure --with-cocoa --prefix=/usr/local --with-macosx-version-min=10.9 --disable-shared + $ make + $ sudo make install + $ export PATH=/usr/local/bin:$PATH + +Check that you got the correct wx-config + + $ which wx-config && wx-config --version-full + +Build Erlang/OTP + + $ export PATH=/usr/local/bin:$PATH + $ cd $ERL_TOP + $ ./configure --enable-shared-zlib + $ make + $ sudo make install + + +#### Pre-built Source Release #### + +The source release is delivered with a lot of platform independent +build results already pre-built. If you want to remove these pre-built +files, invoke `./otp_build remove_prebuilt_files` from the `$ERL_TOP` +directory. After you have done this, you can build exactly the same way +as before, but the build process will take a much longer time. + +> *WARNING*: Doing `make clean` in an arbitrary directory of the source +> tree, may remove files needed for bootstrapping the build. +> +> Doing `./otp_build save_bootstrap` from the `$ERL_TOP` directory before +> doing `make clean` will ensure that it will be possible to build after +> doing `make clean`. `./otp_build save_bootstrap` will be invoked +> automatically when `make` is invoked from `$ERL_TOP` with either the +> `clean` target, or the default target. It is also automatically invoked +> if `./otp_build remove_prebuilt_files` is invoked. + +#### How to Build a Debug Enabled Erlang RunTime System #### + +After completing all the normal building steps described above a debug +enabled runtime system can be built. To do this you have to change +directory to `$ERL_TOP/erts/emulator`. + +In this directory execute: + + $ make debug FLAVOR=$FLAVOR + +where `$FLAVOR` is either `plain` or `smp`. The flavor options will +produce a beam.debug and beam.smp.debug executable respectively. The +files are installed along side with the normal (opt) versions `beam.smp` +and `beam`. + +To start the debug enabled runtime system execute: + + $ $ERL_TOP/bin/cerl -debug - $ bin/erl - Erlang/OTP %OTP-REL% [erts-%ERTS-VSN%] [source] [smp:4:4] [async-threads:0] [kernel-poll:false] +The debug enabled runtime system features lock violation checking, +assert checking and various sanity checks to help a developer ensure +correctness. Some of these features can be enabled on a normal beam +using appropriate configure options. - Eshell V%ERTS-VSN% (abort with ^G) - 1> _ +There are other types of runtime systems that can be built as well +using the similar steps just described. -#### Installing #### + $ make $TYPE FLAVOR=$FLAVOR + +where `$TYPE` is `opt`, `gcov`, `gprof`, `debug`, `valgrind`, or `lcnt`. +These different beam types are useful for debugging and profiling +purposes. -Step 6 is optional. It installs Erlang/OTP at a standardized location (if you -change your mind about where you wish to install you can rerun step 4, -without having to do step 5 again). -##### Alternative Installation Procedures ##### +### Installing ### * Staged install using [DESTDIR][]. You can perform the install phase in a temporary directory and later move the installation into @@ -435,7 +617,7 @@ without having to do step 5 again). if you want to try the system out, running test suites, etc, before doing the real install without `EXTRA_PREFIX`. -### Symbolic Links in --bindir ### +#### Symbolic Links in --bindir #### When doing `make install` and the default installation prefix is used, relative symbolic links will be created from `/usr/local/bin` to all public @@ -449,156 +631,10 @@ passed to `configure`. One can force relative, or absolute links by passing phase. Note that such a request might cause a failure if the request cannot be satisfied. -### Pre-built Source Release ### - -The source release is delivered with a lot of platform independent -build results already pre-built. If you want to remove these pre-built -files, invoke `./otp_build remove_prebuilt_files` from the `$ERL_TOP` -directory. After you have done this, you can build exactly the same way -as before, but the build process will take a much longer time. - -> *WARNING*: Doing `make clean` in an arbitrary directory of the source -> tree, may remove files needed for bootstrapping the build. -> -> Doing `./otp_build save_bootstrap` from the `$ERL_TOP` directory before -> doing `make clean` will ensure that it will be possible to build after -> doing `make clean`. `./otp_build save_bootstrap` will be invoked -> automatically when `make` is invoked from `$ERL_TOP` with either the -> `clean` target, or the default target. It is also automatically invoked -> if `./otp_build remove_prebuilt_files` is invoked. - -### Building in Git ### - -When building in a Git working directory you also have to have a GNU `autoconf` -of at least version 2.59 on your system, because you need to generate the -`configure` scripts before you can start building. - -The `configure` scripts are generated by invoking `./otp_build autoconf` in -the `$ERL_TOP` directory. The `configure` scripts also have to be regenerated -when a `configure.in` or `aclocal.m4` file has been modified. Note that when -checking out a branch a `configure.in` or `aclocal.m4` file may change -content, and you may therefore have to regenerate the `configure` scripts -when checking out a branch. Regenerated `configure` scripts imply that you -have to run `configure` and build again. - -> *NOTE*: Running `./otp_build autoconf` is **not** needed when building -> an unmodified version of the released source. - -Other useful information can be found at our github wiki: -<http://wiki.github.com/erlang/otp> - -### make and $ERL\_TOP ### - -All the makefiles in the entire directory tree use the environment -variable `ERL_TOP` to find the absolute path of the installation. The -`configure` script will figure this out and set it in the top level -Makefile (which, when building, it will pass on). However, when -developing it is sometimes convenient to be able to run make in a -subdirectory. To do this you must set the `ERL_TOP` variable -before you run make. - -For example, assume your GNU make program is called `make` and you -want to rebuild the application `STDLIB`, then you could do: - - $ cd lib/stdlib; env ERL_TOP=<Dir> make - -where `<Dir>` would be what you find `ERL_TOP` is set to in the top level -Makefile. - -The Erlang/OTP Documentation ----------------------------- - -### How to Build the Documentation ### - -Before you can build the documentation you need to either [native build][] -or [cross build][] the Erlang/OTP system. After this you can build the -documentation as follows. - - $ cd $ERL_TOP - $ make docs - -The documentation can be installed either using the `install-docs` target, -or using the `release_docs` target. - -* If you have installed Erlang/OTP using the `install` target, install - the documentation using the `install-docs` target. Install locations - determined by `configure` will be used. `$DESTDIR` can be used the - same way as when doing `make install`. - - $ make install-docs - -* If you have installed Erlang/OTP using the `release` target, install - the documentation using the `release_docs` target. You typically want - to use the same `RELEASE_ROOT` as when invoking `make release`. - - $ make release_docs RELEASE_ROOT=<release dir> - -#### Build Issues #### - -We have sometimes experienced problems with Sun's `java` running out of -memory when running `fop`. Increasing the amount of memory available -as follows has in our case solved the problem. - - $ export FOP_OPTS="-Xmx<Installed amount of RAM in MB>m" - -More information can be found at -<http://xmlgraphics.apache.org/fop/0.95/running.html#memory>. - -### How to Install the Pre-formatted Documentation ### - -Pre-formatted [html documentation][] and [man pages][] can be downloaded at -<http://www.erlang.org/download.html>. - -For some graphical tools to find the on-line help you have to install -the HTML documentation on top of the installed OTP applications, i.e. - - $ cd <ReleaseDir> - $ gunzip -c otp_html_%OTP-VSN%.tar.gz | tar xf - - -For `erl -man <page>` to work the Unix manual pages have to be -installed in the same way, i.e. - - $ cd <ReleaseDir> - $ gunzip -c otp_man_%OTP-VSN%.tar.gz | tar xf - - -Where `<ReleaseDir>` is - -* `<PrefixDir>/lib/erlang` if you have installed Erlang/OTP using - `make install`. -* `$DESTDIR<PrefixDir>/lib/erlang` if you have installed Erlang/OTP - using `make install DESTDIR=<TmpInstallDir>`. -* `RELEASE_ROOT` if you have installed using - `make release RELEASE_ROOT=<ReleaseDir>`. - -Support for SMP (Symmetric Multi Processing) --------------------------------------------- - -An emulator with SMP support will be built by default on most platforms -if a usable POSIX thread library or native Windows threads is found. -You can force building of an SMP emulator, by using -`./configure --enable-smp-support`. However, if configure does not -automatically enable SMP support, the build is very likely to fail. +### Running ### -Use `./configure --disable-smp-support` if you for some reason do not -want to have the emulator with SMP support built. - -If SMP support is enabled, support for threaded I/O will also be turned on -(also in the emulator without SMP support). - -The `erl` command will automatically start the SMP emulator if the -computer has more than one logical processor. You can force a start -of the emulator with SMP support by passing `-smp enable` as -command line arguments to erl, and you can force a start of the -emulator without SMP support by passing `-smp disable`. - -GS (Graphic System) -------------------- - -GS now Tcl/Tk 8.4. It will be searched for when starting GS. - -Using HiPE ----------- +#### Using HiPE #### HiPE supports the following system configurations: @@ -619,12 +655,12 @@ HiPE supports the following system configurations: * FreeBSD: FreeBSD 6.1 and 6.2 in 32-bit and 64-bit modes should work. - * MacOSX/Darwin: Darwin 9.8.0 in 32-bit mode should work. + * OS X/Darwin: Darwin 9.8.0 in 32-bit mode should work. * PowerPC: All 32-bit 6xx/7xx(G3)/74xx(G4) processors should work. 32-bit mode on 970 (G5) and POWER5 processors should work. - * Linux (Yellow Dog) and Mac OSX 10.4 are supported. + * Linux (Yellow Dog) and OS X 10.4 are supported. * SPARC: All UltraSPARC processors running 32-bit user code should work. @@ -643,11 +679,11 @@ HiPE is automatically enabled on the following systems: * x86 in 32-bit mode: Linux, Solaris, FreeBSD * x86 in 64-bit mode: Linux, Solaris, FreeBSD -* PowerPC: Linux, MacOSX +* PowerPC: Linux, Mac OSX * SPARC: Linux * ARM: Linux -On other supported systems you need to `./configure --enable-hipe`. +On other supported systems, see [Advanced Configure][] on how to enable HiPE. If you are running on a platform supporting HiPE and if you have not disabled HiPE, you can compile a module into native code like this from the Erlang @@ -659,7 +695,7 @@ or 1> c(Module, [native|OtherOptions]). -Using the erlc program, write like this: +Using the erlc program, write like this $ erlc +native Module.erl @@ -674,88 +710,103 @@ Use `hipe:help_options/0` to print out the available options. 1> hipe:help_options(). -Mac OS X (Darwin) ------------------ - -Make sure that the command `hostname` returns a valid fully qualified host -name (this is configured in `/etc/hostconfig`). - -If you develop linked-in drivers (shared library) you need to link using -`gcc` and the flags `-bundle -flat_namespace -undefined suppress`. You also -include `-fno-common` in `CFLAGS` when compiling. Use `.so` as the library -suffix. - -Install `Xcode` from the `AppStore` if it is not already installed. - -If you have Xcode 4.3, or later, you will also need to download -"Command Line Tools" via the Downloads preference pane in Xcode. - -### Building with wxErlang ### +#### Running with GS #### -If you want to build the `wx` application, you will need to get wxWidgets-3.0 (or later) -(`wxWidgets-3.0.0.tar.bz2` from <http://sourceforge.net/projects/wxwindows/files/3.0.0/>) -or get it from github: - $ git clone [email protected]:wxWidgets/wxWidgets.git +The `gs` application requires the GUI toolkit Tcl/Tk to run. At least +version 8.4 is required. -Be aware that the wxWidgets-3.0 is a new release of wxWidgets, it is not as matured -as the old releases and the MacOsX port still lags behind the other ports. - -Configure and build wxWidgets (on Mavericks - 10.9): - $ ./configure --with-cocoa --prefix=/usr/local - or without support for old versions and with static libs - $ ./configure --with-cocoa --prefix=/usr/local --with-macosx-version-min=10.9 --disable-shared - $ make - $ sudo make install - $ export PATH=/usr/local/bin:$PATH +Known platform issues +--------------------- -Check that you got the correct wx-config +* Suse linux 9.1 is shipped with a patched GCC version 3.3.3, having the + rpm named `gcc-3.3.3-41`. That version has a serious optimization bug + that makes it unusable for building the Erlang emulator. Please + upgrade GCC to a newer version before building on Suse 9.1. Suse Linux + Enterprise edition 9 (SLES9) has `gcc-3.3.3-43` and is not affected. - $ which wx-config +* `gcc-4.3.0` has a serious optimizer bug. It produces an Erlang emulator + that will crash immediately. The bug is supposed to be fixed in + `gcc-4.3.1`. -### Finish up ### +* FreeBSD had a bug which caused `kqueue`/`poll`/`select` to fail to detect + that a `writev()` on a pipe has been made. This bug should have been fixed + in FreeBSD 6.3 and FreeBSD 7.0. NetBSD and DragonFlyBSD probably have or + have had the same bug. More information can be found at: -Build Erlang + * <http://www.freebsd.org/cgi/cvsweb.cgi/src/sys/kern/sys_pipe.c> + * <http://lists.freebsd.org/pipermail/freebsd-arch/2007-September/006790.html> - $ export PATH=/usr/local/bin:$PATH - $ cd $ERL_TOP - $ ./configure --enable-shared-zlib - $ make - $ sudo make install +* `getcwd()` on Solaris 9 can cause an emulator crash. If you have + async-threads enabled you can increase the stack size of the + async-threads as a temporary workaround. See the `+a` command-line + argument in the documentation of `erl(1)`. Without async-threads the + emulator is not as vulnerable to this bug, but if you hit it without + async-threads the only workaround available is to enable async-threads + and increase the stack size of the async-threads. Oracle has however + released patches that fixes the issue: -How to Build a Debug Enabled Erlang RunTime System --------------------------------------------------- + > Problem Description: 6448300 large mnttab can cause stack overrun + > during Solaris 9 getcwd -After completing all the normal building steps described above a debug -enabled runtime system can be built. To do this you have to change -directory to `$ERL_TOP/erts/emulator`. + More information can be found at: + * <https://getupdates.oracle.com/readme/112874-40> + * <https://getupdates.oracle.com/readme/114432-29> -In this directory execute: +* `sed` on Solaris seem to have some problems. For example on + Solaris 8, the BSD `sed` and XPG4 `sed` should be avoided. + Make sure `/bin/sed` or `/usr/bin/sed` is used on the Solaris + platform. - $ make debug FLAVOR=$FLAVOR -where `$FLAVOR` is either `plain` or `smp`. The flavor options will -produce a beam.debug and beam.smp.debug executable respectively. The -files are installed along side with the normal (opt) versions `beam.smp` -and `beam`. +Daily Build and Test +-------------------- +At Ericsson we have a "Daily Build and Test" that runs on: -To start the debug enabled runtime system execute: +* Solaris 8, 9 + * Sparc32 + * Sparc64 +* Solaris 10 + * Sparc32 + * Sparc64 + * x86 +* SuSE Linux/GNU 9.4, 10.1 + * x86 +* SuSE Linux/GNU 10.0, 10.1, 11.0 + * x86 + * x86\_64 +* openSuSE 11.4 (Celadon) + * x86\_64 (valgrind) +* Fedora 7 + * PowerPC +* Fedora 16 + * x86\_64 +* Gentoo Linux/GNU 1.12.11.1 + * x86 +* Ubuntu Linux/GNU 7.04, 10.04, 10.10, 11.04, 12.04 + * x86\_64 +* MontaVista Linux/GNU 4.0.1 + * PowerPC +* FreeBSD 10.0 + * x86 +* OpenBSD 5.4 + * x86\_64 +* OS X 10.5.8 (Leopard), 10.7.5 (Lion), 10.9.1 (Mavericks) + * x86 +* Windows XP SP3, 2003, Vista, 7 + * x86 +* Windows 7 + * x86\_64 - $ $ERL_TOP/bin/cerl -debug +We also have the following "Daily Cross Builds": -The debug enabled runtime system features lock violation checking, -assert checking and various sanity checks to help a developer ensure -correctness. Some of these features can be enabled on a normal beam -using appropriate configure options. +* SuSE Linux/GNU 10.1 x86 -> SuSE Linux/GNU 10.1 x86\_64 +* SuSE Linux/GNU 10.1 x86\_64 -> Linux/GNU TILEPro64 -There are other types of runtime systems that can be built as well -using the similar steps just described. +and the following "Daily Cross Build Tests": - $ make $TYPE FLAVOR=$FLAVOR +* SuSE Linux/GNU 10.1 x86\_64 -where `$TYPE` is `opt`, `gcov`, `gprof`, `debug`, `valgrind`, or `lcnt`. -These different beam types are useful for debugging and profiling -purposes. Authors ------- @@ -764,6 +815,7 @@ Authors are mostly listed in the application's `AUTHORS` files, that is `$ERL_TOP/lib/*/AUTHORS` and `$ERL_TOP/erts/AUTHORS`, not in the individual source files. + Copyright and License --------------------- @@ -784,30 +836,25 @@ under the License. %CopyrightEnd% -More Information ----------------- - -More information can be found at <http://www.erlang.org>. -Modifying This Document ------------------------ - -Before modifying this document you need to have a look at the -[$ERL_TOP/HOWTO/MARKDOWN.md][] document. [$ERL_TOP/HOWTO/INSTALL-CROSS.md]: INSTALL-CROSS.md [$ERL_TOP/HOWTO/INSTALL-WIN32.md]: INSTALL-WIN32.md [DESTDIR]: http://www.gnu.org/prep/standards/html_node/DESTDIR.html - [Building in Git]: #How-to-Build-and-Install-ErlangOTP_Building-in-Git - [Pre-built Source Release]: #How-to-Build-and-Install-ErlangOTP_Prebuilt-Source-Release - [make and $ERL_TOP]: #How-to-Build-and-Install-ErlangOTP_make-and-ERLTOP + [Building in Git]: #Advanced-configuration-and-build-of-ErlangOTP_Building_Within-Git + [Advanced Configure]: #Advanced-configuration-and-build-of-ErlangOTP_Configuring + [Pre-built Source Release]: #Advanced-configuration-and-build-of-ErlangOTP_Building_Prebuilt-Source-Release + [make and $ERL_TOP]: #Advanced-configuration-and-build-of-ErlangOTP_make-and-ERLTOP [html documentation]: http://www.erlang.org/download/otp_doc_html_%OTP-VSN%.tar.gz [man pages]: http://www.erlang.org/download/otp_doc_man_%OTP-VSN%.tar.gz [the released source tar ball]: http://www.erlang.org/download/otp_src_%OTP-VSN%.tar.gz + [System Principles]: ../system_principles/system_principles + [Known platform issues]: #Known-platform-issues [native build]: #How-to-Build-and-Install-ErlangOTP [cross build]: INSTALL-CROSS.md - [$ERL_TOP/HOWTO/MARKDOWN.md]: MARKDOWN.md - - [?TOC]: true + [Required Utilities]: #Required-Utilities + [Optional Utilities]: #Optional-Utilities + [Building on a Mac]: #Advanced-configuration-and-build-of-ErlangOTP_Building_OS-X-Darwin + [Building with wxErlang]: #Advanced-configuration-and-build-of-ErlangOTP_Building_Building-with-wxErlang diff --git a/Makefile.in b/Makefile.in index c667eb5f79..4dc5ebac40 100644 --- a/Makefile.in +++ b/Makefile.in @@ -422,6 +422,10 @@ endif cd $(ERL_TOP)/system/doc && \ PATH=$(BOOT_PREFIX)"$${PATH}" \ ERL_TOP=$(ERL_TOP) $(MAKE) TESTROOT="$(RELEASE_ROOT)" DOCGEN=$(BOOTSTRAP_ROOT)/bootstrap/lib/erl_docgen $@ +ifneq ($(OTP_SMALL_BUILD),true) + echo "OTP doc built" > $(ERL_TOP)/make/otp_doc_built +endif + mod2app: PATH=$(BOOT_PREFIX)"$${PATH}" escript $(BOOTSTRAP_ROOT)/bootstrap/lib/erl_docgen/priv/bin/xref_mod_app.escript -topdir $(ERL_TOP) -outfile $(ERL_TOP)/make/$(TARGET)/mod2app.xml @@ -447,6 +451,7 @@ else $(make_verbose)cd lib && \ ERL_TOP=$(ERL_TOP) PATH=$(BOOT_PREFIX)"$${PATH}" \ $(MAKE) opt BUILD_ALL=true + echo "OTP built" > $(ERL_TOP)/make/otp_built endif kernel: $(make_verbose)cd lib/kernel && \ @@ -12,6 +12,15 @@ middle-ware to develop these systems. It includes its own distributed database, applications to interface towards other languages, debugging and release handling tools. +ERTS and BEAM +------------- +**BEAM** is the name of the virtual machine where all Erlang code is executed. +Every compiled Erlang file has the suffix .beam. The virtual machine +is sometimes referred to as the emulator. + +**ERTS** is the Erlang Runtime System where the BEAM, kernel and +standard libraries amongst others are included. + More information can be found at [erlang.org] [1]. Building and Installing @@ -57,7 +66,7 @@ Copyright and License > %CopyrightBegin% > -> Copyright Ericsson AB 2010-2012. All Rights Reserved. +> Copyright Ericsson AB 2010-2014. All Rights Reserved. > > The contents of this file are subject to the Erlang Public License, > Version 1.1, (the "License"); you may not use this file except in diff --git a/erts/doc/src/erlang.xml b/erts/doc/src/erlang.xml index 7aaded200c..e7e9b218f2 100644 --- a/erts/doc/src/erlang.xml +++ b/erts/doc/src/erlang.xml @@ -1784,6 +1784,15 @@ os_prompt% </pre> </desc> </func> <func> + <name name="is_map" arity="1"/> + <fsummary>Check whether a term is a map</fsummary> + <desc> + <p>Returns <c>true</c> if <c><anno>Term</anno></c> is a map; + otherwise returns <c>false</c>.</p> + <p>Allowed in guard tests.</p> + </desc> + </func> + <func> <name name="is_number" arity="1"/> <fsummary>Check whether a term is a number</fsummary> <desc> @@ -2220,6 +2229,17 @@ os_prompt% </pre> </desc> </func> <func> + <name name="map_size" arity="1"/> + <fsummary>Return the size of a map</fsummary> + <desc> + <p>Returns an integer which is the number of key-value pairs in <c><anno>Map</anno></c>.</p> + <pre> +> <input>map_size(#{a=>1, b=>2, c=>3}).</input> +3</pre> + <p>Allowed in guard tests.</p> + </desc> + </func> + <func> <name name="max" arity="2"/> <fsummary>Return the largest of two term</fsummary> <desc> diff --git a/erts/emulator/Makefile.in b/erts/emulator/Makefile.in index 58639c7190..7145824f91 100644 --- a/erts/emulator/Makefile.in +++ b/erts/emulator/Makefile.in @@ -1039,7 +1039,6 @@ $(BINDIR)/$(EMULATOR_EXECUTABLE): $(INIT_OBJS) $(OBJS) $(DEPLIBS) $(LCF) else $(BINDIR)/$(EMULATOR_EXECUTABLE): $(INIT_OBJS) $(OBJS) $(DEPLIBS) - echo $(DEPLIBS) $(ld_verbose)$(PURIFY) $(LD) -o $(BINDIR)/$(EMULATOR_EXECUTABLE) \ $(HIPEBEAMLDFLAGS) $(LDFLAGS) $(DEXPORT) $(INIT_OBJS) $(OBJS) \ $(STATIC_NIF_LIBS) $(STATIC_DRIVER_LIBS) $(LIBS) diff --git a/erts/emulator/beam/beam_emu.c b/erts/emulator/beam/beam_emu.c index 9634faff1d..1026e5f649 100644 --- a/erts/emulator/beam/beam_emu.c +++ b/erts/emulator/beam/beam_emu.c @@ -3525,7 +3525,7 @@ get_map_elements_fail: erts_post_nif(&env); #ifdef ERTS_DIRTY_SCHEDULERS if (is_non_value(nif_bif_result) && c_p->freason == TRAP) { - Export* ep = (Export*) c_p->psd->data[ERTS_PSD_DIRTY_SCHED_TRAP_EXPORT]; + Export* ep = ERTS_PROC_GET_DIRTY_SCHED_TRAP_EXPORT(c_p); ep->code[0] = I[-3]; ep->code[1] = I[-2]; } diff --git a/erts/emulator/beam/erl_nif.c b/erts/emulator/beam/erl_nif.c index 063dba056e..ff551ea3af 100644 --- a/erts/emulator/beam/erl_nif.c +++ b/erts/emulator/beam/erl_nif.c @@ -1515,26 +1515,35 @@ int enif_consume_timeslice(ErlNifEnv* env, int percent) #ifdef ERTS_DIRTY_SCHEDULERS +/* NIFs exports need one more item than the Export struct provides, the + * erl_module_nif*, so the DirtyNifExport below adds that. The Export + * member must be first in the struct. + */ +typedef struct { + Export exp; + struct erl_module_nif* m; +} DirtyNifExport; + static void -alloc_proc_psd(Process* proc, Export **ep) +alloc_proc_psd(Process* proc, DirtyNifExport **ep) { int i; if (!*ep) { - *ep = erts_alloc(ERTS_ALC_T_PSD, sizeof(Export)); - sys_memset((void*) *ep, 0, sizeof(Export)); + *ep = erts_alloc(ERTS_ALC_T_PSD, sizeof(DirtyNifExport)); + sys_memset((void*) *ep, 0, sizeof(DirtyNifExport)); for (i=0; i<ERTS_NUM_CODE_IX; i++) { - (*ep)->addressv[i] = &(*ep)->code[3]; + (*ep)->exp.addressv[i] = &(*ep)->exp.code[3]; } - (*ep)->code[3] = (BeamInstr) em_call_nif; + (*ep)->exp.code[3] = (BeamInstr) em_call_nif; } - (void) ERTS_PROC_SET_DIRTY_SCHED_TRAP_EXPORT(proc, ERTS_PROC_LOCK_MAIN, *ep); + (void) ERTS_PROC_SET_DIRTY_SCHED_TRAP_EXPORT(proc, ERTS_PROC_LOCK_MAIN, &(*ep)->exp); } static ERL_NIF_TERM execute_dirty_nif_finalizer(ErlNifEnv* env, int argc, const ERL_NIF_TERM argv[]) { Eterm* reg = ERTS_PROC_GET_SCHDATA(env->proc)->x_reg_array; - ERL_NIF_TERM result = (ERL_NIF_TERM) reg[0]; + ERL_NIF_TERM result, dirty_result = (ERL_NIF_TERM) reg[0]; typedef ERL_NIF_TERM (*FinalizerFP)(ErlNifEnv*, ERL_NIF_TERM); FinalizerFP fp; #if HAVE_INT64 && SIZEOF_LONG != 8 @@ -1544,7 +1553,11 @@ execute_dirty_nif_finalizer(ErlNifEnv* env, int argc, const ERL_NIF_TERM argv[]) ASSERT(sizeof(fp) <= sizeof(unsigned long)); enif_get_ulong(env, reg[1], (unsigned long *) &fp); #endif - return (*fp)(env, result); + result = (*fp)(env, dirty_result); + if (erts_refc_dectest(&env->mod_nif->rt_dtor_cnt, 0) == 0 + && env->mod_nif->mod == NULL) + close_lib(env->mod_nif); + return result; } #endif /* ERTS_DIRTY_SCHEDULERS */ @@ -1560,7 +1573,7 @@ enif_schedule_dirty_nif(ErlNifEnv* env, int flags, erts_aint32_t state, n, a; Process* proc = env->proc; Eterm* reg = ERTS_PROC_GET_SCHDATA(proc)->x_reg_array; - Export* ep = NULL; + DirtyNifExport* ep = NULL; int i; int chkflgs = (flags & (ERL_NIF_DIRTY_JOB_IO_BOUND|ERL_NIF_DIRTY_JOB_CPU_BOUND)); @@ -1585,17 +1598,20 @@ enif_schedule_dirty_nif(ErlNifEnv* env, int flags, if (a == state) break; } - if (!(ep = ERTS_PROC_GET_DIRTY_SCHED_TRAP_EXPORT(proc))) + if (!(ep = (DirtyNifExport*) ERTS_PROC_GET_DIRTY_SCHED_TRAP_EXPORT(proc))) alloc_proc_psd(proc, &ep); ERTS_VBUMP_ALL_REDS(proc); - ep->code[2] = argc; + ep->exp.code[2] = argc; for (i = 0; i < argc; i++) { reg[i] = (Eterm) argv[i]; } - proc->i = (BeamInstr*) ep->addressv[0]; - ep->code[4] = (BeamInstr) fp; + proc->i = (BeamInstr*) ep->exp.addressv[0]; + ep->exp.code[4] = (BeamInstr) fp; + ep->m = env->mod_nif; proc->freason = TRAP; + erts_refc_inc(&env->mod_nif->rt_dtor_cnt, 1); + return THE_NON_VALUE; #else return (*fp)(env, argc, argv); @@ -1609,17 +1625,17 @@ enif_schedule_dirty_nif_finalizer(ErlNifEnv* env, ERL_NIF_TERM result, #ifdef USE_THREADS Process* proc = env->proc; Eterm* reg = ERTS_PROC_GET_SCHDATA(proc)->x_reg_array; - Export* ep; + DirtyNifExport* ep; erts_smp_atomic32_read_band_mb(&proc->state, ~(ERTS_PSFLG_DIRTY_CPU_PROC |ERTS_PSFLG_DIRTY_IO_PROC |ERTS_PSFLG_DIRTY_CPU_PROC_IN_Q |ERTS_PSFLG_DIRTY_IO_PROC_IN_Q)); - if (!(ep = ERTS_PROC_GET_DIRTY_SCHED_TRAP_EXPORT(proc))) + if (!(ep = (DirtyNifExport*) ERTS_PROC_GET_DIRTY_SCHED_TRAP_EXPORT(proc))) alloc_proc_psd(proc, &ep); ERTS_VBUMP_ALL_REDS(proc); - ep->code[2] = 2; + ep->exp.code[2] = 2; reg[0] = (Eterm) result; #if HAVE_INT64 && SIZEOF_LONG != 8 ASSERT(sizeof(fp) <= sizeof(ErlNifUInt64)); @@ -1628,8 +1644,8 @@ enif_schedule_dirty_nif_finalizer(ErlNifEnv* env, ERL_NIF_TERM result, ASSERT(sizeof(fp) <= sizeof(unsigned long)); reg[1] = (Eterm) enif_make_ulong(env, (unsigned long) fp); #endif - proc->i = (BeamInstr*) ep->addressv[0]; - ep->code[4] = (BeamInstr) execute_dirty_nif_finalizer; + proc->i = (BeamInstr*) ep->exp.addressv[0]; + ep->exp.code[4] = (BeamInstr) execute_dirty_nif_finalizer; proc->freason = TRAP; return THE_NON_VALUE; diff --git a/erts/emulator/beam/erl_process.c b/erts/emulator/beam/erl_process.c index 8bf0cc9491..b4b97d7df1 100644 --- a/erts/emulator/beam/erl_process.c +++ b/erts/emulator/beam/erl_process.c @@ -9207,7 +9207,6 @@ Process *schedule(Process *p, int calls) */ pick_next_process: { erts_aint32_t psflg_band_mask; - erts_aint32_t running_flag; int prio_q; int qmask; @@ -9269,12 +9268,6 @@ Process *schedule(Process *p, int calls) state = erts_smp_atomic32_read_nob(&p->state); } - - if (state & ERTS_PSFLG_ACTIVE_SYS) - running_flag = ERTS_PSFLG_RUNNING_SYS; - else - running_flag = ERTS_PSFLG_RUNNING; - while (1) { erts_aint32_t exp, new, tmp; tmp = new = exp = state; @@ -9284,8 +9277,12 @@ Process *schedule(Process *p, int calls) tmp = state & (ERTS_PSFLG_SUSPENDED | ERTS_PSFLG_PENDING_EXIT | ERTS_PSFLG_ACTIVE_SYS); - if (tmp != ERTS_PSFLG_SUSPENDED) - new |= running_flag; + if (tmp != ERTS_PSFLG_SUSPENDED) { + if (state & ERTS_PSFLG_ACTIVE_SYS) + new |= ERTS_PSFLG_RUNNING_SYS; + else + new |= ERTS_PSFLG_RUNNING; + } } state = erts_smp_atomic32_cmpxchg_relb(&p->state, new, exp); if (state == exp) { diff --git a/erts/emulator/test/binary_SUITE.erl b/erts/emulator/test/binary_SUITE.erl index 938aac6a0e..7aba367e33 100644 --- a/erts/emulator/test/binary_SUITE.erl +++ b/erts/emulator/test/binary_SUITE.erl @@ -64,7 +64,7 @@ -export([sleeper/0,trapping_loop/4]). suite() -> [{ct_hooks,[ts_install_cth]}, - {timetrap,{minutes,2}}]. + {timetrap,{minutes,4}}]. all() -> [copy_terms, conversions, deep_lists, deep_bitstr_lists, @@ -1266,7 +1266,7 @@ deep(Config) when is_list(Config) -> deep_roundtrip(T) -> B = term_to_binary(T), - T = binary_to_term_stress(B). + T = binary_to_term(B). obsolete_funs(Config) when is_list(Config) -> erts_debug:set_internal_state(available_internal_state, true), diff --git a/erts/emulator/test/match_spec_SUITE.erl b/erts/emulator/test/match_spec_SUITE.erl index 330bef7104..8038888796 100644 --- a/erts/emulator/test/match_spec_SUITE.erl +++ b/erts/emulator/test/match_spec_SUITE.erl @@ -42,7 +42,7 @@ -export([init_per_testcase/2, end_per_testcase/2]). init_per_testcase(Func, Config) when is_atom(Func), is_list(Config) -> - Dog=?t:timetrap(?t:seconds(10)), + Dog=?t:timetrap(?t:seconds(30)), [{watchdog, Dog}|Config]. end_per_testcase(_Func, Config) -> diff --git a/erts/emulator/test/port_SUITE.erl b/erts/emulator/test/port_SUITE.erl index 202a8b7537..e01b2f253b 100644 --- a/erts/emulator/test/port_SUITE.erl +++ b/erts/emulator/test/port_SUITE.erl @@ -1,7 +1,7 @@ %% %% %CopyrightBegin% %% -%% Copyright Ericsson AB 1997-2013. All Rights Reserved. +%% Copyright Ericsson AB 1997-2014. All Rights Reserved. %% %% The contents of this file are subject to the Erlang Public License, %% Version 1.1, (the "License"); you may not use this file except in @@ -1698,12 +1698,13 @@ otp_5119(Config) when is_list(Config) -> Path = ?config(data_dir, Config), ok = load_driver(Path, "exit_drv"), PI1 = port_ix(otp_5119_fill_empty_port_tab([])), - PI2 = port_ix(erlang:open_port({spawn, "exit_drv"}, [])), + Port2 = erlang:open_port({spawn, "exit_drv"}, []), + PI2 = port_ix(Port2), {PortIx1, PortIx2} = case PI2 > PI1 of true -> {PI1, PI2}; false -> - {port_ix(otp_5119_fill_empty_port_tab([PI2])), + {port_ix(otp_5119_fill_empty_port_tab([Port2])), port_ix(erlang:open_port({spawn, "exit_drv"}, []))} end, MaxPorts = max_ports(), @@ -2318,7 +2319,7 @@ close_deaf_port(Config) when is_list(Config) -> test_server:timetrap_cancel(Dog), Res. -close_deaf_port_1(1000, _) -> +close_deaf_port_1(200, _) -> ok; close_deaf_port_1(N, Cmd) -> Timeout = integer_to_list(random:uniform(5*1000)), diff --git a/erts/preloaded/ebin/erl_prim_loader.beam b/erts/preloaded/ebin/erl_prim_loader.beam Binary files differindex 26f851fd7a..f1e588320b 100644 --- a/erts/preloaded/ebin/erl_prim_loader.beam +++ b/erts/preloaded/ebin/erl_prim_loader.beam diff --git a/erts/preloaded/ebin/erlang.beam b/erts/preloaded/ebin/erlang.beam Binary files differindex 344176b71b..b4e22f6d74 100644 --- a/erts/preloaded/ebin/erlang.beam +++ b/erts/preloaded/ebin/erlang.beam diff --git a/erts/preloaded/ebin/erts_internal.beam b/erts/preloaded/ebin/erts_internal.beam Binary files differindex bd402f9a98..d41c833e05 100644 --- a/erts/preloaded/ebin/erts_internal.beam +++ b/erts/preloaded/ebin/erts_internal.beam diff --git a/erts/preloaded/ebin/init.beam b/erts/preloaded/ebin/init.beam Binary files differindex 5d49eed469..7f2d2740e1 100644 --- a/erts/preloaded/ebin/init.beam +++ b/erts/preloaded/ebin/init.beam diff --git a/erts/preloaded/ebin/otp_ring0.beam b/erts/preloaded/ebin/otp_ring0.beam Binary files differindex 2607739d12..4d22d8bace 100644 --- a/erts/preloaded/ebin/otp_ring0.beam +++ b/erts/preloaded/ebin/otp_ring0.beam diff --git a/erts/preloaded/ebin/prim_eval.beam b/erts/preloaded/ebin/prim_eval.beam Binary files differindex d665d35ab1..efc8347b6e 100644 --- a/erts/preloaded/ebin/prim_eval.beam +++ b/erts/preloaded/ebin/prim_eval.beam diff --git a/erts/preloaded/ebin/prim_file.beam b/erts/preloaded/ebin/prim_file.beam Binary files differindex 201252de34..6c49b5185e 100644 --- a/erts/preloaded/ebin/prim_file.beam +++ b/erts/preloaded/ebin/prim_file.beam diff --git a/erts/preloaded/ebin/prim_inet.beam b/erts/preloaded/ebin/prim_inet.beam Binary files differindex ff8265414e..fe5431c5ff 100644 --- a/erts/preloaded/ebin/prim_inet.beam +++ b/erts/preloaded/ebin/prim_inet.beam diff --git a/erts/preloaded/ebin/prim_zip.beam b/erts/preloaded/ebin/prim_zip.beam Binary files differindex f80d4a96e5..73be297bbb 100644 --- a/erts/preloaded/ebin/prim_zip.beam +++ b/erts/preloaded/ebin/prim_zip.beam diff --git a/erts/preloaded/ebin/zlib.beam b/erts/preloaded/ebin/zlib.beam Binary files differindex dba3ec9546..193cebdc31 100644 --- a/erts/preloaded/ebin/zlib.beam +++ b/erts/preloaded/ebin/zlib.beam diff --git a/erts/preloaded/src/erlang.erl b/erts/preloaded/src/erlang.erl index fbc37bd955..cabbbd191f 100644 --- a/erts/preloaded/src/erlang.erl +++ b/erts/preloaded/src/erlang.erl @@ -1745,9 +1745,9 @@ is_pid(_Term) -> erlang:nif_error(undefined). %% Shadowed by erl_bif_types: erlang:is_map/1 --spec is_map(Map) -> boolean() when - Map :: map(). -is_map(_Map) -> +-spec is_map(Term) -> boolean() when + Term :: term(). +is_map(_Term) -> erlang:nif_error(undefined). %% Shadowed by erl_bif_types: erlang:is_port/1 diff --git a/erts/test/otp_SUITE.erl b/erts/test/otp_SUITE.erl index cd5cfcbab4..229d10ccee 100644 --- a/erts/test/otp_SUITE.erl +++ b/erts/test/otp_SUITE.erl @@ -392,12 +392,18 @@ is_bad_encoding(File) -> end. runtime_dependencies(Config) -> + %% Ignore applications intentionally not declaring dependencies + %% found by xref. + IgnoreApps = [diameter], + + %% Verify that (at least) OTP application runtime dependencies found %% by xref are listed in the runtime_dependencies field of the .app file %% of each application. Server = ?config(xref_server, Config), {ok, AE} = xref:q(Server, "AE"), SAE = lists:keysort(1, AE), + put(ignored_failures, []), {AppDep, AppDeps} = lists:foldl(fun ({App, App}, Acc) -> Acc; ({App, Dep}, {undefined, []}) -> @@ -409,8 +415,45 @@ runtime_dependencies(Config) -> end, {undefined, []}, SAE), - [] = check_apps_deps([AppDep|AppDeps]), - ok. + [] = lists:filter(fun ({missing_runtime_dependency, + AppFile, + common_test}) -> + %% The test_server app is contaminated by + %% common_test when run in a source tree. It + %% should however *not* be contaminated + %% when run in an installation. + case {filename:basename(AppFile), + is_run_in_src_tree()} of + {"test_server.app", true} -> + false; + _ -> + true + end; + (_) -> + true + end, + check_apps_deps([AppDep|AppDeps], IgnoreApps)), + case IgnoreApps of + [] -> + ok; + _ -> + Comment = lists:flatten(io_lib:format("Ignored applications: ~p " + "Ignored failures: ~p", + [IgnoreApps, + get(ignored_failures)])), + {comment, Comment} + end. + +is_run_in_src_tree() -> + %% At least currently run_erl is not present in <code-root>/bin + %% in the source tree, but present in <code-root>/bin of an + %% ordinary installation. + case file:read_file_info(filename:join([code:root_dir(), + "bin", + "run_erl"])) of + {ok, _} -> false; + {error, _} -> true + end. have_rdep(_App, [], _Dep) -> false; @@ -424,28 +467,43 @@ have_rdep(App, [RDep | RDeps], Dep) -> have_rdep(App, RDeps, Dep) end. -check_app_deps(_App, _AppFile, _AFDeps, []) -> +check_app_deps(_App, _AppFile, _AFDeps, [], _IgnoreApps) -> []; -check_app_deps(App, AppFile, AFDeps, [XRDep | XRDeps]) -> - ResOtherDeps = check_app_deps(App, AppFile, AFDeps, XRDeps), +check_app_deps(App, AppFile, AFDeps, [XRDep | XRDeps], IgnoreApps) -> + ResOtherDeps = check_app_deps(App, AppFile, AFDeps, XRDeps, IgnoreApps), case have_rdep(App, AFDeps, XRDep) of true -> ResOtherDeps; false -> - [{missing_runtime_dependency, AppFile, XRDep} | ResOtherDeps] + Failure = {missing_runtime_dependency, AppFile, XRDep}, + case lists:member(App, IgnoreApps) of + true -> + put(ignored_failures, [Failure | get(ignored_failures)]), + ResOtherDeps; + false -> + [Failure | ResOtherDeps] + end end. -check_apps_deps([]) -> +check_apps_deps([], _IgnoreApps) -> []; -check_apps_deps([{App, Deps}|AppDeps]) -> - ResOtherApps = check_apps_deps(AppDeps), +check_apps_deps([{App, Deps}|AppDeps], IgnoreApps) -> + ResOtherApps = check_apps_deps(AppDeps, IgnoreApps), AppFile = code:where_is_file(atom_to_list(App) ++ ".app"), {ok,[{application, App, Info}]} = file:consult(AppFile), case lists:keyfind(runtime_dependencies, 1, Info) of {runtime_dependencies, RDeps} -> - check_app_deps(App, AppFile, RDeps, Deps) ++ ResOtherApps; + check_app_deps(App, AppFile, RDeps, Deps, IgnoreApps) + ++ ResOtherApps; false -> - [{missing_runtime_dependencies_key, AppFile} | ResOtherApps] + Failure = {missing_runtime_dependencies_key, AppFile}, + case lists:member(App, IgnoreApps) of + true -> + put(ignored_failures, [Failure | get(ignored_failures)]), + ResOtherApps; + false -> + [Failure | ResOtherApps] + end end. %%% diff --git a/lib/asn1/doc/src/asn1_ug.xml b/lib/asn1/doc/src/asn1_ug.xml index ee54fdffd7..020e58c615 100644 --- a/lib/asn1/doc/src/asn1_ug.xml +++ b/lib/asn1/doc/src/asn1_ug.xml @@ -34,23 +34,25 @@ <section> <title>Features</title> - <p>The Asn1 application provides: - </p> + <p>The Asn1 application provides:</p> <list type="bulleted"> <item>An ASN.1 compiler for Erlang, which generates encode and decode functions to be used by Erlang programs sending and receiving ASN.1 specified data.</item> <item>Run-time functions used by the generated code.</item> - <item>The supported encoding rules are: + <item>Support for the following encoding rules: <list> <item> Basic Encoding Rules (<em>BER</em>) </item> <item> - Distinguished Encoding Rules (<em>DER</em>), a specialized form of BER that is used in security-conscious applications. + Distinguished Encoding Rules (<em>DER</em>), a specialized + form of BER that is used in security-conscious + applications. </item> <item> - Packed Encoding Rules (<em>PER</em>) both the aligned and unaligned variant. + Packed Encoding Rules (<em>PER</em>); both the aligned and + unaligned variant. </item> </list> </item> @@ -59,71 +61,41 @@ <section> <title>Overview</title> - <p>ASN.1 (Abstract Syntax Notation 1) is a formal language for describing data structures to be exchanged between distributed computer systems. - The purpose of ASN.1 is to have - a platform and programming language independent notation to express - types using a - standardized set of rules for the transformation of values of - a defined type, into a stream of bytes. This stream of bytes - can then be sent on a communication channel set up by the - lower layers in the stack of communication protocols e.g. - TCP/IP or encapsulated within UDP packets. This way, two - different applications written in two completely different - programming languages running on different computers with - different internal representation of data can exchange - instances of structured data types (instead of exchanging - bytes or bits). This makes programming faster and easier since no code - has to be written to process the transport format of the - data. - </p> - <p>To write a network application which processes ASN.1 encoded - messages, it is prudent and sometimes essential to have a set - of off-line development tools such as an ASN.1 compiler which - can generate the encode and decode logic for the specific ASN.1 - data types. It is also necessary to combine this with some - general language-specific runtime support for ASN.1 encoding and - decoding. - </p> - <p>The ASN.1 compiler must be directed towards a target language - or a set of closely related languages. This manual describes a - compiler which is directed towards the functional language - Erlang. In order to use this compiler, familiarity with the - language Erlang is essential. Therefore, the runtime support for ASN.1 is - also closely related to the language Erlang and - consist of a number of functions, which the - compiler uses. The types in ASN.1 and how to represent - values of those types in Erlang are described in this manual. - </p> - <p>The following document is structured so that the first part describes - how to use ASN.1 compiler, and then there are descriptions of all - the primitive and constructed ASN.1 types and their representation - in Erlang, - </p> + <p>ASN.1 (Abstract Syntax Notation One) is a formal language for + describing data structures to be exchanged between distributed + computer systems. The purpose of ASN.1 is to have a platform + and programming language independent notation to express types + using a standardized set of rules for the transformation of + values of a defined type into a stream of bytes. This stream of + bytes can then be sent on any type of communication + channel. This way, two applications written in different + programming languages running on different computers with + different internal representation of data can exchange instances + of structured data types.</p> </section> <section> <title>Prerequisites</title> - <p>It is assumed that the reader is familiar with the ASN.1 notation - as documented in the standard definition [<cite id="X.680"></cite>] which is - the primary text. It may also be helpful, but not necessary, - to read the standard definitions - [<cite id="X.681"></cite>] [<cite id="X.682"></cite>] [<cite id="X.683"></cite>] - [<cite id="X.690"></cite>] [<cite id="X.691"></cite>]. </p> - <p>A very good book explaining those reference texts is - [<cite id="DUBUISSON"></cite>], free to download at - <url href="http://www.oss.com/asn1/dubuisson.html">http://www.oss.com/asn1/dubuisson.html </url>. + <p>It is assumed that the reader is familiar with the ASN.1 + notation as documented in the standard definition [<cite + id="X.680"></cite>] which is the primary text. It may also be + helpful, but not necessary, to read the standard definitions + [<cite id="X.681"></cite>] [<cite id="X.682"></cite>] [<cite + id="X.683"></cite>] [<cite id="X.690"></cite>] [<cite + id="X.691"></cite>]. </p> + <p>A good book explaining those reference texts is + [<cite id="DUBUISSON"></cite>], which is free to download at + <url href="http://www.oss.com/asn1/dubuisson.html">http://www.oss.com/asn1/dubuisson.html</url>. </p> </section> <section> - <title>Capability</title> + <title>Capabilities</title> <p>This application covers all features of ASN.1 up to the 1997 - edition of the specification. In the 2002 edition of ASN.1 a number of - new features where introduced of which some are supported while - others are not. For example the - ECN (Encoding Control Notation) and XML notation are still - unsupported. Though, the other features of the 2002 edition are - fully or partly supported as shown below:</p> + edition of the specification. In the 2002 edition of ASN.1 a + number of new features were introduced. The following features + of the 2002 edition are fully or partly supported as shown + below:</p> <list type="bulleted"> <item> <p>Decimal notation (e.g., "1.5e3") for REAL values. The @@ -131,7 +103,7 @@ supported.</p> </item> <item> - <p>The RELATIVE-OID type for relative object identifiers are + <p>The RELATIVE-OID type for relative object identifiers is fully supported.</p> </item> <item> @@ -141,16 +113,16 @@ constraint is not a PER-visible constraint.</p> </item> <item> - <p>The subtype constraint by regular expressions (PATTERN) for character string types is parsed when compiling, but no further action is taken. This constraint is not a PER-visible constraint.</p> + <p>The subtype constraint by regular expressions (PATTERN) + for character string types is parsed when compiling, but no + further action is taken. This constraint is not a + PER-visible constraint.</p> </item> <item> <p>Multiple-line comments as in C, <c>/* ... */</c>, are supported.</p> </item> </list> - <p>It should also be added here that the encoding formats - supported are <em>BER</em>, <em>DER</em>, <em>PER aligned - basic</em> variant and <em>PER unaligned basic</em> variant.</p> </section> </section> @@ -162,19 +134,17 @@ <title>A First Example</title> <p>The following example demonstrates the basic functionality used to run the Erlang ASN.1 compiler.</p> - <p>First, create a file called <c>People.asn</c> containing the following:</p> + <p>Create a file called <c>People.asn</c> containing the following:</p> <pre> -People DEFINITIONS IMPLICIT TAGS ::= - +People DEFINITIONS AUTOMATIC TAGS ::= BEGIN -EXPORTS Person; - -Person ::= [PRIVATE 19] SEQUENCE { - name PrintableString, - location INTEGER {home(0),field(1),roving(2)}, - age INTEGER OPTIONAL } + Person ::= SEQUENCE { + name PrintableString, + location INTEGER {home(0),field(1),roving(2)}, + age INTEGER OPTIONAL + } END </pre> - <p>This file (<c>people.asn</c>) must be compiled before it can be + <p>This file (<c>People.asn</c>) must be compiled before it can be used. The ASN.1 compiler checks that the syntax is correct and that the text represents proper ASN.1 code before generating an abstract @@ -186,14 +156,14 @@ END </pre> The following shows how the compiler can be called from the Erlang shell:</p> <pre> -1><input>asn1ct:compile("People", [ber]).</input> +1><input> asn1ct:compile("People", [ber]).</input> ok 2> </pre> <p>The <c>verbose</c> option can be given to have information about the generated files printed:</p> <pre> -2><input>asn1ct:compile("People", [ber,verbose]).</input> +2><input> asn1ct:compile("People", [ber,verbose]).</input> Erlang ASN.1 compiling "People.asn" --{generated,"People.asn1db"}-- --{generated,"People.hrl"}-- @@ -201,17 +171,17 @@ Erlang ASN.1 compiling "People.asn" ok 3> </pre> - <p>The ASN.1 module People is now accepted and the abstract syntax tree - is saved in the <c>People.asn1db</c> file, the - generated Erlang code is compiled using the Erlang compiler and - loaded into the Erlang runtime system. Now there is a user interface - for <c>encode/2</c> and <c>decode/2</c> in the module People, - which is invoked by: - <br></br> -<c><![CDATA['People':encode(<Type name>,<Value>),]]></c> <br></br> - + <p>The ASN.1 module <c>People</c> is now accepted and the + abstract syntax tree is saved in the <c>People.asn1db</c> file; + the generated Erlang code is compiled using the Erlang compiler + and loaded into the Erlang run-time system. Now there is an API + for <c>encode/2</c> and <c>decode/2</c> in the module + <c>People</c>, which is invoked by: <br></br> + <c><![CDATA['People':encode(<Type name>, <Value>)]]></c> + <br></br> or <br></br> -<c><![CDATA['People':decode(<Type name>,<Value>),]]></c></p> +<c><![CDATA['People':decode(<Type name>, <Value>)]]></c></p> + <p>Assume there is a network application which receives instances of the ASN.1 defined type Person, modifies and sends them back again:</p> @@ -234,8 +204,7 @@ receive constructed and encoded using <c>'People':encode('Person',Answer)</c> which takes an instance of a defined ASN.1 type and transforms it to a - binary according to the BER or PER - encoding-rules. + binary according to the BER or PER encoding rules. <br></br> The encoder and the decoder can also be run from the shell.</p> @@ -252,13 +221,13 @@ The encoder and the decoder can also be run from <section> <title>Module dependencies</title> - <p>It is common that asn1 modules import defined types, values and - other entities from another asn1 module.</p> - <p>Earlier versions of the asn1 compiler required that modules that + <p>It is common that ASN.1 modules import defined types, values and + other entities from another ASN.1 module.</p> + <p>Earlier versions of the ASN.1 compiler required that modules that were imported from had to be compiled before the module that - imported. This caused problems when asn1 modules had circular + imported. This caused problems when ASN.1 modules had circular dependencies.</p> - <p>Now are referenced modules parsed when the compiler finds an + <p>Referenced modules are now parsed when the compiler finds an entity that is imported. There will not be any code generated for the referenced module. However, the compiled module rely on that the referenced modules also will be compiled.</p> @@ -310,7 +279,7 @@ erlc -o ../asnfiles -I ../asnfiles -I /usr/local/standards/asn1 Person.asn </item> <tag><c>-I IncludeDir</c></tag> <item> - <p>Where to search for <c>.asn1db</c> files and asn1 + <p>Where to search for <c>.asn1db</c> files and ASN.1 source specs in order to resolve references to other modules. This option can be repeated many times if there are several places to search in. The compiler will always @@ -322,26 +291,26 @@ erlc -o ../asnfiles -I ../asnfiles -I /usr/local/standards/asn1 Person.asn </item> <tag><c>+asn1config</c></tag> <item> - <p>This functionality works together with the flags - <c>ber</c>. It enables the + <p>This functionality works together with the + <c>ber</c> option. It enables the specialized decodes, see the <seealso marker="asn1_spec">Specialized Decode</seealso> chapter. </p> </item> <tag><c>+undec_rest</c></tag> <item> - <p>A buffer that holds a message, being decoded may - also have some following bytes. Now it is possible to get - those following bytes returned together with the decoded - value. If an asn1 spec is compiled with this option a tuple - <c>{ok,Value,Rest}</c> is returned. <c>Rest</c> may be a - list or a binary. Earlier versions of the compiler ignored - those following bytes.</p> + <p>A buffer that holds a message being decoded may also have + trailing bytes. If those trailing bytes are important they + can be returned along with the decoded value by compiling + the ASN.1 specification with the <c>+undec_rest</c> option. + The return value from the decoder will be + <c>{ok,Value,Rest}</c> where <c>Rest</c> is a binary + containing the trailing bytes.</p> </item> <tag><c>+'Any Erlc Option'</c></tag> <item> <p>You may add any option to the Erlang compiler when compiling the generated Erlang files. Any option - unrecognised by the asn1 compiler will be passed to the + unrecognized by the ASN.1 compiler will be passed to the Erlang compiler.</p> </item> </taglist> @@ -366,10 +335,6 @@ asn1ct:compile("H323-MESSAGES.asn1",[ber]). </pre> asn1ct:compile("H323-MESSAGES.asn1",[per]). </pre> <p>The generic encode and decode functions can be invoked like this:</p> <pre> -asn1ct:encode('H323-MESSAGES','SomeChoiceType',{call,"octetstring"}). -asn1ct:decode('H323-MESSAGES','SomeChoiceType',Bytes). </pre> - <p>Or, preferable like:</p> - <pre> 'H323-MESSAGES':encode('SomeChoiceType',{call,"octetstring"}). 'H323-MESSAGES':decode('SomeChoiceType',Bytes). </pre> </section> @@ -389,7 +354,7 @@ asn1ct:decode('H323-MESSAGES','SomeChoiceType',Bytes). </pre> compile time appear on the screen together with a line number indicating where in the source file the error was detected. If no errors are found, an Erlang ASN.1 module will - be created as default.</p> + be created.</p> <p>The run-time encoders and decoders execute within a catch and returns <c>{ok, Data}</c> or <c>{error, {asn1, Description}}</c> where @@ -400,18 +365,18 @@ asn1ct:decode('H323-MESSAGES','SomeChoiceType',Bytes). </pre> <section> <marker id="inlineExamples"></marker> - <title>Multi File Compilation</title> - <p>There are various reasons for using a multi file compilation:</p> + <title>Multi-file Compilation</title> + <p>There are various reasons for using multi-file compilation:</p> <list type="bulleted"> - <item>You want to choose name for the generated module by - any reason. Maybe you need to compile the same specs for - different encoding/decoding standards.</item> + <item>You want to choose the name for the generated module, + perhaps because you need to compile the same specs for + different encoding rules.</item> <item>You want only one resulting module.</item> </list> - <p>You need to specify which asn1 specs you will + <p>You need to specify which ASN.1 specs you will compile in a module that must have the extension <c>.set.asn</c>. You chose name of the module and provide the - names of the asn1 specs. For instance, if you have the specs + names of the ASN.1 specs. For instance, if you have the specs <c>File1.asn</c>, <c>File2.asn</c> and <c>File3.asn</c> your module <c>MyModule.set.asn</c> will look like:</p> <pre> @@ -422,11 +387,45 @@ File3.asn </pre> <code type="none"> ~> erlc MyModule.set.asn </code> <p>the result will be one merged module <c>MyModule.erl</c> with - the generated code from the three asn1 specs. + the generated code from the three ASN.1 specs. </p> </section> <section> + <title>A quick note about tags</title> + + <p>Tags used to be important for all users of ASN.1, because it + was necessary to manually add tags to certain constructs in order + for the ASN.1 specification to be valid. Here is an example of + an old-style specification:</p> + + <pre> +Tags DEFINITIONS ::= +BEGIN + Afters ::= CHOICE { cheese [0] IA5String, + dessert [1] IA5String } +END </pre> + + <p>Without the tags (the numbers in square brackets) the ASN.1 + compiler would refuse to compile the file.</p> + + <p>In 1994 the global tagging mode AUTOMATIC TAGS was introduced. + By putting AUTOMATIC TAGS in the module header, the ASN.1 compiler + will automatically add tags when needed. Here is the same + specification in AUTOMATIC TAGS mode:</p> + + <pre> +Tags DEFINITIONS AUTOMATIC TAGS ::= +BEGIN + Afters ::= CHOICE { cheese IA5String, + dessert IA5String } +END +</pre> + + <p>Tags will not be mentioned any more in this manual.</p> + </section> + + <section> <marker id="ASN1Types"></marker> <title>The ASN.1 Types</title> <p>This section describes the ASN.1 types including their @@ -497,7 +496,7 @@ Operational ::= BOOLEAN --ASN.1 definition </pre> <p>In Erlang code it may look like:</p> <pre> Val = true, -{ok,Bytes}=asn1rt:encode(MyModule,'Operational',Val), </pre> +{ok,Bytes} = MyModule:encode('Operational', Val), </pre> <p>Below follows a description of how values of each type can be represented in Erlang. </p> @@ -563,20 +562,18 @@ T6value3 = white <section> <marker id="REAL"></marker> <title>REAL</title> - <p>In this version reals are not implemented. When they are, - the following - ASN.1 type is used:</p> + <p>The following ASN.1 type is used for real numbers:</p> <pre> R1 ::= REAL </pre> - <p>Can be assigned a value in Erlang as:</p> + <p>It can be assigned a value in Erlang as:</p> <pre> -R1value1 = 2.14, +R1value1 = "2.14", R1value2 = {256,10,-2}, </pre> <p>In the last line note that the tuple {256,10,-2} is the real number 2.56 in a special notation, which will encode faster than simply - stating the number as 2.56. The arity three tuple is + stating the number as <c>"2.56"</c>. The arity three tuple is <c>{Mantissa,Base,Exponent}</c> i.e. Mantissa * Base^Exponent.</p> </section> @@ -736,13 +733,11 @@ O2Val = <<"must be exactly 28 chars....">>,</pre> specified for a type are especially important for PER, where they affect the encoding. </p> - <p>Please note that <em>all</em> the Character strings are - supported and it is possible to use the following ASN.1 type - definitions:</p> + <p>Here are some examples:</p> <pre> Digs ::= NumericString (SIZE(1..3)) TextFile ::= IA5String (SIZE(0..64000)) </pre> - <p>and the following Erlang assignments:</p> + <p>with corresponding Erlang assignments:</p> <pre> DigsVal1 = "456", DigsVal2 = "123", @@ -755,70 +750,86 @@ TextFileVal2 = [88,76,55,44,99,121 .......... a lot of characters here ....] characters are all represented by quadruples beginning with three zeros like {0,0,0,65} for the 'A' character. When decoding a value for these strings the result is a list of - quadruples, or integers when the value is an ASCII character. - The following example shows how it works:</p> - <p>In a file <c>PrimStrings.asn1</c> the type <c>BMP</c> is defined as - <br></br> -<c>BMP ::= BMPString</c> then using BER encoding (<c>ber</c> - option)the input/output format will be:</p> + quadruples, or integers when the value is an ASCII character.</p> + + <p>The following example shows how it works. We have the following + specification in the file <c>PrimStrings.asn1</c>.</p> + <pre> +PrimStrings DEFINITIONS AUTOMATIC TAGS ::= +BEGIN + BMP ::= BMPString +END + </pre> + + <p>Encoding and decoding some strings:</p> + <pre> -1> <input>{ok,Bytes1} = asn1rt:encode('PrimStrings','BMP',[{0,0,53,53},{0,0,45,56}]).</input> -{ok,[30,4,"55-8"]} -2> <input>asn1rt:decode('PrimStrings','BMP',list_to_binary(Bytes1)).</input> +1> <input>asn1ct:compile('PrimStrings', [ber]).</input> +ok +2> <input>{ok,Bytes1} = 'PrimStrings':encode('BMP', [{0,0,53,53},{0,0,45,56}]).</input> +{ok,<<30,4,53,54,45,56>>} +3> <input>'PrimStrings':decode('BMP', Bytes1).</input> {ok,[{0,0,53,53},{0,0,45,56}]} -3> <input>{ok,Bytes2} = asn1rt:encode('PrimStrings','BMP',[{0,0,53,53},{0,0,0,65}]).</input> -{ok,[30,4,[53,53,0,65]]} -4> <input>asn1rt:decode('PrimStrings','BMP',list_to_binary(Bytes2)).</input> +4> <input>{ok,Bytes2} = 'PrimStrings':encode('BMP', [{0,0,53,53},{0,0,0,65}]).</input> +{ok,<<30,4,53,53,0,65>>} +5> <input>'PrimStrings':decode('BMP', Bytes2).</input> {ok,[{0,0,53,53},65]} -5> <input>{ok,Bytes3} = asn1rt:encode('PrimStrings','BMP',"BMP string").</input> -{ok,[30,20,[0,66,0,77,0,80,0,32,0,115,0,116,0,114,0,105,0,110,0,103]]} -6> <input>asn1rt:decode('PrimStrings','BMP',list_to_binary(Bytes3)).</input> +6> <input>{ok,Bytes3} = 'PrimStrings':encode('BMP', "BMP string").</input> +{ok,<<30,20,0,66,0,77,0,80,0,32,0,115,0,116,0,114,0,105,0,110,0,103>>} +7> <input>'PrimStrings':decode('BMP', Bytes3).</input> {ok,"BMP string"} </pre> - <p>The UTF8String is represented in Erlang as a list of integers, - where each integer represents the unicode value of one - character. When a value shall be encoded one first has to - transform it to a UTF8 encoded binary, then it can be encoded by - asn1. When decoding the result is a UTF8 encoded binary, which - may be transformed to an integer list. The transformation - functions, <c>utf8_binary_to_list</c> and - <c>utf8_list_to_binary</c>, are in the <c>asn1rt</c> module. In - the example below we assume an asn1 definition <c>UTF ::= UTF8String</c> in a module <c>UTF.asn</c>:</p> + + <p>The UTF8String type is represented as a UTF-8 encoded binary in + Erlang. Such binaries can be created directly using the binary syntax + or by converting from a list of Unicode code points using the + <c>unicode:characters_to_binary/1</c> function.</p> + + <p>Here are some examples showing how UTF-8 encoded binaries can + be created and manipulated:</p> + + <pre> +1> <input>Gs = "Мой маленький Гном".</input> +[1052,1086,1081,32,1084,1072,1083,1077,1085,1100,1082,1080, + 1081,32,1043,1085,1086,1084] +2> <input>Gbin = unicode:characters_to_binary(Gs).</input> +<<208,156,208,190,208,185,32,208,188,208,176,208,187,208, + 181,208,189,209,140,208,186,208,184,208,185,32,208,147, + 208,...>> +3> <input>Gbin = <<"Мой маленький Гном"/utf8>>.</input> +<<208,156,208,190,208,185,32,208,188,208,176,208,187,208, + 181,208,189,209,140,208,186,208,184,208,185,32,208,147, + 208,...>> +4> <input>Gs = unicode:characters_to_list(Gbin).</input> +[1052,1086,1081,32,1084,1072,1083,1077,1085,1100,1082,1080, + 1081,32,1043,1085,1086,1084] + </pre> + + <p>See the <seealso marker="stdlib:unicode">unicode</seealso> module + for more details.</p> + + <p>In the following example we will use this ASN.1 specification:</p> <pre> -1> <input>asn1ct:compile('UTF',[ber]).</input> -Erlang ASN.1 version "1.4.3.3" compiling "UTF.asn" -Compiler Options: [ber] ---{generated,"UTF.asn1db"}-- ---{generated,"UTF.erl"}-- +UTF DEFINITIONS AUTOMATIC TAGS ::= +BEGIN + UTF ::= UTF8String +END + </pre> + + <p>Encoding and decoding a string with Unicode characters:</p> + + <pre> +5> <input>asn1ct:compile('UTF', [ber]).</input> +ok +6> <input>{ok,Bytes1} = 'UTF':encode('UTF', <<"Гном"/utf8>>).</input> +{ok,<<12,8,208,147,208,189,208,190,208,188>>} +7> <input>{ok,Bin1} = 'UTF':decode('UTF', Bytes1).</input> +{ok,<<208,147,208,189,208,190,208,188>>} +8> <input>io:format("~ts\n", [Bin1]).</input> +Гном ok -2> <input>UTF8Val1 = "hello".</input> -"hello" -3> <input>{ok,UTF8bin1} = asn1rt:utf8_list_to_binary(UTF8Val1).</input> -{ok,<<104,101,108,108,111>>} -4> <input>{ok,B}='UTF':encode('UTF',UTF8bin1).</input> -{ok,[12, - 5, - <<104,101,108,108,111>>]} -5> <input>Bin = list_to_binary(B).</input> -<<12,5,104,101,108,108,111>> -6> <input>{ok,UTF8bin1}='UTF':decode('UTF',Bin).</input> -{ok,<<104,101,108,108,111>>} -7> <input>asn1rt:utf8_binary_to_list(UTF8bin1).</input> -{ok,"hello"} -8> <input>UTF8Val2 = [16#00,16#100,16#ffff,16#ffffff].</input> -[0,256,65535,16777215] -9> <input>{ok,UTF8bin2} = asn1rt:utf8_list_to_binary(UTF8Val2).</input> -{ok,<<0,196,128,239,191,191,248,191,191,191,191>>} -10> <input>{ok,B2} = 'UTF':encode('UTF',UTF8bin2).</input> -{ok,[12, - 11, - <<0,196,128,239,191,191,248,191,191,191,191>>]} -11> <input>Bin2 = list_to_binary(B2).</input> -<<12,11,0,196,128,239,191,191,248,191,191,191,191>> -12> <input>{ok,UTF8bin2} = 'UTF':decode('UTF',Bin2).</input> -{ok,<<0,196,128,239,191,191,248,191,191,191,191>>} -13> <input>asn1rt:utf8_binary_to_list(UTF8bin2).</input> -{ok,[0,256,65535,16777215]} -14> </pre> +9> <input>unicode:characters_to_list(Bin1).</input> +[1043,1085,1086,1084] + </pre> </section> <section> @@ -853,9 +864,11 @@ OidVal1 = {1,2,55}, <section> <marker id="Object Descriptor"></marker> <title>Object Descriptor</title> - <p>Values of this type can be assigned a value as an ordinary string i.e. <br></br> + <p>Values of this type can be assigned a value as an ordinary string + like this:</p> - "This is the value of an Object descriptor"</p> + <pre> + "This is the value of an Object descriptor"</pre> </section> <section> @@ -898,19 +911,31 @@ Pdu ::= SEQUENCE { <pre> MyPdu = #'Pdu'{a=22,b=77.99,c={0,1,2,3,4},d='NULL'}. </pre> <p>The decode functions will return a record as result when decoding - a <c>SEQUENCE</c> or a <c>SET</c>. - <marker id="DEFAULT"></marker> -</p> - <p>A <c>SEQUENCE</c> and a <c>SET</c> may contain a component with a - <c>DEFAULT</c> key word followed by the actual value that is the - default value. In case of BER encoding it is optional to encode the - value if it equals the default value. If the application uses the - atom asn1_DEFAULT as value or if the value is a primitive value - that equals the default value the encoding omits the bytes for - this value, which is more efficient and it results in fever - bytes to send to the receiving application.</p> - <p>For instance, if the following types exists in a file "File.asn":</p> + a <c>SEQUENCE</c> or a <c>SET</c>.</p> + + <p>A <c>SEQUENCE</c> and a <c>SET</c> may contain a component + with a <c>DEFAULT</c> key word followed by the actual value that + is the default value. The <c>DEFAULT</c> keyword means that the + application doing the encoding can omit encoding of the value, + thus resulting in fewer bytes to send to the receiving + application.</p> + + <p>An application can use the atom <c>asn1_DEFAULT</c> to indicate + that the encoding should be omitted for that position in + the SEQUENCE.</p> + + <p>Depending on the encoding rules, the encoder may also compare + the given value to the default value and automatically omit the + encoding if they are equal. How much effort the encoder makes to + to compare the values depends on the encoding rules. The DER + encoding rules forbids encoding a value equal to the default value, + so it has a more thorough and time-consuming comparison than the + encoders for the other encoding rules.</p> + + <p>In the following example we will use this ASN.1 specification:</p> <pre> +File DEFINITIONS AUTOMATIC TAGS ::= +BEGIN Seq1 ::= SEQUENCE { a INTEGER DEFAULT 1, b Seq2 DEFAULT {aa TRUE, bb 15} @@ -920,131 +945,50 @@ Seq2 ::= SEQUENCE { aa BOOLEAN, bb INTEGER } - </pre> - <p>Some values and the corresponding encoding in an Erlang terminal - is shown below:</p> + +Seq3 ::= SEQUENCE { + bs BIT STRING {a(0), b(1), c(2)} DEFAULT {a, c} +} +END </pre> + <p>Here is an example where the BER encoder is able to omit encoding + of the default values:</p> <pre> -1> <input>asn1ct:compile('File').</input> -Erlang ASN.1 version "1.3.2" compiling "File.asn1" -Compiler Options: [] ---{generated,"File.asn1db"}-- ---{generated,"File.hrl"}-- ---{generated,"File.erl"}-- +1> <input>asn1ct:compile('File', [ber]).</input> ok -2> <input>'File':encode('Seq1',{'Seq1',asn1_DEFAULT,asn1_DEFAULT}).</input> -{ok,["0",[0],[[],[]]]} -3> <input>lists:flatten(["0",[0],[[],[]]]).</input> -[48,0] -4> <input>'File':encode('Seq1',{'Seq1',1,{'Seq2',true,15}}).</input> -{ok,["0","\\b",[[],["\\241",[6],[[[128],[1],"\\377"],[[129],[1],[15]]]]]]} -5> <input>lists:flatten(["0","\\b",[[],["\\241",[6],[[[128],[1],"\\377"],[[129],[1],[15]]]]]]).</input> -[48,8,161,6,128,1,255,129,1,15] -6> </pre> - <p>The result after command line 3, in the example above,shows that the - encoder omits the encoding of default values when they are specific - by asn1_DEFAULT. Line 5 shows that even primitive values that equals - the default value are detected and not encoded. But the constructed - value of component <c>b</c> in <c>Seq1</c> is not recognized as the - default value. Checking of default values in <c>BER</c> is not done - in case of complex values, because it would be to expensive. - <marker id="DEFAULT DER"></marker> -</p> - <p>But, the DER encoding format has stronger requirements regarding - default values both for SET and SEQUENCE. A more elaborate and time - expensive check of default values will take place. The following is - an example with the same types and values as above but with der - encoding format.</p> - <pre> -1> <input>asn1ct:compile('File',[der]).</input> -Erlang ASN.1 version "1.3.2" compiling "File.asn1" -Compiler Options: [der] ---{generated,"File.asn1db"}-- ---{generated,"File.hrl"}-- ---{generated,"File.erl"}-- +2> <input>'File':encode('Seq1', {'Seq1',asn1_DEFAULT,asn1_DEFAULT}).</input> +{ok,<<48,0>>} +3> <input>'File':encode('Seq1', {'Seq1',1,{'Seq2',true,15}}).</input> +{ok,<<48,0>>} </pre> + + <p>And here is an example with a named BIT STRING where the BER + encoder will not omit the encoding:</p> + <pre> +4> <input>'File':encode('Seq3', {'Seq3',asn1_DEFAULT).</input> +{ok,<<48,0>>} +5> <input>'File':encode('Seq3', {'Seq3',<<16#101:3>>).</input> +{ok,<<48,4,128,2,5,160>>} </pre> + + <p>The DER encoder will omit the encoding for the same BIT STRING:</p> + <pre> +6> <input>asn1ct:compile('File', [ber,der]).</input> ok -2> <input>'File':encode('Seq1',{'Seq1',asn1_DEFAULT,asn1_DEFAULT}).</input> -{ok,["0",[0],[[],[]]]} -3> <input>lists:flatten(["0",[0],[[],[]]]).</input> -[48,0] -4> <input>'File':encode('Seq1',{'Seq1',1,{'Seq2',true,15}}).</input> -{ok,["0",[0],[[],[]]]} -5> <input>lists:flatten(["0",[0],[[],[]]]).</input> -[48,0] -6> - </pre> - <p>Line 5 shows that even values of constructed types is checked and if - it equals the default value it will not be encoded.</p> +7> <input>'File':encode('Seq3', {'Seq3',asn1_DEFAULT).</input> +{ok,<<48,0>>} +8> <input>'File':encode('Seq3', {'Seq3',<<16#101:3>>).</input> +{ok,<<48,0>>} </pre> </section> <section> <marker id="SET"></marker> <title>SET</title> - <p>The SET type is an unusual construct and normally the SEQUENCE - type is more appropriate to use. Set is also inefficient compared with SEQUENCE, as the components can be in any order. Hence, it must be possible - to distinguish every component in 'SET', both when - encoding and decoding a value of a type defined to be a SET. - The tags of all components must be different from each other - in order to be easily recognizable.</p> - <p>A SET may be defined as:</p> - <pre> -Pdu2 ::= SET { - a INTEGER, - b BOOLEAN, - c ENUMERATED {on(0),off(1)} } </pre> - <p>A SET is represented as an Erlang record. - For each SEQUENCE and <c>SET</c> in - an ASN.1 module an Erlang record declaration is generated. For - <c>Pdu2</c> above a record is defined like this:</p> - <pre> --record('Pdu2',{a, b, c}). </pre> - <p>The record declarations for a module <c>M</c> are placed in a - separate <c>M.hrl</c> file.</p> - <p>Values can be assigned in Erlang as demonstrated below:</p> - <pre> -V = #'Pdu2'{a=44,b=false,c=off}. </pre> - <p>The decode functions will return a record as result when decoding - a SET. - </p> - <p>The difference between SET and SEQUENCE is that the order of - the components (in the BER encoded format) is undefined for SET - and defined as the lexical order from the ASN.1 definition for - SEQUENCE. The ASN.1 compiler for Erlang will always encode a - SET in the lexical order. The decode routines can handle SET - components encoded in any order but will always return the - result as a record. Since all components of the SET must be - distinguishable both in the encoding phase as well as the - decoding phase the following type is not allowed in a module - with EXPLICIT or IMPLICIT as tag-default :</p> - <p></p> - <pre> -Bad ::= SET {i INTEGER, - j INTEGER } </pre> - <p>The ASN.1 to Erlang compiler rejects the above type. We - shall not explain the concept of tag further here, we refer to - [<cite id="X.680"></cite>]. - </p> - <p>Encoding of a SET with components with DEFAULT values behaves - similar as a SEQUENCE, <seealso marker="#DEFAULT">see above</seealso>. The DER encoding format restrictions on DEFAULT - values is the same for SET as for SEQUENCE, and is supported by - the compiler, <seealso marker="#DEFAULT DER">see above</seealso>.</p> - <p>Moreover, in DER the elements of a SET will be sorted. If a - component is an un-tagged choice the sorting have to take place - in run-time. This fact emphasizes the following recommendation - if DER encoding format is used.</p> - <p>The concept of SET is an unusual - construct and one cannot think of one single application - where the set type is essential. (Imagine if someone - "invented'' the shuffled array in 'C') People tend to think - that 'SET' sounds nicer and more mathematical than 'SEQUENCE' - and hence use it when 'SEQUENCE' would have been more - appropriate. It is also most inefficient, since every correct - implementation of SET must always be prepared to accept the - components in any order. So, if possible use SEQUENCE instead - of SET.</p> + <p>In Erlang, the SET type is used exactly as SEQUENCE. Note + that if the BER or DER encoding rules are used, decoding a + SET is slower than decoding a SEQUENCE because the components + must be sorted.</p> </section> <section> - <title>Notes about Extend-ability for SEQUENCE and SET</title> + <title>Notes about extensibility for SEQUENCE and SET</title> <p>When a SEQUENCE or SET contains an extension marker and extension components like this:</p> <pre> @@ -1071,51 +1015,28 @@ SExt ::= SEQUENCE { <marker id="CHOICE"></marker> <title>CHOICE</title> <p>The CHOICE type is a space saver and is similar to the concept of a - 'union' in the C-language. As with the previous SET-type, the - tags of all components of a CHOICE need to be distinct. If - AUTOMATIC TAGS are defined for the module (which is - preferable) the tags can be omitted completely in the ASN.1 - specification of a CHOICE. - </p> + 'union' in the C language.</p> <p>Assume:</p> <pre> +SomeModuleName DEFINITIONS AUTOMATIC TAGS ::= +BEGIN T ::= CHOICE { - x [0] REAL, - y [1] INTEGER, - z [2] OBJECT IDENTIFIER } - </pre> + x REAL, + y INTEGER, + z OBJECT IDENTIFIER } +END </pre> <p>It is then possible to assign values:</p> <pre> TVal1 = {y,17}, TVal2 = {z,{0,1,2}}, </pre> - <p>A CHOICE value is always represented as the tuple + <p>A CHOICE value is always represented as the tuple <c>{ChoiceAlternative, Val}</c> where <c>ChoiceAlternative</c> - is an atom denoting the selected choice - alternative. - </p> - <p>It is also allowed to have a CHOICE type tagged as follow:</p> - <p></p> - <pre> -C ::= [PRIVATE 111] CHOICE { - C1, - C2 } - -C1 ::= CHOICE { - a [0] INTEGER, - b [1] BOOLEAN } - -C2 ::= CHOICE { - c [2] INTEGER, - d [3] OCTET STRING } </pre> - <p>In this case, the top type C appears to have no tags at all in - its components, however, both C1 and C2 are also defined as - CHOICE types and they have distinct tags among themselves. - Hence, the above type C is both legal and allowed. + is an atom denoting the selected choice alternative. </p> <section> - <title>Extendable CHOICE</title> + <title>Extensible CHOICE</title> <p>When a CHOICE contains an extension marker and the decoder detects an unknown alternative of the CHOICE the value is represented as:</p> <pre> @@ -1192,26 +1113,29 @@ Arr2Val = ["abc",[14,34,54],"Octets"], </pre> Where <c>Value</c> may be a value of yet another type T2.</p> <p>For example:</p> <pre> +EmbeddedExample DEFINITIONS AUTOMATIC TAGS ::= +BEGIN B ::= SEQUENCE { a Arr1, - b [0] T } + b T } Arr1 ::= SET SIZE (5) OF INTEGER (4..9) T ::= CHOICE { - x [0] REAL, - y [1] INTEGER, - z [2] OBJECT IDENTIFIER } </pre> - <p>The above example can be assigned like this in Erlang:</p> + x REAL, + y INTEGER, + z OBJECT IDENTIFIER } + END </pre> + <p>The SEQUENCE b can be encoded like this in Erlang:</p> <pre> -V2 = #'B'{a=[4,5,6,7,8], b={x,7.77}}. - </pre> +1> 'EmbeddedExample':encode('B', {'B',[4,5,6,7,8],{x,"7.77"}}). +{ok,<<5,56,0,8,3,55,55,55,46,69,45,50>>} </pre> </section> </section> <section> <title>Naming of Records in .hrl Files</title> - <p>When an asn1 specification is compiled all defined types of + <p>When an ASN.1 specification is compiled all defined types of type SET or SEQUENCE will result in a corresponding record in the generated hrl file. This is because the values for SET/SEQUENCE as mentioned in sections above are represented as records.</p> @@ -1227,8 +1151,8 @@ V2 = #'B'{a=[4,5,6,7,8], b={x,7.77}}. Emb ::= SEQUENCE { a SEQUENCE OF OCTET STRING, b SET { - a [0] INTEGER, - b [1] INTEGER DEFAULT 66}, + a INTEGER, + b INTEGER DEFAULT 66}, c CHOICE { a INTEGER, b FooType } } @@ -1299,7 +1223,7 @@ PType{T} ::= SEQUENCE{ <p>Types may refer to themselves. Suppose:</p> <pre> Rec ::= CHOICE { - nothing [0] NULL, + nothing NULL, something SEQUENCE { a INTEGER, b OCTET STRING, @@ -1331,7 +1255,7 @@ tt TT ::= {a 77,b {"kalle","kula"}} </pre> Firstly, it could be used as the value in some DEFAULT component:</p> <pre> SS ::= SET { - s [0] OBJECT IDENTIFIER, + s OBJECT IDENTIFIER, val TT DEFAULT tt } </pre> <p>It could also be used from inside an Erlang program. If the above ASN.1 code was defined in ASN.1 module <c>Values</c>, then the ASN.1 value @@ -1365,8 +1289,8 @@ SS ::= SET { <marker id="Information Object"></marker> <title>ASN.1 Information Objects (X.681)</title> <p>Information Object Classes, Information Objects and Information - Object Sets, (in the following called classes, objects and - object sets respectively), are defined in the standard + Object Sets (in the following called classes, objects and + object sets respectively) are defined in the standard definition [<cite id="X.681"></cite>]. In the following only a brief explanation is given. </p> <p>These constructs makes it possible to define open types, @@ -1435,9 +1359,26 @@ StartMessage ::= SEQUENCE { <p><c>StartMessage</c> can in the <c>content</c> field be encoded with a value of any type that an object in the <c>GENERAL-PROCEDURES</c> object set has in its <c>NEW MESSAGE</c> field. This field refers to a type field - <c><![CDATA[&Message]]></c> in the class. The <c>msgId</c> field is always + <c>&Message</c> in the class. The <c>msgId</c> field is always encoded as a PrintableString, since the field refers to a fixed type in the class.</p> + <p>In practice, object sets are usually declared to be extensible so + so that more objects can be added to the set later. Extensibility is + indicated like this:</p> + <pre> +GENERAL-PROCEDURES GENERAL-PROCEDURE ::= { + object1 | object2, ...} </pre> + <p>When decoding a type that uses an extensible set constraint, + there is always the possibility that the value in the UNIQUE + field is unknown (i.e. the type has been encoded with a later + version of the ASN.1 specification). When that happens, the + unencoded data will be returned wrapped in a tuple like this:</p> + + <pre> +{asn1_OPENTYPE,Binary}</pre> + <p>where <c>Binary</c> is an Erlang binary that contains the encoded + data. (If the option <c>legacy_erlang_types</c> has been given, + just the binary will be returned.)</p> </section> <section> @@ -1466,132 +1407,11 @@ T1 ::= General{PrintableString} T2 ::= General{BIT STRING} </pre> <p>An example of a value that can be encoded as type T1 is {12,"hello"}.</p> - <p>Observe that the compiler not generates encode/decode functions for - parameterized types, only for the instances of the parameterized - types. So, if a file contains the types General{}, T1 and T2 above, + <p>Note that the compiler does not generate encode/decode functions for + parameterized types, but only for the instances of the parameterized + types. Therefore, if a file contains the types General{}, T1 and T2 above, encode/decode functions will only be generated for T1 and T2. </p> </section> - - <section> - <title>Tags</title> - <p>Every built-in ASN.1 type, except CHOICE and ANY have a universal tag. - This is a unique number that clearly identifies the type. <br></br> - - It is essential for all users of ASN.1 to - understand all the details about tags.</p> - <p>Tags are implicitly encoded in the BER encoding as shown below, but - are hardly not accounted for in the PER encoding. In PER tags are - used for instance to sort the components of a SET.</p> - <p>There are four different types of tags.</p> - <taglist> - <tag><em>universal</em></tag> - <item> - <p>For types whose meaning is the same in all - applications. Such as integers, sequences and so on; that is, all the built in - types.</p> - </item> - <tag><em>application</em></tag> - <item> - <p>For application specific types for example, the types in - X.400 Message handling service have this sort of tag.</p> - </item> - <tag><em>private</em></tag> - <item> - <p>For your own private types.</p> - </item> - <tag><em>context</em></tag> - <item> - <p>This is used to distinguish otherwise indistinguishable - types in a specific context. For example, if we have two - components of a - CHOICE type that are both <c>INTEGER</c> values, there is no - way for the decoder to - decipher which component was actually chosen, since both - components will be - tagged as <c>INTEGER</c>. When this or similar situations occur, - one or both of the components should be given a context specific - to resolve the ambiguity.</p> - </item> - </taglist> - <p>The tag in the case of the 'Apdu' type [PRIVATE 1] is encoded to a - sequence of bytes making it possible for a - decoder to look at the (initial) bytes that arrive and determine - whether the rest of the bytes must be of the type associated - with that particular sequence of bytes. This means that each - tag must be uniquely associated with <em>only</em> one ASN.1 - type. - </p> - <p>Immediately following the tag is a sequence of bytes - informing the decoder of the length of the instance. This is - sometimes referred to as TLV (Tag length value) encoding. - Hence, the structure of a BER encoded series of bytes is as shown in the table below.</p> - <p></p> - <table> - <row> - <cell align="left" valign="middle">Tag</cell> - <cell align="left" valign="middle">Len</cell> - <cell align="left" valign="middle">Value</cell> - </row> - <tcaption>Structure of a BER encoded series of bytes</tcaption> - </table> - </section> - - <section> - <title>Encoding Rules</title> - <p>When the first recommendation on ASN.1 was released 1988 it was - accompanied with the Basic Encoding Rules, BER, as the only - alternative for encoding. - BER is a somewhat verbose protocol. It adopts a so-called TLV (type, - length, value) approach to encoding in which every element of the - encoding carries some type information, some length information and - then the value of that element. Where the element is itself - structured, then the Value part of the element is itself a series of - embedded TLV components, to whatever depth is necessary. In summary, - BER is not a compact encoding but is relatively fast and easy to - produce.</p> - <p>The DER (Distinguished Encoding Rule) encoding format was included in - the standard in 1994. It is a specialized form of BER, which gives - the encoder the option to encode some entities differently. For - instance, is the value for TRUE any octet with any bit set to one. But, - DER does not leave any such choices. The value for TRUE in the DER - case is encoded as the octet <c>11111111</c>. So, the same value - encoded by two different DER encoders must result in the same bit - stream.</p> - <p>A more compact encoding is achieved with the Packed Encoding - Rules PER which was introduced together with the revised - recommendation in 1994. PER takes a rather different approach from - that taken by BER. The first difference is that the tag part in - the TLV is omitted from the encodings, and any tags in the - notation are not encoded. The potential ambiguities are resolved - as follows:</p> - <list type="bulleted"> - <item> - <p>A CHOICE is encoded by first encoding a choice index which - identifies the chosen - alternative by its position in the notation.</p> - </item> - <item> - <p>The elements of a SEQUENCE are transmitted in textual - order. OPTIONAL or DEFAULT elements are preceded by a bit map - to identify which elements are present. After sorting the - elements of a SET in the "canonical tag order" as defined in - X.680 8.6 they are treated as a SEQUENCE regarding OPTIONAL - and DEFAULT elements. A SET is transferred in the sorted - order.</p> - </item> - </list> - <p>A second difference is that PER takes full account of the sub-typing - information in that the encoded bytes are affected by the constraints. - The BER encoded bytes are unaffected by the constraints. - PER uses the sub-typing information to for example omit length fields - whenever possible. </p> - <p>The run-time functions, sometimes take the constraints into account - both for BER and PER. For instance are SIZE constrained strings checked.</p> - <p>There are two variants of PER, <em>aligned</em> and <em>unaligned</em>. - In summary, PER results in compact encodings which require much more - computation to produce than BER. - </p> - </section> </chapter> diff --git a/lib/asn1/doc/src/asn1ct.xml b/lib/asn1/doc/src/asn1ct.xml index 4d5a1a402a..32ff2d52cf 100644 --- a/lib/asn1/doc/src/asn1ct.xml +++ b/lib/asn1/doc/src/asn1ct.xml @@ -45,10 +45,11 @@ <p>By default in OTP 17, the representation of the BIT STRING and OCTET STRING types as Erlang terms have changed. BIT STRING values are now Erlang bitstrings and OCTET STRING values - are binaries. For details see <seealso - marker="asn1_ug#BIT STRING">BIT STRING</seealso> and <seealso - marker="asn1_ug#OCTET STRING">OCTET STRING</seealso> in User's - Guide.</p> + are binaries. Also, an undecoded open type will now be wrapped in + a <c>asn1_OPENTYPE</c> tuple. For details see <seealso + marker="asn1_ug#BIT STRING">BIT STRING</seealso>, <seealso + marker="asn1_ug#OCTET STRING">OCTET STRING</seealso>, and + <seealso marker="asn1_ug#Information%20Object">ASN.1 Information Objects</seealso> in User's Guide.</p> <p>To revert to the old representation of the types, use the <c>legacy_erlang_types</c> option.</p> </note> diff --git a/lib/common_test/doc/src/ct_hooks.xml b/lib/common_test/doc/src/ct_hooks.xml index 859ff9df14..cab6dfea51 100644 --- a/lib/common_test/doc/src/ct_hooks.xml +++ b/lib/common_test/doc/src/ct_hooks.xml @@ -450,12 +450,15 @@ </func> <func> - <name>Module:on_tc_fail(TestcaseName, Reason, CTHState) -> + <name>Module:on_tc_fail(TestName, Reason, CTHState) -> NewCTHState</name> <fsummary>Called after the CTH scope ends</fsummary> <type> - <v>TestcaseName = init_per_suite | end_per_suite | - init_per_group | end_per_group | atom()</v> + <v>TestName = init_per_suite | end_per_suite | + {init_per_group,GroupName} | {end_per_group,GroupName} | + {FuncName,GroupName} | FuncName</v> + <v>FuncName = atom()</v> + <v>GroupName = atom()</v> <v>Reason = term()</v> <v>CTHState = NewCTHState = term()</v> </type> @@ -463,14 +466,16 @@ <desc> <p> OPTIONAL </p> - <p>This function is called whenever a testcase fails. - It is called after the post function has been called for - the testcase which failed. i.e. - if init_per_suite fails this function is called after + <p>This function is called whenever a test case (or config function) + fails. It is called after the post function has been called for + the failed test case. I.e. if init_per_suite fails, this function + is called after <seealso marker="#Module:post_init_per_suite-4"> - post_init_per_suite</seealso>, and if a testcase fails it is called + post_init_per_suite</seealso>, and if a test case fails, it is called after <seealso marker="#Module:post_end_per_testcase-4"> - post_end_per_testcase</seealso>.</p> + post_end_per_testcase</seealso>. If the failed test case belongs + to a test case group, the first argument is a tuple + <c>{FuncName,GroupName}</c>, otherwise simply the function name.</p> <p>The data which comes with the Reason follows the same format as the <seealso marker="event_handler_chapter#failreason">FailReason @@ -481,12 +486,14 @@ </func> <func> - <name>Module:on_tc_skip(TestcaseName, Reason, CTHState) -> + <name>Module:on_tc_skip(TestName, Reason, CTHState) -> NewCTHState</name> <fsummary>Called after the CTH scope ends</fsummary> <type> - <v>TestcaseName = end_per_suite | {init_per_group,GroupName} | - {end_per_group,GroupName} | atom()</v> + <v>TestName = init_per_suite | end_per_suite | + {init_per_group,GroupName} | {end_per_group,GroupName} | + {FuncName,GroupName} | FuncName</v> + <v>FuncName = atom()</v> <v>GroupName = atom()</v> <v>Reason = {tc_auto_skip | tc_user_skip, term()}</v> <v>CTHState = NewCTHState = term()</v> @@ -495,14 +502,17 @@ <desc> <p> OPTIONAL </p> - <p>This function is called whenever a testcase is skipped. - It is called after the post function has been called for the - testcase which was skipped. - i.e. if init_per_group is skipped this function is called after - <seealso marker="#Module:post_init_per_suite-4">post_init_per_group - </seealso>, and if a testcase is skipped it is called after - <seealso marker="#Module:post_end_per_testcase-4">post_end_per_testcase - </seealso>.</p> + <p>This function is called whenever a test case (or config function) + is skipped. It is called after the post function has been called + for the skipped test case. I.e. if init_per_group is skipped, this + function is called after + <seealso marker="#Module:post_init_per_group-4"> + post_init_per_group</seealso>, and if a test case is skipped, + it is called after + <seealso marker="#Module:post_end_per_testcase-4"> + post_end_per_testcase</seealso>. If the skipped test case belongs to a + test case group, the first argument is a tuple <c>{FuncName,GroupName}</c>, + otherwise simply the function name.</p> <p>The data which comes with the Reason follows the same format as <seealso marker="event_handler_chapter#tc_auto_skip">tc_auto_skip diff --git a/lib/common_test/doc/src/ct_run.xml b/lib/common_test/doc/src/ct_run.xml index 39259b092a..d8e79ca80e 100644 --- a/lib/common_test/doc/src/ct_run.xml +++ b/lib/common_test/doc/src/ct_run.xml @@ -108,6 +108,7 @@ EvHandler2 InitArg2 and .. EvHandlerN InitArgN] [-include InclDir1 InclDir2 .. InclDirN] [-no_auto_compile] + [-abort_if_missing_suites] [-muliply_timetraps Multiplier] [-scale_timetraps] [-create_priv_dir auto_per_run | auto_per_tc | manual_per_tc] @@ -144,6 +145,7 @@ EvHandler2 InitArg2 and .. EvHandlerN InitArgN] [-include InclDir1 InclDir2 .. InclDirN] [-no_auto_compile] + [-abort_if_missing_suites] [-muliply_timetraps Multiplier] [-scale_timetraps] [-create_priv_dir auto_per_run | auto_per_tc | manual_per_tc] @@ -171,6 +173,7 @@ [-decrypt_key Key] | [-decrypt_file KeyFile] [-include InclDir1 InclDir2 .. InclDirN] [-no_auto_compile] + [-abort_if_missing_suites] [-muliply_timetraps Multiplier] [-scale_timetraps] [-create_priv_dir auto_per_run | auto_per_tc | manual_per_tc] diff --git a/lib/common_test/doc/src/event_handler_chapter.xml b/lib/common_test/doc/src/event_handler_chapter.xml index 47d0ba59fb..45f01c12ec 100644 --- a/lib/common_test/doc/src/event_handler_chapter.xml +++ b/lib/common_test/doc/src/event_handler_chapter.xml @@ -227,11 +227,13 @@ <item> <marker id="tc_auto_skip"></marker> - <c>#event{name = tc_auto_skip, data = {Suite,Func,Reason}}</c> + <c>#event{name = tc_auto_skip, data = {Suite,TestName,Reason}}</c> <p><c>Suite = atom()</c>, the name of the suite.</p> - <p><c>Func = atom() | {end_per_group,GroupName}</c>, the name of the test case - or configuration function.</p> - <p><c>GroupName = atom()</c>, name of the group.</p> + <p><c>TestName = init_per_suite | end_per_suite | + {init_per_group,GroupName} | {end_per_group,GroupName} | + {FuncName,GroupName} | FuncName</c></p> + <p><c>FuncName = atom()</c>, the name of the test case or configuration function.</p> + <p><c>GroupName = atom()</c>, the name of the test case group.</p> <p><c>Reason = {failed,FailReason} | {require_failed_in_suite0,RequireInfo}</c>, reason for auto skipping <c>Func</c>.</p> @@ -252,21 +254,26 @@ <c>init_per_group</c>, a failed <c>require</c> in <c>suite/0</c>, or a failed test case in a sequence. Note that this event is never received as a result of a test case getting skipped because of <c>init_per_testcase</c> failing, since that information is carried with - the <c>tc_done</c> event. + the <c>tc_done</c> event. If a failed test case belongs to a test case group, the second + data element is a tuple <c>{FuncName,GroupName}</c>, otherwise simply the function name. </p></item> - + <item> <marker id="tc_user_skip"></marker> - <c>#event{name = tc_user_skip, data = {Suite,Func,Comment}}</c> + <c>#event{name = tc_user_skip, data = {Suite,TestName,Comment}}</c> <p><c>Suite = atom()</c>, the name of the suite.</p> - <p><c>Func = atom() | {end_per_group,GroupName}</c>, the name of the test case - or configuration function.</p> - <p><c>GroupName = atom()</c>, name of the group.</p> + <p><c>TestName = init_per_suite | end_per_suite | + {init_per_group,GroupName} | {end_per_group,GroupName} | + {FuncName,GroupName} | FuncName</c></p> + <p><c>FuncName = atom()</c>, the name of the test case or configuration function.</p> + <p><c>GroupName = atom()</c>, the name of the test case group.</p> <p><c>Comment = string()</c>, reason for skipping the test case.</p> <p>This event specifies that a test case has been skipped by the user. It is only ever received if the skip was declared in a test specification. Otherwise, user skip information is received as a <c>{skipped,SkipReason}</c> - result in the <c>tc_done</c> event for the test case. + result in the <c>tc_done</c> event for the test case. If a skipped test case belongs + to a test case group, the second data element is a tuple <c>{FuncName,GroupName}</c>, + otherwise simply the function name. </p></item> <item><c>#event{name = test_stats, data = {Ok,Failed,Skipped}}</c> diff --git a/lib/common_test/doc/src/run_test_chapter.xml b/lib/common_test/doc/src/run_test_chapter.xml index a4a77ee400..864f82cb63 100644 --- a/lib/common_test/doc/src/run_test_chapter.xml +++ b/lib/common_test/doc/src/run_test_chapter.xml @@ -59,7 +59,15 @@ <p>If compilation should fail for one or more suites, the compilation errors are printed to tty and the operator is asked if the test run should proceed without the missing suites, or be aborted. If the operator chooses to proceed, - it is noted in the HTML log which tests have missing suites.</p> + it is noted in the HTML log which tests have missing suites. If Common Test is + unable to prompt the user after compilation failure (if Common Test doesn't + control stdin), the test run will proceed automatically without the missing + suites. This behaviour can however be modified with the + <c><![CDATA[ct_run]]></c> flag <c><![CDATA[-abort_if_missing_suites]]></c>, + or the <c><![CDATA[ct:run_test/1]]></c> option + <c><![CDATA[{abort_if_missing_suites,TrueOrFalse}]]></c>. If + <c><![CDATA[abort_if_missing_suites]]></c> is set (to true), the test run + will stop immediately if some suites fail to compile.</p> <p>Any help module (i.e. regular Erlang module with name not ending with "_SUITE") that resides in the same test object directory as a suite @@ -167,6 +175,7 @@ <seealso marker="ct_hooks_chapter#builtin_cths">Built-in Common Test Hooks</seealso>. Default is <c>true</c>.</item> <item><c><![CDATA[-include]]></c>, specifies include directories (see above).</item> <item><c><![CDATA[-no_auto_compile]]></c>, disables the automatic test suite compilation feature (see above).</item> + <item><c><![CDATA[-abort_if_missing_suites]]></c>, aborts the test run if one or more suites fail to compile (see above).</item> <item><c><![CDATA[-multiply_timetraps <n>]]></c>, extends <seealso marker="write_test_chapter#timetraps">timetrap timeout</seealso> values.</item> <item><c><![CDATA[-scale_timetraps <bool>]]></c>, enables automatic <seealso marker="write_test_chapter#timetraps">timetrap @@ -589,8 +598,8 @@ Common Test will either execute one test run per specification file, or join the files and perform all tests within one single test run. The first behaviour is the default one. The latter requires that the start - flag/option <c>join_suites</c> is provided, e.g. - <c>run_test -spec ./my_tests1.ts ./my_tests2.ts -join_suites</c>.</p> + flag/option <c>join_specs</c> is provided, e.g. + <c>run_test -spec ./my_tests1.ts ./my_tests2.ts -join_specs</c>.</p> <p>Joining a number of specifications, or running them separately, can also be accomplished with (and may be combined with) test specification @@ -744,6 +753,9 @@ {auto_compile, Bool}, {auto_compile, NodeRefs, Bool}, + {abort_if_missing_suites, Bool}, + {abort_if_missing_suites, NodeRefs, Bool}, + {config, ConfigFiles}. {config, ConfigDir, ConfigBaseNames}. {config, NodeRefs, ConfigFiles}. diff --git a/lib/common_test/src/ct.erl b/lib/common_test/src/ct.erl index e6732f7fc7..241cd928b7 100644 --- a/lib/common_test/src/ct.erl +++ b/lib/common_test/src/ct.erl @@ -150,7 +150,8 @@ run(TestDirs) -> %%% {silent_connections,Conns} | {stylesheet,CSSFile} | %%% {cover,CoverSpecFile} | {cover_stop,Bool} | {step,StepOpts} | %%% {event_handler,EventHandlers} | {include,InclDirs} | -%%% {auto_compile,Bool} | {create_priv_dir,CreatePrivDir} | +%%% {auto_compile,Bool} | {abort_if_missing_suites,Bool} | +%%% {create_priv_dir,CreatePrivDir} | %%% {multiply_timetraps,M} | {scale_timetraps,Bool} | %%% {repeat,N} | {duration,DurTime} | {until,StopTime} | %%% {force_stop,ForceStop} | {decrypt,DecryptKeyOrFile} | diff --git a/lib/common_test/src/ct_framework.erl b/lib/common_test/src/ct_framework.erl index 7d577462b0..9ef917a507 100644 --- a/lib/common_test/src/ct_framework.erl +++ b/lib/common_test/src/ct_framework.erl @@ -1274,7 +1274,7 @@ report(What,Data) -> ct_util:set_testdata({What,Data}), ok; tc_start -> - %% Data = {Suite,{Func,GroupName}},LogFileName} + %% Data = {{Suite,{Func,GroupName}},LogFileName} Data1 = case Data of {{Suite,{Func,undefined}},LFN} -> {{Suite,Func},LFN}; _ -> Data diff --git a/lib/common_test/src/ct_netconfc.erl b/lib/common_test/src/ct_netconfc.erl index 6fc840745d..a3861dc745 100644 --- a/lib/common_test/src/ct_netconfc.erl +++ b/lib/common_test/src/ct_netconfc.erl @@ -536,7 +536,7 @@ send(Client, SimpleXml) -> Client :: client(), SimpleXml :: simple_xml(), Timeout :: timeout(), - Result :: ok | {error,error_reason()}. + Result :: simple_xml() | {error,error_reason()}. %% @doc Send an XML document to the server. %% %% The given XML document is sent as is to the server. This function @@ -556,7 +556,7 @@ send_rpc(Client, SimpleXml) -> Client :: client(), SimpleXml :: simple_xml(), Timeout :: timeout(), - Result :: ok | {error,error_reason()}. + Result :: [simple_xml()] | {error,error_reason()}. %% @doc Send a Netconf <code>rpc</code> request to the server. %% %% The given XML document is wrapped in a valid Netconf @@ -635,7 +635,7 @@ get(Client, Filter) -> Client :: client(), Filter :: simple_xml() | xpath(), Timeout :: timeout(), - Result :: {ok,simple_xml()} | {error,error_reason()}. + Result :: {ok,[simple_xml()]} | {error,error_reason()}. %% @doc Get data. %% %% This operation returns both configuration and state data from the @@ -661,7 +661,7 @@ get_config(Client, Source, Filter) -> Source :: netconf_db(), Filter :: simple_xml() | xpath(), Timeout :: timeout(), - Result :: {ok,simple_xml()} | {error,error_reason()}. + Result :: {ok,[simple_xml()]} | {error,error_reason()}. %% @doc Get configuration data. %% %% To be able to access another source than `running', the server @@ -759,7 +759,7 @@ action(Client,Action) -> Client :: client(), Action :: simple_xml(), Timeout :: timeout(), - Result :: {ok,simple_xml()} | {error,error_reason()}. + Result :: {ok,[simple_xml()]} | {error,error_reason()}. %% @doc Execute an action. %% %% @end diff --git a/lib/common_test/src/ct_telnet.erl b/lib/common_test/src/ct_telnet.erl index bc22a99741..c9dc2338cd 100644 --- a/lib/common_test/src/ct_telnet.erl +++ b/lib/common_test/src/ct_telnet.erl @@ -46,100 +46,80 @@ %% %% == Logging == %% -%% `ct_telnet' can be configured to uses the `error_logger' for logging telnet -%% traffic. A special purpose error handler is implemented in -%% `ct_conn_log_h'. To use this error handler, add the `cth_conn_log' -%% hook in your test suite, e.g: -%% +%% The default logging behaviour of `ct_telnet' is to print information +%% to the test case HTML log about performed operations and commands +%% and their corresponding results. What won't be printed to the HTML log +%% are text strings sent from the telnet server that are not explicitly +%% received by means of a `ct_telnet' function such as `expect/3'. +%% `ct_telnet' may however be configured to use a special purpose event handler, +%% implemented in `ct_conn_log_h', for logging <b>all</b> telnet traffic. +%% To use this handler, you need to install a Common Test hook named +%% `cth_conn_log'. Example (using the test suite info function): %% %% ``` %% suite() -> -%% [{ct_hooks, [{cth_conn_log, [{conn_mod(),hook_options()}]}]}]. -%%''' +%% [{ct_hooks, [{cth_conn_log, [{conn_mod(),hook_options()}]}]}]. +%% ''' %% %% `conn_mod()' is the name of the common_test module implementing %% the connection protocol, i.e. `ct_telnet'. %% -%% The hook option `log_type' specifies the type of logging: -%% -%% <dl> -%% <dt>`raw'</dt> -%% <dd>The sent and received telnet data is logged to a separate -%% text file as is, without any formatting. A link to the file is -%% added to the test case HTML log.</dd> +%% The `cth_conn_log' hook performs unformatted logging of telnet data to +%% a separate text file. All telnet communication is captured and printed, +%% including arbitrary data sent from the server. The link to this text file +%% can be found on the top of the test case HTML log. %% -%% <dt>`html (default)'</dt> -%% <dd>The sent and received telnet traffic is pretty printed -%% directly in the test case HTML log.</dd> -%% -%% <dt>`silent'</dt> -%% <dd>Telnet traffic is not logged.</dd> -%% </dl> -%% -%% By default, all telnet traffic is logged in one single log -%% file. However, it is possible to have different connections logged -%% in separate files. To do this, use the hook option `hosts' and -%% list the names of the servers/connections that will be used in the -%% suite. Note that the connections must be named for this to work +%% By default, data for all telnet connections is logged in one common +%% file (named `default'), which might get messy e.g. if multiple telnet +%% sessions are running in parallel. It is therefore possible to create a +%% separate log file for each connection. To configure this, use the hook +%% option `hosts' and list the names of the servers/connections that will be +%% used in the suite. Note that the connections must be named for this to work %% (see the `open' function below). %% -%% The `hosts' option has no effect if `log_type' is set to `html' or -%% `silent'. +%% The hook option named `log_type' may be used to change the `cth_conn_log' +%% behaviour. The default value of this option is `raw', which results in the +%% behaviour described above. If the value is set to `html', all telnet +%% communication is printed to the test case HTML log instead. %% -%% The hook options can also be specified in a configuration file with -%% the configuration variable `ct_conn_log': +%% All `cth_conn_log' hook options described above can also be specified in +%% a configuration file with the configuration variable `ct_conn_log'. Example: %% %% ``` -%% {ct_conn_log,[{conn_mod(),hook_options()}]}. -%% ''' -%% -%% For example: -%% -%% ``` -%% {ct_conn_log,[{ct_telnet,[{log_type,raw}, -%% {hosts,[key_or_name()]}]}]} +%% {ct_conn_log, [{ct_telnet,[{log_type,raw}, +%% {hosts,[key_or_name()]}]}]} %% ''' %% %% <b>Note</b> that hook options specified in a configuration file -%% will overwrite any hardcoded hook options in the test suite. +%% will overwrite any hardcoded hook options in the test suite! %% -%% === Logging example 1 === +%% === Logging example === %% -%% The following `ct_hooks' statement will cause raw printing of -%% telnet traffic to separate logs for the connections named -%% `server1' and `server2'. Any other connections will be logged -%% to default telnet log. +%% The following `ct_hooks' statement will cause printing of telnet traffic +%% to separate logs for the connections named `server1' and `server2'. +%% Traffic for any other connections will be logged in the default telnet log. %% %% ``` %% suite() -> -%% [{ct_hooks, [{cth_conn_log, [{ct_telnet,[{log_type,raw}}, -%% {hosts,[server1,server2]}]} -%% ]}]}]. +%% [{ct_hooks, +%% [{cth_conn_log, [{ct_telnet,[{hosts,[server1,server2]}]}]}]}]. %%''' %% -%% === Logging example 2 === -%% -%% The following configuration file will cause raw logging of all -%% telnet traffic into one single text file. +%% As previously explained, the above specification could also be provided +%% by means of an entry like this in a configuration file: %% %% ``` -%% {ct_conn_log,[{ct_telnet,[{log_type,raw}]}]}. +%% {ct_conn_log, [{ct_telnet,[{hosts,[server1,server2]}]}]}. %% ''' %% -%% The `ct_hooks' statement must look like this: +%% in which case the `ct_hooks' statement in the test suite may simply look +%% like this: %% %% ``` %% suite() -> -%% [{ct_hooks, [{cth_conn_log, []}]}]. +%% [{ct_hooks, [{cth_conn_log, []}]}]. %% ''' %% -%% The same `ct_hooks' statement without the configuration file would -%% cause HTML logging of all telnet connections into the test case -%% HTML log. -%% -%% <b>Note</b> that if the `cth_conn_log' hook is not added, telnet -%% traffic is still logged in the test case HTML log file (on the legacy -%% `ct_telnet' format). %% @end %% @type connection_type() = telnet | ts1 | ts2 @@ -205,6 +185,7 @@ open(Name) -> %%% Name = target_name() %%% ConnType = ct_telnet:connection_type() %%% Handle = ct_telnet:handle() +%%% Reason = term() %%% %%% @doc Open a telnet connection to the specified target host. open(Name,ConnType) -> @@ -234,6 +215,7 @@ open(KeyOrName,ConnType,TargetMod) -> %%% TargetMod = atom() %%% Extra = term() %%% Handle = handle() +%%% Reason = term() %%% %%% @doc Open a telnet connection to the specified target host. %%% @@ -295,7 +277,8 @@ open(KeyOrName,ConnType,TargetMod,Extra) -> %%%----------------------------------------------------------------- %%% @spec close(Connection) -> ok | {error,Reason} -%%% Connection = ct_telnet:connection() +%%% Connection = ct_telnet:connection() +%%% Reason = term() %%% %%% @doc Close the telnet connection and stop the process managing it. %%% @@ -330,6 +313,7 @@ cmd(Connection,Cmd) -> %%% Cmd = string() %%% Timeout = integer() %%% Data = [string()] +%%% Reason = term() %%% @doc Send a command via telnet and wait for prompt. cmd(Connection,Cmd,Timeout) -> case get_handle(Connection) of @@ -350,6 +334,7 @@ cmdf(Connection,CmdFormat,Args) -> %%% Args = list() %%% Timeout = integer() %%% Data = [string()] +%%% Reason = term() %%% @doc Send a telnet command and wait for prompt %%% (uses a format string and list of arguments to build the command). cmdf(Connection,CmdFormat,Args,Timeout) when is_list(Args) -> @@ -360,6 +345,7 @@ cmdf(Connection,CmdFormat,Args,Timeout) when is_list(Args) -> %%% @spec get_data(Connection) -> {ok,Data} | {error,Reason} %%% Connection = ct_telnet:connection() %%% Data = [string()] +%%% Reason = term() %%% @doc Get all data which has been received by the telnet client %%% since last command was sent. get_data(Connection) -> @@ -374,6 +360,7 @@ get_data(Connection) -> %%% @spec send(Connection,Cmd) -> ok | {error,Reason} %%% Connection = ct_telnet:connection() %%% Cmd = string() +%%% Reason = term() %%% @doc Send a telnet command and return immediately. %%% %%% <p>The resulting output from the command can be read with @@ -391,6 +378,7 @@ send(Connection,Cmd) -> %%% Connection = ct_telnet:connection() %%% CmdFormat = string() %%% Args = list() +%%% Reason = term() %%% @doc Send a telnet command and return immediately (uses a format %%% string and a list of arguments to build the command). sendf(Connection,CmdFormat,Args) when is_list(Args) -> @@ -765,8 +753,8 @@ check_if_prompt_was_reached(Data,_) when is_list(Data) -> check_if_prompt_was_reached(_,_) -> false. -%%% @hidden -%% Functions for logging ct_telnet reports and telnet data +%%%----------------------------------------------------------------- +%%% Functions for logging ct_telnet reports and telnet data heading(Action,undefined) -> io_lib:format("~w ~w",[?MODULE,Action]); @@ -776,6 +764,8 @@ heading(Action,Name) -> force_log(State,Action,String,Args) -> log(State,Action,String,Args,true). +%%%----------------------------------------------------------------- +%%% @hidden log(State,Action,String,Args) when is_record(State, state) -> log(State,Action,String,Args,false); log(Name,Action,String,Args) when is_atom(Name) -> @@ -783,6 +773,8 @@ log(Name,Action,String,Args) when is_atom(Name) -> log(TelnPid,Action,String,Args) when is_pid(TelnPid) -> log(#state{teln_pid=TelnPid},Action,String,Args,false). +%%%----------------------------------------------------------------- +%%% @hidden log(undefined,String,Args) -> log(#state{},undefined,String,Args,false); log(Name,String,Args) when is_atom(Name) -> @@ -790,6 +782,8 @@ log(Name,String,Args) when is_atom(Name) -> log(TelnPid,String,Args) when is_pid(TelnPid) -> log(#state{teln_pid=TelnPid},undefined,String,Args). +%%%----------------------------------------------------------------- +%%% @hidden log(#state{name=Name,teln_pid=TelnPid,host=Host,port=Port}, Action,String,Args,ForcePrint) -> Name1 = if Name == undefined -> get({ct_telnet_pid2name,TelnPid}); @@ -839,6 +833,8 @@ log(#state{name=Name,teln_pid=TelnPid,host=Host,port=Port}, end end. +%%%----------------------------------------------------------------- +%%% @hidden start_gen_log(Heading) -> %% check if output is suppressed case ct_util:is_silenced(telnet) of @@ -846,6 +842,8 @@ start_gen_log(Heading) -> false -> ct_gen_conn:start_log(Heading) end. +%%%----------------------------------------------------------------- +%%% @hidden end_gen_log() -> %% check if output is suppressed case ct_util:is_silenced(telnet) of diff --git a/lib/common_test/src/cth_conn_log.erl b/lib/common_test/src/cth_conn_log.erl index 0e6c877c5d..1e60f2751e 100644 --- a/lib/common_test/src/cth_conn_log.erl +++ b/lib/common_test/src/cth_conn_log.erl @@ -91,12 +91,15 @@ merge_log_info([{Mod,ConfOpts}|ConfList],HookList) -> {value,{_,HookOpts},HL1} -> {ConfOpts ++ HookOpts, HL1} % ConfOpts overwrites HookOpts! end, - [{Mod,get_log_opts(Opts)} | merge_log_info(ConfList,HookList1)]; + [{Mod,get_log_opts(Mod,Opts)} | merge_log_info(ConfList,HookList1)]; merge_log_info([],HookList) -> - [{Mod,get_log_opts(Opts)} || {Mod,Opts} <- HookList]. + [{Mod,get_log_opts(Mod,Opts)} || {Mod,Opts} <- HookList]. -get_log_opts(Opts) -> - LogType = proplists:get_value(log_type,Opts,html), +get_log_opts(Mod,Opts) -> + DefaultLogType = if Mod == ct_telnet -> raw; + true -> html + end, + LogType = proplists:get_value(log_type,Opts,DefaultLogType), Hosts = proplists:get_value(hosts,Opts,[]), {LogType,Hosts}. diff --git a/lib/common_test/src/unix_telnet.erl b/lib/common_test/src/unix_telnet.erl index b05386a5ab..10666b979d 100644 --- a/lib/common_test/src/unix_telnet.erl +++ b/lib/common_test/src/unix_telnet.erl @@ -17,8 +17,8 @@ %% %CopyrightEnd% %% -%%% @doc Callback module for ct_telnet for talking telnet -%%% to a unix host. +%%% @doc Callback module for ct_telnet, for connecting to a telnet +%%% server on a unix host. %%% %%% <p>It requires the following entry in the config file:</p> %%% <pre> @@ -28,15 +28,15 @@ %%% {password,Password}, %%% {keep_alive,Bool}]}. % optional</pre> %%% -%%% <p>To talk telnet to the host specified by +%%% <p>To communicate via telnet to the host specified by %%% <code>HostNameOrIpAddress</code>, use the interface functions in -%%% <code>ct</code>, e.g. <code>open(Name), cmd(Name,Cmd), ...</code>.</p> +%%% <code>ct_telnet</code>, e.g. <code>open(Name), cmd(Name,Cmd), ...</code>.</p> %%% %%% <p><code>Name</code> is the name you allocated to the unix host in %%% your <code>require</code> statement. E.g.</p> -%%% <pre> suite() -> [{require,Name,{unix,[telnet,username,password]}}].</pre> +%%% <pre> suite() -> [{require,Name,{unix,[telnet]}}].</pre> %%% <p>or</p> -%%% <pre> ct:require(Name,{unix,[telnet,username,password]}).</pre> +%%% <pre> ct:require(Name,{unix,[telnet]}).</pre> %%% %%% <p>The "keep alive" activity (i.e. that Common Test sends NOP to the server %%% every 10 seconds if the connection is idle) may be enabled or disabled for one @@ -62,20 +62,18 @@ -define(prx,"login: |Password: |\\\$ |> "). %%%----------------------------------------------------------------- -%%% @hidden %%% @spec get_prompt_regexp() -> PromptRegexp %%% PromptRegexp = ct_telnet:prompt_regexp() %%% %%% @doc Callback for ct_telnet.erl. %%% -%%% <p>Return the prompt regexp for telnet connections to the -%%% interwatch instrument.</p> +%%% <p>Return a suitable regexp string that will match common +%%% prompts for users on unix hosts.</p> get_prompt_regexp() -> ?prx. %%%----------------------------------------------------------------- -%%% @hidden %%% @spec connect(ConnName,Ip,Port,Timeout,KeepAlive,Extra) -> %%% {ok,Handle} | {error,Reason} %%% ConnName = ct:target_name() @@ -83,14 +81,15 @@ get_prompt_regexp() -> %%% Port = integer() %%% Timeout = integer() %%% KeepAlive = bool() -%%% Extra = {Username,Password} +%%% Extra = ct:target_name() | {Username,Password} %%% Username = string() %%% Password = string() %%% Handle = ct_telnet:handle() +%%% Reason = term() %%% %%% @doc Callback for ct_telnet.erl. %%% -%%% <p>Setup telnet connection to a UNIX host.</p> +%%% <p>Setup telnet connection to a unix host.</p> connect(ConnName,Ip,Port,Timeout,KeepAlive,Extra) -> case Extra of {Username,Password} -> diff --git a/lib/common_test/test/ct_telnet_SUITE.erl b/lib/common_test/test/ct_telnet_SUITE.erl index f5cff76fd1..84e69c2b54 100644 --- a/lib/common_test/test/ct_telnet_SUITE.erl +++ b/lib/common_test/test/ct_telnet_SUITE.erl @@ -180,14 +180,20 @@ telnet_config(unix_telnet, legacy) -> {ct_conn_log,[]}]; %% LogType same as GroupName telnet_config(unix_telnet, LogType) -> + LogTypeTerm = if LogType == raw -> []; + true -> [{log_type,LogType}] + end, [{unix, ct:get_config(unix)}, {ct_conn_log, - [{ct_telnet,[{log_type,LogType}, - {hosts,[telnet_server_conn1, - telnet_server_conn2, - telnet_server_conn3, - telnet_server_conn4]}]}]}]; + [{ct_telnet, LogTypeTerm ++ + [{hosts,[telnet_server_conn1, + telnet_server_conn2, + telnet_server_conn3, + telnet_server_conn4]}]}]}]; telnet_config(_, LogType) -> + LogTypeTerm = if LogType == raw -> []; + true -> [{log_type,LogType}] + end, [{unix,[{telnet,"localhost"}, {port, ?erl_telnet_server_port}, {username,?erl_telnet_server_user}, @@ -202,11 +208,11 @@ telnet_config(_, LogType) -> [{ct_conn_log,[]}]; true -> [{ct_conn_log, - [{ct_telnet,[{log_type,LogType}, - {hosts,[telnet_server_conn1, - telnet_server_conn2, - telnet_server_conn3, - telnet_server_conn4]}]}]}] + [{ct_telnet, LogTypeTerm ++ + [{hosts,[telnet_server_conn1, + telnet_server_conn2, + telnet_server_conn3, + telnet_server_conn4]}]}]}] end]. %%%----------------------------------------------------------------- diff --git a/lib/compiler/src/beam_disasm.erl b/lib/compiler/src/beam_disasm.erl index 4bdfe4e0c2..c45596f236 100644 --- a/lib/compiler/src/beam_disasm.erl +++ b/lib/compiler/src/beam_disasm.erl @@ -1030,6 +1030,7 @@ resolve_inst({gc_bif2,Args},Imports,_,_) -> [F,Live,Bif,A1,A2,Reg] = resolve_args(Args), {extfunc,_Mod,BifName,_Arity} = lookup(Bif+1,Imports), {gc_bif,BifName,F,Live,[A1,A2],Reg}; + %% %% New instruction in R14, gc_bif with 3 arguments %% @@ -1146,21 +1147,17 @@ resolve_inst({put_map_assoc,Args},_,_,_) -> [FLbl,Src,Dst,{u,N},{{z,1},{u,_Len},List0}] = Args, List = resolve_args(List0), {put_map_assoc,FLbl,Src,Dst,N,{list,List}}; - resolve_inst({put_map_exact,Args},_,_,_) -> [FLbl,Src,Dst,{u,N},{{z,1},{u,_Len},List0}] = Args, List = resolve_args(List0), {put_map_exact,FLbl,Src,Dst,N,{list,List}}; - -resolve_inst({is_map,Args0},_,_,_) -> +resolve_inst({is_map=I,Args0},_,_,_) -> [FLbl|Args] = resolve_args(Args0), - {test, is_map, FLbl, Args}; - + {test,I,FLbl,Args}; resolve_inst({has_map_fields,Args0},_,_,_) -> [FLbl,Src,{{z,1},{u,_Len},List0}] = Args0, List = resolve_args(List0), {test,has_map_fields,FLbl,Src,{list,List}}; - resolve_inst({get_map_elements,Args0},_,_,_) -> [FLbl,Src,{{z,1},{u,_Len},List0}] = Args0, List = resolve_args(List0), diff --git a/lib/compiler/src/cerl.erl b/lib/compiler/src/cerl.erl index e400e4f185..ed11c8de4d 100644 --- a/lib/compiler/src/cerl.erl +++ b/lib/compiler/src/cerl.erl @@ -136,10 +136,6 @@ c_literal/0, c_map_pair/0, c_module/0, c_tuple/0, c_values/0, c_var/0, cerl/0, var_name/0]). -%% HiPE does not understand Maps -%% (guard functions is_map/1 and map_size/1 in ann_c_map/3) --compile(no_native). - -include("core_parse.hrl"). -type c_alias() :: #c_alias{}. diff --git a/lib/dialyzer/src/dialyzer_analysis_callgraph.erl b/lib/dialyzer/src/dialyzer_analysis_callgraph.erl index 2a633c5e37..6a33a2acb3 100644 --- a/lib/dialyzer/src/dialyzer_analysis_callgraph.erl +++ b/lib/dialyzer/src/dialyzer_analysis_callgraph.erl @@ -249,7 +249,7 @@ compile_and_store(Files, #analysis_state{codeserver = CServer, timing_server = Timing, parent = Parent} = State) -> send_log(Parent, "Reading files and computing callgraph... "), - {T1, _} = statistics(runtime), + {T1, _} = statistics(wall_clock), Callgraph = dialyzer_callgraph:new(), CompileInit = make_compile_init(State, Callgraph), {{Failed, NoWarn, Modules}, NextLabel} = @@ -272,13 +272,13 @@ compile_and_store(Files, #analysis_state{codeserver = CServer, [[Reason || {_Filename, Reason} <- Failed]]), exit({error, Msg}) end, - {T2, _} = statistics(runtime), + {T2, _} = statistics(wall_clock), Msg1 = io_lib:format("done in ~.2f secs\nRemoving edges... ", [(T2-T1)/1000]), send_log(Parent, Msg1), Callgraph = ?timing(Timing, "clean", _C2, cleanup_callgraph(State, CServer2, Callgraph, Modules)), - {T3, _} = statistics(runtime), + {T3, _} = statistics(wall_clock), Msg2 = io_lib:format("done in ~.2f secs\n", [(T3-T2)/1000]), send_log(Parent, Msg2), {Callgraph, sets:from_list(NoWarn), CServer2}. @@ -620,9 +620,9 @@ dump_callgraph(CallGraph, State, #analysis{callgraph_file = File} = Analysis) -> Extension = filename:extension(File), Start_Msg = io_lib:format("Dumping the callgraph... ", []), send_log(State#analysis_state.parent, Start_Msg), - {T1, _} = statistics(runtime), + {T1, _} = statistics(wall_clock), dump_callgraph(CallGraph, State, Analysis, Extension), - {T2, _} = statistics(runtime), + {T2, _} = statistics(wall_clock), Finish_Msg = io_lib:format("done in ~2f secs\n", [(T2-T1)/1000]), send_log(State#analysis_state.parent, Finish_Msg), ok. diff --git a/lib/dialyzer/src/dialyzer_cl.erl b/lib/dialyzer/src/dialyzer_cl.erl index e013d39a0e..3e7d9dfa99 100644 --- a/lib/dialyzer/src/dialyzer_cl.erl +++ b/lib/dialyzer/src/dialyzer_cl.erl @@ -504,9 +504,7 @@ hipe_compile(Files, #options{erlang_mode = ErlangMode} = Options) -> _ -> Mods = [lists, dict, digraph, digraph_utils, ets, gb_sets, gb_trees, ordsets, sets, sofs, - %cerl, % uses maps instructions - %erl_types, % uses maps instructions - cerl_trees, erl_bif_types, + cerl, erl_types, cerl_trees, erl_bif_types, dialyzer_analysis_callgraph, dialyzer, dialyzer_behaviours, dialyzer_codeserver, dialyzer_contracts, dialyzer_coordinator, dialyzer_dataflow, dialyzer_dep, diff --git a/lib/diameter/doc/src/diameter_make.xml b/lib/diameter/doc/src/diameter_make.xml index 13ec5bbfc1..0c7e6b794d 100644 --- a/lib/diameter/doc/src/diameter_make.xml +++ b/lib/diameter/doc/src/diameter_make.xml @@ -52,7 +52,7 @@ under the License. <p> The function &codec; is used to compile a diameter &dictionary; into Erlang source. -The resulting source implements the interface diameter required +The resulting source implements the interface diameter requires to encode and decode the dictionary's messages and AVPs.</p> <p> diff --git a/lib/diameter/doc/src/notes.xml b/lib/diameter/doc/src/notes.xml index cd14195e78..e2390df37c 100644 --- a/lib/diameter/doc/src/notes.xml +++ b/lib/diameter/doc/src/notes.xml @@ -11,7 +11,7 @@ <header> <copyright> <year>2011</year> -<year>2013</year> +<year>2014</year> <holder>Ericsson AB. All Rights Reserved.</holder> </copyright> <legalnotice> @@ -73,12 +73,6 @@ first.</p> <p> Own Id: OTP-11361</p> </item> - <item> - <p> - Fix silent make rules (Thanks to Anthony Ramine)</p> - <p> - Own Id: OTP-11514</p> - </item> </list> </section> diff --git a/lib/diameter/src/Makefile b/lib/diameter/src/Makefile index 578bbaee2e..9afccf298c 100644 --- a/lib/diameter/src/Makefile +++ b/lib/diameter/src/Makefile @@ -1,7 +1,7 @@ # # %CopyrightBegin% # -# Copyright Ericsson AB 2010-2013. All Rights Reserved. +# Copyright Ericsson AB 2010-2014. All Rights Reserved. # # The contents of this file are subject to the Erlang Public License, # Version 1.1, (the "License"); you may not use this file except in @@ -41,7 +41,7 @@ INCDIR = ../include ABS_EBIN := $(shell cd $(EBIN) && pwd) # Where make should look for dependencies. -VPATH = .:base:compiler:transport:gen +VPATH = .:base:compiler:transport:gen:info # ---------------------------------------------------- # Target specs @@ -55,13 +55,13 @@ DICT_ERLS = $(DICT_MODULES:%=%.erl) DICT_HRLS = $(DICT_MODULES:%=%.hrl) # Modules to build before compiling dictionaries. -COMPILER_MODULES = $(notdir $(filter compiler/%, $(CT_MODULES))) \ - $(DICT_YRL) +COMPILER_MODULES = $(notdir $(CT_MODULES)) $(DICT_YRL) # All handwritten modules from which a depend.mk is generated. MODULES = \ $(RT_MODULES) \ - $(CT_MODULES) + $(CT_MODULES) \ + $(INFO_MODULES) # Modules whose names are inserted into the app file. APP_MODULES = \ @@ -72,6 +72,7 @@ APP_MODULES = \ TARGET_MODULES = \ $(APP_MODULES) \ $(CT_MODULES) \ + $(INFO_MODULES) \ $(DICT_YRL:%=gen/%) # What to build for the 'opt' target. @@ -147,14 +148,19 @@ gen/$(DICT_YRL).erl: compiler/$(DICT_YRL).yrl $(ERLC) -Werror -o $(@D) $< # Generate the app file. -$(APP_TARGET): $(APP_SRC) ../vsn.mk modules.mk +$(APP_TARGET): $(APP_SRC) ../vsn.mk modules.mk app.sed $(gen_verbose) \ M=`echo $(notdir $(APP_MODULES)) | tr ' ' ,`; \ + C=`echo $(COMPILER_MODULES) | tr ' ' ,`; \ + I=`echo $(notdir $(INFO_MODULES)) | tr ' ' ,`; \ R=`echo $(REGISTERED) | tr ' ' ,`; \ sed -e 's;%VSN%;$(VSN);' \ -e "s;%MODULES%;$$M;" \ + -e "s;%COMPILER%;$$C;" \ + -e "s;%INFO%;$$I;" \ -e "s;%REGISTERED%;$$R;" \ - $< > $@ + $< \ + | sed -f app.sed > $@ $(APPUP_TARGET): $(APPUP_SRC) ../vsn.mk $(vsn_verbose) \ @@ -177,6 +183,8 @@ info: @echo @$(call list,CT_MODULES) @echo + @$(call list,INFO_MODULES) + @echo @$(call list,TARGET_MODULES) @echo @$(call list,TARGET_DIRS) @@ -216,7 +224,7 @@ dialyze: opt $(PLT) -Wno_improper_lists \ $(EBIN)/diameter_gen_base_rfc3588.$(EMULATOR) \ $(patsubst %, $(EBIN)/%.$(EMULATOR), \ - $(notdir $(RT_MODULES) $(CT_MODULES))) + $(notdir $(RT_MODULES) $(CT_MODULES) $(INFO_MODULES))) # Omit all but the common dictionary module since these # (diameter_gen_relay in particular) generate warning depending on how # much of the included diameter_gen.hrl they use. diff --git a/lib/diameter/src/app.sed b/lib/diameter/src/app.sed new file mode 100644 index 0000000000..7916f65002 --- /dev/null +++ b/lib/diameter/src/app.sed @@ -0,0 +1,40 @@ +# +# %CopyrightBegin% +# +# Copyright Ericsson AB 2014. All Rights Reserved. +# +# The contents of this file are subject to the Erlang Public License, +# Version 1.1, (the "License"); you may not use this file except in +# compliance with the License. You should have received a copy of the +# Erlang Public License along with this software. If not, it can be +# retrieved online at http://www.erlang.org/. +# +# Software distributed under the License is distributed on an "AS IS" +# basis, WITHOUT WARRANTY OF ANY KIND, either express or implied. See +# the License for the specific language governing rights and limitations +# under the License. +# +# %CopyrightEnd% + +# +# Generate runtime_dependencies from applications to avoid having to +# specify the same application more than once. +# + +/{runtime_dependencies,/b v +/{[-a-z]*, "[0-9.]*"}/!b +/{vsn,/b + +/%%/!H +s/{\([^,]*\)[^}]*}/\1/g +s/%%/%,/ +b + +:v + +p +x +s/\n// +s/%//g +s/\n */ /g +s/{\([^,]*\), "\([^"]*"\)}/"\1-\2/g diff --git a/lib/diameter/src/base/diameter_service.erl b/lib/diameter/src/base/diameter_service.erl index 1274e0fc48..8914992f17 100644 --- a/lib/diameter/src/base/diameter_service.erl +++ b/lib/diameter/src/base/diameter_service.erl @@ -1389,6 +1389,8 @@ pick_peer(Local, Remote, Pid, _SvcName, #diameter_app{mutable = true} = App) case call_service(Pid, {pick_peer, Local, Remote, App}) of {TPid, _} = T when is_pid(TPid) -> T; + false = No -> + No; {error, _} -> false end; diff --git a/lib/diameter/src/compiler/diameter_codegen.erl b/lib/diameter/src/compiler/diameter_codegen.erl index 22422f2ef2..5a068c1a25 100644 --- a/lib/diameter/src/compiler/diameter_codegen.erl +++ b/lib/diameter/src/compiler/diameter_codegen.erl @@ -1,7 +1,7 @@ %% %% %CopyrightBegin% %% -%% Copyright Ericsson AB 2010-2013. All Rights Reserved. +%% Copyright Ericsson AB 2010-2014. All Rights Reserved. %% %% The contents of this file are subject to the Erlang Public License, %% Version 1.1, (the "License"); you may not use this file except in @@ -31,7 +31,8 @@ %% on the beam file of another dictionary. %% --export([from_dict/4]). +-export([from_dict/4, + is_printable_ascii/1]). %% used by ?TERM/1 in diameter_forms.hrl -include("diameter_forms.hrl"). -include("diameter_vsn.hrl"). @@ -121,6 +122,9 @@ eraser(Key) -> %% =========================================================================== %% =========================================================================== +is_printable_ascii(C) -> + 16#20 =< C andalso C =< 16#7F. + get_value(Key, Plist) -> proplists:get_value(Key, Plist, []). diff --git a/lib/diameter/src/compiler/diameter_forms.hrl b/lib/diameter/src/compiler/diameter_forms.hrl index 9b14c1715a..dd03401b9e 100644 --- a/lib/diameter/src/compiler/diameter_forms.hrl +++ b/lib/diameter/src/compiler/diameter_forms.hrl @@ -1,7 +1,7 @@ %% %% %CopyrightBegin% %% -%% Copyright Ericsson AB 2010-2013. All Rights Reserved. +%% Copyright Ericsson AB 2010-2014. All Rights Reserved. %% %% The contents of this file are subject to the Erlang Public License, %% Version 1.1, (the "License"); you may not use this file except in @@ -57,4 +57,6 @@ -define(FIELDS(Fs), [{?record_field, ?ATOM(F), V} || {F,V} <- Fs]). %% Literal term. --define(TERM(T), erl_parse:abstract(T, ?LINE)). +-define(TERM(T), erl_parse:abstract(T, [ + {line, ?LINE}, + {encoding, fun diameter_codegen:is_printable_ascii/1}])). diff --git a/lib/diameter/src/compiler/diameter_make.erl b/lib/diameter/src/compiler/diameter_make.erl index adc7808e49..72f5d36da4 100644 --- a/lib/diameter/src/compiler/diameter_make.erl +++ b/lib/diameter/src/compiler/diameter_make.erl @@ -232,21 +232,29 @@ identify([Vsn | [T|_] = ParseD]) identify({path, File} = T) -> {T, File}; identify(File) -> - Bin = iolist_to_binary([File]), - case is_path(Bin) of + case is_path([File]) of true -> {{path, File}, File}; - false -> {Bin, ?DEFAULT_DICT_FILE} + false -> {File, ?DEFAULT_DICT_FILE} end. -%% Interpret anything containing \n or \r as a literal dictionary, -%% otherwise a path. (Which might be the wrong guess in the worst case.) -is_path(Bin) -> - try - [throw(C) || <<C>> <= Bin, $\n == C orelse $\r == C], - true - catch - throw:_ -> false - end. +%% Interpret anything containing \n or \r as a literal dictionary. + +is_path([<<C,B/binary>> | T]) -> + is_path([C, B | T]); + +is_path([[C|L] | T]) -> + is_path([C, L | T]); + +is_path([C|_]) + when $\n == C; + $\r == C -> + false; + +is_path([_|T]) -> + is_path(T); + +is_path([]) -> + true. make(File, Opts, Dict) -> ok(lists:foldl(fun(M,A) -> [make(File, Opts, Dict, M) | A] end, diff --git a/lib/diameter/src/diameter.app.src b/lib/diameter/src/diameter.app.src index 509de9e595..ac1d847753 100644 --- a/lib/diameter/src/diameter.app.src +++ b/lib/diameter/src/diameter.app.src @@ -1,7 +1,7 @@ %% %% %CopyrightBegin% %% -%% Copyright Ericsson AB 2010-2013. All Rights Reserved. +%% Copyright Ericsson AB 2010-2014. All Rights Reserved. %% %% The contents of this file are subject to the Erlang Public License, %% Version 1.1, (the "License"); you may not use this file except in @@ -20,11 +20,27 @@ {application, diameter, [{description, "Diameter protocol"}, {vsn, "%VSN%"}, - {modules, [%MODULES%]}, + {modules, [ + %MODULES% + %,%COMPILER% + %,%INFO% + ]}, {registered, [%REGISTERED%]}, - {applications, [stdlib, kernel]}, + {applications, [ + {stdlib, "2.0"}, {kernel, "3.0"}%, {erts, "6.0"} + %% {syntax-tools, "1.6.14"} + %% {runtime-tools, "1.8.14"} + %, {ssl, "5.3.4"} + ]}, {env, []}, {mod, {diameter_app, []}}, - {runtime_dependencies, ["syntax_tools-1.6.14","stdlib-2.0","ssl-5.3.4", - "runtime_tools-1.8.14","kernel-3.0","erts-6.0"]} + {runtime_dependencies, [ + ]} + %% + %% Note that ssl is only required if configured on TCP transports, + %% and syntax-tools and runtime-tools are only required if the + %% dictionary compiler and debug modules (respectively) are + %% needed/wanted at runtime, which they typically aren't. These + %% modules are the two commented lines in the 'modules' tuple. + %% ]}. diff --git a/lib/diameter/src/base/diameter_dbg.erl b/lib/diameter/src/info/diameter_dbg.erl index 5b0ac3a3b6..b5b3983afa 100644 --- a/lib/diameter/src/base/diameter_dbg.erl +++ b/lib/diameter/src/info/diameter_dbg.erl @@ -1,7 +1,7 @@ %% %% %CopyrightBegin% %% -%% Copyright Ericsson AB 2010-2011. All Rights Reserved. +%% Copyright Ericsson AB 2010-2014. All Rights Reserved. %% %% The contents of this file are subject to the Erlang Public License, %% Version 1.1, (the "License"); you may not use this file except in @@ -17,20 +17,22 @@ %% %CopyrightEnd% %% +%% +%% Information and debug functions. +%% + -module(diameter_dbg). -export([table/1, tables/0, fields/1, - help/0, modules/0, versions/0, version_info/0, compiled/0, procs/0, latest/0, - nl/0, - log/4]). + nl/0]). -export([diameter_config/0, diameter_peer/0, @@ -52,11 +54,9 @@ tp/1]). -include_lib("diameter/include/diameter.hrl"). --include("diameter_internal.hrl"). - --define(INFO, diameter_info). --define(SEP(), ?INFO:sep()). +-define(APP, diameter). +-define(I, diameter_info). -define(LOCAL, [diameter_config, diameter_peer, @@ -68,27 +68,16 @@ -define(VALUES(Rec), tl(tuple_to_list(Rec))). -log(_Slogan, _Mod, _Line, _Details) -> - ok. - -%%% ---------------------------------------------------------- -%%% # help() -%%% ---------------------------------------------------------- - -help() -> - not_yet_implemented. - -%%% ---------------------------------------------------------- -%%% # table(TableName) -%%% -%%% Input: TableName = diameter table containing record entries. -%%% -%%% Output: Count | undefined -%%% ---------------------------------------------------------- +%% ---------------------------------------------------------- +%% # table(TableName) +%% +%% Pretty-print a diameter table. Returns the number of records +%% printed, or undefined. +%% ---------------------------------------------------------- table(T) when (T == diameter_peer) orelse (T == diameter_reg) -> - ?INFO:format(collect(T), fields(T), fun ?INFO:split/2); + ?I:format(collect(T), fields(T), fun ?I:split/2); table(Table) when is_atom(Table) -> @@ -96,7 +85,7 @@ table(Table) undefined = No -> No; Fields -> - ?INFO:format(Table, Fields, fun split/2) + ?I:format(Table, Fields, fun split/2) end. split([started, name | Fs], [S, N | Vs]) -> @@ -107,9 +96,9 @@ split([[F|FT]|Fs], [Rec|Vs]) -> split([F|Fs], [V|Vs]) -> {F, Fs, V, Vs}. -%%% ---------------------------------------------------------- -%%% # TableName() -%%% ---------------------------------------------------------- +%% ---------------------------------------------------------- +%% # TableName() +%% ---------------------------------------------------------- -define(TABLE(Name), Name() -> table(Name)). @@ -121,16 +110,15 @@ split([F|Fs], [V|Vs]) -> ?TABLE(diameter_service). ?TABLE(diameter_stats). -%%% ---------------------------------------------------------- -%%% # tables() -%%% -%%% Output: Number of records output. -%%% -%%% Description: Pretty-print records in diameter tables from all nodes. -%%% ---------------------------------------------------------- +%% ---------------------------------------------------------- +%% # tables() +%% +%% Pretty-print diameter tables from all nodes. Returns the number of +%% records printed. +%% ---------------------------------------------------------- tables() -> - ?INFO:format(field(?LOCAL), fun split/3, fun collect/1). + ?I:format(field(?LOCAL), fun split/3, fun collect/1). field(Tables) -> lists:map(fun(T) -> {T, fields(T)} end, lists:sort(Tables)). @@ -138,66 +126,66 @@ field(Tables) -> split(_, Fs, Vs) -> split(Fs, Vs). -%%% ---------------------------------------------------------- -%%% # modules() -%%% ---------------------------------------------------------- +%% ---------------------------------------------------------- +%% # modules() +%% ---------------------------------------------------------- modules() -> - Path = filename:join([appdir(), atom_to_list(?APPLICATION) ++ ".app"]), - {ok, [{application, ?APPLICATION, Attrs}]} = file:consult(Path), + Path = filename:join([appdir(), atom_to_list(?APP) ++ ".app"]), + {ok, [{application, ?APP, Attrs}]} = file:consult(Path), {modules, Mods} = lists:keyfind(modules, 1, Attrs), Mods. appdir() -> - [_|_] = code:lib_dir(?APPLICATION, ebin). + [_|_] = code:lib_dir(?APP, ebin). -%%% ---------------------------------------------------------- -%%% # versions() -%%% ---------------------------------------------------------- +%% ---------------------------------------------------------- +%% # versions() +%% ---------------------------------------------------------- versions() -> - ?INFO:versions(modules()). + ?I:versions(modules()). -%%% ---------------------------------------------------------- -%%% # versions() -%%% ---------------------------------------------------------- +%% ---------------------------------------------------------- +%% # versions() +%% ---------------------------------------------------------- version_info() -> - ?INFO:version_info(modules()). + ?I:version_info(modules()). -%%% ---------------------------------------------------------- -%%% # compiled() -%%% ---------------------------------------------------------- +%% ---------------------------------------------------------- +%% # compiled() +%% ---------------------------------------------------------- compiled() -> - ?INFO:compiled(modules()). + ?I:compiled(modules()). -%%% ---------------------------------------------------------- -%%% procs() -%%% ---------------------------------------------------------- +%% ---------------------------------------------------------- +%% procs() +%% ---------------------------------------------------------- procs() -> - ?INFO:procs(?APPLICATION). + ?I:procs(?APP). -%%% ---------------------------------------------------------- -%%% # latest() -%%% ---------------------------------------------------------- +%% ---------------------------------------------------------- +%% # latest() +%% ---------------------------------------------------------- latest() -> - ?INFO:latest(modules()). + ?I:latest(modules()). -%%% ---------------------------------------------------------- -%%% # nl() -%%% ---------------------------------------------------------- +%% ---------------------------------------------------------- +%% # nl() +%% ---------------------------------------------------------- nl() -> lists:foreach(fun(M) -> abcast = c:nl(M) end, modules()). -%%% ---------------------------------------------------------- -%%% # pp(Bin) -%%% -%%% Description: Pretty-print a message binary. -%%% ---------------------------------------------------------- +%% ---------------------------------------------------------- +%% # pp(Bin) +%% +%% Description: Pretty-print a message binary. +%% ---------------------------------------------------------- %% Network byte order = big endian. @@ -207,7 +195,7 @@ pp(<<Version:8, MsgLength:24, HbHid:32, E2Eid:32, AVPs/binary>>) -> - ?SEP(), + ?I:sep(), ppp(["Version", "Message length", "[Actual length]", @@ -227,7 +215,7 @@ pp(<<Version:8, MsgLength:24, HbHid, E2Eid]), N = avp_loop({AVPs, MsgLength - 20}, 0), - ?SEP(), + ?I:sep(), N; pp(<<_Version:8, MsgLength:24, _/binary>> = Bin) -> @@ -328,23 +316,23 @@ ppp(Fields, Values) -> ppp({Field, Value}) -> io:format(": ~-22s : ~p~n", [Field, Value]). -%%% ---------------------------------------------------------- -%%% # subscriptions() -%%% -%%% Output: list of {SvcName, Pid} -%%% ---------------------------------------------------------- +%% ---------------------------------------------------------- +%% # subscriptions() +%% +%% Returns a list of {SvcName, Pid}. +%% ---------------------------------------------------------- subscriptions() -> diameter_service:subscriptions(). -%%% ---------------------------------------------------------- -%%% # children() -%%% ---------------------------------------------------------- +%% ---------------------------------------------------------- +%% # children() +%% ---------------------------------------------------------- children() -> diameter_sup:tree(). -%%% ---------------------------------------------------------- +%% ---------------------------------------------------------- %% tracer/[12] @@ -430,7 +418,7 @@ peers(Name, Ts) -> mk_peers(Name, [_, {type, connect} | _] = Ts) -> [[Name | mk_peer(Ts)]]; -mk_peers(Name, [R, {type, listen}, O, {accept = A, As}]) -> +mk_peers(Name, [R, {type, listen}, O, {accept = A, As} | _]) -> [[Name | mk_peer([R, {type, A}, O | Ts])] || Ts <- As]. %% This is a bit lame: service_info works to build this list and out %% of something like what we want here and then we take it apart. @@ -485,13 +473,12 @@ fields(diameter_service) -> [started, name, record_info(fields, diameter_service), + watchdogT, peerT, - connT, - share_peers, - use_shared_peers, shared_peers, local_peers, - monitor]; + monitor, + options]; ?FIELDS(diameter_event); ?FIELDS(diameter_uri); diff --git a/lib/diameter/src/base/diameter_info.erl b/lib/diameter/src/info/diameter_info.erl index 39d32d07cd..10972f3231 100644 --- a/lib/diameter/src/base/diameter_info.erl +++ b/lib/diameter/src/info/diameter_info.erl @@ -1,7 +1,7 @@ %% %% %CopyrightBegin% %% -%% Copyright Ericsson AB 2010-2011. All Rights Reserved. +%% Copyright Ericsson AB 2010-2014. All Rights Reserved. %% %% The contents of this file are subject to the Erlang Public License, %% Version 1.1, (the "License"); you may not use this file except in @@ -17,6 +17,11 @@ %% %CopyrightEnd% %% +%% +%% Generic functions for formatting table listings and more. Used by +%% diameter_dbg. +%% + -module(diameter_info). -export([usage/1, @@ -573,12 +578,7 @@ sys_info() -> {A,V}. os_info() -> - {os:version(), case os:type() of - {_Fam, _Name} = T -> - T; - Fam -> - {Fam, ""} - end}. + {os:version(), os:type()}. chomp(S) -> string:strip(S, right, $\n). diff --git a/lib/diameter/src/modules.mk b/lib/diameter/src/modules.mk index f8d3cf1d6f..a2a7a51892 100644 --- a/lib/diameter/src/modules.mk +++ b/lib/diameter/src/modules.mk @@ -1,8 +1,7 @@ -#-*-makefile-*- ; force emacs to enter makefile-mode # %CopyrightBegin% # -# Copyright Ericsson AB 2010-2013. All Rights Reserved. +# Copyright Ericsson AB 2010-2014. All Rights Reserved. # # The contents of this file are subject to the Erlang Public License, # Version 1.1, (the "License"); you may not use this file except in @@ -64,16 +63,19 @@ RT_MODULES = \ transport/diameter_transport \ transport/diameter_transport_sup -# Handwritten (compile time) modules not included in the app file. +# Handwritten compiler modules not included in the app file. CT_MODULES = \ - base/diameter_dbg \ - base/diameter_info \ compiler/diameter_codegen \ compiler/diameter_exprecs \ compiler/diameter_dict_scanner \ compiler/diameter_dict_util \ compiler/diameter_make +# Info/debug modules, also not included in the app file. +INFO_MODULES = \ + info/diameter_dbg \ + info/diameter_info + # Released hrl files in ../include intended for public consumption. EXTERNAL_HRLS = \ diameter.hrl \ diff --git a/lib/diameter/test/diameter_app_SUITE.erl b/lib/diameter/test/diameter_app_SUITE.erl index 1e262895a6..f68a18b5c2 100644 --- a/lib/diameter/test/diameter_app_SUITE.erl +++ b/lib/diameter/test/diameter_app_SUITE.erl @@ -1,7 +1,7 @@ %% %% %CopyrightBegin% %% -%% Copyright Ericsson AB 2010-2013. All Rights Reserved. +%% Copyright Ericsson AB 2010-2014. All Rights Reserved. %% %% The contents of this file are subject to the Erlang Public License, %% Version 1.1, (the "License"); you may not use this file except in @@ -50,7 +50,7 @@ diameter_exprecs, diameter_make]). --define(HELP_MODULES, [diameter_dbg, +-define(INFO_MODULES, [diameter_dbg, diameter_info]). %% =========================================================================== @@ -99,13 +99,13 @@ vsn(Config) -> %% # modules/1 %% %% Ensure that the app file modules and installed modules differ by -%% compiler/help modules. +%% compiler/info modules. %% =========================================================================== modules(Config) -> Mods = fetch(modules, fetch(app, Config)), Installed = code_mods(), - Help = lists:sort(?HELP_MODULES ++ ?COMPILER_MODULES), + Help = lists:sort(?INFO_MODULES ++ ?COMPILER_MODULES), {[], Help} = {Mods -- Installed, lists:sort(Installed -- Mods)}. @@ -158,12 +158,15 @@ appvsn(Name) -> %% # xref/1 %% %% Ensure that no function in our application calls an undefined function -%% or one in an application we haven't specified as a dependency. (Almost.) +%% or one in an application we haven't declared as a dependency. (Almost.) %% =========================================================================== xref(Config) -> App = fetch(app, Config), - Mods = fetch(modules, App), + Mods = fetch(modules, App), %% modules listed in the app file + + %% List of application names extracted from runtime_dependencies. + Deps = lists:map(fun unversion/1, fetch(runtime_dependencies, App)), {ok, XRef} = xref:start(make_name(xref_test_name)), ok = xref:set_default(XRef, [{verbose, false}, {warnings, false}]), @@ -178,7 +181,9 @@ xref(Config) -> [?APP, erts | fetch(applications, App)]), {ok, Undefs} = xref:analyze(XRef, undefined_function_calls), - {ok, Called} = xref:analyze(XRef, {module_call, ?COMPILER_MODULES}), + {ok, RTmods} = xref:analyze(XRef, {module_use, Mods}), + {ok, CTmods} = xref:analyze(XRef, {module_use, ?COMPILER_MODULES}), + {ok, RTdeps} = xref:analyze(XRef, {module_call, Mods}), xref:stop(XRef), @@ -190,18 +195,41 @@ xref(Config) -> Undefs), %% diameter_tcp does call ssl despite the latter not being listed %% as a dependency in the app file since ssl is only required for - %% TLS security: it's up to a client who wants TLS it to start - %% ssl. - - [] = lists:filter(fun is_bad_dependency/1, Called). - -%% It's not strictly necessary that diameter compiler modules not -%% depend on other diameter modules but it's a simple source of build -%% errors if not encoded in the makefile (hence the test) so guard -%% against it. -is_bad_dependency(Mod) -> - lists:prefix("diameter", atom_to_list(Mod)) - andalso not lists:member(Mod, ?COMPILER_MODULES). + %% TLS security: it's up to a client who wants TLS to start ssl. + + %% Ensure that only runtime or info modules call runtime modules. + %% It's not strictly necessary that diameter compiler modules not + %% depend on other diameter modules but it's a simple source of + %% build errors if not properly encoded in the makefile so guard + %% against it. + [] = (RTmods -- Mods) -- ?INFO_MODULES, + + %% Ensure that runtime modules don't call compiler modules. + CTmods = CTmods -- Mods, + + %% Ensure that runtime modules only call other runtime modules, or + %% applications declared as in runtime_dependencies in the app + %% file. Note that the declared application versions are ignored + %% since we only know what we can see now. + [] = lists:filter(fun(M) -> not lists:member(app(M), Deps) end, + RTdeps -- Mods). + +unversion(App) -> + T = lists:dropwhile(fun is_vsn_ch/1, lists:reverse(App)), + lists:reverse(case T of [$-|TT] -> TT; _ -> T end). + +is_vsn_ch(C) -> + $0 =< C andalso C =< $9 orelse $. == C. + +app('$M_EXPR') -> %% could be anything but assume it's ok + "erts"; +app(Mod) -> + case code:which(Mod) of + preloaded -> + "erts"; + Path -> + unversion(lists:nth(3, lists:reverse(filename:split(Path)))) + end. add_application(XRef, App) -> add_application(XRef, App, code:lib_dir(App)). diff --git a/lib/diameter/test/diameter_compiler_SUITE.erl b/lib/diameter/test/diameter_compiler_SUITE.erl index df4dde6240..08ffe5981d 100644 --- a/lib/diameter/test/diameter_compiler_SUITE.erl +++ b/lib/diameter/test/diameter_compiler_SUITE.erl @@ -120,6 +120,16 @@ {avp_has_duplicate_flag, " -", " MM"}, + {ok, + "@vendor 0", + "@vendor 10415"}, + {ok, + [{"@vendor 0", "@vendor 10415"}, + {"Proxy-Info .*M$", "&V"}, + {"Proxy-Info ::= [^>]*", "& 10415 "}]}, + {grouped_vendor_id_without_flag, + [{"@vendor 0", "@vendor 10415"}, + {"Proxy-Info ::= [^>]*", "& 10415 "}]}, {avp_has_vendor_id, "@avp_types", "@avp_vendor_id 667 Class\n&"}, diff --git a/lib/edoc/src/edoc_types.erl b/lib/edoc/src/edoc_types.erl index af8f1230fb..d4e00d3ecd 100644 --- a/lib/edoc/src/edoc_types.erl +++ b/lib/edoc/src/edoc_types.erl @@ -66,6 +66,7 @@ is_new_predefined(boolean, 0) -> true; is_new_predefined(byte, 0) -> true; is_new_predefined(iodata, 0) -> true; is_new_predefined(iolist, 0) -> true; +is_new_predefined(map, 0) -> true; is_new_predefined(maybe_improper_list, 0) -> true; is_new_predefined(maybe_improper_list, 2) -> true; is_new_predefined(mfa, 0) -> true; diff --git a/lib/hipe/cerl/erl_types.erl b/lib/hipe/cerl/erl_types.erl index 28281a2fac..47b8dc766a 100644 --- a/lib/hipe/cerl/erl_types.erl +++ b/lib/hipe/cerl/erl_types.erl @@ -126,6 +126,8 @@ t_is_instance/2, t_is_integer/1, t_is_integer/2, t_is_list/1, + t_is_map/1, + t_is_map/2, t_is_matchstate/1, t_is_nil/1, t_is_nil/2, t_is_non_neg_integer/1, @@ -148,6 +150,8 @@ t_list/1, t_list_elements/1, t_list_elements/2, t_list_termination/1, + t_map/0, + t_map/1, t_matchstate/0, t_matchstate/2, t_matchstate_present/1, @@ -208,20 +212,12 @@ lift_list_to_pos_empty/1, is_opaque_type/2, is_erl_type/1, - atom_to_string/1, - - t_is_map/2, - t_map/1, - t_map/0 + atom_to_string/1 ]). %%-define(DO_ERL_TYPES_TEST, true). -compile({no_auto_import,[min/2,max/2]}). -%% HiPE does not understand Maps -%% (guard function is_map/1 in t_from_term/1) --compile(no_native). - -ifdef(DO_ERL_TYPES_TEST). -export([test/0]). -else. @@ -1733,11 +1729,16 @@ lift_list_to_pos_empty(?list(Content, Termination, _)) -> t_map() -> ?map([]). --spec t_map([{erl_type(),erl_type()}]) -> erl_type(). +-spec t_map([{erl_type(), erl_type()}]) -> erl_type(). t_map(_) -> ?map([]). +-spec t_is_map(erl_type()) -> boolean(). + +t_is_map(Type) -> + t_is_map(Type, 'universe'). + -spec t_is_map(erl_type(), opaques()) -> boolean(). t_is_map(Type, Opaques) -> @@ -1746,7 +1747,6 @@ t_is_map(Type, Opaques) -> is_map1(?map(_)) -> true; is_map1(_) -> false. - %%----------------------------------------------------------------------------- %% Tuples %% @@ -2167,10 +2167,10 @@ t_from_term(T) when is_function(T) -> {arity, Arity} = erlang:fun_info(T, arity), t_fun(Arity, t_any()); t_from_term(T) when is_integer(T) -> t_integer(T); +t_from_term(T) when is_map(T) -> t_map(); t_from_term(T) when is_pid(T) -> t_pid(); t_from_term(T) when is_port(T) -> t_port(); t_from_term(T) when is_reference(T) -> t_reference(); -t_from_term(T) when is_map(T) -> t_map(); t_from_term(T) when is_tuple(T) -> t_tuple([t_from_term(E) || E <- tuple_to_list(T)]). @@ -2570,7 +2570,7 @@ force_union(T = ?function(_, _)) -> ?function_union(T); force_union(T = ?identifier(_)) -> ?identifier_union(T); force_union(T = ?list(_, _, _)) -> ?list_union(T); force_union(T = ?nil) -> ?list_union(T); -force_union(T = ?number(_,_)) -> ?number_union(T); +force_union(T = ?number(_, _)) -> ?number_union(T); force_union(T = ?opaque(_)) -> ?opaque_union(T); force_union(T = ?remote(_)) -> ?remote_union(T); force_union(T = ?map(_)) -> ?map_union(T); @@ -3581,6 +3581,8 @@ t_subtract(?product(Elements1) = T1, ?product(Elements2)) -> _ -> T1 end end; +t_subtract(?map(_) = T, _) -> % XXX: very crude; will probably need refinement + T; t_subtract(?product(P1), _) -> ?product(P1); t_subtract(T, ?product(_)) -> diff --git a/lib/hipe/icode/hipe_beam_to_icode.erl b/lib/hipe/icode/hipe_beam_to_icode.erl index 81249c958e..dcd547fd5f 100644 --- a/lib/hipe/icode/hipe_beam_to_icode.erl +++ b/lib/hipe/icode/hipe_beam_to_icode.erl @@ -509,6 +509,10 @@ trans_fun([{test,test_arity,{f,Lbl},[Reg,N]}|Instructions], Env) -> I = hipe_icode:mk_type([trans_arg(Reg)],{tuple,N}, hipe_icode:label_name(True),map_label(Lbl)), [I,True | trans_fun(Instructions,Env)]; +%%--- is_map --- +trans_fun([{test,is_map,{f,Lbl},[Arg]}|Instructions], Env) -> + {Code,Env1} = trans_type_test(map,Lbl,Arg,Env), + [Code | trans_fun(Instructions,Env1)]; %%-------------------------------------------------------------------- %%--- select_val --- trans_fun([{select_val,Reg,{f,Lbl},{list,Cases}}|Instructions], Env) -> diff --git a/lib/hipe/icode/hipe_icode.hrl b/lib/hipe/icode/hipe_icode.hrl index 25deac5152..46c04beb40 100644 --- a/lib/hipe/icode/hipe_icode.hrl +++ b/lib/hipe/icode/hipe_icode.hrl @@ -1,7 +1,7 @@ %% %% %CopyrightBegin% %% -%% Copyright Ericsson AB 2004-2013. All Rights Reserved. +%% Copyright Ericsson AB 2004-2014. All Rights Reserved. %% %% The contents of this file are subject to the Erlang Public License, %% Version 1.1, (the "License"); you may not use this file except in @@ -61,8 +61,8 @@ | 'op_exact_eqeq_2' | 'suspend_msg_timeout'. -type icode_type_test() :: 'atom' | 'bignum' | 'binary' | 'bitstr' | 'boolean' - | 'cons' | 'fixnum' | 'float' - | 'function' | 'function2' | 'integer' | 'list' | 'nil' + | 'cons' | 'fixnum' | 'float' | 'function' + | 'function2' | 'integer' | 'list' | 'map' | 'nil' | 'number' | 'pid' | 'port' | 'reference' | 'tuple' | {'atom', atom()} | {'integer', integer()} | {'record', atom(), non_neg_integer()} @@ -108,7 +108,6 @@ length :: non_neg_integer(), cases :: [icode_switch_case()]}). - -record(icode_type, {test :: icode_type_test(), args :: [icode_term_arg()], true_label :: icode_lbl(), diff --git a/lib/hipe/icode/hipe_icode_type.erl b/lib/hipe/icode/hipe_icode_type.erl index 65876b83ea..ebeb5e2c10 100644 --- a/lib/hipe/icode/hipe_icode_type.erl +++ b/lib/hipe/icode/hipe_icode_type.erl @@ -84,15 +84,15 @@ t_fun/0, t_fun/1, t_fun/2, t_fun_args/1, t_fun_arity/1, t_inf/2, t_inf_lists/2, t_integer/0, t_integer/1, t_is_atom/1, t_is_any/1, - t_is_binary/1, t_is_bitstr/1, t_is_bitwidth/1, t_is_boolean/1, - t_is_fixnum/1, t_is_cons/1, + t_is_binary/1, t_is_bitstr/1, t_is_bitwidth/1, + t_is_boolean/1, t_is_fixnum/1, t_is_cons/1, t_is_map/1, t_is_maybe_improper_list/1, t_is_equal/2, t_is_float/1, t_is_fun/1, t_is_integer/1, t_is_non_neg_integer/1, t_is_number/1, t_is_matchstate/1, - t_is_nil/1, t_is_none/1, t_is_port/1, t_is_pid/1, + t_is_none/1, t_is_port/1, t_is_pid/1, t_is_reference/1, t_is_subtype/2, t_is_tuple/1, t_limit/2, t_matchstate_present/1, t_matchstate/0, - t_matchstate_slots/1, t_maybe_improper_list/0, + t_matchstate_slots/1, t_maybe_improper_list/0, t_map/0, t_nil/0, t_none/0, t_number/0, t_number/1, t_number_vals/1, t_pid/0, t_port/0, t_reference/0, t_subtract/2, t_sup/2, t_to_tlist/1, t_tuple/0, t_tuple/1, t_tuple_sizes/1]). @@ -213,7 +213,7 @@ analyse_blocks(Work, State, MFA) -> {NewState, NewLabels} = try analyse_block(Label, Info, State) catch throw:none_type -> - %% io:format("received none type at label: ~p~n",[Label]), + %% io:format("received none type at label: ~p~n", [Label]), {State,[]} end, NewWork2 = add_work(NewWork, NewLabels), @@ -265,7 +265,7 @@ analyse_insn(I, Info, LookupFun) -> do_move(I, Info); #icode_call{} -> NewInfo = do_call(I, Info, LookupFun), - %%io:format("Analysing Call: ~w~n~w~n", [I,NewInfo]), + %% io:format("Analysing Call: ~w~n~w~n", [I, NewInfo]), update_call_arguments(I, NewInfo); #icode_phi{} -> Type = t_limit(join_list(hipe_icode:args(I), Info), ?TYPE_DEPTH), @@ -788,16 +788,16 @@ test_record(Atom, Size, Var, VarInfo, TrueLab, FalseLab, Info) -> end. test_type(Test, Type) -> - %%io:format("Test is: ~w\n", [Test]), - %%io:format("Type is: ~s\n", [format_type(Type)]), + %% io:format("Test is: ~w\n", [Test]), + %% io:format("Type is: ~s\n", [format_type(Type)]), Ans = case t_is_any(Type) of true -> maybe; false -> TrueTest = true_branch_info(Test), Inf = t_inf(TrueTest, Type), - %%io:format("TrueTest is: ~s\n", [format_type(TrueTest)]), - %%io:format("Inf is: ~s\n", [format_type(Inf)]), + %% io:format("TrueTest is: ~s\n", [format_type(TrueTest)]), + %% io:format("Inf is: ~s\n", [format_type(Inf)]), case t_is_equal(Type, Inf) of true -> not t_is_none(Type); @@ -895,11 +895,12 @@ test_type0(boolean, T) -> t_is_boolean(T); test_type0(list, T) -> t_is_maybe_improper_list(T); -test_type0(cons, T) -> - t_is_cons(T); -test_type0(nil, T) -> - t_is_nil(T). - +%% test_type0(cons, T) -> +%% t_is_cons(T); +%% test_type0(nil, T) -> +%% t_is_nil(T). +test_type0(map, T) -> + t_is_map(T). true_branch_info(integer) -> t_integer(); @@ -931,22 +932,24 @@ true_branch_info(reference) -> t_reference(); true_branch_info(function) -> t_fun(); -true_branch_info(cons) -> - t_cons(); -true_branch_info(nil) -> - t_nil(); +%% true_branch_info(cons) -> +%% t_cons(); +%% true_branch_info(nil) -> +%% t_nil(); true_branch_info(boolean) -> t_boolean(); +true_branch_info(map) -> + t_map(); true_branch_info(T) -> - exit({?MODULE,unknown_typetest,T}). + exit({?MODULE, unknown_typetest, T}). %% _________________________________________________________________ %% %% Remove the redundant type tests. If a test is removed, the trace -%% that isn't taken is explicitly removed from the CFG to simpilify +%% that isn't taken is explicitly removed from the CFG to simplify %% the handling of Phi nodes. If a Phi node is left and at least one -%% branch into it has disappeared, the SSA propagation pass can't +%% branch into it has disappeared, the SSA propagation pass cannot %% handle it. %% %% If the CFG has changed at the end of this pass, the analysis is diff --git a/lib/hipe/llvm/hipe_llvm_main.erl b/lib/hipe/llvm/hipe_llvm_main.erl index e911fb89c9..0e50c9539b 100644 --- a/lib/hipe/llvm/hipe_llvm_main.erl +++ b/lib/hipe/llvm/hipe_llvm_main.erl @@ -342,8 +342,8 @@ create_sdesc_list([{ExnLbl, SPOff} | MoreExnAndSPOffs], %% (thus, some of their arguments are passed to the stack). Because of the %% Reserved Call Frame feature that the LLVM uses, the stack descriptors %% are not correct since at the point of call the frame size is reduced -%% proportionally to the number of arguments that are passed on the stack. -%% Also the offsets of the roots need to be re-adjusted. +%% by the number of arguments that are passed on the stack. Also, the +%% offsets of the roots need to be re-adjusted. fix_stack_descriptors(_, _, [], _) -> []; fix_stack_descriptors(RelocsDict, Relocs, SDescs, ExposedClosures) -> @@ -427,30 +427,22 @@ find_offsets([{Off,Arity}|Rest], Offsets, Acc) -> [I | RestOffsets] = lists:dropwhile(fun (Y) -> Y<Off end, Offsets), find_offsets(Rest, RestOffsets, [{I, Arity}|Acc]). -%% The functions below correct the arity of calls, that are identified -%% by offset, in the stack descriptors. +%% The function below corrects the stack descriptors of calls with arguments +%% that are passed on the stack (more than NR_ARG_REGS) by subtracting the +%% number of stacked arguments from the frame size and from the offset of the +%% roots. fix_sdescs([], SDescs) -> SDescs; fix_sdescs([{Offset, Arity} | Rest], SDescs) -> case lists:keyfind(Offset, 2, SDescs) of false -> fix_sdescs(Rest, SDescs); - {?SDESC, Offset, SDesc} -> - {ExnHandler, FrameSize, StkArity, Roots} = SDesc, - DecRoot = fun(X) -> X-Arity end, - NewRootsList = lists:map(DecRoot, tuple_to_list(Roots)), - NewSDesc = - case length(NewRootsList) > 0 andalso hd(NewRootsList) >= 0 of - true -> - {?SDESC, Offset, {ExnHandler, FrameSize-Arity, StkArity, - list_to_tuple(NewRootsList)}}; - false -> - {?SDESC, Offset, {ExnHandler, FrameSize, StkArity, Roots}} - end, - RestSDescs = lists:keydelete(Offset, 2, SDescs), - fix_sdescs(Rest, [NewSDesc | RestSDescs]) + {?SDESC, Offset, {ExnHandler, FrameSize, StkArity, Roots}} -> + FixedRoots = list_to_tuple([Ri - Arity || Ri <- tuple_to_list(Roots)]), + FixedSDesc = + {?SDESC, Offset, {ExnHandler, FrameSize - Arity, StkArity, FixedRoots}}, + fix_sdescs(Rest, [FixedSDesc | lists:keydelete(Offset, 2, SDescs)]) end. - %%------------------------------------------------------------------------------ %% Miscellaneous functions %%------------------------------------------------------------------------------ diff --git a/lib/hipe/llvm/hipe_rtl_to_llvm.erl b/lib/hipe/llvm/hipe_rtl_to_llvm.erl index ba76e1d815..d7d8d1b049 100644 --- a/lib/hipe/llvm/hipe_rtl_to_llvm.erl +++ b/lib/hipe/llvm/hipe_rtl_to_llvm.erl @@ -430,12 +430,13 @@ trans_call_name(RtlCallName, Relocs, CallArgs, FinalArgs) -> case RtlCallName of PrimOp when is_atom(PrimOp) -> LlvmName = trans_prim_op(PrimOp), - Relocs1 = relocs_store(LlvmName, {call, {bif, PrimOp, length(CallArgs)}}, - Relocs), + Relocs1 = + relocs_store(LlvmName, {call, {bif, PrimOp, length(CallArgs)}}, Relocs), {"@" ++ LlvmName, [], Relocs1}; {M, F, A} when is_atom(M), is_atom(F), is_integer(A) -> - LlvmName = trans_mfa_name({M,F,A}), - Relocs1 = relocs_store(LlvmName, {call, {M,F,A}}, Relocs), + LlvmName = trans_mfa_name({M, F, A}), + Relocs1 = + relocs_store(LlvmName, {call, {M, F, length(CallArgs)}}, Relocs), {"@" ++ LlvmName, [], Relocs1}; Reg -> case hipe_rtl:is_reg(Reg) of diff --git a/lib/hipe/main/hipe.erl b/lib/hipe/main/hipe.erl index d47eced6d8..539ce883c0 100644 --- a/lib/hipe/main/hipe.erl +++ b/lib/hipe/main/hipe.erl @@ -200,6 +200,7 @@ compile_core/4, file/1, file/2, + llvm_support_available/0, load/1, help/0, help_hiper/0, @@ -648,7 +649,18 @@ run_compiler_1(DisasmFun, IcodeFun, Options) -> %% The full option expansion is not done %% until the DisasmFun returns. {Code, CompOpts} = DisasmFun(Options), - Opts = expand_options(Options ++ CompOpts), + Opts0 = expand_options(Options ++ CompOpts), + Opts = + case proplists:get_bool(to_llvm, Opts0) andalso + not llvm_support_available() of + true -> + ?error_msg("No LLVM version 3.4 or greater " + "found in $PATH; aborting " + "native code compilation.\n", []), + ?EXIT(cant_find_required_llvm_version); + false -> + Opts0 + end, check_options(Opts), ?when_option(verbose, Options, ?debug_msg("Options: ~p.\n",[Opts])), @@ -1537,4 +1549,22 @@ check_options(Opts) -> ok end. +-spec llvm_support_available() -> boolean(). + +llvm_support_available() -> + get_llvm_version() >= 3.4. + +get_llvm_version() -> + OptStr = os:cmd("opt -version"), + SubStr = "LLVM version ", N = length(SubStr), + case string:str(OptStr, SubStr) of + 0 -> % No opt available + 0.0; + S -> + case string:to_float(string:sub_string(OptStr, S + N)) of + {error, _} -> 0.0; %XXX: Assumes no revision numbers in versioning + {Float, _} -> Float + end + end. + %%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%% diff --git a/lib/hipe/rtl/hipe_icode2rtl.erl b/lib/hipe/rtl/hipe_icode2rtl.erl index 6ab40adcc8..483d0b37f7 100644 --- a/lib/hipe/rtl/hipe_icode2rtl.erl +++ b/lib/hipe/rtl/hipe_icode2rtl.erl @@ -437,6 +437,8 @@ gen_type_test([X], Type, TrueLbl, FalseLbl, Pred, ConstTab) -> {hipe_tagscheme:test_integer(X, TrueLbl, FalseLbl, Pred), ConstTab}; list -> {hipe_tagscheme:test_list(X, TrueLbl, FalseLbl, Pred), ConstTab}; + map -> + {hipe_tagscheme:test_map(X, TrueLbl, FalseLbl, Pred), ConstTab}; nil -> {hipe_tagscheme:test_nil(X, TrueLbl, FalseLbl, Pred), ConstTab}; number -> diff --git a/lib/hipe/rtl/hipe_tagscheme.erl b/lib/hipe/rtl/hipe_tagscheme.erl index 4725889d8d..c27c682915 100644 --- a/lib/hipe/rtl/hipe_tagscheme.erl +++ b/lib/hipe/rtl/hipe_tagscheme.erl @@ -39,7 +39,7 @@ test_tuple/4, test_atom/4, test_bignum/4, test_pos_bignum/4, test_any_pid/4, test_any_port/4, test_ref/4, test_fun/4, test_fun2/5, test_matchstate/4, - test_binary/4, test_bitstr/4, test_list/4, + test_binary/4, test_bitstr/4, test_list/4, test_map/4, test_integer/4, test_number/4, test_tuple_N/5]). -export([realtag_fixnum/2, tag_fixnum/2, realuntag_fixnum/2, untag_fixnum/2]). -export([test_two_fixnums/3, test_fixnums/4, unsafe_fixnum_add/3, @@ -112,13 +112,15 @@ -define(TAG_HEADER_EXTERNAL_PID, ((16#C bsl ?TAG_PRIMARY_SIZE) bor ?TAG_PRIMARY_HEADER)). -define(TAG_HEADER_EXTERNAL_PORT,((16#D bsl ?TAG_PRIMARY_SIZE) bor ?TAG_PRIMARY_HEADER)). -define(TAG_HEADER_EXTERNAL_REF, ((16#E bsl ?TAG_PRIMARY_SIZE) bor ?TAG_PRIMARY_HEADER)). +-define(TAG_HEADER_MAP, ((16#F bsl ?TAG_PRIMARY_SIZE) bor ?TAG_PRIMARY_HEADER)). -define(TAG_HEADER_MASK, 16#3F). -define(HEADER_ARITY_OFFS, 6). %%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%% -mk_header(SZ,TAG) -> (SZ bsl ?HEADER_ARITY_OFFS) + TAG. +mk_header(SZ, TAG) -> (SZ bsl ?HEADER_ARITY_OFFS) + TAG. + mk_arityval(SZ) -> mk_header(SZ, ?TAG_HEADER_ARITYVAL). size_from_header(Sz, Header) -> @@ -132,9 +134,9 @@ mk_var_header(Header, Size, Tag) -> mk_fixnum(X) -> (X bsl ?TAG_IMMED1_SIZE) + ?TAG_IMMED1_SMALL. -define(NIL, ((-1 bsl ?TAG_IMMED2_SIZE) bor ?TAG_IMMED2_NIL)). -mk_nil() -> ?NIL. -%% mk_atom(X) -> (X bsl ?TAG_IMMED2_SIZE) + ?TAG_IMMED2_ATOM. -mk_non_value() -> ?THE_NON_VALUE. +mk_nil() -> ?NIL. +%% mk_atom(X) -> (X bsl ?TAG_IMMED2_SIZE) + ?TAG_IMMED2_ATOM. +mk_non_value() -> ?THE_NON_VALUE. -spec is_fixnum(integer()) -> boolean(). is_fixnum(N) when is_integer(N) -> @@ -252,6 +254,15 @@ test_tuple_N(X, N, TrueLab, FalseLab, Pred) -> hipe_rtl:mk_branch(Tmp, 'eq', hipe_rtl:mk_imm(mk_arityval(N)), TrueLab, FalseLab, Pred)]. +test_map(X, TrueLab, FalseLab, Pred) -> + Tmp = hipe_rtl:mk_new_reg_gcsafe(), + HalfTrueLab = hipe_rtl:mk_new_label(), + MapMask = ?TAG_HEADER_MASK, + [test_is_boxed(X, hipe_rtl:label_name(HalfTrueLab), FalseLab, Pred), + HalfTrueLab, + get_header(Tmp, X), + mask_and_compare(Tmp, MapMask, ?TAG_HEADER_MAP, TrueLab, FalseLab, Pred)]. + test_ref(X, TrueLab, FalseLab, Pred) -> Hdr = hipe_rtl:mk_new_reg_gcsafe(), Tag = hipe_rtl:mk_new_reg_gcsafe(), diff --git a/lib/hipe/test/Makefile b/lib/hipe/test/Makefile index cedb150b5d..acb2849d0d 100644 --- a/lib/hipe/test/Makefile +++ b/lib/hipe/test/Makefile @@ -8,6 +8,10 @@ include $(ERL_TOP)/make/$(TARGET)/otp.mk MODULES= \ hipe_SUITE +# .erl files for these modules are automatically generated +GEN_MODULES= \ + bs_SUITE + ERL_FILES= $(MODULES:%=%.erl) TARGET_FILES= $(MODULES:%=$(EBIN)/%.$(EMULATOR)) @@ -40,6 +44,8 @@ EBIN = . make_emakefile: $(ERL_TOP)/make/make_emakefile $(ERL_COMPILE_FLAGS) -o$(EBIN) $(MODULES) \ > $(EMAKEFILE) + $(ERL_TOP)/make/make_emakefile $(ERL_COMPILE_FLAGS) -o$(EBIN) $(GEN_MODULES) \ + >> $(EMAKEFILE) $(ERL_TOP)/make/make_emakefile $(ERL_COMPILE_FLAGS) -o$(EBIN) '*_SUITE_make' \ >> $(EMAKEFILE) diff --git a/lib/hipe/test/hipe_testsuite_driver.erl b/lib/hipe/test/hipe_testsuite_driver.erl index c8fdf1600c..5f05a716bc 100644 --- a/lib/hipe/test/hipe_testsuite_driver.erl +++ b/lib/hipe/test/hipe_testsuite_driver.erl @@ -85,7 +85,7 @@ list_testcases(Dirname) -> list_dir(Dir, Extension, Dirs) -> case file:list_dir(Dir) of - {error, _} = Error-> Error; + {error, _} = Error -> Error; {ok, Filenames} -> FullFilenames = [filename:join(Dir, F) || F <- Filenames], Matches1 = case Dirs of @@ -173,10 +173,29 @@ run(TestCase, Dir, _OutDir) -> %% end, DataFiles), %% try ok = TestCase:test(), - HiPEOpts = try TestCase:hipe_options() catch _:_ -> [] end, + HiPEOpts = try TestCase:hipe_options() catch error:undef -> [] end, {ok, TestCase} = hipe:c(TestCase, HiPEOpts), - ok = TestCase:test(). + ok = TestCase:test(), + case is_llvm_opt_available() of + true -> + {ok, TestCase} = hipe:c(TestCase, [to_llvm|HiPEOpts]), + ok = TestCase:test(); + false -> ok + end. %% after %% lists:foreach(fun (DF) -> ok end, % = file:delete(DF) end, %% [filename:join(OutDir, D) || D <- DataFiles]) %% end. + + +%% This function, which is supposed to check whether the right LLVM +%% infrastructure is available, should be probably written in a better +%% and more portable way and moved to the hipe application. + +is_llvm_opt_available() -> + OptStr = os:cmd("opt -version"), + SubStr = "LLVM version ", N = length(SubStr), + case string:str(OptStr, SubStr) of + 0 -> false; + S -> P = S + N, string:sub_string(OptStr, P, P + 2) >= "3.4" + end. diff --git a/lib/inets/src/http_server/httpd_util.erl b/lib/inets/src/http_server/httpd_util.erl index b0b18b9c3d..0d04a75205 100644 --- a/lib/inets/src/http_server/httpd_util.erl +++ b/lib/inets/src/http_server/httpd_util.erl @@ -1,7 +1,7 @@ %% %% %CopyrightBegin% %% -%% Copyright Ericsson AB 1997-2011. All Rights Reserved. +%% Copyright Ericsson AB 1997-2014. All Rights Reserved. %% %% The contents of this file are subject to the Erlang Public License, %% Version 1.1, (the "License"); you may not use this file except in @@ -206,9 +206,6 @@ message(413, Reason,_) -> "Entity: " ++ html_encode(Reason); message(414,ReasonPhrase,_) -> "Message " ++ html_encode(ReasonPhrase) ++ "."; -message(416,ReasonPhrase,_) -> - html_encode(ReasonPhrase); - message(500,_,ConfigDB) -> ServerAdmin=lookup(ConfigDB,server_admin,"unknown@unknown"), "The server encountered an internal error or " @@ -233,7 +230,9 @@ message(501,{Method, RequestURI, HTTPVersion}, _ConfigDB) -> end; message(503, String, _ConfigDB) -> - "This service in unavailable due to: " ++ html_encode(String). + "This service in unavailable due to: " ++ html_encode(String); +message(_, ReasonPhrase, _) -> + html_encode(ReasonPhrase). maybe_encode(URI) -> Decoded = try http_uri:decode(URI) of diff --git a/lib/inets/test/httpd_SUITE.erl b/lib/inets/test/httpd_SUITE.erl index 3eb8a0818f..4be20d3a69 100644 --- a/lib/inets/test/httpd_SUITE.erl +++ b/lib/inets/test/httpd_SUITE.erl @@ -168,7 +168,12 @@ init_per_group(http_1_1, Config) -> init_per_group(http_1_0, Config) -> [{http_version, "HTTP/1.0"} | Config]; init_per_group(http_0_9, Config) -> - [{http_version, "HTTP/0.9"} | Config]; + case {os:type(), os:version()} of + {{win32, _}, {5,1,2600}} -> + {skip, "eaddrinuse XP problem"}; + _ -> + [{http_version, "HTTP/0.9"} | Config] + end; init_per_group(http_htaccess = Group, Config) -> Path = ?config(doc_root, Config), catch remove_htaccess(Path), @@ -1099,7 +1104,7 @@ do_max_clients(Config) -> BlockRequest = http_request("GET /eval?httpd_example:delay(2000) ", Version, Host), {ok, Socket} = inets_test_lib:connect_bin(Type, Host, Port, transport_opts(Type, Config)), inets_test_lib:send(Type, Socket, BlockRequest), - ct:sleep(100), + ct:sleep(100), %% Avoid possible timing issues ok = httpd_test_lib:verify_request(Type, Host, Port, transport_opts(Type, Config), @@ -1112,6 +1117,7 @@ do_max_clients(Config) -> ok end, inets_test_lib:close(Type, Socket), + ct:sleep(100), %% Avoid possible timing issues ok = httpd_test_lib:verify_request(Type, Host, Port, transport_opts(Type, Config), diff --git a/lib/kernel/doc/src/application.xml b/lib/kernel/doc/src/application.xml index 016151891c..7664fda4db 100644 --- a/lib/kernel/doc/src/application.xml +++ b/lib/kernel/doc/src/application.xml @@ -4,7 +4,7 @@ <erlref> <header> <copyright> - <year>1996</year><year>2013</year> + <year>1996</year><year>2014</year> <holder>Ericsson AB. All Rights Reserved.</holder> </copyright> <legalnotice> @@ -459,8 +459,7 @@ Nodes = [cp1@cave, {cp2@cave, cp3@cave}]</code> <name>Module:start(StartType, StartArgs) -> {ok, Pid} | {ok, Pid, State} | {error, Reason}</name> <fsummary>Start an application</fsummary> <type> - <v>StartType = normal | {takeover,Node} | {failover,Node}</v> - <v> Node = node()</v> + <v>StartType = <seealso marker="#type-start_type">start_type()</seealso></v> <v>StartArgs = term()</v> <v>Pid = pid()</v> <v>State = term()</v> diff --git a/lib/kernel/src/application.erl b/lib/kernel/src/application.erl index 76a80553b0..c4bef5188a 100644 --- a/lib/kernel/src/application.erl +++ b/lib/kernel/src/application.erl @@ -30,6 +30,8 @@ -export([get_application/0, get_application/1, info/0]). -export([start_type/0]). +-export_type([start_type/0]). + %%%----------------------------------------------------------------- -type start_type() :: 'normal' @@ -58,8 +60,7 @@ %%------------------------------------------------------------------ --callback start(StartType :: normal | {takeover, node()} | {failover, node()}, - StartArgs :: term()) -> +-callback start(StartType :: start_type(), StartArgs :: term()) -> {'ok', pid()} | {'ok', pid(), State :: term()} | {'error', Reason :: term()}. -callback stop(State :: term()) -> diff --git a/lib/public_key/doc/src/cert_records.xml b/lib/public_key/doc/src/cert_records.xml index 79e5cb219d..397c13b463 100644 --- a/lib/public_key/doc/src/cert_records.xml +++ b/lib/public_key/doc/src/cert_records.xml @@ -5,7 +5,7 @@ <header> <copyright> <year>2008</year> - <year>2013</year> + <year>2014</year> <holder>Ericsson AB, All Rights Reserved</holder> </copyright> <legalnotice> @@ -39,7 +39,7 @@ The intent is to describe the data types and not to specify the meaning of each component for this we refer you to <url href="http://www.ietf.org/rfc/rfc5280.txt">RFC 5280</url> and - <url href="http://www.rsa.com/rsalabs/node.asp?id=2124">PKCS-10</url>. + <url href="http://www.ietf.org/rfc/rfc5967.txt">PKCS-10</url>. </p> <p>Use the following include directive to get access to the diff --git a/lib/public_key/doc/src/part.xml b/lib/public_key/doc/src/part.xml index 51e628aa90..73146c8e2a 100644 --- a/lib/public_key/doc/src/part.xml +++ b/lib/public_key/doc/src/part.xml @@ -5,7 +5,7 @@ <header> <copyright> <year>2008</year> - <year>2013</year> + <year>2014</year> <holder>Ericsson AB, All Rights Reserved</holder> </copyright> <legalnotice> @@ -34,7 +34,7 @@ <p> This application provides an API to public key infrastructure from <url href="http://www.ietf.org/rfc/rfc5280.txt">RFC 5280</url> (X.509 certificates) and public key formats defined by - the <url href="http://www.rsa.com/rsalabs/node.asp?id=2124"> + the <url href="http://en.wikipedia.org/wiki/PKCS"> PKCS-standard</url></p> </description> <xi:include href="introduction.xml"/> diff --git a/lib/public_key/doc/src/public_key.xml b/lib/public_key/doc/src/public_key.xml index fc3479cb64..172a2faba0 100644 --- a/lib/public_key/doc/src/public_key.xml +++ b/lib/public_key/doc/src/public_key.xml @@ -5,7 +5,7 @@ <header> <copyright> <year>2008</year> - <year>2013</year> + <year>2014</year> <holder>Ericsson AB, All Rights Reserved</holder> </copyright> <legalnotice> @@ -48,12 +48,12 @@ <item>Supports <url href="http://www.ietf.org/rfc/rfc5280.txt">RFC 5280 </url> - Internet X.509 Public Key Infrastructure Certificate and Certificate Revocation List (CRL) Profile </item> - <item>Supports <url href="http://www.rsa.com/rsalabs/node.asp?id=2125"> PKCS-1 </url> - RSA Cryptography Standard </item> + <item>Supports <url href="http://www.ietf.org/rfc/rfc3447.txt"> PKCS-1 </url> - RSA Cryptography Standard </item> <item>Supports <url href="http://csrc.nist.gov/publications/fips/fips186-3/fips_186-3.pdf"> DSS</url>- Digital Signature Standard (DSA - Digital Signature Algorithm)</item> - <item>Supports <url href="http://www.rsa.com/rsalabs/node.asp?id=2126"> PKCS-3 </url> - Diffie-Hellman Key Agreement Standard </item> - <item>Supports <url href="http://www.rsa.com/rsalabs/node.asp?id=2127"> PKCS-5</url> - Password-Based Cryptography Standard </item> - <item>Supports <url href="http://www.rsa.com/rsalabs/node.asp?id=2130"> PKCS-8</url> - Private-Key Information Syntax Standard</item> - <item>Supports <url href="http://www.rsa.com/rsalabs/node.asp?id=2132"> PKCS-10</url> - Certification Request Syntax Standard</item> + <item>Supports <url href="http://www.emc.com/emc-plus/rsa-labs/standards-initiatives/pkcs-3-diffie-hellman-key-agreement-standar.htm"> PKCS-3 </url> - Diffie-Hellman Key Agreement Standard </item> + <item>Supports <url href="http://www.ietf.org/rfc/rfc2898.txt"> PKCS-5</url> - Password-Based Cryptography Standard </item> + <item>Supports <url href="http://www.ietf.org/rfc/rfc5208.txt"> PKCS-8</url> - Private-Key Information Syntax Standard</item> + <item>Supports <url href="http://www.ietf.org/rfc/rfc5967.txt"> PKCS-10</url> - Certification Request Syntax Standard</item> </list> </section> @@ -461,11 +461,14 @@ <p>The fun should be defined as:</p> <code> -fun(OtpCert :: #'OTPCertificate'{}, Event :: {bad_cert, Reason :: atom()} | - {extension, #'Extension'{}}, +fun(OtpCert :: #'OTPCertificate'{}, + Event :: {bad_cert, Reason :: atom()} | + {extension, #'Extension'{}}, InitialUserState :: term()) -> - {valid, UserState :: term()} | {valid_peer, UserState :: term()} | - {fail, Reason :: term()} | {unknown, UserState :: term()}. + {valid, UserState :: term()} | + {valid_peer, UserState :: term()} | + {fail, Reason :: term()} | + {unknown, UserState :: term()}. </code> <p>If the verify callback fun returns {fail, Reason}, the @@ -511,7 +514,8 @@ fun(OtpCert :: #'OTPCertificate'{}, Event :: {bad_cert, Reason :: atom()} | <item> <p>The fun has the following type spec:</p> - <code> fun(#'DistributionPoint'{}, #'CertificateList'{}) -> #'CertificateList'{}</code> + <code> fun(#'DistributionPoint'{}, #'CertificateList'{}) -> + #'CertificateList'{}</code> <p>The fun should use the information in the distribution point to acesses the lates possible version of the CRL. If this fun is not specified @@ -519,6 +523,21 @@ fun(OtpCert :: #'OTPCertificate'{}, Event :: {bad_cert, Reason :: atom()} | </p> <code> fun(_DP, CRL) -> CRL end</code> </item> + + <tag>{issuer_fun, fun()}</tag> + <item> + <p>The fun has the following type spec:</p> + + <code> +fun(#'DistributionPoint'{}, #'CertificateList'{}, + {rdnSequence,[#'AttributeTypeAndValue'{}]}, term()) -> + {ok, #'OTPCertificate'{}, [der_encoded]}</code> + + <p>The fun should return the Root certificate and certificate chain + that has signed the CRL. + </p> + <code> fun(DP, CRL, Issuer, UserState) -> {ok, RootCert, CertChain}</code> + </item> </taglist> </desc> </func> diff --git a/lib/ssl/doc/src/ssl.xml b/lib/ssl/doc/src/ssl.xml index 4bc1a9a644..ffee4bd1af 100644 --- a/lib/ssl/doc/src/ssl.xml +++ b/lib/ssl/doc/src/ssl.xml @@ -768,39 +768,45 @@ fun(srp, Username :: string(), UserState :: term()) -> </func> <func> - <name>ssl_accept(ListenSocket) -> </name> - <name>ssl_accept(ListenSocket, Timeout) -> ok | {error, Reason}</name> - <fsummary>Perform server-side SSL handshake</fsummary> + <name>ssl_accept(Socket) -> </name> + <name>ssl_accept(Socket, Timeout) -> ok | {error, Reason}</name> + <fsummary>Perform server-side SSL/TLS handshake</fsummary> <type> - <v>ListenSocket = sslsocket()</v> + <v>Socket = sslsocket()</v> <v>Timeout = integer()</v> <v>Reason = term()</v> </type> <desc> - <p>The <c>ssl_accept</c> function establish the SSL connection - on the server side. It should be called directly after - <c>transport_accept</c>, in the spawned server-loop.</p> + <p> Performs the SSL/TLS server-side handshake <c>Socket</c> is a socket as returned + by <seealso + marker="#transport_accept-2">ssl:transport_accept/[1,2]</seealso> + </p> </desc> </func> <func> - <name>ssl_accept(ListenSocket, SslOptions) -> </name> - <name>ssl_accept(ListenSocket, SslOptions, Timeout) -> {ok, Socket} | {error, Reason}</name> - <fsummary>Perform server-side SSL handshake</fsummary> + <name>ssl_accept(Socket, SslOptions) -> </name> + <name>ssl_accept(Socket, SslOptions, Timeout) -> {ok, Socket} | ok | {error, Reason}</name> + <fsummary>Perform server-side SSL/TLS handshake</fsummary> <type> - <v>ListenSocket = socket()</v> + <v>Socket = socket() | sslsocket() </v> <v>SslOptions = ssloptions()</v> <v>Timeout = integer()</v> <v>Reason = term()</v> </type> <desc> - <p> Upgrades a gen_tcp, or - equivalent, socket to an ssl socket i.e. performs the - ssl server-side handshake.</p> + <p> If <c>Socket</c> is a socket() - upgrades a gen_tcp, or equivalent, socket to an ssl socket + i.e. performs the SSL/TLS server-side handshake and returns the ssl socket. + </p> + <warning><p>Note that the listen socket should be in {active, false} mode before telling the client that the server is ready to upgrade - and calling this function, otherwise the upgrade may + by calling this function, otherwise the upgrade may or may not succeed depending on timing.</p></warning> + + <p> If <c>Socket</c> is an sslsocket() - provides additional SSL/TLS options to those specified in <seealso + marker="#listen-2">ssl:listen/2 </seealso> and then performs the SSL/TLS handshake. + </p> </desc> </func> @@ -842,33 +848,38 @@ fun(srp, Username :: string(), UserState :: term()) -> </func> <func> - <name>transport_accept(Socket) -></name> - <name>transport_accept(Socket, Timeout) -> + <name>transport_accept(ListenSocket) -></name> + <name>transport_accept(ListenSocket, Timeout) -> {ok, NewSocket} | {error, Reason}</name> <fsummary>Accept an incoming connection and prepare for <c>ssl_accept</c></fsummary> <type> - <v>Socket = NewSocket = sslsocket()</v> + <v>ListenSocket = NewSocket = sslsocket()</v> <v>Timeout = integer()</v> <v>Reason = reason()</v> </type> <desc> <p>Accepts an incoming connection request on a listen socket. - <c>ListenSocket</c> must be a socket returned from - <c>listen/2</c>. The socket returned should be passed to - <c>ssl_accept</c> to complete ssl handshaking and - establishing the connection.</p> + <c>ListenSocket</c> must be a socket returned from + <seealso + marker="#listen-2"> ssl:listen/2</seealso>. + The socket returned should be passed to + <seealso marker="#ssl_accept-2"> ssl:ssl_accept[2,3]</seealso> + to complete handshaking i.e + establishing the SSL/TLS connection.</p> <warning> - <p>The socket returned can only be used with <c>ssl_accept</c>, - no traffic can be sent or received before that call.</p> + <p>The socket returned can only be used with + <seealso marker="#ssl_accept-2"> ssl:ssl_accept[2,3]</seealso> + no traffic can be sent or received before that call.</p> </warning> <p>The accepted socket inherits the options set for - <c>ListenSocket</c> in <c>listen/2</c>.</p> + <c>ListenSocket</c> in <seealso + marker="#listen-2"> ssl:listen/2</seealso>.</p> <p>The default - value for <c>Timeout</c> is <c>infinity</c>. If - <c>Timeout</c> is specified, and no connection is accepted - within the given time, <c>{error, timeout}</c> is - returned.</p> + value for <c>Timeout</c> is <c>infinity</c>. If + <c>Timeout</c> is specified, and no connection is accepted + within the given time, <c>{error, timeout}</c> is + returned.</p> </desc> </func> diff --git a/lib/ssl/src/ssl.erl b/lib/ssl/src/ssl.erl index a88bf45293..743753bf7d 100644 --- a/lib/ssl/src/ssl.erl +++ b/lib/ssl/src/ssl.erl @@ -195,7 +195,8 @@ transport_accept(#sslsocket{pid = {ListenSocket, -spec ssl_accept(#sslsocket{} | port(), timeout()| [ssl_option() | transport_option()]) -> ok | {ok, #sslsocket{}} | {error, reason()}. --spec ssl_accept(port(), [ssl_option()| transport_option()], timeout()) -> + +-spec ssl_accept(#sslsocket{} | port(), [ssl_option()] | [ssl_option()| transport_option()], timeout()) -> {ok, #sslsocket{}} | {error, reason()}. %% %% Description: Performs accept on an ssl listen socket. e.i. performs @@ -210,6 +211,15 @@ ssl_accept(#sslsocket{} = Socket, Timeout) -> ssl_accept(ListenSocket, SslOptions) when is_port(ListenSocket) -> ssl_accept(ListenSocket, SslOptions, infinity). +ssl_accept(#sslsocket{} = Socket, [], Timeout) -> + ssl_accept(#sslsocket{} = Socket, Timeout); +ssl_accept(#sslsocket{} = Socket, SslOptions, Timeout) -> + try + {ok, #config{ssl = SSL}} = handle_options(SslOptions, server), + ssl_connection:handshake(Socket, SSL, Timeout) + catch + Error = {error, _Reason} -> Error + end; ssl_accept(Socket, SslOptions, Timeout) when is_port(Socket) -> {Transport,_,_,_} = proplists:get_value(cb_info, SslOptions, {gen_tcp, tcp, tcp_closed, tcp_error}), diff --git a/lib/ssl/src/ssl_connection.erl b/lib/ssl/src/ssl_connection.erl index ed9e4d344f..c2810a199f 100644 --- a/lib/ssl/src/ssl_connection.erl +++ b/lib/ssl/src/ssl_connection.erl @@ -36,7 +36,7 @@ -include_lib("public_key/include/public_key.hrl"). %% Setup --export([connect/8, ssl_accept/7, handshake/2, +-export([connect/8, ssl_accept/7, handshake/2, handshake/3, socket_control/4]). %% User Events @@ -100,6 +100,20 @@ handshake(#sslsocket{pid = Pid}, Timeout) -> Error -> Error end. + +%%-------------------------------------------------------------------- +-spec handshake(#sslsocket{}, #ssl_options{}, timeout()) -> ok | {error, reason()}. +%% +%% Description: Starts ssl handshake with some new options +%%-------------------------------------------------------------------- +handshake(#sslsocket{pid = Pid}, SslOptions, Timeout) -> + case sync_send_all_state_event(Pid, {start, SslOptions, Timeout}) of + connected -> + ok; + Error -> + Error + end. + %-------------------------------------------------------------------- -spec socket_control(tls_connection | dtls_connection, port(), pid(), atom()) -> {ok, #sslsocket{}} | {error, reason()}. @@ -650,6 +664,10 @@ handle_sync_event({start, Timeout}, StartFrom, StateName, State) -> {next_state, StateName, State#state{start_or_recv_from = StartFrom, timer = Timer}, get_timeout(State)}; +handle_sync_event({start, Opts, Timeout}, From, StateName, #state{ssl_options = SslOpts} = State) -> + NewOpts = new_ssl_options(Opts, SslOpts), + handle_sync_event({start, Timeout}, From, StateName, State#state{ssl_options = NewOpts}); + handle_sync_event(close, _, StateName, #state{protocol_cb = Connection} = State) -> %% Run terminate before returning %% so that the reuseaddr inet-option will work @@ -1855,3 +1873,14 @@ make_premaster_secret({MajVer, MinVer}, rsa) -> <<?BYTE(MajVer), ?BYTE(MinVer), Rand/binary>>; make_premaster_secret(_, _) -> undefined. + +%% One day this can be maps instead, but we have to be backwards compatible for now +new_ssl_options(New, Old) -> + new_ssl_options(tuple_to_list(New), tuple_to_list(Old), []). + +new_ssl_options([], [], Acc) -> + list_to_tuple(lists:reverse(Acc)); +new_ssl_options([undefined | Rest0], [Head1| Rest1], Acc) -> + new_ssl_options(Rest0, Rest1, [Head1 | Acc]); +new_ssl_options([Head0 | Rest0], [_| Rest1], Acc) -> + new_ssl_options(Rest0, Rest1, [Head0 | Acc]). diff --git a/lib/ssl/src/ssl_internal.hrl b/lib/ssl/src/ssl_internal.hrl index cec5d8fbb1..8bf5b30a83 100644 --- a/lib/ssl/src/ssl_internal.hrl +++ b/lib/ssl/src/ssl_internal.hrl @@ -101,7 +101,6 @@ reuse_sessions :: boolean(), renegotiate_at, secure_renegotiate, - debug, %% undefined if not hibernating, or number of ms of %% inactivity after which ssl_connection will go into %% hibernation diff --git a/lib/ssl/test/make_certs.erl b/lib/ssl/test/make_certs.erl index c438ae2b87..0947657ca7 100644 --- a/lib/ssl/test/make_certs.erl +++ b/lib/ssl/test/make_certs.erl @@ -344,7 +344,7 @@ req_cnf(C) -> "default_bits = ", integer_to_list(C#config.default_bits), "\n" "RANDFILE = $ROOTDIR/RAND\n" "encrypt_key = no\n" - "default_md = sha1\n" + "default_md = md5\n" "#string_mask = pkix\n" "x509_extensions = ca_ext\n" "prompt = no\n" @@ -390,7 +390,7 @@ ca_cnf(C) -> ["crl_extensions = crl_ext\n" || C#config.v2_crls], "unique_subject = no\n" "default_days = 3600\n" - "default_md = sha256\n" + "default_md = md5\n" "preserve = no\n" "policy = policy_match\n" "\n" diff --git a/lib/ssl/test/ssl_basic_SUITE.erl b/lib/ssl/test/ssl_basic_SUITE.erl index 0148e1f5bc..8e3d2e4b80 100644 --- a/lib/ssl/test/ssl_basic_SUITE.erl +++ b/lib/ssl/test/ssl_basic_SUITE.erl @@ -119,7 +119,8 @@ options_tests() -> ]. api_tests() -> - [connection_info, + [new_options_in_accept, + connection_info, peername, peercert, peercert_with_client_cert, @@ -325,6 +326,37 @@ alerts(Config) when is_list(Config) -> end end, Alerts). %%-------------------------------------------------------------------- +new_options_in_accept() -> + [{doc,"Test that you can set ssl options in ssl_accept/3 and not tcp upgrade"}]. +new_options_in_accept(Config) when is_list(Config) -> + ClientOpts = ?config(client_opts, Config), + ServerOpts = ?config(server_opts, Config), + {ClientNode, ServerNode, Hostname} = ssl_test_lib:run_where(Config), + Server = ssl_test_lib:start_server([{node, ServerNode}, {port, 0}, + {from, self()}, + {ssl_opts, [{versions, [sslv3]}, + {ciphers,[{rsa,rc4_128,sha}]}]}, %% To be set in ssl_accept/3 + {mfa, {?MODULE, connection_info_result, []}}, + {options, ServerOpts}]), + + Port = ssl_test_lib:inet_port(Server), + Client = ssl_test_lib:start_client([{node, ClientNode}, {port, Port}, + {host, Hostname}, + {from, self()}, + {mfa, {?MODULE, connection_info_result, []}}, + {options, [{versions, [sslv3]} | ClientOpts]}]), + + ct:log("Testcase ~p, Client ~p Server ~p ~n", + [self(), Client, Server]), + + ServerMsg = ClientMsg = {ok, {sslv3, {rsa, rc4_128, sha}}}, + + ssl_test_lib:check_result(Server, ServerMsg, Client, ClientMsg), + + ssl_test_lib:close(Server), + ssl_test_lib:close(Client). +%%-------------------------------------------------------------------- + connection_info() -> [{doc,"Test the API function ssl:connection_info/1"}]. connection_info(Config) when is_list(Config) -> diff --git a/lib/ssl/test/ssl_test_lib.erl b/lib/ssl/test/ssl_test_lib.erl index 7d8ece8d19..59f10d53a6 100644 --- a/lib/ssl/test/ssl_test_lib.erl +++ b/lib/ssl/test/ssl_test_lib.erl @@ -106,7 +106,8 @@ connect(#sslsocket{} = ListenSocket, Opts) -> Node = proplists:get_value(node, Opts), ReconnectTimes = proplists:get_value(reconnect_times, Opts, 0), Timeout = proplists:get_value(timeout, Opts, infinity), - AcceptSocket = connect(ListenSocket, Node, 1 + ReconnectTimes, dummy, Timeout), + SslOpts = proplists:get_value(ssl_opts, Opts, []), + AcceptSocket = connect(ListenSocket, Node, 1 + ReconnectTimes, dummy, Timeout, SslOpts), case ReconnectTimes of 0 -> AcceptSocket; @@ -121,24 +122,30 @@ connect(ListenSocket, Opts) -> [ListenSocket]), AcceptSocket. -connect(_, _, 0, AcceptSocket, _) -> +connect(_, _, 0, AcceptSocket, _, _) -> AcceptSocket; -connect(ListenSocket, Node, N, _, Timeout) -> - ct:log("~p:~p~nssl:transport_accept(~p)~n", [?MODULE,?LINE, ListenSocket]), + +connect(ListenSocket, Node, N, _, Timeout, []) -> + ct:log("ssl:transport_accept(~p)~n", [ListenSocket]), {ok, AcceptSocket} = rpc:call(Node, ssl, transport_accept, [ListenSocket]), ct:log("~p:~p~nssl:ssl_accept(~p, ~p)~n", [?MODULE,?LINE, AcceptSocket, Timeout]), case rpc:call(Node, ssl, ssl_accept, [AcceptSocket, Timeout]) of ok -> -ct:log("~p:~p~nok from ssl:ssl_accept@~p",[?MODULE,?LINE, Node]), - connect(ListenSocket, Node, N-1, AcceptSocket, Timeout); + connect(ListenSocket, Node, N-1, AcceptSocket, Timeout, []); Result -> -ct:log("~p:~p~nssl:ssl_accept@~p ret ~p",[?MODULE,?LINE, Node,Result]), + ct:log("~p:~p~nssl:ssl_accept@~p ret ~p",[?MODULE,?LINE, Node,Result]), Result - end. + end; +connect(ListenSocket, Node, _, _, Timeout, Opts) -> + ct:log("ssl:transport_accept(~p)~n", [ListenSocket]), + {ok, AcceptSocket} = rpc:call(Node, ssl, transport_accept, + [ListenSocket]), + ct:log("ssl:ssl_accept(~p,~p, ~p)~n", [AcceptSocket, Opts, Timeout]), + rpc:call(Node, ssl, ssl_accept, [AcceptSocket, Opts, Timeout]), + AcceptSocket. - remove_close_msg(0) -> ok; remove_close_msg(ReconnectTimes) -> diff --git a/lib/stdlib/src/erl_pp.erl b/lib/stdlib/src/erl_pp.erl index 9dbe89da91..82bc2c1460 100644 --- a/lib/stdlib/src/erl_pp.erl +++ b/lib/stdlib/src/erl_pp.erl @@ -256,6 +256,10 @@ ltype({type,_Line,nonempty_list,[T]}) -> {seq,$[,$],[$,],[ltype(T),leaf("...")]}; ltype({type,Line,nil,[]}) -> lexpr({nil,Line}, 0, options(none)); +ltype({type,Line,map,any}) -> + simple_type({atom,Line,map}, []); +ltype({type,_Line,map,Pairs}) -> + map_type(Pairs); ltype({type,Line,tuple,any}) -> simple_type({atom,Line,tuple}, []); ltype({type,_Line,tuple,Ts}) -> @@ -289,6 +293,15 @@ binary_type(I1, I2) -> E2 = [[leaf("_:_*"),lexpr(I2, P, options(none))] || U], {seq,'<<','>>',[$,],E1++E2}. +map_type(Fs) -> + {first,[$#],map_pair_types(Fs)}. + +map_pair_types(Fs) -> + tuple_type(Fs, fun map_pair_type/1). + +map_pair_type({type,_Line,map_field_assoc,Ktype,Vtype}) -> + {seq,[],[]," =>",[ltype(Ktype),ltype(Vtype)]}. + record_type(Name, Fields) -> {first,[record_name(Name)],field_types(Fields)}. diff --git a/lib/stdlib/src/io_lib_pretty.erl b/lib/stdlib/src/io_lib_pretty.erl index 4057abd8d5..aece06afa6 100644 --- a/lib/stdlib/src/io_lib_pretty.erl +++ b/lib/stdlib/src/io_lib_pretty.erl @@ -25,8 +25,6 @@ -export([print/1,print/2,print/3,print/4,print/5,print/6]). --compile(no_native). - %%% %%% Exported functions %%% diff --git a/lib/stdlib/src/maps.erl b/lib/stdlib/src/maps.erl index 1f94d9e69d..fd6d56fa47 100644 --- a/lib/stdlib/src/maps.erl +++ b/lib/stdlib/src/maps.erl @@ -43,8 +43,6 @@ values/1 ]). --compile(no_native). - -spec get(Key,Map) -> Value when Key :: term(), Map :: map(), diff --git a/lib/stdlib/test/erl_pp_SUITE.erl b/lib/stdlib/test/erl_pp_SUITE.erl index 390322a5fa..babf3a49eb 100644 --- a/lib/stdlib/test/erl_pp_SUITE.erl +++ b/lib/stdlib/test/erl_pp_SUITE.erl @@ -993,6 +993,16 @@ maps_syntax(Config) when is_list(Config) -> ok = pp_expr(<<"#{ a => 1, <<\"hi\">> => \"world\", 33 => 1.0 }">>), ok = pp_expr(<<"#{ a := V1, <<\"hi\">> := V2 } = M">>), ok = pp_expr(<<"M#{ a => V1, <<\"hi\">> := V2 }">>), + F = <<"-module(maps_type_syntax).\n" + "-compile(export_all).\n" + "-type t1() :: map().\n" + "-type t2() :: #{ atom() => integer(), atom() => float() }.\n" + "-spec f1(t1()) -> 'true'.\n" + "f1(M) when is_map(M) -> true.\n" + "-spec f2(t2()) -> integer().\n" + "f2(#{a := V1,b := V2}) -> V1 + V2.\n" + "\n">>, + ok = pp_forms(F), ok. diff --git a/lib/test_server/doc/src/test_server_ctrl.xml b/lib/test_server/doc/src/test_server_ctrl.xml index f4aae724e0..0cda531716 100644 --- a/lib/test_server/doc/src/test_server_ctrl.xml +++ b/lib/test_server/doc/src/test_server_ctrl.xml @@ -773,11 +773,14 @@ test_server_ctrl:cross_cover_analyse(Level,[{s1,S1LogDir},{s2,S2LogDir}]) <p><c>What = tests_start, Data = {Name,NumCases}</c><br></br> <c>What = loginfo, Data = [{topdir,TestRootDir},{rundir,CurrLogDir}]</c><br></br> <c>What = tests_done, Data = {Ok,Failed,{UserSkipped,AutoSkipped}}</c><br></br> - <c>What = tc_start, Data = {{Mod,Func},TCLogFile}</c><br></br> - <c>What = tc_done, Data = {Mod,Func,Result}</c><br></br> - <c>What = tc_user_skip, Data = {Mod,Func,Comment}</c><br></br> - <c>What = tc_auto_skip, Data = {Mod,Func,Comment}</c><br></br> + <c>What = tc_start, Data = {{Mod,{Func,GroupName}},TCLogFile}</c><br></br> + <c>What = tc_done, Data = {Mod,{Func,GroupName},Result}</c><br></br> + <c>What = tc_user_skip, Data = {Mod,{Func,GroupName},Comment}</c><br></br> + <c>What = tc_auto_skip, Data = {Mod,{Func,GroupName},Comment}</c><br></br> <c>What = framework_error, Data = {{FWMod,FWFunc},Error}</c></p> + <p>Note that for a test case function that doesn't belong to a group, + <c>GroupName</c> has value <c>undefined</c>, otherwise the name of the test + case group.</p> </desc> </func> <func> diff --git a/lib/tools/test/eprof_SUITE.erl b/lib/tools/test/eprof_SUITE.erl index 1227d5b841..04b522de4a 100644 --- a/lib/tools/test/eprof_SUITE.erl +++ b/lib/tools/test/eprof_SUITE.erl @@ -104,7 +104,7 @@ basic(Config) when is_list(Config) -> profiling = eprof:profile([A]), true = exit(A, kill_it), profiling_stopped = eprof:stop_profiling(), - {error,_} = eprof:profile(fun() -> a = b end), + {error,_} = eprof:profile(fun() -> a = id(b) end), %% with mfa @@ -149,8 +149,7 @@ basic_option_1(Config) -> % vanilla {ok, _} = eprof:profile(fun() -> eprof_test:do(10) end, [{set_on_spawn, true}]), - [{_, MfasDo1},{_, MfasLists1}] = eprof:dump(), - Mfas1 = MfasDo1 ++ MfasLists1, + Mfas1 = lists:foldl(fun({_,Mfas},Out) -> Mfas ++ Out end, [], eprof:dump()), {value, {_, {11, _}}} = lists:keysearch({eprof_test,dec,1}, 1, Mfas1), {value, {_, { 1, _}}} = lists:keysearch({eprof_test, go,1}, 1, Mfas1), @@ -159,8 +158,7 @@ basic_option_1(Config) -> {ok, _} = eprof:profile(fun() -> eprof_test:do(10) end, [set_on_spawn]), - [{_, MfasDo2},{_, MfasLists2}] = eprof:dump(), - Mfas2 = MfasDo2 ++ MfasLists2, + Mfas2 = lists:foldl(fun({_,Mfas},Out) -> Mfas ++ Out end, [], eprof:dump()), {value, {_, {11, _}}} = lists:keysearch({eprof_test,dec,1}, 1, Mfas2), {value, {_, { 1, _}}} = lists:keysearch({eprof_test, go,1}, 1, Mfas2), {value, {_, { 9, _}}} = lists:keysearch({lists, split_2,5}, 1, Mfas2), @@ -255,3 +253,5 @@ ensure_eprof_stopped() -> Pid -> stopped=eprof:stop() end. + +id(I) -> I. @@ -52,9 +52,6 @@ usage () echo " release [-a] <target_dir> - creates full release to <target_dir>" echo " smp [-a] - build an Erlang system, smp flavor only" echo " tests <dir> - Build testsuites to <dir>" - echo " patch_app [-f] <target_dir> <app1>... - build given apps to <target_dir>" - echo " If core apps are patched, new start scripts will be created" - echo " and 'Install' must be run again." echo "" echo "These are for cleaning up an open source distribution" echo "with prebuilt files, so that it resembles the clean developers" @@ -1192,141 +1189,6 @@ do_release () release || exit 1 } -do_patch_app () -{ - # If target dir exists and has an installation of same major release, then - # build given apps. - # If patch includes erts, kernel, stdlib, sasl, then find latest - # erts, kernel, stdlib, sasl and create .rel files. - # Create .script/.boot - - if [ $# -lt 2 ]; then - usage - exit 1 - fi - - setup_make - if [ X`$MAKE is_cross_configured` = Xyes ]; then - TARGET=`$MAKE target_configured` - fi - if [ "x$1" = "x-f" ]; then - force="-force" - shift - else - force= - fi - target_dir=$1 - if [ ! -d "$target_dir/releases/$otp_major_vsn" ]; then - echo "No OTP $otp_major_vsn installation in $target_dir" 1>&2 - exit 1 - fi - - shift - - otp_version=`cat "$target_dir/releases/$otp_major_vsn/OTP_VERSION"` || { echo "Not able to read $target_dir/releases/$otp_major_vsn/OTP_VERSION" 1>&2; exit 1; } - { echo "$otp_version" | sed "s|^\([^\*]*\)\**|\1\*\*|g" > "$target_dir/releases/$otp_major_vsn/OTP_VERSION"; } 2>/dev/null || { echo "Not able to update $target_dir/OTP_VERSION" 1>&2; exit 1; } - - PATH="$ERL_TOP/bootstrap/bin:$PATH" $ERL_TOP/make/verify_runtime_dependencies -release "$otp_major_vsn" -source "$ERL_TOP" -target "$target_dir" $force "$@" - - if [ $? -ne 0 ]; then - exit $? - fi - - # Build all applications to target - for app in "$@"; do - if [ "$app" = "erts" ] && [ -d $ERL_TOP/$app ]; then - (cd $ERL_TOP/$app && $MAKE MAKE="$MAKE" TARGET=$TARGET \ - TESTROOT="$target_dir" release) || exit 1 - elif [ "$app" != "erts" ] && [ -d $ERL_TOP/lib/$app ]; then - (cd $ERL_TOP/lib/$app && $MAKE MAKE="$MAKE" TARGET=$TARGET \ - TESTROOT="$target_dir" release) || exit 1 - else - echo "Invalid application $app" 1>&2 - exit 1 - fi - done - - # If erts, kernel, stdlib or sasl is included, find versions - for app in "$@"; do - if [ "$app" = "erts" ]; then - erts_vsn=`grep '^VSN' erts/vsn.mk | sed "s|^VSN.*=[^0-9]*\([0-9].*\)$|\1|g"` - update_rel=true - elif [ "$app" = "kernel" ]; then - kernel_vsn=`sed "s|^KERNEL_VSN[^=]*=[^0-9]*\([0-9].*\)$|\1|g" lib/kernel/vsn.mk` - update_rel=true - elif [ "$app" = "stdlib" ]; then - stdlib_vsn=`sed "s|^STDLIB_VSN[^=]*=[^0-9]*\([0-9].*\)$|\1|g" lib/stdlib/vsn.mk` - update_rel=true - elif [ "$app" = "sasl" ]; then - sasl_vsn=`sed "s|^SASL_VSN[^=]*=[^0-9]*\([0-9].*\)$|\1|g" lib/sasl/vsn.mk` - update_rel=true - fi - done - - # and find the old versions for those not included - if [ "X$update_rel" != "X" ]; then - if [ "X$erts_vsn" = "X" ]; then - erts_vsns=`ls -d "$target_dir"/erts-* | sed "s|$target_dir/erts-\([0-9\.].*\)|\1|g"` - erts_vsn=`echo "$erts_vsns" | sort -t '.' -g | tail -n 1` - fi - if [ "X$kernel_vsn" = "X" ]; then - kernel_vsns=`ls -d "$target_dir"/lib/kernel-* | sed "s|$target_dir/lib/kernel-\([0-9\.].*\)|\1|g"` - kernel_vsn=`echo "$kernel_vsns" | sort -t '.' -g | tail -n 1` - fi - if [ "X$stdlib_vsn" = "X" ]; then - stdlib_vsns=`ls -d "$target_dir"/lib/stdlib-* | sed "s|$target_dir/lib/stdlib-\([0-9\.].*\)|\1|g"` - stdlib_vsn=`echo "$stdlib_vsns" | sort -t '.' -g | tail -n 1` - fi - if [ "X$sasl_vsn" = "X" ]; then - sasl_vsns=`ls -d "$target_dir"/lib/sasl-* | sed "s|$target_dir/lib/sasl-\([0-9\.].*\)|\1|g"` - sasl_vsn=`echo "$sasl_vsns" | sort -t '.' -g | tail -n 1` - fi - - # Generate .rel, .script and .boot - to tmp dir - start_clean="{release, {\"Erlang/OTP\",\"$otp_major_vsn\"}, {erts, \"$erts_vsn\"},\n [{kernel,\"$kernel_vsn\"},\n {stdlib,\"$stdlib_vsn\"}]}.\n" - start_sasl="{release, {\"Erlang/OTP\",\"$otp_major_vsn\"}, {erts, \"$erts_vsn\"},\n [{kernel,\"$kernel_vsn\"},\n {stdlib,\"$stdlib_vsn\"},\n {sasl,\"$sasl_vsn\"}]}.\n" - - tmp_dir="$target_dir/tmp"; - if [ ! -d "$tmp_dir" ]; then - mkdir "$tmp_dir" - fi - echo "$start_sasl" > "$tmp_dir/start_sasl.rel" - echo "$start_clean" > "$tmp_dir/start_clean.rel" - echo "$start_clean" > "$tmp_dir/no_dot_erlang.rel" - - erlc="$ERL_TOP/bootstrap/bin/erlc" - if [ ! -x "$erlc" ]; then - echo "erlc not found, can not create .script and .boot files" 1>&2 - exit 1 - fi - - $erlc -I"$target_dir"/lib/*/ebin -o$tmp_dir $tmp_dir/start_sasl.rel || exit 1 - $erlc -I"$target_dir"/lib/*/ebin -o$tmp_dir +no_warn_sasl $tmp_dir/start_clean.rel || exit 1 - $erlc -I"$target_dir"/lib/*/ebin -o$tmp_dir +no_warn_sasl +no_dot_erlang $tmp_dir/no_dot_erlang.rel || exit 1 - - # Generate RELEASES file - erl="$ERL_TOP/bootstrap/bin/erl" - if [ ! -x "$erl" ]; then - echo "erl not found, can not create RELEASES file" 1>&2 - exit 1 - fi - "$erl" -noinput +B -eval "release_handler:create_RELEASES(\"%ERL_ROOT%\", \"$tmp_dir\", \"$tmp_dir/start_sasl.rel\", []), halt()" || exit 1 - - # If all good so far, move generated files into target area - mv "$tmp_dir/RELEASES" "$target_dir/releases/RELEASES.src" - mv "$tmp_dir"/* "$target_dir/releases/$otp_major_vsn" - rmdir "$tmp_dir" - - # Remove old start scripts (forces a new run of Install) - rm -f "$target_dir"/releases/RELEASES - rm -f "$target_dir"/bin/*.script - rm -f "$target_dir"/bin/*.boot - rm -f "$target_dir"/bin/erl - fi - -} - - do_tests () { setup_make @@ -1587,9 +1449,6 @@ case "$1" in shift fi; do_release "$2";; - patch_app) - shift; - do_patch_app "$@";; tests) if [ $minus_a_flag = true ]; then shift diff --git a/system/doc/design_principles/applications.xml b/system/doc/design_principles/applications.xml index 228ca1f2bf..7b030115df 100644 --- a/system/doc/design_principles/applications.xml +++ b/system/doc/design_principles/applications.xml @@ -4,7 +4,7 @@ <chapter> <header> <copyright> - <year>1997</year><year>2013</year> + <year>1997</year><year>2014</year> <holder>Ericsson AB. All Rights Reserved.</holder> </copyright> <legalnotice> @@ -157,8 +157,9 @@ ch_app:stop([])</code> all applications have dependencies to at least <c>kernel</c> and <c>stdlib</c>.</item> </taglist> - <p>The syntax and contents of of the application resource file - are described in detail in <c>app(4)</c>.</p> + <note><p>The syntax and contents of of the application resource file + are described in detail in the<seealso marker="kernel:app"> + Application resource file reference</seealso>.</p></note> </section> <section> diff --git a/system/doc/getting_started/seq_prog.xml b/system/doc/getting_started/seq_prog.xml index 3830a34e5a..fd49102263 100644 --- a/system/doc/getting_started/seq_prog.xml +++ b/system/doc/getting_started/seq_prog.xml @@ -419,6 +419,129 @@ list_length([First | Rest]) -> </section> <section> + <title>Maps</title> + <p>Maps are a set of key to value associations. These associations + are encapsulated with "#{" and "}". To create an association from + "key" to value 42, we write:</p> +<code type="none"> +> #{ "key" => 42 }. +#{"key" => 42}</code> + <p>We will jump straight into the deep end with an example using some + interesting features.</p> + <p>The following example shows how we calculate alpha blending using + maps to reference color and alpha channels:</p> + <code type="none"> +-module(color). + +-export([new/4, blend/2]). + +-define(is_channel(V), (is_float(V) andalso V >= 0.0 andalso V =< 1.0)). + +new(R,G,B,A) when ?is_channel(R), ?is_channel(G), + ?is_channel(B), ?is_channel(A) -> + #{red => R, green => G, blue => B, alpha => A}. + +blend(Src,Dst) -> + blend(Src,Dst,alpha(Src,Dst)). + +blend(Src,Dst,Alpha) when Alpha > 0.0 -> + Dst#{ + red := red(Src,Dst) / Alpha, + green := green(Src,Dst) / Alpha, + blue := blue(Src,Dst) / Alpha, + alpha := Alpha + }; +blend(_,Dst,_) -> + Dst#{ + red := 0.0, + green := 0.0, + blue := 0.0, + alpha := 0.0 + }. + +alpha(#{alpha := SA}, #{alpha := DA}) -> + SA + DA*(1.0 - SA). + +red(#{red := SV, alpha := SA}, #{red := DV, alpha := DA}) -> + SV*SA + DV*DA*(1.0 - SA). +green(#{green := SV, alpha := SA}, #{green := DV, alpha := DA}) -> + SV*SA + DV*DA*(1.0 - SA). +blue(#{blue := SV, alpha := SA}, #{blue := DV, alpha := DA}) -> + SV*SA + DV*DA*(1.0 - SA).</code> + <p>Compile (file <c>color.erl</c>) and test:</p> + <pre> +> <input>c(color).</input> +{ok,color} +> <input>C1 = color:new(0.3,0.4,0.5,1.0).</input> +#{alpha => 1.0,blue => 0.5,green => 0.4,red => 0.3} +> <input>C2 = color:new(1.0,0.8,0.1,0.3).</input> +#{alpha => 0.3,blue => 0.1,green => 0.8,red => 1.0} +> <input>color:blend(C1,C2).</input> +#{alpha => 1.0,blue => 0.5,green => 0.4,red => 0.3} +> <input>color:blend(C2,C1).</input> +#{alpha => 1.0,blue => 0.38,green => 0.52,red => 0.51} +</pre> + <p>This example warrant some explanation:</p> + <code type="none"> +-define(is_channel(V), (is_float(V) andalso V >= 0.0 andalso V =< 1.0)).</code> + <p> + First we define a macro <c>is_channel</c> to help with our guard tests. + This is only here for convenience and to reduce syntax cluttering. + + You can read more about <seealso marker="doc/reference_manual:macros">Macros</seealso> + in the Erlang Reference Manual. + </p> + <code type="none"> +new(R,G,B,A) when ?is_channel(R), ?is_channel(G), + ?is_channel(B), ?is_channel(A) -> + #{red => R, green => G, blue => B, alpha => A}.</code> + <p> + The function <c>new/4</c> creates a new map term with and lets the keys + <c>red</c>, <c>green</c>, <c>blue</c> and <c>alpha</c> be associated + with an initial value. In this case we only allow for float + values between and including 0.0 and 1.0 as ensured by the <c>?is_channel/1</c> macro + for each argument. Only the <c>=></c> operator is allowed when creating a new map. + </p> + <p> + By calling <c>blend/2</c> on any color term created by <c>new/4</c> we can calculate + the resulting color as determined by the two maps terms. + </p> + <p> + The first thing <c>blend/2</c> does is to calculate the resulting alpha channel. + </p> + <code type="none"> +alpha(#{alpha := SA}, #{alpha := DA}) -> + SA + DA*(1.0 - SA).</code> + <p> + We fetch the value associated with key <c>alpha</c> for both arguments using + the <c>:=</c> operator. Any other keys + in the map are ignored, only the key <c>alpha</c> is required and checked for. + </p> + <p>This is also the case for functions <c>red/2</c>, <c>blue/2</c> and <c>green/2</c>.</p> + <code type="none"> +red(#{red := SV, alpha := SA}, #{red := DV, alpha := DA}) -> + SV*SA + DV*DA*(1.0 - SA).</code> + <p> + The difference here is that we check for two keys in each map argument. The other keys + are ignored. + </p> + <p> + Finally we return the resulting color in <c>blend/3</c>. + </p> + <code type="none"> +blend(Src,Dst,Alpha) when Alpha > 0.0 -> + Dst#{ + red := red(Src,Dst) / Alpha, + green := green(Src,Dst) / Alpha, + blue := blue(Src,Dst) / Alpha, + alpha := Alpha + };</code> + <p> + We update the <c>Dst</c> map with new channel values. The syntax for updating an existing key with a new value is done with <c>:=</c> operator. + </p> + </section> + + <section> <title>Standard Modules and Manual Pages</title> <p>Erlang has a lot of standard modules to help you do things. For example, the module <c>io</c> contains a lot of functions to help diff --git a/system/doc/installation_guide/Makefile b/system/doc/installation_guide/Makefile index 6923f52d8a..83210bd21f 100644 --- a/system/doc/installation_guide/Makefile +++ b/system/doc/installation_guide/Makefile @@ -1,7 +1,7 @@ # # %CopyrightBegin% # -# Copyright Ericsson AB 1996-2012. All Rights Reserved. +# Copyright Ericsson AB 1996-2014. All Rights Reserved. # # The contents of this file are subject to the Erlang Public License, # Version 1.1, (the "License"); you may not use this file except in @@ -58,8 +58,7 @@ XML_FILES = \ GENERATED_XML_FILES = \ INSTALL.xml \ INSTALL-CROSS.xml \ - INSTALL-WIN32.xml \ - MARKDOWN.xml + INSTALL-WIN32.xml # ---------------------------------------------------- @@ -74,8 +73,7 @@ REDIRECT_HTML_DIR = $(HTMLDIR)/source REDIRECT_HTML_FILES = \ $(REDIRECT_HTML_DIR)/INSTALL.html \ $(REDIRECT_HTML_DIR)/INSTALL-CROSS.html \ - $(REDIRECT_HTML_DIR)/INSTALL-WIN32.html \ - $(REDIRECT_HTML_DIR)/MARKDOWN.html + $(REDIRECT_HTML_DIR)/INSTALL-WIN32.html # ---------------------------------------------------- # FLAGS diff --git a/system/doc/installation_guide/install-binary.xml b/system/doc/installation_guide/install-binary.xml index 07027a39f6..af7dab6e44 100644 --- a/system/doc/installation_guide/install-binary.xml +++ b/system/doc/installation_guide/install-binary.xml @@ -4,7 +4,7 @@ <chapter> <header> <copyright> - <year>2000</year><year>2013</year> + <year>2000</year><year>2014</year> <holder>Ericsson AB. All Rights Reserved.</holder> </copyright> <legalnotice> @@ -31,117 +31,40 @@ <rev>C</rev> <file>install-binary.xml</file> </header> - - <section> - <title>UNIX</title> - - <section> - <title>Introduction</title> - <p>The system is delivered as a single compressed tar file.</p> - <p>To browse the on-line HTML documentation, Netscape or an equivalent - browser supporting frames is needed.</p> - </section> - - <section> - <title>Installation Procedure</title> - <p>When installed, the entire system, except for a small start-up - script, resides in a single directory tree. The location of this - directory tree can be chosen arbitrarily by the installer, and it - does not need to be in the user's <c>$PATH</c>. The only requirements - are that the file system where it is placed has enough free space, - and that the users who run Erlang/OTP have read access to it. In the - example below, the directory tree is assumed to be located at - <c>/usr/local/erlang</c>, which is here called the <em>top-level directory</em>.</p> - <p>It is assumed that you have the compressed tar file, the name of - which is <c><![CDATA[<PREFIX>.tar.gz]]></c>, where <c><![CDATA[<PREFIX>]]></c> is a string - denoting the particular Erlang/OTP release, e.g. - <c>otp_LXA_11930_sunos5_R9B</c>.</p> - <p>Wherever the string <c><![CDATA[<PREFIX>]]></c> is used below, it should - be replaced by the actual name prefix of the compressed tar file.</p> - <p>The tape archive file does not have one single directory in which - all other files are rooted. Therefore the tape archive file must be - extracted into an empty (newly created) directory.</p> - <list type="ordered"> - <item> - <p>If the <em>top-level directory</em> does not already exist, - create it:</p> - <pre> -mkdir /usr/local/erlang</pre> - </item> - <item> - <p>Change the current directory to the <em>top level directory</em>:</p> - <pre> -cd /usr/local/erlang</pre> - </item> - <item> - <p>Create the <em>installation directory</em> with an appropriate - name. For example:</p> - <pre> -mkdir otp_r7b</pre> - </item> - <item> - <p>Change to the <em>installation directory</em>, e.g.</p> - <pre> -cd otp_r7b</pre> - </item> - <item> - <p>Assuming the compressed tar file resides in the directory - <c><![CDATA[<SOME-DIR>]]></c>,. extract the compressed tar file into the - current directory:</p> - <pre> -gunzip -c <SOME-DIR>/<PREFIX>.tar.gz | tar xfp -</pre> - </item> - <item> - <p>Read the <c>README</c> file in the installation directory for - last minute updates, before proceeding.</p> - </item> - <item> - <p>Run the <c>Install</c> script in the installation directory, - with the absolute path of the installation directory as argument,</p> - <pre> -./Install /usr/local/erlang/otp_r7b</pre> - <p>and supply answers to the prompts.</p> - <p>In most cases, there is a default answer in square brackets - (<c>[]</c>). If the default is satisfactory, just press - <c><![CDATA[<Return>]]></c>. In general you are only prompted for one thing:</p> - <list type="bulleted"> - <item> - <p>"Do you want to use a minimal system startup instead of the - SASL startup?" <br></br> - - In a minimal system, only the Kernel and STDLIB applications - are loaded and started. If the SASL startup is used, the SASL - application is included as well. Normally, the minimal system - is enough.</p> - </item> - </list> - </item> - <item> - <p>Make Erlang/OTP available for users, either by putting the path - <c>/usr/local/erlang/otp_r7b/bin</c> in users <c>$PATH</c> - variable, or link the executable - <c>/usr/local/erlang/otp_r7b/bin/erl</c> accordingly, e.g.:</p> - <pre> -ln -s /usr/local/erlang/otp_r7b/bin/erl /usr/local/bin/erl </pre> - </item> - </list> - </section> - </section> - <section> <title>Windows</title> <section> <title>Introduction</title> - <p>The system is delivered as a single <c>.exe</c> file.</p> - <p>To browse the on-line HTML documentation, Netscape or an equivalent - browser supporting frames is needed.</p> + <p>The system is delivered as a Windows Installer executable. + Get it from our <url href="http://www.erlang.org/download.html">download page</url>.</p> </section> <section> - <title>Installation Procedure</title> + <title>Installation</title> <p>The installation procedure is is automated. Double-click the <c>.exe</c> file icon and follow the instructions.</p> </section> + <section> + <title>Verification</title> + <list type="bulleted"> + <item> + <p>Start Erlang/OTP by double-clicking on the Erlang shortcut icon on the + desktop.</p> + <p>Expect a command line window to pop up with an output looking something like this:</p> + <pre> + Erlang/OTP 17 [erts-6.0] [64-bit] [smp:2:2] + + Eshell V6.0 (abort with ^G) + 1></pre> + </item> + <item> + <p>Exit by entering the command <c>halt()</c>,</p> + <pre> + 2> <input>halt().</input></pre> + <p>which will close the Erlang/OTP shell. </p> + </item> + </list> + </section> </section> </chapter> diff --git a/system/doc/installation_guide/part.xml b/system/doc/installation_guide/part.xml index 19808fd165..02bf98db7c 100644 --- a/system/doc/installation_guide/part.xml +++ b/system/doc/installation_guide/part.xml @@ -4,7 +4,7 @@ <part xmlns:xi="http://www.w3.org/2001/XInclude"> <header> <copyright> - <year>2000</year><year>2013</year> + <year>2000</year><year>2014</year> <holder>Ericsson AB. All Rights Reserved.</holder> </copyright> <legalnotice> @@ -32,9 +32,7 @@ <p>How to install Erlang/OTP on UNIX or Windows.</p> </description> <xi:include href="install-binary.xml"/> - <xi:include href="verification.xml"/> <xi:include href="INSTALL.xml"/> <xi:include href="INSTALL-CROSS.xml"/> <xi:include href="INSTALL-WIN32.xml"/> - <xi:include href="MARKDOWN.xml"/> </part>
\ No newline at end of file diff --git a/system/doc/installation_guide/verification.xml b/system/doc/installation_guide/verification.xml deleted file mode 100644 index 391ddfb7b8..0000000000 --- a/system/doc/installation_guide/verification.xml +++ /dev/null @@ -1,102 +0,0 @@ -<?xml version="1.0" encoding="utf-8" ?> -<!DOCTYPE chapter SYSTEM "chapter.dtd"> - -<chapter> - <header> - <copyright> - <year>2000</year><year>2013</year> - <holder>Ericsson AB. All Rights Reserved.</holder> - </copyright> - <legalnotice> - The contents of this file are subject to the Erlang Public License, - Version 1.1, (the "License"); you may not use this file except in - compliance with the License. You should have received a copy of the - Erlang Public License along with this software. If not, it can be - retrieved online at http://www.erlang.org/. - - Software distributed under the License is distributed on an "AS IS" - basis, WITHOUT WARRANTY OF ANY KIND, either express or implied. See - the License for the specific language governing rights and limitations - under the License. - - </legalnotice> - - <title>Installation Verification</title> - <prepared>Peter Högfeldt</prepared> - <responsible>Peter Högfeldt</responsible> - <docno></docno> - <approved>(Peter Högfeldt</approved> - <checked></checked> - <date>1997-05-26</date> - <rev>C</rev> - <file>verification.xml</file> - </header> - <p>This chapter is about verifying your installation by performing - a few simple tests to see that your system is properly installed.</p> - - <section> - <title>UNIX</title> - <list type="bulleted"> - <item> - <p>Start Erlang/OTP from the command line,</p> - <pre> - unix> erl</pre> - <p>Expect the following output:</p> - <pre> - Erlang (BEAM) emulator version 5.0.1 [threads] - - Eshell V5.0.1 (abort with ^G) - 1></pre> - </item> - <item> - <p>Start the GS-based toolbar from the Erlang shell, </p> - <pre> - 1> <input>toolbar:start().</input></pre> - <p>and check that the toolbar window pops up. - </p> - <p><em>Note:</em> The trailing full stop (<c>"."</c>) is an end marker - for all commands in the Erlang shell, and must be entered for a - command to begin execution.</p> - </item> - <item> - <p>Exit by entering the command <c>halt()</c>,</p> - <pre> - 2> <input>halt().</input></pre> - <p>which should end both the toolbar window and the command line window. </p> - </item> - </list> - </section> - - <section> - <title>Windows</title> - <list type="bulleted"> - <item> - <p>Start Erlang/OTP by double-clicking on the Erlang shortcut icon on the - desktop.</p> - <p>Expect a command line window to pop up with the following output,</p> - <pre> - Erlang (BEAM) emulator version 5.0.1 [threads] - - Eshell V5.0.1 (abort with ^G) - 1></pre> - </item> - <item> - <p>Start the GS-based toolbar from the Erlang shell, </p> - <pre> - 1> <input>toolbar:start().</input></pre> - <p>and check that the toolbar window pops up. - </p> - <p><em>Note:</em> The trailing full stop (<c>"."</c>) is an end marker - for all commands in the Erlang shell, and must be entered for a - command to begin execution.</p> - </item> - <item> - <p>Exit by entering the command <c>halt()</c>,</p> - <pre> - 2> <input>halt().</input></pre> - <p>which should end both the toolbar window and the command line window. </p> - </item> - </list> - </section> -</chapter> - diff --git a/system/doc/installation_guide/xmlfiles.mk b/system/doc/installation_guide/xmlfiles.mk index 3995c607af..c443334cd7 100644 --- a/system/doc/installation_guide/xmlfiles.mk +++ b/system/doc/installation_guide/xmlfiles.mk @@ -1,7 +1,7 @@ # # %CopyrightBegin% # -# Copyright Ericsson AB 2009-2013. All Rights Reserved. +# Copyright Ericsson AB 2009-2014. All Rights Reserved. # # The contents of this file are subject to the Erlang Public License, # Version 1.1, (the "License"); you may not use this file except in @@ -18,8 +18,6 @@ # INST_GUIDE_CHAPTER_FILES = \ install-binary.xml \ - verification.xml \ INSTALL.xml \ INSTALL-CROSS.xml \ - INSTALL-WIN32.xml \ - MARKDOWN.xml + INSTALL-WIN32.xml diff --git a/system/doc/reference_manual/data_types.xml b/system/doc/reference_manual/data_types.xml index 8c690d6b86..0031664dfb 100644 --- a/system/doc/reference_manual/data_types.xml +++ b/system/doc/reference_manual/data_types.xml @@ -190,6 +190,38 @@ adam </section> <section> + <title>Map</title> + <p>Compound data type with a variable number of key-value associations:</p> + <pre> +#{Key1=>Value1,...,KeyN=>ValueN}</pre> + <p>Each key-value association in the map is called an + <em>association pair</em>. The key and value parts of the pair are + called <em>elements</em>. The number of association pairs is said to be + the <em>size</em> of the map.</p> + <p>There exists a number of BIFs to manipulate maps.</p> + <p>Examples:</p> + <pre> +1> <input>M1 = #{name=>adam,age=>24,date=>{july,29}}.</input> +#{age => 24,date => {july,29},name => adam} +2> <input>maps:get(name,M1).</input> +adam +3> <input>maps:get(date,M1).</input> +{july,29} +4> <input>M2 = maps:update(age,25,M1).</input> +#{age => 25,date => {july,29},name => adam} +5> <input>map_size(M).</input> +3 +6> <input>map_size(#{}).</input> +0</pre> + <p>A collection of maps processing functions can be found in + the STDLIB module <seealso marker="stdlib:maps"><c>maps</c></seealso>.</p> + <p>Read more about <seealso marker="maps">Maps</seealso>.</p> + <note> + <p>Maps are considered experimental during OTP 17.</p> + </note> + </section> + + <section> <title>List</title> <p>Compound data type with a variable number of terms.</p> <pre> diff --git a/system/doc/reference_manual/expressions.xml b/system/doc/reference_manual/expressions.xml index e9de3e006e..37208710fe 100644 --- a/system/doc/reference_manual/expressions.xml +++ b/system/doc/reference_manual/expressions.xml @@ -56,7 +56,7 @@ Expr1 + Expr2</code> <marker id="term"></marker> <title>Terms</title> <p>The simplest form of expression is a term, that is an integer, - float, atom, string, list or tuple. + float, atom, string, list, map or tuple. The return value is the term itself.</p> </section> @@ -1348,6 +1348,9 @@ end</pre> <cell align="left" valign="middle"><c>is_list/1</c></cell> </row> <row> + <cell align="left" valign="middle"><c>is_map/1</c></cell> + </row> + <row> <cell align="left" valign="middle"><c>is_number/1</c></cell> </row> <row> @@ -1398,6 +1401,9 @@ end</pre> <cell align="left" valign="middle"><c>length(List)</c></cell> </row> <row> + <cell align="left" valign="middle"><c>map_size(Map)</c></cell> + </row> + <row> <cell align="left" valign="middle"><c>node()</c></cell> </row> <row> diff --git a/system/doc/reference_manual/maps.xml b/system/doc/reference_manual/maps.xml new file mode 100644 index 0000000000..78808ce4a2 --- /dev/null +++ b/system/doc/reference_manual/maps.xml @@ -0,0 +1,274 @@ +<?xml version="1.0" encoding="utf-8" ?> +<!DOCTYPE chapter SYSTEM "chapter.dtd"> + +<chapter> + <header> + <copyright> + <year>2014</year> + <holder>Ericsson AB. All Rights Reserved.</holder> + </copyright> + <legalnotice> + The contents of this file are subject to the Erlang Public License, + Version 1.1, (the "License"); you may not use this file except in + compliance with the License. You should have received a copy of the + Erlang Public License along with this software. If not, it can be + retrieved online at http://www.erlang.org/. + + Software distributed under the License is distributed on an "AS IS" + basis, WITHOUT WARRANTY OF ANY KIND, either express or implied. See + the License for the specific language governing rights and limitations + under the License. + </legalnotice> + + <title>Maps</title> + <prepared></prepared> + <docno></docno> + <date></date> + <rev></rev> + <file>maps.xml</file> + </header> + + <note> + <p>Maps are considered experimental during OTP 17 and may be subject to change.</p> + <p>The documentation below describes it being possible to use arbitrary + expressions or variables as keys, this is <em>NOT</em> implemented in the current + version of Erlang/OTP.</p> + <p>Exceptions returns <c>badarg</c> instead of <c>badmap</c>, this will change in + the future releases.</p> + </note> + + <section> + <title>Creating Maps</title> + <p> + Constructing a new map is done by letting an expression <c>K</c> be associated with + another expression <c>V</c>: + </p> + <code>#{ K => V }</code> + <p> + New maps may include multiple associations at construction by listing every + association: + </p> + <code>#{ K1 => V1, .., Kn => Vn }</code> + <p> + An empty map is constructed by not associating any terms with each other: + </p> + <code>#{}</code> + <p> + All keys and values in the map are terms. Any expression is first evaluated and + then the resulting terms are used as <em>key</em> and <em>value</em> respectively. + </p> + <p> + Keys and values are separated by the <c>=></c> arrow and associations are + separated by <c>,</c>. + </p> + + <p> + Examples: + </p> + <code> +M0 = #{}, % empty map +M1 = #{a => <<"hello">>}, % single association with literals +M2 = #{1 => 2, b => b}, % multiple associations with literals +M3 = #{k => {A,B}}, % single association with variables +M4 = #{{"w", 1} => f()}. % compound key associated with an evaluated expression</code> + <p> + where, <c>A</c> and <c>B</c> are any expressions and <c>M0</c> through <c>M4</c> + are the resulting map terms. + </p> + <p> + If two matching keys are declared, the latter key will take precedence. + </p> + <p> + Example: + </p> + +<pre> +1> <input>#{1 => a, 1 => b}.</input> +#{1 => b } +2> <input>#{1.0 => a, 1 => b}.</input> +#{1 => b, 1.0 => a} +</pre> + <p> + The order in which the expressions constructing the keys and their + associated values are evaluated is not defined. The syntactic order of + the key-value pairs in the construction is of no relevance, except in + the above mentioned case of two matching keys. + </p> + </section> + + <section> + <title>Updating Maps</title> + <p> + Updating a map has similar syntax as constructing it. + </p> + <p> + An expression defining the map to be updated is put in front of the expression + defining the keys to be updated and their respective values. + </p> + <code>M#{ K => V }</code> + <p> + where <c>M</c> is a term of type map and <c>K</c> and <c>V</c> are any expression. + </p> + <p> + If key <c>K</c> does not match any existing key in the map, a new association + will be created from key <c>K</c> to value <c>V</c>. If key <c>K</c> matches + an existing key in map <c>M</c> its associated value will be replaced by the + new value <c>V</c>. In both cases the evaluated map expression will return a new map. + </p> + <p> + If <c>M</c> is not of type map an exception of type <c>badmap</c> is thrown. + </p> + <p> + To only update an existing value, the following syntax is used, + </p> + <code>M#{ K := V } </code> + <p> + where <c>M</c> is an term of type map, <c>V</c> is an expression and <c>K</c> + is an expression which evaluates to an existing key in <c>M</c>. + </p> + <p> + If key <c>K</c> does not match any existing keys in map <c>M</c> an exception + of type <c>badarg</c> will be triggered at runtime. If a matching key <c>K</c> + is present in map <c>M</c> its associated value will be replaced by the new + value <c>V</c> and the evaluated map expression returns a new map. + </p> + <p> + If <c>M</c> is not of type map an exception of type <c>badmap</c> is thrown. + </p> + <p> + Examples: + </p> + <code> +M0 = #{}, +M1 = M0#{a => 0}, +M2 = M1#{a => 1, b => 2}, +M3 = M2#{"function" => fun() -> f() end}, +M4 = M3#{a := 2, b := 3}. % 'a' and 'b' was added in `M1` and `M2`.</code> + <p> + where <c>M0</c> is any map. It follows that <c>M1 .. M4</c> are maps as well. + </p> + <p> + More Examples: + </p> +<pre> +1> <input>M = #{1 => a}.</input> +#{1 => a } +2> <input>M#{1.0 => b}.</input> +#{1 => a, 1.0 => b}. +3> <input>M#{1 := b}.</input> +#{1 => b} +4> <input>M#{1.0 := b}.</input> +** exception error: bad argument +</pre> + <p> + As in construction, the order in which the key and value expressions + are evaluated is not defined. The + syntactic order of the key-value pairs in the update is of no + relevance, except in the case where two keys match, in which + case the latter value is used. + </p> + </section> + + <section> + <title>Maps in Patterns</title> + <p> + Matching of key-value associations from maps is done in the following way: + </p> + + <code>#{ K := V } = M</code> + <p> + where <c>M</c> is any map. The key <c>K</c> has to be an expression with bound + variables or a literals, and <c>V</c> can be any pattern with either bound or + unbound variables. + </p> + <p> + If the variable <c>V</c> is unbound, it will be bound to the value associated + with the key <c>K</c>, which has to exist in the map <c>M</c>. If the variable + <c>V</c> is bound, it has to match the value associated with <c>K</c> in <c>M</c>. + </p> + <p> Example: </p> +<code> +1> <input>M = #{"tuple" => {1,2}}.</input> +#{"tuple" => {1,2}} +2> <input>#{"tuple" := {1,B}} = M.</input> +#{"tuple" => {1,2}} +3> <input>B.</input> +2.</code> + <p> + This will bind variable <c>B</c> to integer <c>2</c>. + </p> + <p> + Similarly, multiple values from the map may be matched: + </p> + <code>#{ K1 := V1, .., Kn := Vn } = M</code> + <p> + where keys <c>K1 .. Kn</c> are any expressions with literals or bound variables. If all + keys exist in map <c>M</c> all variables in <c>V1 .. Vn</c> will be matched to the + associated values of their respective keys. + </p> + <p> + If the matching conditions are not met, the match will fail, either with + </p> + <list> + <item> + a <c>badmatch</c> exception, if used in the context of the matching operator + as in the example, + </item> + <item> + or resulting in the next clause being tested in function heads and + case expressions. + </item> + </list> + <p> + Matching in maps only allows for <c>:=</c> as delimiters of associations. + The order in which keys are declared in matching has no relevance. + </p> + <p> + Duplicate keys are allowed in matching and will match each pattern associated + to the keys. + </p> + <code>#{ K := V1, K := V2 } = M</code> + <p> + Matching an expression against an empty map literal will match its type but + no variables will be bound: + </p> + <code>#{} = Expr</code> + <p> + This expression will match if the expression <c>Expr</c> is of type map, otherwise + it will fail with an exception <c>badmatch</c>. + </p> + <section> + <title>Matching syntax: Example with literals in function heads</title> + <p> + Matching of literals as keys are allowed in function heads. + </p> + <code> +%% only start if not_started +handle_call(start, From, #{ state := not_started } = S) -> +... + {reply, ok, S#{ state := start }}; + +%% only change if started +handle_call(change, From, #{ state := start } = S) -> +... + {reply, ok, S#{ state := changed }};</code> + </section> + </section> + <section> + <title>Maps in Guards</title> + <p> + Maps are allowed in guards as long as all sub-expressions are valid guard expressions. + </p> + <p> + Two guard BIFs handles maps: + </p> + <list> + <item> + <seealso marker="erts:erlang#is_map/1">is_map/1</seealso> + </item> + <item> + <seealso marker="erts:erlang#map_size/1">map_size/1</seealso> + </item> + </list> + </section> +</chapter> diff --git a/system/doc/reference_manual/modules.xml b/system/doc/reference_manual/modules.xml index cd4c3a1b1b..f0ec7ef165 100644 --- a/system/doc/reference_manual/modules.xml +++ b/system/doc/reference_manual/modules.xml @@ -53,10 +53,10 @@ fact(0) -> % | <pre> -Tag(Value).</pre> <p><c>Tag</c> must be an atom, while <c>Value</c> must be a literal - term. As a convenience in user-defined attributes, the literal term - <c>Value</c> the syntax <c>Name/Arity</c> - (where <c>Name</c> is an atom and <c>Arity</c> a positive integer) - will be translated to <c>{Name,Arity}</c>.</p> + term. As a convenience in user-defined attributes, if the literal term + <c>Value</c> has the syntax <c>Name/Arity</c> + (where <c>Name</c> is an atom and <c>Arity</c> a positive integer), + the term <c>Name/Arity</c> will be translated to <c>{Name,Arity}</c>.</p> <p>Any module attribute can be specified. The attributes are stored in the compiled code and can be retrieved by calling diff --git a/system/doc/reference_manual/part.xml b/system/doc/reference_manual/part.xml index ee8f3dd7eb..36fb888748 100644 --- a/system/doc/reference_manual/part.xml +++ b/system/doc/reference_manual/part.xml @@ -36,6 +36,7 @@ <xi:include href="typespec.xml"/> <xi:include href="expressions.xml"/> <xi:include href="macros.xml"/> + <xi:include href="maps.xml"/> <xi:include href="records.xml"/> <xi:include href="errors.xml"/> <xi:include href="processes.xml"/> diff --git a/system/doc/reference_manual/typespec.xml b/system/doc/reference_manual/typespec.xml index 71aec732cf..cc35c6eb21 100644 --- a/system/doc/reference_manual/typespec.xml +++ b/system/doc/reference_manual/typespec.xml @@ -100,6 +100,7 @@ | Fun | Integer | List + | Map | Tuple | Union | UserDefined %% described in Section 6.3 @@ -126,10 +127,17 @@ | nonempty_improper_list(Type1, Type2) %% Type1 and Type2 as above | nonempty_list(Type) %% Proper non-empty list + Map :: map() %% stands for a map of any size + | #{} %% stands for a map of any size + | #{PairList} + Tuple :: tuple() %% stands for a tuple of any size | {} | {TList} + PairList :: Type => Type + | Type => Type, PairList + TList :: Type | Type, TList @@ -275,6 +283,10 @@ Records have been extended to possibly contain type information. This is described in the sub-section <seealso marker="#typeinrecords">"Type information in record declarations"</seealso> below. </p> + <note> + <p>Map types, both <c>map()</c> and <c>#{ ... }</c>, are considered experimental during OTP 17.</p> + <p>No type information of maps pairs, only the containing map types, are used by Dialyzer in OTP 17.</p> + </note> </section> <section> diff --git a/system/doc/reference_manual/xmlfiles.mk b/system/doc/reference_manual/xmlfiles.mk index 6886c8c7cf..181e6f8042 100644 --- a/system/doc/reference_manual/xmlfiles.mk +++ b/system/doc/reference_manual/xmlfiles.mk @@ -24,6 +24,7 @@ REF_MAN_CHAPTER_FILES = \ functions.xml \ expressions.xml \ macros.xml \ + maps.xml \ records.xml \ errors.xml \ processes.xml \ diff --git a/system/doc/system_principles/system_principles.xml b/system/doc/system_principles/system_principles.xml index 4f2202fdd1..70c69b1dab 100644 --- a/system/doc/system_principles/system_principles.xml +++ b/system/doc/system_principles/system_principles.xml @@ -4,7 +4,7 @@ <chapter> <header> <copyright> - <year>1996</year><year>2013</year> + <year>1996</year><year>2014</year> <holder>Ericsson AB. All Rights Reserved.</holder> </copyright> <legalnotice> @@ -34,13 +34,12 @@ <p>An Erlang runtime system is started with the command <c>erl</c>:</p> <pre> % <input>erl</input> -Erlang (BEAM) emulator version 5.2.3.5 [hipe] [threads:0] +Erlang/OTP 17 [erts-6.0] [hipe] [smp:8:8] -Eshell V5.2.3.5 (abort with ^G) +Eshell V6.0 (abort with ^G) 1> </pre> <p><c>erl</c> understands a number of command line arguments, see - <c>erl(1)</c>. A number of them are also described in this - chapter.</p> + <c>erl(1)</c>. A number of them are also described in this chapter.</p> <p>Application programs can access the values of the command line arguments by calling one of the functions <c>init:get_argument(Key)</c>, or <c>init:get_arguments()</c>. diff --git a/system/doc/system_principles/versions.xml b/system/doc/system_principles/versions.xml index 2bf0d18010..c63913d867 100644 --- a/system/doc/system_principles/versions.xml +++ b/system/doc/system_principles/versions.xml @@ -61,14 +61,13 @@ <c>filename:join([<seealso marker="kernel:code#root_dir/0">code:root_dir()</seealso>, "releases", <seealso marker="erts:erlang#system_info_otp_release">erlang:system_info(otp_release)</seealso>, "OTP_VERSION"]).</c></p> <p>If the version read from the <c>OTP_VERSION</c> file in a development system has a <c>**</c> suffix, the system has been - patched using the - <seealso marker="doc/installation_guide:PATCH-APP"><c>$ERL_TOP/otp_build patch_app</c></seealso> - tool. In this case, the system consists of application versions from - multiple OTP versions. The version preceding the <c>**</c> + patched using the <c>otp_patch_apply</c> tool available to + licensed customers. In this case, the system consists of application + versions from multiple OTP versions. The version preceding the <c>**</c> suffix corresponds to the OTP version of the base system that has been patched. Note that if a development system is updated by - other means than <c>$ERL_TOP/otp_build patch_app</c>, the - <c>OTP_VERSION</c> file may identify wrong OTP version.</p> + other means than <c>otp_patch_apply</c>, the <c>OTP_VERSION</c> file + may identify wrong OTP version.</p> <p>No <c>OTP_VERSION</c> file will be placed in a <seealso marker="create_target">target system</seealso> created |